String.Index is not valid after mutating its associated string

Great little util, thanks for it! ❤
I've just run into this issue with longer Cyrillic text (and a simple <b>...</b> in there):
https://forums.swift.org/t/string-index-is-not-valid-after-mutating-its-associated-string/30851
It's an issue as text gets mutated between didStartElement and didEndElement calls.
So instead of storing String.Index, I'm recording the Int position in Element (text.count) and creating the NSRange based on that in didEndElement.
     private struct Element {
         let name: String
-        let startIndex: String.Index
+        let startPosition: Int
         let attributes: [NSAttributedString.Key: Any]
     }
...

     func parser(_ parser: XMLParser,
                 didStartElement elementName: String,
                 namespaceURI: String?,
                 qualifiedName qName: String?,
                 attributes attributeDict: [String: String] = [:]) {
         var attributes = style.getAttributes(for: elementName) ?? [:]
         if elementName == "a", let url = attributeDict["href"].map(URL.init(string:)) {
             attributes[.link] = url
         }
-        let element = Element(name: elementName, startIndex: text.endIndex, attributes: attributes)
+        let element = Element(name: elementName, startPosition: text.count, attributes: attributes)
         elements.append(element)
     }

     func parser(_ parser: XMLParser, didEndElement elementName: String, namespaceURI: String?, qualifiedName qName: String?) {
         guard let element = elements.popLast() else {
             return assertionFailure("No opening tag for \(elementName)")
         }
         guard element.name == elementName else {
             return assertionFailure("Closing tag mismatch. Expected: \(element.name), got: \(elementName)")
         }
-        let range = NSRange(element.startIndex..<text.endIndex, in: text)
+        let range = NSRange(location: element.startPosition, length: text.count - element.startPosition)
         attributes.append((range, element.attributes))
     }

Hope this helps

Thanks, can you share the example text so that I could add a unit test for this?
I haven't seen this issue yet, my assumption was that String doesn't invalidate its indices after appending. If it does, that's going to be a problem. Yeah, switching to offset-based indexing will fix it.

Thanks, can you share the example text so that I could add a unit test for this?
I haven't seen this issue yet, my assumption was that String doesn't invalidate its indices after appending. If it does, that's going to be a problem. Yeah, switching to offset-based indexing will fix it.

This piece of string caused a crash: "Если вы забыли свой пароль на новом устройстве, вам также будет отправлен 6-значный код проверки"
While a similar string worked: "Вы получите временный 4-значный пароль через SMS, который вам нужно будет изменить" ...so not quite sure if it's language related or just my environment somehow.

This piece of string caused a crash: "Если вы забыли свой пароль на новом устройстве, вам также будет отправлен 6-значный код проверки"
While a similar string worked: "Вы получите временный 4-значный пароль через SMS, который вам нужно будет изменить" ...so not quite sure if it's language related or just my environment somehow.

I opened a PR https://github.com/kean/Formatting/pull/2. I could reproduce the crash, but it doesn't happen every time.

I opened a PR https://github.com/kean/Formatting/pull/2. I could reproduce the crash, but it doesn't happen every time.

Looking good 👍  Strings, especially the indexing, in Swift is 🤯 (or more like 💩). Add some compiler optimisation to the mix and we end up with weird stuff like this. I didn't expect indexes to get invalidated when a string "grows" either.

Looking good 👍  Strings, especially the indexing, in Swift is 🤯 (or more like 💩). Add some compiler optimisation to the mix and we end up with weird stuff like this. I didn't expect indexes to get invalidated when a string "grows" either.

Exactly, I have a lot of experience working with string indices and I still fell for that. It didn't even cross my mind that append could invalidate existing indexes. What is most puzzling is that I tested it against hundreds of both short and long English strings and never encountered this issue.
Thanks for reporting, I hope this fully resoles it.

Exactly, I have a lot of experience working with string indices and I still fell for that. It didn't even cross my mind that append could invalidate existing indexes. What is most puzzling is that I tested it against hundreds of both short and long English strings and never encountered this issue.
Thanks for reporting, I hope this fully resoles it.


Support input selection

It works also for me. I do not why, but when I use the power button of the media player, it goes to HOME option, similar to config screen and not able to play directly from last input selected before switching off, for example TV input.
If I use the power button of remote, it works and TV input is selected by default.
El 7 oct 2019 23:26 +0200, Matthieu DUVAL<EMAIL_ADDRESS>escribió:

Tested and work for me.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Originally posted by @patrul in https://github.com/alexmohr/media_player.sony/issues/2#issuecomment-539385214

Could you post your device type?
It's strange that you have issues like that because the device gets the command as a string and not via an ID which could be missmatched.
Have you tried the TV & Sideview App from Sony and if yes is it working?

My device is a Home Cinema BDV-N7200W
Let me check if that application works for me
El 8 oct 2019 10:11 +0200, Alex , escribió:

Could you post your device type?
It's strange that you have issues like that because the device gets the command as a string and not via an ID which could be missmatched.
Have you tried the TV & Sideview App from Sony and if yes is it working?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

The application works in the same way.
I can control remotely but when I connect, it goes to HOME and not to the last input select, so I need to select the proper source/input (TV in my case to use Digital input/Toslink). It is not a problem, one additional step in the sequence. What I am not sure if it is currently implemented on sony media player.
El 8 oct 2019 12:22 +0200, Richard Peña<EMAIL_ADDRESS>escribió:

My device is a Home Cinema BDV-N7200W
Let me check if that application works for me
El 8 oct 2019 10:11 +0200, Alex , escribió:

Could you post your device type?
It's strange that you have issues like that because the device gets the command as a string and not via an ID which could be missmatched.
Have you tried the TV & Sideview App from Sony and if yes is it working?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

If the TV & SideView app shows the same wrong behavior than the issue is in the firmware of your device. Make sure you've installed the latest firmware.
You could solve this easily via a home assistant script which will be triggered when the state of the devices changes.

I have just used the mobile app to select “Software Update” on TV screen but it reports the last version it is installed.
I could try to program this sequence, but how could I change the source from Home Assistant? It is implemented on sony media player? I cannot see corresponding buttons
El 8 oct 2019 13:39 +0200, Alex<EMAIL_ADDRESS>escribió:

If the TV & SideView app shows the same wrong behavior than the issue is in the firmware of your device. Make sure you've installed the latest firmware.
You could solve this easily via a home assistant script which will be triggered when the state of the devices changes.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

No you are right this functionality is currently not implemented.
I do not have a device which supports selecting sources. I can implement this feature but I would need some XML files from your device to make sure the implementation is working and to write tests against that.

To provide the file take a look at https://gist.github.com/kalleth/e10e8f3b8b7cb1bac21463b0073a65fb
I'd need the complete response for the method getCurrentExternalInputsStatus

It would be great.
Please, let me know how can I help. I have very basic knowledge of development, I do not remember any language but I studied it and started with Basic, and teached about HTML so many years ago.
El 8 oct 2019 13:57 +0200, Alex<EMAIL_ADDRESS>escribió:

No you are right this functionality is currently not implemented.
I do not have a device which supports selecting sources. I can implement this feature but I would need some XML files from your device to make sure the implementation is working and to write tests against that.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Please let me know if the linked gist above does not provide you enough information to acquire the files.

Sure, I will do all my best
There is a problem:
curl: (7) Failed to connect to <IP_ADDRESS> port 80: Conexión rehusada
I guess it is the same cause giving these error in my logs, showed before
2019-10-08 13:13:19 ERROR (SyncWorker_11) [sonyapilib.device] HTTPError: 403 Client Error: Forbidden for url: http://<IP_ADDRESS>:50002/getRemoteCommandList
2019-10-08 13:13:19 ERROR (SyncWorker_11) [custom_components.sony.media_player] 9830
2019-10-08 13:13:19 ERROR (SyncWorker_11) [custom_components.sony.media_player] Amplificador

curl: (7) Failed to connect to <IP_ADDRESS> port 80: Conexión rehusada is due to using the wrong port. There is no service listening on port 80.
Try a curl on http://<IP_ADDRESS>:52323/dmr.xml and post the output here.
The sony api is a bit wacky from time to time, espescially when trying random services. A restart of the device often helps.
To access things via curl it is necessary to attach the correct headers to the device or a 403 is raised. Look for \"headers\": {\"Authorization\": \"Basic XXXXX=\", \ in sony.conf file to get the authorization for curl.
Some services do not require authentication these depend on the model of the device. As you are not seeing the device as paired something went wrong there. You can reset the home assistant configuration by deleting sony.conf in the home assistant configuration folder, after that the device can be paired again.

curl http://<IP_ADDRESS>:52323/dmr.xml
<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0" xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/11" xmlns:df="http://schemas.microsoft.com/windows/2008/09/devicefoundation">
  <specVersion>
    <major>1</major>
    <minor>0</minor>
  </specVersion>
  <device>
    <deviceType>urn:schemas-upnp-org:device:MediaRenderer:1</deviceType>
    <friendlyName>BDV-N7200W</friendlyName>
    <manufacturer>Sony Corporation</manufacturer>
    <manufacturerURL>http://www.sony.net/</manufacturerURL>
    <modelName>BDV-N7200W</modelName>
    <modelNumber>BDV-2014</modelNumber>
    <UDN>uuid:00000000-0000-1010-8000-104fa844bd2e</UDN>
    <dlna:X_DLNADOC xmlns:dlna="urn:schemas-dlna-org:device-1-0">DMR-1.50</dlna:X_DLNADOC>
    <dlna:X_DLNACAP xmlns:dlna="urn:schemas-dlna-org:device-1-0">playcontainer-1-0</dlna:X_DLNACAP>
    <iconList>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_large.jpg</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_large.png</url>
      </icon>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_small.jpg</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_small.png</url>
      </icon>
    </iconList>
    <serviceList>
      <service>
        <serviceType>urn:schemas-upnp-org:service:RenderingControl:1</serviceType>
        <serviceId>urn:upnp-org:serviceId:RenderingControl</serviceId>
        <SCPDURL>/RenderingControlBdvSCPD.xml</SCPDURL>
        <controlURL>/upnp/control/RenderingControl</controlURL>
        <eventSubURL>/upnp/event/RenderingControl</eventSubURL>
      </service>
      <service>
        <serviceType>urn:schemas-upnp-org:service:ConnectionManager:1</serviceType>
        <serviceId>urn:upnp-org:serviceId:ConnectionManager</serviceId>
        <SCPDURL>/ConnectionManagerSCPD.xml</SCPDURL>
        <controlURL>/upnp/control/ConnectionManager</controlURL>
        <eventSubURL>/upnp/event/ConnectionManager</eventSubURL>
      </service>
      <service>
        <serviceType>urn:schemas-upnp-org:service:AVTransport:1</serviceType>
        <serviceId>urn:upnp-org:serviceId:AVTransport</serviceId>
        <SCPDURL>/AVTransportSCPD.xml</SCPDURL>
        <controlURL>/upnp/control/AVTransport</controlURL>
        <eventSubURL>/upnp/event/AVTransport</eventSubURL>
      </service>
    </serviceList>
    <av:X_StandardDMR xmlns:av="urn:schemas-sony-com:av">1.1</av:X_StandardDMR>
    <microsoft:magicPacketWakeSupported xmlns:microsoft="urn:schemas-microsoft-com:WMPNSS-1-0">
        1
    </microsoft:magicPacketWakeSupported>
    <pnpx:X_compatibleId>MS_DigitalMediaDeviceClass_DMR_V001</pnpx:X_compatibleId>
    <pnpx:X_deviceCategory>MediaDevices</pnpx:X_deviceCategory>
    <pnpx:X_hardwareId>VEN_0106&DEV_0400&REV_01</pnpx:X_hardwareId>
    <df:X_deviceCategory>Multimedia.DMR</df:X_deviceCategory>
  </device>
</root>

Correctly registered

some services do not require authentication these depend on the model of the device. As you are not seeing the device as paired something went wrong there. Altough it is strange that most services seem to work. You can reset the home assistant configuration by deleting sony.conf in the home assistant configuration folder, after that the device can be paired again.

Thanks for posting the file.
I need http://<IP_ADDRESS>:50001/Iirc.xml and http://<IP_ADDRESS>/sony/getCurrentExternalInputsStatus

curl does not get anything from both URL

If I get in this way, the result shows error 415 Unsupported Media Type
curl -v -XPOST http://<IP_ADDRESS>:50001/Iirc.xml -d '{"method":"","params":[""],"id":1,"version":"1.0"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Expire in 0 ms for 6 (transfer 0x11d3880)
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x11d3880)
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 50001 (#0)
> POST /Iirc.xml HTTP/1.1
> Host: <IP_ADDRESS>:50001
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 50
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 50 out of 50 bytes
< HTTP/1.1 415 Unsupported Media Type
< Connection: close
< Date: Wed, 09 Oct 2019 06:59:13 GMT
< Server: Linux/2.6 UPnP/1.0 Sony-BDV/2.0
<
* Closing connection 0

And if I try to connect ofher port it opens connection but says 404 Not found
curl -v -XPOST http://<IP_ADDRESS>:50002/Iirc.xml -d '{"method":"","params":[""],"id":1,"version":"1.0"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Expire in 0 ms for 6 (transfer 0x201a880)
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x201a880)
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 50002 (#0)
> POST /Iirc.xml HTTP/1.1
> Host: <IP_ADDRESS>:50002
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 50
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 50 out of 50 bytes
< HTTP/1.1 404 Not Found
< Connection: close
< Date: Wed, 09 Oct 2019 07:00:18 GMT
< Server: Linux/2.6 Sony-BDV/2.0
<
* Closing connection 0

This should be the correct URL http://<IP_ADDRESS>:50001/Iircc.xml (I forgot the 2nd 'c') The port 50001 is correct but you should use GET for both requests not POST

It might be necessary to authenticate against the device when access this URLs. You must send either a pre shared key which you have to configure on your device or authenticate via basic auth. The data for this can be taken from your home assistant config in the sony.conf file

When I try to connect with browser or curl to http://<IP_ADDRESS>:50001/Iircc.xml
It says HTTP ERROR 404

Third time's the charm... http://<IP_ADDRESS>:50001/Ircc.xml
But please post the other XML I've requested as well the second one is actually more important.
In the sony TV & SideView app do you have a list of available sources or is it just an input button which cycles the inputs through?

curl -X GET http://<IP_ADDRESS>:50001/Ircc.xml
<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
  <specVersion>
    <major>1</major>
    <minor>0</minor>
  </specVersion>
  <device>
    <deviceType>urn:schemas-upnp-org:device:Basic:1</deviceType>
    <friendlyName>BDV-N7200W</friendlyName>
    <manufacturer>Sony Corporation</manufacturer>
    <manufacturerURL>http://www.sony.net/</manufacturerURL>
    <modelDescription></modelDescription>
    <modelName>BDV</modelName>
    <modelURL></modelURL>
    <UDN>uuid:00000003-0000-1010-8000-104fa844bd2e</UDN>
    <iconList>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_large.jpg</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_large.png</url>
      </icon>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_small.jpg</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/bdv_nv_device_icon_small.png</url>
      </icon>
    </iconList>
    <serviceList>
      <service>
        <serviceType>urn:schemas-sony-com:service:IRCC:1</serviceType>
        <serviceId>urn:schemas-sony-com:serviceId:IRCC</serviceId>
        <SCPDURL>/IRCCSCPD.xml</SCPDURL>
        <controlURL>/upnp/control/IRCC</controlURL>
        <eventSubURL></eventSubURL>
      </service>
    </serviceList>
    <presentationURL></presentationURL>
    <av:X_IRCC_DeviceInfo xmlns:av="urn:schemas-sony-com:av">
      <av:X_IRCC_Version>1.0</av:X_IRCC_Version>
      <av:X_IRCC_CategoryList>
        <av:X_IRCC_Category>
          <av:X_CategoryInfo>AAIAAAAF</av:X_CategoryInfo>
        </av:X_IRCC_Category>
        <av:X_IRCC_Category>
          <av:X_CategoryInfo>AAIAAADQ</av:X_CategoryInfo>
        </av:X_IRCC_Category>
        <av:X_IRCC_Category>
          <av:X_CategoryInfo>AAMAAAIQ</av:X_CategoryInfo>
        </av:X_IRCC_Category>
        <av:X_IRCC_Category>
          <av:X_CategoryInfo>AAMAAAYQ</av:X_CategoryInfo>
        </av:X_IRCC_Category>
        <av:X_IRCC_Category>
          <av:X_CategoryInfo>AAMAAA4Q</av:X_CategoryInfo>
        </av:X_IRCC_Category>
      </av:X_IRCC_CategoryList>
    </av:X_IRCC_DeviceInfo>
    <av:X_UNR_DeviceInfo xmlns:av="urn:schemas-sony-com:av">
      <av:X_UNR_Version>1.3</av:X_UNR_Version>
      <av:X_CERS_ActionList_URL>http://<IP_ADDRESS>:50002/actionList</av:X_CERS_ActionList_URL>
    </av:X_UNR_DeviceInfo>
    <av:X_RDIS_DeviceInfo xmlns:av="urn:schemas-sony-com:av">
      <av:X_RDIS_Version>1.0</av:X_RDIS_Version>
      <av:X_RDIS_SESSION_CONTROL>false</av:X_RDIS_SESSION_CONTROL>
      <av:X_RDIS_ENTRY_PORT>50004</av:X_RDIS_ENTRY_PORT>
    </av:X_RDIS_DeviceInfo>
  </device>
</root>

curl -X GET http://<IP_ADDRESS>/sony/getCurrentExternalInputsStatus
curl: (7) Failed to connect to <IP_ADDRESS> port 80: Conexión rehusada
No answer with 50001 or 50002 port
More info
curl -X GET http://<IP_ADDRESS>:50002/actionList
<?xml version="1.0" encoding="UTF-8"?>
<actionList>
  <action name="register" mode="3" url="http://<IP_ADDRESS>:50002/register"/>
  <action name="getText" url="http://<IP_ADDRESS>:50002/getText"/>
  <action name="sendText" url="http://<IP_ADDRESS>:50002/sendText"/>
  <action name="getContentInformation" url="http://<IP_ADDRESS>:50002/getContentInformation"/>
  <action name="getSystemInformation" url="http://<IP_ADDRESS>:50002/getSystemInformation"/>
  <action name="getRemoteCommandList" url="http://<IP_ADDRESS>:50002/getRemoteCommandList"/>
  <action name="getStatus" url="http://<IP_ADDRESS>:50002/getStatus"/>
  <action name="getHistoryList" url="http://<IP_ADDRESS>:50002/getHistoryList"/>
  <action name="getContentUrl" url="http://<IP_ADDRESS>:50002/getContentUrl"/>
  <action name="sendContentUrl" url="http://<IP_ADDRESS>:50002/sendContentUrl"/>
</actionList>

Thanks.
In the sony TV & SideView app do you have a list of available sources or is it just an input button which cycles the inputs through?
I'd like to see the output of http://<IP_ADDRESS>:50002/getRemoteCommandList
For this URL you need to provide authentication data.
I just checked again and the other command I've requested before is only available on newer devices. I have to do some digging to find out where the inputs are listed. I'm not quite sure if home assistant supports cycling through inputs without a list.
That's now nearly all data I should need :)

In the sony app, I can see a list of available inputs: HDMI 1, HDMI2, TV, Bluetooth and AUDIO
For the command, no output.
My device has no option to setup PSK. Anyway, I tried different options according to sony.conf ("pin": "5125", "psk": null,) and the no output. I am sending all curl request from authorized device (terminal from RPI4 where Home Assistant is running)

The device itself is not authorized. The sony.conf contains the authorization. This header has to be send via curl along with the request. I'll send you a detailed how to later today because I have to try it out with my device.

I will keep trying ...
curl -i -H 'Accept:application/json' -H 'Authorization:Basic OjUxMjU=' -X GET
http://<IP_ADDRESS>:50002/getRemoteCommandList
HTTP/1.1 403 Forbidden
Connection: close
Date: Wed, 09 Oct 2019 08:59:49 GMT
Server: Linux/2.6 Sony-BDV/2.0

Don't worry I'll attach a detailed explanation later today :)

curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' http://<IP_ADDRESS>:50002/getSystemInformation
curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' http://<IP_ADDRESS>:50002/getRemoteCommandList
curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' http://<IP_ADDRESS>:50002/getContentInformation
curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' http://<IP_ADDRESS>:50001/IRCCSCPD.xml

I hope all commands are spelled correctly. I must guess a little because I'm not sure which XML will contain the relevant information.
X-CERS-DEVICE-ID:Bluray Must be replaced with the id you can find in your sony.conf

Many thanks!!! I hope this output makes sense
$ curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray'
http://<IP_ADDRESS>:50002/getSystemInformation
HTTP/1.1 200 OK
Connection: close
Cache-Control: no-cache
Pragma: no-cache
Transfer-Encoding: chunked
Content-Type: text/xml
Date: Wed, 09 Oct 2019 20:51:12 GMT
Server: Linux/2.6 Sony-BDV/2.0

<?xml version="1.0" encoding="UTF-8"?>
<systemInformation>
  <name>BDTheatreSystem</name>
  <generation>2014</generation>
  <remoteType>RM-ADP089</remoteType>
  <remoteType>RM-ADP090</remoteType>
  <remoteType>RM-ADP091</remoteType>
  <remoteType>RM-ADP092</remoteType>
  <remoteType>RM-ADP093</remoteType>
  <remoteType bundled="true">RM-ADP093W</remoteType>
  <remoteType>RM-ADP095</remoteType>
  <remoteType>RM-ADP095W</remoteType>
  <remoteType>RM-ADP096</remoteType>
  <remoteType>RM-ADP097W</remoteType>
  <remoteType>RM-ADP099W</remoteType>
  <remoteType>RM-ADP101</remoteType>
  <remoteType>RM-ADP072</remoteType>
  <remoteType>RM-ADP073</remoteType>
  <remoteType>RM-ADP074</remoteType>
  <remoteType>RM-ADP075</remoteType>
  <remoteType>RM-ADP076</remoteType>
  <remoteType>RM-ADP077</remoteType>
  <actionHeader name="CERS-DEVICE-ID"/>
  <supportContentsClass>
    <class>video</class>
    <class>music</class>
  </supportContentsClass>
  <supportSource>
    <source>BD</source>
    <source>DVD</source>
    <source>CD</source>
    <source>Net</source>
  </supportSource>
  <supportFunction>
    <function name="Notification"/>
    <function name="WOL">
      <functionItem field="MAC" value="10-4f-a8-44-bd-2e"/>
    </function>
  </supportFunction>
</systemInformation>

$ curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' ```
http://<IP_ADDRESS>:50002/getRemoteCommandList
HTTP/1.1 403 Forbidden
Connection: close
Date: Wed, 09 Oct 2019 20:52:16 GMT
Server: Linux/2.6 Sony-BDV/2.0

$ curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' ```
http://<IP_ADDRESS>:50002/getContentInformation
HTTP/1.1 403 Forbidden
Connection: close
Date: Wed, 09 Oct 2019 20:52:47 GMT
Server: Linux/2.6 Sony-BDV/2.0

$ curl -i -H 'Authorization:Basic OjUxMjU=' -H 'X-CERS-DEVICE-ID:Bluray' ```
http://<IP_ADDRESS>:50001/IRCCSCPD.xml
HTTP/1.1 200 OK
Accept-Ranges: none
Content-Length: 1760
Content-Type: text/xml; charset="utf-8"
Last-Modified: Tue, 11 Dec 2018 02:02:05 GMT
Connection: close
Date: Wed, 09 Oct 2019 20:53:06 GMT
Server: Linux/2.6 UPnP/1.0 Sony-BDV/2.0

1
0

X_SendIRCC

IRCCCode
in
X_A_ARG_TYPE_IRCCCode

<action>
  <name>X_GetStatus</name>
  <argumentList>
    <argument>
      <name>CategoryCode</name>
      <direction>in</direction>
      <relatedStateVariable>X_A_ARG_TYPE_Category</relatedStateVariable>
    </argument>
    <argument>
      <name>CurrentStatus</name>
      <direction>out</direction>
      <relatedStateVariable>X_A_ARG_TYPE_CurrentStatus</relatedStateVariable>
    </argument>
    <argument>
      <name>CurrentCommandInfo</name>
      <direction>out</direction>
      <relatedStateVariable>X_A_ARG_TYPE_CurrentCommandInfo</relatedStateVariable>
    </argument>
  </argumentList>
</action>

X_A_ARG_TYPE_IRCCCode
string

X_A_ARG_TYPE_Category
string

X_A_ARG_TYPE_CurrentStatus
string

X_A_ARG_TYPE_CurrentCommandInfo
string

```

Yes thank you for providing the files. I'm still not sure how to read the sources list. Probably it is hidden away in one of these soap services. It might take some time for me to come up with something.

Thanks in advance.
Please let me know if there is anything I can do

I cannot find a way to list the sources with the api version your device has. I'd need a wireshark trace between the app and the device to find out which calls are. Other implementations for the API like this use the API  located at http://<IP_ADDRESS>/sony which you've reported to be non working on your device. You could give the linked repository a try to communicate with your device and list the source. Maybe I am missing something.

I am trying to get that implementation working but I cannot, due to my lack knowledge on programming. I'm if I continue googling I may get it work, but there is a risk to broke something with so many tries and errors.
I downloaded the other API, installed with sudo python setup.py install, but after that I´m unable to configure, so I´m just blocked out and I prefer to ask you.

from SonyAPI import SonyAPI

device = SonyAPI(ip_address='<IP_ADDRESS>', pin=XXXX)
for s in device.source_list():
    print(s)

The script above should do it.

If I copy the text to a sh script and execute it shows the error:
from: can't read /var/mail/SonyAPI
./SonyAPI: línea 5: error sintáctico cerca del elemento inesperado `('
./SonyAPI: línea 5: `device = SonyAPI(ip_address='<IP_ADDRESS>', pin=5125)'

Run it with python 3 and put it next to the SonyApu folder from the linked repository

Something went wrong. It seems related to connection and closed port
pi@Atico-HA:~/SonyAPI-master/SonyAPI $ python sonyapi.py
Traceback (most recent call last):
  File "sonyapi.py", line 3, in <module>
    device = SonyAPI(ip_address='<IP_ADDRESS>', pin=5125)
  File "build/bdist.linux-armv7l/egg/SonyAPI/__init__.py", line 184, in __init__
    self.pin = pin
  File "build/bdist.linux-armv7l/egg/SonyAPI/__init__.py", line 378, in pin
    'Unknown Request Error: ' + traceback.format_exc()
SonyAPI.RegisterError: Unknown Request Error: Traceback (most recent call last):
  File "build/bdist.linux-armv7l/egg/SonyAPI/__init__.py", line 322, in pin
    headers=headers
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='<IP_ADDRESS>', port=80): Max retries exceeded with url: /sony/accessControl (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xb613e190>: Failed to establish a new connection: [Errno 111] Connection refused',))

That's what I expected.
The app somehow uses another channel to get the available inputs.
Without a wireshark trace (or similar) of the communication between your device and the app I' unable help you. Sorry.

I am trying to get a wireshark trace but my network is switched and the router is not managed so I cannot mirror the port where Sony device is connected
I am still thinking on a shortcut

Finally I got it
Here is the trace:
After selection a source, I selected these secuence of POST and Response
No.     Time               Source                Destination           Protocol Length Info
    486 18:59:04,901887552 <IP_ADDRESS>         <IP_ADDRESS>          HTTP/XML 316    POST /upnp/control/AVTransport HTTP/1.1

Frame 486: 316 bytes on wire (2528 bits), 316 bytes captured (2528 bits) on interface 0
Ethernet II, Src: Apple_ec:70:0d (40:9c:28:ec:70:0d), Dst: Sony_44:bd:2e (10:4f:a8:44:bd:2e)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 55600, Dst Port: 52323, Seq: 240, Ack: 1, Len: 250
[2 Reassembled TCP Segments (489 bytes): #485(239), #486(250)]
Hypertext Transfer Protocol
    POST /upnp/control/AVTransport HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): POST /upnp/control/AVTransport HTTP/1.1\r\n]
            [POST /upnp/control/AVTransport HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: POST
        Request URI: /upnp/control/AVTransport
        Request Version: HTTP/1.1
    Host: <IP_ADDRESS>:52323\r\n
    User-Agent: DLNADOC/1.50\r\n
    Content-Type: text/xml; charset="utf-8"\r\n
    SOAPACTION: "urn:schemas-upnp-org:service:AVTransport:1#Stop"\r\n
    Content-Length: 250\r\n
    Connection: close\r\n
    \r\n
    [Full request URI: http://<IP_ADDRESS>:52323/upnp/control/AVTransport]
    [HTTP request 1/1]
    [Response in frame: 488]
    File Data: 250 bytes
eXtensible Markup Language
    <s:Envelope
        s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
        xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
        <s:Body>
            <u:Stop
                xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
                <InstanceID>
                    0
                    </InstanceID>
                </u:Stop>
            </s:Body>
        </s:Envelope>

No.     Time               Source                Destination           Protocol Length Info
    488 18:59:05,234115438 <IP_ADDRESS>          <IP_ADDRESS>         HTTP/XML 327    HTTP/1.1 200 OK

Frame 488: 327 bytes on wire (2616 bits), 327 bytes captured (2616 bits) on interface 0
Ethernet II, Src: Sony_44:bd:2e (10:4f:a8:44:bd:2e), Dst: Apple_ec:70:0d (40:9c:28:ec:70:0d)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 52323, Dst Port: 55600, Seq: 324, Ack: 490, Len: 261
[2 Reassembled TCP Segments (584 bytes): #487(323), #488(261)]
Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Response Version: HTTP/1.1
        Status Code: 200
        [Status Code Description: OK]
        Response Phrase: OK
    Content-Length: 261\r\n
    Content-Type: text/xml; charset="utf-8"\r\n
    EXT: \r\n
    Connection: close\r\n
    Date: Sat, 19 Oct 2019 16:59:04 GMT\r\n
    Server: Linux/2.6 UPnP/1.0 Sony-BDV/2.0\r\n
    X-AV-Server-Info.sony.com: av=5.0; cn="Sony Corporation"; mn="BDV-N7200W"; mv="2.0";\r\n
    X-AV-Physical-Unit-Info.sony.com: pa="BDV-N7200W";\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.332227886 seconds]
    [Request in frame: 486]
    [Request URI: http://<IP_ADDRESS>:52323/upnp/control/AVTransport]
    File Data: 261 bytes
eXtensible Markup Language
    <?xml
        version="1.0"
        ?>
    <s:Envelope
        xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
        s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
        <s:Body>
            <u:StopResponse
                xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
                </u:StopResponse>
            </s:Body>
        </s:Envelope>

No.     Time               Source                Destination           Protocol Length Info
    490 18:59:05,248202435 <IP_ADDRESS>         <IP_ADDRESS>          HTTP/XML 462    POST /upnp/control/AVTransport HTTP/1.1

Frame 490: 462 bytes on wire (3696 bits), 462 bytes captured (3696 bits) on interface 0
Ethernet II, Src: Apple_ec:70:0d (40:9c:28:ec:70:0d), Dst: Sony_44:bd:2e (10:4f:a8:44:bd:2e)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 55601, Dst Port: 52323, Seq: 253, Ack: 1, Len: 396
[2 Reassembled TCP Segments (648 bytes): #489(252), #490(396)]
Hypertext Transfer Protocol
    POST /upnp/control/AVTransport HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): POST /upnp/control/AVTransport HTTP/1.1\r\n]
            [POST /upnp/control/AVTransport HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: POST
        Request URI: /upnp/control/AVTransport
        Request Version: HTTP/1.1
    Host: <IP_ADDRESS>:52323\r\n
    User-Agent: DLNADOC/1.50\r\n
    Content-Type: text/xml; charset="utf-8"\r\n
    SOAPACTION: "urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"\r\n
    Content-Length: 396\r\n
    Connection: close\r\n
    \r\n
    [Full request URI: http://<IP_ADDRESS>:52323/upnp/control/AVTransport]
    [HTTP request 1/1]
    [Response in frame: 495]
    File Data: 396 bytes
eXtensible Markup Language
    <s:Envelope
        s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
        xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
        <s:Body>
            <u:SetAVTransportURI
                xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
                <InstanceID>
                    0
                    </InstanceID>
                <CurrentURI>
                    local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_07_7_23_0_0
                    </CurrentURI>
                <CurrentURIMetaData>
                    </CurrentURIMetaData>
                </u:SetAVTransportURI>
            </s:Body>
        </s:Envelope>

No.     Time               Source                Destination           Protocol Length Info
    492 18:59:05,336434746 <IP_ADDRESS>          <IP_ADDRESS>         HTTP/XML 414    NOTIFY / HTTP/1.1

Frame 492: 414 bytes on wire (3312 bits), 414 bytes captured (3312 bits) on interface 0
Ethernet II, Src: Sony_44:bd:2e (10:4f:a8:44:bd:2e), Dst: Apple_ec:70:0d (40:9c:28:ec:70:0d)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 39656, Dst Port: 55567, Seq: 385, Ack: 1, Len: 348
[2 Reassembled TCP Segments (732 bytes): #491(384), #492(348)]
Hypertext Transfer Protocol
    NOTIFY / HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): NOTIFY / HTTP/1.1\r\n]
            [NOTIFY / HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: NOTIFY
        Request URI: /
        Request Version: HTTP/1.1
    Host: <IP_ADDRESS>:55567\r\n
    Date: Sat, 19 Oct 2019 16:59:04 GMT\r\n
    User-Agent: UPnP/1.0\r\n
    X-AV-Physical-Unit-Info: pa="BDV-N7200W";\r\n
    X-AV-Server-Info: av=5.0; cn="Sony Corporation"; mn="BDV-N7200W"; mv="2.0";\r\n
    Content-Type: text/xml; charset="utf-8"\r\n
    Content-Length: 348\r\n
    NT: upnp:event\r\n
    NTS: upnp:propchange\r\n
    SID: uuid:5e2036e0-f291-11e9-8000-104fa844bd2e\r\n
    SEQ: 43\r\n
    \r\n
    [Full request URI: http://<IP_ADDRESS>:55567/]
    File Data: 348 bytes
eXtensible Markup Language
    <?xml
        version="1.0"
        ?>
    <e:propertyset
        xmlns:e="urn:schemas-upnp-org:event-1-0">
        <e:property>
            <LastChange>
                <Event xmlns="urn:schemas-upnp-org:metadata-1-0/AVT/">\n  <InstanceID val="0">\n    <TransportState val="TRANSITIONING"/>\n  </InstanceID>\n</Event>\n
                </LastChange>
            </e:property>
        </e:propertyset>

No.     Time               Source                Destination           Protocol Length Info
    493 18:59:05,341411260 <IP_ADDRESS>         <IP_ADDRESS>          HTTP     104    HTTP/1.1 200 OK

Frame 493: 104 bytes on wire (832 bits), 104 bytes captured (832 bits) on interface 0
Ethernet II, Src: Apple_ec:70:0d (40:9c:28:ec:70:0d), Dst: Sony_44:bd:2e (10:4f:a8:44:bd:2e)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 55567, Dst Port: 39656, Seq: 1, Ack: 733, Len: 38
Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Response Version: HTTP/1.1
        Status Code: 200
        [Status Code Description: OK]
        Response Phrase: OK
    Content-Length: 0\r\n
    \r\n
    [HTTP response 1/1]
    [Request URI: http://<IP_ADDRESS>:55567/]

No.     Time               Source                Destination           Protocol Length Info
    495 18:59:06,397620373 <IP_ADDRESS>          <IP_ADDRESS>         HTTP/XML 353    HTTP/1.1 200 OK

Frame 495: 353 bytes on wire (2824 bits), 353 bytes captured (2824 bits) on interface 0
Ethernet II, Src: Sony_44:bd:2e (10:4f:a8:44:bd:2e), Dst: Apple_ec:70:0d (40:9c:28:ec:70:0d)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 52323, Dst Port: 55601, Seq: 324, Ack: 649, Len: 287
[2 Reassembled TCP Segments (610 bytes): #494(323), #495(287)]
Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Response Version: HTTP/1.1
        Status Code: 200
        [Status Code Description: OK]
        Response Phrase: OK
    Content-Length: 287\r\n
    Content-Type: text/xml; charset="utf-8"\r\n
    EXT: \r\n
    Connection: close\r\n
    Date: Sat, 19 Oct 2019 16:59:05 GMT\r\n
    Server: Linux/2.6 UPnP/1.0 Sony-BDV/2.0\r\n
    X-AV-Server-Info.sony.com: av=5.0; cn="Sony Corporation"; mn="BDV-N7200W"; mv="2.0";\r\n
    X-AV-Physical-Unit-Info.sony.com: pa="BDV-N7200W";\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 1.149417938 seconds]
    [Request in frame: 490]
    [Request URI: http://<IP_ADDRESS>:52323/upnp/control/AVTransport]
    File Data: 287 bytes
eXtensible Markup Language
    <?xml
        version="1.0"
        ?>
    <s:Envelope
        xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
        s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
        <s:Body>
            <u:SetAVTransportURIResponse
                xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
                </u:SetAVTransportURIResponse>
            </s:Body>
        </s:Envelope>

No.     Time               Source                Destination           Protocol Length Info
    497 18:59:07,435817788 <IP_ADDRESS>          <IP_ADDRESS>         HTTP/XML 456    NOTIFY / HTTP/1.1

Frame 497: 456 bytes on wire (3648 bits), 456 bytes captured (3648 bits) on interface 0
Ethernet II, Src: Sony_44:bd:2e (10:4f:a8:44:bd:2e), Dst: Apple_ec:70:0d (40:9c:28:ec:70:0d)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 39657, Dst Port: 55567, Seq: 385, Ack: 1, Len: 390
[2 Reassembled TCP Segments (774 bytes): #496(384), #497(390)]
Hypertext Transfer Protocol
    NOTIFY / HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): NOTIFY / HTTP/1.1\r\n]
            [NOTIFY / HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: NOTIFY
        Request URI: /
        Request Version: HTTP/1.1
    Host: <IP_ADDRESS>:55567\r\n
    Date: Sat, 19 Oct 2019 16:59:04 GMT\r\n
    User-Agent: UPnP/1.0\r\n
    X-AV-Physical-Unit-Info: pa="BDV-N7200W";\r\n
    X-AV-Server-Info: av=5.0; cn="Sony Corporation"; mn="BDV-N7200W"; mv="2.0";\r\n
    Content-Type: text/xml; charset="utf-8"\r\n
    Content-Length: 390\r\n
    NT: upnp:event\r\n
    NTS: upnp:propchange\r\n
    SID: uuid:5e2036e0-f291-11e9-8000-104fa844bd2e\r\n
    SEQ: 44\r\n
    \r\n
    [Full request URI: http://<IP_ADDRESS>:55567/]
    File Data: 390 bytes
eXtensible Markup Language
    <?xml
        version="1.0"
        ?>
    <e:propertyset
        xmlns:e="urn:schemas-upnp-org:event-1-0">
        <e:property>
            <LastChange>
                 [truncated]<Event xmlns="urn:schemas-upnp-org:metadata-1-0/AVT/">\n  <InstanceID val="0">\n    <CurrentTrackURI val="local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_07_7_23_0_0"/>\n  </Instan
                </LastChange>
            </e:property>
        </e:propertyset>

No.     Time               Source                Destination           Protocol Length Info
    498 18:59:07,447868295 <IP_ADDRESS>         <IP_ADDRESS>          HTTP     104    HTTP/1.1 200 OK

Frame 498: 104 bytes on wire (832 bits), 104 bytes captured (832 bits) on interface 0
Ethernet II, Src: Apple_ec:70:0d (40:9c:28:ec:70:0d), Dst: Sony_44:bd:2e (10:4f:a8:44:bd:2e)
Internet Protocol Version 4, Src: <IP_ADDRESS>, Dst: <IP_ADDRESS>
Transmission Control Protocol, Src Port: 55567, Dst Port: 39657, Seq: 1, Ack: 775, Len: 38
Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Response Version: HTTP/1.1
        Status Code: 200
        [Status Code Description: OK]
        Response Phrase: OK
    Content-Length: 0\r\n
    \r\n
    [HTTP response 1/1]
    [Request URI: http://<IP_ADDRESS>:55567/]

I think you are right about the local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_07_7_23_0_0 everthing else does not really hint at sources or inputs.

Do you remember between which sources you switched?

Hey @alexmohr,
I was follwoing this topic for the last few days and making some research, so I just continued from where patrul stopped,
Then using the Wireshark I managed to find how the TV and Video app send the HDMI1/2 command to the device.
I have sent them direct to the device using a Rest client and it worked.
So the thing is that I could not really find a way to find the list it self, so I believe that would be kind of possible to hardcode the input or something, since it may be a standard for some models (I have the same code local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_07_7_23_0_0 as patrul which hefers to HDMI2).
Unfortunatly it is not so simple as the IRCC commands but you may know how to implement it on the API.
Just one thing, I am not an expert on any programming language :)
Hopefully we can try something together here.

First of all unfortunately I don't have a lot of time to work at this at the moment so progress will be slow if any.
I'm not a huge fan of hard coding these things but we could solve it with a configuration option and use the values you've provided as default.
Please post the requests you've made in your REST client. With these it should be more or less easy to implement the source selection.

That is fine, thanks for the help.
So, from my understanding of this function, just made in steps below,
STEP 1
TVsideview apps sends a POST request http://<IP_ADDRESS>:64321/upnp/control/ContentDirectory to get ContentDirectory:
POST /upnp/control/ContentDirectory HTTP/1.1
Host: <IP_ADDRESS>:64321
Content-Type: text/xml; charset="utf-8"
SOAPAction: "urn:schemas-upnp-org:service:ContentDirectory:1#GetSystemUpdateID"

<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:GetSystemUpdateID xmlns:u="urn:schemas-upnp-org:service:ContentDirectory:1">
</u:GetSystemUpdateID>
</s:Body>
</s:Envelope>

Then the device returns an ID "2459". Not really sure why but it may be some reference for the input addresses.
MAybe this can be skiped for now...
STEP 2
This is something I did not find througt the app but it is necessary sending the device to the HOME page, since step 3 just works if you are on it. This can be done with already implemented IRCC command HOME
STEP 3
The APP sends kind of a input selection to /52323/upnp/control/AVTransport
POST /upnp/control/AVTransport HTTP/1.1
Host: <IP_ADDRESS>:52323
Content-Type: text/xml; charset="utf-8"
SOAPAction: "urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"

<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
        <s:Body>
            <u:SetAVTransportURI
                xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
                <InstanceID>
                    0
                    </InstanceID>
                <CurrentURI>
                    local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_06_6_23_0_0
                    </CurrentURI>
                <CurrentURIMetaData>
                     [truncated]<DIDL-Lite xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:metadata-1-0/upnp/" xmlns:dlna="urn:schemas-dln
                    </CurrentURIMetaData>
                </u:SetAVTransportURI>
            </s:Body>
        </s:Envelope>

Where local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_06_6_23_0_0 means select HDMI1 and local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_07_7_23_0_0 means select HDMI2.
I have not tested the other sources yet.
STEP 4
The app sends a command to actually change to the selected source as you can see below.
POST /upnp/control/AVTransport HTTP/1.1
Host: <IP_ADDRESS>:52323
Content-Type: text/xml; charset="utf-8"
SOAPAction: "urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"

<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:Play xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
<InstanceID>0</InstanceID>
<Speed>1</Speed>
</u:Play>
</s:Body>
</s:Envelope>

Then from here we get the source change.
I just found these things by trying and reading a bit around. So I cant give much technical details
I also made some changes on my local API device.py file in order to get this to work, but as I mentioned before, I'm not the programing guy and don't know how to add and reference these addresses into a config file so I hardcoded the addresses and for sure the code is not optimized. So far it is working for me, maybe it may save some time if you want to implement it properly.
    def input_hdmi1(self):
        """Send HDMI input selection to the connected device"""

        self.home()

        time.sleep(1)

        data = """<u:SetAVTransportURI xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
            <InstanceID>0</InstanceID>
            <CurrentURI>local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_06_6_23_0_0</CurrentURI>
            <CurrentURIMetaData>[truncated]<DIDL-Lite xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:metadata-1-0/upnp/" xmlns:dlna="urn:schemas-dln</CurrentURIMetaData>
            </u:SetAVTransportURI>"""

        action = "urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"

        content = self._post_soap_request(
            url=self.av_transport_url, params=data, action=action)

        self.input_play()

        return "HDMI 1"

    def input_hdmi2(self):
        """Send HDMI input selection to the connected device"""

        self.home()

        time.sleep(1)

        data = """<u:SetAVTransportURI xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
            <InstanceID>0</InstanceID>
            <CurrentURI>local://<IP_ADDRESS>:60151/I_14_02_0_-1_00_07_7_23_0_0</CurrentURI>
            <CurrentURIMetaData>[truncated]<DIDL-Lite xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:metadata-1-0/upnp/" xmlns:dlna="urn:schemas-dln</CurrentURIMetaData>
            </u:SetAVTransportURI>"""

        action = "urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"

        content = self._post_soap_request(
            url=self.av_transport_url, params=data, action=action)

        self.input_play()

        return "HDMI 2"

    def input_play(self):
        """Send input select to the connected device"""
        data = """<u:Play xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">
            <InstanceID>0</InstanceID>
            <Speed>1</Speed>
            </u:Play>"""

        action = "urn:schemas-upnp-org:service:AVTransport:1#Play"

        content = self._post_soap_request(
            url=self.av_transport_url, params=data, action=action)

Just need to find out how to implement it on home assistant component.

Thanks that looks amazing. This should be enough to implement the source selection. Maybe we could find out how the urls for the source are generated if we can get them for other inputs too.  I currently have access to a TV which supports this API aswell so maybe I can find out something about the url creation.

I registered my bluray again and tried to use the wireshark to follow the all communication. Unfortunately I could no find  anything, maybe because of lack of knowledge. :)
Anyway, I made a hard coded list (as I know what my device supports) and managed to add the source function on home assistant component.
Once we find a way to find the url and input capabilities, then it should be ok to implement them.
So, so far I have my Bluray integrated with my HA and managed to make some automation to change the source. It is working fine.

Sorry for the late reply, I am very busy writing my thesis at the moment. Just wanted to let you know that you are not forgotten and I'll work on integrating your proposal in october.
If you're up to making to a pull request I'd be happy to accept it.

No problem,
We all have  things to do.
I will try to make a PR for both API and HA component along this week.
Thanks.

@gohlas Did you make any progress yet you want to share? Even if it's not done yet you could open the PR and I'll take over from there.

Sorry, totally forgot about it with busy days. I will make the PR as soon as possible.

Available input sources can be acquired via DLNA (which is currently not implemented).
https://github.com/KHerron/SonyAPILib/blob/444d315a8f869127646383c0a93ab22e59355797/SonyAPILib/SonyAPILib/sonyAPILib.cs#L2467


access violation when message send to wrong partition

Hi
I faced an issue when 3 things are happening:

i'm supplying headers for each message
i'm sending to non existing partition
before sending second message i'm waiting for CB to be triggered for first message

Please find a code sample that reproduce error below, it's a 'example_KafkaProducer_EnableManualEventsPoll.cc' from examples with small additions.
failing_example.txt
The result is when second send method is called, it fails in KafkaProducer.h #393 with error:
Exception thrown: read access violation
it was 0XFFFFFFFFFFFFFF
My environment is Windows 10 x64, VS 2019
Thanks in advance

Please be aware of the memory lifetime management, -- if you're not quite sure about it, just use SendOption::ToCopyRecordValue as the option for send(...)

i tried copy option too. Actually i just took example from this  library and provided headers, no changes of example logics was done


Fix new flow errors

@keybase/react-hackers
This updates our flow version and fixes new errors.
There is a weird bug with implied true values in jsx filed here: https://github.com/facebook/flow/issues/1906

👍

retest this please

(That was directed at Jenkins!)

retest this please

:ship:


Timeouts and proper way of retrieving multiple types of nodes?

Hi, I'm building a simple scraper to check an internal project, but am fairly new to Go/chromedp. First of all, here's a sample block of code:
    var h3Tags []*cdp.Node
    var h3Contents []string
    newCtxt, cancel := chromedp.NewContext(ctxt)
    err = chromedp.Run(newCtxt, chromedp.Nodes(`h3`, &h3Tags, chromedp.ByQueryAll))
    totalh3 := len(h3Tags)

    if err != nil {
        log.Println("Could not retrieve any H3 tags.")
        pageStats["has_h3"] = false
    } else if totalh3 > 0 {
        log.Println("Found " + strconv.Itoa(totalh3) + " H3 tag(s).")
        pageStats["has_h3"] = true
    }

    pageStats["total_h3"] = totalh3
    pageStats["h3_contents"] = h3Contents

    // check if any H1 tags are present
    var h1Tags []*cdp.Node
    var h1Contents []string
    newCtxt,  = chromedp.NewContext(ctxt)
    err = chromedp.Run(newCtxt, chromedp.Nodes(`h1`, &h1Tags, chromedp.ByQueryAll))
    totalh1 := len(h1Tags)

    if err != nil {
        log.Println("Could not retrieve any H1 tags.")
        pageStats["has_h1"] = false
    } else if totalh1 > 0 {
        log.Println("Found " + strconv.Itoa(totalh1) + " H1 tag(s).")
        pageStats["has_h1"] = true
    }

My main issue at the moment is, if it doesn't find any H3 tags, it just keeps running until the context timeout. I guess this is fine, but if that happens, any other chromedp commands after that automatically fail (context deadline exceeded).
What is the proper way of making it so it "resets" the context deadline with each step, so the evaluation of one item does not affect the rest below it? I'm not sure if I'm doing this correctly. Would appreciate some guidance.
Cheers!

Please see https://github.com/chromedp/chromedp/issues/379; because of how contexts work, it's possible to only add a timeout or cancel one of the sub-tasks, and not an entire tab/browser.

Please see #379; because of how contexts work, it's possible to only add a timeout or cancel one of the sub-tasks, and not an entire tab/browser.

Excellent! This is exactly what I needed. I didn't realize it was possible to just create a child context and set a deadline there. Thanks for the quick response!

We made use of contexts partly for this reason. It makes the API simpler and nicer to use, but only if you're used to the context package. So it's both good and bad.


Option to disable scrolling simulation of the scroll view when its content dimension is changed

Hi Team,
I have prepared a simple application with a scrollable container in the horizontal direction. The scrolling simulation is applied when I changed the content dimension of the scrollable container after the scroll position is reached to the maxScrollExtend position. This is the default behavior in flutter scroll view widget. But, I need to restrict the scrolling simulation.
Please, let me know the workaround of how to restrict the scrolling simulation or how to customize the scrolling without the simulation behavior.
Steps to Reproduce

Run the below simple sample.
Scroll fully in the horizontal direction manually and reach the end.
Tap the floating button to change the size at runtime.

Code snippets:

class ScrollingSimulationDemo extends StatefulWidget {
  ScrollingSimulationDemo({Key? key}) : super(key: key);

  @override
  _ScrollingSimulationDemoState createState() => _ScrollingSimulationDemoState();
}

class _ScrollingSimulationDemoState extends State<ScrollingSimulationDemo> {
  double width = 500;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: SingleChildScrollView(
        scrollDirection: Axis.horizontal,
        child: Container(
          color: Colors.lightBlue[100],
          height: 350,
          width: width,
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () {
          width = 250.0;
          setState(() {});
        },
      ),
    );
  }
}

Expected results: The scroll view content dimension should be changed without the simulation effect.
Actual results: The scrolling simulation is applied to the scroll view content when it changed at runtime.

Flutter Doctor
[√] Flutter (Channel stable, 2.2.3, on Microsoft Windows [Version 10.0.18363.1440], locale en-US)
    • Flutter version 2.2.3 at C:\src\flutter
    • Framework revision f4abaa0735 (9 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\AshokKuvaraja\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop for Windows (Visual Studio Professional 2019 16.4.5)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional
    • Visual Studio Professional 2019 version 16.4.29806.167
    • Windows 10 SDK version 10.0.18362.0

[√] Android Studio (version 3.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 34.0.1
    • Dart plugin version 182.5215
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[√] VS Code, 64-bit edition (version 1.58.0)
    • VS Code at C:\Program Files\Microsoft VS Code
    • Flutter extension version 3.25.0

[√] Connected device (2 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.18363.1440]
    • Chrome (web)      • chrome  • web-javascript • Google Chrome 92.0.4515.159

Hi @ashok-kuvaraja
I don't think it is possible to remove this as of right now, add physics: NeverScrollableScrollPhysics() would remove all scrolling physics including when the dimension is changed, that's probably not what you want

Hi @TahaTesser ,
Thanks for your rapid response,
I ensured the same behavior with the NeverScrollableScrollPhysics. But, The simulation is applied to the scroll view even if use NeverScrollableScrollPhysics.
Regards,
Ashok K

/cc @Piinks

Hey @ashok-kuvaraja it looks like the animation you are seeing is the scroll position animating in response to the new, smaller size. Since it is positioned at the end of the larger sized content, when it becomes smaller it moves to be at the correct end position of the new content. It might be possible to configure this to jump to the new position instead of animating if someone wanted to do that.

Hi @Piinks,
Thanks for the update.
You said that the behavior might be possible. Thus, I tried this by creating a custom simulation and custom scroll position. Unfortunately, I couldn't able to achieve this from my side. I am stuck with achieving this. Can you please let me know any workaround or any suggestion for achieving this?
Regards,
Ashok K

Hi, Any update on this?


Added documentation

Changes:

Added report by @ishika1727
YouTube Video by @ishika1727

@ishika1727 Please review this. I have improved documentation.md a bit.

The changes look great 👍  @vinitshahdeo

@Kashish121 Changes look good to you?

Waiting for your approval!

@Kashish121 Review now, updated the PR.


AsyncStorage Couldn't read row 0, col 0 from CursorWindow

This issue was originally created by @JAStanton as facebook/react-native#12529.

Description
When reading values that are too large using AsyncStorage on Android I run into this error:
I wrote a large value with AsyncStorage and later tried to read it, but it crashed.
BaseError: Couldn't read row 0, col 0 from CursorWindow.  Make sure the Cursor is initialized correctly before accessing data from it.
  at e (/node_modules/react-native/Libraries/Storage/AsyncStorage.js:423:22)
  at None (/node_modules/react-native/Libraries/Storage/AsyncStorage.js:416:71)
  at map ([native code])
  at errors (/node_modules/react-native/Libraries/Storage/AsyncStorage.js:416:51)
  at slicedToArray (/node_modules/react-native/Libraries/Storage/AsyncStorage.js:54:33)

Reproduction
It looks like on iOS we write to disk when the value is above a threshold and on Android we write to sqllite always.
let x = 'x';
while (x.length <= 3194304) {
  x = `${x}${x}`;
}
await AsyncStorage.setItem('@@GARBAGE@@', x); // works
const garbage = await AsyncStorage.getItem('@@GARBAGE@@'); // throws

Solution
In order for me to fix this I'll need to re-write AsyncStorage to write to disk instead of writing to sql. https://github.com/mvayngrib/react-native-safe-async-storage <--- this guys has the right idea but his code is super old and broken.
Additional Information

React Native version: 0.39.2
Platform: Android
Operating System: Samsung Galaxy S4

Workaround:
async getSafeItem(key) {
  return new Promise((resolve, reject) => {
    if (!key) {
      reject('Invalid key');
    }
    AsyncStorage.getItem(createKey(key)).then((value) => {
      resolve(value);
    }, () => {// Couldn't read row 0, col 0 from CursorWindow
      resolve(null); // Force not to break
    });
  });
}

@Krizzu why API: AsyncStorage label was removed if it is related?

@juliopiubello 	Because this whole module is about AS. Since everything is related I just remove redundant labels

@juliopiubello were you able to solve it? I'm having the same issue

According to this comment on SO, CursorWindow size limit  is 2MB per row (I couldn't find official docs saying that but let's assume it for now).
Repro steps by OP gives us a 4194304 length string. According to ECMAScript specification, each character takes 2 bytes of memory, so whole is something around 8.38 MB.
This is way over the limit of CursorWindow, hence the crash.
Now, to work around this:

Try not to store your whole object as a single entity (by stringifying objects) - divide it into smaller pieces, then store in at different AS keys

Write a helper that will save large data into filesystem, instead to AS

thanks.

2MB seems to be correct. CursorWindow's default constructor just pulls the number from the internal configuration.
@Krizzu Would it make sense to have a warning here in dev mode to make people more aware of the limitation?

@tido64 You mean like checking for string length before calling Native Module? We could do it, I can imagine those JSON strings parsed all the time :)

If you need more help with this particular issue, please do let us know.

Try not to store your whole object as a single entity (by stringifying objects) - divide it into smaller pieces, then store in at different AS keys

https://github.com/react-native-community/async-storage/pull/99
will this update help me to over the limit of CursorWindow?


JDBCToGCS Template timestamp datatype are wrongly converted to integer in avro files

JDBCToGCS Template (can be true for other templates too)
Timestamp datatype are wrongly converted to integer in avro files
Replication steps:

Create sample data in MySQL with timestamp values.
Run template to export data.
Use BigQuery to read the generated avro files.

@shashank-google tried to recreate the scenario. I can verify the Avro files created by the templates is having appropriate datatypes as per the create table DDL:

Create Table DDL:
CREATE TABLE employee
(
   EmpCode      INT(4),
   EmpFName     VARCHAR(15),
   EmpLName     VARCHAR(15),
   Job          VARCHAR(45),
   SCE      CHAR(4),
   HireDate     DATE,
   Salary       INT(6),
   Commission   BIGINT(8),
   DEPTCODE     BIGINT(8),
   t1 TIMESTAMP
);

Working on the # 3 reading generated Avro files at BQ. Will update the thread soon.

@shashank-google Trying to replicate the issue, the behaviour of converting the timestamp & date type fields to INTEGER while impoorting directly to BQ is expected. This is because BQ does not support the AVRO logical data types by default, until specified explicitly.
ref. https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#logical_types
As mentioned unless we explicitly set the value of use_avro_logical_types or useAvroLogicalTypes (while using bq command line tool), the conversion from Avro logical types to their corresponding BigQuery data types won't happen.
Alternatively, you can also create an external table by setting above property as follows:
CREATE EXTERNAL TABLE test.test_avro_ext_logical_types
OPTIONS (
  format = 'AVRO',
  uris = ['gs://test/out_jdbc/*.avro']
  enable_logical_types = TRUE
)

This does not hold true for other templates loading data to BQ from Avro files.
The explicit conversion of Avro logical types to BigQuery data types is not required as BQ takes schema details from the Spark DataFrame directly without referring the underlying Avro files metadata.
Batch ID: 6f925bcc5d0f48fca5d335279ad21595 (testing Avro to BQ, having date and timestamp fields)

Interesting!
Context: I spotted this issue in exporting data from MySql table into avro and then importing into Cloud Spanner.
So what you seem to be saying that there is no issue in generating the avro files. But mainly when importing / loading data in target system like Spanner. Also please test the same for GCSToJDBC template as well.
Can you please figure out solution so that Spanner or JDBC schema is generated with Timestamp column instead of Integer.

@shashank-google The type conversion from date/ timestamp to integer is bound to happen with BigQuery only, if we do not specify explicitly to use logical types.
This would not be the case for GCStoJDBC or GCStoSpanner templates. To verify the same I ran the respective templates and found the data & timestamp fields inserting correctly and retaining their datatypes.
Batch ID: 1f779a4bd80a474e93edfe87792b206c [GCStoSpanner]
Batch ID: 9540c182720245a584c5cc2d04704e6c [GCStoSpanner]
Output Table: avrodemo


Client ID description misleading?

Are we sure the Client ID has anything to do with the ones you can generate in the Console, and not localist entirely to Google Ads? Per the piece of code describing client_id, it is the API Client ID that you have to generate once. This is also repeated in README.MD.
https://github.com/adswerve/GA4-Measurement-Protocol-Python/blob/c42cb0f62be6d7fea2f96e880559e513d7707672/ga4mp/ga4mp.py#L32-L33
I'm not entirely sure that is right however.
GA v4 [0] describes client_id as "Uniquely identifies a user instance of a web client", in contrast to user_id which is "A unique identifier for a user". So it's identifying the user + device combination.
GA v3 [1] describes cid as "This field is required if User ID (uid) is not specified in the request. This pseudonymously identifies a particular user, device, or browser instance. For the web, this is generally stored as a first-party cookie with a two-year expiration. For mobile apps, this is randomly generated for each particular instance of an application install. The value of this field should be a random UUID...", which echoes the definition above, also greatly expanding it.
From both of those definitions, the Client ID field is a different thing to Google Console API Client ID. In fact, the wizard for creating the API keys does not even have options for Ads/Analytics/GA, or other similar keywords I tried. Providing a random value in the field would work, but it wouldn't be helpful for collating and aggregating the data as GA would be attempting to.
Mind you, I have roughly zero expectations in terms of accuracy or up-to-date-ness when it comes to Google documenting their APIs and products, so this could all well be completely wrong.
[0] https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=gtag#payload
[1] https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters
UPDATE: Found some documentation further describing the expectations for the client_id field: https://developers.google.com/analytics/devguides/collection/protocol/ga4/verify-implementation?client_type=gtag#client_send

You're correct. The previous version's description of what was expected for the client_id field was incorrect. This has been fixed in the newest major update.


compute_mean_contributions_over_time not found

From: https://discourse.pymc.io/t/pymc-marketing-example-method-not-present-in-package/12017
am trying to reproduce the analysis done in the Example notebook 1 for the new Pymc-marketing package.
However, some of the methods used seem to not be present in the package. in particular the DelayedSaturatedMMM.compute_mean_contributions_over_time, is used in the example, it is listed in the docs, but is not present in the actual package.
when running
mmm.compute_mean_contributions_over_time()

returns
AttributeError: 'DelayedSaturatedMMM' object has no attribute 'compute_mean_contributions_over_time'

This is not a bug, but rather a new method which is available in the main branch but not in the latest release ;l)


WIP: [MSINTEROP-720] Add functionality to parse services file

Adds a class method from_file to the Database.Services class that is used to parse the "services" file and ensure the services defined in the file are populated in the database.

Also adds a queries column to the database to hold a JSON blob of queries defined in the file. This is used to match queries to the defined services.

Adds verify_queries to ensure only blocker and critical-blocker queries will be defined by users. New unit tests for this are added.
Adds documentation for populating the "services" file.

/wip

Report bugs in Issues
The following are automatically added:

Add reviewers from OWNER file (in the root of the repository) under reviewers section.
Set PR size label.
New issue is created for the PR. (Closed when PR is merged/closed)
Run pre-commit if .pre-commit-config.yaml exists in the repo.

Available user actions:

To mark PR as WIP comment /wip to the PR, To remove it from the PR comment /wip cancel to the PR.
To block merging of PR comment /hold, To un-block merging of PR comment /hold cancel.
To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
verified label removed on each new commit push.
To cherry pick a merged PR comment /cherry-pick <target branch to cherry-pick to> in the PR.

Multiple target branches can be cherry-picked, separated by spaces. (/cherry-pick branch1 branch2)
Cherry-pick will be started when PR is merged

To build and push container image command /build-and-push-container in the PR (tag will be the PR number).
To add a label by comment use /<label name>, to remove, use /<label name> cancel
To assign reviewers based on OWNERS file use /assign-reviewers
To check if PR can be merged use /check-can-merge

Supported /retest check runs

Supported labels

hold
verified
wip
lgtm

/wip cancel

/wip cancel

/verified

/verified

/verified


added custom recorder type

custom recorder type
+new property value: recorder.type=CUSTOM
+new property: recorder.class  //fully-qualified name of a class that extends VideoRecorder

@githubCast could you describe which recorder might be used as custom?

Hi @SergeyPirogov
It can be any tool that extends com.automation.remarks.video.recorder.VideoRecorder.
I'm going to use it with our internal recorder.
For smoke tests your implementations of VideoRecorder can be used: MonteRecorder, WindowsFFmpegRecorder ...
video.properties:
recorder.type=CUSTOM
recorder.class=com.automation.remarks.video.recorder.ffmpeg.WindowsFFmpegRecorder


Add Huanglong spear skill2

This is a Draft PR test (a.k.a WIP PR pattern)
I will clean-up those commits and logs later.
※ Draft PR のテストのため、自分のレポジトリ宛てPR

#1
#2
see also Project Board
https://github.com/kei-gbf/motocal/projects/1

TODO: メイン装備時のみ

テスト終わり。upstreamにPR済


[MM-9685] Update channel header

Summary
Update channel header based on:

UX Design: 4.8 v3 - https://projects.invisionapp.com/share/2JFXM7453AF#/screens/280506516
follow up discussion at https://pre-release.mattermost.com/core/pl/y7x4in1nxpfzpkcxjoa6g5q49c

Ticket Link
Jira ticket: MM-9685
Checklist

[x] Ran make check-style to check for style errors (required for all pull requests)
[x] Ran make test to ensure unit and component tests passed
[x] Added or updated unit tests (required for all new features)
[x] Has UI changes
[x] Includes text changes and localization file (.../i18n/en.json) updates

Setup Test Server label detected. Spinmint test server created if build succeeds (checks pass and no conflicts with base branch).

Spinmint test server created at: http://i-040e9a050193cf0bb.spinmint.com
Test Admin Account: Email<EMAIL_ADDRESS>| Password: user-0
Test User Account: Email<EMAIL_ADDRESS>| Password: user-1
Instance ID: i-040e9a050193cf0bb

Spinmint test server destroyed

Setup Test Server label detected. Spinmint test server created if build succeeds (checks pass and no conflicts with base branch).

Spinmint test server created at: http://i-0176cf15e2c7995a7.spinmint.com
Test Admin Account: Email<EMAIL_ADDRESS>| Password: user-0
Test User Account: Email<EMAIL_ADDRESS>| Password: user-1
Instance ID: i-0176cf15e2c7995a7

Spinmint test server destroyed

Setup Test Server label detected. Spinmint test server created if build succeeds (checks pass and no conflicts with base branch).

Spinmint test server created at: http://i-0a5376c188dc8e7c2.spinmint.com
Test Admin Account: Email<EMAIL_ADDRESS>| Password: user-0
Test User Account: Email<EMAIL_ADDRESS>| Password: user-1
Instance ID: i-0a5376c188dc8e7c2

Spinmint test server destroyed


[Feature]: Improve Gallery Design and Implement Automatic Transitions in Gallery Section of About us

🚀 Feature Request
📝 Feature Description
The current Gallery section on the About Us page lacks visual appeal and does not have automatic transitions or slides for the card elements. Enhancing this section will improve user engagement and provide a better overall experience.
🔧 How Will This Enhance the App?

Enhance Aesthetics:

Update the styling of the Gallery section to make it visually appealing. Consider using a card layout with images, titles, and descriptions that fit the theme of the website.
Implement hover effects to provide feedback to users when they interact with the cards.

Add Automatic Transitions:

Introduce functionality for automatic transitions or slides between the gallery cards at a defined interval (e.g., every 3 seconds).
Allow users to pause the automatic transitions on hover, and provide navigation controls (e.g., next and previous buttons).

https://github.com/user-attachments/assets/aabc020b-6345-4a96-9d51-e55d7b949c87

@Vin205 hey can you help setting it up on my local.

these errors showed up when I tried setting it up.
I tried npm install and npm start in frontend dir

@Vin205 please reply, as I'm ready to work on the issue.


[10.x] Adds FormRequest@getRules() method

What
A way to override how the rules for a form request are retrieved without having to touch the createDefaultValidator() function.
Why?
 I don't know why that would be a big deal, but maybe you don't.
For my team, we are reimagining our API design so that multiple user roles can access common endpoints (/transactions, /media, etc). However, the list transactions endpoint for one user type will have a totally different set of filters than those of another user type. (For instance, admin can search across all user's transactions but filter by sending bank, whereas that option wouldn't be visible for the average user).
class ListTransactionsRequest extends FormRequest
{
    public function rules(): array
    {
        if ($this->user()->role == 'admin') {
            return [
                'sender_bank_id' => ['nullable', 'int'],
                'user_status' => ['nullable', Rule::in(['deliquent', 'active', 'expired'])],
                // ...
            ];
        }
        else if ($this->user()->role == 'payer') {
            return [
                'date' => ['date:Y-m-d'],
                // ...
            ];
        }
        else if ($this->user()->role == 'user') {
            // ...
        }
    }
}

We have hundreds of requests in our application, so this pattern is tricky and to be avoided.  For us, it's a lot easier to do something like this:
class ListTransactionsRequest extends FormRequest
{
    protected function getRules(): array
    {
        return match($this->user()->role) {
            'payer' => $this->rulesForPayer(),
            'admin' => $this->rulesForAdmin(),
            // ...
        }
    }

    private function ruleForPayer()
    {
        return [
            'date' => ['date:Y-m-d'],
            // ...
        ];
    }
}

Potential Side Effects
Most common users, this should have no impact. The only place where it might be a problem is if a user has a child class (extending from FormRequest) which has its own getRules() method on it prior to this change.

@mateusjunges why the thumbs down?

I might be missing something here but can't you just rename the getRules method in your example to rules to achieve the same?
class ListTransactionsRequest extends FormRequest
{
    protected function rules(): array
    {
        return match($this->user()->role) {
            'payer' => $this->rulesForPayer(),
            'admin' => $this->rulesForAdmin(),
            // ...
        }
    }

    // ...
}

I might be missing something here but can't you just rename the getRules method in your example to rules to achieve the same?
class ListTransactionsRequest extends FormRequest
{
    protected function rules(): array
    {
        return match($this->user()->role) {
            'payer' => $this->rulesForPayer(),
            'admin' => $this->rulesForAdmin(),
            // ...
        }
    }

    // ...
}

Absolutely! However, we were hoping to make use of interfaces to handle this. I suppose that would be another approach we could take.
We were looking to make these changes gradually to existing endpoints. If we do it this way, we would have to change the completed 40 requests and modify all of them.

@cosmastech I gave a thumbs down for the reason described by @rojto. I don't think this method is needed if you can use the existing rules method to achieve the same thing.

We were looking to make these changes gradually to existing endpoints. If we do it this way, we would have to change the completed 40 requests and modify all of them to route the rules() to difference functions.

I also don't understand this. Why do you have to change all 40 request classses at once? Why can't you change one and keep the others unchanged?

@cosmastech I gave a thumbs down for the reason described by @rojtjo. I don't think this method is needed if you can use the existing rules method to achieve the same thing.

Within our project, all requests extend from a BaseRequest class (which itself extends from FormRequest).
What I was hoping to do is something like this:
class BaseRequest extends FormRequest
{
    // ...
    protected function createDefaultValidator(ValidationFactory $factory)
    {
        $methodName = match(true) {
            $this instanceof AuthorizesAdminRequest && $this->user()->role === 'admin' => 'adminRules',
            $this instanceof AuthorizesPayerRequest && $this->user()->role === 'payer' => 'payerRules',
            // ...
            default => 'authorize',
        };

        $rules = method_exists($this, $methodName) ? $this->container->call([$this, $methodName]) : [];
        // everything below here is copied directly from the parent class' method
        $validator = $factory->make(
            $this->validationData(),
            $rules,
            $this->messages(),
            $this->attributes()
        )->stopOnFirstFailure($this->stopOnFirstFailure);

        if ($this->isPrecognitive()) {
            $validator->setRules(
                $this->filterPrecognitiveRules($validator->getRulesWithoutPlaceholders())
            );
        }

        return $validator;
    }
}

If we already have a ListTransactionsRequest with rules() that was designed to work with users, but not necessarily admins or payers, we can leave the request alone and just implement an interface on the class:
interface AuthorizesAdminRequest
{
    public function adminRules();
}

And all we have to do to the existing request would be to tack on implements and then add adminRules():
class ListTransactionsRequest extends BaseRequest implements AuthorizesAdminRequest
{
    public function adminRules()
    {
        return [
            'filter_only_applies_to_admins' => ['rules-go-here'],
            // ...
        ];
    }
}

We're fine if it doesn't get added, but would be a lot easier.

I might be missing something, but I still don't understand why you can't use the rules method.

Please undo all unrelated coding style changes.

The sequence for these calls seem to impact some people: https://github.com/laravel/framework/issues/50083


september to nyc broken

Sends you north past mta.

j:hits_mta_east was missing a direction. I added it, and it should work now.


Convert more percent formatting and upgrade semgrep to 0.14.0

Semgrep has gotten a little more clever at applying the percent formatting rule.

Hmm, semgrep brought back the linter spam:
semgrep-py| running 9 rules...

Feel free to merge the first commit.

I suppressed the linter spam using the new suppress_line parameter from zulip/zulint#13.

Ci seems to be failing

Fixed.

Merged as f364414cb9f1626ba7069cb6477a9a86b19b2004 after a trivial rebase, thanks @andersk!


Logs to console

Will it be possible in the client to add support for having a choice if you want the client to print error messages to sys.stdout?
It could be added as an option where you setup the client to specify where to send the logging information.
In the beginning it could be a start just to have a bool to state wether or not to log to sys.stdout?

pygnmi is consistently using the python logging framework (the only exception being some print()s when you add debug=True to some calls I hink). What I do in my code to get rid of the log message is to raise the log level to CRITICAL. You could also set it to higher than that to avoid any messages.
from pygnmi.client import gNMIclient
from pygnmi.client import logger as gnmi_logger

gnmi_logger.setLevel(logging.CRITICAL + 1)
client = gNMIclient(target=..., ...)
client.connect()

I'm not against adding a feature switch for this (and would have gladly used this myself if I'd seen it available), but this might help with your immediate problem.


Best way to handle audio interruptions?

Please fill the template to help you out. Also, please try the Example project compare before submiting the issue when you have certain issue with your project setup.

Version of react-native-audio-recorder-player
3.6.7 and 3.6.10
I suddenly have started receiving pings from users indicating interruptions are deleting their recordings. This seems to be happening all at once so either something changed in my app (most likely) or perhaps in iOS?
I was wondering if anyone could share how they handle interruptions on the react native side. As it stands, interruptions seem to restart the recording and discard the audio up to that point. Could anyone share an example of how they handle this?
Version of React Native
Platforms you faced the error (IOS or Android or both?)
Expected behavior
Actual behavior
Steps to reproduce the behabior

I am facing the same problem.

For now ive moved to expo-av which has gotten better with Expo 51 and handled interruptions a little more elegantly.

I messed with the code a bit. It calls to pause and resume based on the interrupt event beginning and end. The problem is, in some cases, you would just want to stop the player and save the file recorded so far. It seems that if you call stopRecorder while it is in the paused stage, you lose the file.

I was Having the Exact same issue,

I downgraded the lib to before the fix 3.6.6
-I had to make a module on my native side to handle system audio alert interruptions this rippled a trigger on my ReactNative side, which then paused the recording; BUT and there's a big one, the issue was on IOS;

so I had to put on higher stakes:

the bug is a normal behavior on audio implementation dictionary that can be seen here it calls the prepareRecord function that zeroes our current audio;
I altered the implementation to keep 2 separate stages, 1 for paused/interrupted audios inside an easily accessible folder; and one for stopped audios (or consolidated);
this worked, but I still had audio pieces and not a complete audio, So I stitched them together after hitting stop, using native code;

I was Having the Exact same issue,

I downgraded the lib to before the fix 3.6.6
-I had to make a module on my native side to handle system audio alert interruptions this rippled a trigger on my ReactNative side, which then paused the recording; BUT and there's a big one, the issue was on IOS;

so I had to put on higher stakes:

the bug is a normal behavior on audio implementation dictionary that can be seen here it calls the prepareRecord function that zeroes our current audio;
I altered the implementation to keep 2 separate stages, 1 for paused/interrupted audios inside an easily accessible folder; and one for stopped audios (or consolidated);
this worked, but I still had audio pieces and not a complete audio, So I stitched them together after hitting stop, using native code;

I did the same, i use ffmpeg-kit to merge all the pieces.

Downgrading from 3.6.10 to 3.6.7 fixed this for me.


Support for sensor filters?

Hi,
thanks a lot for this add-on to ESPHome! It really helped me to finally read my EBZ DD3 which doesn't support SML. Does this add-on support sensor filtering? I'd like to reduce the amount of data it sends to HA. Especially the kWh I don't need more than every 60s.
I've used
    filters:     - throttle: 60s
from https://esphome.io/components/sensor/index.html#throttle
But it still seems to send the data every second though.
Thanks!

No, I didn't implement any kind of filtering. According to the docu, throttleshould do the job.
But I don't think that it is worth the spend the effort. I would assume that the overhead for a WLAN message (with all the values in it) is much bigger than the few bytes that are necessary to add an extra value.


Bank Dropdown doesn't work

menu doesn't open at all

can't duplicate, may have been caused by people being stuck in combat


[NEW] Integrate a new dialog Decorator, maybe?

Having seen the Read Only Dialog sample in Serene demo, is it possible to set the code that does it as a part of Serenity.Decorators in the next version?
so that it will be alongside the  @Serenity.Decorators.maximizable() and @Serenity.Decorators.responsive() , etc.
so that we can just set dialogstaht serve only for viewing as @Serenity.Decorators.readonly().

Maybe but not sure it will be so useful feature. Next you'll ask to make it conditional, then ask to switch it by a button so on.


AVRO-4033: [C++] Filter out redundant union classes generated by avrogencpp

AVRO-4033
What is the purpose of the change
Currently avrogencpp generates a class for each union type that is encountered in the schema. Even if there is a class that represents the exact same union, a new type will be generated.
Example Schema:
{
    "type": "record",
    "doc": "Top level Doc.",
    "name": "RootRecord",
    "fields": [
        {
            "name": "nullable_string_1",
            "type": [
                "null",
                "string"
            ]
        },
        {
            "name": "nullable_string_2",
            "type": [
                "null",
                "string"
            ]
        },
        {
            "name": "nullable_string_3",
            "type": [
                "null",
                "string"
            ]
        }
    ]
}

The generated RootRecord will look like this:
struct RootRecord {
    typedef _union_test_json_Union__0__ nullable_string_1_t;
    typedef _union_test_json_Union__1__ nullable_string_2_t;
    typedef _union_test_json_Union__2__ nullable_string_3_t;
    nullable_string_1_t nullable_string_1;
    nullable_string_2_t nullable_string_2;
    nullable_string_3_t nullable_string_3;
    RootRecord() :
        nullable_string_1(nullable_string_1_t()),
        nullable_string_2(nullable_string_2_t()),
        nullable_string_3(nullable_string_3_t())
        { }
}

Each field uses a unique union class, but their code is identical. With the suggested changes, only one union class will be generated.
This can help to reduce the header size for schemas that contain many unions.
Verifying this change
This change added tests and can be verified as follows:

added unit test to check that c++ types are the same for some union types
added a unit test with encode/decode for the schema used in test above

Documentation

Does this pull request introduce a new feature? only a small improvement in the generated code
If yes, how is the feature documented? added release note in JIRA ticket

Merged main into this branch to fix conflicts

Thank you, @hwse !


Single line break (\n rule)

It would be useful to make a "\n" or "%n" rule in the date format, which would allow users to split text into several lines.
Demonstration:
Before (current build):

After (lock screen preview):

Also, \n rule is available in song info format, but the implementation is broken:


dnf/yum update is the same as dnf upgrade -- it updates all packages

No reason to run it before installing or upgrading httpie.
This is not apt.

Added here together with the READMe changes, so it comes in a single commit.

Perfect!
Thanks a lot :) :pie:


Group Reverse SOP

Source:
0 0 1 1 0 1
Target:
1 0 1 1 0 0

Used @numelem instead of @numvtx due to a bug in 17.5.388

Used @numelem instead of @numvtx due to a bug in 17.5.388

The mistake in docs*

[ ] Groups do not appear in group menu

Realise "Bad pattern" warning. Group SOP style.

Toolbox

Waiting for #76


Improve memory usage in features pipeline

The pipeline now has an option for specifying the output dtype to allow reducing of memory needed for the data.
Despite the recent improvements to MergeEOPatchTask it causes some noticeable memory spikes due to data copying. For this reason parts of the pipeline have been linearized by hand (since it does not change the functionality) to deliver a less volatile memory consumption.

The tests fail because setting the dtype relies on functionality that has been recently added to the eo-learn develop :cry:

The tests fail because setting the dtype relies on functionality that has been recently added to the eo-learn develop 😢

As that seems to be the only problem, I say it can be merged. Looking at the code everything seems ok.


Check config.json URLs for corresponding files

Fixes #101
When loading the docs/config.json file for each version of the docs
site, we have some logic in normalizeDocsUrl that validates URLs
(i.e., checks if the URL has a trailing slash) and adds a
version-specific path segment to the URL path.
This changes normalizeDocsUrl to also check if a file exists at the
given URL path. This way, docs site builds fail if (a) a navigation
entry exists for a nonexistent docs page; or (b) a redirect points to a
nonexistent docs page.
Unfortunately, throughout the code that handles config.json, there is
logic that assumes the file lives in
content/<version>/docs/config.json. To test this change on a test data
directory, rather than an actual docs site directory, we would need to
refactor a lot of server/config-docs.ts and the code that imports from
it.
As a result, in order to test only normalizeDocsUrl, I had to make it
an exported function. I also had to make assumptions in the tests that
there would be a content directory populated with per-version
subdirectories.

The build is also going to fail until we merge these PRs in gravitational/teleport:

https://github.com/gravitational/teleport/pull/15219
https://github.com/gravitational/teleport/pull/15218
https://github.com/gravitational/teleport/pull/15217
https://github.com/gravitational/teleport/pull/15216

@avatus @hatched This is currently failing yarn test on the CI server because (I'm guessing) it doesn't load the git submodules before running the tests. I'm going to see if I can rework the file-checking code to not require the actual docs content directory for testing, which is really the approach I should have taken from the start but avoided to get this done more quickly.

@ptgott FYI this commit leads to the following error when trying to starting the docs with yarn dev locally:
➜  docs git:(main) ✗ yarn dev
yarn run v1.22.17
warning ../../../../../../package.json: No license field
$ yarn preflight && next dev
warning ../../../../../../package.json: No license field
$ yarn build-node && yarn add-symlinks && yarn generate-article-links
warning ../../../../../../package.json: No license field
$ rm -rf .build && tsc --build tsconfig.node.json && tsc-esm-fix --target='.build' --ext='.mjs'
warning ../../../../../../package.json: No license field
$ node ./scripts/add-symlinks.mjs
(node:52424) [DEP0147] DeprecationWarning: In future versions of Node.js, fs.rmdir(path, { recursive: true }) will be removed. Use fs.rm(path, { recursive: true }) instead
(Use `node --trace-deprecation ...` to show where the warning was created)
warning ../../../../../../package.json: No license field
$ node ./scripts/generate-article-links.mjs
file:///Users/r0mant/code/go/src/github.com/gravitational/docs/.build/server/config-docs.mjs:257
        throw new Error("The following navigation slugs or redirect destinations do not" +
              ^

Error: The following navigation slugs or redirect destinations do not correspond to an actual MDX file:
	- /kubernetes-access/guides/migration/
	- /trustedclusters/
	- /enterprise/sso/ssh-google-workspace/
	- /machine-id/guides/ansible/
    at loadConfig (file:///Users/r0mant/code/go/src/github.com/gravitational/docs/.build/server/config-docs.mjs:257:15)
    at file:///Users/r0mant/code/go/src/github.com/gravitational/docs/.build/server/paths.mjs:105:41
    at Array.map (<anonymous>)
    at file:///Users/r0mant/code/go/src/github.com/gravitational/docs/.build/server/paths.mjs:104:40
    at Array.forEach (<anonymous>)
    at generateArticleLinks (file:///Users/r0mant/code/go/src/github.com/gravitational/docs/.build/server/paths.mjs:102:14)
    at file:///Users/r0mant/code/go/src/github.com/gravitational/docs/scripts/generate-article-links.mjs:15:1
    at ModuleJob.run (node:internal/modules/esm/module_job:198:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:409:24)

Node.js v18.0.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Checking out commit before this PR works.