Commissioning fails on Linux

Hi all!
I really appreciate your work on this crate; The functionality that is already implemented looks quite promising to me!
Unfortunately, I was not yet able to commission the example as described in the README because the CHIP tool crashes. Here is an extract of the log:
[1662967379.124620][23231:23237] CHIP:EM: Received message of type 0x40 with protocolId (0, 0) and MessageCounter:10578812 on exchange 33111i
[1662967379.124623][23231:23237] CHIP:EM: Found matching exchange: 33111i, Delegate: 0x7fed2800c860
[1662967379.124626][23231:23237] CHIP:EM: Rxd Ack; Removing MessageCounter:223883550 from Retrans Table on exchange 33111i
[1662967379.124628][23231:23237] CHIP:EM: Removed CHIP MessageCounter:223883550 from RetransTable on exchange 33111i
[1662967379.124634][23231:23237] CHIP:SC: Received error (protocol code 2) during PASE process: ../Projects/connectedhomeip/examples/chip-tool/third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:756: CHIP Error 0x00000038: Invalid PASE parameter
[1662967379.124637][23231:23237] CHIP:IN: SecureSession[0x7fed2800a0e0]: Released - Type:1 LSID:28137
[1662967379.124641][23231:23237] CHIP:SC: Failed during PASE session setup: ../Projects/connectedhomeip/examples/chip-tool/third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:756: CHIP Error 0x00000038: Invalid PASE parameter
[1662967379.124644][23231:23237] CHIP:TOO: Secure Pairing Failed
[1662967379.124654][23231:23237] CHIP:TOO: Pairing Failure: ../Projects/connectedhomeip/examples/chip-tool/third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:756: CHIP Error 0x00000038: Invalid PASE parameter
[1662967379.124657][23231:23237] CHIP:EM: Sending Standalone Ack for MessageCounter:10578812 on exchange 33111i
[1662967379.124662][23231:23237] CHIP:IN: Prepared unauthenticated message 0x7fed3e255fd0 to 0x0000000000000000 (0)  of type 0x10 and protocolId (0, 0) on exchange 33111i with MessageCounter:223883551.
[1662967379.124668][23231:23237] CHIP:IN: Sending unauthenticated msg 0x7fed3e255fd0 with MessageCounter:223883551 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>495F84 msec
[1662967379.124696][23231:23237] CHIP:EM: Flushed pending ack for MessageCounter:10578812 on exchange 33111i
[1662967379.124778][23231:23231] CHIP:CTL: Shutting down the commissioner
[1662967379.124794][23231:23231] CHIP:CTL: Stopping commissioning discovery over DNS-SD
[1662967379.124830][23231:23231] CHIP:CTL: Shutting down the controller
[1662967379.124838][23231:23231] CHIP:IN: Expiring all sessions for fabric 0x1!!
[1662967379.124845][23231:23231] CHIP:FP: Forgetting fabric 0x1
[1662967379.124856][23231:23231] CHIP:TS: Pending Last Known Good Time: 2022-09-06T15:14:17
[1662967379.124921][23231:23231] CHIP:TS: Previous Last Known Good Time: 2022-09-06T15:14:17
[1662967379.124928][23231:23231] CHIP:TS: Reverted Last Known Good Time to previous value
[1662967379.124945][23231:23231] CHIP:CTL: Shutting down the commissioner
[1662967379.124950][23231:23231] CHIP:CTL: Stopping commissioning discovery over DNS-SD
[1662967379.124968][23231:23231] CHIP:CTL: Shutting down the controller
[1662967379.124973][23231:23231] CHIP:CTL: Shutting down the System State, this will teardown the CHIP Stack
[1662967379.125221][23231:23231] CHIP:DMG: IM WH moving to [Uninitialized]
[1662967379.125228][23231:23231] CHIP:DMG: IM WH moving to [Uninitialized]
[1662967379.125233][23231:23231] CHIP:DMG: IM WH moving to [Uninitialized]
[1662967379.125239][23231:23231] CHIP:DMG: IM WH moving to [Uninitialized]
[1662967379.125246][23231:23231] CHIP:DMG: All ReadHandler-s are clean, clear GlobalDirtySet
[1662967379.125263][23231:23231] CHIP:BLE: BleConnectionDelegate::CancelConnection is not implemented.
[1662967379.125345][23231:23231] CHIP:DL: writing settings to file (/tmp/chip_counters.ini-6rOj5p)
[1662967379.127292][23231:23231] CHIP:DL: renamed tmp file to file (/tmp/chip_counters.ini)
[1662967379.127299][23231:23231] CHIP:DL: NVS set: chip-counters/total-operational-hours = 0 (0x0)
[1662967379.127302][23231:23231] CHIP:DL: Inet Layer shutdown
[1662967379.127304][23231:23231] CHIP:DL: BLE shutdown
[1662967379.127307][23231:23231] CHIP:DL: System Layer shutdown
[1662967379.127316][23231:23231] CHIP:TOO: Run command failure: ../Projects/connectedhomeip/examples/chip-tool/third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:756: CHIP Error 0x00000038: Invalid PASE parameter

The Rust example itself logs two error messages:
[2022-09-12T07:22:53Z ERROR matter::sys::sys_linux] Linux is not yet supported for MDNS Service

[2022-09-12T07:22:59Z ERROR matter::secure_channel::core] PASE Not enabled

From briefly looking at the code, I assume that you are working on MacOS? I am on Ubuntu 22.04, so this may be part of the problem. For testing purpose, I added an MDNS implementation for sys_publish_service() to sys/sys_linux.rs. However, the second error remained, so I suppose this is the actual problem. Could you give me some hints?
Thanks four your help!
Regards :)

Ah, could you also share the rust side logs from Linux?

I think what is likely happening is that commissioning worked for you once (after you fixed the mDNS). But then once commissioning has happened matter-rs persistently stored a record of it, and hence the next time the commissioning is not enabled (hence PASE is disabled).

You could either try using the control commands from chip-tool, since both sides have persistently stored information about the successful commissioning
Try deleting the /tmp/plonk_psm directory on Linux, which is where the matter-rs persistently stores information, and then relaunch the matter server

Thank you for your quick reply! This actually did the trick. I have succesfully commissioned the device and toggled the onoff state.
However, there are still some quirks. First off, my sys_publish_service() implementation based on the mdns_sd crate did not do the job (I don't really know why, but it's not such a nice option anyways since it only supports IPv4). Instead, I ended up installing libavahi-compat-libdnssd-dev and basically copied the MacOS implementation. However, this always resulted in a RegistrationError::ServiceError(0), which I imagine should be an Ok state and which I thus treated so. Interestingly, this was already sufficient to make it work. Unfortunately, with this hack, we are not able to gain access to the RegisteredDnsService object. If my theory about the ServiceError(0) sounds reasonable to you, I can create an issue in the astro-dnssd crate.
One final side note: I have to wait a few seconds after cargo running before performing the commissioning, or I will end up with the following error. Is this different on MacOS?
[2022-09-12T13:45:16Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:16Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287484
[2022-09-12T13:45:16Z INFO  matter::transport::session] Creating new session
[2022-09-12T13:45:16Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 32, Flags: R|I|
[2022-09-12T13:45:16Z INFO  matter::transport::exchange] Creating new exchange
[2022-09-12T13:45:16Z DEBUG matter::transport::mgr] Exchange is Exchange { id: 27974, sess_idx: 0, role: Responder, state: Open, data: None, mrp: ReliableMessage { retrans: None, ack: Some(AckEntry { msg_ctr: 86287484, ack_timeout: Sys
temTime { tv_sec:<PHONE_NUMBER>, tv_nsec: 597661544 } }) } }
[2022-09-12T13:45:16Z INFO  matter::secure_channel::core] In PBKDF Param Request Handler
[2022-09-12T13:45:16Z INFO  matter::transport::exchange] Sending with proto id: 0 opcode: 33
[2022-09-12T13:45:16Z INFO  matter::transport::proto_hdr] [encode] ExId: 27974, Proto: 0, Opcode: 33, Flags: R|A|
Message Sent to [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:16Z INFO  matter::transport::mgr] Exchange Mgr: {  Session Mgr: {[
    { peer: [::ffff:<IP_ADDRESS>]:56740
    , peer_nodeid: Some(16523851297399427681), local: 0, remote: 0, msg_ctr: 31518809, mode: PlainText, ts: SystemTime { tv_sec:<PHONE_NUMBER>, tv_nsec: 397715582 }, },
    ], next_sess_id: 2},
      Exchanges: [
    { exch_id: 27974, sess_index: 0, role: Responder, data: None, mrp: ReliableMessage { retrans: Some(RetransEntry { msg_ctr: 31518808 }), ack: None }, state: Open, },
      ]
    }
[2022-09-12T13:45:16Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:16Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287484
[2022-09-12T13:45:16Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 32, Flags: R|I|
[2022-09-12T13:45:16Z DEBUG matter::transport::mgr] Exchange is Exchange { id: 27974, sess_idx: 0, role: Responder, state: Open, data: None, mrp: ReliableMessage { retrans: Some(RetransEntry { msg_ctr: 31518808 }), ack: Some(AckEntry {
 msg_ctr: 86287484, ack_timeout: SystemTime { tv_sec:<PHONE_NUMBER>, tv_nsec: 597839429 } }) } }
[2022-09-12T13:45:16Z INFO  matter::secure_channel::core] In PBKDF Param Request Handler
[2022-09-12T13:45:16Z INFO  matter::secure_channel::pake] Previous session in-progress, denying new request
[2022-09-12T13:45:16Z INFO  matter::transport::exchange] Sending with proto id: 0 opcode: 64
[2022-09-12T13:45:16Z ERROR matter::transport::mrp] Previous retrans entry for this exchange already exists
[2022-09-12T13:45:16Z ERROR matter::transport::mgr] Error in sending msg Invalid
[2022-09-12T13:45:16Z ERROR matter::transport::mgr] Error in handle_rxtx
[2022-09-12T13:45:16Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:16Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287485
[2022-09-12T13:45:16Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 34, Flags: R|A|I|
[2022-09-12T13:45:16Z DEBUG matter::transport::mgr] Exchange is Exchange { id: 27974, sess_idx: 0, role: Responder, state: Open, data: None, mrp: ReliableMessage { retrans: None, ack: Some(AckEntry { msg_ctr: 86287485, ack_timeout: Sys
temTime { tv_sec:<PHONE_NUMBER>, tv_nsec: 602460524 } }) } }
[2022-09-12T13:45:16Z INFO  matter::secure_channel::core] In PASE Pake1 Handler
[2022-09-12T13:45:16Z ERROR matter::transport::mgr] Error in proto_demux InvalidSignature
[2022-09-12T13:45:16Z ERROR matter::transport::mgr] Error in handle_rxtx
[2022-09-12T13:45:16Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:16Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287485
[2022-09-12T13:45:16Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 34, Flags: R|A|I|
[2022-09-12T13:45:16Z ERROR matter::transport::mrp] Previous ACK entry for this exchange already exists
[2022-09-12T13:45:16Z ERROR matter::transport::mgr] Error in recv: Invalid
[2022-09-12T13:45:16Z ERROR matter::transport::mgr] Error in handle_rxtx
[2022-09-12T13:45:17Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:17Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287485
[2022-09-12T13:45:17Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 34, Flags: R|A|I|
[2022-09-12T13:45:17Z ERROR matter::transport::mrp] Previous ACK entry for this exchange already exists
[2022-09-12T13:45:17Z ERROR matter::transport::mgr] Error in recv: Invalid
[2022-09-12T13:45:17Z ERROR matter::transport::mgr] Error in handle_rxtx
[2022-09-12T13:45:17Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:17Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287485
[2022-09-12T13:45:17Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 34, Flags: R|A|I|
[2022-09-12T13:45:17Z ERROR matter::transport::mrp] Previous ACK entry for this exchange already exists
[2022-09-12T13:45:17Z ERROR matter::transport::mgr] Error in recv: Invalid
[2022-09-12T13:45:17Z ERROR matter::transport::mgr] Error in handle_rxtx
[2022-09-12T13:45:18Z INFO  matter::transport::session] Received from src: [::ffff:<IP_ADDRESS>]:56740

[2022-09-12T13:45:18Z INFO  matter::transport::plain_hdr] [decode] flags: SRC_ADDR_PRESENT, session type: None, sess_id: 0, ctr: 86287485
[2022-09-12T13:45:18Z INFO  matter::transport::proto_hdr] [decode] ExId: 27974, Proto: 0, Opcode: 34, Flags: R|A|I|
[2022-09-12T13:45:18Z ERROR matter::transport::mrp] Previous ACK entry for this exchange already exists
[2022-09-12T13:45:18Z ERROR matter::transport::mgr] Error in recv: Invalid
[2022-09-12T13:45:18Z ERROR matter::transport::mgr] Error in handle_rxtx

[1662990308.284262][99161:99161] CHIP:DL: ChipLinuxStorage::Init: Using KVS config file: /tmp/chip_kvs
[1662990308.285457][99161:99161] CHIP:DL: ChipLinuxStorage::Init: Using KVS config file: /tmp/chip_factory.ini
[1662990308.285486][99161:99161] CHIP:DL: ChipLinuxStorage::Init: Using KVS config file: /tmp/chip_config.ini
[1662990308.285497][99161:99161] CHIP:DL: ChipLinuxStorage::Init: Using KVS config file: /tmp/chip_counters.ini
[1662990308.285548][99161:99161] CHIP:DL: writing settings to file (/tmp/chip_counters.ini-rmX3km)
[1662990308.285664][99161:99161] CHIP:DL: renamed tmp file to file (/tmp/chip_counters.ini)
[1662990308.285669][99161:99161] CHIP:DL: NVS set: chip-counters/reboot-count = 84 (0x54)
[1662990308.285939][99161:99161] CHIP:DL: Got Ethernet interface: enp11s0
[1662990308.286084][99161:99161] CHIP:DL: Found the primary Ethernet interface:enp11s0
[1662990308.286243][99161:99161] CHIP:DL: Got WiFi interface: wlp9s0
[1662990308.286702][99161:99161] CHIP:DL: Found the primary WiFi interface:wlp9s0
[1662990308.286718][99161:99161] CHIP:IN: UDP::Init bind&listen port=0
[1662990308.286732][99161:99161] CHIP:IN: UDP::Init bound to port=51726
[1662990308.286734][99161:99161] CHIP:IN: UDP::Init bind&listen port=0
[1662990308.286740][99161:99161] CHIP:IN: UDP::Init bound to port=56740
[1662990308.286742][99161:99161] CHIP:IN: BLEBase::Init - setting/overriding transport
[1662990308.286743][99161:99161] CHIP:IN: TransportMgr initialized
[1662990308.286749][99161:99161] CHIP:FP: Initializing FabricTable from persistent storage
[1662990308.286772][99161:99161] CHIP:TS: Last Known Good Time: 2022-09-12T11:06:50
[1662990308.287045][99161:99161] CHIP:FP: Fabric index 0x1 was retrieved from storage. Compressed FabricId 0xCF158BD9346FE8FF, FabricId 0x0000000000000001, NodeId 0x000000000001B669, VendorId 0xFFF1
[1662990308.287252][99161:99161] CHIP:ZCL: Using ZAP configuration...
[1662990308.287418][99161:99161] CHIP:DL: MDNS failed to join multicast group on enp11s0 for address type IPv4: ../../third_party/connectedhomeip/src/inet/UDPEndPointImplSockets.cpp:758: Inet Error 0x00000110: Address not found
[1662990308.287784][99161:99161] CHIP:DL: MDNS failed to join multicast group on enx00e04c68ed22 for address type IPv4: ../../third_party/connectedhomeip/src/inet/UDPEndPointImplSockets.cpp:758: Inet Error 0x00000110: Address not found
[1662990308.287805][99161:99161] CHIP:CTL: System State Initialized...
[1662990308.287814][99161:99161] CHIP:CTL: Stopping commissioning discovery over DNS-SD
[1662990308.287824][99161:99161] CHIP:CTL: Setting attestation nonce to random value
[1662990308.287830][99161:99161] CHIP:CTL: Setting CSR nonce to random value
[1662990308.287838][99161:99161] CHIP:IN: UDP::Init bind&listen port=5550
[1662990308.287845][99161:99161] CHIP:IN: UDP::Init bound to port=5550
[1662990308.287848][99161:99161] CHIP:IN: UDP::Init bind&listen port=5550
[1662990308.287853][99161:99161] CHIP:IN: UDP::Init bound to port=5550
[1662990308.287854][99161:99161] CHIP:IN: TransportMgr initialized
[1662990308.287890][99161:99166] CHIP:DL: CHIP task running
[1662990308.287934][99161:99166] CHIP:CTL: Stopping commissioning discovery over DNS-SD
[1662990308.287940][99161:99166] CHIP:CTL: Setting attestation nonce to random value
[1662990308.287960][99161:99166] CHIP:CTL: Setting CSR nonce to random value
[1662990308.288136][99161:99166] CHIP:CTL: Generating NOC
[1662990308.288303][99161:99166] CHIP:FP: Validating NOC chain
[1662990308.288593][99161:99166] CHIP:FP: NOC chain validation successful
[1662990308.288618][99161:99166] CHIP:FP: Updated fabric at index: 0x1, Node ID: 0x000000000001B669
[1662990308.288622][99161:99166] CHIP:TS: Last Known Good Time: 2022-09-12T11:06:50
[1662990308.288623][99161:99166] CHIP:TS: New proposed Last Known Good Time: 2021-01-01T00:00:00
[1662990308.288624][99161:99166] CHIP:TS: Retaining current Last Known Good Time
[1662990308.289089][99161:99166] CHIP:FP: Metadata for Fabric 0x1 persisted to storage.
[1662990308.289248][99161:99166] CHIP:TS: Committing Last Known Good Time to storage: 2022-09-12T11:06:50
[1662990308.289396][99161:99166] CHIP:CTL: Joined the fabric at index 1. Compressed fabric ID is: 0x0000000000000000
[1662990308.289401][99161:99166] CHIP:IN: UDP::Init bind&listen port=5550
[1662990308.289411][99161:99166] CHIP:IN: UDP::Init bound to port=5550
[1662990308.289413][99161:99166] CHIP:IN: UDP::Init bind&listen port=5550
[1662990308.289420][99161:99166] CHIP:IN: UDP::Init bound to port=5550
[1662990308.289422][99161:99166] CHIP:IN: TransportMgr initialized
[1662990308.290627][99161:99166] CHIP:IN: SecureSession[0x7f52c000a460]: Allocated Type:1 LSID:53180
[1662990308.290632][99161:99166] CHIP:SC: Assigned local session key ID 53180
[1662990308.290642][99161:99166] CHIP:SC: Including MRP parameters in PBKDF param request
[1662990308.290652][99161:99166] CHIP:IN: Prepared unauthenticated message 0x55a297a0c5a8 to 0x0000000000000000 (0)  of type 0x20 and protocolId (0, 0) on exchange 27974i with MessageCounter:86287484.
[1662990308.290657][99161:99166] CHIP:IN: Sending unauthenticated msg 0x55a297a0c5a8 with MessageCounter:86287484 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A73E92 msec
[1662990308.290711][99161:99166] CHIP:SC: Sent PBKDF param request
[1662990308.290715][99161:99166] CHIP:CTL: Setting attestation nonce to random value
[1662990308.290718][99161:99166] CHIP:CTL: Setting CSR nonce to random value
[1662990308.290722][99161:99166] CHIP:CTL: Commission called for node ID 0x0000000000BC5C01
[1662990313.197467][99161:99166] CHIP:EM: Retransmitting MessageCounter:86287484 on exchange 27974i Send Cnt 1
[1662990313.197546][99161:99166] CHIP:IN: Sending unauthenticated msg 0x55a297a0c5a8 with MessageCounter:86287484 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A751BD msec
[1662990316.397920][99161:99166] CHIP:EM: Received message of type 0x21 with protocolId (0, 0) and MessageCounter:31518808 on exchange 27974i
[1662990316.397950][99161:99166] CHIP:EM: Found matching exchange: 27974i, Delegate: 0x7f52c000ccc0
[1662990316.397966][99161:99166] CHIP:EM: Rxd Ack; Removing MessageCounter:86287484 from Retrans Table on exchange 27974i
[1662990316.397971][99161:99166] CHIP:EM: Removed CHIP MessageCounter:86287484 from RetransTable on exchange 27974i
[1662990316.397983][99161:99166] CHIP:SC: Received PBKDF param response
[1662990316.397997][99161:99166] CHIP:SC: Peer assigned session ID 1
[1662990316.402280][99161:99166] CHIP:EM: Piggybacking Ack for MessageCounter:31518808 on exchange: 27974i
[1662990316.402299][99161:99166] CHIP:IN: Prepared unauthenticated message 0x7f52c0009e28 to 0x0000000000000000 (0)  of type 0x22 and protocolId (0, 0) on exchange 27974i with MessageCounter:86287485.
[1662990316.402312][99161:99166] CHIP:IN: Sending unauthenticated msg 0x7f52c0009e28 with MessageCounter:86287485 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A75E42 msec
[1662990316.402407][99161:99166] CHIP:SC: Sent spake2p msg1
[1662990316.735927][99161:99166] CHIP:EM: Retransmitting MessageCounter:86287485 on exchange 27974i Send Cnt 1
[1662990316.735995][99161:99166] CHIP:IN: Sending unauthenticated msg 0x7f52c0009e28 with MessageCounter:86287485 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A75F8F msec
[1662990317.138635][99161:99166] CHIP:EM: Retransmitting MessageCounter:86287485 on exchange 27974i Send Cnt 2
[1662990317.138700][99161:99166] CHIP:IN: Sending unauthenticated msg 0x7f52c0009e28 with MessageCounter:86287485 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A76122 msec
[1662990317.772590][99161:99166] CHIP:EM: Retransmitting MessageCounter:86287485 on exchange 27974i Send Cnt 3
[1662990317.772660][99161:99166] CHIP:IN: Sending unauthenticated msg 0x7f52c0009e28 with MessageCounter:86287485 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A7639C msec
[1662990318.827299][99161:99166] CHIP:EM: Retransmitting MessageCounter:86287485 on exchange 27974i Send Cnt 4
[1662990318.827368][99161:99166] CHIP:IN: Sending unauthenticated msg 0x7f52c0009e28 with MessageCounter:86287485 to 0x0000000000000000 at monotonic time:<PHONE_NUMBER>A767BB msec
[1662990320.437207][99161:99166] CHIP:EM: Failed to Send CHIP MessageCounter:86287485 on exchange 27974i sendCount: 4 max retries: 4
[1662990366.408509][99161:99166] CHIP:SC: PASESession timed out while waiting for a response from the peer. Expected message type was 35
[1662990366.408574][99161:99166] CHIP:IN: SecureSession[0x7f52c000a460]: Released - Type:1 LSID:53180
[1662990366.408587][99161:99166] CHIP:TOO: Secure Pairing Failed
[1662990366.408637][99161:99166] CHIP:TOO: Pairing Failure: ../../third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:245: CHIP Error 0x00000032: Timeout
[1662990366.408855][99161:99161] CHIP:CTL: Shutting down the commissioner
[1662990366.408883][99161:99161] CHIP:CTL: Stopping commissioning discovery over DNS-SD
[1662990366.408947][99161:99161] CHIP:CTL: Shutting down the controller
[1662990366.408957][99161:99161] CHIP:IN: Expiring all sessions for fabric 0x1!!
[1662990366.408967][99161:99161] CHIP:FP: Forgetting fabric 0x1
[1662990366.408983][99161:99161] CHIP:TS: Pending Last Known Good Time: 2022-09-12T11:06:50
[1662990366.409060][99161:99161] CHIP:TS: Previous Last Known Good Time: 2022-09-12T11:06:50
[1662990366.409066][99161:99161] CHIP:TS: Reverted Last Known Good Time to previous value
[1662990366.409091][99161:99161] CHIP:CTL: Shutting down the commissioner
[1662990366.409095][99161:99161] CHIP:CTL: Stopping commissioning discovery over DNS-SD
[1662990366.409115][99161:99161] CHIP:CTL: Shutting down the controller
[1662990366.409120][99161:99161] CHIP:CTL: Shutting down the System State, this will teardown the CHIP Stack
[1662990366.409504][99161:99161] CHIP:DMG: IM WH moving to [Uninitialized]
[1662990366.409516][99161:99161] CHIP:DMG: IM WH moving to [Uninitialized]
[1662990366.409528][99161:99161] CHIP:DMG: IM WH moving to [Uninitialized]
[1662990366.409534][99161:99161] CHIP:DMG: IM WH moving to [Uninitialized]
[1662990366.409543][99161:99161] CHIP:DMG: All ReadHandler-s are clean, clear GlobalDirtySet
[1662990366.409577][99161:99161] CHIP:BLE: BleConnectionDelegate::CancelConnection is not implemented.
[1662990366.409800][99161:99161] CHIP:DL: writing settings to file (/tmp/chip_counters.ini-jjTc3U)
[1662990366.410086][99161:99161] CHIP:DL: renamed tmp file to file (/tmp/chip_counters.ini)
[1662990366.410108][99161:99161] CHIP:DL: NVS set: chip-counters/total-operational-hours = 0 (0x0)
[1662990366.410115][99161:99161] CHIP:DL: Inet Layer shutdown
[1662990366.410120][99161:99161] CHIP:DL: BLE shutdown
[1662990366.410125][99161:99161] CHIP:DL: System Layer shutdown
[1662990366.410152][99161:99161] CHIP:TOO: Run command failure: ../../third_party/connectedhomeip/src/protocols/secure_channel/PASESession.cpp:245: CHIP Error 0x00000032: Timeout

I have to wait a few seconds after cargo running before performing the commissioning

I haven't noticed this problem on the Mac, I usually switch the terminal and run the commissioning command, but let me keep trying.
From the logs, what it looks like is that the PBKDFResponse from Plonk takes longer to reach the chip-tool and hence it triggers a retransmission (chip-tool timestamps:<PHONE_NUMBER>.290657 and<PHONE_NUMBER>.197467). This retransmission probably is causing the state machine to end up in a weird state. I will create a separate issue to track this.

Unfortunately, with this hack, we are not able to gain access to the RegisteredDnsService object. If my theory about the ServiceError(0) sounds reasonable to you, I can create an issue in the astro-dnssd crate.

I am not too familiar with astro_dnssd, but is it likely that this condition gets hit during the execution on Linux?
https://github.com/AstroHQ/astro-dnssd/blob/develop/src/os/apple/register.rs#L211

Unfortunately, with this hack, we are not able to gain access to the RegisteredDnsService object. If my theory about the ServiceError(0) sounds reasonable to you, I can create an issue in the astro-dnssd crate.

I am not too familiar with astro_dnssd, but is it likely that this condition gets hit during the execution on Linux? https://github.com/AstroHQ/astro-dnssd/blob/develop/src/os/apple/register.rs#L211

Sounds reasonable to me.
Anyway, I don't really feel like this avahi apple compatibility layer is really the way to go, with this warning being presented on every run:
*** WARNING *** The program 'onoff_light' uses the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/blog/projects/avahi-compat.html>

I therefore experimented with the libmdns crate instead. This works quite well for commissioning, but fails to perform correctly when toggling the onoff state. I am rather clueless here, but will quickly open a PR and also put the respective error messages in there.

Unfortunately, with this hack, we are not able to gain access to the RegisteredDnsService object. If my theory about the ServiceError(0) sounds reasonable to you, I can create an issue in the astro-dnssd crate.

I am not too familiar with astro_dnssd, but is it likely that this condition gets hit during the execution on Linux? https://github.com/AstroHQ/astro-dnssd/blob/develop/src/os/apple/register.rs#L211

Sounds reasonable to me.
Anyway, I don't really feel like this avahi apple compatibility layer is really the way to go, with this warning being presented on every run:
*** WARNING *** The program 'onoff_light' uses the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/blog/projects/avahi-compat.html>

I therefore experimented with the libmdns crate instead. This works quite well for commissioning, but fails to perform correctly when toggling the onoff state. I am rather clueless here, but will quickly open a PR and also put the respective error messages in there.

Hi, I've been experimenting with your implementation of #18
I figured out how toggling and commands work but I have no idea why.
If you send a toggle command, it times out retrying. But if you use a mdns scanner and request the mdns records of _matter._tcp while it's trying to send a command it goes through. Tested with the IOs app Discovery. I think we're missing the first time chip-tool requests the records for some reason and the pipeline to handle a command only goes through if you poke it a second time.
I'm looking into fixing this.
Also, would you mind opening this PR on this repo: https://github.com/project-chip/matter-rs ?
Seems that one is under active development now.


Installed @actions/glob

Current dependencies on/for this PR:

main

PR #513   👈

PR #514

This comment was auto-generated by Graphite.


This isn't due to Go 1.15, but the newer version of OpenTracing. We should do a bump of OpenTracing in this repo too, and fix the tests.

Oh okay, sorry then, we always try to get the  latest version in Fedora, sometimes it causes API breakage. Hopefully when we move to modules, we could have more fine tuning over dependencies version.

@eclipseo it's fixed on master


Usage in Nuxt 3

Trying to use the component with nuxt 3 renders a warning and then an error.
[SSR] Error transforming ./node_modules/v-lazy-image/dist/v-lazy-image.es.js:
Error: Missing "./dist/v-lazy-image.es.js" export in "v-lazy-image" package

[Vue warn]: Invalid vnode type when creating vnode: undefined.
Invalid value used as weak map key
  at WeakMap.set (<anonymous>)

Should be fixed in latest v2.1.0. Please reopen if it's not the case


New _chart module feedback

Following from #238, #239, #328.  In particular:

I prefer the way @PythonCreator27 outputs the data (arrays with { date: ... } rather than Objects with timestamp keys).

and let's discuss any other feedback.

Agreed - I thought the output format @PythonCreator27 chose was generally easy to work with.
FWIW my client code combines the historical prices, dividends, and splits into a giant 2D array so I can operate on the entire chunk of data efficiently, but I'm not sure that's a common enough use case to consider just having the chart module output that format. I think as it is now is better because it's more explicit (and thus understandable) to users.

Thanks, @huned.
Having also had some time to think about this, I've decided I'd like to offer both formats.  With certain charting libraries, Yahoo's original input may indeed be easier to work with, and I'd like to avoid a situation where to convert to the array format only for the user to then need to write their own conversion code just go get back to the data we already had.
I imagine it will turn out similar to quote, so say:
yf2._chart('AAPL', { period1, return: "array" }); // default
yf2._chart('AAPL', { period1, return: "object" });

I was also considering merging into a single array too, and maybe we can indeed have an option for that.  condense maybe?  What exactly does your array look like, like this?
[
  { date: type: 'quote', /* ... */ },
  { date, type: 'split', /* ... */ },
  { date, type: 'dividend', /* ... */ },
 ]

Thanks again for all your patience and efforts to get things moving on this again... owing to the _ prefix I think / hope we can get this finished up and released in the coming days.

Thanks for your work on this @gadicc.
I transform both quote and _chart output as follows:
[
   ['ticker', 'date',  'open', 'close', 'low', 'high', 'volume', 'dividend', 'split'],
   // and then rows of data, e.g.
   ['MMM', new Date(...), #, #, #, #, #, # or undefined, # or undefined],
   ...
]

Which I write to stdout as CSVs. I'm doing it this way so I can take advantage of many common command line tools (eg, cut, column) and so I can efficiently import into a SQL database. My gut tells me that my use case is probably uncommon and probably shouldn't be supported until more people want it? The hesitation is that this creates a lot more code, tests, and maintenance workload for you for a codepath that might not get used by all that many people.
Are there more users you can think of who might be able to provide feedback on whether you should build it or not?

Oh yes, that's right, you did you say a 2D array.  Ok, on the same page now.
Yes, I agree, that's probably a less commonly used format, and I thank you for your consideration of the project maintainers :)
Umm, I believe the only people interested in this are you, @PythonCreator27, @ZehCoque (hi from July! :sweat_smile:) and now @felixsanz in #334 (intraday interval support).
I'm happy this is still released as _chart, the API is a bit finicky.  Just from playing with the values a bit, came up with quite a few cases that needed schema fine tuning.  Anyway, since you seem happy running off devel, let me know if it handled everything you threw it at, and we can at least get this into an interim release.
Particularly, tradingPeriods is a bit weird, and maybe we should transform it to something more useful.  For example in this test):
{
              "tradingPeriods": {
                "pre": [
                  [
                    {
                      "timezone": "EST",
                      "start":<PHONE_NUMBER>,
                      "end":<PHONE_NUMBER>,
                      "gmtoffset": -18000
                    }
                  ]
                ],
                // ...
}

So it's in a double array.  We should try find an example of more than 1 so we know where the other entries go.  (That's in the original response from Yahoo, we transform the dates of course).

Thanks @gadicc! I'll update my client code to use the devel branch and the new _chart module and will report back on any issues that arise over the next week or so.
Re the tradingPeriods property. I initially thought the array-of-array-containing-one-element thing might be related to securities traded in multiple markets, but I looked at a few (BIDU, BABA, BHP, BP) and all I saw had just the same one element nested arrays as you've observed.
However, an observation: The tradingPeriods data is ALMOST the same as currentTradingPeriods. You'll see that the timestamps are actually different between tradingPeriods and and currentTradingPeriod. However, in all cases, I see that the durations of each period are exactly the same:
(tradingPeriods.pre.end - tradingPeriods.pre.start) / 60 / 60 = 5.5 // 5.5 hours
(currentTradingPeriod.pre.end - currentTradingPeriod.pre.start) / 60 / 60 = 5.5

(tradingPeriods.regular.end - tradingPeriods.regular.start) / 60 / 60 = 6.5
(currentTradingPeriod.regular.end - currentTradingPeriod.regular.start) / 60 / 60 = 6.5

(tradingPeriods.post.end - tradingPeriods.post.start) / 60 / 60 = 4
(currentTradingPeriod.post.end - currentTradingPeriod.post.start) / 60 / 60 = 4

Yet, also, tradingPeriods timestamps and their corresponding currentTradingPeriod counterpart are offset by 24 hours
(tradingPeriods.post.end - currentTradingPeriod.post.end) / 60 / 60 = 24
// same 24 hour offset for `regular`, and `pre` as well

I'm not sure what all this means, but figured it'd be helpful to post the observation.

Thanks, @huned, for your observations.  Glad to have that recorded here.  Ok sounds good, let's see what else comes up with some use :pray:

For those following, this came out (still as _chart) in 2.1.0 today.

Thanks @gadicc 🥇 Been using it daily without problems.

Ok, great!  Thanks amazing news!  And really helpful to know.  Thanks for reporting back :pray:

Hey guys, I started following this closely now and I'm super interested in it as well, my main goal in the future is to be able to request historical data of multiple symbols in one call to avoid Unauthorized error/bans.
I even contacted aforementioned @ZehCoque directly as we're both from the same country and have our own edge cases when it comes to Brazil's stock exchanges (.SA). We are working on similar projects and can benefit from combined knowledge.
I've been delving into the code of this project for the past couple hours and I feel more confident now. I'm not well versed into TypeScript yet but I can help on whatever is needed. I actually never participated actively in the developing of a community library (lone wolf programmer), so this could be a great learning for me as well.

So, I am getting a validation error with symbol NUBR33.SA, sometimes Yahoo have information, but sometimes, like now, they have no historical info, so firstTradeDate is null while it's expecting  date.
I can, of course, skip validation or catch the error, but I just wanted to point that out if you decided to do something about it.
The following result did not validate with schema: #/definitions/ChartResultObject
[
  {
    keyword: 'yahooFinanceType',
    message: "Expecting date'ish but got null",
    params: { schema: 'date', data: null },
    instancePath: '/meta/firstTradeDate',
    schemaPath: '#/properties/firstTradeDate/yahooFinanceType',
    data: null
  }
]

Hey guys, so, still talking about NUBR33.SA, I found something weird that I'm trying to figure out what Yahoo is doing.
If I request a yahooFinance.historical it only gives me one day (the same happens on the the website).
Now, if I request a yahooFinance._chart with 1d intervals, it only return one day (even with longer date ranges), but if I send a request with 5m up to 4h (not 1m) it return all the data with no problem.

https://query1.finance.yahoo.com/v8/finance/chart/SPY?symbol=SPY&period1=1673409759&period2=1673582559&useYfid=true&interval=1m&includePrePost=true&events=div%7Csplit%7Cearn&lang=en-US&region=US&crumb=dOh3M4TA.5w&corsDomain=finance.yahoo.com

If you change period1 from<PHONE_NUMBER> (Tuesday, January 10, 2023 11:02:39 PM GMT-05:00) to<PHONE_NUMBER> (Thursday, January 12, 2023 9:30:00 AM GMT-05:00), the API doesn't properly return volume (0 instead of 7760866).
[
  {
    "i": 0,
    "timestamp": "2023-01-12T14:30:00.000Z",
    "close":<PHONE_NUMBER>550781,
    "open":<PHONE_NUMBER>277344,
    "high":<PHONE_NUMBER>378906,
    "low":<PHONE_NUMBER>964844,
    "volume": 0
  },

versus
{
    "i": 1234,
    "timestamp": "2023-01-12T14:30:00.000Z",
    "close":<PHONE_NUMBER>550781,
    "open":<PHONE_NUMBER>277344,
    "high":<PHONE_NUMBER>378906,
    "low":<PHONE_NUMBER>964844,
    "volume": 7760866
  },

Not a problem with your library since it's the underlying API, but makes using the API "confusing".


.where("id>?", 0) not working

Does active hash have the ability to search by >, < operators? If not, what is the preferred solution?

It doesn't. Simple solution we known is to use Enumerable#select

@joshm1204 Hi, ActiveHash where method now takes range argument so you can do something like foo.where(id: 0..Float::Infinity)


LocalStorage items are able to be tampered with

A user is able to edit the key: value pairs of localStorage items and the CheckComponents function will not realize.

I propose you make a function that will validate the actual content of the key:value pairs

localStorage key:value pairs will now have their keys checked. If a key does not exist in that specific localStorage item, but is supposed to be present, the app will redirect back to app.html with a param respective to the missing localStorage item.


NOISSUE - Influxdb batch add

Summary:
Add batch write to InfluxDB writer. Since official Go InfluxDB client currently supports only batch point add, batch size is increased in order to improve performances. Batch size, as well as the time interval for flushing the batch, can be configured using corresponding env variables.

@dusanb94 Please sync branch

Codecov Report

Merging #394 into master will increase coverage by 0.15%.
The diff coverage is 96.77%.

@@            Coverage Diff            @@
##           master    #394      +/-   ##
=========================================
+ Coverage   89.14%   89.3%   +0.15%
=========================================
  Files          44      44
  Lines        1539    1561      +22
=========================================
+ Hits         1372    1394      +22
  Misses        115     115
  Partials       52      52

Impacted Files
Coverage Δ

writers/influxdb/messages.go
92.98% <96.77%> (+4.41%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c944205...a1b6f21. Read the comment docs.


Issue582 fragpm initialisation

Fixes #582

Codecov Report

Merging #584 (a3fa98d) into master (a23ee0b) will increase coverage by 0.00%.
The diff coverage is 100.00%.

:exclamation: Current head a3fa98d differs from pull request most recent head 72c7132. Consider uploading reports for the commit 72c7132 to get more accurate results

@@           Coverage Diff           @@
##           master     #584   +/-   ##
=======================================
  Coverage   72.62%   72.63%
=======================================
  Files          96       96
  Lines       11194    11197    +3
=======================================
+ Hits         8130     8133    +3
  Misses       3064     3064

Impacted Files
Coverage Δ

R/g.fragmentation.R
93.16% <100.00%> (+0.17%)
:arrow_up:

R/g.part5.R
67.27% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a23ee0b...72c7132. Read the comment docs.


Method interp option order is not taken into account if from_ds is selected

The way interp is implemented right now, if from_ds is selected, the option "order" is not taken into account. Only nearest neighbor is implemented in this case. See e.g. lines 926-930:
if from_ds:
b = self.ds.GetRasterBand(band)
z_interp = np.array([b.ReadAsArray(int(xp),int(yp),1,1)[0,0] for (xp,yp) in zip(xi,yi)])
else:
z_interp = ndimage.map_coordinates(self.r[:,:,bands],
[yi, xi], order=order)
Lines to be updated: 927, 936 and 946.

In the same way, warning on lines 865-866 should be removed.


refactor(backend): deduplicate literal lowering

Create a function, literal_to_py, to convert AST literals (numbers and strings) into corresponding Python literals. Use this function for printf arguments, making the code more readable.

@strager sure! Makes sense to me! Thanks! :+1:


clientv3/integration: do not remove client member in TestMemberRemove

Fixes #4716

lgtm

lgtm. thanks.

@heyitsanthony This does not fix the double panic one mentioned in #4716, right?

@xiang90, the double panic doesn't seem to happen with this fix, but I imagine it's still there if the client asks the server to remove itself. Probably needs a separate test case.


feat(evaluation): Integrate BEIR evaluation

Description
Refer to rendered notebook: https://github.com/jon-chuang/llama_index/blob/jon-chuang/beir/docs/examples/evaluation/BeirEvaluation.ipynb
Fixes: https://github.com/jerryjliu/llama_index/issues/6554
Also partially implements: https://github.com/jerryjliu/llama_index/issues/6797
Type of Change

[x] New feature (non-breaking change which adds functionality)
[ ] This change requires a documentation update

maybe? Or is the notebook in docs/examples good enough?

How Has This Been Tested?

[x] Added new notebook (that tests end-to-end)

Suggested Checklist:

[X] I have performed a self-review of my own code

man, this is neat @jon-chuang! Are you planning on adding a doc explaining as well? Using BEIR comes with pros and cons so we have to make sure the user is aware of those.

Yes, indeed, even interpreting the eval result is non trivial. I intend to provide more details so that it can be useful to the general user. But my sense is that the corpuses are quite large for a quick dev cycle. At least at the current level of embedding speed. If the serving speed can be increased, it might be fine.
For the larger corpuses, what I intend to do is to make a "corpus shrinker". By using the qrels, I can identify which questions depend on which corpus documents. Then, I can shrink the corpus arbitrarily, while maintaining a representative set of questions.
The main workflow that I intend for is for relative comparison between two retrieval methods.

But my sense is that the corpora are quite large for a quick dev cycle

I agree with you on this. It's also not a good practice if people overfit to public datasets because they don't guarantee performance on the user's production distribution but are helpful clues.
for eg if I'm making a medical application, BEIR has a few medical datasets that I can benchmark against and get a score. much better than having no benchmark :)


Can not collapse dom element manually in jsx file

Issue Type: Bug

create a jsx file
write some dom element code in render method, like

  something here

can't collapse  div element when mouse hovers on the line where div is it
can reproduce after disabling all extension

VS Code version: Code 1.22.1 (950b8b0d37a9b7061b6f0d291837ccc4015f5ecd, 2018-04-06T02:26:57.615Z)
OS version: Windows_NT x64 10.0.16299

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-2300 CPU @ 2.80GHz (4 x 2808)

Memory (System)
15.96GB (8.75GB free)

Process Argv
C:\Program Files\Microsoft VS Code\Code.exe

Screen Reader
no

VM
67%

Extensions: none
![image](https://user-images.githubusercontent.com/13702561/38594707-32ec35a0-3d7a-11e8-8dd2-0d9002248a24.png)

Duplicate of https://github.com/Microsoft/TypeScript/issues/23273
Set "typescript.experimental.syntaxFolding": false to disable syntax based folding


Numerically inaccurate scipy.stats.norm.sf

Description
Since a distribution cdf's goes asymptotically to 1 as $x \to \infty$, computing it numerically rounds to 1 at a certain point. For this reason, distributions in scipy.stats also provide the method sf = 1 - cdf that retains the precision.
I was about to use jax.scipy.stats.norm.sf, but I noticed that it is not accurate:
from jax.scipy import stats as jstats
from scipy import stats
import numpy as np

value = np.array(10, np.float32)
print('scipy:', stats.norm.sf(value))
print('jax:', jstats.norm.sf(value))

scipy: 7.61985302416047e-24
jax: 0.0

Indeed, looking at the code, the implementation just does 1 - cdf:
@_wraps(osp_stats.norm.sf, update_doc=False)
def sf(x: ArrayLike, loc: ArrayLike = 0, scale: ArrayLike = 1) -> Array:
  cdf_result = cdf(x, loc, scale)
  return lax.sub(_lax_const(cdf_result, 1), cdf_result)

For the normal distribution, one can use sf(x) = cdf(-x). I have not looked at what jax is doing for other distributions.
What jax/jaxlib version are you using?
0.4.14
Which accelerator(s) are you using?
CPU
Additional system info
Python 3.11.2, macOS 13.4
NVIDIA GPU info
No response

Thanks for the report! #17204 fixes this for norm.sf; I'll look into similar fixes for the other distributions as well.


Methodology questions

Hey friends,
First and foremost, the Methodology doc is great, and there is a lot here to unpack. But before I delve into details, I wanted to ask if the methodology is purposely meant to narrow scope towards Kubernetes, or if it was just a matter of initial scope.
The reason I ask is because the way I read the methodology, is that it could well be tailored to other scenarios such as Cloud PaaS adoption (instead of Kubernetes specifically).
When I look at Konveyor, I see a Platform for Migration and Modernization, where tools and methodologies can help teams achieve their goals regardless of the deployment target; in other words, it may or may not be Kubernetes!
Hope this sparks a great discussion!
Bruno

Hi Bruno,
You are spot on! First of all, let me tell you that this document is quite outdated at the moment, and we haven't made any progress on it for a while. Almost a couple years ago, Konveyor transitioned from being an umbrella community with multiple projects under it (Tackle, Forklift, Crane and Move2kube among others) to the fully fledged and unified project that it is today. The conductive thread across all these projects was Kubernetes adoption, and in the current project it remains a primary target, but things are open ended for the adoption of cloud native technologies in general.
So going back to the methodology itself, yes, I think it's not restricted to k8s for the most part (although there are some specific things related to it), and it could be applied to multiple scenarios. Originally this was thought as a way to somehow open source the migration and modernization methodology we had used at Red Hat Consulting for years, to use it as a starting point for a discussion with other vendors, GSIs and practitioners to come up with something that could be universally used in these scenarios with the support of the Konveyor tooling.
I would be glad to get back to the drawing board and refine (or redefine!) this with your help, and maybe incorporate some of the approaches and techniques your field practitioners use to tackle large migration and modernization engagements.


Stateful predictions of RNN

Hi, I have read the Keras doc about "stateful flag" and several issues. However, I still don't understand how the stateful prediction works.

Notes that the methods predict, fit, train_on_batch, predict_classes, etc. will all update the states of the stateful layers in a model. This allows you to do not only stateful training, but also stateful prediction.

For instance, the size of test data is (2000,10,3), where 2000 is the nb_num,10 is sequence_length,3 is input_dim. The size of test label is (2000,2), where 2 is the label_dim.
When I use predict_classes, every time I label a batch of （１，１０，３）  ｓｅｑｕｅｎｃｅs. However,  the parameters of the network is fixed when predicting, what does the state mean?How can the network pass  the state on to next  batch?
The second question. Think there are several sequences  A,B,C,D...
I want to give sequence B a label based on the label of A. So should I put the sequence A in the first batch, and sequence B in the same place in the second batch? Is there any other method?
The third question.Why does Keras always split the long sequences into shorter ones?If I want to give every timestep a label, I should use TimedistributedDense.But is it possible to process the whole long sequences?
Thank you in advance!

@fchollet @EderSantana I'm sorry to bother you, but can help me with my questions?

I'll try to help:

what does the state mean?How can the network pass the state on to next batch?

when you first create your model, it guesses randomly. after it's trained, it ideally guesses more accurately. the 'state' is the training (the weights that have been set via training). This means you can instantiate a model, train it, and use that instance of the model to perform predictions on properly shaped inputs.

The second question. Think there are several sequences A,B,C,D...
I want to give sequence B a label based on the label of A. So should I put the sequence A in the first
batch, and sequence B in the same place in the second batch? Is there any other method?

If you want A and B to be related as a sequence of data in the prediction of B's label, I think those need to be input together as a sequence. Ideally the order of the data points in the training sequence don't matter at all, and you can shuffle them freely.

The third question.Why does Keras always split the long sequences into shorter ones?If I want to give > every timestep a label, I should use TimedistributedDense.But is it possible to process the whole long
sequences?

This sounds potentially like a programming issue — can you post a gist?

@bhtucker Thank you so much for your help!
I still have something to ask about your reply.

the 'state' is the training
Do you mean that there is no  'state ' when predicting? The network predict every sequence separately? Keras document  says:

Notes that the methods predict, fit, train_on_batch, predict_classes, etc. will all update the states of the stateful layers in a model. This allows you to do not only stateful training, but also stateful prediction.
But I don't understand how stateful prediction works.

I understand your reply about my second question,thank you !
The third question,actually I want to process some prediction problems.Like the one in the Keras document predicting the 11th timestep given the first 10.
X  # this is our input data, of shape (32, 21, 16)
# we will feed it to our model in sequences of length 10
model = Sequential()
model.add(LSTM(32, batch_input_shape=(32, 10, 16), stateful=True))
model.add(Dense(16, activation='softmax'))
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
# we train the network to predict the 11th timestep given the first 10:
model.train_on_batch(X[:, :10, :], np.reshape(X[:, 10, :], (32, 16)))
# the state of the network has changed. We can feed the follow-up sequences:
model.train_on_batch(X[:, 10:20, :], np.reshape(X[:, 20, :], (32, 16)))
In the first  train_on_batch, the network predict the 10th timestep. In the second  train_on_batch, the network predict the 20th timestep. So the network don't need to predict the 2th,3th,4th ...timestep?
Looking forward for your reply.

Here's an example that does character-by-character prediction / text generation via sampling: https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py
In particular maybe this part:
            preds = model.predict(x, verbose=0)[0]
            next_index = sample(preds, diversity)
            next_char = indices_char[next_index]

            generated += next_char

@bhtucker Thank you !

when you first create your model, it guesses randomly. after it's trained, it ideally guesses more accurately. the 'state' is the training (the weights that have been set via training).

You are confusing the 'state' and the 'weights'. Let me try to explain...
On each timestep each node in an RNN receives some input and produces 2 outputs. One output goes to the next network layer, the other is remembered until the next timestep and is then combined with the following input. These 'internal outputs' let it learn relationships in the data that span several timesteps. This internal 'state' is not trainable, just as for the output that is fed into the next network layer, it is the weights in the formulas used to calculate it that are trained.
The best way of understanding the statefulness it to imagine what would happen if each batch contained sequences of only one timestep. You would want the network to remember its internal state between the first batch and the second batch, rather than starting over at each step with its internal state set to all zeros. Otherwise the network would be unable of learning any time-based relationships in the input data.
A non-stateful RNN cannot learn any long running relationships in the input data that are longer than the training sequence length. That is where the statefulness can come in useful. You can feed a stateful RNN the first 10 steps in one batch, then the next ten steps in the second batch, and you can be sure that the internal state of the network after the first ten predictions have been calculated is used to prime the network for the second set of ten timesteps. Thus the network can learn relationships in the input data spanning up to twenty steps even though the sequence length of each batch is only ten steps. If you add a third set of 10 timesteps that follow on from the second set, the network can learn relationships spanning up to thirty steps.
If you had data for each day of the year your first batch could consist of the data for Jan 1st, Feb 1st, Mar 1st, ... Dec 1st. The second batch would then be Jan 2nd, Feb 2nd, ... Dec 2nd. Each element of each batch must follow on from the corresponding element in the previous batch. The batch size in this example is 12 and the sequence length is 1.

@jpeg729 Thanks for the correction — I missed that the question was about the more precise sense of 'state' and not the more general model-weights-as-state.
It looks like in the RNN literature the state at t is typically h_sub_t, so I'll call this recurrent state h. I also have a few questions about h:

Can you set the dimensionality of h? Or is always some function of some other dimensionality parameters?
In a classification task using RNNs, like the imdb sentiment analysis example, is the RNN's output to the next layer simply the value of h at the end of that sequence?
Some RNN papers (like http://arxiv.org/pdf/1409.2329.pdf) refer to having 600+ unit RNN layers. I understand that h is passed along and modified by each unit, but do these units all share values for their other weights (for GRUs, w and u)? Is there a way of creating a 600 unit RNN layer in Keras, or am I misunderstanding something?

Appreciate your help!

As far as I understand it, each GRU node or LSTM node stores one value. The dimensionality of h doesn't seem to be a useful question.
No. At time t the RNN's input is (input_sub_t, h_sub_t_minus_one) and its output is output_sub_t. h_sub_t is stored internally while the network awaits the next input set.
Each LSTM/GRU node stores its own value which it uses on the next timestep. The LSTM/GRU nodes in a layer do not share weights or state, each node has its own weights and its own state. model.add(LSTM(600)) would do the trick.

@jpeg729 Thank you for your help. But I think my problem is still not solved.Can you explain the sentence in Keras document below?Ｔｈａｎｋ you!

Notes that the methods predict, fit, train_on_batch, predict_classes, etc. will all update the states of the stateful layers in a model. This allows you to do not only stateful training, but also stateful prediction.

It means that just as training can be done by feeding the model one sequence step at a time, so you can also predict by just giving the network one sequence step at a time. If the model is stateful, then it can "remember" some aspects of the previous steps in the sequence.
This is rather useful when using the model for real-time predictions of a series.

Any body having character based text classification code in keras like in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/skflow/text_classification_character_rnn.py

Hi @jpeg729 ,
I have a question regarding to batch_shape in the first layer of stateful LSTM network.
As fas as i know, batch_shape specifies (batch_size, time_step, input_dim) for LSTM network, if we do stateful, we can set batch_size to 1, but how about time_step? can we set time_step to 1 as well?
let's say I'm trying to build a character level text classification with stateful LSTM,
in this case should the batch_shape be (1, 1, 26)? (where 26 is number characters)
Thank you!

If you do stateful you can still use batch_size > 1, and time_steps > 1.
The simplest case is of course, batch_size == 1 and time_steps == 1, and in that case you feed the model one character of a single sentence at a time.
If you use time_steps == 10, for example, then you must feed the sentence to the model 10 characters at a time, the first call must provide the first 10 characters of the sentence, the second call must provide the next 10 characters of the sentence and so on.
You can use batch_size > 1, in which case you feed the model several sentences at once. The first call provides the first time_steps characters of each sentence, the second call provides the next time_steps characters and so on.
I hope this is helpful.

Hi @jpeg729 ,
Thank you for your answer! this is very helpful! so my understanding is batch_size won't affect the performance of LSTM, the only different is if we use a large batch size, we would experience some training speed up on GPU?
Regarding to time_steps==1, if we pass one character at a time, it seems like we are not utilizing the advantage of LSTM, because we don't do BPTT in training? I assume the performance of time_steps==1 will be worse than n time steps?
Please correct me if I'm wrong..
Thank you!

I think a smallish batch_size would provide some speedup even on CPU since modern CPUs have some ops for vector/matrix calculations.
I think you are right about loosing BPTT when using time_steps == 1. However, you will get some useful training since the hidden state is preserved.
I have tried batch_size between 10 and 20, with time_steps >= 50. Those settings seem to do useful work at reasonable speed.

Hi @jpeg729 , sorry I'm still confusing about the batch size,
let's say we have a two sentences, one is
A B C D another one is E F G H, and the time step of LSTM is 2, so I have to divide this length 4 sequence into 2 length 2 subsequences. After cut, I got
A B, C D, E F, G H, and my training batch looks like (imagine it already converted to one-hot encoding or labels)
[A B, C D, E F, G H]
here is the part confuse me, should I pass this [A B, C D, E F, G H] to fit() function with batch_size = 2 and reset states every 2 batches (using a call back).
or should I created two chunks:
[A B, E F]
and
[C D, G H] and pass those small batches one after another and reset the state every 2 small batches?
Does it make sense?..
Thank you!

@jpeg729 When predict using stateful Lstm, does predict() set the state variable?  If so, to predict a value, should we predict the past samples N times to get the state produced from the previous predictions? Thanks

@lmxhappy Do you have an answear for your problem yet?


TSC Meeting 2024-04-18

Please respond to this issue with items you would like to see added to the agenda.
Items from attendees of TSC Meetings are prioritized above others, though ad-hoc reprioritization and discussion will occur.
Current agenda below:

ASWF

TAC Project Update occurred, rez renewed another year.

@isohedronpipeline topics:

https://github.com/AcademySoftwareFoundation/rez/pull/1706
https://github.com/AcademySoftwareFoundation/rez/pull/1709

Think we could circle back around to #1668 and see if we can come up with action items?

Also curious about #1475 and if we can get that one closed soon. I know that's a big one.

@maxnbk Will there still be meeting notes and screen recordings? I found that there have been no meeting notes or screen recordings for a long time.
https://github.com/AcademySoftwareFoundation/rez/tree/main/ASWF/TSC/meeting-notes
Thank you very much for the information you recorded before, which allows us to better understand the detailed of TSC discussion.

@loonghao , Yes, we're a bit behind on meeting notes. It's a lot to write ;) We'll try to get the backlog processed sooner than later.


about the  coordinate system for cartographer

hello, How is the coordinate system defined for cartographer?Does this coordinate system correspond to the coordinate system of ROS(x-forward, y-left, z-up) or am I free to define? If it is the latter, how do I define my coordinate system for cartographer?  for another word , if  I do not run cartographer under the ROS system, imu data and laser data are used as inputs，does the coordinate system for cartographer correspond to the coordinate system of my imu (x-left, y-forward, z-up)?

Are you asking about Cartographer or the corresponding ROS node?

yes, i want to know the coordinate system for cartographer,not for the corresponding ros node.

This question is better for the newsgroup. I've been trying to think how to answer your question. My first thought is right-handed Euclidean. You should be free to define the orientation. In answer to the question "should it correspond to the IMU", the answer is yes, but you may rotate the IMU data before sending it in (just don't translate it). All of the inputs need to be in the same coordinate system (so transform them all to match the IMU).

Cartographer uses the ROS coordinate system. This matters primarily for correctly interpreting range finder data.


Use AndroidX instead of Support Library

Fixes #176.

Are you sure this would build without changing the gradle files and it's not only working because of some custom configuration ?

@Titozzz Not sure what you mean. Could you elaborate?

So I checked with some people, we shouldn't not use androidX and old support libraries at the same time. So since react-native did not update yet, this will stay on hold. Will merge as soon as RN gets it :)

Since react-native has updated to androidx, can we expect to see this merge soon?

This was released in version 6.X


fix setting active property

omg, API doesn't allow to set active. ;o)

API fixed, build green ;o)


Unhandled Java Exception in create_output():

I just want to test pdffiler, if my task can be done by this module. BUt after installing pdftk and code the example into my project, I get this error:
> exec error: Error: Command failed: pdftk xxx.pdf fill_form temp_data1508163976446ybvrr.fdf output test_complete.pdf flatten
Unhandled Java Exception in create_output():
java.io.CharConversionException
   at 0x0059a84e (Unknown Source)
   at 0x0059ad42 (Unknown Source)
   at 0x0059adf3 (Unknown Source)
   at 0x005e9bed (Unknown Source)
   at 0x00590c8d (Unknown Source)
   at 0x006ccc5d (Unknown Source)
   at 0x0069eb80 (Unknown Source)
   at 0x005958f6 (Unknown Source)
   at 0x00599492 (Unknown Source)
   at 0x005994cc (Unknown Source)
   at 0x004ba290 (Unknown Source)
   at 0x0046ee85 (Unknown Source)
   at 0x0046e7a0 (Unknown Source)
   at 0x0046deff (Unknown Source)
   at 0x00430bfa (Unknown Source)
   at 0x0043496f (Unknown Source)
   at 0x004344c7 (Unknown Source)

C:\Users\XXX\node_projects\xxx.js:428
        if (err) throw err;
                 ^

Error: Command failed: pdftk xxx.pdf fill_form temp_data1508163976446ybvrr.fdf output test_complete.pdf flatten
Unhandled Java Exception in create_output():
java.io.CharConversionException
   at 0x0059a84e (Unknown Source)
   at 0x0059ad42 (Unknown Source)
   at 0x0059adf3 (Unknown Source)
   at 0x005e9bed (Unknown Source)
   at 0x00590c8d (Unknown Source)
   at 0x006ccc5d (Unknown Source)
   at 0x0069eb80 (Unknown Source)
   at 0x005958f6 (Unknown Source)
   at 0x00599492 (Unknown Source)
   at 0x005994cc (Unknown Source)
   at 0x004ba290 (Unknown Source)
   at 0x0046ee85 (Unknown Source)
   at 0x0046e7a0 (Unknown Source)
   at 0x0046deff (Unknown Source)
   at 0x00430bfa (Unknown Source)
   at 0x0043496f (Unknown Source)
   at 0x004344c7 (Unknown Source)

    at ChildProcess.exithandler (child_process.js:270:12)
    at emitTwo (events.js:125:13)
    at ChildProcess.emit (events.js:213:7)
    at maybeClose (internal/child_process.js:897:16)
    at Socket.stream.socket.on (internal/child_process.js:340:11)
    at emitOne (events.js:115:13)
    at Socket.emit (events.js:210:7)
    at Pipe._handle.close [as _onclose] (net.js:549:12)

My example code looks like this:
   var pdfFiller   = require('pdffiller');     var sourcePDF = "xxx.pdf";     var destinationPDF =  "destination.pdf";     var data = {         "prename": "Name written",         "clientlastname": "lastname written"     };     pdfFiller.fillForm( sourcePDF, destinationPDF, data, function(err) {         if (err) throw err;         console.log("In callback (we're done).");     });
What should I change?

Hi, did you found the solution? I have the same issue

I'm having the same problem, couldn't find a solution

Change the font of the fields to either Helvetica, Times Roman or Courier, pdftk will successfully create a PDF.
https://stackoverflow.com/questions/23948647/font-issue-with-pdftk


feat: improve interaction of Public Datasets selector

Users now receive immediate feedback and reduce confusion when selecting datasets.
Screen Recoding
Before
https://github.com/Kanaries/graphic-walker/assets/125171539/8b1edaec-21ff-4792-9be5-4663e6a808cc
After
https://github.com/Kanaries/graphic-walker/assets/125171539/5db7a49c-d59a-4f1b-b7e4-6e548aec2fb9

Hi, thanks for the PR. It is really a helpful feature. But would you mind keeping the tab size = 4 as default for this commit?

Hi, thanks for the PR. It is really a helpful feature. But would you mind keeping the tab size = 4 as default for this commit?

I just implemented the new formatting, but it seems not correct, let me update the formatting in another PR 🙏.


FIX: Darkness

Before:

After:

And on Linux, before:

After:

FYI this requires darkdetect and qdarkstyle

@hoechenberger now that https://github.com/mne-tools/mne-python/pull/10414 is merged in MNE-Python, you should be able to test here. You should end up with a slightly blue-ish gray instead of plain gray if things are working properly, just like in Brain.

@marsipu this is ready for review/merge from my end now!

That looks great, thank you @larsoner


Clarify Versioning CLI Commands

What was changed

Remove redirect rule reference in commit-build-id, fix spelling
Improve --select-build-id flag description

Why?

commit-build-id does not touch redirect rules, only assignment rules
add optional behavior of --select-build-id

Checklist

Closes

How was this tested:

Any docs updates needed?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix: fix overlap of tracked objects information on rviz

Description
Fix rviz overlap of tracked objects information on rviz.

before

after

Related links

Tests performed

Notes for reviewers

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

How about this image?

@scepter914 thank you!
I think it's good from BEV, but from view point of a third person it would be out of alignment.
It may be better to merge into one string and publish
Same comments :pray:
https://github.com/autowarefoundation/autoware.universe/pull/1014#discussion_r887553023

@yukkysaito
Sorry, the picture is from another autoware environment and I delete the message.
So I'll share the picture again.

It was my misunderstanding. Same result come out and This PR is correct.
(I misunderstood in https://github.com/autowarefoundation/autoware.universe/pull/1014, so I'll fix it.)

It may be better to merge into one string and publish
I'll improve it.

Time has passed, so I will issue another PR.


Could not find 'bundler' (>= 0.a) among 43 total gem(s) (Gem::LoadError)

Gitlab-ce version:8.3.3
CentOS: 6.7 x86_64
$ sudo bundle install --deployment --without development test postgres aws kerberos
/usr/local/lib/ruby/site_ruby/2.2.0/rubygems/dependency.rb:318:in to_specs': Could not find 'bundler' (>= 0.a) among 43 total gem(s) (Gem::LoadError) Checked in 'GEM_PATH=/git/home/.gem/ruby/2.2.0:/usr/local/lib/ruby/gems/2.2.0', execute gem envfor more information         from /usr/local/lib/ruby/site_ruby/2.2.0/rubygems/dependency.rb:327:into_spec'
from /usr/local/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_gem.rb:64:in gem'         from /usr/local/bin/bundle:22:in '
$ gem env
RubyGems Environment:

RUBYGEMS VERSION: 2.5.1
RUBY VERSION: 2.2.4 (2015-12-16 patchlevel 230) [x86_64-linux]
INSTALLATION DIRECTORY: /usr/local/lib/ruby/gems/2.2.0
USER INSTALLATION DIRECTORY: /git/home/.gem/ruby/2.2.0
RUBY EXECUTABLE: /usr/local/bin/ruby
EXECUTABLE DIRECTORY: /usr/local/bin
SPEC CACHE DIRECTORY: /git/home/.gem/specs
SYSTEM CONFIGURATION DIRECTORY: /usr/local/etc
RUBYGEMS PLATFORMS:

ruby
x86_64-linux

GEM PATHS:

/usr/local/lib/ruby/gems/2.2.0
/git/home/.gem/ruby/2.2.0

GEM CONFIGURATION:

:update_sources => true
:verbose => true
:backtrace => false
:bulk_threshold => 1000
:sources => ["https://ruby.taobao.org/"]

REMOTE SOURCES:

https://ruby.taobao.org/

SHELL PATH:

/usr/local/bin
/bin
/usr/bin
/usr/local/sbin
/usr/sbin
/sbin
/git/home//bin

So.. the issue was?


Update Restore-RecoverableItems to include PurgedItems switch

Update Restore-RecoverableItems to include PurgedItems switch with SourceFolder

OPS Build status updates of commit 6ede08d:
:clock10: Incremental building: average incremental building time is 12 min(s) 22 sec(s)

OPS Build status updates of commit 6ede08d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mailboxes/Restore-RecoverableItems.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

as per the article Restore-RecoverableItems, It only support recovering messages from DeletedItems or RecoverableItems.
However when I tested I noticed we can also use Purgeditems as a source folder parameter

OPS Build status updates of commit 7715628:
:clock10: Preparing: average preparing time is 1 min(s) 12 sec(s)

OPS Build status updates of commit 7715628:
:clock10: Incremental building: average incremental building time is 12 min(s) 22 sec(s)

OPS Build status updates of commit ddd39db:
:clock10: Preparing: average preparing time is 1 min(s) 12 sec(s)

OPS Build status updates of commit ddd39db:
:clock10: Incremental building: average incremental building time is 12 min(s) 22 sec(s)

OPS Build status updates of commit ddd39db:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mailboxes/Restore-RecoverableItems.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

OPS Build status updates of commit ff0c55a:
:clock10: Preparing: average preparing time is 1 min(s) 12 sec(s)

OPS Build status updates of commit ff0c55a:
:clock10: Incremental building: average incremental building time is 12 min(s) 22 sec(s)

OPS Build status updates of commit ff0c55a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mailboxes/Restore-RecoverableItems.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Memory leak] OpenGL graphics device retains strong reference to disposed command lists

The OpenGL graphics device retains strong references to disposed command lists.
The _submittedCommandListCounts dictionary is keyed to OpenGLCommandList, and keys are not removed when the command list is disposed.
This results in memory leaks. When using ImageSharpTexture a considerable number of temporary command lists can be created when uploading many textures, resulting in runaway memory allocations that eventually cause an OutOfMemoryException.
The graphics device methods EnqueueDisposal and CheckCommandListDisposal need to be updated to remove the command list from the dictionary.

I made a couple of changes:

OpenGLGraphicsDevice.DecrementCount now removes the CommandList if it's reference count reaches 0.
ImageSharpTexture doesn't use CommandLists / staging Textures anymore. The way it was implemented required a call to GraphicsDevice.WaitForIdle, or resources would not be destroyed even with the first fix above.

I checked the heap snapshot after loading a map, there is now only one command list loaded. Textures look good, so ImageSharpTexture is working as expected as well.
Thanks for the quick response.

Sure thing, thanks again for the bug report.


Encountering a typeerror while fitting the lightweightmmm model

I was just trying to run the example notebook - end_to_end_demo_with_multiple_geos as is without any modification.
The part where I run the following:
mmm.fit(
media=media_data_train,
media_prior=costs,
target=target_train,
extra_features=extra_features_train,
number_warmup=number_warmup,
number_samples=number_samples,
seed=SEED)
It throws an error: Typeerror :
TypeError: where() got some positional-only arguments passed as keyword arguments: 'condition, x, y'
The error is pointing to the following code snippet from the media_transforms.py:
@jax.jit
def apply_exponent_safe(
data: jnp.ndarray,
exponent: jnp.ndarray,
) -> jnp.ndarray:
"""Applies an exponent to given data in a gradient safe way.
More info on the double jnp.where can be found:
https://github.com/tensorflow/probability/blob/main/discussion/where-nan.pdf
Args:
data: Input data to use.
exponent: Exponent required for the operations.
Returns:
The result of the exponent operation with the inputs provided.
"""
exponent_safe = jnp.where(data == 0, 1, data) ** exponent
return jnp.where(data == 0, 0, exponent_safe)
Would appreciate any help in troubleshooting/debugging this. Has anyone faced this issue before?

Hey there! somehow the code installed isn't the one showed on the repo. You can try to upgrade it with
pip install --upgrade git+https://github.com/google/lightweight_mmm.git
with this you should be able to run the notebook


[DRAFT][spi_device] Draft to reproduce bit error

Bit test is not behaving as expected.
Command:
./util/dvsim/dvsim.py hw/ip/spi_device/dv/spi_device_sim_cfg.hjson -i spi_device_byte_transfer --fixed-seed 999 --tool xcelium --waves --xprop-off -v h --max-parallel 1
Signed-off-by: kosta-kojdic<EMAIL_ADDRESS>

@kosta-kojdic
I rerun your test with VCS. From the waveform, it sends 3 bits (data 'b110) in the 1st SPI transfer and then sends another SPI transfer which also starts with 'b110. Also, we can see the async_fifo_level didn't drop after the 1st SPI transfer, and it decreased after each 8 bits transfer. The waveform looks correct to me.

A few other observation.

there are some design assertion errors. You can confirm with Eunchan if it's ok to disable them. Some assertions may assume we always send a full bytes data. We can probably disable them.
This PR has a compile error with VCS. Please run with VCS and fix it, otherwise, CI will fail.

@kosta-kojdic if you want to submit this PR, click ready for review when it's ready. Otherwise, you can close it.

This will be submitted in separate pull requests.


Generate random minutes of the goals

Once the result of a game has been simulated (i.e. 2-3, 3-3, 1-1) it will be necessary to simulate the minutes (1-90) in which the goals have been scored. I think this could be completely random, without any weight; a simple line of thought would be:

select one of the two teams;
for each goal scored by this team, generate a random number in the discrete interval {1, ..., 90};
for the next goal of this team, generate a random number in the same interval, after removing the minute of the previous goal from the interval (we'll make the simplifying assumption that two goals have to be scored in different minutes);
repeat this process for all the goals of the first team.
After that we'll do the same thing for the away team, but we'll start with an interval which already does not contain the minutes of the goals scored by the home team.

This function should be implemented in the simulation.py module.
We can assume to have two input parameters: goals_h: number of goals of the home team : int, goals_a: number of goals of the away team: int.
The function should return two ordered lists: the minutes of the goals of the home team, the minutes of the goals of the away team.

I created a pr on this


Implemented/Fixed infinite scrolling support.

Infinite scrolling now works.
In order for a user to support this they must act on the Select2 page parameter that is automatically sent in each request (except the first request). For example:
public function lookupAction(Request $request)
  $query = $request->query->get('q');
  $limit = $request->query->get('page_limit');
  $page = $request->query->get('page') ?: 1;
  $offset = ($page - 1) * $limit;
  $results = $repo->createQuerybuilder('a')
    ->setFirstResult($offset)
    ->setMaxResults($limit)
    ->where('...')
    ->getQuery()
    ->getResult();

No BC breaks

Thanks.
I've been busy but I will take a look at this and probably merge in the next few days.

I had a play with this. It's very cool but I think there's is a bit of a BC problem with more = results.length == limit. With my test code and data, I can easily make it repeat the first page endlessly. This happens if I enter a query string that returns the page_limit number of results. That assumes more so it asks for the same again. This is existing code which means that my query always starts at offset zero.
I don't think we can have more = results.length == limit or it needs to be a config setting.

I knew that bit might be a problem, but figured it would be harmless if the server side properly obeyed the page parameter. I figured, at worst, it would attempt to fetch the next page and get 0 results.
The only way to possibly work around this is to add a new config option. Maybe a scrolling boolean. When true, the javascript will paginate. When false, it'll only fetch the first set of results.

 If FALSE (default) the behavior will be the same as it was previously. If TRUE, pagination will be enabled in select2 and the server side will receive the page parameter.

Merged, thanks. I removed 'page_limit' from the $varNames array in Select2EntityType.php. We don't need that because page_limit is always in $this->config.
I updated README.md.  I've left it there for now but I don't think your statement "Note: Select2 does not send page on the first request." is true is it? I get page even on the first request. I think your JS code page: params.page || 1 guarantees that.
Sorry to sound picky. Just trying to keep things are clean as possible.

No problem. I'll edit it.


[BEAM-10708] Added an example notebook for beam_sql magic

An example notebook demonstrating the usage of the beam_sql magic.
Supported show/collect for beam_sql outputs.
Added a parser for beam_sql inputs.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @rohdesamuel
PTAL, thx!

Run Python PreCommit

Run PythonDocker PreCommit

Squashed the commits


Transformation functions

Pull Request Test Coverage Report for Build 135

138 of 216 (63.89%)  changed or added relevant lines in 10 files are covered.
6 unchanged lines in 4 files lost coverage.
Overall coverage decreased (-1.2%) to 68.509%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

skensemble_io.go
9
10
90.0%

util/util.go
16
18
88.89%

xgblinear.go
3
5
60.0%

xgblinear_io.go
6
8
75.0%

xgensemble_io.go
11
13
84.62%

leaves.go
29
35
82.86%

lgensemble_io.go
36
61
59.02%

transformation.go
16
54
29.63%

Files with Coverage Reduction
New Missed Lines
%

lgensemble_io.go
1
70.94%

skensemble_io.go
1
75.33%

xgblinear.go
1
65.38%

leaves.go
3
84.11%

Totals

Change from base Build 133:
-1.2%

Covered Lines:
1760

Relevant Lines:
2569

💛  - Coveralls


resolve fails with minimal mdns enabled on test_event_6 branch

While building python contoller with default value of mdns as minimal the resolve is failing
branch: test+event 6 commit id:    96003a703e34b4a663f45e66e4f3de92e5279e66
resolve   should work for minimal mdns as well
chip-device-ctrl > resolve<PHONE_NUMBER>770236299 1
[1633343626.774423][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343626.774831][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343626.775171][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343626.775579][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343627.775242][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343627.775555][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343627.775951][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343627.776307][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343629.777623][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343629.777878][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343629.778183][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343629.778457][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343633.780745][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343633.781056][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343633.781522][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343633.781885][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343641.782493][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343641.782745][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343641.783110][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343641.783380][43783:43791] CHIP:DIS: mDNS broadcast success
[1633343657.784759][43783:43791] CHIP:DIS: Timeout waiting for mDNS resolution.
nrf_ctrl_oct4.txt

Could you provide details to reproduce?
What HW (assume NRF board?) and what did you run where?
Did provisioning succeed before resolution?

Since our timezone does not overlap well, some extra info could help:

how was chip-device-ctrl compiled?
is avahi running on the device that runs chip-device-ctrl?
can avahi discover the given node at all?
if capturing mDNS traffic (port 5353 udp) are the broadcasts visible and does the nrf device reply to them?

Trying to narrow the issue down to something like "conflict with system avahi" or "failure to properly broadcast" or "failure to interpret broadcastt responses"

Linux TE6 branch worked fine for both provisioning and resolution:
chip-device-ctrl > resolve<PHONE_NUMBER>8394890774 178945
[1633551216.959959][15577:15584] CHIP:DIS: Attempt to mDNS broadcast failed:  ../../src/inet/IPEndPointBasis.cpp:902: OS Error 0x02000065: Network is unreachable
[1633551216.960035][15577:15584] CHIP:DIS: Attempt to mDNS broadcast failed:  ../../src/inet/IPEndPointBasis.cpp:902: OS Error 0x02000063: Cannot assign requested address
[1633551216.960265][15577:15584] CHIP:DIS: mDNS broadcast success
[1633551216.960423][15577:15584] CHIP:DIS: mDNS broadcast success
[1633551216.960562][15577:15584] CHIP:DIS: mDNS broadcast success
[1633551216.960653][15577:15584] CHIP:DIS: Attempt to mDNS broadcast failed:  ../../src/inet/IPEndPointBasis.cpp:902: OS Error 0x02000065: Network is unreachable
[1633551216.960796][15577:15584] CHIP:DIS: mDNS broadcast success
[1633551216.960906][15577:15584] CHIP:DIS: mDNS broadcast success
[1633551217.010761][15577:15584] CHIP:DIS: Node ID resolved for 0x000000000002BB01 to [fd3d:e88:d817:1d1f:33a9:cc8f:2fa0:7794]:5540
Node address has been updated

The 'Expected message type 49' is 'CASE_SigmaR2  = 0x31,'


[fix #123] PooledClient now use default_noreply arg set on init

@nichochar please review

This is great, thank you for the patch and tests @kols, it LGTM


Domain list

Add an option to scan a list of domains from a text file

Hello @p0mbe,
The feature is available in v1 but likely to be removed in v2.
Latest stable is v1.11.5 if you want to use it.


Update download page for release 0.6.0

Description
This PR updates download page to include release 0.6.0.
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

[ ] Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

[ ] Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

[ ] Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes
If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release.
If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text
Documentation
If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document

Codecov Report

Merging #6274 (c398dba) into master (1beaab5) will decrease coverage by 2.29%.
The diff coverage is 43.89%.

@@            Coverage Diff             @@
##           master    #6274      +/-   ##
==========================================
- Coverage   66.44%   64.15%   -2.30%
==========================================
  Files        1075     1248     +173
  Lines       54773    59067    +4294
  Branches     8168     8764     +596
==========================================
+ Hits        36396    37893    +1497
- Misses      15700    18411    +2711
- Partials     2677     2763      +86

Flag
Coverage Δ

unittests
64.15% <43.89%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...e/pinot/broker/api/resources/PinotBrokerDebug.java
0.00% <0.00%> (-79.32%)
:arrow_down:

...ot/broker/broker/AllowAllAccessControlFactory.java
71.42% <ø> (-28.58%)
:arrow_down:

.../helix/BrokerUserDefinedMessageHandlerFactory.java
33.96% <0.00%> (-32.71%)
:arrow_down:

...ker/routing/instanceselector/InstanceSelector.java
100.00% <ø> (ø)

...ava/org/apache/pinot/client/AbstractResultSet.java
66.66% <0.00%> (+9.52%)
:arrow_up:

.../main/java/org/apache/pinot/client/Connection.java
35.55% <0.00%> (-13.29%)
:arrow_down:

...inot/client/JsonAsyncHttpPinotClientTransport.java
10.90% <0.00%> (-51.10%)
:arrow_down:

...not/common/assignment/InstancePartitionsUtils.java
73.80% <ø> (+0.63%)
:arrow_up:

.../apache/pinot/common/exception/QueryException.java
90.27% <ø> (+5.55%)
:arrow_up:

...pinot/common/function/AggregationFunctionType.java
98.27% <ø> (-1.73%)
:arrow_down:

... and 1090 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d033a11...c398dba. Read the comment docs.

I've updated the website : https://pinot.apache.org/


Slurm check endpoint

@NatusHonestus Please make sure that all CI tasks are passing

@NatusHonestus lets also be more accurate and specific for description of PR? please describe main points of the PR in the description, what this PR for, etc.


AvatarSettingsButton v2.0.5 - fails to compile

AvatarSettingsButton v2.0.5
Stable 171425 (7ddf2be)
An error was encountered while trying to load this plugin.
BetterDiscord shows a prompt of AvatarSettingsButton fails to compile.

Compilation error of AvatarSettingsButton still present even after disabling every other plugin which could have caused compatibility issues.

Should be fixed now, thanks for notifying me!