vRouer w/DPDK and Mellanox NIC

Hi All,
I am trying to setup vRouter w/DPDK and ConnectX-3-Pro.
Is there any documentation how to?
The problem is that I cannot bring vif to recognize the DPPK enabled interface.
The current state is that install script identifies correctly and interface is binded to the DPDK.
root@Setup-H-C0:/opt/contrail/bin# ./dpdk_nic_bind.py -s
Network devices using DPDK-compatible driver
0000:05:00.0 'MT27520 Family [ConnectX-3 Pro]' drv=igb_uio unused=
Network devices using kernel driver
0000:03:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio Active
0000:03:00.3 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio
Other network devices

Since OFED + PMD drivers are taking over it is not in control of the OS any more and does not show up in ifconfig.
in - /etc/contrail/contrail-vrouter-agent.conf
DPDK or legacy work mode
platform=dpdk
Physical address of PCI used by dpdk
physical_interface_address=0000:05:00.0
MAC address of device used by dpdk
physical_interface_mac=7c:fe:90:b1:ea:50
in - /etc/contrail/agent_param
LOG=/var/log/contrail.log
CONFIG=/etc/contrail/contrail-vrouter-agent.conf
prog=/usr/bin/contrail-vrouter-agent
kmod=vrouter
pname=contrail-vrouter-agent
LIBDIR=/usr/lib64
DEVICE=vhost0
dev=p785p1
vgw_subnet_ip=VGW_SUBNET_IP
vgw_intf=VGW_INTF_LIST
LOGFILE=--log-file=/var/log/contrail/vrouter.log

Adding some more finding...
Continue our earlier call, from contrail point of view it looks all pointing to the right config, pointing to the right PCI bus/slot and MAC address.
When we exec:
root@Setup-H-C0:~# /opt/contrail/bin/dpdk_nic_bind.py -s
Network devices using DPDK-compatible driver
0000:05:00.0 'MT27520 Family [ConnectX-3 Pro]' drv=igb_uio unused=
Network devices using kernel driver
0000:03:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio Active
0000:03:00.3 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio
Other network devices

root@Setup-H-C0:~#
What is the meaning of the Active ????
From the vrouter log we can see that instead of working with 05:00.0 PCI bus/slot as defined in the setup/config it takes 03:00.0 as show in the dpdk_nic_bind.
root@Setup-H-C0:/var/log/contrail# less contrail-vrouter-dpdk-stdout.log
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   Not managed by a supported kernel driver, skipped
Nir.

“Active” might mean that it is in use by the kernel driver (I didn’t confirm by looking at the dpdk_nic_bind script,
but we see it on our setup too). Do you see an vif add messages in contrail-vrouter-dpdk-stdout.log? Does
“contrail-status” report any errors? Also, please look in the agent log for errors (/var/log/contrail/contrail-vrouter-agent.log)
Raja
From: Nir Henn<EMAIL_ADDRESS>Reply-To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Date: Sunday, September 18, 2016 at 7:34 AM
To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Subject: Re: [Juniper/contrail-vrouter] vRouer w/DPDK and Mellanox NIC (#94)
Adding some more finding...
Continue our earlier call, from contrail point of view it looks all pointing to the right config, pointing to the right PCI bus/slot and MAC address.
When we exec:
root@Setup-H-C0:~# /opt/contrail/bin/dpdk_nic_bind.py -s
Network devices using DPDK-compatible driver
0000:05:00.0 'MT27520 Family [ConnectX-3 Pro]' drv=igb_uio unused=
Network devices using kernel driver
0000:03:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio Active
0000:03:00.3 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio
Other network devices
root@Setup-H-C0:~#
What is the meaning of the Active ????
From the vrouter log we can see that instead of working with 05:00.0 PCI bus/slot as defined in the setup/config it takes 03:00.0 as show in the dpdk_nic_bind.
root@Setup-H-C0:/var/log/contrail# less contrail-vrouter-dpdk-stdout.log
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
Nir.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/Juniper/contrail-vrouter/issues/94#issuecomment-247851185, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFni16RBg6CpWbCEMSerS6pK8IQLOM_Iks5qrUwQgaJpZM4J_27s.

You can see that the DPDK boot binds to the wrong interface even that in then config it is the correct one.
Does the DPDK build is linked with Mellanox OFED drivers correctly?

echo 'Thu Sep 15 17:35:44 IDT 2016: Creating vhost0 interface with vif utility...'
Thu Sep 15 17:35:44 IDT 2016: Creating vhost0 interface with vif utility...
'[' -z 7c:fe:90:b1:ea:50 ']'
'[' -z 0000:05:00.0 ']'
++ date
echo 'Thu Sep 15 17:35:44 IDT 2016: Adding p785p1 interface with vif utility...'
Thu Sep 15 17:35:44 IDT 2016: Adding p785p1 interface with vif utility...
vif --add 0 --mac 7c:fe:90:b1:ea:50 --vrf 0 --vhost-phys --type physical --pmd --id 0
vRouter(Response): No such file or directory (2)
'[' 0 '!=' 0 ']'
++ date
echo 'Thu Sep 15 17:35:44 IDT 2016: Adding vhost0 interface with vif utility...'
Thu Sep 15 17:35:44 IDT 2016: Adding vhost0 interface with vif utility...
vif --add vhost0 --mac 7c:fe:90:b1:ea:50 --vrf 0 --type vhost --xconnect 0 --pmd --id 1
vRouter(Response): No such file or directory (2)
'[' 0 '!=' 0 ']'
++ date
echo 'Thu Sep 15 17:35:44 IDT 2016: Done starting vRouter/DPDK.'
Thu Sep 15 17:35:44 IDT 2016: Done starting vRouter/DPDK.
return 0
return 0
Thu Sep 15 17:35:44 IDT 2016: Value VGW_SUBNET_IP

We have not tested with Mellanox NICs. We use DPDK 2.1. I am not sure if it has the drivers for Mellanox NICs.

You can see that the DPDK boot binds to the wrong interface even that in then config it is the correct one.
Does the DPDK build is linked with Mellanox OFED drivers correctly?

echo 'Thu Sep 15 17:35:44 IDT 2016: Creating vhost0 interface with vif utility...' Thu Sep 15 17:35:44 IDT 2016: Creating vhost0 interface with vif utility...
'[' -z 7c:fe:90:b1:ea:50 ']'
'[' -z 0000:05:00.0 ']' ++ date
echo 'Thu Sep 15 17:35:44 IDT 2016: Adding p785p1 interface with vif utility...' Thu Sep 15 17:35:44 IDT 2016: Adding p785p1 interface with vif utility...
vif --add 0 --mac 7c:fe:90:b1:ea:50 --vrf 0 --vhost-phys --type physical --pmd --id 0 vRouter(Response): No such file or directory (2)
'[' 0 '!=' 0 ']' ++ date
echo 'Thu Sep 15 17:35:44 IDT 2016: Adding vhost0 interface with vif utility...' Thu Sep 15 17:35:44 IDT 2016: Adding vhost0 interface with vif utility...
vif --add vhost0 --mac 7c:fe:90:b1:ea:50 --vrf 0 --type vhost --xconnect 0 --pmd --id 1 vRouter(Response): No such file or directory (2)
'[' 0 '!=' 0 ']' ++ date
echo 'Thu Sep 15 17:35:44 IDT 2016: Done starting vRouter/DPDK.' Thu Sep 15 17:35:44 IDT 2016: Done starting vRouter/DPDK.
return 0
return 0 Thu Sep 15 17:35:44 IDT 2016: Value VGW_SUBNET_IP

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/Juniper/contrail-vrouter/issues/94#issuecomment-248061798, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFni13wYKG6f9pm_AIqr8hxcGPEx11wvks5qrsbAgaJpZM4J_27s.

I have also looked in a Juniper Contrail install instance we have and it looks like it uses DPDK 2.1 which do have reference to mlnx4 driver. There is a procedure from Mellanox how to compile it (DODK) with mlnx but it is not clear what will be the Contrail proceedure to update the node with the new setup. Is there dependency with the DKMS ? What is the proceedure to upadate NIC type regardless the Mellanox support?

You should be able to build the contrail-vrouter-dpdk binary (with any DPDK changes if required) and
replace the image in /usr/bin with the new one, then run “service supervisor-vrouter restart”.
Raja
From: Nir Henn<EMAIL_ADDRESS>Reply-To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Date: Monday, September 19, 2016 at 12:39 PM
To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Cc: Raja Sivaramakrishnan<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [Juniper/contrail-vrouter] vRouer w/DPDK and Mellanox NIC (#94)
I have also looked in a Juniper Contrail install instance we have and it looks like it uses DPDK 2.1 which do have reference to mlnx4 driver. There is a procedure from Mellanox how to compile it (DODK) with mlnx but it is not clear what will be the Contrail proceedure to update the node with the new setup. Is there dependency with the DKMS ? What is the proceedure to upadate NIC type regardless the Mellanox support?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/Juniper/contrail-vrouter/issues/94#issuecomment-248100171, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFni13dDNBwlHYHGMW_M0D-tqPWCGPwzks5qruUGgaJpZM4J_27s.

Did anyone get this to work?
We're seeing the same issues and looks like the vRouter is not detecting our Mellanox cards:
EAL: PCI device 0000:02:00.0 on NUMA socket -1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:02:00.1 on NUMA socket -1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   Not managed by a supported kernel driver, skipped
....
Looks like it’s only looking for Intel cards, as Mellanox is being ignored and not added.
root@node-110:~# lspci -s 0000:02:00.0
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
root@node-110:~# lspci -s 0000:01:00.0
01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
in an Intel only setup it's working fine:
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:01:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:0e:00.0 on NUMA socket 0
EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
EAL:   PCI memory mapped at 0x7f5540000000
EAL:   PCI memory mapped at 0x7f5540080000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:0e:00.1 on NUMA socket 0
EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
EAL:   PCI memory mapped at 0x7f5540084000
EAL:   PCI memory mapped at 0x7f5540104000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb

I also gave it a try in the past and after speaking with Mellanox I learned it is not supported.
They might have it in future releases of Contrail.
Nir.
--
Nir Henn
Innovation Coach, AT&T Foundry
Desk   +97297765298
Mobile +972547606867
AMDOCS | EMBRACE CHALLENGE EXPERIENCE SUCCESS
This email was sent using 100% recycled electrons!
STOP texting while driving. Take the pledge at itcanwait.comhttp://www.itcanwait.com/
From: jjgrinwis<EMAIL_ADDRESS>Reply-To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Date: Saturday, February 18, 2017 at 11:37 AM
To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Cc: Nir Henn<EMAIL_ADDRESS>State change<EMAIL_ADDRESS>Subject: Re: [Juniper/contrail-vrouter] vRouer w/DPDK and Mellanox NIC (#94)
Did anyone get this to work?
We're seeing the same issues and looks like the vRouter is not detecting our Mellanox cards:
EAL: PCI device 0000:02:00.0 on NUMA socket -1
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:02:00.1 on NUMA socket -1
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
....
Looks like it’s only looking for Intel cards, as Mellanox is being ignored and not added.
root@node-110:# lspci -s 0000:02:00.0
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
root@node-110:# lspci -s 0000:01:00.0
01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
in an Intel only setup it's working fine:
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:01:00.1 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:0e:00.0 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7f5540000000
EAL: PCI memory mapped at 0x7f5540080000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:0e:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7f5540084000
EAL: PCI memory mapped at 0x7f5540104000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHubhttps://github.com/Juniper/contrail-vrouter/issues/94#issuecomment-280834244, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKB1nrABRshU3o8sSrBU1sTddycrb_Kbks5rdrvJgaJpZM4J_27s.
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at http://www.amdocs.com/email_disclaimer.asp

We have not tested this with Mellanox NICs yet.
Raja
From: jjgrinwis<EMAIL_ADDRESS>Reply-To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Date: Saturday, February 18, 2017 at 1:37 AM
To: Juniper/contrail-vrouter<EMAIL_ADDRESS>Cc: Rajagopalan Sivaramakrishnan<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [Juniper/contrail-vrouter] vRouer w/DPDK and Mellanox NIC (#94)
Did anyone get this to work?
We're seeing the same issues and looks like the vRouter is not detecting our Mellanox cards:
EAL: PCI device 0000:02:00.0 on NUMA socket -1
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:02:00.1 on NUMA socket -1
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
....
Looks like it’s only looking for Intel cards, as Mellanox is being ignored and not added.
root@node-110:# lspci -s 0000:02:00.0
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
root@node-110:# lspci -s 0000:01:00.0
01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
in an Intel only setup it's working fine:
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:01:00.1 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:0e:00.0 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7f5540000000
EAL: PCI memory mapped at 0x7f5540080000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:0e:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7f5540084000
EAL: PCI memory mapped at 0x7f5540104000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/Juniper/contrail-vrouter/issues/94#issuecomment-280834244, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFni19MiSXn9aYLz59KiXEoiDCxp39fMks5rdrvFgaJpZM4J_27s.


Generating new Deck - Play Mode

In play mode when generating a new deck the last two cards turn into black ace of clubs.
Before

After

@JoeBSong this should have been fixed when the other generate deck bug was fixed. Please confirm.

This still happens
Original prefab after generating a new deck

myDeck prefab after generating a new deck

This seems to happen when switch prefab from myDeck and orignal prefab. Confirmed fixed


queue.remove() not working as intended?

Hi! I'm not sure If I'm reading it wrong or using it improperly, but I'm trying to let the users choose what track to remove from the queue and no matter what number the user replies with, it only removes the next track after the current one playing. It does not remove the specified track.

player.queue.remove(args[0])

is what I'm using, if you need more let me know!
Any ideas?  (Using up to date version of erela.js)

The queue#remove method is working perfectly. Note the current playing track is not included when removing tracks from the queue (written in CAPS in the documentation). Queue index starts at 0. Hope this helps!

The queue#remove method is working perfectly. Note the current playing track is not included when removing tracks from the queue (written in CAPS in the documentation). Queue index starts at 0. Hope this helps!

Yeah!
args[0] is whatever the user inputs and i've been putting  higher than 0 and there's 12 songs on the queue. I'm trying to remove the 6th song in the queue, but no matter what number i put into the remove() method, it only removes the first track on the queue.

Yeah!
args[0] is whatever the user inputs and i've been putting  higher than 0 and there's 12 songs on the queue. I'm trying to remove the 6th song in the queue, but no matter what number i put into the remove() method, it only removes the first track on the queue.

Doing some tests it seems like providing undefined which you are most likely doing will remove only the first element in the array, this is because JavaScript treats all falsy values as the number 0 literally. You will probably say you're not giving undefined but JavaScript doesn't care what you do if you try to access a undefined (no value defined) value, this includes array elements and properties. You should check what the user is providing beforehand, if it exists, if it's a number, if it's in range of the queue, if all of those don't fail then remove it from the queue. Another good solution for long time would be to use TypeScript as it will catch basically all these errors and hint towards better code.

Doing some tests it seems like providing undefined which you are most likely doing will remove only the first element in the array, this is because JavaScript treats all falsy values as the number 0 literally. You will probably say you're not giving undefined but JavaScript doesn't care what you do if you try to access a undefined (no value defined) value, this includes array elements and properties. You should check what the user is providing beforehand, if it exists, if it's a number, if it's in range of the queue, if all of those don't fail then remove it from the queue. Another good solution for long time would be to use TypeScript as it will catch basically all these errors and hint towards better code.

I would like to point out that args[0] is probably a string. Try convert it before you remove it. Something like this
const index = number(args[0]) || 0  player.queue.remove(index)
Note that the the || was added incase it failed to convert

I would like to point out that args[0] is probably a string. Try convert it before you remove it.
I wasn't getting the track correctly, and it was returning the first track in the queue, rather than the track I was removing.
Thanks for the help!
I wasn't getting the track correctly, and it was returning the first track in the queue, rather than the track I was removing.
Thanks for the help! Sorry for the bother

I would like to point out that args[0] is probably a string. Try convert it before you remove it. Something like this
const index = Number(args[0]) || 0
player.queue.remove(index)
Note that the the || was added incase it failed to convert

It being a number does not matter with JavaScript, however I will be adding more strict types so this won't be an issue.

I would like to point out that args[0] is probably a string. Try convert it before you remove it. Something like this
const index = Number(args[0]) || 0
player.queue.remove(index)
Note that the the || was added incase it failed to convert

It being a number does not matter with JavaScript, however I will be adding more strict types so this won't be an issue.


OpenTelemetry Extension Should Create Spans for Parsing and Validation

While looking into performance related characteristics of my stack @jkimbo has pointed out that the current OpenTelemetry extension should create spans for other time intensive steps such as:

parsing
validation

Context is on discord: https://discord.com/channels/689806334337482765/890275147955703831/890304451338989670
I'm happy to do this work since I've a setup which allows me to verify it is all working.

Closing this now since #1274 has been merged.


hmac_drbg bounds check off by one

https://github.com/chipsalliance/caliptra-rtl/blob/main/src/hmac_drbg/rtl/hmac_drbg.sv#L370-L376
The sampling needs to be 0 < x < order, so that > needs to be >=

fixed in our internal repo.

This has propagated to main https://github.com/chipsalliance/caliptra-rtl/blame/main/src/hmac_drbg/rtl/hmac_drbg.sv#L370-L376


Philips Hue Bloom LC001

System Health details
System Information

version
core-2024.6.4

installation_type
Home Assistant OS

dev
false

hassio
true

docker
true

user
root

virtualenv
false

python_version
3.12.2

os_name
Linux

os_version
6.6.33-haos

arch
x86_64

timezone
Europe/Amsterdam

config_dir
/config

Home Assistant Community Store

GitHub API
ok

GitHub Content
ok

GitHub Web
ok

GitHub API Calls Remaining
5000

Installed Version
1.34.0

Stage
running

Available Repositories
1463

Downloaded Repositories
36

HACS Data
ok

Home Assistant Cloud

logged_in
true

subscription_expiration
5 juli 2024 om 02:00

relayer_connected
true

relayer_region
eu-central-1

remote_enabled
false

remote_connected
false

alexa_enabled
false

google_enabled
false

remote_server
eu-central-1-13.ui.nabu.casa

certificate_status
ready

instance_id
da1feb313401471ca1bd6d9cc7e03e31

can_reach_cert_server
ok

can_reach_cloud_auth
ok

can_reach_cloud
ok

Home Assistant Supervisor

host_os
Home Assistant OS 12.4

update_channel
stable

supervisor_version
supervisor-2024.06.0

agent_version
1.6.0

docker_version
26.1.4

disk_total
125.3 GB

disk_used
22.4 GB

healthy
true

supported
true

host_connectivity
true

supervisor_connectivity
true

ntp_synchronized
true

virtualization
kvm

board
ova

supervisor_api
ok

version_api
ok

installed_addons
Samba share (12.3.1), Dutch gas prices (2<IP_ADDRESS>), MQTT Explorer (browser-1.0.3), Studio Code Server (5.15.0), Matter Server (6.1.2), Terminal & SSH (9.14.0)

Dashboards

dashboards
4

resources
23

views
22

mode
storage

Recorder

oldest_recorder_run
16 juni 2024 om 09:17

current_recorder_run
25 juni 2024 om 08:57

estimated_db_size
11356.44 MiB

database_engine
sqlite

database_version
3.44.2

Checklist

[X] I have read the FAQ to see if there is a known solution to my problem.
[X] I have enabled debug logging for my installation.
[X] I have filled out the issue template to the best of my ability.
[X] This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).
[X] This issue is not a duplicate issue of currently previous issues..

Describe the issue
I'm using a Philips Hue Bloom Light wich is supported in the docs but i'm getting an error.
Screenshot from your docs:

A screenshot van zigbee2mqtt:

As you can see the model number is different and the link behind the model 7299760PH link  is: https://www.zigbee2mqtt.io/devices/7299760PH.html#philips-7299760ph
Reproduction steps
Just added the lght via yaml (as a package):
powercalc_package:
  powercalc:
    force_update_frequency: 00:01:00
    power_sensor_naming: "{} Powercalc power"
    energy_sensor_naming: "{} Powercalc energy"
    enable_autodiscovery: false
    sensors:
      - entity_id: light.lamp_boven_het_aquarium
        name: "Lamp boven het aquarium"

Debug logs
Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.powercalc.sensors.power
Bron: custom_components/powercalc/sensors/power.py:161
integratie: Powercalc (documentatie, problemen)
Eerst voorgekomen: 08:57:49 (1 gebeurtenissen)
Laatst gelogd: 08:57:49

light.lamp_boven_het_aquarium: Skipping sensor setup: Model not found in library (manufacturer: Signify Netherlands B.V., model: Hue Bloom (7299760PH))

Diagnostics dump or YAML config
No response

Should be fixed now with #2325.
Added missing alias.
Please try restarting HA.

Just restarted HA mutiple times but the error is still there.

Please try again tomorrow, might be some caching issue.
When still not solved let me know then I'll have another look.

Just restarted again and the error is still there..

I'm unsure why the auto discovery is still not working for you.
I would suggest to just select the model manually from library.
Follow these instructions: https://homeassistant-powercalc.readthedocs.io/en/latest/sensor-types/virtual-power-library.html

Followed that and now its working!


Update configure-profile-synchronization-by-using-sharepoint-active-d…

…irectory-import.md
Ref : internal Article https://internal.support.services.microsoft.com/en-us/help/2816050
If you need to import users from multiple domains, you must create multiple synchronization connections. If you have multiple domains to manage, using Profile Synchronization or FIM may be a better option

Please do not copy paste the line from internal Article shared. Editing the existing lines with "Domains" instead of "Forests" is appropriate.

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.


[FEATURE] Enhance materialized views to support streaming JOIN refresh

Is your feature request related to a problem?
Currently, Flint's auto/incremental-refresh materialized views (MVs) are limited to supporting single-table queries with aggregations and windowing functions. This lack of support for JOIN operations restricts the ability to create complex, pre-computed aggregates that involve multiple tables and thus impacts users who need to optimize such queries by a single MV for improved performance.
Specifically, there are constraints in Spark Structured Streaming and Flint SQL regarding streaming join support:

[Spark] Stream-stream joins must be non-aggregate: An exception is thrown if multiple streaming aggregations are attempted with streaming DataFrames/Datasets, resulting in the error message: "Multiple streaming aggregations are not supported with streaming DataFrames/Datasets." (Good to know)

[Spark] Equality join condition required: At least one equality join condition is necessary; cross joins are not supported. An exception message reads: "Stream-stream join without equality predicate is not supported." (Good to know)

[Flint] Window generation without aggregation: Flint currently lacks a way to generate a window without requiring aggregation. The TUMBLE function must be used in the GROUP BY clause, which may not be suitable for all use cases.  (Good to have)

[Flint] Watermark delay configuration for multiple tables: There is a need for a mechanism to configure watermark delays for multiple tables. Currently, the watermark_delay option does not support this flexibility. (Needs to be addressed)

What solution would you like?
I propose enhancing Flint's materialized view functionality to support JOIN operations within the materialized view queries. Specifically, users should be able to define materialized views that incorporate JOIN operations between multiple tables, with the aforementioned constraints alleviated by a newly introduced Windowing Table-Valued Function (TVF) in Flint SQL.
This new TVF function allows users to define windowing without the necessity to group or collapse data, while also enabling them to provide a watermark delay for each table. Below is an example showcasing the new TVF TUMBLE function:

CREATE MATERIALIZED VIEW test
AS
SELECT ...
FROM (
  SELECT
    window.startTime AS startTime,
    ...
  FROM
    TUMBLE(orders, time, '1 Day', '1 Hour')
) AS t1
JOIN (
  SELECT
    window.startTime AS startTime,
    ...
  FROM
    TUMBLE(audit, time, '1 Day', '10 Minutes')
) AS t2
ON
  t1.startTime = t2.startTime
  AND
  t1.id = t2.id
GROUP BY ...

(Appendix)
What alternatives have you considered?
An alternative solution is to create separate materialized views for each table and use the OpenSearch SQL plugin to perform JOIN operations across multiple indices. However, this approach has several drawbacks:

Functionality Limitations: The OpenSearch SQL plugin's JOIN capabilities have limitations that can affect performance and the complexity of JOIN queries, as documented here.

Performance Issues: JOIN operations using the OpenSearch SQL plugin are executed on a single node (the coordinator node) without distributed shuffling. This single-node JOIN approach can lead to performance bottlenecks and inefficiencies, especially for large-scale data or complex JOIN operations.

Increased Space Consumption: This approach requires loading data from each individual table into separate OpenSearch indices, leading to higher storage consumption due to the need for multiple indices and potentially redundant data.

Higher Maintenance Overhead: Managing multiple materialized views and ensuring data consistency across these indices can be complex, labor-intensive and costly.

This alternative does not fully address the need for efficient, complex JOIN operations within materialized views and introduces additional challenges related to storage and maintenance.
Do you have any additional context?
Apache Flint
In Apache Flint, the time column and watermark delay are characteristics of the data source rather than the queries themselves. Therefore, users are required to specify these attributes in the CREATE TABLE statement:

CREATE TABLE Orders (
    `user` BIGINT,
    product STRING,
    order_time TIMESTAMP(3),
    WATERMARK FOR order_time AS order_time - INTERVAL '5' SECOND
) WITH ( . . . );

In queries, window functions are solely used to define windowing. For reference, here’s an example of the syntax used in Apache Flink:

SELECT L.num as L_Num, L.id as L_Id, R.num as R_Num, R.id as R_Id,
 COALESCE(L.window_start, R.window_start) as window_start,
 COALESCE(L.window_end, R.window_end) as window_end
FROM (
   SELECT * FROM TABLE(TUMBLE(TABLE LeftTable, DESCRIPTOR(row_time), INTERVAL '5' MINUTES))
) L
FULL JOIN (
   SELECT * FROM TABLE(TUMBLE(TABLE RightTable, DESCRIPTOR(row_time), INTERVAL '5' MINUTES))
) R
ON L.num = R.num AND L.window_start = R.window_start AND L.window_end = R.window_end;

Flint SQL

CREATE MATERIALIZED VIEW test
AS
SELECT ...
FROM (
  SELECT
    time,
    ...
  FROM orders
) AS t1
JOIN (
  SELECT
    time,
    ...
  FROM audit
) AS t2
ON
  t1.time BETWEEN t2.time AND t2.time + INTERVAL 1 DAY
  AND
  t1.id = t2.id
GROUP BY ...
WITH (
  watermark_delay = {
    orders, time, '1 Hour',
    audit, time, '10 Minutes'
  }
);

  test("create materialized view with join") {
    withTempDir { checkpointDir =>
      sql(s"""
           | CREATE MATERIALIZED VIEW $testMvName
           | AS
           | SELECT
           |   t1.startTime AS eventTime,
           |   t1.name,
           |   t2.productId
           | FROM (
           |   SELECT
           |     window.start AS startTime,
           |     name
           |   FROM $testTable
           |   GROUP BY
           |     TUMBLE(time, '10 Minutes'),
           |     name
           | ) AS t1
           | JOIN (
           |   SELECT
           |     window.start AS startTime,
           |     productId
           |   FROM $testTable2
           |   GROUP BY
           |     TUMBLE(transactionDate, '10 Minutes'),
           |     productId
           | ) AS t2
           | ON t1.startTime = t2.startTime
           | WITH (
           |   auto_refresh = true,
           |   checkpoint_location = '${checkpointDir.getAbsolutePath}',
           |   watermark_delay = '1 Second'
           | )
           |""".stripMargin)

      val job = spark.streams.active.find(_.name == testFlintIndex)
      job shouldBe defined
      failAfter(streamingTimeout) {
        job.get.processAllAvailable()
      }

      flint.queryIndex(testFlintIndex).show()
    }
  }

org.apache.spark.sql.AnalysisException: Multiple streaming aggregations are not supported with streaming DataFrames/Datasets; line 1 pos 0;
Project [startTime#160 AS eventTime#162, name#173, productId#178]
+- Join Inner, (startTime#160 = startTime#161)
   :- SubqueryAlias t1
   :  +- Aggregate [window#187-T1000ms, name#173], [window#187-T1000ms.start AS startTime#160, name#173]
   :     +- Project [named_struct(start, precisetimestampconversion(((precisetimestampconversion(time#172-T1000ms, TimestampType, LongType) - (((precisetimestampconversion(time#172-T1000ms, TimestampType, LongType) - 0) + 600000000) % 600000000)) - 0), LongType, TimestampType), end, precisetimestampconversion((((precisetimestampconversion(time#172-T1000ms, TimestampType, LongType) - (((precisetimestampconversion(time#172-T1000ms, TimestampType, LongType) - 0) + 600000000) % 600000000)) - 0) + 600000000), LongType, TimestampType)) AS window#187-T1000ms, time#172-T1000ms, name#173, age#174, address#175]
   :        +- Filter isnotnull(time#172-T1000ms)
   :           +- EventTimeWatermark time#172: timestamp, 1 seconds
   :              +- SubqueryAlias spark_catalog.default.mv_test
   :                 +- StreamingRelation DataSource(org.apache.spark.sql.test.TestSparkSession@3db2665d,CSV,List(),Some(StructType(StructField(time,TimestampType,true),StructField(name,StringType,true),StructField(age,IntegerType,true),StructField(address,StringType,true))),List(),None,Map(header -> false, delimiter -> 	, path -> file:/Users/daichen/IdeaProjects/opensearch-spark/spark-warehouse/mv_test),None), FileSource[file:/Users/daichen/IdeaProjects/opensearch-spark/spark-warehouse/mv_test], [time#172, name#173, age#174, address#175]
   +- SubqueryAlias t2
      +- Aggregate [window#188-T1000ms, productId#178], [window#188-T1000ms.start AS startTime#161, productId#178]
         +- Project [named_struct(start, precisetimestampconversion(((precisetimestampconversion(transactionDate#177-T1000ms, TimestampType, LongType) - (((precisetimestampconversion(transactionDate#177-T1000ms, TimestampType, LongType) - 0) + 600000000) % 600000000)) - 0), LongType, TimestampType), end, precisetimestampconversion((((precisetimestampconversion(transactionDate#177-T1000ms, TimestampType, LongType) - (((precisetimestampconversion(transactionDate#177-T1000ms, TimestampType, LongType) - 0) + 600000000) % 600000000)) - 0) + 600000000), LongType, TimestampType)) AS window#188-T1000ms, transactionId#176, transactionDate#177-T1000ms, productId#178, productsAmount#179, customerId#180, year#181, month#182]
            +- Filter isnotnull(transactionDate#177-T1000ms)
               +- EventTimeWatermark transactionDate#177: timestamp, 1 seconds
                  +- SubqueryAlias spark_catalog.default.mv_test_2
                     +- StreamingRelation DataSource(org.apache.spark.sql.test.TestSparkSession@3db2665d,CSV,List(),Some(StructType(StructField(transactionId,StringType,true),StructField(transactionDate,TimestampType,true),StructField(productId,StringType,true),StructField(productsAmount,IntegerType,true),StructField(customerId,StringType,true),StructField(year,IntegerType,true),StructField(month,IntegerType,true))),List(),None,Map(header -> false, delimiter -> 	, path -> file:/Users/daichen/IdeaProjects/opensearch-spark/spark-warehouse/mv_test_2),None), FileSource[file:/Users/daichen/IdeaProjects/opensearch-spark/spark-warehouse/mv_test_2], [transactionId#176, transactionDate#177, productId#178, productsAmount#179, customerId#180, year#181, month#182]

  protected def createTimeSeriesTransactionTable(testTable: String): Unit = {
    sql(s"""
      | CREATE TABLE $testTable
      | (
      |   transactionId STRING,
      |   transactionDate TIMESTAMP,
      |   productId STRING,
      |   productsAmount INT,
      |   customerId STRING
      | )
      | USING $tableType $tableOptions
      | PARTITIONED BY (
      |    year INT,
      |    month INT
      | )
      |""".stripMargin)

    // Update data insertion
    // -- Inserting records into the testTable for April 2023
    sql(s"""
      | INSERT INTO $testTable PARTITION (year=2023, month=4)
      | VALUES
      | ('txn001', CAST('2023-04-01 10:30:00' AS TIMESTAMP), 'prod1', 2, 'cust1'),
      | ('txn001', CAST('2023-04-01 14:30:00' AS TIMESTAMP), 'prod1', 4, 'cust1'),
      | ('txn002', CAST('2023-04-02 11:45:00' AS TIMESTAMP), 'prod2', 1, 'cust2'),
      | ('txn003', CAST('2023-04-03 12:15:00' AS TIMESTAMP), 'prod3', 3, 'cust1'),
      | ('txn004', CAST('2023-04-04 09:50:00' AS TIMESTAMP), 'prod1', 1, 'cust3')
      |  """.stripMargin)

    // Update data insertion
    // -- Inserting records into the testTable for May 2023
    sql(s"""
      | INSERT INTO $testTable PARTITION (year=2023, month=5)
      | VALUES
      | ('txn005', CAST('2023-05-01 08:30:00' AS TIMESTAMP), 'prod2', 1, 'cust4'),
      | ('txn006', CAST('2023-05-02 07:25:00' AS TIMESTAMP), 'prod4', 5, 'cust2'),
      | ('txn007', CAST('2023-05-03 15:40:00' AS TIMESTAMP), 'prod3', 1, 'cust3'),
      | ('txn007', CAST('2023-05-03 19:30:00' AS TIMESTAMP), 'prod3', 2, 'cust3'),
      | ('txn008', CAST('2023-05-04 14:15:00' AS TIMESTAMP), 'prod1', 4, 'cust1')
      |  """.stripMargin)
  }

  test("create materialized view with join") {
    withTempDir { checkpointDir =>
      sql(s"""
           | CREATE MATERIALIZED VIEW $testMvName
           | AS
           | SELECT
           |   t1.startTime AS eventTime,
           |   t1.transactionId,
           |   t2.productId
           | FROM (
           |   SELECT
           |     transactionId,
           |     transactionDate AS startTime,
           |     productId
           |   FROM $testTable2
           | ) AS t1
           | JOIN (
           |   SELECT
           |     transactionId,
           |     transactionDate AS startTime,
           |     productId
           |   FROM $testTable3
           | ) AS t2
           | ON
           |   t1.transactionId = t2.transactionId
           |   AND
           |   t1.startTime BETWEEN t2.startTime AND t2.startTime + INTERVAL 1 DAY
           | WITH (
           |   auto_refresh = true,
           |   checkpoint_location = '${checkpointDir.getAbsolutePath}',
           |   watermark_delay = '1 Second'
           | )
           |""".stripMargin)

      val job = spark.streams.active.find(_.name == testFlintIndex)
      job shouldBe defined
      failAfter(streamingTimeout) {
        job.get.processAllAvailable()
      }

      flint.queryIndex(testFlintIndex)
        .select("eventTime", "transactionId", "productId")
        .orderBy("eventTime", "transactionId").show()
    }

+-------------------+-------------+---------+
|          eventTime|transactionId|productId|
+-------------------+-------------+---------+
|2023-04-01 10:30:00|       txn001|    prod1|
|2023-04-01 14:30:00|       txn001|    prod1|
|2023-04-01 14:30:00|       txn001|    prod1|
|2023-04-02 11:45:00|       txn002|    prod2|
|2023-04-03 12:15:00|       txn003|    prod3|
|2023-04-04 09:50:00|       txn004|    prod1|
|2023-05-01 08:30:00|       txn005|    prod2|
|2023-05-02 07:25:00|       txn006|    prod4|
|2023-05-03 15:40:00|       txn007|    prod3|
|2023-05-03 19:30:00|       txn007|    prod3|
|2023-05-03 19:30:00|       txn007|    prod3|
|2023-05-04 14:15:00|       txn008|    prod1|
+-------------------+-------------+---------+


CommanderPi won't start

I tried installing from your install instructions here, as well as installing from Pi Apps (this is the current install).  I get an hourglass, then nothing.  Per # 39, I have included the output from the requested command.
Looking below, I see _tkinter.TclError: couldn't connect to display ":10.0".  I am logged in over a remote session, this is an RPi 4 8GB running 64 bit, I do not have a monitor connected.
pi@FoundryVTTPPE:~ $ sudo python3 ${HOME}/CommanderPi/src/main.py ${HOME}
Here is home_path /home/pi
Here is app-1 Version 1.0
Exist and read
Country code is:
country US: DFS-FCC
Here is ad: pieeprom-2022-03-10.bin
Apr 29 2021
/home/pi/CommanderPi/src
File is exist!
path is /home/pi/CommanderPi/src
No protocol specified
No protocol specified
Traceback (most recent call last):
File "/home/pi/CommanderPi/src/main.py", line 26, in
main()
File "/home/pi/CommanderPi/src/main.py", line 22, in main
start = g.Window()
File "/home/pi/CommanderPi/src/gui.py", line 673, in init
master = tk.Tk()
File "/usr/lib/python3.7/tkinter/init.py", line 2023, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: couldn't connect to display ":10.0"

The following command fixes this particular issue.  If you can include it in your scripts, it might save some users some time, but I'm a linux noob, so I don't know what risk that might pose.
sudo cp ~/.Xauthority ~root/

The above line of code works, it does revert for some reason once you reboot. I fixed it by adding that line  to src/start.sh before running src/main.py.


Remove YoTeams term from glossary

Please remove YoTeam term from glossary. It is unused and not found in the rest of the docs.
Searching for YoTeams in docs only finds the one glossary entry: https://learn.microsoft.com/en-us/search/?terms=YoTeams
Thanks!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ff9690f5-fa8b-98bb-d3b4-ad4ed1030114
Version Independent ID: ff9690f5-fa8b-98bb-d3b4-ad4ed1030114
Content: Microsoft Teams developer documentation- Glossary - Teams
Content Source: msteams-platform/get-started/glossary.md
Service: msteams
GitHub Login: @surbhigupta
Microsoft Alias: surbhigupta

Thank you for reporting this, we will check internally and get the doc updated.

The same issue is being tracked here - https://github.com/MicrosoftDocs/msteams-docs/issues/9477. Hence closing this issue, please follow the above issue for further updates. Thank you.


Illuminate/Http/Request.php misses @method validate() annotation

Laravel Version: 6.4.0
PHP Version: #.#.#
Database Driver & Version:

Description:
Validation # Writing The Validation Logic describes using $request->validate(). Editors with PHP integration will (correctly) mark 'validate' as not found in Illuminate\Http\Request.
I think the annotation should be something like.
/** @method public array validate(array $rules) */

or:
/** @method bool|array|null validate(array $rules, ...$params) */

use Illuminate\Http\Request; was an autogenerated import when creating a Controller with php artisan make:model <name>. E.g. it is not that some Facade should be used. Not that Illuminate\Support\Facades\Request annotates validate(). Maybe add the annotation there too.
Or somewhere deeper in the inheritance tree of course.

I don't think that it should be documented as a docblock in the Illuminate\Http\Request class because that method exists on the request only when it's added as a macro in a full framework setup. That means that method doesn't exist when the illuminate/http component is used as a standalone package.

See the answer above.


Remove impossible selectors

I use partial selectors in SASS which sometimes produces impossible selectors if you're not careful. These selectors are valid syntax but can never match an element, so, can safely be removed.
A few examples I thought of:

tag:not(tag)
:matches(h1):matches(h2)
html html (assuming well-formed HTML)
p div (again, assuming well-formed HTML)
...

Do you really have enough impossible selectors in your codebase to notice a bundle size benefit from removing them?

@chrisdothtml yes. :not especially, as I use %partial:not(%other) in a few places. Sure, there are ways of getting around this and I have also suggested removing these during compilation in sass, but I think that having the option of removing these in minification tools is also a good idea, even if not enabled by default.

Closing this due to being an issue which is likely difficult to implement and which provides very little value.


chore: add license header for main.go.tpl

Description:
add license header for main.go.tpl
[optional Relevant Links:]

Any extra documentation required to understand the issue.

Done in #385


Hidden function scripts in V2 editor

If a question has a function script then the script is not displayed on first entering the question in the V2 editor. You have to click on the content area to see the script.
https://numbas-editor.mas.ncl.ac.uk/question/1496/william-s-copy-of-add-a-custom-ruleset/

This is a bug with the codemirror editor, linked to the bug that makes TinyMCE the wrong size sometimes.


Midi arrangement help

Hi there,
Awesome library and thanks so much for you work.
I'm trying to create a structure like the one below but getting stuck and not sure if it's possible?
I'd like to create either empty midi blocks or it's ok if they have one long note in for the length of the block.
And the ability to add space before and after the blocks of midi.
Is this possible?
Thanks

Hi @gilesbutler,
I'm not sure if I understand your question fully, but each midi file created by this library will create one continuous block of MIDI per track.

Closing this for now @gilesbutler, but let me know if I can help any further.

Thanks for the response @grimmdude 👍 After digging a bit further I think it’s actually a limitation of midi itself, I don’t think the midi spec has the notion of clips. I arrived at the same outcome as your comment that you have to have one continuous block of midi per track. I assume it’s the DAWs that program in support for “midi clips”.


Docker Hub builds

Is this a feature request or a bug?
Feature request
Expected behavior:
Commits to master and git tags build and push images to Docker Hub (see also this script).
Actual behavior:
I've been doing it manually from my machine.
Details
Images could be built and pushed in Circle, but that could lead to credentials leaking. I wonder if automated builds would be better? That would probably require a service account or one of the maintainers or moderators setting it up.

Hey, I am curious to understand, how publishing images from Circle CI   could lead to credentials leaking.

Docker Cloud credentials would have to be made available as environment variables to the build. If someone were to get in some malicious code, they could steal those credentials and have access to hub.docker.com/u/entropicdev.

My understanding is that CI tools make secure env variables available for an only master/development branch. In that case, the malicious code should be merged to master, That won't happen very easily because there will be multiple reviews for each PR.


App Before-Quit Event

It would be useful to add another event to the app module that gets triggered before the windows are closed when a user tries to quit. As it is, the "will-quit" event seems to be the only useful quit-related event, but by the time it gets called, all the data related to the windows is gone. This makes it difficult to preserve the state of the application when quitting.
You could save data in the "close" or "closed" event for the BrowserWindow, but you might only want the data to persist when the whole application quits and not just when an individual window is closed.
Alternatively, you could override the app.quit function and do what you need there before calling the original app.quit function, but "app.quit" doesn't seem to get called when a user quits the application from the dock menu. It also seems a bit hacky to have to override a function from a module like that. It would be easier if there was a "before-quit" event that could save all the window states before closing them and terminating the application.

+1 on @vsaar's comment.
This kind of behavior is very common for apps that run from the tray, (e.g., nearly all persistent apps). There's not a good way to accomplish it right now.

Testing this branch with some example code, run-from-tray behavior is pretty trivial to implement:
hostWindow.on('close', (e) => {
  if (hostWindow.forceClose) return;
  e.preventDefault();
  hostWindow.hide();
});

app.on('before-quit', () => {
  hostWindow.forceClose = true;
});

app.on('activate-with-no-open-windows', () => {
  hostWindow.show();
});

Thoughts @zcbenz?

Fixed by #1176.

@CharlieHess, thanks a lot for implementing this. Much appreciated!

@CharlieHess It appears that e.preventDefault() is no longer preventing the closing of the browser window and as such the "run-from-tray behaviour" isn't working.

@CharlieHess I followed your code but app doesn't get closed when click 'quit' from docker from production build generated by electron builder. any thought?


Why splitting the library into two?

Hi! I like the idea of having a simple pure Python library for TOML! And I understand the motivation of keeping it minimal, however, I am not sure if splitting the package into two is the optimal approach. I would like to share my thoughts on that.

If I am a user that does NOT need  dump(s), what is the benefit of NOT having dump(s)? By looking at tomli-w, it seems that the benefit is the absence of ~150 lines of Python code. If it is true, I personally do not see use cases where this benefit can be important. I understand the goal of "theoretically minimal library". Are you sure that this goal is practical?
If I am a user that needs  dump(s), I have to install two packages instead of one and import two different packages instead of one. This setup does not feel minimal to me (unfortunately, this is exactly my case).
In fact, tomli is not minimal: loads can be replaced by using StringIO and load :) While this is rather a joke, I would like to say that the "minimalism", when not absolute, becomes a subjective thing.

Taking all this into account, merging the two libraries into one looks good to me. What do you think?

An alternative solution could be three libraries: tomli-r, tomli-w and tomli, where the latter one wraps the -r and -w versions.

Hey there, thanks for the discussion starter!
The answer to most of your questions is something along the lines of "yeah, it's subjective" or "I don't know" :smile: I'm still not sure how the setup will look like in say 1 year from now. The options I've had in mind are:

The current state with two packages: one for reading and one for writing
Moving write capability to Tomli so that there's only one package
Tomli-W stays separate but usable as a plugin of sorts: pip install tomli[dumps] will install Tomli-W and inject dumps and dump to tomli namespace

I think much depends on feedback from others (just like what you did here, thanks for that).

what is the benefit of NOT having dump(s)

For a user: the absence of ~150 LOC as you mentioned. For me (the maintainer): the ability to break one package without breaking the other. A theoretical example: if I decide to expand Tomli-W with style editing capabilities and the API grows, I may want to make breaking changes eventually, and don't want that to have an effect on Tomli the read-only package. Especially right now that I'm unsure, the current state gives me the most freedom as a maintainer.

I personally do not see use cases where this benefit can be important

This could be valuable for projects like pip that will be installed in even the most peculiar and resource constrained places. Also size-optimized Docker images, etc... I may also be wrong here.

Are you sure that this goal is practical?

No. :smile:

If I am a user that needs dump(s) ... This setup does not feel minimal to me

Yeah I can see that. Still, pip install tomli tomli-w instead of pip install tomli shouldn't be the end of the world, or?

Taking all this into account, merging the two libraries into one looks good to me. What do you think?

Yeah I'm not totally against it, but as you can probably tell by now, I don't know what the best way is, so as for now, easiest is to not change anything.

An alternative solution could be three libraries: tomli-r, tomli-w and tomli, where the latter one wraps the -r and -w versions.

I'd personally favor the tomli[dumps] install extra approach instead of this, if I go down that road.

One more thing I'd like to say is that my understanding is that write use cases are far fewer than read-only use cases, so you, having to install both packages, may be in the minority. I could be wrong. Also I feel like most write use cases benefit from style preservation, so Tomli-W may not be as valuable as the style-preserving tomlkit. These are further reasons why write capability started off as its own package.

@hukkin thanks for the detailed answer!

shouldn't be the end of the world
...
so you, having to install both packages, may be in the minority

As for pip install, it is not a problem at all; the main inconvenience comes from two imports and two modules tomli and tomli_w instead of one.
The solution tomli[dumps] looks good!

Just to add another possible reason for keeping them separate: some of us hope to one day have a TOML parser in the Python standard library. Given all the possible choices involved in writing TOML, it's conceivable that a parser-only implementation has a better chance of being adopted than one that has both reading and writing.
Of course, even if that's true, it's a fairly minor point, because the parsing parts could always be split off again to join the standard library. But if there's a stable, widely-used parser-only library ready, it would be easier to adopt. And having separate install statistics for the parser and writer might help inform the decision.

The main tomli package should have read and write by default, anything else would be bizarre. The fact that they are not together would lead me to look for a different package that has both. You can still keep good separation inside the package or maintaining.

The main tomli package should have read and write by default, anything else would be bizarre. The fact that they are not together would lead me to look for a different package that has both.

Not all things bizarre are necessarily bad. :wink:   If you need a package that has both read and write, I can recommend TOML Kit.

You can still keep good separation inside the package for maintaining.

You've misunderstood my concern. I was never worried about code separation. In fact, having both in the same package actually removes some maintenance burden. But it puts read and write functionality under shared package and version number, that was my concern.

Now that Tomli will be integrated to the standard library (PEP 680), I think I'll keep Tomli and Tomli-W separate at least as long as Tomli serves as a stdlib backport (approximately October 2026, i.e. Python 3.10 EOL date). An exception being if Tomli-W is added to stdlib too, but I'd be very surprised if that happens.


Add Python Client

Fixes #305
This adds an rgdc package, which contains an Rgdc class. This class is the client class, and contains all the relevant methods for interacting with RGD.
This is just some initial methods, based on #178. Hopefully what I have addresses the use cases correctly, although I'll admit I'm a bit distanced from them.
This PR does not include tests (although I would really like to add them), as that would involve spinning up a test RGD instance in CI. This presents issues regarding both feasibility (e.g. system packages), as well as performance. For now, I've decided to punt on it.

@AlmightyYakob this is great. I see an example in the readme. It would be desired to have an example that first get a list of ids based on some filter (cloud mask, aoi) and then download these tiles. Is it possible to update your example to have these pieces?

@AlmightyYakob this is great. I see an example in the readme. It would be desired to have an example that first get a list of ids based on some filter (cloud mask, aoi) and then download these tiles. Is it possible to update your example to have these pieces?

What endpoint would I use to get the list of IDs? The search endpoint only returns SpatialEntry IDs.

SpatialEntry IDs are fine.

This is waiting on me.
I need to add a few new endpoints to better serialize raster entries, search raster entries specifically, and a few other things
Currently there is no good route to go from the IDs returned by the search endpoint to downloading image files

In the future we want a) given AOI, b) some facet parameter, and c) data type information (raster or vector), the system generates  a list of raster entry ids (in case data type from input is raster type) or vector entry ids, then she can filter based on some criterions by looping over these ids, and finally ask a subset of ids to be downloaded to her desktop with all other metadata files.

@AlmightyYakob, I'm wrapped up with this. Would you do a final look over at my changes and let me know when you think this is good to merge?


Implemented improved setitem semantics

Codecov Report

Merging #264 into master will increase coverage by 0.36%.
The diff coverage is 99.04%.

@@            Coverage Diff             @@
##           master     #264      +/-   ##
==========================================
+ Coverage   88.81%   89.17%   +0.36%
==========================================
  Files          34       34
  Lines        5336     5523     +187
==========================================
+ Hits         4739     4925     +186
- Misses        597      598       +1

Impacted Files
Coverage Δ

panel/tests/test_layout.py
99.47% <100%> (+0.31%)
:arrow_up:

panel/layout.py
95.2% <96.82%> (+0.44%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c92e2b7...f6f1878. Read the comment docs.


set data save path(mysql/redis/memcached..)

save data file to local

Can I know what was the reason for this PR? What benefit it brings?

save database file，e.g:  persisted mysql/redis  data to hard disk


Console will not update from Mod Finder

Mod Finder finds updates, and installs Mewsifer Console Menu, but will not install Mewsifer Console. This is the only mod I've consistently been unable to update. Thought it was Mod Finder, but every other mod updates, while this one does not.

Found what may be the problem. info.json still says it's 1.1.4, not 1.1.5.

Thanks for reporting, fixed.


Added message types to live alert module

Pull request checklist:

[x] CHANGELOG.md was updated, if applicable
[x] Documentation in docs/ or install-docs/ was updated, if applicable
[x] I have tested all changes

It seemed appropriate for this to be announce by default, since live alerts are an announcement. I also went ahead and made the code look cleaner and remove the unnecessary ENABLED_DEFAULT setting (since it's FALSE by default).

Works


Allowing the civic entity short name to be overridden in test

Description

Allowing the civic_entity_short_name to be overridden by an environment variable for testing.
Also changing the default civic entity long/short names to be more generic and not Seattle

This will let the browser tests pass when the deployed environment is not using the default civic_entity_short_name value in the application.conf file. Separate PRs are prepared for the aws and seattle staging repositories.
Release notes:
Allow setting a custom value for the civic_entity_short_name setting for browser tests.
Checklist
General

[x] Added the correct label: < feature | bug | dependencies | infrastructure | ignore-for-release >
[x] Created tests which fail without the change (if possible)

Codecov Report

Merging #3746 (e254462) into main (dd3b45b) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main    #3746   +/-   ##
=========================================
  Coverage     80.54%   80.54%
  Complexity     2431     2431
=========================================
  Files           295      295
  Lines          9626     9626
  Branches        658      658
=========================================
  Hits           7753     7753
  Misses         1620     1620
  Partials        253      253

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Unable to Request Access to Organization after initial Launch

At initial launch there is the initial login flow. This included which organizations to get alerts on.

The request button didn't seem to do anything for me.
After this, I cannot find an opportunity to try requesting this access again.

In further investigation, the "request" did initiate Github to send an email to my account asking to confirm. This seems appropriate, and so this issue would then be that I had received no UI indication that the request was sent.
Github then had me go to https://github.com/settings/connections/applications to request the organization. This could be a useful piece of information to share (perhaps in the settings menu while open requests are still pending).

This is already expected behaviour for dealing with Orgs with 2FA so adding this to the app would create some unecessary clutter. Closing for now.


Revert "Fix LineEdit and TextEdit carets disappearing at theme scales…

@#… below 1.0"

@#… below 1.0"


Ranger authorization for Hadoop SDK

Hello.
We use JuiceFS as a extension of HDFS. And we use Ranger to manage permissions of HDFS path. We hope to use JuiceFS exactly the same with HDFS through JuiceFS Hadoop SDK. Thus supporting ranger authorization is highly important.
Please add ranger support for the Hadoop SDK.
Thank you!

@joey71765 Ranger support is on our roadmap after 1.0 release.

That's great, thanks. when would it be release

Currently, we can not tell the exact date

Hello. We have a strong demand for this feature as well. If JuiceFS could support Ranger, it would be tremendously beneficial for the entire community. We hope you will consider providing support for this as soon as possible. Thank you!


add php solve for 111-999-two-digits-no-zero

add php solve for math/111-999-two-digits-no-zero

Please read the contributors md and add your name to your file name.

Hello
Sorry, I didn't see it
I will close this and open a new request


Connect XPipe to ssh-agent?

I have ssh-agent running in background on macOS. Is it possible to make XPipe connect to that ssh-agent so I don't need to input my key passphrase. Thank you.

I think on macos the passphrases are stored in the macos keychain by the ssh agent. So you should be able to make that work by supplying the option UseKeychain yes through a custom SSH connection.
I'm not sure whether I want to supply this option by default as some people might not like this behavior of automatically filling passphrases without being asked.

This is now released, let me know whether it works for you.

Hello,
I just installed XPipe 10.0.2 and ssh-agent. Using powershell I can connect to an internet host I can only connect with ssh key. With XPipe it don't work
io.xpipe.core.process.ProcessOutputException: Shell opener command was unsuccessful:
Operation was cancelled
Permission denied, please try again.
Operation was cancelled
Permission denied, please try again.
Operation was cancelled
<EMAIL_ADDRESS>Permission denied (publickey,password).  <<  !!!
at io.xpipe.core.process.ProcessOutputException.withPrefix(ProcessOutputException.java:29)
at io.xpipe.ext.proc.util.f.a(SourceFile:321)
at io.xpipe.ext.proc.l.n(SourceFile:2578)
at io.xpipe.ext.proc.ssh.r.n(SourceFile:88)
at io.xpipe.ext.proc.l.C(SourceFile:489)
at io.xpipe.ext.proc.l.start(SourceFile:225)
at io.xpipe.app.util.ScanAlert$Dialog.lambda$content$11(ScanAlert.java:188)
at io.xpipe.app.util.BooleanScope.executeExclusive(BooleanScope.java:18)
at io.xpipe.app.util.ScanAlert$Dialog.lambda$content$12(ScanAlert.java:180)
at io.xpipe.app.util.ThreadHelper.lambda$runFailableAsync$1(ThreadHelper.java:42)
at io.xpipe.app.util.ThreadHelper.lambda$wrap$0(ThreadHelper.java:19)
at java.lang.VirtualThread.run(VirtualThread.java:309)
What's going wrong?


Oscar delete edit limit

Pull Request Test Coverage Report for Build 546

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 77.254%

Totals

Change from base Build 542:
0.0%

Covered Lines:
2391

Relevant Lines:
3095

💛  - Coveralls


LDAP Authentication without groups

Is it possible to use ldap authentication without assigning groups?
Our ldap does not contain groups for certain users and we want to assign all users the same role via authentication. Can we use a kind of wildcard?

Caddy shows no matched groups error. I am not able to sign in.

Notes: if a user authenticated successfully, but the group match failed, the following log messages (specifically LDAP authentication failed: no matched groups) gets generates:
2022/06/11 18:20:17.408 DEBUG   security        LDAP search succeeded   {"server": "ldap://ldap.forumsys.com", "entry_count": 1, "search_base_dn": "DC=EXAMPLE,DC=COM", "search_user_filter": "(&(|(uid=riemann)(mail=riemann))(objectClass=inetOrgPerson))", "users": [{"DN":"uid=riemann,dc=example,dc=com","Attributes":[{"Name":"cn","Values":["Bernhard Riemann"],"ByteValues":["QmVybmhhcmQgUmllbWFubg=="]},{"Name":"uid","Values":["riemann"],"ByteValues":["cmllbWFubg=="]},{"Name":"mail","Values":["riemann@ldap.forumsys.com"],"ByteValues":["cmllbWFubkBsZGFwLmZvcnVtc3lzLmNvbQ=="]}]}]}
2022/06/11 18:20:17.418 WARN    security        Bad Request     {"session_id": "1UOKHLTUlBiAD1DCziSd015yDi2tTO5yzzFlE", "request_id": "fd841467-c8be-4e09-b1eb-d534c2745a76", "error": "LDAP authentication failed: no matched groups", "source_address": "<IP_ADDRESS>"}

I will be adding a new directive - fallback role <role_name> [<role_name>]
        ldap identity store example.com {
            groups {
                "ou=scientists,dc=example,dc=com" authp/user
            }
            fallback role authp/user
        }

This way, if no matched groups appears, the user will be assigned one or more roles specified via fallback role directive.

Documentation update issue: https://github.com/authp/authp.github.io/issues/33

@boesr , please test with the latest release and let me know how it goes,

@greenpau Thanks for the quick adaption. Just tested it. I built caddy with the latest release (fallback directive gets accepted), but I still get the error
{"level":"warn","ts":1655099192.076458,"logger":"security","msg":"Bad Request","session_id":"XXX","request_id":"XXX","error":"LDAP authentication failed: no matched groups","source_address":"XXX.XX.X.X"}

my LDAP config:
ldap identity store informatics.ldap {
			realm informatics.ldap
			servers {
				ldaps://MY_SERVER:636 ignore_cert_errors
			}
			attributes {
				name givenName
				surname sn
				email mail
			}
			username "BIND_DN"
			password "PASSWORD"
			search_base_dn "BASE_DN"
			search_filter "(&(|(sAMAccountName=%s))(objectclass=user))"
			groups {
				"GROUP_DN_CONTAINING_NO_MATCHING_GROUP" admin
			}
            fallback role admin
}
		```

@boesr , for troubleshooting please change to:
“fallback roles authp/admin authp/user”

Hi, this is still a nice to have. A mention in the doc will be also a nice addition. I stumbled upon this issue by chance.

@rizlas , please feel free to contribute to the documentation.

I will.. What's the current behavior of this? I try to recap the current status here:
fallback roles authp/admin

{"level":"warn","ts":1694411713.020804,"logger":"security","msg":"Bad Request","session_id":"2EcSqFPmIMnmpVrQKjODED2yuSwideqJzs532F","request_id":"8b2063b3-b158-4c04-91e8-17bcb9b30b63","error":"LDAP authentication failed: no matched groups","source_address":"*******"}

fallback roles authp/user

{"level":"warn","ts":1694411847.1145182,"logger":"security","msg":"Bad Request","session_id":"2EcSqFPmIMnmpVrQKjODED2yuSwideqJzs532F","request_id":"16bf0eec-c2be-4af4-a585-db62a1611b39","error":"LDAP authentication failed: no matched groups","source_address":"******"}

fallback roles authp/user authp/admin
Login with

  "roles": [
    "authp/admin"
  ]

fallback roles authp/admin authp/user
Login with

  "roles": [
    "authp/user"
  ]

Previous mentioned fallback role directive doesn't exists (https://github.com/authp/authp.github.io/issues/33).
Also, with role admin /settings keep saying that the service is not available


WinHTTP and proxy autodetection

The article states "The WinHTTP stack was designed for use in services and does not support proxy autodetection, PAC scripts or authentication."
At https://docs.microsoft.com/en-us/windows/desktop/WinHttp/what-s-new-in-winhttp-5-1 states that "To ease the configuration of proxy settings for WinHTTP-based applications, WinHTTP now implements the Web Proxy Auto-Discovery (WPAD) protocol, also referred to as autoproxy"
At https://blogs.msdn.microsoft.com/ieinternals/2013/10/11/understanding-web-proxy-configuration/ is stated that "The WinHTTP stack was designed for use in services and other contexts where WinINET would be inappropriate (e.g. because WinINET may show UI or not scale well for unattended use). WinHTTP offers a limited subset of proxy support-- generally speaking, the calling code is expected to set the proxy configuration settings directly on the session or request objects. The user may attempt to configure the "default" proxy for WinHTTP using the proxycfg.exe (XP) or netsh.exe (Vista+) tools, but these tools only support fixed proxy settings (not autodetection or PAC script URLs) and WinHTTP-based applications may or may not use those settings"
Based on the above, it seems that WinHTTP stack itself does support proxy autodetection and therefore PAC scripts exposed through WPAD, and is the use of "netsh winhttp" command that requires to specify a fixed proxy and that cannot be a PAC script URL used in WPAD.
Could you clarify this point?
Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f58247e6-b6a8-e172-941d-b18ee3753a55
Version Independent ID: b05fe5e8-3490-219a-d0dd-cdbd4c387886
Content: Upgrade Readiness data sharing
Content Source: windows/deployment/upgrade/upgrade-readiness-data-sharing.md
Product: w10
GitHub Login: @greg-lindsay
Microsoft Alias: greglin

@greg-lindsay
Any updates with this clarification question regarding documentation content? We would like to follow-up.
Thanks.

@juanand
The issue has been noted and the document will be revised in the light of your feedback.
Thank you.


performance fail on single query search

Expected Behavior
MMseq2 expected to work faster than JackHMMer 3
Current Behavior
MMseq2 works slower than JackHMMer 3
Steps to Reproduce
-Download from Uniprot the Uniref90 database ( ~ 90 millions sequences) and A0A011 sequence in FASTA format
-Create a target database with: mmseqs2/bin/mmseqs createdb uniref90.fasta uniref90_db
-Create a target database index with: mmseqs2/bin/mmseqs createindex uniref90_db tmp -s 4
-Create a query database with: mmseqs2/bin/mmseqs createdb A0A011 A0A011_db
-Compare the execution time of the two following commands:
mmseqs2/bin/mmseqs search A0A011_db uniref90_db A0A011_align_db tmp -a -s 1 --num-iterations 3
jackhmmer --cpu 28 -E 1e-3 --incE 1e-3 --noali -N 3 -A stk_alignment output uniref90.fasta
Context
I've run many times slight variations of the commands listed above, in order to check the differences between the two programs and MMseqs does not seem to be quicker than JackHMMer.
The best time I obtained for MMseq2 is around 16 min while JackHMMer only requires 14 min.
Maybe I'm doing something wrong ... the HMMer version is HMMER 3.1b2
Your Environment
Include as many relevant details about the environment you experienced the bug in.

Git commit used (The string after "MMseqs Version:")
dbeeb55b6cdf16118c221cf7943a70a1d2972daa
Which MMseqs version was used (Statically-compiled, self-compiled, Homebrew, etc.):
Statically-compiled AVX2
Server specifications (especially CPU support for AVX2/SSE and amount of system memory):
CPU: Intel Xeon E7-8860v4
$ cat /proc/cpuinfo | grep avx2
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt ssbd ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts flush_l1d
Cores: 4x18 (of which 28 have been used for the job)
Memory: 3072 GB/node
Operating system and version:
Ubuntu Xenial (16.04 LTS)

The sequence A0A011 is a single query right?
MMseqs2 does not parallelize over single queries, it only parallelizes over query sets. So what jackhmmer does in 14 minutes on 28 cores, MMseqs2 does on a single core. You can improve the speed by making sure the index is forced into the OS' page cache (e.g. using mmseqs touch index_db or using https://github.com/hoytech/vmtouch, the pages cached with touch can be evicted again, while vmtouch the index forced to remain in RAM).

@gabrielepozzati This is crucial for single queries that the database is already in memory. If the database is not in memory, MMseqs2 needs to fetch 100s of GBs each iterations. The tool mmseqs touch loads the database into memory.
My recommended workflow for single queries would be
 mmseqs createdb uniref90.fasta uniref90
 mmseqs createindex uniref90 tmp
 mmseqs touch uniref90.idxdb
 mmseqs search A0A011_db uniref90.idxdb  -a -s 1 --num-iterations 3 --db-load-mode 2

The --db-load-mode 2  relies of the fact that the database was already fetched in to ram. It can speed up single queries substantially.

@milot-mirdita @martin-steinegger I'll check it out, thank you very much for your replies!

We also added an explanation to the wiki on how to solve this issue.

The only similar command in mmseq is touchdb ... I guess its a typo!
Anyway I'm a bit confused, I have to provide as an input to touchdb formatted_database
or formatted_database.idx?

Sorry about the confusion. MMseqs2 will resolve the file endings automatically, you only have to provide it the base name of the database. I edited Martins answer. The wiki entry should be correct.

ok, with the proper set-up it is considerably faster! Thanks!


[FIX] Text does not appear [Spiritual Gems]

🥴 oops... thanks @joaochris. I will prepare a fix for it asap. Currently, I am gradually working as well on the E2E tests for the app that should prevent this from happening. Until then, your feedback is really appreciated.


example tower-client "buffer full; poll_ready must be called first"

Bug Report

Version

│   ├── tonic v0.4.0
├── tonic v0.4.0 (*)
├── tonic-health v0.3.0
│   └── tonic v0.4.0 (*)
│   └── tonic-build v0.4.0
└── tonic-build v0.4.0 (*)

Platform

Darwin local.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec  2 20:39:59 PST 2020; root:xnu-7195.60.75~1/RELEASE_X86_64 x86_64
Description

I tried to run cargo run --bin tower-client and I got a panic.
thread 'main' panicked at 'buffer full; poll_ready must be called first', /Users/smhmayboudi/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.3/src/buffer/service.rs:132:14

This is probably the same issue as https://github.com/tower-rs/tower/issues/547. That issue includes a fix.

This is probably the same issue as https://github.com/tower-rs/tower/issues/547. That issue includes a fix.

Thanks. should I worry about "tower-server" as well. Since it is very similar to this case.
        let mut svc = self.inner.clone();

        Box::pin(async move {
            // Do async work here....

            svc.call(req).await
        })

Thanks. should I worry about "tower-server" as well. Since it is very similar to this case.
        let mut svc = self.inner.clone();

        Box::pin(async move {
            // Do async work here....

            svc.call(req).await
        })

Yes. That kind of usage of tower might panic. You're very welcome to submit a PR that fixes the examples 😊

Yes. That kind of usage of tower might panic. You're very welcome to submit a PR that fixes the examples 😊


Incompatibility with NextJS (11.0.0)

When using the latest version of NextJS (with TS), I am unable to use the @solana/wallet-adapter packages.
***/node_modules/@solana/wallet-adapter-react/lib/index.js:1
export * from './ConnectionProvider';
^^^^^^

SyntaxError: Unexpected token 'export'

I'm quite positive it has to do with how the packages are transpiled, and wouldve hoped adding them to the list of transpiled code in my nextjs project (using next-transpile-modules) would work, but unfortunately I was not able to get them working (edit: see update bellow). Any tips? If I get it working I'd be happy to submit a PR for either a readme update or change.
tsconfig:
{
  "compilerOptions": {
    "target": "es6",
    "lib": [
      "dom",
      "dom.iterable",
      "esnext"
    ],
    "allowJs": false,
    "skipLibCheck": true,
    "strict": false,
    "forceConsistentCasingInFileNames": true,
    "noEmit": true,
    "esModuleInterop": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "jsx": "preserve",
  },
  "include": [
    "next-env.d.ts",
    "**/*.ts",
    "**/*.tsx"
  ],
  "exclude": [
    "node_modules"
  ]
}

nextjs config:
module.exports = {
  webpack5: true
}

Update:
I was finally able to get it working taking quite a detour, I've had to change my nextjs config to the following:
/* eslint-disable @typescript-eslint/no-var-requires */
const withTM = require('next-transpile-modules')([
  '@solana/wallet-adapter-base',
  '@solana/wallet-adapter-bitpie',
  '@solana/wallet-adapter-coin98',
  '@solana/wallet-adapter-ledger',
  '@solana/wallet-adapter-mathwallet',
  '@solana/wallet-adapter-phantom',
  '@solana/wallet-adapter-react',
  '@solana/wallet-adapter-solflare',
  '@solana/wallet-adapter-sollet',
  '@solana/wallet-adapter-solong',
  '@solana/wallet-adapter-torus',
  '@solana/wallet-adapter-wallets',
  '@project-serum/sol-wallet-adapter',
])

module.exports = withTM({
  webpack5: true
})

That allowed me to use the dependencies, but then the next error popped up:

window is not defined

which means of the packages has a dependency on window (not possible on the server) so I had to dynamically load the wallet provider like so:
ClientWalletProvider.tsx:
import type { WalletProviderProps } from '@solana/wallet-adapter-react'
import { WalletProvider } from '@solana/wallet-adapter-react'
import {
  getLedgerWallet,
  getMathWallet,
  getPhantomWallet,
  getSolflareWallet,
  getSolletWallet,
  getSolongWallet,
} from '@solana/wallet-adapter-wallets'
import type { ReactNode } from 'react'
import { useMemo } from 'react'

export function ClientWalletProvider(
  props: Omit<WalletProviderProps, 'wallets'> & { children: ReactNode }
): JSX.Element {
  const wallets = useMemo(
    () => [
      getPhantomWallet(),
      getSolflareWallet(),
      getLedgerWallet(),
      getSolongWallet(),
      getMathWallet(),
      getSolletWallet(),
    ],
    []
  )

  return <WalletProvider wallets={wallets} {...props} />
}

and in my _App.tsx:
import type { AppProps } from 'next/app'
import dynamic from 'next/dynamic'

const WalletProvider = dynamic(() => import('../ClientWalletProvider'), {
  ssr: false,
})

export default function App({ Component, pageProps }: AppProps): JSX.Element {
  return (
    <WalletProvider autoConnect>
      <Component {...pageProps} />
    </WalletProvider>
  )
}

This works for now, but it is quite the detour. Any other ideas?

Thanks for the report and documenting the workaround. I'm a big fan of Next.js! I'll test it using it next week and see if we can provide a starter project using it, and try to rework the window checks so that they won't hard fail with SSR.

Could you check out the code in https://github.com/solana-labs/wallet-adapter/pull/32 and tell me if it resolves the issues with Next.js without these workarounds?

Just to keep the discussion centralized, this is a huge step forward but I think there will always be a need for transpiling the dependencies, perhaps the solution to this problem can be in the form of documentation, as bundling third party libraries might have a negative effect on the final bundle size and could be a bit too opinionated.

I created a throwaway branch using esbuild: https://github.com/solana-labs/wallet-adapter/pull/50
I'm not happy with the result though. There are various issues with the webpack starter projects using it, and from what I can tell we'd lose tree shaking for the wallets package, which is kind of a dealbreaker IMO.

perhaps the solution to this problem can be in the form of documentation

Let's go with this approach. I would be happy to accept any PRs to document use with different frameworks and build systems.

@grumpyoldman-io your fix worked great while developing locally, but once I try to deploy the code it has the window is not defined error again. Were you able to work around that? I'll have to look into it a bit more to figure out a work around. Thanks for sharing your workaround. Made it work here locally. 😃

@Hossman333 can you perhaps share your error? I think it boils down to adding some third party dependencies to the next.config.js until it works. There is stuff in the works to make the wallet adapters more compatible with other frameworks like NextJS

@grumpyoldman-io Oh of course! Sorry I didn't share it.

Awesome, I'm happy to hear that is in the works. I'd love to help, but I'm not completely sure where I'd start. Either way, my next.config.js looks like this(like you shared above):
/* eslint-disable @typescript-eslint/no-var-requires */
const withTM = require("next-transpile-modules")([
  "@solana/wallet-adapter-base",
  "@solana/wallet-adapter-bitpie",
  "@solana/wallet-adapter-coin98",
  "@solana/wallet-adapter-ledger",
  "@solana/wallet-adapter-mathwallet",
  "@solana/wallet-adapter-phantom",
  "@solana/wallet-adapter-react",
  "@solana/wallet-adapter-solflare",
  "@solana/wallet-adapter-sollet",
  "@solana/wallet-adapter-solong",
  "@solana/wallet-adapter-torus",
  "@solana/wallet-adapter-wallets",
  "@project-serum/sol-wallet-adapter",
]);

module.exports = withTM({
  webpack5: true,
});

Thank you for responding and for writing all this up in the issue.

@Hossman333 does that Next config fix your issues, or is that build error happening after adding it?

It fixes my issue when I run npm run dev. I used to get the window error and then that config fixed it. I can connect to the wallets and everything here locally, but when I try and run npm run build that's when the window is not defined error comes up again.
I wonder if the optimized build does something different when loading in the external dependencies?

Which by the way this is what they run:
    "dev": "next dev",
    "build": "next build",

@Hossman333 Hmmmm, I cant replicate the error, but if you add @toruslabs/openlogin to the list in your next.config.js it should transpile it. If you get new warnings like it, keep adding the packages until its gone.

Relevant: https://github.com/project-serum/sol-wallet-adapter/issues/38

Hmmmm thanks for your response! Great comment and thoughts @grumpyoldman-io. I think you're on to something about the wallet adapters being referenced outside of the files. Though I have a very similar situation to you other than I have a few different connectors and using the react-ui library.
This is what my ClientWalletProvider looks like. I wonder if one of these providers has some dependencies that are causing issues when building it.
I think because my Navigation.tsx is being loaded by the ClientWalletProvider.tsx and that's being dynamically loaded it should be fine.
ClientWalletProvider.tsx
import type { WalletProviderProps } from "@solana/wallet-adapter-react";
import { WalletModalProvider } from "@solana/wallet-adapter-react-ui";
import { WalletProvider } from "@solana/wallet-adapter-react";
import {
  getPhantomWallet,
  getSolflareWallet,
  getSolletWallet,
} from "@solana/wallet-adapter-wallets";
import type { ReactNode } from "react";
import { useMemo, useCallback } from "react";
import Notification from "./Notification";
import { WalletError } from "@solana/wallet-adapter-base";
import { WalletAdapterNetwork } from "@solana/wallet-adapter-base";
import Navigation from "./Navigation";
import toast, { Toaster } from "react-hot-toast";
import { ConnectionProvider } from "@solana/wallet-adapter-react";
import { clusterApiUrl } from<EMAIL_ADDRESS>
export default function ClientWalletProvider(
  props: Omit<WalletProviderProps, "wallets"> & { children: ReactNode }
): JSX.Element {
  const network = WalletAdapterNetwork.Devnet;
  const endpoint = useMemo(() => clusterApiUrl(network), [network]);

  const onError = useCallback(
    (error: WalletError) =>
      toast.custom(
        <Notification
          message={
            error.message ? `${error.name}: ${error.message}` : error.name
          }
          variant="error"
        />
      ),
    []
  );
  const wallets = useMemo(
    () => [getPhantomWallet(), getSolflareWallet(), getSolletWallet()],
    [network]
  );

  return (
    <ConnectionProvider endpoint={endpoint}>
      <WalletProvider
        wallets={wallets}
        onError={onError}
        autoConnect
        {...props}
      >
        <WalletModalProvider>
          <Navigation />
        </WalletModalProvider>
        <Toaster position="bottom-left" reverseOrder={false} />
      </WalletProvider>
    </ConnectionProvider>
  );
}

Navigation.tsx
import { WalletMultiButton } from "@solana/wallet-adapter-react-ui";
import { useWallet } from "@solana/wallet-adapter-react";
import React, { FC, Fragment } from "react";
import {
  CloudUploadIcon,
  CogIcon,
  LockClosedIcon,
  RefreshIcon,
  ServerIcon,
  ShieldCheckIcon,
  XIcon,
  ChevronDownIcon,
} from "@heroicons/react/outline";

export default function Navigation() {
  const { wallet } = useWallet();

  return (
    <WalletMultiButton className="hover:bg-teal-400 inline-flex items-center px-4 py-2 border border-transparent text-base  rounded-md shadow bg-gradient-to-r from-teal-500 to-cyan-600 text-white font-medium hover:from-teal-600 hover:to-cyan-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-cyan-400 focus:ring-offset-gray-900" />
  );
}

_app.tsx
import type { AppProps } from "next/app";
import dynamic from "next/dynamic";
import React, { FC, useCallback, useMemo } from "react";
import "./notification.css";

// Work around to use the Solana Wallet Adapter with Next.js
// https://github.com/solana-labs/wallet-adapter/issues/30
const WalletProvider = dynamic(() => import("./ClientWalletProvider"), {
  ssr: false,
});

export default function App({ Component, pageProps }: AppProps): JSX.Element {
  return (
    <div>
      <WalletProvider />
      <Component {...pageProps} />
    </div>
  );
}

Anything super obvious just looking at it? Probably not useful to just share my code like this, but thought maybe you would be able to see something..
When I add in @toruslabs/openlogin to the next.config.js it then starts having a different issue. It goes from window not defined to
<EMAIL_ADDRESS>TypeError: Property left of AssignmentExpression expected node to be of a type ["LVal"] but instead got "BooleanLiteral"

> Build error occurred

Anyways, still trying to dig at it. It's a struggle. Thanks for the help you've given me, it's helped me along so far a lot!

Hmmm, when I do your exact setup up above with a few modifications it still doesn't work when running npm run build. Do you perhaps have something in your config other than what you shared? @grumpyoldman-io

Thought it was because of @solana/wallet-adapter-react-ui, so I added it to the next.config.js file and it still didn't work, so this time I tried just completely removing it and trying to stick as close to your code up above with issues on the build again. So strange that it works when running it locally, but not while building it for prod! There's gotta be something here I'm not understanding.

 Hahah. I moved out the ClientWalletProvider.tsx outside of the pages directory and it worked that time. 🤦🏻♂
Thank you for being my rubber ducky and I'm sorry about the spam. This workaround will work great for now. Thank you

@Hossman333 @grumpyoldman-io would anyone be willing to PR an addition to the readme for Next.js config? Would love to help other users coming to this and be able to close this issue.

Sure thing! I'll have a looksy tomorrow.

Please try out https://github.com/solana-labs/wallet-adapter/pull/57 and let me know if this works for you.

Hey @jordansexton! Sorry I didn't help out with the PR more. You got to it quick! Thanks for doing all of that. I tested it locally and only ran into a few snags that I thought someone else could run into and be confused about. shx wasn't a dev dep and I think it's worth including it if you're going to be using that as part of the clean script otherwise it errors out.
And then simply adding the instructions to install the deps. 🤷🏻♂ Just some thoughts. Other than that everything works as expected!! Good job and thank you for being so on top of everything!
My PR with those additions: https://github.com/solana-labs/wallet-adapter/pull/58/files

Nice, good catch with shx and thanks for the addition to the docs. I've merged and will close this, but please reopen if other issues are found.

https://github.com/solana-labs/wallet-adapter/issues/204 was implemented, which should allow wallets to be safely used in an SSR context because their potentially unsafe dependencies aren't loaded until connect is called.

Hello, all.
I got this error in Next.js while using WalletMultiButton.
Error: Hydration failed because the initial UI does not match what was rendered on the server.
See more info here: https://nextjs.org/docs/messages/react-hydration-error
If anyone who knows about this, hope to get advice from him. Thank you

@Smith0309 If you're using Next.js, you'll want to wrap components that may change from the server to the client with dynamic.

Thank  you for your reply.
Yup, so what is the solution?


document the JSON-RPC API as it currently exists

refs #28

I moved everything to README.md and merged


re-implement contiv-netctl modal in master branch Contiv UI

contiv-netctl modal worked in controlled demo environment in kubecon2018. Need it to work in production deployment in master branch.

will defer to future release


Code Crashes Pico W

Thanks to some back and forth info exchange with @mill1000, I was able to narrow it down to an issue with sscanf and %3d for uint16_t api_code.
The issue appears to be completely fixed (#6 ), but might need to be tested on an ESP32 just to make sure it doesn't cause issues there.

Thanks for finding this latest bug. Looks like I was string to sscanf a 32-bit int into a 16-bit variable. Closed with #6