netkvm: receive/transmit performance vastly different

Describe the bug
iperf3 can send ~10gbit/s to the host from the guest on a single connection:
$ iperf3 --time 30 -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.00  sec  34.3 GBytes  9.83 Gbits/sec  775435825             sender
[  5]   0.00-30.05  sec  34.3 GBytes  9.81 Gbits/sec                  receiver

But less than 10% of that performance when receiving from the host:
$ iperf3 --time 30 --reverse -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.06  sec  2.36 GBytes   675 Mbits/sec    0             sender
[  5]   0.00-30.00  sec  2.36 GBytes   674 Mbits/sec                  receiver

Copying a file over the bridge but between 2 Windows VMs gives me ~1.6Gbit/s and doesn't experience the same issue.
To Reproduce
Steps to reproduce the behaviour:
My Windows iperf3 is on WSL1, so there's no Hyper-V layer in between (but I get to run iperf3). See https://github.com/virtio-win/kvm-guest-drivers-windows/issues/1026#issuecomment-1892173927 for iperf3 using cygwin (no WSL)
I have got a few workarounds:

Increase bridge and tap MTU to 9000.

$ iperf3 --time 30 --reverse -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.04  sec  8.13 GBytes  2.32 Gbits/sec    0             sender
[  5]   0.00-30.00  sec  8.12 GBytes  2.33 Gbits/sec                  receiver

Use --parallel for iperf. Or use SMB multichannel. Guest CPU is highly loaded during iperf run depending on parallelism (parallelism 4 = 4 loaded CPUs, parallelism 8 = 8 loaded CPUs). Notice how adding more parallelism has diminishing returns.

$ iperf3 --time 30 --reverse --parallel 4 -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.04  sec  1.66 GBytes   474 Mbits/sec  101             sender
[  5]   0.00-30.00  sec  1.66 GBytes   474 Mbits/sec                  receiver
[  7]   0.00-30.04  sec   907 MBytes   253 Mbits/sec  145             sender
[  7]   0.00-30.00  sec   903 MBytes   252 Mbits/sec                  receiver
[  9]   0.00-30.04  sec  1.37 GBytes   392 Mbits/sec   68             sender
[  9]   0.00-30.00  sec  1.37 GBytes   392 Mbits/sec                  receiver
[ 11]   0.00-30.04  sec  1.40 GBytes   400 Mbits/sec   45             sender
[ 11]   0.00-30.00  sec  1.40 GBytes   399 Mbits/sec                  receiver
[SUM]   0.00-30.04  sec  5.31 GBytes  1.52 Gbits/sec  359             sender
[SUM]   0.00-30.00  sec  5.30 GBytes  1.52 Gbits/sec                  receiver
$ iperf3 --time 30 --reverse --parallel 8 -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.04  sec  1.51 GBytes   431 Mbits/sec  1449             sender
[  5]   0.00-30.00  sec  1.50 GBytes   431 Mbits/sec                  receiver
[  7]   0.00-30.04  sec   913 MBytes   255 Mbits/sec  982             sender
[  7]   0.00-30.00  sec   909 MBytes   254 Mbits/sec                  receiver
[  9]   0.00-30.04  sec   906 MBytes   253 Mbits/sec  1271             sender
[  9]   0.00-30.00  sec   902 MBytes   252 Mbits/sec                  receiver
[ 11]   0.00-30.04  sec   884 MBytes   247 Mbits/sec  1174             sender
[ 11]   0.00-30.00  sec   880 MBytes   246 Mbits/sec                  receiver
[ 13]   0.00-30.04  sec   974 MBytes   272 Mbits/sec  1462             sender
[ 13]   0.00-30.00  sec   971 MBytes   272 Mbits/sec                  receiver
[ 15]   0.00-30.04  sec   890 MBytes   249 Mbits/sec  1324             sender
[ 15]   0.00-30.00  sec   887 MBytes   248 Mbits/sec                  receiver
[ 17]   0.00-30.04  sec   914 MBytes   255 Mbits/sec  1316             sender
[ 17]   0.00-30.00  sec   910 MBytes   255 Mbits/sec                  receiver
[ 19]   0.00-30.04  sec  1.68 GBytes   481 Mbits/sec  1281             sender
[ 19]   0.00-30.00  sec  1.68 GBytes   480 Mbits/sec                  receiver
[SUM]   0.00-30.04  sec  8.54 GBytes  2.44 Gbits/sec  10259             sender
[SUM]   0.00-30.00  sec  8.51 GBytes  2.44 Gbits/sec                  receiver

Both:

$ iperf3 --time 30 --reverse --parallel 4 -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.04  sec  5.71 GBytes  1.63 Gbits/sec    0             sender
[  5]   0.00-30.00  sec  5.70 GBytes  1.63 Gbits/sec                  receiver
[  7]   0.00-30.04  sec  5.72 GBytes  1.64 Gbits/sec    0             sender
[  7]   0.00-30.00  sec  5.71 GBytes  1.64 Gbits/sec                  receiver
[  9]   0.00-30.04  sec  6.07 GBytes  1.73 Gbits/sec    0             sender
[  9]   0.00-30.00  sec  6.06 GBytes  1.74 Gbits/sec                  receiver
[ 11]   0.00-30.04  sec  6.07 GBytes  1.74 Gbits/sec    0             sender
[ 11]   0.00-30.00  sec  6.07 GBytes  1.74 Gbits/sec                  receiver
[SUM]   0.00-30.04  sec  23.6 GBytes  6.74 Gbits/sec    0             sender
[SUM]   0.00-30.00  sec  23.5 GBytes  6.74 Gbits/sec                  receiver
$ iperf3 --time 30 --reverse --parallel 8 -c HOST
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.02  sec  4.07 GBytes  1.16 Gbits/sec  357             sender
[  5]   0.00-30.00  sec  4.06 GBytes  1.16 Gbits/sec                  receiver
[  7]   0.00-30.02  sec  3.87 GBytes  1.11 Gbits/sec  464             sender
[  7]   0.00-30.00  sec  3.86 GBytes  1.11 Gbits/sec                  receiver
[  9]   0.00-30.02  sec  3.88 GBytes  1.11 Gbits/sec  568             sender
[  9]   0.00-30.00  sec  3.87 GBytes  1.11 Gbits/sec                  receiver
[ 11]   0.00-30.02  sec  4.33 GBytes  1.24 Gbits/sec  578             sender
[ 11]   0.00-30.00  sec  4.33 GBytes  1.24 Gbits/sec                  receiver
[ 13]   0.00-30.02  sec  4.64 GBytes  1.33 Gbits/sec    0             sender
[ 13]   0.00-30.00  sec  4.64 GBytes  1.33 Gbits/sec                  receiver
[ 15]   0.00-30.02  sec  4.65 GBytes  1.33 Gbits/sec    0             sender
[ 15]   0.00-30.00  sec  4.65 GBytes  1.33 Gbits/sec                  receiver
[ 17]   0.00-30.02  sec  4.20 GBytes  1.20 Gbits/sec  426             sender
[ 17]   0.00-30.00  sec  4.20 GBytes  1.20 Gbits/sec                  receiver
[ 19]   0.00-30.02  sec  4.17 GBytes  1.19 Gbits/sec  588             sender
[ 19]   0.00-30.00  sec  4.16 GBytes  1.19 Gbits/sec                  receiver
[SUM]   0.00-30.02  sec  33.8 GBytes  9.67 Gbits/sec  2981             sender
[SUM]   0.00-30.00  sec  33.8 GBytes  9.67 Gbits/sec                  receiver

Expected behavior
Send and receive performance should be similar. Maybe not 1:1 but <10% the performance shows something else is wrong here.
Screenshots
If applicable, add screenshots to help explain your problem.
Host:

Distro: [e.g. Fedora, Ubuntu, Proxmox] Ubuntu 22.04
Kernel version Linux 6.5 (HWE)
QEMU version qemu 6.2
QEMU command line

/usr/bin/qemu-system-x86_64 ... \
-accel kvm \
-cpu host,migratable=off,topoext=on,svm=on,invtsc=on,x2apic=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vpindex=on,hv-runtime=on,hv-synic=on,hv-stimer=on,hv-stimer-direct=on,hv-reset=on,hv-vendor-id=1234567890ab,hv-frequencies=on,hv-reenlightenment=on,hv-tlbflush=on,hv-ipi=on,kvm=off,host-cache-info=on,l3-cache=off ... \
-netdev tap,fds=58:62:63:64:65:66:69:70,id=hostnet0,vhost=on,vhostfds=71:72:73:74:75:76:77:78 \
-device virtio-net-pci,tx=bh,ioeventfd=on,event_idx=on,csum=on,gso=on,host_tso4=on,host_tso6=on,host_ecn=on,host_ufo=on,mrg_rxbuf=on,guest_csum=on,guest_tso4=on,guest_tso6=on,guest_ecn=on,guest_ufo=on,mq=on,vectors=18,rx_queue_size=1024,tx_queue_size=1024,netdev=hostnet0,id=net0,mac=<snip>,bus=pci.1,addr=0x0.0x7 ...

(8 queues, 16 core machine).

libvirt version 8.0.0
libvirt XML file

    <interface type='bridge'>
      <mac address='SNIP'/>
      <source bridge='SNIP'/>
      <model type='virtio'/>
      <driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='on' queues='8' rx_queue_size='1024' tx_queue_size='1024'>
        <host csum='on' gso='on' tso4='on' tso6='on' ecn='on' ufo='on' mrg_rxbuf='on'/>
        <guest csum='on' tso4='on' tso6='on' ecn='on' ufo='on'/>
      </driver>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x7'/>
    </interface>

VM:

Windows version Windows 10 22H2
Which driver has a problem using netkvm version <IP_ADDRESS>00
Driver version or commit hash that was used to build the driver

Additional context
There is a bridge interface on the host, and a tap interface for the Windows guest.
 Not sure if that's related.

I suggest to ran as a benchmark first iperf in the guest without WSL1.
If there are issues, we will dig in. But if it is WSL tap issues, we can at best give some advices for that to look.
Best regards,
Yan.

Wait, there's Windows binaries for iperf? Haha. I'll try that.
iperf is actually to make it more reproducible. What started this was my copies over Samba being slow from Host to Guest.

Can you please share the crash dump?

 Also RX checksumming looks off.
Can you record session with Wireshark? Just to check the TCP packet sizes on receive. With functional RSC (receive side coalescing, that should boost RX performance) we should see 64K packets (or in any case larger than MTU packets)
@ybendito - ideas?

Confirm that RSC is not working. Wireshark shows 1500/9000 sized packets depending on host mtu configuration.

https://github.com/virtio-win/kvm-guest-drivers-windows/wiki/netkvm-RSC-(receive-segment-coalescing)-feature

on host:
 sudo ethtool -K vnet4 tso on
Actual changes:
tx-tcp-segmentation: off [requested on]
tx-tcp-ecn-segmentation: off [requested on]
tx-tcp-mangleid-segmentation: off [requested on]
tx-tcp6-segmentation: off [requested on]
Could not change any device features

Is that what you meant?

Yeah, I've tried setting that using ethtool but it isn't turning on though, where do you suggest I look at next?

Nothing printed there, either.

Yep. I think something's wrong with the TSO on this kernel. But in your case even with TSO off you're still going above 10gbit/s; I'm barely hitting 1gbit/s.
I'm not sure where to look to figure out why TSO isn't turning on though.

I've been digging the kernel code and the qemu code - I can confirm that tap devices can turn on TSO, just not the ones that are currently in use. @ybendito could you share your domain and network libvirt XML please? It looks like libvirt and qemu both have a role to play here in setting the tap device correctly

@lowjoel My results are from plain command line qemu, no libvirt, just -tap,vhost=on,id=..,script= in the command line. Fedora 28, qemu ~6.1, kernel 5.12

could you paste that here and I'll try that as a minimal reproducer please?

@lowjoel Enjoy )
sudo /home/yurib/src/qemu/build/qemu-system-x86_64 -machine q35,accel=kvm --snapshot --trace events=/home/yurib/qemu-events-en -cpu SandyBridge,+kvm_pv_unhalt,hv_spinlocks=0x1fff,hv_relaxed,hv_vapic,hv_time -m 8192 -smp 4 -uuid 1534fa42-4818-4493-9f67-eee5ba758385 -no-user-config -nodefaults -no-hpet -monitor stdio -device ioh3420,bus=pcie.0,id=root0,chassis=1,addr=0xa.0 -device ioh3420,bus=pcie.0,id=root1,chassis=2,addr=0xb.0 -device ioh3420,bus=pcie.0,id=root2,chassis=3,addr=0xc.0 -device ioh3420,bus=pcie.0,id=root3,chassis=4,addr=0xd.0 -device ioh3420,bus=pcie.0,id=root4,chassis=5,addr=0xe.0 -device ioh3420,bus=pcie.0,id=root5,chassis=6,addr=0xf.0 -global ICH9-LPC.disable_s3=0 -global ICH9-LPC.disable_s4=1 -device ahci,id=ahci -device virtio-serial-pci,bus=root1,id=virtio-serial0,max_ports=4,iommu_platform=on,ats=on -chardev spicevmc,name=vdagent,id=vdagent -device virtserialport,nr=2,bus=virtio-serial0.0,chardev=vdagent,name=com.redhat.spice.0 -chardev socket,id=serialp2,host=<IP_ADDRESS>,port=50000,server=on,wait=no -device virtserialport,nr=1,bus=virtio-serial0.0,chardev=serialp2,name=test.0 -netdev tap,id=hostnet10sb,script=/home/yurib/br0-ifup,ifname=nw10sb,vhost=on -device virtio-net-pci,netdev=hostnet10sb,mac=04:54:13:05:10:38,bus=root0,id=poc2,rss=on -device virtio-balloon-pci,bus=root4,iommu_platform=on,ats=on -drive file=/images/vms/2019-q35-usb.qcow2,if=none,id=drive-ide-3,media=disk,format=qcow2,cache=unsafe -device ide-hd,drive=drive-ide-3,id=ide3,bus=ahci.0,bootindex=0 -drive file=/images/iso/ubuntu-18.04.1-desktop-amd64.iso,if=none,id=drive-cd,media=cdrom,format=raw -device qemu-xhci,p2=8,p3=8 -device usb-tablet -device usb-storage,drive=drive-cd,id=xx3,bootindex=1 -vga std -vnc :1 -chardev spicevmc,name=usbredir,id=usbredirchardev1 -device usb-redir,filter=-1:-1:-1:-1:1,chardev=usbredirchardev1,id=usbredirdev1 -chardev spicevmc,name=usbredir,id=usbredirchardev2 -device usb-redir,filter=-1:-1:-1:-1:1,chardev=usbredirchardev2,id=usbredirdev2 -chardev spicevmc,name=usbredir,id=usbredirchardev3 -device usb-redir,filter=-1:-1:-1:-1:1,chardev=usbredirchardev3,id=usbredirdev3 -boot menu=on

@ybendito and how was the tap created? on my fresh Ubuntu install using both ip tuntap add mode tap pi vnet_hdr and using tunctl both create tap devices that I can't enable TSO on. I'm guessing you aren't seeing that same behaviour on your Fedora machine?

Bingo. It wasn't the host side, it's the guest side.
Initially:
> get-netadapterrsc | format-list

Name                 : Ethernet
InterfaceDescription : Red Hat VirtIO Ethernet Adapter
IPv4Enabled          : True
IPv6Enabled          : True
IPv4Supported        : True
IPv6Supported        : True
IPv4OperationalState : False
IPv6OperationalState : False
IPv4FailureReason    : WFPCompatibility
IPv6FailureReason    : WFPCompatibility

WFP is the Windows Filtering Platform. I guess it's the firewall. Disabled the firewall:
> get-netadapterrsc | format-list

Name                 : Ethernet
InterfaceDescription : Red Hat VirtIO Ethernet Adapter
IPv4Enabled          : True
IPv6Enabled          : True
IPv4Supported        : True
IPv6Supported        : True
IPv4OperationalState : True
IPv6OperationalState : True
IPv4FailureReason    : NoFailure
IPv6FailureReason    : NoFailure

Wireshark now shows ~62k sized packets.
 I'll take it.
 Can I help to update the wiki/docs as a form of expressing my thanks? 😄

I didn't expect that the guest can affect the host in this way

Guest is one who requests to enable/disable these options on the host tap. If the driver has started with RSC enabled,  it can dynamically turn it on/off and if the guest turned the RSC on we can turn it off/on in the tap. But if the guest started the device with RSC disabled - this means that the OS is not ready to receive coalesced packets (packet size > MTU), it this case you can't turn it on in the tap. Fortunately.

Incidentally, for those who are seeing this:
> get-netadapterrsc | format-list

Name                 : Ethernet
InterfaceDescription : Red Hat VirtIO Ethernet Adapter
IPv4Enabled          : True
IPv6Enabled          : True
IPv4Supported        : False
IPv6Supported        : False
IPv4OperationalState : False
IPv6OperationalState : False
IPv4FailureReason    : Capability
IPv6FailureReason    : Capability

It means that your libvirt/qemu command line is not enabling any of the offloads. Try the follow under the  definition (for qemu-kvm):
<driver name="vhost" txmode="iothread" ioeventfd="on" event_idx="on" queues="4" rx_queue_size="1024" tx_queue_size="1024">
    <host csum="on" gso="on" tso4="on" tso6="on" ecn="on" ufo="on" mrg_rxbuf="on"/>
    <guest csum="on" tso4="on" tso6="on" ecn="on" ufo="on"/>
</driver>

As promised @YanVugenfirer @ybendito I've updated the wiki with the knowledge from this thread: https://github.com/lowjoel/kvm-guest-drivers-windows-wiki/commit/03d6a935cfbb45b65317433502d7820a446b5e86
Please feel free to integrate the updated docs into the wiki. And also feel free to close this issue since the problem is not with the netkvm driver. Tank you all once again for helping me!

@lowjoel Thanks for the Wiki update!
Just for the statistics - can you tell us why you are testing performance and how you are using the Virtio drivers?

No problem. I have a workstation/server all-in-one setup at home. I use a Windows guest since I'm mostly familiar with it, but on the server side at $DAYJOB I'm more familiar with the Linux stack. The server's just a file server, and I have shares across the host/guest which is why I ran into this specific problem.
I was testing performance because that specific share had my photos on it and transferring them for editing/publishing was unbearably slow 😅

Thanks!

Hi Guys,
I'm trying to reproduce this situation in our internal Host, making the Host as the Server role and the Guest as the Client role(BTW: Do I make a mistake with the server and client roles? Anyway, I want to share it with you to discuss them more.).
But I don't see the Guest affected by the Host (e.g. TSO is always supported on the Host side. No matter how the Guest turns off the firewall or not.), and I don't find the receive/transmit performance as vastly different in my test logs.

C:\> netsh advfirewall set allprofiles state on

Do an iperf Net-Stream test on the guest side.

C:\>  C:\Users\administrator\Desktop\iperf-2.1.6-win.exe  -c %ip% -t 10

Do an iperf Net-Stream test on the guest side with "--reverse."

C:\>  C:\Users\administrator\Desktop\iperf-2.1.6-win.exe  -c %ip% -t 10 --reverse

Check the NetAdapterRsc status by PowerShell.

C:\> Get-NetAdapterRsc | Format-List
Name                 : Ethernet
InterfaceDescription : Red Hat VirtIO Ethernet Adapter #2
IPv4Enabled          : True
IPv6Enabled          : True
IPv4Supported        : True
IPv6Supported        : True
IPv4OperationalState : True
IPv6OperationalState : True
IPv4FailureReason    : NoFailure
IPv6FailureReason    : NoFailure

Do the above commands again with the Windows firewall opened.

balabala

I wrote the BAT script to show the detailed steps.
@echo off
setlocal enabledelayedexpansion

:: Configuration
set ip=<IP_ADDRESS>
set logdir=iperf_logs
set num_tests=100
set iperf_tool=C:\Users\administrator\Desktop\iperf-2.1.6-win.exe

:: Create log folder
if not exist %logdir% mkdir %logdir%

:: Run tests
for /L %%i in (1,1,%num_tests%) do (
    echo Running test %%i...

    :: Enable firewall and test iperf --reverse
    echo Enabling firewall and testing iperf --reverse >> %logdir%\test_%%i.txt
    netsh advfirewall set allprofiles state on
    %iperf_tool% -c %ip% -t 10 --reverse >> %logdir%\test_%%i.txt
    powershell -c "Get-NetAdapterRsc | Format-List | Out-File -FilePath %logdir%\test_%%i.txt" -Append

    :: Enable firewall and test iperf
    echo Enabling firewall and testing iperf >> %logdir%\test_%%i.txt
    %iperf_tool% -c %ip% -t 10 >> %logdir%\test_%%i.txt
    powershell -c "Get-NetAdapterRsc | Format-List | Out-File -FilePath %logdir%\test_%%i.txt" -Append

    :: Disable firewall and test iperf --reverse
    echo Disabling firewall and testing iperf --reverse >> %logdir%\test_%%i.txt
    netsh advfirewall set allprofiles state off
    %iperf_tool% -c %ip% -t 10 --reverse >> %logdir%\test_%%i.txt
    powershell -c "Get-NetAdapterRsc | Format-List | Out-File -FilePath %logdir%\test_%%i.txt" -Append

    :: Disable firewall and test iperf
    echo Disabling firewall and testing iperf >> %logdir%\test_%%i.txt
    %iperf_tool% -c %ip% -t 10 >> %logdir%\test_%%i.txt
    powershell -c "Get-NetAdapterRsc | Format-List | Out-File -FilePath %logdir%\test_%%i.txt" -Append

    :: Add separator
    echo ============================== >> %logdir%\test_%%i.txt
)

echo Testing completed.  Let's check the results.
WFP.zip
There may be something wrong here. Please correct me if I am wrong.


Error fetching OidcKeepAliveServiceWorker.json

If you keep the demo website (https://black-rock-0dc6b0d03.1.azurestaticapps.net/profile) open for long enough (around 5 minutes), then the console will eventually print the following errors:

Even if these errors don't cause any real issue, can this be improved at all?

Hi @beanovar , thank you for the issue.
It may be improve. This error do not cause any problem. The aim of this fetch is to keep the ServiceWorker alive the more longer as possible. It is a trick that i think that it can be improved. I have tested some techniques but i think some to find may be better.


Adjustments to weather API call rates

Tomorrow.io/Climacell seems to limit my free account to 100 API calls a day, not 500 like they advertise. With no channels to contact them since I don't pay them.
I run out of API calls by morning.
Can there be an adjustment setting to how often the weather API is called?
Are there any other alternatives out there after Tomorrow.io took over?

There isn't an adjustment that you can easily make in the app itself, but if you don't mind messing with the code and rebuilding the app with webpack, the update intervals are defined here:
https://github.com/elewin/pi-weather-station/blob/master/client/src/components/WeatherInfo/index.js#L10
Currently it updates the current weather every 10 minutes, the hourly weather every hour, and the daily weather every 24 hours. In total that should be 156 calls / day.
Keep in mind though that this is if you keep the map location stationary, as every time you move the map to a new area it will make a new query for each one of those as it gets the data for the new coordinates you scrolled to.

I'll see about updating the app to make the current weather data interval adjustable from within the settings page to make it easier for people.

Hi @elewin ,
Love this project! 👍
Just wondering if you ever went any further with this?
I'm happy with the Info, Hourly and Daily updates, but would like to reduce the rain chart updates from 10mins to 5mins.
I found the index.js file in pi-weather-station\client\src\components\WeatherMap and edited L10 from 10 minutes to 5 minute and then rebooted, but it made no difference.
const CURRENT_WEATHER_DATA_UPDATE_INTERVAL = 5 * 60 * 1000; //every 5 minutes
I'm not familiar with webpack.
I did successfully change the conversion, Settings button and displayed units for the wind speed from mph to knots by editing a few index.js files and the bundle.min.js file.  :)
Thanks in advance...

If you run webpack from the commandline, it will read the updates you made in the source files and generate a new bundle.min.js file for you, but it looks like you managed to manually do it. Looks great!


Upgrade to CakePHP 3.6

Made compatible with CakePHP 3.6, resolved deprecation errors.

Codecov Report

Merging #5 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##             master     #5   +/-   ##
=======================================
  Coverage       100%   100%
  Complexity      165    165
=======================================
  Files             9      9
  Lines           505    507    +2
=======================================
+ Hits            505    507    +2

Impacted Files
Coverage Δ
Complexity Δ

src/Matcher/Voter/UrlVoter.php
100% <100%> (ø)
8 <0> (ø)
:arrow_down:

src/Renderer/StringTemplateRenderer.php
100% <100%> (ø)
57 <0> (ø)
:arrow_down:

src/TestSuite/RequestFactoryTrait.php
100% <100%> (ø)
7 <0> (+1)
:arrow_up:

src/Matcher/Voter/FuzzyRouteVoter.php
100% <100%> (ø)
16 <0> (-1)
:arrow_down:

src/View/Helper/MenuHelper.php
100% <100%> (ø)
30 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a50566...bc82e11. Read the comment docs.

Thank you for your contribution, it's really appreciated, however I would like to kep the 1.x branch of this plugin backwards compatible.
3.6 deprecation support is already complete (but not yet committed, sorry for the inconvenience), and will be available in the soon to be released 2.0 version, which will be CakePHP 3.4 or 3.5+ only.

Ah, got it! Alright, looking forward for 2.0 release!


Feature/Add option to randomize document order per user

In https://github.com/chakki-works/doccano/issues/201, @rbagd mentioned that having an option to randomize document order per user is a great way to reduce annotation fatigue. This pull request adds a new field to the project model which enables this optional feature.

@rbagd I'd appreciate if you could try out this branch and let me know if this fits your requirements and/or what improvements we could make to this feature. Thanks in advance!

I just tested this and it looks pretty great! Simple, yet effective.
What I like in this simple soluton is that although it can look somewhat random, it is still pretty easy for us to check whether annotation fatigue has a statistical effect after the tagging exercise.


 It's a POC and my attempt to prompt a discussion - it's in no way ready to merge.
Background
I fully expect that anyone reading a PR against this repo will know this already but lets set the scene...
When profiles are retrieved from Salesforce they only contain basic profile information i.e. description, user permissions, etc. If the same retrieve contains metadata types that have profile permissions (e.g. objects, fields, apps, classes, etc), the profile metadata will also contain permissions for that metadata. However, it'll only contain the permissions for metadata contained in that retrieve and any existing profile metadata file will be completely overwritten.
This effectively means you can't use source commands and track profiles. You kind of can, if you maintain a manifest and retrieve all metadata with permissions alongside the profiles but this isn't ideal for so many reasons.
 In the coming years, object and field permissions will be completely removed from profiles.
So why invest time in something that doesn't have a future?

It's going to be years before object permissions are completely removed from profiles (2026 I believe?)
Designing new orgs with Permission Sets + Groups is very sensible but we don't always have that luxury with existing orgs and so have to work with what's there, and that's always profiles
Even when object permissions are removed from profiles - there will still likely be app defaults, layout assignments, etc.

Proposal
When converting from MD API to Source format, merge retrieved profile permissions into the existing profile file. If an existing file doesn't exist, fallback to the way it works at the moment.
There would need to be configuration to determine how to merge the files, I propose the following. Two merge strategies:

Replace - completely replace the existing node with whatever has been retrieved. To be used for metadata that is always retrieved in profiles i.e. user permissions, description, custom, ips, etc.
Merge - interlace retrieved permissions with those that are already there identified by a mapping key. To be used for metadata that is selectively retrieved depending on other retrieved types i.e. field permissions, object permissions, etc.

This is an example of the configuration used in the POC:
  "profile": {
      "id": "profile",
      ...
      "strategies": {
        "adapter": "default",
        "transformer": "merged",
        "transformerConfig": {
          "rootNode": "Profile",
          "defaultHandling": {
            "strategy": "replace",
            "deleteOnEmpty": false
          },
          "nodeHandling": {
            "classAccesses": {
              "strategy": "merge",
              "mappingKey": "apexClass"
            },
            "fieldPermissions": {
              "strategy": "merge",
              "mappingKey": "field"
            },
            "loginIpRanges": {
              "strategy": "replace",
              "deleteOnEmpty": true
            },
            "objectPermissions": {
              "strategy": "merge",
              "mappingKey": "object"
            }
          }
        }
      }
    }

Note, I've also considered an attribute along the lines of 'deleteOnEmpty' (not implemented yet in the POC) to cover metadata that may genuinely have been removed from the org (opposed to just not retrieved) e.g. loginIpRanges.
This does mean there is additional configuration to be maintained. It's not ideal to maintain a list of the permission types in profiles and how to merge them. However, this is mitigated by having a default configuration of replace i.e. do what it does now. Also, the profile metadata type seldom changes.
I've tucked the functionality away behind an environment variable (SF_ENABLE_EXPERIMENTAL_PROFILE_MERGE) - having a flag to explicitly turn this feature on or off is probably a good idea. Though, since profile retrieving doesn't work at the moment, arguably a flag to turn off is probably the better way around.
Limitations / Potential Issues

Deletions - for example, if a field is removed from an object in the org - the retrieved profile won't contain a permission and the result of the merge will be that that field remains unchanged (not removed)
Sorting - we'll have to do sorting of permissions on the client side (i.e. in the library) but that may not exactly match the sorting done server side (i.e. by the MD API) so depending on method of retrieving - metadata may move around slightly

TODO

Discuss this change on the PR or open a discussion in forcedotcom/cli (have started here as there is code)

If the idea is a 'go'er', I'm willing to invest my time to finish off the PR.
That will involve:

Fixing all the TypeScript compile and es-lint errors (I've littered it with ignores to make the build compile, I rarely write TypeScript... and so far I'm not rushing to start a new TypeScript project anytime soon...)
Add remaining nodeHandling configurations
Implement deleteOnEmpty
Write unit tests
Reviews

Conclusion
Ultimately, (and I mean no offence to anyone) I don't think we can make the handling of profiles much worse than it already is - therefore, there is no downside to giving this a shot.
Before:

After:

@gdman We've been discussing this PR some internally.  We also have another proposal that kinda might be closer to what you want for Profiles, but be more broadly useful:  https://github.com/forcedotcom/cli/discussions/2544
It would need some way (possibly a new command) to align profiles with other metadata if there's been deletes (ex: a deleted field).  I had a command like that in my ill-maintained personal plugins...it would compare profile/permSets to the object/field/class etc and remove stuff that wasn't there in the local project.

We're going to do something similar as part of https://github.com/forcedotcom/cli/discussions/2544


Fantasy console theming

Might be worth clarifying what exactly the theme of the interface should be.  What I came up with was less fantasy console-based and closer to Muse.  I'm open to either one, but no sense in either of us wasting time 😄

@micahlt 100%! do you have any code from the preview showed in #gamelab-dev? I love how it looked & if I can use/merge it into what i'm doing that'd really help

@micahlt 100%! do you have any code from the preview showed in #gamelab-dev? I love how it looked & if I can use/merge it into what i'm doing that'd really help

I unfortunately don't.  That's just a mockup I made in Affinity Designer, not actual code.  If I could spin up the code that fast I'd have already submitted a PR 😂

I just got through rebasing this with main. @leomcelroy could we either merge this to main, or promote this to the new dev branch that everything else will merge into before a release bump?


0.3.0 release does not have new import method

I think the 0.3.0 version on npm did not have build run successfully on it before being published - it's missing the new await-free import method from https://github.com/ongteckwu/typeid-ts/pull/7.
If you update a project's dependency to 0.3.0 of this library and install it, then cat node_modules/typeid-ts/build/cjs/src/lib/basics.js, the old const uuidv7_1 = require("uuidv7"); require method is still there.

👋🏼 hello! Indeed, this seems to not have been published correctly.
@ongteckwu would it be possible to publish a new version with a fresh build ? 🙏🏼
Thank you!

@marco-verbeek sure I will do that

published, leet me know if it works marco

@ongteckwu
It seems like I'm still getting the same error on version 0.3.1.
/node_modules/typeid-ts/build/cjs/src/lib/basics.js:5
const uuidv7_1 = require("uuidv7");
                 ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /home/marco/Repositories/easy-auth/node_modules/uuidv7/dist/index.js from /node_modules/typeid-ts/build/cjs/src/lib/basics.js not supported.
Instead change the require of index.js in /node_modules/typeid-ts/build/cjs/src/lib/basics.js to a dynamic import() which is available in all CommonJS modules.

Could you double check if the require is still there on your end as well ? 🙏🏼
Let me know if there's anything I can help with.
I highly appreciate your time, thank you so much!

ok can you try again. Sorry was busy whole week.

Still seeing the error in v0.3.8  @ongteckwu


double-beep: Watch +971555506687

double-beep requests the watch of the watch_number +971555506687. See the MS search here and the Stack Exchange search here.

Approved by Makyen in Charcoal HQ


Write an introduction to Smart Contracts

blockchain
solo node
(Vat/SwingSet ?)
ERTP/Contract host

contract source code (+Jessie)
terms and inviteMaker arguments
provided top-level variables

The provided top-level variables list is a kludge (note the "todo" comment), waiting for integration with our support for es-modules. Once that happens, most of these will be replaced by import statements in the modules themselves.

Once that happens, most of these will be replaced by import statements in the modules themselves.

ok, good to know, thanks!
I feel this only shifts what needs to be documented: instead of top-level variable names available to contracts, it will be available import names available to contracts
So i guess most of th work documenting the top-level variables will be useful after the switch to modules

waiting for integration with our support for es-modules

Do you have a idea (even if just a rough one) of when this will happen? 1/3/6/12/24/+ months?

So i guess most of th work documenting the top-level variables will be useful after the switch to modules

Yes.

Do you have a idea (even if just a rough one) of when this will happen? 1/3/6/12/24/+ months?

Attn @dtribble @michaelfig We should discuss how the actual dedicated time still needed translates into calendar time.

Closing as outdated


Feature request

Hi,
Is there a way already to make the card smaller? I would love to make it in to a chip size card or to integrate the feature in to a chip card. 😊

I don't think that's possible since you can't use custom cards in the chip card.


Background cover.jpg

Hi Kiko
Thanks for aligning the background images for both mobile devices vs desktop/notebook to make them consistent for release 2.5.0. Similarly thanks for fixing the Error bug.
As mentioned earlier, scrolling is required for the Error page, to access the Panic and Safe buttons. I would suggest to reduce the page size, to minimize scrolling, for UX purposes.
Once again, keep up the good work!

My current notebook resolution is 1366 X 768.
My mobile device is Android with 5 inch screen.
Hope this helps?

something that I don't understand is why you say cover.jpg. This file is not more longer necessary, now automatically is resolved from the setting in the admin panel.
btw, what browser do you use? I tested responsive version using chrome dev tools and looks well :-(

The above is the screenshot via my browser. What you see is the whole screen (with the browser parts cropped off), as well as the vertical scrolling bar on the right.
This clearly shows that the web page is too big for my browser even on my resolution mentioned earlier.
Shouldn't the page auto-adjust its size somehow?
For notebook, I am using Chrome v45 on Windows 7.

Lets forget about the cover.jpg. I understand that you have improved the codes. Sorry for the misunderstanding.
If I view the error page on my Android phone using Chrome 45 as well, the same scrolling is required and the last words I could see are 'What happens now?' before I need to scroll down to see the 2 buttons.

the problem is realted with the viewport height, I want to fix quickly :-)

Thanks Kikobeats! All looks good now. I used your latest release :)
You have been very fast and good. Well done!


Allow more concise syntax when creating property graph

CREATE PROPERTY GRAPH MovieRentals
 VERTEX TABLES (
 Customer, Movie
 )
 EDGE TABLES (
 CustSales SOURCE Customer DESTINATION Movie
 );

Would reuse the already existing primary and foreign key constraint that already exist when originally creating the tables

It shouldn't be too hard to implement. During the validation of the edge tables, if there is no pk-fk relationship explicitly described in the query, we can check the edge table if there are any pk-fk relationships with the mentioned vertex table.


HTTP request for passing file upload

@tleyden  -- Can you please help me how to pass the payload for image attachment ?
I've tried with image url string.. It was working good.. I not sure, how to pass the attachment for the api..
Please give me some example..

getting following error :
Error extracting multipart/related parts: Didn't expect to get this far
here is the header I used in my javascript code,
Content-Type:multipart/related; boundary=-----sBOUNDARY
payload (body)
-----BOUNDARY
Content-Type: application/json
{'engine':'tesseract'}
-----BOUNDARY
-----BOUNDARY
Content-Disposition: attachment;
Content-Type: image/*
filename='attachment.txt'.
data:image/jpeg;base64,/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAUAAA/+4ADkFkb2JlAGTAAAAAAf/bAIQAAgICAgICAgICAgMCAgIDBAMCAgMEBQQEBAQEBQYFBQUFBQUGBgcHCAcHBgkJCgoJCQwMDAwMDAwMDAwMDAwMDAEDAwMFBAUJBgYJDQsJCw0PDg4ODg8PDAwMDAwPDwwMDAwMDA8MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwM/8AAEQgNrBOIAwERAAIRAQMRAf/EAPgAAQEBAAEFAQEAAAAAAAAAAAABAgMFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAf/2Q==
-----BOUNDARY

See notes on https://github.com/tleyden/open-ocr/issues/67

@tleyden Should I pass PNG as base64 format like I pass in here ?
data:image/jpeg;base64,/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAUAAA/+4ADkFkb2JlAGTAAAAAAf/bAIQAAgICAgICAgICAgMCAgIDBAMCAgMEBQQEBAQEBQYFBQUFBQUGBgcHCAcHBgkJCgoJCQwMDAwMDAwMDAwMDAwMDAEDAwMFBAUJBgYJDQsJCw0PDg4ODg8PDAwMDAwPDwwMDAwMDA8MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwM/8AAEQgNrBOIAwERAAIRAQMRAf/EAPgAAQEBAAEFAQEAAAAAAAAAAAABAgMFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAf/2Q==

Please share me the real example of passing an image..

@tleyden -- should I pass the image data as base64 ?

Take a look at https://github.com/tleyden/open-ocr/issues/67#issuecomment-248729846

I'm not sure what should we pass it on the body of the payload..
var request = new XMLHttpRequest();
request.open('POST', 'http://openocr.yourserver.com/ocr-file-upload');
request.setRequestHeader('Content-Type', 'multipart/related; boundary=---BOUNDARY');
request.onreadystatechange = function () {
if (this.readyState === 4) {
console.log('Status:', this.status);
console.log('Headers:', this.getAllResponseHeaders());
console.log('Body:', this.responseText);
}
};
var body = "-----BOUNDARY
Content-Type: application/json

{'engine':'tesseract'}
-----BOUNDARY

-----BOUNDARY
Content-Disposition: attachment;
Content-Type: image/png
filename='attachment.txt'.

PNGDATA.........
-----BOUNDARY";
request.send(body);
what should i pass PNGDATA......... \ ? -- Is this base64 image data ?

@tleyden  -- Can you please give me one example ? -- I couldn't upload from file which is the one I expected to do now... Please help me on this..
My question here is :
#1. body payload construction
#2. How do I construct image in the body section ?
I'm not sure, this will support invoking from javascript..
getting several issue.. here is the new issue I've if i provide the payload for upload from javascript,
Error: socket hang up
at createHangUpError (_http_client.js:252:15)
at Socket.socketOnEnd (_http_client.js:344:23)
at emitNone (events.js:91:20)
at Socket.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9)
Please give me one example for javascript end.. that will much helpful.. Thanks in advance...

Please give me one working example with file upload.. this will be much helpful..

I tried https://github.com/tleyden/open-ocr-client this.. it is working great!.. i need to do the samething in Javascript.. Can you please give me the working samples in javascript ?

Does this read base64 image data ? -- Can you please give me your input on this ?

Do you happened to to the error message :
runtime error: invalid memory address or nil pointer dereference
? ?

I know this won't help you direct. I can give you an example how to do that in c#.
https://github.com/alex-doe/open-ocr-dotnet/blob/master/OpenOcrDotNet/Services/RestService.cs
The hard part of writing my c# lib was figuring out how the multipart request works.
I think you had to figure this out, like me, on your own.
Maybe someone on Stackoverflow can help you!

@alex-doe cool!!  I just added a link on https://github.com/tleyden/open-ocr/blob/master/README.md#client-libraries

@alex-doe @tleyden  -- Can you please help me to solve this issue using javascript..? thanks in advance.!!!

I tried with the following way,
genMultipart(image, beforeContent, boundary) {
var raw = atob(image);
var rawLength = raw.length;
var imageBinary = new Uint8Array(new ArrayBuffer(rawLength))
for(i = 0; i < rawLength; i++) {
  imageBinary[i] = raw.charCodeAt(i);
}
var before = [beforeContent, "\n\n"].join('');
var after = '\n' + boundary;
var size = before.length + imageBinary.byteLength + after.length;
var uint8array = new Uint8Array(size);
var i = 0;

// Append the string.
for (; i<before.length; i++) {
  uint8array[i] = before.charCodeAt(i) & 0xff;
}

// Append the binary data.
for (var j=0; j<imageBinary.byteLength; i++, j++) {
  uint8array[i] = imageBinary[j];
}

// Append the remaining string
for (var j=0; j<after.length; i++, j++) {
  uint8array[i] = after.charCodeAt(j) & 0xff;
}
return uint8array.buffer; // <-- This is an ArrayBuffer object!

}
where before string is :
"-----BOUNDARY \n
"Content-Type": "application/json" \n

{"engine":"tesseract"} \n
-----BOUNDARY \n

-----BOUNDARY \n
"Content-Type": image/PNG \n
"Content-Disposition": "attachment; filename=attachment.txt"."
it still not working..


I have implemented a web server with instant streaming output no need wait.

Describe the issue
you can check my project graphrag
you can check demo video: https://www.bilibili.com/video/BV113v8e6EZn/
GraphRAG customized by KylinMountain

I have added websever to support streaming output immediately.
I have fixed error when using local embedding service like LM Studio
I have fixed index error after prompt tune
I have fixed the strategy not loaded when setting entity extraction using NLTK.
I have added advice question api
Support any desktop application or web application compatible with OpenAI SDK, support advice question

GraphRAG 定制版
你可以参考演示视频：https://www.bilibili.com/video/BV113v8e6EZn/

我添加了Web服务器，以支持真即时流式输出。
我修复了使用本地嵌入服务（如LM Studio）时的错误。
我修复了提示调整后索引错误的问题。
我修复了在使用NLTK设置实体提取时策略未加载的问题。
我添加了建议问题API。
支持任意兼容OpenAI大模型桌面应用或者Web应用UI接入, 可以使用Cherry Studio 支持建议问题

Logs and screenshots

Additional Information

GraphRAG Version:
Operating System:
Python Version:
Related Issues:

The link is accessable

Thanks for sharing @KylinMountain! Moving this to a new Community tools Discussion area: https://github.com/microsoft/graphrag/discussions/798 for long-term findability.


OpenAI Dimension size's aren't passed to Ruby Gem or used when creating vector store collection

Now that text-embedding-3-small (and large) is available, we can utilise smaller dimension vector embeddings. However, it seems that the dimension value was not being passed to the OpenAI gem, as all embed requests returned 1536 vectors, irrespective of the dimension size requested.
I've created a PR that resolves the issues here:
https://github.com/andreibondarev/langchainrb/pull/487

@prdanelli It's fixed now.

Yeap similar to this one: https://github.com/andreibondarev/langchainrb/issues/506


Fuchsia SDK -> Engine Roll failing due to Timezone copy ctor changes

PR: https://github.com/flutter/engine/pull/27990/checks
LUCI run: https://ci.chromium.org/ui/p/flutter/builders/try/Linux Fuchsia/29004/overview
Error:
[4028/7224] CXX obj/third_party/dart/runtime/vm/libdart_vm_precompiled_runtime.os_fuchsia.o
FAILED: obj/third_party/dart/runtime/vm/libdart_vm_precompiled_runtime.os_fuchsia.o
/b/s/w/ir/cache/goma/client/gomacc  ../../buildtools/linux-x64/clang/bin/clang++ -MD -MF obj/third_party/dart/runtime/vm/libdart_vm_precompiled_runtime.os_fuchsia.o.d --target=x86_64-fuchsia --sysroot ../../fuchsia/sdk/linux/arch/x64/sysroot  -DUSE_OPENSSL=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FORTIFY_SOURCE=2 -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DTARGET_ARCH_X64 -DFUCHSIA_SDK -DDART_TARGET_OS_FUCHSIA -DDART_PRECOMPILED_RUNTIME -DEXCLUDE_CFE_AND_KERNEL_PLATFORM -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_ENABLE_TRACING=1 -DU_ENABLE_RESOURCE_TRACING=0 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -I../../third_party/dart/runtime -I../.. -Igen -I../../third_party/dart/runtime/include -Igen/build/fuchsia/fidl -I../../fuchsia/sdk/linux/pkg/fidl_cpp/include -I../../fuchsia/sdk/linux/pkg/async/include -I../../fuchsia/sdk/linux/pkg/fit/include -I../../fuchsia/sdk/linux/pkg/stdcompat/include -I../../fuchsia/sdk/linux/pkg/fit-promise/include -I../../fuchsia/sdk/linux/pkg/async-default/include -I../../fuchsia/sdk/linux/pkg/fidl/include -I../../fuchsia/sdk/linux/pkg/fidl_base/include -I../../fuchsia/sdk/linux/pkg/fidl_cpp_sync/include -I../../fuchsia/sdk/linux/pkg/fidl_cpp_base/include -I../../fuchsia/sdk/linux/pkg/zx/include -I../../fuchsia/sdk/linux/pkg/async-loop/include -I../../fuchsia/sdk/linux/pkg/async-loop-default/include -I../../fuchsia/sdk/linux/pkg/inspect/include -I../../fuchsia/sdk/linux/pkg/fdio/include -I../../fuchsia/sdk/linux/pkg/inspect_service_cpp/include -I../../fuchsia/sdk/linux/pkg/async-cpp/include -I../../fuchsia/sdk/linux/pkg/sys_cpp/include -I../../fuchsia/sdk/linux/pkg/sys_service_cpp/include -I../../fuchsia/sdk/linux/pkg/vfs_cpp/include -I../../fuchsia/sdk/linux/pkg/sys_inspect_cpp/include -I../../fuchsia/sdk/linux/pkg/trace-engine/include -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -fno-strict-aliasing -m64 -march=x86-64 -fcolor-diagnostics -Wall -Wextra -Wendif-labels -Werror -Wno-missing-field-initializers -Wno-unused-parameter -Wno-unused-but-set-parameter -Wno-unused-but-set-variable -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-deprecated-copy -Wno-psabi -fvisibility=hidden -Wstring-conversion -Wnewline-eof -Oz -fno-ident -fdata-sections -ffunction-sections -g2 -Werror -Wall -Wextra -Wno-unused-parameter -Wno-unused-private-field -Wnon-virtual-dtor -Wvla -Woverloaded-virtual -Wno-comments -g3 -ggdb3 -fno-rtti -fno-exceptions -Wimplicit-fallthrough -O3 -fno-sanitize=safe-stack -fvisibility-inlines-hidden -std=c++17 -fno-rtti -fno-exceptions -c ../../third_party/dart/runtime/vm/os_fuchsia.cc -o obj/third_party/dart/runtime/vm/libdart_vm_precompiled_runtime.os_fuchsia.o
In file included from ../../third_party/dart/runtime/vm/os_fuchsia.cc:16:
gen/build/fuchsia/fidl/fuchsia/intl/cpp/fidl.h:23:13: error: definition of implicit copy assignment operator for 'TimeZonesError' is deprecated because it has a user-declared copy constructor [-Werror,-Wdeprecated-copy]
  constexpr TimeZonesError(const TimeZonesError& other) = default;
            ^
gen/build/fuchsia/fidl/fuchsia/intl/cpp/fidl.h:74:11: note: in implicit copy assignment operator for 'fuchsia::intl::TimeZonesError' first required here
  *result = value;
          ^
gen/build/fuchsia/fidl/fuchsia/intl/cpp/fidl.h:114:13: error: definition of implicit copy assignment operator for 'SkippedTimeConversion' is deprecated because it has a user-declared copy constructor [-Werror,-Wdeprecated-copy]
  constexpr SkippedTimeConversion(const SkippedTimeConversion& other) = default;
            ^
gen/build/fuchsia/fidl/fuchsia/intl/cpp/fidl.h:160:11: note: in implicit copy assignment operator for 'fuchsia::intl::SkippedTimeConversion' first required here
  *result = value;
          ^
gen/build/fuchsia/fidl/fuchsia/intl/cpp/fidl.h:176:13: error: definition of implicit copy assignment operator for 'RepeatedTimeConversion' is deprecated because it has a user-declared copy constructor [-Werror,-Wdeprecated-copy]
  constexpr RepeatedTimeConversion(const RepeatedTimeConversion& other) = default;
            ^
gen/build/fuchsia/fidl/fuchsia/intl/cpp/fidl.h:214:11: note: in implicit copy assignment operator for 'fuchsia::intl::RepeatedTimeConversion' first required here
  *result = value;
          ^

This should be resolved by adding -Wno-deprecated-copy. We're working on removing the deprecated copy issue but have hit issues with C++14 compatibility, IIRC.


UI partially unresponsive after waking from suspend

Describe the bug
After waking from suspend, I was unable to interact with the app icons but was able to interact with the menu bar.
Tested on commit 4aa33a2d79ccd79861447ada7c55c021fb9f4764 using the Nix build
To Reproduce
Wake from (a long?) suspend.
Expected behavior
UI should be responsive
Version Information:

Device: reMarkable 1
Version 4aa33a2d79ccd79861447ada7c55c021fb9f4764

Could you attach tarnish and oxide logs?

systemctl -u oxide shows no entries, tarnish log is

tarnish log
Dec 30 08:41:51 reMarkable tarnish[2239]: Application "harmony" is not running.
Dec 30 08:41:51 reMarkable tarnish[2239]: Application "harmony" exit code 0
Dec 30 08:41:51 reMarkable tarnish[2239]: Resuming  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 08:41:51 reMarkable tarnish[2239]: Uncompressing screen...
Dec 30 08:41:51 reMarkable tarnish[2239]: Recalling screen...
Dec 30 08:41:51 reMarkable tarnish[2239]: Screen recalled.
Dec 30 08:41:51 reMarkable tarnish[2239]: Resumed  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 08:41:51 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Waiting for application to exit...
Dec 30 08:41:51 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Wifi state: On+Connected+Online
Dec 30 08:41:51 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Application exited 0
Dec 30 08:41:58 reMarkable tarnish[2239]: q qml: Opening settings
Dec 30 08:41:59 reMarkable tarnish[2239]: B Battery temperature:  28
Dec 30 08:42:01 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Show Battery Percentage:  true
Dec 30 08:42:02 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Show Battery Temperature:  true
Dec 30 08:42:04 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Show Wifi DB:  true
Dec 30 08:42:05 reMarkable tarnish[2239]: S Saving configuration...
Dec 30 08:42:05 reMarkable tarnish[2239]: [codes.eeems.oxide 2277]   columns: 6
Dec 30 08:42:05 reMarkable tarnish[2239]: [codes.eeems.oxide 2277]   fontSize: 23
Dec 30 08:42:05 reMarkable tarnish[2239]:   showWifiDb: yes   showWifiDb: yes
Dec 30 08:42:05 reMarkable tarnish[2239]:     showBatteryPercent: yes
Dec 30 08:42:05 reMarkable tarnish[2239]: [codes.eeems.oxide 2277]   showBatteryTemperature: yes
Dec 30 08:42:05 reMarkable tarnish[2239]:   showDate: yes   showDate: yes
Dec 30 08:42:05 reMarkable tarnish[2239]:     autoStartApplication:
Dec 30 08:42:05 reMarkable tarnish[2239]: Auto Sleep 3
Dec 30 08:42:05 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Done saving configuration.
Dec 30 08:42:05 reMarkable tarnish[2239]: qml: Main display qml: Main display
Dec 30 08:42:05 reMarkable tarnish[2239]: E Enabling automatic sleep
Dec 30 08:42:05 reMarkable tarnish[2239]: S Sleep After:  1  minutes
Dec 30 08:42:05 reMarkable tarnish[2239]: S Sleep After:  1  minutes
Dec 30 08:42:05 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] qrc:/widgets/SettingsPopup.qml:44:17: QML SpinBox: Binding loop detected for property "value"
Dec 30 08:43:36 reMarkable tarnish[2239]: Down Qt::Key(Key_PowerOff)
Dec 30 08:43:36 reMarkable tarnish[2239]: Up Qt::Key(Key_PowerOff)
Dec 30 08:43:36 reMarkable tarnish[2239]: Suspending...
Dec 30 08:43:36 reMarkable tarnish[2239]: Pausing  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 08:43:36 reMarkable tarnish[2239]: Saving screen...
Dec 30 08:43:36 reMarkable tarnish[2239]: Compressing data...
Dec 30 08:43:37 reMarkable tarnish[2239]: Screen saved.
Dec 30 08:43:37 reMarkable tarnish[2239]: Paused  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 08:43:38 reMarkable tarnish[2239]: Suspending...
Dec 30 10:04:57 reMarkable tarnish[2239]: Resuming...
Dec 30 10:04:57 reMarkable tarnish[2239]: Resuming  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 10:04:57 reMarkable tarnish[2239]: Uncompressing screen...
Dec 30 10:04:57 reMarkable tarnish[2239]: Recalling screen...
Dec 30 10:04:57 reMarkable tarnish[2239]: Screen recalled.
Dec 30 10:04:57 reMarkable tarnish[2239]: Resumed  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 10:04:57 reMarkable tarnish[2239]: Suspend timer re-enabled due to resume
Dec 30 10:04:57 reMarkable tarnish[2239]: Down Qt::Key(Key_PowerOff)
Dec 30 10:04:57 reMarkable tarnish[2239]: Up Qt::Key(Key_PowerOff)
Dec 30 10:04:57 reMarkable tarnish[2239]: Suspending...
Dec 30 10:04:57 reMarkable tarnish[2239]: Pausing  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 10:04:57 reMarkable tarnish[2239]: Saving screen...
Dec 30 10:04:57 reMarkable tarnish[2239]: Compressing data...
Dec 30 10:04:57 reMarkable tarnish[2239]: Screen saved.
Dec 30 10:04:57 reMarkable tarnish[2239]: Paused  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 10:04:59 reMarkable tarnish[2239]: Suspending...
Dec 30 10:04:59 reMarkable tarnish[2239]: Disconnected from "/codes/eeems/oxide1/network/XX"
Dec 30 10:05:02 reMarkable tarnish[2239]: Resuming  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 10:05:02 reMarkable tarnish[2239]: Uncompressing screen...
Dec 30 10:05:02 reMarkable tarnish[2239]: Recalling screen...
Dec 30 10:05:02 reMarkable tarnish[2239]: Screen recalled.
Dec 30 10:05:02 reMarkable tarnish[2239]: Resumed  "/codes/eeems/oxide1/apps/d3641f0572435f76bb5cc1468d4fe1db"
Dec 30 10:05:02 reMarkable tarnish[2239]: Suspend timer re-enabled due to resume
Dec 30 10:05:02 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Wifi state: On+Disconnected
Dec 30 10:05:02 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Battery level:  67
Dec 30 10:05:04 reMarkable tarnish[2239]: BSS added  -- --
Dec 30 10:05:04 reMarkable tarnish[2239]: Registered "/codes/eeems/oxide1/bss/XX" codes.eeems.oxide1.BSS
Dec 30 10:05:09 reMarkable tarnish[2239]: Connected to "/codes/eeems/oxide1/network/XX"
Dec 30 10:05:09 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Wifi state: On+Connected+Offline
Dec 30 10:05:12 reMarkable tarnish[2239]: [codes.eeems.oxide 2277] Wifi state: On+Connected+Online
Dec 30 10:05:31 reMarkable tarnish[2239]: B Battery temperature:  29
Dec 30 10:06:54 reMarkable tarnish[2239]: Powering off... Powering off...
Dec 30 10:06:54 reMarkable tarnish[2239]: Powering off...
Dec 30 10:06:54 reMarkable tarnish[2239]: Unlocked  /dev/input/event2
Dec 30 10:06:54 reMarkable tarnish[2239]: Killing dbus service
Dec 30 10:06:54 reMarkable tarnish[2239]: Removing all APIs
Dec 30 10:06:54 reMarkable tarnish[2239]: Apps API false
Dec 30 10:06:54 reMarkable tarnish[2239]: Notification API false
Dec 30 10:06:54 reMarkable tarnish[2239]: Screen API false
Dec 30 10:06:54 reMarkable tarnish[2239]: System API false
Dec 30 10:06:54 reMarkable tarnish[2239]: Wifi API false
Dec 30 10:06:54 reMarkable tarnish[2239]: Unregistering all networks
Dec 30 10:06:54 reMarkable tarnish[2239]: Unregistering all BSSs
Dec 30 10:06:54 reMarkable tarnish[2239]: Killing timer
Dec 30 10:06:54 reMarkable tarnish[2239]: Removing all inhibitors
Dec 30 10:06:54 reMarkable systemd[1]: Stopping oxide system service...
Dec 30 10:06:54 reMarkable tarnish[2239]: Killing timer
Dec 30 10:06:54 reMarkable tarnish[2239]:  Recieved signal 15
Dec 30 10:06:54 reMarkable tarnish[2239]:  Exiting
Dec 30 10:06:54 reMarkable tarnish[2239]: Application "codes.eeems.fret" is not running.
Dec 30 10:06:54 reMarkable tarnish[2239]: Application "codes.eeems.fret" exit code 0
Dec 30 10:06:54 reMarkable tarnish[2239]: QDBusAbstractAdaptor: Cannot relay signal AppsAPI::applicationExited(QDBusObjectPath,int): Unregistered input type in parameter list: QDBusObjectPath
Dec 30 10:06:54 reMarkable tarnish[2239]: Application "codes.eeems.oxide" crashed.
Dec 30 10:06:54 reMarkable tarnish[2239]: Application "codes.eeems.oxide" is not running.
Dec 30 10:06:54 reMarkable tarnish[2239]: Application "codes.eeems.oxide" exit code 15
Dec 30 10:06:54 reMarkable tarnish[2239]: QDBusAbstractAdaptor: Cannot relay signal AppsAPI::applicationExited(QDBusObjectPath,int): Unregistered input type in parameter list: QDBusObjectPath
Dec 30 10:06:54 reMarkable tarnish[2239]: Killing child tasks...
Dec 30 10:06:54 reMarkable tarnish[2239]: cat: can't open '/proc/3603/status': No such file or directory
Dec 30 10:06:54 reMarkable tarnish[2239]: Force killing bad child tasks...
Dec 30 10:06:55 reMarkable tarnish[2239]: cat: can't open '/proc/3615/status': No such file or directory
Dec 30 10:06:55 reMarkable systemd[1]: Stopped oxide system service.
-- Reboot --

As you can see it suspended at 08:43:38 and resumed at 10:04:57, and I had to force reboot.

systemctl -u oxide shows no entries, tarnish log is
tarnish log

Did you mean journalctl? And there is no oxide unit, so you wont find results with that.

As you can see it suspended at 08:43:38 and resumed at 10:04:57, and I had to reboot.

I'm not seeing anything here that would indicate why you aren't able to interact with other portions of the screen. The only reason the applications grid would be disabled is if the state is not loaded, so if you had popup open (settings, calendar, wifi menu, or item info) or for some reason it was back in a loading state. There is no code that puts it into a loading state though.
Perhaps it opened a menu but didn't draw it for some reason? That said, the logs don't indicate that.

Ah yes, I meant journalctl. I'll see if the issue occurs again after more use.

@siraben have you been able to replicate this?

Will update and monitor, but I haven't reliably been able to reproduce this.

@siraben With v2.2 are you able to still replicate?

Was not seen since

I've been using the reMarkable actively for the last few months and it does happen. I'm able to obtain logs now with journalctl -u tarnish -f.
Here is me tapping on the 7 key 3 times on the PIN entry with no visible response. It appears that it's being read as a swipe. Later, I tap it with the pen and it registers.
Dec 22 02:38:21 reMarkable tarnish[201]: DOWN (<Touch 876 (514, 322) pressed>)
Dec 22 02:38:21 reMarkable tarnish[201]: UP (<Touch 876 (514, 322) released>)
Dec 22 02:38:21 reMarkable tarnish[201]: Not swiping
Dec 22 02:38:22 reMarkable tarnish[201]: DOWN (<Touch 877 (506, 314) pressed>)
Dec 22 02:38:22 reMarkable tarnish[201]: UP (<Touch 877 (506, 314) released>)
Dec 22 02:38:22 reMarkable tarnish[201]: Not swiping
Dec 22 02:38:23 reMarkable tarnish[201]: Pen state:  Active
Dec 22 02:38:24 reMarkable tarnish[201]: Pen state:  Inactive
Dec 22 02:38:24 reMarkable tarnish[201]: Pen state:  Active
Dec 22 02:38:25 reMarkable tarnish[201]: Pen state:  Inactive
Dec 22 02:38:28 reMarkable tarnish[201]: DOWN (<Touch 878 (519, 324) pressed>)
Dec 22 02:38:28 reMarkable tarnish[201]: MOVE (<Touch 878 (522, 323) pressed>)
Dec 22 02:38:28 reMarkable tarnish[201]: UP (<Touch 878 (522, 323) released>)
Dec 22 02:38:28 reMarkable tarnish[201]: Not swiping
Dec 22 02:38:29 reMarkable tarnish[201]: DOWN (<Touch 879 (0, 321) pressed>)
Dec 22 02:38:29 reMarkable tarnish[201]: Swipe started 2
Dec 22 02:38:29 reMarkable tarnish[201]: UP (<Touch 879 (0, 321) released>)
Dec 22 02:38:29 reMarkable tarnish[201]: Swipe Cancelled
Dec 22 02:38:29 reMarkable tarnish[201]: Write touch move <Touch 879 (0, 321) released>
Dec 22 02:38:29 reMarkable tarnish[201]: Write touch up <Touch 879 (0, 321) released>
Dec 22 02:38:30 reMarkable tarnish[201]: DOWN (<Touch 880 (517, 317) pressed>)
Dec 22 02:38:30 reMarkable tarnish[201]: UP (<Touch 880 (517, 317) released>)
Dec 22 02:38:30 reMarkable tarnish[201]: Not swiping
Dec 22 02:38:31 reMarkable tarnish[201]: Pen state:  Active
Dec 22 02:38:31 reMarkable tarnish[201]: Pen state:  Inactive

@Eeems let me know if this helps

Swipe input is handled by tarnish, actual UI input is handled by whatever application is running. This would not be related.
Could you try to reproduce on v2.3? I've made some changes to how wifi connectivity checking is done, so it will not block for as long when pinging the gateway. It's possible that this was causing the unresponsiveness where the application hadn't resumed yet.

@siraben have you been able to reproduce this with 2.4?

@siraben have you been able to reproduce this with 2.5?

Have not been using Oxide in the last few releases, so I cannot reproduce in the meantime.

Since you haven't been reproducing this, and nobody else is reporting it. I'm closing this issue for now. If you reproduce it again in the most recent release we can open it again.


Oracle: Dependencies tab for procedures/functions

Describe the solution you'd like
It would be nice to be able to view the dependencies (the objects it depends on, as well as other objects that depend on the procedure/function) of a procedure/function in the same way as for example a package. This helps us a lot for debugging/solving problems.
Additional context
We now have to open each function/procedure seperately and navigate through it to find all objects used in it. If we want to adjust a procedure/function, we also don't know where it is being used and what objects depend on the procedure/function. This would help in a big way to know the impact of potential changes.

Implemented:

verified


Add support for git switch and git restore.

System Details

posh-git version/path: 0.7.3  ~\Documents\WindowsPowerShell\Modules\posh-git\0.7.3
PowerShell version: 5.1.17763.771
git version 2.23.0.windows.1
OS: Microsoft Windows NT 10.0.17763.0

Issue Description
Git added two quality of life features in git switch and git restore.
Posh-Git could autocomplete these features.
Git Switch - fast way to switch branches.
Git Restore - fast way to roll files back to the current remote version.

This has been implemented on the master branch.  It is targeted for release in v1.  Unfortunately, to use it now you'd have to git clone the repo and import the module from there.  Hopefully, we can get a v1 beta4 out in the not too distant future.  Then you would be able to install the beta from the PSGallery and try out the support for git restore/switch.


Pytorch building instructions?

Dear ROCm team, congrats with successful Pytorch integration!
I am eager to try it with my AMD gpu, but could not find any instructions.
In particular, default instruction assumes
# Add LAPACK support for the GPU
conda install -c pytorch magma-cuda80 # or magma-cuda90 if CUDA 9
What should I do to install Pytorch with AMD gpu support?

Facebook will post the instruction on their site how to do install   Currently you have to build from Source.


The leaderboard doesn't work

For all the leaderboards it is the same issue

Porque .startsWith está obsoleto nas versões mais recentes de quick.db. Experimente .todos

db.all or only .all?

of course db.all


Waypoint "Target a Specific Child of an Element Using jQuery" has ambiguous instructions

Link to waypoint: http://freecodecamp.com/challenges/waypoint-target-a-specific-child-of-an-element-using-jquery
There have been quite a few people asking for help regarding this waypoint in the Help channel.
It turns out that their code doesn't work because instead of targeting the nth-child of the target class, they are targeting the nth-child of the well class or the left-well/right-well IDs. The reason for this seems to be the previous waypoint, where the campers target the parent of the left-well ID.
The code sample in the waypoint's description uses the target class, but I believe that this is not clear enough. The test cases also do not mention the target class (but rather the well class). As a camper, I myself struggled with this for quite a while before I figured out that I had to use the target class.
The description of the waypoint could be edited to explicitly state that the user must use the target class. The test cases must also be edited to mention the target class.

Thank you for clarifying this for me. I've been stuck on it trying well left-well right-well, all kinds of wells.


How to configure http-client to import from an mlflow server to Databricks

Hello, I am trying to import models from one managed mlflow server to databricks.  I am getting this error after running in databricks notebook:
%sh http-client --api https://<HOST REDACTED>/ --resource 'experiments/list'
error:
WARNING: You haven't configured the CLI yet! Please configure by entering /databricks/python3/bin/http-client configure Traceback (most recent call last):   File "/databricks/python3/bin/http-client", line 8, in <module>     sys.exit(main())   File "/databricks/python3/lib/python3.8/site-packages/click/core.py", line 1128, in __call__     return self.main(*args, **kwargs)   File "/databricks/python3/lib/python3.8/site-packages/click/core.py", line 1053, in main     rv = self.invoke(ctx)   File "/databricks/python3/lib/python3.8/site-packages/click/core.py", line 1395, in invoke     return ctx.invoke(self.callback, **ctx.params)   File "/databricks/python3/lib/python3.8/site-packages/click/core.py", line 754, in invoke     return __callback(*args, **kwargs)   File "/databricks/python3/lib/python3.8/site-packages/mlflow_export_import/common/http_client.py", line 100, in main     client = DatabricksHttpClient() if api == "databricks" else MlflowHttpClient()   File "/databricks/python3/lib/python3.8/site-packages/mlflow_export_import/common/http_client.py", line 74, in __init__     super().__init__("api/2.0/mlflow", host, token)   File "/databricks/python3/lib/python3.8/site-packages/mlflow_export_import/common/http_client.py", line 21, in __init__     raise MlflowExportImportException("MLflow host or token is not configured correctly") mlflow_export_import.common.MlflowExportImportException: MLflow host or token is not configured correctly
How to I correctly configure the http-client to point at the mlflow server that I want to import from?

Hi Andre, thanks for the reply.
I have installed your package in Databricks itself in a notebook. I am able
to export a model from another tracking server (not Databricks) into a
staging area inside Databricks.
My problem I’m having is when I use your import-mode tool. I’m getting a
not authorized error wrapped in html. Have you seen this before?
On Fri, Mar 18, 2022 at 4:10 PM Andre Mesarovic @.***>
wrote:

Its best to use the MLflow CLI for this.
https://mlflow.org/docs/latest/cli.html#mlflow-experiments-list.
To access a Databricks MLflow tracking server from your laptop, see:
https://github.com/amesar/mlflow-resources/blob/master/MLflow_FAQ.md#how-do-i-access-databricks-mlflow-from-outside-databricks
—
Reply to this email directly, view it on GitHub
https://github.com/amesar/mlflow-export-import/issues/48#issuecomment-1072797551,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AF4XOIUUYVM2TBPXPTO2CCLVATWKXANCNFSM5Q7BC73Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

Check to make sure you have correct Databricks personal access token.

@amesar This is the error I'm getting when trying to import into Databricks:
`mlflow.exceptions.MlflowException: API request to endpoint /api/2.0/preview/mlflow/registered-models/create failed with error code 401 != 200. Response body: '

Error 401 Unauthorized

HTTP ERROR 401
Problem accessing /api/2.0/preview/mlflow/registered-models/create. Reason:
    Unauthorized

`
My setup looks like this:
%sh MLFLOW_TRACKING_URI=tracking uri redacted DATABRICKS_HOST=host redacted DATABRICKS_TOKEN=token redacted  import-model --model ExampleModel --input-dir /dbfs/ExampleModel/test --experiment-name test

Are you trying to execute the script from a Databricks notebook? I have never tried that. Try using the export-import notebooks instead.
https://github.com/amesar/mlflow-export-import/tree/master/databricks_notebooks

Will this work if I am importing from a tracking server (not databricks) into a databricks tracking server?

Use the CLI tools to import from OSS tracking server into DB tracking server.

Thank you, I almost have this working fully. I am able to import everything using the CLI tool except I am getting an error when bringing the .pkl (pickle) file over:
ERROR: run_id: <REDACTED> Exception: The following failures occurred while downloading one or more artifacts from s3://<REDACTED>/artifacts: {'<REDACTED>/python_model.pkl': "OSError(95, 'Operation not supported')"}
I am running this CLI command in the notebook:
export MLFLOW_TRACKING_URI=<REDACTED>/ export-run --run-id "$RUN_ID" --output-dir "$FILE_PATH"

Are your Databricks artifacts stored in DBFS or directly in S3? For S3, you need to have your AWS credentials set up correctly. I've only tested with DBFS artifacts, haven't done any direct S3 tests. You may want to verify your S3 artifact with the download_artifacts API method.

A stack trace would also be helpful.

@amesar
Traceback (most recent call last):   File "/databricks/python3/lib/python3.8/site-packages/mlflow_export_import/run/export_run.py", line 72, in export_run     self.mlflow_client.download_artifacts(run.info.run_id,"", dst_path=mk_local_path(dst_path))   File "/databricks/python3/lib/python3.8/site-packages/mlflow/tracking/client.py", line 1413, in download_artifacts     return self._tracking_client.download_artifacts(run_id, path, dst_path)   File "/databricks/python3/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 391, in download_artifacts     return self._get_artifact_repo(run_id).download_artifacts(path, dst_path)   File "/databricks/python3/lib/python3.8/site-packages/mlflow/store/artifact/artifact_repo.py", line 265, in download_artifacts     raise MlflowException( mlflow.exceptions.MlflowException: The following failures occurred while downloading one or more artifacts from s3://<REDACTED>/artifacts: {'<REDACTED>/python_model.pkl': "OSError(95, 'Operation not supported')"} MLflow Version: 1.24.0 MLflow Tracking URI: https://<REDACTED>/ Options:   run_id: <REDACTED>  output_dir: /dbfs/<REDACTED>/   export_metadata_tags: False   notebook_formats:  Databricks REST client: https://<REDACTED>/api/2.0 ERROR: run_id: <REDACTED> Exception: The following failures occurred while downloading one or more artifacts from s3://<REDACTED>/artifacts: {'<REDACTED>/python_model.pkl': "OSError(95, 'Operation not supported')"}

@amesar Also, the artifacts are in S3. The AWS creds are set up correctly because I'm able to pull every file but the pickle file.

Stack trace doesn't really indicate what went wrong. See if this works. mlflow-export-import just calls the public MLflow API.
import sys
import mlflow
run_id = sys.argv[1]
path = sys.argv[2] if len(sys.argv) >= 2 else ""
client = mlflow.tracking.MlflowClient()
client.download_artifacts(run_id, path)


Use ohai root_user attribute in place of root

Description
Allows consumers to override the default attribute of 'root' while not duplicating ohai attributes. This is important for platforms like cygwin that aren't windows but which may not have a root user or with a different superuser.
Issues Resolved
Check List

[x] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[x] New functionality includes testing.
[x] New functionality has been documented in the README if applicable
[x] The CLA has been signed. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

I merged this one without thinking, but root_user is not a thing in Ohai. root_group is.

There is a helper in Poise-land for this but It Gets Complicated™.

We have an open PR to add it to Ohai, but there's a lot of debate on what the root_user is on Windows.


Issue with UTC offsets

Hello! I'm not sure if this is a misinterpretation on my end of "(and automatically applying) UTC Offsets." or an actual bug so bear with me.
I have the following timestamp 2011-06-17T18:30+04:00, this is a Dubai timezone, when converting to UTC, the proper time is: 2011-06-17T14:30:00.000000000Z, however, the crate returns 2011-06-17T22:30:00.000000000Z.
Minimal reproducible example:
fn main() {
    let ts = iso8601_timestamp::Timestamp::parse("2011-06-17T18:30+04:00");
    println!("{:?}", ts);
}

Fixed in 0.2.13, had some logic reversed. Thanks for bringing this up!


Not working for mobile

Hi! I am experiencing problems with this code when viewed on a mobile device.  It works on a tablet, but mobile clicks are not registering for me.
Help would be greatly appreciated  :)

I have figured it out, your site needs to be https for it to show on phone for some reason. Ipad/iphone.
this is my code and it wasnt working for the longest time on phone, then i used ngrok to generate a custom https link for my website and it the modal worked fine on my phone.
<div data-lity data-lity-target="https://player.twitch.tv/?channel={{streamer.streamer_name}}&parent=52ef843bdcb8.ngrok.io" class='p-0 m-0'>                     <img src="{{ streamer.stream_thumbnail }}" class='browse-streamers-thumbnail img-fluid'>                 </div data-lity data-lity-target="">

This is not the case unfortunately.  I have changed my site to https and am having the same problem :-/


IN MY PROJECT NEEDLE IS NOT MOVING

NEEDLE IS NOT WORKING (OR) NOT MOVING WITH ANYGUAGES

Please, share your code or error logs so we can help you.

ok i will mail the code no error is there in the code

thingspeakproject.zip
the code has no error but the needle is not working
I need the gauge to take data from "https://thingspeak.com/channels/362665/fields/1/last" this link and show in the gauge meter but this is not taking the value from http request please help

please help sir for needle function

I'm not able to open source code, please share your code here.
and make sure you have got the right data from your link, use Log.w("data", YOUR_DATA_AS_STRING);


[FLINK-11530] [docs] Support multiple languages for the docs framework

What is the purpose of the change
This PR integrate the jekyll-mulitple-language plugin for flink docs to support Chinese.
And all the xx.zh.mds have been created but keep the original English contents.
Brief change log

Add jekyll-mulitple-language plugin to the Gemfile and _config.yml
Some modifications for sidebar.html, base.html, plain.html, redirect.html to make them support Chinese in case of the current page is a Chinese version.
Copy all the existing markdown files to xx.zh.md (keep the original content and translate later)

Verifying this change
This change is a trivial document rework without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

@flinkbot approve description
@flinkbot approve consensus
@flinkbot approve attention

Thanks for the PR @wuchong.
Can you separate the changes for the multilang support and the copied files into two commits? Makes it easier to review the relevant parts.
Thank you.

I played a bit with the documentation and it seems to work well. Didn't understand (and check) all the variables that are set yet ;-)
Are you planning to add a language switch similar to the project webpage?

Hi @fhueske , thanks for the reviewing and advice.
I have separated the changes into two commits. The first is multi-language support, the second is coping all the files.
The variables are described in the http://jekyll-langs.liaohuqiu.net/#variables
I introduced a new variable baseurl_i18n in sidenav.html.  It is the base url in current language, e.g. {{ site.baseurl }}/zh for Chinese, and {{ site.baseurl }} for English (as default).
Regarding to the language switch:
Yes, I will add a switch similar to the project website, but in a separate JIRA (just created FLINK-11732). This will be added when we satisfied with the translation coverage.
Thanks,
Jark

Btw, the commits to the docs are frequent. I have updated the copied files to the latest commit to docs 70107c4647ecac3df9b1b8c7920e7cb99ad550f1.
https://github.com/apache/flink/commits/master/docs

Hi @wuchong, thanks for splitting commit and the pointers to the variables.
Adding the switch later makes total sense.
While browsing the website, I noticed that all links on Chinese pages that are defined with {{ site.baseurl }} are pointing to the English pages. Do you want to change them to use {{ baseurl_i18n }} (excluding image URLs) when translating the each page or in a bulk fashion?
Either way is fine with me, so +1 to update the last commit and merge the PR.

@flinkbot approve architecture
@flinkbot approve quality

Hi @fhueske , thanks for the suggestion. I think it's a good idea. But I'm afraid we can not use {{ baseurl_i18n }} in markdown files. It is only be declared in sidenav.html and seems that it can't be accessed by documentation files.

Oh, I see. Then links would need to be converted to {{ baseurl_i18n }}/zn or relative paths.
Anyway, I think this can be done later when the pages are translated.
So +1 to merge the PR.

What's the status of this PR?

Hi @rmetzger , this PR needs to be rebased on the latest master. But I think before this PR is merged, we should have a consensus on the review process https://github.com/apache/flink-web/pull/190, otherwise, the Chinese documentation synchronization is hard to maintain. Once https://github.com/apache/flink-web/pull/190 is approved, I will update this PR.

I have updated all the .zh.md files according to the latest docs commit  https://github.com/apache/flink/commit/86bee8679112e76372a84083b1af18722644e1a0 .
Will merge this pull request after compile passed.

@kkrugler FYI


Remaining tasks

Below are the remaining tasks to be completed before the final demo :

[x] Get the followering twitter handles and display it along with their personality type

[x] Improve the UI of the dashboard. For instance - personality traits card is taking a lot of blank space and is looking uneven, remove the purchase banner if possible

[x] Update the Readme with setting up code

[ ] Put screenshots in README

[x] Add a pull request template

[x] Clean up the commented out code

If you guys think there are any other things left you can add it over here.
Edit: I think we can also display the full form of the personality type below the abbreviation for the personality. For instance, INTP will be Introvert-Intuitive-Thinker-Perceiving.

Screenshots for README:


set_latency_timer: allow 0ms duration

The datasheet for the FT2232H says:
"Latency Timer. This is really a feature of the driver and is used
to as a timeout to flush short packets of data back to the PC. The
default is 16ms, but it can be altered between 0ms and 255ms. At 0ms
latency you get a packet transfer on every high speed microframe."

I have observed that using 0 results in significantly improved throughput with small packets on the FT2232H.

Created a new release with this fix, 0.32.3

The quoted text from the datasheet for the FT2232H directly conflicts with the following wording in the official API docs:
In the FT8U232AM and FT8U245AM devices, the receive buffer timeout that is used to flush remaining
data from the receive buffer was fixed at 16 ms. In all other FTDI devices, this timeout is programmable
and can be set at 1 ms intervals between 2ms and 255 ms. This allows the device to be better optimized
for protocols requiring faster response times from short data packets.

This forces us to have to decide which official documentation is correct. I suggest the driver docs should overrule the FT2232H datasheet, as at best the datasheet applies to a single device, but the driver applies to every device. Assuming that the API docs are correct, the removal of the above assertion actually creates an unsoundness issue as values below 2 are explicitly forbidden.
Possible solutions may be:

Reintroduce range checks (would suggest full assertions rather than debug assertions, as perf is probably not an issue)
Mark this function as unsafe
Add an unsafe_set_latency_timer function to allow <2ms timeouts.


ART-6671 Trigger microshift_sync

Ref. https://issues.redhat.com/browse/ART-6671

A test job in hackspace would be nice to verify this doesn't break on anything

A test job in hackspace would be nice to verify this doesn't break on anything

I agree, but it's a simple change and it's mostly about having it run in production. Possibly, a fix will be easy (same for a revert in case or emergencies)

@locriandev If you have the jira ticket in title, the PR would get auto-linked to that ticket, fyi

Does not seem to be working though


[FEATURE REQ] Spring Cloud Azure AD should work with non webapp

Context
The current Spring Cloud Azure  AD only works with the Spring Web, the non webapp scenario should be considered too.
If the scenario is valid, then these features should not be mixed and provided to users, they should be combined with user scenarios.
Refer issue: https://github.com/spring-projects/spring-boot/issues/202
Goal
Provide a way to make the current Spring Security features work in a non webapp environment, also the webapp scenario users also can leverage the features which belong to a non webapp.

nice catch


When duplicating a token, renaming it, and changing the value, the name persists, the value does not

Describe the bug
From Slack:
When duplicating a token, and editing it’s properties. it forgets the value upon saving. Though it does remember the name.
When I duplicate a token, then rename that token and change the token value, the name is saved, but the value is not.
In order to save the new value, I need to Edit the newly-renamed token as I would any other token.
Context:
Sometimes it's convenient to use the Duplicate Token feature to build a new token name out the existing token's naming convention, then updating the value.
To Reproduce
Steps to reproduce the behavior:

Duplicate a token
Change token name
Change token value
Observe name persists, but new value does not

Expected behavior
Both token value and newly changed token name should persist
Screenshots or Screencasts
https://user-images.githubusercontent.com/9900326/185424491-ffcea62d-bdff-4578-a1d2-6cc477af3458.mov

This should've been fixed in https://github.com/six7/figma-tokens/pull/1179 which will be part of 116 (we'll probably do a bugfix release these coming days)


Bump to GDAL 3.9.0

GDAL 3.9.0 was released. The images should be updated accordingly. Still thinking about versioning the tags of the images here. We could increase major version in case we update/upgrade dependencies, thus this would result in mapme-spatial:2.0.0 in this case. Or we only increase the major component in case we change how the image is built, thus updating GDAL here would result in mapme-spatial:1.2.0. @karpfen, any thoughts?

Hi @goergen95, some thoughts, but absolutely no strong opinion.
I'm tending toward mapme-spatial:1.2.0, since it's "just" a dependency version bump.
Thinking about this, maybe it could be useful if the build version reflects the mapme.biodiversity version that is inside, so e.g.  mapme-spatial:0.6.0.x. But I don't know if that is good practice, a good idea or actually helpful for anyone.

Thanks for your comment! I also agree that a "simple" bump of version is somewhat over-to-top for an increase in the major version number.
The idea of reflecting the version of mapme.biodiversity does not make totally sense to me. The main purpose of these images (to me) is to get specific versions of GDAL/PROJ/GEOS to work with either the CRAN or GitHub version of the R package. I think what your are suggesting goes into the direction to build images for each release of mapme.biodiversity, which I currently do not see a need for. You can always install specific versions e.g. via remotes::install_version("mapme.biodiversity", version = "0.5"). The hard part is to control the version of GDAL that might be needed in case you rely on specific functionality, e.g. for cloud provider drivers, only available in recent versions.


clearcache doesn't seem to work

While testing some changes in text searching I'm comparing results between library.bdrc.io while monitoring the fuseki log to see what queries are being issued.
Using:
curl -X POST http://purl.bdrc.io/clearcache

which replies OK, does not appear to clear sufficient caching since when I reissue the query from library.bdrc.io, I don't see the query reissued to fuseki.
I've cleared caches in my browsers and tried various browsers and so on.

I cannot reproduce the issue (I keep trying). Meanwhile, you can check the following page that gives you a full status of the cache and memory. http://purl.bdrc.io/cache

Still no success in reproducing the issue. Below is what I do:
I make this request: http://localhost:8080/query/table/BLMP?L_NAME=("mkhan+chen"+AND+("'od+zer"+OR+"ye+shes"))&LG_NAME=bo-x-ewts&I_LIM=100
and see it in /usr/local/fuseki/tomcat/logs/catalina.out
I make the same request, then check the log file again: it doesn't appear a second time, so fuseki wasn't called.
Then i do this (clear the cache): curl -X POST http://purl.bdrc.io/clearcache and check here : http://purl.bdrc.io/cache that the number of objects in the cache is zero.
Then I resend my request: http://localhost:8080/query/table/BLMP?L_NAME=("mkhan+chen"+AND+("'od+zer"+OR+"ye+shes"))&LG_NAME=bo-x-ewts&I_LIM=100
that appears a second time in fuseki tomcat logs (fuseki was called).
Unless I missed something, I suspect a browser cache issue or some additional caching in library.bdrc.io (which would be very weird !).


[NFC] Complete the IsIUO Request

Because of the cleanup in #27648, it is now possible to just grab the
right TypeRepr here.

And because this was digging around in the pattern before which is super suspicious
@swift-ci please test source compatibility

Build failed
Swift Test OS X Platform
Git Sha - 4c7ab6e7a6390f5fbe4618c860345aef3dd1da2d

Picked up the libSyntax failures and the compatibility UPASS
@swift-ci please smoke test macOS platform

⛵


Improve UI flexibility

For any given shift, ensure that every restaurant can be viewed
alongside any hospital.
Previously, the hospital list dictated the height of each shift. This
accommodated assignment in cases where there were more restaurants than
hospitals as in the following wireframe:
.----------------------------------------------------. |^|
| (shift title)                                      | | |
| .----------------------.                           | | |
| |                    |^|                           | | |
| | (restaurants)      | |    (hospitals)            | | |
| |                    |v|                           | | |
| '----------------------'                           | | |
'----------------------------------------------------' | |
.----------------------------------------------------. | |
| (shift title)                                      | |v|
----------------------------------------------------------
  (tools)                            (budget)

However, when there were more hospitals than restaraunts, every
restaurant could not necessarily be viewed alongside every hospital:
| | (final restaurant)   |                           | |^|
| '----------------------'                           | | |
|                                                    | | |
|                                                    | | |
|                           (final hospital)         | | |
'----------------------------------------------------' | |
.----------------------------------------------------. | |
| (shift title)                                      | |v|
----------------------------------------------------------
  (tools)                            (budget)

In a drag-and-drop interface such as this, it is critical for every
draggable element (restaurants) to be visible alongside every drop
target (hospital).
Alter the layout such that every shift is always completely visible and
that both the list of restaurants and the list of hospitals is presented
with a scroll bar whenever their contents exceed the available vertical
space:
.----------------------------------------------------. |^|
| (shift title)                                      | | |
| .----------------------.  .----------------------. | | |
| |                    |^|  |                    |^| | | |
| | (restaurants)      | |  | (hospitals)        | | | | |
| |                    |v|  |                    |v| | | |
| '----------------------'  '----------------------' | | |
'----------------------------------------------------' | |
.----------------------------------------------------. | |
| (shift title)                                      | |v|
----------------------------------------------------------
  (tools)                            (budget)

Open question: does this solve the min height issue from @susanpark 's design?

I've resolved the conflicts from recent changes to the dev branch and further extended the solution to eliminate unused space.

@jugglinmike - I checkout the branch locally and tried it out in the Capacity Planning Example Base and things look really weird and broken.

Nice catch, @rmeritz. I didn't account for situations with such a small number of consumers. That should be fixed, now--mind taking another look?

@jugglinmike - The Consumers look fine to be now. But now I'm not seeing any Producers at all, but I just fetched the lastest dev and I have the same problem there. I'm not getting any restaurants to show up anymore in block. This is probably good to go my I'm going to figure out what my issue is tomorrow morning with fresh eyes.


buildeng: replace xcopy with robocopy

robocopy is capable of working with paths > MAX_PATH. This replaces all our PostBuildEvent xcopy calls with robocopy.
Need to verify these things before merging:

[ ] - This should fix #70

this doesn't fix the issue, the issue is visual studio's fault. as such, i see no reason to take these changes.


Providing value via cli for input w/ 'docker-image-ref' constraint fails

providing same value via env var works

unable to repro as of 0.1.15-beta.127


[+!LIVE!*Reddit+] Liverpool vs Aston Villa LIVE STreams FREE ON TV Channel 9 November 2024

In the Premier League, Arne Slot‘s in-form Liverpool host Aston Villa who are experiencing a dip in results at the moment. Here’s how to watch and stream live around the world.
Anfield will be up for this one as Unai Emery brings Aston Villa to Merseyside for a Saturday night match against Liverpool, who are currently Europe’s best-performing team.
With Liverpool having won their last six games at Anfield and Aston Villa losing their last three matches in all competitions, the Reds are favourites.
Emery has previously shown, though, that he can set a side up to frustrate, so Liverpool and the crowd need to be ready to put in a display against Aston Villa if they want to remain comfortably top of the Premier League.
Check This Is Anfield for the latest team news ahead of the match.
You can also follow the game by downloading the This is Anfield app, including lineups, live coverage and all the reaction and analysis.
The match gets underway at 8pm (GMT) – or 3pm in New York, 12pm in Los Angeles, 7am (Sunday) in Sydney, 12am (Sunday) in Dubai and 11pm in Nairobi.
UK Viewers
Liverpool vs. Aston Villa is being shown live on TNT Sports 1 in the UK, which is available to live stream here.
Live audio commentary of Liverpool vs. Aston Villa and a full match replay and highlights will also be available on LFCTV GO here.
This Is Anfield will also be providing live text commentary throughout on our website and full match reaction and analysis on the This Is Anfield app for iPhone and Android later.
Download free here.
US Viewers
Liverpool vs. Aston Villa is shown live on Peacock in the US, which is available to live stream with Peacock here.
Live audio commentary of Liverpool vs. Aston Villa and a full match replay and highlights will also be available on LFCTV GO here.
This Is Anfield will also be providing live text commentary throughout on our website and full match reaction and analysis on the This Is Anfield app for iPhone and Android later.
Download free here.
Canada Viewers
Liverpool vs. Aston Villa is being shown live on Fubo TV in Canada, which is available to live stream with Fubo TV here.
Live audio commentary of Liverpool vs. Aston Villa and a full match replay and highlights will also be available on LFCTV GO here.
This Is Anfield will also be providing live text commentary throughout on our website and full match reaction and analysis on the This Is Anfield app for iPhone and Android later.
Download free here.
Worldwide
Live audio commentary of Liverpool vs. Aston Villa and a full match replay and highlights will be available on LFCTV GO here.
You can find live listings for Liverpool vs. Aston Villa around the world at Live Soccer TV’s website, here.
This Is Anfield will also be providing live text commentary throughout on our website and full match reaction and analysis on the This Is Anfield app for iPhone and Android later.

fdgdfgr


Use distroless to build the python container

Fix #28991.
distroless does not provide the images with different python versions. And the Beam python containers also contains extras system packages.
Ideas:

Use FROM python:"${py_version}"-bookworm as python-base as the python base image to install everything Beam needs including both system packages and python packages under /venv
copy all the packages over to the distroless image

Using the python bullseye as the base,

Using the distroless as the base,

Fix #28991

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

What is the underlying motivation here?
On Wed, Oct 18, 2023, 3:24 PM tvalentyn @.***> wrote:

@.**** commented on this pull request.
In sdks/python/container/Dockerfile
https://github.com/apache/beam/pull/29001#discussion_r1364624012:

pip install --upgrade pip setuptools wheel &&
+RUN python -m venv venv-beam

Beam SDK workers create a venv as well, which is superimposed upon the
default environment. I think having Beam dependencies packages in a
separate venv would break this setup.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/29001#pullrequestreview-1686300631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADWVAOZSTTFQKPC3E7CVYLYABJKDAVCNFSM6AAAAAA6BMBU6WVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMOBWGMYDANRTGE
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

I also wonder if the maintainers of python images are aware of  the potential vulnerability concerns and perhaps those issues could be resolved upstream of Beam without us not getting into building Python images from scratch.

What is the underlying motivation here?
…
On Wed, Oct 18, 2023, 3:24 PM tvalentyn @.> wrote: @.* commented on this pull request. ------------------------------ In sdks/python/container/Dockerfile <#29001 (comment)>: > - pip install --upgrade pip setuptools wheel && \ +RUN python -m venv venv-beam Beam SDK workers create a venv as well, which is superimposed upon the default environment. I think having Beam dependencies packages in a separate venv would break this setup. — Reply to this email directly, view it on GitHub <#29001 (review)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADWVAOZSTTFQKPC3E7CVYLYABJKDAVCNFSM6AAAAAA6BMBU6WVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMOBWGMYDANRTGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

We just create the temp venv in the python-base image to install all the packages and later copy them to the distroless image.

Tested the wordcount example using Dataflow. It works.

Run Python_Transforms PreCommit

Run Python ValidatesContainer Dataflow ARM 3.8

Run Python ValidatesContainer Dataflow ARM 3.11

Run Python Dataflow ValidatesContainer

Run Python_Integration PreCommit 3.8

Run Python Dataflow ValidatesContainer

Run Python ValidatesContainer Dataflow ARM 3.11

I can find test results - could you paste a link for container test suites please after we iron out any remaining issues with test infra?

I am not sure why ARM suite is not showing up, it's on github actions, I thought only Jenkins had issues

Run Python ValidatesContainer Dataflow ARM 3.8

Run Python ValidatesContainer Dataflow ARM

Run Python 3.8 Postcommit

Regular postcommits wouldn't pick up these changes. ARM postcommits should

Run Python PostCommit Arm

Close this for now since we plan to support more container images with different purposes.