lxd won't start after upgrade

Required information

Distribution: Ubuntu
Distribution version: 18.10
The output of "lxc info" or if that fails:

Kernel version: 4.18.0-14-generic
LXC version: 3.9
LXD version: 3.9
Storage backend in use:btrfs

Issue description
After upgrading to lxd 3.9 which is only available as snap the lxd daemon won't start.
Steps to reproduce
sudo lxd --debug --group lxd
INFO[02-01|20:17:02] LXD 3.9 is starting in normal mode       path=/var/snap/lxd/common/lxd
EROR[02-01|20:17:02] Failed to start the daemon: failed to init dir /var/snap/lxd/common/lxd/backups: mkdir /var/snap/lxd/common/lxd/backups: read-only file system
INFO[02-01|20:17:02] Starting shutdown sequence
DBUG[02-01|20:17:02] Not unmounting temporary filesystems (containers are still running)
INFO[02-01|20:17:02] Saving simplestreams cache
INFO[02-01|20:17:02] Saved simplestreams cache
EROR[02-01|20:17:02] Failed to cleanly shutdown daemon: open /var/snap/lxd/common/lxd/cache/simplestreams.yaml: no such file or directory
Error: failed to init dir /var/snap/lxd/common/lxd/backups: mkdir /var/snap/lxd/common/lxd/backups: read-only file system

Suggests some filesystem problem as it says that /var/snap/lxd/common/lxd is read-only on your system.
Can you show cat /proc/self/mountinfo?

Hi, thanks for the quick reply
actually that directory in a symlink to another directory (/srv) where the containers are located. It is not read only.
sudo cat /proc/self/mountinfo
22 27 0:21 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
23 27 0:4 / /proc rw,nosuid,nodev,noexec,relatime shared:15 - proc proc rw
24 27 0:6 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev rw,size=31980948k,nr_inodes=7995237,mode=755
25 24 0:22 / /dev/pts rw,nosuid,noexec,relatime shared:3 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
26 27 0:23 / /run rw,nosuid,noexec,relatime shared:5 - tmpfs tmpfs rw,size=6401040k,mode=755
27 0 0:24 /@ / rw,noatime,nodiratime shared:1 - btrfs /dev/sdc2 rw,compress=zlib:3,ssd,space_cache,subvolid=257,subvol=/@
28 22 0:7 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:8 - securityfs securityfs rw
29 24 0:27 / /dev/shm rw,nosuid,nodev shared:4 - tmpfs tmpfs rw
30 26 0:28 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs tmpfs rw,size=5120k
31 22 0:29 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:9 - tmpfs tmpfs ro,mode=755
32 31 0:30 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:10 - cgroup2 cgroup2 rw
33 31 0:31 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,xattr,name=systemd
34 22 0:32 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:12 - pstore pstore rw
35 22 0:33 / /sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime shared:13 - efivarfs efivarfs rw
36 22 0:34 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:14 - bpf bpf rw,mode=700
37 31 0:35 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,blkio
38 31 0:36 / /sys/fs/cgroup/rdma rw,nosuid,nodev,noexec,relatime shared:17 - cgroup cgroup rw,rdma
39 31 0:37 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:18 - cgroup cgroup rw,devices
40 31 0:38 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:19 - cgroup cgroup rw,cpu,cpuacct
41 31 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:20 - cgroup cgroup rw,hugetlb
42 31 0:40 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:21 - cgroup cgroup rw,net_cls,net_prio
43 31 0:41 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:22 - cgroup cgroup rw,perf_event
44 31 0:42 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:23 - cgroup cgroup rw,pids
45 31 0:43 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:24 - cgroup cgroup rw,freezer
46 31 0:44 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:25 - cgroup cgroup rw,memory
47 31 0:45 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:26 - cgroup cgroup rw,cpuset
48 23 0:46 / /proc/sys/fs/binfmt_misc rw,relatime shared:27 - autofs systemd-1 rw,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13874
49 24 0:47 / /dev/hugepages rw,relatime shared:28 - hugetlbfs hugetlbfs rw,pagesize=2M
50 22 0:8 / /sys/kernel/debug rw,relatime shared:29 - debugfs debugfs rw
51 24 0:19 / /dev/mqueue rw,relatime shared:30 - mqueue mqueue rw
52 50 0:11 / /sys/kernel/debug/tracing rw,relatime shared:31 - tracefs tracefs rw
53 22 0:20 / /sys/kernel/config rw,relatime shared:32 - configfs configfs rw
54 22 0:49 / /sys/fs/fuse/connections rw,relatime shared:33 - fusectl fusectl rw
124 27 7:0 / /snap/lxd/9919 ro,nodev,relatime shared:67 - squashfs /dev/loop0 ro
131 27 7:1 / /snap/core/6350 ro,nodev,relatime shared:69 - squashfs /dev/loop1 ro
127 27 0:24 /@home /home rw,noatime,nodiratime shared:71 - btrfs /dev/sdc2 rw,compress=zlib:3,ssd,space_cache,subvolid=258,subvol=/@home
129 27 0:53 / /srv rw,noatime,nodiratime shared:73 - btrfs /dev/sdd1 rw,compress=zlib:3,ssd,space_cache,subvolid=5,subvol=/
134 27 8:33 / /boot/efi rw,relatime shared:75 - vfat /dev/sdc1 rw,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro
122 27 0:52 / /data rw,noatime,nodiratime shared:77 - btrfs /dev/sda rw,compress=zlib:3,space_cache,autodefrag,subvolid=5,subvol=/
339 27 0:63 / /var/lib/lxcfs rw,nosuid,nodev,relatime shared:197 - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
328 26 0:23 /snapd/ns /run/snapd/ns rw,nosuid,noexec,relatime - tmpfs tmpfs rw,size=6401040k,mode=755
358 328 0:3 mnt:[4026532312] /run/snapd/ns/lxd.mnt rw - nsfs nsfs rw
359 26 0:141 / /run/user/1000 rw,nosuid,nodev,relatime shared:204 - tmpfs tmpfs rw,size=6401040k,mode=700,uid=1000,gid=1000

Don't do symlinks in there, it's going to seriously mess up snapd and its confinement, causing such errors.
This is not a setup we can support.
Instead an alternative would be to setup a bind-mount of /srv to /var/snap/lxd/common/lxd which while not something we'd officially support and something you may still run into some corner case issues with, should still work much better in general.
For such setups, you'd also usually get a better chance of it working if you have all of /var/snap/lxd/common on that mountpoint rather than just the lxd directory within it, that's because otherwise you may run into cross-mount issues.
Closing as this is an unsupported setup, but hopefully the above gives you enough pointers to get something more robust in place.

Thanks, the bind-mount helped that issue.
Generally the upgrade from lxd version <3.0 to 3.9 caused some other issues as well.
I am still fightig to get all containers running. Some of them are ok, some just won't work anymore.
This issue is the id mapping.
There is one container which would not start. If I look into the "rootfs" of it from the host, all directories and files are with uid/gid from the host (like 0 for root etc.). The rootfs directory itself is uid/gid=1000000 everything below is like the id-s of the host.
I tried to change the id-s of all the files below "rootfs". I wrote a script that adds this offset to the uid/gid of all files.
I set security.idmap.isolated true and the base to 1000000. Once I start the container it changes back all uid/gid to the host's and won't run.
/etc/subuid, /etc/subgid are set correctly but it shouldn't matter after lxd > 3.0 as far as I know...
Now where else can I have a wrong setting that causes this to happen?

The snap doesn't care about /etc/subuid and /etc/subgid, it's possible that in the middle of this, the container config got wrong.
If you have such containers, it's possible to get LXD to remap things properly by manually editing the volatie.last_state.idmap and volatile.idmap.next, setting the first to the way things are on the filesystem and the latter to the state you'd like things to be in. Upon startup, LXD will remap things for you.
We also have a fuidshift binary as part of the LXD code tree which can be used to manually remap whole filesystem trees though the other approach is certainly preferred.


process.env.TAP_CHILD_ID for parallel processes

process.env.TAP_CHILD_ID provides an integer id for the test process. For instance, given tap -J4, test ids would be 0-3. Test ids could be reused once the given test child process exits. In non-parallel mode, process.env.TAP_CHILD_ID would always === 0.
This could be used to provision resources based on the child id. This would make it easier to turn sequential tests into parallel tests.

in non-parallel mode, a unique id for each child would be useful..what added-value is given by having 0 for every child id?


chore: document new hooks for react setups

🚀 Deployed on https://66ce39cd0d24d786be7bee8b--cozy-fox-0defec.netlify.app


chore: Add frontend

Description
This PR merges annotto-front with annotto-api together in the same repository. Be aware that this is not a monorepo. The intents are to have a single repository but that keeps the frontend independent.
The github workflows now produces one more docker image:

ljnrepo/annotto that starts a self contained Annotto Stack with Keycloak, Postgres, and MongoDB, all together to facilitate the deployment and the launch of Annotto in a quick and easy way.

Type of Change
Please delete options that are not relevant (including this descriptive text).

[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update
[ ] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Tested on a local machine.

In the package.json at the root of the project, shouldn't the frontend test scripts be launched at the same time as the API test scripts?
During a commit on the project the pre-commit hook doesn't launch the tests on the frontend

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Sitl debugging, some bug fixes and various cleanup

This used to be https://github.com/PX4/Firmware/pull/5432
I renamed the branch to sitl_debugging because I think the per-target optimization control is main subject in this all.
Some documentation written for this can be reviewed here:
https://github.com/CarloWood/Devguide/blob/d8c8bd418e6fcef71432a5511a5416e264f5c1f9/simulation-debugging.md
Please see the commit messages for a detailed description of what this does.

Is there anything else that I should do before this can be merged?

The list_cmake_targets target does not work properly for me, as it outputs:
[1/1] List of cmake targets that ...e matched by PX4_NO_OPTIMIZATION:
drivers__boards__sitl\ndrivers__device\ndrivers__gps\ndrivers__pwm_out_sim\ndrivers__sf0x__sf0x_tests\nexamples__attitude_estimator_ekf\nexamples__ekf_att_pos_estimator\nexamples__mc_att_control_multiplatform\nexamples__mc_pos_control_multiplatform\nexamples__px4_simple_app\nlib__DriverFramework__framework\nlib__controllib\nlib__conversion\nlib__ecl\nlib__external_lgpl\nlib__geo\nlib__geo_lookup\nlib__launchdetection\nlib__mathlib\nlib__mathlib__math__filter\nlib__rc\nlib__rc__rc_tests\nlib__runway_takeoff\nlib__tailsitter_recovery\nlib__terrain_estimation\nmodules__attitude_estimator_q\nmodules__commander\nmodules__commander__commander_tests\nmodules__controllib_test\nmodules__dataman\nmodules__ekf2\nmodules__fw_att_control\nmodules__fw_pos_control_l1\nmodules__land_detector\nmodules__local_position_estimator...

If I replace echo with printf it works.
Also, the list gets appended every time cmake re-runs, so I get something like this:
[1/1] List of cmake targets that ...e matched by PX4_NO_OPTIMIZATION:
drivers__boards__sitl
drivers__boards__sitl
drivers__boards__sitl
drivers__boards__sitl
drivers__device
drivers__device
drivers__device
drivers__device
drivers__gps
drivers__gps
drivers__gps
drivers__gps
drivers__pwm_out_sim
drivers__pwm_out_sim
drivers__pwm_out_sim
drivers__pwm_out_sim
...

Ah, good catch thanks! I'll fix that later today.

Is 'printf' POSIX? Does 'echo -e' work for you? It's kinda weird that it works for me and not for you, I wonder where is the difference. Looks like mine gobbles up a backslash more.

Yes printf is posix (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html), but echo -e works too. Not sure where the difference comes from, maybe it's ninja?

The merge conflict is trivial - but how can I do that when I'm not supposed to simply merge master into my branch and push it again?

A rebase on master would do the trick.

You missed to replace echo with printf for the list_cmake_targets target. Other than that it looks good.
If you rebase & force push the CI retest.

rebased & applied

@CarloWood I tried "make posix_sitl_ekf2 jmavsim___gdb" and got no connection to QGC.
There is also no response to ctrl-C: (Ubuntu 15.10)
12:21:00 {(d2626d7...)} ~/git/kd0aij/PX4/Firmware$ make posix_sitl_ekf2 jmavsim___gdb
PX4 CONFIG: /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2
ninja: Entering directory `/home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2'
[1/1] cd /home/markw/git/kd0aij/PX4/Firmware/build_posix_s... /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2
args: /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2/src/firmware/posix/px4 posix-configs/SITL/init/ekf2 gdb jmavsim none /home/markw/git/kd0aij/PX4/Firmware /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2
SITL ARGS
sitl_bin: /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2/src/firmware/posix/px4
rcS_dir: posix-configs/SITL/init/ekf2
debugger: gdb
program: jmavsim
model: none
src_path: /home/markw/git/kd0aij/PX4/Firmware
build_path: /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2
empty model, setting iris as default
SITL COMMAND: /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2/src/firmware/posix/px4 /home/markw/git/kd0aij/PX4/Firmware /home/markw/git/kd0aij/PX4/Firmware/posix-configs/SITL/init/ekf2/iris
GNU gdb (Ubuntu 7.10-1ubuntu2) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2/src/firmware/posix/px4...Buildfile: /home/markw/git/kd0aij/PX4/Firmware/Tools/jMAVSim/build.xml
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
data path: /home/markw/git/kd0aij/PX4/Firmware
commands file: /home/markw/git/kd0aij/PX4/Firmware/posix-configs/SITL/init/ekf2/iris
[New Thread 0x7ffff6d47700 (LWP 18156)]
18446744073709551586 WARNING: setRealtimeSched failed (not run as root?)
[New Thread 0x7ffff7ff4700 (LWP 18157)]
[New Thread 0x7ffff6546700 (LWP 18158)]
[New Thread 0x7ffff7fea700 (LWP 18159)]
[New Thread 0x7ffff5d45700 (LWP 18160)]
[Thread 0x7ffff7fea700 (LWP 18159) exited]
[New Thread 0x7ffff7fe0700 (LWP 18161)]
[New Thread 0x7ffff5544700 (LWP 18162)]
[Thread 0x7ffff7fe0700 (LWP 18161) exited]

______  __   __    ___
| ___ \ \ \ / /   /   |
| |_/ /  \ V /   / /| |
|  __/   /   \  / /_| |
| |     / /^\ \ \___  |
\_|     \/   \/     |_/

px4 starting.

[Thread 0x7ffff7ff4700 (LWP 18157) exited]
[New Thread 0x7ffff7fd6700 (LWP 18163)]
[New Thread 0x7ffff4d43700 (LWP 18164)]
[Thread 0x7ffff7fd6700 (LWP 18163) exited]
INFO  [dataman] Unkown restart, data manager file 'rootfs/fs/microsd/dataman' size is 103090 bytes

make_dirs:

compile:

create_run_jar:
[New Thread 0x7ffff7fcc700 (LWP 18165)]
[Thread 0x7ffff7fcc700 (LWP 18165) exited]
[New Thread 0x7fffeffff700 (LWP 18166)]
INFO  [platforms__posix__drivers__ledsim] LED::init
INFO  [platforms__posix__drivers__ledsim] LED::init
INFO  [simulator] Not using /dev/ttyACM0 for radio control input. Assuming joystick input via MAVLink.
INFO  [simulator] Waiting for initial data on UDP port 14560. Please start the flight simulator to proceed..

copy_res:

BUILD SUCCESSFUL
Total time: 0 seconds
Options parsed, starting Sim.
Starting GUI...
3D [dev] 1.6.0-pre12-daily-experimental daily

Init MAVLink
INFO  [simulator] Got initial simuation data, running sim..
 I don't see anything being compiled when you issue the command though. Is your problem perhaps fixed by doing a 'make clean' first? And/or deleting the build directory...
Anyway, I don't see how the problem that you describe can be related to this patch, as it only changes the build system a bit (ie, the Makefile). Possible expected problems would be of the kind where targets are not built, or give some kind of error when building.

"make clean" before "make posix_sitl_ekf2 jmavsim___gdb" still results in no communication with QGC.
I also get no output from "make list_vmd_make_targets". Perhaps that provides a clue as to what's wrong.

Can you please remove /home/markw/git/kd0aij/PX4/Firmware/build_posix_sitl_ekf2
And then copy&paste the full output of your make command, including the make command and the pwd that you run it from, on a paste site, like I did?

07:00:10 (master) ~/git/kd0aij/PX4/Firmware$ ninja --version
1.6.0
https://gist.github.com/kd0aij/8b0fae37a42f520dbd459b3e706b8b53

06:53:31 {master} ~/git/kd0aij/PX4/Firmware$ make list_cmake_targets
06:53:36 {master} ~/git/kd0aij/PX4/Firmware$ make list_vmd_make_targets
Those are build directory targets, not targets of the root Makefile.
By the way, after my patch of #5491 you'll get:
sean:/usr/src/px4/Firmware/Firmware.git>make list_cmake_targets
Makefile:292: *** "list_cmake_targets cannot be the first argument. Use 'make help|list_config_targets' to get a list of all possible [configuration] targets.".  Stop.

You can run this in the build directory, or you could run:
$ make posix_sitl_ekf2 list_cmake_targets
$ make posix_sitl_ekf2 list_vmd_make_targets
I don't know why you're having problems, can't really think of a reason.
However, I see you have PX4_NO_OPTIMIZATION set (why?).
Do things work when you unset that environment variable?

Thanks, Carlo.
Regarding PX4_NO_OPTIMIZATION, I was following the instructions here:
https://github.com/CarloWood/Devguide/blob/d8c8bd418e6fcef71432a5511a5416e264f5c1f9/simulation-debugging.md#compiler-optimization
Unsetting it seemed to make no difference; QGC doesn't load parameters when
the simulator starts. Do I need to issue any gdb commands first?
https://gist.github.com/kd0aij/cccbecd574ff481d278080898e0d6deb
Also, how would I start the gdb session from within Eclipse (or attach it)?
That would be much preferable to the command line interface.
On Wed, Sep 14, 2016 at 2:08 PM Carlo Wood<EMAIL_ADDRESS>wrote:

06:53:31 {master} ~/git/kd0aij/PX4/Firmware$ make list_cmake_targets
06:53:36 {master} ~/git/kd0aij/PX4/Firmware$ make list_vmd_make_targets
Those are build directory targets, not targets of the root Makefile.
By the way, after my patch of #5491
https://github.com/PX4/Firmware/pull/5491 you'll get:
sean:/usr/src/px4/Firmware/Firmware.git>make list_cmake_targets
Makefile:292: *** "list_cmake_targets cannot be the first argument. Use 'make help|list_config_targets' to get a list of all possible [configuration] targets.".  Stop.
You can run this in the build directory, or you could run:
$ make posix_sitl_ekf2 list_cmake_targets
$ make posix_sitl_ekf2 list_vmd_make_targets
I don't know why you're having problems, can't really think of a reason.
However, I see you have PX4_NO_OPTIMIZATION set (why?).
Do things work when you unset that environment variable?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PX4/Firmware/pull/5443#issuecomment-247137488, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACMZPZgK0fBCGcu3F7psk27KfauB1slXks5qqFQ4gaJpZM4J0Xxo
.

Regarding PX4_NO_OPTIMIZATION, I was following the instructions here:

That's not an instruction is it? It's an example, saying IF you'd do this then the effect is that.
I cannot help you with Eclipse, I never used that. I'm fairly new to this whole project myself, so I'm afraid I can't help. What you pasted looks like it works to my eyes, and certainly the part that is related to my commit. I'd appreciate it if you could take the time to do a git bisect to find out which commit is causing this; are you 100% sure it is caused by my changes?

Do you use QGC with SITL?
On Wed, Sep 14, 2016 at 3:06 PM Carlo Wood<EMAIL_ADDRESS>wrote:

Regarding PX4_NO_OPTIMIZATION, I was following the instructions here:
That's not an instruction is it? It's an example, saying IF you'd do this
then the effect is that.
I cannot help you with Eclipse, I never used that. I'm fairly new to this
whole project myself, so I'm afraid I can't help. What you pasted looks
like it works to my eyes, and certainly the part that is related to my
commit. I'd appreciate it if you could take the time to do a git bisect to
find out which commit is causing this; are you 100% sure it is caused by my
changes?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PX4/Firmware/pull/5443#issuecomment-247153390, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACMZPdTcvwpVr6r55RgyKM6f5EQnlNtQks5qqGHDgaJpZM4J0Xxo
.

@kd0aij I just tested, I can connect QGC to jmavsim with gdb (make posix_sitl_ekf2 jmavsim___gdb)

After discussing this in gitter with Mark, it became apparent that this is probably unrelated to the changes made in this PR: it's simply the first time that Mark tries to use gdb. Nevertheless, with his help I was able to reproduce the issue and I'm willing to investigate it; after all, having a working connection between QGC and the firmware while debugging in gdb helps ;).
So far the following came out of the investigation:
Things work when gdb is not used, and something locks up when gdb is used. This even happens when gdb is attached afterwards.
Not only px4 becomes unresponsive (read: appears to be locked up), even gdb itself is often locked up (ie, doesn't react to control-C).
Sending the px4 process SIGINT signals causes it to continue shortly; ie QGC reacts (sees the process) and I get a pxh> prompt, but immediately after things return to the locked-in state.


⚒ refactored code

Fixed env.example
fixed folder structure
added .nvmrc file for more information on nvm click here
added proper separation of concerns from client/src/index.css to client/src/styles/index.css, fixed all imports in index.js
📝 Documentation update

I am excited to see what concepts you come up with here! One potential concern I have is @intelagense, and I choose BuB/MVC because it's a format we've all worked work and is accessible.
Will these changes be approachable to someone with only BuB experience? What will some of these changes allow us to do that BuB/MVC do not?

Hey @Neoneoexe, just copying over what @Caleb-Cohen has wrote on issue #33 incase you haven't read it yet.

By definition MVC is
“Model–view–controller is a software architectural pattern commonly used for developing user interfaces that divide the related program logic into three interconnected elements. This is done to separate internal representations of information from the ways information is presented to and accepted from the user.“
I don’t see how a folder structure would change a architectural principal.
You can read more on “MVC with mern” here
As for BuB I am not sure since I’ve never used it but if it’s architectural paradigm, I don’t think it would be a good idea to mix 2 of them such as MVC and BuB but again I don’t see how this folder structure changes anything.
The changes I made are:

Made separation of concerns easier by dividing the project into 2 parts frontend(client) and backend(server),
front end runs independently of the backend in every “full stack” application, you connect them by fetching from your backend and in your backend you allow fetching from a specific address and with CORS , read more about CORS and connecting backend with front end here

Added some comments. There's quite a bit of change to unpack here, to be honest, I wish we could have done these refactors progressively.
I will take read over your previous comment and links!

The code that I originally had upload from my pc had given you guys issues with "not giving contributions properly" so I tried editing them from my phone and commited it, unfortunately I didn't check over the package.json and some other files such as the proper formatting of the readme.md put i belive everything else should be fine

Can we set up a time to talk about this in Discord? I want to get a better understanding of what this accomplishes.

Can we set up a time to talk about this in Discord? I want to get a better understanding of what this accomplishes.

for sure.

Hey Albin,
I am going to close this PR for now. I want to break down the ideas into individual PRs for manageability and conflict reduction. Here's the list of feedback that I had from our discussion yesterday. Please let me know if I am missing anything!

Get engines package so we can display node version to prevent conflicts
Add package-lock to .gitignore
Change file management to assist in the separation of concerns between frontend and backend
Update .ENV.Example to have more proper formatting
Consider Vite for more efficient compiling
Consider Nest for more seamless backend communication
Remove the .gitignore folder in frontend


Define feature gates centrally instead of once per component

How to categorize this issue?

/area dev-productivity quality
/kind cleanup
What would you like to be added:
We define feature gates multiple times, once per component which uses it.
E.g. the UseDNSRecords feature gate is defined three times:
https://github.com/gardener/gardener/blob/744e0ff5db4f7f58d70a2f43546d379303f8a2cf/pkg/apiserver/features/features.go#L28
https://github.com/gardener/gardener/blob/744e0ff5db4f7f58d70a2f43546d379303f8a2cf/pkg/controllermanager/features/features.go#L29
https://github.com/gardener/gardener/blob/744e0ff5db4f7f58d70a2f43546d379303f8a2cf/pkg/gardenlet/features/features.go#L35
Also, there is one featuregate.NewFeatureGate() call per component, i.e. one feature gate map per component.
Instead, feature gates should be defined centrally once and added to utilfeature.DefaultMutableFeatureGate like in k/k:
https://github.com/kubernetes/kubernetes/blob/b181e174f25df752eb176c2393b95569646230d3/pkg/features/kube_features.go#L829
https://github.com/kubernetes/kubernetes/blob/b181e174f25df752eb176c2393b95569646230d3/pkg/features/kube_features.go#L822-L824
Why is this needed:
Redefining features gates is a source of programmer errors: one can easily forget to adapt the multiple definitions. Also shared code might be looking at features gates without someone noticing, that the feature gate is not registered with the component's map of feature gates.

I think that there are some benefits when each component has its own map of feature gates. It makes much more easy to check which feature gate is relevant for which component. If we switch to the approach to define them centrally then each component will register all feature gates - which can be misleading -> you will be allowed to configure a feature gate for a component when the component does not such feature gate or does not have handling for it. When the feature gates count grows it is also polluting the help text - for example https://github.com/kubernetes-sigs/cloud-provider-azure/issues/943 where azure CCM gets all kube-apiserver feature gates registered -> help text shows completed unrelated feature gates where most of them do not make any sense to the azure CCM.

We agreed to add documentation about which feature gates are relevant for which of our components. Additionally, we would like to move the FeatureGateSpec (containing information about the maturity level) to a central place (pkg/features) and then just import it in the relevant components.

/assign


More details in the contributing page

For example, a bit of explanation about quarto and how the repository is laid out. Also, a link to the repo 😄

Resolved in 73420c38f60b5233a03ae77dffd3ef1fc92c0c2b


feat: Create ellipse component

Description

Replace this text.

Checklist

[ ] I have followed the Contributor Guide when preparing my PR.
[ ] I have updated/added tests for ALL new/updated/fixed functionality.
[ ] I have updated/added relevant documentation in docs and added dartdoc comments with ///.
[ ] I have updated/added relevant examples in examples or docs.

Breaking Change?

[ ] Yes, this PR is a breaking change.
[ ] No, this PR is not a breaking change.

Related Issues

Can we add some basic tests? You can look at other shapes tests like RectangleComponentTest In particular the intersection method

I can't find the related demo. Could you please provide the specific test demo file location?
In particular, I need to add some basic tests for the intersection method and would like to reference tests like RectangleComponentTest.
Thank you!

Not a demo, just unit tests: packages/flame/test/components/rectangle_component_test.dart

@arthasyou do you need some guidance on how to get started with the tests? The rest of this PR looks great :)

@arthasyou do you need some guidance on how to get started with the tests? The rest of this PR looks great :)

Yes just need a example

@arthasyou here is the simplest one to start with:
https://github.com/flame-engine/flame/blob/refs%2Fheads%2Fmain/packages%2Fflame%2Ftest%2Fcomponents%2Frectangle_component_test.dart
And then intersections etc have to be tested here:
https://github.com/flame-engine/flame/blob/refs/heads/main/packages/flame/test/collisions/collision_detection_test.dart#L259
I think you can just search for RectangleComponent in the test directory and follow the same thing as is done for that one.
A follow up that would be great is if we also got a EllipseHitbox. 😃

@arthasyou here is the simplest one to start with: https://github.com/flame-engine/flame/blob/refs%2Fheads%2Fmain/packages%2Fflame%2Ftest%2Fcomponents%2Frectangle_component_test.dart
And then intersections etc have to be tested here: https://github.com/flame-engine/flame/blob/refs/heads/main/packages/flame/test/collisions/collision_detection_test.dart#L259
I think you can just search for RectangleComponent in the test directory and follow the same thing as is done for that one.
A follow up that would be great is if we also got a EllipseHitbox. 😃

ty I'll do it

adding the test file, pls check if these code can work well. ty

updated test file


Saving sub-scenes will mess up the display of the main scene on the 2D screen

Tested versions
v4.3.dev5.official [c9c17d6ca] good
f47f4a02c bad
System information
Linux Mint 21.3 (Virginia) - X11 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1050 Ti (nvidia; 535.161.07) - Intel(R) Core(TM) i5-7300HQ CPU @ 2.50GHz (4 Threads)
Issue description
Saving the sub-scene and switching to the main scene reloads the scene, and it seems like the position calculation of something is dependent on the current camera.
If there is a enabled (default value) Camera2D node in the main scene, some things in the main scene will be displayed in the wrong position after reloading. For example, the position of the camera screen rectangle. Parallax2D's screen_offset may also change.
https://github.com/godotengine/godot/assets/30386067/869cba0c-e815-47b0-a7dc-5fad8c0f11fa
Closing and reopening the scene will display correctly.
Steps to reproduce

Download and use godot to open the MRP;
Open main.tscn;
Open sub.tscn, and save;
Switch to the tab of main.tscn, it will be automatically reloaded, and the displayed content will be messed up after reloading.

Minimal reproduction project (MRP)
wrong_display.zip

CC @ajreckof @KoBeWi

it seems the Camera is always rendered at the exact same position and everything else is rendered based on there relative position as if it became the actual Camera of the 2D editor.

I discovered that the auto-reload here causes Camera2D::_is_editing_in_editor() (i.e. is_part_of_edited_scene()) to evaluate to false. This causes make_current() to be called when entering the scene tree.
https://github.com/godotengine/godot/blob/e5b4ef8e9522e950033cbece39a31a4a76da19c1/scene/2d/camera_2d.cpp#L314-L316
https://github.com/godotengine/godot/blob/e5b4ef8e9522e950033cbece39a31a4a76da19c1/scene/main/node.cpp#L2573-L2576
This may be due to the current editing scene root not being set to the main scene root after switching tabs but before auto-reloading.

is_part_of_edited_scene() technically returns true for siblings of edited scene, I think we could check straight for the root viewport instead of using scene root parent.

It doesn't seem to be fully fixed by #90317. get_tree()->get_edited_scene_root() has not changed.

It seems that we can not use repace_by, because the diffs have been packed before.
https://github.com/godotengine/godot/blob/a7b860250f305f6cbaf61c30f232ff3bbdfdda0b/editor/editor_data.cpp#L704-L711
So directly call EditorNode::set_edited_scene() to set new_scene, and then delete old_root to complete the replacement.
diff --git a/editor/editor_data.cpp b/editor/editor_data.cpp
index 72225fd454..bdc6504417 100644
--- a/editor/editor_data.cpp
+++ b/editor/editor_data.cpp
@@ -722,8 +722,7 @@ bool EditorData::check_and_update_scene(int p_idx) {

                new_scene->set_scene_file_path(edited_scene[p_idx].root->get_scene_file_path());
                Node *old_root = edited_scene[p_idx].root;
-               edited_scene.write[p_idx].root = new_scene;
-               old_root->replace_by(new_scene, false, false);
+               EditorNode::get_singleton()->set_edited_scene(new_scene);
                memdelete(old_root);
                edited_scene.write[p_idx].selection = new_selection;


fix: dict_source all

What's in this PR:
Selecting dictionary source "all" resulted in zero entries. That has been resolved.

Codecov Report

Merging #1120 (3b3dbdc) into main (67092b5) will decrease coverage by 0.22%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #1120      +/-   ##
==========================================
- Coverage   77.63%   77.41%   -0.23%
==========================================
  Files         154      154
  Lines        5835     5689     -146
  Branches      971      914      -57
==========================================
- Hits         4530     4404     -126
+ Misses       1158     1145      -13
+ Partials      147      140       -7

Impacted Files
Coverage Δ

src/CreeDictionary/API/search/presentation.py
71.70% <0.00%> (-4.32%)
:arrow_down:

src/CreeDictionary/API/search/types.py
88.80% <0.00%> (-0.09%)
:arrow_down:

src/CreeDictionary/CreeDictionary/views.py
62.71% <0.00%> (+0.56%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 67092b5...3b3dbdc. Read the comment docs.


feat(coverage) : improve accuracy for evaluating permission condition

Fixes: #837
/claim #837
@tolgaOzen - Please find below the initial findings and approach, will update as i progress.
Action Items
[ ] -  Review the current implementation of the 'Coverage' command - InProgress
As per the current implementation, only the action names or permission names inside the entity are added to Coverage.Assertions. https://github.com/Permify/permify/blob/60c8fb3ec00f53fc9ebe53d7e056a36c45cf6e52/pkg/development/coverage/coverage.go#L241
This is compared with scenarios->checks->assertions in the input file.
Whereas the conditions inside the action/permission are not accounted, hence the coverage for assertions is inaccurate.
[ ] - Redesign the command to incorporate detailed assessments of each condition part - InProgress
The starting point for the approach is - the permission's or action's conditions are already captured into permission object, inside the Child attribute while parsing the input file. This permission.Child attribute is made use of to capture the assertions from conditions. Refer to the current commit.
Consider this as initial approach to start with. Will refine, redesign and update as I progress with the analysis and considering different scenarios.
Already there is some improvement in the accuracy (with current updates) for the following  sample input file:
schema.yaml.txt
Before updates :

coverage assertions percentage is 50%
After updates from current commit:

The coverage assertions percentage is now 37% , as more uncovered assertions are captured from action/permission conditions.
[ ] - Implement tests and quality checks for the revised 'Coverage' command - To be Started
[ ] - Update documentation to reflect the new standards and procedures - To be Started
You can assign the issue #837 to me.

Proposed design for incorporating detailed assessments of each condition part :
Considering the following condition as provided in the issue -
permission view = system.view or ((is_public or (is_partner and partner) or (viewer or company.maintain or organization.maintain or team.view)) not denied)

Splitting the condition parts

Coverage across all scenarios

Display format

Proposed design for incorporating detailed assessments of each condition part :
Considering the following condition as provided in the issue -
permission view = system.view or ((is_public or (is_partner and partner) or (viewer or company.maintain or organization.maintain or team.view)) not denied)
Following are the design aspects.

Calculate coverage across all scenarios :
The new “condition_parts coverage” would be calculated similar to how "relationships coverage" is calculated currently.
Not w.r.t each scenario. Because a single scenario cannot cover all parts of the condition.

Example:
    permission view = follower or non_follower

In above example, a user cannot be both follower and non_follower.

So, if there are 2 scenarios - one with user as follower and other as non_follower. Then we can say collectively that the above condition is 100% covered.

Splitting the condition parts :
The way we split a condition into parts is important.
Suppose if we just split straight away into individual parts  - system.view, is_public, is_partner, partner,…..
Consider there is one scenario that has is_partner as true (among other relations except partner) and if there is another scenario that has partner as true (but not is_partner).
We may conclude that coverage is 100%.

But actually this is not correct. Because among the input scenarios there is no scenario provided where both is_partner and partner are true and this condition path (is_partner and partner) is never covered.
So we need to have logical splits, i.e. we need to split into each nested condition with union (and) as one unit.
So the correct splits would be -
system.view
is_public not denied
(is_partner and partner) not denied
(viewer or company.maintain or organization.maintain or team.view) not denied
And then check whether each split is covered by atleast one of the several input scenarios.

Display format :

Envisioning the output format for coverage command after implementing condition_parts coverage -

@tolgaOzen , Kindly share your thoughts on this.

Hi @vijayraghav-io, I understand your aspects, but can you clarify with more different examples?

Sure @tolgaOzen , will come up with more examples.

Hi @tolgaOzen ,
I am trying provide a simple example as below, but found that i am not able to provide input for an attribute `loggedIn' inside .yaml file. I referred the docs and existing test cases.
Can you please show how we can input values to attributes in the schema.yam file (that will be input for the coverage command)
Example:
schema: >-
 entity user {}

 entity resource {
    relation viewer  @user
    relation manager @user

    attribute loggedIn boolean

    action edit = manager
    action view = (viewer or manager) and loggedIn
 }

relationships:

“resource:1#viewer@user:1"

scenarios:

name: "scenario 1"
description: "test viewer”
checks:

entity: “resource:1”
subject: "user:1"
assertions:
view: true

For above check, how can i provide the value for loggedIn attribute as true.

Hi @vijayraghav-io, there is a field named 'attributes' in the YAML file, you can retrieve it from there. However, what I don't understand is, does it become 'cover' only when it is set to true?


Spout error CS0234

Hi Keijiro!
I want to use Spout into Phantom, so I replaced Klak with the newer version, and added Spout. However, I got this error:
Assets/Klak/Spout/Internal/PluginEntry.cs: error CS0234: The type or namespace name 'IntPtr' doesnt exist in the namespace 'Klak.System'. Are you missing an assembly reference?
I don't know how to fix this :-/

Did you modified the source code?
I hardly think that this error could happen without modification.

Clean copy of Phantom, then clean copy of Klak over the one already is in, and clean copy of Spout. That's all.

Okay. It reproduced in my side too.
It's caused by the old version of Klak. You have to remove Assets/Klak/System directory to resolve it.
I'm closing this ticket. Please feel free to reopen if you still have issues.


GOV.UK Radios (Checkboxes) Items do not support Hints

Raised by Alex T.
It's currently it is not possible to use hints in items as seen in https://www.gov.uk/log-in-file-self-assessment-tax-return/sign-in/prove-identity
Current work around:
{{ govukRadios({
  "idPrefix": "example",
  "name": "example",
  "fieldset": {
    "legend": {
      "text": "Create a new organisation"
    }
  },
  "items": [
    {
      "value": "Create new organisation",
      "html": "test hint here.",
    },
    {
      "value": "Merge organisations",
      "html": "test hint here",
      "checked": true
    }
  ]
}) }}

for this it would be good to understand what the relationship is between the label text and hint text in the way they're read with screenreaders

This is a relatively common pattern, so we should support it. In the self assessment example above the hint text is not part of the label, it's in a span immediately following it. However, on Voiceover at least it does get read out when you navigate between the radio options, which is probably what we want to happen.

In the self assessment example above the hint text is not part of the label, it's in a span immediately following it. However, on Voiceover at least it does get read out when you navigate between the radio options, which is probably what we want to happen.

This works because the span is associated with the input using aria-describedby, which mirrors how we now associate hints and error messages with inputs generally.


Validator improvements and Rake task

Fixes #822

Coverage remained the same at 100.0% when pulling 43ba02702da7e9bb580d658d577779b25ebc705e on validate-rake into 559eea93e0875ee1ea1a2ab57a49d66ec8ff621d on master.


dependency problems

Hi,
I am trying to install PIE (python version 3.10) but I run into dependency problems:
Is there a workaround?
(venv) jesse@svprre10:~/workspace/pie$ pip install nlp-pie==0.3.8
Collecting nlp-pie==0.3.8
Using cached nlp_pie-0.3.8-py2.py3-none-any.whl (84 kB)
Collecting click<8.0,>=7.0
Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
ERROR: Could not find a version that satisfies the requirement torch<=1.7.1,>=1.3.1 (from nlp-pie) (from versions: 1.11.0, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 2.0.0, 2.0.1)
ERROR: No matching distribution found for torch<=1.7.1,>=1.3.1

Hey :)
I don't know if @emanjavacas still maintains Pie.
In case he does not, I have tried to keep my own version in check with various dependencies ( https://github.com/lascivaroma/PaPie ). Note that it might have some dropped functionalities (as I was not using them and their maintenance was a hassle).

Hey both,
Indeed, I don't have the capacities to keep pie updated to the last pytorch versions. However, I just pip installed it (pip install nlp-pie) and ran the tests and it seems to run fine (using pytorch 2.0). As far as I understand, the update to 2.0 didn't have backwards incompatibilities, so the package should run just fine.
If you don't get to work around this issues " ERROR: No matching distribution found for torch<=1.7.1,>=1.3.1", you could git clone the repo and install it locally.
Let me know if this helps.
Thanks @PonteIneptique for jumping in.

Thanks both!!
I can install @PonteIneptique's version. I run into cuda issues later on, but that is most likely a problem of my local machine.

Hey :) If you do not have cuda, run pip install papie --extra-index-url https://download.pytorch.org/whl/cpu, should fix things ;)

Thanks again! (My machine does have cuda, but it magically gets mixed up on system updates from time to time)


Controlled value is undefined when adding an "onChange" callback to a "dropdown" option

Hello!
I'm experiencing an issue with a dropdown control when I try to add a callback to it:
const { dogFood } = useControls({
  dogFood: {
    options: {
      "meat": "yay",
      "chocolate": "nay",
    },
    onChange: console.log,
  },
  something: True,
});

// "dogFood" is undefined

The callback is called at the correct time, but the value is not usable.
typescript compilation fails with the following error:
Property dogFood does not exist on type { something: boolean; }"

leva: 0.9.35
tsc: 5.4.5
By the way, thanks for the lib! It's really pretty and helpful.

This is a duplicate of #447.
I have been able to hack together a fix for anyone running into this issue:
import { useControls } from "leva";
import { Schema } from "leva/dist/declarations/src/types";

type SanitizeSchema<T> = {
  [K in keyof T]: T[K] extends object ? Omit<T[K], "onChange"> : T[K];
};

const useHook = () => {
  const init = {
    dogFood: {
      options: {
        meat: "yay",
        chocolate: "nay",
      },
      onChange: console.log,
    },
    something: true,
  };

  const { dogFood } = useControls<Schema, Schema, SanitizeSchema<typeof init>>(init);

  return dogFood;
};

I would like to investigate a permanent fix but I'm not a TypeScript expert and have not had the time to investigate.
Perhaps one of the maintainers knows whom I could speak to?


Add spring milestone repositories to sample pom.xml-s

With a clean start (no previous spring-cloud-gcp artifacts built), the starter can't be built in isolation, without running mvn clean install first on the whole big spring-cloud-gcp project.
We can either add the mvn clean install step as a first step everywhere, or include
<repositories>
  <repository>
   <id>spring-milestones</id>
   <name>Spring Milestones</name>
   <url>https://repo.spring.io/milestone</url>
  </repository>
</repositories>

to all the poms.
Error for the spring-cloud-gcp-data-jpa-sample
[ERROR] 'dependencies.dependency.version' for org.springframework.cloud:spring-cloud-gcp-starter-sql-mysql:jar is missing. @ org.springframework.cloud:spring-cloud-gcp-data-jpa-sample:[unknown-version], /Users/balintp/dev/proj/spring-cloud-gcp/spring-cloud-gcp-samples/spring-cloud-gcp-data-jpa-sample/pom.xml, line 21, column 15

I can't reproduce this anymore. Closing the issue.


Difference between text size of anko libs and textsize xml layout

I have use textSize of Anko
textSize = sp(16).toFloat()
But. textSize very Big.
textSize 16sp of xml Layout is smaller textSize of Anko
Difference between text size of anko libs and textsize xml layout ??

This is because TextView's setTextSize already interprets value in scaled pixel units. Instead, you should be using just 16f. This is equivalent to 16sp in xml.

This is because TextView's setTextSize already interprets value in scaled pixel units. Instead, you should be using just 16f. This is equivalent to 16sp in xml.

thanks you.

Solved my problem! Great!

thanks


restore no-std support

no-std support for plonky2 and starky was added by #806, but nobody noticed some breaking changes later on.
This PR restores no-std support, and introduces wasm32-unknown-unknown as an additional target for tests, to avoid making the same mistake again.
https://github.com/0xPolygonZero/plonky2/pull/907#issuecomment-1462939316

The CI isn't running with your changes: https://github.com/0xPolygonZero/plonky2/actions/runs/6782203615.
You may want to try it out locally against your own fork first.

In addition, cargo test cannot be run against a no-std target as it also runs plonky2_evm tests which use std.

This doesn't seem to be doing what's expected still.

If we were running against a wasm32-unknown-unknown target, the current would fail as we didn't deactivate the default features which include std.
I'd recommend you to update the CI until you have a failing job for wasm complaining about std, and only then remove the default features to have it go green.

Thanks @muraca! Tiny final nit, the job is called "Test suite" but we do not run the tests against wasm, so a different name would be preferable.

@Nashtare sorry for the mess.
Looking back at it, I think it would be better to have separate jobs instead of the matrix, since we only have to perform 2 additional checks.
What do you think?

@muraca Yeah, having a dedicated job for wasm32 seems better :)

Hmm, the current command run in the CI job for wasm, cargo check --manifest-path plonky2/Cargo.toml --target wasm32-unknown-unknown doesn't trigger any error on main, although it should if not deactivating the default-features. I'll need to investigate it a bit more.

@muraca Could you add --no-default-features to the no-std check? This is what's currently missing to enforce that we can properly compile in no-std environments (it fails on main without your fixes). Thanks!


Add optional area deviation plot

With the addition of area deviation error analysis capabilities this is plotted when the initial profile is divined with error

Needs changes according to changes in pyroll-project/pyroll-core#83


[2|CBH & HR - UGC & Banned words Dashboard] UGC tables

 🍰 Appetite 🍰

Property
Value

Points
25 points

Difficulty rating
★★★★★

Maximum Assignees
6 person

🤔 Problem Statement

Create a table to show the UGC list. The list will include users' public notes and shared articles. Add pagination to this table. The table will include these fields:

A checkbox field that allows the user to perform action on more than one row at a time.
Note (public notes/user thoughts on an article)
Linked content (where the public note point to/link to the article that the user shared)
Author (Note's author username)
Organization (the organization to which the user belongs to)
Approval & Report

The list will contain UGC  from all organizations if the user is logged in as CBH. If the user is logged in as HR of a company, then only display the UGC from that organization.
If the user is logged in as CBH, for a new post or a post that has only been approved by HR, show the 'approve' button and the 'deny' button in the Approval & Report column. If the UGC is approved by CBH, then show the 'remove' button and a 'report' field to the CBH to remove the post or to see the report count.
If the user is logged in as HR, for a new post or a post that has only been approved by CBH, also show the 'approve' button and the 'deny' button in the Approval & Report column.  If the UGC has been approved by HR, then show the 'remove' button and a 'report' field to HR to remove the post or to see the report count.
For your information: if CBH or HR approves, the post will be on the feed, but if another party denies it after that, the post will be taken down no matter what. (You don't have to do anything here. This is just an explanation of why we have separate approve and deny for CBH and HR)
For every cell in the table, if the content is more than how much a cell can show, when the user clicks on the cell, a pop-up will appear and show the rest of the content.
When the user clicks on the 'Remove' button, a confirmation pop-up will appear. When the user clicks on the 'Deny' button, a pop-up will appear and prompt the user for a reason.
The 'Export selected' button will export a CSV of all chosen Notes. The 'Delete selected' button will delete all of the chosen notes.
🧪 Required Tests
Mainly screenshot tests, and some integration tests for Supabase and Payload CMS.
Review the Code & Testing Guidelines for more info.
⚠ Careful about:
No major rabbit holes identified. But always be careful with scope creep!
🤖 Technologies focused on in this feature

React
Tailwind
Shadcn
Payload CMS
NextJS
Supabase/PostGresQL
AppFlowy

Im gonna work on this. @puma

Can I be assigned here @hzz4343

Hey @PJ-Bernal! Please message me and let me know whether you were able to complete this issue and we will ensure you receive the assigned points. We're temporarily locking it to focus on the priority issues identified on this Discord announcement.

@khnatiuk @PJ-Bernal @ringram58
I have assigned points and merged the pull request. JSON appSmith file has been saved.


grawlix?

Can I use grawlix replacements?

I think this is possible.
The constructor for the word filter accepts placeHolder as one of the options. It defaults to *.  But what we are doing is simply passing that option to string.replace.
See
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace
Since string.replace can also take a function as the replacement argument, so you could do something like so
new Filter({
  placeHolder: function() {
    var grawlix = '#!@$';
    return grawlix[Math.floor(Math.random() * grawlix.length)];
  }
});

I'll update the docs to show this possibility


Cancel-backfill not progressing

Hi
I haven't used the tool before but was pointed here from the mailing list[1]. Not sure if I am just misunderstanding something but cancel-backfill seems to be unable to do any more after taking misplaced down from ~29% to 12.8% the first time I ran it. Running it repeatedly doesn't remove any more of the PGs in backfill or backfill_wait.
Command run:
./pgremapper cancel-backfill --yes --verbose |tee /tmp/pgremapper.out
Current status:
`
[root@lazy ~]# ceph -s
cluster:
id:     8ee2d228-ed21-4580-8bbf-0649f229e21d
health: HEALTH_WARN
1 failed cephadm daemon(s)
nobackfill,norecover,noscrub,nodeep-scrub flag(s) set
Degraded data redundancy: 1/3169196434 objects degraded (0.000%), 1 pg degraded, 3 pgs undersized
services:
mon: 5 daemons, quorum lazy,jolly,happy,dopey,sleepy (age 3d)
mgr: jolly.tpgixt(active, since 3d), standbys: dopey.lxajvk, lazy.xuhetq
mds: 1/1 daemons up, 2 standby
osd: 540 osds: 539 up (since 21h), 539 in (since 5d); 2321 remapped pgs
flags nobackfill,norecover,noscrub,nodeep-scrub
data:
volumes: 1/1 healthy
pools:   15 pools, 10849 pgs
objects: 547.16M objects, 1.1 PiB
usage:   1.9 PiB used, 2.3 PiB / 4.2 PiB avail
pgs:     1/3169196434 objects degraded (0.000%)
405492775/3169196434 objects misplaced (12.795%)
8525 active+clean
2242 active+remapped+backfill_wait
76   active+remapped+backfilling
3    active+recovering
2    active+recovering+undersized+remapped
1    active+recovery_wait+undersized+degraded+remapped
io:
client:   14 MiB/s rd, 82 MiB/s wr, 817 op/s rd, 165 op/s wr
`
ceph.pg.dump.pgs_brief.txt
ceph.osd.tree.txt
ceph.osd.dump.txt
pgremapper.out.txt
[1]<EMAIL_ADDRESS>

On second thought it might just be working as intended. Some of the changes I made to crush would require different placement and current placement would be invalid?

Added 4 hosts to a datacenter bucket (current placement ok)
Moved one host from one datacenter bucket to another (current placement invalid?)
Changed the failure domain for an EC pool from host to datacenter (current placement invalid?)

Hey @torkilt, yeah, based on that sequence I would expect you're hitting a condition where Ceph will not allow upmaps to override CRUSH placement

Thanks for the confirmation, and the tool!


Issue parsing eslint output

Using snazzy to parse eslint output does not seem to honor eslint exit code:
$ eslint .
...
✖ 3 problems (3 errors, 0 warnings)
$ echo $?
1
$ eslint . | snazzy
...
✖ 3 problems (3 errors, 0 warnings)
$ echo $?
0

Using:

<EMAIL_ADDRESS><EMAIL_ADDRESS>

That's because snazzy is designed to convert from "compact" format, not from "stylish" format. By default, eslint outputs "stylish".
Why do you even need snazzy if you're using eslint directly? It doesn't look like it's changing the output at all.

@feross Could you please re-open this issue.
We use snazzy with standardx, the goal is to add warnings on max-len, complexity, no-console, ...

@feross Could you please re-open this issue.
We use snazzy with standardx, the goal is to add warnings on max-len, complexity, no-console, ...

At the end we used directly eslint extending eslint-config-standard with our addition rules and with eslint -f stylish . command.
Sorry for the bothering...

At the end we used directly eslint extending eslint-config-standard with our addition rules and with eslint -f stylish . command.
Sorry for the bothering...


[Event Hubs] Add missing metadata to PartitionContext

Summary
The focus of these changes is to add metadata about the associated Event Hub instance to the PartitionContext used by the consumer and processor clients.  These fields are present in Java, Python, and JavaScript but were missing in .NET; these changes normalize across languages.
References and Related

[FEATURE REQ] Restore metadata in track 2 PartitionData Class (#26629)
Java: PartitionContext
Python: PartitionContext
JS: PartitionContext

API changes have been detected in Azure.Messaging.EventHubs. You can review API changes here
API changes
+         public static PartitionContext PartitionContext(string fullyQualifiedNamespace, string eventHubName, string consumerGroup, string partitionId, LastEnqueuedEventProperties lastEnqueuedEventProperties = default);
+         [EditorBrowsable(EditorBrowsableState.Never)]
+         protected PartitionContext(string fullyQualifiedNamespace, string eventHubName, string consumerGroup, string partitionId);
+         [EditorBrowsable(EditorBrowsableState.Never)]
+         public string ConsumerGroup { get; }
+         public string EventHubName { get; }
+         public string FullyQualifiedNamespace { get; }


Version 1.0.465 removed addUserAgentInfo from ServiceClient which is a breaking change

We generated code using 1.0.455.  This generated code is crashing when our customers install botbuilder, as pulls in the 1.0.465.
Looks like this method was removed with the following commit:
https://github.com/Azure/ms-rest-js/commit/523b891b4de679542f725a96b8c896432b44ecc4#diff-3b5c4e8e88ea4820a88a9584a654fe8cL144
Expected behavior
Following semver rules, patch versions should not be breaking.

Hi sgellock.
Yes, this was a mistake on our side. We were under the impression that the addUserAgentInfo() method was only called internally.
If you are interested in changing the user agent that a ServiceClient adds to outgoing requests, then I recommend using the ServiceClientOptions.userAgent property.
If you don't care about changing the user agent header, then I'd recommend running "autorest --reset" and then trying to regenerate. I just updated the latest version of our TypeScript generator and that should solve the problem.

Related to https://github.com/Azure/autorest.typescript/issues/386.

We pinned to an earlier version that didn’t have the breaking change and published new versions of our bits.
We believe your suggestion will work, but we haven’t tried it.
From: Dan Schulte<EMAIL_ADDRESS>Reply-To: Azure/ms-rest-js<EMAIL_ADDRESS>Date: Monday, November 26, 2018 at 10:25 AM
To: Azure/ms-rest-js<EMAIL_ADDRESS>Cc: Scott Gellock<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [Azure/ms-rest-js] Version 1.0.465 removed addUserAgentInfo from ServiceClient which is a breaking change (#276)
@sgellockhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsgellock&data=02|01|scott.gellock%40microsoft.com|1516c38754a040daa26e08d653cc9551|72f988bf86f141af91ab2d7cd011db47|1|0|636788535477531127&sdata=PATdgCItwAfS4aTqPOdn10g5lK3RAyxG6Qjkwy9Xq8o%3D&reserved=0 Did my previous comment solve the problem for you?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fms-rest-js%2Fissues%2F276%23issuecomment-441744970&data=02|01|scott.gellock%40microsoft.com|1516c38754a040daa26e08d653cc9551|72f988bf86f141af91ab2d7cd011db47|1|0|636788535477541136&sdata=JreaIBDZVXA7nT0Bx2LPO1Q0Zqr3LFlhlq0%2FXe7BgYc%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAsxdxg95w1RZTy7XO9oidiUG9NvP-NDks5uzDIpgaJpZM4YnMLY&data=02|01|scott.gellock%40microsoft.com|1516c38754a040daa26e08d653cc9551|72f988bf86f141af91ab2d7cd011db47|1|0|636788535477541136&sdata=jlZoCJ%2FseZ8ePbd%2BQ9PUP%2BhjklSfXYFQ%2FYYw0gRohYU%3D&reserved=0.

I'm going to close this issue. Feel free to reopen if my proposed solution doesn't work.


Improve vertical alignment of icon buttons within tab

Added styling to 'icon buttons' in horizontal tab so that they remain vertically aligned whether or not active (previously whichever icon was 'active' was aligned so that the 'underline' bottom border aligned with bottom of 'inactive' icons).
This has caused a secondary quirk whereby the containing 'tab-bar' momentarily re-sizes when an icon-button is clicked, which I've not been able to overcome.

Add functionality to game editing to prevent a team playing both home and away (#86)

Also includes a test for CreateLeagueIcon

Responded to changes requested!

@domwakeling Thanks, a couple more changes requested. All our jsx attributes should be in double quotes(help.jsx file). And in the mapping on teams in the form template you don't need to do teams.map() : ''. You can just do teams.map() and if it's an empty array nothing will display. Thanks!

Updated as Slack discussion!


chore: use /usr/bin/env to locate bash (best practice)

#!/usr/bin/env searches PATH for bash, and bash is not always in /bin, particularly on non-Linux systems. For example, on  OpenBSD systems, it's in /usr/local/bin, since it was installed as an optional package.
If you are absolutely sure bash is in /bin and will always be, there's no harm in putting it directly in your shebang—but I'd recommend against it because scripts and programs all have lives beyond what we initially believe they will have.
Particularly, this breaks using the scripts on NixOS, which is not compliant with the Filesystem Hierarchy Standard (FHS) convention. This PR changes all scripts to be independent of this convention.
https://stackoverflow.com/questions/21612980/why-is-usr-bin-env-bash-superior-to-bin-bash

Great suggestion and I have good news for you!
The scripts, etc (among other things) have been re-architected in the next release branch (wisng). This is a much simpler change there :).

Thank you! Keep up the good work!


Make u3m a valid file with physics information only

If u3m can hold physical measurements the user must be able to store a u3m file.
Issue raised in discussions with Assyst and Browzwear.

fixed for U3M 1.1

fixed for U3M 1.1


fix(diff): swagger diff is not working for certain cases

Fixes #2929 and some more fixes

This PR needs to be updated once https://github.com/go-swagger/go-swagger/pull/2933 is merged.

This is ready for merging now. I have updated changes in default values to use warning level.


increase storage retention to 30days

Signed-off-by: tanvigour<EMAIL_ADDRESS>Description
Change prometheus storage retention time to a month so that we get data for entire month for longhaul availability
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

@halspang : please review when you get a chance

@halspang : please review


master

here's the first version of the pattern lab workshopper
future:
-- display the 'example.js' file for each problem to correctly show our solutions, differs from the console output tests in solution.js. currently doesn't do that well, but the files are included anyways
-- update some of the lesson text.
-- more lessons/separate workshopper for more advanced PL concepts?

WOW... will try to start working through this AMAZING pull request and provide feedback!

@illepic / @EvanLovely  When you get a chance can you guys take a pass at the Pattern Lab perspective.  @tjheffner did @illepic talk to you about updating the workshopper-jlord and abstract away the git-it stuff?

He did, and we did. It happened in this commit: https://github.com/tjheffner/pattern-lab-workshop/commit/44fe9c8347c136ddc1e0c81badc01c2d97459f60
and this is the "new" workshopper with updated output changes: https://github.com/tjheffner/workshopper/tree/master
Currently workshopper-jlord points to my updated one, if/until jlord merges the updates
Now anyone using the "new" workshopper framework can spit out any strings they want when a user selects/passes a lesson, not just things related to git-it

@EvanLovely @illepic what is the status of this PR?  would love to get @tjheffner any other feedback and move this from a glimmer in my / bloom's eye to a real released workshop that I can add to nodeschool!

BUMP!! :)

I don't have enough bandwidth to review stuff on this project as my attention's on PL Starter & all the P2 Yeoman generators for Dev Tools; sorry! I'd be happy to look over @illepic's shoulder as he reviews and give my 2¢ though!
Honestly, given the young nature of this codebase, I don't see why we couldn't merge @tjheffner's stuff in as it's obvious he's given it a lot of thought and work; nice job Tanner!

I am going to merge this in @illepic if you want this updated please create an issue and let us know I am going to create a pull request to add this to nodeschool's page.

@robbiethegeek ++


Readme: Fix build instructions for Qemu

Use make to be consistent with the gitlab pipeline.
Without the objcpy after building the loader, qemu fails at startup.
I initially wanted to create the pull request for the devel branch, but the Readme at the devel branch seems to be outdated.

You are right.


Update MS VS ReDist DLLs distributed to "latest stable"

Recent issue highlighted we currently ship an old UCRT DLL that can cause in-compatibilities with certain API calls with the latest Win SDK.
We should move up to a more recent "fixed" stable set, we should also specify the complete set so we are consistent:
Currently this, with the additional DLLs loaded from the installed VS layout:
--with-ucrt-dll-dir='C:/progra~2/wi3cf2~1/10/Redist/10.0.22621.0/ucrt/DLLs/x64'

Should update to use the complete set:
--with-msvcr-dll: points to the VS C runtime main DLL
--with-vcruntime-1-dll: points to the VS C runtime extra DLL
--with-msvcp-dll: points to the VS C++ runtime main DLL
--with-ucrt-dll-dir: point to the directory for the Universal C runtime DLLs (all of them)

Possibly update to latest:

Latest versions from Microsoft: https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170

The full historic list of MSVC++ version are here: https://en.wikipedia.org/wiki/Microsoft_Visual_C++
The current is:
Visual Studio 2022 version 17.10.1 	17.10.1 	2022 	1940 	194033811 	14.40.33811

The latest MSVC Redist files can be downloaded as installer executable file here: https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
When vc_redist.x64.exe will install the redist DLLs into the c:\Windows\System32 directory as long as there isn't a later version of the redist in that folder already.
To package a set of redist DLL's necessary to package with Temurin, I would suggest the following process:

Determine the set of redist files that are packaged with Temurin
On a suitable Windows VM ensure the c:\Windows\System32 directory has all the redist DLLs removed
Then download the required redist version from https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
Run the redist.exe, which will extract the DLLs into the c:\Windows\System32 directory
Zip the necessary redist DLLs into a zip file and store...

For building Temurin at ci.adoptium.net:

Extract the "stored" redist files into a suitable local directory
Configure the openjdk make with necessary configure options pointing at the extract redist local directory:

--with-msvcr-dll: points to the VS C runtime main DLL
--with-vcruntime-1-dll: points to the VS C runtime extra DLL
--with-msvcp-dll: points to the VS C++ runtime main DLL
--with-ucrt-dll-dir: point to the directory for the Universal C runtime DLLs (all of them)

@sxa @steelhead31 Your thoughts please, if we manually package the latest "redist" DLLs, where could we put them for our build nodes?

@andrew-m-leonard I think we could, package up the zip files , and have them deployed by Ansible into a specifie folder on the windows build machines, we could approach this in  a way to allow multiple versions, or alternatively just a single version... something along the lines of..
c:\temurin-msvc-redist\version1.7.0
c:\temurin-msvc-redist\version2.7.0
c:\temurin-msvc-redist\latest
..etc..etc..
Could then define the locations for the parameters listed per version as required ?

The Microsoft Redist "update" package installer seems to only install the vcruntime*, msvcp*.dll and ucrtbase.dll, all of which are put into c:\Windows\System32 when you run say the vc_Redist_x64.exe. It does not install the api-ms-crt*.dll's which seem only present in the Windows Kit/Redist/ucrt/DLLs folder.
Trying to accurately create our own Redist.zip packaging x64,arm64,x86 and all the necessary dlls is going to be tricky, and probably not the right thing to do.
We should probably instead create a "Windows DevKit" for VS2022 using: https://github.com/openjdk/jdk/blob/master/make/devkit/createWindowsDevkit.sh
This requires the "required" versions of VS2022 and Windows Kit SDK to be installed on your machine, it will then create a "DevKit" package from them.
We should thus look to update to the latest(possibly LTSC) MS VS2022 update from:  https://learn.microsoft.com/en-us/visualstudio/releases/2022/release-history
and create a "Windows DevKit" from it.

A Windows DevKit would require further investigation and testing.
To alleviate the user issues found by https://github.com/adoptium/temurin-build/issues/3887
I am going to try a build test using the latest VS2022 17.10 Redist for just the C and C++ "runtime" DLLs:
--with-msvcr-dll: points to the VS C runtime main DLL
--with-vcruntime-1-dll: points to the VS C runtime extra DLL
--with-msvcp-dll: points to the VS C++ runtime main DLL

Hi Andrew
In order to get the UCRT dll to redistribute with the jdk, it seems we have to get them from the Windows SDK:
https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/
The "easiest" to extract a complete collection for these DLLs, is to proceed as follow:

Download an ISO image of the latest SDK from this location: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
Expand the ISO image to disk
Locate the installation bundle "Universal CRT Redistributable-x86_en-us.msi" and de-compile it using the "dark.exe" MSI de-compiler that comes with the Wix Toolkit. I.e.

c:\Tools\wix311\bin\dark.exe -v -o c:\sdk\out\ucrt.wxs  "c:\sdk\26100.1742.240904-1906.ge_release_svc_prod1_WindowsSDK\Installers\Universal CRT Redistributable-x86_en-us.msi" -x c:\sdk\out

The DLLs binaries will have been extracted in "C:\sdk\out\File", but the files are named using the their msi IDs
The Wix sources generated by the de-compiler in c:\sdk\out\ucrt.wxs  contains the correspondence between each ID and the dll names, so it should be easy to write a script that renames them.

While this isn't exactly "easy" (I said "easiest"...), doing things this way presents several advantages over having to run the installer on an (hopefully) clean machine and fish out what we believe are the freshly installed files, mainly that I find it easier to trust we have the extracted the files.

By the way, you can use dark.exe in a similar way to extract the legacy VC runtime DLLs from the VC_redist bundle.

Hi Andrew
In order to get the UCRT dll to redistribute with the jdk, it seems we have to get them from the Windows SDK: https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/#distributing-software-that-uses-the-universal-crt (see point 6: "App-local deployment of the Universal CRT is supported")
The "easiest" way to extract a complete collection for these DLLs, is to proceed as follow:
1. Download an ISO image of the latest SDK from this location: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/

2. Expand the ISO image to disk

3. Locate the installation bundle "Universal CRT Redistributable-x86_en-us.msi" and de-compile it using the "dark.exe" MSI de-compiler that comes with the Wix Toolkit. I.e.

c:\Tools\wix311\bin\dark.exe -v -o c:\sdk\out\ucrt.wxs  "c:\sdk\26100.1742.240904-1906.ge_release_svc_prod1_WindowsSDK\Installers\Universal CRT Redistributable-x86_en-us.msi" -x c:\sdk\out

4. The DLLs binaries will have been extracted in "C:\sdk\out\File", but the files are named using the their msi IDs

5. The Wix sources generated by the de-compiler in c:\sdk\out\ucrt.wxs  contains the correspondence between each ID and the dll names, so it should be easy to write a script that renames them.

~While this isn't exactly "easy" (I said "easiest"...), doing things this way presents several advantages over having to run the installer on an (hopefully) clean machine and fish out what we believe are the freshly installed files, mainly that I find it easier to trust we have the extracted the right files. It might also be simpler to automate the process.~
EDIT: Actually, unlike the VC_redist bundle for the legacy C runtime which dumps everything in system21, running "Universal CRT Redistributable-x86_en-us.msi" installed the DLL in an easily identifiable location in C:\Program Files (x86)\Windows Kits\10\Redist, and it does not overwrite previous versions. So afterall, it might be simpler to just get the files directly from there and avoid having to deal with dark magic.

Thanks @fthevenet, for the methods. For the linked issue we believe we just need to update the C runtime DLLs only. so I am running a test with just those at the moment

To extract the required Redist DLLs, the safest and easiest way is to locally install an "all architecture" installation as follows:

Determine the "Build Tools" version install bootstrapper required and download from here: https://learn.microsoft.com/en-us/visualstudio/releases/2022/release-history#fixed-version-bootstrappers
Determine the required version of the "Windows SDK" from here: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/

Note the version, eg."10.0.26100"

Install locally on a Windows x64 VM using the following command:

./vs_BuildTools.exe --passive --norestart --wait --arch all --add "Microsoft.VisualStudio.Workload.NativeDesktop;includeRecommended;includeOptional" --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.VC.Tools.ARM64 --add Microsoft.VisualStudio.Component.VC.ATL.ARM64 --add Microsoft.VisualStudio.Component.VC.MFC.ARM64

Run the installer manually again to install the required "Windows 11 SDK":

./vs_BuildTools.exe

Select "Individual components"
Find the required "Windows 11 SDK (REQUIRED VERSION from step 2)"

I have created a Redist update using the above method, for the following versions:

Visual Studio 2022 - LTSC 17.10   :  17.10.7   :  September 10, 2024     Build Tools     :  17.10.35230.96
Including "Windows 11 SDK (10.0.26100)"

Test builds running:
x64:
jdk8u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-temurin/514/
jdk11u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-windows-x64-temurin/404/
jdk17u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk17u/job/jdk17u-windows-x64-temurin/531/
jdk21u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-windows-x64-temurin/165/
jdk23u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk23u/job/jdk23u-windows-x64-temurin/1/
x86:
jdk8u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x86-32-temurin/491/
jdk11u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-windows-x86-32-temurin/375/
jdk17u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk17u/job/jdk17u-windows-x86-32-temurin/363/
aarch64:
jdk21u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-windows-aarch64-temurin/3/
jdk23u https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk23u/job/jdk23u-windows-aarch64-temurin/1/

Re-running these 3 failures with a longer timeout of EXTRA_OPTIONS=-Dtest.timeout.factor=20.0:
TEST: java/lang/Thread/virtual/stress/PingPong.java#sq
TEST: java/lang/Thread/virtual/stress/PingPong.java#ltq
TEST: java/util/concurrent/BlockingQueue/MultipleProducersSingleConsumerLoops.java

https://ci.adoptium.net/job/Grinder/11134/

Mauve related tests that nearly always fail:
09:46:25  	MiniMix_5m
09:46:25  	MiniMix_10m
09:46:25  	MiniMix_aot_5m_0
09:45:30  	MauveSingleThrdLoad_HS_5m
09:45:30  	MauveSingleInvocLoad_HS_5m
09:45:30  	MauveMultiThrdLoad_5m

Newly added hotspot_tier1_runtime_0 and hotspot_tier1_serviceability_0 (https://github.com/adoptium/aqa-tests/pull/5648), that always fail:
hotspot_tier1_runtime:
TEST: runtime/CDSCompressedKPtrs/XShareAuto.java
TEST: runtime/cds/TestCDSVMCrash.java
TEST: runtime/cds/CheckDefaultArchiveFile.java
TEST: runtime/ErrorHandling/UncaughtNativeExceptionTest.java
TEST: runtime/ErrorHandling/MachCodeFramesInErrorFile.java
TEST: runtime/memory/ReadFromNoaccessArea.java
TEST: runtime/jni/nativeStack/TestNativeStack.java

hotspot_tier1_serviceability:
TEST: serviceability/sa/ClhsdbFindPC.java#no-xcomp-core
TEST: serviceability/sa/ClhsdbDumpheap.java
TEST: serviceability/sa/ClhsdbFindPC.java#no-xcomp-process
TEST: serviceability/sa/ClhsdbFindPC.java#xcomp-core
TEST: serviceability/sa/ClhsdbCDSJstackPrintAll.java
TEST: serviceability/sa/ClhsdbJdis.java
TEST: serviceability/sa/ClhsdbJstack.java#id1
TEST: serviceability/sa/ClhsdbPmap.java#core
TEST: serviceability/sa/ClhsdbPrintAs.java
TEST: serviceability/sa/ClhsdbPstack.java#core
TEST: serviceability/sa/ClhsdbSource.java
TEST: serviceability/sa/ClhsdbThreadContext.java
TEST: serviceability/sa/ClhsdbWhere.java
TEST: serviceability/sa/TestJhsdbJstackLock.java
TEST: serviceability/sa/TestClhsdbJstackLock.java
TEST: serviceability/jvmti/RedefineClasses/RedefineSharedClass.java

Resolved by https://github.com/adoptium/temurin-build/pull/3981


Complete todo in StringEncoder

Motivation:
Improve performance of StringEncoder.
Modifications:
Store CharsetEncoder in StringEncoder instead of looking it up every encode.
Result:
Less technical debt.

@netty-bot test this please

I ran the verify mojo this time.. :)

I ran the verify mojo this time.. :)

@netty-bot test this please

@netty-bot test this please

Yes

Am 28.10.2019 um 17:50 schrieb Johno Crawford<EMAIL_ADDRESS>
@johnou commented on this pull request.
In codec/src/main/java/io/netty/handler/codec/string/StringEncoder.java:

@@ -68,7 +69,7 @@ public StringEncoder(Charset charset) {
if (charset == null) {
throw new NullPointerException("charset");
}

   this.charset = charset;

   this.encoder = CharsetUtil.encoder(charset);

@normanmaurer so to avoid breaking changes removing the TODO probably makes most sense then?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.


Possible new scraper for Australia

Location name
Australia all states
Source URL
https://www.health.gov.au/sites/default/files/documents/2020/04/coronavirus-covid-19-at-a-glance-coronavirus-covid-19-at-a-glance-infographic_6.pdf
Notes/comments
I saw this report being distributed through the gov coronavirus app. The pdf text can be selected so i assume it can be parsed. The pdf includes cases and deaths per state in a map plus other details this scraper is not collecting like age range, and patients in icu, etc
I proxied the app to see if there were any api's in use but nothing came out. Nothing tells how the app knows how to fetch the document. The report number looks like it increases every day, exploring the same file number but at 2020/03 doesn't show anything, so looks like is only from april
The dates are stamped at the bottom of the document, the earliest one (number 1) is from the 5th of april
@camjc

Yup agreed this is their richest data. I’ve messaged them the other day to try for a better format.
I’ve not scraped a PDF before, if you want to give it a go I’m all for it. Could assert against the existing scraper for cases count even.

I ran the first pdf scraper i found US/KS over QLD a just outputted the raw data
arn run v1.22.4
$ node src/shared/cli/index.js --location AU/QLD
⏳ Fetching scrapers...
✅ Fetched 139 scrapers, filtered to 1.
✅ All scrapers are valid!
  ⚡ Cache hit for https://www.health.gov.au/sites/default/files/documents/2020/04/coronavirus-covid-19-at-a-glance-coronavirus-covid-19-at-a-glance-infographic_6.pdf from coronadatascraper-cache/2020-4-9/28934d116adc677dc257b58cc9a63af8.pdf
[
  { page: 1, x: 2.45, y: 6.451, w: 22.203, text: '6%2C2' },
  { page: 1, x: 3.838, y: 6.451, w: 17.76, text: '03' },
  { page: 1, x: 3.237, y: 12.581, w: 19.92, text: '74' },
  { page: 1, x: 2.533, y: 24.529, w: 26.64, text: '338' },
  { page: 1, x: 4.197, y: 24.529, w: 4.437, text: '%2C' },
  { page: 1, x: 4.475, y: 24.529, w: 26.64, text: '346' },
  { page: 1, x: 37.997, y: 8.656, w: 20.16, text: '506' },
  { page: 1, x: 39.467, y: 8.656, w: 10.596, text: '(6' },
  { page: 1, x: 40.134, y: 8.656, w: 3.996, text: ')' },
  { page: 1, x: 41.456, y: 7.666, w: 6.672, text: '2' },
  { page: 1, x: 41.876, y: 7.666, w: 6.672, text: '7' },
  { page: 1, x: 41.044, y: 9.691, w: 6.672, text: '4' },
  { page: 1, x: 41.464, y: 9.691, w: 6.672, text: '2' },
  { page: 1, x: 41.884, y: 9.691, w: 6.672, text: '8' },
  { page: 1, x: 42.514, y: 9.691, w: 3.996, text: '(' },
  { page: 1, x: 42.761, y: 9.691, w: 6.672, text: '3' },
  { page: 1, x: 43.181, y: 9.691, w: 3.996, text: ')' },
  { page: 1, x: 43.159, y: 8.349, w: 20.16, text: '965' },
  { page: 1, x: 44.629, y: 8.349, w: 3.996, text: '(' },
  { page: 1, x: 44.876, y: 8.349, w: 6.672, text: '5' },
  { page: 1, x: 45.296, y: 8.349, w: 3.996, text: ')' },
  { page: 1, x: 44.021, y: 10.456, w: 50.7, text: '2%2C822 (21' },
  { page: 1, x: 47.201, y: 10.456, w: 3.996, text: ')' },
  { page: 1, x: 40.757, y: 12.386, w: 30.024, text: '1%2C241' },
  { page: 1, x: 42.851, y: 12.386, w: 3.996, text: '(' },
  { page: 1, x: 43.099, y: 12.386, w: 13.44, text: '13' },
  { page: 1, x: 43.939, y: 12.386, w: 3.996, text: ')' },
  { page: 1, x: 45.034, y: 13.548, w: 20.16, text: '111' },
  { page: 1, x: 46.504, y: 13.548, w: 10.596, text: '(3' },
  { page: 1, x: 47.171, y: 13.548, w: 3.996, text: ')' },
  { page: 1, x: 45.664, y: 11.641, w: 20.16, text: '103' },
  { page: 1, x: 47.134, y: 11.641, w: 14.664, text: '(2)' },
  { page: 1, x: 7.506, y: 6.451, w: 17.76, text: '53' },
  { page: 1, x: 11.241, y: 6.451, w: 8.874, text: '3' },
  { page: 1, x: 11.796, y: 6.451, w: 4.437, text: '%2C' },
  { page: 1, x: 12.074, y: 6.451, w: 26.64, text: '141' },
  { page: 1, x: 2.578, y: 26.817, w: 11.697, text: '1.' },
  { page: 1, x: 3.313, y: 26.817, w: 7.806, text: '8' },
  { page: 1, x: 3.8150000000000004, y: 26.817, w: 12.482, text: '%' },
  { page: 1, x: 2.99, y: 18.978, w: 26.64, text: '236' },
  { page: 1, x: 2.915, y: 29.181, w: 20.116, text: '6%2C017' },
  { page: 1, x: 5.624, y: 29.181, w: 29.057, text: '137%2C441' },
  { page: 1, x: 9.066, y: 29.174, w: 20.116, text: '3%2C243' },
  { page: 1, x: 11.924, y: 29.174, w: 24.586, text: '69%2C194' },
  { page: 1, x: 2.975, y: 30.674, w: 11.176, text: '1.7' },
  { page: 1, x: 3.672, y: 30.674, w: 7.148, text: '%' },
  { page: 1, x: 6.081, y: 30.674, w: 18.323, text: '2.1%' },
  { page: 1, x: 9.119, y: 30.674, w: 11.176, text: '0.8' },
  { page: 1, x: 9.816, y: 30.674, w: 7.148, text: '%' },
  { page: 1, x: 12.194, y: 30.674, w: 6.645, text: '1.' },
  { page: 1, x: 12.614, y: 30.674, w: 4.47, text: '4' },
  { page: 1, x: 12.891, y: 30.674, w: 7.148, text: '%' },
  { page: 1, x: 2.855, y: 32.361, w: 24.586, text: '35%2C716' },
  { page: 1, x: 5.969, y: 32.361, w: 20.116, text: '3%2C591' },
  { page: 1, x: 9.096, y: 32.361, w: 24.586, text: '62%2C000' },
  { page: 1, x: 11.999, y: 32.361, w: 24.586, text: '21%2C144' },
  { page: 1, x: 3.057, y: 33.854, w: 18.323, text: '1.2%' },
  { page: 1, x: 6.081, y: 33.861, w: 11.176, text: '3.1' },
  { page: 1, x: 6.779, y: 33.861, w: 7.148, text: '%' },
  { page: 1, x: 9.231, y: 33.861, w: 6.645, text: '2.' },
  { page: 1, x: 9.651, y: 33.861, w: 4.47, text: '0' },
  { page: 1, x: 9.929, y: 33.861, w: 7.148, text: '%' },
  { page: 1, x: 12.374, y: 33.876, w: 11.176, text: '2.4' },
  { page: 1, x: 13.071, y: 33.876, w: 7.148, text: '%' },
  { page: 1, x: 2.765, y: 21.754, w: 4.47, text: '4' },
  { page: 1, x: 4.055, y: 21.754, w: 8.88, text: '83' },
  { page: 1, x: 5.631, y: 21.747, w: 8.88, text: '24' },
  { page: 1, x: 7.146, y: 21.747, w: 4.47, text: '3' },
  { page: 1, x: 7.424, y: 21.747, w: 4.47, text: '6' },
  { page: 1, x: 8.639, y: 21.747, w: 8.88, text: '15' },
  { page: 1, x: 10.244, y: 21.747, w: 4.47, text: '8' },
  { page: 1, x: 11.676, y: 21.747, w: 8.88, text: '43' },
  { page: 1, x: 13.169, y: 21.747, w: 8.88, text: '23' },
  { page: 1, x: 2.72, y: 15.228, w: 4.47, text: '2' },
  { page: 1, x: 4.145, y: 15.221, w: 8.88, text: '29' },
  { page: 1, x: 5.774, y: 15.221, w: 4.47, text: '0' },
  { page: 1, x: 7.139, y: 15.221, w: 8.88, text: '12' },
  { page: 1, x: 8.774, y: 15.221, w: 4.47, text: '6' },
  { page: 1, x: 10.281, y: 15.221, w: 4.47, text: '2' },
  { page: 1, x: 11.631, y: 15.221, w: 8.88, text: '13' },
  { page: 1, x: 13.161, y: 15.221, w: 8.88, text: '10' },
  {
    page: 1,
    x: 26.416,
    y: 35.421,
    w: 394.595,
    text: 'This infographic is updated every afternoon based on the data we receive by 3.00pm from states and territories'
  },
  { page: 1, x: 1.183, y: 35.406, w: 48.722, text: 'Last updated' },
  { page: 1, x: 4.22, y: 35.406, w: 8.88, text: '10' },
  { page: 1, x: 4.91, y: 35.406, w: 36.196, text: 'April 2020' },
  null
]
[
  [ '6%2C2', '03', '53', '3', '%2C', '141' ],
  [
    '506', '(6', ')',
    '2',   '7',  '965',
    '(',   '5',  ')'
  ],
  [ '4', '2', '8', '(', '3', ')', '2%2C822 (21', ')' ],
  [
    '74',  '1%2C241',
    '(',   '13',
    ')',   '111',
    '103', '(3',
    '(2)', ')'
  ],
  [
    '2',  '29', '0',
    '12', '6',  '2',
    '13', '10'
  ],
  [ '236' ],
  [
    '4', '83', '24',
    '3', '6',  '15',
    '8', '43', '23'
  ],
  [ '338', '%2C', '346' ],
  [ '1.', '8', '%' ],
  [ '6%2C017', '137%2C441', '3%2C243', '69%2C194' ],
  [
    '1.7',  '%',
    '2.1%', '0.8',
    '%',    '1.',
    '4',    '%'
  ],
  [ '35%2C716', '3%2C591', '62%2C000', '21%2C144' ],
  [
    '1.2%', '3.1',
    '%',    '2.',
    '0',    '%',
    '2.4',  '%'
  ],
  [
    'Last updated',
    '10',
    'April 2020',
    'This infographic is updated every afternoon based on the data we receive by 3.00pm from states and territories'
  ]
]

Doesn't make sense at all to me unless the structure stays like that forever. Graph titles are not text
  { page: 1, x: 41.456, y: 7.666, w: 6.672, text: '2' }, --|
  { page: 1, x: 41.876, y: 7.666, w: 6.672, text: '7' },   |---> NT 27
  { page: 1, x: 41.044, y: 9.691, w: 6.672, text: '4' }, --|
  { page: 1, x: 41.464, y: 9.691, w: 6.672, text: '2' },    | ---> WA 428
  { page: 1, x: 41.884, y: 9.691, w: 6.672, text: '8' }, --|

This is really annoying :point_up:
Basically they need to get rid of the map, and use a table with proper text fields aligned

Wow that looks very un-fun.
It might be easier to scrape the paragraphs on the page that links to the PDF lol.

Using ImageMagick -threshold and Tesseract OCR,
convert -density 2000 coronavirus-covid-19-at-a-glance-infographic_6.pdf  -resample 200  -quality 75 -colorspace rgb -threshold 70% +antialias coronavirus-covid-19-at-a-glance-infographic_6.png

tesseract coronavirus-covid-19-at-a-glance-infographic_6.png coronavirus-covid-19-at-a-glance-infographic_6.txt -l eng
It'll probably take some more finesse but I was able to glean this:
:smile:

e2a" Australian Government
_ “VERO

Department of Health

CURRENT STATUS OF CONFIRMED CASES

ss Joss

Total cases Total deaths Cases recovered
|
vf 4 eee
CURRENT CASES

INTENSIVE CARE UNITS (ICU)

Exe DS Bl Eee eS Ee

2 29 0 12 6 2 13 10

CURRENT CASES
ADMITTED TO HOSPITALS

ed CS BL ele EE eS Be EZ
4

83 24 36 15 8 43 23

KTS yr
POSITIVE TOTAL TESTS

CONDUCTED

6,017 137,441 3,243 69,194
1.7% 2.1% 0.8% 1.4%
35,/16 3,591 62,000 21,144
1.2% 3.1% 2.0% 2.4%

Last updated 10 April 2020

DAILY NUMBER OF REPORTED CASES

500 ;
400 ;
300 by
200 5
100 +
0 T T =r afa= in
> S$ 8 S$ § > S$ 8 $8 $§ & > Ss § 8s
ob -t wv =) wv =) wn - o “4 re) “4 re _ ) 3d
N wo ~- ~- NN “NN - = N nN mn =

CASES IN AGED CARE SERVICES

Confirmed cases Australia ACT NSW

26 24

[5] (9) [5] (8) (1)
10 0 5 0 1 1 1 2 0
(1) (1)

Cases in care recipients [recovered] (deaths)

CASES BY AGE GROUP AND SEX

@ Male Female

800 -
700 +
600 -
500 +
400 -
300 +
200 -
100 -

0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90+

DEATHS BY AGE GROUP AND SEX

@ Male Female

0-9 10-19 20-29 30-39 40-49 30-59 60-69 70-79 80-89 90+

ete ee ate]

(COovID-19)

CASES (DEATHS) BY STATE AND TERRITORIES

506 (6)

CASES BY SOURCE OF INFECTION

Australia (% of all confirmed cases)

or: AY 23.7%

0 900 1000 1500 2000 2500 3000

NSW
VIC
QLD
WA
SA
TAS

ACT

NT

m Overseas acquired Locally acquired—contact of a confirmed case

= Locally acquired—contact not identified m@ Under investigation

PUBLIC HEALTH RESPONSE MEASURE

Proportion of total cases under investigation

——-ACT -#-NSW -*NT -<QLD -*SA -eTAS VIC WA
12% 5
10% +
8% -
6% -

4% -

2% +

we ce
— — ————s
~ —ao
A.

vs + yw

0% & q * q = qT = x =
4-Apr 5-Apr 6-Apr 7-Apr 8-Apr 9-Apr

10-Apr

This infographic is updated every afternoon based on the data we receive by 3.00pm from states and territories

😬

Cases by state territory retrieves the text only from WA, otherwise looks better than parsed pdf

Queensland updated their webpage showing now a table that can be used better for scraping
https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/current-status-and-contact-tracing-alerts
NSW added a csv
https://data.nsw.gov.au/data/dataset/nsw-covid-19-cases-by-likely-source-of-infection/resource/2f1ba0f3-8c21-4a86-acaf-444be4401a6d?inner_span=True
even an API?

ACT has this dashboard in some sort of web power point
https://www.covid19.act.gov.au/updates/confirmed-case-information
https://app.powerbi.com/view?r=eyJrIjoiM2M4MGIxYTUtZjA1Yy00YTA4LTkwNDMtMDM4ZTdkMGZhN2ZiIiwidCI6ImI0NmMxOTA4LTAzMzQtNDIzNi1iOTc4LTU4NWVlODhlNDE5OSJ9
from inspect chrome looks like it can be scraped, but curl doesn't return the same src

@subzero79  can you please have a look at what we have now vs those sources, and if there’s extra data (eg deaths for a state we don’t have it yet) please open a separate issue for each state. Good investigating :)


Outlier detection needs to log and handle grpc-status sent as trailers

Title: Outlier detection needs to log and handle grpc-status sent as trailers
Description:

Outlier detection currently doesn't check response trailers, which may contain an error grpc-status when a gRPC stream is closed.

Relevant Links:

See the discussion on the original issue for adding grpc-status support in Outlier Detection.

Sorry, I was away - will comment within a day or two. Thanks!

Hi @wenbozhu  and @cpakulski I'm a student in UT Austin's Virtualization & Containers class. Are you still looking for help in adding support for gRPC response trailers? Thanks!

We probably need some design inputs from @cpakulski  first ...
@htuch

Yeah, I think if you want to help drive the design here and propose a strawman @rcash that might be a reasonable start.

@rcash, @wenbozhu Please look at https://github.com/envoyproxy/envoy/pull/7942 as that solved similar case of passing gRPC codes to outlier detector. Let me know if you get stuck in design or understanding the code flow.


feat: migrating to nextjs

https://nextjs.org/docs/migrating/from-gatsby
Move all gatsby links to nextjs links
Removed all gatsby packages
Renamed api --> docs since it's a reserved namespace for nextjs
Added CSS variables rather than relying on "lightMode". https://github.com/pacocoursey/next-themes#avoid-hydration-mismatch . Probably where I will need guidance, as my CSS -foo isn't the best (but there are no hydration mismatches or flashes!)

TODO: I don't think I understand Nav too well unfortunately.
I need to find a way to separate Desktop Nav vs Mobile nav since I get them confused so easily

I could make this even simpler by doing the rename from api -> docs while gatsby is still default. Might focus the dark theme issues more than now

@bluebill1049 can you please approve the deployment?
Fixed up the dark mode issues (that I saw) using global(".dark") and global(".light") CSS modules and I think it's looking pretty good.
Just want to verify from non-dev machine.
Thanks

works great with dark mode 👍

I will review it this week and merge it in.

Let me know if there's anything I need to address. I see that I need to fix light mode select in API page.


Support MongoDB v7

I have a client that wants to upgrade their Atlas Cluster to MongoDB v7, but I’ve asked them to wait until this driver officially supports it. They are of the opinion that there aren’t any breaking changes (relevant for us) in v7, so it will probably work just fine to use the current driver, but I am quite skeptical.
Are there any plans to support v7?

I believe, there is no much difference from the driver's perspective. You can just use it with MongoDB 7. On my local machine I use 7 already.

OK, that’s good to know - thanks! In that case the issue can be closed as far as I am concerned, but do you maybe want to keep it open to track progress on officially supporting v7 at some point?

We can keep it open to remember me to check if something is new. I have a similar issue for MongoDB 6, but didn't find the time to evaluate if there are changes for the driver. :-)

For what it's worth, after switching mongodb_ecto to use this driver (v1.4.0), we were able to add MongoDB 7.0 to the CI tests without any issues: https://github.com/elixir-mongo/mongodb_ecto/blob/04ce48dfc69c06b49d02130d48ae1a236360100a/.github/workflows/ci.yml#L39

@zookzook I think this issue can be closed? With Mongo 8 out, there should probably be a new issue for Mongo 8.