opencv is not working with foxy-base image

Hello @dusty-nv I am working right now with a Jetson nano using the foxy-ros-base-l4t-r35.4.1 after the upgrade I got some issues and I saw that it is recommended to remove old versions with sudo apt-get purge -y '*opencv'. After doing this I got the next error
cv2.error: OpenCV(4.5.0) /opt/opencv/modules/highgui/src/window_gtk.cpp:624: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'
I tried to reinstall opencv and that wasn't work. This is my command to run the docker container sudo docker run --runtime nvidia -it --privileged -v /dev/bus/usb:/dev/bus/usb --network host --name windrobo --device-cgroup-rule='c 189:* rmw' -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix dustynv/ros:foxy-ros-base-l4t-r35.4.1 Do you know what my problem could be?
Thanks.

@JuanCarlos-TiqueRangel are you on the original Jetson Nano, or Jetson Orin Nano?   The original Jetson Nano uses JetPack 4 and doesn't run JetPack 5 containers (so you would use an r32.x.x container on the original Nano, depending on your version of L4T)
What does your cv2.getBuildInformation() show?   This is mine, and it appears to be built with GTK enabled:
General configuration for OpenCV 4.5.0 =====================================
  Version control:               4.5.0

  Extra modules:
    Location (extra):            /opt/opencv_contrib/modules
    Version control (extra):     4.5.0

  Platform:
    Timestamp:                   2021-10-23T16:52:07Z
    Host:                        Linux 5.10.59-tegra aarch64
    CMake:                       3.16.3
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    NEON FP16
      required:                  NEON
      disabled:                  VFPV3

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 9.3.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda/lib64 -L/usr/lib/aarch64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cvv hdf java js julia matlab ovis python2 sfm ts viz
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         YES

  GUI:
    GTK+:                        YES (ver 3.24.20)
      GThread :                  YES (ver 2.64.6)
      GtkGlExt:                  NO
    OpenGL support:              NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 2.3.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      YES (2.2.5)
    FFMPEG:                      YES
      avcodec:                   YES (58.54.100)
      avformat:                  YES (58.29.100)
      avutil:                    YES (56.31.100)
      swscale:                   YES (5.5.100)
      avresample:                YES (4.0.0)
    GStreamer:                   YES (1.16.2)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            TBB (ver 2020.1 interface 11101)

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      YES (/usr/lib/aarch64-linux-gnu/liblapack.so /usr/lib/aarch64-linux-gnu/libcblas.so /usr/lib/aarch64-linux-gnu/libatlas.so)
    Eigen:                       YES (ver 3.3.7)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   YES (ver 11.4, CUFFT CUBLAS FAST_MATH)
    NVIDIA GPU arch:             72 87
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.2.6)

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.8.10)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.8.so (ver 3.8.10)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.17.4)
    install path:                lib/python3.8/dist-packages/cv2/python-3.8

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------

Are you able to view any GUI window or GTK window from your container?

Hi @dusty-nv I am using the original Jetson Nano, so according to your last comment I pulled the container foxy-ros-base-l4t-r32.7.1 but unfortunately I got the same error.
This is what cv2.getBuildInformation() shows.
General configuration for OpenCV 4.5.0 =====================================
  Version control:               4.5.0

  Extra modules:
    Location (extra):            /opt/opencv_contrib/modules
    Version control (extra):     4.5.0

  Platform:
    Timestamp:                   2021-07-19T21:05:54Z
    Host:                        Linux 4.9.201-tegra aarch64
    CMake:                       3.10.2
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    NEON FP16
      required:                  NEON
      disabled:                  VFPV3

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 7.5.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppicom nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda/lib64 -L/usr/lib/aarch64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cvv hdf java js julia matlab ovis python2 sfm ts viz
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         YES

  GUI:
    GTK+:                        YES (ver 3.22.30)
      GThread :                  YES (ver 2.56.4)
      GtkGlExt:                  NO
    OpenGL support:              NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 2.3.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      YES (2.2.5)
    FFMPEG:                      YES
      avcodec:                   YES (57.107.100)
      avformat:                  YES (57.83.100)
      avutil:                    YES (55.78.100)
      swscale:                   YES (4.8.100)
      avresample:                YES (3.7.0)
    GStreamer:                   YES (1.14.5)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            TBB (ver 2017.0 interface 9107)

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      YES (/usr/lib/aarch64-linux-gnu/liblapack.so /usr/lib/aarch64-linux-gnu/libcblas.so /usr/lib/aarch64-linux-gnu/libatlas.so)
    Eigen:                       YES (ver 3.3.4)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   YES (ver 10.2, CUFFT CUBLAS FAST_MATH)
    NVIDIA GPU arch:             53 62 72
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.0.0)

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.6.9)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.13.3)
    install path:                lib/python3.6/dist-packages/cv2/python-3.6

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------

As well as you I have the GTK enabled but I am still with the error. Also when I tried to display a picture with imagemagick I got display-im6.q16: unable to open X server 'localhost:11.0' @ error/display.c/DisplayImageCommand/432. but I added the $DISPLAY and the other commands when I ran the container.

@JuanCarlos-TiqueRangel did you try running sudo xhost +si:localuser:root first before you launched the container?
Here are the flags I run/add when enabling display in container:
https://github.com/dusty-nv/jetson-containers/blob/e46bea804ae31a95485e4ff97e10ea7ba0cd8dc7/run.sh#L16

@dusty-nv after follow your instructions about the sudo xhost +si:localuser:root and I added the flags so, the run command is the next:
sudo docker run --runtime nvidia -it \
	--privileged \
	-v /dev/bus/usb:/dev/bus/usb \
	--network host --name windrobo \
	--device-cgroup-rule='c 189:* rmw' \
	-e DISPLAY=$DISPLAY \
	-v /tmp/.X11-unix:/tmp/.X11-unix \
	-v $XAUTH:$XAUTH \
	-e XAUTHORITY=$XAUTH \
	--env="QT_X11_NO_MITSHM=1" \
	dustynv/ros:foxy-ros-base-l4t-r32.7.1 \

I was into the container and I updated and upgraded it then I restarted the container and know I have the next error:
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/tmp/.docker.xauth" to rootfs at "/tmp/.docker.xauth": mount /tmp/.docker.xauth:/tmp/.docker.xauth (via /proc/self/fd/7), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

Do you know what I did wrong ?

@dusty-nv an update about this issue. After doing a research about the problem I got that to fix it is neccesary to remove the old Xauthority data. However, I used your run.sh file and now I got this error
'''
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/tmp/nv_jetson_model" to rootfs at "/tmp/nv_jetson_model": mount /tmp/nv_jetson_model:/tmp/nv_jetson_model (via /proc/self/fd/7), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
'''
Maybe Do you know what is the problem ? Thank you for your time and help in this matter. Sorry to bother you with this case.

@JuanCarlos-TiqueRangel cat /proc/device-tree/model > /tmp/nv_jetson_model should have been run by run.sh here:
https://github.com/dusty-nv/jetson-containers/blob/02fa7dcb3cc97efb5818ab5789aaa1a6a999829b/run.sh#L45C2-L45C52
Can you try running that?
It's done like that because /proc files can't be mounted

@dusty-nv hi, Thanks for your asnwer. This is my run.sh
#!/usr/bin/env bash
# pass-through commands to 'docker run' with some defaults
# https://docs.docker.com/engine/reference/commandline/run/
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

# check for V4L2 devices
V4L2_DEVICES=""

for i in {0..9}
do
	if [ -a "/dev/video$i" ]; then
		V4L2_DEVICES="$V4L2_DEVICES --device /dev/video$i "
	fi
done

# check for display
DISPLAY_DEVICE=""

if [ -n "$DISPLAY" ]; then
	# give docker root user X11 permissions
	sudo xhost +si:localuser:root

	# enable SSH X11 forwarding inside container (https://stackoverflow.com/q/48235040)
	XAUTH=/tmp/.docker.xauth
	xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -
	chmod 777 $XAUTH

	DISPLAY_DEVICE="-e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix -v $XAUTH:$XAUTH -e XAUTHORITY=$XAUTH"
fi

# check if sudo is needed
if id -nG "$USER" | grep -qw "docker"; then
	SUDO=""
else
	SUDO="sudo"
fi

# run the container
ARCH=$(uname -i)

if [ $ARCH = "aarch64" ]; then

	# this file shows what Jetson board is running
	# /proc or /sys files aren't mountable into docker
	cat /proc/device-tree/model > /tmp/nv_jetson_model

	set -x

	$SUDO docker run --runtime nvidia -it --rm --network host \
		--volume /tmp/argus_socket:/tmp/argus_socket \
		--volume /etc/enctune.conf:/etc/enctune.conf \
		--volume /etc/nv_tegra_release:/etc/nv_tegra_release \
		--volume /tmp/nv_jetson_model:/tmp/nv_jetson_model \
		--volume $ROOT/data:/data \
		--device /dev/snd \
		--device /dev/bus/usb \
		$DATA_VOLUME $DISPLAY_DEVICE $V4L2_DEVICES \
		"$@"

elif [ $ARCH = "x86_64" ]; then

	set -x

	$SUDO docker run --gpus all -it --rm --network=host \
		--shm-size=8g \
		--ulimit memlock=-1 \
		--ulimit stack=67108864 \
		--env NVIDIA_DRIVER_CAPABILITIES=all \
		--volume $ROOT/data:/data \
		$DATA_VOLUME $DISPLAY_DEVICE $V4L2_DEVICES \
		"$@"
fi

As you see, I deleted the --rm flag because I want to continue using the container. But when I ran sudo docker start <container_name> I got the error
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/tmp/nv_jetson_model" to rootfs at "/tmp/nv_jetson_model": mount /tmp/nv_jetson_model:/tmp/nv_jetson_model (via /proc/self/fd/7), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

So having that in mind, What can I do to avoid this error and still have my container ?

@JuanCarlos-TiqueRangel remove the --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model line from run.sh

Hi @dusty-nv I followed some of your advices in this issue #212. However, after the reboot I got the next error again:
cv2.imshow(window_name, image)
cv2.error: OpenCV(4.5.0) /opt/opencv/modules/highgui/src/window_gtk.cpp:624: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'

this is my run.sh file

#!/usr/bin/env bash
# pass-through commands to 'docker run' with some defaults
# https://docs.docker.com/engine/reference/commandline/run/
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

# check for V4L2 devices
V4L2_DEVICES=""

for i in {0..9}
do
        if [ -a "/dev/video$i" ]; then
                V4L2_DEVICES="$V4L2_DEVICES --device /dev/video$i "
        fi
done

# check for display
DISPLAY_DEVICE=""

if [ -n "$DISPLAY" ]; then
        # give docker root user X11 permissions
        sudo xhost +si:localuser:root

        # enable SSH X11 forwarding inside container (https://stackoverflow.com/q/48235040)
        XAUTH=/home/windrobo/Documents/docker_xauth/.docker.xauth
        xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -
        chmod 777 $XAUTH

        DISPLAY_DEVICE="-e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix -v $XAUTH:$XAUTH -e XAUTHORITY=$XAUTH"
fi

# check if sudo is needed
if id -nG "$USER" | grep -qw "docker"; then
        SUDO=""
else
        SUDO="sudo"
fi

# run the container
ARCH=$(uname -i)

if [ $ARCH = "aarch64" ]; then

        # this file shows what Jetson board is running
        # /proc or /sys files aren't mountable into docker
        cat /proc/device-tree/model > /tmp/nv_jetson_model

        set -x

        $SUDO docker run --runtime nvidia -it --name windrobo --network host \
                --env="QT_X11_NO_MITSHM=1" \
                --volume /tmp/argus_socket:/tmp/argus_socket \
                --volume /etc/enctune.conf:/etc/enctune.conf \
                --volume /etc/nv_tegra_release:/etc/nv_tegra_release \
                --volume $ROOT/data:/data \
                --device /dev/snd \
                --device /dev/bus/usb \
                $DATA_VOLUME $DISPLAY_DEVICE $V4L2_DEVICES \
                "$@"

elif [ $ARCH = "x86_64" ]; then

        set -x

        $SUDO docker run --gpus all -it --network=host \
                --shm-size=8g \
                --ulimit memlock=-1 \
                --ulimit stack=67108864 \
                --env NVIDIA_DRIVER_CAPABILITIES=all \
                --volume $ROOT/data:/data \
                $DATA_VOLUME $DISPLAY_DEVICE $V4L2_DEVICES \
                "$@"
fi

At this point I do not know what I am doing wrong. Do yow know what could be the problem ?

@JuanCarlos-TiqueRangel at this point, are you able to display other GUI windows within container, like other GTK GUI apps, GEdit, XTerm, imagemagik, ect?

@dusty-nv No I cannot. After reboot the system If I want to use OpenCV I got this:
cv2.imshow(window_name, image)
cv2.error: OpenCV(4.5.0) /opt/opencv/modules/highgui/src/window_gtk.cpp:624: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'

And if I want to use imagemagik, I got the next error after run display image.jpg:
X11 connection rejected because of wrong authentication.
display-im6.q16: unable to open X server `localhost:10.0' @ error/display.c/DisplayImageCommand/432.

Maybe you know what is the problem ? I think is something with the flags but to be honest I do not know exactly what to do in this case.

Honestly the X11-in-container troubleshooting always baffled me too.   Try it with nvcr.io/nvidia/l4t-jetpack:r35.4.1 too

Hi @dusty-nv the same problem. This first time works good but after reboot the system I got the error
cv2.imshow(window_name, image)
cv2.error: OpenCV(4.5.0) /opt/opencv/modules/highgui/src/window_gtk.cpp:624: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'

Maybe do you have some advice about how to fix this ? It is taking me more time that I thought

This first time works good but after reboot the system I got the error

Ahh okay, my guess is this is related to you removing the --rm and wanting to restart the same container, and potentially the -v /tmp/.X11-unix/:/tmp/.X11-unix file changing after reboot since it's in /tmp.   I see you already changed the XAUTH file to not be in /tmp anymore.
It doesn't seem possible/supported to add a mount dynamically after the container is started:   https://stackoverflow.com/questions/28302178/how-can-i-add-a-volume-to-an-existing-docker-container
TBH, I'm not sure how to fix it without starting new container after reboot, I'm sorry :/

Hello @dusty-nv After doing research about the containers with docker I realize that creating a container for a long term is bad practice. The solution for this issue is first install the necessary packages in the Dockerfile and then mount a volume that has the file that I need in my local and host. Is basically add the next line in the docker run command:
--volume $ROOT/ros2_ws/src:/root/ros2_ws/src \

Thank you for your time and help here.

Yea, making your own Dockerfile is preferred because then you can re-build in an automated way whenever necessary.   Otherwise you can save changes that you made by hand using docker commit


runtime interoperability of cupy and numpy without explicit cupy dependency?

Hello,
I maintain a scientific package that does a significant amount of numerical work.  I allow an optional dependency of numba to fuse CPU kernels and achieve very good performance on CPUs.  I would also like to develop GPU support via cupy.  The idea would be that if a user has just numpy, numpy will be used.  If they have numba, then they will get "numpy+" with better CPU performance.  If they have cupy installed and call something like config.set_backend('cuda'), then cupy would be used, and they can flip between CPU (via numpy+numba) and GPU (via cupy) at a whim.
I have a submodule/file that re-exports common functions (sin, cos, fft, etc) from numpy and import from this throughout my package.  It seems that a variable of type function has pass by value semantics when it comes to imports / is immutable.
For example, a test;
# test_a.py
def _foo():
    print('foo')
    global testd
    testd['bar'] = 'foo'

def _bar():
    print('bar')

def chfoo():
    global foo
    foo = _bar

foo = _foo
testd = {'foo': 'bar'}

#test_b.py
from test_a import foo, chfoo, testd

print(testd)
foo()
print(testd)
chfoo()
foo()

>>> python test_b.py
>>> {'foo': 'bar'}
>>> foo
>>> {'foo': 'bar', 'bar': 'foo'}
>>> foo

So this shows that:

mutable types like dicts can be mutated after import, which allows things like shared config
immutable types like functions cannot be reassigned after import

My prior usage of:
# somewhere in my code
from .mathops import sin, cos, exp, pi

# do something with sin(x)

Could change to
# somewhere in my code
from .mathops import mth

# do something with mth['sin'](x)

But most editors cannot follow this sort of thing to find docstrings and autocompletes, and the syntax is clunky.  I could use a class instead of a dict and have dot syntax, but this feels a bit like a hack.  I could also write out something like:
import numpy as np
import cupy as cp

npmath = {}
cpmath = {}

npmath['sin'] = np.sin
cpmath['sin'] = cp.sin

def sin():  # missing some bits, but hopefully the idea communicates
    if config.backend == 'cuda':
        return cumath['sin']()
    else:
        return npmath['sin']()

But this will incur an extra logical comparison cost at runtime, and also involve an immense amount of boilerplate and care to match arguments (and possibly gloss over small API differences between cupy and numpy).
(how) does cupy handle this?

Update, it seems that there is a slightly non preferred way:
#test_a.py

def _foo():
    print('foo')

def _bar():
    print('bar')

def chfoo():
    global foo
    foo = _bar

foo = _foo

# test_b.py
import test_a as a
a.foo()
a.chfoo()
a.foo()

prints "foo" and then "bar" as desired.
  Perhaps there is a hacky way to overcome this with monkey patching?

Thanks for considering use of CuPy in your package.
I think you can do that like this:
import numpy

try:
    import cupy
except ImportError:
    cupy = None

def get_xp(use_cupy_if_available=True):
    if use_cupy_if_available and cupy is not None:
        return cupy
    return numpy

Then you can use get_xp() (it may return numpy or cupy module) in other places:
xp = get_xp()
xp.sin(...)


Can't run Jocular on Apple Silicon

Having trouble launching Jocular on a Macbook with a M1 chip. It quits soon after launch.
The first time(s) I was able to see the app and even pick the data directory but it quit soon after.
Now it just quits right away.
Below is the output in the terminal.

% /opt/homebrew/bin/jocular --debug
[INFO   ] [Logger      ] Record log in /Users/wmathurin/.kivy/logs/kivy_23-11-07_4.txt
[INFO   ] [Kivy        ] v2.2.1
[INFO   ] [Kivy        ] Installed at "/opt/homebrew/lib/python3.11/site-packages/kivy/init.py"
[INFO   ] [Python      ] v3.11.5 (main, Aug 24 2023, 15:09:45) [Clang 14.0.3 (clang-14<IP_ADDRESS>.1)]
[INFO   ] [Python      ] Interpreter at<EMAIL_ADDRESS>[INFO   ] [Logger      ] Purge log fired. Processing...
[INFO   ] [Logger      ] Purge finished!
[INFO   ] [KivyMD      ] 1.1.1, git-Unknown, 2023-11-04 (installed at "/opt/homebrew/lib/python3.11/site-packages/kivymd/init.py")
[INFO   ] [Factory     ] 190 symbols loaded
[DEBUG  ] [Cache       ] register <kv.resourcefind> with limit=None, timeout=60
[DEBUG  ] [Cache       ] register <kv.image> with limit=None, timeout=60
[DEBUG  ] [Cache       ] register <kv.atlas> with limit=None, timeout=None
[INFO   ] [Image       ] Providers: img_tex, img_imageio, img_dds, img_sdl2, img_pil (img_ffpyplayer ignored)
[DEBUG  ] [Cache       ] register <kv.texture> with limit=1000, timeout=60
[DEBUG  ] [Cache       ] register <kv.shader> with limit=1000, timeout=3600
[DEBUG  ] [Cache       ] register <kv.graphics.texture> with limit=None, timeout=None
[INFO   ] [Text        ] Provider: sdl2
[DEBUG  ] [Cache       ] register <kv.lang> with limit=None, timeout=None
[INFO   ] [Window      ] Provider: sdl2
[INFO   ] [GL          ] Using the "OpenGL ES 2" graphics system
[INFO   ] [GL          ] Backend used
[INFO   ] [GL          ] OpenGL version <b'2.1 Metal - 83.1'>
[INFO   ] [GL          ] OpenGL vendor <b'Apple'>
[INFO   ] [GL          ] OpenGL renderer <b'Apple M1 Max'>
[INFO   ] [GL          ] OpenGL parsed version: 2, 1
[INFO   ] [GL          ] Shading version <b'1.20'>
[INFO   ] [GL          ] Texture max size <16384>
[INFO   ] [GL          ] Texture max units <16>
[DEBUG  ] [Shader      ] Fragment compiled successfully
[DEBUG  ] [Shader      ] Vertex compiled successfully
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.11/site-packages/kivy/data/glsl/default.png>
[INFO   ] [Window      ] auto add sdl2 input provider
[INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
[DEBUG  ] [Cache       ] register <textinput.label> with limit=None, timeout=60.0
[DEBUG  ] [Cache       ] register <textinput.width> with limit=None, timeout=60.0
[INFO   ] [Clipboard   ] Provider: sdl2(['clipboard_nspaste'] ignored)

Thanks for providing the terminal output.  Jocular is not supported on Python 3.11, because Jocular depends on some libraries that must run on Python versions 3.6 - 3.9.  https://jocular.readthedocs.io/en/latest/installation.html. Please try again with one of those versions.

Thanks @steveincolo, switching to 3.9 did the trick.

I closed the issue too quickly.
I can successfully launch the application however it crashes when I pick one of the DSO example - following the steps from the quickstart doc.

Could you please attach the log file?  You can find it in the ‘logs’ subdirectory of the Jocular data directory.

Here you go:
jocular.log

This is a change caused by the latest scikit-image. A fix is to change L62 of monochrome.py to
binned = rescale(im, 1 / binfac, anti_aliasing=True, mode='constant')
(ie removing the multichannel argument)
I'll ensure this is updated on the next release.

It worked. Thank you.


Complete CI tasks

We have recently migrated to Github Actions #339
A few follow up tasks showed up along the way:

Enable flake8 linting. It is disabled for some reason, I am guessing to hide embarrassingly many errors
Revisit command line tests setup. At the moment we have some weird default stuff going on for executing pytest
Sort out GPy somehow. It becomes real pain, I wasn't able to install emukit and GPy for any Python versions other than 3.7. Same error as https://github.com/SheffieldML/GPy/issues/874

Also, https://github.com/SheffieldML/GPy/pull/888 upgrades GPy for 3.9 - 3.8 should have already been working 🤔

So it does. Added 3.8, thanks @ekalosak for pointing that out

Seems that all points are taken care of now


fix: Unhandled Runtime Error

This PR should fix an error about SSR
see https://nextjs.org/docs/messages/react-hydration-error

❤


bump centos images oc's to 3.6 GA

fixes https://github.com/openshift/origin/issues/15656
@openshift/devex @siamaksade

[test]


Add missing test

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (1db45f9) to head (961f553).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #70   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           45        45
  Lines          798       798
  Branches        70        70
=========================================
  Hits           798       798

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


MSBuild step is not a clean MSBuild task

Configuration of a step of type MSBuild requires providing a solution file (*.sln).
So it is not really a MSBuild task.
I would like to run a MSBuild script, not a specific solution.
Is it possible to change the task accordingly? If not, the current one may be renamed and a a new new one created instead.

You should be able to pass any MSBuild script to the MSBuild task, since this will be straightforward passed to msbuild.exe (See https://github.com/Microsoft/vso-agent-tasks/blob/master/Tasks/MSBuild/MSBuild.ps1#L165).

Ok, I'll try if it works.
Anyway, I think you should switch behavior and description: A task which is named "MSBuild" should focus on MSBuild and allow solution files.

I think this can be closed with db9f229ec5ebd7a65786994b88c49cc2e0000ae3


Partial/complete control of mind backend and connection

As a user, I want to completely control the car with my mind to have it driven with only my mind.
As a user, I want to partially control the car with my mind to have it driven with 50% my mind and 50% the manual control

I was already working on this in sprint2. It's about to be merged with the branch pull request I just made. Could you review it maybe? :)


Current URL not checked

Very bad bad situation of current page add in history again
for example
aaaa
bbbb
there is two link
when i click on aaaa
it work ,
but if i click it again , it not check last state , where we are and ...
it add to history  [back / next]

With the new version 8.0.0 you can prevent the "adding to history" situation by using the historyAPIMethod option of the navigate method.
router.navigate('/new-page', { historyAPIMethod: 'replaceState' });

click me

P.S.
Version 8.0.0 is not officially released but can be installed via npm install navigo@beta or yarn add navigo@beta. Migration guide is available here https://github.com/krasimir/navigo/blob/big-rewrite/CHANGELOG.md#migration-guide and the new documentation here https://github.com/krasimir/navigo/blob/big-rewrite/DOCUMENTATION.md.


Engine throttle in replays broken if skipped

If you skip (seek) past the part where someone sets an engine throttle, then the engine throttle that is simulated in the replay after that will be incorrect.

Fixed


Make DateTime timezone aware

#4276
Tested this very briefly on my test environment. May need more testing.

Adding this to 0.10.1 milestone since it fixes an issue in the milestone
There's a unit test failure in QueryGranularityTest

The unit tests were meant to work with date time not being timezone aware. I'm assuming we want to be consistent with 0.9.2 so I've changed the unit tests to work accordingly.

@praveev @jon-wei It looks like #4276 is a regression somewhere else in the code. I'm not sure fixing it in PeriodGranularity is the right place, since it worked fine in 0.9.x

@xvrl this was introduced with support for timezone and refactor of granularity on this PR https://github.com/druid-io/druid/pull/3850 that was released with 0.10.0

@praveev ah yes, I stand corrected, looks like #3850 broke timezones indeed

Tranquility probably cannot be updated to use 0.10.0 without this so it should directly go to 0.10.1


question: Theme documentation link?

Apologize if I'm just completely missing it, but I can't find any sort of documentation anywhere for stuff like fonts and themes and whatnot. I'm trying to make my own theme with a custom font, but I'm not sure where to start

I did find it eventually, it's https://vendetta-themes.gitbook.io/wiki/ for anyone looking for it


Test logging

Codecov Report

Merging #21 (25ed913) into main (38c6a64) will increase coverage by 7.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #21      +/-   ##
==========================================
+ Coverage   80.43%   87.53%   +7.09%
==========================================
  Files          23       23
  Lines         782      786       +4
==========================================
+ Hits          629      688      +59
+ Misses        153       98      -55

Impacted Files
Coverage Δ

tsflex/features/feature_collection.py
80.17% <ø> (+8.62%)
:arrow_up:

tsflex/processing/series_pipeline.py
94.20% <ø> (+14.49%)
:arrow_up:

tsflex/features/__init__.py
100.00% <100.00%> (ø)

tsflex/features/feature.py
97.91% <100.00%> (ø)

tsflex/features/logger.py
88.88% <100.00%> (+43.17%)
:arrow_up:

tsflex/processing/logger.py
100.00% <100.00%> (+48.00%)
:arrow_up:

tsflex/utils/logging.py
100.00% <100.00%> (+72.72%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38c6a64...25ed913. Read the comment docs.


Runtime error when using bevy_winit feature without explicitly adding bevy_window

Bevy version
0.15. This issue appeared at some point during the RC process. LIkely https://github.com/bevyengine/bevy/pull/16401.
What you did
[dev-dependencies.bevy]
version = "0.15"
default-features = false
features = [
    "android_shared_stdcxx",
    "bevy_asset",
    "bevy_core_pipeline",
    "bevy_pbr",
    "bevy_render",
    "bevy_state",
    "bevy_ui",
    "bevy_winit",
    # "bevy_window",
    "default_font",
    "hdr",
    "multi_threaded",
    "tonemapping_luts",
    "ktx2",
    "zstd",
    "x11",
]

And start an app using DefaultPlugins that creates a window.
What went wrong
App immediately crashes with this error, which is produced by a system outside of my app / inside of Bevy.
Resource requested by (bevy_ecs::system::commands::Commands, bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &mut bevy_window::window::Window, core::option::Option<&bevy_window::raw_handle::RawHandleWrapperHolder>), bevy_ecs::query::filter::Added<bevy_window::window::Window>>, bevy_ecs::event::writer::EventWriter<bevy_window::event::WindowCreated>, bevy_ecs::change_detection::NonSendMut<bevy_winit::winit_windows::WinitWindows>, bevy_ecs::change_detection::NonSendMut<bevy_winit::accessibility::AccessKitAdapters>, bevy_ecs::change_detection::ResMut<bevy_winit::accessibility::WinitActionRequestHandlers>, bevy_ecs::change_detection::Res<bevy_a11y::AccessibilityRequested>, bevy_ecs::change_detection::Res<bevy_winit::winit_monitors::WinitMonitors>) does not exist: bevy_ecs::event::collections::Events<bevy_window::event::WindowCreated>

bevy_winit should have a required dependency on bevy_window.


Fix memory leak in show notification when content is object type

What does the pull request do?

Fix memory leak in show notification when content is object type
What is the current behavior?

The NotificationCard is removed after closed and the input content is INotification interface
What is the updated/expected behavior with this PR?

The NotificationCard is removed after closed
How was the solution implemented (if it's not obvious)?

Remove the type check
Checklist

[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Documentation with user documentation

Breaking changes

Obsoletions / Deprecations

Fixed issues

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: ables3000 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

You can test this PR using the following package version. 0.10.999-cibuild0023216-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Thanks!


Graceful closure of websocket connection from script

During certain states like task synchronization, script cannot close the connection when user is trying to end the script.
Expected: In all states of script, it should allow user to override closure using Ctrl+C

Websocket connections gracefully close on

Ctrl + C / SIGINT
Completion of task activity
If websocket connections break and reconnect, further interrupts will terminate normally
5 retries of reconnection of websocket and subsequent failure
Errors from backend


fix 'OptionButton dropdown popup doesn't scale #19451'

This fixes #19451
Demo of the fixed popup: https://youtu.be/BHZZv8wtHs4
Test project adds 4 buttons to a Node2D scene called test. test's _ready function adds 10 items to each option button, then sets the scales on 3 of the 4 buttons.

Actually, I was hasty with my test. This fails to fully resolve the issue because when the button is in a quadrant with negative x, the popup menu is still positioned incorrectly (regardless of the fix). I will cancel this request and try to fix both the size and position in a later request.
Min repro project: https://youlovetocode.com/godot/option_button_test.zip


[kube-prometheus-stack] fix missing unscheduled Pods

What this PR does / why we need it
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer
Checklist

[ ] DCO signed
[ ] Chart Version bumped
[ ] Title of the PR starts with chart name (e.g. [prometheus-couchdb-exporter])

cc @junotx


Return response body

Return response body as before when methods where called through __call

Note that this PR is built on top of PR #21.

:+1: These look great to me. Go for it!
Ideally, we can add a test to the Examples Repo with an example test of OPTIONS.


Request json file response empty content cause by wrong content-type

Steps to Reproduce:

Request URL: http://<IP_ADDRESS>:53640/A7k3KMrG6/file:///F:/angular/angular-electron/dist/assets/i18n/en.json
Request Method: GET
Status Code: 222 unknown
Remote Address: <IP_ADDRESS>:53640
Referrer Policy: no-referrer-when-downgrade
Response Headers
Connection: keep-alive
content-type: text/html
Date: Wed, 09 Dec 2020 05:41:52 GMT
Keep-Alive: timeout=5
Transfer-Encoding: chunked
When I paste the Request URL in chrome browser, response json result correct. I think content-type of this response should be application/json.
Your Environment details:

testcafe version:                   1.9.4
node.js version:                    14.15.1
platform and version:          Windows 10
electron version                   10.1.5
testcafe-browser-provider-electron  0.0.15

@moJiXiang
Hello,
It looks like a CORS failure. Make sure that the route includes CORS headers (for example, "access-control-allow-origin") in its response. If so, and your scenario works successfully without TestCafe, please share a simplified project with us. We will examine it and check for a suitable solution.


Add "select through function"

I would like to retrieve an element based on a custom expression rather than a css selector.
Something like:
const xLink = await page.$( () => document.querySelectorAll('div').filter( el => el.innerText = 'Hello world!'))

await xLink.click()

Is there already a way to do that? This would be extremely useful given that many pages do not have clear ids.

This is something I've been meaning to do. Would definitely be super nice.


Rescue on GetIntoTeachingApiClient::ApiError

The Api Client rescues errors and exposes them as an ApiError, we need to rescue on this instead of the underlying Faraday error.

I don't think it can throw a Faraday error as it swallows them, but I'll double check 👀


feat: allow occam to cope with pack limitations

pack seems to be very limited in the options it will accept when building for multi-arch. It does not
seem to accept a path with a tgz as input and only works properly if you build in an exploded directory. Otherwise it appears to have built the cnb file but the structure of the bin directory is incorrect and the executables cannot be found.
Update the jam packager to work around this by exploding the tgz after creating it and then build in that
exploded directory.
It is still a good idea to build the tgz as that
filters the files based on what is specified in the buildpack.toml.
The unit tests pass, but more integration testing might be good. Once I have confirmation that people
are comfortable withe the PR I'll manually test on a few existing buildpacks in addition to the testing I've
already done on the npm-install buildpack modified for multi-arch -> https://github.com/mhdawson/npm-install/tree/test-multi (Note that this needs an updated jam that includes an updated packit with this change - https://github.com/paketo-buildpacks/packit/pull/604)

Summary

Allow the ability to test buildpacks that have been updated to support multi-arch. Occam will still only test on the one architecture as the machine the tests are run on but with the update can run tests for buildpacks that support multi-arch, or buildpacks that don't.
Use Cases

Testing buildpacks that support multi-arch
Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.
[x] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[x] I have added an integration test, if necessary.
[x] I have reviewed the styleguide for guidance on my code quality.
[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

@robdimsdale any chance you could take a look, we need this to progress on getting Node.js buildpacks to support multi-arch.

@robdimsdale any chance you could take a look?

@ForestEckhardt, @robdimsdale any chance one of you two could take a look?

Should we address this at the source (i.e. jam) by either adding a different flag or something instead of trying to hack around it in occam?

@ForestEckhardt the real fix is for pack to better support multi-arch so that all of the different options/use cases it supports for non-multiarch work but that is a much bigger effort that what is in this PR.
In this case I think jam just calls pack passing through some of the options provided by occam so it seems more confusing to me to try to add the work around in jam that says even through I called you with these pack options, convert the input and options to something else before calling pack.

@mhdawson So jam does not call pack internally at all it does however produce the compressed tar file that you are unwinding. We could add a flag to only produce a directory structure which is compatible with a multi-arch build as far as I understand. We could then set that flag in occam until we reconcile what artifacts we are going to release with our buildpacks going forward.

I'm going to look at updating jam to see how that works out.


Code Size: Pass large loadable types by address instead of by value - Updated Version

radar rdar://problem/28680453
This is an updated version of https://github.com/apple/swift/pull/8909
It addresses the code review comments and splits the PR into multiple commits
Reduce the size of the text area (code size of the compiled output) by changing the way we generate LLVM IR for loadable types:
We do a SIL level transformation, during IRGen, that, from a bird's eye view, does the following:
For every every function that takes as an input a big type, re-write it to take a pointer to a big type instead.
: $@convention(method) (BigStruct) turns into $@convention(method) (@in_constant BigStruct)
There are a few new SIL pieces to support this:

A @in_constant calling convention
New retain_value_addr and release_value_addr SIL instructions that take as an input an address, load the value inside it and call retain_value and release_value respectively
New outlined functions that do retains and releases of large values

Please note:

For outlined retain/release, I currently create a function per large type in IRGen, re-using the retain/releases from the value-witness table is less efficient than a direct call to a smaller function
We currently add new alloc_stack instructions and, in some cases, load of the function arguments to make this work. The current implementation, while it has some peephole optimizations for the most painful cases, is still very conservative: we can update this in the future, further reducing the code size
In case we had call to a function that takes a $BigType, and now takes $*BigType, there are cases wherein we create new alloc_stack and store instructions in order to call said function. Just like the point made above - the current implementation is very conservative / can be further optimized. this will be done in phases

The radar mentioned in this PR contains a test program called BigTypeProg: This program contains a big struct with some loadable types + a class / need for retain and release. Passes it around to different functions that access the struct.
Under -Onone we have 3X binary size reduction, under -O a 2X binary size reduction. The text area is reduced by around 84% and 65% respectively.

@swift-ci Please test

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - db2c59d7198ce3f56fe1396f96ca1ddbb73afd46
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - db2c59d7198ce3f56fe1396f96ca1ddbb73afd46
Test requested by - @shajrawi

@swift-ci Please smoke test

@swift-ci Please clean test

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - db2c59d7198ce3f56fe1396f96ca1ddbb73afd46
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - db2c59d7198ce3f56fe1396f96ca1ddbb73afd46
Test requested by - @shajrawi

@swift-ci Please test OS X Platform

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 6ac4f15e33750e1fc37c475f850cbec697268ec3
Test requested by - @shajrawi

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please test

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 6ac4f15e33750e1fc37c475f850cbec697268ec3
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 6ac4f15e33750e1fc37c475f850cbec697268ec3
Test requested by - @shajrawi

@swift-ci Please smoke test

@swift-ci Please test

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 824d976ff5447f18d68c1968d4571b745cba3bfd
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 824d976ff5447f18d68c1968d4571b745cba3bfd
Test requested by - @shajrawi

@swift-ci Please test

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - aa1cd4e9bc0200da12134977de6cfdb6e3748753
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - aa1cd4e9bc0200da12134977de6cfdb6e3748753
Test requested by - @shajrawi

@swift-ci Please test

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - b49489ac68f0e6b1204d13c09b111acc1be8e933
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - b49489ac68f0e6b1204d13c09b111acc1be8e933
Test requested by - @shajrawi

@swift-ci Please test

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 99266627f34290a9ae723b24265cab71465be710
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 99266627f34290a9ae723b24265cab71465be710
Test requested by - @shajrawi

@swift-ci Please smoke test

@swift-ci Please test

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 5c278743b4c90503b61f26e97087099f45246fd6
Test requested by - @shajrawi

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 5c278743b4c90503b61f26e97087099f45246fd6
Test requested by - @shajrawi

@swift-ci Please smoke test Linux

@swift-ci Please smoke test

The Linux failure here does not have anything to do with these changes.


Create aur repository

Created Arch User Repository.
https://aur.archlinux.org/packages/buildg-git
Arch users can install from aur with yay.

Looks like there is already a similar one (but Apache 2.0) https://aur.archlinux.org/packages/buildg

I saw it but it was invalidating some aur rules (like you should add -git suffix for git sourced repos) and it was outdated.


Unable to ping in 868Mhz band

Hi,
I have two nodes leader and child. There is no output from ping command in leader device. The ping command is successful. But I could see from network analyzer of Silabs IDE, the ping response is sent from leader and child node has replied for the ping response, see below image. In leader device in IP6::HandleDatagram(), HasUnicastAddress() has not passed after ICMP6 reply from child.
Reproduction Steps:
Node A.
channel 0
panid 0xacdb
ifconfig up
thread start
state -> leader
commissioner start
commissioner joiner add  PSK001
ping
Node B.
factoryreset
ifconfig up
joiner start PSK001
thread start
state -> child
ipaddr

router table
| ID | RLOC16 | Next Hop | Path Cost | LQ In | LQ Out | Age | Extended MAC     |
+----+--------+----------+-----------+-------+--------+-----+------------------+
| 16 | 0x4000 |            63 |               0 |         0 |         0 |      0 | ae1fd572816050e9 |
| 32 | 0x8000 |            63 |               0 |         3 |         3 |      0 | 321c723dfec3a1d7 |

Git commit id - 6fda0786edab8e821e1647cc4be08b4a7059cf4a
IEEE 802.15.4 hardware platform - EFR32MG12, BRD4163a Radio board
Build steps:
make -f examples/Makefile-efr32mg12 COMMISSIONER=1 COAP=1 COAPS=1 DTLS=1 JOINER=1 DNS_CLIENT=1 DHCP6_CLIENT=1 DHCP6_SERVER=1 BOARD=BRD4163A

what are the possibilities, where a ping command could fail?
Thanks,
Swathi

@SwathiRamachandra , the BRD4163A board is not a board that is currently included in our repo. I assume there is some custom platform code that you are running?
I noted that the Extended MAC address in the router table does not seem to match the MAC Src/Dest addresses in SiL Network Analyzer. In Thread, the link-local address should be derived from the Extended MAC address.
When processing the ICMPv6 Echo Reply, in Ip6::HandleDatagram(), can you indicate what the datagram's IPv6 Destination Address is? Also, can you provide the output of ipaddr on the Leader? That would allow us to check if there is a mismatch.

@jwhui Sorry, the above image was from older Thread network configuration and the Ext MAC was created with another Thread config. Sorry for the confusion.
Below I have created the Thread network again and the respective network analyzer image.
The datagram's destination address in ICMPv6 ECHO reply is fe80:0:0:0:5481:5727:1cd7:433b
ipaddr of Leader:
fdde:ad00:beef:0:0:ff:fe00:fc30
fdde:ad00:beef:0:0:ff:fe00:fc00
fdde:ad00:beef:0:0:ff:fe00:3000
fdde:ad00:beef:0:836d:6d50:d6ef:5a0a
fe80:0:0:0:3c2f:f3e3:c0ca:1b6c
Done
ipaddr of Child:
fdde:ad00:beef:0:0:ff:fe00:d400
fdde:ad00:beef:0:77a5:45ac:60e9:32d7
fe80:0:0:0:5481:5727:1cd7:433b
Done
router table
| ID | RLOC16 | Next Hop | Path Cost | LQ In | LQ Out | Age | Extended MAC     |
+----+--------+----------+-----------+-------+--------+-----+------------------+
| 12 | 0x3000 |       63 |         0 |     0 |      0 |   0 | 3e2ff3e3c0ca1b6c |
| 53 | 0xd400 |       63 |         0 |     3 |      3 |  26 | 568157271cd7433b |

Thanks,
Swathi

Given that the ICMPv6 Echo Reply's MAC Dest is 3E2FF3E3C0CA1B6C, I would expect the IPv6 Destination Address to be fe80:0:0:0:3c2f:f3e3:c0ca:1b6c. However, you indicated that the IPv6 Destination Address is fe80:0:0:0:5481:5727:1cd7:433b, which is the address assigned to the peer. In that case, the ICMPv6 Echo Reply won't be processed correctly.
Thread uses 6LoWPAN header compression. For link-local addresses, the IPv6 address is derived from the MAC Header on reception. So I'm confused as to why the IPv6 Dest Address is not matching the MAC Dest address. I'm not aware of any issues like this currently in the code base.

@jwhui Thanks for your help,
I am able to ping between the device in 868MHz band now, with the latest OpenThread commit.
Thanks,
Swathi


Can we allow customization of vimrc and/or tmux.conf based on installed packages?

Issue #5 only solved the problem for ~/.bash_profile but can this be implemented for the vimrc and tmux.conf as well?

This is not a high priority. Closing the issues for now.


Add samesite='None' to debugger cookie

This adds samesite='None' to debugger cookie so the debugger can be accessed in an iframe
When entering the debugger pin into the debugger when running in an iframe, Chrome states that "Because a cookie's SameSite attribute was not set or is invalid, it defaults to SameSite=Lax, which prevents the cookie from being set in a cross-site context."

fixes #1912

Looks good to me - given this is a debug cookie I think it can default to the original samesite behavior - will wait a bit to see if anyone disagrees and then merge.

@pgjones the only thing I could see is making it strict, which would be another layer of protection for the debugger. I don't think "run the debugger in an external iframe" is an intended use case for the debugger.
For now, though, keeping the current behavior seems fine.

SameSite=None is no longer valid without the Secure flag, which is only valid on HTTPS. We'll need to change this to Lax or Strict.


[INLONG-10409][Dashboard] Support installing agents by SSH key-based auth

Fixes #10409
Motivation
Implement SSH key-based authentication when installing the agent via SSH.   This PR will provide a dashboard UI that allows users to retrieve the SSH public key and then manually upload it to the agent node.
Modifications

Identification Type Addition: Introduced identify type to distinguish between password and SSH key authentication methods.
SSH Key Tips: Added tips to guide users on uploading the SSH authentication key when selecting the SSH key type.
Test Connection Feature: Implemented a 'Test Connection' button, allowing users to verify the correctness of their configuration settings.
Localization Enhancements: Enhanced descriptions for both Chinese (CN) and English (EN) localizations to improve user understanding and accessibility.

Verifying this change
(Please pick either of the following options)

[x] This change is already covered by existing tests, such as:
(please describe tests)
Successfully tested both SSH password type and SSH key type connections to the agent node. This verification confirms that the new identification types and the 'Test Connection' feature are functioning as expected.

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (not documented)

@wohainilaodou PTAL


add username and password options

Add options to config username and password.

In creating this, I noticed that this is a bad idea. It is a bad idea, is because it can be changed later in a second derivation, with guaranteed success via SSH, the same is true of disk size configuration. I have explained this before.
Adding more characters at this stage via QEMU key inputs is likely to increase failure rates, and if you feed invalid characters we won't be able to catch the error, and the build will just hang forever. Whereas if we allow it to be configured in a second stage we get proper errors that can be handled.


Improve Documentation

Add a small notice to the slack configuration that the channel name have to contain the '#'.

Thanks


Initialization of property in base class not recognized by sub class after redefinition

TypeScript Version:  3.7.3

Search Terms: extends, constructor, class, overriding, assigned, properties, jsdoc, documentation
I'm not sure if this is an actual bug as this is related to #1617 but is not exactly the same case.
I have a base class and a sub class which extends the base one. My goal is to override the JSDoc of one of the base's properties but keeping everything else intact. So I simply thought of declaring the exact same property on the subclass and change the JSDoc:
Code
class BaseClass {
    /**
     * Val in base
     */
    public val: number;
    constructor() {
        this.val = 5;
    }
}
class SubClass extends BaseClass {
    /**
     * Val in sub
     */
    public val: number;
    constructor() {
        super();
    }
}

Expected behavior: I'd expect everything to work since the constructor of the base class already sets val to 5 and is clearly called in the constructor.
Actual behavior: I get a Property 'val' has no initializer and is not definitely assigned in the constructor. ts(2564) in the sub class.
In a way, this makes sense because this error refers to val of the subclass, which isn't being assigned. But at runtime, everything will work fine (thanks to the prototype chain).
If this is intended, I'd like to know if there's any way to achieve the same result but without having to re-assign val inside the constructor. Essentially, I just want to replace the JSDoc of val in the sub class.
Playground Link: https://www.typescriptlang.org/play/?ts=Nightly#code/MYGwhgzhAEBCkFMDC4rQN4Cho+gegCoDtdoDoA1MEaASwDtoAjRE3AvN6ABwFcmQtYNABu1AFzR6vALZMEAJwDcXYAHt6EAC4LewLWoUAKAJQYupLQAtaEAHRiaAXmgBWFaQC+mb6EgwAZX4Uf2gEAA8tBHoAExh4CGRUGCxSQmJSMkpqOkYIfgsOLj4BIVEJKVl5ZVUNbV19Q1NzTNx87kVTD1xvTyA
Related Issues: #1617, #21775, #20911 (this issue might be a duplicate of one of these, sorry)

I also run into the same situation and look for a solution.
In contrast to @PandawanFr , I run into this situation because I want to apply a class property decorator to a parent class property.
Suppose I have a parent class Entity which has decorators applied to properties for validation. A child class TypeOrmEntity applies additional decorators to properties.
abstract class Entity {

  @IsString()
  @MaxLength(24)
  public id: string;

  constructor(id: string) {
    this.id = id;
  }
}

abstract class TypeOrmEntity extends Entity {

  @PrimaryColumn()
  public id: string; // strict property initialization warning

  constructor(id: string) {
    super(id);
  }
}

@PandawanFr — I don’t think it’s good practice to redeclare an inherited field on a subclass. To fix the error, you can remove the redeclaration. If the field is public/protected, you can still access it in the subclass.
class BaseClass {
    /**
     * Val in base
     */
    public val: number;
    constructor() {
        this.val = 5;
    }
}
class SubClass extends BaseClass {
    /**
     * no need to redeclare Val in sub
     */
    // [REMOVE] public val: number;
    constructor() {
        super();
    }
    test(): void {
        console.log(this.val); // still visible within subclass
    }
}

I know, but as stated in my original question, my goal is to “override the JSDoc of one of the base's properties but keeping everything else intact.”
I’m pretty sure the only way to change the JSDoc of a property in TS is to have it right before the property declaration, therefore I HAVE to redeclare it in the subclass.
Also, @rafaelkallis is encountering a similar issue, he wants to apply an extra decorator to a property that is defined in the base class.

You can write
    declare public val: number;

to do this

That solved it for me, thanks! (Sorry for bothering you with that). I’ll wait a bit before closing in case @rafaelkallis has something to say.

@RyanCavanaugh if that does what it looks like it does, I think I'm going to use this a lot.  Thanks!  I wrote a bunch here about how I couldn't find the docs for it but then I stumbled across this part of the 3.7 release notes.  👍


Add full type hover popup to VS Code commands

Several issues have been opened for this problem, but they have been closed as a duplicate of an issue which barely mentions the problem, let alone addresses it: https://github.com/microsoft/vscode/issues/64566
The closed issues are:

https://github.com/microsoft/vscode/issues/66405
https://github.com/microsoft/vscode/issues/76480
https://github.com/microsoft/vscode/issues/66405

The problem is simple. The three dots means you can't copy and paste definitions like you used to be able to. This is an absolutely essential feature because TypeScript isn't even close to being used everywhere, and the intellisense of Javascript files inside TypeScript files is way worse than inside Javascript files. This basically means the easiest way to create a d.ts file is to copy the type definition in the hover inside Javascript and then paste it into a d.ts file so TypeScript files can use it. It also helps me debug overloads and unions when things are going awry.
Even just a command that would popup the hover with the full definition instead of the abbreviated definition would be incredibly useful. Currently the shortcut is Ctrl+k, Ctrl+i for the abbreviated definition, so I take it there is a command behind it that can be typed into the command list (ctrl+shift+p). So even just adding the command for people to manually add a shortcut for would be really useful.
I can work around it, but I spend hours more time per week because of it.

@Arlen22 This is what #64566 is about: being able to expand the error to see more detail.Even in JS there are types that are too large to display (or infinite), so it is not possible to just always expand the ... eagerly.
Can you provide sample code that shows when the ... gets in the way

An infinite recursion should simply be marked as infinite (same color as never) indicating that some level of recursion is involved, I suppose, and the user could then end the recursive reference in some way in order to get a complete type output.
Normally the types I work with are not too large to display. If they are I can usually break them apart into smaller chunks in one way or another, either by making minor edits to the Javascript or by handwriting a Record, or stuff like that.
It's just that the ... prevents this from working at all, especially since I don't really know whether all ... are referring to the exact same type or what that type is. And then someone mentioned | ... and 4 more ... | in a string union, which obviously completely breaks copy and paste.

Can you provide some specific code examples?

Array<{
  "server_id": number; "server_uri": string; "itemType": string; "key": string;
  "title": string; "summary": string; "thumbnail": string; "videocover": string;
  "url": { "H264": string[]; "location": string; };
} | { "itemType": string; ... 7 more ...; "url"?: undefined; }>.shift(): {
    "server_id": number;
    "server_uri": string;
    "itemType": string;
    "key": string;
    "title": string;
    "summary": string;
    "thumbnail": string;
    "videocover": string;
    "url": {
        "H264": string[];
        "location": string;
    };
} | {
    "itemType": string;
    ... 7 more ...;
    "url"?: undefined;
}

ceTouchableFeedback<{
    onPress: () => void;
    object_id: number;
    name: "header" | "italic" | "bold" | "underline" | "repeat" | "link" | "search" | "image"
| "key" | "code" | "map" | "table" | "th" | "circle" | ... 771 more ... | "snowflake-o";
    style: {
        ...;
    };
}>(options: TouchableFeedbackOptions<...>, ...children: any[]): any (+1 overload)

The name field is obvious and useless and I would just remove the contents or change them to a named reference, but the generic inside options: TouchableFeedbackOptions gives me no information, and I critically need to know what the exact properties are being brought in.


tk: use aqua backend on darwin

Most users don’t have xquartz, so let’s use the default window system
for macOS.

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Successfully builds on my mac (Darwin 17.7.0, macOS 10.13.6), and I typed in a bit of a Tk sample app into wish, and it looked like it was working.


how to use rmate in different clients  with a serve

This is my problem: I remote the server with default configuration while my friend also remote the same server(we are different users), I am the first login in the server and I can rmate file in my vscode, when my friend rmate his file then the file unfold on my computer. That’s wield, how can I rmate my file on my computer while my friend rmate his file on his computer and how can I set the configuration?

That is happening because both of you are using the same port.

And excuse me , how could I solve this problem , should I set the port in the configuration which is after localhost or -R, or all these port?

To avoid any issue, is just easier to change all ports to a new one.

Thank you very much! I have already solve my problem with changing all the ports where it exists.


Render temporary html in PDF_DIR_PATH + removed encode() for Python3

Hello,
For my needs, I had to implement the suggestion from issue #2
At the same time, I had to remove the call to encode('utf-8') when using render_template to make it work on my Python 3.5 environment.
Best regards

Please cancel this pull request. I observed a regression on my windows environment, and wrongly reverted the commit. I will submit a proper Pull request once fully tested on my side. Sorry for the inconvenience.


Will there be any swift3 version?

The title says it all. I like your library, and I can convert it but just wondering if you have that in mind?

Hey,
Yes, I have. I will do this next week or you need this earlier?
Regards,

No rush, whenever you can. :bowing_man:

Done ;)

I am going to close the issue.


Implement antibody CRUD api

@bandrow since we can have more than entry for the same antibody id, what do we show when going to antibodyregistry.org/[ID]?

Hi @Filippo Ledda @.***> yes, we can have more than one
entry. We never explicitly mapped this case in the UI. A mistake on my part.
We should add the additional entries (and we can have multiples) as
secondary data about the AB_ID.
For the UI:
The core record should be visible, the other associated records should be
closed (invisible; with a note to "see other versions of this record" as an
option). Clicking on the "see other versions of this record" should open
all of the other versions; this can be pretty or just text, but all of
the fields should be there for each entry.
*How to choose the main record for display, if there are multiples. *
We have a column "of_record" that we don't use for anything any longer, but
was intended for this purpose.
We can reuse that field if that is helpful. You can wipe it and start
new, as I said we used it in the past, but haven't for a long time so the
data in the field is bad.
So you must choose one record to display as the "main record", but how?

set all of_record = 0
If there is more than 1 record where status = 'CURATED' for a given
ab_id, then choose the most recent curate_date set of_record = 1 else set
of_record = 0.

do you need me to write that SQL?
On Thu, Sep 15, 2022 at 6:27 AM Filippo Ledda @.***>
wrote:

@bandrow https://github.com/bandrow since we can have more than entry
for the same antibody id, what do we show when going to
antibodyregistry.org/[ID] http://antibodyregistry.org/[ID]?
—
Reply to this email directly, view it on GitHub
https://github.com/MetaCell/scicrunch-antibody-registry/issues/27#issuecomment-1248101676,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNSWNM7VLMMJSVWGF2RFMLV6MP45ANCNFSM575VKN4Q
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
All key biological entities deserve an #RRID!
orcid.org/0000-0002-5497-0243

Thanks @bandrow I think the logic is clear. If the of_record field is not updated in the current database I think we can just use the logic such as "the last curated" as you suggest without the need of maintaining an additional field.
I'm more concerned about the ui of this, as we didn't really took that into account in the design so need some loop back


Test OOM in evaluateMeasure tests

Describe the bug
A clear and concise description of what the bug is.
Component
Core library, SDC library, reference app, or SDC gallery app
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Smartphone (please complete the following information):

Device: [e.g. Pixel4a emulator]
Android version: [e.g. Settings -> About phone -> Android version]
Build number: [e.g. Settings -> About phone -> Build number]
Version [e.g. 22]

Additional context
Add any other context about the problem here.
Would you like to work on the issue?
Please state if this issue should be assigned to you or who you think could help to solve this issue.

@ktarasenko , did you get a chance to progress on this issue?

Yes, json parser is leaking memory. Finalizing the discussions on possible solutions in #1280

closing, tracking duplicated issue #1336


Make pip installable and add CLI entrypoints

This PR closes #5 by putting the code in a directory structure that's required by python packages, by adding a minimal setup.py/setup.cfg.
It also closes #6 by encapsulating the main functionality (that used to live under the if __name__ == '__main__' block) with functions, then by adding the [options.entry_points] header to setup.cfg.
I also added an example in the README on how to run the code on one of the examples presented in your paper. I'm a big fan of DeepWalk and related methods, and I appreciate the incredible work you did to push the theoretical side of it. I hope this PR is helpful for you, and many others!

How unfortunate that the authors have abandoned their work. Thank you for your efforts @cthoyt!


network disconnect every 3 minutes on: BIO_read returned a system error 110

As the title suggests the problem seems severe under the following conditions. On [VPN]:(https://github.com/yuezk/GlobalProtect-openconnect)
$ xfreerdp --version
This is FreeRDP version 2.8.1 (2.8.1)

xfreerdp /size:1200x800 /smart-sizing /u:USER /v:SERVER /cert-ignore +auto-reconnect /auto-reconnect-max-retries:2 -glyph-cache

[15:18:19:101] [48830:48831] [INFO][com.freerdp.channels.rdpsnd.client] - [static] Loaded fake backend for rdpsnd
[15:18:19:101] [48830:48831] [INFO][com.freerdp.channels.drdynvc.client] - Loading Dynamic Virtual Channel rdpgfx
[15:18:21:894] [48830:48831] [INFO][com.freerdp.client.x11] - Logon Error Info LOGON_FAILED_OTHER [LOGON_MSG_SESSION_CONTINUE]

[15:20:39:072] [48830:48831] [ERROR][com.freerdp.core.transport] - BIO_read returned a system error 110: Connection timed out

[15:20:39:072] [48830:48831] [ERROR][com.freerdp.core] - transport_read_layer:freerdp_set_last_error_ex ERRCONNECT_CONNECT_TRANSPORT_FAILED [0x0002000D]

[15:20:39:072] [48830:48831] [INFO][com.freerdp.client.common] - Network disconnect!
[15:20:39:072] [48830:48831] [INFO][com.freerdp.client.common] - Attempting reconnect (1 of 2)

[15:20:41:975] [48830:48831] [INFO][com.freerdp.channels.rdpsnd.client] - [static] Loaded fake backend for rdpsnd
[15:20:41:975] [48830:48831] [INFO][com.freerdp.channels.drdynvc.client] - Loading Dynamic Virtual Channel rdpgfx

This issue seems related wrt errno.

Normal operation, you have a slow network and the requests time out.
Try ```
/timeout:             Advanced setting for high latency links:
Adjust connection timeout, use if you
encounter timeout failures with your


Make System responsible for updating its own archetypes

Objective

Make it possible to use Systems outside of the scheduler/executor without having to define logic to track new archetypes and call System::add_archetype() for each.

Solution

Replace System::add_archetype(&Archetype) with System::update_archetypes(&World), making systems responsible for tracking their own most recent archetype generation the way that SystemState already does.

This has minimal (or simplifying) effect on most of the code with the exception of FunctionSystem, which must now track the latest ArchetypeGeneration it saw instead of relying on the executor to do it.

This was suggested in the discussion on https://discord.com/channels/691052431525675048/947636849139150868 with @alice-i-cecile and others - it turned out to be pretty simple, so sending this to see what folks think.

Instead of replacing the new_archetype method, can we just add your update_archetypes method?
Some users (and mocking up tests) might still want to register only specific archetypes.

Instead of replacing the new_archetype method, can we just add your update_archetypes method?
Some users (and mocking up tests) might still want to register only specific archetypes.

I'm not sure how common a use case that would be - it seems like it makes the API rather less clear to have both? The interaction between the two isn't really obvious. If there's a general consensus that it's likely to be useful then I can restore it, and just try to describe the behaviour more thoroughly, though.
The other comments all make sense; I agree that better naming would help here, especially if keeping both functions. To propose yet another combination, register_archetype and update_registered_archetypes might work? Though even register_archetype or similar might not be ideal - it's really only notifying the system about the existence of the archetype, since the SystemParam still actually decides whether the archetype is relevant or not.
Since it seems like there's general agreement that this is desirable for the API, I'll see about benchmarking it and adding test coverage.

Still need to work on tests/docs but wanted to see what folks think of this basic set of benchmarks, which don't show any difference outside of noise for me.

Some users (and mocking up tests) might still want to register only specific archetypes.

This seem extremely implausible to me. IMO if this is the best argument in favor of keeping it it should be cut for sure.

(also this is the first time I've actually used github PR code review so am not entirely sure of expectations around resolving comments etc)

wanted to see what folks think of this basic set of benchmarks, which don't show any difference outside of noise for me.

Could you post the benchmark results in the PR description? It's helpful context for reviewers: no change is good news!

(also this is the first time I've actually used github PR code review so am not entirely sure of expectations around resolving comments etc)

Resolve comments once the changes are addressed :) If there are interesting but unactionable comments / call-outs, keep those around so folks can see those on later review / just before merge.

Results of the benchmark without the change:
no_archetypes           time:   [39.805 us 39.956 us 40.119 us]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

added_archetypes        time:   [304.90 us 306.26 us 307.82 us]
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

and with the change:
no_archetypes           time:   [40.220 us 40.335 us 40.454 us]
                        change: [+0.2379% +0.6498% +1.0508%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

added_archetypes        time:   [307.27 us 308.21 us 309.15 us]
                        change: [+0.1588% +0.7505% +1.3797%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

I re-ran this on my giant desktop instead of my ultrabook and there things are less noisy, so it does become statistically significant.

Hm. Actually it's still pretty noisy on this machine too even with CPU frequency locked.

I figured there wouldn't be much of a change. We're still registering archetypes the same number of times. The method is just on the system now. Good to have some confirmation, though.

So with the updated benchmark here I'm actually seeing performance on added_archetypes with my change included being anything up to 10-15% faster, though the results are still pretty noisy and I'm not sure how to get criterion to generate a nice report with a comparison. So, hopefully it's at least no worse in reality?
I'm still thinking about tests to add here but am pretty happy with the form it's in now; thanks for the continued discussion all!

though the results are still pretty noisy and I'm not sure how to get criterion to generate a nice report with a comparison

critcmp is a pretty common and standard solution here.

Ah, I have a hypothesis for why the performance might have actually improved for large numbers of new archetypes - previously the update loop in the executor was iterating over all the new archetypes and passing each to every system in turn, but this is now iterating over the systems and having each deal with all the new archetypes, which seems likely to have better locality?

Updated the benchmark slightly to use iter_batched to properly separate setup and execution, which made it a little less noisy, and also poked the intel_pstate driver on my machine to get it to hopefully stop messing with clocks, and the results look like:
group                                     before                                 torne
-----                                     ------                                 -----
added_archetypes/archetype_count/100      1.34    334.5±3.64µs        ? ?/sec    1.00    250.2±4.17µs        ? ?/sec
added_archetypes/archetype_count/1000     1.86  1876.6±28.12µs        ? ?/sec    1.00  1006.8±14.60µs        ? ?/sec
added_archetypes/archetype_count/10000    1.58     23.9±0.29ms        ? ?/sec    1.00     15.1±0.20ms        ? ?/sec
added_archetypes/archetype_count/200      1.62   516.2±23.51µs        ? ?/sec    1.00   318.1±11.73µs        ? ?/sec
added_archetypes/archetype_count/2000     1.94      3.6±0.04ms        ? ?/sec    1.00  1842.7±37.29µs        ? ?/sec
added_archetypes/archetype_count/500      1.92  1021.5±66.94µs        ? ?/sec    1.00   532.3±13.40µs        ? ?/sec
added_archetypes/archetype_count/5000     1.82      9.7±0.13ms        ? ?/sec    1.00      5.4±0.10ms        ? ?/sec
no_archetypes/system_count/0              1.00      6.4±0.31µs        ? ?/sec    1.00      6.4±0.22µs        ? ?/sec
no_archetypes/system_count/100            1.01     90.7±1.35µs        ? ?/sec    1.00     90.1±0.98µs        ? ?/sec
no_archetypes/system_count/20             1.06     25.0±0.30µs        ? ?/sec    1.00     23.7±0.23µs        ? ?/sec
no_archetypes/system_count/40             1.02     42.0±1.18µs        ? ?/sec    1.00     41.4±0.54µs        ? ?/sec
no_archetypes/system_count/60             1.00     61.5±0.68µs        ? ?/sec    1.01     62.3±0.96µs        ? ?/sec
no_archetypes/system_count/80             1.01     77.3±0.74µs        ? ?/sec    1.00     76.4±0.92µs        ? ?/sec

So, on the set where no archetypes have changed the differences are all noise, and on the set where archetypes have changed my version appears to be a lot faster - anywhere from 30-90% improvement?
I also tried just flipping the parallel_executor loop around to test my hypothesis that the speed impact comes from the other order of iteration there, and that did improve performance, but only by 5-10%, much less than my full change.
Soooo that's interesting, and makes me want to write better tests to verify that this isn't producing the wrong results somehow, because it being that much faster seems surprising :)

@tornewuff nice work! I'm optimistic that these performance gains are real, although they're surprising to me too. Can you rebase this? I'd like to see if we can get this merged next week.

Yeah this being faster really surprises me. This moves O(NEW_ARCHETYPES) work from "once per frame" to "one per system", making this O(NEW_ARCHETYPES * SYSTEM_COUNT). In a "new archetypes are rare" world, this isn't terrible. But if we ever decide to enable "values can fragment archetypes" (ex: the old relations proposal), this type of overhead is something we need to consider seriously.
Part of me suspects that the "empty" system is letting rustc optimize out these updates entirely, as their side effects aren't being "used" anywhere. I think we should try adding a Query<&A::<0>>::len() to the benchmark system (which will run in constant + consistent time for each system, regardless of how many archetypes there are). We should also put a block_box call around the len() call. That way we "force" the systems to "actually observe" the results of the archetype updates. I have a relatively strong feeling that this will turn this "performance win" into a "performance loss", especially for LARGE_NUMBER_OF_SYSTEMS  X LARGE_NUMBER_OF_NEW_ARCHETYPES.

Correction : I was thinking of query.iter().len(), and that won't be constant time. It does work per archetype. Sorry for the confusion. We might need to expose a query.matched_archetype_count() function to make this "constant time" for the benchmark.

The change made in this PR should have the same complexity as before. It just permutes two for loops.
BEFORE
for archetype in new_archetypes.iter() {
    for system in systems.iter_mut() {
        // update system's archetype component access
    }
}

AFTER
for system in systems.iter_mut() {
    for archetype in new_archetypes.iter() {
        // update system's archetype component access
    }
}

I'm assuming "after" has better locality since now we only visit each system once instead of pointer-chasing between systems for each new archetype.

Woops yeah I was definitely wrong about the O(S*A) complexity change. That was an embarrassing brain fart. Thanks for bearing with me :smile:
It does increase the complexity slightly because we've moved world validation and generation tracking + range calculations to systems, but thats just a O(S) increase (and also wasn't what I was trying to call out in my previous comment ... thats overhead on a level im not sure its productive to worry about).
I do think we should still update the test to make it harder for rust to optimize out this stuff + increase confidence that this is a win (or an acceptable loss). But my confidence in this being "fine" is now high. Thanks again for the correction!

@tornewuff I'd like to prioritize this: could you rebase and resolve the merge conflicts? :)

Sorry - was too distracted to play with hobby stuff. I did rebase it but now it needs rebasing again :)
Also, I tried a benchmark where the system does something as suggested above and my version is still significantly faster, by a similar factor, even though the total amount of work being done is now much higher. This is.. pretty suspicious to be honest. I also already tried just flipping the loop as described above to see if it's just locality and while that does improve performance it improves it much less than my change.
I think I still need to write some more concrete test cases here to verify this is still doing what it's supposed to. Will see about that tonight if I can.

Well, with the simple unit test I added above it does look like things are working; if folks have suggestions for more things to test I can give it a shot?
While I'm still a little sceptical of the benchmark results being particularly meaningful, it doesn't appear to have broken anything? :)

OK; I'm considering this done. If folks have other suggestions I can take a look.

bors r+

bors r+

bors r+

Thanks; I didn't intend to leave the println calls in there but since they get swallowed during normal test runs I forgot :)


MZAppearance

There's no MZAppearance file in the project when I download it, so I get an error.

pod install


Support deleting the cluster from Rancher

kind/feature
What this PR does / why we need it:
This PR adds a support for deleting the CAPI cluster from Rancher, so that it is no longer listed in Rancher by introducing a
clusterImportedAnnotation.  The cluster creation and deletion then as following:

renames existing importAnnotationName to importAnnotation to align with new annotation
set clusterImportedAnnotation annotation on cluster creation
while cluster deletion, it checks for annotation and delete the cluster only if it has  clusterImportedAnnotation annotation
adds simple tests to check the deletion

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #7
Special notes for your reviewer:
Checklist:

[x] squashed commits into logical changes
[ ] includes documentation
[x] adds unit tests
[ ] adds or updates e2e tests

Shouldn't we delete Rancher cluster? CAPI will manage deletion itself.

and if we plan to do something custom on CAPI cluster deletion, we might need a finalizer to ensure that CAPI doesn't delete the cluster before we finish cleaning up.

Shouldn't we delete Rancher cluster? CAPI will manage deletion itself.

what do you mean by CAPI will manage deletion itself? we already have a rancher cluster delete in https://github.com/rancher-sandbox/rancher-turtles/blob/main/internal/rancher/cluster.go#L127-L139

@furkatgofurov7 If a user manually deletes CAPI cluster, CAPI controller will handle all deletion but what is rancher-turtles expected to do?

Based on the discussion in the meeting, the deletion could be handled like this:

Set the ownership on the Rancher cluster from CAPI cluster (CAPI owns Rancher) when we create it..
Set the annotation on Rancher cluster deletion to the CAPI cluster manifest.
If the annotation is present, don’t re-import the CAPI cluster next time. If the annotation is not present, cluster could be imported.

Use cases:

User manually deletes CAPI cluster. k/k garbage collection will remove the rancher cluster based on resource ownership.
User removes the rancher cluster from UI. Annotation is being set on CAPI cluster, preventing automatic re-import. If the annotation is removed, cluster will be reimported.

Potential things to add:
Alex: set the finalizer on CAPI cluster we create, to prevent automatic deletion. The finalizer will be unset by the operator during processing the deletion logic for CAPI cluster.

Based on the discussion in the meeting, the deletion could be handled like this:

Set the ownership on the Rancher cluster from CAPI cluster (CAPI owns Rancher) when we create it..
Set the annotation on Rancher cluster deletion to the CAPI cluster manifest.
If the annotation is present, don’t re-import the CAPI cluster next time. If the annotation is not present, cluster could be imported.

Use cases:

User manually deletes CAPI cluster. k/k garbage collection will remove the rancher cluster based on resource ownership.
User removes the rancher cluster from UI. Annotation is being set on CAPI cluster, preventing automatic re-import. If the annotation is removed, cluster will be reimported. CAPI cluster and underlying infrastructure is still present. (Externally Managed mode)

Potential things to add:
Alex: set the finalizer on CAPI cluster we create, to prevent automatic deletion. The finalizer will be unset by the operator during processing the deletion logic for CAPI cluster.

@Danil-Grigorev @alexander-demicev @richardcase  the current state of the PR should follow the above notes we have agreed offline, PTAL, thanks!


Rename master -> main

Would you be up for renaming the master branch to main? This would reduce cognitive load for devs (or at least for me), because most repos use main nowadays.

sure, let's do it!
@holgern would you create a main branch and set it as default?


tag job is failing for some Java repos

This behavior seems similar to what @Neenu1995 described yesterday:
Working set:
 * chore: release v1.3.0-alpha: https://github.com/googleapis/java-compute/pull/488
 * chore: release 0.118.1-SNAPSHOT: https://github.com/googleapis/java-compute/pull/485
 * chore: release 0.157.0: https://github.com/googleapis/java-cloud-bom/pull/2269
Processing chore: release v1.3.0-alpha: https://github.com/googleapis/java-compute/pull/488
command github-release failed with status 404
---------
GitHubAPIError: Not Found
    at GitHub.getFileContentsWithSimpleAPI (/tmpfs/src/github/releasetool/node_modules/release-please/build/src/github.js:1379:23)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async /tmpfs/src/github/releasetool/node_modules/release-please/build/src/github.js:385:24
    at async GitHub.getFileContentsOnBranch (/tmpfs/src/github/releasetool/node_modules/release-please/build/src/github.js:1372:20)
    at async GitHub.getFileContents (/tmpfs/src/github/releasetool/node_modules/release-please/build/src/github.js:1325:16)
    at async JavaYoshi.buildReleaseForVersion (/tmpfs/src/github/releasetool/node_modules/release-please/build/src/release-pr.js:338:36)
    at async GitHubRelease.createRelease (/tmpfs/src/github/releasetool/node_modules/release-please/build/src/github-release.js:35:25)
    at async GitHubRelease.run (/tmpfs/src/github/releasetool/node_modules/release-please/build/src/github-release.js:47:38)
CalledProcessError(1, ['npx', 'release-please', 'github-release', '--token=/tmp/tmp32nfzx7x', '--default-branch=master', '--release-type=java-yoshi', '--bump-minor-pre-major=true', '--repo-url=googleapis/java-compute', '--package-name=', '--debug'])

The tag jobs did fail yesterday evening for java-compute(v1.3.0), java-cloud-bom(v0.157.0) and one other repo. I manually tagged all these releases. Hence these jobs should be cancelled and not retriggered.
release 0.118.1-SNAPSHOT for java compute is a SNAPSHOT release and does not require tagging.
Also, the failure in tag jobs could not have been caused by any java specific issue and is likely caused by the kokoro periodic job failure.

@chingor13 Please can you take a look? We are out of SLO.


add support for application_insights_web_test

Issue-id
PR Checklist

[x] I have added example(s) inside the [./examples/] folder
[x] I have added the example(s) to the integration test list for normal (~30 minutes) or long runner >30 minutes
[x] I have checked the coding conventions as per the wiki
[x] I have checked to ensure there aren't other open Pull Requests for the same update/change?

Description

Adding support for Application insights web test -> https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/application_insights_web_test

This requires https://github.com/aztfmod/terraform-provider-azurecaf/pull/211
Does this introduce a breaking change

[ ] YES
[x] NO

Testing

thanks for your contribution to Microsoft CAF @sschne!


relevant_tests_of_loaded_classes

From the fork, https://github.com/speezepearson/defects4j, in framework/projects/, these is the folder, relevant_tests_of_loaded_classes, what does it mean? https://github.com/rjust/defects4j does not have it.

If you are asking about a different fork, then I suggest that you ask on that fork rather than this one.