mostly silent failure in uvcvideo patch process

Camera Model: R200
Firmware Version: <IP_ADDRESS>
Operating System & Version: Linux 14.04.04 x86_64
Kernel Version (Linux Only): 4.4
I am only able to visualize the rbg camera stream. I don't see depth and IR streams. When I run cpp-multicam, this is the terminal output
xxx@xxx:~/librealsense/bin$ ./cpp-multicam
Starting Intel RealSense R200... done.
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 1 bad magic number 0x0
...
[this warning keeps repeating]
I am able to visualize the IR and depth streams using mplayer. So my realsense device is working correctly. Something should be going wrong with the driver.
Is it a syncing issue that makes the driver drop frames?

Hi, I strongly recommend using a Windows machine to update your firmware. <IP_ADDRESS> is the latest and 1.0.71.xx has known issues. Installing the DCM on a Windows 10 machine with a camera plugged in will update the firmware. If you don't have Win10 handy, I can dig up the utility for Win 8.1 (seems like they recently removed it!).

@prarobo so I accidentally replicated this issue today installing a new machine and saw this exact behavior (RGB only, bad magic numbers for all streams) and it was because I forgot to run the 4.4 uvcvideo patching script:
./scripts/patch-uvcvideo-4.4.sh v4.4-wily

@ddiakopoulos Thanks for your response. I updated the firmware to <IP_ADDRESS>. I also reapplied the uvcvideo patch. I am still seeing the same error. Bad magic number and no depth stream.
Any thoughts? I am able to see the individual streams (color, IR and depth through mplayer) but cpp-multicam shows only color stream.

@prarobo what about cpp-capture with a single camera?

Only color image shows, no depth image when running cpp-capture. This is the terminal output.
Capturing DEPTH at 480 x 360, fov = 55.1 x 42.8, distortion = NONE
Capturing COLOR at 640 x 480, fov = 54.5 x 41.8, distortion = MODIFIED_BROWN_CONRADY
Capturing INFRARED at 480 x 360, fov = 55.1 x 42.8, distortion = NONE
Capturing INFRARED2 at 480 x 360, fov = 55.1 x 42.8, distortion = NONE
rs.warn: Subdevice 0 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x20001c
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0xf000e
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x12000d
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x150012
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0xd000d
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x110012
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0xe0013
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x130013
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x120010
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0xf000e
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x11000e
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x110011
......
[repeats]

This is the exact behavior of a system that hasn't been correctly patched. I'm not sure the script is successfully being applied on your system. Can you run the following commands and post your dmesg output?
sudo modprobe -r uvcvideo
dmesg (only copy relevant lines here, should notate that you removed the kernel object)
sudo modprobe uvcvideo
dmesg (should let you know that it re-inserted correctly)
Also after streaming, could you most the last few lines of dmesg? Unknown frame formats are also a telltale sign.

This what I am seeing from dmesg (see below). I am seeing an unknown video format. I am applying the patch the though. Is there a way to check if the patch is getting applied?
[  368.323530] usb 4-3.3: new SuperSpeed USB device number 5 using xhci_hcd
[  368.342214] usb 4-3.3: New USB device found, idVendor=8086, idProduct=0a80
[  368.342217] usb 4-3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  368.342218] usb 4-3.3: Product: Intel RealSense 3D Camera R200
[  368.342220] usb 4-3.3: Manufacturer: Intel Corp
[  368.342221] usb 4-3.3: SerialNumber: SN_2191301595
[  368.344361] uvcvideo: Unknown video format 49323159-0000-0010-8000-00aa00389b71
[  368.344363] uvcvideo: Unknown video format 20493859-0000-0010-8000-00aa00389b71
[  368.344368] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[  368.347695] uvcvideo: Unknown video format 2036315a-0000-0010-8000-00aa00389b71
[  368.347699] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[  368.348934] uvcvideo: Unable to create debugfs 4-5 directory.
[  368.349007] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb4/4-3/4-3.3/4-3.3:1.2/input/input40
[  368.351038] uvcvideo: Unknown video format 30315752-0000-0010-8000-00aa00389b71
[  368.351041] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[  368.352331] uvcvideo: Unable to create debugfs 4-5 directory.
[  368.352401] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb4/4-3/4-3.3/4-3.3:1.4/input/input41

You need to monitor what is failing in the patch script. The patch is not being applied correctly. When the script is finished, it should copy a new version of uvcvideo.ko to your system directory and then you manually re-load it with the sudo modprobe uvcvideo command. The output above is definitely from an unpatched version which explains all of your issues.

Hello, I have the same problem, executing cpp-capture I only see rgb image:
vigus@NUC-VIGUS1:~/librealsense-master/bin$ ./cpp-capture
Capturing DEPTH at 480 x 360, fov = 55.5 x 43.1, distortion = NONE
Capturing COLOR at 640 x 480, fov = 53.8 x 41.3, distortion = MODIFIED_BROWN_CONRADY
Capturing INFRARED at 480 x 360, fov = 55.5 x 43.1, distortion = NONE
Capturing INFRARED2 at 480 x 360, fov = 55.5 x 43.1, distortion = NONE
rs.warn: Subdevice 0 bad magic number 0xf0006
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0xf0014
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x130015
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x150013
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x26001e
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x400034
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x3d0035
rs.warn: Subdevice 1 bad magic number 0x0
rs.warn: Subdevice 0 bad magic number 0x3f0036
.
.
.
I have applied several times the patch uvcvideo-4.4.sh v4.4-wily and I haven't resolved the issue. The dmesg info:
vigus@NUC-VIGUS1:~/librealsense-master/bin$ sudo dmesg | tail -n 50
[ 1235.233618] usb 3-1: Product: Intel RealSense 3D Camera R200
[ 1235.233620] usb 3-1: Manufacturer: Intel Corp
[ 1235.233622] usb 3-1: SerialNumber: SN_2431013021
[ 1235.235958] uvcvideo: Unknown video format 49323159-0000-0010-8000-00aa00389b71
[ 1235.235961] uvcvideo: Unknown video format 20493859-0000-0010-8000-00aa00389b71
[ 1235.235964] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1235.239232] uvcvideo: Unknown video format 2036315a-0000-0010-8000-00aa00389b71
[ 1235.239235] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1235.240600] uvcvideo: Unable to create debugfs 3-3 directory.
[ 1235.240673] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.2/input/input19
[ 1235.242552] uvcvideo: Unknown video format 30315752-0000-0010-8000-00aa00389b71
[ 1235.242554] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1235.243971] uvcvideo: Unable to create debugfs 3-3 directory.
[ 1235.244042] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.4/input/input20
[ 1323.356012] uvcvideo: Non-zero status (-71) in video completion handler.
[ 1323.368013] uvcvideo: Non-zero status (-71) in video completion handler.
[ 1724.867882] usbcore: deregistering interface driver uvcvideo
[ 1748.602913] media: Linux media interface: v0.10
[ 1748.606820] Linux video capture interface: v2.00
[ 1748.699088] uvcvideo: Unknown video format 49323159-0000-0010-8000-00aa00389b71
[ 1748.699091] uvcvideo: Unknown video format 20493859-0000-0010-8000-00aa00389b71
[ 1748.699095] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1748.701889] uvcvideo: Unknown video format 2036315a-0000-0010-8000-00aa00389b71
[ 1748.701892] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1748.703314] uvcvideo: Unable to create debugfs 3-3 directory.
[ 1748.703446] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.2/input/input21
[ 1748.704778] uvcvideo: Unknown video format 30315752-0000-0010-8000-00aa00389b71
[ 1748.704782] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1748.706203] uvcvideo: Unable to create debugfs 3-3 directory.
[ 1748.706262] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.4/input/input22
[ 1748.707193] usbcore: registered new interface driver uvcvideo
[ 1748.707195] USB Video Class driver (1.1.1)
[ 1808.188787] usb 3-1: USB disconnect, device number 3
[ 1817.500891] usb 3-2: new SuperSpeed USB device number 4 using xhci_hcd
[ 1817.519540] usb 3-2: New USB device found, idVendor=8086, idProduct=0a80
[ 1817.519544] usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1817.519546] usb 3-2: Product: Intel RealSense 3D Camera R200
[ 1817.519548] usb 3-2: Manufacturer: Intel Corp
[ 1817.519550] usb 3-2: SerialNumber: SN_2431013021
[ 1817.521902] uvcvideo: Unknown video format 49323159-0000-0010-8000-00aa00389b71
[ 1817.521906] uvcvideo: Unknown video format 20493859-0000-0010-8000-00aa00389b71
[ 1817.521911] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1817.525353] uvcvideo: Unknown video format 2036315a-0000-0010-8000-00aa00389b71
[ 1817.525357] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1817.526752] uvcvideo: Unable to create debugfs 3-4 directory.
[ 1817.526864] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.2/input/input23
[ 1817.528755] uvcvideo: Unknown video format 30315752-0000-0010-8000-00aa00389b71
[ 1817.528758] uvcvideo: Found UVC 1.10 device Intel RealSense 3D Camera R200 (8086:0a80)
[ 1817.530175] uvcvideo: Unable to create debugfs 3-4 directory.
[ 1817.530243] input: Intel RealSense 3D Camera R200 as /devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.4/input/input24
The patch results:
vigus@NUC-VIGUS1:~/librealsense-master$ ./scripts/patch-uvcvideo-4.4.sh v4.4-wily
--2016-03-19 17:12:13--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/SOURCES
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 231
Grabando a: “SOURCES.2”
100%[===============================================>] 231         --.-K/s   en 0s
2016-03-19 17:12:14 (38,7 MB/s) - “SOURCES.2” guardado [231/231]
Shallow cloning Linux source repository... (~100mb)
Clonar en «linux-v4.4-wily»...
remote: Counting objects: 55383, done.
remote: Compressing objects: 100% (51945/51945), done.
remote: Total 55383 (delta 6436), reused 22448 (delta 2638)
Receiving objects: 100% (55383/55383), 148.99 MiB | 1.39 MiB/s, done.
Resolving deltas: 100% (6436/6436), done.
Checking connectivity... hecho.
Note: checking out 'afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
Checking out files: 100% (52221/52221), done.
--2016-03-19 17:14:39--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: no especificado [text/html]
Grabando a: “index.html”
[ <=>                                            ] 9.003       --.-K/s   en 0,005s

2016-03-19 17:14:39 (1,73 MB/s) - “index.html” guardado [9003]
--2016-03-19 17:14:39--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/linux-headers-4.4.0-040400_4.4.0-040400.201601101930_all.deb
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 9758244 (9,3M) [application/x-debian-package]
Grabando a: “linux-headers-4.4.0-040400_4.4.0-040400.201601101930_all.deb”
100%[===============================================>] 9.758.244   1,40MB/s   en 6,9s
2016-03-19 17:14:46 (1,35 MB/s) - “linux-headers-4.4.0-040400_4.4.0-040400.201601101930_all.deb” guardado [9758244/9758244]
--2016-03-19 17:14:46--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/linux-headers-4.4.0-040400-generic_4.4.0-040400.201601101930_amd64.deb
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 741242 (724K) [application/x-debian-package]
Grabando a: “linux-headers-4.4.0-040400-generic_4.4.0-040400.201601101930_amd64.deb”
100%[===============================================>] 741.242      981KB/s   en 0,7s
2016-03-19 17:14:47 (981 KB/s) - “linux-headers-4.4.0-040400-generic_4.4.0-040400.201601101930_amd64.deb” guardado [741242/741242]
(Leyendo la base de datos ... 331704 ficheros o directorios instalados actualmente.)
Preparing to unpack linux-headers-4.4.0-040400_4.4.0-040400.201601101930_all.deb ...
Unpacking linux-headers-4.4.0-040400 (4.4.0-040400.201601101930) over (4.4.0-040400.201601101930) ...
Preparing to unpack linux-headers-4.4.0-040400-generic_4.4.0-040400.201601101930_amd64.deb ...
Unpacking linux-headers-4.4.0-040400-generic (4.4.0-040400.201601101930) over (4.4.0-040400.201601101930) ...
Configurando linux-headers-4.4.0-040400 (4.4.0-040400.201601101930) ...
Configurando linux-headers-4.4.0-040400-generic (4.4.0-040400.201601101930) ...
The Branch Is:  v4.4
HEAD se encuentra en afd2ff9... Linux 4.4
--2016-03-19 17:14:51--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/0001-base-packaging.patch
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 18903697 (18M) [text/x-diff]
Grabando a: “0001-base-packaging.patch”
100%[===============================================>] 18.903.697  1,40MB/s   en 13s
2016-03-19 17:15:04 (1,37 MB/s) - “0001-base-packaging.patch” guardado [18903697/18903697]
--2016-03-19 17:15:04--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/0002-debian-changelog.patch
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
Petición HTTP enviada, esperando respuesta... 200 OK
Longitud: 426590 (417K) [text/x-diff]
Grabando a: “0002-debian-changelog.patch”
100%[===============================================>] 426.590      740KB/s   en 0,6s
2016-03-19 17:15:05 (740 KB/s) - “0002-debian-changelog.patch” guardado [426590/426590]
--2016-03-19 17:15:05--  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-wily/0003-configs-based-on-Ubuntu-4.4.0-0.10.patch
Resolviendo kernel.ubuntu.com (kernel.ubuntu.com)... <IP_ADDRESS>
Conectando con kernel.ubuntu.com (kernel.ubuntu.com)[<IP_ADDRESS>]:80... conectado.
 I believe this is identical to the error found here: https://github.com/intel-ros/realsense/issues/12#issuecomment-198079546
This is where it's failing: https://github.com/IntelRealSense/librealsense/blob/master/scripts/patch-uvcvideo-4.4.sh#L50. You'll need manually execute the rest of the steps in a separate shell in order to complete the patch process. Unfortunately you'll need to unwind some of the variables like KBASE, but you could run the script once again and add some print statements to see what they are if you are having trouble.

Problem solved, many thanks for your time.

I solved my problem by using the uvcvideo.ko file generated from a different identical system. I just copied and pasted this into the appropriate location. Do you see any caveats in this approach @ddiakopoulos ? Will I run into problems if I do this for several robots?

@prarobo should be fine so long as the uname r strings are **absolutely **identical. Minor kernel patch versions will cause the kernel to reject the ko. Otherwise you're fine!

I had the same problem. Fixed it by deleting lines 50 & 51 in the patch_uvcvideo-4.4.sh script and used the following instead:
CONFIG_LOCATION=/usr/src/linux-headers-$(uname -r)

@bsbc I integrated this change in a new commit: 76b33a02640e6adab496238574af07be6855dbe1

Hi, I was having some problems with the camera but thanks to this issue and to this other issue I was able to fix it. However, I wonder why the changes that fix the problem in the script have been undone.
Thanks!

When you remove the lines 50 & 51 then the two files are not copied i.e. missing.
CONFIG_LOCATION=/usr/src/linux-headers-$(uname -r)
Is needed, though.
Thomas

Hi, I'm running into a similar problem with the SR300: modprobe reveals repeated warnings involving uvcvideo, and if I try and start up a window with a stream (eg. using cpp-tutorial-2-streams) neither colour, depth or IR come up at all (not even a black window*). I'm using the Video4Linux backend, to minimize potential issues. I tried both the 4.4 kernel script and the mainline script patch, on clean installs, but had the same problem on both.
Since it seems related, I thought I'd post here as it could well be a patch error, given the uvcvideo warnings. However none of the fixes listed here seem to work - let me know if it's more appropriate to start a new issue.
Changing the patch script as suggested above simply throws up more errors. I ran the SR300 live unit tests script and it reported no problems ("all tests passed), however a further modprobe of uvc after running the script showed multiple "uvcvideo: Non-zero status" and "uvcvideo: Failed to query UVC control" errors (modprobe output copied below, sorry for the length).

I suppose there is a slight chance this is actually a glfw problem, since the window output seems problematic in a different way to that reported here and elsewhere. However I ran a short glfw test program to check this, and had no issues.

I should note that I have previously set up this same SR300 using identical hardware and an identical installation process, and it worked fine (still does!). Completely baffled at this point.
[  213.505694] uvcvideo: Found UVC 1.10 device Intel(R) RealSense(TM) Camera SR300 (8086:0aa5) [  213.507160] input: Intel(R) RealSense(TM) Camera S as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2.1/2-2.1:1.0/input/input16 [  213.508672] uvcvideo: Unknown video format 435a4e49-b602-480f-978c-e4e88ae89b89 [  213.508677] uvcvideo: Unknown video format 52494150-8536-4841-b6bf-8fc6ffb083a8 [  213.508750] uvcvideo: Found UVC 1.10 device Intel(R) RealSense(TM) Camera SR300 (8086:0aa5) [  213.509571] uvcvideo: Unable to create debugfs 2-3 directory. [  213.509727] input: Intel(R) RealSense(TM) Camera S as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2.1/2-2.1:1.2/input/input17 [  213.510639] usbcore: registered new interface driver uvcvideo [  213.510642] USB Video Class driver (1.1.1) [  221.259752] uvcvideo: Non-zero status (-71) in video completion handler. [  223.558650] uvcvideo: Non-zero status (-71) in video completion handler. [  237.014474] uvcvideo: Non-zero status (-71) in video completion handler. [  239.620868] uvcvideo: Non-zero status (-71) in video completion handler. [  241.937123] uvcvideo: Non-zero status (-71) in video completion handler. [  245.509028] uvcvideo: Non-zero status (-71) in video completion handler. [  249.089001] uvcvideo: Non-zero status (-71) in video completion handler. [  296.357967] uvcvideo: Non-zero status (-71) in video completion handler. [  298.981624] uvcvideo: Non-zero status (-71) in video completion handler. [  301.293752] uvcvideo: Non-zero status (-71) in video completion handler. [  304.878369] uvcvideo: Non-zero status (-71) in video completion handler. [  308.452756] uvcvideo: Non-zero status (-71) in video completion handler. [  310.783042] uvcvideo: Non-zero status (-71) in video completion handler. [  317.040678] uvcvideo: Non-zero status (-71) in video completion handler. [  323.271887] uvcvideo: Non-zero status (-71) in video completion handler. [  329.518868] uvcvideo: Non-zero status (-71) in video completion handler. [  335.655802] uvcvideo: Non-zero status (-71) in video completion handler. [  341.829035] uvcvideo: Non-zero status (-71) in video completion handler. [  341.834817] uvcvideo: Non-zero status (-71) in video completion handler. [  344.605358] uvcvideo: Non-zero status (-71) in video completion handler. [  344.611159] uvcvideo: Non-zero status (-71) in video completion handler. [  347.050978] uvcvideo: Non-zero status (-71) in video completion handler. [  347.058422] uvcvideo: Non-zero status (-71) in video completion handler. [  349.509821] uvcvideo: Non-zero status (-71) in video completion handler. [  349.520519] uvcvideo: Non-zero status (-71) in video completion handler. [  351.976343] uvcvideo: Non-zero status (-71) in video completion handler. [  351.982767] uvcvideo: Non-zero status (-71) in video completion handler. [  354.405937] uvcvideo: Non-zero status (-71) in video completion handler. [  354.411688] uvcvideo: Non-zero status (-71) in video completion handler. [  356.900522] uvcvideo: Non-zero status (-71) in video completion handler. [  360.453683] uvcvideo: Non-zero status (-71) in video completion handler. [  362.344638] uvcvideo: Non-zero status (-71) in video completion handler. [  363.973986] uvcvideo: Non-zero status (-71) in video completion handler. [  371.600058] uvcvideo: Non-zero status (-71) in video completion handler. [  371.607259] uvcvideo: Non-zero status (-71) in video completion handler. [  374.130385] uvcvideo: Non-zero status (-71) in video completion handler. [  374.140093] uvcvideo: Non-zero status (-71) in video completion handler. [  385.937575] uvcvideo: Non-zero status (-71) in video completion handler. [  385.944819] uvcvideo: Non-zero status (-71) in video completion handler. [  386.957043] uvcvideo: Non-zero status (-71) in video completion handler. [  386.963308] uvcvideo: Non-zero status (-71) in video completion handler. [  398.282402] uvcvideo: Non-zero status (-71) in video completion handler. [  399.281976] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 6: -32 (exp. 1). [  399.347338] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 6: -32 (exp. 1). [  399.399697] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 6: -32 (exp. 1). [  399.640214] uvcvideo: Non-zero status (-71) in video completion handler. [  400.638965] uvcvideo: Failed to query (GET_CUR) UVC control 2 on unit 6: -32 (exp. 1). [  400.690542] uvcvideo: Failed to query (GET_CUR) UVC control 2 on unit 6: -32 (exp. 1). [  400.742860] uvcvideo: Failed to query (GET_CUR) UVC control 2 on unit 6: -32 (exp. 1). [  400.795269] uvcvideo: Failed to query (GET_CUR) UVC control 2 on unit 6: -32 (exp. 1). [  401.034500] uvcvideo: Non-zero status (-71) in video completion handler. [  402.032581] uvcvideo: Failed to query (GET_CUR) UVC control 3 on unit 6: -32 (exp. 1). [  402.084033] uvcvideo: Failed to query (GET_CUR) UVC control 3 on unit 6: -32 (exp. 1). [  402.135683] uvcvideo: Failed to query (GET_CUR) UVC control 3 on unit 6: -32 (exp. 1). [  402.187753] uvcvideo: Failed to query (GET_CUR) UVC control 3 on unit 6: -32 (exp. 1). [  402.239991] uvcvideo: Failed to query (GET_CUR) UVC control 3 on unit 6: -32 (exp. 1). [  402.293057] uvcvideo: Failed to query (GET_CUR) UVC control 3 on unit 6: -32 (exp. 1). [  402.604806] uvcvideo: Non-zero status (-71) in video completion handler. [  403.601853] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  403.653161] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  403.704155] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  403.755393] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  403.807400] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  403.858431] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  403.909868] uvcvideo: Failed to query (GET_CUR) UVC control 5 on unit 6: -32 (exp. 1). [  404.104304] uvcvideo: Non-zero status (-71) in video completion handler. [  405.104474] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 6: -32 (exp. 1). [  405.875166] uvcvideo: Non-zero status (-71) in video completion handler. [  408.059490] uvcvideo: Non-zero status (-71) in video completion handler. [  408.068448] uvcvideo: Non-zero status (-71) in video completion handler. [  410.359062] uvcvideo: Non-zero status (-71) in video completion handler. [  410.366652] uvcvideo: Non-zero status (-71) in video completion handler. [  412.788960] uvcvideo: Non-zero status (-71) in video completion handler. [  412.797656] uvcvideo: Non-zero status (-71) in video completion handler. [  415.275198] uvcvideo: Non-zero status (-71) in video completion handler. [  415.411975] uvcvideo: Non-zero status (-71) in video completion handler.

For whatever reason, the patch isn't being applied correctly on your system (unknown video formats are a telltale sign). Follow the script closely, and don't forget to sudo modprobe uvcvideo at the end

I tried again on a completely clean Ubuntu 14.04 install, to make sure I hadn't made an error in the process. The last few lines of dmesg after patch application and modprobe is
[  241.789686] uvcvideo: module verification failed: signature and/or required key missing - tainting kernel
[  241.790724] usbcore: registered new interface driver uvcvideo
[  241.790729] USB Video Class driver (1.1.1)

update: tried on a completely clean Ubuntu 14.04 install on a different machine - no disturbing messages from uvcvideo in dmesg, and the camera seems to be working. No doubt I did something careless during the patch application on the previous system, I will roll back (again!) and see if I can get it working.

Further update: re-installing Ubuntu and running the installation procedure exactly as on the first and third machine (which both work fine) still produces no useable image.
After applying the patch, the modprobe reveals nothing unusual (except the module verification failure, but this seems to happen whether or not the patch succeeds as I get the same message on other machines)
[  351.929478] uvcvideo: module verification failed: signature and/or required key missing - tainting kernel
[  351.931636] usbcore: registered new interface driver uvcvideo
[  351.931643] USB Video Class driver (1.1.1)
(..)
After plugging in the realsense and running cpp-tutorials-2-streams, I get nothing at all in the glfw window and running modprobe produces the following dmesg output:
[  662.018483] uvcvideo: Unknown video format 435a4e49-b602-480f-978c-e4e88ae89b89
[  662.018490] uvcvideo: Unknown video format 52494150-8536-4841-b6bf-8fc6ffb083a8
[  662.018561] uvcvideo: Found UVC 1.10 device Intel(R) RealSense(TM) Camera SR300 (8086:0aa5)
[  662.019470] uvcvideo: Unable to create debugfs 2-3 directory.
[  662.019700] input: Intel(R) RealSense(TM) Camera S as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2.1/2-2.1:1.2/input/input17
So at this point it looks like there's something more fundamental going on, and the problem is not with the patch itself. I am getting a weirdly persistent grub screen on boot, which might mean there's another kernel installed, although I am using the same installation USB as on the working machines, and when I reinstalled ubuntu I erased the disk. I will check the partitions and try again I suppose.

when running all patches, etc... I got the same issue in msg:
[ 351.929478] uvcvideo: module verification failed: signature and/or required key missing - tainting kernel

I too have the error
uvcvideo: Non-zero status (-71) in video completion handler.
in dmesg. I also had an issue with /dev/video0 permissions (remedied by adding the user to the group 'video').
Without the permissions fix, the error behavior was "no device detected". After the permissions fix, I get the -71 error in dmesg when I start up any camera-using software (eg. cpp-headless), and the infrared and depth images appear to be all zeroes.
Perhaps the most interesting thing here is that I had been using the camera fine the previous day, then after a reboot of the system both of these problems arose. There may have been an apt-get upgrade before the reboot - is that expected to break the driver?

Rerunning the patch script did not fix the issue described above, but rerunning it, telling it 'yes' when it asks a question about the patch already being applied that means "do you want to unpatch instead of patch" (I forget how it's phrased, but not very helpfully!), then rerunning it a second time, allows the script to run to completion, which fixed my issue.
I think the apt-get upgrade was the cause of the problem - it continues to work after a subsequent reboot.

mine, it's still not working!

I also had the issue of unknown video format after running the patching script and loading the uvcvideo module.
It turns out that the script was not running successfully for me even though no errors were shown. This was due to the 'realsense-camera-format patch' having a relative address (line 47). Re-running the script from the 'librealsense' dir rather than 'librealsense/scripts' fixed this issue for me.

I also had the issue of unknown video format after running the patching script and loading the uvcvideo module.
It turns out that the script was not running successfully for me even though no errors were shown. This was due to the 'realsense-camera-format patch' having a relative address (line 47) and was not being found. Re-running the script from the 'librealsense' dir rather than 'librealsense/scripts' fixed this issue for me.

I did everything from the begining in another folder and I always get this when trying to run any example:
rs_error was raised when calling rs_create_context(api_version:11201):
UVCIOC_CTRL_QUERY:UVC_GET_CUR error 56, Invalid request code


The behavior of if...else... and ? : is inconsistent.

Phenomena:

Expected:
Both Case 1 and Case 2 conform to static check.
Actual:
Only Case 2 conforms to static check.
Complete sample code:
abstract class IUnit {}
abstract class IGroupableUnit extends IUnit {}
abstract class ILayoutableUnit extends IGroupableUnit {}
abstract class IGroup extends IUnit {}
abstract class IDevice extends IGroupableUnit {}
abstract class IRxDevice extends IDevice {}

abstract class IRepository<E> {}
abstract class IUnitRepository<E extends IUnit> extends IRepository<E> {}
abstract class IGroupableRepository<E extends IGroupableUnit, G extends IGroupRepository> extends IUnitRepository<E> {}
mixin ILayoutableRepository<E extends ILayoutableUnit, G extends IGroupRepository> on IGroupableRepository<E, G> {}
abstract class IGroupRepository<E extends IGroup> extends IUnitRepository<E> {}
abstract class IDeviceRepository<E extends IDevice, G extends IGroupRepository> extends IGroupableRepository<E, G> {}
abstract class IRxDeviceRepository<E extends IRxDevice, G extends IGroupRepository> extends IDeviceRepository<E, G> {}
abstract class IMRxRepository<E extends MRxDevice, G extends IGroupRepository> extends IRxDeviceRepository<E, G> with ILayoutableRepository<E, G> {}

class RxDevice extends IRxDevice {}
class MRxDevice extends IRxDevice implements ILayoutableUnit {}

class OutputGroup extends IGroup {}

class GroupRepository<E extends IGroup> extends IGroupRepository<E> {}
class RxRepository<E extends IRxDevice> extends IRxDeviceRepository<E, GroupRepository<OutputGroup>> {}
class MRxRepository<E extends MRxDevice> extends IMRxRepository<E, GroupRepository<OutputGroup>> {}

void main() {
  RxRepository rxRepository = RxRepository();
  MRxRepository mRxRepository = MRxRepository();
  IRxDevice device = RxDevice();

  // Case 1
  IRxDeviceRepository repository;
  repository = device is MRxDevice ? mRxRepository : rxRepository;
  print(repository.runtimeType);

  // Case 2
  if (device is MRxDevice) {
    repository = mRxRepository;
  } else {
    repository = rxRepository;
  }
  print(repository.runtimeType);
}

I think that's a consequence of how LUB is defined. /cc @eernstg to confirm. Tentantively closing as working as intended.
Note that these two pieces of code are very different. When you write x ? a : b compiler has to find some static type for this expression which both a and b are assignable to. Currently if inheritance is too complex or involves unrelated generics the compiler will give up and infer Object. When you write: if (x) { v = a; } else { v = b; } compiler just has to check if a and b are both assignable to v which is a different (simpler) problem.

(Sorry about the delay, I was on vacation ;-).
@mraleph already explained what's going on, but I could add a couple of words about the background:
The conditional expression needs to have a type (for instance, var x = b ? e1 : e2; will use the type of b ? e1 : e2 as the inferred type of x), and that type is computed using the type function UP. UP cannot compute a least upper bound of the two operands in all cases; that's not because it's hard, it's because the least upper bound does not exist:
abstract class I {}
abstract class J {}
class A implements I, J {}
class B implements I, J {}

In this example, A is a subtype of both I and J, and so is B. The least upper bound T of A and B must be a supertype of both (that is, we must have A <: T and B <: T), and it must be the least among such supertypes (that is, if A <: T2 and B <: T2 then it must be true that T <: T2). But I and J are both the immediate supertypes of both A and B, and they are not related (so we don't have I <: J and we don't have J <: I), and hence there is no type which is the least upper bound of A and B.
So UP has to compute something which is not always the least upper bound of its operands (even though we'd want to compute the least upper bound whenever possible). This means that there will be a slack, that is, we will have some situations where the type of b ? e1 : e2 is not a subtype of some of the shared upper bounds of the type of e1 and e2.
Your example shows such a case: UP(MRxRepository, RxRepository) (where MRxRepository means MRxRepository<MRxDevice> and RxRepository means RxRepository<IRxDevice>) is Object, even though IRxDeviceRepository (meaning IRxDeviceRepository<IRxDevice, IGroupRepository<IGroup>>) is a supertype of both.
In this case the slack is particularly large (we jump up to Object), because the computation of UP uses a Dart 1 algorithm when the operands are distinct interface types (that is, types of the form C1<...> and C2<...>, where C1 and C2 are not the same class).
We considered changing the algorithm when switching to Dart 2.0, but it was considered to be too much of a breaking change. This Dart 1 algorithm is special in that it does not call itself recursively on type arguments (so UP(C1<T1>, C2<T2>) will find a common supertype of C1<T1> and C2<T2> without computing UP(T1, T2)). This in turn was done because the algorithm in other languages (in particular, Java) was known to have termination problems in a similar situation.
In any case, you can actually come up with examples where the UP function computes a surprisingly suboptimal result (often Object), and it is not something for which we have a good, non-breaking solution, even though we've had a rather large number of discussions and proposals about it.
In contrast, the if-statement doesn't ever need to find a shared supertype of MRxRepository<MRxDevice> and RxRepository<IRxDevice>, it just needs to check that each of them is a subtype of IRxDeviceRepository<IRxDevice, IGroupRepository<IGroup>>.
If you do encounter this kind of issue (that is, the type of a b ? e1 : e2 expression is not a subtype of a given type S ever though both the type of e1 and the type of e2 is a subtype of S), then you can always tell UP which way to go by casting one of the branches to the desired type S:
class I {}
class J {}
class A implements I, J {}
class B implements I, J {}

var b = true;

void main() {
  var i = b ? A() : B() as I;
}


Remove unused dependency

It seems that removed dependency is unused.
 mvn clean package test

completes without errors with that dependency removed.

STOP!

I'm not sure anymore how this is exactly used, but probably via reflection in java-sdp-nist-bridge. It is used for the javax.sdp.* stuff, which is still referenced. It's possible that java-sdp-nist-bridge pulls in the dependency anway, or possibly an older version that had bugs with "o=". This needs to be checked more thoroughly than just "it compiles".

@ibauersachs seems like the dependency I've removed is anyway come with java-sdp-nist-bridge artifact:
https://github.com/opentelecoms-org/java-sdp-nist-bridge/blob/c3b3dd6dc5bc8112357d435f3eb05c217b823101/pom.xml#L45-L49
but has slightly different version.
If change does something wrong I'd like to update it/create new PR with comment explanation of such an implicit usage.

Jenkins test this please


GitHub checks details link to cats

When clicking on the links in this picture, it redirects to https://giphy.com/search/random-cats 🐱
I'm using the IP address of an ingress I set up on GKE, so this might be what happens when PAC cannot resolve a route url.

You need to configure it and set the url of the dashboard as documented here: https://pipelinesascode.com/docs/install/kubernetes/#tekton-dashboard-integration

Would it be possible to instead display some message explaining that the dashboard needs to be set up?

yes that would be perhaps better, there is some call in Github checks API expecting a real URL (which we could set to tekton.cd or something) but for the task breakdown we can just avoid showing  a link and show a message otherwise....


Please add LÖVE

Hi mate, I'm missing the LÖVE engine in your list.
If you don't like Lua, well then it might not be your cup of tea, but if you do like it I suggest you look into it.
Really like it's simple and elegant design which makes it perfect for prototyping (or even for full releases judging by the list of actually released games).
It's cross platform, doesn't need an IDE, open source but the license allows to release commercial games.
And I don't quite understand what you mean by "Coding is in Lua, no easier alternatives", etc. Lua must be one of the easiest languages out there (all power to Lua tables :smile_cat:).
And if you are concerned about performance, LÖVE (and probably other Lua based game engines too) use LuaJIT, which results in performance comparable to compiled languages, and often faster than Java.

Thanks very much for the feedback! I realise that maybe the rejected engines info wasn't very easy to follow. I have put it into table form to make it clearer. I've added Love to that table unfortunately, but hopefully it's clearer why now along with the others :). I added a smaller section near the top about language choice too and included Lua.


refactor assert_is_allowed_filename and included tests

refactor assert_is_allowed_filename and included tests
Description
Fixes # (issue)

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

Test A - lind_project/tests/test_cases/test_a.c
Test B - lind_project/tests/test_cases/test_b.c

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been added to a pull request and/or merged in other modules (native-client, lind-glibc, lind-project)

I'm going to close this PR since we have and are continuing to overhaul this file, a lot of the changes are on lines we've now deleted. Sorry for the confusion!


✨ Support upload YouTube shorts

@ibra-hassan  please resolve the conflicts that has been occurred ..

@luin please check this after conflicts are resolved.. it contains a minor changes.

Hi @nawfelsekrafi
I'm sorry for the confusion, but I created a new PR.
https://github.com/slab/quill/pull/4345


Set Up Development Environment for Customer Accounts Microservice

As a developer
I need to set up the development environment for the customer microservice
So that So that i can start testing and developing the microservice
Details and Assumptions
* A local development environment is created.
* All necessary dependencies are installed.
* Environment variables are configured properly.
* Documentation is provided for setting up the environment.

Acceptance Criteria
gherkin
Given a fresh development machine
When I follow the setup instructions
Then I should have a working development environment ready for the customer accounts microservice


Config to pipe to less or any other viewer

This implements the "pipe to less by default" requirement of #14

This is actually calling itself recursively... fixing


WIP: update with CTBase 0.10, CTDirect 0.7, CTFlows 0.4 and CTProblems 0.7

The CI is ok but we need to update all the tutorials.

Ci not ok. Need to remove tests with julia 1.9.

https://github.com/control-toolbox/OptimalControl.jl/pull/185#issuecomment-2185923135

Ci not ok. Need to remove tests with julia 1.9.

CTDirect has been requiring julia 1.10 for a few releases yes. The performance gains are such that I see no reason to support 1.9 unless someone specifically asks for it.

Ci not ok. Need to remove tests with julia 1.9.

CTDirect has been requiring julia 1.10 for a few releases yes. The performance gains are such that I see no reason to support 1.9 unless someone specifically asks for it.

OK
@PierreMartinon Performance where? Loading / precompilation?

We should

The CI is ok but we need to update all the tutorials.

@ocots can you please elaborate a bit?

Basic example: add using Plots, remove size kwarg in plot function.
Basic example functional: add using Plots, remove size kwarg in plot function. Update the definition of the ocp from the new API.
Double integrator: time minimisation: add using Plots, remove size kwarg in plot function.
Initial guess: update the tutorial according to https://github.com/control-toolbox/OptimalControl.jl/issues/170.
Plot a solution: add info about series attributes, labelling, time normalisation.
LQR example: add using Plots. Use the time normalisation within the plot function using time=:normalized instead of defining a rescaling function.
Indirect simple shooting: add using DifferentialEquations, present MINPACK and NonlinearSolver (MINPACK cannot be used with arm).
Batch processing: add using Plots, remove size kwarg in plot function. Add a label for the two plots using solution_label kwarg.
Goddard problem: add using Plots, remove size kwarg in plot function. Add a label for the two plots (direct and indirect) using solution_label kwarg. Add using DifferentialEquations.

Ci not ok. Need to remove tests with julia 1.9.

CTDirect has been requiring julia 1.10 for a few releases yes. The performance gains are such that I see no reason to support 1.9 unless someone specifically asks for it.

OK
@PierreMartinon Performance where? Loading / precompilation?

I assume so. This was the time when new warnings appeared during precompilation.


Coercion from Ghost lemmas to Lemmas

I am looking for ways to use "ghost lemmas" into lemmas or actual code. So far I have come up with something like the following:
effect GLemma (a:Type) (pre:Type) (post:Type) (pats:list pattern) =
       Ghost a pre (fun r -> post)

assume val coerce: pre:Type -> post:Type -> pats:list pattern -> g:(unit -> Ghost unit pre (fun r -> post)) -> Pure unit pre (fun r -> post)

val dummy_ghost_lemma: n:nat -> GLemma unit (requires (True)) (ensures (n > -1)) []
let dummy_ghost_lemma n = ()

val dummy_lemma: n:nat -> Lemma (n > -1)
let dummy_lemma n =
  coerce (requires (True)) (ensures (n > -1)) [] (fun _ -> dummy_ghost_lemma n)

Which works. So I have a couple questions:

is it sound (I restricted the coercion function to unit types only) ?
if so could it be added to FStar.Ghost so as to allow one to write lemmas reasoning on ghost values ?
is there a more appropriate way to write the effect and the coercion function above ?

Thanks!

This has been implemented. Closed.


cloud.gov training for recent new hires

Several new folks have or will be joining the team over the next month or so, and they'll all need to complete the following training:

Contingency Plan Training (overview and discussion of our plan)
Incident Response Training
Non-public information and Security Awareness Training

We try and batch these up so that they're completed at once for everyone as a cohort, which is helpful because it usually yields more useful discussion and identified improvements as well.
Note that new notes docs will have to be created in their respective sub-folders within our main training folder.
Acceptance Criteria

[ ] Training is scheduled and completed by the due dates listed in the training tracker for all new employees.

Security considerations

Helps keep the program in good operational standing with our compliance requirements and informs new team members of our security practices, policies, and procedures.

I'm holding CP training with Shea this week, trying for Thursday 4/7 in the
a.m.
On Mon, Apr 4, 2022 at 5:46 PM Carlo Costino @.***>
wrote:

Several new folks have or will be joining the team over the next month or
so, and they'll all need to complete the following training:

Contingency Plan Training (overview and discussion of our plan)
https://cloud.gov/docs/ops/contingency-plan/
Incident Response Training
https://docs.google.com/presentation/d/1AZjQE8zBzMRWZIFUuJPkJLted1ykGtALrLPoPRx5Vls/edit#slide=id.p
Non-public information and Security Awareness Training
https://docs.google.com/presentation/d/1uB4MlGCu8ZYUxjKVZKwicQ95MvLxaT4Mh93y6w79GPw/edit#slide=id.g1a2cf8d6b3_0_193

We try and batch these up so that they're completed at once for everyone
as a cohort, which is helpful because it usually yields more useful
discussion and identified improvements as well.
Acceptance Criteria

Training is scheduled and completed by the due dates listed in the training
tracker
https://docs.google.com/spreadsheets/d/1hqU6cNeEB293OT0j3OvbdAFRkrf2zDOrPVxGfnr4sSw/edit#gid=0
for all new employees.

Security considerations

Helps keep the program in good operational standing with our
compliance requirements and informs new team members of our security
practices, policies, and procedures.

—
Reply to this email directly, view it on GitHub
https://github.com/cloud-gov/product/issues/1713, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJHWCT3HAA47IICYL64PNDVDNPJRANCNFSM5SQ2TEAA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Peter Burkholder |  *
cloud.gov https://cloud.gov ISSO, compliance & security lead
please use @.** for cloud.gov matters
202-709-2028 <(202)%20209-2028> | @.**
@.***> *
| pronouns he-him https://www.mypronouns.org/he-him
Free/Busy Calendar
@.*>

@Chiakao and I noticed that @melanienleopold  and @kelleyconfer  have not had their IR training. We've decided that they should get that training, but then going forward, the business team should have an abbreviated training (self-guided?) since they only need training in incident recognition and reporting, not in actual response.

Training complete, tracker updated, notes documents are in the right places, next training (5/25) scheduled with @CloudEthos and @mheadd .


SSLEngine.unwrap seems to lose data if the source network data being unwrapped spans two/multiple TCP packets

I have started experimenting the usage of WildFly OpenSSL library as a SSL provider in context of a runtime server/project (not related to WildFly server). I have stumbled upon, what appears to be some kind of issue, in WildFly OpenSSL or OpenSSL. Given my lack of knowledge in either of these projects or the underlying code, I haven't been able to fully narrow this issue down in terms of what pieces of code contributes to it. However, here's what I think is happening based on analyzing logs and wireshark captures from different completes across multiple runs:

The server in question uses WildFly OpenSSL provider and internally uses SSLEngine API to wrap/unwrap data on an underlying SocketChannel using Java NIO selectors.

A typical "write" looks something like this which uses the SSLEngine:

	final ByteBuffer networkWriteBuffer = ...
	final ByteBuffer[] srcs = ...
    for (i = 0;  i < length; i++) {
        if (srcs[i].hasRemaining()) {
			SSLEngineResult wrapResult = sslEngine.wrap(src[i], networkWriteBuffer);
    		// deal with the result...
    		// write out to the underlying socketchannel, the "wrapped" data
    		socketChannel.write(networkWriteBuffer);
    		...
        }
        ....
    }

Please consider this a pseudo-code and assume that the necessary NIO selector semantics and the ByteBuffer usage semantics are being propely handled in the real code.
So what's essentially going on in the send side is that for each source data, the SSLEngine is used to wrap the data and write it over the socket channel.

A typical "read" on the other side of the SocketChannel uses the SSLEngine to unwrap the data and looks like:

        final ByteBuffer networkReadBuffer = ...
	final ByteBuffer appData = ....
	// read the data from the socket channel
	socketChannel.read(networkReadBuffer);
	// now use the SSLEngine to unwrap the data
        SSLEngineResult unwrapResult = sslEngine.unwrap(networkReadBuffer, appData);
        // deal with the result....

So the read side just keeps reading of the socket channel (whenever the relevant Selector is notified about data being available) and then unwrapping that data into a internal application specific buffer.
The issue I keep running into seems to be on the unwrapping part (or at least that's where I can "notice" it). Based on my wireshark captures and log analysis what I see is happening is that if the write side happens to (wrap and) send more than one packet of TCP data on the socket channel (I have tested consistently with two packets) and if the read side happens to read both those TCP packets in one go and pass it to sslEngine for unwrapping, the engine seems to unwrap just one packet and return back a OK result and the second packet is never returned back into the target buffer that's sent to the unwrap call. To explain a bit more concisely, consider the following sequence of events:
- Sender:
	T1: SSLEngine.wrap(dataOne) generates 33 bytes of data
	T2: write dataOne (==33 bytes) on to the socket channel
	T3: SSLEngine.wrap(dataTwo) generates 65 bytes of data
	T4: write dataTwo (==65 bytes) on to the socket channel

- Receiver:
	T5: read into incomingData from socket channel. Reads 98 bytes of data into the incomingData
	T6: SSLEngine.unwrap(incomingData of 98 bytes, dest). This ends up with the `dest` buffer being updated with 4 bytes of data and a SSLEngineResult which looks like:

		Status = OK HandshakeStatus = NOT_HANDSHAKING bytesConsumed = 98 bytesProduced = 4

The unwrap ends up producing wrong (lesser amount) of data in the above case. The important piece here is that this seems to happen only when the unwrap is fed with incoming data that belonged to 2 separate TCP packets that were read off the socket channel (since it was available on the socket). If it so happens that the socket read returns just one packet and that then is passed on to the unwrap API, one each at a time, then it ends up processing it correctly. In other words, the following sequence works:
- Sender:
	T1: SSLEngine.wrap(dataOne) generates 33 bytes of data
	T2: write dataOne (==33 bytes) on to the socket channel
	T3: SSLEngine.wrap(dataTwo) generates 65 bytes of data
	T4: write dataTwo (==65 bytes) on to the socket channel

- Receiver:
	T5: read into incomingData from socket channel. Reads 33 bytes of data into the incomingData
	T6: SSLEngine.unwrap(incomingData of 33 bytes, dest). This ends up with the `dest` buffer being updated with (correct) 4 bytes of data and a (correct) SSLEngineResult which looks like:

		Status = OK HandshakeStatus = NOT_HANDSHAKING bytesConsumed = 33 bytesProduced = 4

	T7: go back to reading off the socket channel. Reads 65 bytes of data into the incomingData
	T8: SSLEngine.unwrap(incomingData of 65 bytes, dest). This ends up with the `dest` buffer being updated with (correct) 36 bytes of data and a (correct) SSLEngineResult which looks like:

		Status = OK HandshakeStatus = NOT_HANDSHAKING bytesConsumed = 65 bytesProduced = 36

I hope this information is good enough to probably understand where the issue might be. So far, based on what I see in the server's usage of SSLEngine APIs and its usage of Java NIO selector APIs, I don't see anything obviously incorrect that it's doing. In fact, it's currently fully functional with the regular SSLEngine that's shipped in Java runtime (currently using Java 8). Having said that, if this appears to be some kind of an issue with the API usage, do let me know, I'll try and dig deeper in that part of their code. Unfortunately, I don't have an easy way to reproduce this outside of that library to narrow it down to this implementation of the SSL engine.
Environment details:

Operating system: Mac OS
Java version:

java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

OpenSSL version (both 1.0.2 and 1.1.0): OpenSSL 1.0.2l  25 May 2017 and OpenSSL 1.1.0f  25 May 2017
WildFly OpenSSL version: Latest upstream (1.0.3.Final-SNAPSHOT)

I think the issue may be that we do not 100% honour the contract of SSLEngine, and can potentially buffer data in the engine. Can you test out this branch:
https://github.com/stuartwdouglas/ssl-experiments/commits/fixes
I had someone else report this, but they never responded when I asked them to test out the fix and I forgot about it (as Undertow will defensivly always unwrap till the engine returns no more data this issue should not affect undertow).

Can you test out this branch:

https://github.com/stuartwdouglas/ssl-experiments/commits/fixes

I just did a very quick test building that branch from that repo. That indeed helped and I no longer see this issue. I am going to do a more thorough test in a few hours, but I think those fixes indeed gets me past this. Thanks very much Stuart.
P.S: I see that, that branch is behind on some of the commits that have happened here in this repo. If my testing goes fine, I will try and merge it on top of this repo and see if it still passes fine (right now, my attempts to merge it showed some conflicts and I was in a hurry so didn't pay much attention to the nature of the conflicts).

P.S: I see that, that branch is behind on some of the commits that have happened here in this repo. If my testing goes fine, I will try and merge it on top of this repo and see if it still passes fine (right now, my attempts to merge it showed some conflicts and I was in a hurry so didn't pay much attention to the nature of the conflicts).

So the merge conflict was due to my local testing related changes than a real one. So I went ahead and brought in those commits from the ssl-experiments/fixes branch and applied it to latest upstream master. It merged cleanly. Here's the branch in my repo which contains them https://github.com/jaikiran/wildfly-openssl/commits/ssl-unwrap-fixes. If necessary, I can open a PR with these changes, if it looks fine.
My testing so far has gone well and I will soon be creating some kind of initial basic performance comparison between this and the regular Java SSLEngine, which was the whole goal of this exercise.

@stuartwdouglas , my initial tests show that WildFly OpenSSL (with the fixes noted in the above comments) performs better than the SSL engine shipped in JRE.  If there's any help that I can provide in testing any additional fixes that you might have in mind, in the context of this issue, do let me know.

I have merged them upstream

Hi Jaikiran,
Would you mind if we aggregated your blog entry on wildfly.org?
-Jason

Oops left off the reference, sorry @jaikiran , see comment above

@n1hility

Hi Jaikiran,
Would you mind if we aggregated your blog entry on wildfly.org?

Hi Jason,
Feel free to include it :)

I have merged them upstream

Hi @stuartwdouglas, I don't see them in this upstream repo commits. Forgot to push maybe?

Oops, it is fixed now.
On Tue, Oct 31, 2017 at 3:06 PM, Jaikiran<EMAIL_ADDRESS>wrote:

I have merged them upstream
Hi @stuartwdouglas https://github.com/stuartwdouglas, I don't see them
in this upstream repo commits. Forgot to push maybe?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wildfly/wildfly-openssl/issues/26#issuecomment-340654359,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUDe3l181_srcECavYhe6VN83JQ6sE3ks5sxpzWgaJpZM4QEJPh
.

Thank you!
Closing this, now that the changes have been committed https://github.com/wildfly/wildfly-openssl/commit/91a15b7f7f69b5af706caf818fbe363396f38443


Remove unpacked connect/disconnect overloads from SignalSource

These overloads are currently unused.
A DelegateX object can be preconstructed at the point of call by enclosing the two parameters with braces {}, which can then be passed to the remaining overload. That perhaps adds clarity as well, since the this pointer and the member function pointer are closely related, so it makes sense to pass them as a compound object.
Reducing the number of overloads makes it easier to add debug code to just a single overload, which should help with occasional event handling issues.
Reference: https://github.com/lairworks/nas2d-core/pull/1055, https://github.com/OutpostUniverse/OPHD/pull/1271, https://github.com/OutpostUniverse/OPHD/issues/1269

Example of possible debug code:
void connect(DelegateType delegate, const char* caller = __builtin_FUNCTION())
{
	std::cout << "Call to connect from: " << caller << " to " << this << std::endl;
	// ...

Output (from OPHD):

Call to connect from: StateManager to 0x7e0a80
Call to connect from: initialize to 0x7e09c0
Call to connect from: initialize to 0x7e0a08
Call to connect from: UIContainer to 0x7e0a08
Call to connect from: Window to 0x7e0a20
Call to connect from: Window to 0x7e0a50
Call to connect from: Button to 0x7e0a08
...

Could probably get a bit more detail about the caller with some additional work.


[ClangImporter] Look for platform modulemaps in the resource dir first

This applies https://github.com/swiftlang/swift/pull/74814 for all targets except Android.
This fixes a common source of build failures on Linux, where shim functions from CxxStdlibShim would not be found by the hosttools Swift compiler when building the CxxStdlib overlay.
This change is keeping Android intact to keep the existing Android SDK working.

@swift-ci please test


component's props change can't update dom

hello!
Vue.js version
1.0.23
Reproduction Link

https://jsfiddle.net/3zw9u5ym/10/
Steps to reproduce
click the "reverse" buttons, component will reverse the props.
one of them will update dom, the other one won't
What is Expected?
all of them will reverse the array and update dom automaticly.
What is actually happening?
if I set the component's props with the vm's data, when I change the props inside component, dom will update automaticly.
but if I set the props with the vm's computed data, when I change the props inside component, dom won't update. I try to use $set and arr.splice, they don't work as before.

Computed properties are derived data - they will be re-created any time the source changes, so mutating them doesn't make sense. If the source value never changes, them it should not be computed; if it does change, the child should emit an event to let the parent mutate the source instead.

ok, thank you!


[Bug]: Hard Refresh on Saving index.js and App.js with rsbuild

Version
System: Window
Browser: Chrome
npmPackages:
react-router-dom:4.0
and others node js 16.13

Details
Description:
After migrating my project from react-scripts v4 to rsbuild (currently using version 0.1.0-rc.1), I noticed the following issues:
Hard Refresh on index.js and App.js Changes:
When making changes to the index.js or App.js files, the application performs a full page reload instead of utilizing Hot Module Replacement (HMR).
Expected Behavior: The changes should trigger HMR, similar to how it works for nested components and routes.
Current Behavior: HMR works as expected for child components called via routes or nested within App.js. However, any changes in index.js and App.js result in a hard refresh.
No Errors for Undeclared Variables:
Another critical issue is that undeclared variables are not throwing any errors during the build process.
Expected Behavior: The build should fail or at least throw an error if there are undeclared variables in the code.
Current Behavior: The build process completes successfully even when there are undeclared variables, which leads to potential runtime issues.
Steps to Reproduce:
Migrate a React project from react-scripts v4 to rsbuild version 0.1.0-rc.1.
Make a change in the index.js or App.js files and observe the behavior on save.
Introduce an undeclared variable in the codebase and attempt to build the project.
Additional Information:
This issue is critical as it affects the development experience and the reliability of the build process.
The behavior observed is inconsistent with the expected functionality of rsbuild, especially regarding error handling for undeclared variables.
Environment:
rsbuild version: 1.0.0-rc.1
React version: Migrated from react-scripts v4
Node.js version: 16
Reproduce link
N/A
Reproduce Steps
Steps to Reproduce:
Issue 1: Hard Refresh on index.js and App.js Changes
Setup:
Start with a React project that was originally using react-scripts v4.
Migrate the project to use rsbuild version 1.0.0-rc.1.
Observe HMR:
Run the project using rsbuild.
Make a simple change in any nested component (e.g., a component loaded via routes within App.js).
Observe that Hot Module Replacement (HMR) works as expected—no hard refresh occurs, and the change is reflected immediately.
Trigger the Issue:
Now, make a change in the index.js file (e.g., modifying an import or adding a console log).
Save the file and observe the behavior in the browser.
Similarly, make a change in App.js (e.g., modifying a state or prop) and save.
Observe that both changes result in a full page reload instead of using HMR.
Issue 2: No Errors for Undeclared Variables
Setup:
Ensure the project is set up with rsbuild version 1.0.0-rc.1.
Introduce an Error:
Open any JavaScript or JSX file in the project.
Introduce an undeclared variable (e.g., use a variable that hasn’t been defined or imported).
Trigger the Build:
Attempt to build the project using the rsbuild command.
Observe the build process to see if any errors related to the undeclared variable are thrown.
Review the Build Output:
Notice that the build completes successfully, and no errors are reported for the undeclared variable.

if we just migrate it removing react-script version 4 we can and implement some changes in index.js or app.js it get's hard refresh, HMR doesn't work root level files.

config file: >>

import { defineConfig, loadEnv } from '@rsbuild/core';
import { pluginReact } from '@rsbuild/plugin-react';
import { pluginSvgr } from '@rsbuild/plugin-svgr';

const { publicVars } = loadEnv({ prefixes: ['REACT_APP_'] });

export default defineConfig({
  plugins: [pluginReact(), pluginSvgr({ mixedImport: true })],
  html: {
    template: './public/index.html',
  },
  output: {
    distPath: {
      root: 'build',
    },
  },
  source: {
    define: publicVars,
  },
  server: {
    open: 'http://localhost:<port>/',
  },
  dev: {
    hmr: true,
    client: {
      host: 'localhost',
      protocol: 'ws',
      overlay: true,
    },
  },
  tools: {
    rspack: {
      module: {
        rules: [
          {
            // Match .png asset
            // You can change this regular expression to match different types of files
            test: /\.png$/,
            type: 'asset/resource',
            generator: {
              filename: 'static/media/[name].[hash][ext]',
            },
          },
        ],
      },
    },
  },
});

package.json

{
  "name": "visadone-dashboard",
  "version": "3.4.27",
  "private": true,
  "dependencies": {
    "@ant-design/icons": "^4.3.0",
    "@date-io/date-fns": "^1.3.13",
    "@fortawesome/fontawesome-free": "^5.15.1",
    "@material-ui/core": "^4.11.0",
    "@material-ui/icons": "^4.9.1",
    "@material-ui/lab": "^4.0.0-alpha.57",
    "@material-ui/pickers": "^3.2.10",
    "@stripe/stripe-js": "^1.11.0",
    "@testing-library/jest-dom": "^5.11.5",
    "@testing-library/react": "^11.1.0",
    "@testing-library/user-event": "^12.1.10",
    "antd": "^4.8.4",
    "array-find-index": "^1.0.2",
    "axios": "^0.21.1",
    "base-64": "^1.0.0",
    "bootstrap": "^4.5.3",
    "capitalize-the-first-letter": "^1.0.8",
    "chart.js": "^3.6.2",
    "dangerously-set-html-content": "^1.0.9",
    "dompurify": "^2.3.3",
    "formik": "^2.2.9",
    "history": "^5.0.0",
    "js-file-download": "^0.4.12",
    "mdbreact": "^4.27.0",
    "moment": "^2.29.1",
    "mui-formik": "^0.1.1",
    "multiselect-react-dropdown": "^1.6.3",
    "otp-generator": "^2.0.0",
    "otp-timer": "^1.1.5",
    "qrcode.react": "^3.0.1",
    "react": "^16.13.1",
    "react-axios": "^2.0.4",
    "react-bootstrap": "^1.4.0",
    "react-chartjs-2": "^4.0.0",
    "react-countdown": "^2.3.2",
    "react-cropper-image-editor": "^1.0.10",
    "react-date-range": "^1.4.0",
    "react-datepicker": "^4.8.0",
    "react-datetime-picker": "^3.0.4",
    "react-dom": "^16.13.1",
    "react-draggable-resizable-modal": "^0.1.2",
    "react-file-picker": "0.0.6",
    "react-ga": "^3.3.1",
    "react-hook-form": "^7.52.2",
    "react-image-gallery": "^1.0.8",
    "react-image-magnify": "^2.7.4",
    "react-otp-input": "^2.4.0",
    "react-otp-timer": "^0.1.0",
    "react-pdf-js": "^5.1.0",
    "react-quill": "^2.0.0",
    "react-router": "^5.2.0",
    "react-script-tag": "^1.1.2",
    "react-scripts": "4",
    "react-simple-keyboard": "^3.4.70",
    "react-timer": "^1.1.1",
    "reactjs-tag-input": "^2.0.15",
    "rn-otp-timer": "^1.0.1",
    "sha256": "^0.2.0",
    "simple-keyboard": "^3.4.56",
    "simple-keyboard-layouts": "^3.1.58",
    "styled-components": "^5.3.6",
    "sweetalert2": "^10.14.1",
    "web-vitals": "^0.2.4",
    "xlsx": "^0.16.9",
    "yup": "^0.32.11"
  },
  "scripts": {
    "dev": "env-cmd -f .env rsbuild dev",
    "rsbuild": "env-cmd -f .env rsbuild build",
    "rsbuild:stage": "env-cmd -f .env.stage rsbuild build",
    "rsbuild:uat": "env-cmd -f .env.uat rsbuild build",
    "rsbuild:prod": "env-cmd -f .env.prod rsbuild build",
    "inspect": "rsbuild inspect --mode production",
    "preview": "rsbuild preview",
    "start": "env-cmd -f .env react-scripts start",
    "build:stage": "env-cmd -f .env.stage react-scripts build",
    "build:uat": "env-cmd -f .env.uat react-scripts build",
    "build:prod": "env-cmd -f .env.prod react-scripts build",
    "build": "env-cmd -f .env.stage react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  },
  "devDependencies": {
    "@biomejs/biome": "1.8.3",
    "@rsbuild/core": "^1.0.1-rc.0",
    "@rsbuild/plugin-react": "^1.0.1-rc.0",
    "@rsbuild/plugin-svgr": "^1.0.1-rc.0",
    "env-cmd": "^10.1.0"
  }
}

trying to migrate this project in rsbuild and this is the production level code currently using rsbuild for staging and development purpose, once it is stable we will remove react-scripts.

Undeclared variable errors are reported by eslint. You can use rsbuild-plugin-eslint  and add no-undef rule.

Oops! Thanks for the suggestion, I have been getting those errors recently and without even those errors i was able to make the build.
again Thanks for the undeclared errors help

I don't know if this is the issue of my config or not but I guess I am using wrong configuration, right now until it will get stable i want to use react-scripts as well as rsbuild once it is stable I will remove the react scripts.

0",

is there any biom plugin available for this?
because I see there is a performance issue when we use eslint plugin for this.
I resolved all the issues coming from the eslintPlugin,
Additionally, is there any way to override all and just use no-undef rule
// .eslintrc

{
  "parser": "@babel/eslint-parser",
  "parserOptions": {
    "sourceType": "module",
    "requireConfigFile": false,
    "babelOptions": {
      "presets": ["@babel/preset-react"]
    }
  },
  "settings": {
    "react": {
      "version": "detect"
    }
  },
  "env": {
    "browser": true,
    "es2021": true
  },
  "extends": [
    "plugin:react/recommended",
    "standard",
    "plugin:react/jsx-runtime"
  ],
  "rules": {
    "no-undef": "error",
    "quotes": "off",
    "semi": "off",
    "comma-dangle": "off",
    "no-console": "off",
    "prefer-const": "off",
    "no-unused-vars": "off",
    "eol-last": "off",
    "object-curly-spacing": "off",
    "quote-props": "off",
    "indent": "off",
    "no-trailing-spaces": "off",
    "no-debugger":"off",
    "spaced-comment": "off",
    "space-before-function-paren": "off",
    "camelcase": "off",
    "no-multiple-empty-lines": "off",
    "padded-blocks": "off",
    "key-spacing": "off",
    "lines-between-class-members": "off",
    "react/no-unescaped-entities":"off",
    "no-useless-return":"off",
    "node/handle-callback-err": "off",
    "comma-spacing": "off",
    "no-useless-escape": "off",
    "block-spacing": "off",
    "no-empty": "off",
    "no-extra-boolean-cast": "off",
    "comma-style": "off",
    "no-mixed-spaces-and-tabs": "off",
    "no-unneeded-ternary": "off",
    "no-prototype-builtins": "off",
    "object-curly-newline": "off",
    "dot-location": "off",
    "import/no-duplicates": "off",
    "no-tabs": "off",
    "arrow-spacing": "off",
    "no-useless-computed-key": "off",
    "prefer-regex-literals": "off",
    "operator-linebreak": "off",
    "multiline-ternary": "off",
    "brace-style":"off",
    "no-multi-spaces":"off",
    "object-property-newline": "off",
    "semi-spacing": "off",
    "no-lone-blocks": "off",
    "no-redeclare": "off",
    "dot-notation": "off",
    "react/jsx-key": "off",
    "space-in-parens": "off",
    "array-callback-return": "off",
    "no-return-assign": "off",
    "space-infix-ops": "off",
    "space-before-blocks": "off",
    "keyword-spacing": "off",
    "react/no-unknown-property": "off",
    "eqeqeq": "off",
    "array-bracket-spacing": "off",
    "no-dupe-keys": "off",
    "react/jsx-no-duplicate-props": "off",
    "react/no-string-refs": "off",
    "curly": "off",
    "no-fallthrough": "off",
    "no-sparse-arrays": "off",
    "no-mixed-operators": "off",
    "no-unreachable-loop": "off",
    "react/jsx-no-target-blank": "off",
    "no-dupe-class-members": "off",
    "no-whitespace-before-property":"off",
    "use-isnan": "off",
    "no-self-assign": "off",
    "no-use-before-define": "off",
    "space-unary-ops": "off",
    "one-var": "off",
    "no-var": "off",
    "yoda": "off",
    "no-constant-condition": "off",
    "react/display-name": "off",
    "react/no-direct-mutation-state": "off",
    "react/prop-types": "off" // Disable PropTypes validation
  }
}

I created Eslint config and some rules had been added for warn but there are performance issue while render or performing Eslint operations so I just turned all off. Please correct me if I am wrong.


Goalie Strafe

This needs to go in a separate file, at the moment this overwrites the parameters used in the walk


Allow prefixing webjars path for swagger

Describe the feature
Currently webjars path goes to /webjars/swagger-ui. Please allow prefixing it with a string so that swagger docs don't break when Javalin app is deployed with reverse proxy
Additional context
If the Javalin app is deployed with reverse proxy which prepends a context path to the URL say /fooservice then swagger docs break because they go to root / instead of /fooservice. Allowing prefixes for webjars will allow us to redirect it to /fooservice/webjars and this not break swagger docs.

@tipsy could you transfer this issue?

✅

@tipsy could you transfer this issue?

Maybe this could be automated with an OpenApi label 🤔

@dzikoysk Sorry, I see this issue was closed, but I do not see the resolution. What am I missing?

This change is already available in 5.3.0-alpha.5-SNAPSHOT and will be available in 5.3.0-alpha.5. Commit that closes this issue introduces basePath property in Swagger plugin configuration:
https://github.com/javalin/javalin-openapi/blob/9f7cc7b825c6ba987fa7141ea22cb60bafd0977c/javalin-plugins/javalin-swagger-plugin/src/main/kotlin/io/javalin/openapi/plugin/swagger/SwaggerPlugin.kt#L17-L18
And here's the example as test case scenario:
https://github.com/javalin/javalin-openapi/blob/9f7cc7b825c6ba987fa7141ea22cb60bafd0977c/javalin-plugins/javalin-swagger-plugin/src/test/kotlin/io/javalin/openapi/plugin/swagger/SwaggerPluginTest.kt#L29-L49

Thank you @dzikoysk! Is there documentation where I can find the release cadence? I want to figure out when to expect 5.3.0 to be out in case there's resistance to using an alpha version.

"Stable" versions are published per each official release of Javalin, but because this plugin receives significantly more updates these days, I'm releasing current versions as alphas, so users can already use non-snapshot artifacts from maven-central.
I decided to use alpha identifier instead of rc, because I want to leave room for changes between alpha releases, plugin is developed on top of the poorly documented Java API for annotation processors and covers quite complex specification. It's also worth to mention I mostly develop it on my own, so it's pretty good trade-off at this moment. Thanks to that approach users usually get what they want in a few days rather than months and for now I didn't have any reports that it's inconvenient for either party.

That sounds like a very good strategy. How about the schedule for non-snapshot versions?
Also, I wanted to test it out. But the latest version seems to be: 5.3.0-alpha.4. How do I get: 5.3.0-alpha.5-SNAPSHOT

I usually release alphas at the end of the day when there're changes addressed by users. As I'm still working on some changes today, it'll probably do it in the evening.
Speaking of the snapshot builds, those are available in maven.reposilite.com/snapshots:

https://github.com/javalin/javalin-openapi/wiki/1.-Installation

Thank you! Again, that makes sense and sounds like a good strategy!

Sure, I hope it'll work properly with your reverse proxy service now :)

@dzikoysk I can verify that swagger docs are working. But /redoc fails for the same reason.  I will create a new ticket for the same.

Yup, this change is not yet implemented in ReDoc. I'll take a look on that a little bit later


Optimize PR validation

Currently we validate PRs by building and testing the world.  We should be smarter about what we build and test based on what's changed.
Goals:

Ideally PR validation takes 5 minutes.
PR validation only validates what's changed, not everything.

I believe that his one is done - although the builds still take some time (but it would be even worse if we built everything :P).


RqMultipart must be an interface

RqMultipart must be an interface and its implementation must be in its nested class RqMultipart.Base. Let's refactor it accordingly. The interface should look like:
public interface RqMultipart extends Request {
  Iterable<Request> part(String name);
  Iterable<String> names();
}

@yegor256 OK, I labeled it as bug

@yegor256 thanks for this report, I added 15 mins to your account, in transaction 000-0a5504bb

@ndbroadbent this task is for you, keep these principles in mind, and don't hesitate to ask any technical questions; The budget here is 30 mins, which is exactly how much time will be paid for, when the task is completed

@davvd - Here is the pull request: #178

@rultor don't close it, ask @yegor256 to do do it, since he created it, thanks!

@yegor256 there is no milestone yet, so I set it to 1.0

Hello @yegor256, please close this issue since it is now completed.

@ndbroadbent thanks!

@ndbroadbent you're holding this ticket for 15 days already. If it is not finished (and closed) in the next 48 hours, it will be re-assigned to someone else, see No Obligations principle (this article should help if you're stuck)
added -30 to your rating, now it is equal to -30

Hi @davvd , I'm pretty confused, this ticket should have been closed a very long time ago...

@yegor256 - can you please close this ticket

@ndbroadbent closed

@elenavolokhova please, let us know what do you think about this ticket, according to our QA rules

@davvd It looks good.

@davvd It looks good.

@elenavolokhova thank you, good quality is always good :)

@davvd It looks good.

@elenavolokhova thanks a lot :)

@davvd It looks good.

@elenavolokhova thank you :)

@davvd It looks good.

@elenavolokhova thank you

@davvd It looks good.

@elenavolokhova thank you

@davvd I think your script is broken :)

@davvd It looks good.

@elenavolokhova thanks for the review

@davvd It looks good.

@elenavolokhova thanks!

@ndbroadbent I added 10 mins to @elenavolokhova (for QA review) in transaction 56813626... thanks, I just added 1 hour to your account, payment AP-2DC814547C481642K, 370 hours and 10 mins spent... there is a bonus for fast delivery (age=1)... added +60 to your rating, now it is equal to +30


Tensorflow Lite not support Siamese network with parameter sharing

Hi,
I found that the Tensorflow Lite did not support Siamese network with parameter sharing of conv2d layers and batch normalization layers yet. (e.g. EnsureBiasVectors will fail to deal with different names of conv output layer with the same kernel.) Is there any plan to support it?
Thanks.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

Hi, sorry for that. And I found the problems were caused by the transformation (e.g. removing of original kernel of convolution after transformation) of operators.
Have I written custom code
No
OS Platform and Distribution
Mac OS
TensorFlow installed from
Compilation from github
TensorFlow version
r1.6
Bazel version
N/A
CUDA/cuDNN version
CPU only
GPU model and memory
N/A
Exact command to reproduce
bazel-bin/tensorflow/contrib/lite/toco/toco

@guohengkai, it's difficult to help you without a reproducible test case. Could you provide one?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 29 days with no activity and the awaiting response label was assigned. Is this still an issue?

@aselle Sorry for the late response. The following codes will save the model into "test.pbtxt", which will cause this problem.
`import tensorflow as tf
with tf.Graph().as_default() as graph:
# Input
input_placeholder = tf.placeholder(tf.float32, shape=(1, 20, 20, 2))
input_split_1, input_split_2 = tf.split(input_placeholder, 2, axis=3)
# Stream 1
conv1 = tf.layers.conv2d(input_split_1, 8, kernel_size=3, strides=1,
                        padding='SAME', activation=None, name='conv1')
bn1 = tf.layers.batch_normalization(conv1, axis=3, momentum=0.997,
                                    epsilon=1e-5, center=True, scale=True,
                                    training=True, fused=True, name='bn1')
relu1 = tf.nn.relu(bn1)
conv2 = tf.layers.conv2d(relu1, 8, kernel_size=3, strides=1,
                        padding='SAME', activation=None, name='conv2')
bn2 = tf.layers.batch_normalization(conv2, axis=3, momentum=0.997,
                                    epsilon=1e-5, center=True, scale=True,
                                    training=True, fused=True, name='bn2')
relu2 = tf.nn.relu(bn2)

# Stream 2
conv1_2 = tf.layers.conv2d(input_split_2, 8, kernel_size=3, strides=1,
                        padding='SAME', activation=None, name='conv1',
                        reuse=True)
bn1_2 = tf.layers.batch_normalization(conv1_2, axis=3, momentum=0.997,
                                    epsilon=1e-5, center=True, scale=True,
                                    training=True, fused=True, name='bn1',
                                    reuse=True)
relu1_2 = tf.nn.relu(bn1_2)
conv2_2 = tf.layers.conv2d(relu1_2, 8, kernel_size=3, strides=1,
                        padding='SAME', activation=None, name='conv2',
                        reuse=True)
bn2_2 = tf.layers.batch_normalization(conv2_2, axis=3, momentum=0.997,
                                    epsilon=1e-5, center=True, scale=True,
                                    training=True, fused=True, name='bn2',
                                    reuse=True)
relu2_2 = tf.nn.relu(bn2_2)

# Final
concat = tf.concat(axis=3, values=[relu2, relu2_2])
flatten = tf.layers.flatten(concat)
fc = tf.layers.dense(flatten, 200)

tf.train.write_graph(graph.as_graph_def(), '.', 'test.pbtxt')

`

Nagging Assignee @aselle: It has been 15 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 30 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

It looks like you will have to freeze your graph. Does https://www.tensorflow.org/extend/tool_developers/#freezing help?

@andrehentz Thanks for your kind response. Indeed I have frozen my graph. The codes I provided are just for network structure.

Nagging Assignee @aselle: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

We will try this, can you please share the error message if you have it handy ?

@anitha-v Thanks for your response. I do not have any error message handy now. Sorry for that. The errors are something like dimension inconsistent with conv due to the trimming operation. But I think it's easy to reproduce.

I'll take a look when I can. You may be able to work around this, by creating a new network that duplicates the weights.

@aselle OK. Thanks.

Have you created an eval graph? Your example seems to use batch normalization with training. Can you try with Training=False?

@raghuraman-k Hi, the provided example is just a demo to show the network structure I used because my codes are more complicated. I have converted it into an eval graph.

Nagging Assignee @aselle: It has been 17 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 33 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 49 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 64 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 79 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 94 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @aselle: It has been 109 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Hi there! I'll close this issue for now. Please open a separate issue if you are still encountering this problem with the latest code.

still not working though.
cannt work on siamese network with tflite


yaml string type not reserved & empty array and map not reserved

PR for https://github.com/mircodz/tojson/issues/7
+
Indentation/white spaces fixed automatically by VsCode save

Could you open a PR with just the fix to #7 without the linting? I'd prefer  to keep the current formatting.


Installation failed Mac M1

Installation
Machine:
Mac M1 2021
Installation:
Miniconda env.
python: 3.9.5
tensorflow: 2.7 (via conda -c apple)
Tried to install both from within jupyter:
!pip install tensorflow_similarity
from terminal in conda env:
pip install tensorflow_similarity

import tensorflow as tf
tf.version
'2.7.0'

The error I got is:
ERROR: Cannot install tensorflow-similarity==0.13.0, tensorflow-similarity==0.13.1, tensorflow-similarity==0.13.10, tensorflow-similarity==0.13.11, tensorflow-similarity==0.13.12, tensorflow-similarity==0.13.13, tensorflow-similarity==0.13.14, tensorflow-similarity==0.13.15, tensorflow-similarity==0.13.17, tensorflow-similarity==0.13.18, tensorflow-similarity==0.13.2, tensorflow-similarity==0.13.20, tensorflow-similarity==0.13.21, tensorflow-similarity==0.13.22, tensorflow-similarity==0.13.23, tensorflow-similarity==0.13.24, tensorflow-similarity==0.13.25, tensorflow-similarity==0.13.26, tensorflow-similarity==0.13.27, tensorflow-similarity==0.13.28, tensorflow-similarity==0.13.29, tensorflow-similarity==0.13.3, tensorflow-similarity==0.13.30, tensorflow-similarity==0.13.31, tensorflow-similarity==0.13.32, tensorflow-similarity==0.13.33, tensorflow-similarity==0.13.34, tensorflow-similarity==0.13.35, tensorflow-similarity==0.13.36, tensorflow-similarity==0.13.37, tensorflow-similarity==0.13.38, tensorflow-similarity==0.13.39, tensorflow-similarity==0.13.4, tensorflow-similarity==<IP_ADDRESS>, tensorflow-similarity==0.13.40, tensorflow-similarity==0.13.41, tensorflow-similarity==0.13.42, tensorflow-similarity==0.13.43, tensorflow-similarity==0.13.44, tensorflow-similarity==0.13.45, tensorflow-similarity==0.13.5, tensorflow-similarity==0.13.6, tensorflow-similarity==0.13.7, tensorflow-similarity==0.13.8, tensorflow-similarity==0.13.9, tensorflow-similarity==0.14, tensorflow-similarity==0.14.1, tensorflow-similarity==0.14.2, tensorflow-similarity==0.14.3, tensorflow-similarity==0.14.4, tensorflow-similarity==0.14.5, tensorflow-similarity==0.14.6, tensorflow-similarity==0.14.7 and tensorflow-similarity==0.14.8 because these package versions have conflicting dependencies.
...
To fix this you could try to:

loosen the range of package versions you've specified
remove package versions to allow pip attempt to solve the dependency conflict

Tried solutions:
Install from conda
conda install tensorflow_similarity

Hi,
I don't have an M1 to test on at the moment, but I was able to install on a 2019 MacBookPro OS 12.0.1 using the following:
python3 -m venv .env
. .env/bin/activate
pip install --upgrade pip
pip install tensorflow_similarity

This may be an issue related to how Conda handles the older packages. I'll try and get a Conda env setup on my machine to test. The current package setup.py is fairly relaxed on the requirements, and only needs tf >= 2.4 and tfds >= 4.2, but I'll let you know what I find once I get Conda setup or access to an M1.
You could also try pulling from the nightly development build. pip install tfsim-nightly

Hey,
Thank you for the follow-up.
I've done what you suggest, the answer back is:
The conflict is caused by:     tensorflow-similarity 0.14.8 depends on tensorflow>=2.4     tensorflow-similarity 0.14.7 depends on tensorflow>=2.4
But when running:
`macbook dive_model % python
Python 3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:24:55)
[Clang 11.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
tf.version
'2.7.0'`

Don't understand though.
Any idea ?

Hi, I moved the tensorflow dep to extras_require. Seems like this is a common pattern to avoid tensorflow install issues.
tensorflow_similarity 0.14.9 contains the fix. Please let me know if this resolves the issue.
Best,

Hi, thank you for the follow-up.
Just tried a new installation using pip install tensorflow-similarity == 0.14.9 as suggested.
several errors during the process due to wheels that it cannot install.
It ends with the following:
Successfully built pandas umap-learn pynndescent scikit-learn
Failed to build nmslib numba numpy llvmlite
ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
When I run a conda install numpy, it says the requirements is already satisfied (numpy is installed along with tensorflow.)
Thanks again for your helps, it seems it's moving in the right direction :)
Best

I have solve this issue by install nmslib by CFLAGS="-mavx -DWARN(a)=(a)" pip install nmslib then install TensorFlow Similarity by this CFLAGS="-mavx -DWARN(a)=(a)" pip install tensorflow-similarity and it works.


GPT2BPETokenizer produce strange symbol

Description
I ran code piece from
https://gluon-nlp.mxnet.io/model_zoo/bert/index.html
and the GPT2BPETokenizer produce a strange symbol Ġ
Error Message
In [1]: import gluonnlp as nlp; import mxnet as mx;
...: model, vocab = nlp.model.get_model('roberta_12_768_12', dataset_name='openwebtext_ccnew
...: s_stories_books_cased', use_decoder=False);
...: tokenizer = nlp.data.GPT2BPETokenizer();
...: text = [vocab.bos_token] + tokenizer('Hello world!') + [vocab.eos_token];
...: seq_encoding = model(mx.nd.array([vocab[text]]))
...:
...:
In [2]: print(text)
['<s>', 'Hello', 'Ġworld', '!', '</s>']
command and paste the outputs below:

Thank you for reporting the bug @hymzoque.
@eric-haibin-lin are you aware of this issue? Could it be related to https://github.com/pytorch/fairseq/blob/master/fairseq/models/roberta/hub_interface.py#L38-L56

Also, currently GPT2BPETokenizer hardcodes the vocabulary. However, for example to add XLM-R https://github.com/pytorch/fairseq/tree/master/examples/xlmr we need to add support for a separate vocabulary.

I think that is how it encodes space character. As far as I can tell, it doesn't lead to any error. For instance, if you apply nlp.data.GPT2BPEDetokenizer on the tokenized text, you successfully retrieve the original text.

Yes, that's working as intended. (cc @sxjscience)

We are now considering to revise the GPT2Tokenzier to make it more similar to https://github.com/sxjscience/gluonnlp-gpt2. For this issue, it’s not a bug because the tokenizer in GPT2 is based on bytes.

@szha @sxjscience is there an open issue which I can follow for supporting separate vocabulary for GPT2BPETokenizer as mentioned by @leezu ? Or is it on the backburner since there is only one trained dataset for both GPT and RoBERTa?

I don't think it's working as intended. The user experience is currently not good. At the very least we have to fix the example to use the GPT2BPEDetokenizer instead of simply looking up tokens in the vocab.
@zeeshansayyed let's open a second issue to track it. It's a real issue as other models use the same tokenizer with a separate vocab.

@leezu Should we close this?

Fixed in master


[DRAFT] Add delay logic between droplet download retries

What is this change about?
Adding a delay between droplet downloads
What problem it is trying to solve?
When the Rep is downloading droplets from the blobstore, in some cases the Hyperscaller may apply throttling. The problem is described in detail in
this issue.
What is the impact if the change is not made?
Some requests will continue to be terminated with "503 ServerBusy"

Hi @prycey77,
I took a quick look at the PR and added a few comments.
In regards to your unit test questions from the slack thread: I recommend following the following  contributor docs as they make running the complete suite of tests a bit more manageable.
Thanks,
Marc

The current configuration allows for 4 retries with gaps of 500ms, 1s, 2s and 4s.


Locale is considered when providing keywords to WAYF

When constructing the WAYF data, previously the current locale was not considered and the method would just in order use the first available of en, nl, or pt. With this change, Engine will first see if there are keywords for the current locale of the WAYF, if not, use en as a fallback, otherwise return Undefined as before.
Closes: #1303

Have applied this patch in our environment and found it to be working just fine!


[Good First Issue]🌄Implement a better demo data module

🚅Search before asking
I have searched for issues similar to this one.
🚅Description
Currently, demo data can only be obtained through function sdgx.utils.io.csv_utils.get_demo_single_table, and only one adult data sets are supported. In this issue, please implement a more scientific demonstration data management module.
🏕Solution
We recommend stripping this moudule out of script sdgx/utils/io/csv_utils.py and implementing a separate script in the sdgx/utils/io/ directory。
We recommend creating a file demo_data.py and implementing the functions or class in this file.
🍰 Example
We provide a class example for your reference:
# ISSUE DESCRIPTION A DemoData example
class DemoData(object):
    def  __init__(self, dataset_name) -> None:
        # ISSUE DESCRIPTION
        # the dataset name should be checked
        pass

    def get_data(self, offline_path = None) -> pd.DataFrame:
        # ISSUE DESCRIPTION
        # if offline_path is not None value,
        # read data from the input path
        pass

    def download_data(self) -> None:
        #
        pass

⚙ Detail
Some operations that enhance user experience are also worthwhile, such as:

When we support many datasets, it is unreasonable to put each dataset in the dataset/ directory. We expect to support downloading the target dataset from the Internet,this helps reduce the size of the entire git repository.
Due to the network speed in mainland China, you can ask the development team to use network resources to upload and provide download links for some larger data sets. The speed of these download links will be faster than the original links of the data sets.

We will provide demo load functions/classes as the project develops.


Issue 3496 - Formatting and a lot of whitespace

The actual changes are all in the first commit. All the rest is whitespace.

Better commit messages:
Scala Common Enrich: configure automated code formatting (closes #3496)
Scala Common Enrich: apply automated code formatting (closes #xxxx)

@chuwy It is not their fault. I have added some of the more exaggerated configurations (look at .scalafmt.conf align.tokens ). Overall we can play with other configurations and if they we need to switch them off somewhere as an exception we can always go //format: off.  Have a look here for configurations: http://scalameta.org/scalafmt/#Configuration


fix(combineLatest): Don't mutate array of observables passed to combineLatest

Description:
combineLatest was mutating the array of observables passed to it. This is a very simple fix to clone the array before modifying it.
See this codepen for example of the problem: https://codepen.io/anon/pen/qRNBVJ?editors=1111

Changes Unknown when pulling f6bb9cab17855818197760c3133da3fc559d5e96 on extremeandy:master into ** on ReactiveX:master**.

@extremeandy ... good catch. This looks like a side-effect from the time we added the ability to do both combineLatest(a, b, c) and combineLatest([a, b, c]) (originally it was just the former, which used rest arguments).
LGTM! :+1:


Skipping text inside dropcap or wordpress custom short codes

I am skipped

"I am skipped" text is skipped when parsing.

Thank you for filing this issue but this report does not nearly have enough context for me to debug.
How text is skipped or chosen into the final extracted summarized text format isn't just based on a single HTML tag, but the structure of HTML tags and many other tags around your text.