COM Port Access in WSL

In an attempt to use the com ports from WSL, I encountered the following:
public@systemhost:/dev$ ls ttyS4 -l  crw-r----- 1 root root 4, 68 Apr 14 21:23 ttyS4
public@systemhost:/dev$ cu /dev/ttyS4  cu: /dev/ttyS4: System not found
public@systemhost:/dev$ sudo cu /dev/ttyS4  cu: /dev/ttyS4: System not found
This is a USB to serial adapter. I've tried to open each of the ports present on my system, but no such luck.

There is a feature request for that somewhere on uservoice, but sadly, accessing the windows USB/com ports is afaik not yet possible.

Serial port access has been added in Insider build 16176. See https://blogs.msdn.microsoft.com/wsl/2017/04/14/serial-support-on-the-windows-subsystem-for-linux/
Note that screen and minicom don't work yet. I'm trying to get cu to work, but haven't gotten the permissions right yet.

@dhalbert: Thanks for the heads up. Just to make sure I read this right: this is not part of CU right?

Using 16176 and it is supposed to work. It's not part of cu, it's part of the WSL, exposing serial ports as device nodes. There's a couple of missing IOCTLS that prevent minicom from working, but cu should.

I tried using cu and had to chmod 666 /dev/ttySxx (where xx is the COM port I wanted) and also run cu as root to avoid permission problems. Then it seemed to open the port but I couldn't get any input or output. I'm not sure if this is a problem with my use of cu or something else. I was talking to a serial-to-USB device that works fine with Putty or Tera Term.

Thanks for trying out the serial support.  Can you please try the follow:
1.) Please double check that your COM port shows up in the registry under HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\SERIALCOMM
2.) The ttyS number matches the COM port number (e.g. ttyS1->COM1, ttyS2->COM2, ...)
3.) No other process has the COM port open (Windows only allows a single open).
4.) Try running with an elevated command prompt and as root

Can confirm that no other process is opening any of my COM ports.
In an elevated bash (run as admin):
sudo cu /dev/ttyS4
cu: /dev/ttyS4: System not found

Am I missing something?

You have to specify the tty as the -l or --line option:
sudo cu -l /dev/ttyS4

Appreciate the feedback, I've never used cu before.
sudo cu -l /dev/ttyS4
cu: open (/dev/ttyS4): Permission denied
cu: /dev/ttyS4: Line in use
Same stands for all of my COM ports; in the elevated bash, using sudo.

sudo chmod 666 /dev/ttyS4
sudo cu -l /dev/ttyS4

That should get rid of the permission error.

@dhalbert, were you able to resolve the issues that you were hitting?  If not, can you give me a quick rundown of your setup?  I have some serial hardware and if it matches I can try to repro.
Locally I used cu to connect to a RaspPi, cu -l /dev/ttyS2 -s 9600.  I may have also set the serial configuration with sty -F /dev/ttyS2 raw -echo -echoe -echok -echoctl -echoke -iexten -onlcr cs8 crtscts since some of the Windows serial ports had non-conventional initial settings.  I'll give it a try tomorrow and report back.

Thanks, @dhalbert - your suggestion worked, sorta. I'm able to connect to the 'line' now, but still not seeing any of the serial traffic. Probably related to what @stehufntdev mentioned about line settings, which I've tried his but to no avail... need to read up on this a bit more and make sure I have the settings correct.

@nosajthenitram: Yes, that's the problem I see.
@stehufntdev:
I am trying to connect to an Adafruit M0 Feather board which is running CircuitPython (variant of MicroPython), talking to the REPL. I did this:
$ sudo chmod 666 /dev/ttyS4
$ cu -l /dev/ttyS4

At that point cu just appears to hang. I try typing '~.and then return to close the connection and nothing happens. I have to kill the process from another bash. When I do kill the process, the~. shows up as input to bash, so it appears it was never read.
I can ask cu to turn on debugging, and I get a little info.
$ cu -l /dev/ttyS4 --debug all
cu: fconn_open: Opening port /dev/ttyS4 (default speed)
cu: fsserial_open: Baud rate is 9600
cu: fconn_set: Changing setting to 0, 0, 2
Connected.

I can connect to the same COM port with Tera Term or putty without difficulty, also at 9600 baud (that probably doesn't matter since it's USB to serial. I tried your stty flags but the problem remained.

Thanks @dhalbert, appreciate the extra details!  I was able to cu fine with a raspberry pi over a usb to serial connector which is the closest thing I have, and wanted to share out the notes.  On my machine, the raspberry pi is connected on COM5. Running as a Windows administrator should only be needed if your COM port is only accessible by a Windows administrator.
1.) sudo chmod 666 /dev/ttyS5 - This is required since cu changes it's uid which drops capabilities even when running as root.  Not all serial programs have the same behavior.
2.) stty -F /dev/ttyS5 -a .  Optionally check your current serial settings before updating.
3.) Set your serial settings, depending on your application this usually will be raw or sane.  Either one seems to work fine with cu:
3.a) stty -F /dev/ttyS5 sane 9600
3.b) stty -F /dev/ttyS5 raw 9600 -echo -echoe -echok -echoctl -echoke -iexten -onlcr cs8 crtscts
4.) cu -l /dev/ttyS5 -s 9600
5.) Hit enter to refresh the cu console
Here's the output:
stehufntdev@stehuf-test-6:/mnt/c/Users/stehuf$ sudo chmod 666 /dev/ttyS5
[sudo] password for stehufntdev:
stehufntdev@stehuf-test-6:/mnt/c/Users/stehuf$ stty -F /dev/ttyS5 -a
speed 1200 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^; erase = ^?; kill = ^U; eof = ; eol = ; eol2 = ; swtch = ;
start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs7 hupcl -cstopb cread clocal crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc
stehufntdev@stehuf-test-6:/mnt/c/Users/stehuf$ stty -F /dev/ttyS5 sane 9600
stehufntdev@stehuf-test-6:/mnt/c/Users/stehuf$ cu -l /dev/ttyS5 -s 9600
Connected.
Raspbian GNU/Linux 8 raspberrypi ttyAMA0
raspberrypi login: pi
Password:
Last login: Sat Dec 17 01:41:41 UTC 2016 on ttyAMA0
Linux raspberrypi 4.4.34-v7+ #930 SMP Wed Nov 23 15:20:41 GMT 2016 armv7l
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
SSH is enabled and the default password for the 'pi' user has not been changed.
This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.
pi@raspberrypi:~$

Not sure which of the above stty settings did it, but something pushed it to work. :)
public@systemhost:/dev$ sudo cu -l /dev/ttyS4 -s 9600
[sudo] password for public:
Connected.
Raspbian GNU/Linux

Great to hear, thanks for the confirmation!  Not sure why some serial devices initialize with non-conventional configurations.  For the most part, WSL reports back the current configuration since there isn't really a canonical serial device configuration with the proliferation of serial devices.  I'll see if we can update the blog with some of the commands above since it will probably be a common question.
@dhalbert, did the steps above resolve the issue you were seeing?

@stehufntdev I tried both the sane and raw settings you gave above. I still see neither input nor output. I wrote a small program on the board in question to print continuously to its serial port. But I also don't seem to be able to get output by doing just cat /dev/ttyS4. And in CMD, doing type COM4 just returns immediately. I also tried some PowerShell reading from COM4 as described here: https://blogs.msdn.microsoft.com/powershell/2006/08/31/writing-and-reading-info-from-serial-ports/, and that didn't work either: it just hung on the ReadLine(). So if I can't get I/O from basic Windows functions, that seems odd. It still works fine in Tera Term and Putty.
The device as shown in Device Manager is just "USB Serial Device (COM4)". The driver is WINDOWS\system32\DRIVERS\usbser.sys, version 10.0.16176.1000. The device installed with no errors, using usbser.inf to install. The Feather M0 board I'm using has native USB support on its Atmel SAMD21 processor.

@dhalbert, thanks for the update and trying out the different scenarios!  It's really strange that you can't read from the serial port with powershell, and I haven't come across a device with that behavior.  I'll ask around to see if I can get something similar from our hardware lab.
One thing that might be good to check is to see if Tera Term or Putty are setting different serial port settings which cause the device to function.  Can you please try the following to see if you can use powershell\WSL with the settings populated by Tera Term or Putty:
1.) Load up Tera Term or Putty so the serial device is functional
2.) Forcibly terminate Tera Term or Putty so the serial device settings are not reset
3.) Load up powershell or WSL and connect to the device without modifying the serial device settings.

@stehufntdev: I'll ask around to see if I can get something similar from our hardware lab.

Try anything you can load CircuitPython on MicroPython on. An Arduino Zero might be more readily available. I haven't tried a BBC micro:bit but that might exhibit the same behavior when loaded with MicroPython if it has a REPL. These MS guys might be able to help: https://makecode.com/blog/one-chip-to-flash-them-all, https://github.com/Microsoft/uf2.
I will test as you suggested and maybe try writing some simple test programs as well.

@stehufntdev - No luck on killing Tera Term and then using its settings (which are not unusual: 9600 baud, 8N1, no flow control).
Also tried various things from WSL using Python. I set up a program on the board to continuously output some text with numbers separated by \r\n, and then tried to read the characters from Python:
>>> t = open('/dev/ttyS3', "rb")
>>> t.read(10)

[hangs]
I also tried the lower-level os.open() interface with Python, which uses file descriptors directly. Same issue. I booted up native Ubuntu on the same computer and had no trouble getting the same Python code above to read output from the board. The stty settings on Ubuntu were stock, and I also tried raw successfully.
I have a feeling this is due to something about WSL's interactions with the stock usbser.sys' driver, as opposed to sttysettings. If you were talking to the RPi with a USB-to-serial cable, that adapter is probably using an FTDI or similar chip with a proprietary driver, not the older MSusbser.sys` driver.

@dhalbert, thanks for all of your persistence here really appreciate the help!
I borrowed a Arduino Mega 2560 which uses usbser.sys as well, and was able to go through the steps above for cu and write basic commands like volt and port and read the results back just fine.  For comparison testing I used putty to make sure the behavior matched.  Is there anything in this setup I can change to be closer to your repro?
Since both powershell and WSL don't work on your device with reasonable configuration settings, I'm starting to wonder if your device doesn't report read available correctly.  If you have time to further debug, could you try out the MSDN sample code just below the "...one oversight covered later in this section" line?  That block of code waits for read available and then reads the data which is what is done in WSL.  Based on your results we can rule out a read available issue with the device.
Today I'll see if I can get ahold of a Adafruit M0 Feather board.  If you can share out any details on your setup that might be relevant that would be helpful since I haven't worked with these devices previously :).

@stehufntdev - Thanks for looking at this. The Mega uses an Atmel 328 microcontroller, which does not have native USB support, so the board provides USB connectivity with another small microcontroller that just manages the USB port. The M0 boards uses an Atmel 32u4-based-board which have native USB support.
I will try the MSDN code, though I have to get VS installed on the Insider machine. If you have a precompiled test to the read-available issue, I could use that, but don't spend a lot of time on building it.
To test with a Feather M0:

Download https://github.com/adafruit/circuitpython/releases/download/0.9.5/adafruit-circuitpython-feather_m0_basic-0.9.5.bin from https://github.com/adafruit/circuitpython/releases.
Download and unpack https://github.com/shumatech/BOSSA/releases/download/1.7.0/bossac-1.7.0-mingw32.tar.gz from https://github.com/shumatech/BOSSA/releases.
Connect the M0 to a USB port and double-click its reset button to activate the bootloader. Its LED should start pulsing slowly (fading in and out). In CMD, run
bossac.exe -evwR adafruit-circuitpython-feather_m0_basic-0.9.5.bin
It should scan the COM ports and find the right one. If you need to specify the COM port, do
bossac.exe -evwRp COMx adafruit-circuitpython-feather_m0_basic-0.9.5.bin
Now CircuitPython should be on the board. (You may need to press the reset button once again, but I think not.) The board will come up and will report a composite USB device that is a CDC serial device, an MSC device (CIRCUITPY will mount on D:\ or whatever), and mouse and keyboard HID devices.
Connect with, say, Putty, and hit return. You should be in the Python REPL. You can type ctrl-D to have it restart. Try it out. Then close Putty and try it using cu in WSL, which shouldn't work.
If you want to get continuous output from the board, go back to Putty and write a small program in the REPL like:

while True do:
    i += 1
    print(i)

So now the board will print numbers continuously and you can test reading in WSL or in Windows proper.
There is an outdated tutorial here, but the above should duplicate what I'm doing.

@dhalbert, thanks again!  All of my serial test code was kernel mode so we couldn't reuse it here.  Let me know if getting the sample setup proves to be painful and I can share out the code and a binary :).
Thanks for the details, I have a lead on a Feather M0 but waiting to hear back.

@stehufntdev - I hacked up a test program to read chars from the Feather M0 running the Python print loop shown above. I included the problematic character reading loop as described in the MSDN sample code you referenced.
Here's the program (as .txt due to GitHub upload restrictions). I started with someone else's sample serial reading code, no points for style. I removed some COMMTIMEOUT setting code which seemed unnecessary: it did not change the result:
serial_read.cpp.txt
I'm not quite sure what I'm testing here, but I hope this helps. I did not actually run it on build 16176 or 16179 because VS was taking to long to install on my wimpy Insider Build machine: it was run on another machine running the vanilla Creators Update. Since I wasn't testing WSL I hope that's OK. Happy to run more tests like this.
When run, the programs reads and prints about 100 lines of increasing numbers (about 1k chars?), and then hangs. When it starts, it reads some old output that seem to have been previously buffered and then skips ahead to the latest set of consecutive numbers:
Port COM4 Opened
 5461192
5461193
[...]
5461201
56641910
6641911
6641912
[...]
6642002
6642003
6642 [hangs here]

@dhalbert, thanks for sharing out the results.  Based on the above the device reported back read available but it's concerning that it hung.  Going through your sample code I realized we wanted to try out the block without the caveat :(, sorry for the oversight there.  If it's not too much trouble could you re-run with the code in the block just below in the sample that has a do while loop on the read?  I'm hoping to rule out the hang is a result of read available.
Also, thanks for your tip on the https://github.com/Microsoft/uf2 project.  I spoke with that team and was able to borrow a Feather M0 to setup a local repro.  I should be able to start debugging today and will report back.

@stehufntdev - With the do-while loop added, the sample program works just fine. It continuously reads my continuously-printing python loop above.
serial_read2.cpp.txt
Glad you got in contact with the uf2 people. Their bootloader is very nice.

Thanks @dhalbert, appreciate all your help here.  I was able to reproduce the hang with the device and tried a few different scenarios.  Here's one that was particularly concerning:
1.) Reboot machine
2.) Run serial_read2.exe
3.) serial_read2.exe doesn't show any output
However, if I then connect with putty, putty shows the output.  If I then close putty and rerun serial_read2.exe it shows the output just fine.  Can you confirm if you are seeing the same behavior?
Based on the above, the serial device doesn't output with standard settings and putty is setting something that makes it work.  Not sure I would go as far to say that the device is buggy since it does work with putty and I'll do some more digging to see if I can narrow down the setting.

as I think my previous comment two weeks ago was missunderstood: Accessing com ports is not possible in the Creators Update but only  in a later Insider Preview.

@MikeGitb, yes serial port support is only in the latest insider preview (post Creator's update).

@stehufntdev - I duplicated your results. Note that you don't have to reboot: pulling the USB plug and then replacing it (and waiting until the CIRCUITPY drive shows up) will reset the port.
I looked at the putty source code to see what it was setting, and I've found the change that will make serial_read2.exe work! You have to set .fDtrControl to DTR_CONTROL_ENABLE. It must be that value and no other. Here's the putty source code, and the line in question: https://github.com/Yasushi/putty/blob/master/windows/winser.c#L109
Add the .fDtrControl setting to serial_read2.cpp:
	dcbSerialParams.BaudRate = CBR_9600;
	dcbSerialParams.ByteSize = 8;
	dcbSerialParams.StopBits = ONESTOPBIT;
	dcbSerialParams.Parity = NOPARITY;
	dcbSerialParams.fDtrControl = DTR_CONTROL_ENABLE;   // NEW - Makes it work
	//dcbSerialParams.fDtrControl = DTR_CONTROL_HANDSHAKE;  // Does not work.
	//dcbSerialParams.fDtrControl = DTR_CONTROL_DISABLE;  // Does not work.

So the serial code on the M0 is expecting that the host will assert DTR (Data Terminal Ready), and does not transmit until DTR is enabled.
stty used to list an option cdtrdsr to toggle DTR support. However, it's not in the current one shipped with Ubuntu. stty was changed not to include unsupported options in 2014. I think this is because stty uses tcsetattr() to change serial device modes. To toggle DTR, it looks like you need to use  ioctl() with TIOCIM_DTR.
So maybe this all has to do with the ioctl() stuff you haven't implemented yet. It may be that DTR is asserted by default on Linux systems when a serial port is opened, so this problem doesn't show up there.

@stehufntdev: Thanks for the clarification (disappointing as it is)

@mikegitb - worry not! Windows is targeting a release twice a year so users on stable builds won't have to wait long.

Thanks @dhalbert, I narrowed it down to the DTR bit as well and also discovered the mode command :).  Here's a basic repro using your test:
hang:
mode com3 dtr=off data=8 parity=n
serial_read2_settings.exe
no hang:
mode com3 dtr=on data=8 parity=n
serial_read2_settings.exe
As you suspected DTR is part of the ioctl support and isn't included in termios.  I started working on the fix and will report back when it is finished.  Thanks again for brining this to our attention!

@stehufntdev, great!
Are you going to try to emulate how serial ports come up on Linux? Or is that not really worth it? It does appear that they start with DTR (and maybe other things) asserted.  Maybe one way to do that would be that the first time /dev/ttySx is opened, you would set DTR, etc. as it would be on Linux.
As we saw usbser.sys does not do this. I tried a couple of other proprietary Windows drivers: the FTDI driver used for a USB-to-serial cable (e.g. for RPi) does turn on RTS. The driver supplied by Silicon Labs for their CP2104 USB-to-serial chip (used for example here) does not.

@dhalbert, to err on the safe side we need to rely on the Windows serial driver to set the correct defaults when the serial port is opened.  I thought about adding initialization code in our layer, but there are so many different serial configurations that any defaults we picked would be the wrong ones.
It looks like some versions of stty support configuring DTR, but I'm not seeing that option in the version included with Xenial.  If this ends up causing a lot of issues we can take  another look at a default configuration but I think those changes should really be in the serial driver.

@dhalbert, I'm hitting another strange issue with the device.  If I update your sample code to not call SetCommState, the device hangs even if I set everything up through mode before hand.  I would have expected the device to work just fine in this scenario.
Might be worthwhile to contact the company or search around to see if anyone else is having issues when reading from the device in code.  I think supporting this in WSL might be tricky since the device seems to be very sensitive to the settings.

@stehufntdev: Could you give the the MODE settings output before you run serial_read2 without no SetCommState() and get the hang? I will check it out. Can you still read from the device via Putty or Tera Term? I suspect it might be yet another setting.
The serial code used is the standard library supplied by Atmel for the microcontroller, incorporated into CircuitPython. It's all open-source and I'm helping to debug on a volunteer basis to debug all this, so I'll look deeper.

@dhalbert, yes I can still read fine from the device via putty.  To follow-up on the experiment above, I updated your sample code to call SetCommState without changing the values from GetCommState (i.e. set to the current).  After this, the device worked fine so the board seems to have some requirement that after opening the state has to be set.

@dhalbert, I was able to narrow down the device hang some more.  SetCommState calls EscapeCommFunction to set\clear RTS and DTR so the line matches the value passed to SetCommState.  When the feather M0 is opened, the DTR line is clear and needs to be set to transfer data.  To double check I updated the same code to call EscapeCommFunction with SETDTR instead of SetCommstate and it worked fine.  SetCommState has a comment that this is required for Win95 compatibility :), but I don't know the history of that code.
If you can update the serial code on the devices, my recommendation would be to have the device come up with the correct serial settings for transfer.  Also, DTR doesn't seem to have first class support everywhere (e.g. Linux) but I imagine switching over to another mechanism may be costly.
Going through the serial layer in WSL, I think we can support this device but it will likely be easier to configure from Windows first since the Linux tty tools I found don't have support for DTR.  Here's a quick summary of the changes I will make:
1.) Match SetCommState's behavior of updating the RTS and DTR line to match the configuration.
2.) Add support for ioctls to configure DTR and other settings
With some quick testing I was able to open the device with cu once it was configured.

@stehufntdev, thanks for the update. I looked at the source code for this device and also did some other research. In general, the libraries for these kinds of devices (not just Atmel) use DTR and/or RTS to determine whether they should send output. If DTR is not asserted, they assume they are not connected. Changing this might break a bunch of assumptions, so I'm leery of that.
On Linux, DTR appears to always be asserted when a USB CDC device is connected, at least for the generic CDC driver. On Windows, usbser.sys, the generic driver,  does not assert DTR automatically. So whatever program uses the port needs to do the right thing for the device. So Putty, etc. all set DTR and things are fine.
The best thing may be for me just to have a little utility to do the ioctl() to set DTR. Terminal programs do not necessarily do this: I looked at the source for minicom and screen and they don't appear to control DTR (though minicom will report its value!). And since cu is an antique program meant for modems and real serial ports, it never really had to deal with this, it seems.
One thing I stumbled on about using EscapeCommFunction() to set DTR and RTS: In the past, and maybe currently, usbser.sys does not let you assert RTS before asserting DTR. So you have to call it twice. I saw this mentioned several places. It is described most clearly here: http://dangerousprototypes.com/forum/viewtopic.php?f=29&t=2435:

there is a bug in the windows CDC driver in that it does not pass on the value of the RTS signal until DTS is also updated.

I don't know if this is really a bug or is by design, but you might need to watch out for this.

@dhalbert, thanks all sounds reasonable to me.
I tested with a few more serial devices and submitted the changes described above.  This should work on a future insider build and the test program could use mode from cmd.exe or from WSL you can use the TIOCMSET\TIOCMBIS ioctls with TIOCM_DTR.  Once the changes are out I'll update the blog post with the devices and some more examples.

I am getting
fcntl.ioctl(self.fd, TIOCMBIS, TIOCM_DTR_str)
OSError: [Errno 25] Inappropriate ioctl for device

I guess this is already been taken care for future release, if not then we need to make DTR control work not sure RTS works or not

The fix hasn't made it out to insider builds yet, but I will update the tag on the thread when it does and the blog post to include the supported ioctls.

Hi Everyone,
Have you find a solution to connect a dongle USB in the WSL ?
I would like to connect a dongle USB Wifi on a Kali distribution.
Thank's ;-)

@Raphdeumax - If your USB WiFi device is getting enumerated by Windows, conceptually it should work within bash. Is the WiFi device not showing up in ip addr show or ifconfig? Also, this looks dupe of #1077. Might be better to post there.

HI @sunilmut and thanks for your response.
I'm trying to connect a wifi dongle on a Kali but the interface "wlan0" does not appear in "wifite" or "ifconfig". Do you think it is possible to install a driver to interpret the Wifi dongle ?

@Raphdeumax - Is the interface showing up on Windows, in ipconfig /all?

@stehufntdev & @dhalbert  - I have been following this issue since I tried to work with my microPython device earlier this year through my WSL setup and stumbled upon this issue. I previously got to the "Connected" message after launching cu, but no in- or output was displayed.
I just installed the FCU. After uninstalling the previous "Bash on Ubuntu on Windows" system, and installing Ubuntu WSL through the Windows Store I can confirm this issue has been resolved and in- as well as output to the usb serial device works perfectly! Well done!

Thanks @TimV81. Closing this out. 16176 and FCU.

Hello!
I have the same problem that I cannot get data from a serial port in WSL.
@TimV81 , what does FCU mean in full?
I also got connected message from "cu" command, but no input or output obtained fro mthe serial port in WSL. How to solve this issue?


trying to ingest files w similar names results in errors

redo the form parsing for ingestor so that filenames that contain the names of other files, eg
thisIsAFile.mov
AndthisIsAFile.mov
currently this tries to get both files even if only one is selected and there's a problem parsing stuff down the line since the ingestor doesn't know which is which.

i think this is happening in the parsing of toIngest list in ingest.views.py

This is solved on ingest-refactor where the ingest form parsing is handled differently


Create TopVisitedPagesTable

Currently open to discussion on what kinds of tech to use for the table. Styling is also up for debate.

@suvasan19 think this is almost good to go!

Looks Great 🔥 🔥 🔥 !!! Awesome work!


Add more list sorting tests and fix many many through sorting

Issue: https://github.com/silverstripe/silverstripe-framework/issues/7981
This separates the sorting assertions out of an existing test into its own method, and adds another sorting test that doesn't use table aliases in the sort field.
I've also added some tests for sorting HasManyLists

This now fixes #7981 - a ManyManyThroughList will now use its joinClass's default_sort if nothing else has been provided already

Self merging with @ScopeyNZ and @kinglozzer's approval


[Chatllama] Generate Data Error on tutorial

I run the GetStart but the dataset has the error
(llama) qiuyc@qiuyc-3090:~/code/Chatllama/chatllama-nebullvm$ python artifacts/main.py artifacts/config/config.yaml --type ALL
Current device used :cuda
No previous model found at ./models/reward for model gpt2-large.pt
Loading dataset from ./datasets/reward_training_data.json
Loaded 200 samples
Start Training the Reward Model
Looking for checkpoints...
Found checkpoint for epoch 1, step 10...
Loading ...
Epoch: 1/32, Iteration: 11/25, Training Loss: 3.4106693267822266
prediction [4.451936  3.87707   5.655977  4.2465377 2.3933697 5.1995487 4.347823
5.8636036] target [4.  3.  2.  3.  4.  4.5 4.  3. ]
Epoch: 1/32, Iteration: 12/25, Training Loss: 2.217592477798462
prediction [2.7735171 5.110946  5.260797  5.696887  4.2918286 2.9091988 5.3532596
4.6064796] target [3. 3. 4. 4. 3. 4. 3. 4.]
Epoch: 1/32, Iteration: 13/25, Training Loss: 3.16831636428833
prediction [2.6523368 5.0899324 5.2925634 4.878763  4.6527834 5.071644  4.953468
4.7791476] target [4. 2. 3. 4. 3. 4. 4. 3.]
Epoch: 1/32, Iteration: 14/25, Training Loss: 1.3716436624526978
prediction [4.5529017 4.795724  5.041059  4.568875  2.6253047 5.424052  4.8917665
3.5276096] target [4.  4.5 4.  3.  3.  3.  4.  4. ]
Epoch: 1/32, Iteration: 15/25, Training Loss: 1.31028413772583
prediction [5.3360367 4.746885  3.1630383 4.666844  4.6595182 2.9819224 4.9737635
4.466095 ] target [4. 4. 2. 3. 3. 4. 5. 4.]
Traceback (most recent call last):
File "artifacts/main.py", line 57, in
reward_trainer.train()
File "/home/qiuyc/code/Chatllama/chatllama-nebullvm/chatllama/rlhf/reward.py", line 404, in train
for i, inputs in enumerate(self.train_dataloader):
File "/home/qiuyc/anaconda3/envs/llama/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 652, in next
data = self._next_data()
File "/home/qiuyc/anaconda3/envs/llama/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 692, in _next_data
data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
File "/home/qiuyc/anaconda3/envs/llama/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/qiuyc/anaconda3/envs/llama/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/qiuyc/code/Chatllama/chatllama-nebullvm/chatllama/rlhf/reward.py", line 239, in getitem
score = float(self.data[idx]["score"])
TypeError: float() argument must be a string or a number, not 'NoneType'

Hi @yicheng-2019
Yes this error is a result of the fact that it may happen that Davinci is unable to assign a score to a text and then when training the reward model the score is still None and training fails.
We have added additional checks on the reward training dataset so that it shouldn't happen anymore.
You can already try the fixed version from the PR #233 .
Thanks for your input and let us know if the problem persist.

Ok, thanks for your help


SPUserProfileServiceApp throws access denied when using the FarmAccount as the PsDscRunAsCredential

Details of the scenario you try and problem that is occurring:
I've executed the SPUserProfileServiceApp resource with the SharePoint Setup account as the PsDscRunAsCredential, also the account used in the examples. When the resource executes it thows an error that it should be the farm account.
I then changed the PsDscRunAsCredential to the FarmAccount, i then get the following:
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Creating user profile service application User Profile Service Application
Exception calling "Add" with "1" argument(s): "Access is denied.
"
+ CategoryInfo          : NotSpecified: (:) [], CimException
+ FullyQualifiedErrorId : CatchFromBaseAdapterMethodInvokeTI
+ PSComputerName        : SPDFSP2
Service 'SharePoint Timer Service (SPTimerV4)' cannot be stopped due to the following error: Cannot open SPTimerV4 service on computer '.'.
+ CategoryInfo          : CloseError: (System.ServiceProcess.ServiceController:) [], CimException
+ FullyQualifiedErrorId : CouldNotStopService,Microsoft.PowerShell.Commands.RestartServiceCommand
+ PSComputerName        : SPDFSP2
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Executing as the local run as user CONTOSO\spfarm
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Leaving BeginProcessing Method of Get-SPServiceApplication.
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Leaving ProcessRecord Method of Get-SPServiceApplication.
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Leaving EndProcessing Method of Get-SPServiceApplication.
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Leaving BeginProcessing Method of New-SPProfileServiceApplication.
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Performing the operation "New-SPProfileServiceApplication" on target "User Profile Service Application".
VERBOSE: [SPDFSP2]:                            [[SPUserProfileServiceApp]UserProfileServiceApp] Leaving ProcessRecord Method of New-SPProfileServiceApplication.
VERBOSE: [SPDFSP2]: LCM:  [ End    Set      ]  [[SPUserProfileServiceApp]UserProfileServiceApp]  in 65.1490 seconds.
PowerShell DSC resource MSFT_SPUserProfileServiceApp  failed to execute Set-TargetResource functionality with error message: Access denied.  Only machine administrators are allowed to create administration service job definitions of type:
Microsoft.SharePoint.Administration.SPServiceApplicationInstanceProvisioningJobDefinition, Microsoft.SharePoint, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=71e9bce111e9429c.
+ CategoryInfo          : InvalidOperation: (:) [], CimException
+ FullyQualifiedErrorId : ProviderOperationExecutionFailure
+ PSComputerName        : SPDFSP2
I suspect the issue is that the FarmAccount cannot add itself to the Admin group. The FarmAccount is not an administrator on any of the SP Servers, only the SetupAccount.
The DSC configuration that is using the resource:
SPUserProfileServiceApp UserProfileServiceApp
{
Ensure               = "Present"
Name                 = $ConfigurationData.NonNodeData.SharePoint.UserProfileService.Name
ApplicationPool      = $ConfigurationData.NonNodeData.SharePoint.Services.ApplicationPoolName
MySiteHostLocation   = $ConfigurationData.NonNodeData.SharePoint.UserProfileService.MySiteUrl
ProfileDBName        = $ConfigurationData.NonNodeData.SharePoint.UserProfileService.ProfileDB
ProfileDBServer      = $ConfigurationData.NonNodeData.SQLServer.ServiceAppDatabaseServer
SocialDBName         = $ConfigurationData.NonNodeData.SharePoint.UserProfileService.SocialDB
SocialDBServer       = $ConfigurationData.NonNodeData.SQLServer.ServiceAppDatabaseServer
SyncDBName           = $ConfigurationData.NonNodeData.SharePoint.UserProfileService.SyncDB
SyncDBServer         = $ConfigurationData.NonNodeData.SQLServer.ServiceAppDatabaseServer
PsDscRunAsCredential = $SPSetupAccount
DependsOn            = @('[SPServiceAppPool]MainServiceAppPool', '[SPManagedMetaDataServiceApp]ManagedMetadataServiceApp', '[SPManagedAccount]WebPoolManagedAccount')
}
Version of the Operating System and PowerShell the DSC Target Node is running:
Windows 2016
Version of the DSC module you're using:
<IP_ADDRESS>

It appears that our only option (if we keep CredSSP out of the loop), is to let the Set-TargetResource function of the SPUserProfileServiceApp would need addition logic to handle the following flow:

Add the farm account to local group
Provision the Service App
Remove the farm account from the local group

This way, we would not have any conflicting resource blocks, and the Test-TargetResource would not always report it as not compliant. When you really think about it, the provisioning portion ABSOLUTELY requires the Farm admin to be local admin for it to work, there it makes sense that this resource handles the add/remove process as well instead of depend on other resources to do stuff it requires.

Actually, to add to my point, a colleague just pointed out to me that we are doing the same thing in the SPUserProfileSyncService resource. We call Add-SPDSCUserToLocalAdmin, start the service and then call Remove-SPDSCUserToLocalAdmin once everything has been provisioned. We should be reusing the same logic here.

We have two issues with this solution:

When the Farm account isn't local admin, it cannot add itself to the local administrators group since it does not have the permissions to do so.
When you change permissions of an account, you need to login again. If we find another way to add the farm account to the local administrators group from inside the resource, the code still isn't able to provision the service application right away. This because the Set method was still running as the farm account and didn't re-logon. The provision first has to fail and then it will work on the second attempt. With the assumption that as soon as it fails on the first attempt, the code doesn't remove the account again ;-)

This leads me to believe that re-adding a farm account parameter again, running the resource as the install account, adding the farm account and then executing the service application provision using CredSSP is the only option that will work.

@ykuijs

Can't we use PsDscRunAsCredential = SP_Setup to add the Farm admin to the local admin group? the setup account should already be a local admin at that point, no?

2 - Wouldn't calling into a separate Invoke-SPDSCCommand script block pick up the new permissions without having to log out/on? How are we doing it in the User Profile Sync Service?
Thanks

@NikCharlebois I could certainly be wrong, but on first glance, it looks to me like the difference is that the sync service doesn't try to run any process AS the farm account. Instead, it looks up the farm account name and password and passes those values to $ups.SetSynchronizationMachine where $ups is the object returned from Get-SPServiceApplication. So the DSC module isn't trying to create a new process using different credentials, which would violate the 2-hop rule that CredSSP was being used for previously.
On the other hand, the SPUserProfileServiceApp resource needs to run New-SPProfileServiceApplication under the context of the farm credential. Assuming we run the resource as the setup account (because of local admin), the resource would then need to invoke another process under the farm credential in order to do this.
I think that, at least in the near term, @ykuijs is right in that re-enabling CredSSP is the best/only way to get this working again. Maybe we can figure out a way to get around this in the future, but I think the minimal risk tied to CredSSP is acceptable to get this high profile functionality working again. Maybe we could CYA by linking to articles/documentation about CredSSP that explains the risks and how to set it up properly to only allow delegating across secure channels/servers.

The Sync Service resource does two things:

It runs as the Farm Account to create the Sync service. It is using the InstallAccount or PsDscRunAsCredential for that.
It uses the credentials in the FarmAccount parameter to configure the service application. We had to use the FarmAccount parameter, because when you use PsDscRunAsCredential there is no way to retrieve the password from code unfortunately.

@NikCharlebois:

That is exactly what we did pre 2.0. This does however mean that you have to use CredSSP to run other code as the Farm account. I do believe that currently this is the only way to solve this issue. I am working on implementing this solution, but couldn't get it implemented in time for v2.1 :-(

@ronx:
Good idea to add the CredSSP info to the Readme.md of both resources and the Wiki. Will add that as well.

Closed via PR #775


Docs: Add resource handler topic

Convert community forum post into draft topic for "Add resource handler to data source plugins".
Fixes #
Issue 66401

Changes look good 👍 Still some unresolved comments. Let me know if you need help with the removal of fetch example

I think I got the unresolved comments resolved now.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Fixed issues #2497 and #2501 (Text-fit bugs)

This fixed issues #2497 and #2501.
/play - items

/play - heroes

Awesome–thanks @laituan245!

Hm...
i'm confused with green "SAVE" button ("сохранить" in ru-locale)
Could you make it with normal size?
And once again, as i asked in https://github.com/codecombat/codecombat/issues/1918 - is it real to split "unlock" text and unlock price to 2 lines?

As you can mention - quick fix with https://github.com/codecombat/codecombat/pull/2503


Upgrade clap version?

Current clap version is 3.1.6 (as of 2022 Mar 14th). If it's worth to upgrade the clap version, let's do it

handled in pr #53


[BUG] Chatting without having entered any API key fails silently

Description

Build and run locally
Escape the dialog to enter an API key
Send a chat text

Note how no response is returned (predictably), but the UI doesn't complain in any way that there's no API key. So the user might've spent time crafting a complex prompt, and after sending it, the prompt will be lost. There is no history of it.
This makes for a disappointing first-time new user experience
Device and browser
Firefox 123 on Linux
Screenshots and more
No response
Willingness to Contribute

[ ] 🙋♂ Yes, I would like to contribute a fix.

Thanks @dandv very true.


Feature request: add option to always query IPQualityScore and pass settings

IPQualityScore provides a lot of useful information on abuse IP. Please, add option to always query it. Also add an option to pass settings string, like strictness=1&allow_public_access_points=false
For example, abuser IP <IP_ADDRESS> currently displays REP ✓ NONE, but with enabled IPQualityScore it will display REP ❌ BAD (Threat Score 87%)  RECENT ABUSER   PROXY. Passing strictness=1 will give even better results: REP ❌ BAD (Threat Score 100%)  RECENT ABUSER   BOT   PROXY .

Thank you!
I've noticed that default strictness=0 doesn't detect BOT in my test, seems because it accounts only 24-48 hours reputation, and levels 1-3 have longer period.

Thank you! I've noticed that default strictness=0 doesn't detect BOT in my test, seems because it accounts only 24-48 hours reputation, and strictness 1-3 have longer period.

Very interesting. I had never digged in-depth into IQS settings, but that sounds like something you'd want as a default. Will consider doing so in the future. Thanks!


[Question]: How to get detail message about abort commit

Description
When using direct load to Oceanbase, the insert process generally does not report an error, but when committing, an abort occurs, yet there is no clear information indicating whether the data types do not correspond. How can one see the specific reason for the abort?

Perhaps this issue should be discussed at https://github.com/oceanbase/obkv-table-client-java, the direct loading client is maintained there.


Add Japanese

Added jp.json

thanks for your contribution @risafj. language code is ja. Published in version 1.63.0


Get rid of PathExt

It's already in std::path::Path in rust 1.5

Looks like Rust guys deprecated PathExt,  advised to use std::fs::metadata instead. Do you have any plans around that?

Rust never had PathExt in stable rust. So vagga doesn't use it. There is our own PathExt in src/path_util.rs
Most methods that was on unstable PathExt (and that are on our path_util::PathExt) are actually stabilized in new rust on Path type directly without an extension trait (exists(), is_dir(), ..etc)

So the issue stays the same. We still looking for a volunteer to do the work.

@tailhook Please close this isse

Yup. Closed in #198


GitHub Addition

GitHub Addition
User Story
AS A developer
I WANT a README generator
SO THAT I can quickly create a professional README for a new project
Acceptance Criteria
GIVEN a command-line application that accepts user input
WHEN I enter my GitHub username
THEN this is added to the section of the README entitled Questions, with a link to my GitHub profile

user name inputted for link to github


add EBUCore to help

Thanks.
Tiny request: next time do a git squash of such commits.

Thanks for the hint. (BTW: I didn't know at all this possibility before Peter's tweet.)

(BTW: I didn't know at all this possibility before Peter's tweet.)

There is definitely a lack of documentation about output formats. It is mainly visible in the Windows GUI for the moment (Linux/Mac GUIs in the next version thanks to some feature sponsoring). I take any help proposal for documentation ;-).
You can try some other "magic" values.

I will download, and read it during 11 hours flight or so ;-)


Module 'httpcore' has no attribute 'Request'

I installed oterm via pip and got this error.
Traceback (most recent call last):
  File "/usr/local/bin/oterm", line 5, in <module>
    from oterm.cli.oterm import cli
  File "/usr/local/lib/python3.10/dist-packages/oterm/cli/oterm.py", line 6, in <module>
    from oterm.app.oterm import app
  File "/usr/local/lib/python3.10/dist-packages/oterm/app/oterm.py", line 7, in <module>
    from oterm.app.chat_edit import ChatEdit
  File "/usr/local/lib/python3.10/dist-packages/oterm/app/chat_edit.py", line 14, in <module>
    from oterm.ollama import OllamaAPI
  File "/usr/local/lib/python3.10/dist-packages/oterm/ollama.py", line 4, in <module>
    import httpx
  File "/usr/local/lib/python3.10/dist-packages/httpx/__init__.py", line 48, in <module>
    from ._main import main
  File "/usr/local/lib/python3.10/dist-packages/httpx/_main.py", line 111, in <module>
    def format_request_headers(request: httpcore.Request, http2: bool = False) -> str:
AttributeError: module 'httpcore' has no attribute 'Request'

After investigating the issue, it was discovered that the problem was caused by conflicting version requirements of httpx between two dependent packages: mugmoment required httpx<0.18.0, while oterm required httpx>=0.25.0.
To resolve the conflict, I downgraded httpx to version 0.25.1, which satisfied the version requirement of both packages. I ran the following command to downgrade httpx:
pip install "httpx<0.26.0"

After downgrading httpx, I was able to run oterm successfully.

 It's never a good idea to have anything really other than pip and pipx installed globally.
Use a virtual environment instead or pipx!


Change notebook dir of auto started notebook server

I would really like to supply my own --NotebookApp.notebook_dir argument for the jupyter notebook server which gets auto started. The reason for this is that I eventually want to banish all .ipynb files to /tmp/, such that only the *.ju.py files are kept and the ipynb is generated from it every time I open the file.
Thank you for this awesome project! It truly is a dream come true!

Sure, will add this option soon!

I'll add an option in the configuration default_notebook_dir and you will be able to set it to "{GIT}", "{FILE}", "{TMP}", or any other path you want e.g. `"{GIT}/ipynb"

I've added an option notebook_dir and you can set it to /tmp. I didn't add variable options like GIT_ROOT to make it simple for now.

Thanks!


BAU: Extract BASE_URL from the environment

Proposed changes
What changed
We're using the BASE_URL env variable to construct a redirect URL to
send to the credential issuer, but never actually extracting it from the
environment. This resulted in redirect URL's with undefined as the
host.

Closing in favour of https://github.com/alphagov/di-ipv-core-front/pull/40


I2C communication bus speed path not found.

I am having issue using the i2c communication on the Jetson Nano Orin. My jetpack version is 6.0-b52 and the os version is ubuntu 22.04.4 LTS.
I am getting this error when I try to open the I2C bus : Not possible to read current bus speed. Segmentation fault.
By looking into the orin.c it seems that the file path for the bus speed file is no longer the same for with version I use. I will keep looking for the rigth path.
Thanks in advance for your help.

You are right, on the latest jetpack 6 the sysfs has changed, this is going to be interesting, lets see if i found another way or maybe i have to remove the option on jetpack 6 onwards

Not a solution to your problem but here it goes a rant.
So this is happening in other platforms as well, just checked some Odroid forum, is the Linux kernel people, nothing to do with Nvidia. Basically the only option open seems to be changing the device tree and reboot, no more dynamic changing. If there is no easy option I will have to disable the speed option for Jetpacks > 6.
By the way, the device tree is overused more and more and again the problem is that in principle the device tree doesn't allow changes at runtime. It can be done but only from a kernel module (double the work because then you will have to implement ioctls or something to be able to do stuff from user space).
Lets see if i can find some shortcut, if not the i2c will be functional but no more speed changes.

@remilamoureuxlevesque
can you make me a favour and run this on your terminal? uname -r

yes no problem, the output is :
5.15.122-tegra

Should be good now, no crashes. Flags will not have any effect for machines with Jetpack >= 6

Thank for you responses, I update the library and now it works.


Update EasyRSA to 3.0.5

It's breaking something:
<ca>
</ca>
<cert>
</cert>
<key>
</key>

The release archive is not yet available.


unexpected lost spacing

This is a bit of a weird one, somewhat of a continuation of #729.  I'm struggling to get a simple test case.
In https://github.com/krautzource/sre-to-tree/issues/39, I have a test that passes if run individually but fails if run all tests.
The failing case is
  expected: |-
      'left brace StartLayout 1st Row  with Label left parenthesis 1 right parenthesis EndLabel b EndLayout'
    actual: |-
      'left braceStartLayout1st Row  with Labelleft parenthesis1right parenthesisEndLabelbEndLayout'

Slightly more minimized: I run the mathjax+sre setup in the repository with first 'f(x) = y' and then \begin{empheq} [left = \empheqlbrace \,]{align} b \tag{1}\end{empheq}, the resulting top-level label for the second sample will be different. (And by "label" I mean data-semantic-speech for the "true" semantic root, cf #729.)

Is it possible that you are running this in parallel with Braille translation? That changes the speech separator from ' ' to ''.

Thanks for taking a look!

Is it possible that you are running this in parallel with Braille translation? That changes the speech separator from ' ' to ''.

Yes. The tests in the repo use the setup in https://github.com/krautzource/sre-to-tree/blob/master/test/tex2svg.js.
I'm switching back and forth  there. Perhaps I'm not resetting SRE correctly. I recall from another project (with a similar setup) that SRE's config had changed.
I'll take a look and report back.

I find the configuration options hard to understand.

Just to expand. I obviously found it difficult to understand SRE's behavior with these options. Primarily, I don't understand why braille is a modality - it seems distinct from the other modalities (if I read "speech" as "full string") but the result also doesn't seem useful for braille output.
In any case, I didn't meant to come across snarky - this was just an FYI.

SRE starts in a default configurations. The setupEngine changes these parameters, but it does not reset them. I.e., once modality is braille it remains so, until it is set to something else. Hence the effect you see.
I understand that the braille modality appears out of place. Initially it was thought that locales would come with speech, braille, etc. But that is rubbish, as tactile languages are locales in their own right. As such functionality that decides on modality === braille should be refactored into the locale setup in the future.

Thanks, Volker.


Add [v1] events commands

This adds new top-level events commands which use v1 of the API. The beta events commands are extant but now print a deprecation warning to stderr. I did not spend time removing duplication between the two; the old code exists in new files which is easily removed once we're ready to.
Reference docs: https://foxglove.dev/docs/api#tag/Events

Should we retire "beta events" right away, or add support for this new thing and deprecate the old thing for a while (perhaps with a warning to stderr or something)? The old beta endpoint will presumably be around for some time, since we have old client versions.

I updated to keep the beta version around with a deprecation warning.

@bryfox how does the map appear in the rendering?

how does the map appear in the rendering?

JSON, same as the original version:
https://github.com/foxglove/foxglove-cli/blob/1c547c28f011481e3e201a034eac978bddc9eaec/foxglove/console/api.go#L175-L176

I resolved conflicts with attachments additions on main.


Unblock Click to Load content

Task/Issue URL: https://app.asana.com/0/0/1204326356868967/f
Description
Steps to test this PR
Feature 1

[ ]
[ ]

UI changes

Before
After

!(Upload before screenshot)
(Upload after screenshot)

Current dependencies on/for this PR:

feature/josh/display-click-to-load-placeholders

PR #3142   👈

This comment was auto-generated by Graphite.


API - machines projection and datacenter

The API does not expose infrastructure (datacenter) information for Machines in the list projection.
This means the API cannot produce infrastructure filterable machine lists without an expensive number of additional API requests.
Request a projection to include linked key values (don't need linked lookups) in machines list.

Requirement met via new API.


feat: base package upstream of atchops and atcommons to carry shared platform macro resolution and utilities

Create a base package below atcommons / atchops which carries:

Naming ideas:

atc
atstdlib
atbase
atcore
NOT atsdk as that is a reserved name for the linked target of atlogger, atchops, atcommons & atclient

Utility functions like long_to_strlen for computing the length of a long when printed as a string
Platform specific override configuration can be in ONE place
Memory management utility functions can be added and have a single home
We have a place to extract and properly unit test repeated patterns that are not package specific

Memory management utility functions can be added and have a single home

As a side note, I'm very aware that this could cause A LOT of problems if done wrong. Especially if it's very macro dependent. I am in the process of researching if it's even a good idea, but my thought process is that if we use a consistent way to allocate memory. Then we can do analysis on all other packages to check if malloc/calloc/realloc are being called. This way we know that there is a single, consistent, and well tested way of handling memory allocations.


request: body format / add ticket(s) to body

Description
The auto-detection of tickets from the branch is such a great feature. Our team would love to use it, but we would want the ticket to be a part of the body and not the header.
Proposed solution
Allow the commit body also to be templatized like the header.
For example:
Given the following config:
"""
const czvinylConfig: Config = {
  headerFormat: '{type}({scope}): {subject}',
  bodyFormat: '{body}\n\nrelates to {ticketId}',
  skipTicketId: false,
};
"""
When the user provides the following answers:
   - type: feat
   - scope: config
   - ticketId: TICKET-123
   - subject: a sample commit
   - body: longer description
Then the final commit message should be:
"""
feat(config): a sample commit

longer description

relates to TICKET-123
"""

I haven't fully thought through how to handle an empty ticketId in json format. Ideally, the line using the variable would not be present. The alternative would be a better approach for us.
Alternatives considered
Have a new configuration field, bodyFormat that supports a function. The function would be fed all answers and return the final formatted body string. This would allow the implementor to be able to handle missing information gracefully.
Given the following config:
"""
const czvinylConfig: Config = {
  headerFormat: '{type}({scope}): {subject}',
  bodyFormat: (answers: Answers) => `${answers.body}\n\nrelates to ${answers.ticketId}`,
  skipTicketId: false,
};
"""
When the user provides the following answers:
   - type: feat
   - scope: config
   - ticketId: TICKET-123
   - subject: a sample commit
   - body: longer description
Then the final commit message should be:
"""
feat(config): a sample commit

longer description

relates to TICKET-123
"""

Do you want to create a pull request?
Yes

@jromero-pg
I cannot find any reason to allow a function input in the bodyFormat field.
Why would you expect an empty ticket ID if you ask for one in your configuration?
An empty ticket ID is not possible either way (if you ask for in the configuration..)

In an ideal world, the ticket would be optional. The function input would allow for the handling of any optional field.

In an ideal world, the ticket would be optional. The function input would allow for the handling of any optional field.

So if ticket ID can be empty, the headerFormat should accept a function as well.
Will be released in the next minor version.


Status bar style gets reset on pop

In our app we use UIKit Navigation + SwiftUI views wrapped inside UIHostingController
For some screens we need to override default status bar style by means of changing it inside UIHostingController instance
Using new SwiftUIIntrospect on some of the subviews makes status bar lose its style on popping from previous screens. Removing introspection resolves the issue.
Minimal example attached
IntrospectBug.zip
https://github.com/siteline/SwiftUI-Introspect/assets/61315957/4d7250a2-419a-46ac-b085-63485a175d77

Thanks for the exhaustive report, I'll look into it as soon as I can and get back to you.

@VladOrackle thanks for your patience. I was able to come up with a fix in #313. Before I merge it, I'd appreciate if you could confirm it works for you too by pointing to the branch fix-wrong-status-bar-style. Cheers.

@davdroman
It does work, but only sometimes.
In my minimal example I introspected viewController, but if we change it to some other component, scrollView for example, the fix strangely stops working
Updated example attached
IntrospectBug.zip

Oh right I see, it goes deeper than I thought then. I'll keep digging. Thanks for the heads up!

@davdroman forgot to mention that on iOS 15 & 17 it does work as it should without any changes. The problem seems to affect only iOS 16 as you mentioned before

@VladOrackle cool, good to confirm. I think I found the solution for all view types, if you point to the latest commit (b220886) in the branch and check I'd appreciate it.

@VladOrackle cool, good to confirm. I think I found the solution for all view types, if you point to the latest commit (b220886) in the branch and check I'd appreciate it.

Checked, and it works perfectly! Many thanks

Excellent! I'll write a UI test, merge, and tag a release shortly.


Problems with Sass

Here is a link to the warnings that arise when working

From this discussion it appears that this is an issue with Susy 2's set of mixins. I'll upgrade this to Susy 3 when I get the time, thanks!


Update dsn_levels.py (v.<IP_ADDRESS>)

guidelines removed
levels staggers added and tested
level spiral is there for collision logic testing

Also let's make the version update 0.1.8 next time instead of adding more digits

good idea but I would like to make more levels and fix the ordering of the levels in an easier way for the next patch


404 on RBAC page

https://kubernetes.io/docs/admin/authorization/rbac/ returns a 404
the page is still present and linked in _data/reference.yml, the same as ABAC, which works correctly:
https://kubernetes.io/docs/admin/authorization/abac/
any ideas about how to correct this?

Looks like a publish failed to roll out to all CDNs. Republishing fixed the issue.

For future reference---kicking off a redeploy at https://app.netlify.com/sites/kubernetes-io-master-staging/deploys fixes things.


HTTP Basic auth does not correctly decode passwords if e.g. "Ä" or some similar characters are included

First check

[x] I added a very descriptive title to this issue.
[x] I used the GitHub search to find a similar issue and didn't find it.
[x] I searched the FastAPI documentation, with the integrated search.
[x] I already searched in Google "How to X in FastAPI" and didn't find any information.
[x] I already read and followed all the tutorial in the docs and didn't find an answer.
[x] I already checked if it is not related to FastAPI but to Pydantic.
[x] I already checked if it is not related to FastAPI but to Swagger UI.
[x] I already checked if it is not related to FastAPI but to ReDoc.
[x] After submitting this, I commit to one of:

Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
Implement a Pull Request for a confirmed bug.

Example
Here's a self-contained, minimal, reproducible, example with my use case:
import uvicorn
from fastapi import FastAPI, Depends, HTTPException
from fastapi.security import HTTPBasicCredentials, HTTPBasic
from starlette import status

app = FastAPI()

async def auth(creds: HTTPBasicCredentials = Depends(HTTPBasic())):
    username = creds.username
    password = creds.password
    print(password)
    if password == "test" or password == "ä":
        return "OK"  # will not be returned, when bug occurs

    raise HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Incorrect email or password",
        headers={"WWW-Authenticate": "Basic"},
    )

@app.get("/", dependencies=[Depends(auth)])
def read_root():
    return {"Hello": "World"}

uvicorn.run(f"{__name__}:app", host="<IP_ADDRESS>", port=5000, reload=False)

Description
run following in linux terminal (or open page in browser, reset session between calls (private window))
syntax: command # expected response
curl -u username:password http://localhost:5000; echo  # should be '{"detail":"Invalid authentication credentials"}'
curl -u username:test http://localhost:5000; echo  # should be '{"Hello":"World"}'
curl -u username:ä http://localhost:5000; echo  # should be '{"Hello":"World"}'

Environment

OS: [e.g. Linux / Windows / macOS]: Linux
FastAPI Version [e.g. 0.3.0]: 0.65.1

To know the FastAPI version use:
python -c "import fastapi; print(fastapi.__version__)"

Python version: 3.9.4

To know the Python version use:
python --version

Additional context

The error is caused by the decode call after b64decode, because "ascii" is used as encoding:
try:
    data = b64decode(param).decode("ascii")
except (ValueError, UnicodeDecodeError, binascii.Error):
     raise invalid_user_credentials_exc

Can be found here

I am having the same issue, solved it by using custom HTTPBasic class that uses UTF-8 instead of ASCII (and I removed some checks I don't need):
from base64 import b64decode
from typing import Optional

from fastapi import HTTPException, status
from fastapi.security import HTTPBasic, HTTPBasicCredentials
from fastapi.security.utils import get_authorization_scheme_param
from starlette.requests import Request

class UTF8HTTPBasic(HTTPBasic):
    """Modified `fastapi.security.http.HTTPBasic` code."""

    async def __call__(  # type: ignore
        self, request: Request
    ) -> Optional[HTTPBasicCredentials]:
        authorization: str = request.headers.get("Authorization")
        scheme, param = get_authorization_scheme_param(authorization)
        unauthorized_headers = {"WWW-Authenticate": "Basic"}
        invalid_user_credentials_exc = HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Invalid authentication credentials",
            headers=unauthorized_headers,
        )
        if not authorization or scheme.lower() != "basic":
            raise HTTPException(
                status_code=status.HTTP_401_UNAUTHORIZED,
                detail="Not authenticated",
                headers=unauthorized_headers,
            )

        try:
            data = b64decode(param).decode("utf-8")
        except ValueError:
            raise invalid_user_credentials_exc

        username, separator, password = data.partition(":")
        if not separator:
            raise invalid_user_credentials_exc

        return HTTPBasicCredentials(username=username, password=password)

Just encountered the same issue. My application was asking again and again my credentials and I didn't get why at first. While debugging, I indeed saw that ASCII is expected. Maybe an encoding parameter could be added to the HTTPBasic.__init__ method? That would also impact the username though 😕


startAngle on radar chart value out of chart

i try to rotate my chart using startAnlge by 90 the value mark is running out the chart. Not sure it is i miss out any option setting to the chart but i did some google unable find any soultion to it.

@jadenjm do happen to have a code sample? It would help out a lot

yes i trying on sample code radar-skip-points.html with some edit.
i had tried on sample code with adding startAngle: 90 in the option it will show the bug too.
Here is the code:
<!doctype html>

Radar Chart

<script>

var config = {
    type: 'radar',
    data: {
        labels: ["Right", "Right-Bottom", "Left-Bottom", "Left", "Left-Top", "Right-Top"],
        datasets: [{
            label: "Skip first dataset",
            borderColor: 'rgb(255, 0, 0)',
            backgroundColor: "rgba(255,255,0,0.5)",
            pointBackgroundColor: "rgba(220,220,220,1)",
            data: [10, 20, 30, 40, 50, 60]
        }]
    },
    options: {
		startAngle: 90,
        title:{
            display:true,
            text:"Chart.js Radar Chart - Skip Points"
        },
        elements: {
            line: {
                tension: 0.0,
            }
        },
        scale: {
            beginAtZero: false,
            reverse: false,
			ticks: {
			//display: false,

			}

        },

		//animation: {
			//onComplete: function (animation) {
			//this.toBase64Image();
			//console.log(animation.chartInstance.toBase64Image();
			//},

    //},
	}
};

window.onload = function() {
    window.myRadar = new Chart(document.getElementById("canvas"), config);

};

//save64Img(myRadar.toBase64Image());
</script>

<?php

//$data = base64_decode(preg_replace('#^data:image/\w+;base64,#i', '', $img));
//file_put_contents('../image.png', $data);
?>

Is there anyway we could make this optional? I like having the labels be vertical despite the angle of the chart. Maybe even have a tick option startAngle where we could angle the chart one way and reangle the ticks a different way. Thoughts?


crop faces

log size and location of bounding box, rotation angle, etc. get distributions on these statistics
compute diversity metrics on cropped/centered/rotated/aligned faces
yo. i bet i could invert idea and use a facial pose controlnet to just get ultra standardized face portraits.
that would run the risk of introducing new biases from the controlnet. could benchmark against non-controlnet methods (i.e. gptdiversity128)

i bet with a controlnet, we could compute strong metrics on just a handful of faces.
oh, the controlnet version might also make DINOv2 [CLS] a more viable representation. would be nice to be able to evaluate this with a fairly "independent" representation, which CLIP definitely is not (as a component of SD, it's text encoder, i.e. the CLIP representational space is part of what SD is learning/aware of, so it makes sense that images cluster so well in it).


[PL-47012] PL 1.25.5 Release Notes

PL-47012
https://harness.atlassian.net/wiki/spaces/PD/pages/21650997249/NG+Release+SAAS+February+12th+2024+for+1.25.x-+Build
xBUILD LINK HEREx
https://65d2ee1fb12513d783bdb862--harness-developer.netlify.app/
What Type of PR is This?

[ ] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[x] Tested Locally
[ ] Optional Screenshot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cb3aa51229917e7e04db6c--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cb5997a8788a007f526884--harness-developer.netlify.app

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cc8c43c5803d907b44b632--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cc8d47cf4a75919f3a8d27--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cc99f8cab67e9c3215acf0--harness-developer.netlify.app

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cdc1976215e77d163d505d--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cddee4d2a5fb9c7d14aaab--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cddf1f7769f297300d1530--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cde5726215e7a2f23d523f--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cde66e7769f29f040d14c9--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65ce31261ee81402bd3c69d1--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65ce43ca9607ec111ccf887d--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cefbb20eb88281e5cea45f--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65cefc1165788b818d36c0a1--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65d2ee1fb12513d783bdb862--harness-developer.netlify.app

@pratmit Requesting review

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65d34328e9eda6007a03bd65--harness-developer.netlify.app

I have updated the "This item is available with Harness Platform version 1.25.5 and does not require a new delegate version. For information about Harness Delegate features that require a specific delegate version, go to the Delegate release notes."
Need approval again. Thanks
@pratmit @brian-f-harness

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65d46902c50afca3ad87ae75--harness-developer.netlify.app


Dev/aozgaa/tsserver project test organization

Example change of what is proposed in #20290.
Most of the diff is reindentation -- look at the first commit for the semantically meaningful changes.

@weswigham @mhegazy does this look good?


Stars for completing a day in under 3 hours doesn't work

The feature to show different stars depending on different conditions which was introduced in PR #8 also added an ✨ emoji when the Day was completed in under 3 hours. but since some time the emoji doesn't work anymore and instead a normal ⭐ is shown, though it did work when it was merged. The problem was first noticed yesterday ( 2020-12-11 ) but may have been unnoticed for longer.
Since PR #19 hasn't changed anything which is currently implemented and only added a file and a dependency we can be pretty sure this is not the cause.

It may be a calculation error or something

at first I thought it could be this commit: 3cc00f06130e82ea7486e02b69e8eb2f5a0b8648 , since it moved the function to get the Date object of the beginning of the challenge into the new common.ts file, so both Leaderboard.ts and ChannelUpdater.ts share the function and don't have to have their own implementation of it. But when looking at it, the common.ts file is imported, but not used, - or im just blind. the only changes in the Leaderboard file are formatting changes... so maybe it is not this commit... tough it seemed most likely at first.

That's the issue then I think 😆


Update to reflect GSoC acceptance, idea change

The doc site proposal has been moved to another section, to reflect feedback
from Google that it isn't a good match for GSoC.
Signed-off-by: Brian Warner<EMAIL_ADDRESS>

👍


Dashboard: 42Crunch Integration

As a user I would like to be able to detect security vulnerabilities in my API.
UX Design:
Add policy
https://www.figma.com/file/wecNPLnwPgaQiZaXDWolSa/Kusk-Gateway-Dashboard?node-id=3740%3A45114&t=q1QoyQzWlDExegJk-4
View 42Crunch link and details
Option 1 (my perference)
https://www.figma.com/file/wecNPLnwPgaQiZaXDWolSa/Kusk-Gateway-Dashboard?node-id=3811%3A44575&t=q1QoyQzWlDExegJk-4
Option 2
https://www.figma.com/file/wecNPLnwPgaQiZaXDWolSa/Kusk-Gateway-Dashboard?node-id=3811%3A44634&t=q1QoyQzWlDExegJk-4

Implementation details are still unknown while we are waiting on their documentation. Initial version implementation proposal is here (WIP) https://kubeshop.atlassian.net/l/cp/RsKCdfho

@mortada-codes I believe this is not done

blocked on #998


Machine-readable progress updates

This adds a progress tracker to benchmark runs, which can be used to print out machine-readable progress updates (e.g. for modellab) by setting the --machine-reports flag. If this flag is set, none of the regular print statements will be logged either, so as to not confuse the machine. To accomplish that, I had to replace the click.echo statements with logging statements.
Right now the progress tracker just prints a dictionary containing % complete. But we can easily add more information to it in the future if needed.
Progress can only be measured at the test-level at the moment. If we decide that this generally the direction we want to go in for progress reporting, then I can go ahead and update modelgauge so that progress can be calculated at the test-item level.
In addition to higher-level feedback, I’d love to hear input on some annoyingly small stuff:

Is there a better place for ProgressTracker other than utilities? It feels weird putting it in it’s own file since it’s such a small class.
Is the param name machine-reports an adequate descriptor

:)

I notice, when running this using --parallel True, that the completion doesn't always increment upward.

@dhosterman  Ah good catch, I'll fix that


Backend Docker/keycloak

containers... the present docker-compose here needs to be changed to use treafik.

minio
traefik (what we use)
keycloak (hosted at NCSA, or a docker

Notes on how the UI and minio should interact:
Minio docker needs to be configured/populated with the policy to restrict users to a 'home' directory
keycloak, because that will handle the registering of users, I believe. Otherwise we need to create a quick api to create users. We can reuse keycloak as a user login for jena fuseki in the shiro.conf (I think)
trafik examples in geodes:
https://github.com/earthcube/geodex
Postman tests for the policies are in:
docker/MinioTests.postman_collection.json

Policies
By User
For a user with no group, this will set a policy that publishes to thier directory
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": ["s3:ListBucket"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::users"],
      "Condition": {"StringLike": {"s3:prefix": ["${aws:username}/*"]}}
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::users/${aws:username}/*"]
    }
  ]
}

Group
this will restrict a group to a folder called genericGroup, in users
when creating a new policy, we need to change the name genericGroup.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": ["s3:ListBucket"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::users"],
      "Condition": {"StringLike": {"s3:prefix": ["{{GROUPNAME}}/*"]}}
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::users/{{GROUPNAME}}/*"]
    }
  ]
}

Minio possible setup:
https://stackoverflow.com/questions/66412289/minio-add-a-public-bucket-with-docker-compose
https://www.keycloak.org/getting-started/getting-started-docker

If you go to this page, you can see a keycloack realm
https://iam.scigap.org/auth/realms/ddstudio/
{"realm":"ddstudio","public_key":"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA65jQRrUptVEX0aeAcvbDkXSNQzbzEGz2nH+9odiGmmOA/Ao9P9400dqHmF2NImOCIrTAMjMV+/rNyvA/xT+BjF+ycu+QdD+UjalntzZ2op9biQKbYWbjWbrBjVoIZ+r7Mr8llGU7J12yti4NmSOhF0FKZHrxgTiTaYboI7miGXMaVN2KGinbWvreh29r6cM5+6bPWVN3RHilu4zJX/b6WDk1yxy5skmjXws+/DYk0LJ/lCgLrkbuvdlPqH1zIVqPm2LYfP2y4t2ylWbxa51YHgGoCgZ9anHlvOxQJa/58MkJgWGGZbobuuwGtw94tCs1emBNHISqFFrIjIIlNXQNoQIDAQAB","token-service":"https://iam.scigap.org/auth/realms/ddstudio/protocol/openid-connect","account-service":"https://iam.scigap.org/auth/realms/ddstudio/account","admin-api":"https://iam.scigap.org/auth/admin","tokens-not-before":0}

Create a user self registration: https://www.baeldung.com/keycloak-user-registration
keycloak vue: https://github.com/dsb-norge/vue-keycloak-js


Can I inference models with XNNPACK backend through tensorflow lite Python APIs?

One of my areas is model compression for deployment. Previously, I worked on QNNPACK and its performance was amazing. I knew that the main author of QNNPACK went to Google from FB, and he continues his work based on QNNPACK. A new library named XNNPACK was created. I am very excited to see the Sparse ConvNets feature of XNNPACK. XNNPACK suggests the TensorFlow lite in its readme. However, I did not find a specific doc or APIs for this feature. So, can you provide some examples of the usage of XNNAPCK?
Specifically, I want to inference the models with XNNPACK backend on CPU no matter what platform I am. Like QNNPACK, I can do some experiments on x86, while I can run on Arm (such as NVIDIA Xavier, TX2, etc) when I want to deploy models.
For your reference, you may learn something from PyTorch quantization feature, by which I can use same APIs to invoke QNNPACK backend on different platforms (x86 or Arm).

This isn't yet possible, but we plan on enabling XNNPACK by default in the near future, so you shouldn't need to do anything explicitly on your end.

@jdduke Is there a way to invoke xnnpack through load_delegate()? Namely, what shared lib should be pointed to and how do we build it, if required. I'd built from master with the tflite_with_xnnpack flag.

If you build with the --define tflite_with_xnnpack=true flag, then you shouldn't have to do anything manually. The delegate should be applied by default.

@jdduke, hey, it’s been a year — what’s the current status of the delegate? Is it enabled by default or still requires building?
Also — is it possible to see TF logs while executing a TF Lite model via the Python API? I see a silent output while executing so I’m not sure what’s going on under the hood.

I'll let @srjoglekar246 and @multiverse-tf provide an update.

Just checked, it still requires building from src to enable XNNPACK

Hi @vippeterhou !
We can build and run XNNPack delegates for  Desktops now with x86/x86_64 flag now.
#Build command with different architecture
bazel build -c opt --fat_apk_cpu=x86,x86_64,arm64-v8a,armeabi-v7a \
  --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
  --define tflite_with_xnnpack=true \
  //tensorflow/lite/java:tensorflow-lite

TensorFlow Lite interpreter will use the XNNPACK backend by default in Desktops.
For detailed usage , Please check official documentation on XNNPack.
Thank you!


index out of bound when build sampling pattern in dtObstacleAvoidanceQuery::sampleVelocityAdaptive

This function declare a array of float pat float pat[(DT_MAX_PATTERN_DIVS*DT_MAX_PATTERN_RINGS+1)*2]; with DT_MAX_PATTERN_DIVS is 4 and DT_MAX_PATTERN_RINGS is 32., so the max valid index should be 257.
When we set nr to 4 and nd to 32.  The last iteration to fill pat would cause index out of bound.
if ((nd&1) == 0)
{
        // before exec the npat is 4*32  = 128
	pat[npat*2+2] = last2[0]*ca - last2[1]*sa; // index is 258
	pat[npat*2+3] = last2[0]*sa + last2[1]*ca; // index is 259
	npat++;
}

The right code should be
if ((nd&1) == 0)
{
	pat[npat*2+0] = last2[0]*ca - last2[1]*sa;
	pat[npat*2+1] = last2[0]*sa + last2[1]*ca;
	npat++;
}

It's true. The last iteration is almost never reached, and so it goes unnoticed.


[Bug] [Module Name] Cron manage: no instance host

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
I have set a timing scheduler for a task. However, dolphinscheduler cannot dispatch this task within the set time. The status of the task is running but the log shows an error: no instance host. The magic is that I can execute this task directly by the START button and everything is normal except I cannot manage it using the time scheduler
What you expected to happen

How to reproduce
I doubt the time scheduling settings after upgrading from v2.0.6 are not compatible with v3.0.x. It happens in the originally v2 designed time schedule, there is no problem in recreating one with v3
Anything else
No response
Version
3.0.x
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

can you post some more detailed logs?

Please make sure this issue exists in dev and 31x.

My version is 30x. But someone said it also appears in 31x

@songjianet 3.1.0 version has the same error, hope fix because this  is important use in work. thanks a lot
file LoggerServiceImpl.java

getHost() is NULL, host field in table t_ds_task_instance is NULL


feat: reload job status every 5 sec. and show monitoring message

Reload job status
Closes #72
Changes made:

Add effect to reload job status every 5 sec. The time is stored in PAGE_REFRESH_RATE_MS constant
Show message "Monitoring state change..."
Created DotsLoader component to show the message
Not related: increased width of workflow section to fit all buttons in one row

Example

@sverhoeven I have refactored code to use env variable HADDOCK3WEBAPP_REFRESH_RATE_MS.
The variable is added to .env file in the deploy/arq.
Let me know if you have any other suggestions


Make the secondary buttons more obvious

It was kind of hard to tell that they were buttons before.
I also de-emphasized the Browse button on selected rows to help suggest that most users don’t need to use it. (Maybe it should say something else like “Custom Path…”?)

The general UI of the installer is meant to match the design of the site. I'll consider making some changes to the secondary buttons to make them more obvious, but generally this isn't what im aiming for.

Will keep this open for discussion since this is a subjective thing.


Move assets (e.g. fonts) into the service

Move assets into the service so issues with external services don't affect ours. We've experienced some issues with page load being blocked, waiting on fonts.gstatic.com.

Created a new branch: https://github.com/StatCan/kubeflow/tree/9-add-fonts-to-assets where I created a new folder in components/centraldashboard/public/assets containing all the .woff2 files for the fonts used.
Then, referenced those instead of the remote ones in components/centraldashboard/public/styles.css.
Haven't created a PR since I can't fully test this yet.


v0.13.0 在树莓派上部署后，memos显示时间不对

Describe the bug
Using the latest image arm/v7 v0.13.0 after Raspberry Pi deployment, the creation time is displayed incorrectly
Steps to reproduce
docker-compose.yaml
version: "3.0"
services:
  memos:
    image: neosmemo/memos:0.13.0
    container_name: memos
    restart: unless-stopped
    volumes:
      - ./data/:/var/opt/memos
    ports:
      - 5230:5230

Screenshots or additional context

建议尝试部署环境系统时间

建议尝试检查部署环境系统时间

The host must have date and time set-up correctly. Use a ntp server to keep host time accurate.
Try to specify the timezone manually and point the host localtime to the container, like this:
version: "3.0"
services:
  memos:
    image: neosmemo/memos:0.13.0
    container_name: memos
    restart: unless-stopped
    environment:
      - TZ=Asia/Shanghai
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/:/var/opt/memos
    ports:
      - 5230:5230

Note that the displayed date is formatted by the browser in use, not the server. You can test using another browser and/or changing memos language and refreshing the page to check if the date is shown correctly.
Also, you may use Alt+Click over the date to change it and check if, somehow, this fixes the display.

@lincolnthalles  Thank you for your reply, but I tried this way, the result is still wrong. I have also run the linux/amd64 docker image, but it is normal.

It seems you are not the only one having this issue.
Unfortunately, I don't own a Pi or other ARM SBC to help investigating this.
Important question: This issue affects every created memo or only the sample memos?
It would be helpful to confirm if the memo table is being populated with proper unix timestamps.
The values in those database columns must be something like "1684156803" (2023-05-15 10:20:03), which can be converted to a readable date and time this way:
# Linux bash
date +"%Y-%m-%d %H:%M:%S" -d @1684156803

# Windows PowerShell
[System.DateTimeOffset]::FromUnixTimeSeconds(1684156803).DateTime

Confirming the container date is current
sudo docker exec -it memos date

This should output something like "2023-05-15 19:40:03"
Inspecting memo created and updated timestamps in the database
# Run this in the docker host to get a shell inside the container
sudo docker exec -it memos /bin/sh
# The shell prompt should become "/usr/local/memos"

# install SQLite shell
apk add sqlite

# load memos database in sqlite shell
sqlite3 /var/opt/memos/memos_prod.db

# inside the sqlite shell (sqlite> prompt):
SELECT id, created_ts, updated_ts FROM memo;

The output should be in this format:

If the output format is different, it means the database is being populated incorrectly and that would be a good lead to developers sort the issue.

@lincolnthalles Thank you for your reply. All created memo time displays are wrong. But now I don't have time to explore the specific cause of the error, I will try to look at the source code on the weekend.

@gabstehr  You are correct.
You mentioned libseccomp2 reminds me of the problem that the container cannot start when using the node-red v2.x version of the container image, which is also caused by libseccomp2.
In addition to the solution you gave, I also thought of another : use --security-opt=seccomp=unconfined
version: "3.0"
services:
  memos:
    image: neosmemo/memos:0.13.0
    container_name: memos
    restart: unless-stopped
    security_opt:
      - seccomp:unconfined
    volumes:
      - ./data/:/var/opt/memos
    ports:
      - 5230:5230

After running it is normal time, except for the first default memo.


DRILL-8443: Upgrade netty due to CVE

Description
https://github.com/apache/drill/security/dependabot/45
Testing
CI build

Some tests failing with this error - IO Running in secure mode, but config doesn't have a keytab
Seems like a Hadoop issue. We may need to upgrade Hadoop or at least review the Hadoop config used in the tests.

@pjfanning I think we may have a CI issue.  @vvysotskyi @jnturton Any ideas here?  Could this be related to that issue we encountered before with the CI and connections?

Now tests for this PR fail for other reason (likely due to changes in the PR):
Error:  Failures:
Error:    TestResultSetLoaderOmittedValues.testOmittedValuesAtEndWithOverflow:264 Row 0 col d should be null
Error:    TestResultSetLoaderOverflow.testBatchSizeLimit:164 expected:<16385> but was:<8193>
Error:    TestOutputBatchSize.testSizerRepeatedRepeatedList:2922 expected:<1048576> but was:<1048560>
Error:  Errors:
Error:    TestResultSetLoaderOmittedValues>SubOperatorTest.classTeardown:39 » IllegalState Allocator[ROOT] closed with outstanding buffers allocated (9).

@pjfanning @cgivre, @rymarm has made progress on this in #2857. Shall we close this one and try to get that one over the line?

sure - I haven't been looking at this


Dropdown erro

Hello, my dropdown doesnt work.
I'm getting a firebase list and trying to generate a Dropdown list but this error returns to me.
Already researched in several forums and no way solved my problem.
Error:
The following assertion was thrown building Cadastro(dirty, dependencies: [MediaQuery, _EffectiveTickerMode], state: _CadastroState#295e7(tickers: tracking 2 tickers)):
There should be exactly one item with [DropdownButton]'s value: .
Either zero or 2 or more [DropdownMenuItem]s were detected with the same value
'package:flutter/src/material/dropdown.dart':
Failed assertion: line 1478 pos 15: 'items == null || items.isEmpty || value == null ||
items.where((DropdownMenuItem item) {
return item.value == value;
}).length == 1'
The relevant error-causing widget was:
Cadastro file:///home/ir4ever/Documentos/rise_delivery/lib/RouteGenerator.dart:69:19
When the exception was thrown, this was the stack:
#2      new DropdownButtonFormField (package:flutter/src/material/dropdown.dart:1478:15)
#3      _CadastroState.buildCadastro (package:rise_delivery/telas/Cadastro/Cadastro.dart:910:57)
#4      _CadastroState.build (package:rise_delivery/telas/Cadastro/Cadastro.dart:510:14)
#5      StatefulElement.build (package:flutter/src/widgets/framework.dart:4744:28)
#6      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4627:15)
...
My code:
List<DropdownMenuItem> _listaItensDropBairros = List<DropdownMenuItem>();
List valorDropDownBairro = List();
String _controllerBairro;
FirebaseFirestore db = FirebaseFirestore.instance;
await db.collection('disponiveis').doc('estados').get().then((value){
    valorDropDownBairro = List<String>.from(value.data()[_controllerUF.toLowerCase()][_controllerCidade.text.toLowerCase()])
                                                    ?? List<String>(); //returns the list correctly
  if(valorDropDownBairro.isNotEmpty){
     for (String bairro in valorDropDownBairro) {
     _listaItensDropBairros.add(DropdownMenuItem<String>(child: Text(bairro), value: bairro));
     }

  } else{...}
 }

DropdownButtonFormField(
validator: (valor) {
return Validador()
.add(Validar.OBRIGATORIO, msg: 'Campo Obrigatório')
.valido(valor);
},
value: _controllerBairro,
style: TextStyle(
fontSize: size.textPequeno,
color: Colors.grey),
hint: Text('Bairro ou Distrito'),
onChanged: (valor) {
setState(() {
_controllerBairro =
valor;
});
},
items: _listaItensDropBairros,
)

@Raphael-Jose-Ferraro-IRForever
It seems you are asking for assistance on personal code. Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow and if you need help with your code, please see https://www.reddit.com/r/flutterhelp/
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


Can the ends of the progress bar be round?

Can the ends of the progress bar be round?

Hello,
At the moment isn't possible, but feel free to contribute :)
Thanks.

Now available in v1.3.3


Can I use Putty for tunneling?

Hi,
I'm working under Win 10, and totally new to vs code.
I have Ubuntu VM running as a guess os, and I want to edit my app that deployed there.
What I have done:

Your plugin installed in my vscode
I installed rmate in my Ubuntu
I configured my User Settings in my vscode

but I confused with:
ssh -R 52698:<IP_ADDRESS>:52698<EMAIL_ADDRESS>Should I install ssh in windows in order to make tunneling command works? I have Putty installed already.
Can I just use Putty tunneling instead?
Thanks
P.S. Nice work :)

Just use Putty tunneling, should be the same.