Block devices, (i.e. /dev/sda) ?

Is there any ETA on when we can see block devices such as sda, for tools like hdparm ?
Thanks

Thanks for the feedback. @russalex or @bitcrazed might be able to comment on any ETA's (or the lack of it). Also, please also make any feature requests on our User Voice Page and help us prioritize.

Unfortunately I can't give an ETA on this one.  We know there is a desire to be able to access block devices from WSL but it is not terribly high on our priority list at the moment.  As @sunilmut points out, the best thing to do here would be to add something to our User Voice page.  Little secret, one thing that always helps the requests there is explaining the actual scenario you're looking to enable.

@russalex Is there really a point to this kind of thing? It's in the nature of the thing that it is going to be served a pre-assembled abstracted filesystem.  Loading up something like dd while Windows mounts the disk (in non-raw mode) seems like it could be a problem, and this kind of thing seems to require effectively writing hardware drivers that negotiate hardware access between WSL and WSWin32

@fpqc, While we know there is some desire for this feature, it's not been enough of a priority for us to discuss planning or implementation.
What is interesting to me is to hear what people want to do with these types of features.  The users of this forum often surprise me with their creativity and the more information we get the better.  I have found that it never hurts to at least hear someone out.  If something is compelling enough, and enough people want it, then we should at least look into feasibility and see where we may be able to fit it into plans.

I would love to have access to block devices from WSL but it would require strict restrictions (like disallow write from devices mounted from windows/wsl). There is a high risk of corruption if a peripheral is already mounted :(

Well actually, raw disk access would make it possible to leverage device-mapper, mdraid and LVM to access any existing linux disks/logical volumes in/on your system.
I agree that it should be restricted to disks not already mounted.

@finkelamein idk about that, I think the LVM driver is baked into the Linux Kernel.

It's device-mapper driver that's in the kernel, but yeah, that would be needed.
~$ sudo pvs
/dev/mapper/control: mknod failed: Function not implemented
Failure to communicate with kernel device-mapper driver.
Check that device-mapper is available in the kernel.
/proc/devices: fopen failed: No such file or directory
Failed to create lvm type filter

But there would be need of something similar anyway to support access to disks like in the original question by OP.

I would love to have access to partition types that don´t have native support on Windows.( e.g. mount a ext4 partition)
Another user case is to use "parted" for modify removable disks (after desatach them from  using the disk manager, of course)

@jdevora This would necessarily be beyond the scope of WSL.  The first thing would be the extension of WSL's ability to mount different partition types in general, maybe with a way to query the NT kernel about "FS capabilities" (which I suspect already exists, but probably does not exist with information about Linux FS feature support).  Then you would also need to write a kernel-mode driver (or maybe somehow use FUSE?) supporting ext4. The first thing might be within the scope of WSL.  The second probably isn't.

At least fuse will be great. With fuse we will be able to implement\emulate somehow everything other linux specific filesystems.

On Cygwin I can use/dev/sdXX files directly.
This allows me to manages the fileystems of embedded devices runing linux directly (for example I can run something like mkfs.ext2 /dev/sdc1or copy a firmware to the ꜱᴅ card using dd)
Would it be possible to have the same feature on Windows subsystem for Linux ?
Please note I don’t want to be able to mount the filesystems, but just read and write to their device files. Like on Cywgin.
Otherwise, wsl isn’t a good platform for building and loading firmwares on the ꜱᴅ card.

Am I understanding properly that even if MS found a way to allow the subsystem direct device access, standard linux filesystem kernel modules would not be usable with the subsystem's customized kernel?
I was hoping we could mount the foreign filesystems which windows does not support otherwise and then export them with smbd - like it was possible to do with the old colinux project - but I guess this is too far off into the future for now.
If fuse support in LXSS is easier to implement, being a single thing as opposed to the various possible linux kernel filesystem drivers, it could be pretty great - even though core linux filesystems that have kernel modules don't seem to have fuse drivers yet because there was never any need for them, they can probably be developed once there is a use-case for them in LXSS.

Yes, as it's not a real kernel, filesystems included from linux kernel will not be usable (or MS have to rewrite them). But if they implement fuse, I think we will see ext2/3/4 comming to fuse :)
And yes, it would be exportable to SMB :D

@georgir That's my general understanding. LXSS does not have a kernel and integrating linux kernel modules with the Windows security-model would be really difficult.

If folks here would like FUSE support (I would), you can up-vote it here:
https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/13522845-add-fuse-filesystem-in-userspace-support-in-wsl

One use would be to mount micropython devices so that you can drop copy .py files from the local filesystem to the on-board memory without having to have another linux machine lying around.

Pretty cool use case eevans - I haven't thought of anything like that.
FUSE in userspace is also great in general.

@eeevans Can't you do that using the new support for removable drives?

@sundhaug92 probably not. That is specifically in the case of fat32 and ntfs removable media.

My use for such devices is to be able to use tools like cdparanoia  and abcde so that I can rip my audio cds directly in Windows vs. using a Linux box.

@res0nat0r - as a matter of interest, if you're running on Windows, why not just use one of the multitude of audio free and/or open-source tools for Windows?

@bitcrazed Honestly it was just so that I could re-use my abcde.conf. I can rip/tag everything with a simple abcde -N without doing anything else. I'm trying to rip my ~1000 cd collection so that I can get rid of all of the discs and was wanting to do it without rebooting my Windows box into Ubuntu.
EAC + Picard on Windows though will also do the trick.

Understood. Supporting USB CD/DVD drives isn't high on our backlog - I recommend you look to other tools; there are plenty for Windows too ;)

No problem, thanks for the feedback!

@res0nat0r - for ripping cd's I recommend CygWin, for the time being.
Bash on Ubuntu on Windows has its own advantages, but in CygWin, block device access works. To rip a CD in your primary CD drive to an iso file in D:\Temp, just do something like
dd if=/dev/sr0 of=/cygdrive/d/Temp/filename.iso bs=2K
I've created and restored images of SD "drives" for Raspberry Pi in the same way, just picking the right raw disk in /dev.
You can install Bash on Ubuntu on Windows and CygWin side by side on the same machine, no problems encountered so far.

@lucvdv Thanks!

Use case: being able to inspect the handful of bootable usb thumb drives I have lying around to see what distro of linux/bsd they contain instead of windows being all "this drive must be formatted before it can be used"

Thanks @sqweek - we agree that'd be a useful use-case, but it'd also mean we'd have to implement EXT2/3/4, RaiserFS, UFS, ZFS, etc. which, as you can imagine, is not a simple task.

but it'd also mean we'd have to implement EXT2/3/4, RaiserFS, UFS, ZFS, etc

Or implement that "other" popular request. Or address #579. Or implement ptrace(PTRACE_SYSEMU) and address #2555. In decreasing order of complexity from re-implementing ext234.

There's so much more to it than that though. Even EXT2FSD disclaims:

Don't use Ext2Fsd 0.68 or earlier versions with latest Ubuntu or
Debian systems. Ext2Fsd 0.68 cannot process EXT4 with 64-BIT mode
enabled, then it could corrupt your data. Very sorry for this
disaster issue, I'm working on an improvement.

Open source is great & all, but customers have expectations of software shipped by the likes of Microsoft. "No Data Loss" is one of them ;)

customers have expectations of software shipped by the likes of Microsoft.

Nah, that's clearly specious. Ext2Fsd is not a Microsoft product and you are not (would not be) shipping it. You can install all manner of things (both from MSFT and not) that will scramble your data. Under this line of reasoning MSFT would disallow 3rd party filesystems in Windows. If a person doesn't like the quality of a particular gratis software implementation, then you pay for it and call a customer service representative.
[I should clarify that if you have Ext2Fsd or paragon then squeek's use-case is addressed with or without WSL. #579 was brought up because it is an outstanding bug and related to his ask, but isn't directly related to the OP's request for block devices. The other two routes forward that I mentioned would require block devices, but not an ext234 re-implementation.]

Thanks @sqweek - we agree that'd be a useful use-case, but it'd also mean we'd have to implement EXT2/3/4, RaiserFS, UFS, ZFS, etc. which, as you can imagine, is not a simple task.

Why??
I think you that you only have to provide a way to inform Windows to not handle certain drive and let the WSL Kernel to handle it directly.
IIRC, this "don't handle this drive" is already possible in Windows; you can use it for allow a Virtualization Software to handle an specific drive directly.
On the WSL side, the file system will be handled by the same old and trusted code that does it  on any Linux distribution; with the same "guaranties" you get from a Native Linux OS.

@jdevora WSL does not use the Linux kernel and thus cannot use Linux kernel drivers

Is there any chance that usb devices will be accessible from WSL in the near future? My case is to use SSH with private key from yubikey - it would be nice to be able to use just WSL for that.

My case is to use SSH with private key from yubikey

That's #2568 aka #412

@sundhaug92 You are right, I forgot about that :-(
Isn't there an easy way to implement some kind of proxy that will avoid to duplicate coder (aka File Systems) that have been working for years?

easy way to implement some kind of proxy

Easy, no. Possible, sure. USB over IP is a thing. But nothing consumer friendly you can drop into WSL that I know of.

I think people are conflating things quite a bit (accessing HID-devices like Yubikey, reading Linux filesystems (filesystem drivers => unrelated to WSL), accessing block devices under /dev (dd), sending SCSI commands to devices (hdparm)).
I think of these different requests, reading and writing block devices like Cygwin can do is not an unrealistic thing to ask for. I have looked at the Cygwin source some time ago, because I needed to learn how it is done for some other project.
I don't know how easy it would be to support hdparm because I guess there might also be some ioctls involved that would all need to be implemented realistically.
Together with FUSE-support one could even solve the filesystem-issue, because there are working FUSE implementations of Linux filesystems.

Thanks @sqweek - we agree that'd be a useful use-case, but it'd also mean we'd have to implement EXT2/3/4, RaiserFS, UFS, ZFS, etc. which, as you can imagine, is not a simple task.

ZFS has already been ported to Windows as a proof of concept. It is in early development.
https://github.com/openzfsonwindows/ZFSin
If you get FUSE support working (with character and block device support working via FUSE too) on top of implementing block device support, there is a good chance that the ZFS support will resolve itself via that. ;)
That said, I found this issue because I wanted to know whether smartctl to get SMART status and ddrescue to image a disk could be run from the WSL for data recovery purposes on a modern Windows machine. There is no chance of that happening without block device support though.

Is it possible to use something like /mnt/c/{DEADBEEF-DEAD-BEEF-DEAD-BEEFDEADBEEF} to get access to \. (such that you can then access \.\RawDevices) in a completely unsupported and potentially dangerous way?

As it was already explained elsewhere, those magic names are interpreted by Explorer only.

I need this feature to flash firmware on block devices using Unix tools!
Please please please 😃

hdparm would be the only way to send certain important low level commands to devices, like set readahead, lookahead, APM, acoustic mgmt, prefetch, write-caching, and especially issue the ATA security-erase command with
--security-erase    PASSWD  Erase a (locked) drive.
--security-erase-enhanced PASSWD   Enhanced-erase a (locked) drive.

Thanks @sqweek - we agree that'd be a useful use-case, but it'd also mean we'd have to implement EXT2/3/4, RaiserFS, UFS, ZFS, etc. which, as you can imagine, is not a simple task.

Well, I for one WISH that windows WOULD EXPAND their filesystem support beyond the old and crufty FAT(and variants), and NTFS(and variants).  At a minimum it would be nice to have ext3, and ext4 support builtin(standard). This is really an area that the NT kernel and subsystems needs a major overhaul in... actually now that I think of it, the entire NT kernel probably could do with an overhaul rather than spending the time on WSL... WSL is nice and convenient, but I can ALWAYS run a VM or just boot into linux(like I usually do since WSL still isn't much more than a lab rat even now...)
Use case, dd to create and write imaged storage from and to block devices.  Most windows utilities in existence don't always seem to be, well, reliable, although that may be on windows itself... granted these are ALL niche use cases

I would absolutely love to see this implemented so i can stop fiddling with VMs and passthrough and dualbooting and all of that.

I ran into this today trying to create a bootable USB stick to update the BIOS on an old computer. Tried to do it in Windows 10 but not working, so took a run at it in Ubuntu 18.04.3 LTS running in WSL according to these intructions: https://tojaj.com/lenovo-biosuefi-update-from-usb-stick-i-e-without-bootable-cd/.  I ran aground because the USB stick does not show up as a device in WSL. Now I have to go find another old computer and set it up as a Linux box so I can do this.

Now I have to go find another old computer and set it up as a Linux box so I can do this.

That's one way to go. You'll probably want a bootable USB stick to install Linux on that old box. Rufus is pretty popular for doing that.

I have a use case for this although it's not a strong need. I develop for embedded devices that run Linux and I use WSL for compiling and running unit tests because it's much faster than using a VM. For one particular task I need to hash the MBR as a security audit for the embedded devices. I can not currently run the unit test for this because I can't discover anything at /dev/sda. If I just had read access to the MBR on C drive that would be possible. I don't need support for any filesystem that might actually exist on the drive, just block (S_ISBLK) device access.

If all you need is the hash the mbr, in the unit test, stick it on /dev/loop*, which is available in WSL2.

Allow for audio playback. For example a remote audio player.

This was the main reason I opted into Insiders and moved to WSL2, it's the only thing I still use Cygwin for and I assumed it would be possible in WSL2 so I can finally retire Cygwin. I regularly make use of low-level access to drives to securely wipe them: dd for SDDs and hdparm for everything else. hdparm is currently the only CLI tool that is capable of sending the SECURE ERASE and SANITISE firmware commands to SSDs, which mean it's currently the only (non-proprietary) way to securely erase any SSD, and it was retired on Cygwin due to inactive maintainers, which means WSL2 is currently the only way to do this while still staying on Windows. Please bring this to Version 2004.

Some BIOS on Supermicro motherboards don't like MBR or GPT bootable USB so I have to dd the ISO directly to them. That, and wiping drives. DBAN bootable USB is often the tool of choice, but if data is not sensitive dd is usually good enough to render a disk unbootable.

As explained in  #4197 usb access to programmers would make embedded development where direct flashing of targets is involved a lot easier.

You can connect pretty much anything via USB in WSL2. You only need an USB/IP application (I use Virtualhere) and reconfigure the WSL2 kernel if the device needs it.
For example this user connected his iPhone to WSL2 with Virtualhere but needed to enable ipheth module in the WSL2 kernel first.
Connecting iPhone with WSL 2 (Ubuntu)

That seems like a lot of overhead and slow (I need to do debugging). Also don't want to recompile my kernel. WSL2 should be able to do that out of the box.

Of course, I was talking about as a workaround. USB access and more should come after GUI and GPU support.

Another use: f3probe seems to only work with block device access, and I'd like to use it to test my flash drive on Windows

Use case: Access LUKS encrypted data such as an lvm volume with partition(s) inside.
I can build and use a custom kernel with the necessary dm-crypt module support, e.g. https://superuser.com/a/1499035/1182194
If my LUKS partition/volume is on a USB drive, I can serve host's USB drive to WSL2 via iSCSI (see UPDATE 2 in the same answer referenced above) and go from there.
However if my lvm volume is on the same physical drive where Windows is, I don't know of a workaround today. (Is there one?)

Another use case: while CD mounting is already possible, I'm unable to use Linux tools to rip audio CDs since programs like abcde/cdparanoia/cd-discid/cd-info reads data directly from the (currently inexisting) raw block device.

Another use case: Bootstrapping a dual-boot to a newly installed device (e.g. “sudo su; mount -t btrfs /dev/sdc2 target; mkdir -p target/boot/efi; mount -t vfat /dev/sdc1 target/boot/efi; debootstrap --arch=amd64 focal target; chroot target apt-get install ubuntu-desktop”) is currently not possible with the NT kernel’s limited number of supported file systems

This is blocking me to go all-in with Windows and WSL 2 for all my desktop needs. There are 3 SSDs in my machine, and I cannot access 2 of them because they have Linux filesystems (which I cannot change).

My use case is wanting to connect my phone so I can adb install the apk I just generated for work and debug it

I agree with @kissgyorgy and @sevaseva : lack of block device access is currently a show stopper for WSL 2. I have a machine that already has a luks volume with my user data-- I should be able to dual boot windows with WSL and linux, and when in windows just mount using the partition designator of the block device. This would allow using the same user home directory in both boot environments...which is pretty much required for switching over to WSL as a daily driver.
There are other technical reasons for this too that have been mentioned above. Another that hasn't been brought up is designating swap space inside of windows for the WSL instance. Having a nvme drive with space allocated for swap would reduce memory pressure, and there's no reason to provision another virtual partition when one already exists specifically for that purpose on the system.

Use case: flash and debug my microcontroller dev board: STM32 F3 Discovery Board

Use case: flash and debug my microcontroller dev board: STM32 F3 Discovery Board

Different board, but for the rest want to be able to write images to flash devices.
For now I'm still forced to use a Linux VM in VMWare to do it.

Here is our use case:
As a software/hardware development company we've had some success using WSL2 as a development environment for a lot of our software. We would like to be able to access simple usb hardware devices from within the WSL2 environment so we can continue this trend and not have to move over to a linux box for these activities.

... also, tried to follow the requests higher up in this thread but:

Since user voice links have expired. I would like to add additional information in this feature request as recommeneded by @sunilmut @russalex in 2016. I have been unable to find a new WSL User Voice  option. Sorry if this not the location to store this info but thought this might help other poeple doing a Bing/Google search.
I would use this feature with the Yubikey for SSH PIV.
At the moment it hard to this kind of setup really working. It not important feature as there are workarounds just would be nice to have similair setup for Yubikey SSH on WSL.
Provided some resource links to explain this in more detail and also show other possible solutions.
Resource Links:
How to setup yubikey pkcs11
Securing SSH with OpenPGP or PIV
SSH authentication using a YubiKey on Windows
Using YubiKey from Windows Subsystem for Linux (WSL)

I noticed that the WSL2 kernel configuration seems to support ext and btrfs. Just run zcat /proc/config.gz and filter for EXT and BTRFS

# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT2=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_ENCRYPTION is not set
# CONFIG_EXT4_DEBUG is not set

CONFIG_BTRFS_FS=y
CONFIG_BTRFS_FS_POSIX_ACL=y
# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
# CONFIG_BTRFS_DEBUG is not set
# CONFIG_BTRFS_ASSERT is not set
# CONFIG_BTRFS_FS_REF_VERIFY is not set

@Dimensional WSL2 root filesystem is ext4 but what does it prove?
mount | column -t
/dev/sdb     on  /                          type  ext4         (rw,relatime,discard,errors=remount-ro,data=ordered)

@Dimensional WSL2 root filesystem is ext4 but what does it prove?
mount | column -t
/dev/sdb     on  /                          type  ext4         (rw,relatime,discard,errors=remount-ro,data=ordered)

That at least it has ext4 and btrfs support?

Use case: format a USB flash drive with format-udf.sh.

Is there any way to mount an usb drive under WSL2? In WSL1 it works without any problem, but not in WSL2. Any ETA when that's gonna be made possible?

I would like to have native fs performance on my btrfs-formatted second nvme and share the filesystem with Windows via Samba.

Is there any way to mount an usb drive under WSL2? In WSL1 it works without any problem, but not in WSL2. Any ETA when that's gonna be made possible?

This works in both WSL1 and 2 via Ubuntu 20.04: sudo mount -t drvfs H: /mnt/h

Is there any way to mount an usb drive under WSL2? In WSL1 it works without any problem, but not in WSL2. Any ETA when that's gonna be made possible?

This works in both WSL1 and 2 via Ubuntu 20.04: sudo mount -t drvfs H: /mnt/h

Can't confirm for WSL2, in WSL2, when I typed in “mount” on the console, it said “9p” as the file system type it mounted the windows drives under (C, D and also the USB drive that was plugged in at boot time). When I tried to boot an USB stick with the “9p” file system it said that file system is unknown or something, same for drvfs. It is the main reason I downgraded to WSL1 on because I need WSl to be able to mount USB drives.

Is there any way to mount an usb drive under WSL2? In WSL1 it works without any problem, but not in WSL2. Any ETA when that's gonna be made possible?

This works in both WSL1 and 2 via Ubuntu 20.04: sudo mount -t drvfs H: /mnt/h

Can't confirm for WSL2, in WSL2, when I typed in “mount” on the console, it said “9p” as the file system type it mounted the windows drives under (C, D and also the USB drive that was plugged in at boot time). When I tried to mount an USB stick with the “9p” file system it said that file system is unknown or something, same for drvfs. It is the main reason I downgraded to WSL1 on because I need WSl to be able to mount USB drives.

Both DrvFs and 9p filesystem require that the host Windows operating system mount and understand the filesystem itself. There does not appear to be a real way to access raw block devices from WSL, 1 or 2.

Is there any way to mount an usb drive under WSL2? In WSL1 it works without any problem, but not in WSL2. Any ETA when that's gonna be made possible?

This works in both WSL1 and 2 via Ubuntu 20.04: sudo mount -t drvfs H: /mnt/h

Can't confirm for WSL2, in WSL2, when I typed in “mount” on the console, it said “9p” as the file system type it mounted the windows drives under (C, D and also the USB drive that was plugged in at boot time). When I tried to mount an USB stick with the “9p” file system it said that file system is unknown or something, same for drvfs. It is the main reason I downgraded to WSL1 on because I need WSl to be able to mount USB drives.

Both DrvFs and 9p filesystem require that the host Windows operating system mount and understand the filesystem itself. There does not appear to be a real way to access raw block devices from WSL, 1 or 2.

The USB stick was mounted just fine under Windows, it's just that WSL2 didn't want to play ball with it.

Well, my objective for it would have been the ability to mount a kernel supported filesystem directly from the WSL2 environment, rather than requiring a Windows filesystem driver for it first. But it turns out, I didn't need to do that anyway, as I already had the data elsewhere.

Mounting ext4 file system feature is available from Windows insider build 20190 but it is not documented as of now. Wait for 1 or 2 builds.

If support for linux FS is added natively into Windows, it certainly
would allow more use of accessing an external partition instead of
having to do some tricks with something like VirtualBox or VMware
connecting directly to the block device. Heck, if it supports direct
block device access, then in theory WSL2 could even mount LUKS encrypted
partitions. And then it would be easier for MS to see everything we
have. That last sentence is purely sarcasm, in case the joke got missed.
Don't want any misinterpretation there.
On 8/19/2020 1:35 AM, Biswapriyo Nath wrote:

Mounting ext4 file system feature is available from Windows insider
build 20190 but it is not documented as of now. Wait for 1 or 2 builds.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/WSL/issues/689#issuecomment-675881558,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABGLLYMPBEWIWTAJ5NNQHI3SBNXCZANCNFSM4CKRECFA.

If support for linux FS is added natively into Windows, it certainly would allow more use of accessing an external partition instead of having to do some tricks with something like VirtualBox or VMware connecting directly to the block device. Heck, if it supports direct block device access, then in theory WSL2 could even mount LUKS encrypted partitions. And then it would be easier for MS to see everything we have. That last sentence is purely sarcasm, in case the joke got missed. Don't want any misinterpretation there.

No idea why they didn't do it yet, the specifications for ext2/3/4 are freely available on the net.

Guys, I thinks this is what you want:
https://github.com/MicrosoftDocs/WSL/pull/824/files

Guys, I thinks this is what you want:
https://github.com/MicrosoftDocs/WSL/pull/824/files

That looks fine for fixed disks, but not for USB (removable) disks. What I've been waiting for is a way to partition and format those from within WSL[2].
We use dozens of Raspberry Pi's, which will grow to hundreds of Raspberry Pi CM modules by early/mid 2021. I still have to prepare the flash images in a Linux VM in VMWare. Having to use VMWare also keeps me locked out from WSL2.

can't google chrome give websites raw USB device access if they ask for it? we're kinda at the point where we can run a whole fully functional VM inside a web browser with all the APIs we have today. so you'd just need to build the flash images and load them into a website?

Guys, I thinks this is what you want:
https://github.com/MicrosoftDocs/WSL/pull/824/files

That looks fine for fixed disks, but not for USB (removable) disks. What I've been waiting for is a way to partition and format those from within WSL[2].
We use dozens of Raspberry Pi's, which will grow to hundreds of Raspberry Pi CM modules by early/mid 2021. I still have to prepare the flash images in a Linux VM in VMWare. Having to use VMWare also keeps me locked out from WSL2.

According to the docs, USB HDD is supported and USB stick support is coming soon.

Forgive my ignorance here, but is the wsl --mount feature only available to Windows Insider builds, or can I get it on Version 2004 (build 19041) by just upgrading WSL separately?

Is there any technical reason a disk couldn't be mounted to WSL1? I see the docs are only for WSL2.

Discs can be mounted in disc drives, WSL1 is not a disc drive.

@espg : We've made this change and it's working its way through the Windows release pipeline.  WSL1 does not have any infrastructure for that, so that would be a completely different problem to solve.
We have no plan to implement wsl --mount for WSL1 at this time.

Not concerned with wsl --mount being implemented, just wsl --mount --bare. Obviously that's the part you're suggesting takes the most effort though. Would it be that different than how any sort of usb image writer app takes exclusive mode of a drive? Or is it trickier because the virtual Linux file system has to map it?

There is no block device to map, which is what this issue is about.

wsl --mount is now available in the latest insider preview build (see blog post).
@lmcdougald : Making the device readable and writable in WSL1 wouldn't really be useful unless WSL1 could also parse partitions (GPT / MBR) and read filesystems. That's the reason why wsl --mount is only available in WSL2.

Enabling the former scenario will allow the latter. The file system aspect for example could be covered by: https://github.com/billziss-gh/winfuse/releases
From<EMAIL_ADDRESS>Sent: Thursday, September 10, 2020 12:28 PM
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [microsoft/WSL] Block devices, (i.e. /dev/sda) ? (#689)
wsl --mount is now available in the latest insider preview build (see blog posthttps://devblogs.microsoft.com/commandline/access-linux-filesystems-in-windows-and-wsl-2/).
@lmcdougaldhttps://github.com/lmcdougald : Making the device readable and writable in WSL1 wouldn't really be useful unless WSL1 could also parse partitions (GPT / MBR) and read filesystems. That's the reason why wsl --mount is only available in WSL2.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/WSL/issues/689#issuecomment-690661803, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEFCKDE4NSBINSK42VS6FEDSFESEFANCNFSM4CKRECFA.

My use cases

To use a single windows 10 machine with WSL enabled to complete linux imaging for a small company assembly line process with linux-illiterate assembly folks. As of now I have to provide a secondary computer with linux running in order to access the raw drives for dd imaging.

If I was doing a forensic recovery with dd, testdisk, photorec it would be useful for me to stay in one OS instead of splitting myself between two so access to a raw drive again is very useful.

So at the end of the day WSL adds nothing for me that I can't get quickly from a secondary OS or from Docker for Desktop containers at this point. The differentiator for me is to be able to eliminate at least one or both of the aforementioned options.

@cephas0 : You can do both of these things with wsl --mount.
wsl --mount --bare allows a physical disk to be attached to WSL2 and accessible as a block device. Once attached, tools like dd, parted & others can be used.

@OneBlue except this don t allows partitions thus preventing it to work with one drive slot computers, whereas a Linux native system maintains full access to acl of ntfs partitions.

@ytrezq that's right, but @cephas0's scenarios are about imaging disks, not accessing the boot disk.

There is a workaround with StarWind VSAN. It supports whole device passthrough or partition/virtual disk block access over iSCSI. Configuration guide  here - https://www.starwindsoftware.com/blog/enable-block-storage-devices-in-wsl2

Every version of WSL I try to like but keep going back to cygwin because stuff like this...
In this case I would like to find the offset of a pattern on an SD card so I can dump it into the memory of an embedded device...
I am root, let me do what I want!!

It would appear that the WSL2 not having passthrough for Serial Ports is also bundled under this issue.
Please advise, as it would be nice to run WSL2 for MicroPython on my ESP32's.  As it is, I will have to restart my machine in full Linux mode.
Is there any plan for passthrough for COM ports in WSL2?  If not, why not?  It worked fine on WSL1.

It would appear that the WSL2 not having passthrough for Serial Ports is also bundled under this issue.
Please advise, as it would be nice to run WSL2 for MicroPython on my ESP32's. As it is, I will have to restart my machine in full Linux mode.
Is there any plan for passthrough for COM ports in WSL2? If not, why not? It worked fine on WSL1.

That sounds like a separate issue, if so you should probably create one as such

and the lsblk command errors out because of a missing file

What distro did you install? Ik have that missing file error in Debian 11, but it works in Ubuntu 18.

The default Ubuntu distro.
~~ Robert.
Science tells us we are merely beasts, but we don't feel like that. We feel
like angels trapped inside the bodies of beasts, forever craving
transcendence.
(VS Ramachandran, cognitive neuroscientist)
On Fri, Feb 4, 2022 at 4:04 AM lucvdv @.***> wrote:

and the lsblk command errors out because of a missing file
What distro did you install? Ik have that missing file error in Debian 11,
but it works in Ubuntu 18.
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/WSL/issues/689#issuecomment-1029780513, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAE5V2JF4CXZDKEMSFLQD3UZOJBJANCNFSM4CKRECFA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

For me, I just want to use dd without having to go to Linux. I can't use a livecd easily because I don't have a flash drive, and my usb drive isn't recognizing on windows tools (though I can see G/: in the explorer, though it freezes). So this was hopefully my last attempt to get it to work.
For others, it's also useful for formatting a drive using linux,

For me, I just want to use dd without having to go to Linux. I can't use a livecd easily because I don't have a flash drive, and my usb drive isn't recognizing on windows tools (though I can see G/: in the explorer, though it freezes). So this was hopefully my last attempt to get it to work. (Not sure it'll necessarily work, but there are similar use-cases to this)
For others, it's also useful for formatting a drive using linux, (without having to go through the trouble of a dual boot / livecd)

Maybe Cygwin will be able to help you with that. I vaguely remember dd working there, but it has been a long time.

For me, I just want to use dd without having to go to Linux. I can't use a livecd easily because I don't have a flash drive, and my usb drive isn't recognizing on windows tools (though I can see G/: in the explorer, though it freezes). So this was hopefully my last attempt to get it to work. (Not sure it'll necessarily work, but there are similar use-cases to this)
For others, it's also useful for formatting a drive using linux, (without having to go through the trouble of a dual boot / livecd)

Maybe Cygwin will be able to help you with that. I vaguely remember dd working there, but it has been a long time.

Can confirm that it works fine, I've been using it for half a decade. I tried my best to move to WSL2 but it has too many failings compared to Cygwin, so I'll probably stay on Cygwin for at least another 5 years.

Just wanted to say - I've been tethered to a virtual machine at work for years, needing a working Linux dev environment for an embedded product. Decided to migrate to WSL2 earlier this year when I'd seen how far it had come. Somehow, I never ran into this issue before now, and man it is very disappointing to have to fire that VM up again.

Just wanted to say - I've been tethered to a virtual machine at work for years, needing a working Linux dev environment for an embedded product. Decided to migrate to WSL2 earlier this year when I'd seen how far it had come. Somehow, I never ran into this issue before now, and man it is very disappointing to have to fire that VM up again.

same here. wsl got everything done in the petalinux toolchain, only to be punted back to a vm to format the sd card :(

@cheesyh I actually managed to get around this by using usbipd.

While usbipd helps immensely for mounted file systems shared with the host (Win11 OS) it does not help with physical device (as in /dev/mmblk0p1) access as much.
Tools that embedded devs use are
dd
cfdisk
and scripts that build images for the target system.
Windows commands to get the physical device name (like \.\PHYSICALDRIVE2 ) - don't really translate well to handles WSL 2 clients use
EX:
c:> GET-CimInstance -query "SELECT * from Win32_DiskDrive"
DeviceID           Caption                          Partitions Size          Model

\.\PHYSICALDRIVE2 TS-RDF8 SD  Transcend USB Device 1<PHONE_NUMBER>0   TS-RDF8 SD  Transcend USB Device
\.\PHYSICALDRIVE1 TS-RDF8 CF  Transcend USB Device 0                        TS-RDF8 CF  Transcend USB Device
\.\PHYSICALDRIVE0 CT2000P5PSSD8                    5<PHONE_NUMBER>280 CT2000P5PSSD8
c:> Get-WmiObject Win32_diskdrive | Select Caption,DeviceID,InterfaceType,Size | Where-Object {$_.InterfaceType -eq "USB"}
Caption                           DeviceID           InterfaceType        Size

TS-RDF8 SD  Transcend USB Device  \.\PHYSICALDRIVE2 USB<PHONE_NUMBER>0
Linux File-Stor Gadget USB Device \.\PHYSICALDRIVE4 USB             115153920
TS-RDF8 CF  Transcend USB Device  \.\PHYSICALDRIVE1 USB
freescale USB Device              \.\PHYSICALDRIVE3 USB<PHONE_NUMBER>
Resorting to creating a local image file and then mounting (each filesystem needed in the boot disk)  it, formatting, and placing images in the image file (that is partitions for different file system) rather than to the direct physical drive is an option.
Then copying it to windows and using an image writer tool like rufus or others seems the only workaround.
Man, computers make things easier, Microsoft could wrap (https://en.wikipedia.org/wiki/Amoeba consume?) Linux systems when they get  this right. Yes, I realize they have Linux source code - but this would give them instant access to all of what free software foundation folks have been working on forever.

@RGulde I'm not sure I follow. The way I use usbipd, the filesystem is not shared with Windows, and it does show up as /dev/<blah> inside of WSL. I can use dd just fine.
C:\Users> usbipd share -b <bus id>
C:\Users> usbipd attach -b <bus id> -w

Sometimes, for reasons I have not yet been able to discover, you need
C:\Users> usbipd bind --force -b <bus id>

Which will necessitate a reboot, and prevent Windows from accessing that device at all. At the end, though, it just shows up as a block device in WSL:
user in ~ λ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 389.8M  1 disk
sdb      8:16   0     2G  0 disk [SWAP]
sdc      8:32   0     1T  0 disk /snap
                                 /mnt/wslg/distro
                                 /
sdd      8:48   1     0B  0 disk
sde      8:64   1  14.8G  0 disk <-- Here!
└─sde1   8:65   1  14.8G  0 part

In order to get this working, I did need to custom compile the WSL kernel. If I recall correctly, Microsoft has made the irritating decision to support this out of the box on Win11, but not backport the kernel config to Win10 WSL. The usbipd wiki has more information.

@RGulde I'm not sure I follow. The way I use usbipd, the filesystem is not shared with Windows, and it does show up as /dev/<blah> inside of WSL. I can use dd just fine.
In order to get this working, I did need to custom compile the WSL kernel. If I recall correctly, Microsoft has made the irritating decision to support this out of the box on Win11, but not backport the kernel config to Win10 WSL. The usbipd wiki has more information.

I'm on Windows 11 and it does not show in lsblk, only lsusb. I would very much like to know what you did to your WSL kernel to get this block device to show up. I did force-bind and attach the USB MSD.
part of my dmesg:
[   92.370629] vhci_hcd vhci_hcd.0: pdev(0) rhport(0) sockfd(3)
[   92.370634] vhci_hcd vhci_hcd.0: devid(393220) speed(5) speed_str(super-speed)
[   92.370691] vhci_hcd vhci_hcd.0: Device attached
[   92.718151] usb 2-1: SetAddress Request (2) to port 0
[   92.718161] usb 2-1: new SuperSpeed USB device number 2 using vhci_hcd
[   92.769955] usb 2-1: New USB device found, idVendor=8564, idProduct=4000, bcdDevice= 0.2a
[   92.769961] usb 2-1: New USB device strings: Mfr=3, Product=4, SerialNumber=5
[   92.769964] usb 2-1: Product: Transcend
[   92.769965] usb 2-1: Manufacturer: TS-RDF8
[   92.769967] usb 2-1: SerialNumber: 000000079

@crackwitz If you're not seeing any block devices in /dev, make sure your kernel was compiled with USB mass storage drivers enabled. Check the config in /proc/config.gz

I follow this for disks such as hard drives, SSDs, etc, but I'm trying to mount a DVD drive as a block device. I tried using the physical path of \.\CdRom0\ and either get an "Access is denied" or a "Incorrect Function", I even removed the drive letter from Windows just in case. Is there something different or are optical drives not available via "wsl --mount" ?


Faded text is difficult to read

Checklist

[X] I'm reporting a problem with Chatterino
[X] I've verified that I'm running the most recent nightly build or stable release
[X] I've looked for my problem on the wiki
[X] I've searched the issues and pull requests for similar looking reports

Describe your issue
Currently, when you open Chatterino, all the history before you connected will be faded (has half opacity). I understand it was meant to signify that it is history, however, I feel like this is an inaccessible solution. Faded out text is very difficult to parse if you rely on text contrast due to poor vision or if you have reading problems and need to see characters clearly. One could say ‘oh just adjust your monitor settings’, but this does not improve the situation at all and actually makes the image for anything else worse.
This issue becomes even more severe when the person who was in history gets muted, Chatterino then double fades out their messages, so they become almost invisible.
Screenshots

OS and Chatterino Version
Chatterino Nightly 2.3.4 (commit e2eb73d81, built on ); Windows 10 20H2 (19042)

Thinking of solutions that can solve this:

Have an option to use a tiny strikethrough effect for muted accounts instead of fading.

Do not fade away the history at all, instead create a special block line that would separate history from the actual chat.
E.g. [                      ▲ Chat history before 27/01/22 07:44 ▲                      ]

This option is available on Dankerino's fork

I see that you're on Chatterino Nightly so keep in mind if you opt to use Danekrino for this option that it won't always be up to date. For example the last pull from Chatterino was quite a while ago 65298302da5c8ab2f348bc2c2349a2c004188007

I would like to work on this issue if possible.

Thinking of solutions that can solve this:

Have an option to use a tiny strikethrough effect for muted accounts instead of fading.
Do not fade away the history at all, instead create a special block line that would separate history from the actual chat.
E.g. [~                     ~ ▲ Chat history before 27/01/22 07:44 ▲ ~                     ~]

I was thinking of taking the path of the second solution described here by @Brawaru and this option could be deactivated as it can be done in Dankerino.

I would like to work on this issue if possible.

Go right ahead. 🙂


obtuse and confusing error with local feature under WSL #834

Issue 834
Confusing error message modified to more understandable message.

Hi @samruddhikhandale ,
I have addressed review comments and also modified the error message as suggested by @chrmarti .
Kindly review and approve the same.


Local fetch from index.svelte's load function fails on server

Describe the bug
In my project's root-level index.svelte, fetch to a local resource (endpoint) within the load function returns 404 even though the resource exists. This only occurs when the server is trying to render the page; if I navigate to my project's homepage from another page, the client-side fetch works fine and the page loads as expected.
Excerpt:
<script lang="ts" context="module">
  import type { Load } from "@sveltejs/kit";

  export const load: Load = async function load({ fetch }) {
    const res = await fetch("/post-list.json");
    console.log(res.status);
    ...
  }
</script>

--> logs 404, even though /post-list.json (from post-list.json.ts endpoint) renders just fine when you navigate to it directly (or load it from other pages).
This issue occurs with<EMAIL_ADDRESS>but not when I downgraded to<EMAIL_ADDRESS>I think the root cause was introduced in #583 and specifically I think it's coming from https://github.com/sveltejs/kit/blob/8a88fad3118a7ddec3ef6728cc1b5a74fc18e0f7/packages/kit/src/runtime/server/index.js#L30-L38
options.initiator in this case seems to correspond to the index.svelte route, which is the first route that gets checked, so we get 404 immediately (before other routes are checked); this problem seems like it could occur with fetches from other routes, but I guess I've noticed it here because index.svelte is the first route listed in the manifest.
To Reproduce
Use<EMAIL_ADDRESS>(issue does not occur with @sveltejs/kit@1.0.0-next.54).

Create a new project (with TypeScript) using npm init svelte@next
Replace the contents of src/routes with these 3 minimal files: https://gist.github.com/alecglassford/f1498cc6af3d2eb90ed5eb0940ff3024
Run npm run dev and navigate to localhost:3000. You'll see 404.
Navigate to localhost:3000/other and then click the link to go to the homepage. You'll now see 200.

Expected behavior
Fetch does not 404 when endpoint exists.
Information about your SvelteKit Installation:

The output of npx envinfo --system --npmPackages svelte,@sveltejs/kit --binaries --browsers
  System:
    OS: macOS 11.2.3
    [omitted]
  Binaries:
    Node: 14.16.0 - ~/.nvm/versions/node/v14.16.0/bin/node
    npm: 7.6.3 - ~/.nvm/versions/node/v14.16.0/bin/npm
  Browsers:
    [omitted]
  npmPackages:
    @sveltejs/kit: ^1.0.0-next.55 => 1.0.0-next.55
    svelte: ^3.29.0 => 3.35.0

Your browser: Firefox

Your adapter (e.g. Node, static, Vercel, Begin, etc...): static (but the reproduction example uses node)

Severity
Can't build the site, and things aren't loading properly on the dev server Somewhat severe.

Thanks — your diagnosis was spot on. Fixed in #615


[BUG] Total Applied Jobs Counter Shows "undefined"

Total Applied Jobs Counter Shows "undefined"
Description
The Total Applied Jobs counter in the Dashboard is not functioning correctly and displays "undefined" instead of the actual number of applications.
Current Behavior

Dashboard shows "Total Applied Jobs: undefined"
Counter not initializing properly
No numerical value displayed
Not updating with new applications
No loading state or error message shown

Expected Behavior

Display accurate count of submitted job applications
Update automatically when new applications are submitted
Show "0" for no applications
Display loading state while counting
Show error message if count fails

Steps to Reproduce

Login to account
Navigate to Dashboard
View Total Applied Jobs section
Observe "undefined" displayed instead of number

Impact

Users cannot track their application count
No way to monitor application progress
Creates uncertainty about application status
Poor user experience

Priority
High - Essential dashboard functionality broken
Suggested Solution

Initialize counter properly with actual database count
Add proper error handling
Implement loading state
Add automatic updates when new applications submitted

Additional Notes

Check API endpoint connectivity
Verify data formatting from backend
Add error logging for debugging

can you assign me @Devmangrani

Hi @Furiousfox-01
Go for it! Looking forward to seeing your contribution!

i couldn't find that bug, seems to be working fine.

If the bug isn’t appearing on your end, you’re welcome to look into other open issues in the project. Let me know if you’d like any suggestions for specific issues to tackle. Thanks for checking!


Update vendor packages

Updates vendor package versions.

Updates the grpc gateway which allows us to make calls like curl -X GET http://localhost:8000/v1/tasks?view=BASIC


argmax or random.choice in generate?

In generate.py, it use random.choice with scaled_prediction to predict next sample, i'm confused about why it doesn't use argmax to choose the highest prediction every time?
i have tried it but it doesn't work, always silence the whole time. anyone have any idea? thanks

Wavenet predicts a probability distribution for the next sample of the Waveform.  In general, always picking the mode of a probability distribution will result in a sample which is very unrealistic.
To see why you are getting silence, suppose that you train on a dataset where the first 500ms is silence and then there is speech in the second 500ms.  If Wavenet sees that there was silence in its input, it will predict that the next sample is very likely to be silence, but there is some small probability that it is not silence because the speech has to start somewhere, after all.  If you randomly sample from this probability distribution, you will find that you get silence for a little while, and then at some point you get not silence (which will hopefully sound like speech).  But if you are always picking the most likely value, you will always pick silence and you will never get speech.

Wavenet预测下一个波形样本的概率分布。通常，始终选择概率分布的模式将导致样本不现实。
要了解为什么您会变得沉默，请假设您在一个数据集上进行训练，其中前500毫秒是沉默，而后500毫秒是语音。如果Wavenet看到其输入中存在静音，它将预测下一个样本很可能是静音，但是由于语音必须从某处开始，因此很有可能不是静音。如果从该概率分布中随机抽样，您会发现您沉默了一会儿，然后在某些时候您没有沉默（希望听起来像语音）。但是，如果您始终选择最可能的价值，那么您将始终选择沉默，并且您永远也不会说话。

How not to choose the most likely value？

Wavenet predicts a probability distribution for the next sample of the Waveform. In general, always picking the mode of a probability distribution will result in a sample which is very unrealistic.
To see why you are getting silence, suppose that you train on a dataset where the first 500ms is silence and then there is speech in the second 500ms. If Wavenet sees that there was silence in its input, it will predict that the next sample is very likely to be silence, but there is some small probability that it is not silence because the speech has to start somewhere, after all. If you randomly sample from this probability distribution, you will find that you get silence for a little while, and then at some point you get not silence (which will hopefully sound like speech). But if you are always picking the most likely value, you will always pick silence and you will never get speech.

How not to choose the most likely value

Tensorflow translation example brought me here (encoder decoder ) :https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/nmt_with_attention.ipynb
I'm not very sure how @joe-antognini 's answer applies to translation but I like the idea of "always picking the mode of a probability distribution will result in a sample which is very unrealistic" especially when we're talking about human related themes such as language etc ..


Crashes when processing a shell script

The #! line causes it to fail. Removing it, processing, then adding it back in works.

@allain Do you mean running xto6 on a file which includes hashbang? I'll take a look.


Run a single controller for watching multicluster secrets

This is briefly described in this doc.
Currently we run two controllers watching for remote-secrets for cross-cluster API server access. This on it's own is odd, but it also means that we create 2x kube clients for the same remote clusters.

dupe of https://github.com/istio/istio/issues/33370


Could browse group be a facet?

We're going to bring collections over as JSON from plum instead of this.


Opening text file in datalab results in 404 error

I'm unable to open/edit a text file (or any file that does not have extension *.ipynb) in Datalab. When I click on the file, I see a blank page instead of an interface to edit a text file.
I'm trying to open a text file called '1.txt' in the folder 'InitialSetup' . Looking at the log, I see there is a 404 error when trying to access the following url : '/edit/InitialSetup/1.txt'
Output (see last entry with 404 error)
{"name":"app","hostname":"<XXXXXX>","pid":48,"type":"request","level":30,"url":"/api/sessions?_=<XXXXXX>","method":"GET","status":200,"msg":"","time":"2016-03-05T18:20:04.454Z","v":0}
{"name":"app","hostname":"<XXXXXX>","pid":48,"type":"jupyter","level":30,"msg":"[9000]: 200 GET /api/sessions?_=<XXXXXX> (<IP_ADDRESS>) 1.72ms\n","time":"2016-03-05T18:20:04.456Z","v":0}

{"name":"app","hostname":"<XXXXXX>","pid":48,"type":"request","level":30,"url":"/api/contents/InitialSetup?type=directory&_=1457201868796","method":"GET","status":200,"msg":"","time":"2016-03-05T18:20:04.486Z","v":0}
{"name":"app","hostname":"<XXXXXX>","pid":48,"type":"jupyter","level":30,"msg":"[9000]: 200 GET /api/contents/InitialSetup?type=directory&_=<XXXXXX> (<IP_ADDRESS>) 3.23ms\n","time":"2016-03-05T18:20:04.487Z","v":0}

{"name":"app","hostname":"<XXXXXX>","pid":48,"type":"request","level":30,"url":"/edit/InitialSetup/1.txt","method":"GET","status":404,"msg":"","time":"2016-03-05T18:20:06.123Z","v":0}

Fixed in 6bbe1fde


Fixed: writing in RTL causes text to appear twice

In addition, 2 small fixes: added the option to show mentions list on the bottom of the editor and two small css fixes
(apologize for mixing these)

The main change fixes this one: https://github.com/mrazadar/react-native-mentions-editor/issues/22


docs(GlobalHeader): Polish GlobalHeader / AppHeader docs

Overview

Touch up Storybook documentation for all GlobalHeader and AppHeader components.

No code changes.
Please look over the Storybook docs for all header components in Labs > Experimental > Molecules. Would love to hear any suggestions for improvement because I know I have blindspots, having worked on these components for awhile now!
PR Checklist

[ ] Related to designs:
[x] Related to JIRA ticket: EGG-660
[ ] I have run this code to verify it works
[ ] This PR includes unit tests for the code change

📬Published Alpha Packages<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6033c823963d4c40af48d69e--gamut-preview.netlify.appDeploy Logs

Looks good to me!! couple of v small things.
I would suggest requesting another Egg to review as well since I have a lot f knowledge of these components + practice using them!

📬Published Alpha Packages<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6033d27e24700c3efd488d37--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6033e4e15ded7d00d529a98a--gamut-preview.netlify.appDeploy Logs

@dreamwasp yeah, I think that spacing is a storybook thing 😊 if you go to the Canvas tab (vs the Docs tab), you'll see a better representation of the components!

📬Published Alpha Packages<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6033f3e321875621760460c7--gamut-preview.netlify.appDeploy Logs


Fix Logo Size on Mobile & Narrow Screens

[ ] Figure out why my logo is so big on mobile screens

Hey Lynx! Looks like it’s this line in your style.css:
https://github.com/michaellunzer/michaellunzer-portfolio/blob/238296286fdd63f2e1b9dddc37fac3b2abc263c1/src/css/style.css#L192
It’s setting the the logo to be 100% of its container’s width, minus 30px. Since your logo is squareish it’ll  make the height naturally larger, and then big logo happens.
I dunno Gatsby so I’m not sure how the css plays into the framework, but you could adjust that value to be 50% or an explicit value like 150px.

Thanks for looking into it @charleskantz -- I appreciate the explanation and advice 😎
It's all fixed and deployed now!


Implement OneAnd[F, A] instances for collection-like F

Addresses #170
Some reasoning behind changes:

I'm targeting only collection-like Fs as it's not perfectly clear to me what should be a generic representation of OneAnd[F, A] where F is just some arbitrary type constructor. Anyway all the fun starts only when F is at least Foldable.
I decided to target collections from std lib, instead of implementing instances in terms of cats typeclasses for a couple reasons:

Getting instances for NonEmptyList/Vector/Stream will require end-users to import cats.std.list/vector/stream._ which is not convenient. Targeting IsTraversableOnce on the other hand will automagically generate instances for standard collections, which is a major use-case AFAIU.
Just replacing an instance for NonEmptyList[A] with an instance for OneAnd[F, A] can break code in a subtle way if automatic derivation is involved. Consider the following snippet:

import io.circe._
import io.circe.generic.auto._

val nel: NonEmptyList[Int] = NonEmptyList(1, 2)

// Before: encoder for NEL is implemented directly and doesn't require any additional imports
val e = Encoder[NonEmptyList[Int]]
e(nel) // "[1, 2]"

// After: encoder for NEL is implemented in terms of OneAnd and requires import from cats
val e = Encoder[NonEmptyList[Int]] // generic derivation kicks in because no foldable for list is provided
e(nel) // "{head: 1, tail: [2]}"

Might not be very generic. But on the bright-side we get very good performance comparable to that of regular collections. Locally I got ~3% difference:
[info] Benchmark                          Mode  Cnt     Score    Error  Units
[info] EncodingBenchmark.encodeFooNelsC  thrpt   20  4875.958 ± 74.492  ops/s
[info] EncodingBenchmark.encodeFoosC     thrpt   20  5039.914 ± 72.708  ops/s

Current coverage is 77.52%

Merging #178 into master will decrease coverage by -0.02% as of ff7a5b2

Powered by Codecov. Updated on successful CI builds.

Note - this a comment for my own learning, not a review comment.
Per your comment in #170, I thought that you needed MonadCombine? Searching for it in this page (in my browser), I don't see it. Why not?

@kevinmeredith yep, no MonadCombine here.
Basically you can implement Decoder for OneAnd in terms of MonadCombine, but the consequences are:

library end-users will have to import cats.std.list/vector/stream._ to to get MonadCombine instance in scope for standard library collections,
just replacing current decoder for NonEmptyList with the one based on MonadCombine will break existing code (again, because of the required extra import).
there will probably be some performance penalties, however I would not like to speculate on this wo/ any benchmarks.

CanBuildFrom[Nothing, A, List[A]] on the other hand is kind of analogous to MonadCombine in a sense that it can give us an empty collection of a given type and append new elements to the collection. But in addition it's also automagically available in the implicit scope for all standard collections, which is quite handy.
Of course, we can have CanBuildFrom based implementation for std-lib collections and also MonadCombine based implementation for other stuff, but that would require rearranging priority of implicits to avoid ambiguity in resolution, and I'm not sure whether this is really needed atm.
Does it make sense?

Thanks, @ArtemPyanykh—this looks perfect to me, with the one exception that Encoder.fromFoldable is part of the public API and potentially has uses other than the internal one that's going away, so I think we should keep it around.

@travisbrown right, was a bit too eager. I put it back.

@travisbrown is there anything else missing here?

@ArtemPyanykh Nope, this looks good, thanks—I've just been swamped with Cats release stuff. Merging now.

@travisbrown cool! Noticed interesting activity in Cats -- gotta check out recent changes.


Testing with Postman

I've successfully used your docker image to /ping and too /connect
I'm now trying to /v1/procedure within postman.
I constantly return the response "message": "Handling the PROCEDURE"
I believe I'm successfully creating the authorization header
Postman Pre-request Script
    var crypto = require('crypto-js');

    const key = "sqlrestTestKey";

    var body = pm.request.body.toString();

    function createHmac(message) {
        var hmac = crypto.HmacSHA256(message,key)
        return hmac
    }

    function setAuthHeader(message) {
        const realm = "testing-func"
        let hmac = ''
        if (message) {
            hmac = createHmac(message)
        }
        const nonce = crypto.lib.WordArray.random(16)
        const timestamp = new Date().getTime()
        pm.environment.set("timestamp", timestamp);
        const authHeader = `${realm}:${hmac}:${nonce}:${timestamp}`

        return authHeader
    }

    my_auth_header = setAuthHeader(body)

    pm.environment.set("Authorization",my_auth_header)

Any ideas on how to return a result please?

Hey @gwsampso Thanks for trying this project out.
It seems that the auto-build from Docker had broke some time ago, and I was not aware. The delete and procedure endpoints were implemented after the 0.2 image.
I've just manually created the burtonr/sqlrest:03image and pushed it to Docker Hub. Please update your image and try it again.
Do let me know if you're still having troubles with it

Thanks @burtonr I thought something didn't look right. Thanks greatly for updating the build.
docker pull burtonr/sqlrest:0.3
docker run -d -p 5050:5050 -e DATABASE_USERNAME=username-e DATABASE_PASSWORD=password -e DATABASE_SERVER=ipaddress -e SQLREST_ALLOWED_REALMS=test-func,qa-func -e SQLREST_API_KEY=sqlrestTestKey --name sqlrest burtonr/sqlrest:0.3

However not looking good on early testing
Test: /ping and /connect
Result: HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
Similarly v1/query and v1/procedure fail
Could you please look into when you get the chance, I'm going to have a crack today at debugging but not 100% sure where to start.
Reason: We were hoping to maybe utilize this with openfaas.
Thanks

Sorry it is working now with the updated C# example - thanks
I will have to work on my postman effort to add the authorization headers which look to be required for all endpoints now, which is totally fine
There are small changes required to the main readme.md to bring inline with the examples

Hey @gwsampso Glad it's working for you! Please let me know if you run in to any other issues, or think of something that could be added


The 'Clear completed' button does nothing in Firefox

I'm able to reproduce this issue with Firefox 47 (the latest). In chrome it seems to be working fine.

Is this still true? I just produced a new version here. If so, what OS are you on? When you open the developer console, are you seeing any errors?

I bet this is resolved by #46, which I brought in a different way. I'll close. Reopen if there's still a problem (assuming you do a hard refresh to make sure you are getting the new resources).

I confirm, that now it's working fine.


Local book readme fix

Some small fixes to the instructions for building jupyterbook locally:

It was a bit unclear where all the commands should be run. So added a cd course-content.
Therefore, symlinks in book/tutorials and book/projects should point to tutorials and projects in the current directory i.e. course-content (instead of a level-up).
tutorials did not have the precourse content. So added explicit instructions to copy those directories (namely W0D*) from ../precourse/tutorials to tutorials in the current directory.
../nmaci/scripts/generate_book.py needs an argument. So changed it to python ../nmaci/scripts/generate_book.py .
Fixed a typo.

Making this proposed change makes the process fail locally.
ln -s ../tutorials book/tutorials
ln -s tutorials book/tutorials

Please file an issue if you have any suggestions and not a PR via forks. See here for our contributors' guidelines.


Add Holt Winter's model and its variations

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Before submitting (must do checklist)

[x] Did you read the contribution guide?
[x] Did you update the docs? We use Numpy format for all the methods and classes.
[x] Did you write any new necessary tests?
[x] Did you update the CHANGELOG?

Type of Change

[ ] Examples / docs / tutorials / contributors update
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Improvement (non-breaking change which improves an existing feature)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Proposed Changes
Look #481
Related Issue
#481
Closing issues
Closes #481.

Codecov Report

Merging #502 (be48d47) into master (c777b30) will decrease coverage by 33.86%.
 Consider uploading reports for the commit db9a300 to get more accurate results

@@             Coverage Diff             @@
##           master     #502       +/-   ##
===========================================
- Coverage   87.80%   53.94%   -33.87%
===========================================
  Files         115      116        +1
  Lines        5454     5537       +83
===========================================
- Hits         4789     2987     -1802
- Misses        665     2550     +1885

Impacted Files
Coverage Δ

etna/models/holt_winters.py
80.00% <80.00%> (ø)

etna/models/__init__.py
100.00% <100.00%> (ø)

etna/commands/__init__.py
0.00% <0.00%> (-100.00%)
:arrow_down:

etna/commands/backtest_command.py
0.00% <0.00%> (-96.43%)
:arrow_down:

etna/commands/forecast_command.py
0.00% <0.00%> (-92.00%)
:arrow_down:

etna/commands/__main__.py
0.00% <0.00%> (-87.50%)
:arrow_down:

etna/commands/resolvers.py
0.00% <0.00%> (-80.00%)
:arrow_down:

etna/analysis/outliers/density_outliers.py
22.44% <0.00%> (-75.52%)
:arrow_down:

etna/transforms/timestamp/time_flags.py
26.02% <0.00%> (-73.98%)
:arrow_down:

etna/transforms/timestamp/fourier.py
27.08% <0.00%> (-72.92%)
:arrow_down:

... and 67 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c777b30...db9a300. Read the comment docs.


enhance XML output to include the type of attributes;

Reported by Ansley B Manke on 4 May 2007 22:22 UTC
Roland's virtual data sets had an incorrect missing_value, listed as a striong.
To best fix this he says the following.  This is all simple with the
attribute-access routines introduced in v6.0 Ferret.
One thing that we might need (should) do on the Ferret side is improve the XML
output to take the guess work out of the attribute types.
For example a variable in the Ferret XML might look like:

deg C
surface heat content
   <_FillValue>-1.000000E+34
-1.000000E+34

XAXLEVITR
YAXLEVITR

The , <long_name>, <_FillValue> and <missing_value> all derive from
netCDF attributes.  And as netCDF attributes the have a type and a dimension.
This information gets lost in the translation.  It would be nice to get it back,
but having the output look something like:

deg C

surface heat content

-1.000000E+34

-1.000000E+34

XAXLEVITR
YAXLEVITR

The reason to go with the  element is in the case of an array attribute
you can list each element.  E.g.
<attribute name="actual_range" type="float">
   <value index="0">-5.0</value>
   <value index="1">32.0</value>

Listing an index is probably overkill, but technically speaking the value
element could come in the content in any oder and you'd need the index to
guarantee you know which is the first value and which is the second and so on.
And, finally it be nice if for "non-virtual" variables we'd get all the
attributes in the netCDF file included in this XML output rather than just the
few the Ferret cares about or makes up.
For, now I will just guess by parsing the contents and if it parses as an
integer or long, I'll keep it as such. If not, I'll try float and double.  Then
if not, I'll keep it as a "string".  However, if the information is known to
Ferret it's be nice to have it in the XML.
Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/1508

Modified by ansley.b.manke on 5 Jan 2011 18:21 UTC

Comment by ansley.b.manke on 8 Feb 2011 01:35 UTC
Done. The XML lists the type of each attribute.


More convenient make test

What do these changes do?

Allow to pass TESTS argument to make test and make vtest targets.
For example:

Following pytest documentation at
http://doc.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests
Run only selected test case: make test TESTS=test_worker.py::test_run
Run only selected test module: make test TESTS=test_worker.py
Run all tests: make test.

Documented in "Contributing" section.

Are there changes in behavior for the user?

This makes development faster in some cases.
E.g. when you want to run only one test case or one test module quickly to check your changes in the code or in the test itself and don't want to run all tests after each little change it's more convenient and fast to run only the test case/module you're interested in.
Related issue number

None
Checklist

[x] I think the code is well written
[ ] Unit tests for the changes exist
[x] Documentation reflects the changes

Current coverage is 98.15% (diff: 100%)

Merging #1126 into master will not change coverage

@@             master      #1126   diff @@
==========================================
  Files            28         28
  Lines          6387       6387
  Methods           0          0
  Messages          0          0
  Branches       1073       1073
==========================================
  Hits           6269       6269
  Misses           63         63
  Partials         55         55

Powered by Codecov. Last update 19c0f31...e3153f4

Sorry, no.
aiohttp had functionaly like this ages ago but later we've removed it.
Most likely you need to provide more than just test selector, e.g. -s and -p no:timeout flags.
Even for selecting tests you may use -k param.
Moreover, for sake of shortening single test running time you most likely want to skip flake8/isort checks etc.
Conclusion: if you need something more complex than just make test does please don't afraid to run py.test tests directly.

@asvetlov Got you. Actually I already use raw py.test command to run a single test. Thanks for the explanation.


(FACT-1112) Move curl wrapper classes to Leatherman

Updates facter to use Leatherman's curl wrapper library.

CLA signed by all contributors.


Add OKEx chain which is an evm chain

i want to add okex chain but after making changes, home screen is blank and app keeps crashing

How are you adding it?
See this commit, and duplicate:
https://github.com/AlphaWallet/alpha-wallet-android/commit/44df65f7d864bd57222758dafa9c04b0fc2b5c07

I want to add it to the main code and later display it in the custom view.

Yes, just following the 44df65f commit (just change the Ids and find the OKX API calls) will allow you to do that. If you try to do it in a different way it'll probably crash - as you found out.

Hi @Mantisrei did you get anywhere with this one? In particular did you find:

Working and stable RPC node
Etherscan/Blockscout/Covalent equivalent
Transaction API (Etherscan/Blockscout compatible preferred)
SVG logo for OKX
Exchange listing prices for OKX chain tokens

@Mantisrei I looked into it, but couldn't find any Etherscan/Blockscout API. There is a custom API - oklink - but this provides a different style of query. Were you able to find anything better?
I found these instructions which give a very basic interface, but not a good experience as it won't discover an NFT/Tokens etc.
https://decentralizedcreator.com/add-okt-to-metamask/

Ok I have got the API routes from the OKX team.

Closing as we are handling this internally.
@Mantisrei it'll be in a release which will be out later this month.


Part of date dispensary when date is having double digits

Ex -  select a date 20/12/2015

I dont have the heart to edit the title of this :laughing:

Fixed with 5e1646d0e7b25d49903c6a24d706a788e6148813


build: bump tools version to 5.3

This is needed to gain access to Windows support.
[One line description of your change]
Motivation:
[Explain here the context, and why you're making that change. What is the problem you're trying to solve.]
Modifications:
[Describe the modifications you've done.]
Result:
[After your change, what will change.]

Can't this be achieved with a<EMAIL_ADDRESS>manifest?

I don't think we can really take this approach as it breaks previous versions of swift.

Okay, I'll fork the current package.swift then.  Does s-p-m pickup the file automatically?  Seems that the same argument as with CMake applies to keeping the old definition: you have two definitions.

Yes, SPM will pick up the appropriate version.

I think we agreed we don't need this?

I think for now we do not.  We can always revisit this if necessary.


debounce validates validators

Why does debounce validate validators until the debounce timeout is hit?
If i set debounce to 500 on a validator, the validators all are deemed valid until 500 milliseconds have passed.
That allows for the user to submit (via enter) a form after every keystroke for 500 milliseconds.

Although isValid might be true, isTruelyValid will only be true once the
debounce promise has been resolved and the validator returns true.
On Sunday, March 13, 2016, Bas Kamer<EMAIL_ADDRESS>wrote:

Why does debounce validate validators until the debounce timeout is hit?
If i set debounce to 500 on a validator, the validators all are deemed
valid until 500 milliseconds have passed.
That allows for the user to submit (via enter) a form after every
keystroke for 500 milliseconds.
—
Reply to this email directly or view it on GitHub
https://github.com/offirgolan/ember-cp-validations/issues/136.

--
Thanks,
Offir Golan

no isTruelyInvalid?
There is no easy way to use the the not operator in handlebars attributes, is there?

https://www.npmjs.com/package/ember-truth-helpers 😁
On Monday, March 14, 2016, Bas Kamer<EMAIL_ADDRESS>wrote:

no isTruelyInvalid?
There is no easy way to use the the not operator in handlebars attributes,
is there?
—
Reply to this email directly or view it on GitHub
https://github.com/offirgolan/ember-cp-validations/issues/136#issuecomment-196209673
.

--
Thanks,
Offir Golan

thx! that's helpful


Changes done after Pavel review

@ness-vaibhavtyagi Thank you for your interest in contributing to Flink.
Please note, however, that we don't intend for the exercise classes in this repository to provide solutions to the exercises. The repository already includes reference solutions for each exercise. For an example of this, see https://github.com/apache/flink-training/blob/master/long-ride-alerts/src/solution/java/org/apache/flink/training/solutions/longrides/LongRidesSolution.java.


Fixed channel override

Fixed channel overwrite not working, the channel parameter must be passed in the body, not the query string. The channel is also optional since a default channel is required when setting up a Slack webhook.
Also updated the documentation. The channel name must either be a normal channel prefixed with # or a direct message to a user @username.

Looks like this was somewhat fixed in commit 63505c50583e76ed6682426504678a251f82157e, but this PR is a bit more complete (updated documentation and checking for the channel parameter as it is indeed required by the Slack API). You want to merge anything from here or shall we just close it?

@phw Please fix your merge conflicts before submitting pull requests, thank you

The merge conflict was the result of the later commit 63505c50583e76ed6682426504678a251f82157e  (that's not my fault). The only thing this pull request did more than that commit was also updating the documentation and making the channel optional (as one can set a default channel for a web hook in the slack UI).
I happily rework this PR if you intent to merge it, though.

The channel parameter not being required was also already fixed in aaa616eb17d6e6331ed7a3e82ab5153f2169c5a0 . This PR has become obsolete beside the documentation update, but you can easily fix that yourself. Closing this

thank you @phw :heart:


Update composer.json to allow twig 3.0

With twig 3.0 out, can you update the composer.json dependency to allow it? This is for dev-master with cakephp 4.0.
^2.4.3|^3.0
https://packagist.org/packages/twig/twig#v3.0.0
Unless you're using the non-namespaced classes, I don't think there will be an issue.
https://symfony.com/blog/preparing-your-applications-for-twig-3

Looks like twig-extensions needs a branch for twig 3.0 support as they have 1.x support in the master branch still.
https://github.com/jasny/twig-extensions

jasny/twig-extensions now have v1.3 released which supports Twig 3, we can now move ahead with updating the twig version constraint for TwigView.

The other two dependencies aptoma/twig-markdown and asm89/twig-cache-extension will also need a new releases :/

https://github.com/aptoma/twig-markdown/pull/34

There seems to be an official markdown plugin too https://github.com/twigphp/markdown-extra, so perhaps aptoma/twig-markdown should be replaced with that?
asm89/twig-cache-extension also seems to be unmaintained. It was meant to be replaced by https://github.com/twigphp/twig-cache-extension but even that doesn't have any activity and no releases.

I'm using the markdown-extra package with twig 3.

Any objections against going 3.0 only?

Any objections against going 3.0 only?

None from me. Twig 3 requires 7.2+ which is inline with Cake 4.

@WyriHaximus Is the asm89/twig-cache-extension dependency essential for TwigView's working? Cause I don't see it getting updated to support Twig 3 anytime soon.

Unless someone has the time and motivation to get asm89/twig-cache-extension or twigphp/twig-cache-extension to do a Twig 3 compatible release the best course of action right now would be simply replace aptoma/twig-markdown with twigphp/markdown-extra and release TwigView 5 without support for Twig 3.
Currently Bake 2.0.0 had to be tagged with while using dev-master of this plugin which isn't very nice. So we need to come up with a short term solution quickly and look into supporting Twig 3 later.

@ADmad planned to do it today after dinner :). And no we don't need the cache extensions to work.

@WyriHaximus Great to hear.

WIP PR: https://github.com/WyriHaximus/TwigView/pull/224

@ADmad locally all tests are passing, only need to get the markdown plugin in

5.0 is out :tada:


add cases for MAC addresses in EVE-JSON - v2

This cleans the previous PR #197 of unnecessary comments and brings it up to date with the current master.

Does this depend upon a Suricata PR ? Which one ?

Does this depend upon a Suricata PR ? Which one ?

That's https://github.com/OISF/suricata/pull/4975.

Merged in #291, thanks!


Buff the AME until somebody fixes engineering

All the engineering numbers are complete unusable garbage and it's apparently completely impossible to properly set up power on some maps.

I do not know why they first removed half of the AME components, then also the possibility of buying AME from cargo and all for the sake of AME getting buffed 100 times

If you need help understanding things ask on discord.


update group index to start at 1 rather than 0

to resolve #54

This looks good to me. All tests are passing!


Session Secret Generation

The backend makes use of a SESSION_SECRET variable in the .env file. The default value should be randomized so clients are not able to decrypt and tamper with the session cookie data.
# This is used in encrypting the session cookie sent to the client. Please change this.
SESSION_SECRET=PleaseChangeMe!

Done in ef0178cab5da2651c0bbc4c09ad8ceebe63d3539
I decided to load a session secret (randomly generated once) from an env file for the master instance, and leave the dev instances as the default as that might be useful for debugging, and shouldn't actually pose any security risks in a test environment.


Unable to Lock/Unlock Car

Please check Services, Known Bug / Issues and Troubleshooting over here first: https://github.com/dahlb/ha_kia_hyundai/blob/master/README.md
Region and Brand of car
Describe the bug
When I clock the lock/unlock toggle switch in Home Assistant I get the following error:
Failed to call service lock/lock. Lock.async_lock() got an unexpected keyword argument 'entity_id'
Debug logs if an error occurred
Outlined here: https://github.com/dahlb/ha_kia_hyundai/blob/master/README.m
Nothing reflected in the logs
To Reproduce
Steps to reproduce the behavior:
Click on the Lock toggle switch in HA and the pop up appears
Expected behavior
Car to lock and unlock
Screenshots

Additional context
Not 100% sure this is a ha_kia_hyundai issue or home assistant issue.. but I believe it started after the 2023.2.0 release..

thanks for writing up this detailed error report, I think it is fixed in the linked commit.  It was caused by a change in ha with a deprecation warning.
fixed in 1.7.2

Thank you! I will test when I get back home. Your addon to HA has given me so much. I love that I can auto lock my car at night and auto start the climate when its cold to go pickup up my kid from school. 👍🏼


hugo 0.34 'server' does not regenerate when rss template updated

seems to work fine in 0.30.2

You need to be more specific.

Is this with fastRenderMode enabled?
What do you eman by "does not regenerate"?

I'm not sure if that is enabled - have always used Hugo defaults - so if the default behavior changed from 0.30.2 to .34 I would have taken that change without knowing.
Does not regenerate => hugo server shows that the file changed, but does not output "generated items" output and the file does not change to reflect changes to the rss template.  on 0.30.2 - it shows the file was edited and does re-output the "generated items" report and the file does change to reflect changes to the rss template.

I think it is more general to any output formats template.
Live Reload does not trigger when saving changes on a output format template. You have to manually refresh the page.


Bump torch-mlir to 45bb17e

llvm/torch-mlir@45bb17e

I can't merge this. @ScottTodd can you hit the green button for me?


Library in dist folder not up to date

Hi,
I discovered that in your demo, the latest version is used of grapes-mjml.min.js.
The version in the demo is:  0.0.24.
The dist folder is still not updated with this version and its stuck on 0.0.12.
I have no issues when using the latest version, the only problem is that i don't get it minified :/
Could you please update the dist folder with the latest minified version of grapes-mjml.in.js?
Kind regards,
Stephan Hovius

I also found out that version 0.0.24 give some error about unescaped backslashes.
for example --> row 38144: "Backslash": "",
this should be "Backslash": "\",
also setminus, setmn, smallsetminus have this problem.

@StephanHovius this plugin comes with MJML library inside and what you mention is from their code. I've also disabled the minification as it was failing with the mjml code. Unfortunately, I didn't have much time to investigate it

should be fixed and we provide minification nowadays


eventChannel emit END, unsubscribe not called

In the advanced docs for using channels, I don't understand why the clearInterval is written in 2 places.
Here's the code for reference:
import { eventChannel, END } from 'redux-saga'

function countdown(secs) {
  return eventChannel(emitter => {
      const iv = setInterval(() => {
        secs -= 1
        if (secs > 0) {
          emitter(secs)
        } else {
          // this causes the channel to close
          emitter(END)
          clearInterval(iv)
        }
      }, 1000);
      // The subscriber must return an unsubscribe function
      return () => {
        clearInterval(iv)
      }
    }
  )
}

As I understand it, calling emitter(END) causes the channel to close. And according to the docs, the unsubscribe function is "used by the channel to unsubscribe before the event source complete."
So, in the case of the above code, I would expect calling emitter(END) to cause the unsubscribe function to be executed since we just closed the channel by emitting the END event. However, this is not the case, as clearInterval is coded in both places, i.e. emitting END event does NOT call the unsubscribe function.
Is there something I'm not understanding about emitting an END event? Does the unsubscribe function only get called when the consumer saga calls channel.close?

You are right, according to the source code the unsubscribe function will be called automaticallt when emiting END. Therefore doubling this clearInterval is futile. Would you like to send a PR with a docs change? Or should I update it myself?
Thanks for noticing!

I would like to provide a PR for this, though I may need some coaching since it will be my first 😄

Well, basically you need to fork this repo from its main page on github. And then clone locally your version of it and create a new branch from the master.
Once you push your changes out you will see a possibility to make a pull request on the repository's site.
Changes need to be made in the place I've linked in my previous comment. Would advise just to extract close function from the returning object and put it before const unsubscribe. Then just call this instead of chan.close.

Awesome, I will submit a PR by the end of the day! Thanks for walking me through the approach.

You can also include in your commit's message something like "fixes #875", so this issue will be automatically closed when your PR gets merged in.
Also remember about updating the code snippet in the docs :) (the one which made you noticing this issue)


Handle NaN, Infinity in simulation results in API and apps

See: https://github.com/biosimulations/Biosimulations/issues/2737

As discussed today, VCell also uses NaN to report partially successfully simulations (individual tasks which partially succeeded). From HDFGroup/hsds#87, it sounds like the HSDS returns a superset of JSON similar to JSON5. It doesn't sound like we want to use the ignore_nan HSDS option because it sounds like this also converts +/- Inf to null, which would lose information.
When an HDF5 data set has NaN or Inf, the HSDS responses likely look like this:
{
  "values": [1, NaN, Inf, -Inf]
}

Because this is not valid JSON, this needs to handled when the response is parsed inside the HSDS client library. This can't be post-processed outside the library. There are NPM libraries which can parse this, including json5.

The HSDS appears to return data in a format compatible with JSON5. When Axios receives output from the HSDS which contains NaN or Inf (JSON5), the data attribute of Axios' response is an instance of string rather than an instance of object. This string can then be parsed to an object by JSON5.parse.
I incorporated JSON5.parse into libs/hsds/client/src/lib/dataset.service.ts. This seems to work with the example in #2737 and the example below.
Code to test HSDS
Create dataset with NaN and Inf
from biosimulators_utils.sedml.data_model import Report, DataSet
from biosimulators_utils.report.data_model import DataSetResults
from biosimulators_utils.report.io import ReportWriter, ReportReader
import base64
import numpy
import requests

report = Report(
    id='report',
    data_sets=[
        DataSet(
            id='d1',
            label='D1',
        ),
        DataSet(
            id='d2',
            label='D2',
        ),
        DataSet(
            id='d3',
            label='D3',
        ),
    ],
)
results = DataSetResults({
    'd1': numpy.array([1., 2., 3.]),
    'd2': numpy.array([4., numpy.NaN, 6.]),
    'd3': numpy.array([7., numpy.inf, 9.]),
})
base_path = '.'
rel_path = 'sim.sedml'
ReportWriter().run(report, results, base_path, rel_path)

Load HDF5 into HSDS
runId=61c22ea9b44ecd73d2c9736c
hsload \
    --endpoint https://data.biosimulations.dev  \
    --username ****** \
    --password ****** \
    --verbose \
    reports.h5 \
    /results/${runId}

Get HDF5 from HSDS
hsget --endpoint https://data.biosimulations.dev  \
    --username ****** \
    --password ****** \
    --verbose \
    /results/${runId} \
    2/reports.h5

Read data HDF5
base_path_2 = '2'
results_2 = ReportReader().run(report, base_path_2, rel_path)

Get list of datasets
response = requests.get('https://data.biosimulations.dev/datasets',
    params={
        'domain': run_id + '.results',
    },
    headers={
        'Authorization': auth_string,
        'Accept': 'application/json',
    },
)
response.raise_for_status()
data_set_ids = response.json()['datasets']
print(data_set_ids)

Read data directly from HSDS
username = '******'
password = '******'
auth_string = username + ':' + password
auth_string = auth_string.encode('utf-8')
auth_string = base64.b64encode(auth_string)
auth_string = b"Basic " + auth_string

run_id = '61c22ea9b44ecd73d2c9736c'
data_set_id = 'd-e9e8ad56-ef34fa21-bce7-b90872-0ae8d2'

response = requests.get('https://data.biosimulations.dev/datasets/' + data_set_id + '/value',
    params={
        'domain': run_id + '.results',
    },
    headers={
        'Authorization': auth_string,
        'Accept': 'application/json',
    },
)
response.raise_for_status()
response.json()

:tada: This issue has been resolved in version 8.4.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Database design (MERISE)

[ ] Data dictionary
[ ] Functional dependency matrix
[ ] Conceptual data model
[ ] Logical data model


README: Use SVG badges

This PR changes the build badges to SVG for sharper readability ✨
Also: Drop defunct Gemnasium badge.

Coverage decreased (-0.05%) to 98.188% when pulling 8dc3335e12d218198cd0612968ebcbfd509894ea on olleolleolle:patch-1 into 185f2c007911f627a6592f3a4a978bfe6673993b on adhearsion:develop.


force update is not being triggered

4afd5060

other users that have old backend and don't get ForceUpdate are on plugin version 2.0.319 or lower. in this version the local settings had no minimal backend version. it was added only in 2.0320.
github
and force update does not return force update for the plugin


Adding pivotal story-type label to the pull request.

@simarpreets the changes looks fine. Can you please update the readme file as well?

done.


threads does not work

Hi,
I find threads doesn't work under the following case and I must set -j > 30 to run this command:

Is it a bug of snakemake or something related to cellranger (an tool developed by 10x Genomics Co.,Ltd. that used to process singlecell RNAseq data).

Hi- What do you mean by this command only runs when -j is more than 30? What happens If you run snakemake -j 10? That rule is never executed and snakemake hangs forever?
cellranger shouldn't be aware of how snakemake submits it and snakemake doesn't know what happens inside the shell directive, snakemake only checks that the shell exit code is 0. So, if it is a bug, I would imagine it would happen also with something like:
...
shell:
    "echo {threads}" > {output}

I suspect there is something wrong with your code rather than snakemake or cellranger, maybe try to post more detail and run snakemake with -p option.

Yes,
There are two rules in my snakemake scripts

When I run snakemake -j 10 ...

This is expected behavior
When a rule requests more threads (threads: 30) than provided to snakemake (-j 20), it is scaled down to match number of threads provided to snakemake, so it essentially becomes: threads: 20. Now since you only give snakemake 20 threads, and the cellranger rule requests all 20 threads, it cannot be run at the same time as any other rules. That's why the fastqc rule runs first and cellranger rule only starts after the fastqc rule has completed.
If you run snakemake with -j 31, and assuming your cellranger rule does not dependon output of fastqc rule, you should see both rules start in parallel.

I am still confused. Why fastqc runs first ? Can we run cellranger first and then fastqc ?
When fastqc runs (2 cores were used), there are still 18 cores left, cellranger can use the remaining cores to fully utilize the available resource, is it more reasonable ?

You can set rule priority to prefer one rule over another in execution. Otherwise the order of execution is random among rules that are ready at any given time.

When fastqc runs (1 cores were used), there are still 19 cores left, cellranger can use the remaining cores to fully utilize the available resource, is it more reasonable ?

How would Snakemake know that a rule that is specified to be run with 20 threads can also be run at 19 threads? How do we know that running rule A with 1 thread and rule B with 19 threads would be faster than running rule A with 1 thread till completion and then running rule B with all 20 threads? This really depends on the nature of each rule, how long they generally take to run, and how well they are multi-threaded. There really is no way for Snakemake to know.

Got it, thanks!


Enhancement YAML file validation

type:
enhancement, tests

description:
This PR introduces enhancements to the YAML file validation process in the 'hostsensordeploy.go' file. It also includes the addition of new test cases in 'hostsensordeploy_test.go' to ensure the correct functioning of the updated validation process. The main changes include:

Validation for empty YAML files.
Validation for non-YAML files.
Addition of test cases for the new validations.
Update of test file paths in the test cases.
Addition of new test data files.

main_files_walkthrough:
 files:

core/pkg/hostsensorutils/hostsensordeploy.go: Added validation for empty YAML files and non-YAML files in the 'loadHostSensorFromFile' function.
core/pkg/hostsensorutils/hostsensordeploy_test.go: Updated the file paths in the test cases and added new test cases for the new validations.
core/pkg/hostsensorutils/testdata/hostsensor.yaml: Added a new test data file 'hostsensor.yaml'.
core/pkg/hostsensorutils/testdata/notAYamlFile.txt: Added a new test data file 'notAYamlFile.txt' to test the validation for non-YAML files.

/describe

/describe

the PR is ready btw @matthyx


Spinner Customization

Is it possible to use custom spinners? if so how can I set up mine? I wanted to set this one.
{ "󰨔", "󰕞", "󰕣", "󱡗", "󱡖", "󱡗", "󰕣", "󰕞" }

progress_icon = { pattern = { "󰨔", "󰕞", "󰕣", "󱡗", "󱡖", "󱡗", "󰕣", "󰕞" } },

Glad you figured it out!
Related: #161


[WIP] Release 1.3

Hi Matteo, do you have an ETA to merge 1.3 to master?

When it's officially released

Will prioritize this in the coming days


chore(tracing): do not error on too short trace id + tests

Summary
This is a forward port of kong-ee/pull/8767 the actual fix is not needed because already covered by the new propagation module.
This commit ports the updates to the pdk (no longer throw error on too short id) and tests.
Checklist

[x] The Pull Request has tests
[x] (no) A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
[x] (no) There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference
KAG-4218

@samugi Should there be a changelog for this PDK related change? It should be a fix.


Support deferred loading snapshots in the Dart VM

Currently, it's not possible to use Dart's deferred loading feature in snapshots. This means that programs that sometimes use large chunks of code have to pay a substantial price up front even when that code isn't used (see issue #20470). It also means that snapshots aren't a reliable means of precompiling user code (issue #20482 and issue #20483), since that code may include deferred imports.

Marking this as "not planned" effectively means that deferred loading is invalid in any code that might possibly be used in a command-line context. Are we okay with that? @dgrove @kasperl


Rework code yii3.

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
✔

Tests pass?
✔

[x] fix phpdocs.
[x] rework code.

Thanks!


feat(dev): Add fixtures for testing traces_sampler

Extracted from https://github.com/getsentry/sentry-python/pull/863.

Can't we just include this as part of the PR where the functions are actually used? I think I am reviewing this PR without context

Sure, if you'd rather. I've been getting coaching to keep my PR's a manageable size, so I've tried to pull out from the main PR anything which could be thought of on its own and/or which was a non-behavior-changing prerequisite for the main changes. I recognize that for that to work there's got to be more cross-referencing and possibly more faith that such prereq changes are going to be used in a reasonable way. Still trying to find the balance, I guess. (And very open to advice in this arena!)
Perhaps rather than mushing these fixtures back into the main PR, I could pull the tests out of that and include them here, and just mark them all xfail for the moment?

Perhaps rather than mushing these fixtures back into the main PR, I could pull the tests out of that and include them here, and just mark them all xfail for the moment?

That's also fine, though I think anything under 400loc is reviewable.