BTRFS with optional USB thumb drive as RAID1

To protect LND data better against data curruption and data loss (because thats were your funds are) its to research the idea to add two 4GB micro usb sticks as a RAID to the raspberryPi - like this:

Its a super cheap option (both sticks for around 10 USD) and flash drive should be more secure against undervolstages.
I Raid USB setup on the Pi is new to me .. so all research and tips are welcome. I have this as a starting point: https://unix.stackexchange.com/questions/120874/how-to-setup-a-raid-system-using-usb-sticks-as-storage-media

see ;) https://twitter.com/CandleHater/status/1097671449207324672

Hardware improvement - I've ran a FreeNAS server for ~5 years now booting off a ZFS mirror volume on USB flash drives. They tend to fail between 6-18 months. I got tired of replacing them and bought cheap used intel SSDs.
This might be an affordable solution:
USB to m.2 SATA adaptor ($15 each)
https://www.amazon.com/SHINESTAR-Adapter-Portable-Performance-Samsung/dp/B0768V1SK7#customerReviews
m.2 SATA SSD ($22 each)
https://www.amazon.com/Transcend-MTS400-Solid-State-TS64GMTS400S/dp/B077H276GQ/

Software improvement - software raid should feature a modern file systemZFS or btrfs. These file systems checksum data, and can fix corruption on the fly by comparing with mirror/parity devices. Raid alone is obsolete (start at 5:05).

I run a raspberrypi 2 with Btrfs raid for years. Scrubbing takes a long time, but otherwise it works great.
ZFS or Btrfs would provide snapshots without the need to shut down lnd or bitcoin demon. This way we could have versioned backups in case of file corruption.
With tools like btrbk, we could manage backups to network or attached storage.

$50 for this 128 GB USB SSD: https://www.sandisk.com/home/usb-flash/extremepro-usb … Quarter capacity for half price might still be feasible. Will keep an eye out if someone makes them.

$50 for this 128 GB USB SSD: https://www.sandisk.com/home/usb-flash/extremepro-usb. There is no smaller size available unfortunately. A good alternative to used SSD-s and messy USB adapters.

or these are cheap, small and more error prone: SanDisk Cruzer Fit 16GB USB 2.0 Flash Drive https://www.amazon.co.uk/dp/B07MDXBT87/ref=cm_sw_r_cp_apa_i_UBIBCbAHGSSEZ. Storing only the LND dir on the flash RAID is nowhere near as a heavy write load as booting from it or using as swap. They would probably last that 6-18 months  until channel state backups are sorted out :).

or these are cheap, small and more error prone: SanDisk Cruzer Fit 16GB USB 2.0 Flash Drive https://www.amazon.co.uk/dp/B07MDXBT87/ref=cm_sw_r_cp_apa_i_UBIBCbAHGSSEZ. Storing only the LND dir on the flash RAID is nowhere near as a heavy write load as booting from it or using as swap. They would probably last that 6-18 months  until channel state backups are sorted out.

Is there any documentation about SSD being significantly more reliable and durable than USB stock though? Last time I looked for data between HDD and SSD and didn't find any reliable data about this. If the SSD if 3 times more reliable than USB stick but 5 times the price, what's the point. Also USB sticks take less space.

Info on how to make a ZFS raid: https://tutorials.ubuntu.com/tutorial/setup-zfs-storage-pool#0

FYI: I am going at the moment with a BTRFS Raid1 with those two small USB thumb drives from the picture on the first post. BTRFS looks good, because it also has self-healing features.

Yay! I'd love to experiment with that. I have a little experience with Btrfs. Maybe the setup could detect if there is one or two usb-sticks present and offer to use them. We could make guesses based on the size of the disks. If we have 500 GB or more, that's your /mnt/hdd. If we have less then it is an sd-device.
How would you recommend changing the lnd path? Should /mnt/hdd/lnd be the mount point, so that all the scripts run without modification? Or is there one central setting to announce that e.g. /mnt/ssd/lnd is the new home?

@raumi75 the detection works already quite well in my test script - I search for two devcies with the same size :) I am fresh new to the BTRFS so it would be great if someone can take a look once I have a first prototype and knows how to optimize it (versioning, etc.).
For the beginning/testing I want to make it as a extra option you do after the setup and make it possible in the config script that you can switch between HDD and USB-DATASTORAGE. When the testing gets positive then this can be worked into the setup/update process.
I think I will check all scripts & config again that they use the home/bitcoin/.lnd path for LND. Thats already a linked directory to the HDD folder. So when the USB-DATASTORAGE gets switchen on/off it just needs to update that one link and we should be good.
Also I am thinking that it would make sense to move more personal data (tor, config, etc.) over to the USB-DATASTORAGE .. so that the HDD is just a large public data store (blockchain, torrents, etc). So that in a worst case scenario: The HDD, sd-card and even the RaspberryPi board it self can broken and you just take your USB-DATASTORAGE plug it into a fresh setup and it will recover and take off from where it stopped.

I will take a look. Just tell me when you are ready.
Btrfs opens many possibilities. We could take hourly snapshots without stopping lnd. Could even copy them to the hdd in a Btrfs send/receive type format. There are scripts that do that (btrbk for example)
If I understand correctly, old channel-state doesn't do any good. Maybe we could even create snapshots every 10 minutes and delete them soon.

This is great, my two 16 GB USB drives are waiting to be tested. One comment about the channel states: currently it is not useful to have a snapshot without stopping LND. Just lost my channels due to this problem, could not restore despite having multiple on-the-go scp backups. It is the right thing that the channel.db won't get corrupted due to having it mirrored, but some information is in the RAM and not written until LND is stopped. For snapshots LND must be stopped,  wallet.db and channel.db saved and restarted. It could be done after every state change if possible.
Found this good collection on the issues: https://gist.github.com/bretton/22f628caffde79390a796e75ea528053

Are you sure this is true for zfs and btrfs? I'm not an expert and don't mean to spread half-truths, but my understanding is that you can't backup virtual machine images, database files while the application is running, because the files might change DURING the copying. A (zfs-/btrfs-)snapshot is created in that instant, because it's a CoW (Copy on Write)-filesystem. The snapshot (which is practically forzen in time) can then be copied to a different device.
Not sure how lnd behaves so please please don't be reckless. All I'm saying is this needs more research, but we should not dismiss the possibility that this works. We could stress test this on testnet.

Are you sure this is true for zfs and btrfs? I'm not an expert and don't mean to spread half-truths, but my understanding is that you can't backup virtual machine images, database files while the application is running, because the files might change DURING the copying. A (zfs-/btrfs-)snapshot is created in that instant, because it's a CoW (Copy on Write)-filesystem. The snapshot (which is practically forzen in time) can then be copied to a different device.
Not sure how lnd behaves so please please don't be reckless. All I'm saying is this needs more research, but we should not dismiss the possibility that this works. We could stress test this on testnet.

Thanks for explaining this. The problem with the on-the-go SCP backup might be indeed that the data changes while being copied. Restoring a frozen snapshot should not be different than powering up after a power outage, which is usually not a problem. Need to be tested in any case and I am up for that too!

The USB-Raid should focus on providing reliable hardware storage first ... the Backup is a feature hopefully comming from LND itself soon: https://github.com/lightningnetwork/lnd/pull/2313

Just thinking, possibly too far: once the LND directory and other personal data is moved from the HDD, setting up a mirrored filesystem of the blockchaindata would make it really easy to clone these devices (even without stopping). Or just could leave two HDD-s running mirrored as well adding further redundancy.

To investigate: check comments on LND data storage in this thread: https://twitter.com/AllYourBanks/status/1100883762261475328

Linking a different approach mirroting an extra partition on the HDD and one on the SDcard. https://twitter.com/vindaRd/status/1103327828546859008?s=19
Might be useful for the OdroidHC1/HC2 which only have 1 USB port to avoid using a USB hub.

TODO: Maybe even move the chainstate to the usb-raid: https://github.com/rootzoll/raspiblitz/issues/413

TODO: Scrub data with background task every hour:

btrfs scrub start /mnt/raid/is better than fsck. Will start a check on a mounted partition. It's done in the background and checks each checksum of every block. If one is wrong, Btrfs will fix it by taking the block from the good disk and copy it to the bad one. You could run this periodically with cron or as a menu item.

@rootzoll I have built from the 1.1 dev branch to my Odroid XU4 on DietPi and had some errors installing btrfs at
# prepare for BTRFS data drive raid
sudo apt-get install -y btrfs-tools

Unpacking btrfs-tools (4.7.3-1) ...
Processing triggers for initramfs-tools (0.130) ...
ln: failed to create hard link '/boot/initrd.img-4.14.66+.dpkg-bak' => '/boot/initrd.img-4.14.66+': Operation not permitted

and

Processing triggers for initramfs-tools (0.130) ...
ln: failed to create hard link '/boot/initrd.img-4.14.66+.dpkg-bak' => '/boot/initrd.img-4.14.66+': Operation not permitted

See the full output of the build_sdcard.sh: https://gist.github.com/openoms/a0f4fd750e3c24c123b8eea0a7b0dbb0
Not sure if it is causing any problems, but will just leave this here for future reference.

Related to this and "fsck" of the data drive which was recently added  -  ie, the bigger subject of data corruption it might help to add additional checks for inadequate power noted here: https://github.com/rootzoll/raspiblitz/issues/474#issue-427212699

Just a quick note about btrfs, ZFS, journaled filesystems and USB storage in general. What you need is correct write/flush barrier semantics otherwise you'll get data corruption or loose your btrfs filesystem or ZFS pools in case of crashes or power losses. Some information (garnished with a lot of FUD/BS) here

ln: failed to create hard link '/boot/initrd.img-4.14.66+.dpkg-bak' => '/boot/initrd.img-4.14.66+': Operation not permitted

DietPi folks (well, Daniel -- it's more or less a one man show) are ignorant. They put /boot/ on a FAT partition for no reason which breaks Debian package management in general.
RPi Trading employees since having to deal with a FAT partition (the VideoCore IV can't cope with Linux filesystems) therefore use a tool called dpkg-divert to manage their bootloader and kernel package updates on the RPi which makes all those updates insanely slow (though they prefer to blame dpkg instead).
There's no reason to put /boot/ on a FAT partition (not even on the RPi but there RPi folks decided to combine primary OS -- ThreadX -- and settings for the secondary OS in one single partition to not overwhelm their user base and so ended up with all the stuff on a FAT partition) and as such what you see is as expected. It's not related to btrfs but the result of anything calling update-initramfs since /boot on non POSIX compliant filesystems is broken by design with Debian and derivates.

Thanks for all the input. Will need some more time to process the details. To not delay the release of v1.2 I will move the issue to a future release. But its still on high prio.

Creating a RAID1 array  is relatively easy with a package called "mdadm".  But now that LND 0.6 has it's own channel state backup this may not be as important. https://github.com/lightningnetwork/lnd/releases

apt install mdadm
mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/sda1 /dev/sdb1
mkdir -p /mnt/raid1
mkfs.ext4 /dev/md0
mount /dev/md0 /mnt/raid1/
echo "/dev/md0 /mnt/raid1/ ext4 defaults,noatime 0 1" | sudo tee -a /etc/fstab
mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf
cd /mnt/raid1
ls

Restoring the SCB in lnd0.6 still means force closing all the channels and paying the on-chain transaction fees. It is still important to avoid data loss.
Creating a BTRFS RAID is equally simple and possibly superior due to the ability to create versioned atomic snapshots (https://github.com/digint/btrbk thanks @raumi75 ).
I am currently running the 1.1  RaspiBlitz on an Odroid XU4 Cloudshell2 with two 500gb SATA HDD-s in BTRFS RAID1. Aiming to create a routing grade node.
Have no experience with ZFS and did not find an similarly easy way to set up yet, but could be interesting.

Looking at these lately: https://www.aliexpress.com/item/Fast-Speed-2-Bay-USB3-0-HDD-Enclosure-Dual-2-5-Inch-Hard-Disk-Drive-Raid/32969221268.html
Option to connect two 2.5" HDD/SSD through one USB3 connector. With PM mode disks appear as two individual disks and can be run as a software BTRFS RAID1. Needs it`s own extra power supply. No SBC can power two HDD-s from USB only.

Nice find!  That is a cool product for the $.  Other links for it:
www.gearbest.com/hdd-enclosure/pp_1042363.html
https://www.amazon.com/MAIWO-USB3-0-2-5inch-Enclosure-K25682/dp/B01M7PPCRH

I still think we can keep the HDD single for blockchain storage when we add two USB thumb drives as RAID for LND and other critical data. First of all this is the cheapest option and does not need additional power supply - as two HDD would need.
The only problem left with one HDD is when we run into a data corruption of the blockchain the redownload/sync of the blockchain is needed which is a long off-time for the node. But this could be fixed with the "Background Torrent Seed" feature, which will help seeding on the one hand and keeps a backup version of the blockchain ready to go/replace in the case of blockchain data corruption and would minimize downtime in such cases. A first experimental version of "Background Torrent Seed" feature will be part of v1.2 - but it needs to be optimized to run in the background. If we can get that feature working for v1.3 release it would be the perfect companion for this 2-Thumb-Drive-Raid feature that is also on high prio for the v1.3 release.

I still think we can keep the HDD single for blockchain storage when we add two USB thumb drives as RAID for LND and other critical data. First of all this is the cheapest option and does not need additional power supply - as two HDD would need.
The only problem left with one HDD is when we run into a data corruption of the blockchain the redownload/sync of the blockchain is needed which is a long off-time for the node. But this could be fixed with the "Background Torrent Seed" feature, which will help seeding on the one hand and keeps a backup version of the blockchain ready to go/replace in the case of blockchain data corruption and would minimize downtime in such cases. A first experimental version of "Background Torrent Seed" feature will be part of v1.2 - but it needs to be optimized to run in the background. If we can get that feature working for v1.3 release it would be the perfect companion for this 2-Thumb-Drive-Raid feature that is also on high prio for the v1.3 release.

I'm wondering if there might be a better, more resource efficient way to create a blockchain backup.  Basically if there is enough space, lets say 1TB or more, then cannot a background script/process just fsync to a backup partition lets say once per hour or once per day?  fsync is ifficient that it will see and copy only changed files and can retain all file/directory names, rights, timestamps, etc.  I'm pretty sure that fsync also might not require the source to be unmounted/un-used.
This does not preclude the possible torrenting of the blockchain - say from the backup partition/copy.

@fluidvoice such a blockchain backup with rsync (stopping bitcoind, syncing, restarting bitcoind) is a good idea. Will keep that in mind on this issue.

IDEA: Switch to BTRFS as the default file system for the HDD.

HDD-s from older RaspiBlitz versions would need to be converted to BTRFS from EXT4 to benefit from the added features.
This is possible with btrfs-convert (https://www.howtoforge.com/how-to-convert-an-ext3-ext4-root-file-system-to-btrfs-on-ubuntu-12.10) although file system operations are risky.
A reliable complete backup of the LND dir is a must before starting a conversion, we can use:
/home/admin/config.scripts/lnd.rescue.sh backup (https://github.com/rootzoll/raspiblitz/blob/master/FAQ.md#1-recover-lnd-data.)

once the HDD is converted to BTRFS a resize is quick and straightforward:
sudo btrfs filesystem resize -20g /mnt/hdd

a second 15GB BTRFS partition could be used for LND. This can be mirrorred as a RAID1 to an extra 16GB USB drive.

a small third EXT4 partition would be needed for SWAP because BTRFS does not support a swap function yet and also the swap should not be mirrored.

BTRFS could achieve easy and cheap data redundancy for LND needing only one extra USB drive.
The LND RAID1 setup would never rely completely on flash drives as it would be the case when mirroring between 2 USB-s.
Having the blockchaindata (+ optional databases like electrs) on BTRFS would make it possible to clone a RaspiBlitz HDD on-the-fly by adding a new HDD as a mirrored drive. This could be an alternative to scp/rsync copy of the blockchain from one RaspiBlitz to an other.

RESEARCH:  Vagrant as per @stadicus3000 tweet
https://twitter.com/fluidvoice/status/1130445221832081412
Vagrant - tool to easily build consistent VM development environments on Virtualbox, VMWare, Docker, etc.: 3 part, short YT vid intro

@fluidvoice until the X86 support is merged in by @rootzoll we should continue to discuss about the VM build here: https://github.com/openoms/raspiblitz/issues/46

IDEA: Switch to BTRFS as the default file system for the HDD.
* HDD-s from older RaspiBlitz versions would need to be converted to BTRFS from EXT4 to benefit from the added features.
  This is possible with `btrfs-convert` (https://www.howtoforge.com/how-to-convert-an-ext3-ext4-root-file-system-to-btrfs-on-ubuntu-12.10) although file system operations are risky.
  A reliable complete backup of the LND dir is a must before starting a conversion, we can use:
  `/home/admin/config.scripts/lnd.rescue.sh backup` (https://github.com/rootzoll/raspiblitz/blob/master/FAQ.md#1-recover-lnd-data.)

* once the HDD is converted to BTRFS a resize is quick and straightforward:
  `sudo btrfs filesystem resize -20g /mnt/hdd`

* a second 15GB BTRFS partition could be used for LND. This can be mirrorred as a RAID1 to an extra 16GB USB drive.

* a small third EXT4 partition would be needed for SWAP because BTRFS does not support a swap function yet and also the swap should not be mirrored.

BTRFS could achieve easy and cheap data redundancy for LND needing only one extra USB drive.
The LND RAID1 setup would never rely completely on flash drives as it would be the case when mirroring between 2 USB-s.
Having the blockchaindata (+ optional databases like electrs) on BTRFS would make it possible to clone a RaspiBlitz HDD on-the-fly by adding a new HDD as a mirrored drive. This could be an alternative to scp/rsync copy of the blockchain from one RaspiBlitz to an other.

Thinking further:
2 partitions on the HDD would be the most simple and beneficial.
Could keep the EXT4 filesystem for the blockchaindata and optionally have an mdamd RAID1.
The mdamd RAID1 would achieve:

protection from a HDD drive failure (no downtime)
on-the-fly HDD cloning option when adding a new HDD
room for the swapfile
in a hybrid SSD+HDD RAID1 case mdamd can mitigate the speed difference with --write-mostly and --write-behind (https://unix.stackexchange.com/questions/250789/can-i-get-ssd-rw-performance-while-keeing-data-security-if-i-combine-ssd-hdd-i)
no safety from bitrot (https://en.wikipedia.org/wiki/Data_degradation), but the blockchain is redundant in the network, so can be redownloaded, corrected any time

A new setup could set two partitions during the 30initHDD.sh and existing BLOCKCHAIN partitions could be shrinked (http://www.microhowto.info/howto/reduce_the_size_of_an_ext2_ext3_or_ext4_filesystem.html) to give space for 15 GB BTRFS partition which then can be mirrored to a cheap USBdrive or the second HDD.
The BTRFS RAID1 for the LND dir would achieve:

some protection against bitrot for the most important data
on-the-fly HDD cloning option when adding a new HDD.

Converting file systems is not an option: https://askubuntu.com/questions/1073428/where-to-get-btrfs-convert-on-18-04. convert-btrfs is not even shipped with Debian any more, because it was producing broken filesystems.
Converting to both mdamd or btrfs RAID1 would require:

using mkfs on the new disk create a RAID capable filesystem btrfs or ext4 (https://www.nachum234.com/storage/mdadm/001-coverting-linux-system-to-software-raid-mdadm/)

copy the data over to the second disk

destroy the file system of the original with mkfs and add build the raid.

once this is done one of the disks should be ok to be removed or changed in case of failure.

To have the RAID1 option the HDD needs to be created as a single disk (degraded) mdamd RAID1 or use BTRFS from start.

A freshly updated lengthy comparison and test between mdadm, btrfs and zfs:
http://www.unixsheikh.com/articles/battle-testing-data-integrity-verification-with-zfs-btrfs-and-mdadm-dm-integrity.html

Mdadm is truly inferior compared to the two copy-on-write filesystems.
ZFS is not needing a lot of RAM if set up correctly and the author has successfully ran ZFS on the RPi
BTRFS is not broken and being actively developed, has a chance to catch up to ZFS in marurity

@openoms thats a great testing report. I still lean to the BTRFS because on my first try it seems easier to setup on Raspbian. Would you agree fo that decision or is there a real downside compared with ZFS that could bite us in the long way?
Also I dont think to provide a "conversion" on updating your RaspiBlitz from ext4 to BTRFS. If someone wants to go this way - I think its cleaner make a Backup (LNDRESCUE LND tar.gz-Backupfile), format the HDD and start a fresh RaspiBlitz (witch then will start by default with a BTRFS) and importing the Backupdata during the setup.

@openoms thats a great testing report. I still lean to the BTRFS because on my first try it seems easier to setup on Raspbian. Would you agree fo that decision or is there a real downside compared with ZFS that could bite us in the long way?

Completely agree that setting up BTRFS is much more user simple as it is included in Debian by default. It seems to be actively developed as well (https://btrfs.wiki.kernel.org/index.php/Status),  can`t see any significant downsides vs ZFS (especially that we are not running on BSD).
If we switch to any of those a separate SWAP partition will be needed on the HDD/SSD. This make sense anyway as it would be unnecessary to mirror the swap.

Also I dont think to provide a "conversion" on updating your RaspiBlitz from ext4 to BTRFS. If someone wants to go this way - I think its cleaner make a Backup (LNDRESCUE LND tar.gz-Backupfile), format the HDD and start a fresh RaspiBlitz (witch then will start by default with a BTRFS) and importing the Backupdata during the setup.

Sure, forcing the backup is the way to go when playing with the file system. Starting from scratch guarantees a clean disk as well. +1

To make multiple partitions of the HDD and combine one of that (that with the raspiblitz/lnd data) as part of a RAID with a USB 3.0 stick seems to be the best option to go. Lets try to get this into the next release.

TODO: When BTRFS dont offer torrent or move torrent download to storage drive.

TODO: When BTRFS with RAID1 - check if volume can still be temp mounted (without USB thumb drive) on a fresh setup

Why did this get closed? Not possible? Compatibility issues?

Its implemented and in a experimental stage (BTRFS and Raid1 with a USB drive) ... but it seemed a bit early and needs more testing. USB thumb drives may not be best candidates to work with a SSD in a RAID 1. Its not high on the prio list for the coming releases to finish this (other things are more pressuring) ... but of anybody wants to catch up on this, the experimental implementation is there to test & improve.


"Class A source" or "A Class source"

On the first screen to register a new Seedlot, just below the Seedlot Species, there's the source. I wonder if we should fix the order, or is this correct?

@SLDonnelly is the fix necessary here? If so, it's a 1 point task we could fill in sprint 15 :)

Hi @mgaseta, I will reach out to the business area and confirm whether they want A class or Class A. We can keep this as a 1 point filler for 15 for the time being

@SLDonnelly any updates on this one?

Hello @RMCampos and @mgaseta, this inquiry has actually sparked a bit of a debate!! there's been some back and forth about switching to orchard seed and natural stand seed but nothing confirmed. I think for now we can stick with "Class A source"

Np. Closing this issue then, for now. 👍🏻 😃


Hotfix/update packages

This is a package update and I've added myself to the license; refactored the readme;
This PR is in preparation to append a set of slides to this repo to better describe what's going on
https://docs.google.com/presentation/d/1OPEDASQ9N6jYVYcHEBOFCCC_zszxuJXher114DHx3DY/edit?usp=sharing

Good job! I know there has been some confusion of how everything has been structured here, and helping clearing things up and improving the overall codebase is always appreciated. I'm looking forward to your next PR 😃


Refactor APB package to have a public exposed API and internal API that handles shared behavior

Right now bind/unbind/provision/update/deprovsion will all have some similar logic that is not being captured. examples are checking if the namespace exists, checking if the spec has an image, executing the APB cleaning up the sandbox. Some of them (provision/bind) will have to extract the credentials.
I think that this is tied to #248 and we should also think about this from unit testing of callers standpoint (make it easily overridable so that the broker can be unit tested properly).

We're going to create a new issue after an in depth vendoring discussion.


Cc0 updates

First edits – will figure out how to make the others :)

Going to hold on deleting branch until double check.


Move stargaze node

Move stargaze api node to juno-tick api node.
Reconnect monitoring
Change api domain redirection

Stargaze API node has been moved to juno-tick api server.
Appropriate boards have been changed.


vectorization improvements for TPS transform(), apply(), and inverse()

let's not merge this just yet. I found a scipy thin plate radial basis function that is even faster...

@RussTorres i'm good with merging this are you?

I mentioned this before, but not as a review -- this PR makes scipy an actual dependency.  If we're okay with that (and I think we are) we should add it to the requirements file.

I will make that change in the PR once Forrest confirms he is ok with the dependency

there do appear to be a couple of numpy only ideas instead of scipy.spatial.cdist:
https://stackoverflow.com/questions/52030458/vectorized-spatial-distance-in-python-using-numpy


ESLint config should also support CommonJS environment

Sadly not all tools support ESM. Therefore, our eslint config should allow using CommonJS environment variables like module.

Also important for Jest snapshot tests.


fix: Update dependencies and node-sdk

Codecov Report

Merging #167 into master will decrease coverage by 8.83%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master     #167      +/-   ##
==========================================
- Coverage   97.72%   88.88%   -8.84%
==========================================
  Files           4        4
  Lines          44       45       +1
  Branches        3        2       -1
==========================================
- Hits           43       40       -3
- Misses          1        5       +4

Impacted Files
Coverage Δ

config/error-handler.js
100% <100%> (ø)
:arrow_up:

app.js
75% <60%> (-25%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a60e722...b85fa0f. Read the comment docs.


ferretdb: 1.10.1 -> 1.11.0

Diff: https://github.com/FerretDB/FerretDB/compare/v1.10.1...v1.11.0
Changelog: https://github.com/FerretDB/FerretDB/releases/tag/v1.11.0
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 257803 run on aarch64-darwin 1

1 package built:

ferretdb


Inconsistent behavior for queries with very small time units

I'm finding it difficult to get consistent behavior when querying Influxdb (from grafana fwiw) for very finely grained data. See below for data written approx 50,000 times per second, where I am trying to create a scatter plot at sub-millisecond resolution:
> SELECT COUNT(value) FROM "latency_micros" WHERE  time ><PHONE_NUMBER>s and time <<PHONE_NUMBER>s GROUP BY time(1ms) LIMIT 3
name: latency_micros
--------------------
time                            count
2015-10-29T16:06:00Z            51
2015-10-29T16:06:00.001Z        50
2015-10-29T16:06:00.002Z        51

> SELECT COUNT(value) FROM "latency_micros" WHERE  time ><PHONE_NUMBER>s and time <<PHONE_NUMBER>s GROUP BY time(500u) LIMIT 3
ERR: time dimension must have one duration argument
> SELECT COUNT(value) FROM "latency_micros" WHERE  time ><PHONE_NUMBER>s and time <<PHONE_NUMBER>s GROUP BY time(500000n) LIMIT 3
ERR: error parsing query: found n, expected ) at line 1, char 112

Is there a different level of support for different time units?
Puzzled,

@jackzampolin can you investigate and try to repro?

@beckettsean On it!

Data Set:
> select * from latency_micros where time > now() - 10m
name: latency_micros
--------------------
time			    latency	   value
1446575910000000000	one	        1
1446575910000000001	two	        1
1446575910000000002	one	        2
1446575910000000003	two	        2
1446575910000000004	one	        3
1446575910000000005	two	        3
1446575910000000006	one	        4
1446575910000000007	two	        4
1446575910000000008	one	        5
1446575910000000009	two	        5

Queries
GROUP BY time(1ms)
> select count(value) from latency_micros where time ><PHONE_NUMBER>999999999 group by time(1ms) limit 3
name: latency_micros
--------------------
time			    count
1446575910000000000	9
1446575910001000000
1446575910002000000

GROUP BY time(1u)
> select count(value) from latency_micros where time ><PHONE_NUMBER>999999999 group by time(1u) limit 3
name: latency_micros
--------------------
time		  	    count
1446575910000000000	9
1446575910000001000
1446575910000002000

GROUP BY time(500n)
> select count(value) from latency_micros where time ><PHONE_NUMBER>999999999 group by time(500n) limit 3
ERR: error parsing query: found n, expected ) at line 1, char 91

Per the querying time syntax documentation it does not appear that we support nanosecond precision in queries.
I was not able to reproduce the microsecond error. What version of InfluxDB are you running?

This is an older issue and has probably been fixed in a newer version. I'm going to close this issue. Please try a newer version and comment on this issue if you still are experiencing this issue. Thank you.


I tried running your code. Im getting this error. Any insights?

Hi I am trying to run your code for wordnet-verbs in colab. This is the error Im getting, any insights on how to solve this?
Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/drive/My Drive/LorentzianDistanceRetrieval-master/lorentzian_model.py", line 235, in getitem
if n not in self._weights[t]:
KeyError: tensor(4160)

Hi,
Are you sure you're using pytorch 0.3.1? The way key indices were coded changed in pytorch 0.4 and later....

Hi!
You're right changed the pytorch version and it started working! Thank you!


Unable to install traefik v2 as a dependency in the helm chart.

I am using traefik v1 helm chart as a dependency within my helm chart.
dependencies:
- name: traefik

  version: '1.87.1'
  repository: 'https://kubernetes-charts.storage.googleapis.com'

I am not able to find the documentation to use traefik v2 in the same way. The official documentation doesn't help much. Could you please help me with this?
Thanks in Advance.

dependencies:
  - name: traefik
    version: 9.3.0
    repository: https://helm.traefik.io/traefik

As this is answered, I'll gonna close that one. Feel free to reach out again (=


Idempotency Concern of the ChatFunction

Description:
I would like to kindly bring attention to a potential issue in the ChatFunction, which is invoked by the chatRule IoT TopicRule. This function calls the IotData.publish to send chat messages to the frontend, where the frontend appends the new messages with previous ones. However, this message sending is not idempotent. Suppose the IoT-triggered function crashes after successfully sending a chat message to the frontend-facing IoT topic (but before acknowledging the event source that the function has finished). In that case, when the function is retried, another message will be sent and received by the frontend.
Suggested Fix:
Although there's no serious harm in sending two identical chat messages, this duplicate sending due to untimely retries can be prevented. Specifically, one can attach a unique id to each chat message when the message is being generated. Before the frontend appends the new message to the message list, it should check whether the message id is already present in a deduplication set for holding all previous chat message ids. If the id is already present, don't append the message. Otherwise, append the message as usual.
Thank you for considering this feedback. I hope that this suggestion could help improve the idempotency and user experience of the ChatFunction. Please feel free to reach out if you have any questions or concerns.

Thanks, that's a very good point! If you'd like to contribute a PR with your suggested fix, I'd be happy to merge it.


Downloading Images

Is there anyway we could download the images as well when we download the selected data ?

Yes, we should be able to add a "download images" button next to the download table. You can see how the button is added to the layout here:
https://github.com/microscopium/microscopium/blob/db7da4391a694052a6a63273721de05721683ac3/microscopium/serve.py#L363
implemented here:
https://github.com/microscopium/microscopium/blob/db7da4391a694052a6a63273721de05721683ac3/microscopium/serve.py#L261-L275
using some javascript here:
https://github.com/microscopium/microscopium/blob/master/microscopium/js/download_data.js
I have to say I'm not super sure about how to implement it, as you'd want a bit of Python first — to bundle them in a zip file, for example, and serve the zip file.
I did a quick search of the Bokeh discourse and found the "BokehGarden" tool that implements a download button:
https://github.com/emerald-geomodelling/BokehGarden/blob/master/examples/grib_info_download.py
So potentially we can use that? 🤷
@Jamwali I don't have a lot of time to investigate right now, so I wonder if you think you'd be up for making a pull request to add this feature?


DatePicker: cannot use other language than English

Hi,
I tried below code in my app:
<DatePicker
            required
            invalidText="Champs requis"
            datePickerType="single"
            locale="fr"
            dateFormat="d/m/Y"
            bind:value={contact.birth_date}
            style="margin-bottom:1rem"
            >
            <DatePickerInput
                labelText="Date de naissance"
                placeholder="jj/mm/aaaa"
            />
</DatePicker>

But date picker is not french is not displayed. Any other language than english is not working.
I checked that flatpicker node module is properly installed and it's ok, with locale files available.

Did you import the locale file into your svelte component? I do this inside the <script> tag:
import 'flatpickr/dist/l10n/ro.js'

It defintely works! many thanks @weaseldotro . Short reference inside documentation may help others ;-)


When multiple dialogs are open, closing one restores Focus to the parent instead of the next dialog in the stack

Bug, feature request, or proposal:
Bug
What is the expected behavior?
I would expect the next dialog to get focus.
What is the current behavior?
The parent element get focus
What are the steps to reproduce?
Our case is if we have a dialog open and some event happens that wants to replace the currently shown dialog with a new one we will close the current dialog using the dialogRef and then open the new one.
What we see is the new dialog getting focus and then after a short delay you see the dialog lose focus and the parent restores focus to the last element that had it. I assume this happens because in some cases the animation for opening the new dialog finishes before the closing animation finishes.
I've created a stackblitz that isn't our exact case but it exaggerates what I believe our root issue is.
https://stackblitz.com/edit/multi-dialog-restorefocus
What is the use-case or motivation for changing an existing behavior?
In our case the dialogs that are open have the backdrop up to prevent the user from performing an action on the screen behind the dialogs. The current behavior allows the user to tab around on the screen behind the dialog and perform invalid operations.
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
I would assume all but we are using:
-Angular 6.07
-Material 6.3.1 (also tested with 6.4.7)
-Typescript 2.72
-OSX (Haven't confirmed but we suspect this is also root for some issues we are seeing on iOS)
-Chrome
Is there anything else we should know?

The logic behind MatDialog is that each dialog tries to restore focus to the element that was focused before it was opened which is the correct behavior in the majority of cases. Since yours is a special case, you can use the restoreFocus option to disable focus restoration and then handle it yourself. You can get a hold of all the open dialogs from MatDialog.openDialogs.

@crisbeto do you have any recommendations on how to work around the issue in angular 6?


PR to add document on how to write documents.

Also: still trying to do everything in CLI so experimenting.

Closing - to clean up backlog. QA and docs have reviewed the documentation and setup.


GT.M ipv6 support in VISTA

GT.M already supports IPV6. Just need to make sure that VistA knows that.

Fixed.


Diskus Scan - Error http 403

Source information
Diskus Scan - português - 1.4.36
Source language
Português
Steps to reproduce

Webview its work
Error

Expected behavior
Error HTTP 403
Actual behavior
Error site or extension
Tachiyomi version
0.16.1
Android version
Android 13
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have updated the app to version 0.15.3.
[X] I have updated all installed extensions.
[X] I have tried the troubleshooting guide.
[X] I have admitted that I am a clown by having checked this box, as I have not read these acknowledgements.
[X] If this is an issue with the app itself, I should be opening an issue in the app repository.
[X] I will fill out all of the requested information in this form.

@deividgabrielpeira this issue was automatically closed because:

You have not carefully read the acknowledgements.

do not close.


Build failure: [darwin] python3.12-curl-cffi-0.7.1.drv

Steps To Reproduce
Steps to reproduce the behavior:

Run MacOS / Darwin
Run nix flake update
Run darwin-rebuild switch --flake .

Build log
Here's the part that errors out:
error: builder for '/nix/store/37qyknpbxhhkvn2n85vkpjzyflk7qd3l-python3.12-curl-cffi-0.7.1.drv' failed with exit code 137;
       last 10 log lines:
       > calling 'postFixup' function hook '_makeSymlinksRelativeInAllOutputs'
       > calling 'postFixup' function hook '_multioutPropagateDev'
       > Running phase: installCheckPhase
       > no Makefile or custom installCheckPhase, doing nothing
       > Running phase: pythonCatchConflictsPhase
       > Running phase: pythonRemoveBinBytecodePhase
       > Running phase: pythonImportsCheckPhase
       > Executing pythonImportsCheckPhase
       > Check whether the following modules can be imported: curl_cffi
       > /nix/store/xbnb1944q4bg3blhv17fgwr7cx4f0x46-python-imports-check-hook.sh/nix-support/setup-hook: line 15: 71823 Killed: 9               /nix/store/3swy1vadi125g0c1vxqp8ykdr749803j-python3-3.12.4/bin/python3.12 -c 'import os; import importlib; list(map(lambda mod: importlib.import_module(mod), os.environ["pythonImportsCheck"].split()))'
       For full logs, run 'nix log /nix/store/37qyknpbxhhkvn2n85vkpjzyflk7qd3l-python3.12-curl-cffi-0.7.1.drv'.
error: 1 dependencies of derivation '/nix/store/zlrr7qlg6j82kg9f4syjhnwdma8hlp2f-yt-dlp-2024.7.25.drv' failed to build
error: 1 dependencies of derivation '/nix/store/l3c2lc8p374968lnb28frsfwpxhbzw2n-system-applications.drv' failed to build
error: 1 dependencies of derivation '/nix/store/v06gkwgiw41i4n9c319j0mmai72iarg3-darwin-system-24.11.20240728.038fb46+darwin4.0413754.drv' failed to build

Additional context
N/A - this was a regular system rebuild.
Notify maintainers
@chuangzhu
@r-ryantm

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"

these 12 paths will be fetched (0.81 MiB download, 11.29 MiB unpacked):
  /nix/store/18fccp44v47b1pr7g7yjmbsf7wld1hbv-adv_cmds-119-locale
  /nix/store/04w1mgkhawycg64s4q3n8mkcvma5ppx0-apple-framework-CoreFoundation-11.0.0
  /nix/store/gw9ps4037al7ccccb7yq8hfapmp6xq8z-bzip2-1.0.8-bin
  /nix/store/s886gz4z355r8zw9hvp8x3vngh4m267p-ed-1.20.2
  /nix/store/cnb7qaqj61vldzk88hcc8fhwad4ifn0x-file-5.45
  /nix/store/hbsgsww2qiaa6b6gjffnvqc0w419wc4h-gnu-config-2024-01-01
  /nix/store/c4301v1sjrp1779626h44pl91jbnpn5r-gnumake-4.4.1
  /nix/store/jis2zvqwsjr7c9i4n48avrqfjcqki9hr-patch-2.7.6
  /nix/store/gwkch2vkywxcdxy4wjvim78ffcrh7zfy-sdkroot-11.0
  /nix/store/89ghmy7qxyggz4sycc9qr1klinmrlgbx-stdenv-darwin
  /nix/store/sjjp9j2nfl66rl19d5mbh0xajfanlall-update-autotools-gnu-config-scripts-hook
  /nix/store/2ix8n2873iib59yp7is3grmrmlx8sv2p-xz-5.6.2-bin
copying path '/nix/store/18fccp44v47b1pr7g7yjmbsf7wld1hbv-adv_cmds-119-locale' from 'https://cache.nixos.org'...
copying path '/nix/store/c4301v1sjrp1779626h44pl91jbnpn5r-gnumake-4.4.1' from 'https://cache.nixos.org'...
copying path '/nix/store/gwkch2vkywxcdxy4wjvim78ffcrh7zfy-sdkroot-11.0' from 'https://cache.nixos.org'...
copying path '/nix/store/04w1mgkhawycg64s4q3n8mkcvma5ppx0-apple-framework-CoreFoundation-11.0.0' from 'https://cache.nixos.org'...
copying path '/nix/store/gw9ps4037al7ccccb7yq8hfapmp6xq8z-bzip2-1.0.8-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/cnb7qaqj61vldzk88hcc8fhwad4ifn0x-file-5.45' from 'https://cache.nixos.org'...
copying path '/nix/store/2ix8n2873iib59yp7is3grmrmlx8sv2p-xz-5.6.2-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/s886gz4z355r8zw9hvp8x3vngh4m267p-ed-1.20.2' from 'https://cache.nixos.org'...
copying path '/nix/store/hbsgsww2qiaa6b6gjffnvqc0w419wc4h-gnu-config-2024-01-01' from 'https://cache.nixos.org'...
copying path '/nix/store/sjjp9j2nfl66rl19d5mbh0xajfanlall-update-autotools-gnu-config-scripts-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/jis2zvqwsjr7c9i4n48avrqfjcqki9hr-patch-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/89ghmy7qxyggz4sycc9qr1klinmrlgbx-stdenv-darwin' from 'https://cache.nixos.org'...
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.5.0, macOS 14.5`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.18.5`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

Issue seems to be solved for me with yesterday update (nixpkgs unstable).

Confirmed.
Odd though, since the derivation for this pkg hasn't been updated.


TimeZone problem

Hello,
I have installed the application and everything works correctly in France and I have employees who are located in Malaysia, US, Canada, Australia... and there I have a TimeZone problem.
The application is useless internationally, they don't have the same calendar because of the time difference so they can't even create leave requests.
The example:

A day in France is 9AM - 1AM => 2 AM-6 PM

A day in Malaysia is showed 4 PM => 8 PM (the equivalent to 9AM => 1AM in France)

which create a conflict in the balance
Thanks,

For now, the only solution is to change their timezone to France does anyone have any ideas
I wanted to generate a new calendar for working days but i can't find how.


Add toggle command

I added the following things to toggle the BufExplorer list on and off.

the command :ToggleBufExplorer
the map <Leader>bt, and
the function ToggleBufExplorer()

The function works by simply calling the BufExplorer() or s:Close() function, depending on the current state of BufExplorer. Now one can easily see which buffers are open (with a mapped function key, ideally), and then easily close the buffer list with the same key.
I believe this pull request will satisfy the request in Issue #3.

Reopening after restoring branch.

Thanks, Jeff. You've written a great plugin, and I'm glad I was able to contribute to it.


Problem with 'PX' in Elements

Let's say I want to create a custom tag like:
<custom></custom>

The I want to animate 2 properties:
<custom data-0="x:0" data-500="x:500"></custom>

The problem is that Skroller is forcing values in pixels and I want the as integer.
<custom x="145px"></custom>

I just don't want the 'px' there.

The problem is that Skroller is forcing values in pixels and I want the as integer.

it does not. If it would, then things like font-size and z-index couldn't be animated

Yes something like that. Does that  work? Does it mean I can animate any property without using pixels? If that is the case, then there is no bug :)
-- Suggestion to make this easier to find in the docs.


Search feature?

Is there an example on how to implement search feature using this plugin? Thanks!

There's no way to search using this package directly, you would need to combine this widget with a separate service (like the google maps / google places API) and use that.


type inferred on the default

We need an example where the type is inferred on the value of the default.
You don't have to be explicit about it in this case.
( count = 0 )
//would be equivalent to
( count = 0: Number )

Slightly off topic
@ericelliott If you have a function that always return the same value can you do something like
( param ) => String = 'example'
//or
( param ) => 'example': String

//shorthand
( param ) => 'example'

?

If you have a function that always return the same value

…or if you have a function which always expects the same value on a property. For example @Matt-Esch does that on virtual-dom to ensure backwards-compat: https://github.com/Matt-Esch/virtual-dom/blob/903d884a8e4f05f303ec6f2b920a3b5237cf8b92/vnode/vnode.js#L71-L72.
This would be a trickier one because = is already taken in param context.

The spec already has support for native type literals, including both parameters and return values.
(someThing: 'SpecificThing') => 'Always returns this string';

Destructuring parameter exceptions: To disambiguate from destructuring, only simple type literals are supported in parameters:
wrong(thing1: { a: 'a' }, thing2: ['b']) => Any

interface A:  { a: 'a' }
interface B: ['b']
better(thing1: A, thing2: B) => Any

I'm willing to look at a PR to clarify.

@ericelliott This issue is not about that.

Oh, I see. We need to explicitly document that we can infer types from default values. Yes. Feel free to PR. =)

@ericelliott  #32
Ill let you add the part about the return values with a constant value.


Markdown Preview compatible: Markdown

How can the issue be reproduced
markdown_preview="markdown" and autolink="true"

has 2 same headers in content.
example:
## 05/05/2020
some text
## 05/05/2020

What was expected
TOC should be:
- [05/05/2020](#05052020)
- [05/05/2020](#05052020_1)

What actually occurred
- [05/05/2020](#05052020)
- [05/05/2020](#05052020-1)

The difference is the _ and - character on the second link #05052020_1 and #05052020-1
What was the version of the involved component
markdownTOC: v4.0.1
MarkdownPreview: v2.2.5

@anlv
Thank you for reporting and sorry for late reply.
This problem may be fixed in 4.0.2.
Please check.


Build error with MSYS2

I can't build library after #59. Building failed with an error:
--- stdout
cargo:rustc-link-lib=static=lua
OPT_LEVEL = Some("0")
PROFILE = Some("debug")
TARGET = Some("x86_64-pc-windows-gnu")
debug=true opt-level=0
HOST = Some("x86_64-pc-windows-gnu")
TARGET = Some("x86_64-pc-windows-gnu")
TARGET = Some("x86_64-pc-windows-gnu")
HOST = Some("x86_64-pc-windows-gnu")
CC_x86_64-pc-windows-gnu = None
CC_x86_64_pc_windows_gnu = None
HOST_CC = None
CC = None
TARGET = Some("x86_64-pc-windows-gnu")
HOST = Some("x86_64-pc-windows-gnu")
CFLAGS_x86_64-pc-windows-gnu = None
CFLAGS_x86_64_pc_windows_gnu = None
HOST_CFLAGS = None
CFLAGS = None

--- stderr
thread 'main' panicked at 'Error: The system cannot find the file specified. (os error 2)', build.rs:122
stack backtrace:
   0:           0x491402 - <unknown>
   1:           0x48f19a - <unknown>
   2:           0x48fa6a - <unknown>
   3:           0x48f906 - <unknown>
   4:           0x48f819 - <unknown>
   5:           0x40f4fa - <unknown>
   6:           0x48f6d8 - <unknown>
   7:           0x499858 - <unknown>
   8:           0x48e773 - <unknown>
   9:           0x40f65a - <unknown>
  10:           0x4013b4 - <unknown>
  11:           0x4014e7 - <unknown>
  12:     0x7ffbb6318101 - <unknown>

I found that new build.rs uses CARGO_MANIFEST_DIR which returns C:\..... path, but for msys2 all paths starts with /c/....
My environment:
MINGW64_NT-10.0 DESKTOP-5U3PIP1 2.5.2(0.297/5/3) 2016-07-15 08:31 x86_64 Msys
rustc 1.13.0-nightly (1576de0ce 2016-08-21)

@SpaceManiac Maybe does CARGO_MANIFEST_DIR work wrong for MSYS2? Do you have any idea how can we fix it?

@SpaceManiac I've resolved it! Sorry me, but your PR is perfect. I have build errors because make wasn't installed! I was creating a new building environment and was confused by the lack of "unknown" file.


Document issues with SELECT * in views

I recently created a view that was meant to be the entirety of one table plus some aggregate data on each row. So something like this simplified, incomplete example:
SELECT
    posts.*,
    approved_comments.count AS approved_comments_count,
    deleted_comments.count AS deleted_comments_count,
FROM
    posts
LEFT JOIN (SELECT ... ) as approved_comments ON ...
LEFT JOIN (SELECT .. ) as deleted_comments ON ...

Then I created a model like so:
class PostWithStatistics < Post
end

This allows my PostWithStatistics to act as a Post model and lets me opt in to when I want the extra associated data. Everything works reasonably well until...
class AddFooToPosts < ActiveRecord::Migration
  def change
    add_column :posts, :foo, :string
  end
end

The posts table now has a new column and I'd expect my posts_with_statistics view to reflect that because I used SELECT *. Unfortunately, it doesn't work this way because when the view is created with SELECT *, Postgres bakes into the schema the actual column names represented by *. To pick up the new column I'd have to use update_view and pass the same version number that is already deployed.
For a similar reason, this migration would also be a problem
class RemoveBarFromPosts < ActiveRecord::Migration
  def change
    remove_column :posts, :bar, :string
  end
end

This migration will fail because Postgres knows that there's a view that explicitly lists the column bar as part of its schema. This dependency will prevent the column from being removed. The workaround is:
class RemoveBarFromPosts < ActiveRecord::Migration
  def up
    drop_view :posts_with_statistics
    remove_column :posts, :bar, :string
    create_view :posts_with_statistics
  end
end

I don't think there's anything we can do from the scenic perspective to handle this. I think we likely just need to document this (and potentially link to this issue for a full description).
Anyone have any other ideas?

I could swear we've seen this in an issue and documented it before. I know it but me. Best way I've found to "fix" it is drop and add the same view to pick up new columns.


[Question] Build Streamx - Sign in Failed

General Question

I use independent packaging mode，The front-end and back-end deployment are completed and started normally

But, When I log in to the home page, I am prompted with failure, and The front-end error  like this (failed)net::ERR_CONNECTION_REFUSED

Additional questions
I used 1.2.2 by Mixed package， home page can Login Success，Only Independent packaging Login Failed

see: https://www.streamxhub.com/docs/user-guide/deployment#2-前端打包

Closed as answered.


PHANTOMPEAKQUALTOOLS throws stack overflow exception

Description of the bug
PHANTOMPEAKQUALTOOLS throws a stack overflow exception on certain types of input data. This issue suggests a fix by increasing the default --max-ppsize. This is already implemented in MULTIQC_CUSTOM_PHANTOMPEAKQUALTOOLS and should be added to PHANTOMPEAKQUALTOOLS as well.
Command used and terminal output
nextflow run main.nf -profile janelia --input samples.csv --read_length 50 --outdir ./results --fasta internal.fasta --gff internal.gff
ERROR ~ Error executing process > 'NFCORE_CHIPSEQ:CHIPSEQ:PHANTOMPEAKQUALTOOLS (neg_rep1)'

Caused by:
  Process `NFCORE_CHIPSEQ:CHIPSEQ:PHANTOMPEAKQUALTOOLS (neg_rep1)` terminated with an error exit status (1)

Command executed:

  RUN_SPP=`which run_spp.R`
  Rscript  -e "library(caTools); source(\"$RUN_SPP\")" -c="neg_rep1.mLb.clN.sorted.bam" -savp="neg_rep1.spp.pdf" -savd="neg_rep1.spp.Rdata" -out="neg_rep1.spp.out" -p=6

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_CHIPSEQ:CHIPSEQ:PHANTOMPEAKQUALTOOLS":
      phantompeakqualtools: 1.2.2
  END_VERSIONS

Command exit status:
  1

Command output:
  ################
  ChIP data: neg_rep1.mLb.clN.sorted.bam
  Control data: NA
  strandshift(min): -500
  strandshift(step): 5
  strandshift(max) 1500
  user-defined peak shift NA
  exclusion(min): 10
  exclusion(max): NaN
  num parallel nodes: 6
  FDR threshold: 0.01
  NumPeaks Threshold: NA
  Output Directory: .
  narrowPeak output file name: NA
  regionPeak output file name: NA
  Rdata filename: neg_rep1.spp.Rdata
  plot pdf filename: neg_rep1.spp.pdf
  result filename: neg_rep1.spp.out
  Overwrite files?: FALSE

  Reading ChIP tagAlign/BAM file neg_rep1.mLb.clN.sorted.bam
  opened /tmp/RtmpEulNd9/neg_rep1.mLb.clN.sorted.tagAlign2961e49a6f
  done. read 326050 fragments

Command error:
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  WARNING: Skipping mount /var/apptainer/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  Loading required package: Rcpp
  Error: protect(): protection stack overflow
  Execution halted

Relevant files
No response
System information
No response

Fixed by #371.


[Form] Skip autofilling select input if it was changed already

Reviewer:  @shakyShane
Asana: https://app.asana.com/0/1200930669568058/1204544631545889/f
Description
select elements were simply autofilled every time, without considering if the input was already changed by a user action. This PR extends the behavior and adds logic to skip select type input, if the input was added to the Set touched.
Additionally, select type elements were not added to this.touched, as the handler that attaches it was intentionally skipped in the code. The new approach creates a separate handler called handleSelect in Form.js to simply add the select element to the touched Set.
Steps to test
(Pre-requisite: Have an identity added to autofill)

Locally build your mac+autofill environment,
Go to https://fill.dev/form/identity-simple in the locally built browser,
Change a select field (country/state),
Autofill one of the text field inputs (name, email e.g),
The select field should not get autofilled,

Credit card
(Pre-requisite: have credit card added to your autofill)

Go to https://fill.dev/form/credit-card-simple
Change the select field (expiry date),
Autofill on card number,
Expiry date should get autofilled.

Example videos
Amazon
https://github.com/duckduckgo/duckduckgo-autofill/assets/3018923/4971cc27-0c35-4cbe-8700-c75ee064803d
Amazon credit card
https://github.com/duckduckgo/duckduckgo-autofill/assets/3018923/7456b367-290b-4b7c-9798-5d99972c605d
Zip code search
https://github.com/duckduckgo/duckduckgo-autofill/assets/3018923/921aa9dd-02fe-4c8e-9f44-c0eda52e390d
fill.dev
https://github.com/duckduckgo/duckduckgo-autofill/assets/3018923/3c0d2037-a4bf-4a6b-b545-fc51e17cd1a3

Thank you for helping me step up so quickly with the feedback!


Add native popover component

Details
On tablets we were using popovers like on a desktop due to screen size and that doesn't work very well in that environment. This ensures that we always show popovers as bottom docked regardless of native platform.
Fixed Issues
https://github.com/Expensify/Expensify.cash/pull/1991#issuecomment-821735996
Tests/QA

Log in and select a conversation
Click the emoji button
If you are on a native app or mobile web the picker should be bottom docked. Otherwise it should be a popup

Tested On

[x] Web
[x] Mobile Web
[ ] Desktop
[x] iOS
[ ] Android

Screenshots

@shawnborton any idea why the styling breaks on ipad? Looks like we do a static 12.5% so i'm not sure why there would be extra space on the right

Hmm I think we would need to do some kind of text-align: center in each column - the space on the right is actually just white space from the width of each column, where the emoji is on the left edge.

Interesting! Why an empty file src/components/Popover/index.native.js?

Because I'm dumb and forgot to push the actual content :joy:

Updated to fix style

🚀 Deployed to staging 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

popovers like on a desktop due to screen size and that doesn't work very well in that environment

Are we planning to fix that and revisit the popover for use on tablets? And if so, is there some context we can leave here about why it's not working in that environment and what specifically is not working?

IDK is having a desktop-like popup a normal pattern on tablet apps?
The specific problems we were having were:

The picker is giant for some reason
The picker is based on the click/tap location of the trigger button. If the keyboard is open and you click the emoji button the keyboard closes, but the popover is still way up at the top of the screen because that's where the button was when you pressed it even though now it's at the bottom.

is having a desktop-like popup a normal pattern on tablet apps

Thanks! Yeah, I can't ever remember what the status is on this, but I thought we wanted to have all "wide screens" have more or less the same appearance across platforms. With talk about iPad Pros replacing macs lately it seems like it's becoming more "normal".
Not that we need to do anything about it now. Just thought it would be good to leave some breadcrumbs.

The picker is giant for some reason

That seems kind of unexpected.

The picker is based on the click/tap location of the trigger button. If the keyboard is open and you click the emoji button the keyboard closes, but the popover is still way up at the top of the screen because that's where the button was when you pressed it even though now it's at the bottom.

This also seems like a problem with the Popover. I noticed some other issues related to the popover not re-positioning itself so maybe that problem would be resolved after they are addressed.


Personal Ship changed nomenclature after 9/13/23

pre patch, i had hijacked a crimson fleet phantom to see of the ship type causes UC attacks during regular gameplay.
today (9/13) when i attempted to fast travel back to my lading point, my ship was renamed and reskinned to an "ecliptic combat A01"
dunno if this is a bug or an unintended consequence of the last patch

Game Version
<IP_ADDRESS>
New Game
No
Reproduction Steps
n/a

is possible that the "dead body removal process" via the shipbuilding modular swap, also altered the ownership, because i haven't (yet) become a member of the Crimson fleet, so it defaulted to a basic modular part?


Want to add Dino Run game

want to add Dino Run game

Add the github repo link as well.

Yes I will

Add the github repo link, here in the issue as well.

https://github.com/ufrshubham/dino_run


[5.1] Added verbose method on command class

Allows you to call
$this->verbose('some message');

inside commands.

Fixed typo, removed alignment

heh, first timer here. i'll read the docs for next time. sorry about that

@timfeid you should squash down all those commits.

squashed

I don't really feel like the method name communicates what is happening here. It doesn't communicate that the method is dependent on the verbosity level that is passed in.

Output already has isQuiet(), isVerbose(), isVeryVerbose() and isDebug() methods which represent verbosity levels.

Symfony supports $options argument (which is ignored in Laravel Command) which allows to pass verbosity levels as a bitmask (to use together with output styles). But it is good if using as a parameter. And it already has a comparing condition inside.

$output->writeln('something', OutputInterface::VERBOSITY_DEBUG);
$output->writeln('something', OutputInterface::VERBOSITY_VERBOSE);

So we should better just add $options argument to comment, line etc.
The only thing that may be simplified here is using aliases
'v' => VERBOSITY_VERBOSE,
'vv' => VERBOSITY_VERY_VERBOSE,
'vvv' => VERBOSITY_DEBUG,
'quiet' => VERBOSITY_QUIET,

So it will be used as
$this->comment('oops!', 'vvv');
$this->error('oops!', 'vv');
$this->line('oops!', OutputInterface::VERBOSITY_VERBOSE); // both ways should work

I'm all for an "options" argument. I just took what I am using now, but I agree, it does not really communicate what is going on clearly.

@timfeid Are you going to implement this yourself?

Closing for now since it sounds like we might take another approach.


Unable to Login; Service May Be Down Since Update?

Nratings runs normally up until the point it's time to "import real life data," when the user is flooded with error messages and is then led to a login page that does not function properly as it doesn't log the user in, but says it did. Never had this issue before with the software. 1/2/2022

UPDATE: I've got the software functioning properly now by deleting the usersettings file in documents as suggested in another thread but the error messages still continue. I'd prefer to get rid of them but at least the program runs properly now.

Are you still getting these other error message you speak of ? If so, could you tell me exactly what they say or post a screenshot. Or if this is solved, please let me know too so I can close this issue. Thanks.

Hi Mathieu hopefully this message gets to you. Sorry it took me so long to get back to you.
Thanks for reaching out, but no I am no longer receiving any error messages as of last night. The program runs perfectly normal now so whatever fix you guys did seemed to do the trick. You could probably close the ticket if you haven't already. I'll be making another donation soon due to your kindness. Have a great day!

From: Mathieu @.>
Sent: Tuesday, January 18, 2022 4:00 PM
To: 64Soft/NRatings.Client @.>
Cc: KBMfan @.>; Author @.>
Subject: Re: [64Soft/NRatings.Client] Unable to Login; Service May Be Down Since Update? (Issue #29)
Are you still getting these other error message you speak of ? If so, could you tell me exactly what they say or post a screenshot. Or if this is solved, please let me know too so I can close this issue. Thanks.
—
Reply to this email directly, view it on GitHubhttps://github.com/64Soft/NRatings.Client/issues/29#issuecomment-1015872514, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AXEBFVZEVPDPMRHAAGZZVTLUWXPGHANCNFSM5LEBCENQ.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID: @.***>


implement detection model in pipeline using hydra

implement detection model in overall pipeline by using standard hydra config format that we've been using fro SemiF-AnnotationPipeline and Field-DataExploration

create the 4 main outputs as discussed:

cropout
cutout
cutout mask
cutout metadata (this can be simplified)


[Enhancement] Please provide a way of purchasing Brefoil for devices without gapps

I don't know if this is actually implemented already. It would be extremely useful for people with phones which do not have Google Play Services, or for those who choose not to use them for their own reasons.
One way to do this would be to include offline codes in updates to the app, which you would then provide to users if they contacted you, and then users could pay using Paypal, Bitcoin, you name it... I'm not sure how that would work but I got the Idea from the peeps at Titanium Backup who do something similar. They're really nice, I'm sure they would be happy to help if you asked.
Or another way would be to release a separate "premium" version of the app, and make it available for purchase on XDA labs. Personally, I like this idea better, as it would mean I could buy it sooner (presumably it wouldn't take as long to release)
This would also benefit the development, since Google wouldn't be able to take a cut :)
Whatever way you choose, I hope you'll consider this soon :)

good

Brevent only support:

Google Play
Wechat, a China payment


Rename Render* to Gpu*

Objective
Improve the consistency of the render module. See #7157 for more context
Solution

This PR only focuses on renaming the type and variable names.

renamed type names Render{Device, Queue, Adapter, AdapterInfo, Instance, Settings} to GPU{Device, Queue, Adapter, AdapterInfo, Instance, Settings}

renamed variable names render_{device, queue, adapter, adapter_info, instance, context} to gpu_{device, queue, adapter, adapter_info, instance, context}

maybe slightly controversial: I have additionally renamed a few occurrences of the variable name device and queue, which refer to a GPUDevice or GPUQueue, to gpu_device and gpu_queue for consistency's sake.

many small changes and renames for consistency's sake

Before we merge this, we should have a comprehensive migration guide ready :) I understand wanting to hold off until the bikeshedding is done though.

On that list, we should probably just close the rename Render to Gpu PR at this point. I was in favor of it, but it's been too long and too many things changed so whether we decided to do it or not it will need to a new PR

@IceSentry on Discord.


Broken Link in README

Hello, the following link on this page is not working:
Try clicking on documentation.
{{cookiecutter.description}}. Check out the project's documentation.
Broken Link: http://%7B%7Bcookiecutter.github_username%7D%7D.github.io/{{cookiecutter.github_repository_name}}/
Thanks for the repo, btw. Keep up the awesome work.

Hey @ademidun thanks for opening up an issue! That link is actually correct; it gets templated by cookiecutter when you scaffold a project.
When scaffolding, cookiecutter will render your github username and repo  in the links address so it will point to your projects documentation hosted on github pages.  You can this out in the example repo here.


Database.lastInsertRowId should return int?, instead of int

Hi,
I noticed,
If NO INSERT op has happened on a newly created table, 0 is returned, which is correct.
But, 0 is also returned, when the insert op is the first insert of the session, may or may not be first row of the table, which is incorrect, and null should be returned, in this case, if possible, else please add docs.
thanks

We call sqlite3_last_insert_rowid under the hood, which always returns an integer and so we can't return null.

But, 0 is also returned, when the insert op is the first insert of the session, may or may not be first row of the table, which is incorrect

After the insert it should actually return the rowid of that insert, which is unlikely to be zero since these usually start at 1. Unless you're inserting to a WITHOUT ROWID or some virtual tables. I have clarified the documentation.


Fix: dont confetti unless on head

Confetti was firing on a changeset when deleting a component that was added in the changeset.

bors merge

bors merge


Add speculative decoding example to OpenVINO quantized generation notebook

What does this PR do?

Add support and example for running speculative decoding generation with OpenVINO.
This PR is depending on PR #620

Fixes # (issue)
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @echarlaix, updated the notebook per your suggestions. Thanks!


Safe wrapper for the video extension

Hello everyone, as a member of vulkan video working group under Khronos I would like contribute to this project by providing safe wrapper for the video extension. Spec for the encode is still in flux but I volunteer to ensure compatibility in future releases.
For I while Im have ash fork with added extensions:
https://github.com/neurotok/ash.git
And a basic encode example:
https://github.com/neurotok/ash-video.git
But the problem which I cannot overcome is a panic when calling even basic functions from the encode:
https://docs.rs/ash/latest/ash/vk/struct.KhrVideoQueueFn.html#
(using my ash-video example)
I had some success when I wrote this wrapper as an instance extension, but I'm running out of ideas on how the wrapper should be designed as device one.
@MarijnS95 , @MaikKlein ? Could you please take a look at my ash fork and provided example ?

What is the panic message?

Hi @Ralith Im getting "thread 'main' panicked at 'Unable to load get_physical_device_video_capabilities_khr'"
This is caused by calling get_physical_device_video_capabilities in ash-video\src\main.rs:224
This function is implemented here ash/src/extensions/khr/video_encode_queue.rs:25
To run the examples you'll need nvidia beta drivers with video support

@neurotok Do you want to use this as tracking issue or can we close it now that the initial problem has been resolved (nothing to do with Ash it seems, just general Vulkan usage)?
Also keep in mind that the Video bindings are currently generated from the headers because they landed in an era where video.xml with bitfield struct members did not yet exist. We might change that in the future, though most likely with the generator rewrite and not in the current setup.

Hi @MarijnS95, of course you were right,  I already update m fork but let's keep this as a issue tracker for now please. At least until I have a working sample project with decoding.
Do you want some help with the generator to pull video extension from xml?

We already generate the full video extension bindings, don't need help for that but it'll likely change with the generator rewrite. In the event that you do need to make changes, use this tracking issue to discuss so first please.

@pingpongcat another user seems to have beat you to submitting the Video bindings in #916, though in a less mergeable state than your fork would have been (a year ago when it was proposed; we've made significant breaking/simplification changes to hand-written extension wrappers now).
Would you be able to help reviewing and testing the proposed implementation, once all basic review is carried out?

Sure I'll look at it @MarijnS95 the main reason the further progress for this extension to be developed was the lack of crates that could handle video file demuxing and the extraction of the SPA/PPS parameters from the video bytestrem , year ago there was no crates that could handle this which meant I wasn't 100% sure that the bindings would work in real life. Vulkan video extension itself have no ability to do this as a part some API it expect these parameters to be provided by the developer . Hey @Cach30verfl0w did you have a chance to test these extension binding in "action" ? and if so how you solved the SPS/PPS parameters issue ?

Sure I'll look at it @MarijnS95 the main reason the further progress for this extension to be developed was the lack of crates that could handle video file demuxing and the extraction of the SPS/PPS parameters from the video bytestrem , year ago there was no crates that could handle this which meant I wasn't 100% sure that the bindings would work in real life. Vulkan video extension itself have no ability to do this as a part some API call and expecting these parameters to be provided by the developer . Hey @Cach30verfl0w did you have a chance to test these extension binding in "action" ? and if so how you solved the SPS/PPS parameters issue ?

I am currently working on a project where these extension bindings will definitely be used. But I haven't had a chance to use the bindings in action yet.

I haven't yet played with this either, but please let me know (and maybe we could list it in the readme somewhere) about good crates that fulfill this purpose!

I use the encode extensions in my project. I could upstream the wrappers I use, if that's what's meant by "safe wrappers"?


PV State

A PV may have a particular value (represented by a VType), but that doesn't include whether the PV is connected or whether it is readonly. These are additional information associated with each PV.
At the moment in the store we have
export interface PvState {
  value: VType;
  connected: boolean;
}

We could add readonly: boolean to this. Looking at the PV.java class in Phoebus, the only other state associated with the PV is the list of subscribers. We currently have this as a separate object in the store.
Another important point is that the vtype may be null (this happens for example when a local PV has not been initialised). That suggests that the pv state should be:
export interface PvState {
  value?: VType;
  connected: boolean;
  readonly: boolean;
}

with a value that may be undefined.

How does Phoebus tackle this problem then? Or are you saying this IS a state stored with the PV state information in Phoebus?

Phoebus has a PV object, which has:

a value (VType) that may be null
isDiconnected()
isReadonly()

Main work here would be adding it in connectionWrapper.tsx and giving a sensible name?
Are there any places where this would be expected to cause big problems?
Happy to take this on as I haven't looked at this part of the program too much.

I think this would be quite simple. I'm not sure what you mean by 'giving a sensible name' though.

Will this need to be optional? Can a PV which is not connected know whether it is readonly or not? Should this default to True if there is no connection or other information from the PV?

Same in CS-Studio.

Yea I'm happy with that. Are we envisioning that this information will eventually be sent from the EPICS interface (ie coniql) ?

Yes, it comes through coniql. It will be false for local PVs, but we could have sim pv for which it is true.

From: TimGuiteDiamond<EMAIL_ADDRESS>Sent: 10 October 2019 16:07
To: dls-controls/cs-web-proto<EMAIL_ADDRESS>Cc: Rogers, Will (DLSLtd,RAL,LSCI<EMAIL_ADDRESS>Assign<EMAIL_ADDRESS>Subject: Re: [dls-controls/cs-web-proto] PV State (#53)
Yea I'm happy with that. Are we envisioning that this information will eventually be sent from the EPICS interface (ie coniql) ?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://github.com/dls-controls/cs-web-proto/issues/53?email_source=notifications&email_token=AASTW523CRFB53VQRW7NZUTQN5AMJA5CNFSM4I7LSKKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA4WHXA#issuecomment-540632028, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASTW56QEXPDFTYW4MNV453QN5AMJANCNFSM4I7LSKKA.
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom


Firmware update USB device sample - link is broken

"Firmware update USB device sample" link doesn't lead to a valid sample app. Through some searching found the following archived sample app but I am not sure if it is still valid. I am looking for a working sample for DeviceServicingTrigger. Any help is appreciated.
https://github.com/microsoftarchive/msdn-code-gallery-microsoft/tree/master/Official Windows Platform Sample/Windows 8.1 Store app samples/[C%23]-Windows 8.1 Store app samples/Firmware Update USB Device sample/C%23.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e46710c3-f9c4-279f-7134-2ef0610ea028
Version Independent ID: 7210cc0b-ef33-0beb-4e0c-7a4490b82038
Content: Device sync and update for Store device apps in Windows 8.1 - Windows drivers
Content Source: windows-driver-docs-pr/devapps/device-sync-and-update-for-uwp-device-apps.md
Product: windows-hardware
Technology: windows-devices
GitHub Login: @EliotSeattle
Microsoft Alias: eliotgra

@tewarid . I conformed that the link is broken . i could not able to find an alternative link

Thanks, @tewarid for bringing this to my attention.
This older sample is, indeed, no longer in the catalogue. I am researching if it was removed purposefully or if this was an error.
Eliot

Hi @tewarid thanks for reporting this, and apologies for the delay. I checked around and was unable to find the sample in question, so I have removed those links.


Fix pyspark local execution

I've tried this with scale 5, 10, and 100 scale data w/o trouble after verifying the current state is broken.
My machine has 62G so the logic gives  ~23G to executors, and have tried capping that to 5G per executor and it still worked.
I somewhat arbitrarily assigned n_executors = 2, could very well be a single executor or something else. I'm not sure what might be preferred here by others.

I'm running on a M1 mac book and when I close most of my browser tabs and my IDE I have about 9.4GiB of available memory. When I initially executed this script, I essentially had almost no available memory and the total_executor_memory_g math produced negative values for me.
d)

Regardless of the settings. I tried a couple of different configs and this doesn't work for me. Even on scale1 I have a job that appears to be stuck

I get that spark may be slow but I don't believe that it can't handle 1GB locally.

Ya, I'm slightly baffled. I was concentrating on scale 100 to get it running for the comparison. But now am experiencing a similar scenario on scales 1, 5, 10. Queries will fail, while on scale 100 all work fine. :face_exhaling:

2024-04-03 10:12:43,053 - distributed.nanny.memory - WARNING - Worker tcp://<IP_ADDRESS>:35463 (pid=148634) exceeded 95% memory budget. Restarting...

This occurs on the lower scales for me, but again, not on scale 100.


add behave test for list, query and update

also fixed some typo

Codecov Report

Merging #83 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #83   +/-   ##
=======================================
  Coverage   93.91%   93.91%
=======================================
  Files           4        4
  Lines         230      230
=======================================
  Hits          216      216
  Misses         14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d20d3f6...861234f. Read the comment docs.


makeWrapper: document invocation flags

Motivation for this change
Things done

[x] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

I always forget how to use makeWrapper. Instead of scanning other
packages for the usage or reading the code, document the usage directly
in the header of makeWrapper.

Please add this information to the Nixpkgs manual instead. Otherwise, every change to the documentation produces a mass-rebuild.

I have a semi-irrational dislike of touching the XML doc so I might just close the PR. I don't think the script's doc would change that much unless you find a typo or a new feature is added.

@zimbatm Maybe you'll write it in Markdown? We have pages written in Markdown already, for example Haskell and Python language framework manuals -- see doc/languages-frameworks/*.md. See doc/default.nix for examples on how to hook it into the documentation.


Release doesnt not show in deployment section when we take existing build and link it to development

Document URL: https://docs.microsoft.com/en-us/azure/devops/boards/work-items/work-item-deployments-control?view=azure-devops
I have integration turned on in my releases. However i linked a PR and build to a different work item than the one which i intended to. So my deployment section has releases for the wrong work item. i went ahead and added the links for PR and build to my WI which i want the development to show. But it still doesn't reflect the change?

@santhoshrajr  Thank you for the feedback.
Unfortunately, issues here are for documentation concerns and this is more of a product issue.
Please get in touch with the product team using Report a problem in the Developer Community and link back to this issue.


Prerequisites for getting-started are not clearly defined

At File: get-started/02_our_app.md it was not clear to me what yarn is.  Evidently I can install it via npm that comes with my node installation.  People that aren't familiar with javascript will not know what yarn is or how to install it, and I see nothing on the page that clearly states the pre-requisite software apps that must be installed.

I can't reproduce it on my home computer now, even after uninstalling yarn which is odd.  I might have made a bad assumption.  I saw this line in the docker file which appears to require yarn ( RUN yarn install --production ).  What does that line do?  Is that command simply causing yarn to be installed within the container itself?  On my work computer I first had to fix https certificate issues with NPS to get farther.  I will need to try this again on my work computer on Monday to get you the error message.  Perhaps it is something unrelated to yarn and another issue with something blocked by my corporate firewall.  If I can't reproduce it by Monday evening I'll recommend we close this as unreproducible.

Here is the error, but I agree now that it isn't yarn related.
Microsoft Windows [Version 10.0.19043.1766]
(c) Microsoft Corporation. All rights reserved.
V:\TrustedData\Personal-Projects\Docker-TODO-List\app>docker build -t getting-started .
Sending build context to Docker daemon  4.654MB
Step 1/6 : FROM node:12-alpine
---> bb6d28039b8c
Step 2/6 : RUN apk add --no-cache python2 g++ make
---> Running in 0125c7d7c285
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
139750553811784:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.15/main: Permission denied
139750553811784:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.15/community: Permission denied
ERROR: unable to select packages:
g++ (no such package):
required by: world[g++]
make (no such package):
required by: world[make]
python2 (no such package):
required by: world[python2]
The command '/bin/sh -c apk add --no-cache python2 g++ make' returned a non-zero code: 3
V:\TrustedData\Personal-Projects\Docker-TODO-List\app>


Stencil 1.16.2 - Job Output failed to validate.

[Nx Workspace]

@nrwl/**: 9.5.1
typescript: 3.9.6

[Screenshot]

Hi, the error is known. The plugin don't work with 1.16 up, see #35
The stencil compiler api changed and I talk already to Adam what need to be re-added

There's Version 1.1.0 which uses 1.16.3


tests should not fail based on actor timeouts

tests should not fail based on actor timeouts
Summary:
tests should not fail based on actor timeouts
JIRA issues:

@kensipe looking at the failing test in this PR, I think we need to backport my test fixes to this branch... I"ll try to do that during this day.
Thanks for the patience config fix!