feat: Refactor Dockerfile

Summary
TL;DR:

4GB disk usage down to 1.7GB (some extra weight externalized via cache mounts).
68MB binary now builds to smaller 24MB size.
Layer invalidation concerns reduced, especially with use of cache mounts (less meaningful to CI, but good for local use).
Additional changes detailed below.

The first commit introduces the larger diff as adopting HereDoc feature introduces indents and merging of some RUN layers.

I've also relocated and grouped some directives where appropriate, minimizing layer invalidation.
HereDoc usage brings the added benefit of better formatting for multi-line RUN content.
There doesn't seem to be a need for the early --target until explicitly adding the target triple.

2nd commit (will follow shortly) addresses excess disk weight for the builder image layers that's less obvious from the multi-stage build.

When ignoring the final stage, the total disk usage for the builder image is about 4GB.

Expand to see breakdown (4.11GB image)

80MB => Ubuntu base image (~116MB after apt update)
+450MB => apt update + system deps
+13MB => pip /root/.venv creation
+390MB => pip install cargo-zigbuild

306MB /root/.venv/lib/python3.12/site-packages/ziglang
79MB /root/.cache/pip/http-v2 (can be skipped with pip install --no-cache-dir)
2.7MB /root/.venv/bin/cargo-zigbuild

+1.3GB => rustup + toolchain + target setup

15MB rustup install
1.3GB rust toolchain target add, (Over 600MB is from toolchain installing docs at ~/.rustup/toolchains/1.78-x86_64-unknown-linux-gnu/share/doc/rust/html, the minimal rustup profile doesn't apply due to rust-toolchain.toml)

+1.8GB => Build cache for uv:

270MB /root/.cache/zig from cargo-zigbuild
300MB /root/.cargo/registry/ (51MB cache/, 21MB index/, 225MB src)
1.2GB target/ dir

360MB target/release/{build,deps} (180MB each)
835MB target/<triple>/release/ (168MB build/, 667MB deps/)

68MB uv (copied from target build dir)

The RUN --mount usage allows for caching useful data locally that would otherwise be invalidated across builds when you'd rather have it available. It also minimizes the impact of unused layers from past builds piling up until a prune is run. This is mostly a benefit for local usage rather than CI runners.

With the fix for the rustup profile and proper handling of cacheable content, the image size falls below 1.6GB and iterative builds of the image are much faster.

The 3rd / 4th commits simplify the cross-platform build. There isn't a need for the conditional block to install a target.

Since the image is to be built from the same build host (FROM --platform=${BUILDPLATFORM}), you can just install both targets in advance? Then use TARGETPLATFORM ARG to implicitly get the preferred platform  binary build.
Now the builder stage shares all the same common layers, instead of diverging at the TARGETPLATFORM ARG. This is viable due to cargo-zigbuild supporting cross-platform builds easily.

The 5th commit tackles the inline comment about optimizing the build size of the binary, reducing it from the current 68MB to 24MB, better than the current uv binary distributed via other install methods. You can use UPX to reduce this furth from 23.6MB to 6.7MB (<10% current published size for the image), at the expense of 300ms startup latency and potentially false positive to AV scanners.

Test Plan
I've built the image locally, you still get the binary output successfully. Let me know if you'd like me to test something specific?
# Using Docker v25 on Windows 11 via WSL2, with docker-container driver builder:
$ docker buildx create --name=container --driver=docker-container --use --bootstrap

# Excess platforms omitted:
$ docker buildx ls
NAME/NODE     DRIVER/ENDPOINT             STATUS  BUILDKIT PLATFORMS
container *   docker-container
  container0  unix:///var/run/docker.sock running v0.13.2  linux/amd64, linux/arm64

# Building with both platforms and outputting the image contents from both platforms:
$ docker buildx build --builder=container --platform=linux/arm64,linux/amd64 --tag local/uv --output type=local,dest=/tmp/uv .
# ... Build output ...
 => exporting to client directory
 => => copying files linux/arm64 20.98MB
 => => copying files linux/amd64 23.67MB

# Exported contents:
$ ls /tmp/uv/*
/tmp/uv/linux_amd64:
io  uv
/tmp/uv/linux_arm64:
io  uv

# AMD64:
$ du -h /tmp/uv/linux_amd64/uv
23M     /tmp/uv/linux_amd64/uv
$ file /tmp/uv/linux_amd64/uv
/tmp/uv/linux_amd64/uv: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped

# ARM64:
$ du -h /tmp/uv/linux_arm64/uv
20M     /tmp/uv/linux_arm64/uv
$ file /tmp/uv/linux_arm64/uv
/tmp/uv/linux_arm64/uv: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped

# Equivalent output to running `docker run --rm -it local/uv`
$ /tmp/uv/linux_arm64/uv
Usage: uv [OPTIONS] <COMMAND>

Commands:
  pip      Resolve and install Python packages
  venv     Create a virtual environment
  cache    Manage the cache
  version  Display uv's version
  help     Print this message or the help of the given subcommand(s)

Options:
  -q, --quiet                  Do not print any output
  -v, --verbose...             Use verbose output
      --color <COLOR_CHOICE>   Control colors in output [default: auto] [possible values: auto, always, never]
      --native-tls             Whether to load TLS certificates from the platform's native certificate store [env: UV_NATIVE_TLS=]
  -n, --no-cache               Avoid reading from or writing to the cache [env: UV_NO_CACHE=]
      --cache-dir <CACHE_DIR>  Path to the cache directory [env: UV_CACHE_DIR=]
  -h, --help                   Print help (see more with '--help')
  -V, --version                Print version

Tagging @konstin for this one.

Hi, and thanks for looking into our dockerfile! I have two main concerns with the changes:
Docker is a secondary target for us (most users are using the curl, powershell or pip installer), so we'd like to keep it simple to maintain. The main audience are projects with a fully dockerized CI pipeline who want to run ruff or uv as a docker step; to people creating docker images we recommend the pip or curl installers inside the dockerfile. We don't intend for users to build the uv container themselves, so we haven't optimized for builder size. There is one big exception though: Building the image for the release job is the bottleneck in our release process, if that could be improved that would be amazing!
Performance is crucial for us, and we optimize performance over binary size.

TL;DR: I'm happy to adjust the Dockerfile if you'd still like to keep the build process in there.
I can look into optimizing the workflow to bring down that bottleneck time with compilation, but I suspect the pragmatic approach might just be to take an existing uv build from CI and publish an image with that?

so we'd like to keep it simple to maintain.

Sure! I can make an adjustment that prioritizes that over the other optimizations 👍

There is one big exception though: Building the image for the release job is the bottleneck in our release process, if that could be improved that would be amazing!

Could you please elaborate a bit more here? Is it with Github Actions and the time it takes to build the image?
I looked at the recent workflow runs and see the bulk of the time is spent compiling amd64 + arm64 stages concurrently, taking about 20+ minutes to complete.
As long as future runs wouldn't trigger compiling everything all over again if you had the target/ dir for each platform cached, that'd speed up that step quite a bit.
However, you're already building uv with these binaries outside of Docker in CI? Why not just use FROM scratch + COPY like the final stage of the current Dockerfile, skipping the bottleneck concern?
I prefer having a Dockerfile as an easy way to build a project and not worry about dependencies or other setup, but since you don't seem to have an audience for that and the build step is purely for your CI... might as well skip all this since you want to prioritize keeping maintenance simple? (unless the build stage is useful to maintainers or users outside of CI?)

However, you're already building uv with these binaries outside of Docker in CI? Why not just use FROM scratch + COPY like the final stage of the current Dockerfile, skipping the bottleneck concern?

We would like to retain the isolation of the Dockerfile for building the image. The build stage isn't used by us otherwise, we just want to publish the final image, everything else goes through cargo build --release or maturin build --release. Anything that makes the build faster (incl. caching) would be great. Note that we update dependencies often, so Cargo.lock doesn't work as a cache key unfortunately

I've tried to summarize the considerations in the TLDR, let me know which direction you'd like to take.
TL;DR:

The build stage in the Dockerfile could be kept for potential use elsewhere, but an alternative stage that is easy to maintain could be used purely by the release CI to provide existing release binaries already built, completely removing the bottleneck.
Otherwise, to optimize the cache you need to only cache what is relevant but presently you cache image layers that aren't helping that much and cannot cache the data of interest due to layer invalidation. You will need the cache mount syntax or similar approach to restore target/ from cache.
sscache is probably needed for the above cache optimization until cargo moves away from mtime based invalidation (which makes target/ cache likely ignored).
Splitting the image builds by target to separate runners and caching the final output is an alternative approach. It'll shift the maintenance complexity to the GH workflow where it needs to merge the two final images into a single multi-platform image manifest (which is currently done implicitly).

We would like to retain the isolation of the Dockerfile for building the image.
Anything that makes the build faster (incl. caching) would be great.

Just to confirm that isolation isn't serving any real purpose though? You could use the CI binaries you already built for AMD64/ARM64 if you wanted to right?
If you just want the build stage to remain in the Dockerfile as a useful isolated build environment for reproductions / documentation or something like that, that's fine 👍

You could still use that stage for testing PRs or anything else in CI if you find a use for it. However for the release workflow, where it's not really needed, you could have an alternative stage that does take an external binary as input.
As you stated you don't really have any actual need or value with the build stage for the release as you're only publishing an image with the binary, you might as well use the same one that you distribute elsewhere?

At least that is just the more pragmatic / simple approach to take and doesn't complicate maintenance. To optimize the build stage for CI will require cache optimizations such as with the cache mounts I showed earlier, otherwise the layer that creates target/ during build is invalidated and this data is discarded... which has no benefit to cache import/export like you presently do.

Anything that makes the build faster (incl. caching) would be great. Note that we update dependencies often, so Cargo.lock doesn't work as a cache key unfortunately

Cargo.lock doesn't matter much here, it's the target/ dir you want to cache, not the crates source. You can optionally create a cache by any content/input you'd like to, and when that has no cache hit GH will use the latest cache item instead IIRC.
You're looking at 1GB roughly after compilation. That can be compressed fairly quickly with zstd down to 300MB, there isn't much value in higher compression levels (the tradeoff is barely worth it for 50MB-ish less). So 600MB for both targets to build with. There is little value in caching anything else in the image itself, the toolchain and crates all install fairly fast that you're just wasting the 10GB cache storage available by doing that. I would only cache what is useful and would assist with the speedup.
I haven't tried sscache, but this might be worthwhile to look into. A key problem with builds in CI is that git clone doesn't preserve mtime attribute on files, yet cargo relies on that for it's own "cache key", thus even with the target/ restored, since the source files have different mtime I think it'll still perform quite a bit of unnecessary compilation. By leveraging sscache instead, I think we can avoid the overhead that introduces.
Finally one other trick you can use is to split it across two runners, one for each target. In your case this works quite well due to the minimal image you actually care about. If the cache upload is only your final scratch image with just the binary, then you can quite easily leverage cache to retrieve both platforms (using the release tag as the cache key for example) and have these two separate images combined into a multi-platform manifest (which is what's already happening implicitly). The added complexity here splits the bottleneck of 20+ min build time in half, so that it more closely aligns with the other jobs running. You may not need this optimization though if that sscache approach works well enough.

Splitting the image builds by target to separate runners and caching the final output is an alternative approach. It'll shift the maintenance complexity to the GH workflow where it needs to merge the two final images into a single multi-platform image manifest (which is currently done implicitly).

Splitting into two separate jobs would be a good idea.
I don't have experience with caching docker builds beyond layer caching, if you know a way i'm happy to use that.

You could still use that stage for testing PRs or anything else in CI if you find a use for it. However for the release workflow, where it's not really needed, you could have an alternative stage that does take an external binary as input.

We only build the docker container for release, we don't use those images ourselves otherwise.

You're looking at 1GB roughly after compilation. That can be compressed fairly quickly with zstd down to 300MB, there isn't much value in higher compression levels (the tradeoff is barely worth it for 50MB-ish less). So 600MB for both targets to build with. There is little value in caching anything else in the image itself, the toolchain and crates all install fairly fast that you're just wasting the 10GB cache storage available by doing that. I would only cache what is useful and would assist with the speedup.

The github docs [says]:

GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 10 GB. Once a repository has reached its maximum cache storage, the cache eviction policy will create space by deleting the oldest caches in the repository.

If we get into a release cadence of once a week the cache will already be evicted by then, i'm afraid this wouldn't work for us.

TL;DR:

You can build and cache on a schedule (daily) or when main branch is pushed to. Since it's only intended for releases, daily builds might work well? (ideally minimizing cache storage use to not impact other workflows usage)
Splitting the build to separate runners should be simple enough I think.
Cache mounts to speed up the build may be viable, I'll look into trying it with a test repo this weekend.
You could alternatively consider depot.dev as they seem to have an open-source tier which may like other services equate to free use. They're apparently quite simple to switch to and claim to speed up image builds considerably (especially ARM64 due to native build nodes).

If we get into a release cadence of once a week the cache will already be evicted by then, i'm afraid this wouldn't work for us.

Technically you could probably have a scheduled build job that just builds and caches the image, or triggers when the main branch is updated (something we have at docker-mailserver, except we publish a preview/beta image). You don't have to publish that way, but it should keep the cache available for an actual release.
That said, I don't know if there's any nice ways to partition/manage the CI cache... and wow, I just looked at your current cache use:

Your largest cache items are about 1GB of which multiple were uploaded within the same hours. So not quite sure what's happening there, perhaps a bit of a bug on GH side. You have one item (2MB) from almost a week ago, while everything else is within the past 2 days, so it's not like it's evicting the old items regularly when exceeding the 10GB capacity limit 🤔

Splitting into two separate jobs would be a good idea.

The way this is handled for docker-mailserver I explained here with some rough examples demonstrated. We have it modularized into separate build and publish workflows, which a much smaller and simpler workflow(s) can then call (GH Actions calls these "re-usable" workflows).
That way you can separately build and cache the two images on a schedule or when commits are pushed to main branch like described above, with the separate publishing workflow triggered only on tagging new releases?
You wanted to keep the Dockerfile simple to maintain, so I'm not sure how you feel about this kind of change for the CI?

I don't have experience with caching docker builds beyond layer caching, if you know a way i'm happy to use that.

I don't have any workflow examples to cite for this as I haven't done it yet myself. There is a GHA cache exporter for BuildKit that can be used.. oh you're already using it here:
https://github.com/astral-sh/uv/blob/0a055b79420fdf1342c278e97d0c78cc487df89b/.github/workflows/build-docker.yml#L64-L65
It was experimental when I implemented the workflows for docker-mailserver. There's a separate action that allows leveraging that cache method when building a Dockerfile that uses cache mounts. Seems like there was a related effort for this approach 4 months ago. This is the action I had in mind for using to support cache mounts.

Alternatively, you could apply to depot.dev for an open-source plan to be granted for UV? They claim to be considerably faster at builds, automate the cache for you and offer native ARM64 hosts instead of slower QEMU emulation we presently are limited with on GH.
So from a maintenance point of view, depot.dev may be a simpler option available to you (assuming like other services like Netlify and DockerHub that offer similar plans that haven't cost docker-mailserver anything to adopt):

The cache mounts seem nice to have but yeah per my earlier attempt in #995 probably not worth it in GitHub Actions. I did consider https://github.com/reproducible-containers/buildkit-cache-dance but it didn't seem worth the dependency and complexity for us. It seems like the ecosystem is lagging a bit here and I've just been waiting for those upstream issues to be fixed. We could consider depot.dev, I'm not sure if we'd get a discount since we're VC funded but the project is open source 🤷♀ I'm hesitant to administer an external service for something that's not causing us problems right now, but perhaps that's the simplest path forward.
Sorry about the back and forth and hesitant here. This is a tough pull request to review, we appreciate that you're investing time into it but the release artifacts are a critical part of our software. And, since we're a small team, we're careful about taking on more maintenance and services to manage.


Removed close icon.

I do not want to remove the close icon.

Soory, I pulled the request by mistake:)-
On Wed, Jan 25, 2017 at 5:25 PM, Mark Murphy<EMAIL_ADDRESS>wrote:

I do not want to remove the close icon.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/commonsguy/cwac-cam2/pull/304#issuecomment-275089573,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ANCuE5dA_FRuQ9BOCacahY0FD-pHprhXks5rVzgmgaJpZM4LtNjq
.


error when using chain syntax in static site (ssr: false)

Hi there,
Just wondering does this plug-in support chain syntax? e.g. useChain(myArray).find({id: 2}).get("val").value()
I am facing a very strange issue, the code running fine when in Dev mode but not in production. Once SPA site generated, it doesn't like chain syntax. The error pops up as below:
TypeError: (intermediate value)(intermediate value)(intermediate value)(...).find is not a function
Nuxt3 example code - example.vue
<template>

{{ useChain(myArray).find({id: 2}).get("val").value() }}

{{ filtered }}

</template>

Hopefully, someone can guide me to solve this issue, maybe a config problem??
Cheers

Have the same issue, ssr: false and works fine in Dev, production cant find chain.

Not possible as chain needs almost all lodash functions and they are tree shaken on build. You can try flow solution as mentioned here https://stackoverflow.com/a/45464732/10745364

@cipami
maybe isposiable add a warning when use useChain
warning useChain can't use on production ( miss bundle )
If not warned like this, we would discover, after completing a significant amount of code, that everything is a mess only when we are waiting to release the nuxt build version.

@cipami
Perhaps it is possible to add a warning when using useChain. Please note that useChain cannot be used in a production environment (bundle missing).
If not warned in this way, we would realize, after completing a substantial amount of code, that everything is a mess only when we are waiting to release the Nuxt build version.

I'm in that exact situation right now – have I had known that in the beginning my architecture wouldn't rely so much on chain
@cipami is there no way to tell the app to keep the "used sub functions" of chain? I only use 5-6 functions (uniq, reject, pick, omit, filter, map)


Typescript extension using 100% CPU for one minute on every change

Type: Bug
On one proprietary Typescript project I am working on, every change causes the Typescript extension to use up one CPU core for about one minute. So, in effect, when working, one CPU core is on 100% all the time. Autocompletion, code hovering and type checking does not work during this period. Happens with disabled extensions and insiders as well.
Output of code --status
I also have a tsserver.log but I'd rather send it per email than make it public.
Please let me know if there is any more information you need.
VS Code version: Code 1.81.1 (6c3e3dba23e8fadc360aed75ce363ba185c49794, 2023-08-09T22:40:25.698Z)
OS version: Darwin arm64 22.5.0
Modes:

System Info

Item
Value

CPUs
Apple M2 (8 x 24)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (1.58GB free)

Process Argv
--disable-extensions

Screen Reader
no

VM
0%

Extensions disabled

any help?

@sagarpanchal 5.1.6

@dschuessler
https://github.com/microsoft/vscode/issues/191283 <- This issue of mine could be similar
https://github.com/microsoft/vscode/issues/191283#issuecomment-1702691297 <- This is the temporary solution that I found. (Extension bisect didn't help)

@dschuessler Have you tried this tool?
https://github.com/microsoft/typescript-analyze-trace

@sagarpanchal Sorry for not getting back. I worked around the problem by selecting the TypeScript version of my workspace (4.8.4) rather than VSCode's one and then forgot about it. That works for me.
 If you are interested in me trying to reproduce the problem again and analyzing the trace, let me know.

@dschuessler I'm not associated with this project. I was trying to figure out what the issue was.
What I think I have figured out is that it's quite easy to overwhelm typescript by using large unions and complex types.

@dschuessler Still running into this. If you are, please send the logs to matb at Microsoft dot com

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


add nginxModules.brotli

This adds the nginx module for static and on the fly brotli compression. It needs libbrotli which needs a newer version of brotli. Because of that, brotliUnstable is included.
On my side of the internet, all packages (including nginx with the module) compile fine.

Please split the commit into separate ones for each package, with commit messages according to our guidelines. For example, brotliUnstable: init at 0.3.0, nginxModules.brotli: init at 788615e (revision is fine if we don't have a specific version) and so on.

It seems like brotliUnstable could simply be brotli.overrideDerivation { src = ... }, given that (I think) it's identical outside of the source link.

Hm, I remember vaguely that we discourage use of overrideDerivation in nixpkgs, but I can't remember for sure.

Perhaps, but I also hate copying and pasting code :smile:
It doesn't seem like the usual concerns about overrideDerivation blowup would really apply here.

It isn't even 24h since I pushed brotli :-)
What perplexes me about libbrotli is that the github repo and homepage seem not to exist (anymore).

Moreover, you only use src from brotliUnstable so I'm not sure if it's worth to keep it around.

@knupfer One thing that I've failed to mention -- if there is a name attribute where you can actually specify some version (e.g. nginx-1.8.0) we prefer to have dates of the source revision there. For example, if you use revision 123bcde which was committed at 2015-12-24 for package nginx, the name would be nginx-20151224. This applies to brotliUnstable and libbrotli, but not to nginxModules.brotli since there is no place to specify a version there.

@vcunat the repo and homepage exist fine at https://github.com/bagder/libbrotli
Actually I was writing the nix brotli exactly the same time you pushed yours (I finished 30 min after your push). I would also say, that brotliUnstable is useless, should we just raise the version of brotli from v0.3 to head? Brotli isn't finalized, so there's actually no point in having a "stable" version.

Oh, in there is actually written badger and not bagder in the homepage link. (It's the author of curl)

I've fixed the typo and changed the names. How should we proceed? Should I remove brotliUnstable and update brotli?
I've tested the module on my server and it works fine, albeit the algo is disappointingly slow for on the fly compression

LGTM now, let's resolve what should we do with brotli and merge. Thanks!

@knupfer do you strictly need brotliUnstable with libbrotly ? Just checked but google hasn't made another official release since last month.

Yes, without it libbrotli doesn't compile. I would suggest that we use simply only the most recent brotli. The old version is as well a beta version, so it wouldn't be "unstabler".

Alright. Nothing is depending on brotli so it's fine to go unstable I think. Ping me here or on IRC once you've done the change.

@knupfer ping

Travis passed, thanks.


[Gradle JDK] Fix when palantir-gradle-jdks is not installed and jdks are missing

Before this PR

If a user didn't have the Intellij plugin palantir-gradle-jdks installed and if one of the jdks is not installed, they would get a failure in Intellij saying Failed to find the toolchain at path= .... It isn't clear how to unblock.
After this PR

If one of the configured jdks is not installed, the settings plugin will call the gradle/gradle-jdks-setup.sh script. Thsi should only happen if the developer doesn't have the Intellij plugin installed.
[Small cleanup, happy to move it to another PR] Changed the default log message for the Intellij Gradle JDK setup configuration run. Default was: Process finished with exit code

==COMMIT_MSG==
[Gradle JDK] Fix when palantir-gradle-jdks is not installed and jdks are missing
==COMMIT_MSG==
Possible downsides?

:+1:

Released 0.42.0


Review *_count methods in the API for 4.0

Follow-up to #23832 and linked discussion on IRC, and related to #16863.
We have a number of int get_something_count() methods which simply call get_somethings().size() with Array get_somethings() being exposed too. In other places, we don't have the _count methods and only the method returning the array of objects.
We should review these APIs and make everything consistent for Godot 4.0, either no _count method when it doesn't provide added value (e.g. faster retrieval of the number of objects), or add them everywhere for consistency.

I couldn't prioritize working on this and it's not particularly critical. So it could still be done, but it's not worth keeping an issue open for it, especially since it doesn't actually provide details on which methods could be renamed.


File doesnot contain read-type classification. Employing pairtools

I'm implementing the tool on the cloud, but I'm having some problems. Could you explain what they are?
2022/11/28 10:48:48 Starting container setup.
2022/11/28 10:48:50 Done container setup.
2022/11/28 10:48:52 Starting localization.
2022/11/28 10:49:02 Localization script execution started...
2022/11/28 10:49:02 Localizing input gs://fc-00bcc31d-9daf-40fa-bca6-2c6aa332a71c/submissions/ed43243a-456f-4c23-aa8f-d9ab68f35c44/HiTea/7162fb12-ce28-4739-b4b7-27411fadbb73/call-Transposable_Element/script -> /cromwell_root/script
2022/11/28 10:49:04 Localizing input gs://fc-00bcc31d-9daf-40fa-bca6-2c6aa332a71c/NDX-22-829-004.bam -> /cromwell_root/fc-00bcc31d-9daf-40fa-bca6-2c6aa332a71c/NDX-22-829-004.bam
2022/11/28 10:51:36 Localization script execution complete.
2022/11/28 10:51:48 Done localization.
2022/11/28 10:51:55 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint=/bin/bash 4dndcic/hitea@sha256:ed6f807c778fddeca22297132e9c82bed7042d24ba6c76a0fceb63a13c632f97 /cromwell_root/script

I/O and paramters:
-i --inputs /cromwell_root/fc-00bcc31d-9daf-40fa-bca6-2c6aa332a71c/NDX-22-829-004.bam
-o --outprefix ./test
-w --workdir ./here
-m --enzyme MboI
-index --index /usr/local/bin/hg38/TEConsensus_Hs.fa
-repbase --repbase /usr/local/bin/hg38/Repbase_Hs.fa
-x --wgs F
-remap --remap T
-bgAnnotations --bgAnnotations /usr/local/bin/hg38/bgRepeats_hg38.bed.gz
-chr_sizes --chrsize
-anchor_mapq --anchor_mapq 28
-clip --clip 20
-gap gap 2
-read-length inferred-readlength 98.49
-chunksize chunk size 1995M
-ncores (available) --ncores 4
-ncores (used) 4
-threads (available) 8
-threads (used) 8

command: hitea -p 4 -e MboI -q 28 -o ./test -s 20 -g hg38 -w ./here -i /cromwell_root/fc-00bcc31d-9daf-40fa-bca6-2c6aa332a71c/NDX-22-829-004.bam
sourceDirectory /usr/local/bin

Start-Time 00:00:00

# Step1 : checking dependancies ...
great!! all dependancies are present!

running pre-run checks..

1. retrieving header and chr_sizes information

2. creating background annotation files using the input file..

3. confirming TE family and subfamily names..
perl /usr/local/bin/src/prechecks.pl -index /usr/local/bin/hg38/TEConsensus_Hs.fa -polym /usr/local/bin/hg38/Polymorphs_Hs.fa -repbase /usr/local/bin/hg38/Repbase_Hs.fa -bgAnnotations ./here/bgAnnotations -enzyme MboI
(perl prechecks.pl -index /usr/local/bin/hg38/TEConsensus_Hs.fa -polym /usr/local/bin/hg38/Polymorphs_Hs.fa -repbase /usr/local/bin/hg38/Repbase_Hs.fa -bgAnnotations bgAnnotations -enzyme MboI)
L1~L1HS L1HS 6151bp
SVA~SVA_F SVA_F 1395bp
SVA~SVA_D SVA_D 1406bp
SVA~SVA_B SVA_B 1403bp
Alu~AluYb10 AluYb10 293bp
Alu~AluYk13 AluYk13 287bp
Alu~AluYb8 AluYb8 294bp
Alu~AluYb8a1 AluYb8a1 292bp
Alu~AluYb9 AluYb9 294bp
SVA~SVA_C SVA_C 1404bp
Alu~AluYa5 AluYa5 287bp
SVA~SVA_E SVA_E 1402bp
Alu~AluYb11 AluYb11 294bp
SVA~SVA_A SVA_A 1664bp

4. genrating index for TE-consensus..
[bwa_index] Pack FASTA... 0.00 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 0.00 seconds elapse.
[bwa_index] Update BWT... 0.00 sec
[bwa_index] Pack forward-only FASTA... 0.00 sec
[bwa_index] Construct SA from BWT and Occ... 0.00 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index ./here/indexC/TEConsenssus.fa
[main] Real time: 0.021 sec; CPU: 0.006 sec

5. genrating index for Repbase sequences..
[bwa_index] Pack FASTA... 0.01 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 0.27 seconds elapse.
[bwa_index] Update BWT... 0.01 sec
[bwa_index] Pack forward-only FASTA... 0.01 sec
[bwa_index] Construct SA from BWT and Occ... 0.09 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index ./here/repbase/repbase.fa
[main] Real time: 0.414 sec; CPU: 0.386 sec

6. genrating index for Polymorphic sequences..
[bwa_index] Pack FASTA... 0.00 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 0.00 seconds elapse.
[bwa_index] Update BWT... 0.00 sec
[bwa_index] Pack forward-only FASTA... 0.00 sec
[bwa_index] Construct SA from BWT and Occ... 0.00 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index ./here/indexP/Polymorph.fa
[main] Real time: 0.023 sec; CPU: 0.009 sec

pre-run checks completed successfully.

# Step2 : Splitting input file and generating fastq records ...
working on bam input (NDX-22-829-004, size: 10 Gb)
File doesnot contain read-type classification. Employing pairtools
Traceback (most recent call last):
File "/anaconda3/bin/pairtools", line 11, in <module>
sys.exit(cli())
File "/anaconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/anaconda3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/anaconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/anaconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/pairtools/__init__.py", line 113, in wrapper
return func(*args, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/pairtools/pairtools_split.py", line 46, in split
split_py(pairsam_path, output_pairs, output_sam, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/pairtools/pairtools_split.py", line 109, in split_py
sam2 = cols.pop(sam2col)
IndexError: pop index out of range
Input bam file: - input is pipe/sam Use of uninitialized value $flag in numeric ge (>=) at /usr/local/bin/src/parse.pl line 137, <IN> line 3378.
at /usr/local/bin/src/parse.pl line 22, <IN> line 3378.
main::__ANON__("Use of uninitialized value \$flag in numeric ge (>=) at /usr/l"...) called at /usr/local/bin/src/parse.pl line 137
main::bam_read("-") called at /usr/local/bin/src/parse.pl line 93
Command exited with non-zero status 1
0.30user 0.17system 0:00.62elapsed 75%CPU (0avgtext+0avgdata 19112maxresident)k
0inputs+200outputs (0major+34951minor)pagefaults 0swaps
Something went wrong while parsing the input file/s. Exiting!
Completed parsing input file
Time-elapsed 00:00:05
Summary is already displayed in a log file
/usr/local/bin/hitea: line 298: ./here/./test.ReadSummary.logs: No such file or directory
Completed merging back the input parts
Time-elapsed 00:00:06

# Step3 : Extracting clusters ...
Mapping merged fastq file and extracting clusters

bwa mem -t 8 -a -v 1 -k 13 -T 20 ./here/indexC/TEConsenssus.fa ./here/./test_1.fq.gz | perl /usr/local/bin/src/find_breaks.pl -bam - -outprefix ./test_1 -wd ./here -hd ./here/headers.sam -remap T -te 1
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -t 8 -a -v 1 -k 13 -T 20 ./here/indexC/TEConsenssus.fa ./here/./test_1.fq.gz
[main] Real time: 0.001 sec; CPU: 0.002 sec
[extract_clusters=Start_Time]: 0 seconds
Input is pipein
Total number of clip read positions in the file: 0
Total badlines in the file: 0
[extract_clusters=End_Time]: 0 seconds

bwa mem -t 8 -a -v 1 -k 13 -T 20 ./here/indexP/Polymorph.fa ./here/./test_1.1.temp.remap.fq.gz | perl /usr/local/bin/src/find_breaks.pl -bam - -outprefix ./test_1 -wd ./here -hd ./here/headers.sam -remap F -te 2
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -t 8 -a -v 1 -k 13 -T 20 ./here/indexP/Polymorph.fa ./here/./test_1.1.temp.remap.fq.gz
[main] Real time: 0.001 sec; CPU: 0.002 sec
[extract_clusters=Start_Time]: 0 seconds
Input is pipein
Total number of clip read positions in the file: 0
Total badlines in the file: 0
[extract_clusters=End_Time]: 0 seconds

bwa mem -t 8 -a -v 1 -k 19 -T 30 ./here/indexC/TEConsenssus.fa ./here/./test_2.fq.gz | perl /usr/local/bin/src/find_breaks.pl -bam - -outprefix ./test_2 -wd ./here -hd ./here/headers.sam
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -t 8 -a -v 1 -k 19 -T 30 ./here/indexC/TEConsenssus.fa ./here/./test_2.fq.gz
[main] Real time: 0.001 sec; CPU: 0.002 sec
[extract_clusters=Start_Time]: 0 seconds
Input is pipein
Total number of clip read positions in the file: 0
Total badlines in the file: 0
[extract_clusters=End_Time]: 0 seconds
Completed extracting cluster locations
Time-elapsed 00:00:06

can this work on exome WGS bam files, Illumina paired end


Safely open / close netCDF files without resource locking

Code Sample, a copy-pastable example if possible
(essentially the same to #1629)
Opening netCDF file via xr.open_dataset locks a resource, preventing to write a file with the same name (as pointed out and answered as an expected behavior in #1629).
import xarray as xr
ds = xr.Dataset({'var': ('x', [0, 1, 2])})
ds.to_netcdf('test.nc')

ds_read = xr.open_dataset('test.nc')
ds.to_netcdf('test.nc')   # -> PermissionError

ds_read = xr.open_dataset('test.nc').load()
ds.to_netcdf('test.nc')   # -> PermissionError

ds_read = xr.open_dataset('test.nc').load()
ds_read.close()
ds.to_netcdf('test.nc')  # no error

Problem description
Another program cannot write the same netCDF file that xarray has opened, unless close method is not called.

-- EDIT --
close() method does not return the object, thus it cannot be put in the chain call, such as
some_function(xr.open_dataset('test.nc').close())

It is understandable when we do not want to load the entire file into the memory.
However, sometimes I want to read the file that will be updated soon by another program.
Also, I think that many users who are not accustomed to netCDF may expect this behavior (as np.loadtxt does) and will be surprised after getting PermissionError.
I think it would be nice to have an option such as load_all=True or even make it a default?

This pattern should work:
with xr.open_dataset('test.nc') as ds_read:
    ds_read.load()
ds.to_netcdf('test.nc')

OK. But what about the usual scientist workflow where you work in multiple cells
ds.load()

# do computation

next cell:
do more computation

...
ds.close()  # is this right?
ds.to_netcdf('test.nc')

I wonder if we should add autoclose as a kwarg for to_netcdf. If autoclose=True, remove that file from the LRU cache if present, and then write to file.

I think this is more of a limitation of netCDF-C / HDF5 than xarray. For example, this example works if you use SciPy's netCDF reader/writer:
import xarray as xr
ds = xr.Dataset({'var': ('x', [0, 1, 2])})
ds.to_netcdf('test.nc', engine='scipy')

ds_read = xr.open_dataset('test.nc', engine='scipy')
ds.to_netcdf('test.nc', engine='scipy')

I didn't notice that with statement can be used for open_dataset.
Thanks for the comments.
I personally want to a simpler function for the daily (not so big data) analysis without caring open / close stuff.
(and probably also for new users)
Is it too much if we add load_dataset function that works similar to np.load_txt in terms of the file handling?

Just to clarify, load_dataset would be equivalent to:
def load_dataset(*args, **kwargs):
    with xarray.open_dataset(*args, **kwargs) as ds:
        return ds.load()

This also seems pretty reasonable to be. I’ve a version of this utility function a handful of times, so I at least would find it useful.
Would we also want load_dataarray?

I used to use the autoclose=True argument to open_dataset() all the time to avoid this issue. Then it was deprecated with the LRU introduction, and I now do a two-line ds = open_dataset().load() and then ds.close(). Just adding my use-case and what I found to be a regression in terms of usability. Something like load_dataset() would be helpful but I wonder if adding an argument to the existing open_dataset() would be better.

Just to clarify, load_dataset would be equivalent to:
def load_dataset(*args, **kwargs):
with xarray.open_dataset(*args, **kwargs) as ds:
return ds.load()

Yes, that is actually in my mind.
I added a tag good first issue for this.
I love to implement this by myself but I don't think I have enough free time near future...

Took a stab at implementing these functions.


Harpoon 2 - How Highlight Groups Works?

Harpoon 2 - How Highlight Groups Works?
I can't find in the documentation any example of highlight groups being used.
I'm not sure where can I ask this, sorry if the issues is not the right place.

you can set any color you want on those groups like this
        vim.api.nvim_set_hl(0, 'HarpoonBorder', { fg = c.bg_dark, bg = c.bg_dark })
        vim.api.nvim_set_hl(0, 'HarpoonWindow', { fg = c.fg, bg = c.bg_dark })

Thank you so much


PP-6257 Add live services CSV download

Add a button the the "Platform services" page to download a CSV listing the active live services in the format expected by the performance platform.

Great work


Add new compaction abstraction, simulator, and implementation.

Rebased version of #5234, part of #6768
This consists of three parts:

A refactoring and new contract for implementing and testing compaction.

The logic is now in a separate crate, with no dependency on the 'pageserver' crate. It defines an interface that the real pageserver must implement, in order to call the compaction algorithm. The interface models things like delta and image layers, but just the parts that the compaction algorithm needs to make decisions. That makes it easier unit test the algorithm and experiment with different implementations.
I did not convert the current code to the new abstraction, however. When compaction algorithm is set to "Legacy", we just use the old code. It might be worthwhile to convert the old code to the new abstraction, so that we can compare the behavior of the new algorithm against the old one, using the same simulated cases. If we do that, have to be careful that the converted code really is equivalent to the old.
This inclues only trivial changes to the main pageserver code. All the new code is behind a tenant config option. So this should be pretty safe to merge, even if the new implementation is buggy, as long as we don't enable it.

A new compaction algorithm, implemented using the new abstraction.

The new algorithm is tiered compaction.  It is inspired by the PoC at PR #4539, although I did not use that code directly, as I needed the new implementation to fit the new abstraction. The algorithm here is less advanced, I did not implement partial image layers, for example. I wanted to keep it simple on purpose, so that as we add bells and whistles, we can see the effects using the included simulator.
One difference to #4539 and your typical LSM tree implementations is how we keep track of the LSM tree levels. This PR doesn't have a permanent concept of a level, tier or sorted run at all. There are just delta and image layers. However, when compaction starts, we look at the layers that exist, and arrange them into levels, depending on their shapes. That is ephemeral: when the compaction finishes, we forget that information. This allows the new algorithm to work without any extra bookkeeping. That makes it easier to transition from the old algorithm to new, and back again.
There is just a new tenant config option to choose the compaction algorithm. The default is "Legacy", meaning the current algorithm in 'main'. If you set it to "Tiered".

A simulator, which implements the new abstraction.

The simulator can be used to analyze write and storage amplification, without running a test with the full pageserver. It can also draw an SVG animation of the simulation, to visualize how layers are created and deleted.
To run the simulator:
./target/debug/compaction-simulator run-suite

cc @hlinnaka
I pushed this with the last published state of #5234 . Have only spent 4-5 hours today rebasing it and adjusting it for the changes that happened since, not done anything else (large part was waiting for the compiler).
Next is to address the review comments on #5234 by @koivunej and me, or put them as followups.

Force pushed to get rid of the stray file


Query limit and ordering

This PR adds an ability to sort the query results and limit them to a specified amount of entries.
The ordering is specified using a new Order message, and limit is specified using page_size parameter in the Pagination message. Both are attributes of a Query.
The QueryBuilder was extended to support newly introduced parameters.
Their support was also implemented for an existing in-memory storage implementation.

@armiol, PTAL.


Add support for SPARCv9 and RISC-V (32 and 64-bit)

Please provide a succinct description of the issue.
I would like to learn how to add support for these two open-source instruction set architectures, if the compiler can't already compile to these targets.
Provide the steps required to reproduce the problem

Step A

Step B

Expected behavior
Provide a description of the expected behaviour.
Actual behavior
Provide a description of the actual behaviour observed.
Known workarounds
Provide a description of any known workarounds.
Related information
Provide any related information

Operating system
Branch
.NET Runtime, CoreCLR or Mono Version
Editing Tools (e.g. Visual Studio Version)
Links to F# RFCs or entries on http://fslang.uservoice.com
Links to performance testing scripts
Indications of severity

This isn't an F# issue -- support for new processor architectures (like SPARCv9, RISC-V, MIPS, POWER) would need to be added in mono or coreclr. When either of those projects supports the architecture(s) you want, it shouldn't require much work (if any) to get F# running on them.
Consider opening an issue on those projects to ask about support for SPARCv9 or RISC-V. Mono still has codegen support for older SPARC architectures; it looks like the code hasn't been touched in a while though, and may need to be updated for SPARCv9.

Closing as this is a Mono or CoreCLR issue


TabularPredictor: load_X_train, load_y_train, load_X_val, load_y_val

Issue #, if available:
#249, #428, #429
Description of changes:
Added load_X_train, load_y_train, load_X_val, load_y_val functions to TabularPredictor.
Now when users use transform_features(), they can get the y labels as well by calling these functions.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-434-1 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-434/1/index.html

Might these methods (load_X_train, load_y_train, load_X_val, load_y_val) might be better as internal methods (e.g. in Learner)?  I'm just worried if the Predictor offers too many methods, then UX may become more complicated than necessary.
If these methods become internal, there could just be certain args in transform_features() to invoke each (combination of) these.  For example: dataset arg in transform_features() could have special str options train & validation.  I also don't see why transform_features() shouldn't automatically return a tuple with both X and y (the latter whenever it's available in data, otherwise returned y = None; or could even have a boolean arg transform_label in transform_features() which determines whether y is returned or not).
Another related point that the documentation might explicitly state somewhere is:   transform_features() (and these new load_* methods) allow you to train other 3rd party ML models using the same data representation used internally by AutoGluon.
If this is the main intended use-case, I'd also consider rethinking whether transform_features() returns the train or validation DataFrame by default (when dataset=None), as I think returning train might make more sense in this context (or even both train and val).  If the above suggestion is adopted with special dataset options train & validation, then you could also just set dataset=train as the default in transform_features().

Addressed comments based on feedback, combined functionality into a single function.

Job PR-434-2 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-434/2/index.html


Weather Underground PWS

Please can you add the following to Weather Underground service as it currently only does temperature and humidity ...

pressure
wind speed
wind direction
wind gust
rain
rain rate

Thank you

Another issue with Weather Underground updates is that Weather Underground needs all sensor values passed at once. At the moment MyController is sending individual sensor updates each time that particular sensor changes. This causes Weather Underground to show "--" for all other sensor values.


Cannot download the dataset.

Hi all.
Thanks for the interesting work.
The link " https://oval.cs.stanford.edu/datasets/pldi2019-quote-free.zip" doesn't seem to work.
Thanks!

Thank you, the correct link is https://oval.cs.stanford.edu/datasets/pldi19-quote-free.zip
I fixed the instructions.


[WIP] - Complete Test Suite

WORK IN PROGRESS - DO NOT MERGE
This PR adds a complete automated test suite for DriveShare. It depends on the branch gordonwritescode/refactor/allthethings (currently mirrored on develop).
Run the unit tests with:
npm run test-unit

Run the integration tests with:
npm run test-integration

Run all the tests:
npm test

TODO
Unit Tests

[x] dataserv.js
[ ] installer.js
[x] logger.js
[x] tab.js
[x] updater.js
[x] userdata.js

Integration Tests (views)

[ ] setup
[ ] logs
[ ] updater
[ ] about
[ ] main
[ ] footer

This will ultimately close #67

Fixes #85

Closes #80

Will close #79

Will close #84

100% TEST COVERAGE NERDS!

:shipit:


Challenge test regex doesn't recognize dash

Challenge Add Images to your Website has an issue.
User Agent is: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:

<link href="https://fonts.googleapis.com/css?family=Lobster" rel="stylesheet" type="text/css">

CatPhotoApp

Kitty ipsum dolor sit amet, shed everywhere shed everywhere stretching attack your ankles chase the red dot, hairball run catnip eat the grass sniff.

Purr jump eat the grass rip the couch scratched sunbathe, shed everywhere rip the couch sleep in the sink fluffy fur catnip scratched.

does not recognize the alt tag even tho user typed it in the image tag

@austinHeisleyCook thanks for the issue. The real issue is that it doesn't recognize the dash - within your alt tag. I'm not sure why isn't not. I don't see anything wrong with the challenge test regex.
Regex101: https://regex101.com/r/0LCoJt/2
cc/ @freeCodeCamp/moderators

If you remove one letter from your image src , you can see that your alt value will be shown on the screen. You can write cat or kit-kat, both will be shown on the screen but it won't let you pass if you use - anywhere in the alt tags.
Tested on : Firefox - version 52
Chromium - version 57
By the way, alt = "this.that" and alt="this_that" works.

@erictleung I dig a little digging into the source code and found that the challenge is currently using the old test regex, which is alt\s*?=\s*?(\"|\')[\s\w\.\,\!]+?(\"|\'). It is evident that the regex does not capture dashes within the square brackets.
Here is the regex test: https://regex101.com/r/0LCoJt/2
Seems like an easy enough fix, maybe we can open this up to first-timers? 😄

The regex has already been improved on staging (beta) to account for any character except line terminators. You can test it here: http://beta.freecodecamp.com/en/challenges/basic-html-and-html5/add-images-to-your-website
Since updates to the production site are now limited to "critical issues", I vote we mark this as resolved in staging.

@dhcodes thanks for checking on beta! I forgot to check there :sweat_smile: I'll close this then and leave the resolved in staging label.


[Request/Suggestion] Add new sorting filter and provider

Hello,
First, thank you very much for your work on this add-on.
I'd like to know if its possible to add a new sorting filter. I'd prefer a recent torrent over an old one to avoid to be "flashed" by the local authority. Or at least if its possible to show the date of the upload.
Also would it be possible to add yggtorrent as a new french provider ? I know its been requested in the past and there's a lot of demand. Its a private french provider which has a lot of content. Since its a private one, it would requiere username and password options.
Sometimes the address can change because of DNS block, the domain can be monitored here : https://yggland.fr/FAQ-Tutos/#status
Ressources :
From Elementum -
https://raw.githubusercontent.com/elgatito/script.elementum.burst/master/burst/providers/providers.json
From JimmyLaurent -
https://github.com/JimmyLaurent/torrent-search-api
Regards,

Hi, as was answered before, private trackers will never be part of torrentio.


Update the tech-docs-gem to the newest version of GOV.UK Frontend

What
Update the tech-docs-gem to use version 3.12.0 of GOV.UK Frontend.
Why
The tech-docs-gem is using version 3.6.0 of GOV.UK Frontend, which is a couple of minor releases behind. We will be releasing the new link styles in version 3.12.0 and think this will be a good time to update the tech-docs-gem to use this version so everyone benefits from the new link styles.
Who needs to know about this
Developers
Done when

[ ] The tech docs gem is using GOV.UK Frontend 3.12.0
[ ] The tech docs gem is opted into the new link styles
[x] We should probably check the changelog between 3.6.0 and 3.12.0 to see if there are any other changes we need to make in the tech docs gem
[ ] Roll this change out to Frontend Docs?

I've opened a PR to update GOV.UK Frontend in the tech-docs-gem repo: alphagov/tech-docs-gem#232. This PR includes updating the link styles to use the new styles and mixins in v3.12.0 (thanks for the help with that @36degrees), and also changes the Sass imports to take advantage of the quicker CSS compiles from v3.7.0.
Once that is merged I will be able to update the gem itself by opening another PR to bump the version and do a release.

PR to release new version of tech docs gem https://github.com/alphagov/tech-docs-gem/pull/234

Ruby gem govuk_tech_docs v2.3.0 has been released 🎉 Going to update frontend docs now.

Updated frontend docs, new link styles are live at frontend.design-system.service.gov.uk. Calling this done.


Getting index from inner object

I having trouble with getting the index of an object inside an array to the template. I want to have a field in my JSON named FAQ_ID with the index of the object as its value. I have no trouble getting the index in the headerTemplate, but cannot access the index from the FAQ_ID property. My JSON schema is as follows:
 FAQ: {
                      type: "array",
                      format: "tabs",
                      description: "Frequent Questions",
                      items: {
                          title: "FAQ",
                          id: "FAQItem",
                          headerTemplate: "{{i}} - {{self.Category}}",
                          type: "object",
                              properties: {
                                FAQ_ID: {
                                    type: "string",
                                    format: "hidden",
                                    headerTemplate: " ",
                                    template: "{{myVar}}",
                                    watch: {
                                        myVar: "FAQItem.i"
                                    }
                                },
                                Category: {
                                  type: "string"
                                },
                                Text: {
                                    type: "string",
                                    format: "textarea"
                                }
                              }
                      }

I have tried exchanging myVar with different alternatives like FAQItem.{{i}} , but I was not succesful. Is it possible to access the index from inside the array properties?

Sorry to hijack; Aaes, did you find a workaround for this?


[WIP] Remove block from MiqQueue#put_unless_exists

Summary
MiqQueue#put_unless_exists adds a unique message to the MiqQueue.
This PR change:

Make it easier to declare which parameters are used for uniqueness (the _unless_exists part) and which are used for creation (put part).
Prep for: change the SQL SELECT, INSERT queries to a single INSERT statement. (all data in sql friendly hashes, and not looking at returning object)

Before
We used a block that was clunky, nuanced, and preventing us from tuning the sql:
MiqQueue.put_unless_exists(
  :class_name  => "Metric",
  :method_name => "purge_hourly",
) do |_msg, find_options|
  find_options.merge(:args => [timestamp])
end

What does this do?

If a message for Metric#purge_hourly already exists, then don't create one. Ignore the :args value since it is a timestamp and probably won't match anyway.
If a message for Metric#purge_hourly does not already exist, then create one. The new record will include the :args value provided.

Stated another way:

:class_name and :method_name are part of the uniqueness constraint.
:class_name, :method_name, and :args are used for creation.

After
We can leverage an active record mechanism created_with to convert these parameters into SQL. On one hand, it does not introduce any change to MiqQueue, but it is more closely coupled with ActiveRecord:
MiqQueue.create_with(:args => [timestamp]).put_unless_exists(
  :class_name  => "Metric",
  :method_name => "purge_hourly"
)

Alt
We can add another parameter to put_unless_exists. This interface is not as elegant, requires a change to MiqQueue, but is less coupled with ActiveRecord:
MiqQueue.put_unless_exists({
  :class_name  => "Metric",
  :method_name => "purge_hourly"
  }, :args => [timestamp]
)

This is starting scope creep - pulled out last commit into own PR. this should simplify it (at least a little)

@gtanzillo Of the 31 instances of put_unless_exist, 8 took a block. This pr changes the 5 that manipulate the hash in the block:
#before
MiqQueue.put_unless_exists(unique_params) do |msg|
  msg.merge(create_params)
end
# after
MiqQueue.create_with(create_params).put_unless_exists(unique_params)
# or
MiqQueue.create_with(create_params).put_unless_exists(unique_params) do |msg|
  _logger.debug("created a record #{msg.id}")
end

There are some put_or_update that don't manipulate the queue entries. So in #14569 I changed them over to put_unless_exist and the create_with pattern

WIP:

purging goes into separate pr / and becomes just a put
clean up extra commits that got into here
ensure callbacks don't get botched up

fixed cops - I'll pull out anything that isn't straight forward

In ActiveRecord, just as where(:a => 5).order(:c) is 2 method calls but an atomic method call.
ActiveRecord does create_with(:c => 5).create_or_update(:a => 5).
True, current implementation has the second as 2 different database queries, but that is what monkey patching / PRs is about

ActiveRecord does create_with(:c => 5).create_or_update(:a => 5).

@kbrock That is fair.  I guess that goes to show you my lack of keeping up with the new ActiveRecord interfaces.  Anyway, looks good.

@kbrock Please also don't forget to check for put_unless_exists in the split out repos.  Not sure they exist or not, but we can't forget those.


CALL LIST: Don't let me put patients onto my call list twice

We should have a safety valve in place so that we can't add individual patients to a call list twice.

Creatively fixed in @ajohnson052 's A+ work in #114, since objects are getting associated with current_user. Closing.


DOT 1.74: args.rellinks does not generate related links correctly for subjectScheme topics: html plugin

Where a subjectScheme item links to a topic AND has a parent and children subjectScheme items, the related links are not generating correctly and not correctly controlled by the three values of args.rellinks.
In the following image:

The 'What is Shelf Review?' topic has a parent and children. It has related links entitled 'Parent Topic' but they are both Parent and Child topics.
No value of arg.rellinks will turn off these auto-generated links whilst allowing user created ones to be included.

Marking this as closed - no sample was provided to reproduce the issue, and it has been marked inactive since October 2014. If a sample is provided we can take another look.


Feature: support for touch slide?

Could you add support for touch slide just like the CupertinoSlidingSegmentedControl?

@savadodesigns i would like to do this, but now there is not enough time, as soon as there is time I will try to add


PDF processing of links from the same topic to the same topic has changed between 2.3.3 and 2.4.2

This is really difficult to explain...
The link processing has changed somewhere between DITA-OT 2.3.3 and DITA-OT 2.4.2. With the DITA-OT 2.3.3, multiple links to the same target in the same topic are always processed. It does not matter, whether it is a @href or @keyref.
With DITA-OT 2.4.2 only the first link from the same topic to the same topic is generated correctly. Maybe this is not a bug, but a feature. It may make sense, because you probably do not need multiple references to the same topic in the same topic.

Output with DITA-OT 2.3.3
Output with DITA-OT 2.4.2

bookmap-2.3.3.pdf
bookmap-2.4.2.pdf

But: Unfortunately this also affects conkeyrefed modules, e.g. conkeyrefed <xref> elements or conkeyrefed modules that contain a <xref> element. So, if you, for example, conkeyref a <note> element with a <xref> element in two separate topics, the <xref> is only processed in the first topic. It does not matter, whether there are 0...n chapters in between.
test-files.zip

I cannot reproduce this with DITA-OT 2.4.5 and therefore close this issue. :smile: :+1:


uninitialised variable warning with "ruby -v"

/usr/local/lib/ruby/gems/1.9.1/gems/uuidtools-2.1.2/lib/uuidtools.rb:399: warning: instance variable @string not initialized

@gfim I've submitted a pull request for this fix for the warnings https://github.com/monde/uuidtools/commit/5515316154f431cfdf8f12c8437b43aacab5c5e6

Looks good. Thanks.


Download GIF first frame only

DFImageRequestOptions could contain an option GIFFirstFrameOnly
which downloads only the first GIF frame
for situation, if you want to have a play button on GIF and display the first frame placeholder
(optional, in the progressHandler you could retrive the first frame once available)

for situation, if you want to have a play button on GIF and display the first frame placeholder

This is very application specific. Here is what you can do with available classes:
DFImageRequest *request = ...;
DFImageTask *task = [[DFImageManager sharedManager] imageTaskForRequest:request completion:^(UIImage *__nullable image, NSError *__nullable error, DFImageResponse *__nullable response, DFImageTask *__nonnull completedTask) {
   // Decoded GIFs are wrapped into UIImage subclass (DFAnimatedImage)
    DFAnimatedImage *animatedImage = image;

    // FLAnimatedImageView is used for GIF playback
    FLAnimatedImageView *imageView = ...;

    // DFAnimatedImage can be used as a UIImage with a single frame
    // Settings .image property of FLAnimatedImageView will display first GIF frame and won't start playback
    imageView.image = animatedImage;
}];

- (void)buttonPlayTapped {
    DFAnimatedImage *image = self.animatedImage;
    // Set animated image to start GIF playback
    self.animatedImage = image;
}

The only drawback is that placeholder would not be displayed until the entire GIF is downloaded. This is again very application/server specific. You can workaround that by having two separate image requests. First one would download a GIF placeholder. The second one would download the entire GIF (you could start second request when Play button is tapped).

(optional, in the progressHandler you could retrive the first frame once available)

That's exactly what I'm working on right now. I'm refactoring some parts of the framework to add progressive JPEG support #28, #41. It would enable your use case too (display first GIF frame while file is still downloading).

I've almost finished working on #28 (progressive image decoding) in develop branch.
It already enables progressive GIF decoding. In order to implement this case you would do something like this:

Enable progressive image on DFImageManager. This step makes it possible to implement progressive image decoding in a way that has zero overhead if it is not required.

DFImageManagerConfiguration *conf = ...
conf.allowsProgressiveImage = YES;
[DFImageManager setSharedManager:[[DFImageManager alloc] initWithConfiguration:conf]];

Create image request that allows progressive image

DFMutableImageRequestOptions *options = [DFMutableImageRequestOptions new];
options.allowsProgressiveImage = YES;
DFImageRequest *request = // create request with given options

Start image task with a given request

FLAnimatedImageView *imageView = ...

DFImageTask *task = [[DFImageManager sharedManager] imageTaskForRequest:request completion:^(UIImage *__nullable image, NSError *__nullable error, DFImageResponse *__nullable response, DFImageTask *__nonnull imageTask){
    FLAnimatedImage *animatedImage  = [((DFAnimatedImage *)image) animatedImage];
    imageView.image = animatedImage; // Start GIF playback as soon as full data is loaded.
    // Alternatively, wait until user tapes "Play" button
}];

task.progressiveImageHandler = ^(UIImage *__nonnull image){
    if (!imageView.image) {
        imageView.image = image; // This call would not start GIF playback
    }
};

[task resume];

Closing this task. If you have more general questions, please use Stack Overflow. I'm subscribed for dfimagemanager tag.


Environment is not showing for Folder on latest release

Expected Behavior
Trying to update the environment for a folder and there is not option.

Actual Behavior
Nothing is showing for Environment
Reproduction Steps
right click on folder...option is not present
Is there an existing issue for this?

[X] I have searched the issue tracker for this problem.

Additional Information
n/a
Insomnia Version
9.3.2
What operating system are you using?
Windows
Operating System Version
Windows 10
Installation method
Download
Last Known Working Insomnia version
No response

If you click on the folder (like you would do for a request), do you see this section?

Closing as the environment for folders can be found in the folder view


Remove cc toolchain registration from rules_cc in MODULE.bazel

The same toolchain is registered by @bazel_tools already, we should not register it again to prevent competing with the apple cc toolchain from apple_support.
Context: https://github.com/bazelbuild/rules_swift/issues/1115#issuecomment-1728219962

In principle: I'd say the toolchain falls naturally into rules_cc and not bazel_tools. I think the registration should be remove from bazel_tools instead.
Is this a temporary solution? Or will Bazel users when bazel_tools are gone, register C++ toolchain in their repos?

Is this a temporary solution? Or will Bazel users when bazel_tools are gone, register C++ toolchain in their repos?

I think in long term, yes, we should move the toolchain registration to rules_cc and remove the ones in bazel_tools.
But until we figure out a better way to specify toolchain registration priority, I believe it's safer to keep it in bazel_tools. Because we can make sure bazel_tools is always appended at the end of the root modules' MODULE.bazel file, but have no control on the order of rules_cc and apple_support.
/cc @katre @Wyverald

Is this a temporary solution? Or will Bazel users when bazel_tools are gone, register C++ toolchain in their repos?

I think in long term, yes, we should move the toolchain registration to rules_cc and remove the ones in bazel_tools.
But until we figure out a better way to specify toolchain registration priority, I believe it's safer to keep it in bazel_tools. Because we can make sure bazel_tools is always appended at the end of the root modules' MODULE.bazel file, but have no control on the order of rules_cc and apple_support.
/cc @katre @Wyverald

Wait. Don't apple users have complete control? MODULE.bazel is the last. Anybody using @rules_apple, just need to put it after @rules_cc.

Anybody using @rules_apple, just need to put it after @rules_cc.

That's true! But with this change, they don't need to do that at all. I don't feel super strong with this change, as there is a workaround, @keith @brentelyjones

The toolchain in rules_cc is problematic in any case as it diverged from the one in Bazel itself and may no longer work well with more recent versions of Bazel. I even tried to remove it at some point to prevent confusion.
I would thus recommend:

Remove the toolchain registration in rules_cc now. Whoever wants the vanilla toolchain even with apple_support should just register the one from bazel_tools anyway.
Move the vanilla C++ toolchain from bazel_tools to rules_cc.
Come up with a better long-term solution that allows choosing between multiple registered toolchains.

Is there a case that the end user would actually prefer the vanilla cc toolchain over the apple one?

For rules_apple users, no. And most won't have a rules_cc bazel_dep, so they won't be able to change the order of it.

The toolchain in rules_cc is problematic in any case as it diverged from the one in Bazel itself and may no longer work well with more recent versions of Bazel. I even tried to remove it at some point to prevent confusion.
I would thus recommend:

Remove the toolchain registration in rules_cc now. Whoever wants the vanilla toolchain even with apple_support should just register the one from bazel_tools anyway.
Move the vanilla C++ toolchain from bazel_tools to rules_cc.
Come up with a better long-term solution that allows choosing between multiple registered toolchains.

Is it possible to copy Bazel's toolchain over rules_cc?
I'd prefer if the toolchain is removed from bazel_tools, like that was done for Java.
The reason that this is moving in the right direction and not leaving more things to do later.

Is it possible to copy Bazel's toolchain over rules_cc?

Yes, but probably not in time for Bazel 6.4.0. :-)
If it is added to rules_cc with automatic registration, then users will still run in the situation where bazel_dep order matters. That essentially breaks our story around MODULE.bazel being descriptive rather than imperative, so I think we need to tackle 3. before doing 2. and keeping the automatic registration.

could we remove the bazel_tools toolchain registration before bazel 7.x? so that way we'd end up in the good final state here sooner rather than later

I'd personally lean towards removing the cc toolchain registration from bazel_tools (and keeping this one in rules_cc) too. It's conceptually cleaner and removes mental burden.

If it is added to rules_cc with automatic registration, then users will still get into the situation where bazel_dep order matters. That essentially breaks our story around MODULE.bazel being descriptive rather than imperative

I wouldn't say that. Declarative =/= order-insensitive. It's not perfect that we don't yet have a more explicit way to choose from registered toolchains, but the BFS order was a conscious design decision from day 1.

I wouldn't say that. Declarative =/= order-insensitive. It's not perfect that we don't yet have a more explicit way to choose from registered toolchains, but the BFS order was a conscious design decision from day 1.

Sorry, I didn't mean to say that this is somehow not working as designed. In the absence of any domain-specific information, BFS is a great default and much better than WORKSPACE's "first come, first serve" semantics.
I just think that BFS works much better between different graph depths (e.g. my direct deps' toolchains win over my transitive deps's toolchains) than on the same depth (reorder modules to decide which toolchain wins), in particular when we are considering adding import capabilities to module files. Still a decent fallback, but IMO we need something better.

could we remove the bazel_tools toolchain registration before bazel 7.x? so that way we'd end up in the good final state here sooner rather than later

@keith I can try to move cc toolchains to rules_cc before Bazel 7 cut, but that means we do need to depend on rules_cc and apple_support order. Can you help document this somewhere for apple users until we figure out a better solution to control the toolchain registration order?
Since we are heading to the final state anyway, I'll give up on this PR.

is the ordering issue only if the user directly depends on both? vs bazel depending on rules_cc but not apple_support, and even apple_support depending on rules_cc (which we could add for this if it would help)

is the ordering issue only if the user directly depends on both?

yes -- if the user only directly depends on apple_support, everything's fine. The other two cases you mentioned aren't affected.

https://github.com/bazelbuild/apple_support/pull/265

Thanks for updating the doc!


Support schedulers other than kubernetes

It would be really cool if torus can be generalized to support schedulers other than kubernetes. For example, nomad and mesos.

Right now there's nothing kubernetes-scheduler specific; it's just an app running in a pod, with the mount tool supporting the flexvolume plugin. Extending the mount capabilities for other platforms is completely open, and this is a good tracking issue for it.

@F21 Kubernetes has the concept of mounting storage for a container, which is what the flexvolume integration is. Can you research what the alternatives would be for nomad or mesos?

@philips
For nomad, there is no way to mount storage volumes yet. Because nomad supports docker, qemu, raw exec and a few other task drivers to run tasks, the devs want to implement 1 interface/api that will support storage volumes for all task drivers. The tracking issue is here: https://github.com/hashicorp/nomad/issues/150
Mesos supports persistent volumes since 0.23. The behavior and schematics appears to be quite similar to how kubernetes handles volumes: http://mesos.apache.org/documentation/latest/persistent-volume/ Since 0.27, volumes can be managed using the HTTP API.

@F21 can you do the work on either of these?

@philips Unfortunately, I don't have enough bandwidth at the moment, however, I am happy to help look at any proposed designs.
Having said that, I think the nomad team at hashicorp probably wants to work on their volume capability internally rather than source contributions from the community.


Implemented type of Value mismatch

There's an extra Integer field before the maps, in Figure 1 of GL-D1 Value is a derived type of PolicyID → (AssetName → Quantity).
https://github.com/input-output-hk/cardano-ledger-specs/blob/a55a0458008bfe23a92fe25bb81b7b1c96f73827/shelley-ma/impl/src/Cardano/Ledger/Mary/Value.hs#L122
I believe this is used for the ADA quantity (for convenience perhaps?).
https://github.com/input-output-hk/cardano-ledger-specs/blob/a55a0458008bfe23a92fe25bb81b7b1c96f73827/shelley-ma/impl/src/Cardano/Ledger/Mary/Value.hs#L158
The spec alluded that it should be theoretically included in the map with (adaPolicy, adaName).

Yes, this is an intentional difference in the formal/executable specs. One of the reasons for this difference is memory efficiency: We require Ada deposits for outputs as protection against spam, and the smaller we can make an output, the smaller the required deposit is. This saves a ScriptHash on every output, which is quite substantial.
We probably don't want to change the executable spec, but what we could do is make Value abstract in the formal spec, and just require it to be a monoid with a function Value -> P PolicyID and morphisms valueToCoin and coinToValue that are inverses in one direction. Then, we could move the definition of Value into an appendix and explicitly mark it as a suggestion. I think I've suggested this before but it wasn't popular because this is more or less the key definition of the entire spec.

I like your suggestion @WhatisRT


BUG: Unable to launch apps from UI and also from the menu

I am using distrobox v<IP_ADDRESS> and boxbuddy v2.2.4 on NixOS 24.05 (Gnome 46 + XORG)
I installed distrobox from the NixOS repository...
I installed ubuntu 24.04 and added some apps...
When I try to open any of the installed applications using the UI, nothing happens.
Also when I send the apps to the menu and try to launch them from there, nothing happens.
Also I do not get any error message..
Only through the Console I am able to start all applications without problems.

This is my system:

Hello
I do not package or support NixOS. Please open an issue against the nix package

Okay, just want to share some extra infos, I also tried using the Flatpak version, and also got the same behaviour.
I am getting the following errors for each of the installed applications using the Flatpak version of BoxBuddy:
Error for "4K Video Downloader+":
[2024-09-08 18:11:25.751419] [0x00007ff3f0ff96c0] [info]    started
QEventLoop: Cannot be used without QApplication
[2024-09-08 18:11:25.751451] [0x00007ff3ef7f66c0] [info]    started
[2024-09-08 18:11:25.751472] [0x00007ff3f07f86c0] [info]    started
[2024-09-08 18:11:25.751489] [0x00007ff3efff76c0] [info]    started
[2024-09-08 18:11:25.751506] [0x00007ff3eeff56c0] [info]    started
[2024-09-08 18:11:25.751535] [0x00007ff3ee7f46c0] [info]    started
terminate called after throwing an instance of 'std::runtime_error'
  what():  locale::facet::_S_create_c_locale name not valid
Aborted (core dumped)

Error for "Gdebi Package installer":
Error: crun: executable file 'gdebi-gtk %f' not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
Error for "TeXInfo":
info: Terminal type 'dumb' is not smart enough to run Info
Error for "Synaptic Package Manager":
The value for the SHELL variable was not found in the /etc/shells file

This incident has been reported.

Error for "Vim":
Error: crun: executable file 'vim' not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
Error for "Zutty":
Warning: could not set locale!
Expect broken international characters (or fix your locale)!
E [fontpack.cc:218] Error: No Regular variant of the requested font '9x18' could be identified.
terminate called after throwing an instance of 'std::runtime_error'
  what():  No suitable files for '9x18' found!


Data.JSString.pack throws RangeError on lists with certain thunk structures

Might be related to https://github.com/ghcjs/ghcjs-base/issues/128
I encountered it in the wild by way of showFFloat (Just 6) (2.2e-2), here's a reproduction case
import Numeric (showFFloat)
import Data.JSString (pack, unpack)

main :: IO ()
main = do
  putStrLn . unpack . pack $ showFFloat (Just 2) 2.2e-2 ""

ghcjs ./test.hs && node test.jsexe/all.js results in:
Haskell main thread: RangeError: Invalid code point: NaN
RangeError: Invalid code point: NaN
    at /home/ross/3rd/reflex-platform/test.jsexe/all.js:20550:29
    at h$jsstringPack (/home/ross/3rd/reflex-platform/test.jsexe/all.js:20711:36)
    at h$$Kt (/home/ross/3rd/reflex-platform/test.jsexe/all.js:57018:11)
    at h$runThreadSlice (/home/ross/3rd/reflex-platform/test.jsexe/all.js:6845:11)
    at h$runThreadSliceCatch (/home/ross/3rd/reflex-platform/test.jsexe/all.js:6814:12)
    at Immediate.h$mainLoop [as _onImmediate] (/home/ross/3rd/reflex-platform/test.jsexe/all.js:6809:9)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)

via node and something similar via Safari.
this is using GHCJS <IP_ADDRESS> (GHC 8.6.5), as packaged with reflex-platform 58873162d542c14cf198e9711986c046b06a18da
when compiled via GHC with Data.Text (pack, unpack), instead the output is the expected
0.022000

using Safari devtools, the exception stack is:

https://github.com/ghcjs/ghcjs-base/blob/master/jsbits/jsstring.js#L64

arguments: Arguments (8)
  0 {d1: null, d2: null, f: function, m: 3}
  1 46
  2 48
  3 50
  4 50
  5 48
  6 48
  7 48
codePoint: NaN
index: 0
length: 8

https://github.com/ghcjs/ghcjs-base/blob/master/jsbits/jsstring.js#L234

pushing on the test case a bit, I found that reproduction is strangely fragile. this doesn't blow up:
{-# OPTIONS_GHC -Wall #-}
import Data.Text (pack, unpack)
import GHC.Float
import GHC.Show (intToDigit)
import System.IO (hFlush, stdout)

test :: String
test =
  let
    (is, e) = floatToDigits 10 (2.2e-2 :: Double)
    (ei,is') = roundTo 10 6 (replicate (-e) 0 ++ is)
    d:ds' = map intToDigit (if ei > 0 then is' else 0:is')
  in
    d : (if null ds' then "" else '.':ds')

main :: IO ()
main = do
  putStrLn . unpack . pack $ test
  --- hFlush stdout

and I see the correct output (0.022000) but it blows up if I uncomment the flush


Prevent error where invalid timestamp is passed to extrapolateTime()

UNTESTED!

I'll test

Works for me!


[ios][API 0.20.0]Assertion failure in -[RCTTouchEvent coalesceWithEvent:]

It has a crash when I used 2 CRTRootView and move them at the same time.

Hey, could you include a stack trace? That'd help a lot!

trunk/node_modules/react-native/React/Base/RCTTouchEvent.m:48
(lldb) bt

thread #1: tid = 0x1eb121, 0x0000000182a33f48 libobjc.A.dylibobjc_exception_throw, queue = 'com.apple.main-thread', stop reason = breakpoint 1.2   frame #0: 0x0000000182a33f48 libobjc.A.dylibobjc_exception_throw
frame #1: 0x00000001833c57d0 CoreFoundation+[NSException raise:format:arguments:] + 108   frame #2: 0x0000000183d38a08 Foundation-[NSAssertionHandler handleFailureInFunction:file:lineNumber:description:] + 88

frame #3: 0x000000010132c85c Circle-[RCTTouchEvent coalesceWithEvent:](self=0x000000012cfac250, _cmd="coalesceWithEvent:", newEvent=0x000000012e144b60) + 824 at RCTTouchEvent.m:48 frame #4: 0x0000000101319b74 Circle-[RCTEventDispatcher sendEvent:](self=0x000000012cebc8a0, _cmd="sendEvent:", event=0x000000012e144b60) + 300 at RCTEventDispatcher.m:152
frame #5: 0x00000001012e8eec Circle-[RCTTouchHandler _updateAndDispatchTouches:eventName:originatingTime:](self=0x000000012ceecfb0, _cmd="_updateAndDispatchTouches:eventName:originatingTime:", touches=1 object, eventName=@"touchMove", originatingTime=335132.74809829169) + 1344 at RCTTouchHandler.m:203 frame #6: 0x00000001012e92e4 Circle-[RCTTouchHandler touchesMoved:withEvent:](self=0x000000012ceecfb0, _cmd="touchesMoved:withEvent:", touches=1 object, event=0x000000012ce54d90) + 244 at RCTTouchHandler.m:269
frame #7: 0x00000001886353c8 UIKit-[UIGestureRecognizer _touchesMoved:withEvent:] + 96 frame #8: 0x00000001880e8600 UIKit-[UIWindow _sendGesturesForEvent:] + 588
frame #9: 0x00000001880e7e1c UIKit-[UIWindow sendEvent:] + 764 frame #10: 0x00000001880b84cc UIKit-[UIApplication sendEvent:] + 248
frame #11: 0x00000001880b6794 UIKit_UIApplicationHandleEventQueue + 5528 frame #12: 0x000000018337cefc CoreFoundationCFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION + 24
frame #13: 0x000000018337c990 CoreFoundation__CFRunLoopDoSources0 + 540 frame #14: 0x000000018337a690 CoreFoundation__CFRunLoopRun + 724
frame #15: 0x00000001832a9680 CoreFoundationCFRunLoopRunSpecific + 384 frame #16: 0x00000001847b8088 GraphicsServicesGSEventRunModal + 180
frame #17: 0x0000000188120d90 UIKitUIApplicationMain + 204 frame #18: 0x0000000100084c44 Circlemain(argc=1, argv=0x000000016fdafa80) + 124 at main.m:16
frame #19: 0x0000000182e4a8b8 libdyld.dylib`start + 4

https://github.com/mannyvergel/ReactNativeMVTableView

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/iosapi-0200assertion-failure-in-rcttouchevent-coalescewithevent
ProductPains helps the community prioritize the most important issues thanks to its voting feature.
It is easy to use - just login with GitHub. GitHub issues have voting too, nevertheless
Product Pains has been very useful in highlighting the top bugs and feature requests:
https://productpains.com/product/react-native?tab=top
Also, if this issue is a bug, please consider sending a pull request with a fix.
We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.


FIPS-10 Region Codes

One should add support for FIPS-10 Region codes. To implement this feature, we need:

[ ] A machine-readable source for FIPS-10 codes and FIPS-10 conversions into one of the already supported code maps (ISO 3166-2, ISO 3166-3, BCP47).
[ ] Downloader(s) and Processor(s) for the Setup subsystem, using the source(s).
[ ] Integration code for the Lingua\Locale\Attribute\Region class.
[ ] An adaption of the UML graphics for the Setup and Locale subsystems.
[ ] Unit tests
[ ] Documentation

The FIPS-10 codes are included in the ICU and the OKFN sources, which are already available as downloaders.


Automatically selected location is appearing to candidate on a submitted application

Context
We made some mistakes when implementing the conditional location information in the Candidate & Provider Interfaces.
Changes proposed in this pull request
Apply

Page
Auto Selected
User Selected

Details of Offer

Are you sure you want to decline this offer?

Manage

Page
Auto Selected
User Selected

"Offer"

Guidance to review

Things to check

[ ] If the code removes any existing feature flags, a data migration has also been added to delete the entry from the database
[ ] This code does not rely on migrations in the same Pull Request
[ ] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[ ] If this code adds a column to the DB, decide whether it needs to be in analytics yml file or analytics blocklist
[ ] API release notes have been updated if necessary
[ ] If it adds a significant user-facing change, is it documented in the CHANGELOG?
[ ] Required environment variables have been updated added to the Azure KeyVault
[ ] Inform data insights team due to database changes
[ ] Make sure all information from the Trello card is in here
[x] Rebased main
[x] Cleaned commit history
[x] Tested by running locally
[x] Add PR link to Trello card

 Automatically selected Location is appearing to Candidate on a submitted application


feat(react-i18n): upgrade to React 16.4 and use context API

Upgrade to React 16.4 and use context API
Use react 16.4 Context API. Does not include breaking change except react dependency.
TODO

[x] upgrade to react 16.4
[x] use the new context API
[x] remove dead code from debug folder
[x] tests
[ ] doc

😍😍😍

There was also a lot of dead code to remove 😅


AudioKit.start() crash

Hi there,
Getting a lot of crashes when I call AudioKit.start() upon opening my app. The weird thing is that it doesn't crash every single time, most of the time it appears to be working perfectly. It's affecting 6% of my app's users and I can't replicate it. It's spread out pretty evenly across all devices and iOS versions so it's not a device specific or OS specific thing.
Here is the file in question in the project (you can clone and run the project)
The crash is occurring at line 32 when I call AudioKit.start():
    self.mixer = AKMixer(tuner.silence)

    bank = AKOscillatorBank(waveform: AKTable(.triangle), attackDuration: 0.06, releaseDuration: 0.06)
    mixer.connect(bank)

    AudioKit.output = mixer
    try! AKSettings.session.overrideOutputAudioPort(AVAudioSessionPortOverride.speaker)
    AudioKit.start() --- CRASH

This method is only called once when the user opens the app.
Stack trace:
com.danielkuntz.pitchtuner_issue_9_crash_ebfaf35c1f82427d84ac5fb7086e6228_edee4442519f4755ba14518d373a3033_0_v1 (1).txt

I have been getting some rather similar crash reports from users of my own AudioKit app. I wonder if this might be explained by audio session types and the user's particular configuration.

@dkun7944 We saw that Apple featured your app in the "New apps we love" category.  Congratulations!  I hope it translates to a lot of downloads.  I do believe these bugs are related to session management.  What version of AudioKit is your app using?  If its on 3.5.2 I'd definitely recommend updating to 3.5.3.

@aure Thanks! I believe I'm on 3.5.2. I'll try updating to 3.5.3 and let you know if that clears things up.

@megastep Glad to know I'm not the only one with this issue. I'll let you know what happens after updating to 3.5.3

FYI my own crashes look like this: http://crashes.to/s/952f9695c5a
I see that you are also using Crashlytics so maybe you can compare with the error codes from the exceptions.

@megastep I'm getting error code 561017449.
http://crashes.to/s/f3a95c95692

@dkun7944, congrats on the Apple love. And, it was super cool of you to upload your code to Github so others can learn from it. You should join the AudioKit slack. We're cheering you on there.

@dkun7944 That's code for AVAudioSessionErrorInsufficientPriority, FYI

@megastep thanks. That's a step in the right direction.
@swiftcodex Thanks! I learned a lot from open sourced iPhone apps in my early programming days so I thought I'd give back. How do I join the slack channel?

@dkun7944 Just drop me an email and I'll add you to our slack group.

@aure Done.
In regards to the error code, I understand it means that the category AudioKit is trying to set is already being used by another app.
How should I work around this since the code for setting the category is being called within the AudioKit framework? Should I set the category manually at first in a try-catch block and then do AudioKit.start()?

It would be awesome if you could, for the purposes of the good of all, build your changes into the framework.  It would mean not useing the framework file and instead building with an embedded AudioKit for iOS.xcodeproj.  Once you got it working, then it could be compiled and you could go back to using the framework.
If it makes sense, to close this issue and start a new one, please do.  Long issue-based discussions that go off the initial issue are not so good for new users looking through the archives for previously solved issues.  Thanks!

Also "AudioKit.start() crash" is just a scary title. :)

@aure Ok, sounds good.


[Feature 5] ML Parameters

Menu at the top in which the user can select to play, restart, the epoch step, and the parameters (Learning rate, activation, regularization...etc)

/close by https://github.com/CSC207-2023Y-UofT/ai-playground/pull/42


Best way to importing existing stripe customers & products to firebase?

Question
Is there a suggested method of importing an existing set of stripe customers to an empty firestore?
What about importing products and prices?
Wondering how much will be taken care of via webhook updates if I change to the live keys where my customers are.

You would have to write a script that adds (or you manually add) the necessary mapping in the customers collection:

Path: /customers/{uid}
Data:

{ stripeId: 'cus_xxx' }

As long as this information is present, the extension will be able to perform all actions for authenticated users with the correct Stripe customer ID.


is_valid and is_simple are True on some self-intersecting Polygons, LineStrings, and LinearRings

Expected behavior and actual behavior
Expected behavior: All self-intersecting objects should have is_valid and is_simple return False. In particular, the documentation for is_simple states that it should:

Returns True if the feature does not cross itself.

Similarly, the documentation for is_valid states:

A valid LinearRing may not cross itself or touch itself at a single point. A valid Polygon may not possess any overlapping exterior or interior rings. A valid MultiPolygon may not collect any overlapping polygons. Operations on invalid features may fail.

Actual behavior: For some geometries with repeated vertices, is_valid and is_simple return True. However, geometries with repeated consecutive vertices, by definition, intersect themselves as they overlap at one point. This behavior violates user expectations and should either be documented or changed.
Steps to reproduce the problem

In [1]: from shapely.geometry import Polygon, MultiPolygon, LineString, LinearRing

In [2]: Polygon([(0,0), (0,0), (1,0), (1,1)]).is_simple
Out[2]: True

In [3]: Polygon([(0,0), (0,0), (1,0), (1,1)]).is_simple
Out[3]: True

In [4]: LineString([(0,0), (0,0), (1,0), (1,1)]).is_valid
Out[4]: True

In [5]: LineString([(0,0), (0,0), (1,0), (1,1)]).is_simple
Out[5]: True

In [6]: LinearRing([(0,0), (0,0), (1,0), (1,1)]).is_valid
Out[6]: True

In [7]: LinearRing([(0,0), (0,0), (1,0), (1,1)]).is_simple
Out[7]: True

In [8]: LinearRing([(0,0), (1,0), (1,1), (0,0)]).is_simple
Out[8]: True

Notes and thoughts
This sounds like an upstream issue -- I'm not sure. There may be other circumstances where is_simple and is_valid do not behave as expected (e.g. on other shapes like MultiPolygons, etc. or on other geometries). The vertices do not have to be consecutive for unexpected behavior to occur (see last example).
Slight tangent -- Shapely is an awesome, important library -- keep up the good work!
Operating system
Pop!_OS 20.04 LTS (Ubuntu Focal)
Shapely version and provenance
Shapely-1.7.1-1-cp39-cp39-manylinux1_x86_64 installed from PyPI using pip on Python 3.9.4

Repeated points are considered valid and simple in JTS/GEOS by design.   This behaviour will not be changed.

@dr-jts to clarify, do you mean all repeated points (the OpenGIS spec only refers to repeated endpoints). If so, I'm happy to close this issue,

@dr-jts to clarify, do you mean all repeated points (the OpenGIS spec only refers to repeated endpoints). If so, I'm happy to close this issue,

Yes, I mean any and all repeated points, whether endpoints or points internal to a line.


Cargo gets lost in the root of a Linux filesystem

Problem
cargo doc will mysteriously hang when run from the root of a Linux filesystem. The problem is that the fingerprinting code tries to find the file with the newest mtime by walking the entire filesystem. Unfortunately there are some paths that are infinite black holes.
An example is /sys/block. This tree has a complex set of cyclical symbolic links. Cargo follows a cycle until it reaches ELOOP (too many symbolic links), and then starts following another cycle. There is a permutation of cycles, and so cargo essentially gets lost forever.
I believe this also affects "old" style build scripts which also use LocalFingerprint::Precalculated.
Steps

Start a bash prompt with the rust:latest docker image.
In the root directory, run cargo init --name foo
Run cargo doc. This step will never finish.

Possible Solution(s)
Uncertain.  A few ideas:

Use walkdir for walking, which has symbolic link cycle detection.
Check for special devices and ignore them.
Detect running in root, and return an error.
Set an upper bound on the number of files the walker will visit before it gives up.

As a side note, another problem with the implementation is that it collects the path of every file it visits in memory. This is done with the assumption that PathSource::list_files will return a reasonable number of files. However, the mtime computation code doesn't need a full list, but instead just needs to know which file has the newest mtime. Perhaps that code could be changed to take a callback instead of accumulating all the files in memory.
Notes
Output of cargo version:
cargo 1.54.0-nightly (e931e4796 2021-05-24)
Linux 4.19.121

@ehuss it seems strange to me that cargo looks at all files in the first place - I can look into adding --emit=dep-info to rustdoc if that would help.

Perhaps that code could be changed to take a callback instead of accumulating all the files in memory.

I prototyped an iterative version, but I don't feel comfortable about its complexity. 😞
Is there anything can do to make it simpler?

Faced the same issue during cargo build when I copy sources into root in docker. I see endless wall of warnings with the original repro.
Workaround for me was to create directory and copy sources there instead of root.
Will it be better to terminate with an error instead of being stuck with warnings?


unlock is missing?

https://github.com/go-ocf/go-coap/blob/d0a8af7f0e1459d17fbcd6e0c7a3f45d02fbdc78/server.go#L652

Thx for a report bug -> it is fixed by https://github.com/go-ocf/go-coap/pull/62


Edited Video Section (added border to videos and labels to videos)

Is your feature request related to a problem? Please describe.

assign this task to me
Describe the solution you'd like.
No response
Describe alternatives you've considered.
No response
Additional context.
No response
Show us the magic with screenshots
No response
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I want to work on this issue. (optional)

can you do shadow instead of giving border on hover? will assign to you after confirmation.

Yes, I've implemented the shadow effect instead of giving a border on hover. Here's a screenshot showcasing the updated UI.

Please assign me this issue


Added EnableVgwRoutePropagation to network role

Also cleaned the role with current errors. A typo and some reference to
non existing permissions in guardduty.

/assign @rogbas

Closing in favor of #353