systemd: 253.5 -> 254

Description of changes
systemd-repart relies on cryptsetup to perform LUKS operations,
therefore, it's not suitable for systemdMinimal.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

TODO:

[ ] investigate hibernate test
[ ] run it on actual machine:

[ ] laptop
[ ] server

[ ] reflow patches properly
[ ] more tests?

We will also need to mention in the changelog that boot.resumeDevice now must be set to use hibernation when not EFI booting.

These changes are also required:
diff --git a/nixos/modules/system/boot/systemd/initrd.nix b/nixos/modules/system/boot/systemd/initrd.nix
index 3f40a5b2dfa0..7476a4f9040a 100644
--- a/nixos/modules/system/boot/systemd/initrd.nix
+++ b/nixos/modules/system/boot/systemd/initrd.nix
@@ -57,7 +57,6 @@ let
     "systemd-ask-password-console.service"
<EMAIL_ADDRESS>     "systemd-halt.service"
-<EMAIL_ADDRESS>     "systemd-journald-audit.socket"
     "systemd-journald-dev-log.socket"
     "systemd-journald.service"
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 5e28c064212e..2787a5d0d970 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -28481,6 +28481,7 @@ with pkgs;
     withTpm2Tss = false;
     withUserDb = false;
     withUkify = false;
+    withBootloader = false;
   };
   systemdStage1 = systemdMinimal.override {
     pname = "systemd-stage-1";

These changes are also required:
diff --git a/nixos/modules/system/boot/systemd/initrd.nix b/nixos/modules/system/boot/systemd/initrd.nix
index 3f40a5b2dfa0..7476a4f9040a 100644
--- a/nixos/modules/system/boot/systemd/initrd.nix
+++ b/nixos/modules/system/boot/systemd/initrd.nix
@@ -57,7 +57,6 @@ let
     "systemd-ask-password-console.service"
<EMAIL_ADDRESS>     "systemd-halt.service"
-<EMAIL_ADDRESS>     "systemd-journald-audit.socket"
     "systemd-journald-dev-log.socket"
     "systemd-journald.service"
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 19226904e0e0..6ca73349ccb4 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -28515,6 +28515,7 @@ with pkgs;
     withTpm2Tss = false;
     withUserDb = false;
     withUkify = false;
+    withBootloader = false;
   };
   systemdStage1 = systemdMinimal.override {
     pname = "systemd-stage-1";

Applied. @flokli Patches reflowed according to the guidelines, let me know if I messed up something.

$ sudo env NIXOS_INSTALL_BOOTLOADER=1 /nix/var/nix/profiles/system/bin/switch-to-configuration boot
Copied "/nix/store/9qpbi9wp8rvy6mjlv06f6c7nxwdp1938-systemd-254/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/systemd-bootx64.efi".
Copied "/nix/store/9qpbi9wp8rvy6mjlv06f6c7nxwdp1938-systemd-254/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/BOOTX64.EFI".
⚠ Mount point '/boot' which backs the random seed file is world accessible, which is a security hole! ⚠
⚠ Random seed file '/boot/loader/random-seed' is world accessible, which is a security hole! ⚠
Random seed file /boot/loader/random-seed successfully refreshed (32 bytes).
Created EFI boot entry "Linux Boot Manager".

Looks like we've got a new warning to account for in systemd-boot-builder.py

Also, I'm not sure why yet, but it appears the 254 update has caused /dev/zfs to have 0600 permissions instead of the usual 0666, so e.g. regular users can't zpool status now.

@nlewo I'm seeing osqueryd issues with systemd v254:
août 04 00:47:07 core01 osqueryd[973941]: osqueryd started [version=5.5.1]
août 04 00:47:07 core01 osqueryd[973943]: E0804 00:47:07.948038 973943 init.cpp:700] An error occurred during extension manager startup: Extension socket directory missing: /var/osquery/osquery.em
août 04 00:47:07 core01 osqueryd[973943]: E0804 00:47:07.949370 973943 shutdown.cpp:79] Cannot activate filesystem logger plugin: Could not create file: /var/log/osquery/osqueryd.results.log
août 04 00:47:07 core01 osqueryd[973943]: Cannot activate filesystem logger plugin: Could not create file: /var/log/osquery/osqueryd.results.log
août 04 00:47:10 core01 osqueryd[973941]: E0804 00:47:10.672145 973942 shutdown.cpp:79] Worker returned exit status
août 04 00:47:10 core01 osqueryd[973941]: Worker returned exit status
août 04 00:47:10 core01 systemd[1]: osqueryd.service: Main process exited, code=exited, status=78/CONFIG

Please take a look if possible.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/breaking-changes-announcement-for-unstable/17574/29

Additional bug during a ZFS unlock in systemd stage 1: keymap went back to us instead of the specified fr-latin1.

So here are the things I see needing to still be done:

We need to address the change in hibernation behavior from systemd. Now, if you don't use EFI, systemd will refuse to hibernate unless you have a resume= cmdline parameter. If you do use EFI, systemd will set some EFI variables, expecting them to be used for resume (unless resume= is present), which we don't respect. We could use these instead of the logic we use to auto-detect the hibernate image device.
We need a solution to https://github.com/systemd/systemd/issues/28653. I know that issue is about ZFS specifically, so it seems like we could add a simple workaround, but it's actually a much broader problem, so we should fix it completely. We can either revert https://github.com/systemd/systemd/commit/b42482af904ae0b94a6e4501ec595448f0ba1c06, or we can backport their fix https://github.com/systemd/systemd/pull/28784#issuecomment-1685238230
We have an issue with vconsole not working right with systemd stage 1. This is because the udev rules now start systemd-vconsole-setup.service instead of whatever they did before. Luckily all this means is that now we have to add that service to boot.initrd.systemd.additionalUpstreamUnits in nixos/modules/config/console.nix.
Finally, we still have the warning about globally readable /boot to address. I'm not entirely sure what to do here. Maybe change some permissions in systemd-boot-builder.py?

There's also some majorly quirky nuance with respect to systemd stage 1 and the new hibernation situation. So far, we've simply not supported not setting boot.resumeDevice (a.k.a. resume=) when using hibernation and systemd stage 1. We could simply continue this practice. But if we want to support systemd's expected behavior around these new EFI variables, we need to use an EFI-enabled systemd build in stage 1, and we need to include the efivarfs kernel module in initrd.
This does create a minor snag though. We currently have systemd-gpt-auto-generator in the initrd, and once efivarfs is there and an EFI-enabled build is used, that generator will take over the rootfs unless you have root= on the cmdline or rd.systemd.gpt_auto=false. This is a self imposed snag though; we can just remove the generator, since we don't actually use it.
By eliminating that snag and enabling EFI in systemd stage 1, we can drop the requirement for boot.resumeDevice with systemd stage 1 as long as the user has EFI! Which, now that systemd mandates the same anyway, is true for scripted initrd as well, bringing them to parity.
I would like to have scripted initrd check those EFI variables and use the resume= parameter rather than the way it currently does everything, but we can save that for later.

There's also some majorly quirky nuance with respect to systemd stage 1 and the new hibernation situation. So far, we've simply not supported not setting boot.resumeDevice (a.k.a. resume=) when using hibernation and systemd stage 1. We could simply continue this practice. But if we want to support systemd's expected behavior around these new EFI variables, we need to use an EFI-enabled systemd build in stage 1, and we need to include the efivarfs kernel module in initrd.
This does create a minor snag though. We currently have systemd-gpt-auto-generator in the initrd, and once efivarfs is there and an EFI-enabled build is used, that generator will take over the rootfs unless you have root= on the cmdline or rd.systemd.gpt_auto=false. This is a self imposed snag though; we can just remove the generator, since we don't actually use it.
By eliminating that snag and enabling EFI in systemd stage 1, we can drop the requirement for boot.resumeDevice with systemd stage 1 as long as the user has EFI! Which, now that systemd mandates the same anyway, is true for scripted initrd as well, bringing them to parity.
I would like to have scripted initrd check those EFI variables and use the resume= parameter rather than the way it currently does everything, but we can save that for later.

I'm not sure if I understand well your proposal.
Can you clarify if you propose to:
(1) Use an EFI enabled build for systemd (which we do in this PR)
(2) Add efivarfs in initrd (which we don't)
(3) Set rd.systemd_gpt_auto=false always or not?
(4) Dropping boot.resumeDevice need?
(5) Changing scripted initrd to adhere to the new logic you propose at the end in another PR.
Am I right?

So here are the things I see needing to still be done:

We need to address the change in hibernation behavior from systemd. Now, if you don't use EFI, systemd will refuse to hibernate unless you have a resume= cmdline parameter. If you do use EFI, systemd will set some EFI variables, expecting them to be used for resume (unless resume= is present), which we don't respect. We could use these instead of the logic we use to auto-detect the hibernate image device.

I would love if you can push a commit for this one, I am afraid I don't have all the context you had on this one.

We need a solution to permissions issues with openzfs systemd/systemd#28653. I know that issue is about ZFS specifically, so it seems like we could add a simple workaround, but it's actually a much broader problem, so we should fix it completely. We can either revert systemd/systemd@b42482a, or we can backport their fix [Udev vs tmpfiles take 3 systemd/systemd#28784 (comment)]
(https://github.com/systemd/systemd/pull/28784#issuecomment-1685238230)

Implemented.

We have an issue with vconsole not working right with systemd stage 1. This is because the udev rules now start systemd-vconsole-setup.service instead of whatever they did before. Luckily all this means is that now we have to add that service to boot.initrd.systemd.additionalUpstreamUnits in nixos/modules/config/console.nix.

Implemented.

Finally, we still have the warning about globally readable /boot to address. I'm not entirely sure what to do here. Maybe change some permissions in systemd-boot-builder.py?

I guess we can fixup the permissions, but that won't work for, e.g. lanzaboote. Maybe this should be a user activation script for fixup, but heh.

We need a solution to permissions issues with openzfs systemd/systemd#28653. I know that issue is about ZFS specifically, so it seems like we could add a simple workaround, but it's actually a much broader problem, so we should fix it completely. We can either revert systemd/systemd@b42482a, or we can backport their fix [Udev vs tmpfiles take 3 Udev vs tmpfiles take 3 systemd/systemd#28784 (comment)]
(Udev vs tmpfiles take 3 systemd/systemd#28784 (comment))

Implemented.

I believe backporting their fix requires adding a new unit systemd-tmpfiles-setup-dev-early.service to the default upstream units.

I'm not sure if I understand well your proposal.
Can you clarify if you propose to:
(1) Use an EFI enabled build for systemd (which we do in this PR)
(2) Add efivarfs in initrd (which we don't)
(3) Set rd.systemd_gpt_auto=false always or not?
(4) Dropping boot.resumeDevice need?
(5) Changing scripted initrd to adhere to the new logic you propose at the end in another PR.
Am I right?

We do not currently use an EFI enabled build for systemd in stage 1.
Right
No, I think it would work to get rid of systemd-gpt-auto-generator instead, because we don't use that and actually actively don't want it doing anything anyway.
Sort of. We would be adding that as a requirement for non-EFI systems, because that's now required by systemd. But with the previous points, we can remove it as a requirement for systemd stage 1 on EFI systems.
Yea, I think that would be another PR.

Alternatively we can just take no action now and make improvements later. No matter what we do, we have to tighten the constraint and say that non-EFI users must have boot.resumeDevice set for hibernation to work. Otherwise, everything else is status quo.

because we don't use that and actually actively don't want it doing anything anyway.

I do use it and we should at least add an option to keep it

Can someone test the NixOS containers? I do not use this feature and do not really care much about it, but it would be nice to verify we are not breaking it.

@GrahamcOfBorg test containers

@GrahamcOfBorg test containers-bridge containers-custom-pkgs containers-ephemeral containers-extra_veth containers-hosts containers-imperative containers-ip containers-macvlans containers-names containers-nested containers-physical_interfaces containers-portforward containers-reloadable containers-restart_networking containers-tmpfs containers-unified-hierarchy

Oh of course ofborg won't work as this targets staging.. sigh. not sure how to cancel these jobs now

@cole-h Did we do a big mistake? Let us know if we can cancel them, apologies for the inconveniences.

I wouldn't worry about it :)
(Mostly because there's no way to cancel them :)

@nikstur How are you using it? Without patching nixpkgs, NixOS currently generates errors if you try to configure gpt-auto for the root fs. If you're not talking about the root fs, then we're talking about two different things.

If you're not talking about the root fs, then we're talking about two different things.

Yes, you're right. Although I'd like to potentially do that (although as you said we'd need to remove the error when you don't set fileSystems."/" first.)

Whats the status of this

Whats the status of this

As you can see, some points are left to be solved, anyone is free to PR them, untangle them. We coordinate in the NixOS systemd Matrix room.
As you can also see on my profile, I'm on vacations for now :).

I fixed this PR up to use 254.3 and fixed the conflicts here: https://github.com/nikstur/nixpkgs/tree/systemd-254
As far as I understand the comments in this history, we should be good to go.
@RaitoBezarius please let me know how you want me to incorporate these changes into your branch.
We have these action items on our todo list:

Improve the hibernation/resume situation
Improve the permission situation on /boot

systemd on staging-next is broken on armv7l-linux https://hydra.armv7l.xyz/build/17881/nixlog/2
[662/1780] Generating src/boot/efi/addonarm.efi.stub with a custom command
FAILED: src/boot/efi/addonarm.efi.stub
/build/source/tools/elf2efi.py --version-major=254 --version-minor=0 --efi-major=1 --efi-minor=1 --subsystem=10 --minimum-sections=15 src/boot/efi/addonarm.elf.stub src/boot/efi/addonarm.efi.stub
Traceback (most recent call last):
  File "/build/source/tools/elf2efi.py", line 601, in <module>
    main()
  File "/build/source/tools/elf2efi.py", line 597, in main
    elf2efi(parser.parse_args())
  File "/build/source/tools/elf2efi.py", line 499, in elf2efi
    pe_reloc_s = convert_elf_relocations(elf, opt, sections)
  File "/build/source/tools/elf2efi.py", line 388, in convert_elf_relocations
    raise RuntimeError("ELF .dynamic section is missing.")
RuntimeError: ELF .dynamic section is missing.
[663/1780] Linking target src/boot/efi/linuxarm.elf.stub
[664/1780] Compiling C object src/import/libimport-common.a.p/import-common.c.o
[665/1780] Compiling C object src/boot/efi/systemd-bootarm.elf.p/boot.c.o
ninja: build stopped: subcommand failed.

Cross to riscv64 is broken for the same reason:
FAILED: src/boot/efi/addonriscv64.efi.stub
/build/source/tools/elf2efi.py --version-major=254 --version-minor=0 --efi-major=1 --efi-minor=1 --subsystem=10 --minimum-sections=15 src/boot/efi/addonriscv64.elf.stub src/boot/efi/addonriscv64.efi.stub
Traceback (most recent call last):
  File "/build/source/tools/elf2efi.py", line 601, in <module>
    main()
  File "/build/source/tools/elf2efi.py", line 597, in main
    elf2efi(parser.parse_args())
  File "/build/source/tools/elf2efi.py", line 499, in elf2efi
    pe_reloc_s = convert_elf_relocations(elf, opt, sections)
  File "/build/source/tools/elf2efi.py", line 388, in convert_elf_relocations
    raise RuntimeError("ELF .dynamic section is missing.")
RuntimeError: ELF .dynamic section is missing.

Is this an upstream error or something that we have messed up?

Is this an upstream error or something that we have messed up?

I think it's an upstream issue. Likely related: https://github.com/systemd/systemd/issues/27821, https://github.com/systemd/systemd/pull/26641

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/breaking-changes-announcement-for-unstable/17574/33

Arch Linux ARM builds addonarm.efi.stub successfully so I don't think this is an upstream issue.

The root cause of this bug appears to be that gcc doesn't support the -static-pie argument on 32-bit ARM or RISC-V. The argument is accepted, but doesn't actually do anything. On x86_64, -static-pie causes -static -pie --no-dynamic-linker -z text to be passed to the linker (with slight variations on the other supported architectures).
In systemd, they noticed something wasn't quite right with -static-pie, and therefore they manually pass most of these arguments to the linker, see:
https://github.com/systemd/systemd/blob/cb88da825460c410139a9a3041af849ee2d01d5b/src/boot/efi/meson.build#L156-L158
Notably though, they don't pass -pie. The reason for this is that pretty much every distribution except for nixpkgs builds gcc with --enable-default-pie, so -pie is passed to the linker even without -static-pie.
This raises several questions about how this should be fixed:

If manually passing -pie --no-dynamic-linker -z text is enough to make static PIE work, why doesn't gcc support -static-pie on more architectures?
Should systemd pass -Wl,-pie to work around this gcc issue?
Why don't we build gcc with --enable-default-pie in nixpkgs? We have the pie hardening option that is disabled by default (except on musl) because it apparently caused problems. Would these same problems occur with --enable-default-pie?

It looks like -static-pie support on the gcc side is as simple as passing those flags to the linker. Some more work is probably needed in glibc to support it, but that is not relevant to systemd.
In fact, someone submitted a patch to gcc to add -static-pie support on ARM, but it was ignored<EMAIL_ADDRESS>


[v4] navbar order is different from _meta

investigate why here order is different

Hello, I've fixed this issue!
This is what I've done:
I investigated the issue with the sidebar order in the Nextra docs theme. The problem appears to be in the Menu component within the sidebar.tsx file. To fix this, I've modified the Menu component to respect the order of items as they are passed in the directories prop.
Here's what I changed:

Added a sort function to sort the directories based on their sort property if available.
Modified the Menu component to use this sorting function before rendering the items.

This change ensures that the sidebar items are displayed in the correct order as specified in the configuration.
I also created pull request: https://github.com/shuding/nextra/pull/3624

[!CAUTION]
Disclaimer: This fix was created by Latta AI and you should never merge before you check the correctness of generated code!

This bug was fixed for free by Latta AI - https://latta.ai/ourmission


Add RouteTokenTransformerConvention documentation

@JameNK ... If you don't mind, it's faster if I make a quick grammar pass on this with a direct edit over typing nit comments. Cool with you?

all good in the hood

Thanks ... nits only. Check tho to make sure you agree with the changes.

@JamesNK 	 can you add the code for SlugifyParameterTransformer and please check my edits.

Looks good. Ship it

There's an 85 char limit on code lines, so you should break the long one on the last commit.

G2G. I'll let you guys merge it in case there is any docs ceremony I don't know about 😬 (some sort of pagan sacrifice I assume)

Sometimes, we sacrifice a mouse from a certain well-branded manufacturer to the :pray: server gods :pray:, but this isn't one of those scenarios. :smile:


Session API slow to upload a chunk

I have made a repository to demonstrate.
Summary:
I have a readable stream that goes through a transformation stream (which buffers up data, /src/transform.js) and then passes it to dropbox upload stream (/src/dropbox.js) that runs the session dropbox API internaly.
Sending 8 MB chunks which takes about ~45s to upload.
Repository: https://github.com/AlesMenzel/dropbox-session-test
IMPORTANT prerequisites:

Include a file to be read into a stream (see index.js - './samples/sample-10MB.mp4')
Please set DROPBOX_TOKEN in the /src/dropbox.js
Please set your DEBUG env to stream-to-dropbox:* to see whats happening.

Thanks for putting this together!
I just ran that with a 10 MB sample file at ./samples/sample-10MB.mp4 though, and it consistently ran in about 8 seconds for me. That being the case, it sounds like variation in network connection is likely the cause here.
Your connection speed to Dropbox depends on the routing you get between your ISP and our servers, and may be slower than your ISP's rated speeds.
Sometimes resetting or retrying your connection gets you a different route and better speeds, but that is outside of our control. Some ISPs also throttle sustained connections so if you see an initial high connection speed followed by lower speeds, that could be the reason.
That's not to see there isn't room for improvement on our side of course; there certainly is, and I've sent this along as feedback to improve performance on our side. Currently though, it seems this is working as expected, albeit subject to the network connection, which is outside our control.

@greg-db
Thank you for your answer. I am going to try from a different device.
One more question though about the Session Upload API. Can I be uploading multiple chunks at once? As there is the cursor: offset property that corresponds to total uploaded size.
Example:
Session Start -> upload 8 MB -> offset 8 MB
Session Append -> upload 8 MB -> offset 16 MB
Session Append -> upload 8 MB -> offset 24 MB
Session Finish
As opposed to:
Session Start -> upload 8 MB -> offset 8 MB
Run in parallel
->  Session Append -> upload 8 MB -> offset 16 MB
->  Session Append -> upload 8 MB -> offset 24 MB << this might finish before the above
Session Finish

Similarly Twitter Session API lets you do that: https://dev.twitter.com/rest/reference/post/media/upload-append.html by specifiing chunk index instead of length.

Anyways seems like i will have to change a provider as paying for Dropbox Plus is not worth for my use case as Dropbox API is just too slow (even your runs prove my point - 10MB/8s is outrageous in 2017).
Comparison with Google Drive:
Upload:
Dropbox: 1.5GB file = ~ 20min
Google Drive: 1.5Gb file = ~ 3min

Thanks for the feedback!
And no, you can't upload multiple chunks for a single upload session in parallel. They have to be uploaded sequentially. (You can run multiple upload sessions, i.e., for different files, at the same time though. Check out the data ingress guide for information on how to do that.)


cmd/compile/internal/gc: esc.go -m=2 reports invalid sink pos for some nodes

cmd/compile/internal/gc: esc.go -m=2 reports invalid sink pos for some nodes
Given this code (extracted from $GOROOT/test/escape_param.go):
package example

var sink interface{}

func param1(p1, p2 *int) (*int, *int) {
	return p1, p2
}

func caller1() {
	i := 0
	j := 0
	sink, _ = param1(&i, &j)
}

Execute the command $ go tool compile -m=2 example.go.
The output on tip is (important bits are in bold):

example.go:5:6: can inline param1 as: func(*int, *int) (*int, *int) { return p1, p2 }
example.go:9:6: can inline caller1 as: func() { i := 0; j := 0; sink, _ = param1(&i, &j) }
example.go:12:18: inlining call to param1 func(*int, *int) (*int, *int) { return p1, p2 }
example.go:5:13: leaking param: p1 to result ~r2 level=0
example.go:5:13: 	from ~r2 (return) at example.go:6:2
example.go:5:17: leaking param: p2 to result ~r3 level=0
example.go:5:17: 	from ~r3 (return) at example.go:6:2
example.go:12:10: (*int)(~r2) escapes to heap
example.go:12:10: 	from sink (assign-pair) at example.go:3:5
example.go:12:19: &i escapes to heap
example.go:12:19: 	from p1 (assign-pair) at example.go:12:18
example.go:12:19: 	from ~r2 (assign-pair) at example.go:12:18
example.go:12:19: 	from (*int)(~r2) (interface-converted) at example.go:12:10
example.go:12:19: 	from sink (assign-pair) at example.go:3:5
example.go:10:2: moved to heap: i
example.go:12:23: caller1 &j does not escape

Note that the assignment target is reported to be sink (correct) and the location is example.go:3:5 (not correct). During flow graph construction, it's clear that the location is example.go:12:10.
Before describing what's the reason behind that, we can compare OAS2 handling with OAS.
Here is slightly modified example:
package example2

var sink interface{}

func caller1() {
	i := 0
	sink = &i
}

The output for -m2 is:

example2.go:5:6: can inline caller1 as: func() { i := 0; sink = &i }
example2.go:7:7: &i escapes to heap
example2.go:7:7: 	from sink (assigned to top level variable) at example2.go:7:7
example2.go:7:9: &i escapes to heap
example2.go:7:9: 	from &i (interface-converted) at example2.go:7:7
example2.go:7:9: 	from sink (assigned to top level variable) at example2.go:7:7
example2.go:6:2: moved to heap: i

Nevermind that it prints &i escapes to heap twice, that's not relevant for this issue.
What is important is that it points to example2.go:7:7, the location where the assignment actually happens, not to the line where sink is declared.
Would be good to be consistent here and print assignment location for OAS too.
The code that causes problem is this:
if n.List.Len() == n.Rlist.Len() {
	rs := n.Rlist.Slice()
	for i, n := range n.List.Slice() {
		e.escassignWhyWhere(n, rs[i], "assign-pair", n)
	}
}

Note that n is OAS2, and inner n is an element from LHS and it shadows outer n.
So, instead of marking "assign-pair" with assignment position, we annotate it with LHS[i] position.
90% sure this is wrong and that error probably comes from copy/paste error (other e.escassignWhyWhere calls use n parameter as well, but there is no for loop that shadows it).
So, the proposed solution is to pass proper where argument to escassignWhyWhere.
Fixed output is:

example.go:5:6: can inline param1 as: func(*int, *int) (*int, *int) { return p1, p2 }
example.go:9:6: can inline caller1 as: func() { i := 0; j := 0; sink, _ = param1(&i, &j) }
example.go:12:18: inlining call to param1 func(*int, *int) (*int, *int) { return p1, p2 }
example.go:5:13: leaking param: p1 to result ~r2 level=0
example.go:5:13: 	from ~r2 (return) at example.go:6:2
example.go:5:17: leaking param: p2 to result ~r3 level=0
example.go:5:17: 	from ~r3 (return) at example.go:6:2
example.go:12:10: (*int)(~r2) escapes to heap
example.go:12:10: 	from sink (assign-pair) at example.go:12:10
example.go:12:19: &i escapes to heap
example.go:12:19: 	from p1 (assign-pair) at example.go:12:18
example.go:12:19: 	from ~r2 (assign-pair) at example.go:12:18
example.go:12:19: 	from (*int)(~r2) (interface-converted) at example.go:12:10
example.go:12:19: 	from sink (assign-pair) at example.go:12:10
example.go:10:2: moved to heap: i
example.go:12:23: caller1 &j does not escape

Note that now it points to example.go:12:10.
I find old output confusing and inconsistent, hence I'm proposing fixing it.

CL with a fix will be submitted soon.

Moved to 1.12; this is not critical.


[Post release v0.5.0] Remove serviceType

Why are these changes needed?
For KubeRay, the configuration YAML files (e.g. files in ray-operator/config/samples) in the master branch must be compatible with:

Nightly KubeRay operator
Latest release of KubeRay operator (i.e. v0.5.0 at this moment)

The field serviceType is a required field before v0.5.0, but becomes optional from v0.5.0. Hence, when v0.5.0 becomes the "latest release of KubeRay operator", we can remove it from sample YAML files.
We only keep the optional fields in ray-cluster.complete.yaml, ray-cluster.complete.large.yaml, and RayCluster chart's values.yaml.
Related issue number
Closes #856

Checks

[ ] I've made sure the tests are passing.
Testing Strategy

[x] Unit tests
[x] Manual tests
[ ] This PR is not tested :(

KubeRay operator tests (i.e. xxxx.go)
# path: kuberay/ray-operator
make test

Test RayCluster YAML files
RAY_IMAGE=rayproject/ray:2.3.0 OPERATOR_IMAGE=kuberay/operator:v0.5.0 python3 tests/test_sample_raycluster_yamls.py 2>&1 | tee lo

Python client: KubeRay CI

The sample YAML tests failed because the CI tested with both the KubeRay release v0.4.0 and the nightly version. In the v0.4.0 CRDs, the serviceType field was required, which caused an error to be reported. The commit https://github.com/ray-project/kuberay/pull/1013/commits/94e643fcc1a86c97140487e400e1626047215625 fixes the issue.
cc @architkulkarni


fix: upgrade expo config plugins

replaces https://github.com/stripe/stripe-react-native/pull/935

hey @charliecruzan-stripe! any chance you can back port this to 0.6.x? we included that version in the sdk 45 release and it looks like there are some breaking changes between 0.6.0 and 0.9.0 that would impact expo go


add ytt config for configuring kp config at install

The reason for this change is to allow users to template the kp_default_repository creds as well as proxy values by using the carvel ytt tool.

@tomkennedy513 I'm about to give this a look. Could you provide more context on this PR for why you want to do this and what this helps accomplish?

Can I close this @tomkennedy513


About the running time of ssclp

Hello~ When I ran this command (python deepconcolic.py —model ../saved_models/mnist_complicated.h5 —criterion ssclp —mnist-data —outputs outs) to run Concolic Sign-sign-coverage (MC/DC) for DNNs on the MNIST model, I ran it for two days and still stuck in the place where the constraints were solved, as shown in the figure below.

Hi,
The size of the LP problems can be quite large indeed; also the performance of LP solvers may vary significantly on those problems. By default, DeepConcolic selects a solver (CBC) that requires dumping each LP problem to a file in order to solve it, and this makes the whole process even more time and resource heavy.
DeepConcolic offers several possibilities to select a solver among a pre-defined list, like CPLEX for instance. Yet, using that solver requires installing and a bit of configuration. The first step for starts here, and after that normally setting PYTHONPATH  to some value that depends on where you install CPLEX should suffice.


Add match_time to alert

Currently, Kibana can only query alerts by their creation time, not the time of the event. The event time is stored in the match_body, however that is not indexed.
This change adds the match_time field to an alert, and adds it as indexed field to the ElasticSearch index. In order to make use of the indexin feature, you will have to reindex.

@Qmando There is some great ElastAlert in Kibana update coming, would you be able to merge this into trunk?


MSI-Installer: Text is cut off and not readable

Please provide a brief summary of the bug
The help text for at least one element that can be installed is cut off (notice the "...").

Please provide steps to reproduce where possible

On Windows, start the installer for Temurin JDK 17
Continue until the page where it is possible to select elements to be installed
Select "JavaSoft (Oracle) registry keys"
See text on the right

Expected Results
Text should be short enough to be fully readable. Or better: Provide a link to a web page that has more information.
Actual Results
Text is cut off (to be precise: something is left out from the middle) and in this form makes no sense at all.
What Java Version are you using?
17
What is your operating system and platform?
No response
How did you install Java?
No response
Did it work before?
No response
Did you test with the latest update version?
No response
Did you test with other Java versions?
No response
Relevant log output
No response

As noted here
https://github.com/adoptium/installer/blob/ce1b4858058ff90ca5866db66274d05f780170f7/wix/Lang/OpenJDK.Base.fr-fr.wxl.template#L19
Description must be limited to 255 chars (spec that I found somewhere one time) .
This one on en-us is  258 chars after substituting {vendor_branding}) by "Eclipse Temurin"
In fr-fr around 256 with {vendor_branding}) replaced by "Eclipse Temurin"
I think I remember that the full text is show when the cursor go in top of the text.
Feel free to propose a shorter description for each language to fit into 255 chars

There is even a validation warning when building the msi:
warning SMOK1076 : ICE03: String overflow (greater than length permitted in column); Table: Feature, Column: Description, Key(s): FeatureOracleJavaSoft

I shortened the text under 255 length but it still doesn't show properly.

I propose to shorten the string, so it can fit and also to get rid of the validation warning.
If the string cannot fit, it adds ellipsis (...) and you won't understand anything anyway.
@douph1 what do you say?

I propose to shorten the string, so it can fit and also to get rid of the validation warning. If the string cannot fit, it adds ellipsis (...) and you won't understand anything anyway.
@douph1 what do you say?

If you just remove Eclipse or Temurin, would that fit?

If you just remove Eclipse or Temurin, would that fit?

Eclipse Temurin is a variable {vendor_branding}. But even if I remove that, it still doesn't fit completely.

This is the maximum we can fit (159 characters)

PR already merged


Feature/1004 deadline better handling of pype

Adds custom event listener to use extractenvironments command to pull Pype environments and push them into render process.
Updated existing submits to Deadline to include all required AVALON_* env vars.
Added missed custom plugins of deadline to vendor

Changed approach to Deadline plugin. Needs to be configured same way as custom event.
Needs https://github.com/pypeclub/pype/issues/1069 to be fixes first. (Or manually hardcode _SCHEMA_DIR in avalon-core\avalon\schema.py:100 , set _SCHEMA_DIR to pype/pype-v3.0.0/pype/schema directory of unpacked Pype folder.
(eg. something like _SCHEMA_DIR = "c:/Users/YOURUSER/AppData/Local/pypeclub/pype/pype-v3.0.0/pype/schema"

I started testing, but we should remove PypeMaya with this. it won't be needed anymore

Reimplemented with usage of GlobalJobPreLoad.


Update secedit-configure.md

note on detail of secedit behavior added that may confuse customers.

Learn Build status updates of commit d826186:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

WindowsServerDocs/administration/windows-commands/secedit-configure.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

#assign: @robinharwood, @Xelu86
@robinharwood @Xelu86
Can you review the proposed changes?
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Override to delete meta property?

I have an Helmet element and want to use it to delete a meta property (og:image:height) set in a parent component.
I've tried:
<Helmet
      meta={{ property: 'og:image:height', content: null }}
>

But can still see the og:image:height meta tag in the generated HTML. I was wondering if there's any way to remove it entirely.

@olalonde Thanks for your interest in Helmet.  In your example, the meta prop needs to be an array of objects.  Aside from that, the API doesn't offer a way to remove parent tags, but only a way to override them with different values.  If you need your tag to only show up in certain children components, you may need to move your tag down a level into the specific children you would like to use it.

@cwelch5
So, I stumbled on this today and was curious if this might be a viable solution.
const elem = document.querySelectorAll('meta[name=apple-itunes-app]')[0];
elem.remove();

The issue I'm having is that I want this tag present on almost every single view and I'd love to reduce boilerplate. I could, of course, use a shared <DefaultHelmet /> and inject on the top level; just curious why this feature isn't present or lacking, or if you might have any other recommended solutions.

I came across this today , wanted to remove the meta tags added by Gatsby, the only option was to override it with an empty value. I wish there was a way to delete a property.

I came across this today , wanted to remove the meta tags added by Gatsby, the only option was to override it with an empty value. I wish there was a way to delete a property.

When using Gatsby, you may want to customise your html.js file by cloning the .cache/default-html.js to your src/ directory.
You can achieve that by running the command line below at the root of your project;
cp .cache/default-html.js src/html.js

Then, edit your src/html.js to add or remove meta tags as needed.
You may refer to my Stack Overflow post on How to add mandatory AMP HTML attribute to HTML tag (for Gatsby) as sample reference.


Timing prescaler (PREC) assertion is panicking on 170Mhz

The timing prescaler is out of bounds even on 400_000 Fast i2c. I think they should be match-case for the correct dividers instead of a hardcoded 2 bit one, but not sure.

I'm testing it on real hardware, based on SM32G474RE

Awesome! Now works


[DOC] Feedback on ML Commons

What do you want to do?

[x] Request a change to existing documentation

Tell us about your request. Provide a summary of the request and all versions that are affected.
Community feedback
What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

[ ] 20230831 | /docs/latest/ml-commons-plugin/ml-framework/ | The request examples on this page are quite distinct from the ones on the api reference page - https://opensearch.org/docs/latest/ml-commons-plugin/api/#request-fields. The ones on the api request page work. These ones do not.
[ ] 20230830 | /docs/latest/ml-commons-plugin/ml-framework/ | The example request for uploading "all-MiniLM-L6-v2" requires "model_content_hash_value" to be included
[ ] 20230823 | /docs/latest/ml-commons-plugin/pretrained-models/ | I don't know what to do with the "config_url"
[ ] 20230928 | /docs/latest/ml-commons-plugin/api/ | Missing connector APIs
[ ] 20230928 | /docs/latest/ml-commons-plugin/extensibility/connectors/ | Example of predict API calls for each connector would be helpful.
[x] 20231005 | /docs/latest/ml-commons-plugin/conversational-search/ | "persistent": {"plugins.ml_commons.rag_pipline_feature_enabled": "true" has a typo in pipline
[ ] 20231007 | /docs/2.10/ml-commons-plugin/conversational-search/ | Does this work with any other hugging face model? or only with OpenAI and Cohere and Sagemaker?
[ ] 20231013 | /docs/2.5/ml-commons-plugin/cluster-settings/ | Have users asking about how to run these settings because they are unfamiliar with cluster settings.

Assigning to you, @kolchfa-aws , as this is ML-related. Thanks!


KIC: Introduce annotations available for gateway API resources

Where is the problem?
https://docs.konghq.com/kubernetes-ingress-controller/latest/reference/annotations/
What happened?
There is no introduction on annotations could be used on gateway API resources, such as HTTPRoute in the annotations page. Basically, HTTPRoute supports the same annotations as Ingress, but since some features like method match and header match are available in specs of HTTPRoute, so we disabled the annotations (when expression router enabled). We need to add introductions about the differences.
What did you expect to happen?
There is a section to introduce annotations could be applied on gateway API resources like HTTPRoute.
Code of Conduct and Community Expectations

[X] I agree to follow this project's Code of Conduct
[X] I agree to abide by the Community Expectations

Any update on this?


feature(dspy): Support more embedding models and return all fields

The PGVector RM currently only supports a single OpenAI embedding model and although it allows users to input a list of fields to select from PGVector, it doesn't use those additional fields in the dspy.Examples output. This modification allows users to specify:

Any OpenAI embedding model to be used. Still will default to the current model so there's no regression.
Return data within the specified fields in the dspy.Examples. It will now use the column names in the examples rather than the hardcode fields the current RM specifies.

This PR still needs to be tested. I'll create a local PGVector DB and test once a maintainer has notified me that the goal of this change is desired.

Thanks @Joshmantova ! These are great additions. Feel free to test (and add this to tests/) and lmk once it's ready to merge (post-merge conflict resolution).

Thanks @Joshmantova ! These are great additions. Feel free to test (and add this to tests/) and lmk once it's ready to merge (post-merge conflict resolution).

Hi @arnavsinghvi11, it's tested and ready to go. I don't think unit tests are very useful here because I'd have to mock over nearly everything. It also looks like most if not all the tests are unit tests. Given that we can't run necessary tests without setting up postgres, do you think it's worth it to add tests to the repo for this?

Hi @Joshmantova , looks great! yeah I think you can add some base integration tests and leave them out from the CI for now. Also, are the changes to poetry.lock needed? seems like they are not relevant to the changes but just want to confirm if you can remove that from the next commit. Thanks!

Hi @Joshmantova , looks great! yeah I think you can add some base integration tests and leave them out from the CI for now. Also, are the changes to poetry.lock needed? seems like they are not relevant to the changes but just want to confirm if you can remove that from the next commit. Thanks!

I'll add those integration tests and ping you when they're in. It'll likely be today. The changes to the poetry lock file were due to me adding the requirements to integrate PGVector. This RM depends on some packages that were not in the list of dependencies so I added them. After that, I re-compiled the lock file. I'm on a M1 mac so I'm not sure if that caused the lock file to change more than expected. Do you want me to remove the added dependencies?

Hi @Joshmantova , looks great! yeah I think you can add some base integration tests and leave them out from the CI for now. Also, are the changes to poetry.lock needed? seems like they are not relevant to the changes but just want to confirm if you can remove that from the next commit. Thanks!

@arnavsinghvi11 - woops! Looks like I accidentally upgraded the dependencies. I'll get that fixed up today as well, thanks for catching that.

Hi @Joshmantova , looks great! yeah I think you can add some base integration tests and leave them out from the CI for now. Also, are the changes to poetry.lock needed? seems like they are not relevant to the changes but just want to confirm if you can remove that from the next commit. Thanks!

@arnavsinghvi11 - I added a few integration tests including some instructions on how to set up PGVector. They're all passing at the moment. It turns out that I did not upgrade any dependencies. Installing the postgres related extras is causing the diff in the poetry lock file. I think it's wise to keep these dependency extras in the PR. Do you agree? If so, it's good to merge.

Hi @Joshmantova , thanks for adding the tests!
If you could resolve the merge conflict with the dependencies in poetry.lock and pyproject.toml files, and also run ruff check . --fix-only, the PR is good to go!

@arnavsinghvi11 - Btw, my other open PR will implement a solution that allows developers to easily install a valid dev environment including installation of all git hooks so we won't have to worry about these manual linting fixes moving forward. If you're able to take a look at that one as well, I'd appreciate it! There will be some merge conflicts that I'll resolve shortly after this PR goes in.

Thanks @Joshmantova !


Set up Codecov (#56)

#56
Is this okay?
This repo first needs to be linked to Codecov.io choosing it from the list of repos and generating a token.
Also, we can add coverage reports right in the GitHub pull requests by adding a codecov.yml file in the root of the project.
Like:

Should that be done too?

Hi @s0mnaths :wave:, thanks for your PR! The report extension would be a nice addition to the PR, but is not really required. If you can add it then why not.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@a2a7279). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #65   +/-   ##
=========================================
  Coverage          ?   86.02%
=========================================
  Files             ?       12
  Lines             ?      415
  Branches          ?        0
=========================================
  Hits              ?      357
  Misses            ?       58
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2a7279...fd433ca. Read the comment docs.


[BUG] GUI: Problems connecting to wallet

Describe the bug
I can't connect to wallet after restart computer
Screenshots

Desktop

OS: windows 10 pro
OS Version/Flavor: 20h2
CPU: 11700 intel

Additional context

This is almost certainly caused by Windows reserving the 55400 port the daemon uses by default.
You can either change the daemon port in config.yaml, or you can modify windows to not reserve this port:
dism.exe /Online /Disable-Feature:Microsoft-Hyper-V
reboot
netsh int ipv4 add excludedportrange protocol=tcp startport=55400 numberofports=1
reboot
dism.exe /Online /Disable-Feature:Microsoft-Hyper-V
reboot


An additional slash at the end of the hostname makes some operations unworkable

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

An additional slash at the end of the hostname makes some operations unworkable.
Current Behaviour
Using some operations from high-level API does not work when an additional slash at the end of the hostname is added. For example:

download_backup
export_dataset
create_task_from_data

However, other operations like downloading independent images (download_frames) or getting information about projects or tasks work properly.
Steps to Reproduce (for bugs)
Using the following code, having a slash at the end of the hostname, can be used to reproduce the error
from cvat_sdk import make_client, models
from cvat_sdk.core.proxies.tasks import ResourceType, Task

with make_client(host="http://localhost:8080/") as client:
    # Authorize using the basic auth
    client.login(('user', 'password'))

    # Models are used the same way as in the layer 1
    task_spec = {
        "name": "example task 2",
        "labels": [
            {
                "name": "car",
                "color": "#ff00ff",
                "attributes": [
                    {
                        "name": "a",
                        "mutable": True,
                        "input_type": "number",
                        "default_value": "5",
                        "values": ["4", "5", "6"],
                    }
                ],
            }
        ]
    }
    task = client.tasks.create_from_data(
        spec=task_spec,
        resource_type=ResourceType.LOCAL,
        resources=["image00200.png", "image01000.png"],
    )

Context
This is a minor bug that led me to try many other things before I saw where the problem was. The solution should be simple and I believe it could avoid a lot of headaches.
Your Environment

Git hash commit: ba4175bf5b4467bf0fbf89a3fd645cceef7f7c23
Docker version docker version: 19.03.13-ce
Operating System and version: Linux (AWS EC2 Amazon Linux)

@zhiltsov-max  Do you have any ideas here?

Hi, thanks for reporting the problem. Yes, looks like something to be fixed.


Repeated instances found in the tasks

@kurbster Looks task 77 merged in  #15 has some repeated instances. For example, check the following input which appears twice:

Step 1: For each row in Movie table, find the corresponding rows in Rating table.
Step 2: find each value of director in the results of step 1 along with the minimum stars of the corresponding rows to each value

Note that the two outputs corresponding to this input are different. Assuming that both these outputs are valid outputs, you'd need to group them to form one single instance (one input and two distinct valid outputs).
Please send a PR with the fix.

@shailaja183 your task also has repeated instances.
Check for Passage: Sandra journeyed to the bathroom. Daniel moved to the garden. Question: Where is Daniel? in the following task:  task083_babi_t1_single_supporting_fact_answer_generation

@kurbster also see the following repeated line:
 {'input': '[14, -18]', 'output': ['[13, -20]']}

in
task096_conala_list_index_subtraction.json

@shailaja183 Also see
 {'input': 'Passage: Fact 1- John went to the kitchen. Fact 2- John went to the hallway. Question: Where is John? Answer: hallway', 'output': ['Fact 2']}

in: task084_babi_t1_single_supporting_fact_identify_relevant_fact.json

@kurbster See
 {'input': '[0, 2, 4, 4, 1, 5]', 'output': ['[0, 2, 1, 5]']}

in task097_conala_remove_duplicates.json

@kurbster and
 {'input': "['H', 'T']", 'output': ['HT']}

in
task079_conala_concat_strings.json

@shailaja183
 {'input': 'Passage: John went to the kitchen. John went to the hallway.', 'output': ['Where is John?']}

in
task082_babi_t1_single_supporting_fact_question_generation.json

@shailaja183 @kurbster just checking if you have seen my earlier comments.

@kurbster , also when you fix these rename task074 to latest task numbers (107 at this point), task074 is repeated with another submitted task

I have fixed babi tasks regarding repetition of instances. Let me know if the issue persists on my end.
New commit: repeat instances fix - fa802e9

@shailaja183 you gotta put that commit on a branch and send a PR (so that we can merge it with our main branch).

Yes, I fixed the errors and made a python script to check for repetitions. I will make a PR.

Thank you all!


Split UG into separate files by H1, copied res dir into Documentation

Description

Divide the MSDK user guide into multiple .md files, based on H1 headings in the markdown.md file.
Copy image files in /res folder to /Documentation folder so image links don't break in new docs.
This task will facilitate ease of editing as we update the user guide.

Checklist Before Requesting Review

[x] PR Title follows correct guidelines.
[x] Description of changes and all other relevant information.
[ ] (Optional) Link any related GitHub issues using a keyword
[ ] (Optional) Provide info on any relevant functional testing/validation.  For API changes or significant features, this is not optional.

Good start! A few suggestions:

We're going to have multiple "books" in here so I suggest that we move these files into a user-guide sub-folder and remove the ug- prefix.
One of these files needs to be named index.md and serve as the first page of the guide. Maybe overview.md?
The file Documentation/index.md will stay where it is and be the landing page that people see when the get to the root of the docs. We should update that to point to user-guide/index.md.

Changes made per Oct. 2 recommendations, please review and we should be good to go.


parser: opertion kinds in grants for queries\commands are mixed up

see test:

for queries:

IsOperationAllowed(OPKind_Select) does not work
IsOperationAllowed(OPKind_Execute) works

for commands:

IsOperationAllowed(OPKind_Insert) does not work
IsOperationAllowed(OPKind_Execute) works

whereas, see here:
GRANT SELECT ON ALL QUERIES WITH TAG WithoutAuthTag TO sys.Anyone;
GRANT INSERT ON ALL COMMANDS WITH TAG WithoutAuthTag TO sys.Anyone;

I'm expecting:
GRANT EXECUTE ON ALL QUERIES WITH TAG WithoutAuthTag TO sys.Anyone;
GRANT EXECUTE ON ALL COMMANDS WITH TAG WithoutAuthTag TO sys.Anyone;

but that does not work

Done with #2726


Dispatching effects throw an error

Which @ngneat/effects-* package(s) are the source of the bug?
effects
Is this a regression?
No
Description
Effects are throwing an error when dispatched.
When following the Documentation example:
  loadTodos$ = createEffect(actions => actions.pipe(     ofType(loadTodos),     switchMap((todo) => this.todosApi.loadTodos())   ));
Please provide a link to a minimal reproduction of the bug
https://stackblitz.com/edit/angular-mctwxe?file=src/main.ts
Please provide the exception or error you saw
ERROR
Error: Make sure to provide a valid action type or set the option {dispatch: false}

Please provide the environment you discovered this bug in
"@ngneat/effects": "2.1.0",
        "@ngneat/effects-ng": "3.0.0",

Anything else?
No response
Do you want to create a pull request?
Yes

@EricPoul, Do we handle it in your last PR that we merged?

Nope, it's a bug. I see it. I'll make a pr to fix it


Incorrect decompilation of PowerPC:BE:64:VLEALT-32addr

Describe the bug
I've noticed that in PowerPC VLE mem copy function is not decompiled correctly showing infinity loop
-> Ghidra_9.1-BETA_DEV_20190923

void FUN_00000004(uchar *dest,uchar *src,ulonglong counter)
{
  int i;

  if (0 < (longlong)counter) {
    do {
      counter = counter & 0xffff; // <-- counter is not decreased
      i = (int)counter;
      dest[i] = src[i];
    } while (i != 0);  //<--infinity loop
  }
  return;
}

To Reproduce

Save as binary file

echo GADQABgA0AB8oCt54QotBgRgAOB8pACufKMBrioA5fkABBgA0AAYANAA | base64 -d > decompile_ppc_vle.bin

Open the binary as PowerPC:BE:64:VLEALT-32addr (1.5)
Decompile

On a side note  when creating a function at offset 0x0 there seems to be an issue with renaming variables in this decompiled code

Expected behavior
void copy(uint8 *pDest, const uint8 *pSrc, unsigned __int32 Count)
{
  int i; // r0

  LOWORD(i) = Count;
  if ( (int)Count > 0 )
  {
    do
    {
      i = (unsigned __int16)(i - 1);
      pDest[i] = pSrc[i];
    }
    while ( i > 0 );
  }
}

Screenshots
N/A
Attachments
N/A
Environment

Ghidra Version: ghidra_9.1-BETA_DEV_20190923

Additional context
N/A

Looks like a problem with the 'se_bmaski' instruction. You can try the following patch on your Ghidra installation as an immediate fix.
diff --git a/Ghidra/Processors/PowerPC/data/languages/ppc_vle.sinc b/Ghidra/Processors/PowerPC/data/languages/ppc_vle.sinc
index 587a4e4e..f3d4a42f 100644
--- a/Ghidra/Processors/PowerPC/data/languages/ppc_vle.sinc
+++ b/Ghidra/Processors/PowerPC/data/languages/ppc_vle.sinc
@@ -720,9 +720,8 @@ IMM16B: val                                         is IMM_0_10_VLE & IMM_16_20_VLE [ val = (IMM_16_20_VLE << 11) |

 :se_bmaski RX_VLE,OIM5_VLE             is $(ISVLE) & OP6_VLE=11 & BIT9_VLE=0 & RX_VLE & OIM5_VLE {
        RX_VLE = ~0;
-       tmp:1 = OIM5_VLE;
-       if (tmp == 0) goto inst_next;
-       RX_VLE = RX_VLE >> ($(REGISTER_SIZE) - tmp);
+       sa:4 = (8 * $(REGISTER_SIZE) - OIM5_VLE) * zext( OIM5_VLE != 0:1 );
+       RX_VLE = RX_VLE >> sa;
 }

 :se_bseti RX_VLE,OIM5_VLE              is $(ISVLE) & OP6_VLE=25 & BIT9_VLE=0 & RX_VLE & OIM5_VLE {


feat(typings): use exclude TS compilation instead of manual file list

The newly published webpack v1 skeleton used the annoying method of manual file inclusion for compilation, which required to add files-to-be-compiled one by one to the tsconfig.json.
It is better to utilize the capabilities of Typings software to simply link to those d.ts. files inside of the npm packages. I have corrected this bug by adding the proper references to the typings.json file. We do not need to double the d.ts references as Typings will always install the latest versions installed under node_modules.

The issue here is that the versions in the node_modules folder can be different than the versions in the typings.json which is always the latest version.

@arjendeblok I believe you are incorrect. With the definition as in the pull request, Typings will use the versions from the node_modules folder, not the latest version from NPM. The modules need to be installed prior to the typings install is run.


Something wrong with "composer require zizaco/entrust": "5.2.x-dev"

My Laravel version is 5.5.*,when I run "php artisan entrust:migration" command,an exception occurred;
[ReflectionException]   Method Zizaco\Entrust\MigrationCommand::handle() does not exist
So,I open the composer.json file,change "5.2.x-dev" to "master".
Then,the exception disapear!

I think the reason is in #836. 5.2.x-dev seems to work with old Laravel - maybe just outdated readme?

Yes,you are right! The README is outdated.


Timetable of paper and code release

Hi Haitao,
Thank you for your excellent work!
Could you please tell me the time that I can access the paper and code?
Regards,
Rui

Hi, Thank you for your interest in our work!
For the paper, we have updated the paper on the arXiv website recently. Here is the link
For the code, we are currently organizing the code, and we will release the code later. We will inform you once the code is publicly available.

Hi, Haitao
Thank you for your reply. How I can find the Appendix part mentioned in the paper?
Best,
Rui

Hi,
I will update the Appendix to the GitHub soon.

Hi, I have uploaded the Appendix supp on the Github.

期待您将神经网络部分开源

@scq6688 谢谢，我们开源后会第一时间告知您的

Hi Hetolin,
I find your approach really interesting and would like to try out the code. I hope find the time to upload it soon!
Thanks!

Hi, the code is released now


update to 3.0 spec

@devoncarew

Not sure about the failure:
Dart process started - port 40753
     [java] Sample out: Observatory listening on http://<IP_ADDRESS>:40753
     [java] Exception in thread "main" java.lang.RuntimeException: Failed to connect to the VM vmService service
     [java] 	at org.dartlang.vm.service.VmServiceTest.vmConnect(Unknown Source)
     [java] 	at org.dartlang.vm.service.VmServiceTest.main(Unknown Source)
     [java] Caused by: java.io.IOException: Failed to connect: ws://localhost:40753/ws
     [java] 	at org.dartlang.vm.service.VmServiceBase.connect(VmServiceBase.java:118)
     [java] 	at org.dartlang.vm.service.VmServiceBase.localConnect(VmServiceBase.java:173)

The Dart version is able to connect when running its smoke test.

And the java test runs fine on my desktop, so why does it fail here?

Does travis have any inter-process communication restrictions?
Or any restrictions on port numbers used by the test?

I don't know - I'd guess anything above 1024 is legal -

Super awesome that we're able to update both libraries to the new spec just w/ additional code gen.
Perhaps try switching travis over to using the dev build of the sdk?

Add the line dart: dev into the travis file, or test on both sdks at the same time by adding two lines: https://github.com/dart-lang/dartdoc/blob/master/.travis.yml#L3

@devoncarew Apparently hard coding the port to 7575 works for travis. PTAL

lgtm!

Per "dart: dev" comment above... https://github.com/dart-lang/vm_service_drivers/pull/64


problem in installing widget

using the following command I get the following result:
sudo composer require 2amigos/yii2-date-time-picker-widget:~1.0
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Problem 1
- 2amigos/yii2-date-time-picker-widget 1.0.x-dev requires bower-asset/smalot-bootstrap-datetimepicker 2.3.1 -> no matching package found.
- 2amigos/yii2-date-time-picker-widget 1.0.2 requires bower-asset/smalot-bootstrap-datetimepicker 2.3.1 -> no matching package found.
- 2amigos/yii2-date-time-picker-widget 1.0.1 requires bower-asset/smalot-bootstrap-datetimepicker 2.3.1 -> no matching package found.
- 2amigos/yii2-date-time-picker-widget 1.0.0 requires bower-asset/smalot-bootstrap-datetimepicker 2.3.1 -> no matching package found.
- Installation request for 2amigos/yii2-date-time-picker-widget ~1.0 -> satisfiable by 2amigos/yii2-date-time-picker-widget[1.0.0, 1.0.1, 1.0.2, 1.0.x-dev].
Potential causes:

A typo in the package name
The package is not available in a stable-enough version according to your minimum-stability setting
see https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion for more details.

Read https://getcomposer.org/doc/articles/troubleshooting.md for further common problems.
Installation failed, reverting ./composer.json to its original content.

I have the same problem

I can not install this widget too.

As https://github.com/smalot/bootstrap-datetimepicker is not registered in https://packagist.org/, it is showing this error.
We can do following:

Install smalot-bootstrap-datetimepicker#2.3.1 using following command
bower install smalot-bootstrap-datetimepicker#2.3.1
This will copy required files in \vendor\bower\smalot-bootstrap-datetimepicker directory.

Clone https://github.com/2amigos/yii2-date-time-picker-widget.git

Remove "bower-asset/smalot-bootstrap-datetimepicker": "2.3.1" from cloned project's composer.json.
It should look like this after removing
"require": {
    "yiisoft/yii2": "~2.0",
    "yiisoft/yii2-bootstrap": "~2.0.0"
},

Commit changes
git add .
git commit -m "remove dependency."

Update yii2's composer.json to following: (Please change url according to path where you have cloned repo.)
"repositories": [
    {
        "type":"vcs",
        "url":"file://D:\\programs\\xampp\\htdocs\\packages\\yii2-date-time-picker-widget-master"
    }
],
"require": {
    ...
    "2amigos/yii2-date-time-picker-widget": "dev-master"
},

run composer update --prefer-source

Now datetimepicker widget is installed.

This issue is quite old (sorry for being unresponsive). I am going to close it as we have already solved what was the subject of it.

in composer.json write :
"minimum-stability": "dev",
"prefer-stable": true,
and in CMD
composer require 2amigos/yii2-date-picker-widget:"*"


MacOS system Ruby version (2.6) being used and prevents the extension from working

Description
Ruby LSP Information

Ruby LSP Information
VS Code Version
1.95.3
Ruby LSP Extension Version
0.8.14
Ruby LSP Server Version
0.21.3
Ruby LSP Addons

Ruby LSP RSpec
Reek: Code smell detector for Ruby
Ruby LSP Rails

Ruby Version
3.2.5
Ruby Version Manager
mise
Installed Extensions

Click to expand

auto-rename-tag (0.1.10)
copilot (1.246.0)
copilot-chat (0.22.4)
debugpy (2024.12.0)
gitlab-workflow (5.19.0)
gitlens (16.0.4)
prettier-vscode (11.0.0)
python (2024.20.0)
remote-containers (0.391.0)
ruby-lsp (0.8.14)
sqltools (0.28.3)
sqltools-driver-pg (0.5.4)
vim (1.28.1)
vscode-docker (1.29.3)
vscode-eslint (3.0.10)
vscode-html-css (2.0.11)
vscode-pull-request-github (0.101.2024111810)
vscode-pylance (2024.11.3)
vscode-tailwindcss (0.12.15)

Ruby LSP Settings

Click to expand
Workspace
{}

User
{
  "enabledFeatures": {
    "codeActions": true,
    "diagnostics": true,
    "documentHighlights": true,
    "documentLink": true,
    "documentSymbols": true,
    "foldingRanges": true,
    "formatting": true,
    "hover": true,
    "inlayHint": true,
    "onTypeFormatting": true,
    "selectionRanges": true,
    "semanticHighlighting": true,
    "completion": true,
    "codeLens": true,
    "definition": true,
    "workspaceSymbol": true,
    "signatureHelp": true,
    "typeHierarchy": true
  },
  "featuresConfiguration": {},
  "addonSettings": {},
  "rubyVersionManager": {
    "identifier": "mise"
  },
  "customRubyCommand": "",
  "formatter": "auto",
  "linters": null,
  "bundleGemfile": "",
  "testTimeout": 30,
  "branch": "",
  "pullDiagnosticsOn": "both",
  "useBundlerCompose": false,
  "bypassTypechecker": false,
  "rubyExecutablePath": "",
  "indexing": {},
  "erbSupport": true,
  "useLauncher": false,
  "featureFlags": {}
}

Reproduction steps

Open VSC on MacOS using Finder/Spotlight with the Ruby LSP extension
Open a Ruby file
Notice it does not work and tries to use the MacOS 2.6 ruby version:

Failed to setup the bundle: Command failed: gem list ruby-lsp language_server-protocol prism rbs sorbet-runtime /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby: invalid option --yjit (-h will show valid options) (RuntimeError) . See [Troubleshooting](https://shopify.github.io/ruby-lsp/troubleshooting.html) for help

Run VSC on MacOS using code . from your terminal
Notice it works fine and doesn't try to use the 2.6 ruby version, but correctly uses 3.2.5 from Mise version manager

Basically it looks like asyncExec in vscode/src/workspace.ts:271 uses the MacOS built-in 2.6 Ruby version by default to run gem list. But when I run VSC from my terminal, it correctly loads the 3.2.5 Ruby version from Mise.

Yes it was, I will try your suggestion


question(urlMatcher): i18n the app paths

Hello everybody,
First of all, ui-router is really a huge improvement on any AngularJS app that pretends to bem flexible and well structured. Thank you for your effort!
In an application that I'm working on, we need to provide i18n of its paths.
Due to the nature of it, we need to deal with an endpoint that brings the available languages on the app.
So, the user enters on the app, and we have the following possibilities:

Without a locale: http://www.app.com(/);
With a valid locale: http://www.app.com/:i18n(/);
With an invalid locale string: http://www.app.com/articles(/) - The :i18n path param is interpreted as articles;
With a valid locale but not available: http://www.app.com/ru(/) - The :i18n path param is an unavailable locale;

I need to be able to:

Load the API endpoint that gives the available languages;
Parse the URL provided, change if necessary, and if so, replace it;
During the app runtime, the possibility to change the i18n param through something like a button and:
Inherit the current params, replacing the i18n one;
Trigger the elements that need to know about the i18n change (can be resolves, on an abstract route, or even the current;
With the ii, the possibility to deal with this without reloading the entire app (be able to only change the data used on the view; I'll use a feedback to inform the load of the new data);

One point is that, for example, if the user changes the locale to one that doesn't have content for the desired language, the app needs to redirect to a state that informs it to the user. I've thought, if its possible to trigger the elements on i18n change, and there's resolves on the state, on a rejection of one, it can dispatch a $stateChangeError, or something.
Ultimately, if I'm dealing with something that can be much simpler, please let me know... The use of a server configuration (the final server of this app will be an Apache), isn't that possible and, because the locales available comes from an API, I guess there's not much to deal with a .htaccess, isn't right?
Thanks!

This is a better question for stackoverflow. Try asking there.

http://stackoverflow.com/questions/24281652/localize-urls-with-ui-router-and-angular-translate

Ah, ok, I've placed one on there. The intention to ask here was about the core structure of ui-router, which there maybe not the view of the others.
Thanks for your time.


Added Docstring/Example for :meth:always_redraw

Overview: What does this pull request change?

Added an example to always_redraw
Changed lambda m: mob.become(func()) to lambda _: mob.become(func()) for clarity

Motivation and Explanation: Why and how do your changes improve the library?

There wasn't much of an example for always_redraw, which might've helped with learning how to use updators and such.
Links to added or changed documentation pages

https://manimce--3312.org.readthedocs.build/en/3312/reference/manim.animation.updaters.mobject_update_utils.html

Reviewer Checklist

[ ] The PR title is descriptive enough for the changelog, and the PR is labeled correctly
[ ] If applicable: newly added non-private functions and classes have a docstring including a short summary and a PARAMETERS section
[ ] If applicable: newly added functions and classes are tested

@jsonvillanueva I took your suggestion. Any other commends or todo's?

@behackl I upstate my PR to reflect suggestions.

@behackl Implemented your suggestions!


Release Spring boot 2 compatible version

Version 3 is in milestone stage for quite a while and spring boot 2 is out now.
It would be nice to have a stable release based on the current version 3.x of spring social, because especially for the facebook connector versions below 3.x don't work because the API has changed

+1

+1

Where is milestone 3?  I only see 2.0.0 in http://repo.spring.io/milestone.  Is social 3.x compatible with spring boot 2.0?

+1

+1

No reaction at all from the maintainers? :-(

It looks like spring social is in the attic now.
Login is possible via spring security oauth client, if you want to do more, you might want to have a look at restFB

There is no need to +1 here, spring social is dead


Fix wrong calculation of average precision

Description
Just realized that my implementation of average precision in #14189 was incorrect. This PR fixes that error.
Type of Change
Please delete options that are not relevant.

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Added new unit/integration tests
[ ] Added new notebook (that tests end-to-end)
[X] I stared at the code and made sure it makes sense

Suggested Checklist:

[X] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added Google Colab support for the newly added notebooks.
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
X] I ran make format; make lint to appease the lint gods

@DarkLight1337 Thanks for the correction.
Average Precision (AP) is defined as the precision calculated at each point where a relevant document is retrieved. These precision values are then averaged over the total number of relevant documents.
Is this the definition you’re referring to in the correction?

@DarkLight1337 Thanks for the correction.
Average Precision (AP) is defined as the precision calculated at each point where a relevant document is retrieved. These precision values are then averaged over the total number of relevant documents.
Is this the definition you’re referring to in the correction?

Yeah, previously I didn't factor in the case where the retrieved document isn't relevant in the calculation.


eligius_pushtx fail

show below message:
Trying to send...
NULL
Response = 0Sorry, function disabled.  Contact wizkid057 directly if you need a transaction mined.

How did you signed the transaction?


Feature request: Support new version of VSTest shipping with VS2017

Hi all,
in order to get coverage reports for .NET Core, it is now recommended to use VSTest in a newer version (https://github.com/Microsoft/vstest-docs/blob/master/docs/analyze.md#coverage).
This version can be found in Common7\IDE\Extensions\TestPlatform\vstest.console.exe compared to Common7/IDE/CommonExtensions/Microsoft/TestWindow.
Some parameters have changed as well (it is /collect:"Code Coverage" instead of /EnableCodeCoverage). Also VSTestFrameworkVersion would need to be extended ...
What would be the best approach? Extending the existing VSTestRunner with the changes or create a new version for this? Since I am already tweaking my local version of cake, I'd be happy to help on a PR ;)

Forget about it, I found it in DotNetCore/VsTest ... Didn't see that before :(


RSpec won't use syntax_suggest

I wrote a sample RSpec code to test syntax_suggest and Ruby 3.2.0rc1: https://github.com/JunichiIto/syntax_suggest_sandbox
I have a wrong spec file:
https://github.com/JunichiIto/syntax_suggest_sandbox/blob/main/spec/sample_spec.rb
RSpec.describe 'sample' do
  exampledo
    expect(1 + 1).to eq 2
  end
end

I added gem "syntax_suggest" to Gemfile and --require syntax_suggest to .rspec according to README.

https://github.com/JunichiIto/syntax_suggest_sandbox/blob/main/Gemfile
https://github.com/JunichiIto/syntax_suggest_sandbox/blob/main/.rspec

I ran bundle exec rspec but RSpec won't use syntax_suggest:
$ bundle exec rspec

An error occurred while loading ./spec/sample_spec.rb.
Failure/Error: __send__(method, file)

SyntaxError:
  /Users/jnito/dev/sandbox/syntax_suggest_sandbox/spec/sample_spec.rb:5: syntax error, unexpected `end'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2117:in `load'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2117:in `load_file_handling_errors'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1617:in `block in load_spec_files'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1615:in `each'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1615:in `load_spec_files'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:102:in `setup'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:86:in `run'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/gems/rspec-core-3.12.0/exe/rspec:4:in `<top (required)>'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/bin/rspec:25:in `load'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0+3/bin/rspec:25:in `<top (required)>'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/cli/exec.rb:58:in `load'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/cli/exec.rb:58:in `kernel_load'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/cli/exec.rb:23:in `run'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/cli.rb:486:in `exec'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/cli.rb:31:in `dispatch'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/cli.rb:25:in `start'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/gems/3.2.0+3/gems/bundler-2.4.0.dev/libexec/bundle:48:in `block in <top (required)>'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/3.2.0+3/bundler/friendly_errors.rb:120:in `with_friendly_errors'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/lib/ruby/gems/3.2.0+3/gems/bundler-2.4.0.dev/libexec/bundle:36:in `<top (required)>'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/bin/bundle:25:in `load'
# /Users/jnito/.rbenv/versions/3.2.0-rc1/bin/bundle:25:in `<main>'
#
#   Showing full backtrace because every line was filtered out.
#   See docs for RSpec::Configuration#backtrace_exclusion_patterns and
#   RSpec::Configuration#backtrace_inclusion_patterns for more information.
No examples found.

Finished in 0.00002 seconds (files took 0.09691 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

CLI command works fine:
$ syntax_suggest spec/sample_spec.rb
--> /Users/jnito/dev/sandbox/syntax_suggest_sandbox/spec/sample_spec.rb

Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?

  1  RSpec.describe 'sample' do
> 2    exampledo
> 4    end
  5  end

How can I use syntax_suggest when I ran bundle exec rspec ?

The gem is included in Ruby 3.2.0 you don’t need to add it to your Gemfile.
can you check with Ruby 3.2.0 and see if it still fails?

I removed "syntax_suggest" from Gemfile and ran against Ruby 3.2.0:
https://github.com/JunichiIto/syntax_suggest_sandbox/commit/1a734575e3624554f00a2b532adf7f1c4584e4c5
But the result did not change:
$ ruby -v
ruby 3.2.0 (2022-12-25 revision a528908271) [arm64-darwin21]
$ bundle exec rspec

An error occurred while loading ./spec/sample_spec.rb.
Failure/Error: __send__(method, file)

SyntaxError:
  /Users/jnito/dev/sandbox/syntax_suggest_sandbox/spec/sample_spec.rb:5: syntax error, unexpected `end'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2117:in `load'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2117:in `load_file_handling_errors'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1617:in `block in load_spec_files'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1615:in `each'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1615:in `load_spec_files'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:102:in `setup'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:86:in `run'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/gems/rspec-core-3.12.0/exe/rspec:4:in `<top (required)>'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/bin/rspec:25:in `load'
# /Users/jnito/dev/sandbox/syntax_suggest_sandbox/vendor/bundle/ruby/3.2.0/bin/rspec:25:in `<top (required)>'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/cli/exec.rb:58:in `load'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/cli/exec.rb:58:in `kernel_load'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/cli/exec.rb:23:in `run'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/cli.rb:491:in `exec'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/cli.rb:34:in `dispatch'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/cli.rb:28:in `start'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/bundler-2.4.1/libexec/bundle:45:in `block in <top (required)>'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/3.2.0/bundler/friendly_errors.rb:117:in `with_friendly_errors'
# /Users/jnito/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/bundler-2.4.1/libexec/bundle:33:in `<top (required)>'
# /Users/jnito/.rbenv/versions/3.2.0/bin/bundle:25:in `load'
# /Users/jnito/.rbenv/versions/3.2.0/bin/bundle:25:in `<main>'
#
#   Showing full backtrace because every line was filtered out.
#   See docs for RSpec::Configuration#backtrace_exclusion_patterns and
#   RSpec::Configuration#backtrace_inclusion_patterns for more information.
No examples found.

Finished in 0.00002 seconds (files took 0.09088 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

ruby command works fine:
$ ruby spec/sample_spec.rb
spec/sample_spec.rb: --> spec/sample_spec.rb
Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?
  1  RSpec.describe 'sample' do
> 2    exampledo
> 4    end
  5  end
spec/sample_spec.rb:5: syntax error, unexpected `end' (SyntaxError)

Thank you for the error report 🙏 I found the problem and possible solution. It's written here https://github.com/rspec/rspec-core/issues/2990. TLDR is that rspec needs to re-raise the SyntaxError or the contents won't be written to the screen.
I'm out on vacation at the moment but I hope there is enough information for someone to pick up the work and move forward.


filter parameters

Hi!
I have got some results by analyzing 10X SC data. I've compared it with 10X results and I found that the TRUST4 results are much more than 10X. In my opinion, it's not normal. So I want to filter TRUST4 results. Is there any recommended filtering parameters for TRUST4?
Thank you!

I observed the same case in several samples too. However, if we focus on the cells passed Seurat QC, 10x V(D)J still found more CDR3s. So the filtration should be using conventional single-cell RNA-seq QC, not through TRUST4.

Thank you for your quick reply. I will try it later.


Improve performance/memory usage of HashJoin datastructure (5-15% improvement on selected TPC-H queries)

Which issue does this PR close?
Closes #https://github.com/apache/arrow-datafusion/issues/6700

Benchmark results (TPC-H SF=1 in memory, average of 20 runs): q5, q7, q17, q18, and q21 show some improvement, most queries do not challenge the join or this code path (building the hashmap) of the join too much.

query
main
PR

5
58.74
54.98

7
104.14
92.41

17
332.83
320.09

18
244.41
218.97

21
152.79
132.93

Rationale for this change
Currently, we're using a smallvec to keep the indices with the same value. This hurts performance for cases when we have multiple values as it slows down insertion as well as retrieval of the indices (if the smallvec allocates, it's not cache-efficient).
Also the memory usage of the in-memory structure is reduced by paying the Vec-overhead only once rather than per-value: required memory is roughly reduced by 4x for storing the indices (8 bytes per value vs 32 bytes)
We can use a chained-list datastructure in a preallocated array where the next indices are stored (adapted from Balancing vectorized query execution with bandwidth-optimized storage)

What changes are included in this PR?

Are these changes tested?
Yes, by existing tests.

Are there any user-facing changes?
No

FYI @berkaysynnada I am trying to make some improvements to the hash join datastructure.
It seems the datastructure changes are not really compatible with the symmetric hash join (as it needs to be mutated during the probing process) - does it make sense to "duplicate" the old structure and code and use that inside the symmetric hash join?

@metesynnada, PTAL. Let's collaborate with @Dandandan on this as this is related to your work area.

@metesynnada let me know what you think and how we can cooperate. Next up I'm investigating improvements that could be done to speed up / vectorize collision checks.

We will check this PR tomorrow and comment on it. Thanks for working on this.

@metesynnada let me know what you think and how we can cooperate. Next up I'm investigating improvements that could be done to speed up / vectorize collision checks.

I'm excited to collaborate on optimizing the collision checks. As @ozankabak tells, I'll start my deep dive into this PR and further tomorrow.

Here are my measurements which I think are consistent with what is on this PR.
This is very exciting to see
--------------------
Benchmark tpch_mem.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃ main_base ┃ adapt_datastructure ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  517.86ms │            827.70ms │  1.60x slower │
│ QQuery 2     │  278.64ms │            534.33ms │  1.92x slower │
│ QQuery 3     │  186.82ms │            178.34ms │     no change │
│ QQuery 4     │  114.30ms │            112.48ms │     no change │
│ QQuery 5     │  480.70ms │            436.67ms │ +1.10x faster │
│ QQuery 6     │   40.20ms │             40.14ms │     no change │
│ QQuery 7     │ 1212.07ms │            963.47ms │ +1.26x faster │
│ QQuery 8     │  248.13ms │            251.24ms │     no change │
│ QQuery 9     │  613.36ms │            614.93ms │     no change │
│ QQuery 10    │  344.36ms │            339.00ms │     no change │
│ QQuery 11    │  212.06ms │            211.54ms │     no change │
│ QQuery 12    │  165.50ms │            167.81ms │     no change │
│ QQuery 13    │  673.03ms │            697.19ms │     no change │
│ QQuery 14    │   53.17ms │             50.71ms │     no change │
│ QQuery 15    │   88.08ms │             96.08ms │  1.09x slower │
│ QQuery 16    │  251.78ms │            200.89ms │ +1.25x faster │
│ QQuery 17    │ 2692.16ms │           2694.20ms │     no change │
│ QQuery 18    │ 2814.05ms │           2642.08ms │ +1.07x faster │
│ QQuery 19    │  167.80ms │            167.04ms │     no change │
│ QQuery 20    │  865.79ms │            852.77ms │     no change │
│ QQuery 21    │ 1446.79ms │           1268.84ms │ +1.14x faster │
│ QQuery 22    │  100.54ms │            102.31ms │     no change │
└──────────────┴───────────┴─────────────────────┴───────────────┘

Thanks for running the benchmarks @alamb  - are you sure of the accuracy of the slower running queries?

Especially query 1 is suspicious, as it doesn't have a join @alamb ;)

Especially query 1 is suspicious, as it doesn't have a join @alamb ;)

Wow. So there is a huge noise in the benchmark?

Especially query 1 is suspicious, as it doesn't have a join @alamb ;)

Wow. So there is a huge noise in the benchmark?

I ran the benchmark manually, averaged over 20 runs, which has only minimal noise. But running once might add some more noise

My results using the bench script with default parameters:
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃     main ┃ adapt_datastructure ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 187.78ms │            188.39ms │     no change │
│ QQuery 2     │  70.28ms │             68.41ms │     no change │
│ QQuery 3     │  52.51ms │             53.41ms │     no change │
│ QQuery 4     │  39.13ms │             38.95ms │     no change │
│ QQuery 5     │ 126.46ms │            113.56ms │ +1.11x faster │
│ QQuery 6     │  11.12ms │             11.14ms │     no change │
│ QQuery 7     │ 277.91ms │            232.85ms │ +1.19x faster │
│ QQuery 8     │  80.91ms │             81.30ms │     no change │
│ QQuery 9     │ 174.15ms │            163.58ms │ +1.06x faster │
│ QQuery 10    │ 104.81ms │            104.20ms │     no change │
│ QQuery 11    │  53.20ms │             54.47ms │     no change │
│ QQuery 12    │  70.71ms │             71.59ms │     no change │
│ QQuery 13    │ 214.11ms │            202.77ms │ +1.06x faster │
│ QQuery 14    │  14.09ms │             13.13ms │ +1.07x faster │
│ QQuery 15    │  23.20ms │             23.99ms │     no change │
│ QQuery 16    │  67.71ms │             51.89ms │ +1.30x faster │
│ QQuery 17    │ 718.41ms │            705.77ms │     no change │
│ QQuery 18    │ 732.54ms │            607.96ms │ +1.20x faster │
│ QQuery 19    │  61.01ms │             61.89ms │     no change │
│ QQuery 20    │ 214.62ms │            218.56ms │     no change │
│ QQuery 21    │ 412.88ms │            335.48ms │ +1.23x faster │
│ QQuery 22    │  33.44ms │             34.58ms │     no change │
└──────────────┴──────────┴─────────────────────┴───────────────┘

Currently, I am reviewing the code. The algorithm is neat, however, I want to find a good way to integrate the symmetric hash join into this algorithm. I think it can be possible.

@ozankabak I am not too familiar with the symmetric hash join, but one complexity seemed to be in reallocating/updating the next datastructure for the symmetric hash join.

I think changing the chain start pointer and a deletion offset might be the solution, I am working on that. We can address this in another PR. Since the performance gain is obvious, we can move on with this algorithm.
If you change the memory reservation calculation, LGTM.

32 bytes per `(u64, SmallVec<[u64; 1]>)

Sounds good :). Yes will do!

I had run the test on my local, no performance downgrade, this PR is great !!


Allow access to PolicyMap in AuthorizationOptions

This request is related to AuthorizationOptions class in Microsoft.AspNetCore.Authorization library.
Due to architectural reasons, we need to access registered policies that are stored in AuthorizationOptions class as:
private IDictionary<string, AuthorizationPolicy> PolicyMap { get; } = new Dictionary<string, AuthorizationPolicy>(StringComparer.OrdinalIgnoreCase);
This class has method allows retrieval of single policy by its name:
    public AuthorizationPolicy GetPolicy(string name)
        {
            if (name == null)
            {
                throw new ArgumentNullException(nameof(name));
            }

            return PolicyMap.ContainsKey(name) ? PolicyMap[name] : null;
        }

What we need is a similar method that would return all policies, something as simple as:
        public IDictionary<string, AuthorizationPolicy> GetPolicies()
        {
            return PolicyMap;
        }

Or, even easier version - PolicyMap property can be changed to have public gettter.
If this request is approved, I'd like to create pull request and make change myself.

@atuzovic can you share more info about why you need to list them? This is probably something we can implement, but the more we know about the scenario we can make sure we offer the right solution.

Scenario that we want to support is to implement visibility management on certain components in AngularJS based on server-side policies.
When the user logs in, names of the policies that are satisfied are returned in the response.
On the client side, AngularJS components are secured like this:
<some-component policy="RequiredPolicyName"></some-component>
In order for us to return policies that are satisfied, we need to access them from the code and validate if they are satisfied which we currently cannot.

@atuzovic feel free to submit a PR for this if you like as a starting point for discussion

We can do this in 2.1 as its not a breaking change

Closing because we think this change along would be insufficient for proper case-insensitivity support.

Did you finally decided to not implement this?
I think it's essential for the case @atuzovic explained: https://github.com/aspnet/Security/issues/1011#issuecomment-264634640
And it's a very common scenario for a SPA.


Mechanical Miner and Quantum quarry unusable on SMP?

Referencing issue #52 , the miner and quantum quarry seem to do nothing and are also not break-able on an SMP, even with version 1.1.3 of the mod.

hmm, can you check logs?

dont forget to post here

Sorry to come back to you so late, I will send the server and client-logs in here today, since I don't have direct access to the server logs.

fixed in dev


Added [?] operator for arrays

Scope & Purpose

Added experimental [?] array operator, which works as follows:

nonArray[?]: returns false
nonArray[? FILTER CURRENT ...]: returns false
array[?]: returns false if array is empty, true otherwise
array[? FILTER CURRENT ...]: returns false if no array member
satisfies the filter condition, returns true if at least one
member satistifies it.

[ ] :hankey: Bugfix (requires CHANGELOG entry)
[x] :pizza: New feature (requires CHANGELOG entry, feature documentation and release notes)
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification
[x] :book: CHANGELOG entry made

Backports:

[x] No backports required

Related Information
(Please reference tickets / specification / other PRs etc)

[ ] Docs PR:
[ ] Enterprise PR:
[ ] GitHub issue / Jira ticket number:
[ ] Design document:

Testing & Verification
(Please pick either of the following options)

[ ] This change is a trivial rework / code cleanup without any test coverage.
[ ] The behavior in this PR was manually tested
[ ] This change is already covered by existing tests, such as (please describe tests).
[ ] This PR adds tests that were used to verify all changes:

[ ] Added new C++ Unit tests
[ ] Added new integration tests (e.g. in shell_server / shell_server_aql)
[ ] Added new resilience tests (only if the feature is impacted by failovers)

[ ] There are tests in an external testing repository:
[ ] I ensured this code runs with ASan / TSan or other static verification tools

Suggestion from gnusi:
Also support quantifiers for the operation, such as:
[? 2 FILTER ...]
[???? 1..3 FILTER ...]
[? ANY FILTER ...]
[?? NONE FILTER ...]
[? ALL FILTER ...]


[request tool] Preclean text input so it can be successfully converted into a JSON object

Sometimes when chaining API calls, we have to preclean the output of the preceding step to be successfully converted into a JSON object. This PR adds a _preclean_input function that strips out backticks and newlines.

Backticks are not valid JSON IIRC. For newlines, perhaps better to replace them with whitespace, I don't think newlines (should) have semantic meaning in a value though. What do you suggest?

Backticks are not valid JSON IIRC. For newlines, perhaps better to replace them with whitespace, I don't think newlines (should) have semantic meaning in a value though. What do you suggest?

im assuming you want to strip new lines from the start and end? so maybe just do .strip()? or a regex?
either way, would be great to add tests for this

@drdee  Hi , could you, please, resolve the merging issues and address the last comments? After that ping me and I push this PR for the review. Thanks!

Closing due to inactivity and library being a different directory structure now, and feel free to reopen against the current state of the repo!