systems/architecture: bump default architecture to x86-64-v2

Compile everything by default with SSE4.2 to gain free performance.
This also removes support for any CPU older than westmere Nehalem.
Nehalem was chosen because the bootstrap gcc is to old to know westmere.
TODO:

[x] Do we want to enable this?
[x] Do all the hydra builders support this?
[x] Write a changelog entry

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Just drafting so that no one merges it by accident but I still want feedback and discuss this.

RHEL is on x86_64-v2 by default now: https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level

Also we should probably add the x86_64-vX feature levels instead of using uarch names, because we want the featureset without the uarch specific codegen.

The baseline is mostly defined by the gcc-N package, which is configured to produce baseline binaries when options like -march= are not used. - https://wiki.debian.org/ArchitectureSpecificsMemo#Architecture_baselines

Is that what this change does?

x86-64-v2 (this PR) should be safe-ish to enable and brings a good performance boost.
I think Arch uses x86-64-v3 (haswell), but maybe this is too much.

x86-64-v2 (this PR) should be safe-ish to enable and brings a good performance boost.
I think Arch uses x86-64-v3 (haswell), but maybe this is too much.

AFAIK, most of Arch packages are still using x86-64, though x86_64_v3 architecture was added to devtools. The RFC is to keep both x86_64 and x86_64_v3 packages, instead of bumping the default architecture.

Also we should probably add the x86_64-vX feature levels instead of using uarch names, because we want the featureset without the uarch specific codegen.

I added a suggestion from me to this PR.

Is that what this change does?

Yes, on first look.

x86_64_v3 packages, instead of bumping the default architecture.

We are currently not doing this because it would be equivalent of adding a new architecture because we would build everything twice.

Some more cpu features shoul be added: POPCNT, LAHF-SAHF, CMPXCHG16B...
https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels

Another one: https://www.phoronix.com/news/openSUSE-Tumbleweed-x86-64-v2

Also, looks like we're still not actually setting -march with this, which we probably should do.

Also, looks like we're still not actually setting -march with this, which we probably should do.

If we can configure our compilers to do that (produce baseline binaries when options like -march= are not used) by default, this would be redundant.

Some more cpu features should be added: POPCNT, LAHF-SAHF, CMPXCHG16B...

I am not sure if this will change much because I think gcc is already implying that but I'll leave the definitive answer to people that know a lot more than me about compilers.

Also, looks like we're still not actually setting -march with this, which we probably should do.

Do you think we need to set nixpkgs.hostPlatform.gcc.march/mtune?

Some more cpu features should be added: POPCNT, LAHF-SAHF, CMPXCHG16B...

I am not sure if this will change much because I think gcc is already implying that but I'll leave the definitive answer to people that know a lot more than me about compilers.

Also, looks like we're still not actually setting -march with this, which we probably should do.

Do you think we need to set nixpkgs.hostPlatform.gcc.march/mtune?
Going to undraft to collect more feedback because most of the open questions I cannot comfortable answer.
From my local testing I so far only found some breakages in already brittle python packages with comments like round to 4 digests because that works for aarch64-linux and some strange interaction with znver3 but at this point I am sure I messed something in my hacky configuration up.

Setting march would be a good idea, because in that case you can just set: -march=x86-64-v2

Do you think we need to set nixpkgs.hostPlatform.gcc.march/mtune?

I like generic mtune, as we tend to default to relatively recent GCC.

As new processors are deployed in the marketplace, the behavior of this option will change.  Therefore, if you upgrade to a newer version of GCC, code generation controlled by this option will change to reflect the processors that are most common at the time that version of GCC is released.

gcc -march=x86-64-v2 -Q --help=target | rg -e march -e mtune
  -march=                               x86-64-v2
  -mtune-ctrl=
  -mtune=                               generic
  Known valid arguments for -march= option:
  Known valid arguments for -mtune= option:

Do all the hydra builders support this?

All x86 HW on Hydra.nixos.org is recent.  The oldest is probably a Skylake machine used for the metrics job.  (Obviously, if builder features would be required, that would need some re-deploys.)

We have a problem already demonstrated by ofborg:
error: a 'x86_64-linux' with features {gccarch-x86_64-v2} is required to build '/nix/store/1d3i945hp1cgiwnrb50y80ps7d69qc29-python-setup-hook.sh.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}

I am not sure how to properly solve this.

It would be possible to extend the default set in nixos/modules/services/misc/nix-daemon.nix, but that would probably need some advance time.  I expect it might be better to assume v2 by default for this purpose.

(There's probably just this chicken-egg problem, according to the code in nix-daemon.nix.

Yes
I just discovered another problem: The gcc used in bootstrap stage0 is to old to know about x86_64-v2 (or znver2)
 ▶ nix log /nix/store/rc71dmi1y059z137pn4vgnf9mr90grvh-expand-response-params.drv
https://cache.nixos.org/vjvl14cl72pdf2vhkzkfz2l4sjzl7j94.narinfo
https://nix-community.cachix.org/vjvl14cl72pdf2vhkzkfz2l4sjzl7j94.narinfo
https://tum-dse.cachix.org/vjvl14cl72pdf2vhkzkfz2l4sjzl7j94.narinfo
unpacking sources
patching sources
configuring
no configure script, doing nothing
building
cc1: error: bad value ('x86_64-v2') for '-march=' switch
cc1: note: valid arguments to '-march=' switch are: nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 cannonlake icelake-client icelake-server bonnell atom silvermont slm knl knm x86-64 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 btver1 btver2 native; did you mean 'x86-64'?

I think we would need at least gcc11 in boostrapping. I also just discovered that x86-64-v1 is x86-64.
So what do we do now? Do we want to disable gcc.arch in boostrapping?

So what do we do now? Do we want to disable gcc.arch in boostrapping?

I don't think there is any reason to set -march during bootstrap. It is not like this will be the compiler that will be used to build the packages.

Some of the packages built by that compiler will get retained into the final package set, as it is now.  For example, glibc.

Found the problem. x86**-**64 not underscore

We could use march westmere and mtune generic instead.

This could help with packages hard-coded to build on older GCCs.

It evaled 🎉

Found one problem. x86 - 64 not underscore
We could use march westmere and mtune generic instead.
Edit: or not
--- old 2022-11-29 22:25:32.132595592 +0100
+++ new 2022-11-29 22:25:48.454569891 +0100
@@ -24,7 +24,7 @@
   -mamx-int8                       [disabled]
   -mamx-tile                       [disabled]
   -mandroid                        [disabled]
-  -march=                          x86-64-v2
+  -march=                          westmere
   -masm=                           att
   -mavx                            [disabled]
   -mavx2                           [disabled]
@@ -130,7 +130,7 @@
   -mpc32                           [disabled]
   -mpc64                           [disabled]
   -mpc80                           [disabled]
-  -mpclmul                         [disabled]
+  -mpclmul                         [enabled]
   -mpcommit                        [disabled]
   -mpconfig                        [disabled]
   -mpku                            [disabled]

We should use nehalem instead of westmere, it's one instruction closer to x86-64-v2:
❯ diff westmere nehalem
27c27
<   -march=                     		westmere
---
>   -march=                     		nehalem
133c133
<   -mpclmul                    		[enabled]
---
>   -mpclmul                    		[disabled]
187c187
<   -mtune=                     		westmere
---
>   -mtune=                     		nehalem

Maybe we should update the bootstrap GCC instead? This should allow us to get GCC 11 on aarch64 too (though the underlying issue is still not resolved).

We should use nehalem instead of westmere, it's one instruction closer to x86-64-v2:

Good catch!

Maybe we should update the bootstrap GCC instead?

Lets do that in a follow step. So far we do not know if this even work completely and bumping the bootstrap things is always a bit more involved.

aes is not present in any Microarchitecture level, but here it is included in 3 of them, it should be removed. Not even building with -march=nehalem enables aes:

2. LAHF-SAHF, CMPXCHG16B.. etc should be enabled too. In fact:
   gcc -msse3 -mssse3 -msse4.2 -msse4.1 -Q --help=target

This is not for what that list is used for. It does not directly map to march or mtune and also does not impact it. The main purpose of it is to feed predicates at the end of the file and turn on aes if the processor supports it. This is then used as a boolean stdenv.hostPlatform.avx2Support inside of a package.
Since --march=broadwell also does not enable the aes but it is listed above I just keep it like it is right now. If it turns out to be a problem we can change it in staging-next or the next staging run.

x86-64-v2 (this PR) should be safe-ish to enable and brings a good performance boost.

Does anyone have a reference about the performance boost?
I looked through threads around OpenSUSE and RHEL, without really finding any interesting measurements (SPEC isn't interesting to me for this purpose).  I fear that the difference for practical workloads might be non-measurable, except some very specific cases.  I mean, performance is about the only positive thing that can come from this PR, right?

I only have a system fully compiled with march=skylake. First of all booting seems to be way faster on zfs with a luks beneath. Both systems used nearly the same flack.lock with the exception of some build fixes.
--- none    2022-12-04 12:11:01.707783764 +0100
+++ skylake 2022-12-04 12:21:26.912490938 +0100
@@ -1,2 +1,2 @@
-Startup finished in 13.787s (firmware) + 2.206s (loader) + 11.297s (kernel) + 2.295s (userspace) = 29.587s
-graphical.target reached after 2.295s in userspace.
+Startup finished in 6.539s (firmware) + 1.525s (loader) + 11.370s (kernel) + 2.240s (userspace) = 21.675s
+graphical.target reached after 2.240s in userspace.

I don't have any other idea how to provide meaningful other results that are not purely benchmarks.

I only have a system fully compiled with march=skylake. First of all booting seems to be way faster on zfs with a luks beneath. Both systems used nearly the same flack.lock with the exception of some build fixes.
--- none    2022-12-04 12:11:01.707783764 +0100
+++ skylake 2022-12-04 12:21:26.912490938 +0100
@@ -1,2 +1,2 @@
-Startup finished in 13.787s (firmware) + 2.206s (loader) + 11.297s (kernel) + 2.295s (userspace) = 29.587s
-graphical.target reached after 2.295s in userspace.
+Startup finished in 6.539s (firmware) + 1.525s (loader) + 11.370s (kernel) + 2.240s (userspace) = 21.675s
+graphical.target reached after 2.240s in userspace.

Also add a small difference for me entering the encryption password. Not sure how meaningful that data is. Could also be that my method was not very reproducible.
I don't have any other idea how to provide meaningful other results that are not purely benchmarks.
 I wanted to try that for a znver2 but fail on libxcrypt (where you pointed out that there are some changes on staging). I haven't invested to much time yet and currently cannot provide error logs (as I am not at home) but thought maybe a quick question could help me out there.
Thanks in advance and thanks for the work to bring a newer architecture level to the binary cache ♥

I used this branch https://github.com/SuperSandro2000/nixpkgs/compare/nixos-unstable...SuperSandro2000:nixpkgs:nixos-unstable-simd which indeed contains commits for libxcrypt which are right now only on staging.

@SuperSandro2000 Is there anything preventing us from merging this PR? Thank you a lot!

For compiling gcc-11:
nixos-unstable: 30m 30s
nixos-unstable-simd with march skylake: 35m 45s

Maybe I'm being stupid, but the numbers sound like slow-down, not speed-up.

For compiling gcc-11:
nixos-unstable: 30m 30s
nixos-unstable-simd with march skylake: 35m 45s

Maybe I'm being stupid, but these numbers sound like slow-down, not speed-up.

Compilation times may be slower for some packages, I'd be interested in the compile times of what that gcc compiles

Compile everything by default with SSE4.2 to gain free performance.

Was this assertion verified? How much are we talking?
Furthermore, do we have an idea of how popular or how unpopular the CPU generations left behind are with our users? I'm thinking mainly about server-ish hardware, e.g. inexpensive colocated server offers, re-purposed older computers. Desktop and laptops from Core i[0] and before are not really likely to be still in use, I suppose.
How likely are we to suddenly make NixOS unusable on someone's deployment?
Do we have a migration path to prevent activating a generation that can't be used on a incompatible hardware? It would be awkward if a remote system that may not have proper remote console access is updated and now suddenly can't run the software.
Shouldn't this go through an RFC beforehand?

Maybe I'm being stupid, but these numbers sound like slow-down, not speed-up.

The numbers are not representative. They are over one build each and there might be other factors involved that influenced them.

Was this assertion verified? How much are we talking?

Generally you can say depends. Some software benefits majorly from being compiled with newer instructions while others sees no change.

Furthermore, do we have an idea of how popular or how unpopular the CPU generations left behind are with our users?

No, we have nothing like debian popcon.

How likely are we to suddenly make NixOS unusable on someone's deployment?

Should be pretty unlikely. We are talking about CPUs from the core 2 duo era, before the i branding was a thing. The better Xeon CPUs have a whopping 4 threads and max 16 GB of RAM across all slots.

Do we have a migration path to prevent activating a generation that can't be used on a incompatible hardware? It would be awkward if a remote system that may not have proper remote console access is updated and now suddenly can't run the software.

No, like for any other software update or change we do. This is already a gamble with every update and eventually it will fail. Right now nothing prevents programs from enabling sse3/4 or avx in their build scripts and some already do that.

Shouldn't this go through an RFC beforehand?

I am not volunteering for that.

Was this assertion verified? How much are we talking?

Generally you can say depends. Some software benefits majorly from being compiled with newer instructions while others sees no change.

So this is done without any metrics for benefits, got it. I don't mind the change if there is a good rationale and actual day-to-day gains, but otherwise it sounds like cargo-culting and/or superstition.

Furthermore, do we have an idea of how popular or how unpopular the CPU generations left behind are with our users?

No, we have nothing like debian popcon.

How likely are we to suddenly make NixOS unusable on someone's deployment?

Should be pretty unlikely. We are talking about CPUs from the core 2 duo era, before the i branding was a thing. The better Xeon CPUs have a whopping 4 threads and max 16 GB of RAM across all slots.

This includes the very start of the i branding, unless for some reason Nehalem, which is older than Westmere, doesn't count... it is unclear. I'm going from the PR description which states older than Westmere.
Though this is exactly the kind of hardware I expect to see sitting as the way older system running one's on-the-cheap NAS or home server. Not everyone is upgrading every few years and left with recent hardware to run as a home server.
This is also the kind of hardware that was heavily advertised by some cloud operators, and still is! I can start renting a "fresh new" (used) dedicated server without SSE4.x support without actually searching for it. In fact it's the first result from that vendor. Atom N2800, cedarview, is newer than Westmere by approximately a year, and yet still doesn't support SSE4.

Also this won't happen suddenly. unstable is unstable and breakages can and happened in the past and for a stable update people are expected to read the changelog and this change is the first entry.

This will happen suddenly. The change will happen suddenly. The whole world will be rebuilt. What may not be sudden is software that stops working. It will not include any sort of verification that requires the CPU to have the required instructions.
So in many ways, unless I'm wrong, this is way worse, since things will appear to work after an update, but some things will throw SIGILL hapazardly. A clear "this CPU cannot run this software" would be preferable here.
And "unstable is unstable" is a bad take. Its name is a shame, but it should be named rolling, which is rolling-up changes to the next release. For all intents and purpose, it is a rolling release, and should be treated as always green. Breakage is not fine.
Another way to see this, while you say "this won't happen suddenly", this will happen for the next release, so within six months someone with such hardware will have no way forward, even on stable. Suddenly at release time they are left with about a month of updates to figure out a solution.

Do we have a migration path to prevent activating a generation that can't be used on a incompatible hardware? It would be awkward if a remote system that may not have proper remote console access is updated and now suddenly can't run the software.

No, like for any other software update or change we do. This is already a gamble with every update and eventually it will fail. Right now nothing prevents programs from enabling sse3/4 or avx in their build scripts and some already do that.

And yes, this is already an issue. We've had software fail to build hapazardly with SIGILL since it built on older hardware than dependencies did on our diverse builders previously.
So since we already have a problem with unbounded impurities from the ambiant environment, it's fine to now break it in another way? This is a weird leap of logic, unless I'm missing what you meant here.

Shouldn't this go through an RFC beforehand?

I am not volunteering for that.

Right, but shouldn't this go through an RFC beforehand?

This is also the kind of hardware that was heavily advertised by some cloud operators, and still is! I can start renting a "fresh new" (used) dedicated server without SSE4.x support without actually searching for it. In fact it's the first result from that vendor. Atom N2800, cedarview, is newer than Westmere by approximately a year, and yet still doesn't support SSE4.

Would be curious to know which distros this cloud operators supports. Like someone said above, RHEL already dropped x86_64-v2 it seems, and considering they're one of the slowest moving distros out there I would assume others distros already did the same or will do soon(TM).

RHEL rationale: https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level#background_of_the_x86_64_microarchitecture_levels
Archlinux RFC: https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/2/diffs (mentions of improvements)
OpenSUSE: https://www.theregister.com/2022/11/30/opensuse_drops_x86_32/
Here: https://www.theregister.com/AMP/2022/07/26/suse_adaptable_linux_platform/ it is said x86_64-v2 processors are around ~2008-2011.
In my personal experience of homelabbing and cloud providers (famous Atom processors), those processors are in circulation, taking a PowerEdge R710 (which are now VERY old, power hungry, etc.), the processors in there are also of Nehalem generation.
Certainly, PowerEdge 2850 (Intel Xeon SL8MA TD428 for example.) and similar servers are not going to support NixOS with this change.
But I am in favor of getting those machines recycled honestly, I am not sure keeping them around is benefitting anyone, be it electricity, ecology, maintenance people, inherit risk of having them in your assets.
That being said, on my NextCloud virtual machine on an x86_64-v3 baremetal server (PowerEdge R720), I do spend a lot of time in the kernel doing crypto_aes_encrypt (measured with perf). I think having a aes-ni-enabled OpenSSL would bring "easy" improvements though I did not measure them yet.
If we do decide to stay on x86_64, I would argue, given the fact that everyone who switched to that level, including enterprise distributions, the burden of the proof, i.e. collecting the fact that there are still too many offerings for this machine and potential actual users, are on the people who want to maintain this status-quo.
Of course, our alternatives could be:

to duplicate the jobsets if we can get more hands and money to have builders for that, but if we could do that, I would spend that money on getting x86_64-v3.
offering a non-rolling release Hydra for x86_64 (sounds reasonable to me)

etc.

@SuperSandro2000 Could we split the changes and upstream on master the bare minimum to support the final switch (without introducing unwanted changes?)?
I would like to make it easy to propagate the switch and try it.

So this is done without any metrics for benefits

Do you have a suggestion how to get some meaningful metrics? We could do some benchmarks but they are meaningless for day to day usage. I don't think you can generally say this will improve things in every possible scenario but overhaul it should have a positive impact.

'm going from the PR description which states older than Westmere.

forgot to update that

Though this is exactly the kind of hardware I expect to see sitting as the way older system running one's on-the-cheap NAS or home server. Not everyone is upgrading every few years and left with recent hardware to run as a home server.

I also expect those systems to not have enough RAM to evaluate themselves and need to be remotely deployed.

This is also the kind of hardware that was heavily advertised by some cloud operators, and still is! I can start renting a "fresh new" (used) dedicated server without SSE4.x support without actually searching for it. In fact it's the first result from that vendor. Atom N2800, cedarview, is newer than Westmere by approximately a year, and yet still doesn't support SSE4.

Which? One of the best price/performance provider in Germany, Hetzner, is giving out skylake for their virtual machines. I also checked ovh and the oldest server I could find had a CPU from 2011.

This will happen suddenly. The change will happen suddenly. The whole world will be rebuilt. What may not be sudden is software that stops working. It will not include any sort of verification that requires the CPU to have the required instructions.

For other major breaking changes we also only wrote an entry into the changelog, unless I am forgetting one where we took special care to handle it.
We could add a check to the activation script to just fail if the cpu is to old.

So in many ways, unless I'm wrong, this is way worse, since things will appear to work after an update, but some things will throw SIGILL hapazardly. A clear "this CPU cannot run this software" would be preferable here.

Then people did not read the changelog. Also after this PR it is clear what the oldest supported CPU arch is. Right now it is unclear and in reality some software already is not working on old hardware.

Right, but shouldn't this go through an RFC beforehand?

I like to spend my time on other tasks where I directly feel the impact instead of spending on discussions.

to duplicate the jobsets if we can get more hands and money to have builders for that, but if we could do that, I would spend that money on getting x86_64-v3.

I personally think this would be a waste of builders and storage if we duplicate everything.

Could we split the changes and upstream on master the bare minimum to support the final switch (without introducing unwanted changes?)?

#208398 not much, we still have 4 commits in this PR.

From the discussion, I see the following points:

We lack a good set of benchmarks. :)
It would be nice, if people could try the different settings for their own workloads.
There is still a bit of 10+ years hardware in service that would be affected. The upgrade experience for those people would be poor.

Personally, I think one way forward would be:
Make different default optimization settings easy to try out (like @RaitoBezarius said). Otherwise, it's hard to get any data. Then make an informed decision based on that.

I think doing it like Arch folks plan to is better. Instead of dropping x86-64-v1, make caches for both. Also, instead of -v2 it would be better to jump straight to -v3 which supports eg. AVX2. Benchmark results comparing v1, v2 and v3 are on Arch mailing list

We can't keep making all binaries for both.  It would be way too expensive, in comparison to the advantages.

We can't keep making all binaries for both. It would be way too expensive, in comparison to the advantages.

Sadly, that's expected. I guess what would be left from my idea is comparing gcc-built and llvm-built -v2. But I don't have resources to do that :(

We lack a good set of benchmarks. :)

The problem with benchmarks is, that they do not represent real world and any benchmark that uses multi step instructions from the last 15 years will show a significant difference. Based on our test selection this will either bring the performance boost of the century or not change anything at all.

It would be nice, if people could try the different settings for their own workloads.

Not possible without a complete rebuild. replaceRuntimeDependencies is also not an option because it forces hydra to build packages at evaluation time.

There is still a bit of 10+ years hardware in service that would be affected.

We are talking about 15+ years. Nehalem is from 2008.

We can't keep making all binaries for both. It would be way too expensive, in comparison to the advantages.

Yes, we would waste a lot of compute and storage on that. I think it is feasible to build unstable-small or only stdenv to make using older versions less painful though.

my idea is comparing gcc-built -v2 and llvm-built -v2. But I don't have resources to do that :(

That is out of scope of this PR and won't solve the problems we are discussing. Many programs for linux expect gcc and llvm has subtle differences that would require manual fixing.

NixOS is a fast moving distro, especially compared to LTS enterprise Distros like Ubuntu, Suse or RedHat. If they can bump the default architecture, then we can do it, too.

If they can bump the default architecture, then we can do it, too.

IIRC Suse will keep providing all binaries for older machines as well, so IMHO that's a key difference.  (I don't recall situation in other distros.)

Yes, we would waste a lot of compute and storage on that. I think it is feasible to build unstable-small or only stdenv to make using older versions less painful though.

I think it is enough for the scenarios in which these old servers will be used.

About the question with the bootstrapping -march, this commit add the microarchitectures also to the cc-wrapper logic (which is not a bad idea anyways), and that seems to solve the bootstrapping issue on my setup.
I am currently building some x86-64-v3 to play around. Before i had a commit which removed the -march and -mtune from the boostrapping phase, but i wasnt really happy with that.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/best-practice-to-automaticly-build-all-installed-packages-to-put-them-into-a-binary-cache/25527/6

This change would mean I need to replace my firewall. It's not the end of the world of course, but I am definitely in the "let's get some numbers"-camp before moving ahead with this.
And asking people how they "feel" things are before and after this change is completely and utterly worthless.
To me, this comes down to:

either we can measure the resulting performance boost (and as it turns out, some workloads seem to be slower too, so we absolutely need to know what those are), or
we have architectural reasons for doing this

If it's none of the above, what exactly do we gain?

Drafting since this is not currently mergable from a community perspective.

Nehalem was chosen because the bootstrap gcc is to old to know westmere.

Now that we have the bootstrap files totally isolated from the built stdenv, we can one set of -march= flags when calling the bootstrap compiler and a different set when calling xgcc and stdenv.cc.
This is now possible because we never link binaries built by the bootstrap compiler to binaries built by xgcc or stdenv.cc.

For what it’s worth, I wanted to know how fast I can get made a -march=znver3 to compile clang.
My benchmark – compiling SuperTuxKart (2 runs) – went from 30 s (clang compiled in default stdenv) to 31 s (clang compiled in stdenv with znver3).
At least in this case, there wasn’t much to gain.

The gain(s), if any, aren't likely in compile time. As znver3 has another instruction set, which may result in "faster" execution time in some cases. Interesting benefits are likely on all those SIMD. As example this PR (defaulting to x86-64-v2) would enable compilation with SSE3 instructions. There are usecases that will possibly benefit in runtime execution if that instructions are usable.
If there is a measurable and reliable performance increase
on the daily jobs is another question. (Which was the main discussion on the PR so far). Because enabling that instructions will result that older CPUs without those instructions will not be able to run the code.
Tho that is not really related to this PR as znverX is already possible (even without that PR) but it's not the default (which i obviously a good idea, as not everyone has a znver3 compatible CPU), but you can still use that by setting GCC arch with the respective value, but you will result in compile everything by yourself when setting that globally.

To clarify, I did compare the runtime performance of an application compiled with gcc and znver3.
That application happened to be clang (because compilation is where I care most about performance). The flags I passed to clang to compile SuperTuxKart were the same in both cases I compared.
I was hoping that any improvement I see with znver3 is at least as good as x86-64-v2, which should be a subset/more general optimizations.
Apparently, clang is not a workload that benefits from that.

The difference between v2 and higher feature levels is mostly vector extensions, so it's to be expected a compiler (which is mostly pointer chasing) doesn't gain much here.

Here are some benchmarks from arch where the person running it kindly did a tl;dr which I am pasting here:
- there is no or negligible performance benefit of *-march=nehalem*, which corresponds to x86_64-v2,

- there is a moderate benefit of *-march=haswell* (x86_64-v3) - of around 10%-20% as compared to baseline for the tests performed

Link: https://lists.archlinux.org/pipermail/arch-general/2021-March/048739.html
I know it's a sample size of 1, but I would argue that the burden of proof with regards to demonstrating any benefit now lies with those proposing this change. The argument that "well, everyone else is doing it" doesn't carry much weight in this.

One thing I forgot (apologies for the comment spam) is if we do start looking at potential benefits and where we stand to gain something from this (full disclosure: I'm very much not a compiler guy).

either the act of building software is faster because the compilers benefit from this (use less energy, can get critical fixes shipped faster, can build more with less and so on), or
the resulting software runs faster (or uses fewer resources) - who doesn't like free speed?

I would imagine, that most of the software that really stands to benefit already has optimizations for various cpus/cpu features where we would gain nothing from fiddling with the baseline flags.
So could we compile variants of the compilers with a better baseline and use that when available? This way there would be no sudden breakage for end users?

You can't do this "when available" that easily.  What would people with older computers do?  Impurely switch their compiler variant?  Or would we compile everything twice (e.g. base + -v3)?

Here are some benchmarks from arch where the person running it kindly did a tl;dr which I am pasting here:
- there is no or negligible performance benefit of *-march=nehalem*, which corresponds to x86_64-v2,

- there is a moderate benefit of *-march=haswell* (x86_64-v3) - of around 10%-20% as compared to baseline for the tests performed

Link: lists.archlinux.org/pipermail/arch-general/2021-March/048739.html
I know it's a sample size of 1, but I would argue that the burden of proof with regards to demonstrating any benefit now lies with those proposing this change. The argument that "well, everyone else is doing it" doesn't carry much weight in this.

This was mentioned in https://github.com/NixOS/nixpkgs/pull/202526#issuecomment-1364066124 :).
It is not sufficient though.

I would imagine, that most of the software that really stands to benefit already has optimizations for various cpus/cpu features where we would gain nothing from fiddling with the baseline flags.

So could we compile variants of the compilers with a better baseline and use that when available? This way there would be no sudden breakage for end users?

Someone needs to do the rigorous work of looking into which packages benefit the most currently, see Guix prior art on that.

Or would we compile everything twice (e.g. base + -v3)?

Compile the compilers twice and abuse FOD…
I said it was an idea - not that it was a good idea.

(full disclosure: I'm very much not a compiler guy).

There's a bit of a Schrodinger's Developer problem here; the people who really care about this stuff are already building everything themselves.
My nix is patched to remove the Hydra key, so I don't really care.

It looks like CentOS is investigating what a jump to v3 might look like: https://blog.centos.org/2023/08/centos-isa-sig-performance-investigation/

I'd be intrerested to revisit reasons of other distros for switching to x86-64-v2, e.g. RedHat. Do they apply in NixOs' case? Or are they completely unapplicable to the NixOs? It's "do it because everyone else is doing it" kind of argument, but my point is why is everyone else doing it? They stand dropping support for old hardware just like NixOs does but still they decide to go ahead. What makes NixOs's situation different from everyone else's?

Right now, in NixOS, we decided to postpone as no one came with compelling
evidence that v2 provides serious benefits.
I am trying to assemble a recent unstable v2 and v3 system and build a
"relevant" benchmark to see what's going on.
But it takes time and resources.
In the meantime, there's no reason to break older hardware I suppose.
Le mar. 29 août 2023 à 02:26, Sergey Vinokurov @.***> a
écrit :

I'd be intrerested to revisit reasons of other distros for switching to
x86-64-v2, e.g. RedHat. Do they apply in NixOs' case? Or are they
completely unapplicable to the NixOs? It's "do it because everyone else is
doing it" kind of argument, but my point is why is everyone else doing it?
They stand dropping support for old hardware just like NixOs does but still
they decide to go ahead. What makes NixOs's situation different from
everyone else's?
—
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/202526#issuecomment-1696558456,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACMZRHTFF4Y6VGVGFERS2TXXUSKRANCNFSM6AAAAAASI77324
.
You are receiving this because you were mentioned.Message ID:
@.***>

AFAIK the other distros don't do it without replacement and still provide also binaries for older CPUs.  This PR's proposal didn't do that.  (And actually I don't think this is worth doubling build and storage costs for the x86 infra parts.)

https://www.phoronix.com/news/CentOS-ISA-Experiment-Perform
CentOS is looking at v3 baseline

As I said on Matrix, CentOS can afford to do that because CentOS has very long release cycles, so people on CentOS N-1 will remain supported for many years to come. For NixOS, the switchover would take at most 6 months, which is not nearly enough time to cut off support for all pre-2015 hardware.

Isn't there some kind of cflag to automatically create variants that use these features and decide in runtime which function sets will be used?
The examples cited here [1] have like explicit definitions but is there something that can kind of ramp up to more recent instructions?
For example, a math heavy library could generate more than one variant of loop heavy objects and some kind of logic to decide at runtime using the CPUID instruction which code variant will run.
IDK tbh if this is possible in single file binaries. The common approach for BLAS-like libraries for example is to generate a dynamic library for each variant then decide at runtime which one to dlopen.
[1] https://lwn.net/Articles/691932/

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/pre-rfc-moving-nixos-x86-64-baseline-to-x86-64-v3/35924/2

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/pre-rfc-gradual-transition-of-nixos-x86-64-baseline-to-x86-64-v3-with-an-intermediate-step-to-x86-64-v2/35924/1

Any news on this? Would really like to see a v2 baseline for Nixos.

See the discourse thread above.
To summarise:
To our current knowledge, there is no good data that proves a benefit from bumping to x86_64-v2. There might be a benefit but we don't actually know whether it exists or not. If it exits, it will be rather small though. (Note that there is a decent amount of bad data on this.)
There is no good data on how many users a bump beyond x86_64-v1 would exclude either. Anectdata suggests that there are at least some.
IMHO, this bump will require an RFC. Perhaps we should close this to signal this a bit clearer.

What about duplicated nixos channels (at least for stable), where one is the generic one and the other is compiled with x86-64-v3? This wouldn't deprecate any platform and it is also the way cachyos handles this.
The problem with selectively compiling packages with x86-64-v3, which could improve the most, is that if I for example compile gcc this will retrigger a recompilation of my whole system. I did not find a way around this problem.
Maybe there should be a flag to prevent such rebuild at the cost of reproducability?

We do not have the hardware to build multiple channels. You can use system.replaceRuntimeDependency if you really want to. But also, x86_64-v3 also does not improve things much, and we can likely get very close in terms of performance with some dynamic dispatch.


NOT WORK in Swift 3

incorrectly calculated touch
Test.zip

I'm facing this issue too. Version 1.0.2 works ok.

Hey guys, sorry for the long delay. The bug in the zip you provided is fixed in 1.0.4.


newsflash: init at 1.0.1

Motivation for this change
Adds NewsFlash. A modern feed reader designed for the GNOME desktop, and the spiritual successor to FeedReader.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

I tested with nix-shell -p nixpkgs-review --run "nixpkgs-review pr 93622" and it builds fine. However I could not find an executable to run.

@dasj19 The executable is named com.gitlab.newsflash

@danieldk I've opened an issue asking upstream to add the lock file: https://gitlab.com/news-flash/news_flash_gtk/-/issues/112.

@dasj19 The executable is named com.gitlab.newsflash

Yes, that works fine... You can thick the 3rd checkbox in the description ("Tested compilation of all pkgs that depend on this change using...")

@danieldk Thanks! I didn't know about nixpkgs-fmt, but I will use it in all my commits now.


Not all secrets/resources getting created

I have directly installed terrakube via helm on my dev and it works perfectly.
When I try to install it via argoCD in prod via helm it seems it is not able to create some services and secrets. I am not sure why that is as I am using the same values.yaml which works in dev.
Below is the values.yaml
name: "terrakube"

## Terrakube Storage
storage:
  existingSecret: terrakube-minio-manual
  defaultStorage: true
  minio:
    bucketName: "iaac-infrastructure"
    endpoint: ""
  default:
    endpoint: ""

security:
  dexClientId: "keycloak"
  dexIssuerUri: https://<>/dex
  adminGroup: terrakubeAdmin
  existingSecret: terrakube-ca-secrets-manual

minio:
  existingSecret: terrakube-minio-manual
  defaultBuckets: "iaac-infrastructure"

## API properties
api:
  version: "2.19.2"
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "Fortinet.pem"
          path: "Fortinet.pem"
        - key: "type"
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true

registry:
  version: "2.19.2"
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "Fortinet.pem"
          path: "Fortinet.pem"
        - key: "type"
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true

executor:
  version: "2.19.2"
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "Fortinet.pem"
          path: "Fortinet.pem"
        - key: "type"
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true

dex:
  config:
    issuer: https://<>/dex
    storage:
      type: memory
    oauth2:
      responseTypes: ["code", "token", "id_token"]
    web:
      allowedOrigins: ['*']

    staticClients:
      - id: keycloak
        redirectURIs:
        - 'https://<>'
        - 'http://localhost:3000'
        - 'http://localhost:10001/login'
        - 'http://localhost:10000/login'
        - '/device/callback'
        name: 'keycloak'
        public: true

    connectors:
      - type: oidc
        id: keycloak
        name: keycloak
        config:
          issuer: https://<>
          clientID: terrakube
          clientSecret: <>
          redirectURI: https://<>/dex/callback
          scopes:
            - openid
            - profile
            - email
          insecureSkipEmailVerified: true
          insecureEnableGroups: true
          userIDKey: email
          userNameKey: email

ingress:
  useTls: true
  includeTlsHosts: true
  ui:
    enabled: true
    domain: "<>"
    path: "/"
    pathType: "Prefix"
    ingressClassName: "nginx"
    tlsSecretName: tls-secret-ui-terrakube
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
  api:
    enabled: true
    domain: "terrakube-api.<>"
    path: "/"
    pathType: "Prefix"
    ingressClassName: "nginx"
    tlsSecretName: tls-secret-api-terrakube
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  registry:
    enabled: true
    domain: "terrakube-reg.<>"
    path: "/"
    pathType: "Prefix"
    ingressClassName: "nginx"
    tlsSecretName: tls-secret-reg-terrakube
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  dex:
    enabled: true
    path: "/dex/"
    pathType: "Prefix"
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"

this is what gets created in prod with argocd helm in prod

and this is what gets created for the same values.yaml file in dev but without argocd but a direct apply via helm

I am not really sure about your error maybe double check that you are using the same helm chart version......

Thanks...The problem was something else.


Allow expunging a VM on a deleted host when using host cache and ConfigDrive userdata service

Description
This PR fixes an issue observed during expunging a VM deployed on a network using COnfigDrive userdata service and host cache
2022-04-08 07:24:14,834 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-8:ctx-e4d333b0 job-78) (logid:a6b8b1aa) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin
com.cloud.utils.exception.CloudRuntimeException: Unable to get an answer to handle config drive deletion for vm: i-2-8-VM on host: 1
        at com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIsoOnHostCache(ConfigDriveNetworkElement.java:569)
        at com.cloud.network.element.ConfigDriveNetworkElement.deleteConfigDriveIso(ConfigDriveNetworkElement.java:638)
        at com.cloud.network.element.ConfigDriveNetworkElement.postStateTransitionEvent(ConfigDriveNetworkElement.java:321)
        at com.cloud.network.element.ConfigDriveNetworkElement.postStateTransitionEvent(ConfigDriveNetworkElement.java:89)
        at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:142)
        at com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachineManagerImpl.java:2129)
        at com.cloud.vm.VirtualMachineManagerImpl$5.doInTransactionWithoutResult(VirtualMachineManagerImpl.java:2160)
        at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:25)
        at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:21)
        at com.cloud.utils.db.Transaction.execute(Transaction.java:40)
        at com.cloud.vm.VirtualMachineManagerImpl.destroy(VirtualMachineManagerImpl.java:2150)

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[X] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[X] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?
Post Fix:
Successfully expunges the VM
022-04-08 08:22:29,583 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-2:ctx-d3f0bdc8 job-94 ctx-6bcc9fd2) (logid:4d070eb7) Asking ConfigDrive to release NicProfile {"broadcastUri":null,"iPv4Address":"<IP_ADDRESS>","id":18,"reservationId":"9b5d5de0-a880-4ee4-b906-6fa086b19010","vmId":8}
2022-04-08 08:22:29,584 DEBUG [c.c.n.e.ConfigDriveNetworkElement] (API-Job-Executor-2:ctx-d3f0bdc8 job-94 ctx-6bcc9fd2) (logid:4d070eb7) Deleting config drive ISO for vm: i-2-8-VM on host: 1
2022-04-08 08:22:29,587 WARN  [c.c.n.e.ConfigDriveNetworkElement] (API-Job-Executor-2:ctx-d3f0bdc8 job-94 ctx-6bcc9fd2) (logid:4d070eb7) Host 1 appears to be available, skipping deletion of config-drive ISO on host cache
2022-04-08 08:22:29,595 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-2:ctx-d3f0bdc8 job-94 ctx-6bcc9fd2) (logid:4d070eb7) Sync job-96 execution on object VmWorkJobQueue.8
2022-04-08 08:22:31,498 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-dd836c4c) (logid:00038e1f) Execute sync-queue item: SyncQueueItemVO {id:34, queueId: 9, contentType: AsyncJob, contentId: 96, lastProcessMsid:<PHONE_NUMBER>4091, lastprocessNumber: 10, lastProcessTime: Fri Apr 08 08:22:31 UTC 2022, created: Fri Apr 08 08:22:29 UTC 2022}
2022-04-08 08:22:31,500 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-dd836c4c) (logid:00038e1f) Schedule queued job-96
2022-04-08 08:22:31,508 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-5:ctx-a7f3e64f job-94/job-96) (logid:030ae26c) Add job-96 into job monitoring
2022-04-08 08:22:31,511 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:ctx-a7f3e64f job-94/job-96) (logid:4d070eb7) Executing AsyncJobVO: {id:96, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo: rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAACHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:<PHONE_NUMBER>4091, completeMsid: null, lastUpdated: null, lastPolled: null, created: Fri Apr 08 08:22:29 UTC 2022, removed: null}

@blueorangutan package

@Pearl1594 a Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3101

@blueorangutan test

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3845)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31590 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6234-t3845-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File


CORE: Get weight of food per person per day -->Calculate total weight of all food on trip, then divide that by numberofpeople x numofdays.

Got this to render on page, took me hours!


Ayush/feat/news feed

This PR closes the issue:

#3935

Changes

Added news feed on right panel above 944px and at bottom below 944px
Collapse of left panel at 1250px
News feed updates at start only
Added a new newsfeed model and reducer to set the news feed only at the start of the app. Everytime the startpage is opened again, the feed is fetched from the state and not via API call. API call is made only the initial start time.

How to test it

Run the electron app and in StartPage.tsx, add a console.log("API function is called") just below where the API call is made. And then route to different panels and cluster modes. You will see that API call will only be made the initial time.
Also try to resize the window to different widths to check the UI

Screenshots
https://github.com/kubeshop/monokle/assets/72748253/8cb42e32-9adf-4fa6-91da-2c51691e3175
Checklist

[x] tested locally
[ ] added new dependencies
[ ] updated the docs
[ ] added a test

I had to create a separate PR in order to continue working on this - https://github.com/kubeshop/monokle/pull/4106
Will close this one, thanks for your amazing work!

No problem @devcatalin !🙌🏼 Once #4106 is done, I will look at it too to improve my future contributions in Monokle. Thank you for guiding always!😄


Unexpirable API keys for AWS Amplify Appsync GraphQL API

Is this related to a new or existing framework?
React Native
Is this related to a new or existing API?
GraphQL API
Is this related to another service?
Appsync
Describe the feature you'd like to request
It would be really great if there's a way to have an unexpirable api key so i don't have to get a 401 response from the app when requesting queries, mutations or anything related to that.
Describe the solution you'd like
I think that an unexpirable API Key would be the best solution
Describe alternatives you've considered
Or maybe have it as a built-in function to repeatedly request for the latest API key if its not expired otherwise generate one when it does.
Additional context
@chrisbonifacio mentioned to me this article about creating a lambda function for this feature to happen but i wanted it to be included with Amplify instead
https://www.cloudxs.ch/2021/06/rotate-appsync-api-key-in-an-aws-amplify-project/
Is this something that you'd be interested in working on?

[ ] 👋 I may be able to implement this feature request
[ ] ⚠ This feature might incur a breaking change

Hey, there is a effort currently in place to support rotatable API key with Amplify CLI. Please refer to https://github.com/aws-amplify/amplify-category-api/issues/598 for additional information.


How to accelerate my own text to video model?

Thanks for your awesome work!
If I want to accelerate my own t2v model use this repository, how should I do? Which configuration files should I modify? I hope you can give me some advice, thanks a lot!

Hi there, I think you may need to make two major changes: 1. organize your dataset following the webdataset format (see our guideline), and 2. define and modify the teacher and student models.
I believe most of the training code does not require major changes.
Thanks!

Thanks for your guidance! I will take this steps later.


fix(layout): remove focusWindow calls from onWindowCreatedTiling

Abstract
Remove calls to focusWindow() from all implementations of onWindowCreatedTiling()
Bug fixed
commit 05047f60f4a6303a26f5ecef712f249b770f27b9 (#2630) introduced a bug where the active workspace incorrectly changes when using silent workspace methods or rules if the target workspace contains an unlocked group.
Current behaviour
When

using the moveActiveToWorkspaceSilent dispatcher to move the active window to a target workspace
or using the [workspace silent] rules to create a window in the target workspace

and the target workspace contains an unlocked group, the active workspace incorrectly changes to the target workspace instead of remaining in the current workspace as intended (not silent).
Expected behaviour
Active workspace should remain the same when moveActiveToWorkspaceSilent or [workspace silent] are used, regardless the content of target workspace.
Changes
Upon initial static analysis and debugging results, I respectfully suggest that all direct or indirect callers of onWindowCreatedTiling() appear to be calling focusWindow() themselves, or implying that their operand is active within the context. Therefore, the following lines may be redundant or betray the intention of their direct or indirect callers.
https://github.com/hyprwm/Hyprland/blob/1a6f961de2eff84e4725255f09d33395c4a52fcc/src/layout/DwindleLayout.cpp#L344
https://github.com/hyprwm/Hyprland/blob/1a6f961de2eff84e4725255f09d33395c4a52fcc/src/layout/MasterLayout.cpp#L123
Thus they are removed in this patch.
Please let me know if I have overlooked any significant details or drawn any erroneous assumptions in my conclusions.

Conflict with #3229 resolved. Ready


NSI suggestions are treated as valid when they are often not

URL
https://www.openstreetmap.org/edit?editor=id&node=3196433293#map=20/53.95187/-1.08486
How to reproduce the issue?
(related to https://github.com/openstreetmap/iD/issues/9528)
When iD makes an incorrect NSI suggestion, the language iD uses suggests that the NSI suggestion is likely correct (which is often untrue).
To reproduce:
Pick a cafe named after one of the most popular surnames in the country (Robinson).
Edit with iD.
iD says that it "looks like a common feature with nonstandard tags" and offers me the chance to "upgrade the tags"
This is wrong.  Sometimes NSI suggestions could be considered as "upgrades" (adding a wikidata field maybe, or pushing a particular tag schema).  This is not what is being suggested here - instead the suggestion is to change something mapped as one feature to another.
Screenshot(s) or anything else?
https://www.openstreetmap.org/node/3196433293/history
Which deployed environments do you see the issue in?
Released version at openstreetmap.org/edit
What version numbers does this issue effect?
2.25.1
Which browsers are you seeing this problem on?
Other (please specify)

That's the screen that I see too, but all of the text that iD displays here is wrong:

Cafe Robinsons is not a standard feature with nonstandard tags - it's a different feature entirely to the one added to the NSI at https://github.com/osmlab/name-suggestion-index/pull/7007/files ).
The suggested change is not an "upgrade"; it is a change from feature X to feature Y
"Tag is not the same Robinsons" is also wrong, since the tag does not match!

See also https://github.com/osmlab/name-suggestion-index/issues/7844 , although as noted there the NSI update (that there is a chain of pubs in the northwest of England called Robinson's) is mostly correct.

When adding a POI from the Feature Type, an available branded logo will appear, which makes it a lot easier to see what the suggestion is.
In this case, however, while iD shows the suggested tags, I can see from a new user perspective that it's not overly clear that the new tag suggestions represent the brand Robinsons.
Would it be possible to apply any available logo within the warning / suggested area as well?


test

test

Coverage remained the same at 95.833% when pulling b2dbff2c90eb60365399f3503b2889a6451d26f6 on gandhi23:master into a1ae7761d9d462ea8dd11872b7c529b37924e3a9 on openwisp:master.


Charging Stations: Display the closest charging stations according your position

[x] Handle sort/filtering per distance in the backend
[x] Order the chargers by closest to your position (need to enable localization)
[x] Display the distance in charger's header in the list

Done & Tested


how to compile

how can i compile libretower on my windows pc
this is my first time running a game maker studio game so i dont know how to run it

This is very similar to my issue: https://github.com/Applemunch/LibreTower/issues/2

do you mean run it in gamemaker? if so:
download all the files as a zip with the "code" button in the "code" page(your prob already know this)

once downloaded, go to the place where you downloaded the zip file (in my case the downloads folder)(also if you happen to have already installed the game, the image below shows the distinction between the game zip and the source code zip)

Zip the file with anything like Winzip or 7-Zip
Once zipped, open gamemaker studio 2
Click open

go to the file location of the source code (In my case, Documents) and select LibreTower.yyp

then voila! you are now editing LibreTower!
(im not really sure if this is what you meant, but here you go!)

oh yeah and another thing: this is ONLY for windows, i would make some instructions for mac and/or linux but i have neither.


Requests delivered too late

Often our inter-service requests are delivered more than 60s late causing them to fail since the receiver will think the signature is stale.
For instance faucet -> conveyor data transfers failing:
https://www.scalyr.com/events?severity=0&filter="Unable to store user data in conveyor"&teamToken=KBolCc6pavrJcaaz%2F4zDPw--&logSource=*production-faucet-001*&startTime=24 hours

Possibly due to https://github.com/steemit/jussi/issues/130

@roadscape @jnordberg  is this still happening?

Seems to have stopped happening since April 13


how to add new provider using android 11 with not root

I created a new server and this json file is perfeccting working on other android device, but now I bought a new tv box with android 11, in this version we haven't permission to access android/data/ folders, I know one way using some file managers apks to see this files but I can't because need a native file manager to provider this permission. My sugestion is one way to upload this file using burst addoon direct. Or if someone knows another option to privide this let me know please, unfortunatly this box isn't avalible to root yet, I m using a A95x F4 with android 11.

With version 0.0.77 there is now a new setting in Burst settings where you can point to a folder where you have overrides.json files saved, so it can be an sdcard.


Update django and python, add github action

Nice. You can close #58 from the community after this as well.
If you're in here, might be worth adding Python 3.12 and Django 5.0 to the test matrix as well since they are out now and there's some OSS usage of this library where folks might be on more cutting edge, non-LTS versions of Py/Django.

Nice. You can close #58 from the community after this as well.
If you're in here, might be worth adding Python 3.12 and Django 5.0 to the test matrix as well since they are out now and there's some OSS usage of this library where folks might be on more cutting edge, non-LTS versions of Py/Django.

Thank you for the input! I would like to get this merged ASAP and don't want to make the reviews stale since I have a dependency on this package that I need to update, but I'll make the updates in a following PR


Add clause reminding people to create file from within project directory

This change reminds folks to create index.html from within their project directory so that a new FileAsset object can be created (and used as a source) successfully in the tutorial.

@davidwrede Now that https://github.com/pulumi/docs/pull/5494 has merged, would you mind resubmitting this on https://github.com/pulumi/pulumi-hugo? Sorry the the inconvenience and happy to help if you need.

Moved this over to pulumi-hugo.
https://github.com/pulumi/pulumi-hugo/pull/16


Redstone Servo Recipe broken

the recipe for the thermal redstone servo gives gregtech covers instead

Known issue. Move the iron plate and then put it back in the same spot.


File Previews Sometimes Disappear

To this

Notice how the 2nd file preview is just gone. It seems to happen in a few seconds after it gets loaded from the picker. Note that the 2 files were files that I have taken from GDrive.

@AminDhouib This is completely out of our hands @MohamedBechirMejri , but w'll focus on it again asap

Not a high priority issue I may add but it's an issue that's worth noting here @bassem97 feel free to focus on other things and keep this issue in mind if you have nothing else to work on


Add Board Configuration of Pico Fighting Board(PFB)

Feature

GP2040-CE support for existing PFB.

Validation

Tested with actual device on hand.

Not equipped with turbo.

Please make these changes and re-submit.  If you need any help please feel free to reach out


change supportsPrewhere() of Buffer and return its destination_table supportsPrewhere

change supportsPrewhere() of table engine Buffer and return its destination_table supportsPrewhere() or not
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Buffer can help users to batch insert records into MergeTree from small inserting with great frequency. And use Prewhere clause in MergeTree can reduce the volume of data to read.
We often use Buffer upon MergeTree for inserting data, and  only query MergeTree table using prewhere condition, because Buffer can't support prewhere. But this method will make us lost the latest data in the query result.
so this PR can improve Buffer engine and solve the above problem.
Category (leave one):

Improvement

Short description (up to few sentences):
...
Detailed description (optional):
...

PREWHERE is disabled for Buffer engine for reason.
When you query Buffer table, it will read from two places:

from the destination table;
from its buffers in memory;

When a query with PREWHERE is executed, filtering by PREWHERE expression must be done entirely inside Storage. In contrast to WHERE, it is not performed in InterpreterSelectQuery. It means, the Storage must be able to perform PREWHERE.
But StorageBuffer does not evaluate PREWHERE when reading from its buffers in memory.

Here is a test that will provide incorrect results with your PR:
USE test;
DROP TABLE IF EXISTS dest;
DROP TABLE IF EXISTS buffer;
CREATE TABLE dest (s String, payload DEFAULT arrayMap(x -> s, range(1000))) ENGINE = MergeTree ORDER BY tuple();
CREATE TABLE buffer AS dest ENGINE = Buffer(test, dest, 1, 10, 60, 10000, 100000, 1000000, 10000000);
INSERT INTO dest (s) VALUES ('hello'), ('world');
INSERT INTO buffer (s) VALUES ('hello'), ('world');
SELECT * FROM dest PREWHERE s = 'world';
SELECT * FROM buffer PREWHERE s = 'world';
DROP TABLE dest;
DROP TABLE buffer;

Your own test has also failed.

Nevertheless, you can complete this PR in a proper way.
To do it, you should add interpretation of PREWHERE (create ExpressionBlockInputStream FilterBlockInputStream) inside StorageBuffer::read for processing of buffered data.

Your own test has also failed.

Yes, if I execute my test sql wholly in multi-commands mode, test will be failed. I will take your advice and add interpretation of PREWHERE. Thank you for your advice!

The test failure 00637_sessions_in_http_interface_and_settings - unrelated, already fixed in master.

Now the implementation looks perfect. Thank you!

great job!


GWL Upgrade

Back-end changes to Updated GWL notebook
Making changes to GWL framework updated notebook. Main changes include separating warming levels individually, so that data could be properly sliced into different 30-year chunks in sliced_data.
To test, please run the corresponding updated_warming_levels.ipynb from cae-notebooks.

 Warming Levels Notebook Debugging and Cleaning up Code


WIP for Initia data load

Addresses #72 #53.

FYI I'm working on fixing the tests then will give it a final once-over and merge.

Merged at f8127519c072af685b90b8a779189bd909187aa3!


Add cov

Description:

Add coverage reporting

Changes:

Added the nyc package as a dev dependency

Added the test:coverage and test:coverage:report package scripts

Added 2 new steps in the test action

[x] I have tested all my changes thoroughly.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@075037f). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #100   +/-   ##
=========================================
  Coverage          ?   79.53%
=========================================
  Files             ?      155
  Lines             ?     1793
  Branches          ?      139
=========================================
  Hits              ?     1426
  Misses            ?      345
  Partials          ?       22

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 075037f...198772c. Read the comment docs.


Sort mq-breakpoints when adding breakpoint

When adding a breakpoint with the mq-add-breakpoint mixin, can the function sort the breakpoint list. This prevents libraries that loop over the mq-breakpoints from having breakpoints added later overtaking larger breakpoints.
For instance, in my code I added a phablet breakpoint. It put it on the end of the list so when inuitcss generates the width utilities the phablet one is generated later than the tablet one so it overrides it when I want different widths at the two points but only one shows up.

Thank you for this feature request, it's definitely something that the mixin should be doing.
Reference for later: I've dealt with this in the past when working on o-grid, see https://github.com/Financial-Times/o-grid/blob/master/src/scss/_main.scss#L6

@solleer Hey! Thank you for your patience… I've just released<EMAIL_ADDRESS>let me know how it works for you!


Add HttpTryFrom for Uri

It seems super desirable to be able to do this:
Request::builder()
    .uri(format!("https://{}{}", host, path))

You currently cannot, and need to insert &* before the format.

While at it, probably also &'a String, since the generics on fn uri<T> don't cast the &String as a &str.


PDF-Chatbot-using-Langchain

In this project i have utilized  the power of OpenAI's language model and Langchain to enable users to interactively chat and extract information from multiple PDF documents.
The goal is to make it easier for users to get quick insights from various PDF files without the need to read each document manually.
The Chat with PDFs project is built on top of two powerful technologies:
OpenAI and Langchain.
:p assign me this task under GSSOC'24

cant assign more issues as its limited to 60 points


Compilation doesn't work on OS X with M1 because of different brew paths

on macos brew install glpk is not sufficient
for include, I had to add this manually:  -I /opt/homebrew/Cellar/glpk/5.0/include -L /opt/homebrew/Cellar/glpk/5.0/lib

I think this is the issue:
https://github.com/Homebrew/brew/issues/13481#issuecomment-1207203483

It seems macOS has no way to globally add a directory in the library search path, and so there is way besides explicitly looking in /opt/homebrew/lib (well, except pkg-config maybe) to find Homebrew libs. I blame Apple for this.

So the paths need to be added manually.


Add team application support

Add functionality and rest end points to allow user applications to teams. This needs to include creation of application, approval of application, denial of application. Associated events should be pushed to atomist to allow subatomic to react to these application requests.

This is the corresponding Gluon functionality to enable https://github.com/absa-subatomic/quantum-mechanic/issues/3.
Remember to handle the following:

Only a member that is a Owner of the requested team can Accept/Decline these requests


Infinite Money Glitch (lol)

Reprod steps:
Have more than 7 or so rows on your board.
Navigate to upgrades menu, set buy mode to 'MAX'
Sell all rows
Buy back maximum amount of rows
Expected outcome:
You lose a little bit of cells from selling off the rows but get most of them back
Actual outcome:
You get more money and more rows than you started with. It seems you can sell more rows than you actually have, as well as sell an amount of rows that would put you under the 3x3 board size minimum. This allows you to generate infinite cells.


allow devices without filesystem

Proposal: The lxops (config) file could have a way to attach a device without creating a filesystem for it.
That will provide a quick way of specifying a disk device to a container, by putting it directly in the config file.
I often attach devices to share directories between containers, or between the host and containers.  I've been doing this via profiles, so I have a lot of small profiles which are sometimes used by only one container.
We can use the existing "devices" syntax without the filesystem field:
devices:
   dev1:
     path: /<path-in-container>
     dir: /<path-in-host>

If the filesystem is empty, dir should be absolute.

This is particularly useful in cases where I share a directory in two different ways: readonly and read-write.  I end up creating two profiles.  The read-write profile is used by the container that creates data that multiple other containers can read (but not modify).
The readonly version can be in a profile as before, but the read-write version can be in the config file of the container that needs write permission.

Yet again, putting a device in a profile has distinct advantages.  You can easily move it to another container.  You can keep track of it separately from the container.

Done.  Also added "readonly" attribute.


allocate  with stat keyword must check stat variable

See: https://github.com/byornski/castep-linter/blob/main/src/castep_linter/tests/allocate_stat_checked.py

I proposed a similar thing for open in https://github.com/PlasmaFAIR/fortitude/issues/16, so if we can figure out the logic for one then the other should follow easily.


Feature request: Lan Mode only

Is it possible to force device to operate only in Lan Mode after initial setup?Means to Control it only over Lan and discount it from internet?

You should do it on your router. Or flash custom firmware.

I want to keep the original firmware. By just blocking it from Internet from Router side would it work? I mean the credential validation.

This is not recommended because the firmware doesn't always work well over the LAN. And then the component will be able to send the command via the cloud.
But if you "afraid" clouds - you can block Internet for your devices.


Add motherboard associations support for PCIe devs

Add motherboard associasions support for PCIe devs.
Create association endpoint between PCIe device and available motherboard chassis.

Please also add some description to your commit message. What you are trying to achieve, and how did you test this

race condition, когда интерфейс Item.System создан, а ObjectMapper про него не знает


Compilation error!

Hi there — just came across this course, which looks fantastic... thanks so much for taking the time to put it together and share your knowledge!
I have run into the following running the 'dev' script:
ERROR in ./src/css/app.scss (./node_modules/css-loader/dist/cjs.js??ref--5-1!./node_modules/postcss-loader/dist/cjs.js??ref--5-2!./node_modules/sass-loader/dist/cjs.js??ref--5-3!./src/css/app.scss)
    Module build failed (from ./node_modules/postcss-loader/dist/cjs.js):
    Error: PostCSS plugin autoprefixer requires PostCSS 8.

I tried downgrading autoprefixer as recommended here, but no dice... any idea much appreciated :)

Great tutorials so far!
Just to say, I am up and running, but I had to unhook the app.scss file from webpack, due to all the errors, far quicker(for me at least!) to compile the .scss using VSCode extensions than to try and unpick a whole pile of build errors!
So if anyone else runs into the same issue...

remove this line from web pack.config.js

'./src/css/app.scss',

'manually' compile app.scss in the project folder, each time you make a change in the sass files, obvs!

link to your own compiled css in the head of each html page, ie
<link href="css-compiled/app.css" rel="stylesheet">


[WIP] PUI Plugin Panels

This PR serves as a "proof of concept" as to how client-side data could be served by plugins, in the new react / platform interface.
In this instance, custom "panels" are loaded via the API, and rendered on the page based on what is delivered to the server.
Currently this only serves raw text - much more work would be required to render:

Raw html data
Content rendered on server via template
Content served via /static/ and loaded dynamically
Compiled react components etc

Related Issues

Closes https://github.com/inventree/InvenTree/issues/7709

TODO

[ ] Update documentation
[ ] Include a way to embed javascript / typescript code into the documentation
[ ] Mark old plugin mixin as deprecated (in documentation at least)
[ ] Make a "demo" plugin (published on PIP) which includes staticfiles, compiled javascript, etc
[ ] Work out a way to support "themes" (observe dark mode, etc)
[ ] Test if passing context objects (such as user, navigate, api) works with "compiled" code (currently works with vite dev server)
[ ] Unit testing (playwright, install remote plugin?)

Resources
Pinning some resources here for further reading:

https://stackoverflow.com/questions/44778265/dynamically-loading-react-components
https://stackoverflow.com/questions/50948995/react-application-with-external-plugins
https://stackoverflow.com/questions/40249045/react-native-dynamically-load-modules-plugins
https://samarjit-samanta.medium.com/react-dynamic-plugins-loading-e45905aa00a7
https://webpack.js.org/configuration/target/

Thanks that you already started a draft for this long standing issue. I just want to add my thoughts about this now:
I See two ways how developers could build PUI integrations:
1. They provide everything they need on thrown. E.g. they want to use react, they need to setup their own react spa, bundle it and ship the static files. Inventree PUI just loads embeds a specified entry html document.

✅ Advantages
❌ Disadavantages

Plugin devs are not limited to any particular framework, they could use preact, react, solidjs, svelte, ... and even just plain HTML or vanilla js
If multiple panel plugins using the same framework are installed and being serverd, multiple copies of the core frameworks code is needed to be loaded by the client

Developers have to make their own efforts to align the UI of their plugin panel with the ui design of PUI, and rebuild helper functions, UI elements on their own

To fix these disadvantages, I see two methods:

use a concept called module federation (search for the terms microfrontend, this concept is often used there). That way only one instance of react, mantine, ... needs to be used. But that would have the downside of version conflicts. E.g. if we do some breaking change in a component (e.g. add or remove a required prop), plugin panels using that federated component could just break
ship the inventree ui and utility functions as npm package, panel developers could use. This would have the advantage, that developers have a predictable environment (e.g. they can pin this inventree ui dependency) and this works with all versions.

2. They export a render function which gets called by PUI with a target DOM element they should render their content in. (Also with this method devs could use their framework of choice.) I Already made some tests with this a year ago: https://github.com/wolflu05/spa-preact-test
This would also add us some more possibilities, like we could let plugins provide some other UI things than just custom plugins. E.g. full new pages, custom label/report editors in the admin center (they also need a backchannel to talk to PUI).

Personally, I think I like a combination of all from above. We bundle all inventree components and utilities into a custom npm package developers could use. Yes, this adds the overhead of duplicate code, but I think that's much more reliable than somehow hack module federation to work correctly. Maybe we could module federation for the react dependency which has no breaking changes in minor version changes, and/or provide a global copy of preact, ... . Custom panels (and pages) use the HTML providing method (1), and custom editors could export a function (2).

@wolflu05 some great ideas here. I think that whatever approach we choose:

Enforce a single approach
Provide some well-worked examples
Publish a comprehensive set of common components
Make it future-proof!

@wolflu05 if you can provide a small POC for how we might load an entrypoint into a panel, that would be amazing.

The questions is what approach we should use. For my first suggestion, it would be as easy use using: dangerouslySetInnerHTML
And for the second, I actually made a POC last year linked in my initial post. I also played a bit there with using the main react version + mantine to use in panels (commented out), but that doesn't worked well that way, as react was complaining, that the rendering react version is not the same instance as the one creating elements, but that might be an issue from me to that time.

I would rather not use the first approach. I'll try to get your demo running when I have some time!

I really like @wolflu05 second idea; to keep the UI consistent providing a default set of UI components would be nice

Just one thought, maybe we can build the panel system in a way so that we later could also add those pages to the mobile app via a web view as an option (when plugin devs expose a panel, they could specify the target: web,mobile,...).

@SchrodingersGat regarding the ToDo section: why would we need javascript in the documentation?

@SchrodingersGat regarding the ToDo section: why would we need javascript in the documentation?

I want to be able to link in some source code from the frontend - namely to show what data gets passed through to the plugin context. And then to ensure that the docs stay up to date when we update / change that!

Regarding your is hidden function, can't we just return an object like
{ hidden: true }

from the renderPanel function? We could then use this later to return some additional information too.
And just a question, how flexible will the render system be? I'm thinking about other places where we need to inject custom content, maybe we can build that system in a generic way, so that this can be later easily extended (currently I'm thinking about the following places):
content based

custom panels on all pages using the panel group component (including admin center, system settings)
plugins settings drawer
custom editors/previews for templates
custom routes, e.g. to add completely new pages to the ui

list based

custom table actions, row actions, line actions, ... that have an onclick handler where they could do stuff with the rows (also opening modals, drawers so they are half list, half content based)
include custom entries in the navigation

@SchrodingersGat that sounds like a good way to have the docs break / be un-reproducible every few weeks. Is this required or could we get by with a similar mechanism as the settings json to render the settings? HTML-only docs have proven to be very stable and I fear that adding another dependency to our data structures will make it even more difficult to maintain.

@matmair to clarify, I do not want to run any javascript in the docs, just render code snippets (like we already do for our python source code)

And just a question, how flexible will the render system be?

@wolflu05 I am designing this to be the first part of a flexible system, defined via /api/plugin/ui/.... I think it should be reasonably generic.

Dynamic rendering from an imported javascript module is now working quite smoothly:

Still need to work out how to make them style nicely - i.e. build out a full working example that uses mantine and observes the correct theme.

This looks very promising although some parts seem a bit brittle - is unit and e2e testing part of the scope for this PR?

is unit and e2e testing part of the scope for this PR?

Some testing will definitely be added! What parts in particular do you think look brittle?

@SchrodingersGat
I apologize if this has been addressed elsewhere... but having trouble finding it if it is. I have a collection of CUI plugins (including panel plugins) that I've written and continue to maintain for my InvenTree instance. I use many "InvenTree builtins", like inventreeTable(), in my plugins in order to take advantage of all the reusability it has to offer. The switch to PUI, while exciting, makes me nervous that I'll have to rewrite all my plugins all at once.
Is there a world where switching to PUI for the server and updating my plugins to also use PUI can be incremental for me? In other words, would it ever be possible for me to continue using my existing CUI plugins with PUI on the instance?

@jacobfelknor this is a good point. I am going to write some documentation around panel plugin migration, before we force the switch.

I think first we need to publish the inventree npm package with the set of components, so that we can migrate e.g. the old form framework (I have used this in a panel) to the new react form framework. And same with other things.


Add windows install script/support

The v3.0.0-alpha1 will have a Windows build, so follow along with #266 for updates.


Added missed API visibility annotations for public / experimental APIs

Description
Added missed API visibility annotations for public / experimental APIs
Related Issues
Fixing warnings from OpenSearch API annotation processor:
Note: The element org.opensearch.http.HttpServerTransport is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.plugins.SecureTransportSettingsProvider)
Note: The element org.opensearch.transport.TcpTransport is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.plugins.SecureTransportSettingsProvider)
Note: The element org.opensearch.semver.SemverRange is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.plugins.PluginInfo)
Note: The element org.opensearch.rest.RestHandler.Route is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.rest.action.admin.indices.RestViewAction.CreateViewHandler)
Note: The element org.opensearch.rest.BaseRestHandler.RestChannelConsumer is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.rest.action.admin.indices.RestViewAction.SearchViewHandler)
Note: The element org.opensearch.common.cache.RemovalListener is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.common.cache.store.config.CacheConfig)
Note: The element org.opensearch.common.cache.serializer.Serializer is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.common.cache.store.config.CacheConfig)
Note: The element org.opensearch.common.cache.policy.CachedQueryResult.PolicyValues is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.common.cache.store.config.CacheConfig)
Note: The element org.opensearch.common.lucene.index.OpenSearchDirectoryReader.DelegatingCacheHelper is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.common.lucene.index.OpenSearchDirectoryReader)
Note: The element org.opensearch.index.store.RemoteSegmentStoreDirectoryFactory is part of the public APIs but is not marked as @PublicApi, @ExperimentalApi or @DeprecatedApi (referenced by org.opensearch.index.store.RemoteSegmentStoreDirectory)

Check List

[X] New functionality includes testing.

[X] All tests pass

[X] New functionality has been documented.

[X] New functionality has javadoc added

[X] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[X] Commits are signed per the DCO using --signoff
[X] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[X] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

If these builds are now clean, what do you think about enforcing the annotation process' checks and failing the build?

Thanks @peternied , so there was one change that held me back [1] but I am not sure anymore we should wait for it, I will work on that  to have clean pass (and make it mandatory)
[1] https://github.com/opensearch-project/OpenSearch/pull/10921


Serial loading of file parts

I would like to have not only sequential download of files, but also sequential download of parts of one file. This would be useful when downloading a video file so that I can start watching a few minutes after the start of the download. In order not to have to wait for the file to download completely.

@vktr please take a look at this issue and add the desired label.


PL[3.1.16] GTIN/NTIN resolution - Get Responsible MAH

As a Patient or HCP
I want to be able to scan the barcode and the GTIN/ NTIN resolver must resolve to to the responsible MAH
So That I am provided with the correct information related to my product
User Story Rationale/ Examples:
This user story relates to the resolution of the GTIN/ NTIN. It is important that the GTIN/ NTIN resolver resolves to the responsible MAH so that the correct information can be provided, the correct information could be the most appropriate electronic leaflet or the necessary anticounterfeit checks etc. The GTIN/ NTIN resolver provides the links/ connections between the scanned barcode on the pack and the responsible MAH.
There may be various reasons when the GTIN/ NTIN resolver cannot resolve, for example:
·         The GTIN/ NTIN might be falsified
·         The GTIN/ NTIN might not be maintained in the resolver by the MAH
·         Technical issues
·         The GTIN/ NTIN may resolve but there might not be an ePI available or no possibility to perform Anticounterfeit checks etc. This is different to the above examples but from a User perspective they would not be returned any information so this should be indicated in the error/ warning message displayed.
See Attachment A2 for a conceptual logical overview on how the GTIN/ NTIN resolver would be required to operate
Acceptance Tests
The GTIN/ NTIN resolution must be completed within 1 second.
In the event that the GTIN/ NTIN cannot resolve then the user must be provided with an appropriate and meaningful error message.

https://issues.merck.com/secure/attachment/385979/A2 GTIN  NTIN Resolver Concept v0.1.pdf

https://issues.merck.com/secure/attachment/385979/A2 GTIN  NTIN Resolver Concept v0.1.pdf


Readme steps resulted in errors

When I attempted to build from source I received errors:

No target for EnsureMage.
mage install appeared to be a no-op until mage build was run.

Issue 2 turns out to be PEBKAC, sorry guys! Turns out I just skipped the step that said build :D

@dgannon991 Was this fixed by #10 ?

It was, sorry I didn't close this as well!