Chunk endpoint returns 500 if chunk is not found

Summary
When using the /chunk endpoint with GET method and the requested chunk is not found then the response status is 500 (Internal Server Error).
Steps to reproduce

from the command line (fetch a random hash):

$ curl -v http://localhost:11633/chunks/8016d768c4f4a3f389848d26c5f3a57e0cbec5e8744d382ab0f0a608a4eefd4b
*   Trying ::1:11633...
* Connected to localhost (::1) port 11633 (#0)
> GET /chunks/8016d768c4f4a3f389848d26c5f3a57e0cbec5e8744d382ab0f0a608a4eefd4b HTTP/1.1
> Host: localhost:11633
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json; charset=utf-8
< Date: Wed, 20 Jan 2021 12:29:39 GMT
< Content-Length: 43
<
{"message":"chunk read error","code":500}

Expected behavior
It would be better if the status were 404, 406 or 408 or something similar. 500 suggests that there was an error on the server side and it could not process the request.

I cannot reproduce this. Trying same steps return (expected) 404 error (i.e. {"code": 404,"message": "chunk not found"}).
This is the case when the chunk cannot be found.
From logs I see that my node is trying to retrieve chunk from other connected peers, but fails.
If the error was chunk read error, there must be something else wrong during read process. This does not include trying to fetch chunk from other peers. In the case where recovery from network fails one can also receive 202 response.
For this to be explored more, it would be helpful to have logs that happen during your request. The node must be started with option to log at least "debug" level logs (level >= 4).

I cannot reproduce this. Trying same steps return (expected) 404 error (i.e. {"code": 404,"message": "chunk not found"}).
This is the case when the chunk cannot be found.
From logs I see that my node is trying to retrieve chunk from other connected peers, but fails.
If the error was chunk read error, there must be something else wrong during read process. This does not include trying to fetch chunk from other peers. In the case where recovery from network fails one can also receive 202 response.
For this to be explored more, it would be helpful to have logs that happen during your request. The node must be started with option to log at least "debug" level logs (level >= 4).

So I tried again with the current master (0ead1204) and it's indeed not reproducible. However with 0.4.2 (the currently released stable version) it is, so somewhere in between this behavior has changed. In case you are interested here are the logs with trace loglevel using all zeroes for the address:
TRAC[2021-01-27T12:16:01+01:00] name resolve: valid bzz address "0000000000000000000000000000000000000000000000000000000000000000"
TRAC[2021-01-27T12:16:01+01:00] retrieval: requesting chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 1c352197b7fd89844d5edf62176b6035236106c340d74473cf0a458e975d2b73
DEBU[2021-01-27T12:16:01+01:00] retrieval: failed to get chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 1c352197b7fd89844d5edf62176b6035236106c340d74473cf0a458e975d2b73: read delivery: stream reset peer 1c352197b7fd89844d5edf62176b6035236106c340d74473cf0a458e975d2b73
TRAC[2021-01-27T12:16:01+01:00] retrieval: requesting chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 9c7c5cbf8f2720c8353d9159f92c55fb4cbcadcf9fa34de28e1d94717889a867
DEBU[2021-01-27T12:16:01+01:00] retrieval: failed to get chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 9c7c5cbf8f2720c8353d9159f92c55fb4cbcadcf9fa34de28e1d94717889a867: read delivery: stream reset peer 9c7c5cbf8f2720c8353d9159f92c55fb4cbcadcf9fa34de28e1d94717889a867
DEBU[2021-01-27T12:16:01+01:00] chunk: chunk read error: get closest for address<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000, allow upstream true: no peer found ,addr<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
ERRO[2021-01-27T12:16:01+01:00] chunk: chunk read error
INFO[2021-01-27T12:16:01+01:00] api access                                    duration=0.003628671 ip=<IP_ADDRESS> method=GET proto=HTTP/1.1 size=43 status=500 uri=/chunks/0000000000000000000000000000000000000000000000000000000000000000 user-agent=axios/0.21.1

So I tried again with the current master (0ead1204) and it's indeed not reproducible. However with 0.4.2 (the currently released stable version) it is, so somewhere in between this behavior has changed. In case you are interested here are the logs with trace loglevel using all zeroes for the address:
TRAC[2021-01-27T12:16:01+01:00] name resolve: valid bzz address "0000000000000000000000000000000000000000000000000000000000000000"
TRAC[2021-01-27T12:16:01+01:00] retrieval: requesting chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 1c352197b7fd89844d5edf62176b6035236106c340d74473cf0a458e975d2b73
DEBU[2021-01-27T12:16:01+01:00] retrieval: failed to get chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 1c352197b7fd89844d5edf62176b6035236106c340d74473cf0a458e975d2b73: read delivery: stream reset peer 1c352197b7fd89844d5edf62176b6035236106c340d74473cf0a458e975d2b73
TRAC[2021-01-27T12:16:01+01:00] retrieval: requesting chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 9c7c5cbf8f2720c8353d9159f92c55fb4cbcadcf9fa34de28e1d94717889a867
DEBU[2021-01-27T12:16:01+01:00] retrieval: failed to get chunk<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000 from peer 9c7c5cbf8f2720c8353d9159f92c55fb4cbcadcf9fa34de28e1d94717889a867: read delivery: stream reset peer 9c7c5cbf8f2720c8353d9159f92c55fb4cbcadcf9fa34de28e1d94717889a867
DEBU[2021-01-27T12:16:01+01:00] chunk: chunk read error: get closest for address<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000, allow upstream true: no peer found ,addr<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
ERRO[2021-01-27T12:16:01+01:00] chunk: chunk read error
INFO[2021-01-27T12:16:01+01:00] api access                                    duration=0.003628671 ip=<IP_ADDRESS> method=GET proto=HTTP/1.1 size=43 status=500 uri=/chunks/0000000000000000000000000000000000000000000000000000000000000000 user-agent=axios/0.21.1

I can confirm that on 0.4.2 (commit dd6aa73a99ee4ebf285701295eee626842248768) it does return error 500.
This happened if the case that there are too few peers connected to node that can be contacted for chunk.
One could argue that might be better to return 500 if the peers cannot be reached, in order to check if the chunks actually exists, because at some later time same request may return the chunk (because peers are reachable then).
This changed due to changes I made in regard to preemptive retry for chunk retrieval. In current code, due to the way retry is done internally, some complexities were required to have same behavior as before, which were removed during review. And in reality I think it is better to return 404 in the case where chunk cannot be fetched or confirmed to exist.
@acud, @Eknir would you care to comment in regards to this?

I can confirm that on 0.4.2 (commit dd6aa73a99ee4ebf285701295eee626842248768) it does return error 500.
This happened if the case that there are too few peers connected to node that can be contacted for chunk.
One could argue that might be better to return 500 if the peers cannot be reached, in order to check if the chunks actually exists, because at some later time same request may return the chunk (because peers are reachable then).
This changed due to changes I made in regard to preemptive retry for chunk retrieval. In current code, due to the way retry is done internally, some complexities were required to have same behavior as before, which were removed during review. And in reality I think it is better to return 404 in the case where chunk cannot be fetched or confirmed to exist.
@acud, @Eknir would you care to comment in regards to this?


OSIDB:2701 Fix CVSS score value reset on flaw reset

[OSIDB-ID] [Title]
Checklist:

[x] Linting passed
[x] Type checks passed
[x] Tests suite passed
[x] Commits consolidated
[x] Changelog updated
[x] Test cases added/updated
[x] Jira ticket updated

Summary:
Fix CVSS score value reset on flaw reset
Changes:
Fix CVSS score value reset on flaw reset
Considerations:
[Replace with any additional considerations, notes, or instructions for reviewers.]
Screencast from 2024-06-17 15-35-06.webm

Hi @tboyetteredhat , I checked out the codes and verified the scenario in your screencast.

The flaw has no CVSSv3 vector&score, I did some update then reset
but it failed in below scenarios:
The flaw has a CVSSv3 vector/score, when I did some update then reset, the CVSSv3 vector&score are erased
The flaw has a CVSSv3 vector/score, I clicked reset directly, the CVSSv3 vector&score are erased

Do we need to cover below 2 scenarios in this PR?

I see, We need to resolve those issues. I will fix those issues.

Hi @tboyetteredhat , I checked out the codes and verified the scenario in your screencast.

The flaw has no CVSSv3 vector&score, I did some update then reset
but it failed in below scenarios:
The flaw has a CVSSv3 vector/score, when I did some update then reset, the CVSSv3 vector&score are erased
The flaw has a CVSSv3 vector/score, I clicked reset directly, the CVSSv3 vector&score are erased

Do we need to cover below 2 scenarios in this PR?

I see, We need to resolve those issues. I will fix those issues.

Resolved
Screencast from 2024-06-18 08-02-37.webm


Missing lib on ArchLinux when installing through Cargo

WIP
error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
this error requires libxcrypt-compat
pacman -S libxcrypt-compat

try this?
https://aur.archlinux.org/packages/rye-git
I have tested, namcap did not warn.

Perhaps using a musl build or an extracted appimage would solve the problem.

For now since this all installs via cargo only anyways by compiling, i could also switch to build settings that compile an SSL library in the same go.

Note that this issue is caused by the Python binary, not the Rye binary. I recommend switching the toolchain sources to extracted appimages

I encountered the same error on Amazon Linux 2023.
$ sudo dnf install libxcrypt-compat worked for me.

@quiver I presume this happens from the underlying Python for you?

@quiver I presume this happens from the underlying Python for you? At the very least this needs better documentation. The PyOxidizer builds have that in the quirks section: https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#missing-libcrypt-so-1

I think rye should either bundle the library, or switch to another Python distribution.

@mitsuhiko

I presume this happens from the underlying Python for you?

No, this error happens on rye installation process, i.e., $ curl -sSf https://rye-up.com/get | bash. So I believe this is a rust-specific problem.

There is an appimage distribution. It can be extracted to avoid the libfuse dependency, with Apprun replacing the python binary.
I agree that it usually is the host's responsibility, but if rye is to be a "one command install", this should be accounted for.

@quiver can you paste the full output please?

@mitsuhiko

can you paste the full output please?

Looking at the logs, it's caused by Python installations.
full output

sh-5.2$ sudo su - ec2-user
[ec2-user@ip-172-31-35-58 ~]$ uname -a
Linux ip-172-31-35-58.eu-central-1.compute.internal 6.1.27-43.48.amzn2023.aarch64 #1 SMP Tue May  2 04:53:12 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
[ec2-user@ip-172-31-35-58 ~]$ curl -sSf https://rye-up.com/get | bash
This script will automatically download and install rye (latest) for you.
######################################################################## 100.0%
Welcome to Rye!

This installer will install rye to /home/ec2-user/.rye
This path can be changed by exporting the RYE_HOME environment variable.

Details:
  Rye Version: 0.2.0
  Platform: linux (aarch64)

Continue? yes
Installed binary to /home/ec2-user/.rye/shims/rye
Bootstrapping rye internals
Downloading<EMAIL_ADDRESS>Checking hash
success: Downloaded<EMAIL_ADDRESS><EMAIL_ADDRESS>error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
Error: failed to initialize virtualenv in /home/ec2-user/.rye/self

How to reproduce

Launch Amazon EC2

Amazon Linux 2023
select arm instance type:t4g.micro

connect to EC2 with Session Manager and sudo to ec2-user
run $ curl -sSf https://rye-up.com/get | bash

This is now documented and a better error message is created on start. Generally though this is a archlinux violation of LSB and a bug should be opened against archlinux.


Fix expressions.md confusing directions

This section of the documentation says clearly that GitHub automatically evaluates the IF conditional as an expression so "you may omit the expression syntax ${{ }}.
However right after that tip it shows an example with expression syntax inside an IF.
This PR fixes that by removing the superfluos syntax.
Why:
Closes:  https://github.com/github/docs/issues/26539

@fsgreco Thank you for opening a PR and linking it to your issue! ✨
And welcome to the community!
I'll get this triaged for review! :zap:


missing 'version' attribute with custom emacsWithPackages from emacs-overlay

I'm trying to set up emacs package management in Nix specifically rather than using the builtin nix-straight management, and I noticed the emacsPackage option allows you to provide a custom emacs package to use to derive doom-emacs.
I've added the community emacs-overlay as a flake input (and made sure nix-doom-emacs follows the same pinned version). I've set emacsPackage to a custom build with flycheck installed here. Building that results in a missing version attribute, presumably from the emacs package that's now the base for  doom-emacs.
error: attribute 'version' missing

       at /nix/store/m2vv0bxfchzrjngx8wi0i7dhzb9q2g50-source/pkgs/applications/editors/emacs/elisp-packages/elpa-packages.nix:67:39:

           66|             else super.seq;
           67|       project = if lib.versionAtLeast self.emacs.version "28"
             |                                       ^
           68|                 then null
(use '--show-trace' to show detailed location information)

Tracing the custom base package however shows that the version is set to "28.1" - expected, since the base emacsUnstable from emacs-overlay being used is 28.1.
Does emacsPackage require a base non-derived emacs package for doom-emacs to build correctly? Or am I missing a key part of the process to build a custom emacs package?
For reference, I am using the NixOS wiki Emacs article and the source from the home-manager module option here.
I've tested the custom base package in environment.systemPackages and home.packages without issue, so the error is isolated to using it as a base for doom-emacs.

Try this:
From 99b749e1c5e6d061b376d789c1756ce9b3241ad0 Mon Sep 17 00:00:00 2001
From: ckie<EMAIL_ADDRESS>Date: Fri, 10 Jun 2022 17:50:53 +0300
Subject: [PATCH] home/doom-emacs: maybe fix nix-community/nix-doom-emacs#141

---
 home/doom-emacs/default.nix | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/home/doom-emacs/default.nix b/home/doom-emacs/default.nix
index 1ca9ebb..1f46aab 100644
--- a/home/doom-emacs/default.nix
+++ b/home/doom-emacs/default.nix
@@ -10,7 +10,8 @@ lib.mkMerge [
       {
         enable = true;
         doomPrivateDir = ./doom.d;
-        emacsPackage = ((emacsPackagesFor emacsUnstable).emacsWithPackages (epkgs: with epkgs.melpaStablePackages; [ flycheck ]));
+        emacsPackage = emacsPackagesFor emacsUnstable;
+        extraPkgs = epkgs: [ epkgs.melpaStablePackages.flycheck ];
       };
   }
   (lib.mkIf isLinux {
--
2.36.0

If that still doesn't work you might need to overrideAttrs it a bit like some of these.
(naming your repository nixpkgs is incredibly confusing :P)

I'm facing the same problem but the solution above did not work for me... @remi-gelinas were you able to fix it?

I was taking a look on our code where we get the Emacs derivation to be used... And well, it is complicated:

https://github.com/nix-community/nix-doom-emacs/blob/master/default.nix#L110-L118: here we get an Emacs for building doom-emacs packages
https://github.com/nix-community/nix-doom-emacs/blob/master/default.nix#L209-L220: here we get another Emacs for building the final derivation

This behavior may/may not explains the issue you're seeing, and also explain another issue that I saw when trying vterm module from Emacs. See: https://github.com/thiagokokada/nix-configs/commit/4f845d4766fa5d29b6c178beaf73a3596fe0ca91
I did this emacsPackagesOverlay because Emacs was build with the wrong source code it seems, and know that I took a look at the code this makes sense why it happened.


Merge Develop to main for Sprint 3

Merging develop to main for deployment of sprint 3

will fix bug


Tools rendered white / Crash on resource reload

Minecraft Version
1.18.2
Fabric/Quilt Version
Quilt 0.17.6
Fabric/Quilt API Version
qfapi-1.0.0-beta.28_qsl-1.1.0-beta.26_fapi-0.67.0_mc-1.18.2
Tinkers' Construct Version
1.18.2-<IP_ADDRESS>
Describe your issue
All tool parts are rendered white.
Pressing F3+T to reload sometimes works and the tools are rendered correctly. Sometimes, this does not work and the game crashes with a ConcurrentModificationException.
Crash Report
https://mclo.gs/LAYp5ca
Other mods
Only Hephaestus and Quilt Fabric API (all other mods are disabled as seen in the crash report)
Tried reproducing with just Tinkers?
Yes
Performance Enchancers
None of the above
Searched for known issues?
Searched open issues, Searched closed issues, Checked the FAQ

For anyone else having these crashes with a ConcurrentModificationException either on resource reload or when starting the game. Limiting the number of CPUs visible to Java seemed to be a work around. Just add the following argument to in your launcher configuration:
-XX:ActiveProcessorCount=4
I assume this may impact performance. I guess what is happening here is that Minecraft spawns a number of worker threads depending on the amount of CPUs the machine has and be reducing this, it also reduces the risk for a concurrent access to a certain resource. However, this is all wild guess. Anyway, my friends and I can play with this mod in our modpack now. Feel free to close this issue. Maybe this helps someone.

Fairly sure minecraft runs on a single core, this shouldn't impact performance.

Duplicate of #33


Add storage-shared to get analytics cluster health

Noticed in an issue the pod name is analyt-storage-shared-0 for v10 OVA.
Please add a new condition in the if loop at :
Change
https://github.com/ibm-apiconnect/v10-postmortem/blob/7cba4f8b015158e8fb2e25e816d45124e318938d/generate_postmortem.sh#L1068
to
if [[ "$pod" == *"storage-data"* || "$pod" == *"storage-basic"* || "$pod" == *"storage-shared"* ]]; then

Fixed in commit 67133485b86cee3bb45b7b1b6fa96abe3e04a1d2.


fix(default-theme): adding translated value to show more navigation item

This prevents a fatal error which was introduced with us rewriting the navigation to use translated values.

THAT is really great. This helped a lot. Thanks for your super direct support @elkmod. That left an outstanding impression.


Implement react-query for the wrap page

Remake of previous PR #1455 by @biwano now that #1401 is merged.

This is displaying on frontend, likely related to above comment

@brightiron That popup is the react-query devtools, really handy for debugging queries.

just need to change that wrap layout I commented on

I went ahead & pushed a simple fix for this here https://github.com/OlympusDAO/olympus-frontend/pull/1515/commits/0d8fd7bb82978c88c7b6a8cb4c7c4cc237fbcd6e


Add link options CA and CPP

Problem
EPICS supports link options CA, CP, and CPP which control behavior of a link which is treated as a channel access link. However, epicsdbbuilder supports CP only.
Solution
Add CA and CPP options.
Test
Extend with CA, CP, and CPP and run examples/test.py

Is there anything I should add in this pull request? I was thinking about automated unit tests but I would separate that into a specific task as there are none yet.

I can see nothing wrong with this.  Sorry, have been a bit busy elsewhere!

Thank you.


Small readme changes

And also I have gotten rid of the testing (as not sure who actually uses it?)

Thank you for this MR;
I do like the idea of some basic tests which I have therefor just added.
I furthermore updated the Readme which made this MR conflicting.


Bar Series onValueMouseOut not triggering

I am using vertical clustered bar chart. Rest of the functionality works well, but onValueMouseOut does not trigger when I leave a particular bar.
Here is my snippet from React render:
{
          data.map((item, i) => {
            if (!item.disabled) {
              return (
                <BarSeries
                  key={i}
                  color={item.color}
                  data={item.datum}
                  title={item.title}
                  onValueMouseOver={(datapoint, event) => getHintData(datapoint, event, item.title)}
                  # **onValueMouseOut**={(datapoint, event) => this._removeHintData(datapoint, event)}
                  onValueClick={(datapoint, event, j) => handleBarClick(datapoint, event, j, item.datum)}
                />
              );
            }
            return null;
          })
        }

Hey @ArbaazAnDossani
I did a cursory check on some of the examples and they appear to have their onValueMouseOver/onValueMouseOut/onValueClick working in good order. is there a chance that you animation on? It is a known issue (#381) that animation causes problems here. Additionally you shouldn't be using BarSeries (bar-series) directly, as that will most likely cause problems?

Hey @mcnuttandrew
I am using Vertical Stacked Bar Graph sample, I removed the animation prop and now onMouseOut is working fine.
Can you please explain how are these two issues related and what is the solution when I need both animation as well as onMouseout function?
Thanks!

@ArbaazAnDossani
This problem is caused by the relationship between our Animation component and listeners. As best I can tell the problems arise from higher order components causing problems. In #554 I tried out a different strategey for getting around this, that boils down to adding invisible two copies of the series of concern, like so
<XYPlot>
<VerticalBarSeries animation data={mydata}>
<VerticalBarSeries data={mydata} opacity={0} onMouseOut={() => {...}}>
</XYPlot>

So that the invisible series handles all of the mouse events and the visible series is animated.  If we can't come up with a better solution to this problem, this solution will be integrated into the main library.

@jckr It looks like this was closed, referencing #711, which was closed referencing this bug.  Can we re-open one of them?  And even earlier bug was raised (#381) which was closed as "similar to [this bug]"

I can confirm I've just encountered this bug with <MarkSeries/>. It works fine after removing the animation prop from the <XYPlot/>.

Re-opened. The still best strategy I've thought of is the one deployed in arc series, which puts a secondary ghost series of the top of the animated one for control of the event hooks. I don't really think there will be a better solution, unless there happens to be some way to untangle the way that the animation component works.

I agree with @mcnuttandrew  as a temporary solution for the current issue.
But populating DOM twice is never a great idea.

same problem here!

I got the same problem too.
The workaround does not work so well to be honest.
I think it worths raise a ticket and fix this in some PR.
Cheers

@ArbaazAnDossani
This problem is caused by the relationship between our Animation component and listeners. As best I can tell the problems arise from higher order components causing problems. In #554 I tried out a different strategey for getting around this, that boils down to adding invisible two copies of the series of concern, like so
<XYPlot>
<VerticalBarSeries animation data={mydata}>
<VerticalBarSeries data={mydata} opacity={0} onMouseOut={() => {...}}>
</XYPlot>

So that the invisible series handles all of the mouse events and the visible series is animated.  If we can't come up with a better solution to this problem, this solution will be integrated into the main library.

My workaround was to have animation in state, and set it to true by default, set it to false on onValueMouseOver, and set it back to true on onValueMouseOut.

@mizozobu, your solution works great.  wanted to note that the same issue with annimation exists for higher order components that make width/height flexible.   One may need to remove the HOC to use this solution.

Yes it's not working with HOC makeFlexible but I fix it by using XYPlot as a child component of  ReactResizeDetector
<ReactResizeDetector handleWidth={true} onResize={this.onResize}>                 {(width) =>                 (<XYPlot width={width} height={this.props.height} xType="ordinal" animation={false} >


Codegenerate StructureShape, BlobShape, application, server and Python runtime

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Documentation blocks have also been duplicated from the codegen-server subproject. I would opt for replacing these with a pointer to the server subproject corresponding class/method, and only having comments pertaining to the Python-specific bits.


feat(429 retry after handling): handle 429 retry after from idBroker

add regex to parse API name form idbroker url
add map to store API to retry after time
add methods to check for retry after
add tests

@ianwremmel  here are the basics so far.
Cheers.

Created interceptor and UTs

Addressed code review feedback

@ianwremmel  any further concerns here?
Getting pinged asking when this is getting merged in.

@Ian got rid of the ternary operators; you're making an old embedded systems developer sad, but I can avoid their use in the future if it is going to be so controversial. I honestly find them much more readable, and when they are not nested to an insane degree, I prefer them.
Updated to use lolex, moved some stuff around, and I did add the interceptor in the core.

@ianwremmel Tagged the wrong Ian on last comment....
got rid of the ternary operators; you're making an old embedded systems developer sad, but I can avoid their use in the future if it is going to be so controversial. I honestly find them much more readable, and when they are not nested to an insane degree, I prefer them.
Updated to use lolex, moved some stuff around, and I did add the interceptor in the core.
Cheers.

already merged

Merged. Sorry about skipping cleanup here.


PR build job didn't update commit status during build

When a new pull request is opened in the project and the author of the pull request isn't white-listed, builder will ask "Can one of the admins verify this patch?".
"ok to test" to accept this pull request for testing.
Under this scenario, pr build is launched. However PR build job didn't update commit status during build.
Infor from concole;
FileNotFoundException means that the credentials Jenkins is using is probably wrong. Or the user account does not have write access to the repo.
https://ci.adoptopenjdk.net/view/OpenJDK tests/job/PR_sanity_openjdk_test_x86-64_linux/41/
didn't update status in https://github.com/AdoptOpenJDK/openjdk-tests/pull/88

It's probably because the user is not in whitelist. @renfeiw is now in the whitelist, we will see as he will have another PR in a short time.

What is the status of this issue, is it resolved and can be closed?

I believe this can be closed, as we use Grinders and Sandbox builds to test changes.  @sophia-guo
if you disagree, please re-open.


Added examples to RegEx doc #5827

Do you guys think it's good or does it need improvement?

It looks good to me. Not sure how it will format in the online docs, but we can always adjust it :)


PGP Hook not running

The shell I have looks more or less like this
pkgs.mkShell {
  sopsPGPKeyDirs = [
    "./keys/host"
    "./keys/users"
  ];
  inputsFrom = [ projectEnv ];
  nativeBuildInputs = [
    (sops-nix { inherit pkgs; }).sops-pgp-hook
    spago2nix'
  ];
}

but I also have an .envrc which calls use flake at the end. Long story short, I have the correct .asc files in the above mentioned folders, but the hook doesn't seem to run since the environment variable doesn't exist.
Even if I direnv block, followed by sops secrets.yaml I get

config file not found and no keys provided through command line options

Also not sure what to make of this
$ nix-shell -p sops --run "sops secrets.yaml"
/nix/store/npdq5pilrfsklvhx0ws44lykc1rl1i66-stdenv-linux/setup: line 81: sopsPGPHook: command not found
config file not found and no keys provided through command line options

$ sops secrets.yaml
config file not found and no keys provided through command line options

Ok never mind ... I still forget that when using flakes I need to adjust my import paths a bit. Replacing (sops-nix ...) with sops-nix.packages.x86_64-linux.sops-pgp-hook works


upgrade-assistant 0.5.564 crashes immediately on macOS (x64) running with net8.0  Unable to load shared library 'hostfxr'

Describe the bug
Crashes immediately
To Reproduce
$ upgrade-assistant --version
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'hostfxr' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable:
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/hostfxr.dylib, 1): image not found
dlopen(/Users/xxx/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/hostfxr.dylib, 1): image not found
dlopen(hostfxr.dylib, 1): image not found
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libhostfxr.dylib, 1): image not found
dlopen(/Users/xxx/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/libhostfxr.dylib, 1): image not found
dlopen(libhostfxr.dylib, 1): image not found
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/hostfxr, 1): image not found
dlopen(/Users/xxx/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/hostfxr, 1): image not found
dlopen(hostfxr, 1): image not found
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libhostfxr, 1): image not found
dlopen(/Users/xxx/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/libhostfxr, 1): image not found
dlopen(libhostfxr, 1): image not found

   at Microsoft.Build.Locator.NativeMethods.hostfxr_resolve_sdk2(String exe_dir, String working_dir, hostfxr_resolve_sdk2_flags_t flags, hostfxr_resolve_sdk2_result_fn result)
   at Microsoft.Build.Locator.DotNetSdkLocationHelper.GetDotNetBasePaths(String workingDirectory)+MoveNext()
   at Microsoft.Build.Locator.DotNetSdkLocationHelper.GetInstances(String workingDirectory)+MoveNext()
   at Microsoft.Build.Locator.MSBuildLocator.GetInstances(VisualStudioInstanceQueryOptions options)+MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.FilterForBitness(IEnumerable`1 instances)+MoveNext() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 92
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToList()
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.FindMSBuildPath() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 68
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.RegisterMsbuild() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 28
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.Initialize() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 23
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider.RunCompositionInitializers(IEnumerable`1 initializers) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 86
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider.InitializeContainer(IEnumerable`1 initializers) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 69
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider..ctor(IAppSettings settings, String extensionsPath, IEnumerable`1 initializers, ILogger logger, StatusContext statusContext) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 31
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.<ConfigureExports>b__8_0(StatusContext statusContext) in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 74
   at Spectre.Console.Status.<>c__DisplayClass15_0`1.<Start>b__0(StatusContext ctx) in /_/src/Spectre.Console/Live/Status/Status.cs:line 60
   at Spectre.Console.Status.<>c__DisplayClass17_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:line 120
--- End of stack trace from previous location ---
   at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:line 133
--- End of stack trace from previous location ---
   at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:line 40
   at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:line 116
   at Spectre.Console.Status.StartAsync[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:line 117
   at Spectre.Console.Status.Start[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:line 61
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.ConfigureExports() in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 68
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.Build() in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 57
   at Program.<Main>$(String[] args) in D:\a\_work\1\s\src\cli\Program.cs:line 7
   at Program.<Main>(String[] args)
Abort trap: 6

$ dotnet tool list -g upgrade-assistant
Package Id             Version      Commands
-----------------------------------------------------
upgrade-assistant      0.5.564      upgrade-assistant

$ dotnet --info
.NET SDK:
 Version:           8.0.100
 Commit:            57efcf1350
 Workload version:  8.0.100-manifests.6c33ef20

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx-x64
 Base Path:   /usr/local/share/dotnet/sdk/8.0.100/

.NET workloads installed:
 Workload version: 8.0.100-manifests.6c33ef20
There are no installed workloads to display.

Host:
  Version:      8.0.0
  Architecture: x64
  Commit:       5535e31a71

.NET SDKs installed:
  6.0.417 [/usr/local/share/dotnet/sdk]
  7.0.311 [/usr/local/share/dotnet/sdk]
  8.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.32 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.17 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.25 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.14 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.23 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.32 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.17 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.25 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.14 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

I have the same problem.

I had the same problem but was able to solve it by installing dotnet from https://dotnet.microsoft.com/en-us/download/dotnet/8.0 instead of from homebrew.

Also happening in Alpine Docker Container (pulled latest today)
docker images
REPOSITORY                                    TAG            IMAGE ID       CREATED         SIZE
mcr.microsoft.com/dotnet/sdk                  8.0-alpine     c0c01e15d6aa   11 days ago     694MB

docker run -it --rm mcr.microsoft.com/dotnet/sdk:8.0-alpine "ash"

/ # dotnet --info
.NET SDK:
 Version:           8.0.100
 Commit:            57efcf1350
 Workload version:  8.0.100-manifests.6c33ef20

Runtime Environment:
 OS Name:     alpine
 OS Version:  3.18
 OS Platform: Linux
 RID:         linux-musl-x64
 Base Path:   /usr/share/dotnet/sdk/8.0.100/

.NET workloads installed:
 Workload version: 8.0.100-manifests.6c33ef20
There are no installed workloads to display.

Host:
  Version:      8.0.0
  Architecture: x64
  Commit:       5535e31a71

.NET SDKs installed:
  8.0.100 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

/ # dotnet tool install -g upgrade-assistant
Skipping NuGet package signature verification.
Tools directory '/root/.dotnet/tools' is not currently on the PATH environment variable.
If you are using bash, you can add it to your profile by running the following command:

cat << \EOF >> ~/.bash_profile
# Add .NET Core SDK tools
export PATH="$PATH:/root/.dotnet/tools"
EOF

You can add it to the current session by running the following command:

export PATH="$PATH:/root/.dotnet/tools"

You can invoke the tool using the following command: upgrade-assistant
Tool 'upgrade-assistant' (version '0.5.564') was successfully installed.

/ # export PATH="$PATH:/root/.dotnet/tools"

/ # upgrade-assistant --version
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'hostfxr' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable:
Error loading shared library /usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/hostfxr.so: No such file or directory
Error loading shared library /root/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/hostfxr.so: No such file or directory
Error loading shared library hostfxr.so: No such file or directory
Error loading shared library /usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libhostfxr.so: No such file or directory
Error loading shared library /root/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/libhostfxr.so: No such file or directory
Error loading shared library libhostfxr.so: No such file or directory
Error loading shared library /usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/hostfxr: No such file or directory
Error loading shared library /root/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/hostfxr: No such file or directory
Error loading shared library hostfxr: No such file or directory
Error loading shared library /usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libhostfxr: No such file or directory
Error loading shared library /root/.dotnet/tools/.store/upgrade-assistant/0.5.564/upgrade-assistant/0.5.564/tools/net8.0/any/libhostfxr: No such file or directory
Error loading shared library libhostfxr: No such file or directory

   at Microsoft.Build.Locator.NativeMethods.hostfxr_resolve_sdk2(String exe_dir, String working_dir, hostfxr_resolve_sdk2_flags_t flags, hostfxr_resolve_sdk2_result_fn result)
   at Microsoft.Build.Locator.DotNetSdkLocationHelper.GetDotNetBasePaths(String workingDirectory)+MoveNext()
   at Microsoft.Build.Locator.DotNetSdkLocationHelper.GetInstances(String workingDirectory)+MoveNext()
   at Microsoft.Build.Locator.MSBuildLocator.GetInstances(VisualStudioInstanceQueryOptions options)+MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.FilterForBitness(IEnumerable`1 instances)+MoveNext() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 92
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToList()
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.FindMSBuildPath() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 68
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.RegisterMsbuild() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 28
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.Initialize() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 23
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider.RunCompositionInitializers(IEnumerable`1 initializers) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 86
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider.InitializeContainer(IEnumerable`1 initializers) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 69
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider..ctor(IAppSettings settings, String extensionsPath, IEnumerable`1 initializers, ILogger logger, StatusContext statusContext) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 31
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.<ConfigureExports>b__8_0(StatusContext statusContext) in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 74
   at Spectre.Console.Status.<>c__DisplayClass15_0`1.<Start>b__0(StatusContext ctx) in /_/src/Spectre.Console/Live/Status/Status.cs:line 60
   at Spectre.Console.Status.<>c__DisplayClass17_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:line 120
--- End of stack trace from previous location ---
   at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:line 133
--- End of stack trace from previous location ---
   at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:line 40
   at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:line 116
   at Spectre.Console.Status.StartAsync[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:line 117
   at Spectre.Console.Status.Start[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:line 61
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.ConfigureExports() in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 68
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.Build() in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 57
   at Program.<Main>$(String[] args) in D:\a\_work\1\s\src\cli\Program.cs:line 7
   at Program.<Main>(String[] args)
Aborted
/ #

I am encountering a similar issue with the Upgrade Assistant (version 0.5.711) on a Mac M1 machine. When attempting to use the tool, I receive the following error:
~ % upgrade-assistant

▰▱▱▱▱▱▱ Initializing and loading extensions...
                                              No .NET SDKs were found.

Download a .NET SDK:
https://aka.ms/dotnet/download

Unhandled exception. System.InvalidOperationException: Failed to find an appropriate version of .NET Core MSBuild. Call to hostfxr_resolve_sdk2 failed. There may be more details in stderr.
   at Microsoft.Build.Locator.DotNetSdkLocationHelper.GetDotNetBasePaths(String workingDirectory)+MoveNext()
   at Microsoft.Build.Locator.DotNetSdkLocationHelper.GetInstances(String workingDirectory)+MoveNext()
   at Microsoft.Build.Locator.MSBuildLocator.GetInstances(VisualStudioInstanceQueryOptions options)+MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.FilterForBitness(IEnumerable`1 instances)+MoveNext() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 92
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToList()
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.FindMSBuildPath() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 68
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.RegisterMsbuild() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 28
   at Microsoft.UpgradeAssistant.Cli.Dependencies.MsBuildRegistrationInitializer.Initialize() in D:\a\_work\1\s\src\cli\Dependencies\MsBuildRegistrationInitializer.cs:line 23
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider.RunCompositionInitializers(IEnumerable`1 initializers) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 86
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider.InitializeContainer(IEnumerable`1 initializers) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 69
   at Microsoft.UpgradeAssistant.Cli.Dependencies.ExportProvider..ctor(IAppSettings settings, String extensionsPath, IEnumerable`1 initializers, ILogger logger, StatusContext statusContext) in D:\a\_work\1\s\src\cli\Dependencies\ExportProvider.cs:line 31
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.<ConfigureExports>b__8_0(StatusContext statusContext) in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 73
   at Spectre.Console.Status.<>c__DisplayClass15_0`1.<Start>b__0(StatusContext ctx) in /_/src/Spectre.Console/Live/Status/Status.cs:line 60
   at Spectre.Console.Status.<>c__DisplayClass17_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:line 120
--- End of stack trace from previous location ---
   at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in /_/src/Spectre.Console/Live/Progress/Progress.cs:line 133
--- End of stack trace from previous location ---
   at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync[T](Func`1 func) in /_/src/Spectre.Console/Internal/DefaultExclusivityMode.cs:line 40
   at Spectre.Console.Progress.StartAsync[T](Func`2 action) in /_/src/Spectre.Console/Live/Progress/Progress.cs:line 116
   at Spectre.Console.Status.StartAsync[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:line 117
   at Spectre.Console.Status.Start[T](String status, Func`2 func) in /_/src/Spectre.Console/Live/Status/Status.cs:line 61
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.ConfigureExports() in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 67
   at Microsoft.UpgradeAssistant.Cli.UpgradeCommandAppBuilder.Build() in D:\a\_work\1\s\src\cli\UpgradeCommandAppBuilder.cs:line 56
   at Program.<Main>$(String[] args) in D:\a\_work\1\s\src\cli\Program.cs:line 7
   at Program.<Main>(String[] args)
zsh: abort      upgrade-assistant

Operating System: macOS 14.5 (Mac M1)
.NET SDK Versions Installed:
~ % dotnet --list-sdks
6.0.424 [/usr/local/share/dotnet/sdk]
8.0.303 [/usr/local/share/dotnet/sdk]

.NET Runtime Versions Installed:
~ % dotnet --list-runtimes
Microsoft.AspNetCore.App 6.0.32 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.32 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

MSBuild Version:
% dotnet msbuild -version
MSBuild version 17.10.4+10fbfbf2e for .NET
<IP_ADDRESS>02

I have tried reinstalling the tool and ensuring that all necessary environment variables are correctly set. However, the issue persists.
Steps to Reproduce:

Install Upgrade Assistant version 0.5.711.
Run upgrade-assistant command.
Observe the error message.

Any assistance or guidance on resolving this issue would be greatly appreciated.
Thank you!

Can you try installing the dotnet 8.0 runtime/sdk? I've always run this using dotnet 8.0

Sorry, just noticed you have net8.0 installed

I just made sure to install the exact same versions of the SDK/runtimes and updated my upgrade-assistant as well and am unable to reproduce this issue :(

Thanks for the quick response!
I’ve verified that .NET 8.0 SDK and runtime are installed, as you suggested. Additionally, I’ve tried uninstalling and reinstalling various versions of the Upgrade Assistant.
Since the issue persists, could there be any other underlying factors or configurations that might be impacting the Upgrade Assistant? Any additional guidance or troubleshooting steps you could provide would be greatly appreciated.
Below is the output of dotnet --info for your reference:
~ % dotnet --info
.NET SDK:
 Version:           8.0.303
 Commit:            29ab8e3268
 Workload version:  8.0.300-manifests.c915c39d
 MSBuild version:   17.10.4+10fbfbf2e

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  14.5
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/8.0.303/

.NET workloads installed:
There are no installed workloads to display.

Host:
  Version:      8.0.7
  Architecture: arm64
  Commit:       2aade6beb0

.NET SDKs installed:
  6.0.424 [/usr/local/share/dotnet/sdk]
  8.0.303 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.32 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.32 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

@jstedfast in my case, the issue was related to MSBuildLocator and was fixed by adding /usr/local/share/dotnet to the PATH environment variable as suggested here.
Thanks for your support!


In the Dataset tutorial, the conversion of MapDataset to IterDataset is not quite clear

The infromation in tutorial next:
Any MapDataset can be turned into a IterDataset by calling to_iter_dataset. When possible this should happen late in the pipeline since it will restrict the transformations that can come after it (e.g. global shuffle must come before). This conversion by default skips None elements.
But calling map_dataset.to_iter_dataset() converts it to PrefetchIterDataset class. What the problem? This class is not iterable. Also we can't get state from it. So it's not easy to understand why you mentioned it that way, and why it wasn't used in the tutorial at all.
Example for reproduction:
# !pip install -q jax-ai-stack[grain]==2024.11.1
import chex
import grain.python as pygrain

class Source(pygrain.RandomAccessDataSource):
    def __init__(self, x:chex.Array, y:chex.Array) -> None:
        assert (len(x) == len(y)), "must be the same length"
        self.x = x
        self.y = y

    def __len__(self) -> int:
        return len(x)

    def __getitem__(self, idx: int) -> tuple[chex.Array]:
        return self.x[idx], self.y[idx]

x = range(10)
y = [0]*5 + [1]*5
data_source = Source(x, y)

dataset = (
    pygrain.MapDataset.source(data_source)
    .shuffle(seed=seed)
    .map(lambda x: x)
    .batch(batch_size=5)
)
# from tutorila, works fine
iter_dataset = iter(dataset)
print(iter_dataset.get_state())
print(next(iter_dataset))

# creates PrefetchIterDataset with to_iter_dataset()
iter_dataset2 = dataset.to_iter_dataset()
print(type(iter_dataset2))

# AttributeError: 'PrefetchIterDataset' object has no attribute 'get_state'
print(iter_dataset2.get_state())
# TypeError: 'PrefetchIterDataset' object is not an iterator
print(next(iter_dataset2))

# works fine
iter_dataset2 = iter(iter_dataset2)
print(iter_dataset2.get_state())
print(next(iter_dataset2))

PrefetchIterDataset is an instance of IterDataset that is an Iterable. Note that Iterable and Iterator are different things. Iterable only has to implement __iter__ that will return an Iterator. Iterator implements __next__ and grain's DatasetIterator additionally implements get/set_state. So for iter_dataset2 you are missing iter(iter_dataset) call to produce an iterator that you're expecting.


Different Workspace Location

Hey,
Thank you for creating this great repository!
Is there a way to change the workspace location to be /home/ros/workspace on the docker container? I would like to ask as well what would be the best way to map a directory(on host) for the docker container that contains multiple ROS packages relying on each other? I am aware of the ros2.repos solution you have implemented but most of the packages I would like to use are private repositories and it makes it harder to automatise with the SSH keys.

Glad you figured it out!


Update README.md

Hi @zhangyangyu,
Thanks for your pull request!
tidbcloudy is a community project that provides a Python interface to call TiDB Cloud related APIs.
I have updated the README in https://github.com/Oreoxmt/tidbcloudy/commit/89ce1dd0e5b9f6191f16adfb140fa61b624e8b89 to provide more details and incorporate tidbcloud-cli into the related projects section.
As your changes have now been integrated, I'm going to proceed with closing this pull request. Thanks for your suggestions!


Flexx running on RPi with static IP address?

Hi,
I have a Raspberry Pi 3 (Raspberry Pi OS) running a flexx application with Python 3.7 on port 80.
The application runs without problems when I'm using DHCP to get the IP address, but when I set the Raspberry to a static IP address in /etc/dhcpcd.conf, the application doesn't seem to remember session cookies. The application only remembers cookies on the Raspberry itself if I open the application in Chromium browser and navigate to it (http://localhost), but when trying to connect from a remote machine in a Chromium browser at http://<IP_ADDRESS> that is in the same LAN, the cookies are always None.
I have a simple login page (a PyWidget at http://localhost) that if a user logs in correctly some cookies are set (using self.session.set_cookie), then a redirection opens a second page (another PyWidget at http://localhost/mainpage), then the cookies are read and used for a few things (using self.session.get_cookie).
The problem is that all cookies are None on the second page where they are read when the Raspberry has a static IP address, and I have no idea why.
Any help would be greatly appreciated, as I'm not that knowledgable in this stuff. I may even be doing something wrong, so please correct me, if that is the case.
Thank you
Matic

Are the page that sets the cookies, and the page that tries to load them, both on http://<IP_ADDRESS>/?

Yes, they are http://<IP_ADDRESS>/ and http://<IP_ADDRESS>/mainpage.

And both on port 80, and http (not https), right? In that case I don't see why the cookies should not work. But it looks like a networking issue, not specific to Flexx.

And both on port 80, and http (not https), right?
Yes

I did some further testing and found that setting a static IP in the router's network (192.168.1.X) does not have this problem, it's only when I set it to a different subnet (in my example 192.168.11.X) is when the cookies stop working.

CONCLUSION:
This is not a Raspberry Pi problem, it is a date/time problem:

The problem: cookies do not get stored when using Widget.set_cookie
The reason: Date-and-time difference between RaspberryPi and client are too large
The solution: Update date-and-time on both RaspberryPi and client to be the same, or at least in the cookie expiration time window

Thans for the update. I learned from it as well :)


Autocomplete ahead of cursor is not working

Issue
The autocomplete feature where commands are displayed ahead of the cursor looks super slick, but unfortunately it is not working for me. I'm probably doing something wrong, would love to know what.
Environment
Operating system: Linux and Mac
Terminal emulator: iTerm
ZSH version: 5.0.5 (Linux), 5.2 (Mac)
Relevant .zshrc
# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

# Path to your oh-my-zsh installation.
export ZSH=/Users/olszowy/.oh-my-zsh

# Set name of the theme to load. Optionally, if you set this to "random"
# it'll load a random theme each time that oh-my-zsh is loaded.
# See https://github.com/robbyrussell/oh-my-zsh/wiki/Themes
ZSH_THEME="spaceship"

# Which plugins would you like to load? (plugins can be found in ~/.oh-my-zsh/plugins/*)
# Custom plugins may be added to ~/.oh-my-zsh/custom/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git)

source $ZSH/oh-my-zsh.sh

# User configuration

alias mountfs="~/scripts/mount.sh"

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

source "/Users/olszowy/.oh-my-zsh/custom/themes/spaceship.zsh-theme"

Hey @oldchevy I think you meant zsh-autosuggestions. It's a zsh plugin and it's awesome.

Awesome, thank you :)
I think it may be worth adding a note in the README for people getting started with Zsh, that the autocomplete in the gif is separate from the theme.

Thank you for the suggestion. We'll take a look into it.
See https://github.com/denysdovhan/spaceship-zsh-theme/issues/52#issuecomment-284223729 for author's setup shown in README.


Fix in memory dao code problem

Why?
When trying to login with club credentials, without checking the club box, the app encountered an error. No error would be displayed on the user views, even though it was wrong.
Similarly with students, if you tried signing up with student credentials and checking the club box, the app would cause and error in the terminal.
The user was never warned of the error, and so the screen would remain the same even after causing it.
What changed?
I split the existsByEmail and existsByName each into two separate methods respectively for students and clubs. This also meant I had to do some further refactoring in the dataAccessInterface of select use cases. Furthermore, I added specifications in the java docs of some methods in the form of preconditions, to ensure that the user doesn't use the methods on unexpected inputs.
Finally, I revamped the in memory DAO methods to work with the new changes.
Did I respect CA/SOLID principles?
YES! With all these changes, I made sure to add the necessary abstractions between the use case layer and the data access layer, hence abiding to the CA principles. Moreover, I eliminated all interdependencies, enabling us to expand the code base in the future without having to worry about other parts of the code being affected (Open/Closed and Liskov principles).
Additional information:
Since Kabir will work on the firestore database DAO, I decided to comment out all the pre-existing code in that file in order to not cause any errors. As this will be implemented in the near future, this shouldn't be a problem for now as the app runs on the in memory DAO .
Contributor: Karl

Hi Karl, the updates look good lays a good foundation for implementing the persisting data version on firebase. I will keep in mind whether to split the two types of users into array lists or into firestore collection types to avoid similar bugs.

Hi good job on the code! I don't see any problems with the logic. A concern though is the coupling of both clubs and students in one DAO. So far, it seems the DAO has multiple uses (i.e. storing Student specific things and Club specific things), which correlates to different use cases. This may be in violation of the single responsibility principle. One way I think we can solve this is to use a User DAO interface with a generic value, and creating subclass DAOs while specifying the user type in the generic.
Let me know if you think this is a concern!

Hey Roy. I believe you brought up an amazing point. However, do we really need to worry about solid principles for this DAO if the DAO's sole purpose is testing?  On the CAUser Login, the java doc for the class says "This implementation does NOT persist data between runs of the program.". This suggests that the class should not be used for any other purpose other than testing (which is the case).

Passing the pull request as the only problem is related to coupling issues and Kabir wants to works on it. Keep your eyes for a new pr very soon, this time the right version from Kabir


[Question]The yurtctl binary file build in macOs can't execute

What happened:
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

OpenYurt version:
Kubernetes version (use kubectl version):
OS (e.g: cat /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

others
/kind question

/assign @Peeknut

@mauricewei Would you like to take over this problem and fix it?

@rambohe-ch I am willing to fix it and commit pull request later


[bug/css] page layout depends on number of parks

Current Behavior

Expected Behavior
As you reduce the number of parks, the layout of the page stages the same (the state of the page layout when there are 6 parks).

I'll take this issue.


Bump Dependencies #3575297194

Bumps dependencies for product-is. Link : https://github.com/wso2/product-is/actions/runs/3575297194

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.jenkins-is-staging seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


hotfix/1148-search

What does this pull request aim to achieve? 💯

fixed bug where switching tabs on mobile (after already selecting one) renders the new selected label, but still shows results of previous selected tab.

Please be concise and link any related issue(s):
N/A

[x] I have checked that no one else is working on similar changes.
[x] I have read the latest README and understand the development workflow.
[x] The name of this branch is: feature/<branch_name>, or hotfix/<branch_name> if this is a hotfix, where <branch_name> briefly describes the issue(s) resolved and is prefixed with the issue number(s) (e.g. 1127-1130-update-git-branching-model). The branch is created in a cloned version of this repo, not a forked repo.
[x] This branch is rebased/merged with the latest staging (or the latest production if it is a hotfix).
[x] There are no merge conflicts.
[x] There are no console warnings and errors.
[x] On the right hand side of this PR, I have linked the appropriate issue(s) under "Issues" and added a project progress under "Projects".
[x] The title describes the issue(s) being addressed using format: <issue_numbers> - <brief_issue_description> (e.g. #1127/#1130 - Update Git Branching Model)

@idjevm @joshmorel @rabbi08 please review this PR, it would be good if we can merge it soon.

Verified on hotfix feature branch this is working fine. Thanks.
Please below gif


Update README.md

Integration test for snapshot application-sample-9f5rw and scenario example has passed

Task
Duration
Test Suite
Status
Details

task-success-2
5s

:heavy_check_mark: SUCCESS

task-skipped
5s

:white_check_mark: SKIPPED

task-success
4s

:heavy_check_mark: SUCCESS

Integration test for snapshot application-sample-499rb and scenario example has passed

Task
Duration
Test Suite
Status
Details

task-success
5s

:heavy_check_mark: SUCCESS

task-success-2
4s

:heavy_check_mark: SUCCESS

task-skipped
4s

:white_check_mark: SKIPPED

Integration test for snapshot application-sample-wgt5x and scenario example was deleted before the pipelineRun could finish
Integration test which is running as pipeline run 'application-sample-wgt5x-m6tlc', has been deleted

Integration test for snapshot application-sample-lbl6x and scenario example was deleted before the pipelineRun could finish
Integration test which is running as pipeline run 'application-sample-lbl6x-lkbj7', has been deleted

/retest

/retest

/retest

/retest


apply previous pr's to main branch and update eslint deps

Apply other suggested pr changes for react 17, also update eslint and its dependencies so the precommit hook runs correctly :+1:

bump @iamacup ?


iOS8 crash -[AFURLConnectionOperation connectionDidFinishLoading:]

Hi,
there seems to be frequent crash on iOS8 only.
AFURLConnectionOperation.m line 685
-[AFURLConnectionOperation connectionDidFinishLoading:]

I am i am investigating whats what.

@mattt
the same thing is happening to me and I couldn't find any help.

Same issue. iOS 9.2.x greater and iOS 7 only. (not appear on iOS 8)
EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000000000000000
-[AFURLConnectionOperation connectionDidFinishLoading:] (AFURLConnectionOperation.m:691)


How does file sync tunnel works? port-forward or ssh tunnel?

It uses the same mechanism as kubectl exec except that we use a websocket, not spdy.  So the communication is piped through the k8s api server using standard k8s apis.

Since this is a question, I am going to convert it to a discussion


Axelar - MAINNET QA

Review Info

Description
Info

Subgraph Reviewed
https://subgraphs.messari.io/subgraph?endpoint=https://api.thegraph.com/subgraphs/name/tnkrxyz/axelar-ethereum&tab=protocol

Schema Version
1.1.0

Subgraph Version
1.0.0

Methodology Version
1.0.0

Evidence Spreadsheet
https://docs.google.com/spreadsheets/d/14m3StI-RskdRGi8uh6Qx9zbdnkScknd3/edit?usp=sharing&ouid=104027705798814439445&rtpof=true&sd=true

Metric to Review
Revenue

Axelar does not generate revenue since it does not impose fees, and only the relayer gas fees are collected. You may refer to this link: https://axelar.network/blog/cross-chain-gas-costs-on-satellite-explained
Refer to the sheet 'Protocol'

Total Pool Count and Total Supported Token Count

Axelar's official website displays that there are solely 18 tokens native to Ethereum, which contrasts with the 24 tokens listed on the subgraph.

Please refer to the attached worksheet for more details.

@tnkrxyz QA for axelar.

@tnkrxyz Just wanted to follow up on this QA in case it got buried!

Response to QA comments ** WIP**

Review Info

Metric to Review

Revenue

Axelar does not generate revenue since it does not impose fees, and only the relayer gas fees are collected. You may refer to this link: https://axelar.network/blog/cross-chain-gas-costs-on-satellite-explained

I am aware of this, but I remember seeing somewhere that it is possible to enable protocol fees (current set to 0). Let me see if I can find it.

Total Pool Count and Total Supported Token Count

Axelar's official website displays that there are solely 18 tokens native to Ethereum, which contrasts with the 24 tokens listed on the subgraph.

Please refer to the attached worksheet for more details.

Dashboard linke for ETH:  https://subgraphs.messari.io/subgraph?endpoint=https://api.thegraph.com/subgraphs/id/QmYqtQzyWSc5LPevXUesaw4bP5VzmK6QXVH98s1H9PeNaK&tab=protocol

Looks good :)

Should this be prod now and in PM? @pietydespises @bye43

Yes, I was waiting to do all of axelar, but will do this now that its taking a little longer


Revert "Use AndroidX in new projects by default"

It looks like this broke some test in the device lab.
Reverts flutter/flutter#40925

LGTM. Can we file a bug to describe what went wrong for posterity?

In one of the cases, it looks like just a line in one test.  I'm still investigating.


Release 3.8.0

Codecov Report

Merging #2263 (299dae4) into main (78f6cc1) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 299dae4 differs from pull request most recent head c3daafb. Consider uploading reports for the commit c3daafb to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@           Coverage Diff           @@
##             main    #2263   +/-   ##
=======================================
  Coverage   68.17%   68.17%
=======================================
  Files         138      138
  Lines       16739    16739
=======================================
  Hits        11412    11412
  Misses       5327     5327

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Some small fixes to recipe package

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-375/1 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-375/1/index.html


Is it possible to "decorate" jsonPayload?

I'd like to add some data to all log messages, like a request id for tracing purposed for example. Any idea how I might accomplish that? I tried adding a processor to all log handlers but it seems like that data doesn't get passed to the app['log']->listen. Thanks!

If this feature is supported by the GCP Logging API then we should be able to implement this here. Otherwise, no, since this is just a wrapper for their APIs

If this feature is supported by the GCP Logging API then we should be able to implement this here. Otherwise, no, since this is just a wrapper for their APIs

It's not really a gcp feature, I used the request id as an, but it could really be anything like memory usage, or some detail about the user. In short I'm looking to add extra data to the context of all log messages.

It's not really a gcp feature, I used the request id as an, but it could really be anything like memory usage, or some detail about the user. In short I'm looking to add extra data to the context of all log messages.

I think this is outside the scope of this package. We simply listen for calls to the Laravel log method:
$this->app['log']->listen(function () {
    $args = Arr::first(func_get_args());
    $this->app['Stackdriver\Logger']->log(
        $args->level,
        $args->message,
        $args->context
    );
});

So I guess it is best to find a Laravel compatible solution. Have you checked this?

I think this is outside the scope of this package. We simply listen for calls to the Laravel log method:
$this->app['log']->listen(function () {
    $args = Arr::first(func_get_args());
    $this->app['Stackdriver\Logger']->log(
        $args->level,
        $args->message,
        $args->context
    );
});

So I guess it is best to find a Laravel compatible solution. Have you checked this?

If I'm following, I'm trying to do something very similar... I have a bunch of different queues (one per concurrently-running job) and I'm looking to isolate which job created which job.  I came here to see if there was a similar question already, and well here we all are!
GCP does support structured logs, and the labels property of the LogEntry object allows for a user-defined map of key/val pairs.  So it sounds exactly like what we're after.
I've just not yet traced things back up the stack to see that laravel-stackdriver is using the LogEntry object... or really how this sausage is made.  I'm just starting that research now.

It's looking like the easiest way to pull this off is to tweak the config during runtime, since the static method StackdriverExceptionHandler::report that we all had to add to App\Exceptions\Handler.php is pulling the array of labels from a the stackdriver config file.  In my app I believe I'm going to be able to pull that off with some simple middleware.
Here's a gist for the middleware that I wrote to accomplish this.
Then, standard Laravel (Laravel7 for me) stuff to get it to be called...

Register the middleware in the $routeMiddleware array inside of App\Http\Kernel.php.
Decorate the appropriate routes with the call to the middleware... in my case a handful of route groups.

I ought to be in-testing this method tomorrow... and this is my first Laravel project (hell, my first MVC project) so the usual "zero-warranties" disclaimers apply.

Hey all.  I've abandoned this original idea.
Last night I finally got back to the original idea, and have tried placing the "labels" associative array in a few different places , yet none worked.  So I RTFM'd a bit more, and found (go easy, I'm new to Laravel) that the second parameter to all of the Log::<level> calls (which is named "context") accepts an associative array which behaves exactly as we were describing above.
What happens is that when it's consumed in Stackdriver, each of the keys end up being properties inside of the jsonPayload element, and the value of the property is (unsurprisingly) the value you gave that same property in the associative array.
So, here's what one log entry in my system looks like (where I added the "Process-Job" property):
{
insertId: "rAnd0mN01z3"
  jsonPayload: {
    Process-Job: "53"
    message: "Publishing vendor 75 to store 151 (retailer 3183100)"
  }
  resource: {
    type: "global"
    labels: {
      project_id: "names-redacted-to-protect-the-innocent"
    }
  }
  timestamp: "2021-06-22T04:46:00.742580Z"
  severity: "DEBUG"
  logName: "projects/names-redacted-to-protect-the-innocent"
  receiveTimestamp: "2021-06-22T04:46:15.172304700Z"
}

HTH


build rmw_connext_dynamic_cpp in two parts to match ABI of Connext as well as ROS

While ROS 2 packages on Xenial are always built with the new ABI (_GLIBCXX_USE_CXX11_ABI 1) Connext might use either or. This patch moves all API which interacts with the ROS messages(namely the std::string fields) into a separate compilation unit which is always build with the new ABI. The remaining rmw functions interacting with Connext are built with the same ABI as Connext. Since the interface between the two compilation units doesn't contain anything which is different between the two ABIs it doesn't pose a problem that both are built with a different ABI.

No change with the custom Debian package of Connext: http://ci.ros2.org/job/ci_linux/1624/
Also no change on OS X and Windows:

http://ci.ros2.org/job/ci_osx/1223/
http://ci.ros2.org/job/ci_windows/1590/

And the dynamic tests are passing with the official Connext binaries now:

http://ci.ros2.org/job/ci_linux/1625/ (before http://ci.ros2.org/job/ci_linux/1620/)

The code related to publish and take has only been moved to the separate compilation unit but has not been modified in any way.
Please review.

lgtm, but I still feel that merging https://github.com/ros2/rmw_connext/pull/187 first would be easiest.

Then please go ahead with merging it.

Closing in favor of #191.


Troubles adding an svg

I'm submitting a ...  (check one with "x")
[x] bug report => check the README and search github for a similar issue or PR before submitting
[ ] support request => check the README and search github for a similar issue or PR before submitting
[ ] feature request

Current behavior

add svg to src/client/assets
example css background: url('../../assets/bg.svg') no-repeat center center;
webpack fails to build with error:
/home/<REDACTED>/Projects/<REDACTED>/node_modules/webpack/lib/TemplatedPathPlugin.js:69
                .replace(REGEXP_CHUNKHASH, withHashLength(getReplacer(chunkHash), chunkHashWithLength))
   ^
Error: Path variable [chunkhash] not implemented in this context: assets/bg.[chunkhash].svg

Expected/desired behavior

For the svg to show as a background using relative paths to describe where it should be found.
Minimal reproduction of the problem with instructions

npm run build:spa-hmr
What is the motivation / use case for changing the behavior?

Please tell us about your environment:

Ubuntu 17.04, VsCode, NPM

Angular version: 2.0.X

Angular 4.10

Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]

Language: [all | TypeScript X.X | ES6/7 | ES5]

Node (for AoT issues): node --version =

Seems to be fixed by using [hash] instead of [chunkhash] in webpack.common.js

@jason-candeo thank for reporting this, I'll check this soon.

@jason-candeo  done 👍


User form is empty

What's wrong with mine, why my user form is empty.

Sec, let me check my magic sphere :) Need details, code, errors in console etc.

Thanks, I have found this problem and solve it.


Add subscriptions graphql schema

use processes_resolver in subscription schema to resolve processes of a subscription.

use subscriptions_resolver in processes schema to resolve subscriptions of a process.

add subscriptions db filters and sorting.

add generic filters to remove duplicated code.

TODO: use subscriptions_resolver in subscriptions schema to resolve depends_on and in_use_by subscriptions.

TODO: make date filter better so you don't need an exact isoformat date.

TODO: test subscriptions schema.

Codecov Report

Merging #275 (d6a69d3) into main (4fa647f) will decrease coverage by 0.18%.
The diff coverage is 73.52%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #275      +/-   ##
==========================================
- Coverage   76.12%   75.94%   -0.18%
==========================================
  Files         153      162       +9
  Lines        8217     8455     +238
  Branches     1320     1152     -168
==========================================
+ Hits         6255     6421     +166
- Misses       1772     1842      +70
- Partials      190      192       +2

Impacted Files
Coverage Δ

orchestrator/db/filters/filters.py
90.47% <33.33%> (-7.03%)
:arrow_down:

orchestrator/db/filters/process.py
70.00% <33.33%> (-6.00%)
:arrow_down:

orchestrator/graphql/schemas/subscription.py
59.75% <59.75%> (ø)

...strator/db/filters/generic_filters/range_filter.py
63.82% <63.82%> (ø)

orchestrator/db/filters/subscription.py
71.42% <71.42%> (ø)

orchestrator/graphql/schemas/process.py
97.43% <86.66%> (-2.57%)
:arrow_down:

orchestrator/graphql/resolvers/subscription.py
89.47% <89.47%> (ø)

...rchestrator/db/filters/generic_filters/__init__.py
100.00% <100.00%> (ø)

...estrator/db/filters/generic_filters/bool_filter.py
100.00% <100.00%> (ø)

...rator/db/filters/generic_filters/is_like_filter.py
100.00% <100.00%> (ø)

... and 8 more

... and 7 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Use higher quality .svg icon

Coverage increased (+0.04%) to 94.915% when pulling ea11b06d00a7b68a7c8399d975b8ac983e88d63a on Philippus:patch-1 into c92e0eaab6ed72d73161eb6e9ef82b0e30c307f2 on iheartradio:master.


StrategyQA

@googlebot I signed it!

Thank you for submitting your task! We are in the process of pairing submissions with task reviewers and will be in touch soon.

Thank you so much, @lewkowycz, for reviewing our task and for the useful suggestions!
We were not aware of the append_choices_to_input=False option, I'll make sure to covert the task to this multiple choice format, as you suggested.
Regarding the suggestion of adding a text-to-text format - I completely agree that it would be useful for explainability and evaluation. My only concern is that these facts could be noisy - they were written without many constraints as a "side product" as part of the data collection pipeline. In addition, for some answers, there could be more than a single explanation (so a model could generate a valid answer, but it will get a low score because it doesn't match the facts). Nonetheless, our analyses in the paper show that in most cases, people come up with the same reasoning path for answering strategy questions. Also, in most cases, the facts do provide a valid explanation for the answer.
Overall, I really liked this suggestion as it opens up a new way to evaluate the ability to answer strategy questions. I'll update the task file as you suggested.

@lewkowycz apparently, it is not possible to have both text-to-text and multiple choice evaluation metrics. I am getting this error when running the tests:
Exception: A task can not have both text-to-text and multiple choice metrics.

Therefore, I am reverting to have only a multiple-choice format.

@chiafullo Is it possible to have two formats for the same task? We really liked @lewkowycz's suggestion of having a text-to-text evaluation that considers possible explanations generated by the models, but it would be a shame to ignore the boolean nature of this task (answers are always yes or no).
Thanks!
FYI @yonatansito

@mega002 Thanks a lot for the updates, it looks great! Generative + scoring should be merged today or tomorrow https://github.com/google/BIG-bench/pull/171 , it should be able to pass the tests then, can let you know when it is merged.

@mega002 Just merged that PR, it should work now!

Thanks, @lewkowycz! Updated the task to include the two formats and it passes the tests :+1:

Thanks a lot, it looks great. Almost ready, one last request is if you could add logical reasoning to the keywords. This could be either on top or instead of implicit reasoning, will leave that up to you.

Thanks! I've added it on top because it's a different thing (implicit reasoning means the model needs to infer the reasoning type).

thanks,looks awesome. @chiafullo accept

Thanks to the author for submitting such an interesting task! I'm one of your reviewers.
I like everything as is and have no further suggestions :-)
@chiafullo accept

Thanks a lot @lewkowycz and @nullstring for reviewing our task :pray:


docs: refine README

[ ] test execution description should be before the DAG run. The test downloads additional files. How much disk space, approximately?
[ ] config.yaml can possibly include the same test files as the test run within test/test_singularity_execution/. Alternative: download the additional files as for the test run.
[ ] write how long the test will run on local and/or cluster execution.

after executing bash test/test_singularity_execution/test_local.sh without cleanup, i.e. commenting it out:
cleanup () {
    rc=$?
#    rm -rf .cache/
#    rm -rf .config/
#    rm -rf .snakemake/
#    rm -rf logs/
#    rm -rf results/
    cd $user_dir
    echo "Exit status: $rc"
}

the RCRUNCH total directory size is: 4968M (with du -BM). This contains the test files plus the singularity images.
The test took approx. 8 minutes with the default 4 cores.


[Proposal] Add Support for Custom Memory (Soft) Limit

Heyho,
This proposal PR allows users to set a custom memory limit (RAM, not VRAM).
I have not found a prior issue or PR related to this, but I might have missed it.
Feel free to link any previous discussions.
Context
So far, AutoGluon assumes that the memory limit equals all available memory on the system. AutoGluon uses this memory limit at several points in the program flow to control the fitting behavior at runtime.
This includes (but is likely not limited to): whether to fit certain models at all (based on estimating their required memory) or early stopping models that would go over the memory limit when fitting them further.
Problem & Impact
If AutoGluon is used in an environment with shared resources (e.g., SLURM), or if one would like to avoid fitting very memory-expensive models, then the current behavior is "bugged" or suboptimal, respectively.
From studying some papers that have used AutoGluon, I also suspect several researchers fell into this trap and used AutoGluon incorrectly. To illustrate, in the case of SLURM, this leads to AutoGluon trying to fit models that require more memory than allocated for the SLURM job, resulting in the fit process being killed by SLURM. AutoGluon does not expect this process to be killed, so it cannot just stop fitting early or skip the model in the first place.
Thus, this is quite an important feature we should add and communicate to the user (just like the num_cpus or num_gpus parameters). Moreover, I would love to use this as an official feature instead of my monkey patch workaround, which I have been using for a while now (see below).
Proposed Solution
I opted to use an environment variable in the current format of the resource manager (which is all static).
If we want to make this a parameter like num_cpus or num_gpus, we need to make the resource manager dynamic, requiring many changes. Or add the custom memory limit to all places that require it.
Mokey Ptach Workaround for Custom Memory Limit
To monkey patch the memory limit (e.g., when running on SLURM), call the following function before calling any AutoGluon code or importing AutoGluon. Essentially, call this function before doing anything else in your script. Check the output of AutoGluon to see if the patch worked.
Kudos to @eddiebergman for cleaning up my monkey patch code and turning it into a usable function!
"""Due to AutoGluon's use of psutil to infer memory limit,
we need to patch psutil to return the required memory limit.

This method is easier than patching AutoGluon's internals
and psutil is more likely to be stable.

The use of this code will enforce AutoGluon's memory usage
but may impact any other tools relying on psutil's memory
information.

The use of this function should not be used in production and
is mostly required for systems in which there is a process
enforced memory limit that is not directly accessible to
psutil, i.e. SLURM.
"""

from __future__ import annotations

import warnings

import psutil

def patch_memory_limit(gb: float) -> None:
    """Patch psutil to enforce a maximum memory limit when AutoGluon reads memory
    available.

    Args:
        gb (float): The maximum memory limit in GB.
    """
    psutil._virtual_memory = psutil.virtual_memory  # type: ignore
    MAX_MEMORY_BYTES = int(gb * (1024.0**3))

    def patch_virtual_memory():
        r = psutil._virtual_memory()  # type: ignore
        p = psutil.Process()
        usable_memory = MAX_MEMORY_BYTES - p.memory_info().rss
        return r._replace(
            available=usable_memory,
            free=usable_memory,
            total=MAX_MEMORY_BYTES,
            percent=round(
                (MAX_MEMORY_BYTES - usable_memory) / MAX_MEMORY_BYTES * 100,
                1,
            ),
        )

    psutil.virtual_memory = patch_virtual_memory
    warnings.warn(
        f"Psutil's Max Memory is enforced to be {gb} GB. "
        "This might be higher than the actual available memory, depending on the"
        " machine/SLURM setup.",
        stacklevel=2,
    )

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Very nice! Is there any way that SLURM clusters traditionally deal with this for other packages that have memory checks? Is there any way AutoGluon could automatically pick up that there are limited resources? If not then I would guess this is the best solution.

Re AutoSklearn:
There method there was just to have it be user specified and then use pynisher to set process memory limits. No support for this method on Mac though :(

@eddiebergman
The concept of a wrapper for memory handling is very interesting:
model_trainer = limit(
    train_memory_hungry_model,
    memory=(500, "MB"),
    wall_time=(1.5, "h")  # 1h30m
)

Couple questions:

Does this correctly capture memory usage for models such as LightGBM / CatBoost that are written in C with their own multiprocessing logic?
Does it work for ray worker threads?
Can I use a variant of this logic to store the peak memory usage of a function call?

Very nice! Is there any way that SLURM clusters traditionally deal with this for other packages that have memory checks (Ex: Auto-sklearn)? Is there any way AutoGluon could automatically pick up that there are limited resources? If not then I would guess this is the best solution.

Typically, SLURM workloads are not machine learning tasks but instead other high-performance computing applications. If they are (GPU) machine learning tasks, they tend to use an entire node (i.e., making it non-shared). In these settings, memory checks, tracking, and allocations are usually baked into the system (due to being high-performance jobs).
I am unaware of a general framework that supports this nicely regarding memory. Most people ignore these kinds of problems. It should be possible in Linux, but there is nothing available for Python AFAIK.
If you can use a virtualized run on SLURM (via a resources-limited container), you could fake it so that psutil and similar checks only see as many resources as you would give the container. Unfortunately, resources-limited containers are often not supported by SLURM deployments (due to red tape, not technically limitations).

@eddiebergman
The concept of a wrapper for memory handling is very interesting:
model_trainer = limit(
    train_memory_hungry_model,
    memory=(500, "MB"),
    wall_time=(1.5, "h")  # 1h30m
)

Couple questions:
1. Does this correctly capture memory usage for models such as LightGBM / CatBoost that are written in C with their own multiprocessing logic?

2. Does it work for ray worker threads?

3. Can I use a variant of this logic to store the peak memory usage of a function call? This would be awesome for unit tests / benchmarking / logging (Although I'd guess it comes with a performance hit due to subprocess usage)

Just to give some more details on how it works:
Wrap everything into a subprocess, and the first thing the wrapper will do once launched inside the subprocess is set RLIMIT's (Linux) or communicate to pywin32 to set actual memory limits on the process. Note this isn't a polling system, it's a syscall that literally prevents more allocation than a certain size. For example, np.ones(BIG_BIG_NUMBER) will just not be allowed allocate if it's over the limit. Once this occurs, Python throws an error and the wrapper uses pipes between the main subprocesses and worker subprocess to communicate what happened (could also be a timeout or a more general user/code error).
There's some issues:

If an import causes it to go over the memory limit, you get an ImportError and not a Memory based error, which is hard to track that something when wrong.
The memory limiting on Linux is weird, from what I gathered, the only memory you could limit for a child process is one that includes both its resident memory (that which only the worker has access to) + the memory it shares with the main process from fork or fork-server (the memory not yet copied over to the worker but will be copied as soon as the worker accesses it).
Linux has one way of doing things, Windows has another, and Mac memory-limiting and signals is a dumpster-fire which is better left burning.

Now to the main points:

It should, it uses system's memory for that process, nothing Python specific other than how the syscall is made. (I think it keeps track of children memory too, would need to double check)
The way to approach this would be to send work to the ray worker/thread and have it launch the wrapped function. You could also send over the wrapped function but I'm not sure how pickling works there. In general though, since pynisher will essentially launch a child process just for that function call, it should be fairly okay to just send the wrapped function to the worker and it will handle it there. I'm not a user of Ray but if the ray worker can not spawn subprocesses, you may have issues.
There might be a way where you can set a limit and a signal handler to catch the over memory use, ignore the signal so the process can continue and track the memory somewhere. However I was just using this with polling: https://github.com/automl/pynisher/blob/608ae2f7a102b399d28d7643a7263c6cdff7af10/pynisher/util.py#L218-L240

You might also want to check out this thread: https://github.com/giampaolo/psutil/issues/1096

@eddiebergman Thanks for the detailed reply! I think for now the solution in the PR works for solving the overall problem, although doing direct limiting of process memory would be very interesting as a potential future improvement to AutoGluon's overall resilience to crashes.
Notably, the ability to track overall memory usage (over time and peak) including all child processes will be very useful for stress testing the system, finding memory leaks, and seeing the impact of optimizations. Will hopefully circle back on this later when we find bandwidth.

Happy to chime in where I can :)
I would imagine Ray has some better functionality for this built-in, given it has some C/C++ in there (I think).
If not, all of this information is actually easier to track with conventional tools on Linux for process monitoring given Python is just a C program. Note, tracking, trying to do memory limiting in Python is usually a patchwork of syscalls on the interpreters process then a thing Python can manage itself in Python land.
If I come across a tool for general process monitoring, I'll let you know!


Manual test of run of sandbox provisioning in vSphere

Make the changes needed to be able to provision the sandbox and run through the e2e.sh scripts in an Ubuntu VM on vSphere. This entire flow should be tested:
https://github.com/nephio-project/test-infra/tree/main/e2e/provision
See also https://github.com/nephio-project/nephio/issues/235#issuecomment-1582918293

https://github.com/nephio-project/test-infra/pull/73#issuecomment-1585291441
If there is anything special for vSphere, we should document it as described in that issue.

Manual set up of VM on vSphere and then running the provisioning and e2e scripts.

https://github.com/nephio-project/nephio/issues/317 - blocking issue.

@johnbelamaric just installed nephio in Ubuntu 20.04 VM running in vSphere.
Used this as an entry point.
Executed all steps, including 007 - xNF Deploy. Everything works fine, except for a minor issue in the latest commit in test-infra, where $HOME env variable is mistakingly referenced in init.sh.
How do we want to document this? Or we can just reflect in this page that the procedure is applicable for Ubuntu VM running in vSphere.

Awesome @denysaleksandrov. Yes, I think that page is fine - it will move to the user guide from where it is today.