[No QA] Simplify test workflow using Jest --shard flag

Details
This PR started out as a simplification of the test workflow, but grew with some (valuable) scope-creep. Upgrading Jest was a huge pain, so I'm proud of this PR that gets it done!
The initial changes

Get rid of the unnecessary config job from test.yml, instead define the matrix inline
Separates out the shell tests into their own job (before they were running in all three jobs)

Upgrading Jest

In order to use the --shard feature of jest to split up the tests between multiple runners, we needed to upgrade to at least Jest 28: https://jestjs.io/blog/2022/04/25/jest-28#sharding-of-test-run, so I went for the latest version (Jest 29)
Upgrading Jest presented a number of challenges, each of which I managed to work through (some with lots of frustration) I'll try to explain the more significant learnings below.
Had to add a few more babel plugins to get Jest 29 running (I think they're using some newer JS features)

Fake Timers - why?
One of the biggest changes with Jest 28 was that it changed the "fake timers" module.
I think the idea behind fake timers is to make tests faster and less flaky. Rather than waiting for actual CPU cycles, timers, JavaScript macrotasks, etc... (all things I don't really understand very well), everything is "frozen" by default with fake timers.
Then you get fine-grained control when you want to wait for promises to resolve, when you want to advance the timers and by how much. This speeds things up because you generally never wait for real timeouts, and when the clock needs to be advanced that happens as close to instantly as possible.
So in general using real timers in tests is bad (supposedly), so we enable them globally. The only timer we don't fake globally is nextTick, and I think it has to do with how Onyx intentionally updates subscribers only on the next tick (example). As stated in the code comment there we don't necessarily need to do this anymore and it in my opinion it would be very valuable to clean that up in the future. I think that would also mean we could remove many if instances of waitForPromisesToResolve throughout our tests.
Fixing flaky UnreadActionsTest
The UnreadActionsTest is slow and was pretty flaky (very consistent on my M2 mac, but very inconsistent on the GitHub Actions runners). So I did a few things to make it better:

Moved jest.setTimeout to the correct location. Not sure if something changed in jest or if it just wasn't doing anything before, but according to the jest docs:

Set the default timeout interval (in milliseconds) for all tests and before/after hooks in the test file. This only affects the test file from which this function is called.

So I think that the file root is the correct place (and it resolved timeout errors I was seeing on this PR).

Before we were using jest.advanceTimersByTime(100); to wait for animations to complete. This was flaky even on my computer, and to get it working consistently on the runner I had to increase the timeout to like 500. Maybe something to do with the new timers? Anyways, rather than relying on this sort of hacky/slow/flaky method of waiting for the animation to finish, I decided to use the approach recommended by @testing-library/react-native – using waitFor.

However, waitFor had a gotcha – for now, we need to use the @testing-library/react-native jest preset in order for it to work properly (fixed upstream in React Native 0.71.2, issue created here, TODO left in the code)

In order to use the @testing-library/react-native plugin, I had to upgrade @testing-library/react-native. (More on this below)

Upgrading @testing-library/react-native

v11 of @testing-library/react-native included a breaking change that removed some of the queries we were using, so I had to update those
At this point, since I was updating some queries already, I made the decision to migrate over all our test queries from using the result of a render call to using screen instead. This wasn't strictly necessary, but is recommended by the library maintainers. While it did make the diff of this PR a bit nastier, it's ultimately a simplification because it doesn't require you to maintain and pass around a reference to the render result. I went back and forth on whether this was worth it, but it was just a series of a few simple find-and-replace queries in three files, so I included it.

SessionTest
I'm not sure why this was passing before and not after the Jest upgrade, but it seems like it was a false positive before. Session.signOut alone is not enough to make PushNotification.deregister be called. You need to call signOutAndRedirectToSignIn instead, which in turn clears Onyx, which in turn causes PushNotification.deregister to be called.
Other

I moved jest configs to their own file, mostly because I find the JS more readable than pure JSON and I like being able to jump straight to the jest config file. We could move it to a subdirectory but that causes a few issues if (like me) you run tests using the GUI of your IDE. It also wasn't clear to me if I should put it in config/jest.config.js or jest/config.js, so I just put in the project root because that's the standard place for it.
One small QOL improvement we get from upgrading Jest is that when tests fail you get a nice summary of failing tests, which we didn't have before (instead I always had to search for FAIL tests/ in the GitHub Actions logs). Now we get a concise list of all the tests that failed.

Future Work

The default test environment is jsdom, but GitHub Actions tests run in node, as so should be tested in a node environment rather than jsdom. This can be accomplished with a comment in the first lines of the file, but I think a better solution would be to move those to tests/githubActions and configure jest to use the node runner for tests in that directory, and jsdom for the others. This PR is big enough already so I'm not doing that today.
There are probably a lot of other locations where we could use waitFor instead of advancing timers or using waitForPromisesToResolve, and it might make the tests more readable and intuitive

Fixed Issues
$ https://github.com/Expensify/App/issues/14088
Tests (already done)
Purposely break a Jest test, then push code to this PR. The test workflow should fail. This happened plenty of times during development, as you can see below.

[x] Verify that no errors appear in the JS console

Offline tests
None.
QA Steps
None.

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos
Screenshots are not included, since only test code has been updated and those tests have run in the CI for this PR. For good measure I've included a screenshot of all tests passing locally (both in my terminal and my IDE).

Okay, that test is consistently passing when run in isolation, but I'm able to reproduce the failure (and timeout issue) when running the full npm run test test suite locally. I suspect this means that another test is not cleaning up after itself properly.

@thesahindia @aldo-expensify One of you needs to copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

Removing C+ review here because there's nothing for them to test beyond what we can see in the CI.

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots: N/A

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/roryabraham in version: 1.2.68-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.2.68-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[Feature]: Dynamic input variables

Contact Details
<EMAIL_ADDRESS>Description
As it is now the parameters for To Grid, From Grid, To Battery, From Battery etc is configured as static values in the configuration of the node. If these values could be configured by sending messages to the input of the node, it would be really easy to adjust to different installations and needs and also easy solve some of the other feature requests.
Look at the "Power Saver" node and the "dynamic config":
https://powersaver.no/nodes/ps-strategy-lowest-price.html#description

Suggested Solution
Make the node accept incoming messages to control the node's config parameters dynamically.
Alternatives
No response
Additional Context
No response

It already does that. You can set different msg fields, e.g. inject a msg.tb_max.

That is just fantastic! :)
Sorry I missed that in the documentation.


chore: surface package resolution failures

Description
This will fail the lint check because console.log is used, but I wanted to open to ask what the right way to do this is?
Currently exceptions are being swallowed, which makes it quite hard to tell what is going wrong
Additional context
SvelteKit packages are throwing an exception, but I had no way to know that

What is the purpose of this pull request?

[ ] Bug fix
[ ] New Feature
[ ] Documentation update
[X] Other

Before submitting the PR, please make sure you do the following

[X] Read the Contributing Guidelines.
[X] Read the Pull Request Guidelines and follow the Commit Convention.
[X] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[X] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

I assume you can use our logger

The logger is available in config: ResolvedConfig that is in scope. See
https://github.com/vitejs/vite/blob/fb406ce4c0fe6da3333c9d1c00477b2880d46352/packages/vite/src/node/server/middlewares/proxy.ts#L48

thanks for the pointer! updated


Add beacon2 tutorial

Thanks @khaled196!
This tutorial feels very 'developer-y' and I am a bit afraid it will scare researchers away from Beacons if we put it in the science topic. Is there any way we could have a tutorial just about querying an already existing Beacon here in the variant analysis topic, and then move this full tutorial to admin/dev topic? What do you think?

HI @shiltemann
Both of the ideas are great, and both are doable, but I am concerned about the querying part, I have a beacon for CNV from the 1000hg which does not have the full beacon protocol collection (genomicVariations, runs, biosample, etc). Can the tutorial still be made for just the CNV?
We are working on creating a beacon, but it may take some time.

Hi @khaled196! My feeling is that it would still be a good showcase of how beacons work even if it's only CNV for now. So it would be great to have that, and hopefully we can extend it to more variant types in the future once there is a good 'full' public beacon available we can use.

@shiltemann
I added another tutorial as a prequel. Please have a look at it

HI @shiltemann
I made additional changes. Please take a look and give me your feedback.

Thanks a lot @khaled196!


Consider using inve to activate virtual environments

I've been using it via the pew package, which fills roughly the same niche as virtualenvwrapper, but using the inve approach. My own experience finds it vastly superior to the script-sourcing approach. Explained simply, it starts a subshell instead of changing the current shell. That means de-activating an environment is as simple as the standard exit command in your shell.
https://gist.github.com/datagrok/2199506
https://github.com/berdario/pew

Investigate pew https://github.com/berdario/pew

i think we have all the benefits of this with the current $ pipenv shell command.

i tried using pew, but pew workon does not work on my machine for some reason

going to punt on this for now

That's cool, it's a different model, and it may not be worth the time Right Now.
I'd like to point out, for possible future reference, that the inve model doesn't require using pew, though. inve was originally designed as an activate replacement, that would live in the same place, but open a subshell instead. That concept can be really helpful. pew thought so, and made a virtualenvwrapper alternative built on it, and I think it works excellently.


Lesson counter fixed

Changed the Alignment of lesson counter.

Thanks for creating a branch, I've left you some feedback 😄


frame show bug

like this

Most likely (as far as I can say without seeing the file) it is not a bug. The APNG frame is not a standalone image, but the "patch" to a previously rendered image. So it can look weird being shown in isolation.


Bank customer Segementation

We can make bank customer segmentation using k means dbscan as we can get segregation of customers on the basis of income daily wage spend etc. I will do the needful kindly assign with necessary labels

We can make bank customer segmentation using k means dbscan as we can get segregation of customers on the basis of income daily wage spend etc. I will do the needful kindly assign with necessary labels

Share the details of the proposed issue as per the given issue template.

Evaluate the quality of clusters using internal metrics (e.g., silhouette score) or external metrics if ground truth labels are available.
Compare the results of K-means and DBSCAN to see which algorithm provides better insights for your specific task

Simple machine learning models won't work here.
Thanks for showing your interest. Closing this issue as not required.


Star favourite services

And bump favourite services at the top of the web view.
For now, we could just add a flag in the config file.

Added in the user-panel branch and deployed in the demo server.


Adding healthcheck to docker image

📃 Description
Updated the docker image with a HEALTHCHECK directive to avoid the need for users to specify one themselves in a docker-compose.yml (for example).
Used some typical values for the check, happy to tweak if necessary.
🪵 Changelog
➕ Added
Added HEALTHCHECK to docker/8.3/Dockerfile.

Thanks for the pr, I'm going to close this because the Dockerfile contained here is based off Laravel Sail and I don't want to deviate too far from that base.

Ok, fair enough. 👍


[ADAM-612] Point to proper k-mer counters.

Resolves #612. I also renamed CountKmers to CountReadKmers for clarity.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/625/
Test PASSed.

Thanks, Frank!


Fix folder for the new Band of Blades sheet

@clevett, could you take a look? This entry is currently referring to the wrong underlying sheet.

Greetings,
My apologies. I did add the entry with the wrong folder name. I will ensure this change is correct after the code is live tomorrow.

Thanks Cassie! It seems to be working now.


Fix markdown hyperlink for jax package on frontpage readme

ghcr.io/nvidia/jax automatically redirects to https://github.com/nvidia/JAX-Toolbox/pkgs/container/jax if accessed by a browser. Is this causing issues in some situations?

Strange. For me it gets processed as a relative link in the repo on chrome and gets directed to:
https://github.com/NVIDIA/JAX-Toolbox/blob/main/ghcr.io/nvidia/jax

Oh I see what you are saying. Entering ghcr.io/nvidia/jax directly into the browser redirects, but I think b/c there's no https?:// URI, it gets interpreted as a relative path

Oh I see what you are saying. Entering ghcr.io/nvidia/jax directly into the browser redirects, but I think b/c there's no https?:// URI, it gets interpreted as a relative path

I guess this is where GFM's secret sauce backfires :)


Added customizable timestamp format

Added string property TimeStampFormat. Change this property to customize how date and time are displayed in the LogViewer.

@fraluderin Thanks for the improvement. Besides it's good functionality it gave me also the opportunity to handle a pull request for the first time ever.


Alias bool as u1

@vezenovm can this be closed? Pinging as you recently opened a PR regarding this discrepancy.

Yes this was fixed a while back I believe. Related issue can be found here: https://github.com/noir-lang/noir/issues/277


Bugfix: Initial example fails to resize height

On the initial page load, I almost always see this warning from the iFrameSizer:

which is referencing the example's iframe that initially loaded in the viewport. This example's iframe also fails to resize properly to the content inside of the iframe. Scrolling down to other examples, this warning message does not appear, and these examples do resize properly.
After turning on the log flag in the iFrameResizer, I see that it sends its initial message to the contentWindow but receives no response, and doesn't attempt to send the message again, leading to this warning. I am assuming that this is a race condition between the iFrameResizer host and the contentWindow script, where the host is expecting that the contentWindow script has already loaded when it sends its initial connection message. I haven't looked into that library's code to attempt to resolve that race condition, and it probably is worth letting them know about this condition. But as a hotfix, I did find that wrapping the .iFrameResize call in a setTimeout gave enough time for this race condition to not occur (at least from my ad hoc testing).

Thanks for contributing! This was an annoying bug. I'll follow up with the iFrameResizer maintainer to see if they're aware of this.


Queue functions

Clear queue at reset() in the QueuedAudioPlayer, which happens when stop() is called.
Can clear previous items in the queue with removePreviousItems()

Codecov Report

Merging #32 into master will decrease coverage by 0.17%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master      #32      +/-   ##
==========================================
- Coverage   78.01%   77.84%   -0.18%
==========================================
  Files          17       17
  Lines         828      835       +7
==========================================
+ Hits          646      650       +4
- Misses        182      185       +3

Impacted Files
Coverage Δ

SwiftAudio/Classes/QueuedAudioPlayer.swift
68.51% <0%> (-4.04%)
:arrow_down:

SwiftAudio/Classes/QueueManager.swift
97.7% <100%> (+0.11%)
:arrow_up:

SwiftAudio/Classes/AudioPlayer.swift
80.37% <100%> (ø)
:arrow_up:

...udio/Classes/AVPlayerWrapper/AVPlayerWrapper.swift
86.48% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a0efa5f...5e871fc. Read the comment docs.


[BUG][Spark] maxBytesPerTrigger and maxFilesPerTrigger not working

Bug
Which Delta project/connector is this regarding?

[x] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel

Describe the problem
Adding the option maxFilesPerTrigger or maxBytesPerTrigger doesn't seem to do anything.
I have a streaming reading from and writing to delta. At the beginning, when numBytesOutstanding is very low, the performance is really good. The batch duration and processing rate/second are what I expect. Once the numBytesOutstanding increased (Due to upscaling upstream stream), the performance drops a lot (batch_duration and proc rate). I assume this is because I was not limiting the batch size enough. However, when I do this with maxBytesPerTrigger and maxFilesPerTrigger, I don't see a difference.
I used extreme values with the following:
df = (
    spark.readStream
    .option("maxFilesPerTrigger", 4)
    .option("maxBytesPerTrigger", "1m")
    .format("delta")
    .load(input_folder)
)

I expected this would result in a much lower batch_duration, yet, it still stays stuck. While it is still working on the first batch, in the SQL/DataFrame tab of the Spark UI, I find the following metrics of the Scan parquet step

The number of files read is exactly 1000 (The default value of maxFilesPerTrigger), which makes me assume it doesn't do anything.
In the stage of a batch, the Input Size is also very large (A lot larger than the 1MB I would expect)

This screenshot is with metrics when only finished 32/155 stages.
My transformations are quite complex (Mainly a pandas UDF doing API requests), but I wouldn't expect to impact this.

Steps to reproduce
See above
Observed results
See above
Expected results
Smaller size/less files processed per batch
Further details
Running this on Databricks DBR 14.1
5x workers of 32GB memory, 4 cores. Driver 26GB, 4 cores. No Photon
Environment information

Delta Lake version: 3.0.0
Spark version: 3.5.0
Scala version: 2.12.15
Databricks Runtime System Config

Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

[ ] Yes. I can contribute a fix for this bug independently.
[x] Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
[x] No. I cannot contribute a bug fix at this time.

Willing to contribute, but not sure if enough knowledge.

"I assume this is because I was not limiting the batch size enough. However, when I do this with maxBytesPerTrigger and maxFilesPerTrigger, I don't see a difference."
If there is a growing backlog of data yet to be processed (that is, high numBytesOutstanding), then larger batches with more data in each of them is likely to increase the throughput and processing efficiency. There is some amount of per-batch fixed overheads and so lower number of larger batches reduces the overall overheads.
So while it is worth looking into why maxFilePerTrigger setting is not kicking in ... it may not be the best policy to use it for your use case.

I agree with your logic. However, in my case, the batch duration suddenly increases a lot (from <1min to +20min to currently not being able to process the batch in a reasonable time). The processing rate also decreases in this case (which I find also counterintuitive, but is maybe because my specific transformation logic)

Update from my side: The maxBytesPerTrigger and maxFilesPerTrigger seem to work, but not for the first batch of the stream. Is this expected behaviour or not? If so, is this somewhere in the documentation, I didn't find it anywhere?
The issue is that the maxFilesPerTrigger default of 1000 is way too large. I kept getting OOM errors. I now decreased the spark.sql.files.maxPartitionBytes to +- 40MB, which made it pass the initial batch.

I also encountered this issue, which is quite problematic.
My code is a simple scala job using readStream, writeStream, a forEachBatch and AvailableNow trigger.
My forEachBatch function performs a lot of updates in a delta table, which makes it very sensible to the size of the batches.
A batch of 1000 files (the default value) is too much to handle and fails.
Despite trying many things, I could never get delta to apply the maxBytesPerTrigger or maxFilesPerTrigger options.
I do have a (dirty) workaround though.
I overwrite the default value for maxFilesPerTrigger in DeltaOptions using java/scala reflection.
https://github.com/delta-io/delta/blob/b721e1913f5bdf93e69254333d839fa32673a30a/spark/src/main/scala/org/apache/spark/sql/delta/DeltaOptions.scala#L249
With this overwritten, you need to keep maxBytesPerTrigger empty because of this condition in DeltaSource
https://github.com/delta-io/delta/blob/b721e1913f5bdf93e69254333d839fa32673a30a/spark/src/main/scala/org/apache/spark/sql/delta/sources/DeltaSource.scala#L1260-L1265
This does work for now, but if anyone has a cleaner workaround or a proper patch, that'd be great.
forEachBatch is very convenient because of checkpoints.
I did some debugging but I don't see any obvious fix, it really seems like the option value is not routed to DeltaSource.AdmissionLimits. I'll look into it further if I have the time.


BSOD on Windows 11 21H2

I got BSOD while waking up from sleep. Bugcheck code is WDF_VIOLATION and parameter is 0x4
I'm using the latest release version 1.21.442.0
This is the stack

I have minidump and also a full memory dump but it's too big for me to upload. https://gofile.io/d/Q77Gjl I doubt there's sensitive info in there, so I just uploaded here and I can delete it later just incase

That link doesn't work. Compress the kernel memory dump which will shrink it down to one third its size and upload it here: https://files.nefarius.at/
No need to share the resulting link, only I will see it on the server, tell me when upload is done. It indeed does contain sensitive information so no public sharing pls 😉

Uploaded the minidump. Full dump is still too big and because I almost never shutdown my pc, I have no clue what info might be inside, no offense. There's an unknown function in the stack, I hope you can pinpoint it with the pdbs.

All kernel memory at the point of the crash is in there, which means potential passwords, encryption keys, ...

I've also experienced a BSOD with the identical stack trace. After spending some time analysing my memory dump I've mapped it to the EvtWdfIoPendingNotificationQueueState callback, which receives an invalid this pointer to a PDO object resulting in _UsbInterruptOutBufferQueue member being NULL.  For me it looks like the PDO object has been destroyed after being "Unplugged" but the  _PendingNotificationRequests still exists since its parent is the FDO object.  When Windows wakes up it restarts the _PendingNotificationRequests queue and calls the callback with an pointer to a deleted object. I could be wrong as I am not familiar with driver development, literally knew nothing about it until yesterday.  Is there any reason why the code below uses ParentDevice instead of this->_PdoDevice?
status = WdfIoQueueCreate(
ParentDevice,
&notificationsQueueConfig,
WDF_NO_OBJECT_ATTRIBUTES,
&this->_PendingNotificationRequests
);

Check #139

Have the same issue with latest release.
OS: Windows 11 21H2

I believe I also experienced this. It happened after my DS4 went to sleep due to inactivity. Using Moonlight to stream from Sunshine to an Apple TV 4K. Windows 11 22H2 here. After powering the controller back on, it was not detected again. Tried restarting moonlight and sunshine but no luck then eventually the BSOD and after reboot it's working again

I'm experiencing the same issue (Windows 11 is BSODing/rebooting after waking from sleep) with the same error code of WDF_VIOLATION (10d).
This happens consistently anytime I use sunshine from a moonlight client using PS4 controller via ViGEmBus on the sunshine host system.  The symptoms are always whenever the system enters a sleep state and is woken-up, the same WDF_VIOLATION BSOD occurs and subsequent reboot.
Analyzing the minidumps all show items suggesting that ViGEmBus is involved,
STACK_TEXT:
ffffbe8c`d3614238 fffff807`0ee05b50     : 00000000`0000010d 00000000`00000004 00000000`00000000 fffff807`1e82b226 : nt!KeBugCheckEx
ffffbe8c`d3614240 fffff807`0ee05bdc     : fffff807`1e82b226 ffffe20f`ad215b00 fffff807`1e842288 fffff807`1e836979 : Wdf01000!FxVerifierBugCheckWorker+0x24 [minkernel\wdf\framework\shared\object\fxverifierbugcheck.cpp @ 87]
ffffbe8c`d3614280 fffff807`0edd65e7     : 00000000`00000000 fffff807`1e82d989 00000000`00000000 ffffe20f`ae15dd50 : Wdf01000!FxVerifierNullBugCheck+0x44 [minkernel\wdf\framework\shared\object\fxverifierbugcheck.cpp @ 122]
ffffbe8c`d36142c0 fffff807`1e82b226     : 00001df0`54d9c548 ffffe20f`910ccba0 fffff807`1e842288 fffff807`1e83e718 : Wdf01000!imp_WdfObjectGetTypedContextWorker+0xdd67 [minkernel\wdf\framework\shared\object\handleapi.cpp @ 716]
ffffbe8c`d3614310 00001df0`54d9c548     : ffffe20f`910ccba0 fffff807`1e842288 fffff807`1e83e718 00000000`00000000 : ViGEmBus+0xb226
ffffbe8c`d3614318 ffffe20f`910ccba0     : fffff807`1e842288 fffff807`1e83e718 00000000`00000000 ffff9500`995d1180 : 0x00001df0`54d9c548
ffffbe8c`d3614320 fffff807`1e842288     : fffff807`1e83e718 00000000`00000000 ffff9500`995d1180 00001df0`5b66c408 : 0xffffe20f`910ccba0
ffffbe8c`d3614328 fffff807`1e83e718     : 00000000`00000000 ffff9500`995d1180 00001df0`5b66c408 fffff807`1e8260ac : ViGEmBus+0x22288
ffffbe8c`d3614330 00000000`00000000     : ffff9500`995d1180 00001df0`5b66c408 fffff807`1e8260ac ffffe20f`ad215b00 : ViGEmBus+0x1e718

SYMBOL_NAME:  ViGEmBus+b226
MODULE_NAME: ViGEmBus
IMAGE_NAME:  ViGEmBus.sys
STACK_COMMAND:  .cxr; .ecxr ; kb
BUCKET_ID_FUNC_OFFSET:  b226
FAILURE_BUCKET_ID:  0x10D_4_ViGEmBus!unknown_function
OSPLATFORM_TYPE:  x64
OSNAME:  Windows 10

Hi all,
I raised a discussion earlier and it seems like my issue is somewhat related to this. Mine crashes when resuming from hibernate (rebooting the whole system) losing my hibernated apps.
https://github.com/ViGEm/ViGEmBus/discussions/171

For those experiencing this issue with Sunshine + Moonlight....
It seems I have found a workaround.
The idea of the fix was to restart sunshine services in the background just before the system goes into sleep/hibernate.
Below is a file that you can import into Task Scheduler.
restart_sunshine.zip
Hope this helps anyone who is experiencing the same issue.

Hope this helps anyone who is experiencing the same issue.

Can confirm that this is workaround has the intended affect.  Thank you @chestermw!


Unable to exclude TF modules

Hello,
I'm migrating from the tfsec Github Action to trivy but have been running into some issues. I'm using private terraform modules, and the trivy check is failing on modules which are not even used in that repo.
I have tried adding TRIVY_TF_EXCLUDE_DOWNLOADED_MODULES or TRIVY_TF_EXCLUDE_DOWNLOADED_MODULES, but no luck 😞
As per the GH Actions documentation, I should be able to add extra flags as environment variables.
I can't reproduce the same error locally, only happens in the CICD pipeline.
Current config:
    - name: Run Trivy vulnerability scanner in IaC mode
      uses: aquasecurity/trivy-action@master
      with:
        scan-type: 'config'
        scan-ref: ${{ inputs.terraform-directory }}
        hide-progress: false
        exit-code: '1'
        ignore-unfixed: true
        output: trivy.txt
      env:
        TRIVY_TF_EXCLUDE_DOWNLOADED_MODULES: true

If possible I'd like to avoid having to add extra files to the repo as I have 30+ repos to update. Configuring this via input variable/flag would be ideal

@afdesk could you take a look at it?

Looks like it was solved offline.

Hey folks I'm facing the same issue how did you solve it @AnitaErnszt ?

i also would like to know how this was solved @AnitaErnszt @simar7


The react/jsx-space-before-closing rule is deprecated. Please use the react/jsx-tag-spacing rule with the "beforeSelfClosing" option instead

Reported by @donovanhiland in feross/eslint-config-standard#83

I have the latest version of eslint-config-standard-jsx, but I still see this warning.
I have this line in node_modules/eslint-config-standard-jsx/eslintrc.json:
"react/jsx-space-before-closing": "error"

but not this:
"react/jsx-tag-spacing": ["error", { "beforeSelfClosing": "always" }]

Any fix yet? I haven't specified anything in the rules, and still, I see this warning.

Why was this closed?  Is any workaround available?

This package is used in eslint-config-standard-react which doesn't have the updated dependency version that includes this fix.
Until that happens you can add:
"rules": {
  "react/jsx-tag-spacing": ["error", { "beforeSelfClosing": "always" }]
}

to your .eslintrc which will suppress the error

Hi,
I am getting this error still.. even after adding the rule.. any other pointer..
The react/jsx-space-before-closing rule is deprecated. Please use the react/jsx-tag-spacing rule with the "beforeSelfClosing" option instead.
Cannot read property 'range' of null
Below is my .eslintrc
{
"extends": ["standard", "standard-react"],
"globals": {
"DEVELOPMENT": true,
"CLIENT": true,
"DEVTOOLS": true,
"DISABLE_SSR": true,
"fetch": false,
"fail": false,
"test": false,
"describe": false,
"expect": false,
"jest": false,
"afterAll": false,
"beforeAll": false,
"afterEach": false,
"beforeEach": false
},
"rules": {
"indent" : ["error", 2, { "ignoredNodes": ["JSXElement *"] }],
"object-curly-spacing": ["error", "always", { "objectsInObjects": false }],
"standard/object-curly-even-spacing": "off",
"jsx-quotes": ["error", "prefer-double"],
"react/jsx-tag-spacing": ["error", { "beforeSelfClosing": "always" }],
"react/jsx-curly-spacing": [2, "never", {"allowMultiline": true}],
"react/jsx-indent-props": ["error", 4],
"no-unused-expressions": "off"
},
"settings": {
"react": {
"version": "latest"
}
},
"parser": "babel-eslint"
}


Failing test test_app_rollback_validation

Drone publish failed https://drone-publish.rancher.io/rancher/rancher/1117/1/2

Nothing for QA to test and current builds have not seen the issue.

http://drone-pr.rancher.io/api/repos/rancher/rancher/builds/6854/logs/1/2,
http://drone-pr.rancher.io/api/repos/rancher/rancher/builds/6772/logs/1/2

https://drone-pr.rancher.io/rancher/rancher/6860/2/2


Extra space appears before mention in LHN message

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

from userA send message ( in message type anything and after that mention userB)
check the message in userB

Expected Result:
Should not show extra space
Actual Result:
Extra space appears before mention
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

Version Number: v1.3.13-1
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/43996225/781f750d-07dc-4ff6-b245-c09aed7e3b34
Expensify/Expensify Issue URL:
Issue reported by: @gadhiyamanan
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1683883268543329
View all open jobs on GitHub

Already reported here https://github.com/Expensify/App/issues/18539


chore(ci): fix release-please manifest

It appears that the $schema field was throwing it off. Further, some versions were out-of-date.

I'ma publish a package call $schema


Operations on DataTree objects should not create duplicate coordinates on sub-trees

What is your issue?
This is very obviously an issue using the repr from https://github.com/pydata/xarray/pull/9470:
In [20]: tree = DataTree(Dataset(coords={'x': [1, 2, 3]}))

In [21]: tree['/foo'] = DataTree(Dataset({'bar': ('x', [4, 5, 6])}))

In [22]: tree
Out[22]:
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 3)
│   Coordinates:
│     * x        (x) int64 24B 1 2 3
└── Group: /foo
        Dimensions:  (x: 3)
        Data variables:
            bar      (x) int64 24B 4 5 6

In [23]: tree * 2
Out[23]:
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 3)
│   Coordinates:
│     * x        (x) int64 24B 1 2 3
└── Group: /foo
        Dimensions:  (x: 3)
        Coordinates:
          * x        (x) int64 24B 1 2 3
        Data variables:
            bar      (x) int64 24B 8 10 12

Instead, the result for tree * 2 should look like:
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 3)
│   Coordinates:
│     * x        (x) int64 24B 1 2 3
└── Group: /foo
        Dimensions:  (x: 3)
        Data variables:
            bar      (x) int64 24B 8 10 12

We probably also need to also revisit other Dataset methods that are ported to DataTree via map_over_subtree (xref https://github.com/pydata/xarray/issues/9472). Some of these (e.g., arithmetic, aggregations) can likely easily be corrected simply by mapping over nodes with inherited=False. Others (e.g., indexing) will need more careful consideration.

As a starting point, it might make sense to disable most automatically ported DataTree methods, at least those that can't be implemented via map_over_subtree with inherited=False.

I'm kind of surprised that the existing map_over_subtree tests didn't catch this.

I'm kind of surprised that the existing map_over_subtree tests didn't catch this.

This could be because assert_identical (prior to https://github.com/pydata/xarray/pull/9473) doesn't currently check this.

In the meeting just now we decided that this is a deficiency in the data model, not just in the implementation of map_over_subtree. We should change the model so that identical coordinates in children are automatically removed, so that the above would be automatically de-duplicated at construction time.
This is important to do as otherwise almost every user will run into this problem, especially using this pattern:
dt['lat'] = ...  # coord present on root

ds = dt[path].dataset
result_ds = some_operation(ds)
dt[path].dataset = result_ds  # coord now present on both root and descendant!

Currently this would duplicate the 'lat' coordinate between dt and dt[path], which would lead to very confusing behaviour, e.g.
del dt[path]['lat']
'lat' in dt[path]  # would return true!

(because only the duplicated coordinate on the child would have been deleted, not the original on the root, which would still be inherited).
IO
Disallowing duplicated coordinates in general would subtly affect round-tripping behaviour. It would no longer be possible to write out a netCDF/Zarr file with duplicated coordinates, because DataTree could not represent them in the first place.
Round-tripping is then broken for any file with duplicated coordinates, as reading the file would de-duplicate the coordinates, and the information wouldn't be propagated in order to re-duplicate them before writing.
Mostly this wouldn't matter often - there is not much reason to define duplicated coordinates in netCDF/Zarr in the first place.
It might lead to subtle trickiness for files containing CF metadata that refers to other variables in different groups using absolute paths, but that is not something that xarray promises to handle anyway, as interpreting metadata contents is not part of xarray's data model (beyond what happens during decoding).
However again open_groups could still open such files without removing duplicates, and one could imagine writing out group-by-group to a single output file, and that should roundtrip perfectly.
All of this is a consequence of the fact that whilst very similar, the data models of netCDF, Zarr, and DataTree are not identical.
cc @owenlittlejohns @flamingbear @eni-awowale

Not sure exactly what the internal implementation of de-deduplication would be (perhaps some kind of NonDuplicatedChainMap?), but @shoyer said he is on it 🙂 A small change to map_over_subtree might also still be required to close this issue.

It should be straightforward enough to handle duplicated index coordinates. Non-identical coordinates with an index are already disallowed between parent and child nodes, because the parent and child nodes would fail the alignment check in _pre_attach.
However, there is one tricky case: duplicated coordinates without an associated index. Here's an example of how these currently manifest themselves:
In [33]: tree = DataTree(Dataset(coords={'x': 0}), children={'child': DataTree(Dataset({'y': 1}))})

In [34]: tree
Out[34]:
<xarray.DataTree>
Group: /
│   Dimensions:  ()
│   Coordinates:
│       x        int64 8B 0
└── Group: /child
        Dimensions:  ()
        Data variables:
            y        int64 8B 1

In [35]: tree.map_over_subtree(lambda ds: ds)
Out[35]:
<xarray.DataTree>
Group: /
│   Dimensions:  ()
│   Coordinates:
│       x        int64 8B 0
└── Group: /child
        Dimensions:  ()
        Coordinates:
            x        int64 8B 0
        Data variables:
            y        int64 8B 1

Notice the duplicated coordinates /x and /child/x.
For consistency with the rest of Xarray, we should try to handle this like duplicate coordinates that result from other Xarray operations like arithmetic. As noted in https://github.com/pydata/xarray/issues/9481, the current strategy requires evaluating coordinate values to see if they are equal or not in order to decide where they appear. This is attractive for DataTree as well, but unfortunately is not compatible with lazy evaluation.
Instead, I propose that we switch to the equivalent of compat='override, similar to the proposal for Dataset/DataArray coordinates proposed in https://github.com/pydata/xarray/issues/9481. On DataTree, if an operation results in a node with a (non-index) coordinate with the same name as a parent coordinate, the child coordinate should be silently dropped.
An exception to this rule are operations that explicitly assign a child coordinate, e.g., DataTree.coords.__setitem__. Here we should raise an error (noting the existence of the parent coordinate) rather than being a silent no-op.

Thanks for this great write up! I think in the long run this will probably help to avoid confusion and is probably the best way forward. But from more of a DAAC archiving, service provider and data validation perspective (lol I wear a lot of hats) this makes me a little nervous. At GES DISC we use DataTree to do data validation checks against server hosted services and cloud hosted services. The part that makes me nervous is that using DataTree to open netCDF4 files would be modifying the orginal file by removing the duplicate coordinate variables at each node. I don't think this will actually break anything in the backend for us but there is some complexity especialy with metadata that was mentioned.
Would folks be opposed to some kind of flag that's like inherit=True or de_duplicate=True? I think we could set that to being the default behavior and give the user an option to turn it off.

Would folks be opposed to some kind of flag that's like inherit=True or de_duplicate=True? I think we could set that to being the default behavior and give the user an option to turn it off.

The proposed change here would make such a flag impossible - your intended result of de_duplicate=False would now be forbidden by the DataTree data model.
What we could perhaps do though is add a flag to constructors / openers that would raise on encountering duplicated coordinates instead of silently de-duplicating them. e.g.
dt = open_datatree(unvalidated.nc, duplicate_coords='raise')

where the error message tells you which coordinates are duplicated, and refers you to open_groups. The default could still be 'ignore' or maybe 'warn'. There's precedent for treating potential errors this way, see for example
https://github.com/pydata/xarray/blob/aeaa0820e3092626472c9749f6601563bf20fb3e/xarray/namedarray/core.py#L1001

What we could perhaps do though is add a flag to constructors / openers that would raise on encountering duplicated coordinates instead of silently de-duplicating them. e.g.

Yes, this is a great idea!
I would actually suggest such a stricter mode for open_datatree by default. Users who write data from Xarray will not encounter the issue, because Xarray will not be able to represent such datasets. So it's only something that will effect loading data from an external data provider, and it seems better to force users to make an intentional choice in such cases.
The cases where we should be more lenient by default (automatically dropping conflicts) are situations where users are constructing a new DataTree from a collection of Dataset objects, which were likely created using inherited=True.

If we put the flag in DataTree.from_dict() then it can just be passed from open_datatree immediately, and we can also explicitly write DataTree.from_dict(..., duplicate_coords='ignore') inside map_over_subtree to make intent clear internally 🙂

So summarising another meeting's worth of discussion on this... (including special guest @castelao)
The de-duplication idea has some issues.

map_over_subtree vs DataTree.__setitem__

As implied above, any de-duplication should ideally occur in the data model itself, not as a special feature of map_over_subtree. That means that the de-duplication also has to be applied for DataTree.__setitem__.

De-duplication vs Overriding

Comparing two coordinate variables to decide if they are de-duplicated could just be done by comparing names, as in https://github.com/pydata/xarray/pull/9510#discussion_r1763445717. But removing anything of the same name (and doing so in __setitem__) basically completely forbids "overriding" inherited coordinates with a new coordinate of the same name but different values. This would be a valid data model for DataTree, but an even more restrictive one than what we currently have (the ChainMap implementation we currently have allows for overriding any coordinate).

De-duplicating lazy variables by id

If you don't compare by names you have to compare something else. You could compare the id of the underlying Variable, i.e. id(Variable), but this will not recognize copies as being duplicates, even if their values are the same. e.g.
def mean(ds: Dataset) -> Dataset:
    return ds.mean()  # this creates a shallow copy of all the variables!

dt.map_over_subtree(mean)  # this would therefore still end up with duplicated coordinates

Only inherit index variables?

You can instead identify duplicated coordinates by comparing values directly, but this implies loading the variable into memory. Mostly our coordinate variables will be backed by in-memory indexes, so we can compare those and all would be fine. So a data model of "only inherit index-backed coordinates" works quite nicely, in that you can then always cheaply do comparison of inherited coordinates to check for de-duplication, and you can distinguish overridden from duplicated coordinates.

Accessing non-indexed inherited variables?

The fly in that ointment is that it's possible to have coordinate variables that are not backed by indexes, and as these can still be multi-dimensional they can still lazily point to large amounts of data.
If the data model is now "only inherit coordinate variables backed by indexes", then it limits the usefulness of inheritance.
If we try to do inheritance of non-indexed coordinate variables, we can't use the solution from step (4).

My only concern is what @eni-awowale already mentioned, removing coordinates from child-groups will break the original file data model. Overriding coordinates in child groups is also a CF conventions feature, it should at least be handled correctly via some switch/flag.

I realised that the .sel example in #9501 is interesting because you can navigate around the unintuitive behaviour by indexing in such a way that preserves the IndexVariable:

Click to see example
In [6]: ds = xr.tutorial.open_dataset("air_temperature").drop_attrs()

In [7]: ds_daily = ds.resample(time="D").mean("time")

In [8]: ds_weekly = ds.resample(time="W").mean("time")

In [9]: ds_monthly = ds.resample(time="ME").mean("time")

In [12]: dt = xr.DataTree.from_dict(
  ...:     {
  ...:         "/": ds.drop_dims("time"),
  ...:         "daily": ds_daily.drop_vars(["lat", "lon"]),
  ...:         "weekly": ds_weekly.drop_vars(["lat", "lon"]),
  ...:         "monthly": ds_monthly.drop_vars(["lat", "lon"]),
  ...:     }
  ...: )

In [13]: dt
Out[13]:
<xarray.DataTree>
Group: /
│   Dimensions:  (lat: 25, lon: 53)
│   Coordinates:
│     * lat      (lat) float32 100B 75.0 72.5 70.0 67.5 65.0 ... 22.5 20.0 17.5 15.0
│     * lon      (lon) float32 212B 200.0 202.5 205.0 207.5 ... 325.0 327.5 330.0
├── Group: /daily
│       Dimensions:  (time: 730, lat: 25, lon: 53)
│       Coordinates:
│         * time     (time) datetime64[ns] 6kB 2013-01-01 2013-01-02 ... 2014-12-31
│       Data variables:
│           air      (time, lat, lon) float64 8MB 241.9 242.3 242.7 ... 295.9 295.5
├── Group: /weekly
│       Dimensions:  (time: 105, lat: 25, lon: 53)
│       Coordinates:
│         * time     (time) datetime64[ns] 840B 2013-01-06 2013-01-13 ... 2015-01-04
│       Data variables:
│           air      (time, lat, lon) float64 1MB 245.3 245.2 245.0 ... 296.6 296.2
└── Group: /monthly
      Dimensions:  (time: 24, lat: 25, lon: 53)
      Coordinates:
        * time     (time) datetime64[ns] 192B 2013-01-31 2013-02-28 ... 2014-12-31
      Data variables:
          air      (time, lat, lon) float64 254kB 244.5 244.7 244.7 ... 297.7 297.7

In [14]: dt.sel(lat=75, lon=300)
Out[14]:
<xarray.DataTree>
Group: /
│   Dimensions:  ()
│   Coordinates:
│       lat      float32 4B 75.0
│       lon      float32 4B 300.0
├── Group: /daily
│       Dimensions:  (time: 730)
│       Coordinates:
│           lat      float32 4B 75.0
│           lon      float32 4B 300.0
│         * time     (time) datetime64[ns] 6kB 2013-01-01 2013-01-02 ... 2014-12-31
│       Data variables:
│           air      (time) float64 6kB 242.7 245.6 244.9 249.8 ... 254.8 255.6 256.8
├── Group: /weekly
│       Dimensions:  (time: 105)
│       Coordinates:
│           lat      float32 4B 75.0
│           lon      float32 4B 300.0
│         * time     (time) datetime64[ns] 840B 2013-01-06 2013-01-13 ... 2015-01-04
│       Data variables:
│           air      (time) float64 840B 247.2 251.7 256.2 261.4 ... 249.8 248.2 255.7
└── Group: /monthly
      Dimensions:  (time: 24)
      Coordinates:
          lat      float32 4B 75.0
          lon      float32 4B 300.0
        * time     (time) datetime64[ns] 192B 2013-01-31 2013-02-28 ... 2014-12-31
      Data variables:
          air      (time) float64 192B 254.0 252.8 256.9 258.7 ... 265.1 261.8 251.7

In [15]: dt.sel(lat=[75], lon=[300])
Out[15]:
<xarray.DataTree>
Group: /
│   Dimensions:  (lat: 1, lon: 1)
│   Coordinates:
│     * lat      (lat) float32 4B 75.0
│     * lon      (lon) float32 4B 300.0
├── Group: /daily
│       Dimensions:  (time: 730, lat: 1, lon: 1)
│       Coordinates:
│         * time     (time) datetime64[ns] 6kB 2013-01-01 2013-01-02 ... 2014-12-31
│       Data variables:
│           air      (time, lat, lon) float64 6kB 242.7 245.6 244.9 ... 255.6 256.8
├── Group: /weekly
│       Dimensions:  (time: 105, lat: 1, lon: 1)
│       Coordinates:
│         * time     (time) datetime64[ns] 840B 2013-01-06 2013-01-13 ... 2015-01-04
│       Data variables:
│           air      (time, lat, lon) float64 840B 247.2 251.7 256.2 ... 248.2 255.7
└── Group: /monthly
      Dimensions:  (time: 24, lat: 1, lon: 1)
      Coordinates:
        * time     (time) datetime64[ns] 192B 2013-01-31 2013-02-28 ... 2014-12-31
      Data variables:
          air      (time, lat, lon) float64 192B 254.0 252.8 256.9 ... 261.8 251.7

The first call dt.sel(lat=75, lon=300) behaves counter-intuitively in that it duplicates the lat and lon coords. However dt.sel(lat=[75], lon=[300]) keeps the coordinate inheritance in the way you would expect. This only works because dt.sel(lat=[75], lon=[300]) does not actually get rid of the Index underlying the lat/lon coordinate variables, so the result is still subject to the inheritance rules.
In some ways this is weird, but it also kind of makes sense: dimensions are inherited, and the dt.sel(lat=75, lon=300) call removes the dimensions (by making lat/lon into scalars). dt.sel(lat=[75], lon=[300]) keeps the dimensions by indexing to create 1D arrays instead of scalars.
I wondered what people thought about encouraging these kind of index-preserving operations as an escape hatch, especially @castelao.

For sel() and isel() in particular, I think we should have special case logic to keep the indexed coordinates at the same level where they were defined originally.

I'm not sure I totally follow. You mean so that even if the result of indexing is a non-indexed coordinate (e.g. a scalar), special-case logic in isel/sel de-deuplicates that result coordinate automatically.

I'm not sure I totally follow. You mean so that even if the result of indexing is a non-indexed coordinate (e.g. a scalar), special-case logic in isel/sel de-deuplicates that result coordinate automatically.

Yes, exactly. Coordinates defined at a higher level could be automatically be excluded from the indexing result.
This is similar to how Dataset.sel() is not exactly equivalent to mapping over DataArray.sel()  -- dimensions that do not apply to a particular array as dropped from the indexers. (Incidentally, we should add this feature for DataTree, too!)

(Incidentally, we should add this feature for DataTree, too!)

I think that's already tracked in https://github.com/pydata/xarray/issues/8949. This would be a nice thing to get in before release, because it's definitely going to annoy people who try to use datatrees with inheritance for analysis.

This should be closed by #9555


Release v0.1

I'd like to start working towards the initial 0.1 release.
This issue should serve as a discussion for figuring out what should happen before the 0.1 release.
@0x00002a any inputs from your side?
Features & Fixes

Required

#17
#19

Optional

#24
#25

Great! I'm hoping to pr feat-custom-requests at some point today (hopefully soon), just gotta fix some build issues and finish the writeup. Thats gonna need some review and possibly lots of tweaking and I suggest we sort that out before since its a pretty major change (at least for websockets). If we wanna go ahead with 0.1 without it though that might be a good idea too since it is such a major change and probably has some new bugs.
I'm also planning on setting up a conan package for this, at least hosted on a free artifcatory instance (I could try getting it into the conan central repo but I've never done that before).
For the mingw, I think we should get the CI to build with mingw, that way we would at least have warning that it got broke before you tried to compile. But thats gonna be a real PITA I think, might have to do some copy-pasting of the workflow since I don't think I can do it with the matrix.

I agree. Let's wait for your PR then on #17 then.

@0x00002a anything else you deem important enough to put into v0.1?

Nothing off the top of my head, so no :p.

Well, in that case we can close this issue :p


tour: [REPLACE WITH SHORT DESCRIPTION]

Context: https://tour.golang.org/welcome/1
Change the title above to describe your issue and add your feedback here, including code if necessary

@yan-xue-wen can you please update title and description? Thanks


Observation test -Invalid Observation: Observation: Unrecognised property '@encounter'

Subject of the issue
I was wondering why the Observation test showing this error now, Observation: Unrcognised property '@encounter'
I tested early this morning and was not getting this error on all the Observation tests and then wondering if there was an update and now I get this error on all the Observation tests.  The example is from the Observation Smoking Status test.
Your environment

Edition of inferno (Community or Program): inferno.healthit.gov/community
Version of inferno: 2.11.0
Which browser and version(s) is the bug present on?: Chrome Version 88.0.4324.104

Steps to reproduce
Include:

The selected test suite : Smoking Status Observation tests
The URL of the FHIR server being tested: https://rqhandler-dev.cpsi.com/primary/requesthandler/fhir/R4
The Test ID, if it's a problem with a specific test.
Any required configuration options (client ID, client secret, etc.)
Anything else needed to make the issue occur.

Expected behavior
Should pass USCSSO-01 and USCSSO-02
Actual behavior
USCSSO-01 fails

This was test from this morning that showed it passed:

Saved file with encounter error:
https://inferno.healthit.gov/community/jEO2cY4chVF/

It looks like there was another update.  It looks like it is working again.  Thanks!   This ticket may be closed.

It looks like there was another update.  It looks like it is working again.  Thanks!   This ticket may be closed.


Jschema doesn't emit default values.

JSchema doesn't emit default values. in the SARIF SDK we have to maintain these manually, currently
        /// </summary>
        [DataMember(Name = "columnKind", IsRequired = false, EmitDefaultValue = false)]
        [JsonConverter(typeof(EnumConverter))]
        [System.ComponentModel.DefaultValue(ColumnKind.UnicodeCodePoints)]
        public ColumnKind ColumnKind { get; set; }

Also, make sure that default values are set on deserialization by setting
[JsonProperty DefaultValueHandling = DefaultValueHandling.Include)]

on every property.
@michaelcfanning FYI

Ah! JsonProperty can accommodate this. I think you need the Populate member for the deserialization scenario?

As you have pointed out separately, code gen hints can be used to apply these. More importantly, however, JSchema could populate default values in OM constructor. The attribute hints only work when serializing to and from JSON. Direct creation via the OM is not initializing the default values, requiring users to explicitly know about and set them. This issue is obviously more urgent than the application of attributes (agian, because there is an existing workaround).
    public Location()
    {
        LogicalLocationIndex = -1;


Support config file and keyring

This pull requests merges all my local improvements into this module. It is stacked on the pull request #62 which should be reviewed and merged before this one. Following changes are included:

Add profitbricks.utils containing helper functions: ask() and find_item_by_name
Add ProfitBricksService.get_datacenter_by_name()
Implement ProfitBricksService.wait_for_completion()
Support providing a client user agent
Add config file and keyring support
Add pb-api-shell script to examples

Please review these changes carefully and tell me if you are happy to merge them or if you like to have it implemented differently.

Since pull request #62 was merge, I rebased this set of patches. Please review them.

@bdrung, I can't run the tests. Missing configparser module.
python -m unittest discover tests test_datacenter.py
E
======================================================================
ERROR: test_datacenter (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_datacenter
Traceback (most recent call last):
  File "c:\python27\Lib\unittest\loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "c:\python27\Lib\unittest\loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "C:\workspace\virtualenvs\pbpythonsdk\repo\profitbricks-sdk-python\tests\test_datacenter.py", line 19, in <module>
    from profitbricks.client import Server, Volume
  File "profitbricks\client.py", line 16, in <module>
    import configparser
ImportError: No module named configparser

The code worked on Python 3. In Python 2 the module is called ConfigParser instead of configparser. I changed the patch to work with Python 2 and 3:
try:
    import configparser
except ImportError:
    import ConfigParser as configparser

@bdrung I would like to revisit this. I'm going to leave some remarks in the code.


html-pdf TypeError: callback is not a function

I was following a tutorial on how to use your npm package :
the tutorial url :
https://medium.com/@saumya.ranjan/how-to-generate-pdf-in-nodejs-ejs-and-html-pdf-81a9a154d22d#:~:text=report-template.ejs,-step-6&text=Go to new tab and,response as File created successfully
and when i'm sending my request to lauch the process, this the error i get which comes right from you node module could you tell me if anyone had the same issues ? my code is the same as the one in the tutorial.
-- Edit :
Found out that the problem occurs for this call :
the toFIle() crashes and causes the error :
pdf.create(data, options).toFile(__root, "generatePDF/report.pdf", function (err, data) {    console.log("in toFile :", data);         if (err) {               console.log("error : ", err);                res.send(err);            } else {                res.send("File created successfully");            }  });
`
/Users/axel/Documents/CMF/cmf/cmf_server/node_modules/html-pdf/lib/pdf.js:128
console.log('callback : ',callback(null, data));
^
TypeError: callback is not a function
at respond (/Users/axel/Documents/CMF/cmf/cmf_server/node_modules/html-pdf/lib/pdf.js:128:31)
at Socket.onData (/Users/axel/Documents/CMF/cmf/cmf_server/node_modules/html-pdf/lib/pdf.js:108:17)
at Socket.emit (events.js:203:13)
at addChunk (_stream_readable.js:295:12)
at readableAddChunk (_stream_readable.js:276:11)
at Socket.Readable.push (_stream_readable.js:210:10)
at Pipe.onStreamRead (internal/stream_base_commons.js:166:17)
`

https://github.com/marcbachmann/node-html-pdf#api
The signature of toFile is: toFile(filepath, function callback(err, res) {})
so your __root is extra there and "generatePDF/report.pdf" is in the place of callback which causes the problem you are seeing.

You use three arguments instead of two toFile(filename?: string, callback?: (err: Error, res: FileInfo) => void): void.


on_client_gone will be triggered multiple times after reload lua script

Environment

VerneMQ Version: 1.0.0

If I reload the lua script, and the on_client_gone will be triggered multiple times as same as reload times

unload and load is ok, but reload will trigger this issue

Hi, I cannot reproduce this. I wrote this small script:
function auth_on_register(reg)
   print("auth_on_register")
   return true
end

function on_client_gone(reg)
   print("on_client_gone")
end

function on_client_offline(reg)
   print("on_client_offline")
end

hooks = {
   auth_on_register = auth_on_register,
   on_client_gone = on_client_gone,
   on_client_offline = on_client_offline
}

and then connected with mosquitto_sub -d -t 'topic' and reloaded the script with _build/default/rel/vernemq/bin/vmq-admin script reload path=pwd/test.lua but the on_client_gone hook isn't called and neither is the on_client_offline. When I kill the client the I get the on_client_offline hook.
Could you provide some more information about when you experience this? Since you're getting the on_client_gone hook, you must have set allow_multiple_session = on? Also, could you post your script so we could have a look?

Aah, now I see what you mean - each reload causes the hook to be registered an extra time and then when the client disconnects the hook is executed too many times. I've now done a reload 3 times and the on_client_offline hook get's called 3 times when a client disconnects. Thanks for reporting - we'll see if we can get this fixed soon.

Thanks for quickly fix.


conformance tests for AWS pubsub SQS SNS

Signed-off-by: Amit Mor<EMAIL_ADDRESS>Description
Conformance tests.
Note that ordering of messages is disabled in test setup yaml, as localstack SQS FIFO (that is needed for message ordering) support is broken (not documented but tested).
My tries to use a custom image of localstack with Elasticmq (SQS API but with fifo support) failed as the localstack SNS seems to be dropping a mandatory param (MessageGroupId) as it publishes to SQS. I've verified that when the component is configured with fifo set to true that param is indeed sent off.
localstack in that situation doesn't "complain", but as stated, it doesn't support FIFO.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #1393
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[X] Code compiles correctly
[X] Created/updated tests
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@amimimor can you create a docs PR for the same ?

@amimimor can you create a docs PR for the same ?

Detailing the component to now be "beta"?

@amimimor can you create a docs PR for the same ?

Detailing the component to now be "beta"?

Yes, the component can advance to beta.

@amimimor can you create a docs PR for the same ?

Detailing the component to now be "beta"?

Yes, the component can advance to beta.

Just did. It waits a review. Also another doc pr is in waiting to meet v1.6


feat: remove orphaned blocks from cfg to improve simplify_cfg pass.

Description
Problem*
Resolves
Summary*
Inspired by https://github.com/noir-lang/noir/pull/6184, I noticed that after simplifying a constant jmpif we wouldn't inline the block following the inlined block into the first block, as an example consider the SSA
After Dead Instruction Elimination:
brillig fn main f0 {
  b0(v0: Field, v1: Field):
    constrain v0 == Field 1
    constrain v1 == Field 0
    jmpif u1 0 then: b2, else: b1
  b2():
    v8 = add v0, Field 1
    jmp b3(v0, v8)
  b3(v2: Field, v3: Field):
    v9 = add v0, Field 1
    constrain v2 == v9
    constrain v3 == v0
    constrain v0 == Field 1
    constrain v1 == Field 0
    return
  b1():
    v7 = add v0, Field 1
    jmp b3(v7, v0)
}

Here we can see that we never enter b2, instead going b0 -> b1 -> b3. However when the simplify_cfg pass looks at the predecessors of b3 it sees both b1 and b2 despite b2 not being reachable anymore.
On master we'll simplify the CFG for the SSA to give the program:
After Simplifying:
brillig fn main f0 {
  b0(v0: Field, v1: Field):
    constrain v0 == Field 1
    constrain v1 == Field 0
    v6 = add v0, Field 1
    jmp b1(v6, v0)
  b1(v2: Field, v3: Field):
    v7 = add v0, Field 1
    constrain v2 == v7
    constrain v3 == v0
    constrain v0 == Field 1
    constrain v1 == Field 0
    return
}

This PR checks the previous successors of any block which has it's CFG recomputed to check if they've been left orphaned, if so we remove it from the CFG entirely.
We now instead fully simplify the CFG to give the program:
After Simplifying:
brillig fn main f0 {
  b0(v0: Field, v1: Field):
    constrain v0 == Field 1
    constrain v1 == Field 0
    v4 = add v0, Field 1
    v5 = add v0, Field 1
    constrain v4 == v5
    constrain v0 == Field 1
    constrain v1 == Field 0
    return
}

Additional Context
Documentation*
Check one:

[x] No documentation needed.
[ ] Documentation included in this PR.
[ ] [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

[x] I have tested the changes locally.
[x] I have formatted the changes with Prettier and/or cargo fmt on default settings.

We can maybe pull this extra logic out of recompute_cfg as we only need to run it when simplifying constant jmpifs rather than every time we recompute the CFG for a block. I just left it here as it maintained the same interface.

Seems like the above is required. Marking draft so I can do some cleanup tomorrow.


Edge functions has crashed

Environment

Operating System: Darwin
Node Version:     v18.12.0
Nuxt Version:     3.0.0
Nitro Version:    1.0.0
Package Manager<EMAIL_ADDRESS>Builder:          vite
User Config:      app, components, modules, css, build, runtimeConfig, proxy, graphql-client, vite
Runtime Modules<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Build Modules:    -

Reproduction
Cannot reproduce this issue without using external NITRO_PRESET provider like vercel of netlify
Describe the bug
Edge functions crashes with NITRO_PRESET set to vercel-edge or netlify-edge I tried to deploy my app on both services.
Additional context
No response
Logs
Netlify error:

Mar 24, 02:06:19 PM: error  TypeError: Pn.Debug.extend is not a function
    at file:///root/.netlify/edge-functions/server.js:1:160277Mar 24, 02:06:41 PM: error  TypeError: Pn.Debug.extend is not a function
    at file:///root/.netlify/edge-functions/server.js:1:160277

Vercel error:
[GET] / reason=EDGE_FUNCTION_INVOCATION_FAILED, status=500, user_error=true
TypeError: Sn.Debug.extend is not a function
    at (index.mjs:1:160264)
    at (__nitro:middleware.js:1:17)

Can you please open an issue in nuxt repository with reproduction? 🙏🏼 (from above info, i can only guess nuxt-proxy might cause some issues. we have proxy option in routeRules you can replace it with)


Cannot use website.

Since the changes in eigentrustnetwork/ETN#19 and eigentrustnetwork/ETN#20, you are no longer able to use the website.  The login function works but then it processes the 200 response incorrectly so future calls do not work.

Temporarily rolled back ETN API until the fix is pushed.


ctb: Reset Guardian address

Description
Fixes the OP Sepolia Guardian role to reflect the resetting of the address to the FakeFoundationSafe.

Okay this is still failing because Mode, Metal, and Zora share the foundation guardian. Those should be updated in superchain/extra/addresses/sepolia/metal&mode&zora.json


test(vrt): update visual snapshots for atroncy:master [skip ci]

This PR was generated based on visual changes detected in #4906. Please verify that the updated snapshots look correct before merging this PR into atroncy:master.

Visual changes have been resolved. This PR will be closed and vrt/atroncy/baseweb/master will be deleted. If future commits on atroncy:master trigger visual changes, a new snapshot branch will be created and a new PR will be opened.


Placeholder text for lv_ta

Added a second label to lv_ta, this label is used as text placeholder and will only be visible if the text area contains no text.
LV_TA_STYLE_PLACEHOLDER was added to assign a style to the placeholder label.

I've added some fixes:

If somehow the scrollable has a style with non zero padding the to label needs to positioned to the same coordinates.  It normally not happens with the TA but can happen with an object derived from TA. Test with: lv_page_set_style(ta, LV_PAGE_STYLE_SCRL, &lv_style_plain);
in placeholder_update is used ta_text[0] == '\0' instead of strlen because len was used only to decide the length is 0 or not 0;
use LV_LABEL_LONG_BREAK by default
handle the placeholder in lv_ta_set_one_line()

What do think about the changes?

Maybe we should check the font height of the label and the placeholder in lv_ta_set_one_line, just in case someone uses a bigger font for the placeholder.

Good idea.
However, if the two fonts are different - and we take into account the placeholder's font height too - then the TA height will be changed when the first character is entered and change again when the last character is deleted. IMO it'd look ugly. So I vote for leaving the height according to the main label's font.

Then we leave it as it is, i don't believe that many people would like to use two different fonts or font sizes for the text and the placeholder.

Ok, then I merge it!
Thank you for your PRs for Text area. You made it really better! :+1:


Avoid scan of other filesystems

Hey ! It could be nice to add -mount to find here to avoid scan of large mounted pathes…
https://github.com/bedis/letsencryptforvtm/blob/f57853132020081c2a4c4fd513126938caaddf4c/letsencryptforvtm.sh#L28

In some cases if the VTM is updated probably you see more the one path
To solve it you can find it manually and then add the last version's path

Something like this:
root@BMLb01:~/.acme.sh# find / -name zcli | egrep "/bin/zcli$"
/usr/local/zeus/zxtm-10.0/bin/zcli
/usr/local/zeus/zxtm-11.1/bin/zcli
then add the current version in the line
[ -z "$ZCLI" ] && ZCLI=$(find / -name zcli | egrep "/zxtm-11.1/bin/zcli$")


Why there's no Authentication.Windows package?

There's a  Microsoft.Windows.Compatibility nuget package now, Can we build a asp.net core native Microsoft.AspNetCore.Authentication.Windows now ? or that's some thing must impelement in the Server (eg.IIS,Kestrel,HttpSys) ?

It doesn't exist because currently it's supported only with IIS and HttpSys, where there are options to turn it on. It's not currently supported in Kestrel, but we are tracking that feature request in https://github.com/aspnet/AspNetCore/issues/4662 .

@Eilon

It's not currently supported in Kestrel

Could you answer my last question :  or that's some thing must impelement in the Server (eg.IIS,Kestrel,HttpSys) ?
I see the windows authentication is a module in IIS ,  Can it just be a authentication MiddleWare in Asp.NetCore when it's possible ?
service.AddAuthentication().AddWindows(WindowsAuthenticationOption);

...

app.UseAuthentication() ;

Yes in theory it could be, and that's what the other issue is tracking.


extend -> assign

renames extend
simplifies code (uses deconstructing instead of Object.assign)
adds new testcase
improves the docs

opinions? @1-liners/maintainers

Great! Breaking change though – we should hop on to semver shouldn’t we? 1.0.0?

Yop, agreed. Let's publish this with 1.0.0

:sun_with_face:


fix(build): Revert docker upgrade to fix Darwin builds

Description
Reverts: https://github.com/stackrox/stackrox/pull/8354
Until this commit hits a tag, https://github.com/moby/moby/pull/45724 then builds and tests on darwin are broken
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change
Backed it out then ran a test that was working
Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

/retest


make grid container default size 1440px

This means that each columns width once we hit the upper limit is now an even pixel value. This will help with rounding errors from fractional lengths.
#826

@macandcheese fixed the merge conflict.

🌐


[Github] Update language-widget.html.twig

Add missing space between twig block title and delimiter.

1. Why is this change necessary?
Syntax error.
2. What does this change do, exactly?
Add missing space between twig block title and delimiter.
3. Describe each step to reproduce the issue or behaviour.
4. Please link to the relevant issues (if any).
5. Checklist

[ ] I have rebased my changes to remove merge conflicts
[ ] I have written tests and verified that they fail without my change
[ ] I have created a changelog file with all necessary information about my changes
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Imported from Github. Please see: https://github.com/shopware/shopware/pull/3761

[public] Automated response: This issue is linked to the internal issue https://shopware.atlassian.net/browse/NEXT-36876.
[created from NEXT-36876, comment 484246]


Victor does not work with Gradle 7: Type 'com.trello.victor.RasterizeTask' property 'includeDensities' is missing an input or output annotation

When using Gradle 7, in both the source-code sample project and my source code using victor:1.1.0 I get the following error:
Type 'com.trello.victor.RasterizeTask' property 'includeDensities' is missing an input or output annotation.
I've tested it on a fork here:
https://github.com/arberg/victor/tree/gradle7

Very strange; despite having @Input on includeDensities, it looks like the 1.1.1 build didn't include it somehow.
I'll update to Gradle 7 and also see if a new build puts out the right annotations.

Alright, after a lot of wrangling, I've got Victor building with Gradle 7 (both the library and the sample). Let me know if v1.1.2 works for you or not.

Cool it works, that was really helpful. Do you have anywhere where I can support your work (paypal etc?)? Possibly pm me at<EMAIL_ADDRESS>

I appreciate the gesture but it's not necessary, I'm just glad that it's working again.


Wrong Floating Point Behavior on M1

Reported by @EnzoPlayer0ne while setting up an M1 (Arm64) CI builder for Motoko:
https://github.com/dfinity/motoko/actions/runs/5819381906/job/15777666227
The copySign floating point operation seem to fail specifically on M1.
During the tests of Motoko base library run by the Motoko CI build:
copySign/second argument negative NaN failed:
1.200000 was expected to be -1.200000

copySign/positive infinity and negative NaN failed:
inf was expected to be -inf

copySign/negative infinity and negative NaN failed:
inf was expected to be -inf

It seems that some similar errors still occur on certain M1 machines, even with x64 emulation. Further analysis is needed.

Seems to be solved. The remaining issue was probably due to a version mismatch.


Update flutter create templates and flutter repo to use new Android embedding

Also migrate embedding usage code that is already materialized in the Flutter repo such as devicelab tests and examples/ etc

should we leave this issue open for defaulting this flag to on?

@xster I filed https://github.com/flutter/flutter/issues/43638

Thanks. I added that to our spreadsheet


Could not resolve all artifacts for configuration ':video_player_android:debugUnitTestRuntimeClasspath'.

I have added dependency in my pubspec.yml file and implement the code to pay the video. Its working when I play the app on emulator. But when I try to build the apk it gives following errors on the terminal:
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:lintVitalRelease'.
> Could not resolve all artifacts for configuration ':video_player_android:debugUnitTestRuntimeClasspath'.
   > Failed to transform bcprov-jdk15on-1.68.jar (org.bouncycastle:bcprov-jdk15on:1.68) to match attributes {artifactType=processed-jar, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
      > Execution failed for JetifyTransform: /Users/anukant/.gradle/caches/modules-2/files-2.1/org.bouncycastle/bcprov-jdk15on/1.68/46a080368d38b428d237a59458f9bc915222894d/bcprov-jdk15on-1.68.jar.
         > Failed to transform '/Users/anukant/.gradle/caches/modules-2/files-2.1/org.bouncycastle/bcprov-jdk15on/1.68/46a080368d38b428d237a59458f9bc915222894d/bcprov-jdk15on-1.68.jar' using Jetifier. Reason: IllegalArgumentException, message: Unsupported class file major version 59. (Run with --stacktrace for more details.)
           Suggestions:
            - Check out existing issues at https://issuetracker.google.com/issues?q=componentid:460323&s=modified_time:desc, it's possible that this issue has already been filed there.
            - If this issue has not been filed, please report it at https://issuetracker.google.com/issues/new?component=460323 (run with --stacktrace and provide a stack trace if possible).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 32s

I am facing this exact same issue.

I am using the latest version video_player: ^2.4.7
But, finally I have resolved this issue by
replace build.gradle
from
classpath 'com.android.tools.build:gradle:4.1.0'
To
classpath 'com.android.tools.build:gradle:7.1.2'
And
Also replaced gradle-wrapper.properties
From
distributionUrl=https\://services.gradle.org/distributions/gradle-6.7-all.zip
To
distributionUrl=https\://services.gradle.org/distributions/gradle-7.4-all.zip
And it start working...


CAMEL-20281 - Camel-AWS Components: Make it possible to use AwsSessio…

…nCredentials to support temporary credentials - AWS ECS
Description

Target

[x] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[x] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[x] I checked that each commit in the pull request has a meaningful subject line and body.

[x] I have run mvn clean install -DskipTests locally and I have committed all auto-generated changes

/component-test aws


[MM-17441] Fix bug preventing CMD+~ to switch between open Safari windows

Summary
The PR fixes an issue where a11y_controller.js is preventing CMD+~ from working in Safari to switch between multiple open windows.
Ticket Link
https://mattermost.atlassian.net/browse/MM-17441

@deanwhillier I noticed the Jira ticket is already resolved. Can you help merge and cherry-pick this PR?


Did #2335 break something?

After #2335 I can't build esp32-core:wapi.  I see however that the CI builds it correctly, but I can't locally.
make V=1

...

make[1]: Entering directory '/home/ouss4/NuttX/nuttx/mm'
make[1]: Nothing to be done for 'depend'.
make[1]: Leaving directory '/home/ouss4/NuttX/nuttx/mm'
make[1]: Entering directory '/home/ouss4/NuttX/nuttx/libs/libxx'
make[1]: Nothing to be done for 'depend'.
make[1]: Leaving directory '/home/ouss4/NuttX/nuttx/libs/libxx'
make[1]: Entering directory '/home/ouss4/NuttX/nuttx/wireless'
make makedepfile
make[2]: Entering directory '/home/ouss4/NuttX/nuttx/wireless'
cat   >  Make.dep

The build hangs at the last command.
Reverting #2335 the build continues with no issues.
Can't build in 2 Arch based computers. Building in a Ubuntu based docker image passes correctly, however... Was it only tested on Debian based distros? We had few changes before that broke other non Debian distros.
Some tools used in the build system are implementation dependant and change from distro to distro.  Last time we had an issue with different implementations of touch.

I tested on my Ubuntu, that is the only OS I can test myself.
The make rule being executed there is:
makedepfile: $(CSRCS:.c=.ddc) $(ASRCS:.S=.dds)
	$(call CATFILE, Make.dep, $^)
	$(call DELFILE, $^)

which means that CSRCS/ASRCS appears to be empty, which is strange.
I just reproduced the issue locally, will see what it is about.

@Ouss4 is CONFIG_WIRELESS really needed for this config? It seems that while WIRELESS is turned on, not WIRELESS_* option is enabled so nothing is actually added to the build. I disabled it and it builds OK, but I'm not sure if the option is required
for any other reason.
I think that would be one thing to fix.
As to having empty CSRCS/ARSCS, I'm not sure if that is a normal thing to have and consider or not. I could make the dep rule not run if it is empty somehow, but maybe that would hide an unexpected situation.

@Ouss4 is CONFIG_WIRELESS really needed for this config?

I think it's necessary to include the wireless directory into the build.
https://github.com/apache/incubator-nuttx/blob/cb8d2672305c4a2f8740a2c561dd71a16ab3b7a8/tools/Directories.mk#L147-L151
https://github.com/apache/incubator-nuttx/blob/cb8d2672305c4a2f8740a2c561dd71a16ab3b7a8/tools/FlatLibs.mk#L135-L139

OK, I will change CATFILE definition so that it does not hang on empty list of input files (a sane expectation). I'm not sure how to adapt the windows case, though. This is how the change looks for the Linux case so far:
# CATFILE - Cat a list of files
#
# USAGE: $(call CATFILE,dest,src1,src2,src3,...)

ifeq ($(CONFIG_WINDOWS_NATIVE),y)
define CATFILE
	$(Q) type $(2) > $1
endef
else
define CATFILE
	$(Q) if [ -z $(2) ]; then echo '' > Make.dep; else cat $(2) > $1; fi
endef
endif

BTW, I'm perplexed as to why this didn't fail on CI. My guess is that stdin is not available and thus "cat > Make.dep" would return immediately.

OK, I will change CATFILE definition so that it does not hang on empty list of input files (a sane expectation). I'm not sure how to adapt the windows case, though. This is how the change looks for the Linux case so far:
# CATFILE - Cat a list of files
#
# USAGE: $(call CATFILE,dest,src1,src2,src3,...)

ifeq ($(CONFIG_WINDOWS_NATIVE),y)
define CATFILE
	$(Q) type $(2) > $1
endef
else
define CATFILE
	$(Q) if [ -z $(2) ]; then echo '' > Make.dep; else cat $(2) > $1; fi
endef
endif

Shouldn't cause a hang.  It should generate a (uncaptured) error message and generate an empty file:
C:\Users\spuda>type > foobar
The syntax of the command is incorrect.

C:\Users\spuda>dir foobar
 Volume in drive C has no label.
 Volume Serial Number is 947F-7817

 Directory of C:\Users\spuda

11/25/2020  06:05 PM                 0 foobar
               1 File(s)              0 bytes
               0 Dir(s)  943,090,741,248 bytes free

Ok, then, I'll submit the above change. Thanks for the input Greg.


Feature/dashboard page

finish version 2 of landing page

@boy-warakorn .env.example is missing

added now


Unify pins libraries

This library and the official Rstudio library which is maintained by @javierluraschi have inconsistent behaviour, and this one doesn't work very well with Rstudio Connect. Would it be possible to figure out a way to either unify the two libraries, or change the name of one of them so that the Rstudio one could be pip installed?
I raised a parallel issue on the Rstudio repository: https://github.com/pinsjs/pinsjs/issues/60

There isn't currently an official python pins library from RStudio. The PyPI pins "name" will be passed on to RStudio when one is created.

Oh great, have you communicated that to Javier?

Javier left RStudio some time ago. Hadley is aware though.

Thanks, is there anything I can do to advocate for this? We're an rstudio customer and lack of python support on Rstudio Connect is causing us some headaches.

Would probably be worth letting your CS rep know about your frustrations and maybe raising an issue on the official pins repo if you haven't already. https://github.com/rstudio/pins/issues

The PyPI hosted pins is now an official RStudio project.
Early stages of development, but Connect support already seems to work well.


Supply structural schema without version map

Bug was introduced in #81 that got past review. This change correctly supplies the structural schema. (Version map is no longer required due to https://github.com/kubernetes/kubernetes/pull/121337)

🔥


Fix INCLUDE_RAW = True

Custom String subtypes have been setting INCLUDE_RAW, not ES_INCLUDE_RAW

Correct, not relevant to BQ


bump to version v<IP_ADDRESS>

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

It seems the compiled osx version was compiled on a github actions runner using macos-latest (macos-latest switched to the arm-cpu variant some time ago). See https://github.com/lierdakil/pandoc-crossref/issues/452
I'll mark this PR as a draft for now. Hopefully I'll find some time to make a PR to pandoc-crossref to update the GA builds (if not anyone else beats me to it)


Make sure the webhook is always registered before allowing user to create resources

As of today, knative/pkg's code is the one responsible to register the webhook on kubernetes. This means, the following are possible :

The webhook fails to start (for any reason), it does not get registered, thus it's not applied
The webhook takes time to start (slow network, …), it's not registered quickly enough, a user can create resources

In all those cases, the user can end up creating resources that are either not valid or not mutated (with defaults values)
There is two way to fix these cases:

Move the webhook registration in the yamls, so that the webhook is registered as soon as the CRDs are created, before it started, and thus it will fail if the webhook is not running
Having the controller validate

I have a very very high preference for solution 1. .

/assign

@vdemeester are you still working on this or should we throw it in the backlog instead?

Throwin it in the backlog :sunglasses: But I added it to the "post beta" milestone so we won't forget it!