Update Docker-Moby Client and Server to support zstd

Tool name
Docker-Moby
Tool license
Apache
Add or update?

[ ] Add
[X] Update

Desired version
21+
Approximate size
No response
Brief description of tool
This PR tagged for 21.x adds support for zstd images. Currently you can not setup workflows to use container images compressed with zstd. Havng this would be very desirable as this can greatly reduce time necessary to pull large images.
I'm not sure why docker is being held back at 20, but it would be very nice to have it updated.
URL for tool's homepage
No response
Provide a basic test case to validate the tool's functionality.
Should be able to use a workflow like this:

name: Example
on: [push]
jobs:
  setup:
    name: Setup
    runs-on: ubuntu-latest
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          driver-opts: image=moby/buildkit:master

      - name: Build and push the CI image
        id: build
        uses: docker/build-push-action@v4
        with:
          file: Dockerfile
          push: true
          tags: ghcr.io/username/repo:latest
          outputs: type=image,oci-mediatypes=true,compression=zstd,compression-level=3,force-compression=true

  lint_and_unit_test:
    name: Lint and unit test
    needs: setup
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/username/repo:latest
    steps:
      - name: Sanity Test
        run: echo I should run

Platforms where you need the tool

[ ] Azure DevOps
[X] GitHub Actions

Runner images where you need the tool

[ ] Ubuntu 20.04
[X] Ubuntu 22.04
[ ] macOS 11
[ ] macOS 12
[ ] macOS 13
[ ] Windows Server 2019
[ ] Windows Server 2022

Can this tool be installed during the build?
No, it's needed to pull newer zstd images.
Tool installation time in runtime
No response
Are you willing to submit a PR?
I can submit a PR if this isn't being held back for some special reason.

@TikiTDO ,
we install moby-engine provided by Ubuntu: https://github.com/actions/runner-images/blob/main/images/linux/scripts/installers/docker-moby.sh#L17
if you wish to switch to some other moby package, it is ok for your private repo (just using "apt install ..." as usual), but it probably won't be desired for the rest of users.
please keep it in mind when proposing PR

We are using and providing only the Moby versions supported by the Microsoft's repository (always latest). Please contact https://github.com/microsoft/linux-package-repositories if you have concerns

@ilia-shipitsin In this case my problem happens while the runner is trying to start the image that the actual workflow will run in. The only viable solution for me would be to build my own runner and launch it on a self-hosted machine, which seems like a lot of extra work for something that seems to be due to happen soon anyway. I tend to not like opening issues otherwise because it tends to be a waste of my time.
@mikhailkoliada Based on this link it appears that the newest moby-cli is in the official Microsoft repository.  This issue is more about at least understanding when it might ship because it's affecting a workflow I'm working on, so I want to know if this is something I can wait on imminently, or if I'm going to be waiting another year and can put the feature on the back burner.

@mikhailkoliada , let's keep it open for a while since it is a feature request anyway
I'm against changing moby build to some custom build, but I beleive there might be other options

changing moby during runtime
maybe zstd is an option which can/should be enabled for moby ?
maybe microsoft folks will help us

@ilia-shipitsin It appears that the new version is actually in the MS repo, but it's in testing. No custom builds necessary. I imagine it will be released imminently, but I couldn't find any discussion on it. See this link. That said, the zstd PR is fairly small so it may be possible to backport it to moby 20.


locale nui_warning changes and fixed some typos

How good is the source on those translations?
Need to be native level

I am danish and I can confirm that the da.json changes are native level. I have a polish friend that I just asked and he said that the pl changes are also native level. (Idk about the other ones since Im not the one who made this pr, just wanted to help out since these are some good changes)

I can confirm that it's native level, I'm from Romania, Also what's wrong with the sv.json?


Added method getCurrentRouteName to Zend\Mvc\Router\RouteStack interface.

This feature allows you to add diffrent behaviours to applications (modules) depending of the route name ie. layout swiching (Sorry for my english)

Closing based on DASPRiD's comment.


knife ec2 server create not support AWS longer IDs

I changed on AWS Resource ID length management the resources on my account to support longer IDs (reservation, instance, volume, snapshot). Newly created instances from AWS console recieved the longer ID, but when I created a new instance with knife ec2 server create, the instance goes up with shorter ID like before.
knife ec2 server create -N $NAME -r $RUNLIST -S $KEYPAIR -- subnet $SUBNETID -E $ENV -Z $AVAILABILITYZONE --region $REGION --security-group-id $SECGROUP --bootstrap-version 11.8.2"
#gem list 'knife-ec2'
*** LOCAL GEMS ***
knife-ec2 (0.13.0, 0.12.0)
Is anyone have idea why ?
Thanks!

applying the change with the root account and now it works everywhere as expected.


improvement: Try and remove more macros from parsers

Also some changes to make it cross compile

Ok, it's good now :sweat_smile:


Implement Conventional Commits

We should implement conventional commits in our CI/CD to help keep contributions more organized and uniform. Doing so will also allow us to automate semantic versioning for future releases.
https://www.conventionalcommits.org/en/v1.0.0/

I want to talk through this one a bit. I've got some concerns around their implementation of the concept

@ermish Sure! Can we start a thread for it on Slack since it's hard for us to meet up during the week?

actually, let's just try it as is. 👍
I love their concept of organizing types of commits and the ability to auto-generate changelogs.
My concern was just around the specific naming like feat, chore, etc. which is completely different  what I'm used to and most of my workplaces use.

@theyokohamalife do you have any tool in mind that will generate a changelog based on conventional commits?

I find this a bit cumbersome and better suited to larger software projects. However, if you feel that you want to do this, let's configure husky or another tool so that we guide users to the correct formatting (might require setting up a rule on push)

I find this a bit cumbersome and better suited to larger software projects. However, if you feel that you want to do this, let's configure husky or another tool so that we guide users to the correct formatting (might require setting up a rule on push)

@ann-kilzer  Since you're more familiar with husky could you create an issue to add the conventional commit formatting to it?


Multi user configuration and support, set user name/id

I miss being able to set user name/id in the new BotFx emulator -why was this feature cut?

The new emulator has a different user interface design from its predecessor. Because of this, some of the features do not map 1:1. This is one of those instances. Configurable user name/id is a feature that will be coming in as part of a new management UI for conversations and users in conversations.

Thanks!
I'll use ye ole one until it's complete.

@eanders-MS any update on when this feature will be available?

@garypretty this is being tracked by #522 . It is currently in our backlog of items and has not yet been assigned to a milestone, although, that may change.

Any update on this issue? This is a particularly useful feature to test bot-to-human handoff scenarios.

Controlling the UserId is a critical need for testing authentication and human handoff scenarios. At this time the Emulator is not able to be used for more complex Bot testing and has led to time-consuming workarounds which takes developers out of our tooling workflow.
Linked Accounts based scenarios are a key blocked for our Virtual Assistant customers. In this scenario, a Bot can be linked to other services (e.g. O365, Google) ahead of a voice-led conversation removing the need for a user to be prompted for authentication which in voice first scenarios doesn't work.
This approach works by having a user log-in through a mobile app/website and perform the account linking. The "object identifier" is used as the user identifier which is then also passed on the from.id activity property meaning the two can be matched and tokens retrieved.
I'd suggest we implement two things:

[P0] The emulator generates a random UserId and needs to provide a way (Settings pane ideally) for the User to provide their own UserId (typically a GUID) which is then set on the from.id property.
[P1] When/If the user clicks "signin" within the emulator we retrieve the '"http://schemas.microsoft.com/identity/claims/objectidentifier' claim and set this to the UserId automatically (could prompt for confirmation). This would avoid the need for the user to "find" their UserId when they are using a microsoft account.

@darrenj 👍I like this suggestion. We'll cut a ticket to address the P0

Is this correct in that this is still an open issue, with no way to emulate multiple users within the same conversation?

Hello @alevinetx ,
Yes, this is still currently unsupported in the Emulator.

Hello. We'll close this issue as we have no plans to support multi-user on our 6-12 month roadmap.


Update sbt-coveralls to 1.3.2

Updates org.scoverage:sbt-coveralls from 1.2.7 to 1.3.2.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scoverage", artifactId = "sbt-coveralls" } ]

labels: sbt-plugin-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #121.


Add README, Travis configuration, and more powerful gulp file

Changes:

Make gulp start web server (use this instead of slc run)
Make gulp test start web server and run API tests and integration tests
Make npm test run jshint and tests
Add documentation for installation, usage, and testing to README file
Setup Travis and add Travis badge to README file
Exclude some third-party libraries from JSHint

I merged your changes and some changes to the gulpfile and package.json. Please review.
The gulpfile should support all testing and all platforms.
All dependencies through week 3 should be included in package.json
Known issue: when running gulp test:all there is a significant delay after test:unit before the integration tests start but it does eventually complete successfully.

The unit tests for me fail with:
ERROR [config]: File /home/trey/repos/project/karma.conf.js does not exist!


618 Migrate courses edit age range

Context
The "Courses" section (linked from a page like this https://qa.publish-teacher-training-courses.service.gov.uk/organisations/2GU/2022) needs to be reimplemented in New Publish.
Changes proposed in this pull request
Reimplement the functionality to update the age range from the basic details tab view

Guidance to review
Checklist

[x] Make sure all information from the Trello card is in here
[x] Attach to Trello card
[x] Rebased master
[x] Cleaned commit history
[x] Tested by running locally

 Migrate Courses - Course show page - Updating age range


home: Handle empty HOME on unix

What does this PR try to resolve?
Contrary to the documentation, if the HOME environment variable is empty on Unix, then an empty Path is returned. This is because std::env::home_dir doesn't handle this case.
I've copied the code from standard library & added a .filter(|s| !s.is_empty()) check. The rest of the code is the same.
Also added a test similar to Windows.
How should we test and review this PR?
The current home_dir function doesn't handle an empty HOME environment variable, with this PR, it should.
Additional information
Related discussion: https://github.com/rust-lang/rust/issues/71684#issuecomment-1519151955

:umbrella: The latest upstream changes (presumably #12021) made this pull request unmergeable. Please resolve the merge conflicts.

Can you provide some context here about why this would be changed? What is the importance of how it handles an empty HOME value? When would that happen in practice? How do other ecosystems handle this?

Can you provide some context here about why this would be changed? What is the importance of how it handles an empty HOME value? When would that happen in practice? How do other ecosystems handle this?

The docs falsely claim that if HOME is empty, then it falls back to getpwuid_r.
I hope to see this crate recommended in the standard library & it came up in the linked discussion that this crate doesn't handle this case.
It is not very difficult to get a shell where the HOME variable is not set or set incorrectly. For eg. env -i bash.
A lot of crates outside of cargo & rustup use home as well, so it would be useful to fix it for the ecosystem.

The docs falsely claim that if HOME is empty, then it falls back to getpwuid_r.

Can you say why we wouldn't just change the docs instead?

It is not very difficult to get a shell where the HOME variable is not set or set incorrectly. For eg. env -i bash.

From what I can tell, at least on my systems, that unsets the HOME variable (which should go to the fallback).
As for the ecosystem question, I was asking more along the lines of what other languages and libraries do. For example, Python doesn't handle it, and it essentially becomes unusable. go also seems to fail with an empty HOME. Are there any languages or common tools or libraries that handle this? Or is this requirement a holdover from a documentation mistake when the initial env module was added?
I'm not completely opposed to checking for this case. However, it's a lot of code to add for an edge case that doesn't seem terribly important without some motivation.

The docs falsely claim that if HOME is empty, then it falls back to getpwuid_r.

Can you say why we wouldn't just change the docs instead?

It is not very difficult to get a shell where the HOME variable is not set or set incorrectly. For eg. env -i bash.

From what I can tell, at least on my systems, that unsets the HOME variable (which should go to the fallback).
As for the ecosystem question, I was asking more along the lines of what other languages and libraries do. For example, Python doesn't handle it, and it essentially becomes unusable. go also seems to fail with an empty HOME. Are there any languages or common tools or libraries that handle this? Or is this requirement a holdover from a documentation mistake when the initial env module was added?
I'm not completely opposed to checking for this case. However, it's a lot of code to add for an edge case that doesn't seem terribly important without some motivation.

Hmmm. I can't find any resources on any other languages handling this. On top of that, defining $HOME is a requirement of the POSIX spec, so I guess it should be fine to not handle this. I'll raise a new PR to fix the documentation.
@ehuss It would be nice to get your input on https://github.com/rust-lang/rust/issues/71684 & https://github.com/rust-lang/rust/pull/110665.

@ehuss One thing to note is that this crate currently uses a fallback for Windows, but not for Unix. On Python, it doesn't use a fallback for both unix & windows. Same for Go.
While the documentation could clear that up, it is still unexpected that the 2 platforms are treated differently.


[AMBARI-23106] Optionally generate localjceks://file/{path} url

Backport of #494 to trunk - Generate localjceks://file/{path} urls for local certificate providers
What changes were proposed in this pull request?
Ambari generates 'jceks://file/{filepath}' as a credential provider URL, which force using the JavaKeyStoreProvider - which uses the hdfs API to access the local file system.
This cause problems, when the local file system access from that API is disabled - for security reasons, for example in Oozie. Fortunately, there is a LocalJavaKeyStoreProvider which can be used in this case, which can be accessed with  a 'localjceks://file/{filepath}' URL
How was this patch tested?
Manually tested with Oozie. Before the patch, the generated oozie-site.xml contained jceks://file/usr/hdp/current/oozie-server/conf/oozie-site.jceks, after the patch, it became localjceks://file/usr/hdp/current/oozie-server/conf/oozie-site.jceks

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1041/
Test FAILed.
Test FAILured.

retest this please

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1047/
Test PASSed.


[ Question ] When does the inner state of the LSTMs reset during training?

Just as a sanity check;
The shuffle_on_load flag made me believe, that the LSTMs' hidden state would be reset every sentence; however, diving into the code it seems it actually is not reset during entire training. Is this correct?
Might be a novice question, I'm new to Tensorflow I'm afraid.

That's correct, the LSTM hidden states are never reset during training.

Thanks for the quick answer.
Would you be interested in a PR adding a flag for document-wise training?


How can I test the remote methods?

Testing remote methods should be the same as normal API methods. You can put the URL in the "models" field of the test. If you need more specific advice please let me know what you need to test and what you've tried.


page does not render in IE11

My project doesn't render anymore in IE11. http://reactboilerplate.com/ seems to have the same issue. After loading, the webpage is completely blank and there are no errors in the console.
I've tried using babel-preset-es2015-ie but it didn't help.
Any help or suggestions would be appreciated.

I have a problem with this on Safari also. It just stop working on production. I guess it is about dependencies after it got update.
Btw, i got some error show in my console ststing uncaught promise

@chaintan17 Can you give us more info on the steps to reproduce?
I just pushed a dependency update #803 and I had no problems in Safari.

I spend my whole sunday fixing this bug on Safari :(...
Look like because i install dotenv library and in produciton server, there is no .env file (because it actaully set in ENVIRONMENT variables)
and it annoy Safari ...
So, the problem is not related to the code. I'm not sure about what IE11 problem then.

No... it's not about dotenv
It still happen occasionally. After hit refresh many times, few of the request got blank response with below error in console.log
I will try reproduce it if it occurs in sample boilerplate.

+1 I still see issues when testing http://reactboilerplate.com in ie10, ie11 and refference to #805 didn't really clear up the path I need to take for a fix. @chaintan17 did you manage to overcome it?

This should be fixed in the newest verison of the boilerplate if #805 was indeed the problem. Can you verify @chaintan17?

For me it only worked when removing "transform-react-inline-elements" from package.json

Closing due to inactivity, let me know if you're still running into this!

Does this issue still exist? One of my client recently reported the same problem on IE 11 on my reaact-boierplate, firebase, antd design codebase


Multiple dateTime types in one schema

We have a JSON schema where most of the string date-time properties should be converted into java.time.OffsetDateTime  objects:
        "creationDate": {
            "type": "string",
            "format": "date-time",
            "description": "Timestamp showing when the mediaitem was created."
        },

but one in particular needs to be converted into a java.time.LocalDateTime
        "eventDate": {
            "type": "string",
            "format": "date-time",
            "description": "Timestamp the user sets for displaying when an event happened. Due to the defined Java dateTimeType it is stored as UTC but to be treated as if no timezone."
        },

We have defined the default dateTimeType:
  dateTimeType = "java.time.OffsetDateTime"

We would like to be able to change the type for this one property, but we have not found a way to do that.
Are we missing something, or is this an enhancement that could be included in a future release?

I think one way to solve this would be to add "existingJavaType": "java.time.LocalDateTime" to the schema for eventDate.

existingJavaType

Hi, we tried that (I'm a colleague of himslm01) and unmarshalling worked okay, but marshalling didn't, ended up with something like:
"eventDate":[2015,7,10,8,51,25,648000000]
at a guess due to jackson treating it as a java object rather than a date with a format string. We could write custom serialisers everywhere, but that starts getting a bit nasty. You already have the customDateFormat option to override the date format at the per-field level, so it would seem to make sense to allow a customDateTimeType to override dateTimeType at the per-field level?

customDateFormat option to override the date format at the per-field level, so it would seem to make sense to allow a customDateTimeType to override dateTimeType at the per-field level?

Even if we allow customDateTimeType to override dateTimeType at a field level, this won't solve the problem of marshalling causing that bad output. You shouldn't need to write custom serializers everywhere though, just use https://github.com/FasterXML/jackson-modules-java8.

It may be because I thought that to use existingJavaType, you had to have type="object" rather than type="string".
That may have been my misunderstanding. I've tried this now:
      "eventDate": {             "type": "string",             "format": "date-time",             "existingJavaType": "java.time.LocalDateTime"         }
which results in
   @JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSX", timezone = "UTC")     @JsonProperty("eventDate")     @JsonPropertyDescription("Timestamp showing when an event happened. This could be a transmission event or a recording event. Stored as UTC but to be treated as if no timezone.")     @NotNull     private LocalDateTime eventDate;
but when serialising get: Caused by: java.time.temporal.UnsupportedTemporalTypeException: Unsupported field: OffsetSeconds
So maybe need to do something with a custom date format. Anyway yes, looks like existingJavaType will work, just my misunderstanding that it only applied to object fields.


popover attribute causing error in Chrome version 109

Vue version
3.2.45
Link to minimal reproduction
https://codesandbox.io/s/silly-pike-85lncp?file=/src/App.vue:60-115
Steps to reproduce
User latest chrome version (109) and add an attribute with the name popover to your component.
<calendar :popover="popover"/>

What is expected?
No error in console.
What is actually happening?
Error in console:
Found a 'popover' attribute. If you are testing the popover API, you must enable Experimental Web Platform Features. If not, note that custom attributes must start with 'data-': https://html.spec.whatwg.org/multipage/dom.html#custom-data-attribute. This usage will likely cause site breakage when the popover API ships: https://chromestatus.com/feature/5463833265045504.
System Info
No response
Any additional comments?
Might be related to this issue: https://github.com/vuejs/vue/issues/273

Not sure what should be wrong here. Component props that are not declared in the component will fall through on the component's root element as HTML attributes. That's a Vue behavior ever since Vue 2.
It just so happens that Chrome warns about this particular HTML attribute.
Solution: A) remove it if your don't need it or B) declare it as a prop in the component if you expect it, or C) set ìnheritAttrs: false` in the child.
not a bug.


Borrow limit > 100% view bug

This is kinda impossible on prod, but on testnet I catch it :D

Hey invis97, this is not a bug, it is intended behavior. it's just that there are no liquidations being made on testnet right now!

@kevinhankpark I think UI is showing it in a wrong way. That why I call that "bug"

@invis87 can you elaborate? it is intended to be shown like that, given that the bar should never go beyond 100% (if there are liquidations)

Hmmm, if it is intended, then probably all fine.
I just think it is ugly :) and looks like an "unexpected behavior":

@invis87 Thanks for this report. I'll fix it on next update.


WyzeHacks working with <IP_ADDRESS> confirmed

I can confirm WyzeHacks is working well on these firmware versions.
Working:
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>

Thanks, please keep doing this for new future releases and let me know when it's working or not.

works with RTSP firmware version: <IP_ADDRESS> as well

I ran into an issue where upgrading V2 to <IP_ADDRESS>, it lost telnet. Attempting to remote install didn't work. I performed a factory reset back to <IP_ADDRESS> and attempted to install the hack using remote install, however it seems the file isn't being pushed to the camera even though is shows "Serving firmware file './firmware.bin' as http://:11808". "installation begins" is not triggerred.

I ran into an issue where upgrading V2 to <IP_ADDRESS>, it lost telnet. Attempting to remote install didn't work. I performed a factory reset back to <IP_ADDRESS> and attempted to install the hack using remote install, however it seems the file isn't being pushed to the camera even though is shows "Serving firmware file './firmware.bin' as http://:11808". "installation begins" is not triggerred.

<IP_ADDRESS> works

<IP_ADDRESS> works

Doesn't seem to work on <IP_ADDRESS> v2

Confirming it works on V3 FW 4.36.25 :)  thank you for the great work !!!
Tested on 4 cams.
My setup is 4 x Wyze V3 , NFS share on TrueNAS 12.0 U-4
Setting up the NFS share the right way took me a long time, as i was getting video files with 0 bytes. Problem was the ACL Mode on the Pool which was set to Restricted instead of Passthrough.


validation of transport rules api definition

Signed-off-by: craman<EMAIL_ADDRESS>

@havetisyan can you review?

lgtm. @abvaidya  can you also please review to make sure you're ok with the structs.

LGTM. @chandrasekhar1996 you would want to run the full athenz build locally to generate other bindings like go client


fix: brotli let for reassignment

Description
brotli is re-assigned on line 26
Additional context

What is the purpose of this pull request?

[X] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[ ] Read the Contributing Guidelines.
[ ] Read the Pull Request Guidelines and follow the Commit Convention.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[ ] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

@Niputi something looks odd here, why brotli is initially false and then reassigned to false if it isn't supported? Should we remove this variable or add a comment that we may enable it in the future?

looks like a mistake. should default to true as it goes to false when it's not supported
  // disable Brotli on Node<12.7 where it is unsupported:
  if (!zlib.createBrotliCompress) brotli = false

Yes, was it like that in the WMR version? Maybe they ended up disabling it globally because it didn't work as expected?

looks like they defaulted to false but haven't given context on why https://github.com/preactjs/wmr/blob/7f96b821c41ed178643749be83d262bf9bc8de5c/packages/wmr/src/lib/polkompress.js#L13


Treat both 2XX and 3XX responses as successful.

The 303 See Other response is often used to direct clients to another page after successfully creating a new resource, and such behavior is encouraged by web standards.
Thanks for this module, it's very useful. I'm using it to develop a web application.

I've made both your suggested changes. Thanks for taking a look at this.

Thank you both for the PR, apologies for taking so long to get to it. I'll just wait for the workflows to pass then go ahead with a merge.


Fix LDS, improve test

Description of changes:

The test for LDS was @skip'ed (!!!), now it's enabled
By doing that I realized that the sample method of LDS was broken (the case where num_samples=None was not handled correctly), now the shaping problems are fixed
The batch_shape/event_shape were also set wrong, now they are fixed and a docstring somewhat related to this has been updated
Assertions in the test for LDS where improved/added

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-65/2 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-65/2/index.html

Codecov Report

Merging #65 into master will increase coverage by 1.26%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #65      +/-   ##
==========================================
+ Coverage   77.85%   79.11%   +1.26%
==========================================
  Files         108      108
  Lines        6199     6200       +1
==========================================
+ Hits         4826     4905      +79
+ Misses       1373     1295      -78

Impacted Files
Coverage Δ

src/gluonts/distribution/lds.py
88.28% <100%> (+68.28%)
:arrow_up:

src/gluonts/support/util.py
73.68% <0%> (+3.15%)
:arrow_up:


XDC: Resend the entire history on first replication task after failover

Merge protocol sometimes rely on the entire history being sent on the first replication task.  As sometime conflict resolution could drop certain events and only way to correctly merge execution history is to send the entire history on first replication task.

solved by #1348, #1353, #1356, #1358


dev-cmd/audit: fix cask arg handling

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

This fixes an issue where brew audit --cask --new-cask did not run the check_token_conflicts check as it should have. This was because at some point in the past, args.token_conflicts? started returning true/false instead of true/nil. Since false.nil? is false but false.blank? is true, this wasn't triggering properly. There are a few other implied audit rules that likely were affected as a result of this. that I've fixed here, too.

Review period skipped due to critical label.

I realized that it is valuable to have a distinction between nil and false for testing. That way, false means "this is 100% false never set it to true" and nil means "unset so make whatever changes are needed."
Now, we can set new_cask to true while keeping online off. Before, setting online to false was treated as if --online wasn't passed (because there's no --no-online flag) and so the --new was overriding the value set in the test.
Instead, we handle the change in the brew audit command, itself, by adding || nil to pass nil (i.e. "unset") to the auditor whenever args.foo? returns false.
In other words, the argument parser returns false when a (non-boolean) switch isn't set, and the auditor expects nil to mean unset. So, we change falses to nils before passing them.

--[no-]appcast is excluded from all of the above because the argument parser already returns nil/false/true because it's a boolean switch.


How didn't display custom property type info?

How didn't display custom property type infomation?  for example: custom defined class or enum.

I've no idea what the issue is here - please provide much more detail, including code samples and a screenshot of what you're expecting to see

I've no idea what the issue is here - please provide much more detail, including code samples and a screenshot of what you're expecting to see

Closing due to inactivity


task: update bundle to v99.9.9-beta.9

Created by the ocp_package job in maze88/network-operator OCP bundle CI.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>4
Details

0 of 0   changed or added relevant lines in 0 files are covered.
17 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.4%) to 64.093%

Files with Coverage Reduction
New Missed Lines
%

pkg/state/state_hostdevice_network.go
2
78.38%

controllers/hostdevicenetwork_controller.go
2
89.77%

pkg/state/state_skel.go
4
83.63%

controllers/nicclusterpolicy_controller.go
9
75.31%

Totals

Change from base Build<PHONE_NUMBER>7:
0.4%

Covered Lines:
3263

Relevant Lines:
5091

💛  - Coveralls


Automatic listing refresh

Originally created by Antoine Nguyen on 2010-04-30T15:29:56Z
When listing messages, it would be to usefull to refresh the page frequently in order to check for new messages.
At least one parameter is needed to specify the refresh rate.

This issue was moved to modoboa/modoboa-webmail#2


Update build_release.yml

i need hep building this app

I believe you are confused with how GitHub works; PRs are for making public code changes.
For further clarification, reading through the GitHub wiki about Pull Requests should help.


Example for Like in query

how to use like in moor for search? Do you have any example?
`
Future<List> getProductGroup({int limit = 0, int offset = 40, String search = ""}) async{
print("search  : " + search);
final query = await ((select(productGroupTable)
..limit(limit, offset: offset)
..orderBy([
(t) =>
OrderingTerm(expression: t.name, mode: OrderingMode.asc)
])
)..where((productGroup)=> productGroup.name.like(search)))
.join(
[
leftOuterJoin(brandTable, brandTable.id.equalsExp(productGroupTable.brand)),
leftOuterJoin(typeTable, typeTable.id.equalsExp(productGroupTable.type)),
])..get();
  return query.map((result){
    return ProductGroupWithBrandType(
      productGroup: result.readTable(productGroupTable),
      brand: result.readTable(brandTable),
      type: result.readTable(typeTable),
    );
  }).get();

}
`
This is my code and always return empty. I dont know how to solve this, could you help me?
Thank you..

Try to do the join first, then call the where, something like this in my case:
    final query = select(songs).join(
        [leftOuterJoin(bands, bands.id.equalsExp(songs.bandId))])
      ..where(bands.name.like(searchPattern) | songs.title.like(searchPattern));

    final rows = await query.get();

You're calling ..get(); on the query (note the two dots). Just using .get() should work. See the cascade operator for details.

@erwintan96 Did that solve your problem? I'll close this issue since it looks like there isn't anything substantial to add anymore. If you have more questions, please let me know. Thanks!


Improve C++ Examples

Description
I see that the C++ Examples need some rework. Running Current CPP_Package Lenet Example shows the accuracy is not improving. The network architecture needs to be revisited to improve the accuracy. Logging/Documentation/etc., needs to be improved as well for all the CPP Package Examples.
There are also several other issues with Lenet example(other examples might also be the same) :

Model not converging
Copies data to NDArrays on CPU instead of using Iterators, so this won't work if you don't have enough memory on your deskop
Logging could be improved
Code structure could be improved with comments, modularized.
Example could use some good documentation on how to run.

Accuracy over epochs on the Lenet Example is below
[08:03:11] lenet.cpp:191: Epoch [251] Time cost :2.82437, accuracy: 0.0917187
[08:03:14] lenet.cpp:191: Epoch [252] Time cost :2.80197, accuracy: 0.105
[08:03:17] lenet.cpp:191: Epoch [253] Time cost :2.84335, accuracy: 0.0992187
[08:03:20] lenet.cpp:191: Epoch [254] Time cost :2.84199, accuracy: 0.105
[08:03:23] lenet.cpp:191: Epoch [255] Time cost :2.82093, accuracy: 0.103594
[08:03:26] lenet.cpp:191: Epoch [256] Time cost :2.82659, accuracy: 0.105
[08:03:29] lenet.cpp:191: Epoch [257] Time cost :2.83494, accuracy: 0.0976562
[08:03:32] lenet.cpp:191: Epoch [258] Time cost :2.81344, accuracy: 0.105
[08:03:35] lenet.cpp:191: Epoch [259] Time cost :2.82905, accuracy: 0.0992187
[08:03:38] lenet.cpp:191: Epoch [260] Time cost :2.84072, accuracy: 0.105
[08:03:41] lenet.cpp:191: Epoch [261] Time cost :2.80219, accuracy: 0.111562
[08:03:44] lenet.cpp:191: Epoch [262] Time cost :2.81896, accuracy: 0.095625
[08:03:46] lenet.cpp:191: Epoch [263] Time cost :2.80515, accuracy: 0.0976562
[08:03:49] lenet.cpp:191: Epoch [264] Time cost :2.81536, accuracy: 0.105
[08:03:52] lenet.cpp:191: Epoch [265] Time cost :2.80752, accuracy: 0.100781
[08:03:55] lenet.cpp:191: Epoch [266] Time cost :2.85179, accuracy: 0.0976562
[08:03:58] lenet.cpp:191: Epoch [267] Time cost :2.82427, accuracy: 0.0917187
[08:04:01] lenet.cpp:191: Epoch [268] Time cost :2.85179, accuracy: 0.105
[08:04:04] lenet.cpp:191: Epoch [269] Time cost :2.82831, accuracy: 0.0992187
[08:04:07] lenet.cpp:191: Epoch [270] Time cost :2.84071, accuracy: 0.105
[08:04:10] lenet.cpp:191: Epoch [271] Time cost :2.82893, accuracy: 0.103594
[08:04:13] lenet.cpp:191: Epoch [272] Time cost :2.83461, accuracy: 0.105
[08:04:16] lenet.cpp:191: Epoch [273] Time cost :2.80872, accuracy: 0.0976562
[08:04:19] lenet.cpp:191: Epoch [274] Time cost :2.82435, accuracy: 0.105
[08:04:22] lenet.cpp:191: Epoch [275] Time cost :2.83497, accuracy: 0.0992187
[08:04:24] lenet.cpp:191: Epoch [276] Time cost :2.82968, accuracy: 0.105
[08:04:27] lenet.cpp:191: Epoch [277] Time cost :2.81235, accuracy: 0.111562
[08:04:30] lenet.cpp:191: Epoch [278] Time cost :2.8144, accuracy: 0.095625
[08:04:33] lenet.cpp:191: Epoch [279] Time cost :2.81339, accuracy: 0.0976562
[08:04:36] lenet.cpp:191: Epoch [280] Time cost :2.79739, accuracy: 0.105
[08:04:39] lenet.cpp:191: Epoch [281] Time cost :2.8489, accuracy: 0.100781
[08:04:42] lenet.cpp:191: Epoch [282] Time cost :2.83861, accuracy: 0.0976562
[08:04:45] lenet.cpp:191: Epoch [283] Time cost :2.8226, accuracy: 0.0917187
[08:04:48] lenet.cpp:191: Epoch [284] Time cost :2.8185, accuracy: 0.105
[08:04:51] lenet.cpp:191: Epoch [285] Time cost :2.81521, accuracy: 0.0992187
[08:04:54] lenet.cpp:191: Epoch [286] Time cost :2.82646, accuracy: 0.105
[08:04:57] lenet.cpp:191: Epoch [287] Time cost :2.83055, accuracy: 0.103594
[08:04:59] lenet.cpp:191: Epoch [288] Time cost :2.82807, accuracy: 0.105
[08:05:02] lenet.cpp:191: Epoch [289] Time cost :2.83138, accuracy: 0.0976562
[08:05:05] lenet.cpp:191: Epoch [290] Time cost :2.82849, accuracy: 0.105
[08:05:08] lenet.cpp:191: Epoch [291] Time cost :2.84255, accuracy: 0.0992187
[08:05:11] lenet.cpp:191: Epoch [292] Time cost :2.84767, accuracy: 0.105
[08:05:14] lenet.cpp:191: Epoch [293] Time cost :2.82494, accuracy: 0.111562
[08:05:17] lenet.cpp:191: Epoch [294] Time cost :2.8167, accuracy: 0.095625
[08:05:20] lenet.cpp:191: Epoch [295] Time cost :2.81547, accuracy: 0.0976562
[08:05:23] lenet.cpp:191: Epoch [296] Time cost :2.83213, accuracy: 0.105
[08:05:26] lenet.cpp:191: Epoch [297] Time cost :2.83226, accuracy: 0.100781
[08:05:29] lenet.cpp:191: Epoch [298] Time cost :2.81854, accuracy: 0.0976562
[08:05:32] lenet.cpp:191: Epoch [299] Time cost :2.83164, accuracy: 0.0917187
[08:05:35] lenet.cpp:191: Epoch [300] Time cost :2.83807, accuracy: 0.105
[08:05:37] lenet.cpp:191: Epoch [301] Time cost :2.82923, accuracy: 0.0992187
[08:05:40] lenet.cpp:191: Epoch [302] Time cost :2.83439, accuracy: 0.105
[08:05:43] lenet.cpp:191: Epoch [303] Time cost :2.8648, accuracy: 0.103594
[08:05:46] lenet.cpp:191: Epoch [304] Time cost :2.81611, accuracy: 0.105
[08:05:49] lenet.cpp:191: Epoch [305] Time cost :2.83235, accuracy: 0.0976562
[08:05:52] lenet.cpp:191: Epoch [306] Time cost :2.82705, accuracy: 0.105
[08:05:55] lenet.cpp:191: Epoch [307] Time cost :2.8145, accuracy: 0.0992187
[08:05:58] lenet.cpp:191: Epoch [308] Time cost :2.8112, accuracy: 0.105
[08:06:01] lenet.cpp:191: Epoch [309] Time cost :2.82919, accuracy: 0.111562
[08:06:04] lenet.cpp:191: Epoch [310] Time cost :2.80883, accuracy: 0.095625
[08:06:07] lenet.cpp:191: Epoch [311] Time cost :2.81497, accuracy: 0.0976562
[08:06:10] lenet.cpp:191: Epoch [312] Time cost :2.83363, accuracy: 0.105
[08:06:13] lenet.cpp:191: Epoch [313] Time cost :2.80442, accuracy: 0.100781
[08:06:15] lenet.cpp:191: Epoch [314] Time cost :2.82276, accuracy: 0.0976562
[08:06:18] lenet.cpp:191: Epoch [315] Time cost :2.81923, accuracy: 0.0917187
[08:06:21] lenet.cpp:191: Epoch [316] Time cost :2.86166, accuracy: 0.105
[08:06:24] lenet.cpp:191: Epoch [317] Time cost :2.80451, accuracy: 0.0992187
[08:06:27] lenet.cpp:191: Epoch [318] Time cost :2.82232, accuracy: 0.105
[08:06:30] lenet.cpp:191: Epoch [319] Time cost :2.83088, accuracy: 0.103594

@aaronmarkham - additional input to improve the C++ examples.


Hygieia cloud dashboard

Do hygieia have support to create cloud view dashboard on GCP. Any documentation or roadmap available to support GCP. (Simmillar to AWS collector)

@ajayakumar-jayaraj we do not have the support as of now, However if you have time you can create a collector for populating Data from GCP. The instructions to create a collector can be found at https://hygieia.github.io/Hygieia/create_collector.html


Categorize all Software OTA sensors as diagnostic

The update entity and the Software OTA sensor entities that are disabled by default are categorized as diagnostic, whereas the Software OTA sensor entities that are enabled by default are not.  For consistency, and in the interest of continuing to reduce the number and/or prominence of sensors that are mostly used for testing purposes, I am inclined to categorize the rest of the Software OTA sensor entities as diagnostic.
(Personally, I would disable all of them by default for new installations, and ultimately eliminate them in favor of the new update entity, but I recognize their value for testing purposes, and I'm sure there are folks who still use portions of the (excellent) @tmack8001 dashboards, which rely on the individual Software OTA sensor entities, so I will defer to others on whether and when to disable or deprecate them.)
That said, if the Software OTA - Download Progress entity is going to be enabled by default, I would also enable the Software OTA - Install Progress by default.  That change is also included in this PR.

I was just going to mention this morning that the install progress entity is disabled (rebuilding my dashboards back up).
Prior to having the software entity handle this type of data these individual sensors had value, now most have fallen into that single entity and I'm ok with that. Progress percentages are nice to see on a gauge the progress of a download and installation, not clear to me if that is supported in the software entity or not (maybe, I just haven't experienced an OTA since that addition).
I'm ok with marking them as diagnostics given they have little to no value to the owner's view of the vehicle, only to those looking under the covers for OTA availability information.


Fix the leadership integration test with Java 11

Description
The leadership integration test starts failing with Java 11 such as in here. It is because of the custom node failed with OOM.
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /tmp/java_pid906.hprof ...
Heap dump file created [46020040 bytes in 0.143 secs]
Terminating due to java.lang.OutOfMemoryError: Java heap space

This PR increases the heap from 32m to 64m.

This PR has:

[x] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

As this issue is another blocker for 0.21.0, I couldn't wait for Travis to run, so went ahead and ran the same jobs on the internal CI of Imply. All tests passed except for (Compile=openjdk8, Run=openjdk11) input source integration test, which seems like a temporary connection issue. This test uses https://druid.apache.org/data/wikipedia.json.gz. The detailed error is in below. The same test passed on my local machine.
java.lang.RuntimeException: java.net.ConnectException: Connection timed out (Connection timed out)
	at org.apache.druid.data.input.impl.InputEntityIteratingReader.lambda$read$0(InputEntityIteratingReader.java:81) ~[druid-core-0.22.0-iap1-SNAPSHOT.jar:0.22.0-iap1-SNAPSHOT]
	at org.apache.druid.java.util.common.parsers.CloseableIterator$2.findNextIteratorIfNecessary(CloseableIterator.java:84) ~[druid-core-0.22.0-iap1-SNAPSHOT.jar:0.22.0-iap1-SNAPSHOT]
	at org.apache.druid.java.util.common.parsers.CloseableIterator$2.<init>(CloseableIterator.java:69) ~[druid-core-0.22.0-iap1-SNAPSHOT.jar:0.22.0-iap1-SNAPSHOT]
	at org.apache.druid.java.util.common.parsers.CloseableIterator.flatMap(CloseableIterator.java:67) ~[druid-core-0.22.0-iap1-SNAPSHOT.jar:0.22.0-iap1-SNAPSHOT]
	at org.apache.druid.data.input.impl.InputEntityIteratingReader.createIterator(InputEntityIteratingReader.java:103) ~[druid-core-0.22.0-iap1-SNAPSHOT.jar:0.22.0-iap1-SNAPSHOT]
...```

+1 on not waiting for travis on this change

@jon-wei @clintropolis @suneet-s @maytasm thank you all!


Created cpp-stl-cheetsheet.md

What does this PR do?

Add content(s) : cpp-stl-cheetsheet.md

CPP STL Cheetsheet

Complete C++ STL which include vector, set, unordered set, map, stack, queue, deque, and vector pair.

Screenshots or Links

Link : https://github.com/khushpreet-007/CheatSheets-for-Developers/blob/main/CheatSheets/cpp-stl-cheetsheet.md

Additional context

Created cpp-stl-cheetsheet.md

Checklist:

[] Read our contributing guidelines.
[ ] Search for duplicates.
[ ] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

Contributed cpp-stl-cheetsheet.md. Add this to your repository


e2e testing for Minikube

What this PR does / why we need it?:

Support e2e testing for Minikube

Does this PR introduce a user-facing change?

[ ] User guide
[ ] CHANGELOG.md

Which issue(s) this PR fixes:

Fixes #235

New PR open - updated version of PR: https://github.com/application-stacks/runtime-component-operator/pull/273
All tests, except for stream is working. Support for knative and affinity tests enabled!
Kubernetes version 1.19 to match with OCP 4.6.


#1145 - Task Template - Downloaded Workplan only shows milestones

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

:exclamation: No coverage uploaded for pull request base (develop@16ed717). Click here to learn what that means.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #1657   +/-   ##
==========================================
  Coverage           ?   61.93%
==========================================
  Files              ?      248
  Lines              ?     8201
  Branches           ?        0
==========================================
  Hits               ?     5079
  Misses             ?     3122
  Partials           ?        0

Flag
Coverage Δ

epictrack-api
61.93% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


refactor(formsection): use interface instead of type

:tada: This PR is included in version 4.11.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Switch from http-parser to llhttp

Apparently, http_parser is abandonware and node.js started work on a switching to llhttp.
cc @alyssawilk @htuch @mattklein123

One concern around llhttp is it uses https://github.com/indutny/llparse and a bunch of TypeScript. I don't want to go down the language war path, but it's probably worth considering the increased build complexity (Bazelifying llparser and dependencies) and developer impedance matching when making the call. All things being equal, I'm a fan of higher level solution to parsing, but switching to this is not totally obvious.

If I could be of any help, please do not hesitate to ask me directly.

@indutny would you mind filing a tracking bug for the migration over on the node.js so interested envoy developers can follow it?  If we opt to migrate we'd probably swap shortly after you, and if we don't it's still important to know when node.js stops using http_parser from a security perspective, so it's good to know either way :-)

My opinion on this is that we should take a wait and see approach. TBH, HTTP/1.1 at this layer is basically done. I don't really see what changes we would need to make.

@alyssawilk https://github.com/nodejs/node/issues/24730 - This might do.

Reopening in light of https://github.com/envoyproxy/envoy/issues/6434. Since we didn't get any response from Node.JS security WG when reporting, it seems that http-parser should be considered unsupported. The community did drive a fix when we went public, see https://github.com/nodejs/http-parser/issues/468.
Since http-parser is a codec and hence exposed to untrusted traffic from clients and backends on the data plane, it has a higher standard to live up to than other external dependencies. I would like us to consider only including dependencies that have active security teams that are at least doing CVE management. This is further motivation for moving away from http-parser.

@htuch I'm sorry about the negative experience that you had. I wish we would have seen your security report on our channels before you submitted that github issue. What channel did you use? Was it HackerOne or the mailing list (security@nodejs.org)? I'm having a hard time finding your report...
I am part of Node.js community and one of the maintainers of http-parser. I believe that in our shared effort we managed to fix the issue very promptly after it was disclosed. Sorry again if this caused you any frustration!

As a matter of fact, I'm a maintainer of llhttp too :joy:

@indutny we send two e-mails, one pre and one post disclosure to<EMAIL_ADDRESS>Happy to forward them again. Is HackeOne preferred?
Thanks again for the prompt fix once this went public, the comment above reflects a concern that we didn't have any effective method to coordinate prior to disclosure.

Oh no<EMAIL_ADDRESS>should be forwarding reports to HackerOne. Apparently, something is not working as expected. Again, I'm terribly sorry for this. Will look into it momentarily.

@indutny Will https://hackerone.com/nodejs/hacktivity and https://groups.google.com/group/nodejs-sec be updated retroactively with the CVE?
@htuch it seems like the reporting system would have been or is broken for both http-parser and lhttp (independent of their health it status as sub-projects).

@moderation that's a good question. We can definitely request a CVE for the issue. Let me see what I can do.

cc @rvagg @jasnell, perhaps?

@indutny we sent two e-mails, one pre and one post disclosure to<EMAIL_ADDRESS>Happy to forward them again. Is HackerOne preferred? The title was "Vulnerability in http-parser & embedded NULL header handling".

Hi, I work for @Hacker0x01 and act as our main liaison to the open source community. Hoping I can assist you.
When you e-mailed<EMAIL_ADDRESS>you should have received an e-mail back from HackerOne stating "ACTION REQUIRED: Submit Vulnerability Report". Can you check to see if you received that? If so, you'll need to click that link in order to finalize your report submission.
In any case, you can always submit directly to https://hackerone.com/nodejs.

@reedloden as discussed in https://github.com/nodejs/security-wg/issues/454, this was not received. I'm thinking that this is a process flaw if folks can opt to use e-mail or HackerOne and the e-mail path is not as reliable or requires steps that might result in a dropped report. Would it be better to just drop the e-mail option entirely and have folks use HackerOne directly to avoid potential lost reports?

I haven't been as active in the security triage recently but I don't recall this coming through. My apologies for having missed it.

@htuch I figured out what happened, but until it's fixed (either on our side or on Node.js's), I would recommend that https://hackerone.com/nodejs be utilized. Even if e-mail was working, the web interface is preferred, as e-mail is not secure.

@reedloden thanks for the analysis of the issue. @indutny are you going to followup with the NodeJS security WG on (1) CVE-2019-9900 and (2) how to deal with the e-mail redirect issues? I can do (1) if that helps by filing a HackerOne report.

Is picohttpparser still in the running as a replacement as well? It was mentioned in #2880 & #5536 as a potential perf enhancement for http1.1 .

May be time to revisit this. llhttp now the default in Node 12 and the team are looking to deprecate http-parser https://twitter.com/addaleax/status/1173966000175366144

Yep, it would be great to see a comparison, including feature support, standard compliance, performance, licenses, security process/maturity and so on. http-parser is basically EOLed.

Happy to see an analysis done, but IMO we should probably just use llhttp since it's roughly a drop in API replacement for http_parser.
I've used picohttp in the past and we would have to implement a substantial amount of stuff on top that http_parser/llhttp already implements.

The only thing that slightly sketches me out about llhtttp is the way it pushes us towards having to live in Typescript land. I'm a fan of higher level languages for parsers and codecs, but not such a huge Javascript fan. We can just use the released C implementations though, so build integration shouldn't be an issue. FWIW, the concerns over appropriate handling of vulnerabilities in the Node.js community remains, it took a long time to resolve the http-parser issues I reported.

Started playing around with llhttp, the parser API is pretty much the exact same which is great, but it doesn't expose the URL parsing library like http_parser which we use in a few places as a URL utility.

@derekargueta can we move away from relying on these URL parsing utils and switch to the Chromium URL library as per https://github.com/envoyproxy/envoy/issues/6588? This would allow us to get behind a single source of URL parsing truth, which can only be a good thing for consistency; we've seen quite a few issues with different parts of Envoy not agreeing on things like URL semantics historically.

@htuch sure, I was unaware of sunsetting the Node URL parser.
Also, should the new parser should initially be gated behind a feature flag to allow opt-in before forcing everywhere, similar to when the buffer implementation changed?

@derekargueta I'd recommend a feature flag to allow folks to fall back as needed, but I'd advocate we flip it on eagerly, there isn't a ton of value in remaining with the extant URL parser.

So far I have 73% of http1 codec tests passing, working through them. llhttp introduces slight deviations - stricter header parsing (i.e. this block is no longer required), slightly different handling of upgrade, etc.
Also, I'm going to do the URL utility library as a separate change if no one beats me to it. My updated branch leaves that as is for now.

@derekargueta please let me know if I can be of any help in this endeavor.

FWIW I highly suggest moving from http_parser to llhttp. There was even discussion about changing the defaults in 10.x during LTS due to lack of ability to continue to support the deprecated parser

I will update the title of this issue. We are switching. @derekargueta is working on it.

This change is about ready, however there's one missing feature in llhttp which is the lenient_http_headers option (used here in http-parser). This option toggles strict validation of HTTP header values by ensuring they are printable ASCII characters. We currently run http-parser in lenient mode (which is the default) then provide a runtime switch to enable strict header validation. However, llhttp defaults to strict validation and doesn't provide an option to disable it.
To move forward, we either need to

get a lenient option added in llhttp (there's a TODO for this cc @indutny) or
deprecate envoy.reloadable_features.strict_header_validation and have all traffic use strict header value checking. Based on some conversation on the PR that added strict_header_validation, it seems that we eventually wanted to move this way? If so, I think this option would be better.

https://github.com/envoyproxy/envoy/pull/8849 flips strict validation on by default and the option will be removed entirely (unless folks yell) at the next Envoy release.  I assume we're going to support both parsers in parallel for a bit so I think going 100% strict is probably fine (we can revisit if changing the default causes problems in upcoming weeks)

Awesome stuff. @derekargueta is there any way to support both parsers without a recompile? IMO this should be feature flagged and flippable at startup like we did with the buffer implementation?

@derekargueta any update on status here?

@htuch I've done my part to encourage @derekargueta :wink: https://twitter.com/moderat10n/status/1230125590428053509

No status update at the moment - I have started catching my stale branches up to master. There's been some refactorings of the HTTP codecs that I need to work through

Started re-investigating this. @indutny one issue that came up is that the llhttp release artifact is that it's not built with strict mode enabled, and this option cannot be enabled from the C API since it is a variable in producing the C code. Are there plans to publish release builds with strict mode enabled, or be able to set strict mode through the C-API perhaps using -D like http-parser? This is causing quite a few tests to fail but I really don't want to introduce Typescript tooling in the Envoy build (for running npx to generate the C code with strict mode)

There is a bit of optimizing that has to be done on llhttp's code before compiling it down to C. I don't think that it'd be possible to introduce the configuration defines the same way as they are in http-parser. Runtime checks would likely affect performance so I'd avoid this too.
There are two things that could be done, however:

Bundling two separate builds and switching between them in a build system (Easy, might be aesthetically undesirable)
Combining two generated .c files from strict and loose mode builds into one guarded by a huge top-level #ifdef

I'll look into the latter to see how much churn this would involve in llhttp. Could you consider the former meanwhile?

See: https://github.com/nodejs/llhttp/pull/58

FWIW, that Pull Request has landed and was released in llhttp 2.1.0.

@derekargueta any update on progress here?

Now @derekargueta 's presentation at KubeConEU is done he should have some cycles :smile:

This commit on 2020-10-02 confirms that the last remaining maintainer for http-parser has moved on and http-parser is no longer maintained - https://github.com/nodejs/http-parser/commit/ec8b5ee63f0e51191ea43bb0c6eac7bfbff3141d.
From an @envoyproxy/dependency-shepherds POV it would be great if we could get https://github.com/envoyproxy/envoy/pull/9033 over the line /cc @derekargueta @htuch

I've tagged this as both priority and security in response to the above comment.

I have a "drop-in replacement" branch with everything passing except the new functionality that was introduced via http-parser for allowing transfer-encoding: chunked + content-length: N. There's an llhttp issue for this where I've started a small investigation and the fix seems pretty straightforward: https://github.com/nodejs/llhttp/issues/69

FYI, nodejs/llhttp#69 was just closed with a fix in v3.0.0

I added a link to the release at the now stale WIP PR at https://github.com/envoyproxy/envoy/pull/9033#discussion_r532955677

Confirmed all tests pass with the new llhttp update 🎉 reviving the abstractions I initially had for supporting both parsers through a command line flag

Woohoo

I ran some perf tests and got CPU profiles from Derek's branches (direct swap in with llhttp, and an added parser factory and parser class) here: https://gist.github.com/asraa/8babd3516acd447aa0f2ff5aa61d2e19
It doesn't seem to me that there's significant overhead in adding the factory, and llhttp swap in results in some CPU time differences (IMO insignificant).
If there's anything I can do to make some of these benchmarks more informative please let me know!

I ran some perf tests and got CPU profiles from Derek's branches (direct swap in with llhttp, and an added parser factory and parser class) here: https://gist.github.com/asraa/8babd3516acd447aa0f2ff5aa61d2e19
It doesn't seem to me that there's significant overhead in adding the factory, and llhttp swap in results in some CPU time differences (IMO insignificant).
If there's anything I can do to make some of these benchmarks more informative please let me know!

@asraa looks compelling, i.e. there is little difference. I think the 1M QPS example probably has a ton of error rate reflecting open loop request drops; it might be a good idea when measuring max QPS achievable to switch NH to closed loop mode (at which point latency is not very interesting). I'd also be curious to see if there is any difference for large request, e.g. 1MB body.

@asraa looks compelling, i.e. there is little difference. I think the 1M QPS example probably has a ton of error rate reflecting open loop request drops; it might be a good idea when measuring max QPS achievable to switch NH to closed loop mode (at which point latency is not very interesting). I'd also be curious to see if there is any difference for large request, e.g. 1MB body.

Thanks! More benchmarks added.
Those benchmarks were actually closed loop, that appears the default in the nighthawk test client. I ran with --open-loop again though, and added those benchmarks. Again, didn't see a major difference in open loop latency.
You do see a larger error rate (OR llhttp improvement) in the open loop 500k QPS (1M was enough to break nighthawks stats gathering).

Thanks! More benchmarks added.
Those benchmarks were actually closed loop, that appears the default in the nighthawk test client. I ran with --open-loop again though, and added those benchmarks. Again, didn't see a major difference in open loop latency.
You do see a larger error rate (OR llhttp improvement) in the open loop 500k QPS (1M was enough to break nighthawks stats gathering).

BTW -- added an adaptive load test (with nighthawk) to find optimal RPS (based on mean latency <= 5ms and 95%+ success rate)
@derekargueta

-
BASELINE http_parser
llhttp
Factory with http_parser
Factory with llhttp

Final RPS
20125
20156
19531
19843

the llhttp boost somewhat counteracts the factory, there's a 1% degradation.

BTW -- added an adaptive load test (with nighthawk) to find optimal RPS (based on mean latency <= 5ms and 95%+ success rate)
@derekargueta

-
BASELINE http_parser
llhttp
Factory with http_parser
Factory with llhttp

Final RPS
20125
20156
19531
19843

the llhttp boost somewhat counteracts the factory, there's a 1% degradation.

llhttp 4.0.0 is out - https://github.com/nodejs/llhttp/releases/tag/release%2Fv4.0.0

@asraa Do you know offhand whether or not this new parser implementation will allow Envoy to support custom methods for http 1.1? I'm having to bypass the http layer of envoy in my cluster for a service whose protocol (over which I have no control) uses non-standard http methods. That is quite unfortunate.
I took a quick gander at the llhttp code, and it looked like it may have a similar restriction to the old parser (i.e. a see explicit lists of http methods e.g. https://github.com/nodejs/llhttp/blob/master/src/llhttp/c-headers.ts#L46), which worries me. I could be wrong, though.

@klarose Last I checked llhttp hardcodes the same whitelisted HTTP methods.

@klarose Last I checked llhttp hardcodes the same whitelisted HTTP methods.

Thanks for looking in to it. That's unfortunate. Do you know if there is any way to change that behaviour? It's quite awful for us. We don't have control over the protocol involved here. Having to treat the request as TCP because the envoy http parser cannot handle unknown methods really removes a bunch of its benefits -- we lose the logging, filters, etc that all come with http. All of those things support arbitrary methods. Why not the parser?
There is some discussion here about the meaning of the methods, as well as a registry: https://datatracker.ietf.org/doc/html/rfc7231#section-4. It does say that a method which cannot be handled by a server should be rejected. However, envoy is not an http server; it is a proxy.
It seems to me that the server to which envoy is proxying should utltimately decide whether a method is unsupported.
Perhaps this isn't the proper forum for discussing this, but I feel like it should be taken into consideration when decision to use this parser.

Google has a similar problem where we need to support methods which http_parser does not handle.   cc @yanavlasov maybe link your doc on this thread?

Closing in favor of https://github.com/envoyproxy/envoy/issues/21245


Character editor flicker fix

Fix: #7000
Updates SpriteView.Sprite instead of disposing and adding child on each rebuild. Although we should probably see into why it wasn't working to begin with.
There was also apparently a memory leak because _previewDummy = _entMan.SpawnEntity
At least it works now:

Nevermind _previewDummy is still broken. Gotta fix that too.


'az webapp config ssl list' returns an empty list

Describe the bug
az webapp config ssl list --resource-group <resource-group> returns [] as the response,
while Get-AzWebAppCertificate -ResourceGroupName <resource-group> returns the actual certificates.
It has previously been reported, and supposedly fixed, here:

#18216
#19777

(Although my problem doesn't seem to be a caching issue as described in the first issue because running the Get-AzWebAppCertificate first, then az command doesn't make it work)
Related command
az webapp config ssl list --resource-group <resource-group>
and
Get-AzWebAppCertificate -ResourceGroupName <resource-group>
Errors
No errors, just no results.
Issue script & Debug output
I've added --debug to the az command and -Debug to the Get-AzWebAppCertificate command and noticed that Get-AzWebAppCertificate makes this request 👇 to Management API:
https://management.azure.com/subscriptions/<subscription>/resourceGroups/<resource-group>/resources?$filter=resourceType eq 'Microsoft.Web%2FCertificates'&api-version=2016-09-01

While the az command makes a different one:
https://management.azure.com/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Web/certificates?api-version=2023-01-01

So, maybe the problem is not necessarily the CLI, but that 👆 specific Management API version/request?
Expected behavior
I expect the certificates to be included in the az output.
Environment Summary
I'm running both on Cloud Shell.
> Get-Module -Name Az -ListAvailable

    Directory: /usr/local/share/powershell/Modules

ModuleType Version    PreRelease Name                                PSEdition ExportedCommands
---------- -------    ---------- ----                                --------- ----------------
Script     12.1.0                Az                                  Core,Desk

$ az --version
azure-cli                         2.62.0

core                              2.62.0
telemetry                          1.1.0

Extensions:
ai-examples                        0.2.5
front-door                         1.2.0
ml                                2.27.0
ssh                                2.0.4

Dependencies:
msal                              1.28.1
azure-mgmt-resource               23.1.1

Python location '/usr/bin/python3.9'
Extensions directory '/home/<name>/.azure/cliextensions'
Extensions system directory '/usr/lib/python3.9/site-packages/azure-cli-extensions'

Python (Linux) 3.9.19 (main, Mar 28 2024, 18:56:59)
[GCC 11.2.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context
No response

Thank you for opening this issue, we will look into it.

Also, tried on a different machine (Ubuntu 22.04), with a different version of az, still getting an empty list []; --debug says it makes this call:
https://management.azure.com/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Web/certificates?api-version=2022-03-01

$ az --version
azure-cli                         2.42.0 *

core                              2.42.0 *
telemetry                          1.0.8 *

Extensions:
azure-devops                      0.26.0
containerapp                      0.3.17
front-door                        1.0.17
portal                             0.1.3

Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1

Python location '/opt/az/bin/python3'
Extensions directory '/home/<name>/.azure/cliextensions'

Python (Linux) 3.10.8 (main, Oct 28 2022, 04:19:21) [GCC 11.3.0]

Legal docs and information: aka.ms/AzureCliLegal

You have 3 update(s) available. Consider updating your CLI installation with 'az upgrade'

Hey @yonzhan and @seligj95, any updates on this?
Just a bit of context: We have not updated Azure CLI since it last worked; still using the same version 2.42.0 that used to work. However, the last time it ran successfully we were using Ubuntu-20.04-based self-hosted agents version 2.211.1 on Azure DevOps.
We have since upgraded our agents to Ubuntu-22.04-based version 3.227.2 and run it on the new agent, as well as run it on native Ubuntu 22.04 (local machine), both of which fail.
Other than the OS upgrade, which is unlikely to be the cause, It just randomly stopped working and now our production pipelines are broken due to this issue.
Would appreciate if you could prioritise the issues when Microsoft is the cause.
Looking forward to an update 🙏

I got same issue.
Here is my Azure CLI version on Ubuntu 22.04.4 LTS (WSL2).
$ az --version
azure-cli                         2.61.0 *

core                              2.61.0 *
telemetry                          1.1.0

Extensions:
containerapp                      0.3.28

Dependencies:
msal                              1.28.0
azure-mgmt-resource               23.1.1

Python location '/opt/az/bin/python3'
Extensions directory '/home/<name>/.azure/cliextensions'

Python (Linux) 3.11.8 (main, May 16 2024, 03:47:28) [GCC 11.4.0]

Legal docs and information: aka.ms/AzureCliLegal

You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade'

Since today I also get this issue on an Azure subscription. We did not have this issue in September.
This still works on other Azure subscriptions. I'm baffled.


Collect and export kubernetes events and docker events

InfluxDB supports events natively. Other backends might not. We can place structure around some specific events like container starts, deletions, restarts, etc and export them as metrics, but that might be sub-optimal.
Need to explore how events is currently handled by various monitoring/alerting systems.

@saad-ali is working on this.

#252


tools/villas: improve script behaviour for -h or --help

Quote ${SUBTOOL} in case it is set but empty
Add -e flag to grep so villas --help does not show grep --help
Add -F flag to grep so villas "[ " does not show a regexp error
Don't exit with error when -h or --help was specified
Add -h and --help to Usage in the help message.

Hi @PJungkamp,
thanks a lot :) Seems like you are late up like me coding :D


Combination Sum 2 added Leetcode

Cpp program with only unique sequences allowed

Great work @Atharva-77  please give this repo a star if it really helped you to complete your PR.
Also follow me for follow back if you want to get connected with me for any such open source contribution program. Thank you 😊


Improve the Test

This PR is designed to address the following issues:

Add lint to the project
Make the naming more go idiomatic eg FastOtp to FastOTP as OTP is an abbrevation.
Increase the test coverage
Separate the test process from the build process
Make some strings constant, eg The Type and the Status to prevent users from passing wrong values and protect against typographical errors.
Add comment on all exported functions.
Configure/Tune the http client that is being used.
Create a package wide client, instead of creating new http client per request.

I see the good work you're doing.. but
Why did you decide to mock http request instead of testing against an actual server?  @LordRahl90

I see the good work you're doing.. but Why did you decide to mock http request instead of testing against an actual server? @LordRahl90

I'm also using a mocked server actually.
However, to see how the library reacts to some errors, that's why I'm mocking the httpclient as well. 😄

@LordRahl90 epp me fixx ur merge conflicts

@LordRahl90 epp me fixx ur merge conflicts

Done with some minor additions. Eg:

Unexporting the APIKey, so it can't be modified once it's set.
Making the context on a functional level as the client will be set in a global context.


[Question]Running multiple simulators

@lawrencelomax  I see that one factor that limits number of simulators one can have on machine is CPU used by Native App under test itself. And, I see that same app on simulator takes more CPU than its on actual real device. I see this behavior same for few apps that I tried. Have you guys run into this issue? How many simulators you guys have on a host? what's your host configuration?

One of the biggest limiting factors worth considering is that by performing identical work on multiple simulators, on the same host, at the same time will lead to a very uneven loading of the machine. Booting and launching an App are CPU and IO intensive, performing this work at the same time across multiple simulators will severely limit the amount of Simulators that you can run in parallel.
If you employ a strategy that means that this work is more evenly distributed over time, then you will likely see that you don't have such a spiky load profile. The net result will be that you can get much more out of your machines. Another thing worth bearing in mind is that you can have Simulators 'pre-booted' in anticipation of a test running. This means that the tests don't have to pay the cost of booting the Simulator itself.
There are certain sections of the SeleniumConf video that may be helpful to you.

@lawrencelomax My apology for late response. I totally missed this. We have all our sessions always pre-warmed i.e. we have simulators launched with app install. We do launch app per test and we do not keep app launched. We have 3 simulators running on macmini(i5) and what I have seen is if 3 simulators are happened to be doing scroll or typing actions' (both are CPU intensive jobs) at same time on 2 different simulators then tests start failing. How do you solve this issue or you guys don't face such issue?

3 Simulators should just work fine. It's important to have either a Monitor or some HDMI dongle connected to the Mac Mini to activate its GPU. If you don't have one connected, you could give it a try, connect one, and then see if that changes something. For us this makes a huge difference.

Thanks @plu . I will give that try. We don't have that Monitor or HDMI dongle connected. I will give HDMI dongle a try and see if it helps. I remote into one of such machine and  tried to look into system setting to check whether it is currently using GPU but could not find any setting indicating that. How did you confirm this helps? by measuring test performance?

Yes, measuring test performance. If the HDMI dongle gets removed, our test suite runs much longer and tests start to fail randomly (timeouts). We even have a check in our CI scripts that verify the HDMI dongle is connected, before any tests are executed.
# Verify the HDMI dongle is connected. This activates the GPU and is important for Simulator performance.
system_profiler SPDisplaysDataType | grep -q 'fit Headless' || (echo "Are you sure the fit Headless HDMI dongle is connected?" && exit 1)

Thanks a lot @plu .  I did see 5-7 minutes improvement, not whole lot. Failures due to CPU issues were intermittent so I have to run it for some more time to conclude. But I do see one amazing thing i.e. I was able to run tests across 5 simulators which was just not possible. It did slow down test run time a little but I have not see failures yet. I am going to experiment more around this. Thanks for all help.

Glad to hear it helped!

How many simulators do you guys run in parallel ? What is your machine spec for that? Do you guys use XCUITEST based framework or KIF/EarlGrey?

You should be able to get substantially more than 3 simulators running in parallel on a recent Mac mini. Bear in mind that having one with 8Gb of memory or an SSD will make a big difference to the boot speeds, much of this is IO bound.
Are you splaying out your boots so that they don't execute at the same time? You might see test execution suffer more if you boot all of them at the same time.

Thanks @lawrencelomax Yes we do have 8 GB and SSDs. Our system is such that there could never be simulator boot and test can happen at same time. So that is not case. What I have seen is AUT itself takes lots of CPU sometimes. What @plu  suggested (using display emulator) is helping a little.

Another suggestion is to avoid the need for rendering the display of a Simulator in the first place. If you look at the --direct-launch argument, you'll see that it's possible to boot a Simulator without needing to create an Application and Window for it. This may lower system utilization as you'll likely not need to see the Simulator screen when it's being automated. Instead you can use video recording/streaming/screenshots to view the screen.

Thanks @lawrencelomax . I am okay with not seeing Simulator. I have tried this but surprisingly I don't see difference. Theoretically there has to be some. But it is not visible. I am also passing hidport will that affect performance. I want to be able send command to hid if I am using --direct-launch.

You are only able to send commands to the hid using --direct-launch, you can't use this otherwise. I doubt that connecting the hid will effect performance, since this is what Simulator.app already does.

Thanks, that's what I thought. I am running 5 simulators now and it has been reliable so far. I have one more question : How do you guys run browser tests on multiple simulators on same host? We use magical port 27753 to communicate with webkit to execute test on safari on simulator. But if multiple simulators are launched there is only one simulator has that service running with magical port 27753 to execute test on webkit.

I'm not sure of the use-case of this magical port, but found this which may be what you are looking at.
With regards to additional services that run alongside multiple simulators, a simple way of doing this is to never rely on the default port for any additional services that you may run, and have each simulator own a shard of ports. For example each simulator will have a range of ports reserved:

PORT_OFFSET + 0 the port for --http in fbsimctl
PORT_OFFSET + 1 the port for WebDriverAgent
PORT_OFFSET + 2 the port for your arbitrary service.

This is where PORT_OFFSET = BASE_OFFSET + (SIMULATOR_INDEX * RANGE_SIZE).
If you're using some WebKit proxy service, this would be analogous to running WebDriverAgent on the same host for multiple Simulators. Just make a guarantee that the ports are free for each simulator and always manually specify and calculate the values of these ports. You may find some inspiration in one of our recent posts.

We just released idb. a new tool that improves significantly over fbsimctl. We recommend switching over. As part of our renewed commitment to this repo we are cleaning up old issues and will be monitoring and reviewing more closely from now on. if you still think this issue is relevant feel free to reopen


Implement Gigantamax properly

Gigantamax formes and moves should not be included in the data files (they are not in the game data files, they are not formes/moves by that definition). PS has begun doing some work here, the most obvious being marking the 'formes'/'moves' as Nonstandard and adding Gigantamax: Yes to the importable. The following is all required:

[ ] support Gigantamax: Yes in imports (moveset_import.js)
[ ] remove nonstandard Gigantamax from being special cased for inclusion

[ ] similarly, clean up otherFormes hardcodes (in particular, Toxitricity/Urshifu) to remove the Gmax 'forme'

[ ] Gigantamax should be a checkbox in the UI instead of a 'forme'.
[ ] Set import logic should switch to sets with gigantmax: true instead of the legacy -Gmax name approach

The main question is around moves - how should the Gmax version of a move be handled? This should probably block on PS implementing the Gmax data as a field on the base move.


WIP: Bug/validate array of obj

This PR deals with #710 where if a form has the datatype "array of objects" it would never validate it. It also deals with the string mutation of an array of objects, causing any entered array of objects to be mutated to [object Object].
It also deals with issues from #711 where the spec may not define what kind of items an array may not have.

@LindsayHill Is the circleci issue on my end?
#!/bin/bash -eo pipefail
PKG_VERSION=$(node -e "console.log(require('./package.json').st2_version);")
PKG_RELEASE=$(packagecloud.sh next-revision trusty ${PKG_VERSION} st2web)
echo "export PKG_VERSION=${PKG_VERSION}" >> $BASH_ENV
echo "export PKG_RELEASE=${PKG_RELEASE}" >> $BASH_ENV
/home/circleci/scripts/packagecloud.sh: line 23: PACKAGECLOUD_TOKEN:  PACKAGECLOUD_TOKEN env is required
Exited with code 1

Nah, that’s not on your end. Will have to see if there’s some way of handling that situation. It’s related to “who can see these private tokens - should PRs from forks be able to use them?”

@LindsayHill Has there been any progress on the Circle CI?

@jdmeyer3 sorry, haven't had a chance to look at it. You should be able to run the unit tests locally, following roughly the steps up to here https://github.com/StackStorm/st2web/blob/master/.circleci/config.yml#L59
Are those passing?


Single function API for binning

Functionality to call the binning function directly in one function
call with passing it a list of axes, ranges and nbins
implement from MPES

More or less also address in PR #69 (?)


Improved build caching for s2i local build

Changes

:gift: Improve caching for local s2i builds, by employing RUN --mount=type=cache....

/kind enhancement
improved caching for local s2i builds

PTAL @lkingland @gauron99 @rhuss

Nice finde about RUN --mount ... (didn't know about that). Does it also work with Podman ?

Nice finde about RUN --mount ... (didn't know about that). Does it also work with Podman ?

@rhuss
I works at least in podman +v4.0, it might not work on older versions.
It works on Docker only if BuildKit is enabled.

Looks good to me, however I wonder whether we should hardcode the cache directory to /tmp/artefacts.

From the outside there is not indication that it belongs to func.
It's shared across multiple function projects, which might or might not be desired
Different OSs have different standard location for caches, like on MacOs its ~/Library/Caches. We should respect this.
It should be configurable
/tmp is often a memory filesystem, so it might be too small for caching large dependencies.

The  /tmp/artefacts dir is s2i specific, I did not made that up.

To be clear /tmp/artefacts is path in container not in host.

PTAL @gauron99 @lkingland @rhuss

I thought one of bots was doing tests and created func binaries?
I see it only runs tests and only keep test results? https://github.com/knative/func/actions/runs/11955247512/job/33327188182?pr=2581 ?

@aslom  func.gpg.gz

This PR should at least fix java maven dep caching @aslom . It probably want change much for Go. I just wonder whether this would work on your setup.

@aslom There are also some nightly builds in knative infra, but again there are on main not on every PR.

/lgtm


BGP CP: Add Support for Multi-Pool IPAM

Currently, BGP CP only supports kubernetes IPAM. Add support for running BGP CP with Multi-Pool IPAM. The following is an agent (1.14.0-rc.1) log message when trying to run BGP CP with Multi-Pool IPAM:
level=error msg="Invoke failed" ="cmd.glob..func4 (daemon_main.go:1592)" error="could not build arguments for function \"github.com/cilium/cilium/daemon/cmd\".glob..func4 (/go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:1592): failed to build promise.Promise[*github.com/cilium/cilium/daemon/cmd.Daemon]: could not build arguments for function \"github.com/cilium/cilium/daemon/cmd\".newDaemonPromise (/go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:1628): failed to build *agent.Controller: could not build arguments for function \"github.com/cilium/cilium/pkg/bgpv1/agent\".NewController (/go/src/github.com/cilium/cilium/pkg/bgpv1/agent/controller.go:170): failed to build agent.nodeSpecer: received non-nil error from function \"github.com/cilium/cilium/pkg/bgpv1/agent\".NewNodeSpecer (/go/src/github.com/cilium/cilium/pkg/bgpv1/agent/nodespecer.go:42): BGP Control Plane doesn't support current IPAM-mode: 'multi-pool'" subsys=hive

xref: https://github.com/cilium/cilium/pull/25767

The only cluster-pool-v2beta feature that is lacking in multi-pool IPAM mode is BGP support. We intend to add BGP support to multi-pool IPAM in Cilium v1.15.

In our rough idea, the change should be something look like this.
virtualRouters:
- localASN: 65000
  podCIDRPoolSelector:
    foo: bar

Then it selectively advertises the route from the specified pool. The CiliumNode should contain the information that which prefix is coming from which pool.

/assign


git big add fails when the containing directory is untracked

$ git big add new/path/train2017.zip
new/path/train2017.zip: 100% |#####################################################################| Time: 0:00:58  18.0 GiB
Traceback (most recent call last):
  File "/usr/local/bin/git-big", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/git_big/main.py", line 818, in cmd_add
    App().cmd_add(paths)
  File "/usr/local/lib/python2.7/dist-packages/git_big/main.py", line 442, in cmd_add
    self._add_file(path)
  File "/usr/local/lib/python2.7/dist-packages/git_big/main.py", line 666, in _add_file
    self.repo.index.add([entry.working_path])
  File "/usr/local/lib/python2.7/dist-packages/git_big/main.py", line 47, in add
    git('add', path)
  File "/usr/local/lib/python2.7/dist-packages/git_big/main.py", line 41, in git
    return subprocess.check_output(['git'] + map(str, args), stderr=DEV_NULL)
  File "/usr/lib/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['git', 'add', '/home/user/repo/new/path/train2017.zip']' returned non-zero exit status 1

Here's the underlying error returned from git add: ```
The following paths are ignored by one of your .gitignore files:
new/path/train2017.zip
Use -f if you really want to add them.


[BugFix] Fix CpuUsageRecorderTest failure

Fixes #25263
CpuUsageRecorder cannot sample cpu time and realtime in exactly the same time, so it's possible the cpu_used_permille be greater than 1000, remove the related assert
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.1
[ ] 3.0
[ ] 2.5
[ ] 2.4

https://github.com/Mergifyio backport branch-3.0

https://github.com/Mergifyio backport branch-2.5

https://github.com/Mergifyio backport branch-2.4


Add missing await in skr tests

Description
Changes proposed in this pull request:

Add missing await in skr tests

Related issue(s)

/kind enhancement


Remount issue for AST nodes

The root layout has a unique key which makes the whole component remounting each time outer layer changes

I managed to resolve this by making the root have a key 'root' instead of the getUniqueKey()
and for each ast node, use Type + tokenIndex to avoid remounting issues.
https://www.npmjs.com/package/@ddx0510/react-native-markdown-display


Error running server

I get the following error when trying to start the OpenBazar-Server
Traceback (most recent call last):
  File "openbazaard.py", line 10, in <module>
    from api.ws import WSFactory, AuthenticatedWebSocketProtocol, AuthenticatedWebSocketFactory
  File "/home/jza/Downloads/OpenBazaar-Server/api/ws.py", line 18, in <module>
    from dht.node import Node
  File "/home/jza/Downloads/OpenBazaar-Server/dht/node.py", line 8, in <module>
    from protos import objects
  File "/home/jza/Downloads/OpenBazaar-Server/protos/objects.py", line 17, in <module>
    import countries as countries__pb2
  File "/home/jza/Downloads/OpenBazaar-Server/protos/countries.py", line 23, in <module>
    x0eWESTERN_SAHARA\x10\xf7\x01\x12\n\n\x05YEMEN\x10\xf8\x01\x12\x0b\n\x06ZAMBIA\x10\xf9\x01\x12\r\n\x08ZIMBABWE\x10\xfa\x01\x12\x10\n\x0b\x41\x46GHANISTAN\x10\xfb\x01\x12\x12\n\rALAND_ISLANDS\x10\xfc\x01\x62\x06proto3')
TypeError: __init__() got an unexpected keyword argument 'syntax'

I also get this error. Debian testing amd64

The error went away for me after I installed protobuf==3.0.0b2 and removed my system's python-protobuf (2.6.1-1.3) package.


Why Insider for a Generally Available product?

So why enroll in Insider when mdtap edr is GA for MAC OSX?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c91a5cfb-d32d-3864-c1d5-aec39558f66f
Version Independent ID: dbd2647e-88a6-ff22-4eab-ef628ac2e605
Content: Enable Microsoft Defender ATP Insider Device - Windows security
Content Source: windows/security/threat-protection/microsoft-defender-atp/endpoint-detection-response-mac-preview.md
Product: w10
GitHub Login: @Dansimp
Microsoft Alias: dansimp

Closing via #8518
Thanks


Missing ability to return HTML DOM element in 'render' function

I miss an ability to return HTML DOM element in "render.target" function when I use the dataset API.
Now is possible to return only "string".
I have list of items. I use angular component to display each Item. The component has its logic.
I need compile that component in 'render' function and return element
let template = angular.element("<super-component data-item='item'></super-component>");
return this.$compile(template)(scope);

but its not possible now
I can return only string
let template = angular.element("<super-component data-item='item'></super-component>");
let compiledTemplate = this.$compile(template)(scope);
return $(compiledTemplate).prop("outerHTML");

but it is not sufficient -> this way I will lost component functionality
Now I do not see an option to list JS components (at least angular 1.5 components) by the dataset API

Hi - thanks for raising this. I will be releasing an update in the next days that will allow the render function to return a single DOM element.
Hopefully this will be enough for your angular integration, but I intend to build an angular 2 extension in the next few months that will make this process more seamless.

The ability to return a DOM element has been added in v3.1.4 - let me know if works for you. Otherwise you may need to wait for the angular extension.

Thanks a lot for fast response and the update that resolves the issue. Now it works! :)


Use of nvarchar on AggregateId in EventStores.MsSql could impact indexing

Is unicode needed in the aggregate id and should it be 255 long?
Using unicode in an indexed field does have an impact on the available key space in a data page in sql server which could result in performance issues when handling huge volumes of event data in the event store.

@JC008 Valid point. Any suggestions on how to create a SQL migrate script that could be executed on a running system? We use the MSSQL store in production and thus it would be critical that the operation could be performed without any significant downtime.

I would consider calculating a new AggregateIdHash field for the aggregate id and using it as the base for IX_EventFlow_AggregateId_AggregateSequenceNumber thereby leaving AggregateId intact and allowing for a graceful switch-over.

@rasmus FYI http://stackoverflow.com/questions/32271518/append-only-to-table-in-sql-server-to-record-immutable-events-and-improve-overal

I read the article which basically sums up to this

Ensure ALLOW_ROW_LOCKS is ON
Ensure ALLOW_PAGE_LOCKS is ON
Write SQL for inserting and selecting based on multiple factors including CPU, memory, network

For 1. and 2. ON is the default for the database. As for using varchar instead of nvarchar I'll keep it as it is. I understand the concern for wanting to limit the amount of storage space and memory used, but I don't think limiting the character set for aggregate IDs is a good idea for the default database schema. As you wrote, migrating a live event source is problematic and limiting the character set that you can use for aggregate root IDs isn't worth it. Developers might e.g. at some point want to use e-mail addresses as aggregate root IDs.
I'll keep the nvarchar for the aggregate root IDs, but I do think you have a point regarding performance optimizations in EventFlow. The default behavior of EventFlow should be "good enough", but it should be possible to do performance tweaking. Regarding the SQL schema, you control when, and if, you want to use it and thus you could create your own schema or even your own MSSQL store implementation as its a very few lines of code (see MssqlEventStore). Its important for me that EventFlow "just works" and handles CQRS+ES task very well, without too many surprises whan you start using it.
Again, your point is valid and I suggest to create a "performance tips" guide to EventFlow with specifics on what you could do to improve performance by e.g. imposing limitations on what characters you can put in an aggregate root ID.

I asked the question on SO on Friday in the hope that someone could provide some guidance beyond the obvious but is seems there is none as far as SQL Server is concerned. Postgresql (GreenPlum) supports append only storage and in general, append only structured storage will become more prevalent in the future.
More importantly is awareness of the impact of using the nvarchar but more important than that is how flexible the EventFlow approach and implementation is regarding storage. I will add it to the list of articles I am writing for the EventFlow Guide.

@JC008 Thanks for investigating it and I appreciate the effort, I'll run it by our SO team as well just to be sure.