Helm tests fail on distroless.

This PR fixes the issue #15414. The new distroless images no longer contain the curl command. Therefore, I changed the images, which are used to run curl test commands from proxy to pstauffer/curl:v1.0.3
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[X] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: kramerul
To complete the pull request process, please assign rshriram
You can assign the PR to them by writing /assign @rshriram in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

install/OWNERS
tests/e2e/tests/pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kramerul: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-master

prow/integ-telemetry-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-telemetry-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-master

prow/integ-telemetry-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-telemetry-k8s-presubmit-tests-master

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test e2e-simpleTests-master

/test integ-telemetry-k8s-presubmit-tests-master
/test integ-telemetry-k8s-presubmit-tests-master

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/integ-telemetry-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-telemetry-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

looks good. this is just for the helm test so I'm not concerned about the performance of the image of anything (an extra image to pull, etc)
Do we have other images that are hardcoded? not sure if that will break anyone with restrictive image policies and whatnot (although I would like to avoid adding helm settings as much as possible)

/test e2e-simpleTests-distroless

/test e2e-simpleTests-distroless-master

^ the test was misconfigured to not run automatically but can be manually triggered like above. and Maria has a PR to fix it but we were waiting for it to pass first

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/integ-telemetry-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-telemetry-k8s-presubmit-tests-master

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Looks like the tests enable coredump which fails with      Message:   oci runtime error: container_linux.go:247: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory"

Unfortunately all image names must be configurable as many individuals operate Istio in air gapped environments, and air-gapped installs are a hard requirement.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: kramerul
To complete the pull request process, please assign rshriram
You can assign the PR to them by writing /assign @rshriram in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

install/OWNERS
pilot/OWNERS
tests/e2e/tests/pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/integ-telemetry-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-telemetry-k8s-presubmit-tests-master

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/istio-unit-tests.sh
7cede5f2bb4de96af9495047fa230fd423c777d5
link
/test istio-unit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/integ-telemetry-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-telemetry-k8s-presubmit-tests-master

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/istio-unit-tests.sh
7cede5f2bb4de96af9495047fa230fd423c777d5
link
/test istio-unit-tests-master

prow/codecov.sh
7cede5f2bb4de96af9495047fa230fd423c777d5
link
/test istio-codecov-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sdake, only test images are hard coded now. The image to enable core dumps is now configurable. I would expect that the flag enableCoreDump is used rarely.  Only in that case, you need to set enableCoreDumpImage in an air-gapped environment.
Hopefully, this is enough. Or do we need to make the hub configurable?

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/codecov.sh
7cede5f2bb4de96af9495047fa230fd423c777d5
link
/test istio-codecov-master

prow/istio-unit-tests.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-unit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/istio-unit-tests.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-unit-tests-master

prow/codecov.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-codecov-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/integ-mixer-k8s-presubmit-tests.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test integ-mixer-k8s-presubmit-tests-master

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/istio-unit-tests.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-unit-tests-master

prow/codecov.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-codecov-master

prow/racetest.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-racetest-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/istio-unit-tests.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-unit-tests-master

prow/codecov.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-codecov-master

prow/racetest.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test istio-racetest-master

prow/integ-mixer-k8s-presubmit-tests.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test integ-mixer-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

prow/integ-mixer-k8s-presubmit-tests.sh
1e5b96f2c9e2caed1dd525c6e743f07fe3a420de
link
/test integ-mixer-k8s-presubmit-tests-master

prow/racetest.sh
6f5b4b7cdf58abfd01683ced25d03c5c26706194
link
/test istio-racetest-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

I am fine with this, will hold off for approval from @sdake though

/assign @sdake
/retest

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: kramerul, sdake
To complete the pull request process, please assign costinm
You can assign the PR to them by writing /assign @costinm in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

install/OWNERS [sdake]
pilot/OWNERS
tests/e2e/tests/pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, kramerul, sdake
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS [howardjohn,sdake]
pilot/OWNERS [howardjohn]
tests/e2e/tests/pilot/OWNERS [howardjohn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/test istio-unit-tests-master

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

integ-galley-k8s-presubmit-tests-master
6f5b4b7cdf58abfd01683ced25d03c5c26706194
link
/test integ-galley-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@kramerul: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests-distroless.sh
4a05aeb6374d8aa33bb89fac5d9029fc47653b80
link
/test e2e-simpleTests-distroless-master

integ-mixer-k8s-presubmit-tests-master
6f5b4b7cdf58abfd01683ced25d03c5c26706194
link
/test integ-mixer-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Use GitHub Actions for CI & CD

This change would help to validate, test and release the chart.
It's using GitHub Actions and official Helm related workflows to automate the tasks.

Direct follow up of https://github.com/netbox-community/netbox-chart/issues/162#issuecomment-2061528891

Curious, what are the GPG bits for? I see it's half-commented-out, wasn't sure if there's some part of the workflow that wants it, or if it's vestigial from another version of this that you use.

@RangerRick It's a common setup to sign Helm charts https://helm.sh/docs/topics/provenance/.
That being said, none of this is yet configured for this chart, thus not available for use.
It's left as a usual best practice.

I've gotta figure out some github access stuff, so I can't set up the signing bits yet, but for now this should merge as-is and I can uncomment that stuff when we're ready. Thanks for this!


fix: Make only lower case container image names

Signed-off-by: Ashok Pon Kumar<EMAIL_ADDRESS>

This will be available in v0.3+ builds


vercelPreset for vite disables the ability to preview the production site on localhost

When using the vercelPreset within the vite.config.ts, the build output path is altered. Instead of the usual

./build/server/index.js

, the output path becomes something like

build/server/nodejs-eyJydW50aW1lIjoibm9kZ*****/index.js

This makes it difficult to preview the site on localhost, as the build "start" script needs to be manually updated with the new identifier. Do I really have to call "start": "remix-serve ./build/server/nodejs-eyJydW50aW1lIjoibm9kZ*****/index.js", each time I want to serve it locally or is there something I can do to disable the altering of the output path on building it locally?

Hey, I was facing this issue as well. My solution was to find the file with pattern matching and pass it to my remix-serve command in a bash script, like this:
#!/bin/sh

# Find the directory that matches the pattern
BUILD_DIR=$(find ./build/server -type d -name "nodejs-*")

# Check if the directory was found
if [ -d "$BUILD_DIR" ]; then
  # Construct the full path to the index.js file
  INDEX_FILE="$BUILD_DIR/index.js"

  # Start the server
  remix-serve "$INDEX_FILE"
else
  echo "Build directory not found!"
  exit 1
fi

In package.json:
"start": "./start-server.sh"

I'd be curious about a solution which doesn't require creating a new file!

How about conditionally only add the preset when an env var is set, i.e. process.env.VERCEL?

Thanks, @TooTallNate! I was initially considering checking if it was a local build or a preview using the isPreview flag, but since Remix doesn't utilize that, I couldn't figure out how to implement it. Your comment helped me find the solution, and it works perfectly. For anyone interested, here's the code:
export default defineConfig(() => {
  if (!process.env.VERCEL) {
    return {
      plugins: [remix(), tsconfigPaths()],
    };
  }

  return {
    plugins: [
      remix({
        presets: [vercelPreset()],
      }),
      tsconfigPaths(),
      sentryVitePlugin({
        org: "***",
        project: "***",
      }),
    ],
    build: {
      sourcemap: true,
    },
  };
});


793 pluggable authz retries

I needed to add retry logic, so that dotmesh wouldn't have quite so much trouble if it starts before the gateway is ready. Also found that I'd broken some tests before and not noticed, so I fixed that!

CRed via slack chat with Kai


[W5][W11-2]Wang Zhi Jian

Add listtag command to list all persons with a certain tag.

You restructured the list command and added the JUnit tests together in the same commit.
It might be good to separate two changes into separate commits, in case you mess up your work halfway. Also, this makes it easier for a third party to see what is happening to the code base.

Yup I realised that I missed out a commit for adding JUnit tests after restructuring the code. Will take greater care to commit after making changes next time.

Well done. Some comments:

Nice that you have updated tests.
It is good that you have incremental changes. You can try to have commits done in a batch for each enhancement.
Wildcard * imports is a coding standard violation, done automatically by Intellij. You need to configure Intellij to stop doing that. See Parser.java.

After reading the comments, please close the PR.

Thanks for the review :) Will change my Intellij configuration.


Darwin wrong launchctl load

node_modules/user-startup/lib/startup/darwin.js
line 44:
cp.execSync('launchctl load ' + name);
must be
cp.execSync('launchctl load ' + file);
you load the service plist file, then you can start the service by name

My bad, thank you :)

BTW, great lib, save me lot of work...

Thanks :)


add convenience wrappers for var and vars

A quick shot at addressing the proposal in #33

Looks good to me! Feel free to merge whenever.


[Documentation Generation] make_rst.py fails to generate rst files for any gdscript generates xml files.

Tested versions
4.3.3 - master clone
4.2 - stable clone
System information
Window 11, Python 3.12.3
Issue description
NOTE: The following example gdscript I used here was named parse_script.gd
When using a xml documentation file generated using the --doctool and --gdscript-docs, the generated xml doc file cannot be written to disc. Instead it gives an output like this:
Generating the RST class reference...
Traceback (most recent call last):
  File "C:\Users\Me\Godot\make_rst.py", line 2638, in <module>
    main()
  File "C:\Users\Me\Godot\make_rst.py", line 814, in main
    make_rst_class(class_def, state, args.dry_run, args.output)
  File "C:\Users\Me\Godot\make_rst.py", line 913, in make_rst_class
    with open(
         ^^^^^
OSError: [Errno 22] Invalid argument: 'C:\\Users\\Me\\Godot\docoutputtest\\classes\\class_"parse_script.gd".rst'

More importantly, it appears that when using --dry-run with make_rst.py, no errors are found.
Upon further inspection this appears to be fixed when editing the xml doc file's <class name=""parse_script.gd"" ... to remove the "s.
Most likely this is due to the fact that whenever is used to determine the file name of the .rst file uses the class's name attribute directly, instead of filtering it for characters impossible to use in filenames.
Instead of fixing this directly I'm making this a issue because:

I'm not confident enough in my skills to directly make a pr for a large project's doc generation script like this.
I'm not sure if this would break an expected naming or other formatting conventions expected with Godot's .rst doc formatting.

@NovaDC Please test #98533.

Should the issue be closed now?

No, the issue should be open until the PR is merged.


Fixed --skip and --rules regression from 0.8.0

Coverage increased (+0.06%) to 87.863% when pulling 4c599fc97b124bf3efeb573a6e0192c520e65f77 on bug/issue151 into f4f7d40e1593ffda9d0409ad13403840982b27c1 on master.


[DotNetCoreCLIV2] Can't define runtimeIdentifier to use for restore

Required Information
Bug, or Feature ?
Task Name: DotNetCoreCLIV2
Environment
AzureDevOps
Agent : Hosted Windows 2019
Issue Description
I need to specify the RuntimeIdentifier used by the restore command.
In Build or Publish command it's possible to pass it by using the parameter "Arguments".
But this parameter is not used by restore command (Indicated in the documentation).
So I tried to use the custom command as a workaround but it doesn't handle Azure Artefact authentication like the restore command does so i'm blocked.
So the "Restore" command need to be able to handle the "Arguments" parameter.

Assigned to @infin8x for considering the bug for restore command in DotNetCoreCLIV2 task.

We are currently building a new strategy for authentication to Azure Artifacts from Azure Pipelines. There will be a set of tasks that just configure authentication, allowing you to call tools like dotnet directly from any part of the build, including the various script tasks.
We do not intend to add new packaging features to the existing all-in-one tasks at this time, but if you wish to contribute, we will consider small, targeted pull requests
As a workaround, if you're using a dotnet SDK with NuGet 5.x (see the NuGet 5.0 release notes), you can use artifacts-credprovider with an environment variable to set up authentication for an arbitrary script command. Use the $(System.AccessToken) build variable to get the password. For pipelines built from the designer UI, you need to go into the phase settings and check "Allow scripts to access OAuth token". For pipelines described in YAML, you should be able to use a block like the following:
- powershell: dotnet restore -r win-x64
    env:
        - VSS_NUGET_URI_PREFIXES: https://myazuredevopsorg.pkgs.visualstudio.com/;https://pkgs.dev.azure.com/myazuredevopsorg/
        - VSS_NUGET_ACCESSTOKEN: $(System.AccessToken)

@jmyersmsft Thanks for your help.
I already use the environment variable with the credprovider approach for docker build, I wasn't thinking to use directly dotnet through script, it's a good workaround.
In parallel I also try to implement the change with a PR because the build pipeline is simpler with it.


Update sample config file to match installation guide

See https://github.com/finos/legend/pull/376

Can you also update README.md so the port mentioned there matches the sample config?


Compentizing Inventory

and paving the way for multi account multi cluster pentagon repos and GKE as a first class citizen.
This is a major step to Pentagon 2.0
WIP:

[x] Make the VPN work properly
[x] Figure out how to make this work as a default and a flexible setup
[x] Figure out what to do with Makefile

Closing in favor of https://github.com/reactiveops/pentagon/pull/99


Missing Count User Orgs Endpoint / Method

Is it possible to add this also?

Is this an existing endpoint, or something you want to add?

i suggest adding this, so all get endpoint would have an additional count method?
EDIT: i just thought about it => if this endpoint returns all orgs and is not paginated, than a count would be obsolete and this can be closed. (if you like to add it, you can of cause)

Yes. This is why it wasn't added. Haven't encountered any use cases where a user was in so many orgs that this method needed to be paginated. And was secretly hoping that someone who had the case would PR it for me ;)


Adjust keep times to mitigate missing predecessors

Awhile ago, the amount of time that the Completer holds onto blocks was adjusted in order to resolve a bug dubbed the "flatline bug" (https://github.com/hyperledger/sawtooth-core/commit/df75658011e59dff0be2a8ce45a6ac2770303fec). At that time, the amount of time the Chain Controller holds onto blocks should have also been updated, but this was missed. Because of this, it became much easier for the Chain Controller and Completer to get out of sync with respect to the blocks they thought were in the system. With the two out of sync, it was possible for the Completer to send a block to the Chain Controller that was built on a block that the Chain Controller had dropped. This resulted in a forking bug deemed the "missing predecessor bug" (STL-901). This commit updates the time that the Chain Controller holds onto blocks. This commit is a temporary bandage to mitigate the out of sync problem. A full solution requires the block manager (STL-1189).

Completed overnight stability check at 20 TPS. Network stayed in consensus and did 6.26 TPS. One node was behind but on the same fork.


Remove redundant imports from project components

Removed redundant imports, left from previous refactorings and changes in the codebase, from all project component Cargo.toml files.

I see some things are commented out rather than deleted. What's the motivation for that?

Ah, that's a mistake.  All commented-out lines should be deleted.  I'll push a new version.


Lead for web interface?

David is working on the database functionality but not super keen on developing the web interface. We have AstrodbWeb has a prototype but need someone to take the lead on building a new thing.

Happy to lead this when time allows!

Involvement with exo-mast people may help here majorly


Preventing app startup when no Mongo connection avaialble

If the mongo connection is unavailable, constructing MongoStorage  throws an unhandled exception which prevents the application from starting.
StorageOptions.CheckConnection can be disabled to prevent the ping, but migrations still run which require an active connection. Storageoptions.BypassMigration can also be set to true to disable the migrations. However, MongoMigrationManager is internal so it can't be called later when the app has started and the Mongo connection is available.
Are there any recommendations for registering the dependencies & deferring migration until a connection is established?

@anth12,
I see. Could you create a custom MigrationStrategy which executes the migrations when signaled?
You can set the custom strategy in the MongoMigrationOptions
You can edit the set the "MigrationLockTimeout" in the StorageOptions
I will make the migration public and overridable in the next release
Thanks


fix: TextInput some missing, and deprecated props

updated text input based on the latest changes in React Native Web
ReactNativeWeb:TextInput

@TheEhsanSarshar looks good, lmk when ready

we probably want to deprecate react-native-web-lite, not sure if rnw properly supports concurrent mode on some things still (especially ScrollView) but maintaining the fork is a lot of hassle for us, and looks like they started making some progress there finally too.

@TheEhsanSarshar actually bringing in some of the new style changes to tamagui itself will be a huge help, see these commits:

eg transform string syntax looks cool

@natew the changes to react-native-web-lite is ready, but I should change tamagui itself as well.
I thought rnwl is our source for cross-platform stylings.
where do we use rnwl ? is it for expo web ?

@TheEhsanSarshar nope i just was frustrated the rnw didn't support concurrent mode or tree shaking, and also i needed to use some internal libraries inside it without the rest. so you get a bit better tree shaking and concurrent mode support, but other than that its not necessary. but yes getting tamagui to support all the new RN stuff is great to follow on. i'll merge this and test it out.


Form Error Messages Announced All at Once Overwhelming VoiceOver Users.

Steps to reproduce

create a random form.
activate the voiceover from your mobile accessibility settings.
save the form without filling it or with wrong values.

Expected results
The voiceover announce only the first error message each time the user saves the form, guiding users with disabilities through each field error step-by-step.
Actual results
When validating a form in Flutter, all error messages are concatenated into a single message and announced at once. This behavior overwhelms users relying on voiceover, as they are bombarded with multiple pieces of information simultaneously.
Code sample
This is the code that actually exists in the library:
bool _validate([Set<FormFieldState<Object?>>? invalidFields]) {
    bool hasError = false;
    String errorMessage = '';
    for (final FormFieldState<dynamic> field in _fields) {
      final bool isFieldValid = field.validate();
      hasError = !isFieldValid || hasError;
      errorMessage += field.errorText ?? '';
      if (invalidFields != null && !isFieldValid) {
        invalidFields.add(field);
      }
    }

    if (errorMessage.isNotEmpty) {
      final TextDirection directionality = Directionality.of(context);
      if (defaultTargetPlatform == TargetPlatform.iOS) {
        unawaited(Future<void>(() async {
          await Future<void>.delayed(_kIOSAnnouncementDelayDuration);
          SemanticsService.announce(errorMessage, directionality,
              assertiveness: Assertiveness.assertive);
        }));
      } else {
        SemanticsService.announce(errorMessage, directionality,
            assertiveness: Assertiveness.assertive);
      }
    }
    return !hasError;
  }

Screenshots or Video
https://github.com/user-attachments/assets/aa56b132-92ec-4bd0-88f8-11704ac847ba
Logs
No response
Flutter Doctor output
not related.

@danagbemava-nc would you please assign it to me so I can start working on it, or do I need to wait until the issue is confirmed ?

Reproducible using the code sample below
@azizChebbi, you can start working on it without being assigned

android
iOS

code sample
import 'package:flutter/material.dart';

void main() => runApp(const MyApp());

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    const appTitle = 'Form Validation Demo';

    return MaterialApp(
      title: appTitle,
      home: Scaffold(
        appBar: AppBar(
          title: const Text(appTitle),
        ),
        body: const MyCustomForm(),
      ),
    );
  }
}

// Create a Form widget.
class MyCustomForm extends StatefulWidget {
  const MyCustomForm({super.key});

  @override
  MyCustomFormState createState() {
    return MyCustomFormState();
  }
}

// Create a corresponding State class.
// This class holds data related to the form.
class MyCustomFormState extends State<MyCustomForm> {
  // Create a global key that uniquely identifies the Form widget
  // and allows validation of the form.
  //
  // Note: This is a GlobalKey<FormState>,
  // not a GlobalKey<MyCustomFormState>.
  final _formKey = GlobalKey<FormState>();

  @override
  Widget build(BuildContext context) {
    // Build a Form widget using the _formKey created above.
    return Form(
      key: _formKey,
      autovalidateMode: AutovalidateMode.always,
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.start,
        children: [
          TextFormField(
            // The validator receives the text that the user has entered.
            validator: (value) {
              if (value == null || value.isEmpty) {
                return 'Please enter some text';
              }
              return null;
            },
          ),
          TextFormField(
            // The validator receives the text that the user has entered.
            validator: (value) {
              if (value == null || value.isEmpty) {
                return 'Please enter some text';
              }
              return null;
            },
          ),
          TextFormField(
            // The validator receives the text that the user has entered.
            validator: (value) {
              if (value == null || value.isEmpty) {
                return 'Please enter some other text';
              }
              return null;
            },
          ),
          TextFormField(
            // The validator receives the text that the user has entered.
            validator: (value) {
              if (value == null || value.isEmpty) {
                return 'Please enter some other other text';
              }
              return null;
            },
          ),
          Padding(
            padding: const EdgeInsets.symmetric(vertical: 16),
            child: ElevatedButton(
              onPressed: () {
                // Validate returns true if the form is valid, or false otherwise.
                if (_formKey.currentState!.validate()) {
                  // If the form is valid, display a snackbar. In the real world,
                  // you'd often call a server or save the information in a database.
                  ScaffoldMessenger.of(context).showSnackBar(
                    const SnackBar(content: Text('Processing Data')),
                  );
                }
              },
              child: const Text('Submit'),
            ),
          ),
        ],
      ),
    );
  }
}

flutter doctor -v
[✓] Flutter (Channel stable, 3.24.3, on macOS 15.0.1 24A348 darwin-arm64, locale en-US)
    • Flutter version 3.24.3 on channel stable at /Users/deanli/dev/stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2663184aa7 (4 weeks ago), 2024-09-11 16:27:48 -0500
    • Engine revision 36335019a8
    • Dart version 3.5.3
    • DevTools version 2.37.3

[!] Flutter (Channel master, 3.26.0-1.0.pre.392, on macOS 15.0.1 24A348 darwin-arm64, locale en-US)
    • Flutter version 3.26.0-1.0.pre.392 on channel master at /Users/deanli/dev/master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision bd718ef93b (4 hours ago), 2024-10-08 01:12:58 -0400
    • Engine revision 5923a8b3e6
    • Dart version 3.6.0 (build 3.6.0-329.0.dev)
    • DevTools version 2.40.0

Thanks @danagbemava-nc, I'll start working on it right away.


No "Open Quickly..." menu item

How to I open the feature shown here?

Was expecting this the be listed at File -> Open Quickly..., and for the keyboard shortcut to be shown there.

@fulldecent You're looking for the Fuzzy Finder, I believe: https://github.com/atom/fuzzy-finder. It's a package.

This isn't a terrible idea on OS X to add it as "Open Quickly" since it's basically the same feature

Even if I know it was called Fuzzy Find, this is how I would try to find the keyboard short cut:

This isn't a terrible idea on OS X to add it as "Open Quickly" since it's basically the same feature

@paulcbetts I definitely didn't know "Open Quickly" was a standard OSX thing -- thanks! I've been using OSX for about 2 years and never seen that option myself (just checked Sublime, Skitch and a few other apps which open files). I'm not :-1: on adding such an option -- but not sure what the best place would be to open a feature request for that. Perhaps the Fuzzy Finder package?

Ah, got it I can do that. And to be sure, Atom core would provide the ability to fuzzy find to create a menu item if it wanted, right?

Even if I know it was called Fuzzy Find, this is how I would try to find the keyboard short cut:

@fulldecent The recommended way is to use the Command Palette -- it's mentioned on the Welcome screen when you start Atom for the first time.
You can also find the option in the Help -> Search box by typing "find file".

And to be sure, Atom core would provide the ability to fuzzy find to create a menu item if it wanted, right?

Fuzzy finder already adds a few menu options, just not the ones you expected:
https://github.com/atom/fuzzy-finder/blob/master/menus/fuzzy-finder.cson

Find file is reasonable, I should have found that


Cannot easily use v-tooltip with user-generated content (XSS / HTML injection)

First, thanks for creating this very useful project.
I noticed that it is currently not easily possible to safely put user-generated content into v-tooltip, because the popper option html is always true and I cannot seem to override it. E.g. if you have code such as v-tooltip="user.name", users would be able to enter a name such as <b>Example</b> or worse, leading to XSS vulnerabilities.
I tried the following approaches, but none of them helped:

install VTooltip with { defaultPopperOptions: { html: false } } using Vue.use
pass v-tooltip="{ popperOptions: { html: false }, content: user.name }"
pass v-tooltip="{ html: false, content: user.name }"

I would expect at least the first two options to work.
Right now, the only way seems to be

either to manually sanitize the input text using e.g. the DOM API,
or to use v-popover instead, which is much more complicated compared to v-tooltip.

Personally, I think that the most straightforward way to use v-tooltip (just using the directive attribute) should also be "safe by default". Or is there maybe another simple way to achieve what I am looking for?

@Akryum thanks, I'll make sure to try it out!


WIP: First shot at a SST1M R1Calibrator

(This is based on #668. So the diff should become much smaller once #668 is merged.)
So .. R1Calibrator... I must admit I do not know what I am doing here, but I still wanted to give it a shot, sometimes it is easier to argue about something, if one has an example of how to now do it.
In order to understand what an R1Calibrator is supposed to do, I had a look at the already existing implementation, but I did not understand what they are doing.
So I read the (not yet published?) document named "R1 & DL0 Telescope Event Interfaces
and Prototype Evaluation". which I got by email, so I have no idea how to link this document, I could not even tell if I had the most recent version of it... anyway the version I had, says in chapter 3.1, that the R1 Calibration is supposed to do this:

remove any effects of ADC inhomogeneities in time (ring-sampler corrections)

[x] subtract a pixel-wise rough electronic + NSB noise pedestal value,  so that the values fluctuate
around 0 counts (and record it as MON or SVC data, so it can be un-applied or quality-checked
later)

if multiple gain channels exist, optionally select the appropriate channel such that the readout value
is within a linear range (or mark the pixel as saturated).
if a non-linear gain is used, apply the gain correction function

[x] apply a pixel-wise multiplicative gain correction factor to correct for differences in camera uniformity
[ ] apply a scalar scale and shift the signal so that it efficiently fits in a 16-bit signed integer value with
the required dynamic range (see Section 3.3)
[ ] record all applied corrections in SVC or MON streams

(Those, which have a bullet point and not a tick-box, do not apply for SST1M/digicam, but I though this list is so nice, I wanted to keep it complete.)
I think I did the first 2 points in this PR. I did not yet look-up how to write into SVC or MON streams, so I did not yet do it.
I did not apply any special scaling so that the result fits into 16bit int rage, since it does so by design. The digicam ADC is 12 bit (unisgned int) and by removing the baseline and multiplying with the gain_drop (value between 0 and 1) I am bound to stay somewhere in a 12bit signed integer range. However, one might want to increase resolution and artificially multiply everything by 16 to make use of the 16bit range we have. However, this does not really add anything and I was not sure what is wanted.
I'd be happy for advice, if this is running in a somewhat right direction.

Ah another reason, why I am feeling very unhappy about this is, that it is of course not yet clear how to best perform this R1 Calibration for digicam. We are in a prototype phase and people are working everywhere at the same time.
I personally think that sometimes one wants to see the effect of a modification in R1Calibration on high level output like ... dunno ... size distribution on an example data set.
To study this, a telescope group would implement different R1Calibrators, but all the higher level stuff would stay the same and could be reused. (That is at least my dream ... )
On the other hand one always runs (at least) into the "cleaning-question", which goes like this: "When you modify how the baseline is defined, you also need to optimize the cleaning-levels for this case, otherwise the comparison is not useful at all".
So at least the cleaning levels (maybe the whole cleaning algorithm) needs to be adjusted for different R1-calibrations.
So my question is, how do other telescope groups perform these studies? What role does the CameraR1CalibratorFactory play in such a study, how could one utilize that factory to test different calibrators?  Are there examples for such studies?

The R1 calibration is not well defined yet,  but the basic idea is that all cameras should put out data that is "good enough" to to a basic hillas analysis (for realtime purposes) and so that the data volume reduction can be performed in a generic way at the R1 level.
That means at least applying some rough calibration (this is not the final calibration, so need not be perfect) so that camera images are uniform enough to determine where the shower is so we can drop non-shower pixels later.   It also means that the output should  still  be integers like in R0, not floats, so basically the pedestal and gain corrections just round to the nearest integer (this is necessary to store them in 16-bit ints in DL0).
What is done so far  for MCs in ctapipe is not quite this.
Also keep in mind that the R1 calibration will almost certainly have to be implemented in the camera servers, so won't probably be in python (it's out of scope of ctapipe, except for monte-carlos), but it's useful to have it prototyped here so we can ensure we apply the same calibration to MCs as is done to the real data.

Also the document you refer to is not yet official and was only circulated to the PC members as a status report, and still needs to go through a full review process with the camera groups.   Therefore do not assume anything in it is correct!. I'm also probably going to remove any mention of what R1 calibration means in that document, since that should be in a separate calibration specification.

I #668 Karl made two very helpful comments regarding reading files. I will cite them here for completeness.
start quote:

Any files you load must be recorded in the provenance system, so either use utils.get_dataset() to do it automatically (which also searches automatically everywhere in CTAPIPE_SVC_PATH, which can contain multiple directories), or add the following line afterward:
Provenance().add_input_file(self.dark_baseline_path, role='dl0.tel.svc.sst1m_baseline')

probably best to just use
from ctapipe.utils import get_dataset
...

get_dataset(self.dark_baseline_path, role='dl0.tel.svc.sst1m_baseline')

since having an abstraction for reading files allows us to later switch to reading this from a database, or remote server without changing any code.

end quote.

I have one question regarding the role string: How is this string designed? In your comment you propose:
"dl0.tel.svc.sst1m_baseline" .. one might also assume something like r1.tel.svc or even r0.tel.svc since this baseline is made from r0 level but used for r1.

I have one question regarding the role string: How is this string designed?

So far, we've just been using the data naming convention (DLx.{ARR,SUB,TEL}.{EVT,MON,SVC}.*).  We don't have a good convention yet, but I would suggest using the data level of where it is used, since that may be the most obvious... (though we should think about it)  So it may be R1 instead of DL0 if you are using it to compute the R1 waveforms.  I'd say go with what you think makes sense and we can write a note in the documentation about it, so we use the same convention afterward.

Ah thanks for the clarification on this. I will soon fix that.
So do I understand correctly, that (at the moment) these strings are not to be consumed by some software, but they are for humans? I assumed the purpose of the Provenance information was to provide machine readable information about the Provenance of a certain analysis result, so that it can be reproduced or the same treatment can be applied to similar input-data.
In that case, I assume Provenance must be machine readable and it must be ensured that it is correct by a harness of tests-which make sure that no Provenance information is missing and that it is correct.
Anyway for the time being this seems to be not the case, so I'll try to come up with a useful name and that's it.

these strings are not to be consumed by some software, but they are for humans?

yes, the provenance is machine readable, and is supposed to show the chain of processing in a reproducible way.  The "role" is a required field from the W3C model, intended to describe the reason for the file (like a semantic name, not a file name), similar to a UCD.  How those roles are defined is up to CTA, and we don't yet have a dictionary for them, so so far I'm just using it to define which type of data are loaded. Really we should put "cal" or something in the name as well (e.g r1.tel.svc.cal.XXX so what we understand that this was calibration data for a telescope.

Is this still relevant?


Anti-Cheat Kicking Players

What are the steps that lead to the bug occurring? / How frequently does the bug occur?

Not sure what the steps are for this bug to occur, and it occurs frequently with some users, but not all.

What is the impact of this bug?

Some players aren't able to play the game.

Image for context

Found the detection here.
You should look at your dev console as that might contain the reason why it detects "failed to parse json".

Thanks for your report; we'll look into this issue and update you once a fix has been published.

It seems that this issue is caused by a script in your own game. Check if your game contains any possibly malicious scripts or scripts that may cause the message "failed to parse json" to be outputted on a player's client.
One way that might help is to press Ctrl+Shift+F and search for "failed to parse json".


Don't Push the "Done" Button

Bug Report
Description
Pushing the "done" button on the files upload page. It makes the UI go away so it looks as if you haven't uploaded anything.

To Reproduce
Steps to reproduce the behavior:

Upload a file
Push "done"
Weep.

Expected behavior
The list of files you've uploaded probably shouldn't go away? Or maybe the button doesn't need to be there.

replace uploader?


Permissioned blockchain based databases

Proposed title of article
Permissioned blockchain based databases
Proposed article introduction
The most common blockchain based databases are the permissionless blockchain based databases. General implementation of a permissioned blockchain solution is a tough one which is why most people leverage on the permissionless part of blockchain.
Key takeaways
Readers will be able to understand general blockchain based databases.
Readers will be able to differentiate between permissioned and permissionless blockchain based databases.
Readers will be able to differentiate between traditional databases and permissioned blockchain databases.
Readers will be able to choose the right permissioned blockchain database for a particular solution.
A list of permissioned blockchain databases will be stated.
Conclusion
At the end of this article, blockchain enthusiasts will be able to understand the general blockchain concept and then differentiate between permissioned blockchain databases and permissionless blockchain databases.

@oleey, you should not remove sections of the Topic Approval form such as Article Quality that you are meant to fill in for consideration. Please add these back in and fill out the form correctly so we can appropriately review your article.

@oleey
Closing due to inactivity - if you'd like to continue working on the topic. Please let us know.


[NFC] Re-alphabetize underscored attributes reference

The introductory material says that we're alphabetizing, so let's make sure we stick with it!

@swift-ci smoke test

Hi @typesanitizer, can you point me to who might have oversight over these docs nowadays? Just want to make sure I ping them for awareness.

Sorry, I don't know; you could ask in the forums or look at git log -- docs in case you didn't already. Generally, it should be OK to have someone with some contribution experience + some edits to the same/related files sign off, unless something has changed recently.

Thanks. Let's see if @gottesmm would know as a recent contributor to this file...


Add possibility to erase drawings with eraser.

The possibility to erase your drawings was added today. Check v2.0 update.

The possibility to erase your drawings was added today. Check v2.0 update.


Tweaks for modals on mobile/Android

I'm using modal dialog in an Android Phonegap app, and had to make a few adjustments. Perhaps these can be incorporated or useful somehow.
First, I made a small modal which was half the size by manually overriding the width/margin. There's probably a better more generic way.
Then, I found it was quite hard to click the "x" in Android browser, so I gave the X as much padding as I could and still have it look the same.
My CSS:
.modal {
  width: 280px;
  margin: -140px 0 0 -140px;
  .modal-header {
    padding: 0px;
    .close {
      margin-top: 0px;
      padding-top: 10px;
      text-align: right;
      padding-right: 10px;
      padding-bottom: 10px;
      padding-left: 30px;
    }
    h3 {
      padding-left: 10px;
    }
  }

SGTM
On Mon, Oct 31, 2011 at 6:32 PM, Jacob Thornton
<EMAIL_ADDRESS>wrote:

We'll look at this closer for our 2.0 release which will have media queries and all that mobile type stuff... sound good?
--
Reply to this email directly or view it on GitHub:
https://github.com/twitter/bootstrap/issues/511#issuecomment-2585780

I applied 10px padding to the close button to address that issue—much easier to hit.
For the modal itself, I have removed the fixed width and reset the positioning to always be 20px from the edge. Makes it a bit more flexible for more devices. Still needs more refinement, but it's good enough to close this ticket for now.


KAFKA-15045: (KIP-924 pt. 12) Wiring in new assignment configs and logic

This PR creates the new public config of KIP-924 in StreamsConfig and uses it to instantiate user-created TaskAssignors. If such a TaskAssignor is found and successfully created we then use that assignor to perform the task assignment, otherwise we revert back to the pre KIP-924 world with the internal task assignors.

Test failures are unrelated, merging to trunk


Red background white text on Request Privileges

Red text on Gray background under request Privileges makes it difficult to read. You can uncomment line 186 to get red background on white text

@gustavo-sanchez-bbk Thanks for reporting this. The color scheme for a "destructive" button is defined by Apple. So the current color scheme is the macOS default and therefore we don't want to change it. I agree with you that it is not ideal but we want to stay as close to the standard macOS behavior as possible.


Cannot login/Sign up

When trying to use this web app after entering the email and password, pressing the login button does nothing at all. It remains on the sign in button.
When trying to sign up, you can proceed up to the “beep boop” captcha verification, but the Google Captcha does not work. The sign up portion of this is not a big deal to me, but not being able to login is a pretty big issue.

I'm having this exact same issue, once (on the wrong device lol) it didn't ask me for a captcha and I was able to login without having to do one but every other time I have tried to login it just hasn't done anything

Discord now blocks logins from third-party websites. You'll have to manually set the Discord token (perhaps with a bookmarklet).

How do I manually set the token?


Version Bump to 0.7.0

Update recipe for 0.7.0 version.
Due to some problems with downloading miniconda on AppVeyor (pelson/Obvious-CI#67) feedstock was upgraded with conda-smithy regenerate.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Did you regenerate with the latest version of conda smithy? It in general looks fine, but might want to try again to see about covering py36.

@pmlandwehr I regenerated with conda-smithy 2.0.1. I tried it one more time and no luck — py36 is still not there. Will try to investigate further.

@pmlandwehr supposedly this will start building for py36 when all the dependencies will be build for py36. Since only oauthlib is not yet on 36 I updated it to the latest version and py36. See https://github.com/conda-forge/oauthlib-feedstock/pull/5. Once that PR gets accepted this should also run on py36.

I still cannot find py36 build of 0.7.0 with conda. Could it be that this got merged too quickly, before the py36 oauthlib got available on conda? If so, @pmlandwehr any ideas how to re-trigger the building process to also build py36?

Yeah - still not positive what's up here  and we're getting beyond me. I've gone ahead and submitted a new build that adjusts the template a bit (needed to be brought up to date) and rebuilds again. Other thoughts, @ocefpaf ?

Great, that seems to start building py36 so I guess all is good.

Other thoughts, @ocefpaf ?

I am a bit lost in the issue, but I guess it is all sorted, right? If not feel free to ping me again.


Cannot run Spark 3 tests in IntelliJ IDEA

Describe the issue
I get the following error when trying to run the E2EHyperspaceRulesTest tests:
An exception or error caused a run to abort: org.apache.parquet.hadoop.ParquetOutputFormat.getJobSummaryLevel(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/parquet/hadoop/ParquetOutputFormat$JobSummaryLevel;
java.lang.NoSuchMethodError: org.apache.parquet.hadoop.ParquetOutputFormat.getJobSummaryLevel(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/parquet/hadoop/ParquetOutputFormat$JobSummaryLevel;
	at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.prepareWrite(ParquetFileFormat.scala:131)
	at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:132)
	at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:178)
	at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:108)
	at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:106)
	at org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:131)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:175)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:213)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:210)
	at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:171)
	at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:122)
	at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:121)
	at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:963)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:100)
	at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:160)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:87)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:764)
	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
	at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:963)
	at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:415)
	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:399)
	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:288)
	at org.apache.spark.sql.DataFrameWriter.parquet(DataFrameWriter.scala:848)
	at com.microsoft.hyperspace.SampleData$.save(SampleData.scala:48)
	at com.microsoft.hyperspace.index.E2EHyperspaceRulesTest.beforeAll(E2EHyperspaceRulesTest.scala:50)
	at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
	at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
	at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
	at com.microsoft.hyperspace.index.E2EHyperspaceRulesTest.org$scalatest$BeforeAndAfter$$super$run(E2EHyperspaceRulesTest.scala:34)
	at org.scalatest.BeforeAndAfter.run(BeforeAndAfter.scala:258)
	at org.scalatest.BeforeAndAfter.run$(BeforeAndAfter.scala:256)
	at com.microsoft.hyperspace.index.E2EHyperspaceRulesTest.org$scalatest$BeforeAndAfterAllConfigMap$$super$run(E2EHyperspaceRulesTest.scala:34)
	at org.scalatest.BeforeAndAfterAllConfigMap.liftedTree1$1(BeforeAndAfterAllConfigMap.scala:248)
	at org.scalatest.BeforeAndAfterAllConfigMap.run(BeforeAndAfterAllConfigMap.scala:245)
	at org.scalatest.BeforeAndAfterAllConfigMap.run$(BeforeAndAfterAllConfigMap.scala:242)
	at com.microsoft.hyperspace.index.E2EHyperspaceRulesTest.run(E2EHyperspaceRulesTest.scala:34)
	at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
	at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1346)
	at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1340)
	at scala.collection.immutable.List.foreach(List.scala:431)
	at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
	at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031)
	at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010)
	at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
	at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
	at org.scalatest.tools.Runner$.run(Runner.scala:850)
	at org.scalatest.tools.Runner.run(Runner.scala)
	at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:38)
	at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:25)

To Reproduce
To setup Hyperspace project in IntelliJ IDEA I followed steps provided by @imback82 in https://github.com/microsoft/hyperspace/pull/434#issuecomment-829801997.

Marked src/main/scala and src/main/scala-spark3 as sources.
Marked src/test/scala and src/test/scala-spark3 as test sources.
Marked src/test/resources as test resources.
Compiled from terminal with sbt clean compile.
From IntelliJ IDEA I right clicked on the E2EHyperspaceRulesTest file and chose Run 'E2EHyperspaceRulesTest'...

Expected behaviour
E2EHyperspaceRulesTest to run successfully.
Environment

MacOS Big Sur (11.1)
IntelliJ IDEA 2021.1.1 Ultimate Edition
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09)
Scala 2.12.11
Apache Spark 3.0.1
Local machine

There is another nuisance: after setting the project according to @imback82's #434 (comment), whenever I close the IDE everything needs to set once more. I need to remove once more the root and root-build modules and mark all the sources and tests. This takes time and is a nuisance.

Are you sure that the .idea directory is created in the repository root and the content of the directory is retained when you close and open the IDE again? For me, I don't have to do such things and everything was fine. Also, you don't have to remove root-build project (at least for me).

I'm looking into the issue. As a workaround, you can use the built-in sbt shell and type

testOnly *E2EHyperspaceRulesTest to run the test for all subprojects (or the current project set by the project command)
spark3_0/testOnly *E2EHyperspaceRulesTest to run the test for Spark 3.0
spark2_4/testOnly *E2EHyperspaceRulesTest to run the test for Spark 2.4

Please, don't close this issue. Or if you close it please create another ticket for the creation of the documentation explaining the way to develop on Hyperspace and IntelliJ IDEA (is should contain the way to debug a test - some screenshots would be great also).
It is not the common way of developing and because of that it should be documented.
Not helping the community and letting them bang their heads for hours (or even days) to be able to debug a test it will make the community of developers grow up in reluctance against this project.
If it helps, I can share my way of debugging, which I found by trying out different approaches.

As I said already, if you use the sbt shell in IntelliJ, everything works fine, including debugging with breakpoints. It's been only two months since I started developing in Scala so it should be easy for anyone developing in Scala.

@clee704 In my case is NOT working - even if I use the sbt shell inside IntelliJ. When I add breakpoints an run the test from sbt shell it does NOT stop to the break points. If it works for you it doesn't mean it works for every one.
The only way to make it work was to run sbt shell in terminal with debug ports open, then connect from IntelliJ IDEA with a remote debug configuration, then run the tests from the terminal sbt shell.
There are multiple problems - development-wise - that are now present in the project and those problems are a nuisance for the community.
For example, every time I close IntelliJ IDEA, I have to go through the process of removing root, etc. and mark the source folders once more, which is time consuming and a pain.
I my have different options in IntelliJ but that needs also to be documented.
@imback82, @sezruby, @rapoth: For me, right now, after the support of Spark 3 has been added, it is a pain to develop in Hyperspace project.
What can be done to make it better? How can I help?

I just realized that I omitted a crucial information for debugging in IntelliJ. You should set Test/fork to false in the sbt shell to use breakpoints. This can be done

set Test/fork := false if you've set the current project, or
set <project name>/Test/fork := false, e.g. set spark3_1/Test/fork := false

This is because with Test/fork set to true tests will be run in a forked JVM. As the debugger is attached to the shell process, tests must be run in the same process as the shell. I'll update README with this information.
I acknowledge that the current situation is not the best. If you have any idea to improve IntelliJ experience, please share with us or make a PR.


Clarify in README that posts in the future can be rendered with "--future" flag

Per Jekyll we can append the --future flag when running serve so that blogs posted in the future can be tested. The README should reflect this in the serve instructions.

Placing a link to the docs is fine, but I don't want to re-create the Jekyll docs on our README.
Note: we should also be careful about sending the message that future dated posts will automatically appear - they won't - we're not setup to do that.

Placing a link to the docs is fine, but I don't want to re-create the Jekyll docs on our README.
Note: we should also be careful about sending the message that future dated posts will automatically appear - they won't - we're not setup to do that.

Interesting @stockholmux . Should we set ourselves up to do that?

@elfisher It's a little hard to achieve, but not impossible. Basically it's like a triggered cron job to generate the site, but since the site is entirely on S3, we'd have to either setup something like a server or serverless function to run it.

@elfisher It's a little hard to achieve, but not impossible. Basically it's like a triggered cron job to generate the site, but since the site is entirely on S3, we'd have to either setup something like a server or serverless function to run it.

got it. Sounds like a more involved enhancement. Makes sense to keep it separate from this issue. Also agreed with the earlier comment. Linking to docs makes more sense.

Fixed by #281


Add ARP: Google.AndroidStudio 2<IP_ADDRESS>

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[x] Is there a linked Issue?

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

I'm not sure if those ARP fields should be added to this package.
ProductCode: "Android Studio", may overlap with other AndroidStudio packages such as Google.AndroidStudio.Canary:
--> Installing the Manifest 2<IP_ADDRESS>

Found Android Studio Canary [Google.AndroidStudio.Canary] Version 2<IP_ADDRESS>
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://redirector.gvt1.com/edgedl/android/studio/install/2<IP_ADDRESS>/android-studio-2<IP_ADDRESS>-windows.exe
  ██████████████████████████████  1.14 GB / 1.14 GB
Successfully verified installer hash
Starting package install...
Successfully installed

--> Refreshing environment variables

--> Comparing ARP Entries

DisplayName    DisplayVersion Publisher  ProductCode    Scope
-----------    -------------- ---------  -----------    -----
Android Studio 2024.2         Google LLC Android Studio Machine

Publisher: "Google LLC" isn't necessary as that's already defined in Google.AndroidStudio.locale.en-US.yaml:
https://github.com/microsoft/winget-pkgs/blob/98f18e9ea4276a6180aba8e2eb28c274fd83a80e/manifests/g/Google/AndroidStudio/2<IP_ADDRESS>/Google.AndroidStudio.locale.en-US.yaml#L7

Google's installer already causes some issues for WinGet. They only use MAJOR.MINOR semver for the DisplayVersion and all Android Studio channels write the same DisplayName and ProductCode. (Google doesn't support traditional side-by-side installation.)


Use Construct button is causing page crashes

This line is causing pages to crash when a user navigates to a package with an unsupported language in the params.
Rather than throwing (and subsequently crashing the page), we should only return the installation instructions string if the packageName exists. We have a client-side redirect that will switch the docs to typescript anyways so this won't cause any user-facing issues

Example:
Crashes due to unsupported language: https://constructs.dev/packages/cdk-drift-monitor/v/0.2.20?lang=python
Does not crash: https://constructs.dev/packages/cdk-drift-monitor/v/0.2.20?lang=typescript

Fixed


Fix whisper streaming

Type of Change

[ ] New example
[x] Example updates (Bug fixes, new features, etc.)
[ ] Other (changes to the codebase, but not to examples)

Fixes #732

Even after the updates I have made, I still got error using the fastapi endpoint. Can I have a little help?:
downloading https://www.youtube.com/watch?v=s_LncVnecLA
Expand
streaming transcription of https://www.youtube.com/watch?v=s_LncVnecLA audio to client...
[MainThread] 2024-05-04T12:21:34+0000 Warning: the results of a call to Function.starmap was not consumed, so the call will never be executed. Consider a for-loop like `for x in Function.starmap(...)` or unpacking the generator using `list(...)`
Traceback (most recent call last):
  File "/pkg/modal/_container_io_manager.py", line 488, in handle_input_exception
    yield
  File "/pkg/modal/_container_entrypoint.py", line 244, in run_input
    async for value in res:
  File "/pkg/modal/_asgi.py", line 126, in fn
    app_task.result()  # consume/raise exceptions if there are any!
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 69, in app
    await response(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 266, in __call__
    async with anyio.create_task_group() as task_group:
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 597, in __aexit__
    raise exceptions[0]
  File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 269, in wrap
    await func()
  File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 258, in stream_response
    async for chunk in self.body_iterator:
  File "/root/modal_app.py", line 186, in stream_whisper
    for result in transcribe_segment.starmap.aio(
TypeError: '_WarnIfGeneratorIsNotConsumed' object is not iterable


Rename ListDatabasesAsync to GetDatabasesAsync method in DatabaseApiClient

#47

I should also rename the models. Waiting for #49

I was waiting on #49 to rename the models. Is there a way I can reopen the PR? Maybe by pushing an additional commit.


Failed to read or decrypt the BLS private key

This is similar to a Closed Issue but when running ./run.sh opt-in I'm getting the following error:
Error: failed to read or decrypt the BLS private key: read /app/operator_keys/bls_key.json: is a directory

this is possible if you did not provide correct path to bls key in .env file. Please provide full path correctly. Please re-open if it's still an issue


Fix #746: invalid slice with start or stop =-1 when step<0

this one was tricky. :sob:
I let the tests used to track the bug.

Codecov Report

Merging #766 into master will increase coverage by <.01%.
The diff coverage is 60%.

@@            Coverage Diff             @@
##           master     #766      +/-   ##
==========================================
+ Coverage   61.95%   61.96%   +<.01%
==========================================
  Files          80       80
  Lines       12708    12716       +8
  Branches     2606     2608       +2
==========================================
+ Hits         7873     7879       +6
  Misses       3165     3165
- Partials     1670     1672       +2

Impacted Files
Coverage Δ

vm/src/obj/objsequence.rs
55.32% <60%> (+0.83%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5c9181...7d18a1c. Read the comment docs.

I refactored the tests to autogenerate it.
cpython passes test
rustpython fails.  it's because I import my resource file like this
from resources.cpython_generated_slices import SLICES_RES, START, END, STEP, LL
rustpython seems not be able to read this. I'm not sure where to go to try to fix it in rustpython code

You could use __import__ and just use a direct path to the resources file.

Only coverage is failing, I don't know why.

There's a problem doing a clean build on nightly, so coverage tests are currently failing for everything at the moment.

Is there anything else todo here ?


Update requirements and simlify readme

The install for most people is described in the website.
Developers just need this info to install and run qcodes.
Agree?

Makes sense to me. .travis.yml needs to be updated to install from the new requirements file

ref: #280  god darn it.

Also fixes #280 .


VoiceClient not playing file

My voiceClient isn't working when I try to run $voiceClient->playFile($filePath) and isn't showing any errors... The audio file path is correct (verified it using other functions), the bot connects to the voice channel and everything before and after calling playFile() executes normally (except the then() method).
$ws->on('ready', function () use ($ws, $channel) {
	$ws->joinVoiceChannel($channel)->then(function (VoiceClient $vc) use ($channel) {
		echo 'Joined '.$channel->name.' voice channel'.PHP_EOL; // Runs fine

		$vc->setFrameSize(40)->then(function () use ($vc) {
			$vc->playFile('./audio/test.mp3')->then(function () {
                		echo 'Playing test on '.$channel->name.' voice channel.\r\n'; // Not being executed
           		 });
		});
	});
});

I'm using a mp3 file.
In which cases playFile() wouldn't work?
Edit: Added code.

try appending a -otherwise(function($e){ echo "ERR: ".$e->getMessage(); }) to the end of the playFile section and see if the triggers some sort of error.

Oh, got an Outdated DCA error... I thought it supported mp3 files, seems like I'll need to convert it do dca

DCA updated and now everything works fine.
Thank you.

@bertZ I got the same message. I tried to update DCA and build it with go build. But then I still receive the same message:
You are using an outdated version of DCA. Please make sure you have the latest version from https://github.com/bwmarrin/dca
Could you tell me how you've updated DCA?

@GerritHoekstra I just followed the installing guide provided by Uniquoooo at https://github.com/bwmarrin/dca/blob/master/cmd/dca/README.md and it worked fine.
I did it on Windows 10 Bash (Bash on Ubuntu on Windows as some call it).


Reorder TryDeactivateSymLinkWatcher in call to prevent leak/crash

This is the same fix as #967, reordering function calls to make sure we deactive the symlinkwatcher before destroying the FolderNode. The current order results in an error when deleting folders from a project.

:+1:


OCM-11943 | chore: bump version to 1.2.47-RC1

Bumps version to cut release branch

/test coverage

/lgtm

/test coverage


Adds work view.

closes #33
Why was this change made?
To allow viewing the details for a work.
How was this change tested?
Unit, locally.
Which documentation and/or configurations were updated?
settings.yml


Training RFBNET in colab, couldn't find '/content/coco_cache/Monk_Object_Detection/example_notebooks/sample_dataset_kangaroo_Images_gt_roidb.pkl'

Hi Thanks for your nice work ! When I try to train rfbnet in colab. It gives me this error, it couldn't find the coco cache file
Monk_Object_Detection/8_pytorch_rfbnet/lib/data/coco.py in init(self, root_dir, coco_dir, set_dir, preproc, target_transform, dataset_name)
Monk_Object_Detection/8_pytorch_rfbnet/lib/data/coco.py in _load_coco_annotations(self, coco_name, indexes, _COCO)
FileNotFoundError: [Errno 2] No such file or directory: '/content/coco_cache/Monk_Object_Detection/example_notebooks/sample_dataset_kangaroo_Images_gt_roidb.pkl'

Thank you for pointing out the issue. Will resolve it very soon.

Thanks for your reply very looking foward to your response. I found this is the same for the lcoal system as well

The issue has been resolved. Please check and confirm

The issue has been resolved. Please check and confirm

thanks!. It's working!


HTTPS doesn't work under mirrored network mode.

Windows Version
Microsoft Windows [Version 10.0.22631.3593]
WSL Version
<IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>-2
Distro Version
Ubuntu 24.04
Other Software
Docker Desktop v4.30.0
Repro Steps

Set WSL to use mirrored network mode
Deploy portainer docker run -d -p 8000:8000 -p 9443:9443 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest
Access localhost:9000 and https://localhost:9443

Expected Behavior
Both http and https portainer server should work.
Actual Behavior
https servser shows error while http server page can be reached.

Diagnostic Logs
No response

WslLogs-2024-05-29_10-08-33.zip

I don't see any logs inside that zip? Did you let log collection run while wsl and the web service start up?


WPs created from Personal Notes not saved

In the latest NB after refreshing a cache the WPs created from Personal Notes are visible in the WP-Tab.
After closing and Restart from cgeo the WPs are gone.

This is essentially a duplicate of #1055, so I close here. Please follow up on the mentioned issue.

OK, thank you


Nitj: A compiler to Java code.

Introduce a new compiler nitj that takes nit code and produce Java code.
Passes ~700/950 tests from the base*.nit test suite.
Since the commit serie is rather long, I suggest to read it comit by commit.
Some features still missing (maybe later):

generics
arrays
varargs
strings

Note: There is no concrete motivation beyond this engine. Did it for the lulz.

Nice proof of concept and surprisingly short. There is still a few comments relating to C code instead of Java, but they are still understandable.
Unless you plan on completing the missing features right away, you should probably document in a readme or in the module doc what works and what does not.
+1

Nice step towards Nit Enterprise Edition.
:+1:

Impressive. I can accept as is (once tests pass). +1

document the data structures used in Java to represent things in the documentation of the module
add a todo to remember factorize code between java_compiler and abstract_compiler

@xymus: Removed weird indent. Can you precise where the superfluous curly brackets are?
@privat reroll for:

comments on RuntimeModel and generated structures
indent

TODO

factorize default value (I do not plan to do this in this PR because there is a lot of work to do/redo about default primitive values
use HashSets, maybe, later
is_sys please precise where
the line that seems buggy. Weirdly it passe all related tests. Need to investigate (can be merged before).


v0.1.2

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/1523
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if Julia TagBot is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.2 -m "<description of version>" 9f6036cbccf2f1cd5a7297a31a27b7f1aa192ca9
git push origin v0.1.2

Can I edit files in Google Drive using this? Like a Google Sheet Document?

Not at the moment. Those particular APIs would need to be added.

Can I edit files in Google Drive using this? Like a Google Sheet Document?

Not at the moment. Those particular APIs would need to be added.

Do you have any plans to?
It would be really great.

@RoyiAvital No plans. I've moved on. Feel free to write them :smiley:

Did you leave the Julia eco system? Or just left Google Cloud?

I set both Julia and GCP aside for time being. Both are good and fine. My needs and time have just shifted in different directions. Currently focusing on the Rust ecosystem. I may eventually implement more machine learning with Julia, after re-evaluating machine learning advancements. The GCP account is ready and waiting. :grinning:
What are you goals and interests? That helps me focus on one thing or another.

I just needed something to download a certain Google Sheet, process it as a table and update results back to the online file.
I see many people shift to Rust.
It seems to have a lot of inertia this days.


Does cudnn Batchnorm support 5D input

I am trying to use 3D convolution as well as 3D batch norm, which means their input are 5D tensors. There is nothing wrong with conv layer, but I got an error when using cudnn batch norm. I heard 5D is supported by cudnn bn, but why I got this error? Something wrong with the mxnet implementation?

what's error msg you got?
batchnorm should works for general N-D input: http://ec2-54-152-159-175.compute-1.amazonaws.com///api/python/symbol.html#mxnet.symbol.BatchNorm

I got:
[10:22:55] /home/a1689670/mxnet/dmlc-core/include/dmlc/./logging.h:300: [10:22:55] src/engine/./threaded_engine.h:336: [10:22:55] src/operator/./cudnn_batch_norm-inl.h:59: Check failed: in_data[cudnnbatchnorm::kData].ndim() <= 4 (5 vs. 4)
@mli

Any help please? @piiswrong @tqchen @pluskid

It seems the current cudnn batchnorm implementation requires the input dimension <= 4. Updates are needed to either call cuDNN's general ND tensor API or use get_with_shape to fake 4D tensors out of ND.
https://github.com/dmlc/mxnet/blob/master/src/operator/cudnn_batch_norm-inl.h#L62

For now a workaround is to reshape the 5D data to 4D, perform the BatchNorm and then reshape back to 5D.
You may use the following code:
def batchnorm_5d(data, height, width, name, fix_gamma, eps, momentum):
    data = mx.symbol.reshape(data, shape=(0, 0, -1, width))

    data = BatchNorm(
        data,
        name=name,
        fix_gamma=fix_gamma,
        eps=eps,
        momentum=momentum)

    return mx.symbol.reshape(data, shape=(0, 0, -1, height, width))

This should be equivalent to the way cudnn performs the 5D (spatial) BatchNorm:

CUDNN_BATCHNORM_SPATIAL: Normalization is performed over N+spatial
dimensions. This mode is intended for use after
convolutional layers (where spatial invariance
is desired). In this mode bnBias, bnScale tensor
dimensions are 1xCx1x1.

As cudnn throws CUDNN_STATUS_BAD_PARAM if:

bnScaleBiasMeanVarDesc dimensions are not
1xC(x1)x1x1 for spatial or 1xC(xD)xHxW for
per-activation mode (parenthesis for 5D).

(See the user manual at https://developer.nvidia.com/cudnn )

Has anyone made any progress on this?
The workaround posted will make network creation quite rigid as you need to know the input size and compute the current input size while constructing the network.
Is there any way to compute height and width dynamically so they don't need to be passed in? Also is there any issue with performance with the above workaround?

You can refer to the formulas in https://arxiv.org/abs/1603.07285 and write some helper functions to calculate the shape automatically. Alternatively you can call the infer_shape method on a symbol, passing the input data shapes as keyword arguments. It will return a list of 3 elements, with the output shape being the second one. Based on that you know the arguments for the 5d_batchnorm function. This could be done automatically when constructing your symbols.

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!


Fully deprecate werkzeug.script

werkzeug.string has been deprecated since 2011 and 0.12 now shows huge warnings, I think it's time to remove it.

Can I take this? It looks like some of the example still use ‘from werkzeug import script‘. What should be use instead?

Click is recommended.

Where should I add it to requirements?

No, this would be about removing the module and - if it's referenced in the docs - mention click there instead. There is no need/reason to make click a hard requirement of werkzeug.

I think Antonio is talking about the examples folder in the Werkzeug directory.
Since that one doesn't have a setup.py for each example, I think it's fine to
just not define the dependency anywhere. Perhaps a short notice in
examples/README would suffice.
On Mon, Mar 27, 2017 at 03:23:20PM -0700, Adrian wrote:

No, this would be about removing the module and - if it's referenced in the docs - mention click there instead. There is no need/reason to make click a hard requirement of werkzeug.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/pallets/werkzeug/issues/1079#issuecomment-289604060

Yes, I'm sorry, I was talking about the examples, I didn't mention it. I'll work on this :grin:

@untitaker I think this one can be closed now :ok_hand:


#11919: Enable fast dispatch build-and-unit-tests

Ticket
Link to Github Issue
Problem description
Provide context for the problem.
What's changed
Enabled ttnn and eager tests in fast dispatch. Will selectively choose tests to enable/disable after running some tests
Checklist

[ ] Post commit CI passes
[ ] Blackhole Post commit (if applicable) https://github.com/tenstorrent/tt-metal/actions/runs/10742741287
[ ] Model regression CI testing passes (if applicable)
[ ] New/Existing tests provide coverage for changes

@nvelickovicTT fyi


output command stdout/stderr to specified file

provide option to redirect command's stdout and stderr to a give file
https://rust-lang-nursery.github.io/rust-cookbook/os/external.html

Why not just use the OS-level redirect?


Help on setting intervals on Month Scheduler

Question:
I can't find a clear way on the documentation on how to do the following:
On Month Scheduler mode:
Set programmatically and asynchronously an event (could be one day duration or more) and see it being reflected on the calendar (the corresponding days must be styled accordingly).
A common use case would be:

I click on one of the resource entities (a employee) buttons on the left of the calendar;
A dialog with a form is fired, with event information to be filled;
After submitting the form, the calendar is updated and styled according to the data passed on the submitted form.

I'm a bit lost on which methods and parameters should I use to achieve this behavior, i.e., change asynchronously the appearance of the calendar (in other words, styling and adding text to selected days), when I'm in Month Scheduler mode.

@jonDel Fork this codepen, add what you can and when done ping me and I'll see what help I can provide: https://codepen.io/Hawkeye64/pen/RwwwKQL

Thanks @hawkeye64,
I provided a bit messy pen, but I think from this you can get the overall idea:
https://codepen.io/Dellagostin/pen/oNXdpwL
Obviously, the dialog doesn't do anything right now. The idea is that, after you pick start/end dates for the event, it gets reflected on the calendar, just like the On duty events that are set up initially in the pen.

@jonDel You can do that. Right now, you do not have a few functions referenced in your html code. Like the onSubmit. You need to keep track of who was selected so you can apply the results to them. Because you have hard-coded the data in a function it would be difficult to add new data. You need to split out the data into an object.
props: {
  data: {
    {
      Jon: {
        days: [1, 2, 3, 4, 5, 6, 7, 8]
      }
      Mary: {
        days: [24, 25, 26, 27, 28, 29, 30, 31]
      }
    }
  }
}

Then you can add or remove to this data according to what is captured by your dialog.
And, in your getOnDuty (day, name) method, you can:
getOnDuty (day, name) {
  if (this.data[name]) {
    return this.data[name].days.includes(day.day))
  }
  return false
}

keeps your code short and clean.
So, if you need to add or remove days for Jon. Make sure you use splice so Vue can make the data reactive.

@hawkeye64 , that was just a quick and dirty example for illustrating my point and far away from the final result I want to get. The data manipulation is not what is bugging me, but how to reflect that in the styling of the calendar.
Let's take the following example (I know it is in month view, but the point holds for the scheduler month view):

Now, I want to extend the vacation event till day 31. I could add a dialog with a form and some inputs, get the result and alter some data. That is OK. However, I want to see that vacation event get reflected in the calendar either, I.E, that purple stripe extended and covering day 30 and 31 too.

The length would be something like this:
(100 / days_in_month) * days_of_event

You can get the start and end days of the month in the @change event

But how can I make a redraw in the calendar asynchronously? The only hacky way I figured out is to call the __checkChange() method.

You should be using the slots, unless I am just not understanding what you need.

If your slot data changes, Vue will redraw the calendar.

Of course! My bad, I am still learning vuejs, that must be obvious. The "resources" is hooked to the slot, and the slot will react to changes on "resource". Thanks very much for your patience.


fix: add missing state reset

:tada: This PR is included in version 0.20.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Latest version 0.0.9 does not include the axios upgrade

The PR: https://github.com/yovanoc/mailsac/pull/16 was merged on Apr. 22 but a new version of the dependency was not deployed yet.
Server-Side Request Forgery
Overview
The axios NPM package before 0.21.1 contains a Server-Side Request Forgery (SSRF) vulnerability where an attacker is able to bypass a proxy by providing a URL that responds with a redirect to a restricted host or IP address.

I'm currently doing some cleaning on the repo, I will publish this soon, sorry for the delay


This still work with ah.nl?

Yes, it's still running on a raspberry pi as we speak. Feel free to open a new issue if you face any issues.
Note that the Albert Heijn API is used for barcode (product) lookup, and a local shopping list will be kept. This app doesn't add those product to your personal AH shopping cart (yet).


Request: 1-In-1-Out feature

Hi @lelegard ,
The current functionalities of the TSDuck are impresive. I would like to thank you for the scale of this project. I comment this because new functionalities can open new uses of it.
I'm not sure if this can be done now or not. But I want to use the tsp for the purpose that I will describe now. I prefer to explain the precise use case as I feel you will understand it faster:

Inside another process I want to process just one pid (for example, the 0x16 EIT pid, or the 0x20 TOT/TDT, or whatever) with the tsp tool. So the idea is to capture every single packet inside the process and pass a copy of it to the tsp process over a pipe. Then receive a new TS packet over the pipe and replace the original packet with the received one.

However, what's the problem here? The problem is that the tsp process is oriented to realtime or off-line chunk processing. So a group of packets are processed at some time. But the idea here it's a running mode with 1-In-1-Out TS processing mode. This mode will grant that for every single input packet a new single packet will output. For sure, some time for the processing will be consumed; but this will a problem for the master process, and not for the tsp slave process. Futhermore, I'm not requesting to "advance in time", as it's impossible. So when the processing will starts then the output packets will be NULL packets until the internal processor can output real packets. This doesn't needs to be a real problem.
What do you think of this idea? Is it feasible? Or can it be done now with the current implementation?

Everything is possible. This is just a matter of development. In this case, the packet processing is done in a separate process. I assume that you develop this custom application. Why not developing it directly as a plugin? If you need to communicate with another application of yours, then this is your private protocol.

Hi @lelegard ,
Thank you for the response.
Regarding the App, is not my development. I want to add to some opensource tools (for example, SAT>IP servers and DLNA servers) the option to rewrite/add some PSI tables. In this case it's quite simple to open a child process for an external processor and send to it each single TS packet and receive a new one. This will work even if the main process is writen in a different language (not C/C++). And especially when it comes to scripts that use different tools.
I hope you will understand the target of the idea and the benefit of such operation mode.
In any case, thank you for your support!
Regards.

Hi @lelegard ,
Two additional comments to be more clear:

"and send to it each single TS packet" in this phrase "each" refers to each packet of a given pid. Not each TS packet of the stream. The objective is to only process low bit rate pids, such EIT for example.
"the packet processing is done in a separate process" regarding this, take note that the processing is done inside the master process. The "external processor" will be used only for the small part of specific pids.

In any case, let me ask you directly: Do you have a better idea about this objective?

Hi @lelegard ,
Any idea to aproximate to this behaviour with current implementation?
Perhaps adjusting some buffer sizes.
I hope you want to comment about it.
Thank you!

Duplicate of #553, now fixed


futher simplify remove and shift_back operations in hashmap

simplify the remove operation
simplify shift_back
tweak

Pull Request Test Coverage Report for Build 1568
Details

2 of 3 (66.67%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 89.094%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

hashmap/hashmap.mbt
2
3
66.67%

Totals

Change from base Build 1567:
0.0%

Covered Lines:
2557

Relevant Lines:
2870

💛  - Coveralls

@coderabbitai review

The 50% load factor will result in half of the space being wasted, maybe a higher load factor is better?

The 50% load factor will result in half of the space being wasted, maybe a higher load factor is better?

I am open to change the load factor later, rationale why I prefer 0.5 instead of higher values:

when people use hashtbl, they want the optimal performance, so it is okay to exchange space for speed
the wasted the space is not that bad, since the array only holds a pointer(not the full content)


Docker startup fails with Rancher agent set to "restart: always"

Rancher Version: 0.56.1
Docker Version: 1.9.1, build 7206621
OS: Fedora 23
Steps to Reproduce: Install Fedora 23, expand volume group to get some free space in lvm for docker, start docker (it will create a thin pool in the root volume group), install Rancher agent, stop Docker, start Docker
Results: Docker fails to get startet. I can't pin it down to one single error message, but overall the Docker service fails to come up and isn't able to start anymore until restarting the whole machine, even if the rancher agent isn't set to "restart: always". If I start the rancher-agent manually, a few seconds after the docker service is done starting, everything works without any errors.
Expected: Beeing able to start Docker with rancher-agent set to "restart: always".
I can provide some log files, if needed, but I'm not sure what to include and it should happen on every Fedora 23 host anyway, since i've tried it in different environments many, many times with fresh installations of Rancher and Fedora, it never worked. Not even once. Some other Docker containers I've used have no such problem.

You shouldn't set --restart=always on the rancher/agent command as we actually end up starting 3 containers to get the rancher-agent container running. There is the first container which is a docker named container and can be deleted if you wanted. The second container which is a data container for the rancher/agent that is stopped and named rancher-agent-state The last container which is the one that stays running already has restart: always set and will be called rancher-agent.
If you don't add in the --restart=always, do you have the same issues?

I didn't add anything. I've just copied the command line like it's shown in Rancher. But you've said exactly the same as I did. The rancher-agent is started with restart: always. And that one is causing the problems. I have to change that policy manually, by editing the json file of that container. Otherwise I'm not able to start Docker anymore.

@slaschinski have you tried this on different OS to see if it's an issue only on Fedora 23? We don't face these issues on other OSes. Could you see if this is an issue with Fedora 24?

@slaschinski if this is still an issue, please let me know.

@deniseschannon Shouldn't the rancher agent be on restart always too (I get it about the 3 commands but it should handle the case) ? Because when you reboot, the host won't be recognized by rancher server.
On a blogpost, someone proposes to add an autostart unix service.
http://anlek.com/2016/07/rancher-agent-start-on-reboot/
What is the best practice ? Thx.

@wadjeroudi the container that is named rancher-agent, the final 1 of the 3 start and the one that matters, should and typically does have an always restart policy. If you are seeing something different, its a bug.
docker inspect rancher-agent | grep -i -A3 RestartPolicy
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },

@cjellick Thanks. Don't why but before I was on v1.2.0-pre3, it didn't autostart, it does now. Sorry.