[Secret Manager] Issues with secrets that depend on entries in different manifests

Secret manager is able to handle the case when a secret that depends on another secret gets cycled, as long as the secrets are in the same manifest, but the scenario breaks when the source secret is in a different manifest.
Take this scenario for example:

The helixosautostaging-storage-connection-string-admin secret is referenced in the helixstagingkv.yaml manifest in helix-machines.
This secrets has a reference to the execution-storage-connection-key secret in the HelixStagingService key vault. This secret is managed in the helixstagingservicekv.yaml manifest in the helix-service repository.
When the execution-storage-connection-key secret gets cycled from the helix-service secret sync build, it regenerates the dependent secrets that it knows about, which are the ones in the current manifest, but it won't cycle the helixosautostaging-storage-connection-string-admin secret from helix-machines.
Helix-machines is now using an out of date connection string until we forcibly tell the secret to update.

We need some way to handle these cases.

Release Note Category

[ ] Feature changes/additions
[ ] Bug fixes
[ ] Internal Infrastructure Improvements

Release Note Description

Some options:

Consolidate and enforce that all secrets referenced in a manifest are also managed in that same manifest. This would result in a refactor of our manifests and vault usage.
Add some logic to secret manager so that when it finds a referenced secret that has been cycled recently, it will also force update the dependent secret. For the example, if the connection string was last updated on 11/10 but the referenced storage key was updated on 11/13, secret-manager should force an update of the connection string the next time the synchronization build runs. This won't get rid of the race, but will make it so running the synchronization builds will eventually get us to a consistent state.
Teach secret-manager to search for other manifests outside of the current manifest/repo where it's running during synchronization. This gets rid of the race, but it means that secret-manager needs to know about all the manifests before running, and in that case would it just make sense to consolidate all our secrets in the same place?

in that case would it just make sense to consolidate all our secrets in the same place?

do you mean "use a single manifest" or "use a single key vault (a lower number at least)" or both❔
how difficult would it be to focus on the "single manifest" side of things i.e., does secret-manager solidly support multiple key vaults referenced from the manifest it's given❔
from a usability PoV, our manifest and key vault counts are high. feel like anything we do to lower either or both will help secret-manager and us

do you mean "use a single manifest" or "use a single key vault (a lower number at least)" or both❔

I'm good with either. Consolidating our key vaults seems like a much larger work item though, which is part of what we want to achieve with the pending "secret sweep and clean" epic.

how difficult would it be to focus on the "single manifest" side of things i.e., does secret-manager solidly support multiple key vaults referenced from the manifest it's given❔

My reading of the code says it does not support this yet.

from a usability PoV, our manifest and key vault counts are high. feel like anything we do to lower either or both will help secret-manager and us

Definitely. I'm not sure if these are the ways we want to deal with this issue. I'll keep thinking of more ideas.


license question

Hello,
Are there specific reasons for this project to be using GPL?
Would be a lot more contributor friendly if we go for something like MIT or Apache-2.0.
Love this project - totally useful for what I need.  Would love to understand more about its long term plans.

changed the license to Apache-2.0.

That was fast! Awesome.


Django AttributeError

Just ran pip install google-apis-client-generator then tried to run a couple commands for the generate_library command and got this traceback every time. Any suggestions?
Traceback (most recent call last):
  File "/usr/local/bin/generate_library", line 9, in <module>
    load_entry_point('google-apis-client-generator==1.4.2', 'console_scripts', 'generate_library')()
  File "/Library/Python/2.7/site-packages/googleapis/codegen/script_stubs.py", line 9, in RunGenerateLibrary
    from googleapis.codegen import generate_library
  File "/Library/Python/2.7/site-packages/googleapis/codegen/generate_library.py", line 35, in <module>
    from googleapis.codegen import generator_lookup
  File "/Library/Python/2.7/site-packages/googleapis/codegen/generator_lookup.py", line 10, in <module>
    from googleapis.codegen import csharp_generator
  File "/Library/Python/2.7/site-packages/googleapis/codegen/csharp_generator.py", line 23, in <module>
    from googleapis.codegen import api
  File "/Library/Python/2.7/site-packages/googleapis/codegen/api.py", line 39, in <module>
    from googleapis.codegen import data_types
  File "/Library/Python/2.7/site-packages/googleapis/codegen/data_types.py", line 24, in <module>
    from googleapis.codegen import template_objects
  File "/Library/Python/2.7/site-packages/googleapis/codegen/template_objects.py", line 27, in <module>
    from googleapis.codegen.django_helpers import MarkSafe
  File "/Library/Python/2.7/site-packages/googleapis/codegen/django_helpers.py", line 50, in <module>
    django_template.add_to_builtins(
AttributeError: 'module' object has no attribute 'add_to_builtins'

Downgrade django to 1.7.  See https://github.com/google/apis-client-generator/issues/13

Thanks.


Set the assets URL during e2e tests

Codecov Report

Merging #234 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #234   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          53     53
  Lines        1208   1212    +4
  Branches      174    175    +1
=====================================
+ Hits         1208   1212    +4

Flag
Coverage Δ

#builder
100% <100%> (ø)
:arrow_up:

#runtime
100% <ø> (ø)
:arrow_up:

#srcapp
100% <ø> (ø)
:arrow_up:

Impacted Files
Coverage Δ

cli/build.js
100% <100%> (ø)
:arrow_up:

lib/assets-processor.js
100% <100%> (ø)
:arrow_up:

cli/e2e.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f222a35...98f4924. Read the comment docs.

@Blackbaud-PaulCrowder The feature works great; tested all CLI commands on various SPAs.
I didn't see a unit test where we're passing in a relative value to getAssetsUrl(). Is that important?
Something like...
lib-assets-processor.spec.js
const url = processor.getAssetsUrl(
    {
      runtime: {
        app: {
          base: 'test'
        }
      }
    },
    'https://example.com/',
    '../' // <-- passing in a relative value...
  );


Added parameters "Devices" and "SkipSerialNumber to DiskIO plugin.

"Devices" can be used to specify storage devices on which stats should be reported.
"SkipSerialNumber" can be used to omit the device serial number.
Added tests to verify the new parameters.

thank you @subhachandrachandra!


Add missing part in translation key

A new level has been added to this translation as follows: https://github.com/cortezaproject/corteza-locale/blob/2021.9.x/src/en/corteza-webapp-compose/module.yaml#L89

Thank you!


Add more idiomatic approach to provide certificates to installer

This PR adds a way to provide TLS certificates to the installer without needing them to be on the file system. This is useful for  a third party utilities (e.g. terraform-provider-helm) in cases when certificate data is not available on the local filesystem (e.g. vault).

CLA signed

Hey! Sorry for letting this one fall through the cracks. The feature and tests here look great. If you wouldn't mind rebasing this PR I can take another look at getting this merged. Thanks!

ping

Closing this PR due to inactivity. As per the announcement on the mailing list, Helm 2.15.0 is the last release where we will be merging new features for Helm 2. This feature doens't look like it needs to be ported over to Helm 3 either since it deals with Tiller, which was removed.
Thanks!


docker-compose multirole sometimes stuck on boot

The issue is about docker-compose-multirole.yaml example in temporalio/docker-compose repo. I am posting this here because temporalio/docker-compose does not have an issue page.
Expected Behavior
When I run docker compose -f docker-compose-multirole.yaml up the whole multirole cluster is up and running normally.
Actual Behavior
When I run docker compose -f docker-compose-multirole.yaml up, temporal-history is sometimes stuck at "Waiting for Temporal server to start...", unable to reach the frontend service via nginx. The whole cluster does not seem to be in sync because I cannot connect to the UI service as well.
This does not always happen. So you need to try this multiple times. To me it feels like the chance is about 20%, especially when you stop docker-compose after running the service for a long time.
Restarting temporal-frontend or temporal-frontend2 sometimes make the cluster to the normal state, but not always.
Steps to Reproduce the Problem

git clone https://github.com/temporalio/docker-compose
cd docker-compose
docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
docker compose -f docker-compose-multirole.yaml up
If it works well, press control+C to stop the docker-compose project and wait for a moment. Then run the previous command again until you come across the problem.

Specifications

Version: 1.22.4 (it happens in the previous versions too)
Platform: macOS Sonoma 14.3.1 (newest version)
Docker Desktop: 4.27.2 (newest version)

Here is the following log:
temporal-nginx          | <IP_ADDRESS> - - [27/Feb/2024:13:03:16 +0000] "POST /grpc.health.v1.Health/Check HTTP/2.0" 204 0 "-" "grpc-go/1.59.0"
temporal-nginx          | 2024/02/27 13:03:16 [error] 29#29: *73 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7237", host: "temporal-nginx:7233"
temporal-nginx          | 2024/02/27 13:03:16 [error] 29#29: *73 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7236", host: "temporal-nginx:7233"
temporal-history        | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: unexpected HTTP status code received from server: 204 (No Content); malformed header: missing HTTP content-type
temporal-history        | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
temporal-history        | Waiting for Temporal server to start...
temporal-nginx          | 2024/02/27 13:03:18 [error] 29#29: *76 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7236", host: "temporal-nginx:7233"
temporal-nginx          | 2024/02/27 13:03:18 [error] 29#29: *76 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7237", host: "temporal-nginx:7233"
temporal-nginx          | <IP_ADDRESS> - - [27/Feb/2024:13:03:18 +0000] "POST /grpc.health.v1.Health/Check HTTP/2.0" 204 0 "-" "grpc-go/1.59.0"
temporal-history        | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: unexpected HTTP status code received from server: 204 (No Content); malformed header: missing HTTP content-type
temporal-history        | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
temporal-history        | Waiting for Temporal server to start...
temporal-history        | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: unexpected HTTP status code received from server: 204 (No Content); malformed header: missing HTTP content-type
temporal-history        | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
temporal-nginx          | 2024/02/27 13:03:19 [error] 29#29: *79 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7237", host: "temporal-nginx:7233"
temporal-nginx          | 2024/02/27 13:03:19 [error] 29#29: *79 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7236", host: "temporal-nginx:7233"
temporal-nginx          | <IP_ADDRESS> - - [27/Feb/2024:13:03:19 +0000] "POST /grpc.health.v1.Health/Check HTTP/2.0" 204 0 "-" "grpc-go/1.59.0"
temporal-history        | Waiting for Temporal server to start...
temporal-history        | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: unexpected HTTP status code received from server: 204 (No Content); malformed header: missing HTTP content-type
temporal-history        | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
temporal-nginx          | 2024/02/27 13:03:20 [error] 29#29: *82 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7236", host: "temporal-nginx:7233"
temporal-nginx          | 2024/02/27 13:03:20 [error] 29#29: *82 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: , request: "POST /grpc.health.v1.Health/Check HTTP/2.0", upstream: "grpc://<IP_ADDRESS>:7237", host: "temporal-nginx:7233"
temporal-nginx          | <IP_ADDRESS> - - [27/Feb/2024:13:03:20 +0000] "POST /grpc.health.v1.Health/Check HTTP/2.0" 204 0 "-" "grpc-go/1.59.0"
temporal-history        | Waiting for Temporal server to start...

Setting TEMPORAL_CLI_SHOW_STACKS=true on temporal-history does not help much:
temporal-history        | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: unexpected HTTP status code received from server: 204 (No Content); malformed header: missing HTTP content-type
temporal-history        | Stack trace:
temporal-history        | goroutine 1 [running]:
temporal-history        | runtime/debug.Stack()
temporal-history        |       /opt/hostedtoolcache/go/1.20.11/x64/src/runtime/debug/stack.go:24 +0x64
temporal-history        | runtime/debug.PrintStack()
temporal-history        |       /opt/hostedtoolcache/go/1.20.11/x64/src/runtime/debug/stack.go:16 +0x1c
temporal-history        | github.com/temporalio/cli/app.HandleError(0x40003df438?, {0x2b492a0, 0x40007481e0})
temporal-history        |       /home/runner/work/cli/cli/app/app.go:73 +0x134
temporal-history        | github.com/urfave/cli/v2.(*App).handleExitCoder(0x40009d91e0?, 0x400014de00?, {0x2b492a0?, 0x40007481e0?})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/app.go:452 +0x3c
temporal-history        | github.com/urfave/cli/v2.(*Command).Run(0x40009d91e0, 0x40003eb480, {0x4000b10200, 0x1, 0x1})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/command.go:276 +0x768
temporal-history        | github.com/urfave/cli/v2.(*Command).Run(0x406ef60, 0x40003eb340, {0x4000b01be0, 0x2, 0x2})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/command.go:267 +0x948
temporal-history        | github.com/urfave/cli/v2.(*Command).Run(0x406f900, 0x40003eb280, {0x40009c92f0, 0x3, 0x3})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/command.go:267 +0x948
temporal-history        | github.com/urfave/cli/v2.(*Command).Run(0x40009dab00, 0x40003eb140, {0x400004c0c0, 0x4, 0x4})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/command.go:267 +0x948
temporal-history        | github.com/urfave/cli/v2.(*App).RunContext(0x40008da780, {0x2b6a678?, 0x4000058068}, {0x400004c0c0, 0x4, 0x4})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/app.go:332 +0x568
temporal-history        | github.com/urfave/cli/v2.(*App).Run(0x60?, {0x400004c0c0?, 0x400007c768?, 0x49b54?})
temporal-history        |       /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.7/app.go:309 +0x40
temporal-history        | main.main()
temporal-history        |       /home/runner/work/cli/cli/cmd/temporal/main.go:14 +0x38
temporal-history        | Waiting for Temporal server to start...

Temporal docker-compose-multirole.yaml Issue
This issue is about the docker-compose-multirole.yaml example in the temporalio/docker-compose repository. Here's a breakdown of the issue and potential solutions:
Problem:

Symptom: The temporal-history service gets stuck at "Waiting for Temporal server to start..." and fails to connect to the temporal-frontend service via Nginx. The whole cluster seems out of sync.
Frequency: This issue occurs intermittently, with a reported chance of 20%, especially after stopping and restarting the Docker Compose project.
Log: The logs show connect() failed (111: Connection refused) errors from Nginx and unable to health check errors from the history service.
Version: 1.22.4 (and potentially earlier versions)
Platform: macOS Sonoma 14.3.1 (newest version)
Docker Desktop: 4.27.2 (newest version)

Potential Causes:

Network Issues: The temporal-history service might be unable to connect to the temporal-frontend service due to network connectivity issues or DNS resolution problems.
Race Condition: The temporal-history service might be starting before the temporal-frontend service is fully initialized and listening on the specified port, leading to a connection refusal.
Service Startup Order: Docker Compose's service startup order might not guarantee that the temporal-frontend service is ready before the temporal-history service attempts to connect.
Nginx Configuration: The Nginx configuration might have issues proxying gRPC traffic, leading to connectivity problems for the temporal-history service.
Temporal Server Startup: The Temporal Server might be starting up slowly or encountering issues during startup, causing delays in the history service connecting to the frontend.

Troubleshooting Steps:

Verify Network Connectivity:

Check network connectivity between the temporal-history container and the temporal-frontend container. Use ping or docker exec to run nslookup within the containers to validate DNS resolution.

Adjust Service Startup Order:

In the docker-compose-multirole.yaml file, adjust the service startup order using the depends_on property to ensure the temporal-frontend service starts before the temporal-history service.
Example:services:
  temporal-frontend:
    # ...
    depends_on:
      - temporal-nginx
  temporal-history:
    # ...
    depends_on:
      - temporal-frontend

Increase docker-compose up Timeouts:

If there are issues with service startup delays, increase the timeout value for the docker-compose up command to allow for more time for services to start.
Example:docker compose -f docker-compose-multirole.yaml up -t 300  # Increase the timeout to 5 minutes

Check Nginx Configuration:

Review the docker-compose-multirole.yaml file for any errors in the Nginx configuration. Check the upstream directive in the temporal server block to ensure it is correctly configured and pointing to the temporal-frontend service.
Make sure your Nginx configuration has HTTP/2 enabled for gRPC traffic.

Increase Temporal Server Startup Time:

Review the Temporal Server configuration ( development.yaml file) to ensure there are no delays in the server's startup process. If necessary, adjust values such as the maxJoinDuration or rpcAddress to improve connectivity and reduce startup delays.

Use grpc-health-probe:

Use the grpc-health-probe tool from the grpc-ecosystem to check if the frontend service is healthy and reachable by the history service.

Additional Tips:

Logging: Enable verbose logging for the temporal-frontend, temporal-history, and Nginx services to gain more insights into the issue.
Debugging: Use a debugger tool like VS Code to step through the code and identify potential bottlenecks or errors.

Conclusion:
This issue appears to be related to a combination of network connectivity, race conditions, and potentially service startup order issues. By carefully reviewing the configuration files and adjusting the service startup order, you should be able to mitigate this intermittent problem. If the issue persists, reach out to the Temporal community or support team for assistance.

Thanks for reporting this issue! The logs show that temporal-history is unable to health check the temporal.api.workflowservice.v1.WorkflowService service. This is most likely due to the temporal-frontend service not being available at the expected address, causing the connection to be refused.
The logs show the Frontend Service is running on <IP_ADDRESS>:7237 and <IP_ADDRESS>:7236 while the History service is trying to reach the Frontend Service on <IP_ADDRESS>:7237 and <IP_ADDRESS>:7236.
You may need to review your docker-compose-multirole.yaml to ensure these addresses are properly configured.
To help debug further, could you please tell me:

How did you configure your docker-compose-multirole.yaml file?
What are the IP addresses of your docker containers and their respective ports for temporal-frontend, temporal-frontend2, and temporal-history?
Are you using any custom network settings?

Once I have this information I can provide a more tailored solution for your issue.

Thank you for reporting this issue. We are aware that the docker-compose-multirole.yaml example sometimes experiences a race condition.
The issue occurs when Temporal History service is unable to reach the Temporal Frontend Service via Nginx. This can happen due to a race condition in the startup process.
Here are some troubleshooting steps you can try:

Increase Docker memory: Ensure that Docker has enough memory allocated to avoid resource constraints.
Restart containers: Restart the Temporal History service and Temporal Frontend service containers to ensure that they're properly synchronized.
Increase Nginx worker connections: Adjust the worker_connections setting in Nginx's configuration to increase the number of connections allowed.
Check network connectivity: Verify that the containers can communicate with each other on the appropriate ports.

If these steps don't resolve the issue, please provide more information about your environment, including:

Docker version
Docker Compose version
Host operating system
Network configuration

We can then provide more specific guidance.
For further reference, please refer to Temporal's docker-compose documentation.

Thanks for reporting the issue.
It seems like the issue is due to the temporal-history service failing to connect to the frontend service. This could be caused by a network issue or a problem with the docker-compose configuration.
Could you try running the docker-compose command with the --verbose flag to get more detailed logs?
In the meantime, you could try the following workarounds:

Check the docker-compose logs for the temporal-history and temporal-nginx services. Look for any errors that might indicate a network issue.
Verify that the ports 7236 and 7237 are correctly exposed in your docker-compose configuration.
Restart the temporal-frontend and temporal-frontend2 services.
Try restarting the entire docker-compose project.

If none of these workarounds solve the issue, please provide the following information:

The full docker-compose-multirole.yaml file.
The full logs for the temporal-history and temporal-nginx services.

This information will help us understand the issue and find a solution.

Thanks for reporting this issue! It appears that you are experiencing a problem where the temporal-history service gets stuck when you start your multirole cluster using the example in the temporalio/docker-compose repo.
Could you tell me what version of Temporal Server you are using? I noticed you are using Temporal CLI version 1.22.4.
Also, have you tried running the docker-compose-multirole.yaml example with a different database backend like MySQL or Cassandra?
Here are some additional tips:

Ensure the network connectivity between the temporal-history container and the temporal-frontend container is not blocked by any firewall rules or network configuration.
Check the logs of the temporal-frontend and temporal-nginx containers for any errors or warnings that might be related to the issue.
If the issue persists, you can try restarting the temporal-frontend and temporal-nginx containers.
If you are using an external database, verify the database server is running and accessible.

If the issue is still not resolved, please provide the following information to help us better understand the problem:

The complete logs of the temporal-history, temporal-frontend, and temporal-nginx containers.
The version of the database backend you are using.
The output of the docker compose -f docker-compose-multirole.yaml up command.

Let me know if you have any more questions.


Add CLI option to error on kwargs hash without splat

In Ruby 2.7, passing a keyword arg hash without ** results in a warning:
$ cat badsplat.rb
def f(x, y:)
end

someArgs = {y: 3}
f(6, someArgs)
$ ruby -W badsplat.rb
badsplat.rb:5: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
badsplat.rb:1: warning: The called method `f' is defined here

This will become an error in Ruby 3.0.
It would be nice to have Sorbet detect this. However, we don't want to make it an error by default since that would probably be too disruptive.
This PR adds an error for "legacy" kwarg hashes, hidden behind a --error-kwarg-hash-without-splat CLI option. It makes an autocorrect suggestion (e.g., changing someArgs to **someArgs in the above example) as well, but that autocorrect suggestion doesn't actually work yet, due to a bug in Sorbet.
Motivation
Should be handy for folks upgrading to 2.7 and/or looking ahead to 3.0.
Test plan
See included automated tests. Had to make them CLI tests instead of corpus tests, since I don't think there's any way to pass the flag into the corpus tests.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build result here:
→ https://go/builds/bui_IJph2igyGDSAnb


Code Formatting

Can we standardise the C++ to four spaces please? ^^

We will, even if this is just wrong. :P
I'll need to check how to properly setup per-project formatting configurations in VS. Perhaps it'd be best if I could get clang-format integration to work, so we can have a cross-platform machine-readable formatting specification.

Heresy! But yes, clang format would be good.

I'll be leaving the current code as-is, as it is to be replaced by a new C++ implementation (early peeks are available at https://github.com/luna/Dataframes/tree/apache-arrow ). clang-format causes issues for me (it looks as if it was constantly fighting with my other plugins), but VS respects .editorconfig file, so the indenting issue is solved by this.


Validate unique within region only if field has changed

Every time we save or validate a record, it ensures that the name is unique within a region.
this is only necessary when a record is created, or when the name has changed.
When the field has changed, save makes the same number of queries
When the field has not changed, save makes 1 fewer queries. (this use case the by far the most common)
/credit Taken from a comment by @NickLaMuro

Jason's https://github.com/ManageIQ/manageiq/pull/20526#discussion_r486387645 is correct; I think we want to handle this the same way as https://github.com/ManageIQ/manageiq/pull/20539 which should resolve the failure here on dialogs.


Article on Ramda Transducers

@kevinbeaty, @buzzdecafe, @kedashoe
Are any of you interested in writing up a description of transducers as they're used in Ramda?  I think the topic is a little too large for a blurb on the 0.12.0 upgrade guide (although I'll put something there regardless.)
It could be a blog post wherever you choose, or we could put it on the (mostly moribund) wiki, as you prefer.

@CrossEye I could probably whip something up.  It may be a few days though...

Great!  There are already pretty good articles on Transducers, but more information is never a bad thing.  What would clearly be helpful, though, is how Ramda is changed by the introduction of Transducers, what new possibilities they offer to users, what benefits they offer, and any potential gotchas.

Here is a great article for those who want to get a head start:
http://phuu.net/2014/08/31/csp-and-transducers.html

Yes, that's one of the most readable articles around.   Thanks!

@CrossEye Streaming Logs with Transducers and Ramda.

Excellent!  Thank you very much!

:+1:
i have an idea for a little article, if i find time i'll do that as well


Fix the actions

Roll back pimoroni-pico version to something that works. Pointing at main was probably a bad move anyway.
(Apparently I can't make a PR without finding something else broken in the process...)

Ooof, I had noticed this a few days ago and meant to look into it :grimacing: thank you!


Cache whether we've disabled intrinsic tasks

This took 0.2% of CPU, and 0.1% of allocations in a .NET Standard/.NET Framework mixed project.

I have no idea why this variable exists - I cannot see a usage anywhere internally.

tag @pilchie

@dotnet-bot test Windows_NT Build for CoreCLR please

Updated this to use our Traits class.

@AndyGerlicher can we just get rid of this escape hatch?  Would anyone ever want to disable the MSBuild or CallTarget task?  The whole entire internet hasn't heard of this one:
https://www.google.com/search?q=MSBUILDDISABLEINTRINSICMSBUILDTASK

Sounds good let's just get rid of it.


Jenayah: Watch tezadvise.com

Jenayah requests the watch of the watch_keyword tezadvise\.com. See the MS search here and the Stack Exchange search here.

tezadvise\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Machavity in Charcoal HQ


"Parameter" for example for use with a data transformation

From<EMAIL_ADDRESS>on April 14, 2009 05:43:01
Please indicate the label for the new term parameter (or possibly data transformation parameter)
Please provide any additional information below. (e.g., proposed position
in the IAO hierarchy)
See thread: JM: data transformation parameter
Original issue: http://code.google.com/p/information-artifact-ontology/issues/detail?id=17

From<EMAIL_ADDRESS>on April 14, 2009 06:55:08
I strongly think this should go into OBI, not IAO.

From<EMAIL_ADDRESS>on October 24, 2009 07:02:07
please provide a proposed definition

From<EMAIL_ADDRESS>on October 24, 2009 07:07:52
There is a long thread on this already http://sourceforge.net/tracker/index.php?func=detail&aid=2309976&group_id=177891&atid=886178 The conclusion was that parameter was a type of input, specified in the
'specification' class for the process.

From<EMAIL_ADDRESS>on November 27, 2009 08:41:08
I withdraw my request for parameter
Status: WontFix


Unexpected token (302 Page)

Kinder @ Ferrero

109 child brand product(s) found
24 product(s) to check
22.02% products ratio to check

/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/json/common.rb:155:in `parse': 757: unexpected token at '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> (JSON::ParserError)

<title>302 Found</title>

Found

The document has moved here.

'
	from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/json/common.rb:155:in `parse'
	from /Library/Ruby/Gems/2.0.0/gems/openfoodfacts-0.3.0/lib/openfoodfacts/product.rb:152:in `update'
	from openfoodfacts-corrector.rb:77:in `block (3 levels) in run!'
	from openfoodfacts-corrector.rb:74:in `map'
	from openfoodfacts-corrector.rb:74:in `block (2 levels) in run!'
	from openfoodfacts-corrector.rb:71:in `each'
	from openfoodfacts-corrector.rb:71:in `block in run!'
	from openfoodfacts-corrector.rb:70:in `each'
	from openfoodfacts-corrector.rb:70:in `run!'
	from openfoodfacts-corrector.rb:96:in `'

I didn't succeed to reproduce your error, what command did you tipped (without password)? with which Ruby version? Thanks

I finally succeed to reproduce without the default debug mode.
The problem was coming from endpoint subdomain changes for updates and some other errors from the openfoodfacts gem. Now it works again with the version >= 0.3.1.


Incorporate error into handicap adjustments

Change get_handicap_adjustment to return a tuple with both rating and error, where the error comes from:

±0.5 for rank differences that aren't (almost) the same as the number of handicap stones.
±1 for rank difference of 9 (scaled up/down) for large/extreme handicaps.

Also change Glicko2Entry.copy to add incorporate the error into the rating deviation using root-of-sum-of-squares, which is the usual way to combine standard errors.

Did a sniff test, but not really any analysis of whether the initial defaults are the numbers we actually want.
Let me know if you'd rather default to "off" for these until some analysis is done.

Let me know if you'd rather default to "off" for these until some analysis is done.

I mean, either way there will be analysis to do later, just a matter of the current default on the branch.

What are we looking to improve with this? Comparing the output from with and without this change, the end results are pretty similar, at least on the surface. That leads me to think it's probably fine to merge, but I'm a little hesitant to just click merge without building a bit more intuition about how this is improving results and what, if any, tradeoffs we're making.

From the https://github.com/dexonsmith/goratings/blob/document-ratings-v6/RatingsV6.md the added clarification for the purpose of this is:
Increase effective rating deviation of opponent for larger handicaps and strange komi values.

Reflects higher uncertainty that opponents' rating is correct, the more the board diverges fom "normal".
Reduces impact of high handicap games on ratings, enabling a policy decision to allow rated 19x19 games with 10+ stones.
Reduces impact of reverse-komi games on ratings, enabling a policy decision to allow handicap via reverse komi.

Marking this as draft for the summer so it's not on my "to check on and merge" list until we start revisiting stuff in the fall


Add support for custom validators

And implements the new validators related to ndarray.
[This is part of the ongoing work to support ASDF in jwst_lib.models]

Related to spacetelescope/asdf-standard#85

Incorporates #125.


transcoding with ffmpeg.js (wasm)

Is it possible to use ffmpeg.js on the frontend side and transcode, then send pre-processed video to FFmpeg for relay (-c copy) for the final destination?
This way it won't consume much server resources and also paves the way for sending video directly from a web browser via websockify (it's just an idea) or node app (similar approach)

no it is not - you cannot open a RTMP socket from the browser (yet...)


Define Heading levels in Settings App

Summary of the Pull Request
What is this about?
This PR defines heading levels in the settings app to make scanning more efficient and accessible.
PR Checklist

[x] Applies to #7033
[x] CLA signed. If not, go over here and sign the CLA
[ ] Tests added/passed
[ ] Requires documentation to be updated
[ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Info on Pull Request
What does this include?

The narrator supports this functionality when a visually impaired user would like to skim through the application by reading only it's headings. They can do so by following these steps -

Enable narrator
Press Caps+Space to enable scan mode in the narrator
Press H to read the headings

Before this PR we did not have any headings defined, therefore the narrator would just say no headings defined when H is pressed.
This PR adds two levels of headings, level1 for the main heading and level2 for the sub-headings within the settings app.

Validation Steps Performed
How does someone test & validate?
Follow these steps to validate that headings are defined -

Enable narrator
Press Caps+Space to enable scan mode in the narrator
Press H to read the headings

The pipeline for this PR has passed - https://dev.azure.com/ms/PowerToys/_build/results?buildId=117907&view=results. @crutkas, @ryanbodrug-microsoft it is ready to merge.

@alekhyareddy28 Question: is it also possible to set the HeadingLevel in the style of the TextBlock? This would make it future proof since we use the SettingsGroupStyle.

Thanks @niels9001 that's a great suggestion!
I updated the PR to set the heading level in the style to avoid duplication as you suggested.
The general page, run as admin was however using the default style as we probably don't want a huge margin so I added the heading level2 only for that control. The rest of them are using the heading level set in the GroupStyle.

https://dev.azure.com/ms/PowerToys/_build/results?buildId=117958&view=results is the CI for this

this succeeded BTW

This was successfully built here: https://dev.azure.com/ms/PowerToys/_build/results?buildId=117958&view=results


Concurrency Control !?

As this was the persisting issue with Decap CMS. I'm assuming this has not been fixed yet as well. The problem being that currently when two or people edit the same document, it will overwrite information and disregard the first "person's" changes.
I'm planning to build a tool for my company that somehow prohibits the user when multiple people are editing the document. The key requirements I have are -

Somehow track the changes in the current file and show them to other users.
If one of the change are merged, we somehow rebase all of the changes.

I'm thinking of using Web RTC of the authenticated Oauth User, since static / decap does not have a server, I'm finding it a little overwhelming to figure out a way to start with this.
I would like to contribute this feature to static if it makes sense with everyone else. If maintaners have a idea about how I can think about building this or improving this. Please guide me and help me out in understanding how should I start with understanding the codebase.

Hi, that would be something like Google docs or Miro where multiple users can edit a file and others see their changes immediately. imo easier would be to check if a file has been modified (someone made and published changed earlier than you) and reject your publish unless you merge your changes.

Hey @blahetal
Yes that should be relatively easier. But to add support for live editing or people editing document "concurrently" I would need to somhow ping all connected users while fetching latest branch info every second or so.
As you seem to have worked on the codebase already, can you point me where to start from.

Hey @blahetal
Yes that should be relatively easier. But to add support for live editing or people editing document "concurrently" I would need to somhow ping all connected users while fetching latest branch info every second or so.
As you seem to have worked on the codebase already, can you point me where to start from.

not really, just minor tweaks here and there.
better to ask devs here on discord


Making multiple subscriptions on the same table gets immediately CLOSED with error null

Bug report
Describe the bug
I have created a table countries with a single field name. In the following gist, when I comment out either _subOnCountriesDelete  or _subOnCountriesInsert, everything works as intended and I receive events during the 10 seconds. But when I don't comment out anything in the gist, both the subscriptions just return event: CLOSED error: null.
To Reproduce
This gist
Expected behavior
The expected behavior is that the subscription should only end when removeSubscription() is called.
System information

OS: EndlessOS
Version of supabase-dart: "0.0.1-dev.8"

Additional context
this

I've noticed that everything works fine with only one subscription in this flutter example too.
Maybe having multiple subscriptions on the same table is causing this issue.

Hi @happy-san, you can only have 1 active subscription per unique topic at any given time.

While the client may join any number of topics on any number of channels, the client may only hold a single subscription for each unique topic at any given time. When attempting to create a duplicate subscription, the server will close the existing channel, log a warning, and spawn a new channel for the topic. The client will have their channel.onClose callbacks fired for the existing channel, and the new channel join will have its receive hooks processed as normal.

Oh okay!
Also, can you clarify what do you mean by topic ?

subscription topic is based on your schema and table name.
final topic = tableName == '*' ? 'realtime:$schema' : 'realtime:$schema:$tableName';

The client will have their channel.onClose callbacks fired for the existing channel, and the new channel join will have its receive hooks processed as normal.

If I understood correctly, the second subscription should've been successful but they both get closed in my example.

I've updated my subscription to this and everything is working fine!
      _countriesSub =
          _client.from('countries').on(SupabaseEventTypes.all, (x) {
        switch (x.eventType) {
          case 'INSERT':
            print(
                'on countries.insert: ${x.table} ${x.eventType} ${x.newRecord} ');
            break;

          case 'DELETE':
            print(
                'on countries.delete: ${x.table} ${x.eventType} ${x.oldRecord}');
            break;

          case 'UPDATE':
            print(
                'on countries.insert: ${x.table} ${x.eventType} ${x.oldRecord} -> ${x.newRecord} ');
            break;
        }
      }).subscribe((String event, {String errorMsg}) {
        print('event: $event error: $errorMsg');
      });

...almost
I'm not receiving anything in `oldRecord` in case of `UPDATE`
I/flutter (15683): on countries.delete: countries DELETE {name: Niger}
I/flutter (15683): on countries.insert: countries INSERT {name: Niger}
I/flutter (15683): on countries.insert: countries UPDATE {} -> {name: Mexico}
I/flutter (15683): on countries.insert: countries UPDATE {} -> {name: Mexic}
I/flutter (15683): on countries.insert: countries UPDATE {} -> {name: Mexico}

@phamhieu thank you for your quick response though.

I'm not receiving anything in oldRecord in case of UPDATE

tracking at #9


ArchiveUnpacker now preserves "executable" permissions in Tar and Zip

When unpuzzle extracts the eclipse archives, it doesn't preserve the executable bits.  This means that if you want to use unpuzzle to install a runnable Eclipse, it doesn't work on Linux or Mac.  This PR fixes this issue.

Merged it into snapshot version. Will test and release soon.


Add fighter guide

https://web.archive.org/web/20110130235002/http://calesta.dyndns.org/~genesis/mer/html/index.php?section=guide-fighter


Align query_chw and get_dimensions

We have to proper combination
https://github.com/pytorch/vision/blob/9b82df43341a6891f652be1803abd1d1d05bfbb2/torchvision/transforms/v2/functional/_meta.py#L84
https://github.com/pytorch/vision/blob/9b82df43341a6891f652be1803abd1d1d05bfbb2/torchvision/transforms/v2/utils.py#L35
However, we have the combination below that diverges on the name
https://github.com/pytorch/vision/blob/9b82df43341a6891f652be1803abd1d1d05bfbb2/torchvision/transforms/v2/functional/_meta.py#L15
https://github.com/pytorch/vision/blob/9b82df43341a6891f652be1803abd1d1d05bfbb2/torchvision/transforms/v2/utils.py#L21
I don't recall anymore how we ended up with this, but we should correct it.
My vote is out for also calling it query_dimensions since we already have
https://github.com/pytorch/vision/blob/9b82df43341a6891f652be1803abd1d1d05bfbb2/torchvision/transforms/functional.py#L64
in v1.
cc @vfdev-5

Hello !, I'm looking forward to contribute, how can I do that

Can we get rid of the query* stuff and just let get* accept both single datapoints/tensors and flat inputs?
And BTW, since we're not providing public APIs to flatten / unflatten inputs nor to handle arbitrary structures in custom transforms, I don't think we need to make query* public (should we keep it).


Running regression on a single dataset throws: argument after * must be an iterable, not int

Unsure if this error is meant to occur because the calculation is not possible, or if an actual error is occuring.

It appears that this is occurring only because the calculation at the time is not possible. Should add a new try-catch to just given an error like "Unable to compute for ".


unassigned search

What does this pull request do?
-introduces search feature for the unassigned tab in the dashboard
What is the intent behind these changes?
-users wanted search feature in the unassigned tab in the dashboard

Noticed that referral number tests are missing


Updating parent component state causes child to lose state

Describe the bug
The parent component renders a list of buttons (Child) which if clicked displays a dialog. If the child then updates state in the parent, the dialog is closed.
Your Example Website or App
https://github.com/nmathew98/qwery/blob/dev/packages/example-solid/src/App.tsx#L191
Steps to Reproduce the Bug or Issue

Create a thread
Click 'View whole thread'
Share a thought
Reply to the thought
Result: Dialog closes but reopening the dialog, the reply is there
Expected result: Dialog does not close

Expected behavior
The dialog should not close when the parent state is updated
Screenshots or Videos
https://github.com/solidjs/solid/assets/55116576/060166dc-37a9-4124-bc8c-c53e4c062173
Platform

OS: macOS Sonoma 14.4
Browser: Safari
Version: 17.4

Additional context

The GitHub repo is not a minimal example but the issue as far as I've checked the issue does not stem from the third party state management library which is being used

Components used are shadcn solid which are built upon Kobalte

Data is being updated correctly, just the Child is losing its state

Problematic line is here: https://github.com/nmathew98/qwery/blob/dev/packages/example-solid/src/App.tsx#L191 which causes state in the parent to update

My recommendation would be to start cutting away as much as possible to narrow down further if this relates to the kobalte Dialog implementation, solid-js, or something else.

will get to it, its a rabbit hole I didn't want to go down haha. don't feel like it'll be a kobalte related issue because solid is doing the rerender when state in parent changes

Given the way Solid works, ie without doing re-renders something has to be causing the re-render for the child to lose state. A VDOM framework like React or Vue would do this re-render without you noticing even if it could possibly avoid it but that isn't the case in Solid. So something that seems to work in Vue or React may not even be working optimally and you wouldn't know. Since Solid's re-render works purely on reactive tracking it does more than likely come down to how something is wired together and to understand that would likely require following the path.
That being said working bottom up sometimes is easier or us than top down. Ie tracing up to what re-runs when we don't expect it to. Often you will find you are tracking something you didn't intend.

Given the way Solid works, ie without doing re-renders something has to be causing the re-render for the child to lose state. A VDOM framework like React or Vue would do this re-render without you noticing even if it could possibly avoid it but that isn't the case in Solid. So something that seems to work in Vue or React may not even be working optimally and you wouldn't know. Since Solid's re-render works purely on reactive tracking it does more than likely come down to how something is wired together and to understand that would likely require following the path.
That being said working bottom up sometimes is easier or us than top down. Ie tracing up to what re-runs when we don't expect it to. Often you will find you are tracking something you didn't intend.

okok thanks for that, haven't had time to dig into this yet, but will get time over the weekend. last time something like this (reactivity behaving weirdly) cropped up with svelte (which I think is closer to solid than vue or react) it was likely due to the slot being nested which was a known bug (stripped out the component library and things were working again and then went from there)


Rename cloak to dagger

⚠ This is a significant & disruptive change, please read this message with the care & attention so that the surprises are kept to a minimum ⚠
"Project Cloak" was the codename for what will become Dagger v0.3.x. As we start preparing to ship Dagger v0.3.x pre-releases, we need to use the same "dagger" name everywhere so that:

Users keep the same entrypoint: the dagger binary

The tooling that installs dagger (e.g. HomeBrew, install.sh, install.ps) largely continues working as is. The only difference is that now the tooling will gain the functionality to install pre-releases of dagger.

There is more context here (including what happens after this gets merged): https://github.com/dagger/dagger/issues/3176#issuecomment-1267359683

There is an ongoing incident with GitHub Actions which explains why the checks are not running currently: https://www.githubstatus.com/incidents/smn1qtqvbsb6

@gerhard Do you want to go ahead and merge this? Must be painful to maintain ... /cc @sipsma

I only have one concern, which is that the nodejs and python engine sdks relied on finding the cloak binary in their path, now they rely on dagger. This means that users have to be sure that they have the most recent dagger built from the cloak branch. That's fine, I just fear that it's more likely users have some existing europa dagger binary laying around and now the engine sdk will find that and not work.
There's not all that much we can do here in the immediate term besides document that users need to build dagger from cloak branch and put that in their PATH. Longer term this will be improved by including the binary w/ the npm/python packages.
Above isn't a blocker though, I agree that it will be highly painful to maintain this branch so we should be biased towards merging asap.

@shykes 1. Can we have the checklist in its own issue?

Done: https://github.com/dagger/dagger/issues/3176#issue-1390192888

At what point to we finalize the syntax for the new dagger? The current syntax is ad hoc and we never discussed its design end to end. I would like an opportunity for that.

I expect this to happen over the next week. From a pre-release process perspective, this can happen in parallel - I don't see any coupling. My aim is to get alpha pre-releases published. We don't have to announce them, they just need to be available. People can try them out, but there is no guarantee on anything (the syntax can change as it needs to).
As we make progress over the coming weeks, we can decide to move onto betas or rcs. Beta to me signals one step closer to the GA and less change, but still no guarantees. At that point, we should start involving users and aim to finalise the big items like the syntax things. As soon as we hit the RCs stage, I would expect us to freeze new features and only publish fixes. By then, I would expect the new dagger syntax to be locked in. We can start any of these stages whenever we feel we are ready. We can also skip them. My preference would be not to, but I am easy. As long as we are exercising the release process regularly, I am easy on the steps towards GA. This PR is the first step towards building a release process for v0.3.x.

Do you want to go ahead and merge this? Must be painful to maintain ...

@aluzzardi as soon as we have confidence that the tests pass and everything looks good, yes.
I have a few more questions to answer and figure out why the tests GitHub Action has not run. I am keen on converting this into Ready for review in the next hour or so. Will update if I don't get there within this time-box.

@sipsma I only have one concern, which is that the nodejs and python engine sdks relied on finding the cloak binary in their path, now they rely on dagger. This means that users have to be sure that they have the most recent dagger built from the cloak branch. That's fine, I just fear that it's more likely users have some existing europa dagger binary laying around and now the engine sdk will find that and not work.

WDYT about the SDKs failing if the dagger binary version does not start with 0.3. ?

@sipsma I agree that it will be highly painful to maintain this branch so we should be biased towards merging asap.

I am giving myself a 1h time-box for this.

WDYT about the SDKs failing if the dagger binary version does not start with 0.3. ?

That's a good idea, we can at least have a clear error message then

I am stopping now. I am trying to understand why the buildkitd version is empty:
go test -v -run TestContainerWithMountedDirectory ./core/integration/container_test.go

=== RUN   TestContainerWithMountedDirectory
=== PAUSE TestContainerWithMountedDirectory
=== RUN   TestContainerWithMountedDirectorySourcePath
=== PAUSE TestContainerWithMountedDirectorySourcePath
=== RUN   TestContainerWithMountedDirectoryPropagation
=== PAUSE TestContainerWithMountedDirectoryPropagation
=== CONT  TestContainerWithMountedDirectory
    container_test.go:440:
                Error Trace:    /workdir/core/integration/container_test.go:440
                Error:          Received unexpected error:
                                buildkitd version is empty
                Test:           TestContainerWithMountedDirectory
=== CONT  TestContainerWithMountedDirectoryPropagation
--- FAIL: TestContainerWithMountedDirectory (0.00s)
=== CONT  TestContainerWithMountedDirectorySourcePath
=== CONT  TestContainerWithMountedDirectoryPropagation
    container_test.go:581:
                Error Trace:    /workdir/core/integration/container_test.go:581
                Error:          Received unexpected error:
                                buildkitd version is empty
                Test:           TestContainerWithMountedDirectoryPropagation
=== CONT  TestContainerWithMountedDirectorySourcePath
    container_test.go:517:
                Error Trace:    /workdir/core/integration/container_test.go:517
                Error:          Received unexpected error:
                                buildkitd version is empty
                Test:           TestContainerWithMountedDirectorySourcePath
--- FAIL: TestContainerWithMountedDirectoryPropagation (0.00s)
--- FAIL: TestContainerWithMountedDirectorySourcePath (0.00s)
FAIL
FAIL    command-line-arguments  0.011s
FAIL

I found this https://github.com/dagger/dagger/blob/9fbea2eec8c3840a104dfcae930ada27dfff0af3/internal/buildkitd/buildkitd.go#L87-L100, but this workaround is not being used in the cloak branch when invoked as above (the same failure happens). Am I missing something @sipsma?

I am stopping now. I am trying to understand why the buildkitd version is empty:

I am not at all sure, but I also am not seeing that failure in CI. In CI runs I'm seeing us consistently get:
        	Error:      	Received unexpected error:
        	            	input:5: core.filesystem.file failed to read file: ResourceExhausted: grpc: received message larger than max (10315513 vs. 4194304)
        	Test:       	TestGoGenerate
--- FAIL: TestGoGenerate (64.67s)

Which is... extremely odd. I can reproduce it locally though when pulling this branch, so I will debug and push a fix here.

@gerhard noticed that the PR seemed to accidentally have a go binary committed, that was the cause of the grpc: received message larger than max error. Tests pass for me locally after that.
For the original issue you posted about the buildkitd version error, I can reproduce it if I run your exact command, but if I run the slight variant of go test -v -run TestContainerWithMountedDirectory ./core/integration/ then the test passes.
I think this is because our integration tests rely on some special magic done in an init func in suite_test.go in that package, and without that init func running they can't start buildkitd. There's a whole back story here, happy to share more and dig up the original commits if desired, but suffice it to say that you need to specify the full ./core/integration package when running tests.
That's not obvious though of course. I guess we need some developer documentation. I'll add a bare-bones TESTING.md for now.

@aluzzardi please review and merge if it all looks good to you.
I will add the next PR on top of this one.

WDYT about the SDKs failing if the dagger binary version does not start with 0.3. ?

Alternatively, to avoid an extra roundtrip (e.g. SDK executing an additional "dagger version" command), we could just catch from the SDK "dev: command not found" and rewrite the error, since that's what's going to happen when using the SDK on 0.2.x.


management: Add option to reset all the active realms to a level.

CZO Discussion
This PR adds a --reset-level optional argument to send_zulip_update_announcements management command to reset all the active realms to a given level.

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

Merged, thanks @prakhar1144!


The plot got from muP coord_check seems not horizontal, which may indicates there exits a bug in the muP implementation?

Bug Discription & To Reproduce
The source code is from current main branch, and follow the instructions in the README-MUP.md until this step:

I encounter an error like this:
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /mnt/cache/baoyu/gpt-neox/train.py:27 in <module>                            │
│                                                                              │
│   24 │   neox_args.configure_distributed_args()                              │
│   25 │   neox_args.build_tokenizer()  # tokenizer needs to be build in train │
│   26 │   neox_args.initialize_tensorboard_writer()  # is initialized if tens │
│ ❱ 27 │   pretrain(neox_args=neox_args)                                       │
│   28                                                                         │
│                                                                              │
│ /mnt/cache/baoyu/gpt-neox/megatron/training.py:211 in pretrain               │
│                                                                              │
│   208 │   timers("train/valid/test data iterators").stop()                   │
│   209 │                                                                      │
│   210 │   if neox_args.use_mup and neox_args.coord_check:                    │
│ ❱ 211 │   │   mup_coord_check(neox_args, timers, lr_scheduler, train_data_it │
│   212 │                                                                      │
│   213 │   # Print setup timing.                                              │
│   214 │   print_rank_0("done with setups ...")                               │
│                                                                              │
│ /mnt/cache/baoyu/gpt-neox/megatron/training.py:154 in mup_coord_check        │
│                                                                              │
│   151 │   │   models[hidden_size] = lazy_model(hidden_size)                  │
│   152 │                                                                      │
│   153 │   neox_args.use_mup = True                                           │
│ ❱ 154 │   df_up = get_coord_data(                                            │
│   155 │   │   neox_args, timers, lr_scheduler, models, train_data_iterator,  │
│   156 │   )                                                                  │
│   157 │   neox_args.use_mup = False                                          │
│                                                                              │
│ /mnt/cache/baoyu/gpt-neox/megatron/mup_substitute.py:207 in get_coord_data   │
│                                                                              │
│   204 │   elif optimizer is None:                                            │
│   205 │   │   raise ValueError("optimizer should be sgd|adam|adamw or a cust │
│   206 │                                                                      │
│ ❱ 207 │   data = _get_coord_data(                                            │
│   208 │   │   neox_args, timers, lr_scheduler, models, dataloader, optcls, * │
│   209 │   )                                                                  │
│   210 │   data["optimizer"] = optimizer                                      │
│                                                                              │
│ /mnt/cache/baoyu/gpt-neox/megatron/mup_substitute.py:69 in _get_coord_data   │
│                                                                              │
│    66 │   │   │   │   │   )                                                  │
│    67 │   │   │   │                                                          │
│    68 │   │   │   │   # train for a step                                     │
│ ❱  69 │   │   │   │   loss_dict, skipped_iter = train_step(                  │
│    70 │   │   │   │   │   neox_args=neox_args,                               │
│    71 │   │   │   │   │   timers=timers,                                     │
│    72 │   │   │   │   │   data_iterator=dataloader,                          │
│                                                                              │
│ /mnt/cache/baoyu/gpt-neox/megatron/training.py:695 in train_step             │
│                                                                              │
│   692 │                                                                      │
│   693 │   # Pipeline parallelism schedules forward/backward/step             │
│   694 │   if neox_args.is_pipe_parallel:                                     │
│ ❱ 695 │   │   reduced_loss = train_step_pipe(                                │
│   696 │   │   │   neox_args=neox_args, timers=timers, model=model, data_iter │
│   697 │   │   )                                                              │
│   698 │   else:                                                              │
│                                                                              │
│ /mnt/cache/baoyu/gpt-neox/megatron/training.py:745 in train_step_pipe        │
│                                                                              │
│   742 │   """Single training step with DeepSpeed's pipeline parallel engine. │
│   743 │                                                                      │
│   744 │   assert neox_args.deepspeed                                         │
│ ❱ 745 │   loss = model.train_batch(data_iter=data_iterator)                  │
│   746 │   loss_dict = {"lm_loss": loss}                                      │
│   747 │   # Don't break Megatron's timers because we changed code paths.     │
│   748 │   for t in [                                                         │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/runtime/ │
│ pipe/engine.py:336 in train_batch                                            │
│                                                                              │
│    333 │   │   sched = schedule.TrainSchedule(micro_batches=self.micro_batch │
│    334 │   │   │   │   │   │   │   │   │      stages=self.num_stages,        │
│    335 │   │   │   │   │   │   │   │   │      stage_id=self.stage_id)        │
│ ❱  336 │   │   self._exec_schedule(sched)                                    │
│    337 │   │   self.agg_train_loss = self._aggregate_total_loss()            │
│    338 │   │                                                                 │
│    339 │   │   self.timers('train_batch').stop()                             │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/runtime/ │
│ pipe/engine.py:1307 in _exec_schedule                                        │
│                                                                              │
│   1304 │   │   │   │                                                         │
│   1305 │   │   │   │   # Equivalent to: self._exec_forward_pass(buffer_id=0) │
│   1306 │   │   │   │   self._exec_instr = MethodType(self._INSTRUCTION_MAP[t │
│ ❱ 1307 │   │   │   │   self._exec_instr(**cmd.kwargs)                        │
│   1308                                                                       │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/runtime/ │
│ pipe/engine.py:627 in _exec_forward_pass                                     │
│                                                                              │
│    624 │   │   # tensor changes across batches                               │
│    625 │   │   self._zero_grads(inputs)                                      │
│    626 │   │                                                                 │
│ ❱  627 │   │   outputs = super().forward(inputs)                             │
│    628 │   │                                                                 │
│    629 │   │   # Reset activation checkpointing buffers.                     │
│    630 │   │   # Need to call this between evaluation iterations             │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/utils/nv │
│ tx.py:15 in wrapped_fn                                                       │
│                                                                              │
│   12 │                                                                       │
│   13 │   def wrapped_fn(*args, **kwargs):                                    │
│   14 │   │   get_accelerator().range_push(func.__qualname__)                 │
│ ❱ 15 │   │   ret_val = func(*args, **kwargs)                                 │
│   16 │   │   get_accelerator().range_pop()                                   │
│   17 │   │   return ret_val                                                  │
│   18                                                                         │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/runtime/ │
│ engine.py:1731 in forward                                                    │
│                                                                              │
│   1728 │   │   if self.fp16_auto_cast():                                     │
│   1729 │   │   │   inputs = self._cast_inputs_half(inputs)                   │
│   1730 │   │                                                                 │
│ ❱ 1731 │   │   loss = self.module(*inputs, **kwargs)                         │
│   1732 │   │                                                                 │
│   1733 │   │   if self.zero_optimization_partition_weights():                │
│   1734 │   │   │   # Disable automated discovery of external parameters      │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/torch/nn/modules/m │
│ odule.py:1212 in _call_impl                                                  │
│                                                                              │
│   1209 │   │   │   bw_hook = hooks.BackwardHook(self, full_backward_hooks)   │
│   1210 │   │   │   input = bw_hook.setup_input_hook(input)                   │
│   1211 │   │                                                                 │
│ ❱ 1212 │   │   result = forward_call(*input, **kwargs)                       │
│   1213 │   │   if _global_forward_hooks or self._forward_hooks:              │
│   1214 │   │   │   for hook in (*_global_forward_hooks.values(), *self._forw │
│   1215 │   │   │   │   hook_result = hook(self, input, result)               │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/runtime/ │
│ pipe/module.py:350 in forward                                                │
│                                                                              │
│   347 │   │   │   │   if self._is_checkpointable(funcs):                     │
│   348 │   │   │   │   │   x = self.activation_checkpoint_func(exec_range_fun │
│   349 │   │   │   │   else:                                                  │
│ ❱ 350 │   │   │   │   │   x = exec_range_func(start_idx, end_idx)(*x)        │
│   351 │   │   return x                                                       │
│   352 │                                                                      │
│   353 │   def _partition_layers(self, method='uniform'):                     │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/deepspeed/runtime/ │
│ pipe/module.py:327 in exec_func                                              │
│                                                                              │
│   324 │   │   │   │   │   │   else:                                          │
│   325 │   │   │   │   │   │   │   ds_utils.set_random_seed(new_seed)         │
│   326 │   │   │   │   │                                                      │
│ ❱ 327 │   │   │   │   │   inputs = layer(inputs)                             │
│   328 │   │   │   │   return inputs                                          │
│   329 │   │   │                                                              │
│   330 │   │   │   return exec_func                                           │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/torch/nn/modules/m │
│ odule.py:1215 in _call_impl                                                  │
│                                                                              │
│   1212 │   │   result = forward_call(*input, **kwargs)                       │
│   1213 │   │   if _global_forward_hooks or self._forward_hooks:              │
│   1214 │   │   │   for hook in (*_global_forward_hooks.values(), *self._forw │
│ ❱ 1215 │   │   │   │   hook_result = hook(self, input, result)               │
│   1216 │   │   │   │   if hook_result is not None:                           │
│   1217 │   │   │   │   │   result = hook_result                              │
│   1218                                                                       │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/mup/coord_check.py │
│ :161 in f                                                                    │
│                                                                              │
│   158 │   │   │   │   for i, out in enumerate(output):                       │
│   159 │   │   │   │   │   _ret = copy(ret)                                   │
│   160 │   │   │   │   │   _ret['module'] += f':out[{i}]'                     │
│ ❱ 161 │   │   │   │   │   get_stat(_ret, out, output_fdict)                  │
│   162 │   │   │   elif isinstance(output, dict):                             │
│   163 │   │   │   │   for name, out in output.items():                       │
│   164 │   │   │   │   │   _ret = copy(ret)                                   │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/mup/coord_check.py │
│ :145 in get_stat                                                             │
│                                                                              │
│   142 │   │   │   elif isinstance(x, torch.Tensor):                          │
│   143 │   │   │   │   _d = copy(d)                                           │
│   144 │   │   │   │   for fname, f in fdict.items():                         │
│ ❱ 145 │   │   │   │   │   _d[fname] = f(x).item()                            │
│   146 │   │   │   │   records.append(_d)                                     │
│   147 │   │   │   else:                                                      │
│   148 │   │   │   │   raise NotImplemented(f'Unexpected output type: {type(x │
│                                                                              │
│ /mnt/cache/baoyu/packages/usr/lib/python3.8/site-packages/mup/coord_check.py │
│ :44 in <lambda>                                                              │
│                                                                              │
│    41                                                                        │
│    42 #: dict of provided functions for use in coord check                   │
│    43 FDICT = {                                                              │
│ ❱  44 │   'l1': lambda x: torch.abs(x).mean(),                               │
│    45 │   'l2': lambda x: (x**2).mean()**0.5,                                │
│    46 │   'mean': lambda x: x.mean(),                                        │
│    47 │   'std': lambda x: x.std(),                                          │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: mean(): could not infer output dtype. Input dtype must be either a
floating point or complex dtype. Got: Bool

This is caused by passing a Bool Tensor into the get_stat of mup(maybe the attention mask), but the mup library cannot handle it. In addition, we will also encount an error which is caused by passing None to the get_stat.
In order to solve this problem temporarily, I modify the source code in the file coord_check.py in mup like this:

This time, coord_check ran successfully, it outputs many jpgs, one for each GPU, jpg from different GPUs looks very similar, so I just show one jpg for each paramerization.
Standard Parameterization:

muP Parameterization:

The result looks weird, the SP is more horizontal than muP, which is not expected.
Expected behavior
https://github.com/microsoft/mup#coord-check
An expected behavior should looks like the plots in the above link, in which muP is very horizontal, while SP blows up.
Proposed solution
I check the code related to mup, but don't have a proposal yet, I will try to keep checking it. Maybe contributors in the issue(https://github.com/EleutherAI/gpt-neox/issues/679) can give some comments? @nsarka @Quentin-Anthony @StellaAthena
Thanks a lot!
Environment (please complete the following information):

GPUs: 8 V100 GPU on one node
Configs:python deepy.py train.py configs/2-7B.yml configs/local_setup.yml

I add mup related configs into the configs/local_setup.yml, and keep completely identical to the instructions in README-MUP.md.

Thanks for raising this issue. It looks like you’re correct and we broke the implementation at some point.
One thing we really need to start doing (but haven’t been able to do due to manpower limitations) is build out a robust testing suite that verifies new major changes don’t break old features :S

I was looking into the muP implementation in gpt-neox to contrast it with the Megatron-LM setup and accidentally found this issue :)
I am thinking, could LR schedule be the cause of the problem? By design it overrides the LR values per group (hence overwrites muP changes), and so the way muP scaling was introduced in AnnealingLR() as rescaling by group["width_mult"] in step() here. But I couldn't find that this key was added neither inside mup optimisers nor in gpt-neox codebase, so I am not sure that width_mult rescaling is applied at all.
Also, width_mult rescaling can be applied only for Adam-like optimisers and matrix-like params (as here), while for SGD the rescaling is with different multipliers, and so should be taken into account.
However, neither I found whether AnnealingLR schedule is actually applied during the training, so that might well be that my comment isn't really relevant to the observed behaviour.

I think you are right @ofivite, i don't think the implementation was ever correct since the learning rate wasn't correctly setup from the beginning. After a long and thorough debugging I managed to pass at least the 2 basic sanity checks for mup:

at same width mup doesn't do anything (since all shapes are the same it should coincide with SP)
all the rest being fixed, you only get better by going wider

Issues I have found are:

lr not scaled, added the width_mult key to the groups dicts (as stated above)
MuAdam being used instead of MuAdamW (maybe irrelevant, need to test more)
I have fixed all initializations to be normal with a fixed std. I'm not sure the method is correctly implemented or should be used at all for generic initialization functions, specially those that already scale the std by fan_in since as far as I can see that same std is then scaled again

Found another bug, neox_args.use_mup is set to false before initializing models, which also sets their use_mup attribute to False and therefore always ignore multipliers

And finally there seems to be a bug in the re-initialization of the output layer, after skipping that completely (it should be anyway in the flavour of Table 8) I'm getting these very nice and smooth horizontal lines

@marcobellagente93 Oh yes, now it's indeed nicely flat curves, great ! :)

I'll make a PR as soon as I can

Hi, I’m interested to see the PR as well. When I originally made my PR, the curves looked as expected—flat for mup and blown up for SP. >The use_mup parameter is set to false so the weights never get initializedAre you referring to it being false by default? When calculating the base shapes using mup, two models have to be instantiated, one with use_mup set to false and the other set to true. If I remember correctly, I set use_mup to false by default everywhere and enabled it only when mup was set to true in the config file. Then, when calculating base shapes it’s forced to false.>The learning rate is not scaled by width_multI saw some code being linked to in this thread that’s from the original mup repo from Microsoft. I had to bring in and modify a lot of it. I believe width_mult for the learning ratemay have been set there.On Aug 31, 2023, at 11:17 AM, marcobellagente93 @.***> wrote:
I'll make a PR as soon as I can
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

What I mean is that the main training loops with mup enabled does the following:

set neox_args.use_mup to false
initialize model
set neox_args.use_mup back to true

but at step 1 all parameters get initialized with self.use_mup = neox_args.use_mup (which is false) and causes everything else to be wrong (multipliers not used, 1/d attention not used, ...)

This behavior is expected--it the weights are reinitialized using mup_weights_reinit(neox_args, model)
https://github.com/EleutherAI/gpt-neox/blob/43ea51c2f3aeef2fc642ba401ce08844eb5a0240/megatron/training.py#L446
Or do you mean this function does not get called?


Refactor range proof code

There are 2 files: splitting.rs & padding.rs
There is some shared functionality between the 2 concepts but the way it's coded is not idiomatic and does not emphasize the differences & similarities.
The general idea is that the range proofs can be all individual, all aggregated, or some mix of the 2 (% split defined by calling code). padding.rs takes only 1 aggregated proof while splitting.rs takes many, and this seems to be the main difference between the 2.
splitting.rs does some strange logic when moving between secrets & blinding params and proofs. It's not clear why this logic is needed and how would the calling code know to pack their params in this format?

The num <-> bytes conversions can be changed to new rust syntax: https://stackoverflow.com/questions/29445026/converting-number-primitives-i32-f64-etc-to-byte-representations

Most of the range proof code was rewritten https://github.com/silversixpence-crypto/dapol/pull/48
Just need to do serialization

Serialization done


Update plugins.md

Fixes a misspelled path for the accessibility plugin.
Changes path: 'node_modules/protractor.plugins/accessiblity' to path: 'node_modules/protractor.plugins/accessibility'

Merged in at 6f6e9a4


Reworks the DLL-Plugin so that its possible to explicitly include /

…exclude dependencies.
Note: I've included lodash for ease of use and speed since this was a PoC. If the consensus is to get rid of lodash, we'll need standard functions for pullAll, uniq and defaultDeep.

Hi @strues!
Thanks for making a start on this.  I'll start reviewing and we can work from there.
Off the bat though I would prefer if we dropped the lodash dep.  I'm a ramda man myself, but would prefer the starter be unopinionated in this regard.

Lodash is gone. Check things out. Everything seems to work, but some thorough testing wont hurt.

Awesomeness. I will give it a bash as soon as this config story settles.  Do you mind adding me as being allowed to commit on the fork?

@ctrlplusb its actually a branch in this repo because I didnt realize I wasnt in my fork.

Brilliant, I am going to wait for next to settle down a bit and then mere it into there. :)

Thanks @strues! ❤


Update compilers.md.

Adds the missing getStyleImports function to the Vue example.
See #733.

Thanks @yishuolin! Much appreciated.

:rocket: This pull request is included in v5.37.1. See Release 5.37.1 for release notes.
Using Knip in a commercial project? Please consider becoming a sponsor.


Upgrading to Symfony 2.8.10 breaks functional tests

Stack trace:
24) Tests\PromoBundle\Functional\Controller\Api\PromotionControllerTest::testSchufaInvalidPromotion
Symfony\Component\DependencyInjection\Exception\RuntimeException: You have requested a synthetic service ("request"). The DIC does not know how to construct this service.

/app/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Container.php:313
/app/var/cache/test/appTestProjectContainer.php:42
/app/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Container.php:208
/app/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/DependencyInjection/ContainerAwareHttpKernel.php:82
/app/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php:185
/app/vendor/oro/platform/src/Oro/Bundle/TestFrameworkBundle/Test/Client.php:227
/app/vendor/symfony/symfony/src/Symfony/Component/BrowserKit/Client.php:315
/app/vendor/oro/platform/src/Oro/Bundle/TestFrameworkBundle/Test/Client.php:76
/app/tests/PromoBundle/Functional/Controller/Api/PromotionControllerTest.php:206

Reference: https://github.com/symfony/symfony/issues/19857

See #19857.

Hi @sagikazarmark,
Try to use symfony 2.8.9 instead 2.8.10

I did and it actually works. I realized later that the problem is in Symfony, but it causes some issues here, so I though it might make sense to leave this opened until the referenced PR is merged and tagged. Even then, it might make sense to either make this version conflicting or mention somewhere in the docs that this version does not work.

Just checked and the relevant PR has been merged.


Hello, can I use the custom trained model  as a pre-trained model for training again?

I have many kinds of data. I don't want to train by once, because it will take a lot of time and I need to keep collecting data.

@c7jie  Yes you can.

This is the second time i read about this same topic. Maybe should add this _feature_on docs to avoid future questions.
Should close this issue since it was answered.
Thanks for your answer @OlafenwaMoses !

Alright @rola93 . Will make updates soon. Thanks to you too.

@c7jie Yes you can.
Thanks


Improve some transformers

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (3fe586a) to head (f9817fc).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #354   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           49        49
  Lines         1282      1286    +4
  Branches        52        52
=========================================
+ Hits          1282      1286    +4

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Should request be exported?

In src/lib.rs, mod request isn't pub. Since SQSClient (the API I'm attempting to use) now has the D type parameter (DispatchSignedRequest), I can't figure out how to declare the SQSClient in my code (I want to pass it to different functions). If I export request with pub mod request in src/lib.rs, then I can write functions like:
fn do_something<P,D>(sqs: &SqsClient<P,D>) -> Result<(),AwsError>
    where P: rusoto::ProvideAwsCredentials, D: rusoto::request::DispatchSignedRequest

It's also possible that I'm just making a dumb Rust mistake here.

No mistake, that should definitely be pub.  It already is over in #321, but that's likely not going to be merged down to master for a bit.  I'll open up a separate PR for it.


Follow maximal pheromone value

Same as the other follow pheromone function, but please create a function, so that the ants are looking for the maximum pheromone value.

We discussed the idea again in our meeting and decided, it is not worth to keep trying to implement it.
Closed.


Core-metadata PUT endpoints for Device Admin/OpState

Fix #1365
Send the status data in the body of the request instead of being part of the URL
Signed-off-by: difince<EMAIL_ADDRESS>

@tsconn23 Thanks for the review. I just commits all your points


Class is deprecated

NotificationCompat should be used rather than Notification in the sample.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 14dfecf6-b596-61af-ef6c-c0b65838bd45
Version Independent ID: 7e1e4994-270d-5383-7d2d-f6d1cd1e1992
Content: Foreground Services - Xamarin
Content Source: docs/android/app-fundamentals/services/foreground-services.md
Product: xamarin
GitHub Login: @conceptdev
Microsoft Alias: crdun

@AndrueCope thanks for the feedback. We'll incorporate this fix into the next update of this page.

The example does not work "Bad notification for startForeground"

Thanks for submitting an issue and we apologize for the delayed response. The requested updates haven't been made since the creation of the issue, and the timeline for resolution may vary based on resourcing, so the issue is being moved to an internal repo to better track it.


improvements to the installation script

verify that the requested API version is supported by the underlying Kubernetes.
verify that the Cluster-Insight output contains all expected resources.
document functions.
cleanup parameter matching code.

The updated script looks good Eran.


ci: remove not needed workflow

This PR removes not needed golang workflow.
This change will optimize github actions runtime.
Thanks to this line the file I just removed will not be created here by bot again.

Thanks @codingtenshi !!!
/rtm


Why did I get this result?

---------------------- RESULTS ----------------------
Global FedAvg loss: 2.6197
Global FedAvg accuracy: 0.00%
Localized FedAvg loss: 0.4280
Localized FedAvg accuracy: 20.00%

Weird. I run the code again and get the results as follow:👀
---------------------- RESULTS ----------------------
Global FedAvg loss: 2.2936
Global FedAvg accuracy: 84.40%
Localized FedAvg loss: 0.4877
Localized FedAvg accuracy: 97.00%
Can you describe how you run the code and what args you had changed?

Weird ! The same as you. python main.py in windows11

Emmm... Maybe because our OS are different? I run the code in WSL2(Windows Subsystem for Linux), and that's the reason? 😂 I don't know...

I found that your result's losses are quite similar to mine, maybe the code about displaying results have some bugs?👀 I would check that asap. Please keep tracking on this repo's updating.

I found that your result's losses are quite similar to mine, maybe the code about displaying results have some bugs?👀 I would check that asap. Please keep tracking on this repo's updating.

OKOK

I run it in Win10, and get almost the same results as WSL2's. 😂
---------------------- RESULTS ----------------------
Global FedAvg loss: 2.2951
Global FedAvg accuracy: 84.40%
Localized FedAvg loss: 0.4930
Localized FedAvg accuracy: 97.00%
So I am really confused now.😂

Let me have an another try "from git clone ......"

Maybe it's my gpu's fault. But when I use cpu to train the model, I meet another bug : AttributeError: 'FedAvgTrainer' object has no attribute 'gpu', Luckily, I found the solution: add self.gpu = None to the fedlab/core/model_maintainer.py in line 38.

LOL. I found that bug also, and it's okay to avoid it by defining our own device, like self.device = next(iter(self.model.parameters())).device.
Change the code
loss_g, acc_g = evaluate(model_4_eval, self.valloader, self.criterion, self.gpu)
to
loss_g, acc_g = evaluate(model_4_eval, self.valloader, self.criterion, self.device). And it will run correctly.😄