Remove URL from OnComputId method (#6110)

Fixes #6109

Pull Request Test Coverage Report for Build 285853

0 of 0   changed or added relevant lines in 0 files are covered.
3 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.009%) to 78.671%

Files with Coverage Reduction
New Missed Lines
%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetNextViableDate.cs
1
93.94%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetNextViableTime.cs
1
93.94%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetPreviousViableDate.cs
1
93.94%

Totals

Change from base Build 282079:
0.009%

Covered Lines:
24846

Relevant Lines:
31582

💛  - Coveralls


Can't send POST request through formParams for Base64 encoded .xslx content

Details:
Currently we can't send POST request using formParams for Base64 encoded .xslx content:
Example:
Content Type = "application/x-www-form-urlencoded"
content: data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64, {encodedXlsxFile}  - file has been attached.
Actual result: encoded content corrupts in the request body (we can see that "+"  characters replaces with spaces)
Expected result: content data successfully uploaded without data corruption.
Please look at corrupted data (corrupted_result_data.txt)

corrupted_result_data.txt
encodedXlsxFile.txt

Have you tried using the EncoderConfig and specified the charset for the "application/x-www-form-urlencoded" content-type?

Yes, we tried and it does not work for us as well.


Gaia was not accessible – GetFile and PutFile api was not working

Describe the bug
Gaia was not accessible and all requests to gaia were failing (request timeout) on 20-Sep morning. After an hour, it started working, since they reversed DNS changes.
But we were facing the bug, while using BlockSurvey. Few of requests to gaia were successful and few were failed. Few PutFile request to gaia took long time and timed out. When we retried to save the same document, it threw 409 error (Concurrent write on same document).
Can you please help us to understand, what happened during this time?
To Reproduce
Steps to reproduce the behavior:
I attached the screenshot, when it was occurred. Now it is working.
Expected behavior
First putFile request to gaia should have been succussed. Or when we retried to save again, it should have been succussed.
Screenshots

I'm sorry you had some accessibility issues that morning.
It's already been some time since this happened so its very complicated to trace that particular day. I'm closing the issue but please let us know again if you encounter this issue or any other one again.


Add feature to change in Preferences the background color of queries executed with error in Query Manager

Please consider adding a feature to change in Preferences the background color of queries executed with error.
I found out a lot of bright red color is not convenient for eyes.
I failed to find how to change it in Preferences (maybe it is already presented somewhere?)

Thank you for the idea


I have created a resource list for web development, can we add it also?

Resources

This repo is only for CP.


Documentation fixes, minor Makefile update

Some minor issues we noticed while working on the IncludeOS support: mostly errors/typos in the docs, plus the links to the IncludeOS example in docs/examples does not work on GitHub, possibly because it is added as a submodule? Replaced with links to our example repo.

nice work


Property enumeration order of String exotic objects is wrong

Consider:
let obj = Object('asdf');
obj.a = 1;
console.log(Reflect.ownKeys(obj).map(x => x.toString()).join(','));

Per the overload for [[OwnPropertyKeys]] for String exotic objects, and in accord with every other browser, this should print
0,1,2,3,length,a

But in fact it prints
0,1,2,3,a,length

See also this open spec PR about very large integer indices and this related ChakraCore issue about enumeration order for classes.

Had a quick look at this:

Currently property keys are enumerates as:
i) Integer indexed keys
ii) string indexed keys (excluding builtins)
iii) Builtin keys
iv) Symbol indexed keys

The problem of course is the handling of builtins - in this case length is a builtin.

A simple update to move builtins before other string indexed keys would fix this case but may break other cases

A proper fix would involve thinking through what order should be used for each type of object and then likely special casing the JavascriptObject::CreateKeysHelper method for various of them


Enhance dump/restore github actions (#2437)

Description
Previously, the github action to test database-level dump/restore was only using master database and the assumption was to include more databases into it. But would be a bit optimistic thinking and require individual developers to add new github workflows for new T-SQL databases which adds extra burden on developers.
This commit changes the github action model a bit so that single workflow is able to test database-level dump/restore for all the T-SQL databases prepared by the preparation tests.
Task: BABEL-4053
Signed-off-by: Rishabh Tanwar<EMAIL_ADDRESS>Test Scenarios Covered

Use case based -

Boundary conditions -

Arbitrary inputs -

Negative test cases -

Minor version upgrade tests -

Major version upgrade tests -

Performance tests -

Tooling impact -

Client tests -

Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.005%) to 72.948%

Totals

Change from base Build<PHONE_NUMBER>:
0.005%

Covered Lines:
41194

Relevant Lines:
56470

💛  - Coveralls


[ROCm] Enable ROCm support for "snapshot" op

This PR enables ROCm support for the "snapshot" op.
PR #26722  is a pre-req for this PR, and hence this PR includes commits from that PR.
Only the last commit in this PR should be reviewed here (as all others will be reviewed as part of PR #26722  )

@tatianashp , @whchung : just FYI

rebased to remove merge conflicts

@deven-amd Please do not add Eigen patches to TensorFlow. Submit these to Eigen instead and add me as a reviewer. The same goes for all PRs.

@deven-amd Please do not add Eigen patches to TensorFlow. Submit these to Eigen instead and add me as a reviewer. The same goes for all PRs.

@rmlarsen , cc'd you on the Eigen PR
(PR No 546 : https://bitbucket.org/eigen/eigen/pull-requests/546/rocm-hip-specfic-fixes-updates/diff )

@deven-amd Thanks!

@rmlarsen
Also please advise on how to proceed with PR #26722  (the PR that is tracking the edit to the eigen patch file. this PR requires that PR and hence includes its commits)
Not including the eigen patch update will break the --config=rocm build, which will effect all PRs for which PR #26722 is a pre-requisite.
All of them will need to wait until

the Eigen PR I mentioned above gets merged and
the TF eigen pointer moves to/beyond the commit containing the Eigen PR merge
...that could take a while!

Can we come up with a solution that will allow for minor + temporary updates to the eigen patch file (both for now and going forward?
This is required for future too because:

Sometimes new additions to the TF code break Eigen compilation under HIP, and sometimes new commits to the Eigen codebase do the same.
Since we can only find out about it after the fact, it necessitates the need for temporary updates to the patch file (in the ROCm TF fork) to keep the ROCm TF fork working.

Typically we file a PR to upstream the fix to Eigen, and then remove the patch file once the TF eigen pointer moves to a point, where it is pulling in the fix

Now that we are upstreaming ROCm support, we need to continue to have a similar capability to keep the --config=rocm build working on the tip (when Eigen+HIP compilation gets broken due to TF or eigen updates)

We do run nightly Eigen unit tests (see prj47-rack-50 results in the Eigen dashboard : http://manao.inria.fr/CDash/index.php?project=Eigen),

@deven-amd Yeah, we need to upstream the last few bits in that patch file. I'll do it today or Monday.

rebased to remove merge conflicts

rebased PR to account for updates to PR #26722

rebased PR to resolve merge-conflicts post the merge for PR #26722 .
Note that now PR #28116 is a pre-requisite for this PR. This PR includes commits from PR #28116

rebased PR to the tip of master (since all the pre-reqs are now merged).
the changes in this PR are now trivial...please approve and merge.
thanks
deven


Incorrect autocompletion for module variables type

Prerequisites

[*] Ensure you have latest version of plugin installed
[*] Search for possible issue duplicates

Installation details

[*] IDE version: IDEA 2019.2
[*] intellij-hcl plugin version: 0.7.5
[*] Terraform version: v0.12.6

Terraform Configuration Files
variable "sg_sources_ids" {
  type = list(string)
}

...

module "ig-shared-resources" {
  source = "./ig-shared-resources"

  create           = var.create
  tags             = local.tags
  name             = local.name
  vpc_id           = var.vpc_id
  sg_master_id     = aws_security_group.k8s-master[0].id
  sg_sources_count = var.api_sg_count + 1
  sg_sources_ids   = local.api_sources_sgs_ids
  iam_boundary     = var.iam_boundary
}

Exception
Not related.
Expected Behavior
sg_sources_ids having correct type
Actual Behavior
Type falls back to string
Steps to Reproduce
Check screenshots:

Fixed, will be available in 0.7.6


Multi-line rows for the document type picker

It's no use if the user can't see what they're picking.

Closing as invalid. This should be addressed as part of the iOS and macOS unified app.


Spread the EdgeX Micro Service Management (MSM) API to the SupportScheduler service.

The gist of having EdgeX Micro Service Management (MSM) API spread to the SupportScheduler service is to add the capability of delegating to the SupportScheduler service the request (to the SMA) for that service's configuration info and for its metrics info.
Initially, this existed only for the support-notifications service.

Created this issue for the brand new SupportScheduler service.

Closing this issue in favor of: Issue #697 and Issue #690.


Change the base ports for all services of nctl nodes

See https://github.com/casper-network/casper-node/issues/1478#issuecomment-858592003 for details.

bors try

bors try

I've had a chance to ask @momipsl about this and he seems cautiously optimistic. Merging this!
bors r+


Move to karma/jasmine + jasmine/node combined tests

Similar to https://github.com/GoogleChrome/webidl-diff test setup. This should fix our misleading high coverage metrics (we're missing tests for all the code in #33 but it claims to have no impact on coverage). This will be necessary before we can fix #35.

Fixed in 995594f372165e788e57c5538a2027a24a4fedf1


Multi cluster installs are not working

Bug description
At last check, all 3 of the multi cluster install scenarios are failing on istio.io. This issue is for changes required in the istioctl tooling. There was a beta.3 build last night but I now that the latest installer PR https://github.com/istio/installer/pull/517 is not in that. It is in the 1.4-alpha.b632400c69f127ae453352784567497ddbf05f3a build which I will test with today.
Expected behavior
Documented scenarios should work
Steps to reproduce the bug
Forthcoming with testing later today (testing on latest build above).
There is a sister documentation issue to this: https://github.com/istio/istio.io/issues/5515

#18656 is an earlier issue. It may or may not be resolved with the  istio/installer#517 PR.

Trying the replicated control plane example today using the latest istio 1.4 build (1.4-alpha.b632400c69f127ae453352784567497ddbf05f3a), which has the above PR in it, I get the following missing field:
istioctl manifest apply
-f install/kubernetes/operator/examples/multicluster/values-istio-multicluster-gateways.yaml --set coreDNS.enabled=true
Failed to generate and apply manifests, error: failed to generate manifest: input file failed validation with the following errors: unknown field "podDNSSearchNamespaces" in v1alpha1.GlobalConfig

Some weirdness on the second cluster. I ran the same istioctl manifest apply \ -f install/kubernetes/operator/examples/multicluster/values-istio-multicluster-gateways.yaml --set coreDNS.enabled=true --force
I see this in the output:
Component CoreDNS failed install:
=================================
Error: error running kubectl: exit status 1

Error detail:

Error from server (NotFound): error when creating "STDIN": namespaces "istio-system" not found (repeated 4 times)

clusterrole.rbac.authorization.k8s.io/istiocoredns created
clusterrolebinding.rbac.authorization.k8s.io/istio-istiocoredns-role-binding-istio-system created

I did not pre-create the Istio-system namespace but it was created as part of the install so maybe an ordering issue. When removing the install, the nsamespace was removed.

@ostromart @elfinhe FYI

Current issue is the replicated Control Plane example uses the egress gateway and I am having an issue trying to create it. I updated the command and using istioctl manifest apply -f install/kubernetes/operator/examples/multicluster/values-istio-multicluster-gateways.yaml --set hub=gcr.io/istio-testing --set tag=1.4-alpha.b632400c69f127ae453352784567497ddbf05f3a --set coreDNS.enabled=true   --set values.gateways.istio-egressgateway.enabled=true --force
and I see this in the output:
Component EgressGateway installed successfully:
===============================================
No resources found

Component IngressGateway installed successfully:
================================================

serviceaccount/istio-ingressgateway-service-account unchanged
deployment.apps/istio-ingressgateway configured
destinationrule.networking.istio.io/istio-multicluster-destinationrule unchanged
envoyfilter.networking.istio.io/istio-multicluster-ingressgateway unchanged
gateway.networking.istio.io/ingressgateway unchanged
gateway.networking.istio.io/istio-multicluster-ingressgateway unchanged
sidecar.networking.istio.io/default unchanged
poddisruptionbudget.policy/ingressgateway unchanged
horizontalpodautoscaler.autoscaling/istio-ingressgateway unchanged
service/istio-ingressgateway unchanged

and no egress gateway service or pod.

@ostromart said I should use --set gateways.components.egressGateway.enabled=true and this created the egress gateway but saw following in the output (Note this was ingress):
Component IngressGateway failed install:
========================================
Error: error running kubectl: exit status 1

Error detail:

Error from server (AlreadyExists): error when creating "STDIN": destinationrules.networking.istio.io "istio-multicluster-destinationrule" already exists (repeated 1 times)

serviceaccount/istio-ingressgateway-service-account created
deployment.apps/istio-ingressgateway created
envoyfilter.networking.istio.io/istio-multicluster-ingressgateway created
gateway.networking.istio.io/ingressgateway created
gateway.networking.istio.io/istio-multicluster-ingressgateway created
sidecar.networking.istio.io/default created
poddisruptionbudget.policy/ingressgateway created
horizontalpodautoscaler.autoscaling/istio-ingressgateway created
service/istio-ingressgateway created

I mentioned the incorrect setting I used in prior comment was from https://preliminary.istio.io/docs/tasks/traffic-management/egress/egress-gateway/#deploy-istio-egress-gateway and Martin said he would fix that page.

Issues:

podDNSSearchNamespaces missing in validation schema;
Component EgressGateway is not generated/enabled with values-istio-multicluster-gateways.yaml
"istio-multicluster-destinationrule" already exists (repeated 1 times) ?

I'm not sure if 2 is required or not? I know the egress gateway is used as (an optional)? part of the Replicated Control Plane page so it could be installed in that section? I don't see the word egress in the other 2 multi cluster install scenarios.

I'm not sure if 2 is required or not? I know the egress gateway is used as (an optional)? part of the Replicated Control Plane page so it could be installed in that section? I don't see the word egress in the other 2 multi cluster install scenarios.

For documentation, the egress gateway is not directly related to a multi-cluster setup, kinda orthogonal settings for traffic management.

I was able to get through the Replicated Control Plane page today with some additions (coreDNS and egress gateway) and needing the --force. The uninstall instructions still use helm and will need to be updated. I will work on a doc PR for that this afternoon.
Then next up is shared multi-network. I was blocked yesterday as I didn't have the Istio-multi sa needed to create the kubeconfig to place in the secret. I might be able to use istio-reader-service-account but needing some fixes which hopefully are in the latest build.

I am now creating some PRs to let those cmds running without --force.

@elfinhe The remote cluster on the shared multinetwork has these requiring a --force:
Proceeding despite the following validation errors:
unknown field "istioRemote" in v1alpha1.GlobalConfig
Proceeding despite the following validation errors:
unknown field "istioRemote" in v1alpha1.GlobalConfig
Proceeding despite the following validation errors:
unknown field "remotePilotCreateSvcEndpoint" in v1alpha1.GlobalConfig

I also notice on the master cluster installs fine, no --force`` using istioctl manifest apply --context=$CTX_CLUSTER1
-f install/kubernetes/operator/examples/multicluster/values-istio-multicluster-primary.yamlbut converting that to an unsintall requires a--force`:
Proceeding despite the following validation errors:
unknown field "podDNSSearchNamespaces" in v1alpha1.GlobalConfig
Proceeding despite the following validation errors:
unknown field "podDNSSearchNamespaces" in v1alpha1.GlobalConfig
Proceeding despite the following validation errors:
unknown field "podDNSSearchNamespaces" in v1alpha1.GlobalConfig

I assume the later will be fixed by your #1 above.

Current status on the shared multi-network:

The text bash showing pods for cluster 1 needs updating (no completed pods)
I needed to add --force for cluster 2's istioctl manifest apply
The text bash showing pods for cluster 2 needs updating (no sidecar)
Create the sa in Cluster 2:

kubectl create clusterrole istio-reader-istio-system --verb=get,list,watch \                                                               ✔  16:13:05
    --resource=nodes,pods,endpoints,services,replicasets.apps,replicationcontrollers --dry-run -o yaml \
    | kubectl apply -f -

kubectl create serviceaccount istio-reader-service-account -n istio-system  --dry-run -o yaml \
    | kubectl apply -f -

kubectl create clusterrolebinding istio-reader-istio-system --clusterrole=istio-reader-istio-system \
     --serviceaccount=istio-system:istio-reader-service-account  --dry-run -o yaml \
    | kubectl apply -f -

Create the secret in cluster 1 using something like: istioctl --context $CTX_CLUSTER2 -n istio-system x create-remote-secret --service-account istio-reader-service-account | kubectl -n istio-system --context $CTX_CLUSTER1 apply -f - but this doesn't create a secret named n2-k8s-secret as the scenario uses.

I messed around at this point to try and get the ingress-gateway pod to become READY but have been unsuccessful.

FYI @ayj

https://github.com/istio/istio.io/pull/5530
for Shared control plane (single-network)

Tried the shared multi-network install again on beta.4.  I'm still not able to get the inregess-gateway to become ready on cluster #2 using the instructions as doc'd for the secret changing the service account from istio-multi to istio-reader-service-account

One thing I noted is a change in values-istio-multicluster-primary.yaml to add networks:
    pilot:
      meshNetworks:
        networks:
          network1:
            endpoints:
            - fromRegistry: Kubernetes
            gateways:
            - address: <IP_ADDRESS>
              port: 443
          network2:
            endpoints:
              - fromRegistry: n2-k8s-config
            gateways:
              - address: <IP_ADDRESS>
                port: 443

Still failing the pls I think between pilot and ingress.  Looking into that one more. I think helm is using mutualTLS and not sure of the istioctl, even though the pls values seem to indicate it should be...

Create the secret in cluster 1 using something like: istioctl --context $CTX_CLUSTER2 -n istio-system x create-remote-secret --service-account istio-reader-service-account | kubectl -n istio-system --context $CTX_CLUSTER1 apply -f - but this doesn't create a secret named n2-k8s-secret as the scenario uses.

create-remote-secret uses the cluster ID for the name and key. The cluster name (i.e. context) is added as an annotation. The cluster ID is the UID of the kube-system namespace (kubectl get namespace kube-system -o jsonpath='{.metadata.uid}').

I got the same error when testing shared-control-plane (single network):
In the remote cluster, error:
2019-11-11T21:00:09.585257Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected

Also confirmed that the master cluster got the secret and  is able to discover all injected services in the remote cluster.

Thanks to a bunch of detective work by a number of people I think we have a path forward for the shared control plane (multi-network). There are a few PRs coming for:

Update the values-istio-multicluster-primary.yaml to imbed network1 and 2 inside networks
Create the meshexpansion-vs-pilot
and doc steps to
Add autoInjection to cluster 2

https://github.com/istio/istio.io/pull/5639 to cover Shared control plane (multi-network). @elfinhe will work on Shared control plane (single-network).

I think at this point I will close this one since we probably aren't going to take anymore code changes. The Replicated Control Plane and Shared Control Plane (multi-network) are working with the istioctl install. @elfinhe is still working the Shared Control Plane (multi-network), but we can revert back to the helm install (I believe that still works).

Shared Control Plane (single-network) with istioctl requires some changes in remote.yaml. I am investigating which part is missing. Even if we identifies the issue, more testing is needed.
Agree with @ericvn to revert back to helm in Shared Control Plane (single-network)  scenario, which I've tested with the latest 1.4 branch this morning.


Add singleCellTK and alphabetize web portals and apps section

Thanks, @dfjenkins3 for the first contribution and for the cleanup. Very much appreciated.


allow volume migration on managed storage

Description
This PR...

Fixes: #8190

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[x] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[x] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

How did you try to break this feature and the system with this change?

@blueorangutan package

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✖ el7 ✖ el8 ✖ el9 ✖ debian ✖ suse15. SL-JID 8337

@blueorangutan package

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✖ el8 ✖ el9 ✖ debian ✖ suse15. SL-JID 11531


Oncall task - Set up dev GHA failure to ping oncall

Context:
Staff meeting notes:
https://docs.google.com/document/d/1USpxVJ_ZyObxMXWlYOOMAEHcvqeVuZCPYuQSPf55qsA/edit#bookmark=id.l2vz0jnk6asb
DoD:

Set up a Slack ping in oncall channel when Explorer dev GHA fails after a commit merge. Similar to what Data Portal does for staging GHA

@atarashansky in the staff meeting the team decided to ask the next oncall eng to set this up. I see that you're oncall next week, so please help 😆🙏 Thank you!


is there a way to delete the saved data off computer?

Is there a way to make it when I click fn+profile it doesn't work, and it's like I never used the site? Recently got a ps5 and can't stop the profile switching even after deleting data from site settings. Is it saved to the computer?

I was trying to use the console, but the website made my left stick a lot more than it should. Is there no way to get the data off the  controller itself? Does resetting the controller do anything?


Snapshot testing

The issue
Currently, we have a fragile repository.
Our function library is great for creating shared logic and dramatically improves code reuse, developer ergonomics, and the speed at which we can write new scripts. It allows for more robust code as we can encapsulate common operations in a single, robust function. And it allows us to fix future bugs by updating a single function rather than dozens of scripts.
However, this also produces a fragile ecosystem. The more we rely on these functions, the harder it is to understand the impact of a change to any single function. For instance, as I was redoing our bundler yesterday, I noticed importing a single library function can bring in several indirect dependencies. For example, library.mixin_helper depends on library.mixin, which itself depends on library.general_library. So there are several layers of indirection. So if someone updates the general library, there's no way to ensure every script still works without manually tracking every dependency and checking that every script functions as expected in every single edge case.
While as far as I know, this hasn't been an issue, an adjacent one occurred with the recent bundling PR #240 which had to be rolled back in #245. In this instance, the 3rd-party bundler only bundled the first library script in each file, and also did not bundle indirect dependencies. When I tested the resulting bundles, things happened to work for me since I just happened to pick scripts with a single dependency and no indirect dependencies.
Proposed solution
The industry standard way of fixing this issue is with automated testing. For instance, you can see these automated tests in our new bundling code (merged in PR #253):
https://github.com/finale-lua/lua-scripts/tree/master/.github/actions/bundle/src
These tests are unit tests. They ensure that even as I fix bugs in the bundler, the bundler will continue to work as expected. It also ensures that once a bug is fixed (and a test added), that bug can never resurface.
However, unit tests likely do not work in our use case since the vast majority of our code has side effects inside a Finale file. Additionally, while the bundler has complete unit test coverage, these still cannot guarantee the bundled scripts will run. They only guarantee the bundler works as expected, but not that the expected result produces valid RGP Lua code.
But there's also snapshot testing. With the new "Save document as text file…" script, we have the potential to introduce snapshot testing into the repository. The general workflow would be as follows:

A developer writes the script as normal
A developer creates a file (or multiple files) of the desired before state of the script. This set of files should also include all conceivable edge cases.
The developer may also need to write some code to tell the testing tool how to run the script (e.g., select these measures on these staves, press these dialog buttons)
Our snapshot testing tool would then run these files through the script, and save the resulting output as a txt file
Finally, the snapshot testing tool compares the resulting output with the output from the previous run (usually called a "golden"). If they match, the test passes! If they don't match in a desired way, the developer can update the golden to make sure future tests pass. If they don't match in an undesired way, the test fails.

On every PR, the tests for the entire repository would run.
Benefits

We can ensure all changes to our library (and bundler) only affect scripts in a desired manor (e.g., no unexpected bugs are introduced), leading to increased stability
Future versions of RGP Lua can then be tested against our scripts to ensure there aren't any regressions in its behavior
We can ensure scripts work across multiple different versions of Finale and operating systems, including future versions of Finale, macOS, and Windows, by running the tests on each Finale/OS pair.

Downsides

Setting this up could take quite a bit of engineering time since as far as I can tell, the entire toolchain would effectively need to be custom.
We'd need to find a way to run Finale on GitHub actions. This may not only have technical hurdles, but also potential licensing issues.
These kinds of tests could potentially take quite a bit of time to run (estimating 20–30 minutes if we have hundreds of tests?). This might not be a huge deal as review times are usually measured in hours or days for this repo. However, it could impact someone making a PR since think they're done, only to learn half an hour later they need to make changes. If tests can be run locally, that would help (since our computers are likely much more powerful than the ones GitHub actions uses, leading to faster tests).

Conclusion
Ultimately, I don't see a huge reason to implement this now as the benefits probably don't outweigh the costs just yet. However, as the number of scripts grow, the size of the library grows, the number of first-time coders join, and the more Finale professionals use these scripts, there will be a growing need to ensure stability and reliability.
So right now I'm mainly throwing this idea out there to see what everyone things.
cc @rpatters1 @cv-on-hub @ThistleSifter

The more automated testing, the better, as far as I'm concerned. I agree that we may not have reached the point where it's a priority. One caveat about the Save As Text File is that it doesn't see differences in layout, only differences in notation that would directly affect how the piece is performed. This is by design. (More info in the notes.)
I've been expanding and using the jwluatesttools repo that Jari initially created to regression test RGP Lua. A tool we could use for testing is Lua itself, in the form of test scripts against test docs. That way we wouldn't necessarily be limited to what the Save As Text File script does.

I assume you're referring to https://github.com/jariw2/jwluatesttoolkit?
That could be quite useful. I've not used it and am not too familiar with it. Does it require Finale to run tests? I assume so, but want to double-check.

One caveat about the Save As Text File is that it doesn't see differences in layout, only differences in notation that would directly affect how the piece is performed. This is by design. (More info in the notes.)

Ah, good to know. Should have read the description more carefully.

The current version is a fork of that one: https://github.com/finale-lua/jwluatesttoolkit
It requires Finale to run and so far I run it manually. But I always do this before every release (at a minimum). I'm not necessarily suggesting we use that (although not opposed). It is currently mainly for regression testing the PDK Framework and RGP Lua. But we could either copy its approach or expand it to add library tests for our library.

Ok, I see. I think more thought is needed, at least on my end.


Add backward support for binary maximum

Merged to main  #6003  https://github.com/tenstorrent-metal/tt-metal/pull/6003


Support Expo EAS build via plugin config?

Hi, Would you like to support Expo EAS build via plugin config? I will send a PR for that support.

I think the config plugins are pretty cool and we have good experience with them so far in react-native-firebase. I would be open to a PR here. Please note that I have experienced an increased support load in react-native-firebase related to Expo users starting to take it up, so any PR should have really explicit complete documentation for Expo users, and I personally don't use Expo so a commitment to help maintain the thing would be useful.
Here's the stack of PRs related to this in react-ntaive-firebase of how the plugin went there, including a jest test for it with fixture and instructions on how to update it, all helpful - maybe can serve as inspiration?
https://github.com/invertase/react-native-firebase/pulls?q=is%3Apr+expo+config

Expo has a new core module for integration, I will support them by migrate all plugins to the new system (expo-module-core). Included react-native-firebase

Not stale

Is stale, needs someone to do it! @glautm could you propose a pull request?

hey @mikehardy, The PR is ready.
Sorry for the delay, I have trouble when running it with SDKv44 on Android. Because Expo SDKv44 support<EMAIL_ADDRESS>and it has a bug in onActivityResult that let the FB SDK didn't receive the login result.
The plugin will work fine with the next SDKv45, or upgrade the React-Native version to v0.66

Thanks, @mikehardy. If there is any issue with the Expo please feel free to assign it to me.


mkl: 2019.3 -> 2019.5

Includes bugfixes, new features, and performance improvements.
Full release notes here:
https://software.intel.com/en-us/articles/intel-math-kernel-library-release-notes-and-new-features
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @

For some reason undmg crashes:
unpacking sources
unpacking source archive /nix/store/1jg2vljjzkykw5hil40idqrsfn8i3315-m_mkl_2019.5.281.dmg
/nix/store/x27s6d91scidhvg72hrx94dgwin7fx4a-undmg-1.0.3/nix-support/setup-hook: line 2: 74066 Segmentation fault: 11  undmg < "$curSrc"
do not know how to unpack source archive /nix/store/1jg2vljjzkykw5hil40idqrsfn8i3315-m_mkl_2019.5.281.dmg
builder for '/nix/store/69q9n44kspyakljxxjmb13yxp56nzkzz-mkl-2019.5.281.drv' failed with exit code 1
error: build of '/nix/store/69q9n44kspyakljxxjmb13yxp56nzkzz-mkl-2019.5.281.drv' failed

At the same time, building mkl from the parent commit works.

For some reason undmg crashes

Is there a way I can reproduce this without a Mac?

Is there a way I can reproduce this without a Mac?

Yes. This fails on linux too:
with import <nixpkgs> {};

stdenv.mkDerivation {
  name = "undmg-test";
  src = fetchurl {
    url = "https://registrationcenter-download.intel.com/akdlm/irc_nas/tec/15822/m_mkl_2019.5.281.dmg";
    sha256 = "014c91jzsvbhjpqafgzwvlzh7f8h2mbnmp4mk978xx838apsnxf9";
  };
  nativeBuildInputs = [ undmg ];
}

unpacking sources
unpacking source archive /nix/store/1jg2vljjzkykw5hil40idqrsfn8i3315-m_mkl_2019.5.281.dmg
/nix/store/rbbbb1lqrc1mx1m1qy2w3d5m3b5ygsg4-undmg-1.0.3/nix-support/setup-hook: line 2:   465 Segmentation fault      undmg < "$curSrc"
do not know how to unpack source archive /nix/store/1jg2vljjzkykw5hil40idqrsfn8i3315-m_mkl_2019.5.281.dmg
builder for '/nix/store/7ybah40ig46hsq15m3czymgh02bhsn5l-undmg-test.drv' failed with exit code 1

I think we are facing https://github.com/matthewbauer/undmg/issues/4

We could keep the old mkl version on darwin and update only on linux until the issue is resolved. If we do that, we should leave a comment pointing to the undmg issue.

I've updated the PR to leave Darwin on the old version.  The hash of the Linux package is unchaged from the previous commit at /nix/store/d79x6zfbq3dkvkz19xldamnnias7swd4-mkl-2019.5.281 , so no need to re-test that one; let me know if the Darwin one still works with the refactored package expressions but the unchanged src version.

@bhipple Thank you!


The validation accuracy of Cifar-10 is almost 99%

Hi.
I modified DoReFa-Net and applied it to Cifar10.
And I mistakenly omitted the gradient_override_map and quantized the last activation of the network, but the validation accuracy is over 99%.
I admit that there is a problem with the code, but how do I interpret that the validation accuracy is  over 99%?
Here is the code:

...
...
...
with tf.name_scope('last_layer') as scope:
    l = tf.matmul(l, kernel)
    l = tf.nn.bias_add(l, biases)

    # I omitted the code
    # with G.gradient_override_map({"Round": "Identity"})
    l = tf.round(tf.clip_by_value(l, 0.0, 1.0) * float(2**BITA - 1)) / float(2**BITA - 1)
logits = tf.identity(l)

and results:

[0313 15:03:26 @monitor.py:363] DataParallelInferenceRunner/QueueInput/queue_size: 50
[0313 15:03:26 @monitor.py:363] GPUUtil/0: 15.429
[0313 15:03:26 @monitor.py:363] GPUUtil/1: 24.929
[0313 15:03:26 @monitor.py:363] GPUUtil/2: 26.786
[0313 15:03:26 @monitor.py:363] GPUUtil/3: 63.214
[0313 15:03:26 @monitor.py:363] QueueInput/queue_size: 0.80026
[0313 15:03:26 @monitor.py:363] accuracy: 1
[0313 15:03:26 @monitor.py:363] lr: 0.001
[0313 15:03:26 @monitor.py:363] regularize_loss: 0.090447
[0313 15:03:26 @monitor.py:363] validation_accuracy: 0.99565
[0313 15:03:26 @monitor.py:363] validation_cost: 1.8313
[0313 15:03:26 @group.py:42] Callbacks took 4.208 sec in total. DataParallelInferenceRunner: 4.142sec

I forgot to mention that before the activation quantization I got 91% accuracy and did quantize with a 91% pre-trained model.
Thank you very much for your help
Best Regards

https://github.com/tensorflow/tensorflow/issues/10767

Thank you so much for your help! It is clear now.


Release 1.5.1

feat: only check codemod for rax and ice project #969

Codecov Report

Merging #971 (594c220) into master (0b1c76e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #971   +/-   ##
=======================================
  Coverage   57.14%   57.14%
=======================================
  Files           1        1
  Lines          70       70
  Branches       18       15    -3
=======================================
  Hits           40       40
  Misses         30       30

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0b1c76e...594c220. Read the comment docs.


HDFS-16707. RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus

Description of PR
HDFS-16302 intoduced request record for each namespace, but it is only exposed in /jmx endpoint and in json format, not very convenient.
this patch exposed these metrics in /prom endpoint for Prometheus
How was this patch tested?
manual testing

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
41m 13s

trunk passed

+1 :green_heart:
compile
0m 54s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 48s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 40s

trunk passed

+1 :green_heart:
mvnsite
0m 54s

trunk passed

+1 :green_heart:
javadoc
0m 59s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m 10s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 42s

trunk passed

+1 :green_heart:
shadedclient
24m 17s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 38s

the patch passed

+1 :green_heart:
compile
0m 41s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 41s

the patch passed

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 36s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
0m 22s
/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)

+1 :green_heart:
mvnsite
0m 39s

the patch passed

+1 :green_heart:
javadoc
0m 37s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
0m 54s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 26s

the patch passed

+1 :green_heart:
shadedclient
23m 47s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
31m 50s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 43s

The patch does not generate ASF License warnings.

137m  9s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4665

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 126ff9bc843e 4.15.0-184-generic #194-Ubuntu SMP Thu Jun 2 18:54:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 514fb4f3aee0cd15a790f8089447149ee4b53543

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/2/testReport/

Max. process+thread count
2294 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 50s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
41m  3s

trunk passed

+1 :green_heart:
compile
0m 54s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
compile
0m 48s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
0m 42s

trunk passed

+1 :green_heart:
mvnsite
0m 53s

trunk passed

+1 :green_heart:
javadoc
1m  0s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
1m  5s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 42s

trunk passed

+1 :green_heart:
shadedclient
23m 59s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 38s

the patch passed

+1 :green_heart:
compile
0m 42s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javac
0m 42s

the patch passed

+1 :green_heart:
compile
0m 35s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
0m 35s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 21s

the patch passed

+1 :green_heart:
mvnsite
0m 39s

the patch passed

+1 :green_heart:
javadoc
0m 38s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS>

+1 :green_heart:
javadoc
0m 54s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
1m 27s

the patch passed

+1 :green_heart:
shadedclient
23m 42s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
31m 39s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 43s

The patch does not generate ASF License warnings.

136m 19s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4665

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux d1dc88b8a8f6 4.15.0-184-generic #194-Ubuntu SMP Thu Jun 2 18:54:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 388fb58ef6ad9439e095f74a77ea7b8a2ac57793

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu<IP_ADDRESS> /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/3/testReport/

Max. process+thread count
2317 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Looks good, but have triggered the build again to see if things are still calm. If the build comes green will push this in.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 50s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
42m 27s

trunk passed

+1 :green_heart:
compile
0m 38s

trunk passed

+1 :green_heart:
checkstyle
0m 30s

trunk passed

+1 :green_heart:
mvnsite
0m 42s

trunk passed

+1 :green_heart:
javadoc
1m  2s

trunk passed

+1 :green_heart:
spotbugs
1m 34s

trunk passed

+1 :green_heart:
shadedclient
23m 43s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 33s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 17s

the patch passed

+1 :green_heart:
mvnsite
0m 34s

the patch passed

+1 :green_heart:
javadoc
0m 49s

the patch passed

+1 :green_heart:
spotbugs
1m 23s

the patch passed

+1 :green_heart:
shadedclient
23m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
21m 39s

hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

122m 39s

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4665

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux d39c77316add 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 388fb58ef6ad9439e095f74a77ea7b8a2ac57793

Default Java
Red Hat, Inc.-1.8.0_362-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/4/testReport/

Max. process+thread count
2121 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4665/4/console

versions
git=2.9.5 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Enable store RawRepresentable directly

I need to store enum and optionSet in DefaultsKit without writing duplicated code
After add these two extension methods. Any RawRepresentable can be get and set directly.
public func get<ValueType: RawRepresentable>(for key: Key<ValueType>) -> ValueType? where ValueType.RawValue: Codable
public func set<ValueType: RawRepresentable>(_ value: ValueType, for key: Key<ValueType>) where ValueType.RawValue: Codable

Hi, @wddwycc
Thank you for this wonderful contribution. 🙏
Sorry it took so long to merge.


Add NPCs to derivatives

add NPCs for Loot/xLoot holders
NPCs
Etherscan link: https://etherscan.io/address/0x8a41ded90481367fbf1323f75c88328256ef1f68#writeContract

hi @justuhNPC are you still working on this?

@justuhNPC closing the PR. Feel free to re open.


UK TRE Community Leadership

Goal Title
Provide a space for those involved in building, using and responsible for governance of TREs to discuss and reccomend best practices.
Description

[ ] Host online working spaces, events and workshops to support the UK TRE Community
[ ] Share best practices i.e. for making radiology data available for researchers
[ ] Empower the community to help influence policy decisions

Definition of Done
Permanent scope
Details

RACI

Accountable team or person: RCM
Responsible team or person: RAM, RPM
Informed and consulted people:

Estimated effort in total FTE:

RAM: 0.1
RPM: 0.05
RCM: 0.1

Task and issue breakdown
Breakdown in specific tasks of maximum a month duration, tasks and issues may be added as necessary
Issues may span across repositories when necessary

issue alan-turing-institute/data-safe-haven-team#X
issue alan-turing-institute/data-safe-haven#X
issue alan-turing-institute/trusted-research
issue sa-tre/satre-specification#X
issue sa-tre/satre-team#X

Checklist

[ ] This Story have been agreed with project members, it tackles prioritised work

[ ] If not: Label as ForPrioritisation so it discussed in the next monthly meeting, do not set a status

[ ] I have filled in the Team Accountabe field
[ ] I have included this Story in the agreed upon Milestone, set status as Planned

[ ] If not: set status as Backlog, to be Planned via weekly meetings or async discussions

[ ] The work to be done is likely to span over a month
[ ] I have broken down the work in monthly issues and added them above
[ ] I have labeled this issue according to its main project: SATRE, TRESA, CORE DSH (ELA) or other (please agree and create a new label if necessary)
[ ] If known: sum the total effort (points) of tracked issues and add it on the body
[ ] Select a Story level of effort in the project fields

Reporting
10 July to 14 August

Work is focused on the organisation of the in person event at Swansea #46 , leading orgnaisation in collaboration with DARE UK. Ample communication have been done and a reach out strategy is in place, 100 registered attendees at this point. Agenda and format for the day is set, with specific content to be defined in the following 2 weeks.
Working on the sustainability and governance by exploring how formal support cna be offered to it

@harisood anything you want to highlight at this point?

I'm good unless we need anything specific!

October: after submitting grant and submnitting events we need to write the report, manage the grant response and deliver a community event (lead by others). Prioritatry to do the necessary work but not a lot of dedication

Jan planning
See https://github.com/orgs/uk-tre/projects/1/views/1

February focus: prepare for March event, put together website and Governance processes.

March focus:

Finalise governance docs
March 14th meeting to present governance
Final report

Finalise last bits, wrap up documentation and host the Working Group event.
Very important: Hari knowledge transfer

In May still a priority to:

Navigate the transition (knwoledge, resources, post WG day)
Organisation of June event

Mainly @Davsarper with @jemrobinson willing to help.

In April, transition is important as the community is eating up RPM time. I (David) feel progress in that transition, but we do need someone else present, this can be a topic for Monthly meeting

Continue to be a priority for @Davsarper, focusing on September event, skills grant and general community management. This comes at the expense of RPM time.
WARNING: August I cannot do much of this, we need to focus on project end.

Priority to plan and prepare annual meeting on 2nd september, hope to see you all there.

plans
merch
running it

Also work on skills grant and internal approval process


what is needed for upgrading this code into word2vec version

As is stated in @dennybritz 's blog, this code can also be used together with a pre-trained word2vec.
I am thinking about how to make the modification for that.
The obvious steps that I came across are:

instead of padding and indexing vocabulary, use a fix sized w2v vector to represent the sentence.
remove the embedding layer on TextCNN class.

Is that all? Did I misunderstand something?
Thx

I don't think you need to change the vocabulary building or remove the embedding. I think what you need to do this is:

Set the embedding dimension to 300 (or whatever pre-trained word2vec you want to use)
Build the "initial" vocab_size x embedding_sim embedding matrix based on word2vec. You do that by iterating through the downloaded word2vec vectors and for each vocabulary word set the appropriate column in the matrix. Initialize words that are not in word2vec randomly.
Before training, initialize the embedding matrix with the one you computed in step 2 using W.assign

Hello @dennybritz , I think there will be two steps here:

Change the parameter embedding_dim in file train.py
Change the following code in file text_cnn.py:

# Embedding layer
        with tf.device('/cpu:0'), tf.name_scope("embedding"):
            W = tf.Variable(
                tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0),
                name="W")
            self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)
            self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1)

Instead of building the matrix with random entries, now we build the matrix with row vectors from vectors.bin taken from word2vec, right?
I also have another question. In your code, you initialize matrix entries with random numbers. How that possible? Each time we run ./train.py, a new feature matrix will be created for a sentence because we randomly initialize the vocab matrix, is that right?

Instead of building the matrix with random entries, now we build the matrix with row vectors from vectors.bin taken from word2vec, right?

You can do it either way. You can either change the initialization of the matrix, or you can leave the random initialization and then overwrite the value. Nothing is assigned/initialized in the code snippet you posted, it just builds the computation graph. You may want to read the Tensorflow documentation on how initialization and variable assignment works. Then it will become clear.

I also have another question. In your code, you initialize matrix entries with random numbers. How that possible? Each time we run ./train.py, a new feature matrix will be created for a sentence because we randomly initialize the vocab matrix, is that right?

I'm not sure if I understand the question. Yes, the matrices are initialized randomly. And yes, at each run the random initialization will be different. If you want 100% reproducible results you can use tf.set_random_seed

Nothing is assigned/initialized in the code snippet you posted
Yes because I just copy your code
I'm not sure if I understand the question
I mean for a word, a random vector of embedding_dim is assigned for, but we don't have the correlation between that word and the random values. Can you give me a reason why you do that?

@dennybritz Thx, I've tried. and it works perfects! awesome.

Hi @sjhddh
Would you mind to share the code for embedding the word2vec if you managed to do it?
I have a pretrained word2vec matrix "embedding_matrix" with one text sentence per row and n_dim(300) columns using gensim.
I assign it to "W" after defining it. However I get an error when processing "tf.nn.embedding_lookup(W, self.input_x)"
@dennybritz , I followed your indications assining the pretrained word2vec matrix usint W.assign. Please, let me know if you how to solve it. thanks in advance!
 ```

Embedding layer
    with tf.device('/cpu:0'), tf.name_scope("embedding"):
       W = tf.Variable(tf.random_uniform([embedding_matrix.shape[0], embedding_size], -1.0, 1.0),name="W")
       W.assign(embedding_matrix)

            self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)
            self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1)

        # Create a convolution + maxpool layer for each filter size
        #...

- Error:

tensorflow.python.framework.errors.InvalidArgumentError: indices[0,10] = -1 is not in [0, 50000)
	 [[Node: embedding/embedding_lookup = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@embedding/W"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](embedding/W/read, _recv_input_x_0)]]
Caused by op 'embedding/embedding_lookup', defined at:
 File "/Users/Fer/Development/MASTERBD/TFM/cnn-text-classification-tf/text_cnn.py", line 46, in __init__
    self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)

Hi @sjhddh
Would you mind to share the code for embedding the word2vec if you managed to do it?
I have a pretrained word2vec matrix "embedding_matrix" with one text sentence per row and n_dim(300) columns using gensim.
I assign it to "W" after defining it. However I get an error when processing "tf.nn.embedding_lookup(W, self.input_x)"
@dennybritz , I followed your indications assining the pretrained word2vec matrix usint W.assign. Please, let me know if you how to solve it. thanks in advance!
   # Embedding layer
        with tf.device('/cpu:0'), tf.name_scope("embedding"):
           W = tf.Variable(tf.random_uniform([embedding_matrix.shape[0], embedding_size], -1.0, 1.0),name="W")
           W.assign(embedding_matrix)

            self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)
            self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1)

        # Create a convolution + maxpool layer for each filter size
        #...

Error:

tensorflow.python.framework.errors.InvalidArgumentError: indices[0,10] = -1 is not in [0, 50000)
[[Node: embedding/embedding_lookup = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@embedding/W"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](embedding/W/read, _recv_input_x_0)]]
Caused by op 'embedding/embedding_lookup', defined at:
File "/Users/Fer/Development/MASTERBD/TFM/cnn-text-classification-tf/text_cnn.py", line 46, in init
self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)

Hi @sjhddh
Would you mind to share the code for embedding the word2vec if you managed to do it?
I have a pretrained word2vec matrix "embedding_matrix" with one text sentence per row and n_dim(300) columns using gensim.
I assign it to "W" after defining it. However I get an error when processing "tf.nn.embedding_lookup(W, self.input_x)"
@dennybritz , I followed your indications assining the pretrained word2vec matrix usint W.assign. Please, let me know if you how to solve it. thanks in advance!
   # Embedding layer
        with tf.device('/cpu:0'), tf.name_scope("embedding"):
           W = tf.Variable(tf.random_uniform([embedding_matrix.shape[0], embedding_size], -1.0, 1.0),name="W")
           W.assign(embedding_matrix)

            self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)
            self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1)

        # Create a convolution + maxpool layer for each filter size
        #...

Error:

tensorflow.python.framework.errors.InvalidArgumentError: indices[0,10] = -1 is not in [0, 50000)
[[Node: embedding/embedding_lookup = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@embedding/W"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](embedding/W/read, _recv_input_x_0)]]
Caused by op 'embedding/embedding_lookup', defined at:
File "cnn-text-classification-tf/text_cnn.py", line 46, in init
self.embedded_chars = tf.nn.embedding_lookup(W, self.input_x)

Will the below code work? I test it, and the accuracy rises a little, and I doubt it's correctness.
            sess.run(tf.global_variables_initializer())             if FLAGS.word2vec:                 # initial matrix with random uniform                 initW = np.random.uniform(-0.25,0.25,(len(vocab_processor.vocabulary_), FLAGS.embedding_dim))                 # load any vectors from the word2vec                 print("Load word2vec file {}\n".format(FLAGS.word2vec))                 model = word2vec.load('mini_train_300.bin')                 # index_word = {}                 # enumerate words from the vocab list                 for i in range(len(model.vocab)):                     # get the word's index from the vocab_processor                     idx = vocab_processor.vocabulary_.get(model.vocab[i])                     # give its word vector                     initW[idx] = model.vectors[i]                 sess.run(cnn.W.assign(initW))

Will the below code work? I test it, and the accuracy rises a little, so I doubt its correctness. Apart from this, other codes in the original project are not changed.

            sess.run(tf.global_variables_initializer())
            if FLAGS.word2vec:
                # initial matrix with random uniform
                initW = np.random.uniform(-0.25,0.25,(len(vocab_processor.vocabulary_), FLAGS.embedding_dim))
                # load any vectors from the word2vec
                print("Load word2vec file {}\n".format(FLAGS.word2vec))
                model = word2vec.load('mini_train_300.bin')
                # index_word = {}
                # enumerate words from the vocab list
                for i in range(len(model.vocab)):
                    # get the word's index from the vocab_processor
                    idx = vocab_processor.vocabulary_.get(model.vocab[i])
                    # give its word vector
                    initW[idx] = model.vectors[i]
                sess.run(cnn.W.assign(initW))


BASH_SEVERITY not working

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
BASH_SEVERITY does not filter the lower severity errors (e.g., "info"), when BASH_SEVERITY=warning
Expected Behavior
BASH_SEVERITY should filter the lower severity errors (e.g., "info"), when BASH_SEVERITY=warning
Super-Linter version
slim-v5.0.0

Relevant log output
2024-03-20 17:33:39 [INFO]   File:[/github/workspace/tutorials/distributed-ml/tutorial-0-basics/ddp_slurm.sh]
2024-03-20 17:33:39 [ERROR]   Found errors in [shellcheck] linter!
2024-03-20 17:33:39 [ERROR]   Error code: 1. Command output:
------

In /github/workspace/tutorials/distributed-ml/tutorial-0-basics/ddp_slurm.sh line 27:
source ../../../envAI_hdfml/bin/activate
       ^-- SC1091 (info): Not following: ../../../envAI_hdfml/bin/activate: openBinaryFile: does not exist (No such file or directory)

For more information:
  https://www.shellcheck.net/wiki/SC1091 -- Not following: ../../../envAI_hdf...
------
2024-03-20 17:33:39 [INFO]   ---------------------------

Steps To Reproduce
See this GH action run (e.g., L316): https://github.com/interTwin-eu/itwinai/actions/runs/8363368499/job/22896129844?pr=117
Note the workflow file here: https://github.com/interTwin-eu/itwinai/actions/runs/8363368499/workflow?pr=117
and that BASH_SEVERITY: warning
Anything else?
No response

Hi @matbun !
Can you please try again with the latest super-linter version, and provide a debug log?
Thanks!

Moving from uses: github/super-linter/slim@v5 to uses<EMAIL_ADDRESS>solved the issue.


Add little doc for Rack/Sinatra apps

I have to add these lines to my Sinatra app:
require 'rails_autoscale_agent/middleware'
use RailsAutoscaleAgent::Middleware

require 'rails_autoscale_agent' - didn't work because it only loads middleware if there is a Railtie in the app, which is missing in Sinatra/Rack app.

one more little note - require 'rails_autoscale_agent/logger'is loaded from railtie which doesn't trigger when just middleware included.
But require 'rails_autoscale_agent/logger' also present in reporter which is loaded by middleware - it looks that loading sequence could be improved a bit

@troex Thanks for the report! There is this bit of the README that addresses other (non-Rails) Rack apps.
Regarding how logger is required... It's being required in every file it's used. require is essentially a no-op if it's already been required. Perhaps I'm missing something?

@adamlogic right but using use RailsAutoscaleAgent::Middleware without prior require 'rails_autoscale_agent/middleware' will fail to start the app, it will even fail with require 'rails_autoscale_agent'
what about the logger I guess is just about the style so doesn't really affect

Good point! I've updated the README. Thanks!


WIP: Update styled components example

Based on #2010 this updates the styled-components to use the new renderPage() signature.
This doesn't work yet, it throws an error when rendering on the server: "The default export is not a React Component in page: /"
Any ideas why that could be happening?
/cc @_philpl @jcheroske @iamstarkov

_document.js gets called only ever on the server, so no need to do server side checks.
Also since sheet.getStyleTags() returns an html escaped string, they'll need to be parsed manually.
Using sheet.getStyleElement() works perfectly for me:
static getInitialProps ({ renderPage }) {

    const sheet = new ServerStyleSheet()
    const page = renderPage(App => props => sheet.collectStyles(<App {...props} />))
    const styleTags = sheet.getStyleElement()
    return { ...page, styleTags}
  }

I have gone ahead and created a pull request with those changes in effect. See #2945

Would be great to add an example how to use styled-components with static-export. Looks like it's not working with this example.

Closing in favor of #2945, it was just me being not so smart. Thanks @Shalzz!


Fix extra space added to new line on split

Comment moved from source. Needs some revision and clarification: we need to conditionally trim. But we cannot trim if we are just inserting. Therefore, I think we cannot use insert. We have to use replace. Or we have to consider both, based on whether there is whitespace or not. To test, use a breaking sequence of a couple spaces and notice how the next line is has strange post-prefix indent, because the extra spaces are carried over, but we want to get rid of them.

I think this issue overlaps with the goal of merging tokenize and findContentBreak logic? Maybe revise split to tokenize the text and make decisions based on tokens and then return to this issue?

This one is hilariously more complicated than it seems. First, we have to replace text spanning into the next line, so the current insert approach needs to be tossed. Second, we need to be aware of the next line, so the core of the split function logic needs to be tossed as well. Third, the split function needs the previous/current or current/next line pair instead of just one line. That then means that the iteration code in index.ts is no longer suitable, because split needs to be able to operate on either a one line comment without a subsequent comment or on the subsequent line if one exists.

There are a few edge cases remaining but somewhere in today's series of misc. commits this was corrected.

There are a few edge cases remaining but somewhere in today's series of misc. commits this was corrected.


For Compose Desktop, do not expose Compose Material dependency externally

See https://github.com/JetBrains/compose-multiplatform-core/blob/976f6feab9f7539a02c264c8adc1a6b35eadecd1/compose/desktop/desktop/build.gradle#L50
We should not expose Compose Material dependency externally. This will cause IDEA to import the wrong package by default, if we use Compose Material3.
Maybe it is time to remove DesktopTheme.jvm.kt completely.

Duplicate: https://github.com/JetBrains/compose-multiplatform/issues/497

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Livestream player: please remove blocking console

Every time a livestream video loads, the console is blocked by player-settings. All other debug logs will not appear unless you clear the console first, which causes one to miss important errors.  Getting tired of having to manually clear it.

Let's get rid of that player once and for all! Hopefully Anthony's progress will make that easier.


fix: Use picker for iris grid partition selector

Resolves https://github.com/deephaven/web-client-ui/issues/1970
Changes Implemented:

Replace TableDropdown with Picker for the Partition Selector for Iris Grid
Change corresponding test, since role is no longer "combobox"

Testing Information:
Code used:
from deephaven import empty_table
t = empty_table(15000).update(["x = i", "y = i"]).partition_by(["x"])

Sample visual of new Picker:

@dsmmcken Could you take another look at this whenever you have the time, so that I can merge?

Thanks Don, merging now!


logprobs appears to have expired in vLLM v0.3.0 version (/v1/completions)

In vLLM v0.3.0 version, logprobs appears to have expired. My personal guess is that it's due to the Pydantic version.
In Pydantic2 version, use model_ dump() method doesn't seem to handle logprops very well.

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
    await self.asgi_callable(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 762, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 782, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 299, in app
    raise e
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 294, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 177, in create_completion
    return JSONResponse(content=generator.model_dump())
  File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 180, in __init__
    super().__init__(content, status_code, headers, media_type, background)
  File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 39, in __init__
    self.body = self.render(content)
  File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 183, in render
    return json.dumps(
  File "/usr/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

I have printed the interface and found that the information returned using dict() is correct, but it cannot be converted to JSON format。

In addition, due to the refactoring of the openai server, the promt parameter in the interface log will always be None.
vllm/vllm/entrypoints/openai/serving_completion.py##line 298
vllm/vllm/engine/async_llm_engine.py##line 432

Regarding the logging of the prompt, I can fix it.
Regarding the LogProb, do you have the request to reproduce it? I am getting properly formatted logprob on 0.3.0
Logprobs(text_offset=[0, 6, 10, 12, 15], token_logprobs=[-3.7315573692321777, -1.002060055732727, -0.12878480553627014, -0.8721659183502197, -0.7809031009674072], tokens=['▁Sarah', '▁and', '▁I', '▁am', '▁a'], top_logprobs=[{'▁Sarah': -3.7315573692321777}, {'▁and': -1.002060055732727}, {'▁I': -0.12878480553627014}, {'▁am': -0.8721659183502197}, {'▁a': -0.7809031009674072}])

I can reproduce this problem through vllm/examples/openai_completion_client.py
from openai import OpenAI
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id
stream = False
completion = client.completions.create(
    model=model,
    prompt="Hello, my name is",
    max_tokens=5,
    stream=stream,
    logprobs=1)

print("Completion results:")
if stream:
    for c in completion:
        print(c)
else:
    print(completion)

I added logs to the code and found that the engine can generate output. Due to byte encoding issues, it is not possible to convert it correctly into a JSON object, resulting in interface errors
 Wood.That is', 'logprobs': {'text_offset': [0, 4, 6, 11, 16], 'token_logprobs': [-5.482499122619629, -2.5623340606689453, -9.2454833984375, -12.924626350402832, -1.6724791526794434], 'tokens': [b' Tim', b'my', b' Wood', b'.That', b' is'], 'top_logprobs': [{b' Tim': -5.482499122619629, b' Li': -3.939530372619629}, {b'my': -2.5623340606689453, b'.': -1.0467090606689453}, {b' Wood': -9.2454833984375, b'.': -0.6868898868560791}, {b'.That': -12.924626350402832, b'.': -0.9168136119842529}, {b' is': -1.6724791526794434, b"'s": -0.6568542122840881}]}, 'finish_reason': 'length'}], 'usage': {'prompt_tokens': 5, 'total_tokens': 10, 'completion_tokens': 5}}
{'id': 'cmpl-e09b44854fdd4df2afcaa66ddc64c671', 'object': 'text_completion', 'created': 1258618, 'model': '/models/Qwen-14B-Chat/1', 'choices': [{'index': 0, 'text': ' Timmy Wood.That is', 'logprobs': {'text_offset': [0, 4, 6, 11, 16], 'token_logprobs': [-5.482499122619629, -2.5623340606689453, -9.2454833984375, -12.924626350402832, -1.6724791526794434], 'tokens': [b' Tim', b'my', b' Wood', b'.That', b' is'], 'top_logprobs': [{b' Tim': -5.482499122619629, b' Li': -3.939530372619629}, {b'my': -2.5623340606689453, b'.': -1.0467090606689453}, {b' Wood': -9.2454833984375, b'.': -0.6868898868560791}, {b'.That': -12.924626350402832, b'.': -0.9168136119842529}, {b' is': -1.6724791526794434, b"'s": -0.6568542122840881}]}, 'finish_reason': 'length'}], 'usage': {'prompt_tokens': 5, 'total_tokens': 10, 'completion_tokens': 5}}

INFO:     <IP_ADDRESS>:51680 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)

This is my Python environment：
Package                       Version
----------------------------- ------------
aioprometheus                 23.12.0
aiosignal                     1.3.1
annotated-types               0.6.0
anyio                         4.2.0
attrs                         23.2.0
certifi                       2023.11.17
charset-normalizer            3.3.2
click                         8.1.7
distro                        1.9.0
einops                        0.7.0
exceptiongroup                1.2.0
fastapi                       0.109.0
filelock                      3.13.1
flash-attn                    2.4.2
frozenlist                    1.4.1
fsspec                        2023.12.2
h11                           0.14.0
httpcore                      1.0.2
httptools                     0.6.1
httpx                         0.26.0
huggingface-hub               0.20.3
idna                          3.6
Jinja2                        3.1.3
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
MarkupSafe                    2.1.4
mpmath                        1.3.0
msgpack                       1.0.7
networkx                      3.2.1
ninja                         <IP_ADDRESS>
numpy                         1.26.3
nvidia-cublas-cu12            <IP_ADDRESS>
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu12             <IP_ADDRESS>
nvidia-cufft-cu12             <IP_ADDRESS>
nvidia-curand-cu12            <IP_ADDRESS>
nvidia-cusolver-cu12          <IP_ADDRESS>
nvidia-cusparse-cu12          <IP_ADDRESS>
nvidia-nccl-cu12              2.18.1
nvidia-nvjitlink-cu12         12.3.101
nvidia-nvtx-cu12              12.1.105
openai                        1.10.0
orjson                        3.9.12
packaging                     23.2
pip                           22.0.2
protobuf                      4.25.2
psutil                        5.9.8
pydantic                      2.6.0
pydantic_core                 2.16.1
pynvml                        11.5.0
python-dotenv                 1.0.1
PyYAML                        6.0.1
quantile-python               1.1
ray                           2.9.1
referencing                   0.33.0
regex                         2023.12.25
requests                      2.31.0
rpds-py                       0.17.1
safetensors                   0.4.2
sentencepiece                 0.1.99
setuptools                    69.0.3
sniffio                       1.3.0
starlette                     0.35.1
sympy                         1.12
tiktoken                      0.5.2
tokenizers                    0.15.1
torch                         2.1.2
tqdm                          4.66.1
transformers                  4.37.2
transformers-stream-generator 0.0.4
triton                        2.1.0
typing_extensions             4.9.0
urllib3                       2.2.0
uvicorn                       0.27.0.post1
uvloop                        0.19.0
vllm                          0.3.0
watchfiles                    0.21.0
websockets                    12.0
wheel                         0.37.1
xformers                      0.0.23.post1

And I noticed a question, why is your logprobs parameter of type bool?

The echo parameter may also have some issues.
INFO 02-02 09:05:41 async_llm_engine.py:431] Received request cmpl-e3ab58bfc7834026a608eee91a74dc15-0: prompt: None, prefix_pos: None,sampling params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=1.0, top_k=-1, min_p=0.0, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=5, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True), prompt token ids: [9707, 11, 847, 829, 374], lora_request: None.
INFO 02-02 09:05:42 async_llm_engine.py:110] Finished request cmpl-e3ab58bfc7834026a608eee91a74dc15-0.
None
None
INFO:     <IP_ADDRESS>:46374 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
    await self.asgi_callable(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 762, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 782, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 299, in app
    raise e
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 294, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 163, in create_completion
    generator = await openai_serving_completion.create_completion(
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/serving_completion.py", line 334, in create_completion
    response = request_output_to_completion_response(
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/serving_completion.py", line 177, in request_output_to_completion_response
    top_logprobs = prompt_logprobs + output.logprobs
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

vllm/entrypoints/openai/serving_completion.py"# line 177
top_logprobs = prompt_logprobs + output.logprobs

what's the function of logprobs?

what's the function of logprobs?

According to the description, the logarithmic probability of being able to print out candidate words

The logprobs issue has been resolved because I used the qwen model, and the convert_id_to_token method in the tokenization_qwen. py file on the huggingface was not converted to str. But it was converted into bytes for processing. Modify the file so that the parameters take effect.
examples:
def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
        """Converts an id to a token, special tokens included"""
        if index in self.decoder:
            return self.decoder[index]
        raise ValueError("unknown ids")

->
def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
        """Converts an id to a token, special tokens included"""
        if index in self.decoder:
            return self.decoder[index].decode("utf-8", errors=self.errors)
        raise ValueError("unknown ids")

The logprobs issue has been resolved because I used the qwen model, and the convert_id_to_token method in the tokenization_qwen. py file on the huggingface was not converted to str. But it was converted into bytes for processing. Modify the file so that the parameters take effect.
examples:
def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
        """Converts an id to a token, special tokens included"""
        if index in self.decoder:
            return self.decoder[index]
        raise ValueError("unknown ids")

->
def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
        """Converts an id to a token, special tokens included"""
        if index in self.decoder:
            return self.decoder[index].decode("utf-8", errors=self.errors)
        raise ValueError("unknown ids")

@simon-mo The logprobs parameter has been resolved, it is a tokenizer issue。
"In addition, due to the refactoring of the openai server, the promt parameter in the interface log will always be None.
vllm/vllm/entrypoints/openai/serving_completion.py##line 298
vllm/vllm/engine/async_llm_engine.py##line 432"
I think the echo parameters are also affected by this place.
vllm/vllm/entrypoints/openai/serving_completion.py##line 298
vllm/vllm/entrypoints/openai/serving_completion.py##line 177,I feel like it can add some protection
 top_logprobs = prompt_logprobs if prompt_logprobs else [] + output.logprobs if output.logprobs else []
 output_text = prompt_text if prompt_text else "" + output.text if output.text else ""

Any fixes or PRs here is appreciated.

I stumbled on the same issues. I've created this PR #2995. It should cover both echo and logprobs.

@liangxfng has https://github.com/vllm-project/vllm/pull/3441 solved your issue?


Put all of the information about releases in conf.py.

Previously we had put some hard-coded lists of EOL distributions
in some page templates.  However, it is much better to have
all of the information about distributions in the configuration
file.  To support this, add in a new config variable 'smv_eol_versions'
that has a list of the EOL versions.  When in multiversion mode,
propagate this into the page context so that the pages can just
reference the variable.
Finally, while we are in here, take galactic out of the "released"
list until it is actually released.
Signed-off-by: Chris Lalancette<EMAIL_ADDRESS>

I'm going to go ahead and merge this, as our current documentation site has the wrong status for Galactic.  We can always go back and edit it if we find problems with this setup.


Add DomainState method.

Returns state of the domain.

Can you suggest, how to write the test, please?

Add test function on libvirt_test.go

func TestDomainState(t *testing.T) {
    conn := libvirttest.New()
    l := New(conn)

    wantState := 1
    gotState, err := l.DomainState("test")
    if err != nil {
        t.Error(err)
    }

    if gotState != wantState {
        t.Errorf("expected domain state %d, got %d", wantState, state)
    }
}

Add variable var testDomainStateReply = []byte{} on libvirttest/libvirt.go
Add case on func handleRemote on libvirttest/libvirt.go

func (m *MockLibvirt) handleRemote(procedure uint32, conn net.Conn) {
    switch procedure {
    ...
    case constants.ProcDomainGetState:
        conn.Write(m.reply(testDomainStateReply))
    ...

Right?

@nemca Yep, you've got it. Bonus points for an integration test as well.

How to get var testDomainStateReply = []byte{}?
I should used Wireshark with libvirt dissector?

I should used Wireshark with libvirt dissector?

Yes, that's the best way IMO.

@nemca
I've pulled the hexdump for you, here's the result from DOMAIN_GET_STATE on a running domain:
Libvirt
    length: 36
    program: REMOTE (0x20008086)
    version: 1
    procedure: DOMAIN_GET_STATE (212)
    type: REPLY (1)
    serial: 7
    status: OK (0)
    remote_domain_get_state_ret
        state: 1
        reason: 1

0000   00 00 00 24 20 00 80 86 00 00 00 01 00 00 00 d4
0010   00 00 00 01 00 00 00 07 00 00 00 00 00 00 00 01
0020   00 00 00 01

@benlemasurier Thank you for hexdump!

Guys, I think I corrected all the errors.
Or do you still have any objections?

Nice work! Mind rebasing against master to resolve the merge conflict?

Nice work! Mind rebasing against master to resolve the merge conflict?
I don't know how. :(
I need my branch domainState merge with my master bracnch, fix conflicts and make a push from the my master branch?

Here's a quick guide for updating your fork of digitalocean/go-libvirt. Once completed:
git checkout domainState
git rebase master
git push origin domainState

Done.

thank you!


Update generate-release-notes.py to only include release-note-required label [v1.31]

Cherry-pick of #3282

I'm sorry but I failed delete the branch after merging the pull request.


chore: update game bridge to 1.46.9

Update game bridge (build from ts-immutable-sdk version 1.46.9)

Use https://github.com/immutable/unreal-immutable-sdk/pull/122 instead


Stop overwriting CMake internals/reading variables

it may lead to unstable behaviours on certain CMake versions, just see https://github.com/Devsh-Graphics-Programming/Nabla/blob/criss_baw/cmake/common.cmake#L309
We should not have changed variables that are only being used for reading target property, e.g. TARGET_NAME like above

Might be related:
https://github.com/buildaworldnet/IrrlichtBAW/issues/311
https://github.com/buildaworldnet/IrrlichtBAW/issues/298

Did your recent work fix any of this @AnastaZIuk ?

Did your recent work fix any of this @AnastaZIuk ?

I think most of those issues are fixed


Overlap problem

I created an example where the flows overlap: https://codesandbox.io/s/blazing-dawn-b07djs?file=/src/index.js
The node with the name "bug" has a wrong position. Do you see a way how to fix this except adding more nodes?

No, I don't see a way of fixing it just by configuration :/

@richardtallent-erm What about this one? why is Net Income gone down?

I guess I'm writing about the same issue here: https://github.com/kurkle/chartjs-chart-sankey/discussions/108
Maybe I could solve it by creating additional flow/nodes for the links that overlap (continuing the flow with the same value so that there would be an even amount of nodes on all levels), but that would look kind of silly, especially with the gradients and highlighted nodes.
I wouldn't mind if the nodes would end earlier, like in the sankey here:

Maybe a quick way to do that would be to add a possibility to create invisible links/nodes, for example if you set this in @hanstar17 's chart above:
{from: "Income Tax": to: null, flow: null}
The "Income Tax" node would end at the same level as "Net Income" and there would be an invisible node after the "Income Tax" which wouldn't show any info. At the moment even if you set the flow to 0 it still draws a line.


binding to TCP6 PORTS?

root@5f2be741-a708-4ed3-dcd8-e1d43aface6a:~# netstat -an | grep 5050
tcp6       0      0 :::5050                 :::*                    LISTEN

Sorry, but I'm going to need a bit more information, here. What's the exact issue?
By default, the exporter will listen on all interfaces, both IPv4 and IPv6. Counter-intuitively, this will also happen when <IP_ADDRESS> is specified as listen address -- see golang/go#17615 for more information.

Hello Martin,
This is all good. It's partly my fault. It's all sorted now.


Difference operator

Implement Different and SymmetricDifferent. Unit tests are also provided.
Different(A, B) = Set(all elements in A but not in B)
SymmetricDifferent(A, B) = Set(all element in A or B, but not in both)
check this link for symmetric difference.
Fix #1205

Add gif for DifferenceOperator and SymmetricDifferenceOperator


Stop bubbling from a child in a custom element

I'm working with angular2 and polymer. All seems to wok well but I found a strange behaviour on the <property>-changed propagation. I have a custom element with a property named value and inside it there is an paper-input that has the same property name. When I write in the paper-input the value-changed event propagates outside the custom element that I don't want. Is there a way to avoid the bubbling?
Ciao Angelo

Afaik not. You need to check the rootTarget in your handler. That would be Polymer.dom(e).rootTarget.

Duplicate #3226 et.al.
The check Polymer internally uses is e.path[0] === e.target.
Can be closed?

The problem is the event from the child cannot stopped with stopPropagation since on chrome, after the first stopPropagation, all the same kind of events don't work anymore; this is dued to the caching events management.

I did some check on the specification on ShadowDom and I must say the use of stopPropagation with the cache is buggy. In particular, according to https://www.w3.org/TR/shadow-dom/#event-dispatch

For each object in event path where the relative target is same as the object, invoke its event listeners, with event event, as long as event's stop propagation flag is unset

So if I call stopPropagation the event should be removed from the cache (see the problems with chrome).
Ciao Angelo

In attachment you'll find the patch to let work stopPropagation.
Ciao Angelo
patch.txt

In 2.0 these events no longer bubble. Please reopen this issue if you can supply a JSBin which shows the issue.


Screen reader reads text 3 times

I think because of the animations or some other DOM changes, the role="alert" on the toaster causes the screen reader to read the text 3 times.
I'm not sure the fix for this, but I would think having a nested div for the role="alert" that does not change would fix it.
Example of the issue on the docs but also happens on syntax.fm
https://github.com/user-attachments/assets/d23a1a50-bade-40c6-a04f-ace7af283af9

@wesbos Thanks for reporting this. If anyone can make a PR, happy to merge!


link to github.com/gojp/goreportcard from the site

so people can find the repo and contribute etc.

Fixed in the latest redesign, the link is in the top right corner labeled "GitHub"


Traefik

Is traefik not an example of "Products with Integrated ACME support"?

Sorry I don't use Traefik so you'd have to tell me.
Does it use some other tool for its acme or does it provide that itself? Feel free to submit a PR for the json file change.

Cool, we definitely mention Lego already, but we can also mention Traefik.