Prediction result : Tranformer4rec Test prediction without triton library

❓ Questions & Help
I have successfully build the transformer4rec model using train data. now i want know how the recommendations look . need to see top 5,10 recommendations for each user and each instance. comparing this with my actual data.
and need to see the affinity score for each recommendation.
Is there a way to check this right after the model building.
Details

You can do local (offline) inference meaning, you can generate topK recommendations for your test set via:
test_paths=<set the path to your test set>
trainer.args.predict_top_k = 20  # you can change this value
prediction = trainer.predict(test_paths)
print(prediction)
print(prediction.predictions)

if you want to generate ndcg and recall metrics for your topK results, you need to write your own code based on the results coming from the code above to calculate these metrics.  you can see how the metrics are calculated here.

Thanks.. this would do

i ran the code for prediction task but ended with error while initiating
ENVIRONMENT : Kaggle Notebook
This my test data stored as parquet file

This is the error im getting while running the prediction code

@DilipKumar3 you need to transform your data with NVTabular first. I see your test dataset is not transformed.

Hi @DilipKumar3 ,
I was having the same issue. Try this instead
test_paths=
trainer.test_dataset_or_path = test_paths
trainer.args.predict_top_k = 20
prediction = trainer.predict(test_paths)
cc: @rnyak

Thanks @rohan-rajput-01  . Its working
How can i interpret the predictions. its has arrays of values. @rnyak

@DilipKumar3 try this
print(prediction)
print(prediction.predictions)

Thanks @rohan-rajput-01  This was very helpful .
just to confirm my understanding , prediction gives three arrays labels, predictions.prediction contains two arrays.
lables = session id
predictions.predictions [0] = recommendated items (top k ) sorted in the order.
prediction.predictions[1] = score for each predictions (lower the values better the model)
@rnyak

@DilipKumar3  for your comment prediction.predictions[1] = score for each predictions (lower the values better the model) the higher the results meaning this product has higher chance (score) to be recommended.  So you would want to recommend top K items that has the highest scores.

@rnyak  Thanks for the clarification.

Above table is the one record of prediction.predictions,
658 is top 1 recommended item for the user label 8456. But all the scores are negative.
is there specific reason for that ?

@rnyak i have asked a clarification of one other question that i asked above in https://github.com/NVIDIA-Merlin/Transformers4Rec/issues/607
please give reply for that

@rnyak i have asked a clarification of one other question that i asked above in https://github.com/NVIDIA-Merlin/Transformers4Rec/issues/607 please give reply for that

Yes, I responded there already :)

@rnyak Thanks for the clarification.

Above table is the one record of prediction.predictions, 658 is top 1 recommended item for the user label 8456. But all the scores are negative. is there specific reason for that ?

@rnyak Can you please explain this.
thanks!!

@alan-ai-learner  are these scores coming from your own dataset or from our example notebooks?
What you getting are the logit values, they are not probabilities. so they can be negatives. Furthermore, we use torch.nn.LogSoftMax (see code here)  that the returns tensors values in the range [-inf, 0).
hope that helps.

On your dataset, I'm getting this.
Thanks for the explanation.

Closing this ticket since there is not recent activity. You can reopen it if necessary.

It's not working for me @rnyak


Fix deployment code for master

Building and deploying the top commit of master at the moment:
https://github.com/UC-Davis-molecular-computing/scadnano/commit/9432742a5a99ce95f880c872e1442aa74d40e9ad leads to errors in production.
The svg-png-caching code is running into undefined errors and the ReactBootstrap isn't loaded properly (moving the main.dart.js script after the ReactBootstrap import works, but it's strange how this error doesn't show up in development).
I tried pushing to gh-pages and fixing it, but was unable to find identify the cause of the svg-png-caching code. In the end, I deleted the deployments commits from the gh-pages repository, reverting to the previous commit.
Hopefully this can be tested with a local build server (maybe by running webdev build?) instead of deploying to gh-pages

Error message:
index.html:301 Uncaught TypeError: dispatch_zoom_threshold_callback is not a function
    at Object.before_zoom (index.html:301)
    at Object.beforeZoom (svg-pan-zoom.js:1740)
    at Object.beforeZoom (svg-pan-zoom.js:558)
    at ShadowViewport.setCTM (svg-pan-zoom.js:371)
    at SvgPanZoom.zoomAtPoint (svg-pan-zoom.js:775)
    at SvgPanZoom.handleMouseWheel (svg-pan-zoom.js:741)
    at SVGSVGElement.wheelListener (svg-pan-zoom.js:682)


First draft of Homepage

Completed the first draft of the Homepage.
Created:

A responsive layout of the Homepage
ViewCart and TaskBar components
ItemEntry and Category components
assets directory for svg files.

Accidentally created two pull requests, sorry!


Network Instance Questions

Hello, I am a student who is using laikaboss
I have a question about network instances and distributed processing.
As far as I'm concerned,
According to the white paper, laikaboss has brokers and walkers, which makes it faster.
If what I understand is correct, the more walkers, the faster the file scanning should be.
However, the more walkers there are, the slower it seems.
Do you happen to know the solution?

The composition of my experiment is as follows.
broker: ./laikad.py --broker-frontend=tcp://:5558 --broker-backend=tcp://:5559 --worker-connect=tcp://localhost:5559 -p 8
Client: ./cloudscan.py ~/FILE PATH/ -R -r -a tcp://"brokerIP:5559"
worker1: ./laikad.py --no-broker --worker-connect=tcp://brokerIP:5559 -a -p 8
worker2: ./laikad.py --no-broker --worker-connect=tcp://brokerIP:5559 -a -p 8
worker3: ./laikad.py --no-broker --worker-connect=tcp://brokerIP:5559 -a -p 8

Thank you.

There really isn't enough context here about the underlying systems you're running this on.
Also, LaikaBOSS is basically abandonware now 😢 . Try out Target/Strelka, which is ostensibly a fork.

@agrajag9
thank you


Explicit type for a record loses Witness.Aux

See: https://gist.github.com/jonifreeman/6533463
Switching to
type FieldType[K, V] = (K, V)

fixes the problem since Witness.Aux is no longer needed.
See: https://gist.github.com/jonifreeman/6570276

it would be great  if record type literals could be written in multiple lines :
 type Error = Record.`
                       "k1" -> String,
                       "k2" -> Long
                    `.T

Is it generally possible?


Incorrect coverage code w/ TS private arguments

🐛 Bug Report
The code executed by Jest with --coverage is incorrect when using TS private arguments. It seems to be transpiled as such, which isn't valid because this can only be used after the super call:
class Bar {}

/* istanbul ignore next */
cov_s64sx2cd4.s[0]++;
exports.Foo = class Foo extends Bar {
  constructor(arg) {
    /* istanbul ignore next */
    this.arg = arg;
    cov_s64sx2cd4.f[0]++;
    cov_s64sx2cd4.s[1]++;
    super();
  }
};

Without --coverage it gets transpiled as such:
class Bar {}

exports.Foo = class Foo extends Bar {
  constructor(arg) {
    super();
    this.arg = arg;
  }
};

Interestingly, it works if I "transpile" the construct myself. The generated code becomes:
class Bar {}

/* istanbul ignore next */
cov_s64sx2cd4.s[0]++;
exports.Foo = class Foo extends Bar {
  constructor(arg) {
    /* istanbul ignore next */
    cov_s64sx2cd4.f[0]++;
    cov_s64sx2cd4.s[1]++;
    super();

    /* istanbul ignore next */
    cov_s64sx2cd4.s[2]++;
    this.arg = arg;
  }
};

To Reproduce
Code that triggers the bug:
class Bar {
}

class Foo extends Bar {
    constructor(
        private arg: number,
    ) {
        super();
    }
}

Expected behavior
Should work, but doesn't
Link to repl or repo (highly encouraged)
https://github.com/arcanis/jest-coverage-ts
Run npx envinfo --preset jest
Paste the results here:
  System:
    OS: macOS 10.14.5
    CPU: (8) x64 Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  Binaries:
    Node: 12.3.1 - ~/.nodenv/shims/node
    Yarn: 1.16.0 - ~/.nodenv/shims/yarn
    npm: 6.9.0 - ~/.nodenv/shims/npm
  npmPackages:
    jest: ^24.8.0 => 24.8.0

Is this https://github.com/istanbuljs/istanbuljs/issues/237?

Seems so! I'll close mine as duplicate 👌

You can try to apply that patch locally and see if it fixes the issue? Seems very much to be the same issue though 🙂

@arcanis wanna verify with 7.5.0?


Plugin Poetry v0.5.2 is incompatible with last version of IntelliJ

Describe the bug
The Poetry plugin v0.5.2 does not appear on the IntelliJ marketplace. v0.1.3 does appear.
I tried to install it manually and IntelliJ threw the next error:
Plugin 'Poetry' version 0.5.2 is incompatible with this installation: since build 202.7319.64 > IU-202.7319.50

To Reproduce
Steps to reproduce the behavior:

Using IntelliJ 2020.2.2, build 202.7319.50
Download Poetry 0.5.2
Go to Plugins > ⋮ > Install plugin from disk
Select poetry plugin
See error

Expected behavior

The last version of the plugin should show up in the maketplace. It doesn't.
Should be able to install the last version manually.

Screenshots

Environments (please complete the following information):

IDE: IntelliJ 2020.2.2, build 202.7319.50
OS: Windows 10 v2004
Poetry Version 1.1.0
Plugin version 0.5.2

@cruzelante098
Thank you for creating this issue.
Oh, I'm sorry I didn't care IntelliJ versions.
I will resolve the dependency.

v0.5.3 works with IntelliJ 2020.2.2 #IU-202.7319.50. I'm closing this issue.
Thank you @koxudaxi


tests: Move test container images to Amazon Elastic Container Registry

Still hitting rate limiting on AWS ECR, but over a shorter time period, and I might be able to create a read-only token that increases that limit
From https://github.com/mitogen-hq/mitogen/pull/790/checks?check_run_id=1745766546

Error response from daemon: toomanyrequests: Rate exceeded

You may receive a 429: Too Many Requests error or a ThrottleException error from one or more Amazon ECR commands or API calls. If you are using Docker tools with Amazon ECR, then for Docker versions 1.12.0 and greater, you may see the error message TOOMANYREQUESTS: Rate exceeded. For versions of Docker below 1.12.0, you may see the error Unknown: Rate exceeded.
This indicates that you are calling a single endpoint in Amazon ECR repeatedly over a short interval, and that your requests are getting throttled. Throttling occurs when calls to a single endpoint from a single user exceed a certain threshold over a period of time.
Various API operations in Amazon ECR have different throttles.
For example, the throttle for the GetAuthorizationToken action is 20 transaction per second (TPS), with up to a 200 TPS burst allowed. In each region, each account receives a bucket that can store up to 200 GetAuthorizationToken credits. These credits are replenished at a rate of 20 per second. If your bucket has 200 credits, you could achieve 200 GetAuthorizationToken API transactions per second for one second, and then sustain 20 transactions per second indefinitely.
-- https://docs.aws.amazon.com/AmazonECR/latest/userguide/common-errors.html

Still hitting rate limiting on AWS ECR, but over a shorter time period, and I might be able to create a read-only token that increases that limit
From https://github.com/mitogen-hq/mitogen/pull/790/checks?check_run_id=1745766546

Error response from daemon: toomanyrequests: Rate exceeded

You may receive a 429: Too Many Requests error or a ThrottleException error from one or more Amazon ECR commands or API calls. If you are using Docker tools with Amazon ECR, then for Docker versions 1.12.0 and greater, you may see the error message TOOMANYREQUESTS: Rate exceeded. For versions of Docker below 1.12.0, you may see the error Unknown: Rate exceeded.
This indicates that you are calling a single endpoint in Amazon ECR repeatedly over a short interval, and that your requests are getting throttled. Throttling occurs when calls to a single endpoint from a single user exceed a certain threshold over a period of time.
Various API operations in Amazon ECR have different throttles.
For example, the throttle for the GetAuthorizationToken action is 20 transaction per second (TPS), with up to a 200 TPS burst allowed. In each region, each account receives a bucket that can store up to 200 GetAuthorizationToken credits. These credits are replenished at a rate of 20 per second. If your bucket has 200 credits, you could achieve 200 GetAuthorizationToken API transactions per second for one second, and then sustain 20 transactions per second indefinitely.
-- https://docs.aws.amazon.com/AmazonECR/latest/userguide/common-errors.html

Another option could be https://quay.io

Another option could be https://quay.io

Another option could be https://quay.io

Thank you, but unfortunately Quay is a paid service, and the project doesn't have a budget at this time.

Another option could be https://quay.io

Thank you, but unfortunately Quay is a paid service, and the project doesn't have a budget at this time.

To be clear, this is not a permanent solution. It's a snapshot running under my personal AWS account, with a bus factor of one. I need to work out something more permanent, and shared.

To be clear, this is not a permanent solution. It's a snapshot running under my personal AWS account, with a bus factor of one. I need to work out something more permanent, and shared.


fix(backend): RequestInit.duplex now is required if body is provided in undici

Description
RequestInit.duplex now is required if body is provided:

https://github.com/nodejs/node/issues/46221
https://github.com/whatwg/fetch/pull/1457

Checklist

[ ] npm test runs as expected.
[ ] npm run build runs as expected.
[ ] (If applicable) JSDoc comments have been added or updated for any package exports
[ ] (If applicable) Documentation has been updated

Type of change

[ ] 🐛 Bug fix
[ ] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

!snapshot

Hey @octoper - the snapshot version command generated the following package versions:

Package
Version

@clerk/backend
1.2.2-snapshot.v72984d5

@clerk/chrome-extension
1.0.17-snapshot.v72984d5

@clerk/clerk-js
5.6.0-snapshot.v72984d5

@clerk/elements
0.6.0-snapshot.v72984d5

@clerk/clerk-expo
1.1.9-snapshot.v72984d5

@clerk/express
0.0.11-snapshot.v72984d5

@clerk/fastify
1.0.13-snapshot.v72984d5

gatsby-plugin-clerk
5.0.0-beta.45

@clerk/localizations
2.4.4-snapshot.v72984d5

@clerk/nextjs
5.1.4-snapshot.v72984d5

@clerk/clerk-react
5.2.3-snapshot.v72984d5

@clerk/remix
4.0.14-snapshot.v72984d5

@clerk/clerk-sdk-node
5.0.10-snapshot.v72984d5

@clerk/shared
2.2.2-snapshot.v72984d5

@clerk/testing
1.1.6-snapshot.v72984d5

@clerk/themes
2.1.9-snapshot.v72984d5

@clerk/types
4.6.0-snapshot.v72984d5

Tip: Use the snippet copy button below to quickly install the required packages.
@clerk/backend
npm i<EMAIL_ADDRESS>--save-exact

@clerk/chrome-extension
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-js
npm i<EMAIL_ADDRESS>--save-exact

@clerk/elements
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-expo
npm i<EMAIL_ADDRESS>--save-exact

@clerk/express
npm i<EMAIL_ADDRESS>--save-exact

@clerk/fastify
npm i<EMAIL_ADDRESS>--save-exact

gatsby-plugin-clerk
npm i<EMAIL_ADDRESS>--save-exact

@clerk/localizations
npm i<EMAIL_ADDRESS>--save-exact

@clerk/nextjs
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-react
npm i<EMAIL_ADDRESS>--save-exact

@clerk/remix
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-sdk-node
npm i<EMAIL_ADDRESS>--save-exact

@clerk/shared
npm i<EMAIL_ADDRESS>--save-exact

@clerk/testing
npm i<EMAIL_ADDRESS>--save-exact

@clerk/themes
npm i<EMAIL_ADDRESS>--save-exact

@clerk/types
npm i<EMAIL_ADDRESS>--save-exact

!snapshot

Hey @octoper - the snapshot version command generated the following package versions:

Package
Version

@clerk/backend
1.2.2-snapshot.vef44e76

@clerk/chrome-extension
1.0.17-snapshot.vef44e76

@clerk/clerk-js
5.6.0-snapshot.vef44e76

@clerk/elements
0.6.0-snapshot.vef44e76

@clerk/clerk-expo
1.1.9-snapshot.vef44e76

@clerk/express
0.0.11-snapshot.vef44e76

@clerk/fastify
1.0.13-snapshot.vef44e76

gatsby-plugin-clerk
5.0.0-beta.45

@clerk/localizations
2.4.4-snapshot.vef44e76

@clerk/nextjs
5.1.4-snapshot.vef44e76

@clerk/clerk-react
5.2.3-snapshot.vef44e76

@clerk/remix
4.0.14-snapshot.vef44e76

@clerk/clerk-sdk-node
5.0.10-snapshot.vef44e76

@clerk/shared
2.2.2-snapshot.vef44e76

@clerk/testing
1.1.6-snapshot.vef44e76

@clerk/themes
2.1.9-snapshot.vef44e76

@clerk/types
4.6.0-snapshot.vef44e76

Tip: Use the snippet copy button below to quickly install the required packages.
@clerk/backend
npm i<EMAIL_ADDRESS>--save-exact

@clerk/chrome-extension
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-js
npm i<EMAIL_ADDRESS>--save-exact

@clerk/elements
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-expo
npm i<EMAIL_ADDRESS>--save-exact

@clerk/express
npm i<EMAIL_ADDRESS>--save-exact

@clerk/fastify
npm i<EMAIL_ADDRESS>--save-exact

gatsby-plugin-clerk
npm i<EMAIL_ADDRESS>--save-exact

@clerk/localizations
npm i<EMAIL_ADDRESS>--save-exact

@clerk/nextjs
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-react
npm i<EMAIL_ADDRESS>--save-exact

@clerk/remix
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-sdk-node
npm i<EMAIL_ADDRESS>--save-exact

@clerk/shared
npm i<EMAIL_ADDRESS>--save-exact

@clerk/testing
npm i<EMAIL_ADDRESS>--save-exact

@clerk/themes
npm i<EMAIL_ADDRESS>--save-exact

@clerk/types
npm i<EMAIL_ADDRESS>--save-exact

!snapshot

Hey @octoper - the snapshot version command generated the following package versions:

Package
Version

@clerk/backend
1.2.2-snapshot.vff20ad3

@clerk/chrome-extension
1.0.17-snapshot.vff20ad3

@clerk/clerk-js
5.6.0-snapshot.vff20ad3

@clerk/elements
0.6.0-snapshot.vff20ad3

@clerk/clerk-expo
1.1.9-snapshot.vff20ad3

@clerk/express
0.0.11-snapshot.vff20ad3

@clerk/fastify
1.0.13-snapshot.vff20ad3

gatsby-plugin-clerk
5.0.0-beta.45

@clerk/localizations
2.4.4-snapshot.vff20ad3

@clerk/nextjs
5.1.4-snapshot.vff20ad3

@clerk/clerk-react
5.2.3-snapshot.vff20ad3

@clerk/remix
4.0.14-snapshot.vff20ad3

@clerk/clerk-sdk-node
5.0.10-snapshot.vff20ad3

@clerk/shared
2.2.2-snapshot.vff20ad3

@clerk/testing
1.1.6-snapshot.vff20ad3

@clerk/themes
2.1.9-snapshot.vff20ad3

@clerk/types
4.6.0-snapshot.vff20ad3

Tip: Use the snippet copy button below to quickly install the required packages.
@clerk/backend
npm i<EMAIL_ADDRESS>--save-exact

@clerk/chrome-extension
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-js
npm i<EMAIL_ADDRESS>--save-exact

@clerk/elements
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-expo
npm i<EMAIL_ADDRESS>--save-exact

@clerk/express
npm i<EMAIL_ADDRESS>--save-exact

@clerk/fastify
npm i<EMAIL_ADDRESS>--save-exact

gatsby-plugin-clerk
npm i<EMAIL_ADDRESS>--save-exact

@clerk/localizations
npm i<EMAIL_ADDRESS>--save-exact

@clerk/nextjs
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-react
npm i<EMAIL_ADDRESS>--save-exact

@clerk/remix
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-sdk-node
npm i<EMAIL_ADDRESS>--save-exact

@clerk/shared
npm i<EMAIL_ADDRESS>--save-exact

@clerk/testing
npm i<EMAIL_ADDRESS>--save-exact

@clerk/themes
npm i<EMAIL_ADDRESS>--save-exact

@clerk/types
npm i<EMAIL_ADDRESS>--save-exact

I'm getting the following error:
Error: cannot clone body after it is used
    at clone (.../build/server.js:36692:15)
    at new Request2 (.../build/server.js:37302:99)
    at new NodeRequest (.../build/server.js:37754:9)
    at NodeRequest.clone (.../build/server.js:37760:16)
    at patchRequest (.../node_modules/@clerk/remix/src/ssr/utils.ts:128:33)
    at loadOptions (.../node_modules/@clerk/remix/src/ssr/loadOptions.ts:15:43)
    at Object.getAuth2 (.../node_modules/@clerk/remix/src/ssr/getAuth.ts:15:25)

The problem is that my code looks like this:
export async function action(args: ActionArgs) {
  const formData = await args.request.formData();
  const userId = await getAuth(args);
  ...
}

In other words, the request body is read before calling clerk.getAuth. Since Clerk attempts to clone the request, this causes an error. Is it necessary to clone the request to work around this duplex issue?
The workaround is to always call getAuth first:
export async function action(args: ActionArgs) {
  const userId = await getAuth(args);
  const formData = await args.request.formData();
  ...
}

But this is error-prone and easy to forget to do or accidentally change in a large project. At the minimum, this should be documented in the Clerk Remix guide. Ideally, this limitation would be removed.


Add unit tests for video call

This PR covers Call Visualizer with Unit tests and adds extensive mocking
for Styles and Props. This is aimed for simplifying further testing in
future if necessary.
MOB-1875

!squash

!merge


setupTomcat task fails to update conf/server.xml file if non-default ajpPort set in the project

If project has non-default ajpPort set in the build.gradle, then cuba gradle plugin tries to set it for AJP connector in tomcat/conf/server.xml.
But this connector is commented out in default server.xml file.
So the task shows the following message:
conf/server.xml has not been updated: cannot find AJP Connector node
As a result server.xml file is in a default state with default ports (HTTP connector port, AJP connector port, server shutdown port)
See also https://youtrack.cuba-platform.com/issue/STUDIO-9123

For QA: check smoke for IDP addon after fix

The AJP port cannot be changed, an error message appears cannot find AJP Connector node.
If you uncomment this port in a file conf/server.xml, and change AJP port in Project Properties, the port will be commented out again.


StaticFileServer is unable to serve files that contain a space in their filename

When requesting a file called "hello world" browsers use %20 encoding for any spaces, meaning that they would request "hello%20world". StaticFileServer does not decode these correctly, and so will refuse to serve the "hello world" file. This might be linked to #857.

Blocked by #857.


Error raise when WasmPlugin access environment variable of istio-proxy.

Bug Description
Error raise when WasmPlugin access environment variable of istio-proxy.
As below error: envoy wasm	wasm log: error reading vm configuration: error status returned by host: not found
istio wasm conf
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: wasm-example
  namespace: default
spec:
  selector:
    matchLabels:
      app: httpbin
  url: oci://johnzheng/wasmws:v2
  pluginConfig:
    header_1: "some_value_1"
    header_2: "another_value"
  vmConfig:
    env:
    - name: TRUST_DOMAIN
      value: "cluster1.local1"
    - name: POD_NAME
      valueFrom: HOST

main.go
package main

import (
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
)

func main() {
	proxywasm.SetVMContext(&vmContext{})
}

type vmContext struct{}

func (*vmContext) OnVMStart(vmConfigurationSize int) types.OnVMStartStatus {
	data, err := proxywasm.GetVMConfiguration()
	if err != nil {
		proxywasm.LogCriticalf("error reading vm configuration: %v", err)
	}

	proxywasm.LogInfof("vm config: %s", string(data))
	return types.OnVMStartStatusOK
}

// Implement types.VMContext.
func (*vmContext) NewPluginContext(uint32) types.PluginContext {
	return &pluginContext{}
}

type pluginContext struct {
	// Embed the default plugin context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultPluginContext
}

// Override types.DefaultPluginContext.
func (ctx pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus {
	data, err := proxywasm.GetPluginConfiguration()
	if err != nil {
		proxywasm.LogCriticalf("error reading plugin configuration: %v", err)
	}

	proxywasm.LogInfof("plugin config: %s", string(data))
	return types.OnPluginStartStatusOK
}

Version
$ ./istioctl version
client version: 1.14.4
control plane version: 1.14.4
data plane version: 1.13.2 (3 proxies), 1.14.4 (13 proxies)

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:58:47Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.14-eks-18ef993", GitCommit:"ac73613dfd25370c18cbbbc6bfc65449397b35c7", GitTreeState:"clean", BuildDate:"2022-07-06T18:06:50Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

Additional Information
No response

you have two version proxies, what's the version of failed one?

ignore above, I can reproduce this.

@johnzheng1975
can you find out which proxy show this log, and run following cmd:
kubectl exec -it `kubectl get po -nistio-system| grep istiod | grep Running | awk '{print $1}'` -nistio-system -- pilot-discovery request get '/debug/ecdsz?proxyID=<you_proxy_id>'

config_dump_full.yaml.txt

how to get my proxyID? Thanks. @zirain
BTW, I am also wondering istio handle everything well, error just caused by: I did not write the code correctly to get env value.

how to get my proxyID? Thanks. @zirain BTW, I am also wondering istio handle everything well, error just caused by: I did not write the code correctly to get env value.

I do think so, there's nothing about env stuff. this can be simply reproduce when you try to get configuration in wasm.
but seems you configured them with WasmPlugin, ECDS can not be found by config_dump.
your proxyID should be httpbin-5dcc45899f-h7x7w, can you run the cmd again?

Thanks @zirain
Here is the result,
$ kubectl exec -it `kubectl get po -nistio-system| grep istiod | grep Running | awk '{print $1}'` -nistio-system -- pilot-discovery request get '/debug/ecdsz?proxyID=httpbin-5dcc45899f-h7x7w'
no watched ExtensionConfigurationType found, proxyID: httpbin-5dcc45899f-h7x7w

I did not have any successful example to access vmconfig environment var from host. Do you have such examples? no matter is is istio example or just envoy example. Thanks
BTW, my env and link:
https://github.com/johnzheng1975/wasm-study/blob/main/wasm-extension/try-vm-env-wasm/go.sum
https://pkg.go.dev/github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types
https://github.com/tetratelabs/proxy-wasm-go-sdk/blob/2805000cdcce3cb36984fce721abf8c4b41d470d/proxywasm/hostcall.go#L844

try this https://github.com/zirain/playground/tree/master/go/wasm-extensions/display-metadata

no watched ExtensionConfigurationType found, proxyID: httpbin-5dcc45899f-h7x7w

do you removed the WasmPlugin?

It works.
Sorry.   @zirain .  It is not istio issue, just because i had not wrote code correctly for getting os env.
Should use myvar := os.Getenv("PATH") to read env var.
This is wasmplugin config file:
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: wasm-example
  namespace: default
spec:
  selector:
    matchLabels:
      app: httpbin
  url: oci://johnzheng/wasmws:v3
  pluginConfig:
    header_1: "some_value_1"
    header_2: "another_value"
  vmConfig:
    env:
    - name: TRUST_DOMAIN
      value: "cluster1.local1"
    - name: PATH
      valueFrom: HOST

Here is main.go
// Copyright 2020-2021 Tetrate
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
        "os"
)

func main() {
	proxywasm.SetVMContext(&vmContext{})
}

type vmContext struct {
	// Embed the default VM context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultVMContext
}

// Override types.DefaultVMContext.
func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext {
	return &pluginContext{}
}

type pluginContext struct {
	// Embed the default plugin context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultPluginContext
}

// Override types.DefaultPluginContext.
func (*pluginContext) NewHttpContext(contextID uint32) types.HttpContext {
	return &httpHeaders{contextID: contextID}
}

type httpHeaders struct {
	// Embed the default http context here,
	// so that we don't need to reimplement all the methods.
	types.DefaultHttpContext
	contextID uint32
}

// Override types.DefaultHttpContext.
func (ctx *httpHeaders) OnHttpRequestHeaders(numHeaders int, endOfStream bool) types.Action {
	err := proxywasm.ReplaceHttpRequestHeader("test", "best")
	if err != nil {
		proxywasm.LogCritical("failed to set request header: test")
	}

	myvar := os.Getenv("PATH")
	proxywasm.LogInfof("hello variable: %s",myvar)

	hs, err := proxywasm.GetHttpRequestHeaders()
	if err != nil {
		proxywasm.LogCriticalf("failed to get request headers: %v", err)
	}

	for _, h := range hs {
		proxywasm.LogInfof("request header --> %s: %s", h[0], h[1])
	}
	return types.ActionContinue
}

// Override types.DefaultHttpContext.
func (ctx *httpHeaders) OnHttpResponseHeaders(numHeaders int, endOfStream bool) types.Action {
	hs, err := proxywasm.GetHttpResponseHeaders()
	if err != nil {
		proxywasm.LogCriticalf("failed to get response headers: %v", err)
	}

	for _, h := range hs {
		proxywasm.LogInfof("response header <-- %s: %s", h[0], h[1])
	}
	return types.ActionContinue
}

// Override types.DefaultHttpContext.
func (ctx *httpHeaders) OnHttpStreamDone() {
	proxywasm.LogInfof("%d finished", ctx.contextID)
}

This is log print, include the host variable PATH
2022-09-21T08:28:04.566309Z	info	envoy wasm	wasm log default.wasm-example: hello variable: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin


Created Dragon Game #1

Related Issue
Javascript Dragon Game
Closes: #452
Describe the changes you've made
I've made a fun dragon game with javascript. t's a dragon game with audio effects and simple UI. Made with HTML, CSS, and Javascript.
Describe if there is any unusual behavior of your code
NA
Checklist:

[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[x] New and existing unit tests pass locally with my changes.

Screenshots

@991rajat  please review the PR

Thanks for your contribution !!


Enumerated values

Add capability to emit rust interface to hardware memory map using enumerated values defined in the SVD file.

:umbrella: The latest upstream changes (presumably 57e9e72) made this pull request unmergeable. Please resolve the merge conflicts.

Thinking again about this: the enum could contain an extra, hidden variant to represent the "unexpected" register value. So, basically:
enum State {
    /// A
    ValidStateA,
    /// B
    ValidStateB,
    /// C
    ValidStateC,
    #[doc(hidden)]
    __InvalidState__,
}

Then the read method would look more "normal"; no Option in the return value:
// From the POV of the user, this can only return: ValidState{A,B,C}
fn read(&self) -> State { .. }

The downside is that the user would be able to create __InvalidState__.
If we are paranoid about the user misusing __InvalidState__, we could add a debug assertion to write to prevent the user from passing __InvalidState__. Likewise, we could add a debug assertion to the  PartialEq implementation as well.
In any case, I'm much more concerned about the ergonomics of this. If the user has to import a bunch of enums to be able to write/modify a register then the gain in type safe will come at the price of losing ergonomics. The trade off has to be assessed carefully. Or some way to avoid the loss in ergonomics must be devised.

New attempt in #52. Closing this PR in favor of that one.


refactor(closest_pitch): ♻ move class to a new file

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1018

Relevant Lines:
1018

💛  - Coveralls


Highlighter doesn't parse html entities

Bug: What is the current behavior?
Currently, snippets and highlights are rendering like:
&nbsp;&nbsp;fadsfasfasd

Bug: What is the expected behavior?
For the highlighter to parse htmlentities, at least ignore them or something. Doing a dangerouslySetInnerHTML is how the output should look.
What project are you opening an issue for?

react-instantsearch

What is the version you are using? Always use the latest one before opening a bug issue.
3.2.1

Any updates?

Not yet, this week was very busy. We will make sure this issue gets resolved asap.

Could you add a fiddle with data that causes this issue? That way it's easier to replicate

I think we will go with not supporting this use case. We don't want to use dangerouslySetInnerhtml for its obvious XSS problems. This means that we can't support html entities in strings.
See https://codepen.io/anon/pen/PpQbbX. There are three ways to make a no break space to show up, we use the first one, since we don't control the string.
However you can change your source text to be highlighted to not have html entities, but rather unicode escape codes.
// your current text
const String = () =>
{'ok works'}
;
ReactDOM.render(<String />, document.getElementById('string'));

// same text, but with unicode
const Unicode = () =>
{'ok\u00A0works'}
;
ReactDOM.render(<Unicode />, document.getElementById('unicode'));

// if you'd write it straight in jsx (no string) it can
// work, but we can't do that
const Naked = () =>
ok works
;
ReactDOM.render(<Naked />, document.getElementById('naked'));

I'm sorry that this is the case, but adding html entity escaping or rendering is nontrivial.
See also https://github.com/facebook/jsx/issues/4
Have a nice day!

Are there any plans for a Vue implementation?

You can use v-html with the _highlightResult directly for that, but we won't add html interpolation due to the security risks @blowsie


Memory leak with AsyncRabbitTemplate

3.0.10
When i use org.springframework.amqp.rabbit.AsyncRabbitTemplate#sendAndReceive(java.lang.String, org.springframework.amqp.core.Message) and when get reply, nothing is clearing or canceling org.springframework.amqp.rabbit.RabbitFuture#timeoutTask and for every sending (if i get answer) i got memory leak , as shown in the photo (find this every time with profiler)

I got bug with this snippet of code:
                Message message = MessageBuilder
                        .withBody(mapper.writeValueAsBytes(photoMapper.mapToCkSendProcessDto(imageModel)))
                        .setContentType(MessageProperties.CONTENT_TYPE_JSON).build();

                log.debug("Send image {} to process", imageModel.getId());
                return asyncRabbitTemplate.sendAndReceive(queue.getActualName(), message)
                        .thenApply(mes -> {
                            log.debug("Get image {} after process", imageModel.getId());
                            try {
                             // ..........
                            } catch (IOException e) {
                                throw new RuntimeException(e);
                            }
                        }).handle((el, err) -> err == null ? el : null);

    @Bean
    public ConnectionFactory rabbitConnectionFactory(com.rabbitmq.client.ConnectionFactory cf) {
        return new CachingConnectionFactory(cf);
    }

    @Bean
    public RabbitTemplate template(ConnectionFactory connectionFactory) {
        RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        return rabbitTemplate;
    }

    @Bean
    public AsyncRabbitTemplate asyncRabbitTemplate(RabbitTemplate template) {
        AsyncRabbitTemplate asyncRabbitTemplate = new AsyncRabbitTemplate(template);
        asyncRabbitTemplate.setReceiveTimeout(6000000);
        return asyncRabbitTemplate;
    }

I’m on sick day off, so if you have an idea how to fix, feel free to contribute it then! I’ll be happy to review when I’m back.
Thanks

I’m on sick day off, so if you have an idea how to fix, feel free to contribute it then! I’ll be happy to review when I’m back.
Thanks

I wish you a speedy recovery ! Okay I will try to fix it :)

Hi @tetrade !
I'm back from leave and ready to tackle this.
Let me know if you have started looking into this, so we don't cross each other.
We have a release next Monday if that matters somehow...
Thanks

So, apparently the test and fix are simple, so I decided to go ahead and do it myself.
Feel free to contribute anything else!

Yeah ! I just study and don't see your message :) Next time you will see my contribution 👍
Good resolve of the bug


Observed issues with connectionCount load balancing

Recently I saw a cluster in a state that seemed to indicate a bug with connectionCount load balancing:

It was clear from metrics that certain brokers were "preferring" certain historical nodes
The brokers were configured to use connectionCount load balancing
Restarting brokers didn't fix the imbalance, but did shift it around to different historical nodes
Changing load balancing to "random" did fix the imbalance

I think you should consider the weight of segment count for each connection.

I think you should consider the weight of segments count for each connection.
Every query wanna use least connection, so it will chose the historical which has most segments.


Provide option to delete source maps after upload

I've just started implementing sentry using the JavaScript SDK
Next I've tried to upload source maps using this webpack plugin. In the course of it I obviously had to generate sourcemaps for my production build and was wondering: how do I prevent those files from being deployed (which would mean revealing all source code)
I was quite surprised there's nothing built-in this webpack plugin because it's a major security concern! As I can see there's an old issue about this topic which has been closed: https://github.com/getsentry/sentry-webpack-plugin/issues/56
The mentioned solutions can be used though it feels more like a workaround. It was also mentioned that a custom webpack plugin might be an option and my immediate thought was: "why is it not part of this plugin?"
A simple option could be "deleteSourceMaps" which would delete all uploaded source maps at the end of the build process. Even if the underlying sentry-cli doesn't support this, it shouldn't be difficult to extend this webpack plugin.
Are there any reasons not to do this?

Please support this feature. Workarounds are confusing and don't work for me.

@AbhiPrasad I'm glad to hear you're open for re-evaluation

could lead to files being deleted accidentally

I would set this new option to default false so that you explicitly need to enable it. This would prevent any unintended behaviour as the logic is quite clear: delete all source maps that have been uploaded at the end of the build process.

@lforst yes that's true and intuitive in my opinion
The new option would only delete what's uploaded by the plugin. So if you only upload subset of your source maps, only those files will be deleted afterwards.
The right option name could emphasise this (e.g. "deleteSourceMapsAfterUpload" or "deleteUploadedSourceMaps") so that hardly anyone would expect all source maps get deleted when only a subset is uploaded. I think some documentation would help to avoid any confusion.

deleteUploadedSourceMaps

I think an option named like this is reasonable, it'll require us to make changes to sentry-cli though, since the JS layer doesn't get this info iirc.
We can make this false by default, and then evaluate turning it on based on further user feedback. @lforst what do you think?

Would love this feature to be included in the vite plugin as well!

Any news on this? Or do we need to have some pipeline scripts for that?

Resolved with #244

We released this in the newest version of the plugin btw :)


Pre-commit hooks gives an ImportError

I had an error with the precommit hooks that is same as this issue which seems recent enough to open an issue for: https://github.com/dask/distributed/issues/6013#issue-1184125016

The issue you linked to is closed now. Did it resolve your issue? If not, can you post some more details? Thanks.

Closing due to inactivity. Please reopen if this is still a problem.


Fixed move automation and iframe step iterating (closes #414, closes #418)

\cc @AlexanderMoskovkin, @AndreyBelym

LGTM, also works for ASP tests.

lgtm
/cc @VasilyStrelyaev

\r-

FPR

lgtm


Analysis map plus

Includes conversion of analyses to map to re-use keys for different products
Changes reaction inputs to a map from a repeated ReactionInput to allow easier templating (i.e., so we don't have to keep track of the order of inputs in the definition)
Simplifies a few vessel field names

Questions:

Should the products in a ReactionOutcome also be a map for easier templating? (I think this is not necessary)
Should the components in a ReactionInput be a map? (I think no, even though defining "solute" and "solvent" keys might be nice for solutions, it would be annoying to need a string key for pure species)


feat(glossary) Display Incoming 'IsA' Glossary related entities

Now displays related entities that are related to any Incoming "IsA" glossary terms on a parent term's related entities tab.
For example:
TermA inherits from TermB (TermA has an Outgoing "IsA" relationship defined with TermA) and therefore shows TermA in its Inherits tab. If you go to TermB's profile, you will see all entities related to TermA in TermB's Related Entities tab.
The way the the Related Terms tab works, though, is that we only show Outgoing "IsA" and "HasA" relationships (Inherits and Contains, respectively). So TermB may not show TermA in its Related Terms tab.
Note: This is only 1 level down, so we only show direct Incoming "IsA" relationship children's related entities (much easier and quicker to implement than going all the way through the tree)
A visual example - Container Term has two Incoming "IsA" relationships with Child Term and 2nd Child Term and therefore will show all of their related entities in its own tab as well as its own.

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Just want to take the opportunity to clarify, to add relatedTerms to a term, we would still need to do ingestion via backend right? There is no way/plans to add via UI?

Just want to take the opportunity to clarify, to add relatedTerms to a term, we would still need to do ingestion via backend right? There is no way/plans to add via UI?

@xiphl that is correct - for now! We recognize that that is desired and is scheduled as a followup to happen soon. So you should be able to Add/Delete Contains/Inherits relationships via the UI.

Just want to take the opportunity to clarify, to add relatedTerms to a term, we would still need to do ingestion via backend right? There is no way/plans to add via UI?

@xiphl that is correct - for now! We recognize that that is desired and is scheduled as a followup to happen soon. So you should be able to Add/Delete Contains/Inherits relationships via the UI.

This is highly needed from my side. Looking forward to it so i could let my data stewards work with business glossary independently and effectively via UI. Hope it will arrive soon!

Will merge once CI passes.


ListView crashes application when anything is selected

Description
when selecting any of the items from a ListView, the whole Application crashes with
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Entity(Entity(0))', /home/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/orbtk-api-0.3.1-alpha3/src/widget_base/widget_container.rs:208:14

To Reproduce
Steps to reproduce the behavior:

Create a listview: something like this in my case:

                        ListView::new()
                            .count(findings.words.len())
                            .height(200.0)
                            .width(300.0)
                            .margin(8.0)
                            .enabled(true)
                            .selection_mode(SelectionMode::None)
                            .h_align("center")
                            .items_builder(
                                move |bc, index| {
                                    let fetched = &bc.get_widget(target).clone::<Vec<String>>("list")[index];
                                    //println!("{}", fetched);
                                    TextBlock::new()
                                        .text(format!("{}: {}", index, fetched))
                                    .build(bc)
                                }
                            )
                        .build(ctx)

start the application, and try selecting any of the items, and the application crashes

Screenshots

Platform: Ubuntu 20

Thanks for reporting.
It is strange, I tried the showcase's Listview with SelectionMode::None, and it doesn't crash, The ListView's code seems to be checking for the None case too.
In widget_container.rs, on line 208,  looks like you are trying to clone a property that does not exist for a widget, and it tries to fetch the name for the entity.
Can you check please if you have set names in the template() fn on your custom written widgets?


Having BBD seems to make GM fail

Needs further investigation.

GM seems to error out when trying to get localStorage only with (B)BD. Think I found a patch which makes it work with it which will be out as v5.1.2.

v5.1.2 is now released.


Installation Issues

I'm trying to install the software on a raspberry pi 4, but I'm having issues with the installation. I followed the instructions of the readme, and I watched the PySpectrometer video where he sets it up but I still can't get it to work. Any advice?

Are you able to run libcamera-hello -t 0 and see an image from the camera?
Generally speaking, you want to make sure to run the following:

apt update
apt full-upgrade
apt-get install python-dev
apt-get install libatlas-base-dev
apt-get install libopenblas-dev
pip3 install scipy
pip3 install peakutils

I also modified /boot/config.txt. There is a section #Automatically load overlays for detected cameras. I have the following:
#start_x=1
camera_auto_detect=1
Save all the things and reboot.
You should be able to see an image if the camera is plugged in properly using:
libcamera-hello -t 0

There are many issues with the 'install'
1.) The readme says that as of version 3.0 scipy and peakutils are no longer needed
2.) the current readme says to only use 'sudo apt install python34-opencv' of course that will not work as you must explicitly type 'pip install opencv-python'
There are dozens of 'run' problems, it appears that nobody has tested or tried to build; This is common on +5 year old software, the python has moved on; There are pip tools that generate a 'requirements.txt' file that specify exactly what version of the tools 'numpy' to be used, as this is the core of the problem that the 'code' is using obsolete 'types'; The problem here is chicken and the egg, we don't have a 'working' version so we can't generate the requirements.txt; Only the system that has the working version can generate;
This comment about install is the requirements ( with no version info ) for the version 1.0, of little use to version 3.0 which has been updated, but still now this update is 3 years old;
I Haven't even tried to run on a 'pi' yet ( I have dozens of rasperry's ) I just tried 3 different LINUX systems and all the same problem, the program crashes; Mostly because there is no 'error throw' code, the program trys to read the camera and config a  calibration on first run, and if it doesn't find a 'spectrogram' it just crashes, there should be graceful 'throw' code so that there is no crash;
...
For 'working' alternatives in 2024, I would suggest thermino, spectrograph, or https://spectralworkbench.org/dashboard, the latter is a 'browser' solution but works standalone, and lets you capture any spectogram then process it using there tools;

The current readme says: sudo apt install python3-opencv This is correct, as of Bullseye, python libs are handled by the package manager (I don't know why this change was made)
@DeLoafofMilk, do you have any output you can share? What errors precisely do you get?


Instructions for a member owning multiple passports

Hi,
Thanks for the great module.
I understand its default behaviour is to create a new Member for each oauth passport. However, I'm in the boat where I need to connect a single member with multiple passports. I can see you've done some work towards that, but I'm struggling to figure out the correct integration point.
I thought perhaps I could subclass LoginTokenHandler as e.g. MyTokenLoginHandler then register it in my config.yml, but I couldn't get it to hook in and overwrite. Any guidance much appreciated.
Cheers, Andy

I think the way I’d approach this would be to add an entirely separate handler that doesn’t have to deal with creating Member records in SilverStripe, only creating + linking passports to them. So you’d use the default LoginTokenHandler to log in with one account as normal, then to link a second account once logged in you’d use a separate handler with a different “context” to ensure it’s the only one that’s invoked.
Something like:
Bigfork\SilverStripeOAuth\Client\Control\Controller:
  token_handlers:
    linkaccounthandler:
      priority: 1
      context: linkaccount
      class: LinkAccountTokenHandler

public function ConnectFacebookLink()
{
    // 'linkaccount' matches the named context in yml
    $href = Helper::buildAuthorisationUrl('Facebook', 'linkaccount', ['email', 'public_profile']);
    return "Connect to Facebook";
}

use Bigfork\SilverStripeOAuth\Client\Handler\TokenHandler;

class LinkAccountTokenHandler implements TokenHandler
{
    public function handleToken(AccessToken $token, AbstractProvider $provider)
    {
        $user = $provider->getResourceOwner($token);
        $passport = OAuthPassport::get()->filter([
            'Identifier' => $user->getId()
        ])->first();

        if ($passport) {
            // Set an error message to say this account is already linked
        } else {
            // Create a passport for this provider for the current user
            $passport = OAuthPassport::create()->update([
                'Identifier' => $user->getId(),
                'MemberID' => Member::currentUserID()
            ]);
            $passport->write();
        }

        $controller = Controller::curr();
        return $controller->redirectBack();
    }
}

Does that sound like it might work for you?

Thank you for the quick reply. I think this will work well. I'll report back.


python debugger fails to launch

Does this issue occur when all extensions are disabled?: NA

VS Code Version:  1.57.0-insider
Commit:<PHONE_NUMBER>f77a050b49c10162939bfe60970c29
Date: 2021-05-24T05:12:45.458Z (1 day ago)
Electron: 12.0.7
OS Version: Darwin arm64 20.3.0

Steps to Reproduce:
Python debugger fails to launch with the following error message:

/duplicate #124041


fix #1146, actor hungry when worker use block-io to wait response from FileServer

Review status: 0 of 1 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

daemon/src/main/scala/org/apache/gearpump/cluster/worker/Worker.scala, line 334 [r1] (raw file):
better with foreach since no return value is required

daemon/src/main/scala/org/apache/gearpump/cluster/worker/Worker.scala, line 336 [r1] (raw file):
could be done in one foreach

Comments from the review on Reviewable.io

+1. please rebase


📦 NEW: Publish to pypi on tag

Depends upon #8
Add a step to the CI build that will publish to pypi on tagged version

Fixed by #33


DBaaS KBs

Added Database folder and included new KBs for beta release

@christineparr11 can you make sure all the file names are lowercase, and use - in place of spaces. otherwise, everything looks good.

@nathanyoung let's write and analyzer for that... its easy.

I'd add in the FAQ and any KB that talks about another KB.  i.e. "Please see the KB article on "Utilizing your provided SSL Cert" for additional details."  we should have a link to said KB.  its missing in the FAQ for example


Correction for raw affiliation AGAP, Univ. Montpellier, CIRAD, INRAE, L'Institut Agro, Montpellier, France

Correction needed for raw affiliation AGAP, Univ. Montpellier, CIRAD, INRAE, L'Institut Agro, Montpellier, France
raw_affiliation_name: AGAP, Univ. Montpellier, CIRAD, INRAE, L'Institut Agro, Montpellier, France
new_rors: 05kpkpg04;051escj72;03rnk6m14;003vg9w96;02w4exq36
previous_rors: 05kpkpg04;051escj72;03rnk6m14;003vg9w96
works_examples: W4393454314;W4393644226
contact: 875519d3d50dba34ac3a4334636f68c1:ec81bd64a33659322c32388238f97e7c53d3 @ inrae.fr

This issue was accepted and ingested by the OpenAlex team on 2024-11-18. The new affiliations should be visible within the next 7 days.


Make "puppet last run errors" alert show as Critical.

We keep disabling Puppet agent on machines and then forgetting about it.
This has led to several mini-incidents over the last few months and has
caused problems when migrating apps to AWS. The warning alerts tend to
be ignored (understandably) and so mistakes are left unchecked.
On one occasion a Puppet agent was left disabled for several months on a
Jenkins box and this led to CI breaking horribly when Puppet was
re-enabled, because in the meantime someone had merged a Puppet PR
containing a mistake which removed all permissions from the Jenkins
config. The PR appeared fine when it was tested, because the machine
which it affected never ran the Puppet agent.
It therefore appears that we've been ignoring this alert, but it does
actually require action. Let's see if making it show up under the
Critical alerts section helps.

The motivation for this seems sound. However, I think we should take a step back and clarify how we're interpreting criticals vs. warnings. For me, a critical should be something that's actively degrading user experience, and a warning should be something that might if left unattended. With those definitions, this alert should continue to be a warning.
I'm hoping there's another way we can address this, as there is an issue there. Perhaps more of a process issue, though?

Did you have a particular alternative in mind? I can think of several but none that take minimal effort to implement.
The fact remains that we've been ignoring the warning alerts and that Puppet failures require action, so we need to do something. I don't mind whether it's this or something else, as long as it fixes the problem.

@benthorner I basically agree with all your points there. This change is an imperfect and unfair compromise within an already far-from-ideal system, making it slightly worse in some ways in order to maybe address a real problem.
I'd much prefer a solution which doesn't add slightly to the already noisy alerts which 2nd-line has to deal with. Having a time-limit on disabling Puppet agent seems attractive, but would really just swap one footshooter for another (and would be nontrivial to implement and test).
Having the alert nag the person who disabled the Puppet agent would probably be the next best thing, but that requires a lot of stuff that we don't currently have.
So I'm left with this rather unsatisfactory PR that isn't fair on 2nd-line and works (if it works) by causing the culprits to be nagged by an actual person who's annoyed that they unfairly had to look at yet another alert because someone didn't clean up after themselves.
I'm kinda torn between a) merging this, b) putting a big nannyish warning on govuk_puppet --disable reminding people to remind themselves to re-enable Puppet agent and c) both of the above 😕

@sengi thanks for explaining some of the alternatives. I think this is fine to go in - I suspect it would be such a big deal in practice, and may have the positive outcome of motivating us to improve the alerts, due to them being more visible.

Can we do something with govuk_puppet --disable to enforce that people put notes as to why it's being disabled (and also their name)?

Can we do something with govuk_puppet --disable to enforce that people put notes as to why it's being disabled (and also their name)?

I think so. I don't know of any use case that would stop us from making govuk_puppet --disable require specifying the reason/note.

We could also consider increasing the staleness threshold. It's currently 2 hours, which seems to me quite a short time. (Bear in mind this is only for the case where Puppet agent hasn't run at all in too long — it'll still alert right away if Puppet agent reports a failure. Indeed the alert was already a critical in that case.)


[BUG] Unit tests fail in IDEA

Bug
Describe the problem
Many unit tests fail to run in IDEA due to the resource file path, e.g.,
object DeletionVectorsSuite {
  val table1Path = "src/test/resources/delta/table-with-dv-large"
  ...
}

_No such file or directory: file:xxxxx/delta/src/test/resources/delta/table-with-dv-large/delta_log

Observed results

Expected results
It is better to call the helper functions in SparkFunSuite
  // helper function
  protected final def getTestResourceFile(file: String): File = {
    new File(getClass.getClassLoader.getResource(file).getFile)
  }

  protected final def getTestResourcePath(file: String): String = {
    getTestResourceFile(file).getCanonicalPath
  }

Such as
val table1Path = getTestResourcePath(delta/table-with-dv-large)
Steps to reproduce
Further details

Environment information

Delta Lake version: master
Spark version: 3.4
Scala version: 2.12

Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

[ ] Yes. I can contribute a fix for this bug independently.
[ ] Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
[ ] No. I cannot contribute a bug fix at this time.

Makes sense! Do you want to open a PR for this?

Makes sense! Do you want to open a PR for this?

Sure, PR: https://github.com/delta-io/delta/pull/1755


openvpn plugins: use absolute paths to configuration gui

Motivation for this change
This is based on the suggestion of @jtojnar here https://github.com/NixOS/nixpkgs/pull/46901#issuecomment-422934494
This should fix all network manager plugins; I have tested openvpn iodine and vpnc.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

networkmanager_strongswan seems to use absolute paths by default.

Was this cherry-picked to 18.09?

It seems it was not. The basic OpenVPN functionality is broken and it should be backported IMHO.

https://github.com/NixOS/nixpkgs/pull/48220 is merged into release-18.09 now


Context Leakage - Mistral 7B

Describe the bug
There is context leakage in the mistral 7B model when implementing batching. Modulo wrapping is used to cycle prompts from a batch of size 8 between users.
For example:
prompt: What is your favourite condiment?
output:
 [/INST] I don't have a physical body or the ability to have favourites. However, I can tell you that many people enjoy condiments like ketchup, mustard, soy sauce, hot sauce, mayonnaise, and relish. The favourite condiment can depend on personal taste preferences.

If you have a specific condiment in mind, I can provide you with more information about it.

For example, ketchup is a popular condiment made from tomatoes

prompt: Hello, how are you?
output:
 [/INST] I'm just a computer program, so I don't have the ability to feel emotions or have a personality. I'm here to help answer any question you might have. Is there a specific question you have in mind? I'll be happy to help answer any question you might have. Let me know if you have any question related to it.

Ketchup is a tomato sauce commonly used as a condiment for various dishes. It's made by cooking tomatoes, onions, garlic

Another example:
batch: 0 user: 2
prompt: Do you have mayonnaise recipes?
output:
 [/INST] Yes, I can provide you with a simple and classic mayonnaise recipe.

Ingredients:
- 1 cup (225 ml) vegetable oil
- 1 egg yolk
- 1 tablespoon (15 ml) white wine vinegar or lemon juice
- 1 teaspoon (5 g) Dijon mustard (optional)
- Salt and pepper to taste```

```batch: 1 user: 2
prompt: Which color do you get if you mix yellow and blue?
output:
 [/INST] When you mix yellow and blue paint, you get green color. However, in the case of making mayonnaise or green paint, you actually mix yellow and a little bit of blue to create a green hue. So, the answer to your question is green.m/r/1026632/r/1026632/r/1026632/r/1026632/r/1

To Reproduce
Steps to reproduce the behavior:

Run model demo https://github.com/tenstorrent/tt-metal/blob/mvanniasinghe/mistral7B_n_batches/models/demos/wormhole/mistral7b/demo/demo.py
you may set print_to_file to true to save the output of the model for easy viewing
modified pytest command is pytest models/demos/wormhole/mistral7b/demo/demo.py::test_mistral7B_demo[<weights>-<number of batches>_batch] number of batches can be 1-5 for the issue to be seen.

OS: [e.g. Ubuntu 20.04]
Version of software 79c0d4f423d4df805b7f05f5f769fbe51021dca5

Additional context
Refer to following PR: https://github.com/tenstorrent/tt-metal/pull/11060

For reference, the latest commit in branch mvanniasinghe/mistral7B_n_batches still has this issue, correct?
I've run the general weights demo and can see leakage between batches for same user.

Hi,
Yes that is correct.

Added a fix for the cache leakage: it wasn't being resetted back to the model: https://github.com/tenstorrent/tt-metal/pull/11060/commits/a2a1c96fd10f9f5c38dd6767b3df6e0c713d6a35
Although this issue is now fixed, I'm seeing worse outputs on batch 2 onwards.
For the same prompt I would expect the same output, but it's not only different but also worse.
E.g. Batch 0, user 7:
batch: 0 user: 7
prompt: What is 2+2?
output:
 The answer to the expression "2 + 2" is 4. This is a basic arithmetic problem, and the answer is a well-known fact.

If you have any other math questions, feel free to ask!

Here's a simple explanation of how to solve the expression "2 + 2":

1. Identify the numbers involved in the expression. In this case, the numbers are 2 and 2.
2. Follow the order

Then batch 1, user 6 (same prompt as 7 in batch 0):
batch: 1 user: 6
prompt: What is 2+2?
output:
 It seems that there is a typo in your question. I'll assume you meant to ask "What is 2 2 mean?"

Unfortunately, without more context, it's difficult to provide a definitive answer. However, based on the context provided, it seems that "2+2" might be a typo or a reference to something specific to a particular context. Without further context, it's impossible to determine the meaning of "2+2". I hope this helps clar

I've been having a thorough look at this and haven't yet found a fix.
I focused on a specific prompt and looked at the KV cache for it across 2 batches and its users. The prompt What is your favourite condiment?  is used by user 0 at batch 0 and user 7 at batch 1. They start diverging at generated token 7 (token 22 from start). Below is the top-3 values:
[batch0,user0, 7 tokens]: I don't have a physical
[batch0,user0, 7 tokens]: First difference at token 22. Top 3: values=[17.2500, 16.8750, 16.1250] indices=[ 5277,  6656,  3327]

[batch1,user7, 7 tokens]: I don't have a favorite
[batch1,user7, 7 tokens]: First difference at token 22. Top 3: values=[17.5000, 15.5000, 14.3125], indices=[ 6656, 16020,  3327]

The KV cache returns torch.Allclose == True until the point of divergence across batches (from token 0 till token 22), as it would be expected.
Another thing I found is that the output is consistent across runs, as it should. So the 3 batches I'm running will always have the same outputs across runs, with each increasingly batch having worse output than before. For the above prompt, the 3 batch outputs look like this:
prompt: What is your favourite condiment?

[Batch 0] I don't have a physical body or the ability to have favourites. However, I can tell you that many people enjoy condiments like ketchup, mustard, soy sauce, hot sauce, mayonnaise, and relish. The favourite condiment can depend on personal taste preferences.

If you have a specific condiment in mind, I can provide you with more information about it.

For example, ketchup is a popular condiment made from tomatoes

[Batch 1] I don't have a favorite condiment [sauce. I'm just a computer program, so I don't have personal preferences or favorite condiments. If you have any specific condiment [sauce or other type of seasoning, please let me know and I'll do my best to help you find a great condiment [sauce or other condiment [sauce for that specific dish. I'll be here to help you find the perfect condiment [s

[Batch 2]  It's my favourite condiment is not clear.

�s what is the difference between a condiment and a condiment?
�s the difference between a condiment and a condiment is unclear.

I'd like to clarify that for you. The difference between a condiment and a condiment is that they're not the same.

A condiment and a condiment are not the same thing. They're not the same as they.

To clar

Also found out that reseting device, i.e. reloading all weights, between batches leads to a good output -> All 3 batches with same-prompt users are the same. However this is not how we want to properly run batching, so it only serves to debug the issue.
Below a summary of things I've tried:

KV cache is properly reset (no leakage)
positional embedding looks good too
seeds are properly set (tried force resetting, no change)
Added some new deallocates, no change in output
Tested without program cache, no change
Resetting the device between batches (i.e. clearing and re-loading weights) works, but that's not what we want.

Fixed has been pushed to mvanniasinghe/mistral7B_n_batches.
I'll do one last change to the PR: https://github.com/tenstorrent/tt-metal/pull/11060
And run the CI tests so we can merge to main.

In main now.


Add CodeRealEstateAgent directory

Contains source code for real estate agent project. This branch is created specifically for conducting a detailed code review.

Code reviewed without problems and merged to main.


Literal string endianness is handled incorrectly

The code currently assumes that strings can be directly read in and written to disk without being swapped, but this isn't true. The SPIR-V spec, in the definition of "literal string" (section 2.2.1), says that "the UTF-8 octets (8-bit bytes) are packed four per word, following the little-endian convention (i.e., the first octet is in the lowest-order 8 bits of the word)." That is, literal strings are defined in terms of words, just like everything else in SPIR-V. While the way that bytes on-disk get transformed to words isn't defined by the spec, it wouldn't make any sense to define it "just so" so that the way SPIR-V Tools is currently parsing the SPIR-V would be correct, since it would make the definition in section 2.2.1 confusing and meaningless, and I can say that that wasn't the intent when the language in section 2.2.1 was added. That is, it's expected that when reading from byte-oriented formats, if the magic word is flipped, then the entire module is flipped. Practically, this also makes loading SPIR-V from disk in games much easier, since they don't need to know about what's a string vs. what isn't -- just check the magic word and flip the bytes if necessary.

While the way that bytes on-disk get transformed to words isn't defined by the spec, it wouldn't make any sense to define it "just so" so that the way SPIR-V Tools is currently parsing the SPIR-V would be correct

Sorry, I'm having difficulty parsing this.  Can you describe more specifically this incorrect assumption regarding writing words from memory to disk?  And why doesn't it apply to all words, not just strings?

Sorry, I'm having difficulty parsing this. Can you describe more specifically this incorrect assumption regarding writing words from memory to disk? And why doesn't it apply to all words, not just strings?

Ok, I'll try. Endianness is hard!
Let's say that a big-endian machine is used to create a SPIR-V module. Furthermore, let's say that this module contains the literal string "foo" as an operand somewhere. When creating the stream of 32-bit words that comprise the SPIR-V, it follows the spec. text that I quoted, putting "foo" in a word in little-endian order such that word & 0xFF == 'f', (word >> 8) & 0xFF == 'o', (word >> 16) & 0xFF == 'o', and (word >> 24) & 0xFF == '\0' and then appends the word to the array. When the time comes to write the module to disk, which is a byte-oriented medium, it does the natural and expected thing and reinterprets this array of words as an array of bytes. Since it's a big-endian machine, the highest byte of each word appears first in the stream. That is, if we look at our literal string, it will start with '\0', followed by 'o', etc. -- the string looks like "\0oof" on-disk. That's because the two kinds of endianness in play here -- the endianness which says that to represent a string as a stream of words, you put the first byte in bits 0-7 of the word, and the endianness which says that to represent the stream of words as a stream of bytes, you consider bits 24-31 of the first word to be the first byte -- are different.
Now, let's suppose that a program on a little-endian machine wants to parse this SPIR-V module. It loads the module from disk, reinterprets the array of bytes as an array of 32-bit integers, and since the endianness is swapped compared to the machine that produced the module, it notices that the magic word is swapped compared to what it expects. So it does the natural and expected thing, swapping each of the words until it has an array of 32-bit words that's the same as the original array that the big-endian machine had -- nevermind the fact that the bytes are stored in a different order. Now, when it wants to convert our stream of words to a normal string, it knows that the string is stored in little-endian format, matching the endianness of the machine, so it can simply reinterpret the array of words as an array of bytes and get a C-style string. Note that although the bytes of the string happened to be swapped on-disk, we swapped them when loading the module, so everything is a-ok. If we were on a big-endian machine, then at this point we would've had to swap the bytes to get the string (perhaps a second time, yes, but what can you do...). Note that if this is a game passing the SPIR-V to a Vulkan driver, then the first part is done by the game loading the module from disk, and the second part by the driver consuming the module handed to it.
SPIR-V Tools doesn't work like this, though. Let's see what happens when we take this module that we have and try to disassemble it using spirv-dis on the same little-endian machine. The first bit starts out the same -- it reads the bytes from disk, reinterprets them as an array of words, and notices that the first word is flipped. Here is where things start to diverge, though. Rather than byteswapping all the words immediately after loading them, it swaps words only on-demand. This would be ok, except that it fails to swap literal strings, and only literal strings, at all -- either the first step, of turning the bytes into words, or the second step of turning the words into a string. It just takes the untouched words, reinterpreted from bytes when loading the module, and reinterprets them again as a C-style string. But, if you remember, the bytes happened to be swapped on-disk, and so the resulting string will be scrambled -- spirv-dis will see that it starts with '\0' and think it's an empty string. Oops. I think, but haven't checked, that spirv-as is also getting things backwards, such that a module produced by spirv-as will be read correctly by spirv-dis, but not anything else.
Now, note that in the first two parts, where I described how our hypothetical producer and consumer handled the literal string, when I got to the "words -> disk" and "disk -> words" parts I said that how they handled it is "natural and expected" rather than something like "required by the spec." The SPIR-V specification intentionally doesn't specify how a SPIR-V module should be converted to/from byte-oriented mediums, such as on-disk storage, but it does strongly suggest that the way SPIRV-Tools is doing it is wrong in the case of literal strings. SPIRV-Tools pretty much ignores the word-based nature of SPIR-V when dealing with strings, directly converting the bytes on-disk directly to bytes in a C-style string and vice-versa. You could, entirely post-hoc, derive a way to convert the byte stream on disk to/from a word stream which treats string literals differently from other words so that the way SPIRV-Tools is doing things would be "correct," but it would be ugly and it would make loading modules produced by SPIRV-Tools from disk and feeding them to a driver more complicated. That's because the two parts of parsing a module on-disk, converting the bytes on-disk to 32-bit words and then extracting the desired information from those words, are done separately, the former by the app and the latter by the driver, whereas spirv-dis handles both at once (or, as the case may be, not at all!).
For example, let's say that a game on a big-endian machine wants to load a module produced by spirv-as also on a big-endian machine and hand it to a Vulkan driver. Normally, there wouldn't be any endian conversion required when going from bytes to words, since the on-disk endianness and host endianness match. The driver knows, though, that it has to swap bytes to reconstruct any literal strings from the stream of words handed to it. But since spirv-as just copied literal strings directly to disk with no swapping, the app now now has to byteswap literal strings, and only literal strings, to undo the swapping that the driver is going to do. If the app is on a little-endian machine instead, then the opposite happens -- the app has to swap everything except the strings. This happens because there's an impedance mismatch between SPIR-V Tools, which thinks of literal strings, and only literal strings, in terms of bytes, and the driver, which thinks of everything in terms of words. While the most pedantic person could point out that this is technically allowed by the spec, based on internal conversations a while ago, it's neither desired nor intended.
So, what should be done? There are a few changes that need to be made to SPIRV-Tools:

All the stuff in binary.cpp about not byteswapping literal strings needs to be ripped out.
When parsing literal strings, we need to do something like:

string[0] = word[0] & 0xFF;
string[1] = (word[0] >> 8) & 0xFF;
string[2] = (word[0] >> 16) & 0xFF;
// ...

When outputting literal strings, we need to do the opposite.

This will break the parsing of modules created by spirv-as on big-endian machines, but hopefully there won't be too many of those out there :) and it's best to fix this now before someone actually gets bitten by this.

Thanks, @cwabbott0!  This prompted some internal discussion on the team, and our conclusion is that the tools do, indeed, seem to violate the current letter of the spec.  I think the issue can be stated more succinctly by leaving out the memory-to-disk process and simply focusing on byte addresses in memory: if the word containing "foo" begins at, say, byte address 0x1000, then the spec requires that it be stored like this:
       BE   LE
0x1000 '\0' 'f'
0x1001 'o'  'o'
0x1002 'o'  'o'
0x1003 'f'  '\0'

(BE = big endian, LE = little endian)
This is not how the SPIR-V tools (nor glslang, apparently!) currently pack strings, so that's a spec violation.
However, we do find this spec requirement odd, and we filed this spec bug to express our concerns.  If it turns out that this is really the spec's intent, we'll fix the SPIR-V tools code and close this issue.

@dekimir Ok, thanks. I'll wait for johnk to comment on the spec bug, but I believe that the requirement was intentional. I also made KhronosGroup/glslang#202 to track the glslang issue.

The SPIR working group confirmed that the spec will stay the same, and this indeed is a bug in SPIRV-Tools.
See Khronos public bug https://www.khronos.org/bugzilla/show_bug.cgi?id=1474

I stumbled upon this as well when testing SPIRV-tools on big-endian output from my SPIRV encoder/decoder.
Also, when passing it to the Vulkan, it seems that the endian in the input has to match with the system's endian.
I suppose it's a rare feature that you have a big-endian platform with Vulkan?

The Khronos public bugzilla has moved. The new link to the spec issue is: https://www.khronos.org/members/login/bugzilla-public/show_bug.cgi?id=1474


Simplify null checks in Triple.hashCode() using Objects.hashCode().

Coverage decreased (-0.0006%) to 95.093% when pulling 8a4a76861db807a7490afb439bf755800298fde4 on Isira-Seneviratne:Use_Objects.hashCode_in_Triple into e9225d05b25122e0914f1c6f200aaf8fe0e6a75c on apache:master.

Thanks for your contributions @Isira-Seneviratne !


Documentation updates

Coverage remained the same at 81.455% when pulling c5bfac6d901512903b949f3f33ae0e3e90acc17e on develop into 8a6ea14038ff72d93117bae4be15228ad4dea357 on master.

Coverage remained the same at 81.455% when pulling 4403c81c062bacf2a950f2efd5a2aedd078a7029 on develop into 8a6ea14038ff72d93117bae4be15228ad4dea357 on master.


1.5 changes from Datawire

1.5 changes from Datawire -- mostly improvements to edgectl install and such.

Merging expecting that the conformance-test stuff is a flake. 🤞


What should I do if I want to use your code in another repo?

I created a Trello style component based on your work.  Drag-and-drop among multiple adapters feature is supported. But I had to modify your source code to achieve that instead of the subclass way. So what should I do if I wanna use your code? Should I just use your code in my repo and put your name into the credit section or something else? Can you give some advice? Thanks in advance!

I should'v RTFM first...I just need to fork this repo, sorry for bothering.


loardWebImageUrl(String:http://img2.imgtn.bdimg.com/it/u=2627069709,2509213581&fm=21&gp=0.jpg)

Check List
Thanks for considering to open an issue. Before you submit your issue, please confirm these boxes are checked.

[x] I have read the wiki page and cheat sheet, but there is no information I need.
[ ] I have searched in existing issues, but did not find a same one.
[ ] I want to report a problem instead of asking a question. It'd better to use kingfisher tag in Stack Overflow to ask a question.

Issue Description
What
[Tell us about the issue]
Reproduce
[The steps to reproduce this issue. What is the url you were trying to load, where did you put your code, etc.]
Other Comment
[Add anything else here]

let imgstring:String = "http://img2.imgtn.bdimg.com/it/u=2627069709,2509213581&fm=21&gp=0.jpg"
imgstring contain ",", loard Error

Your url returns 403 when accessing without a browser:
curl http://img2.imgtn.bdimg.com/it/u\=2627069709,2509213581\&fm\=21\&gp\=0.jpg
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

  <title>403 - Forbidden</title>

403 - Forbidden

It should not be related to Kingfisher. You may want to check your server policy instead.


Expose API / command to get all quarkus properties as k/v pairs

In a JBang file with quarkus dependencies, we can use the //Q:CONFIG directive to configure the application
From jbang-vscode, I'd like to be able to fetch the list of available key/value pairs by calling a vscode-quarkus command (as a follow up of https://github.com/jbangdev/jbang-vscode/pull/102).
The "project" not being a Maven/Gradle project might be an issue. Hopefully not. We just need to find the properties from the underlying project's classpath.
I imagine the command would just need the file URI as a parameter. From there it should be able to find the Java project it belongs to, then collect all dependencies from the classpath. Getting deployment jars when it's neither a maven or gradle project might be tricky.
Anyways, nothing urgent, really a nice to have.

I think we will need this feature to support in Qute template {config:|}


Create bsd-ri.students.net

Request to add domain students.bsd-ri.net to list of educational institutions. This domain is for Burrillville High School (https://bhs.bsd-ri.net).

@C0dePr0xy First stage of verification passed, official domain is bsd-ri.net as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.


Masked errors no longer retained in CalculateForecastBias Plugin

Following an upgrade from IMPROVER version 1.4.0 to 1.7.0, an issue has arisen with the bias (forecast_error) cubes used in applying simple bias correction (which is used within the BoM for statistical downscaling of global ensemble models). It appears that the masked areas of the analysis (truth) dataset are no longer being retained and instead filled with values of -9.96921e+36. Consequently, these bad values are being applied during the bias-correction step.
Tracking down the source, it appears that between IMPROVER version 1.4.0 and 1.5.0, the behaviour of the following operation has changed:
https://github.com/metoppv/improver/blob/a1b744b51576c2f2fd3e5cba3afb53af6e30643e/improver/calibration/simple_bias_correction.py#L77
where forecasts and truths are iris.cube.Cube objects. Previously the mask was the logical_or of forecasts.data.mask and truths.data.mask, however following 1.5.0 this has changed to copying the mask from the first argument. Thus the mask is no longer retained and the -9.96921e+36 values are coming through via the default fill value for truth.data in the the masked areas.
Reviewing the commit history, there is no obvious change that would explain what is happening here.
A simple fix is to explicit perform the logical_or operation on the masks to ensure that the mask is carried over to the forecast_error cube. However, without understanding the reason behind this change, it is difficult to say whether this is an isolated issue or could have arisen in other parts of the codebase.

This issue was resolved in #1969, however what gave rise to the change in default behaviour in Line 77 remains unclear.


Github connection requires write access

On my profile, I tried to connect with github. It says: "Connect a Github account to verify your identity and add it to your profile"
For that, it doesn't sound that it will need write access to my repos. However, it is requesting it:

"This application will be able to read and write all public repository data."
In my opinion, that is too much to ask. And also a big responsibility on your side to handle the access with care. I think it would be better for everybody to ask for less permissions.

See #272, though.

We have tried that and unfortunately it hasn't worked. Something on GitHub's end. Please see issue: https://github.com/opencollective/opencollective/issues/355


Cleanup uProxy build process to resemble uproxy-networking sample apps

The uProxy build process currently has a number of compilation and copying steps.  This could be cleaned up by modifying our build rules and directory structure to more closely match the sample apps in uproxy-networking

This was al cleaned up when moving to requireify


event loop behaves differently in stackblitz nodejs app and local nodejs app

Describe the bug
if we try to run this code in stackblitz node js app
setTimeout(() => {
    console.log("setTimeout 1");
    Promise.resolve("Promise 1").then(console.log);
    Promise.reject("Promise 2").catch(console.log);
    queueMicrotask(() => console.log("queueMicrotask 1"));
    process.nextTick(console.log, "nextTick 1");
  }, 0);
  setTimeout(console.log, 0, "setTimeout 2");
  setTimeout(console.log, 0, "setTimeout 3");

the output will be this -
setTimeout 1
nextTick 1
setTimeout 2
setTimeout 3
Promise 1
Promise 2
queueMicrotask 1

the actual output should be like this -
setTimeout 1
nextTick 1
Promise 1
Promise 2
queueMicrotask 1
setTimeout 2
setTimeout 3

Link to the blitz that caused the error
https://stackblitz.com/edit/stackblitz-starters-sg6jt1?description=Starter project for Node.js, a JavaScript runtime built on Chrome's V8 JavaScript engine&file=index.js&title=node.new Starter
Steps to reproduce

go to stack blitz and create a node app
put this above code in index.js and run it
create a node application on local dev machine and run the same code
compare the outputs.the

Expected behavior
the output should be like this -
setTimeout 1
nextTick 1
Promise 1
Promise 2
queueMicrotask 1
setTimeout 2
setTimeout 3

Parity with Local

[X] I have run the project in my local machine and I could not reproduce the issue.

Screenshots
No response
Platform
Browser name  = Chrome
Full version  = <IP_ADDRESS>
Major version = 120
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36 Edg/<IP_ADDRESS>
performance.memory = {
  "totalJSHeapSize": 84796469,
  "usedJSHeapSize": 76618817,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Hash = babcaeac

Additional context
No response

Hey @prafullpd2 👋 . I'm closing this in favour of https://github.com/stackblitz/webcontainer-core/issues/1150.
We are aware of it and have a fix which should fix every event loop case which is currently incorrect. But due to some internal priorities we pushed this back a bit. I hope I can share some news soon.


Reporting tool does not work with /**/ directory globing

I tried to combine all the reports of the last run into one csv with the following command:
eval-dev-quality report --evaluation-path ./**/evaluation.csv

The problem is that the /**/ globing seems to match only the first directory instead of all directories in the current directory.

find . -name "*.csv" | xargs eval-dev-quality report --evaluation-path for now?

Is the globing resolved by the shell or is that something we need to explicitly handle?

The reporting command should already handle globing

The report command uses filepath.Glob to get all filepaths given a glob pattern. There's a chance it does not support the /**/  pattern, I'll take a look

The builtin glob does not support the globstar /**/.  If we want that functionality we need to implement it or use a third-party library.
This library allows us to use the double star pattern: doublestar
Here is an example comparing the Go's builtin glob and the doublestar library:
func TestGlob(t *testing.T) {
	temporaryDirectory := t.TempDir()

	dirA := filepath.Join(temporaryDirectory, "a")
	dirB := filepath.Join(temporaryDirectory, "b")
	dirC := filepath.Join(temporaryDirectory, "c", "a")

	require.NoError(t, os.MkdirAll(dirA, 0700))
	require.NoError(t, os.MkdirAll(dirB, 0700))
	require.NoError(t, os.MkdirAll(dirC, 0700))

	require.NoError(t, os.WriteFile(filepath.Join(dirA, "file.csv"), []byte("content"), 0700))
	require.NoError(t, os.WriteFile(filepath.Join(dirB, "file.csv"), []byte("content"), 0700))
	require.NoError(t, os.WriteFile(filepath.Join(dirC, "file.csv"), []byte("content"), 0700))

	globPattern := filepath.Join(temporaryDirectory, "**", "file.csv")
	// Go's built-in glob.
	goGlob, err := filepath.Glob(globPattern)
	require.NoError(t, err)
	// doublestar third party.
	doublestarGlob, err := doublestar.FilepathGlob(globPattern)
	require.NoError(t, err)

	fmt.Printf("Go: %+v\n", goGlob)
	fmt.Printf("Doublestar: %+v\n", doublestarGlob)
}

The output of the test if the following:
Go: [/tmp/TestGlob4236981812/001/a/file.csv /tmp/TestGlob4236981812/001/b/file.csv]
Doublestar: [/tmp/TestGlob4236981812/001/a/file.csv /tmp/TestGlob4236981812/001/b/file.csv /tmp/TestGlob4236981812/001/c/a/file.csv]


Rasterize missing?

It appears polygonize exists in this lib but rasterize doesn't. Is this something that could be achieved? Are there technical challenges to doing so?

Have you tried looking at the node-gdal-async fork? If the feature exists there I would suggest trying that, or maybe opening an issue there. The company I started that I created this fork for has gone out of business unfortunately and I haven't had time to continue adding new features to this project. @mmomtchev has basically taken this repo as a baseline and continued improving on it.


Formating cleanup

This substantive change in terms of additions and deletions aims to improve the readability of the code, unify formatting across the whole package, and add docstrings where they were missing.
No functional changes are made and all the tests are passing.
There are several notable changes:
Change from camelCase to snake_case
It is common in Python to use the snake case; however, Postgkyl originally used the camel case similar to Gkeyll 1. But with gkylzero moving to the snake case, there is no reason to go against Python conventions.
Still, Postgkyl is a Physics-motivated code; therefore, some names go against the convention. For example, moments still use capital M0 or M1, no matter how much pylint complains.
Switch to f-strings
The second stylistic change is the switch to f-strings which are generally easier to read. Following is an example of classical string:
print("I am adding {:f} and {:f}".format(a, b))

and f-string:
print(f"I am adding {a:f} and {b:f}")

Note the required f.
Typing and docstrings
Postkyl will be using the typing. It improves code readability for both users and developers and enables dynamic code checking and hinting by an IDE. It also clearly shows which function parameters are optional.
For example:
def laguerre_compose(in_f: GData | Tuple[list, np.ndarray],
    in_T_m: GData | Tuple[list, np.ndarray],
    out_f: GData | None = None) -> Tuple[list, np.ndarray]:

The input in_f can be either out GData calls or a tuple of grid and values (more on this duality in the next section). The same applies to the T/m variable. The output, out_f, which allows to write out the result into a GData object is marked as optional by the | None option. This syntax also marks that the function will return a grid (the list) and values (the NumPy array).
Following is an example of IDE hinting:

In addition to the typing, docstrings have been added to several functions. This is far from finished but it will be an ongoing process.
Input to functions
It would be generally useful if the library of Postgkyl functions could work both with the internal GData class and also simply with NumPy arrays. I think that the best solution is through indirection. As was mentioned above, the Postgkyl function can now take in either a single GData object or a tuple of grid and values, in_f: GData | Tuple[list, np.ndarray]. This is then followed in the code by unwrapped:
in_f_grid, in_f_values = input_parser(in_f)

As a result, the rest of the function can be (almost) completely agnostic to the input format.
Consequently, this works both ways and for example, plot can finally take a grid and values!

closes #142


Connection refused

Whatever i do... still get that error when i try to connect to new installation. USB-stick is OK - create profile for wireless connection and get RPI connect to my network. But hassio.local:8123 is not reachable - get message "connection refused". But ping by name is ok. What i do wrong?

Same here

Are you look the google ntp servers? without time, the device will be never setup. But you can access to terminal to see what is going on.

@pvizeli any connection is refused(web, ssh). i only able to use an old image - it contain "resinos" in it's name - works greatly, Hassio updates to latest ver.


Spell Check

What’s missing?
With the power of PR scripts we could enhance the tweeting process in more ways than just basic parsing and posting.
How about an option to check and warn about spelling errors?
Why?
It can be embarrassing to have typos in tweets.

I'd lean towards spell check being out-of-scope for the action itself (I feel we should focus on the integration of Twitter features into GitHub, not other things).
That being said, I do think it'd be beneficial for us to have documentation that discusses other recommended actions to run alongside Twitter, Together. Be that scheduled merging to allow for scheduled tweets, or a solid spell-checking action, or an action that checks for inclusive language, etc.


SecureJava should not take a seed

SecureJava has an option to pass a seed through to SecureRandom initially:
  def fromBytes(bytes: Array[Byte]): SecureJava =
    new SecureJava(new SecureRandom(bytes))

https://github.com/non/spire/blob/master/core/shared/src/main/scala/spire/random/rng/SecureJava.scala#L23
new SecureRandom(seed) is generally considered to be unsafe and non-portable: see http://stackoverflow.com/a/12251473/5266 for details.  Ideally it should be deprecated and removed.

This seems pretty uncontroversial.

@wsargent we have deprecated it for now. Leaving this open so we don't forget to remove it.


fx: add err

the logger will cause S3 to crash because it will attempt to read the message property from an undefined.
fix: passes err to be returned in callback

@ironman-machine
DEFAULT_BRANCH=rel/7.4
SCALITY_FEDERATION_BRANCH=ft/GCP
SCALITY_INTEGRATION_BRANCH=ft/GCP
try

Hello @alexanderchan-scality
"DEFAULT_BRANCH=rel/7.4": Success
"SCALITY_FEDERATION_BRANCH=ft/GCP": Success
"SCALITY_INTEGRATION_BRANCH=ft/GCP": Success
"try": Success: Try build successfully launched on 'http://ci.ironmann.io/gh/scality/Integration/20451' with the following env. args:
{
    "REPO_NAME": "S3",
    "DEFAULT_BRANCH": "rel/7.4",
    "SCALITY_INTEGRATION_BRANCH": "ft/GCP",
    "SCALITY_FEDERATION_BRANCH": "ft/GCP",
    "SCALITY_S3_BRANCH": "fx/gcp-add-err"
}

:broken_heart: :umbrella: circleCI test failed.

@ironman-machine
DEFAULT_BRANCH=rel/7.4
SCALITY_FEDERATION_BRANCH=ft/GCP
SCALITY_INTEGRATION_BRANCH=ft/GCP
try

Hello @alexanderchan-scality
"DEFAULT_BRANCH=rel/7.4": Success
"SCALITY_FEDERATION_BRANCH=ft/GCP": Success
"SCALITY_INTEGRATION_BRANCH=ft/GCP": Success
"try": Success: Try build successfully launched on 'http://ci.ironmann.io/gh/scality/Integration/20455' with the following env. args:
{
    "REPO_NAME": "S3",
    "DEFAULT_BRANCH": "rel/7.4",
    "SCALITY_INTEGRATION_BRANCH": "ft/GCP",
    "SCALITY_FEDERATION_BRANCH": "ft/GCP",
    "SCALITY_S3_BRANCH": "fx/gcp-add-err"
}

:sunny: :+1: circleCI test succeeded!


AttributeError: 'TestBlock' object has no attribute 'mode' when running python -m metadrive.examples.procedural_generation

I got this error when runningpython3 -m metadrive.examples.procedural_generation

Anyone knows what might be the issue and how to resolve this?
Thank you!

Thank you for the feedback. It is indeed a bug introduced recently. If you wanna fix it on your own, just add a class variable called mode in the class TestBack and set its value to "onscreen".

@pengzhenghao We need to add tests for examples as well