CSV node check header properties for ' and "

and add test
to close #3919

[X] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

Proposed changes

Fixes Issue #3919 - if property names contain quotes (unusual but no impossible) then we should output then as the header line of the CSV correctly rather than breaking
Checklist

[x] I have read the contribution guidelines
[ ] For non-bugfix PRs, I have discussed this change on the forum/slack team.
[x] I have run grunt to verify the unit tests pass
[x] I have added suitable unit tests to cover the new/changed functionality

Merging is blocked until #3922 is merged and this branch rebased to pickup that fix.

@dceejay
Dave, there are more issues I'm afraid. Look at the examples below.

[{"id":"0a117283c6ec0e9a","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},{\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}]","payloadType":"json","x":1450,"y":160,"wires":[["e9b9c226cb22fc89"]]},{"id":"e9b9c226cb22fc89","type":"csv","z":"10ece3ab98e93683","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1630,"y":160,"wires":[["8a79b816b21ebf6f"]]},{"id":"8a79b816b21ebf6f","type":"debug","z":"10ece3ab98e93683","name":"debug 28","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1820,"y":160,"wires":[]},{"id":"87702dd4078dd355","type":"comment","z":"10ece3ab98e93683","name":"No col/template object data only.  Outputs invalid header   c\"c,d.d,\"b,b\",e`e,a'a   (RFC4180 2.6/2.7)","info":"#### Demo 1 - no column template\n\nInput data\n```\n[\n    {\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},\n    {\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}\n]\n```\n\nExtracts col template...\n```\n['c\"c', 'd.d', 'b,b', 'e`e', \"a'a\"]\n//c\"c,d.d,b,b,e`e,a'a               as a CSV string\n```\n\nAccording to RFC4180 2.6 & 2.7 the CSV generated for the template is invalid:\n\n> 2.6\n> Fields containing line breaks (CRLF), double quotes, and commas\n> should be enclosed in double-quotes\n\n> 2.7\n> If double-quotes are used to enclose fields, then a double-quote\n> appearing inside a field must be escaped by preceding it with\n> another double quote\n\ni.e. the column template should be...\n\n```\n\"c\"\"c\",d.d,\"b,b\",e`e,a'a\n```","x":1730,"y":120,"wires":[]},{"id":"15e327549fb3d8a8","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},{\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}]","payloadType":"json","x":1450,"y":300,"wires":[["8b44ac2b17a19652"]]},{"id":"8b44ac2b17a19652","type":"csv","z":"10ece3ab98e93683","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"a'a,\"b,b\",c\"c,d.d,e`e","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1630,"y":300,"wires":[["6af93e801ca72838"]]},{"id":"6af93e801ca72838","type":"debug","z":"10ece3ab98e93683","name":"debug 109","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1830,"y":300,"wires":[]},{"id":"4715772ba53f646c","type":"comment","z":"10ece3ab98e93683","name":"Incorrectly parses `col`  a'a,\"b,b\",c\"c,d.d,e`e  - as -  ['a'a,b', 'b,cc', 'd.d', 'e`e']      (RFC4180 2.6)","info":"#### Demo 2\n\nColumn Template\n```\na'a,\"b,b\",c\"c,d.d,e`e\n```\n\nparses as\n```\n['a'a,b', 'b,cc', 'd.d', 'e`e']\n```\n\nAccording to RFC4180 2.6 the CSV for the template is invalid:\n> Fields containing line breaks (CRLF), double quotes, and commas\n>  should be enclosed in double-quotes\n\n\nUsing https://www.npmjs.com/package/@vanillaes/csv\n\n```\ncsv.parse('a\\'a,\"b,b\",c\"c,d.d,e`e')\n//Uncaught Error: CSVError: Illegal state [row:1, col:3]\n```","x":1710,"y":260,"wires":[]},{"id":"f5feaf8eada6049b","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},{\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}]","payloadType":"json","x":1450,"y":700,"wires":[["7b8d6e39682d2307"]]},{"id":"7b8d6e39682d2307","type":"csv","z":"10ece3ab98e93683","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"a'a,\"b,b\",\"c\"\"c\",d.d,e`e","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1630,"y":700,"wires":[["09fed6ead887b782"]]},{"id":"09fed6ead887b782","type":"debug","z":"10ece3ab98e93683","name":"debug 110","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1830,"y":700,"wires":[]},{"id":"e367eaebdec89b4a","type":"comment","z":"10ece3ab98e93683","name":"Valid CSV `col` incorrectly parsed   a'a,\"b,b\",\"c\"\"c\",d.d,e`e  →  a'a,\"b,b\",cc,d.d,e`e","info":"#### Demo 5\n\nColumn Template\n```\na'a,\"b,b\",\"c\"\"c\",d.d,e`e\n```\n\nparses as\n```\n['a'a', 'b,b', 'cc', 'd.d', 'e`e']\n//a\\'a,\"b,b\",cc,d.d,e`e            as a CSV string\n```\n\nAccording to RFC4180 2.7 this should be\n```\n[ \"a'a\", 'b,b', 'c\"c', 'd.d', 'e`e' ]\n// a'a,\"b,b\",\"c\"\"c\",d.d,e`e           as a CSV string\n```\n\n\nUsing https://www.npmjs.com/package/@vanillaes/csv\n\n```\ncsv.parse('a\\'a,\"b,b\",\"c\"\"c\",d.d,e`e')\n// [ \"a'a\", 'b,b', 'c\"c', 'd.d', 'e`e' ]\n```\n\n","x":1670,"y":660,"wires":[]},{"id":"4fe89a759501f96f","type":"csv","z":"10ece3ab98e93683","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"a'a,\"b,b\",c\"\"c,d.d,e`e","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1630,"y":440,"wires":[["01560e6b446b8b27"]]},{"id":"286f0e9e06ada8b0","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},{\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}]","payloadType":"json","x":1450,"y":440,"wires":[["4fe89a759501f96f"]]},{"id":"01560e6b446b8b27","type":"debug","z":"10ece3ab98e93683","name":"debug 108","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1830,"y":440,"wires":[]},{"id":"b971c7dde1d9d5b6","type":"comment","z":"10ece3ab98e93683","name":"Incorrectly parses `col`  a'a,\"b,b\",c\"\"c,d.d,e`e  - as -  ['a'a,b', 'b,cc', 'd.d', 'e`e']      (RFC4180 2.6)","info":"#### Demo 3\n\nColumn Template\n```\na'a,\"b,b\",c\"\"c,d.d,e`e\n```\n\nparses as\n```\n['a'a,b', 'b,cc', 'd.d', 'e`e']\n```\n\nAccording to RFC4180 2.6 the CSV for the template is invalid:\n> Fields containing line breaks (CRLF), double quotes, and commas\n>  should be enclosed in double-quotes\n\n\nUsing https://www.npmjs.com/package/@vanillaes/csv\n\n```\ncsv.parse('a\\'a,\"b,b\",c\"\"c,d.d,e`e')\n// Uncaught Error: CSVError: Illegal state [row:1, col:3]\n```","x":1710,"y":400,"wires":[]},{"id":"7f8ae7950520433e","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},{\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}]","payloadType":"json","x":1450,"y":840,"wires":[["96ca0b519e4fb5cf"]]},{"id":"96ca0b519e4fb5cf","type":"csv","z":"10ece3ab98e93683","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"a'a ,\" b,b\",\" c\"\"c\",d.d, e`e","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1630,"y":840,"wires":[["0432d128217f3f08"]]},{"id":"0432d128217f3f08","type":"debug","z":"10ece3ab98e93683","name":"debug 111","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1830,"y":840,"wires":[]},{"id":"2bf17037679c85db","type":"comment","z":"10ece3ab98e93683","name":"Valid CSV `col` incorrectly parsed     a'a ,\" b,b\",\" c\"\"c\",d.d, e`e  →   ['a'a', 'b,b', 'cc', 'd.d', 'e`e']  (RFC4180 2.4)","info":"#### Demo 6\n\nColumn Template\n```\na'a ,\" b,b\",\" c\"\"c\",d.d, e`e\n```\n\nparses as\n```\n['a'a', 'b,b', 'cc', 'd.d', 'e`e']\n//a'a,\"b,b\",cc,d.d,e`e             as a CSV string\n```\n\nAccording to RFC4180 2.4\n> Spaces are considered part of a field and should not be ignored\n\nSo this template should parse as\n```\n[ \"a'a \", ' b,b', ' c\"c', 'd.d', ' e`e' ]\n// a'a ,\" b,b\",\" c\"\"c\",d.d, e`e          as a CSV string\n```\n\n\nUsing https://www.npmjs.com/package/@vanillaes/csv\n\n```\ncsv.parse('a\\'a ,\" b,b\",\" c\"\"c\",d.d, e`e')\n[ \"a'a \", ' b,b', ' c\"c', 'd.d', ' e`e' ]\n\ncsv.stringify([[ \"a'a \", ' b,b', ' c\"c', 'd.d', ' e`e' ]])\n// a'a ,\" b,b\",\" c\"\"c\",d.d, e`e\n```","x":1750,"y":800,"wires":[]},{"id":"6906c507f6037bf7","type":"csv","z":"10ece3ab98e93683","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"a'a,\"b,b\",c\"\"c,d.d,e`e","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1630,"y":560,"wires":[["e1cc491e0758212f"]]},{"id":"997198c671e92660","type":"inject","z":"10ece3ab98e93683","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"c\\\"c\":3,\"d.d\":4,\"b,b\":2,\"e`e\":5,\"a'a\":1},{\"d.d\":2,\"e`e\":1,\"b,b\":4,\"c\\\"c\":3,\"a'a\":5}]","payloadType":"json","x":1450,"y":560,"wires":[["6906c507f6037bf7"]]},{"id":"e1cc491e0758212f","type":"debug","z":"10ece3ab98e93683","name":"debug 112","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1830,"y":560,"wires":[]},{"id":"721d9bf721697398","type":"comment","z":"10ece3ab98e93683","name":"Incorrectly parses `col`  a'a,\"b,b\",\"c\"c\",d.d,e`e  - as -  a'a,\"b,b\",cc,d.d,e`e      (RFC4180 2.7)","info":"#### Demo 4\n\nColumn Template\n```\na'a,\"b,b\",\"c\"c\",d.d,e`e\n```\n\nincorrectly parses as\n```\na'a,\"b,b\",cc,d.d,e`e\n```\n\nAccording to RFC4180 2.6 the CSV for the template is invalid:\n> Fields containing line breaks (CRLF), double quotes, and commas\n>  should be enclosed in double-quotes\n\nUsing https://www.npmjs.com/package/@vanillaes/csv\n\n```\ncsv.parse('a\\'a,\"b,b\",\"c\"c\",d.d,e`e')\n// Uncaught Error: CSVError: Illegal state [row:1, col:3]\n```","x":1690,"y":520,"wires":[]}]

RFC 4180 Section#2, point#4

Spaces are considered part of a field and should not be ignored

RFC 4180 Section#2, point#6

Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double-quotes

RFC 4180 Section#2, point#7

If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote

Thoughts (IMHO)...

The CSV node should be RFC 4180 compliant (if not by default, then via a checkbox option)

For node-red reputation purposes I think this is important

The built in help tip a wink to the RFC specs
Invalid column template should cause a UI error & if deployed, a runtime error

Perhaps we might consider using a proven NPM import? I tried about 6 promising ones - none handled all 6 tests 100% apart from  https://www.npmjs.com/package/@vanillaes/csv

Demo details...
Demo 1 - no column template
Input data
[
    {"c\"c":3,"d.d":4,"b,b":2,"e`e":5,"a'a":1},
    {"d.d":2,"e`e":1,"b,b":4,"c\"c":3,"a'a":5}
]

Extracts col template...
['c"c', 'd.d', 'b,b', 'e`e', "a'a"]
//c"c,d.d,b,b,e`e,a'a               as a CSV string

According to RFC4180 2.6 & 2.7 the CSV generated for the template is invalid:

2.6
Fields containing line breaks (CRLF), double quotes, and commas
should be enclosed in double-quotes

2.7
If double-quotes are used to enclose fields, then a double-quote
appearing inside a field must be escaped by preceding it with
another double quote

i.e. the column template should be...
"c""c",d.d,"b,b",e`e,a'a

Demo 2
Column Template
a'a,"b,b",c"c,d.d,e`e

parses as
['a'a,b', 'b,cc', 'd.d', 'e`e']

According to RFC4180 2.6 the CSV for the template is invalid:

Fields containing line breaks (CRLF), double quotes, and commas
should be enclosed in double-quotes

Using https://www.npmjs.com/package/@vanillaes/csv
csv.parse('a\'a,"b,b",c"c,d.d,e`e')
//Uncaught Error: CSVError: Illegal state [row:1, col:3]

Demo 3
Column Template
a'a,"b,b",c""c,d.d,e`e

parses as
['a'a,b', 'b,cc', 'd.d', 'e`e']

According to RFC4180 2.6 the CSV for the template is invalid:

Fields containing line breaks (CRLF), double quotes, and commas
should be enclosed in double-quotes

Using https://www.npmjs.com/package/@vanillaes/csv
csv.parse('a\'a,"b,b",c""c,d.d,e`e')
// Uncaught Error: CSVError: Illegal state [row:1, col:3]

Demo 4
Column Template
a'a,"b,b","c"c",d.d,e`e

incorrectly parses as
a'a,"b,b",cc,d.d,e`e

According to RFC4180 2.6 the CSV for the template is invalid:

Fields containing line breaks (CRLF), double quotes, and commas
should be enclosed in double-quotes

Using https://www.npmjs.com/package/@vanillaes/csv
csv.parse('a\'a,"b,b","c"c",d.d,e`e')
// Uncaught Error: CSVError: Illegal state [row:1, col:3]

Demo 5
Column Template
a'a,"b,b","c""c",d.d,e`e

parses as
['a'a', 'b,b', 'cc', 'd.d', 'e`e']
//a\'a,"b,b",cc,d.d,e`e            as a CSV string

According to RFC4180 2.7 this should be
[ "a'a", 'b,b', 'c"c', 'd.d', 'e`e' ]
// a'a,"b,b","c""c",d.d,e`e           as a CSV string

Using https://www.npmjs.com/package/@vanillaes/csv
csv.parse('a\'a,"b,b","c""c",d.d,e`e')
// [ "a'a", 'b,b', 'c"c', 'd.d', 'e`e' ]

Demo 6
Column Template
a'a ," b,b"," c""c",d.d, e`e

parses as
['a'a', 'b,b', 'cc', 'd.d', 'e`e']
//a'a,"b,b",cc,d.d,e`e             as a CSV string

According to RFC4180 2.4

Spaces are considered part of a field and should not be ignored

So this template should parse as
[ "a'a ", ' b,b', ' c"c', 'd.d', ' e`e' ]
// a'a ," b,b"," c""c",d.d, e`e          as a CSV string

Using https://www.npmjs.com/package/@vanillaes/csv
csv.parse('a\'a ," b,b"," c""c",d.d, e`e')
[ "a'a ", ' b,b', ' c"c', 'd.d', ' e`e' ]

csv.stringify([[ "a'a ", ' b,b', ' c"c', 'd.d', ' e`e' ]])
// a'a ," b,b"," c""c",d.d, e`e

Not totally sure about some of those assertions... - Don't forget this is a parser to / from a JS object ... and JS property names cannot start with spaces for example - so we cannot map 1:1 and be expected to do a two way conversion.

JS property names cannot start with spaces for example - so we cannot map 1:1 and be expected to do a two way conversion.

They can...

Not totally sure about some of those assertions... - Don't forget this is a parser to / from a JS object

Yes, but the examples were more about demonstrating things it does wrong (or maybe nicer to say "not RFC compliant")
Lets put it this way Dave, if a user feeds RFC compliant data into the CSV node - currently, there are several ways we "get it wrong"
For example, "c""c",d.d,"b,b",e`e,a'a\n3,4,2,5,1\n3,2,4,1,5 is 100% RFC4180 compliant
PROOF (using  https://www.npmjs.com/package/@vanillaes/csv) ...
csv.parse('"c""c",d.d,"b,b",e`e,a\'a\n3,4,2,5,1\n3,2,4,1,5')
// Results in ...
// [
//   [ 'c"c', 'd.d', 'b,b', 'e`e', "a'a" ],
//   [ '3', '4', '2', '5', '1' ],
//   [ '3', '2', '4', '1', '5' ]
// ]

ALT PROOF (from convertcsv.com/csv-to-json.htm)

Input data for above ↑
"c""c",d.d,"b,b",e`e,a'a
3,4,2,5,1
3,2,4,1,5

CSV NODE...

This can now be rebased with master to pickup the recent fix so we can get the tests passing cleanly before merging.

This can now be rebased with master to pickup the recent fix so we can get the tests passing cleanly before merging.

Nick, I don't think we should merge this until valid CSV data is producible. Also, @dceejay is working on a separate branch : CSV-handle-quotes-and-spaces

Naturally I disagree. This PR does fix the immediate issue that crashes the flow. The other branch to produce "more correct" CSV is a major breaking change that will upset flows, and will need careful introduction.

Naturally I disagree. This PR does fix the immediate issue that crashes the flow.

Sorry Dave, in that sense, I retract my previous statement.
Let me know if you want any help or legs or testing on your other branch.

@knolleary - it is already based on master  - rebasing it does re-run test - but still fails

Coverage increased (+0.01%) to 68.328% when pulling 94690fad7a8a3488f946a5a9424924eb26b4cc0a on CSV-header-props-with-quotes into 32999ffa84e96f5f72a11714055ec3e716af052a on master.


window.showQuickPick is not closed when promise is rejected

When showQuickPick is used with a promise, it shows a nice progress bar which is cool. But when the promise is rejected, it should close the quick pick in my opinion, which is not the case at the moment, the progress bar just keeps spinning.
return window.showQuickPick(new Promise((resolve, reject) => {
	setTimeout(() => {
		reject();
	}, 2000);
}));

// @bpasero

I fixed most of the issues: If there is an error in your promise, the picker will hide and you have a chance to handle the error from the returned promise from the pick() call.
@jrieken is our proxy from main to PH not handling errors? I have the following extension that shows a pick with error but in the console I still get an error about unhandled promise:
export function activate(context: vscode.ExtensionContext) {

	var disposable = vscode.commands.registerCommand('extension.sayHello', () => {
		vscode.window.showQuickPick(new Promise((c, e) => {
			setTimeout(() => {
				e(new Error("does not work"));
			}, 2000);
		})).then(null, (e) => {
			vscode.window.showErrorMessage(e.toString());
		});
	});

	context.subscriptions.push(disposable);
}

Thanks for the quick fix!

@bpasero The reject works as expected but there is an known issue with unhandled promise rejection. Our handler sits here: https://github.com/Microsoft/vscode/blob/master/src/vs/workbench/node/pluginHostProcess.ts#L33 but there is a well known issue with that. Since project rejection and handling can happen async (on different event loop ticks) a promise might be handled some time in the future. Since the node events seem to lack the actual promise causing this we can't have better logging.

@jrieken in the example code I gave, we have a valid use of our API and the console ends up to report unknown errors:

I would think this can happen from any promise that carries an error from the main side to the PH side. If we know this can happen and cannot fix it, we should filter these messages.

It's not possible with the version of nodejs we are using today. Please read the comment and the links I have pointed to.

And since this only happens when the event loop is released between rejection and handling of it I think its the better compromise to see some potential false positive here and there then not seeing any unhandled rejection.

tracking the false positive in unhandled rejection here #963

:+1:


Filler: Refactor complex functions

I'll create a separate PR exploding the filler to multiple files, but will leave this open too, if we decide to keep it in one file.

I'll create a separate PR exploding the filler to multiple files, but will leave this open too, if we decide to keep it in one file.

Filename
Size
Change

dist/chart.esm.js
73.7 kB
+65 B (0%)

dist/chart.js
93.2 kB
+64 B (0%)

dist/chart.min.js
66.2 kB
+14 B (0%)

Filename
Size
Change

dist/chart.esm.js
73.7 kB
+65 B (0%)

dist/chart.js
93.2 kB
+64 B (0%)

dist/chart.min.js
66.2 kB
+14 B (0%)

Closing in favor of #10040


Add a bare-bones utility for generating noise models from calibration data.

Given an Engine instance and a target processor, simple_noise_from_calibration_metrics will generate a depolarizing noise model with per-qubit depolarization probabilities pulled directly from the most recent calibration data.
Known issues/possible fixes:

Assumes the most recent calibration data is valid and relevant to users.

Allow users to specify a calibration run? (i.e. get_calibration)

Requires connection to the engine.

Provide static sample calibration data as an alternative?

Does not incorporate other error channels (dephasing, amplitude damping)

Update as relevant metrics become available.

This change is included as part of #2734.


Models common RMS Norm doesn't support interleaved

Describe the bug
Current rmsnorm implementation doesn't support interleave memory, only sharded. See file models/common/rmsnorm.py.
For Mixtral prefill we rely on the interleaved mode due to different seqlen supported. I've pushed a change that we're using inside the Mixtral prefill PR.
You can check the branch mtairum/interleaved_rmsnorm and the diff here: https://github.com/tenstorrent/tt-metal/compare/main...mtairum/interleaved_rmsnorm

Oh no I knew I forgot something, will fix today.

Merged


Install script should create a universal repo file

Today, the install.sh script for k3s creates a repo file that looks like rancher-k3s-common-<CHANNEL>.repo where <CHANNEL> is one of testing, latest, or stable. We should simply create a rancher-k3s-common.repo file and internally within that file set up the corresponding repository. We should also clean up the old *-<CHANNEL>.repo files so as to prevent conflicting packages when a user decides to change their installed channel.

Validated universal repo file is created, with corresponding repo set up internally.
cat  /etc/yum.repos.d/rancher-k3s-common.repo
[rancher-k3s-common-stable]
name=Rancher K3s Common (stable)
...
sudo yum list|grep rancher
k3s-selinux.noarch                                     0.2-1.el7_8                                       @rancher-k3s-common-stable

 cat  /etc/yum.repos.d/rancher-k3s-common.repo
[rancher-k3s-common-testing]
name=Rancher K3s Common (testing)

sudo yum list|grep rancher
k3s-selinux.noarch                                     0.3~rc1-0.el8                                     @rancher-k3s-common-testing

Validated *-.repo file is deleted upon uninstall.


PowerShell instructions are incorrect, mix AzureRm and Az module syntax

You've copy-pasted the PowerShell script instructions from DigiCert's web site, but those instructions are for the AzureRm PowerShell Module, version 2.18.  You're using Login-AzAccount which is an Az module command, but several of the following commands use AzureRm syntax which will not work with Az module 4.8 under PowerShell 7 (for instance, "-IssuerName" in Set-AzureKeyVaultCertificateIssuer)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 406baadb-00bb-a08c-ebfa-bf5f4981842e
Version Independent ID: 9858967a-6913-ec53-fcdc-868d76759fbb
Content: Integrating Key Vault with DigiCert Certificate Authority
Content Source: articles/key-vault/certificates/how-to-integrate-certificate-authority.md
Service: key-vault
Sub-service: certificates
GitHub Login: @msmbaldwin
Microsoft Alias: sebansal

@cpdohert
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

Document has been updated, changes should go live by end of the day. Thanks.

https://github.com/MicrosoftDocs/azure-docs-pr/pull/134720

#please-close


Appnexus adapter: Placement_id update

Updated description of appnexus parameters according to this issue:
https://github.com/prebid/prebid-server/issues/2103

There doesn't appear to be any code here -- just docs. Merging -- thanks.


'on attach' returning LIBUSB_ERROR_NOT_SUPPORTED (Win 7)

Hi, thanks for your help with my previous issue. The projects build fine now however when adding an event listener , I'm getting the 'not supported' error (I can/open close devices so the drivers are installed , although it shouldn't really matter here ). Are there any known issues on Windows 64 ?
code is pretty simple
let usb = require('usb');
usb.on('attach', function(device) {
console.log("Usb Device attached:\n", device);
});
Thanks

Oh, forgot I was using Cedware's fork (Electron issue), perhaps there's a problem there...

Yes, that fork uses upstream libusb, which does not support hotplug on Windows.

You can try using git master of this repository, which hopefully fixed the build issue that fork was intended to solve.


kpt pkg get doesn't terminate

kpt pkg get https://github.com/kubernetes/website/content/en/examples/application/cassandra cassandra
$ fetching package content/en/examples/application/cassandra from https://github.com/kubernetes/website to ...
Expected behavior
Fetch the package
Actual behavior
Hangs
Information
0.39 mac os
Steps to reproduce the behavior

Note that even though that repo is large, shallow clone terminates in 7s:
 time git clone --depth 1 https://github.com/kubernetes/website.git                                                                                <<<
Cloning into 'website'...
remote: Enumerating objects: 6974, done.
remote: Counting objects: 100% (6974/6974), done.
remote: Compressing objects: 100% (6059/6059), done.
remote: Total 6974 (delta 1191), reused 4021 (delta 468), pack-reused 0
Receiving objects: 100% (6974/6974), 161.19 MiB | 7.36 MiB/s, done.
Resolving deltas: 100% (1191/1191), done.
Updating files: 100% (6510/6510), done.
git clone --depth 1 https://github.com/kubernetes/website.git  7.37s user 3.94s system 40% cpu 27.600 total

We should also consider showing progress similar to clone, it's not good UX to be blocking for more than a few seconds without providing feedback.

So it does actually complete, but it is very slow. There are several reasons for this:

kpt first tries to fetch it using the package-specific ref (which in this case is content/en/examples/application/cassandra/master). But in this case this tag doesn't exist, so the initial attempt fails.
Each attempt actually tries fetching the repo two ways, first by fetching just the specified ref, and if that doesn't work, fetch the full repo and the hard reset to the desired ref.
The second attempt to just use the master ref succeeds, but even then it takes some time since it is a large repo.
kpt also downloads git submodules.

So overall this adds up. We should look into whether there are ways to speed this up.
And yes to providing feedback on progress. This applies to several kpt commands.

May want to split this into 2 issues: speeding up get and showing progress. For this package, it's really don't matter if it takes X minutes or it never terminates, the user won't stick around to find out.

The performance of the example here is improved substantially by the changes in https://github.com/GoogleContainerTools/kpt/pull/1778. There is still room to improve however, and we have a more specific issue to reduce the number of files we need to copy during get/update: https://github.com/GoogleContainerTools/kpt/issues/1807
The UX to show progress has also been improved, but this also remains something we'll continue to improve.
Based on this, I'm closing this issue.


WIP: use chrome-launcher for handling the browser.

this is the launcher module from lighthouse that we maintain.
https://www.npmjs.com/package/chrome-launcher
it defaults to an open port
addresses #21

Hi, can we use chrome-launcher with puppeteer now?

I copy some code from chrome-launcher to make it work, but it looks so dirty. Is there a better way to archive that? Thanks. 😉
const { getPlatform } = require('chrome-launcher/utils')
const chromeFinder = require('chrome-launcher/chrome-finder')
const Puppeteer = require('puppeteer')

async function setOptionsDefaults(options)
{
  if (options.executablePath === undefined) {
    const installations = await chromeFinder[getPlatform()]()
    if (installations.length === 0) {
      throw new Error('No Chrome Installations Found')
    }
    options.executablePath = installations[0]
  }

  return options
}

class Puppeteer2 extends Puppeteer {
  static async launch(options) {
    options = await setOptionsDefaults(options)
    return super.launch(options)
  }
}

module.exports = Puppeteer2


Update Bartholomew to handle relative URL links

ref #426
ref https://github.com/fermyon/bartholomew/issues/69
cc @dicej
Once Bartholomew has a fix, we should update it to handle relative links in Markdown properly.

Looks like this was completed in https://github.com/fermyon/spin/pull/432 🎉 ; closing.


When update to 4.2.2 some dashboard cant get value

With Item like this in version 4.2.1, can show good. Item in / ... / failed to get.

I'm also having this issue.  If you refresh the page, sometimes random items will work.  I have very zabbix-heavy dashboards, and they're basically useless now.  It's been rock solid for me for a while now.

Manual downgrade to 4.2.1 or 4.2.0 and always "plugin unavailable" :( Cant restore last good T.T

@JSylvia007 could you please post debug logs when error happens?

@quangphan1602 could you also try to change a regex? Now it's parsed on the backend and JS regex is not fully compatible with a go engine, so that might cause troubles in case of using (?=) and (?!).

I am facing similar issues. After the upgrade to 4.2.2. A lot of my panel say No data. Did some basic check and found in a lot of places where i select the item as /Interface .*/ fails
I see error "message:"Metric request error" in the query inspector . I then checked the grafana logs for this and found the following warning "Request Origin is not authorized"  and status 403

Similar issue, getting below message in grafana log after upgrading to latest grafana (8.1.5) and Zabbix plugin (4.2.3)
t=2021-09-22T08:03:48+0000 lvl=eror msg="Could not find plugin definition for data source: alexanderzobnin-zabbix-datasource"  t=2021-09-22T08:03:48+0000 lvl=eror msg="Could not find plugin definition for data source: alexanderzobnin-zabbix-datasource"  t=2021-09-22T08:03:48+0000 lvl=eror msg="Could not find plugin definition for data source: alexanderzobnin-zabbix-datasource"  t=2021-09-22T08:03:48+0000 lvl=eror msg="Could not find plugin definition for data source: alexanderzobnin-zabbix-datasource"
Any idea, how this can be fixed, the plugin is no more visible in Grafana GUI.

I've the same Problem. I attached a debug logfile from a request that won't work and a screenshot of the non working and the working example. It works if i remove the regex from the host and display only one host. The regex match all 6 hosts in the group and won't work.
grafana_debug.log

I hope this information can help fixing the issue.

This should be fixed in 4.2.3.

Thank you, It’s now working like a charm. Good job man.


executive_smach_visualization: 2.1.0-0 in 'lunar/distribution.yaml' [bloom]

Increasing version of package(s) in repository executive_smach_visualization to 2.1.0-0:

upstream repository: https://github.com/tork-a/executive_smach_visualization.git
release repository: https://github.com/tork-a/executive_smach_visualization-release.git
distro file: lunar/distribution.yaml
bloom version: 0.5.26
previous version for package: null

executive_smach_visualization

No changes

smach_viewer
* support for Qt5 (Kinetic)
* update CMakeLists.txt, package.xml, setup.py, smach_viewer.py for new xdot structure
* add necessary lines in xdot/__init__.py https://github.com/jbohren/xdot/pull/14
* copy xdot from https://github.com/jbohren/xdot, since system xdot is released in rosdep key https://github.com/ros/rosdistro/pull/4976
* wx viewer: checking to make sure item urls are strings to prevent crash
* Contributors: Jonathan Bohren, Kei Okada

See https://discourse.ros.org/t/releasing-repositories-form-other-people/1797/10


take updates of vscode

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: infoneershalin sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


portia-portia-16.02.2_Error in Run time

Hi I am configuring portia new release "portia-portia-16.02.2" in ubuntu 12.04 LTS OS. The before release "portia-15.12.2" working fine for me. but in new release when i run the command "bin/slyd"
i receive error like this
Traceback (most recent call last):
File "bin/slyd", line 29, in
splash.server.main(server_factory=make_server, argv=[])
TypeError: main() takes no arguments (2 given)
how can i clear it.
thanks in advance.

You will need to run the install steps again. Your version of splash is out of date.

already the installed splash version is 1.8
now the portia UI loading in browser, but when I load the url in UI error received as "internal server error"

How did you install portia?

cd ~portia/portia
pip install --upgrade pip
pip install -r slybot/requirements.txt
pip install -r slyd/requirements.txt
pip install pyopenssl ndg-httpsclient pyasn1
pip install splash
pip install scrapy
pip install jsonschema
pip install scrapely
pip install rfc3987
pip install loginform
pip install autobahn
pip install monotonic
pip install chardet
pip install requests
pip install scrapyjs
pip install dateparser
pip install -e slybot
pip install -e slyd

By installing splash that way you install an older version of it. We are currently using splash-2.0 in Portia, (it is scheduled for release next week). To install splash-2.0 run
pip install  git+https://github.com/scrapinghub/splash

while installing splash-2.0 using the above command
error: The requested URL returned error: 403 while accessing http://github.com/scrapinghub/splash-2.0/info/refs
fatal: HTTP request failed

Did you run the command I added above?

yes i run the above command, it shows error as
error: The requested URL returned error: 403 while accessing http://github.com/scrapinghub/splash-2.0/info/refs
fatal: HTTP request failed

The request failed because the repository https://github.com/scrapinghub/splash-2.0 doesn't exist. Are you adding a -2.0 to the end of the pip install command?

yes i tried like this its installed splash 2.0. and  to run the splash is it required pyqt5, because error comes
File "/usr/local/bin/splash", line 3, in
from splash.server import main
File "/usr/local/lib/python2.7/dist-packages/splash/server.py", line 12, in
from splash.qtutils import init_qt_app
File "/usr/local/lib/python2.7/dist-packages/splash/qtutils.py", line 11, in
from PyQt5.QtCore import (QAbstractEventDispatcher, QDateTime, QObject,
ImportError: No module named PyQt5.QtCore
shall i install pyqt5 to run the splash. is it work?

Yes you need to install pyqt5

is this portia required python3 or else can run in python2.7.6?


Extract unknown map into its own card

Take the warning and the unknowns map card that's currently in the disparity bar chart and pull it into its own card

Have a PR out, still need sign off on the copy/ux/placement of the card. See deploy preview https://deploy-preview-607--health-equity-tracker.netlify.app/exploredata

From Larry:

title = “Share of total Covid-19 cases in United States with Unknown Values for Race and Ethnicity”
Add warnings to BOTH map cards with UNKNOWN or KNOWN in all caps: "/!\ 42.9% of cases in reported had an unknown value for race and ethnicity. This chart displays data for cases where race and ethnicity was UNKNOWN"

Add warning to Share of total cases with unknown values:
/!\ 42.9% of cases in reported had an unknown value for race and ethnicity. This chart displays data for cases where race and ethnicity was UNKNOWN
Update warning in Disparity chart
/!\ 42.9% of cases in reported had an unknown value for race and ethnicity. This chart only displays data for cases where race and ethnicity was KNOWN

Updating per Chi's suggestion:
Add warning to Share of total cases with unknown values:
/!\ 42.9% of cases in reported had an unknown value for race and ethnicity. The chart below displays data for cases where race and ethnicity was UNKNOWN
Update warning in Disparity chart
/!\ 42.9% of cases in reported had an unknown value for race and ethnicity. The chart below only displays data for cases where race and ethnicity was KNOWN

Correcting grammar
Add warning to Share of total cases with unknown values:
/!\ 42.9% of cases reported an unknown value for race and ethnicity. The chart below displays the cases where race and ethnicity was UNKNOWN
Update warning in Disparity chart
/!\ 42.9% of cases reported an unknown value for race and ethnicity. The chart below only displays data for cases where race and ethnicity was KNOWN

Just bumping this to note that we still need to add the warning to the card that maps the unknown values:
/!\ 42.9% of cases reported an unknown value for race and ethnicity. The chart below displays the cases where race and ethnicity was unknown


using wildcard for angular2 peer dependencies

currently updating angular2 is not possible (current version beta15) as ng2-bootstrap has the dependency only to beta14.
suggestion:
change
"peerDependencies": {
"angular2": "2.0.0-beta.14"
},
using ~ or ^ for the version in order to make it independantly of the host

ya ya, I know
as soon as angular2 will get out from beta and will not push breaking changes with prerelease versions


updating the bulk ingest file metadata logic

Resolves this issue https://github.com/arc53/DocsGPT/issues/240#issue-1714665631

Hey @dartpain, would you mind approving the ruff workflow?


Leading comments in base64 encoded irules stripped by as3

Environment

Application Services Version: 3.18.0
BIG-IP Version: BIG-IP <IP_ADDRESS> Build 0.0.4 Point Release 2

Summary
When applying iRules that have the first line as a comment, the leading comment is stripped from the "applied" iRule. This leads to a non-idempotent state where every time I apply an as3 config, these rules get deleted and updated, even when the iRule has not changed.
Steps To Reproduce
Steps to reproduce the behavior:

Submit the following declaration:

{
    "msg": {
        "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/3.18.0/as3-schema.json",
        "action": "deploy",
        "class": "AS3",
        "declaration": {
            "class": "ADC",
            "id": "tenant.dev",
            "tenant": {
                "Shared": {
                    "template": "shared",
                    "test_irule": {
                        "class": "iRule",
                        "iRule": {
                            "base64": "IyB0aGlzIGlzIGEgdGVzdCBjb21tZW50CndoZW4gSFRUUF9SRVFVRVNUIHsKICAgICAgICBIVFRQOjpoZWFkZXIgcmVwbGFjZSBYLVJlYWxtIHN0YW5kYXJkCiAgICB9"
                        }
                    }
                }
            }
        }
    }
}

Observe the following error response:
The response is not an error - as3 accepts a valid declaration.

Expected Behavior
The base64 decode of the above text is: # this is a test comment when HTTP_REQUEST {         HTTP::header replace X-Realm standard     }
I expect that entire contents to be found in the declared iRule.
Actual Behavior
When that declaration is applied, what I see in the irule is:
ltm rule test_irule {
    partition tenant
when HTTP_REQUEST {
        HTTP::header replace X-Realm standard
    }
}

Thank you for reporting this. I have confirmed the bug and added it to our internal product backlog as AUTOTOOL-1513.

I also determined this was happening to trailing comments as well.

Is it possible to get an updated timeline on the expected fix for this issue?

This has been resolved in the 3.33.0 release.


Update to embedded-hal v1.0.0

Having accidentally submitted this PR to another fork here is an actual PR to update the driver to embedded-hal v1.0.0.
This PR:

Uses the new I2C trait
Switches DelayMs to DelayNs

Excellent, thanks! Looking for to see this merged.

This is a must! ping @juliangaal

Just ordered a new set of MPU6050s...
I'm hesitant to merge and/or release on crates without testing myself. Do the examples compile and work as expected?

Tested and working from my side


Cannot publish pages anymore since 3.1

After upgrading to 3.1 I'm unable to publish pages. I get a 500 error with the following trace:
Traceback (most recent call last):
  File "/home/vagrant/env/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 64, in __call__
    return self.application(environ, start_response)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 187, in __call__
    response = self.get_response(request)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 199, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/home/vagrant/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 236, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/home/vagrant/env/lib/python2.7/site-packages/django_extensions/management/technical_response.py", line 5, in null_technical_500_response
    six.reraise(exc_type, exc_value, tb)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/utils/decorators.py", line 105, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 206, in inner
    return view(request, *args, **kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/transaction.py", line 394, in inner
    return func(*args, **kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/reversion/revisions.py", line 297, in do_revision_context
    return func(*args, **kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/cms/admin/pageadmin.py", line 956, in publish_page
    published = page.publish(language)
  File "/home/vagrant/env/lib/python2.7/site-packages/cms/models/pagemodel.py", line 621, in publish
    self.save()
  File "/home/vagrant/env/lib/python2.7/site-packages/cms/models/pagemodel.py", line 492, in save
    super(Page, self).save(**kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/base.py", line 589, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/vagrant/env/lib/python2.7/site-packages/cms/models/pagemodel.py", line 507, in save_base
    return super(Page, self).save_base(*args, **kwargs)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/base.py", line 617, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/base.py", line 679, in _save_table
    forced_update)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/base.py", line 723, in _do_update
    return filtered._update(values) > 0
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/query.py", line 600, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1004, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/home/vagrant/env/lib/python2.7/site-packages/debug_toolbar/panels/sql/tracking.py", line 159, in execute
    return self._record(self.cursor.execute, sql, params)
  File "/home/vagrant/env/lib/python2.7/site-packages/debug_toolbar/panels/sql/tracking.py", line 101, in _record
    return method(sql, params)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/vagrant/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "cms_page_path_518270318703c18f_uniq"
DETAIL:  Key (path)=(0005) already exists.

The weird thing is that if I dump the query and the parameters, I get this:
>>> sql
u'UPDATE "cms_page" SET "path" = %s, "depth" = %s, "numchild" = %s, "created_by" = %s, "changed_by" = %s, "parent_id" = NULL, "creation_date" = %s, "changed_date" = %s, "publication_date" = %s, "publication_end_date" = NULL, "in_navigation" = %s, "soft_root" = %s, "reverse_id" = NULL, "navigation_extenders" = NULL, "template" = %s, "site_id" = %s, "login_required" = %s, "limit_visibility_in_menu" = NULL, "is_home" = %s, "application_urls" = NULL, "application_namespace" = NULL, "publisher_is_draft" = %s, "publisher_public_id" = %s, "languages" = %s, "revision_id" = %s, "xframe_options" = %s WHERE "cms_page"."id" = %s'
>>> params
(u'0005', 1, 0, u'foo', u'bar', u'2015-04-13 17:11:56.884334+00:00', u'2015-04-28 09:04:05.957800+00:00', u'2015-04-27 20:47:06.971613+00:00',   )

And if I run the query manually in a shell, the query succeeds. Maybe that's related to a transaction or something like that?

Running ./manage.py cms fix-tree fixed the issue. Not sure why though. I'll reopen the issue if it persists.

./manage.py cms fix-tree also fixed this problem for me.


Add Kubernetes deployment and Docker configuration for monolith app

This pull request includes significant changes to the deployment configurations for the monolith application, including updates to the monolith/deploy.yaml file and the addition of a new monolith/mysql-deployment.yaml file. The changes primarily focus on updating namespaces and improving secret management for database credentials.
Deployment Configuration Updates:

Namespace Update:

Changed the namespace from monolith to monolith-app in monolith/deploy.yaml. [1] [2]

Secret Management:

Updated database secret references from dbsecret to mysql-secret and changed the keys accordingly in monolith/deploy.yaml.

New MySQL Deployment Configuration:

Persistent Volume Claim:

Added a PersistentVolumeClaim for MySQL storage.

Secret Creation:

Created a new secret mysql-secret with encoded database credentials.

MySQL Service and Deployment:

Added a new service and deployment configuration for MySQL, including environment variables for database credentials and volume mounts for persistent storage.

lgtm merged


[NEW ICON] Swiper, Slick

Skill Name
Swiper, Slick
Why?
We should add these skills to the project because these two libraries are the most popular among frontend developers. They provide convenient and powerful tools for creating various carousels and interactive elements on websites. Using these libraries will help us enhance the functionality and attractiveness of our project, as well as save time in development thanks to their ready-made solutions.
Reference Image

slick no icon in svg. They use a font. Therefore, I will leave the link: https://kenwheeler.github.io/slick/

The 'Swiper JS' library is widely used in the frontend development community. If it is added, it will be great!


[Feature proposal] Allow subclassing of the T util

I'm working on a project that involves needing fallbacks (in some ways similar to #63, but differently).
For our system, we're needing to allow clients to optionally specify certain localized strings for fields in our system.  These strings can change depending on the route.  We wouldn't have to have async support for this, but it might be nice.
Currently, it seems like the best approach to handling this would be to refactor addon/utils/t.js to shift:
result = get(locale, read(path));

to this:

result = this.lookupResult(locale, path);

...

this.lookupResult = function(locale, path) {
  return get(locale, read(path));
};

Subclasses would then be able to handle looking up results in different places.
Is this something you'd want a pull request for?  I'm happy to send in a pull request, but may need a bit of guidance on how you want the subclassing to be allowed in the system (I'm relatively new to the JS/Ember world).
My initial thoughts on how to handle the changes would be to both change the above into a method and then shift T to be an Ember.Object.  That would then seem to allow us to extend the registered utils:t instance through an initializer.
Thoughts?

:+1: good idea, do you want to PR?

@bcardarella Yep, happy to.  Just wanted to make sure you wanted it before I bothered ;-)

@bcardarella Hmm, start wondering if I'm in over my head.  Any pointers on how to get T.create to act as a factory in the container, but also pass it parameters so it can instantiate a subclass?  I don't see any way to pass additional attributes through the Ember container to it ...

I'm unsure what you're trying to do.

You mean you can't read my mind? ;-)
Sorry, let me see if I can state it far clearer (I've been mulling over this for a bit longer now).
What I'm wanting to do (I think I have the proper terminology) is to setup a subclass of the addon/utils/t.js factory, which would allow me to swap in a different implementation of this.lookupResult() (see above).  Here's the code that seems to be causing me problems:
T.create = function(attributes) {
  var t = new T(attributes);
  // this seems to be where I'm tripped up
  var fn = bind(t, t.t);
  fn.destroy = function() {};
  return fn;
};

Currently, we seem to tightly bind the creation of our T factory instance with the binding of the t() method.  Is there any way we can split those into separate operations so that I can swap in a subclass of the factory instance?
I'm envisioning registering the default T 'factory' with the container if there isn't already on setup and then setting up the binding of the factory t() method later on (to an instance of whichever factory is stored in the container).
Am I making sense?

So that crazy code allows us to export the function binding. This way we can just call this.t(...) from controllers, components, routes, etc... instead of exporting the object which would require us to call this.t.t(...) which seems odd.
Let's take another approach to this: why do you want to modify the lookup handler? What is your use-case?

I'd gathered that that was what it was doing (after much head scratching ;-)
Re: use case, here goes.
We're trying to handle swapping out translation strings depending on the route selected (we'll be using different strings depending on the location in the app, even though we're using the same translation keys under the hood).  At the moment we don't have to have async loading of translations, but that will be requested pretty quickly, as our app is pretty heavily dependent on translations.
So what I was aiming for was a way to load in all our additional logic about what translation keys to use when without embedding a lot of that within the main add-on (as I don't see it being a common use case).
I'd thought I'd found a way to do it by having the locale/en.js file be an intelligent model (where we could embed logic about how to resolve the various keys), but the model gets cached in the container after initial lookup and I can't currently trigger a change there (unless I trash it as needed ...  hmm ...)
Thoughts?

I think it might make more sense to extract out the lookup handler as a service that itself can be lookup. Something similar to what happens in ember-validations where it will first attempt to find the service in the app layer, if that is a miss it will fallback to the default lookup handler.

Agreed, that sounds like a good plan.  Any suggestions on how to go about that?  That was the path I started heading down, but the "crazy code" (as you put it) ended up throwing me for a bit of a loop ;-)
Ideally we'd be able to have the lookup handler be extensible by other folks as needed.

I'm currently working on a solution (started last night). It might be a day or two as it has a bunch of edge cases I'm trying to account for.

Understood on the edge cases and appreciated.  I started in down that path before I opened this issue but I broke every test.  That's when I began realizing the complexity of your relatively simple looking code :-)

@bcardarella Any further thoughts on these changes?  I know you guys have been getting hammered weather-wise, so understood if you've had a few other things you've been working on ;-)

@acacheung sorry it took longer than it should have, the PR is above. Once green I'll merge

@bcardarella Sweet!  Thanks very much!


Missing \MongoDb\Operation docs

Can't find \MongoDb\Operation docs anywhere. It shouldn't already be used in production or documentation is missing?

We've documented the methods on the core classes (e.g. MongoDB\Database, MongoDB\Collection), which use these operations internally and serve as the real public-facing API.
I don't intend on documenting the operation classes in the library documentation, as most users will never interact with them directly and there is significant overhead involved with maintaining two sets of documentation for the same operation (e.g. MongoDB\Collection::find() and MongoDB\Operation\Find). That said, I've tried to be thorough with doc block comments across the operation classes so you can rely on those if you like. Additionally, you can rely on BC being preserved for classes annotated with @api (e.g. FindOneAndDelete). Private classes with no API guarantees have been annotated with @internal (e.g. FindAndModify).

I understand @jmikola. Thanks for the info.
Actually I'm looking for info about Map Reduce operations. I see some tests using \MongoDb\Operations\MapReduce here https://github.com/mongodb/mongo-php-library/blob/dcb9d12023ebf9cc4782ffce454b3e191b5d3e8c/tests/Operation/MapReduceTest.php.
Since Map Reduce is widely used (not for internal stuff I guess), shouldn't at least \MongoDb\Operations\MapReduce and \MongoDb\Operations\MapReduceResults operations be documented?

Ah, it looks like I forgot to add documentation in #392. I've re-opened PHPLIB-53 and will make sure we get the Collection::mapReduce() method documented with 1.2.0. MapReduceResult will likely be documented on the same page, as that's the only time users will encounter the object.


Execution failed for task ':mapmyindia_direction_plugin:compileReleaseJavaWithJavac'.

[√] Flutter (Channel stable, 2.5.3, on Microsoft Windows [Version 10.0.22000.348], locale en-US)
[√] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
[√] Chrome - develop for the web
[√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.2)
[√] Android Studio (version 2020.3)
[√] VS Code, 64-bit edition (version 1.62.3)
[√] Connected device (3 available)

• No issues found!```

I got the same issue, do you get the solution?


UI : spelling

Interface a is missing in all this 3 modal

Done,

Verified  this issue on keeper app v 1.1.8(265)

@akshay8551 please check

Resolved,

Completed

Verified this issue on keeper app v 1.1.8(271)


test: add test for json status params

add test covering #4

Should also make the tests a bit less flaky at github actions.


Getting java.io.IOException: Unexpected response code for CONNECT: 400

Hi @MajedurRahmanMmr , Google Bard is only available in US/UK, please make sure the network is connected.

And make sure you have the right Session ID, It a cookie named __Secure-1PSID.

As no response for a long time, will close it.


Use til::some<T,N> to replace the SomeViewports class

Summary of the Pull Request
Let's give it a test drive.

References

PR Checklist

[x] Closes #4162
[x] CLA signed. If not, go over here and sign the CLA
[ ] Tests added/passed
[ ] Requires documentation to be updated
[ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed
Build and run it.

Oops forget some files. Will look into it later.

@msftbot make sure @miniksa signs off on this

:tada: Once again, thanks for the contribution!
This pull request was included in a set of conhost changes that was just
released with Windows Insider Build 19603.


eosio.token account

Hello guys, before deploying contract, we create account to deploy it to:  cleos create account eosio eosio.token PUB_KEY PUB_KEY . the name of the account is "eosio.token". So there is my question? How can I create account with this name? I know that account with this name already exists. Isn't account name unique? I know that something specific is going on, but I can't understand well. Can someone explain me?

The tutorials are for a private testnet, which needs an eosio.token account to operate properly.  Contracts can be deployed to any account with the RAM purchased sufficient to hold the contract, one contract per account.  See the Hello World Contract tutorial for an example of a different account name being used.
Note that even that tutorial is geared for a private testnet—account names with dots in them are reserved for the bidding process on mainnet.  If and when the .code suffix is successfully bid on mainnet, the tutorial name hello.code would be purchased from the owner of that suffix.
To get an account on mainnet, if you didn't buy and register EOS during the ICO, you must purchase one from one of the available services.  The account names that may be bought without auction must be exactly 12 characters long and consist of the digits 0-5 and the letters a-z.  (This restriction is a side effect of the fact that every account 'name' in EOS is actually a base32-encoded 64 bit integer.)  Once you have your new account, you may deploy your contract to it using cleos.


Changed option name?

In https://github.com/seaweedfs/seaweedfs/wiki/Production-Setup#setup-filer
There is defaultReplicaPlacement option, but I think it should be defaultReplication. Is it right?

correct.


export all types in web-features package

We currently have to do some typescript shenanigans to get access to certain types from web-features for use in our code:
// web-features doesn't export these types directly so we need to do a little typescript magic:
import type { features } from "web-features";
type SupportStatus = (typeof features)[keyof typeof features]["status"];
type BrowserIdentifier =
  keyof (typeof features)[keyof typeof features]["status"]["support"];

It would be far nicer if we could simply import the types directly:
import type { SupportStatus, BrowserIdentifier } from "web-features";

This PR enables that, by exporting all the types, and renaming browserIdentifier to the more type-like BrowserIdentifier

@LeoMcA This is a bit stale (one or two of the changes you proposed have been picked up already, I think). Do you want to come back to it or open a superseding PR? Or is this something someone else should take up?

This has gone a bit stale (and some of the proposed changes have already been picked up). Feel free to reopen or send a new PR if this is still relevant. Thank you!


Sending notification to specific user

I am currently working on a queue app and want users to get a notification when they are first in line.
I am very new to React Native and have never used One Signal before, but decided I wanted to use it.
All code below is part of my componentWillMount() method.
The part before the if-sentence is not working the way I want as I am not receiving any notification when OneSingal.postNotification(contents, data, playerId) is called. How should postNotification be formatted? Because right now nothing happens.
The part inside the sentence results in a error in the emulator when ran. Error is stating that the OneSignal.idsAvaliable() is  not a function. I just tried to follow this; https://github.com/geektimecoil/react-native-onesignal/issues/4
  const data = null; // some array
  const contents = 'You are first in line';
  const playerId = '7997ab8f-3029-4e2c-b58e-3072e0284435';
  OneSignal.postNotification(contents, data, playerId);

//Checks current users place in line (if 1 a notification should be sent)
  if (this.props.place === 1) {
    console.log('you are first');
    OneSignal.idsAvailable((idsAvailable) => {
      console.log(idsAvailable.playerId);
      console.log(idsAvailable.pushToken);
      OneSignal.postNotification(contents, data, idsAvailable.playerId);
    });
  }

Any help would be much apreciated. I have gone through basically all information I could find on OneSignal and other similar threads. As I said I dont have much experience and the OneSignal documentation was not very helpful and too abstract. Thanks

You should add the below line to your component will mount function
OneSignal.addEventListener("ids", this.onIds);
and have a function
onIds = device => {
    console.log(device.userId);
    console.log(device.pushToken);
}


Stone cluster size ignores 0

If you set the size of stone cluster to 0, it gets ignored and uses default value.

Makes sense. I'll make it simply not generate but the blocks will still be craftable. Thanks for the recommendation :)


fix: added HTTP to F5 URL

Incorrectly formatted URL broke the table in the docs:

The API checker skipped it, because we currently don't check gRPC.

@andresaiello please, review.


How to send icmpv6NS packets through gopacket

	ethernetLayer := &layers.Ethernet{
		SrcMAC:       srcMac,
		DstMAC:       dstMac,
		EthernetType: layers.EthernetTypeIPv6,
	}

	ipv6layers := &layers.IPv6{
		Version: 6,
		SrcIP: srcIP,
		DstIP: snm,
		NextHeader: layers.IPProtocolICMPv6,

	}

	icmplayers := &layers.ICMPv6NeighborSolicitation{
		TargetAddress: srcIP,

	}

	buffer := gopacket.NewSerializeBuffer()
	opts := gopacket.SerializeOptions{
		FixLengths:       true,
		ComputeChecksums: true,
	}
	err := gopacket.SerializeLayers(buffer, opts,
		ethernetLayer,
		ipv6layers,
		icmplayers,
	)

Through the above construction of data packet transmission, it is found that the data packet is defective in wireshark capture.
Is there something wrong with the icmp-layers construction?

Or how to modify


[HOW-TO] Keep camera acquisition rate using PyQT5 (threads vs processes)

I am developing an app which takes images from a camera, do some processing with each frame and display a GUI for the user. The hardware I am trying this on is a Raspberry Pi 4 model b, an Arducam camera module having camera sensor AR1820HS and, as recommeded, PyQT5 for the GUI. The processing involves face detection, eye landmarks and neural networks inference using Coral Accelerator.
I have already the camera adquisition code working at around 48 fps (1224x922) and image processing code takes 25 ms. I would need more resolution when this works so I can more details. This means 3840x2160px which allows 14,1 fps (YUV420) if done independently.
Well, I have looked at other similar developments and I have beeing doubting between using multiprocessing or multithreading to make all this works efficiently in a concurrent manner. I decided to implement 2 QThreads (apart from GUI thread) using a queue to share the acquired frames. One of them would be the frame producer thread and the other would process the frame and output a result.
Below is the code I wrote...
The problem is that I am getting around 20 fps for 1224x922px and image processing is just returning the average of the frame.
Am I doing something wrong? Is this as high as it can get?
Should I try multiprocessing?
import sys

import atexit

from queue import Queue

from PyQt5.QtCore import Qt, QTimer
from PyQt5.QtWidgets import (
    QApplication,
    QLabel,
    QMainWindow,
    QPushButton,
    QVBoxLayout,
    QWidget,
    QMessageBox,
    QAction,
)

from PyQt5.QtCore import QObject, QThread, pyqtSignal

from picamera2 import Picamera2
import libcamera

from picamera2.controls import Controls

import numpy as np
import cv2 as cv

ANALOG_GAIN = 4.0
EXPOSURE_TIME = 40000
RESOLUTIONS = ["1224x922", "2448x1378", "3840x2160", "4912x3684"]
SELECTED_RES = RESOLUTIONS[0]
#FORMAT = 'RGB888'
FORMAT = 'YUV420'

class Camera_Producer(QObject):
    frame_produced = pyqtSignal(int)

    def __init__(self, buffer_queue):
        """Initial camera configuration
        """
        super().__init__()
        atexit.register(self.cleanup)
        self.buffer_queue = buffer_queue

        self.picam2 = Picamera2()

        resH = int(SELECTED_RES.split('x')[0])
        resV = int(SELECTED_RES.split('x')[1])

        still_config = self.picam2.create_still_configuration(main={"format": FORMAT, "size": (resH, resV)})
        still_config["transform"] = libcamera.Transform(hflip=0, vflip=0)
        self.picam2.configure(still_config)
        self.picam2.start()

        metadata = self.picam2.capture_metadata()
        controls = {c: metadata[c] for c in ["ExposureTime", "AnalogueGain", "ColourGains"]}
        print("Camera controls: {}".format(controls))

        ctrls = Controls(self.picam2)
        ctrls.AnalogueGain = ANALOG_GAIN
        ctrls.ExposureTime = EXPOSURE_TIME
        self.picam2.set_controls(ctrls)

    def cleanup(self):
        print("Camera destroyed")

    def run(self):
        """Adquire images and stores in queue for later processing by the consumer thread.
        It will adquire all the time
        """
        while(1):
            if self.buffer_queue.qsize() == 0:
                frame = self.picam2.capture_array()

                if FORMAT == 'RGB888':
                    img_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
                elif FORMAT == 'YUV420':
                    img_gray = cv.cvtColor(frame, cv.COLOR_YUV420p2GRAY)

                self.buffer_queue.put(img_gray)
                self.frame_produced.emit(1)

class Algorithm_Consumer(QObject):
    frame_processed = pyqtSignal(int)
    mean_image = pyqtSignal(float)

    def __init__(self, buffer_queue):
        """Initial algorithms configuration (ie. load models)
        """
        super().__init__()
        atexit.register(self.cleanup)
        self.buffer_queue = buffer_queue

    def cleanup(self):
        print("Algorithm destroyed")

    def run(self):
        """Grabs image, or images, from camera queue, process it and update mouse position
        """
        while(1):
            if self.buffer_queue.qsize() > 0:
                frame_gray = self.buffer_queue.get()
                self.mean_image.emit(np.mean(frame_gray))
                self.frame_processed.emit(1)

class Window(QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)
        self.clicksCount = 0
        self.setupUi()

        #Creates Queue for sharing data (images)
        queue_buffer = Queue(maxsize=1)

        #Initializes camera and algorithm threads
        self.thread_camera = QThread()
        self.worker_camera = Camera_Producer(queue_buffer)
        self.worker_camera.moveToThread(self.thread_camera)
        self.thread_camera.started.connect(self.worker_camera.run)
        self.thread_camera.start()

        self.thread_algorithm = QThread()
        self.worker_algorithm = Algorithm_Consumer(queue_buffer)
        self.worker_algorithm.moveToThread(self.thread_algorithm)
        self.thread_algorithm.started.connect(self.worker_algorithm.run)
        self.thread_algorithm.start()

        #Way to handle proper closure of all objects
        quit = QAction("Quit", self)
        quit.triggered.connect(self.closeEvent)

        #10 seconds timer to update fps (debugging purposes for now)
        timer = QTimer(self)
        timer.timeout.connect(self.updateFrameRate)
        timer.start(1000) # 1 Second Refesh Rate
        self.counter = 0
        self.camera_counter = 0
        self.algorithm_counter = 0

        self.worker_camera.frame_produced.connect(self.updateCameraCounter)
        self.worker_algorithm.frame_processed.connect(self.updateAlgortithmCounter)
        self.worker_algorithm.mean_image.connect(self.algorithmResult)

        self.result = 0

    def updateCameraCounter(self):
        self.camera_counter += 1

    def updateAlgortithmCounter(self):
        self.algorithm_counter += 1

    def algorithmResult(self, result):
        self.result = result
        #print("Mean image result: {}".format(result))

    def updateFrameRate(self):
        self.counter += 1
        #print("General Counter: {}".format(self.counter))

        #TODO Update value according to interval time defined
        print("Frame Rate - Camera: {} fps".format(self.camera_counter))
        print("Frame Rate - Algorithm: {} fps".format(self.algorithm_counter))

        print("---o---")

        #Resets values
        self.algorithm_counter = 0
        self.camera_counter = 0

    def closeEvent(self, event):
        reply = QMessageBox.question(self, 'Cerrar aplicación?',
                                     'Estás seguro que quieres salir?',
                                     QMessageBox.Yes | QMessageBox.No, QMessageBox.No)

        if reply == QMessageBox.Yes:
            self.thread_camera.quit()
            self.thread_algorithm.quit()

            print("Destroyed all")
            if not type(event) == bool:
                event.accept()
            else:
                sys.exit()
        else:
            if not type(event) == bool:
                event.ignore()

    def setupUi(self):
        self.setWindowTitle("Freezing GUI")
        self.resize(300, 150)
        self.centralWidget = QWidget()
        self.setCentralWidget(self.centralWidget)
        # Create and connect widgets
        self.clicksLabel = QLabel("Counting: 0 clicks", self)
        self.clicksLabel.setAlignment(Qt.AlignHCenter | Qt.AlignVCenter)
        self.stepLabel = QLabel("Long-Running Step: 0")
        self.stepLabel.setAlignment(Qt.AlignHCenter | Qt.AlignVCenter)
        self.countBtn = QPushButton("Click me!", self)
        self.countBtn.clicked.connect(self.countClicks)
        self.longRunningBtn = QPushButton("Long-Running Task!", self)
        self.longRunningBtn.clicked.connect(self.runLongTask)
        # Set the layout
        layout = QVBoxLayout()
        layout.addWidget(self.clicksLabel)
        layout.addWidget(self.countBtn)
        layout.addStretch()
        layout.addWidget(self.stepLabel)
        layout.addWidget(self.longRunningBtn)
        self.centralWidget.setLayout(layout)

    def countClicks(self):
        self.clicksCount += 1
        self.clicksLabel.setText(f"Counting: {self.clicksCount} clicks")

    def reportProgress(self, n):
        self.stepLabel.setText(f"Long-Running Step: {n}")

    def runLongTask(self):
        self.reportProgress(1)

app = QApplication(sys.argv)
win = Window()
win.show()

sys.exit(app.exec())

Hi, there's probably a bit more there than I can reasonably digest, but here are a few things I would check.
Firstly, I'd take the processing after capture_array() our of your Camera_Producer.run() method, just to check that we really can get frames at the rate you want.
If you want greyscale images, YUV420 is a much better format to use. I'd have thought you could just do img_gray = frame[:resV, :resH] which would hopefully just give you a view onto the buffer you already have, rather than copying it again (though I don't know what cv2.cvtColor will do in this case).
I was a bit bothered by the Algorithm_Consumer.run() method which, unless I've misunderstood things, is going to busy wait. I think the test on the queue size may be superfluous as I think queue.get() will block until there's something there.
Also Camera_Producer.run() might also by busy-waiting while there's something in the queue? Again the test on the queue size being zero may be unnecessary. IIRC, you can set up a queue with a max size, so that queue.put() would block until there's space, so that might be a way to stop the queue growing indefinitely.
Multi-processing has to be used with care. Unless you know the runes for getting image buffers across without copying them then it can be quite slow. Though I suppose if the processing is way slower than buffer copying then maybe it's not a big deal.
Multi-threading works well so long as the processing you're doing either blocks for something else to happen (during which it should release the GIL), or calls into a big C/C++ library (in my experience these normally release the GIL while you're in them, though I'm sure there are some that don't...).

Hello @davidplowman, thanks for your help!
Where should I place capture_array() for your test in this multithreaded code? The result I reported (48 fps (1224x922)) is just a simple script where I configure the camera and get the mean time taken to obtain 100 frames.
Regarding the other comments:

For some situations, I might need RGB data. I will use what you suggest for when I need YUV.
Is there a way to tell the thread to yield? The size of the queue was set to 1 and might need to set to 2 in other cases.
Below I leave some code that I am using to the test multiprocesing... I have 2 image processing codes that I would need to use (one or the other which is faster) and one of them takes around 250 ms per frame. I was concerned GIL was causing the issues in multithreading. Is there a way to estimate how much time it would take to share 8 MB (3840x2160, grayscale)?

`from time import perf_counter, sleep
import multiprocessing as mp
from queue import Empty
from picamera2 import Picamera2
import libcamera
from PIL import Image
from picamera2.controls import Controls
import numpy as np
import cv2 as cv
ANALOG_GAIN = 4.0
EXPOSURE_TIME = 40000
RESOLUTIONS = ["1224x922", "2448x1378", "3840x2160", "4912x3684"]
SELECTED_RES = RESOLUTIONS[0]
FORMAT = 'RGB888'
#FORMAT = 'YUV420'
def do_capture(queue, finished):
picam2 = Picamera2()
resH = int(SELECTED_RES.split('x')[0])
resV = int(SELECTED_RES.split('x')[1])

still_config = picam2.create_still_configuration(main={"format": FORMAT, "size": (resH, resV)})
still_config["transform"] = libcamera.Transform(hflip=0, vflip=0)
picam2.configure(still_config)
picam2.start()

metadata = picam2.capture_metadata()
controls = {c: metadata[c] for c in ["ExposureTime", "AnalogueGain", "ColourGains"]}
print("Camera controls: {}".format(controls))

ctrls = Controls(picam2)
ctrls.AnalogueGain = ANALOG_GAIN
ctrls.ExposureTime = EXPOSURE_TIME
picam2.set_controls(ctrls)
while(1):
    if queue.qsize() == 0:
        t1_start = perf_counter()

        frame = picam2.capture_array()

        if FORMAT == 'RGB888':
            img_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
        elif FORMAT == 'YUV420':
            img_gray = cv.cvtColor(frame, cv.COLOR_YUV420p2GRAY)

        t1_stop = perf_counter()
        time_capture = t1_stop - t1_start

        print("Time using for capturing image: {:.2f}ms".format(1000*time_capture))

        queue.put(img_gray)

def do_processing(queue, finished):
while(1):
if queue.qsize() > 0:
frame_gray = queue.get()
print(np.mean(frame_gray))
        sleep(0.1)
        #print('%d: Processing image with size %dx%d' % (
        #    os.getpid(), frame_gray.size[0], frame_gray.size[1]))

if name == 'main':
queue = mp.Queue()
finished = mp.Event()
capture_proc = mp.Process(target=do_capture, args=(queue, finished))
processing_proc = mp.Process(target=do_processing, args=(queue, finished))

processing_proc.start()
capture_proc.start()

processing_proc.join()
capture_proc.join()`

I looked at the documentation and there is actually a way to yield. So, I added "QThread.yieldCurrentThread()" in an else condition within each while(1) and I am getting slighter better results:
25fps instead of 18 fps.
Still not satisfied...

So I understand that sometimes one finds oneself in a situation where "yielding" a thread is necessary, nonetheless it always worries me that it's just masking a different problem. Did you the problem with the busy waiting? Normally, so long as threads sleep properly when they aren't doing anything useful then I would mostly expect things to work.
Also, is your code setting the exposure time to 40000 (i.e. 40ms)? In this case the maximum achievable framerate will be 1000/40 = 25 fps.
I'm afraid I can't tell you for sure how long it will take to copy these buffers between processes. I can only really suggest that you write to code to test the performance - possibly there is even something out there on the web.

Ok, I found the bug... It was in my code. I had to comment the "sleep(0.5)" and it is giving the same results now as in the multithreaded version... I have copied it below...
The 0.5 seconds gives some time to the camera module to acquire the image... I do not know why I added it.
The weird thing now is that the YUV420 takes more time than RGB888. For example, at 2432x1840, it takes 45 ms for RGB888 and 88 ms for YUV. Is this because the data is converted to YUV in the Raspberry and it adds more time? The format for this camera, at this resolution, is 2432x1840-SGRBG10_1X10. The camera mode displayed by --list-cameras says for this resolution: 2432x1840 [20.01 fps - (0, 0)/2432x1840 crop]
`while iterator < IMAGE_COUNT:
    t1_start = perf_counter()
    im_captured = picam2.capture_array()

    #Store image in gray
    if FORMAT == 'RGB888':
        img_gray = cv.cvtColor(im_captured, cv.COLOR_BGR2GRAY)
    elif FORMAT == 'YUV420':
        img_gray = cv.cvtColor(im_captured, cv.COLOR_YUV420p2GRAY)

    t1_stop = perf_counter()
    time_capture = t1_stop - t1_start
    time_capture_array.append(time_capture)

    iterator += 1
    #sleep(0.5)`

Glad it's working better! Using a "still" configuration will by default do extra high quality colour denoising if it thinks it can, which is actually when you ask for a YUV420 image (because for still image capture you care less about framerate, or so it thinks...). To stop it, add controls={'NoiseReductionMode': 1} into your call to create_still_configuration. Or alternatively try create_video_configuration instead.

I need both quality and speed... I will have set a line at some point.
Thanks for your help!


pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)")

I am getting same error
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)")
VM: Ubuntu latest: 20
python: unixodbc, pyodbc installed
command: odbcinst -j
Result:
unixODBC 2.3.7
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /home/vsts/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
command: cat /etc/odbcinst.ini
Result:
[ODBC Driver 17 for SQL Server]
Description=Microsoft ODBC Driver 17 for SQL Server
Driver=/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.8.so.1.1
I am able to execute it locally no issues. Its only when i use Azure pipeline using Ubuntu latest VM unable to connect sql server with above error.
I have tried with
ip and host both.
also, tried with Driver: SQL Server and {ODBC Driver 17 for SQL Server}
conn = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};'
'Server=ip/host;'
'Database=xxx;'
'Trusted_Connection=yes;'
'UID=xxx;'
'PWD=xxx')
Then I have also run below commans:
sudo apt-get update
sudo apt-get install -y unixodbc-dev
Now i am seeing below error:
pyodbc.OperationalError: ('HYT00', '[HYT00] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0) (SQLDriverConnect)')

No luck.

"Login timeout expired" means the server is not accessible for some reason. Can you ping it from that machine? Can you connect to it by other means (such as SSMS)?
This error can be caused by many things, such as server service not running, firewalls etc.

I am having the same exact issue - our production environment was fine until last Friday. Receiving the same error "pyodbc.OperationalError: ('HYT00', '[HYT00] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0) (SQLDriverConnect)')".
Can connect locally, but not on our AWS EC2 instance. We are running an older version of Ubuntu 18.04.5.  Can connect via SSMS. Interested in hearing recommendations for troubleshooting.

Figure out what changed to cause it. This error is usually related to networking.

@ucop-ocio
I had same issue earlier on AWS EC2 VM, Its resolved with below in Python:

conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};'
                              'SERVER=' + server + ';'
                              'DATABASE=' + db + ';'
                              'UID=' + user + ';'
                              'PWD=' + pwd)

Take out 'Trusted_Connection=yes;' in connection. You don't need on external system.


Add missing double overloads for GetAttr, GetNodeAttr, and TryGetNodeAttr

Tangentially related to issue #39206.
For attributes of type double, it is not possible to directly call GetAttr. Instead, a temporary float needs to be created and passed to GetAttr, the float then needs to be cast to double. This PR aims to fix this.

There's some concern that this change is risky, given that existing users of these functions who pass constants will switch to the double version. C++ treats floating point constants as double by default.

I understand that concern. However, I think the missing functionality is just as bad.
I think it also must be said that anyone using the constant 1.0 would be expecting it to be of type double, since that is the default for C++, if they specifically wanted the float then they should be using 1.0f as I did in the unit tests.
Further to that, tensorflow already supports, and distinguishes between, both float32 and float64 data types every where else, not distinguishing between float and double here could be seen as breaking expected functionality and semantics.

For @tensorflow/api-owners:
We agree that this is bad as is but unfortunately, this PR cannot be merged because it would break backwards compatibility. A path forward would be submitting the functionality as a new op.

As an alternative. Would it be reasonable to do the following

Implement a new function called GetAttribute (or similar) that works in the correct way.
Replace every usage of GetAttr with GetAttribute
Make GetAttr deprecated to push people to the new GetAttribute function

That sounds reasonable.
On Mon, Jun 1, 2020 at 8:49 PM Alex Biddulph<EMAIL_ADDRESS>wrote:

As an alternative. Would it be reasonable to do the following

Implement a new function called GetAttribute (or similar) that
works in the correct way.
Replace every usage of GetAttr with GetAttribute
Make GetAttr deprecated to push people to the new GetAttribute
function

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/pull/39452#issuecomment-637254411,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAABHRMDX57Y6JHCFQLRCELRURZGJANCNFSM4M6WD2AA
.

--

Alex

@Bidski Any update on this PR? Please. Thanks!

@Bidski Can you please resolve conflicts? Thanks!

@Bidski Still, conflicts are appearing. Can you please resolve those? Thanks!

@Bidski Can you please resolve conflicts? Thanks!

@Bidski Can you please resolve conflicts? Thanks!

@gbaned conflicts are resolved

@Bidski Can you please resolve conflicts? Thanks!

@Bidski Can you please resolve conflicts? Thanks!

@Bidski Can you please resolve conflicts? Thanks!

@Bidski Can you please resolve conflicts? Thanks!

@Bidski  Can you please resolve conflicts? Thanks!

@Bidski  Can you please resolve conflicts? Thanks!

@Bidski  Can you please check @kkimdev's comments and resolve conflicts?. Thanks!

It has been 15 days with no activity and the awaiting response label was assigned. Is this PR still valid? Assigning the stalled label. Please comment to reassure me that this is still being worked on.

It has been 15 days with no activity and the awaiting response label was assigned. Is this PR still valid? Assigning the stalled label. Please comment to reassure me that this is still being worked on.

@gbaned still working on this

@gbaned still working on this

@Bidski  Can you please review this PR ? Thanks!

The API owners are moving to a rotation based system instead of the meetings. Looks like @jhseu and @rohan100jain are already reviewers. Please review on behalf of the API owners.

@Bidski Any update on this PR? and please resolve conflicts Thanks!

It has been 14 days with no activity and the awaiting response label was assigned. Is this PR still valid? Assigning the stalled label. Please comment to reassure me that this is still being worked on.


Move traditions into mixins

Closes https://github.com/laws-africa/indigo-web/issues/19

Oops, looks like you manage to get .sass-cache committed accindentally.


used WorkerRuntime in standalone

Standalone was giving an error since it was not initializing WorkerRuntime.
I initialized it.
I also disabled fault-tolerance and zoo keeper settings in standalone if they are set.
Since, we are neither supporting fault tolerance nor zoo keeper at standalone yet.

@ahmet-uyar there is a conflict

@ahmet-uyar there is a conflict

fixed it. it was because of the other merged pull request.


[P4_Symbolic] Modified p4_symbolic/symbolic/expected/table.smt2 [Evaluate parsers symbolically.]

Keyword Check:
~/sonic-buildimage/src/sonic-p4rt/sonic-pins$ ~/tools/keyword_checks.sh .
Keyword check Passed.
Build Result:
/sonic/src/sonic-p4rt/sonic-pins$ bazel build $BAZEL_BUILD_OPTS ...
INFO: Analyzed 634 targets (0 packages loaded, 0 targets configured).
INFO: Found 634 targets...
INFO: Elapsed time: 0.771s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
Test Result:
/sonic/src/sonic-p4rt/sonic-pins$ bazel test $BAZEL_BUILD_OPTS --cache_test_results=no ...
INFO: Analyzed 634 targets (0 packages loaded, 0 targets configured).
INFO: Found 423 targets and 211 test targets...
INFO: Elapsed time: 176.439s, Critical Path: 112.17s
INFO: 265 processes: 320 linux-sandbox, 18 local.
INFO: Build completed successfully, 265 total actions
//dvaas:port_id_map_test                                                 PASSED in 0.8s
//dvaas:test_run_validation_golden_test                                  PASSED in 0.1s
//dvaas:test_run_validation_test                                         PASSED in 0.6s
//dvaas:test_run_validation_test_runner                                  PASSED in 0.0s
//dvaas:test_vector_stats_diff_test                                      PASSED in 0.1s
//dvaas:test_vector_stats_test                                           PASSED in 0.1s
//dvaas:test_vector_test                                                 PASSED in 0.7s
//dvaas:user_provided_packet_test_vector_diff_test                       PASSED in 0.1s
//dvaas:user_provided_packet_test_vector_test                            PASSED in 0.1s
//gutil:collections_test                                                 PASSED in 0.5s
//gutil:io_test                                                          PASSED in 0.5s
//gutil:proto_matchers_test                                              PASSED in 0.7s
//gutil:proto_ordering_test                                              PASSED in 0.5s
//gutil:proto_test                                                       PASSED in 0.5s
//gutil:status_matchers_test                                             PASSED in 0.5s
//gutil:test_artifact_writer_test                                        PASSED in 1.1s
//p4rt_app/tests:resource_limits_test                                    PASSED in 2.8s
//p4rt_app/tests:response_path_test                                      PASSED in 5.9s
//p4rt_app/tests:role_test                                               PASSED in 1.5s
//p4rt_app/tests:state_verification_test                                 PASSED in 3.3s
//p4rt_app/tests:vrf_table_test                                          PASSED in 2.6s
//p4rt_app/tests/lib:app_db_entry_builder_test                           PASSED in 0.1s
//p4rt_app/utils:event_data_tracker_test                                 PASSED in 0.0s
//p4rt_app/utils:table_utility_test                                      PASSED in 0.7s
//sai_p4/instantiations/google:clos_stage_test                           PASSED in 0.5s
//sai_p4/instantiations/google:fabric_border_router_p4info_up_to_date_test PASSED in 0.0s
//sai_p4/instantiations/google:middleblock_p4info_up_to_date_test        PASSED in 0.0s
//sai_p4/instantiations/google:sai_nonstandard_platforms_build_test      PASSED in 0.0s
//sai_p4/instantiations/google:sai_nonstandard_platforms_cc_test         PASSED in 0.7s
//sai_p4/instantiations/google:sai_p4info_fetcher_test                   PASSED in 0.7s
//sai_p4/instantiations/google:sai_p4info_test                           PASSED in 1.1s
//sai_p4/instantiations/google:sai_pd_proto_test                         PASSED in 0.0s
//sai_p4/instantiations/google:sai_pd_util_test                          PASSED in 0.6s
//sai_p4/instantiations/google:tor_p4info_up_to_date_test                PASSED in 0.0s
//sai_p4/instantiations/google:union_p4info_up_to_date_test              PASSED in 0.1s
//sai_p4/instantiations/google:wbb_p4info_up_to_date_test                PASSED in 0.0s
//sai_p4/instantiations/google/test_tools:table_entry_generator_helper_test PASSED in 4.4s
//sai_p4/instantiations/google/test_tools:test_entries_test              PASSED in 0.9s
//sai_p4/instantiations/google/tests:p4_fuzzer_integration_test          PASSED in 4.1s
//sai_p4/tools:p4info_tools_test                                         PASSED in 0.8s
//sai_p4/tools:packetio_tools_test                                       PASSED in 0.8s
//tests:thinkit_gnmi_interface_util_tests                                PASSED in 1.1s
//tests/lib:p4info_helper_test                                           PASSED in 0.9s
//tests/lib:p4rt_fixed_table_programming_helper_test                     PASSED in 0.7s
//tests/lib:packet_generator_test                                        PASSED in 26.8s
//tests/lib:switch_test_setup_helpers_golden_test                        PASSED in 0.1s
//tests/lib:switch_test_setup_helpers_golden_test_runner                 PASSED in 0.1s
//tests/qos:gnmi_parsers_test                                            PASSED in 0.1s
//tests/qos:gnmi_parsers_test_runner                                     PASSED in 0.0s
//thinkit:bazel_test_environment_test                                    PASSED in 0.8s
//thinkit:generic_testbed_test                                           PASSED in 0.9s
//thinkit:mock_control_device_test                                       PASSED in 0.6s
//thinkit:mock_generic_testbed_test                                      PASSED in 0.7s
//thinkit:mock_mirror_testbed_test                                       PASSED in 0.6s
//thinkit:mock_ssh_client_test                                           PASSED in 0.0s
//thinkit:mock_switch_test                                               PASSED in 0.7s
//thinkit:mock_test_environment_test                                     PASSED in 0.1s
//thinkit:switch_test                                                    PASSED in 0.7s
//sai_p4/instantiations/google/tests:p4_constraints_integration_test     PASSED in 0.9s
Stats over 5 runs: max = 0.9s, min = 0.7s, avg = 0.8s, dev = 0.0s
//sai_p4/instantiations/google/test_tools:table_entry_generator_test     PASSED in 44.4s
Stats over 50 runs: max = 44.4s, min = 0.8s, avg = 5.6s, dev = 12.9s
Executed 211 out of 211 tests: 211 tests pass.


Configure Jest to use async hook from Nuxt residual logic (e.g. fetch())

Describe
During the development of a trivial fix, it showed up the need to setup Jest for Nuxt functionalities, such like $fetch.
This may be avoided in case of cleaning up leftover logic https://github.com/rancher/dashboard/issues/9711 and stick to Vue lifecycle hooks.
NOTE: This is not an issue about async, rather a lack of the hook implementation in the vue-test-utils.

Example case which may work in some cases (it did not for mine):
Thanks @torchiaf
https://github.com/rancher/dashboard/pull/10493/files#diff-6321b8c64f34671df66c6092445640b8e9da0a2b208c02190f96896f7ab0cb3bR46-R53
  it('should show apps catalog', async() => {
    const wrapper = shallowMount(ClusterTools, mountOptions);

    await (ClusterTools as any).fetch.call(wrapper.vm);
    const cards = wrapper.find('[data-testid^="cluster-tools-app"]');

    expect(cards.exists()).toBe(true);
  });

This issue may be solved with https://github.com/rancher/dashboard/issues/9711


prometheus-node-exporter: 1.6.0 -> 1.6.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/project/prometheus-node-exporter/versions.
meta.description for prometheus-node-exporter is: Prometheus exporter for machine metrics
meta.homepage for prometheus-node-exporter is: https://github.com/prometheus/node_exporter
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.6.1 with grep in /nix/store/f2bd6w68ac5qkqkj61dv4pi3c1x18z46-node_exporter-1.6.1
found 1.6.1 in filename of file in /nix/store/f2bd6w68ac5qkqkj61dv4pi3c1x18z46-node_exporter-1.6.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
prometheus-node-exporter

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/f2bd6w68ac5qkqkj61dv4pi3c1x18z46-node_exporter-1.6.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A prometheus-node-exporter https://github.com/r-ryantm/nixpkgs/archive/b15444007873527231090e09660bcd0dbefcfb6c.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/f2bd6w68ac5qkqkj61dv4pi3c1x18z46-node_exporter-1.6.1
ls -la /nix/store/f2bd6w68ac5qkqkj61dv4pi3c1x18z46-node_exporter-1.6.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

prometheus-node-exporter

Maintainer pings
cc @benley @fpletz @globin @Frostman for testing.

Result of nixpkgs-review pr 245020 run on aarch64-darwin 1

1 package built:

prometheus-node-exporter


Can't change log level using --v cli option

What version of descheduler are you using?
descheduler version:
v0.24.1
Which descheduler CLI options are you using?
          containers:
            - args:
              - --policy-config-file
              - /policy-dir/policy.yaml
              - --logging-format
              - json
              - --v
              - "4"

Please provide a copy of your descheduler policy config file
What k8s version are you using (kubectl version)?
kubectl version Output
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:48:33Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.14-eks-6d3986b", GitCommit:"8877a3e28d597e1184c15e4b5d543d5dc36b083b", GitTreeState:"clean", BuildDate:"2022-07-20T22:05:32Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

What did you do?

I was trying to get pod eviction logs of descheduler, so I increased the log level to 4. But that doesn't seem to work, there were only 3 log events in each descheduler pod logs.
We recently upgraded to descheduler v0.24 and started using --logging-format=json option. So I tried removing --logging-format=json from the cronjob and I started seeing correct logs as per level set in the cli option.
What did you expect to see?
Info level descheduler logs, in json format
What did you see instead?
Logs:

{"ts":1666025401338.376,"caller":"server/secure_serving.go:210","msg":"Serving securely on [::]:10258\n","v":0}
{"ts":1666025401338.4546,"caller":"dynamiccertificates/tlsconfig.go:240","msg":"Starting DynamicServingCertificateController","v":0}
{"ts":1666025405001.4187,"caller":"dynamiccertificates/tlsconfig.go:255","msg":"Shutting down DynamicServingCertificateController","v":0}

@damemi That doesn't work.
I changed the arguments like this:
            - args:
              - --policy-config-file
              - /policy-dir/policy.yaml
              - --logging-format
              - json
              - --v
              - "5"
              - --logtostderr=false

logs after the change:

$ k logs -n kube-system descheduler-cronjob-27768154-qgdg6
Flag --logtostderr has been deprecated, will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components
{"ts":1666089242007.3384,"caller":"server/secure_serving.go:210","msg":"Serving securely on [::]:10258\n","v":0}
{"ts":1666089242007.3962,"caller":"dynamiccertificates/tlsconfig.go:240","msg":"Starting DynamicServingCertificateController","v":0}
{"ts":1666089246026.032,"caller":"dynamiccertificates/tlsconfig.go:255","msg":"Shutting down DynamicServingCertificateController","v":0}

Thanks for confirming that @poojasj, there seems to be some issue with our logging as there have been several reports of odd behavior recently. We'll be looking into this and trying to fix it asap

I also loose eviction logs as soon as i switch to logging-format json in v0.26.0.