Formatted JSON failing tokenparse

Hi Folks,
Processing a minified version of a .json file the tokenparser does not fail, however, if the original file from the Customer comes in formatted,
Then:
https://github.com/juanjoDiaz/streamparser-json/blob/ef9a919957de8a4ef04037c44eb89067b1fbc365/packages/plainjs/src/tokenparser.ts#L328
throws the error
"stack": [
"Runtime.UnhandledPromiseRejection: Error: Error on Readable Stream for s3DropBucket Object acmerepsignature_sampleformatted.json.",
"Error Message: An error has stopped Content Parsing at record 0 for s3 object acmerepsignature_sampleformatted.json.",
    **"Error: Unexpected SEPARATOR (\"\\n\") in state VALUE ",**

    "    at process.<anonymous> (file:///var/runtime/index.mjs:1276:17)",
    "    at process.emit (node:events:517:28)",
    "    at emit (node:internal/process/promises:149:20)",
    "    at processPromiseRejections (node:internal/process/promises:278:11)",
    "    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"
]

I first thought there was a value in the json that contained an embedded newline, however, inspecting the file very closely that is not the case.
Editing to add params:
const jsonParser = new JSONParser( {
// numberBufferSize: 64,               //64, //0, //undefined, // set to 0 to don't buffer.
stringBufferSize: undefined,        //64, //0, //undefined,
separator: '\n',                               // separator between object. For example \n for nd-js.
paths: [ '$' ],                                  //ToDo: Possible data transform oppty
keepStack: false,
emitPartialTokens: false                // whether to emit tokens mid-parsing.
} )
Here's a short sample of the Original file:
[
{
id: '0051N000006sMsYQAU',
email<EMAIL_ADDRESS>first: 'Cheryl',
last: 'Basso',
photo: null,
Photo: null,
brandApprovedPhoto: null,
designations: null,
jobTitle: null,
phone: null,
registeredAdvisorTitle: null,
updatedAt: '2022-01-20T02:28:13.077Z'
},
{
id: '0051N000006sMzcQAE',
email<EMAIL_ADDRESS>first: 'Laura',
last: 'Dern',
photo: null,
Photo: null,
ApprovedPhoto: null,
designations: null,
jobTitle: null,
phone: null,
registeredAdvisorTitle: null,
updatedAt: '2022-01-20T02:28:13.077Z'
},
{
id: '0051N000006sNCiQAM',
email<EMAIL_ADDRESS>first: 'Brad',
last: 'Bean',
photo: null,
Photo: null,
ApprovedPhoto: null,
designations: null,
jobTitle: null,
phone: null,
registeredAdvisorTitle: null,
updatedAt: '2022-01-20T02:28:13.077Z'
}
]
Of note, it would be helpful if the error message included an index to the character/position within the json that causes the exception,
I've parsed the formatted version independently and it does not fail so hoping someone can provide some insight as to how to go about resolving this?
Having the Customer >>not<< format the file is not an option, just in case that was a thought,
Thanks,
KWL

@juanjoDiaz Thanks!
As noted in another issue thread, I'll set the option separator: "" and test that, although Im pretty sure I ran that way in earlier testing and ran into an issue with line-delimited json,
I'll let you know,
Thanks,
KWL

Hey @KWLandry-acoustic how was the testing?
All fixed with ""?

Hi Juanjo,
Apologies for not getting back sooner, the testing encompasses a fair few sets of different files/formats,
In any case, the first run I tried with that Customer’s specific files did not go well, I had to switch back to the line-delimited separator, ‘\n’,
But, wanted to get time to gather more specific info than just that outcome and so set it aside while I tested the other sets,
Will try to get you more specific notes later today,
Thanks,
Kip Landry
Kip W. Landry
WW Solutions Architect
[signature_1720021682]
Email: @.*** @.***>
     How Can Acoustic Help?

Acoustic Help Centerhttps://help.goacoustic.com/hc/en-us
Subscribe for Product Updates and Releaseshttps://acoustic.com/client-signup/
New Product Releases
Acoustic Support Portalhttps://support.goacoustic.com/
Product Status Pages for Outages or Reduced Servicehttps://status.goacoustic.com/
Insights Help Documentationhttps://help.goacoustic.com/hc/en-us/categories/12538307189657-Insights
Acoustic Support Chathttps://support.goacoustic.com/
Insights Academy Traininghttps://learn.goacoustic.com/lms/index.php?r=site/sso&sso_type=saml&id_course=479&utm_source=shareButton&utm_content=course_link
Acoustic Academyhttps://learn.goacoustic.com/learn
Developer - REST API Referencehttps://api2.silverpop.com/restdoc/
Developer - API Referencehttps://developer.goacoustic.com/acoustic-campaign/reference/xml-api-overview-1
Feedback and Product Ideashttps://ideas.goacoustic.com/
Developer - Postman Collectionhttps://developer.goacoustic.com/acoustic-campaign/reference/postman-collection
Developer – API Endpoints and credentialshttps://developer.goacoustic.com/acoustic-campaign/reference/getting-started-with-oauth
From: Juanjo Diaz @.>
Date: Sunday, April 28, 2024 at 2:59 PM
To: juanjoDiaz/streamparser-json @.>
Cc: Kip Landry @.>, Mention @.>
Subject: Re: [juanjoDiaz/streamparser-json] Formatted JSON failing tokenparse (Issue #42)
Hey @KWLandry-acoustichttps://github.com/KWLandry-acoustic how was the testing?
All fixed with ""?
—
Reply to this email directly, view it on GitHubhttps://github.com/juanjoDiaz/streamparser-json/issues/42#issuecomment-2081602108, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AYV2WAG2MCZG2CWI6XHHTCLY7VBJPAVCNFSM6AAAAABGPFW7KWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGYYDEMJQHA.
You are receiving this because you were mentioned.Message ID: @.***>
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain Acoustic proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

Ok, using the separator: '\n' returns the error noted above, for that Customer, for that file, formatted json.
Using the option separator: '' does not return the error for that specific Customer and file,
But does return the following error (below) for a different Customers' files (that have been aggregated by AWS Firehose into a file of lines of:
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
.....
(Which parses just fine when option separator: '\n' is used)
Error:
2024-04-29T18:31:49.346Z	42c98925-0adf-4711-b0e1-947723fcab3c	ERROR	Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError: Unexpected LEFT_BRACE ("{") in state SEPARATOR ","reason":{"errorType":"Error","errorMessage":"Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError: Unexpected LEFT_BRACE ("{") in state SEPARATOR ","stack":["Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error: Unexpected LEFT_BRACE ("{") in state SEPARATOR ","    at JSONParserTransform. (/tmp/tmp8x51hhcp/src/handlers/s3DropBucket.ts:703:31)","    at JSONParserTransform.emit (node:events:529:35)","    at emitErrorNT (node:internal/streams/destroy:151:8)","    at emitErrorCloseNT (node:internal/streams/destroy:116:3)","    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error: Unexpected LEFT_BRACE ("{") in state SEPARATOR ","    at process. (file:///var/runtime/index.mjs:1276:17)","    at process.emit (node:events:517:28)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:278:11)","    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"]}
And, of course, using the separator: ''  - returns only the first record (as noted in the docs but I totally missed it until working through this issue, would recommend pulling that little tidbit up, highlight it more so, in the Documentation)
Going to try specifying an array or separators, see where that gets me,
Thanks,
KWL

Oops, never mind, the separator option as an array is not there yet,
KWL

Hi @juanjoDiaz ,
I've tried every separator option and I can now process every Customer file successfully but still not a Formatted version, such as the original file I noted above.
Hopefully there's something in this latest round of messaging that is helpful,
"errorMessage": "Error on Readable Stream for s3DropBucket Object acme_repsignature_sampleformatted.json.\nError Message: An error has stopped Content Parsing at record 0 for s3 object acme_repsignature_sampleformatted.json. Separator is \n.
Error: Unexpected SEPARATOR ("\n") in state VALUE ",
"stack": [
"Error: Error on Readable Stream for s3DropBucket Object acme_repsignature_sampleformatted.json.",
"Error Message: An error has stopped Content Parsing at record 0 for s3 object acme_repsignature_sampleformatted.json. Separator is ",
".",
"Error: Unexpected SEPARATOR ("\n") in state VALUE ",
"    at JSONParserTransform. (/tmp/tmplji6xj_6/src/handlers/s3DropBucket.ts:718:31)",
"    at JSONParserTransform.emit (node:events:529:35)",
"    at emitErrorNT (node:internal/streams/destroy:151:8)",
"    at emitErrorCloseNT (node:internal/streams/destroy:116:3)",
"    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)"
]

Hi @KWLandry-acoustic ,

Ok, using the separator: '\n' returns the error noted above, for that Customer, for that file, formatted json.
Using the option separator: '' does not return the error for that specific Customer and file, But does return the following error (below) for a different Customers' files (that have been aggregated by AWS Firehose into a file of lines of:
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n

..... (Which parses just fine when option separator: '\n' is used)
Error:
2024-04-29T18:31:49.346Z	42c98925-0adf-4711-b0e1-947723fcab3c	ERROR	Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","reason":{"errorType":"Error","errorMessage":"Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","stack":["Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","    at JSONParserTransform.<anonymous> (/tmp/tmp8x51hhcp/src/handlers/s3DropBucket.ts:703:31)","    at JSONParserTransform.emit (node:events:529:35)","    at emitErrorNT (node:internal/streams/destroy:151:8)","    at emitErrorCloseNT (node:internal/streams/destroy:116:3)","    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","    at process.<anonymous> (file:///var/runtime/index.mjs:1276:17)","    at process.emit (node:events:517:28)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:278:11)","    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"]}

And, of course, using the separator: '' - returns only the first record (as noted in the docs but I totally missed it until working through this issue, would recommend pulling that little tidbit up, highlight it more so, in the Documentation)
Going to try specifying an array or separators, see where that gets me, Thanks, KWL

That errors seems to indicate that you set up a separator but the separator didn't came and a new object was inmediately before the previous.

Ok, using the separator: '\n' returns the error noted above, for that Customer, for that file, formatted json.
Using the option separator: '' does not return the error for that specific Customer and file, But does return the following error (below) for a different Customers' files (that have been aggregated by AWS Firehose into a file of lines of:
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n
{"data";"data","data""data","data":"data",....}\n

..... (Which parses just fine when option separator: '\n' is used)
Error:
2024-04-29T18:31:49.346Z	42c98925-0adf-4711-b0e1-947723fcab3c	ERROR	Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","reason":{"errorType":"Error","errorMessage":"Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.\nError: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","stack":["Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","    at JSONParserTransform.<anonymous> (/tmp/tmp8x51hhcp/src/handlers/s3DropBucket.ts:703:31)","    at JSONParserTransform.emit (node:events:529:35)","    at emitErrorNT (node:internal/streams/destroy:151:8)","    at emitErrorCloseNT (node:internal/streams/destroy:116:3)","    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: Error: Error on Readable Stream for s3DropBucket Object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error Message: An error has stopped Content Parsing at record 1 for s3 object acme_S3DropBucket_Aggregator-8-2024-04-29-18-27-15-020841bf-786e-3e7f-8227-4c843efa7822.json.","Error: Unexpected LEFT_BRACE (\"{\") in state SEPARATOR ","    at process.<anonymous> (file:///var/runtime/index.mjs:1276:17)","    at process.emit (node:events:517:28)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:278:11)","    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"]}

And, of course, using the separator: '' - returns only the first record (as noted in the docs but I totally missed it until working through this issue, would recommend pulling that little tidbit up, highlight it more so, in the Documentation)
Going to try specifying an array or separators, see where that gets me, Thanks, KWL

You data seems totally wrong as it's not valid JSON.
The first key-value pair of each object is separated by a semicolon and the second by nothing.
With a valid dataset it works just fine.
const { Readable } = require("stream");
const { JSONParser } = require("@streamparser/json-node");

const data = `{"data1":"data1","data2":"data2","data3":"data3"}\n
{"data1":"data1","data2":"data2","data3":"data3"}\n
{"data1":"data1","data2":"data2","data3":"data3"}\n
{"data1":"data1","data2":"data2","data3":"data3"}\n
{"data1":"data1","data2":"data2","data3":"data3"}\n`;

const inputStream = new Readable();
inputStream._read = () => {
  /* Do nothing */
};

const jsonparser = new JSONParser({ separator: '' });
inputStream.pipe(jsonparser);

jsonparser.on("data", (parsedElementInfo) => {
    if (parsedElementInfo.parent === undefined) {
        console.log(parsedElementInfo.value);
    }
});
jsonparser.on("error", err => console.error(err));

inputStream.push(data);
inputStream.push(null);

and with formatted data also works:
const data = `{
    "data1": "data1",
    "data2": "data2",
    "data3":"data3"
}\n{
    "data1": "data1",
    "data2": "data2",
    "data3":"data3"
}\n{
    "data1": "data1",
    "data2": "data2",
    "data3":"data3"
}\n{
    "data1": "data1",
    "data2": "data2",
    "data3":"data3"
}\n{
    "data1": "data1",
    "data2": "data2",
    "data3":"data3"
}\n`;

I still need to publish a fix for using \n with a formatted file.
It's ready but I need to cut the release.
If you want more help, I need concrete JSON examples and the exact settings that you use so I can replicate the issue.

version  0.0.21 fixes the issue when using a separator that's made of whitespaces and it's detected just as formatting.

Hi JuanJo,
Certainly, Let me know if you need more than this:
Using:
...
let s3ContentReadableStream = getS3StreamResult.Body as NodeJS.ReadableStream
...
s3ContentReadableStream = s3ContentReadableStream.pipe( jsonParser )
...
const jsonParser = new JSONParser( {
// numberBufferSize: 64,        //64, //0, //undefined, // set to 0 to don't buffer.
stringBufferSize: undefined,        //64, //0, //undefined,
//separator: '\n',               // separator between object. For example \n for nd-js.
//separator: '',               // separator between object. For example \n for nd-js.
separator: sep,               // separator between object. For example \n for nd-js.
paths: [ '$' ],              //ToDo: Possible data transform oppty
keepStack: false,
emitPartialTokens: false    // whether to emit tokens mid-parsing.
} )               //, { objectMode: true })
s3ContentReadableStream = s3ContentReadableStream.pipe( jsonParser )
...
OnData()
...
OnEnd()
....
I've attached a renamed file here, and here's an example of the failing formatted content as well,
acmerepsignature_sampleformatted.json.txt
I've attached a renamed file here, and here's an example of the failing formatted content as well,
[
{
id: '0051N000006sMsYQAU',
email<EMAIL_ADDRESS>first: 'Cheryl',
last: 'Basso',
photo: null,
Photo: null,
brandApprovedPhoto: null,
designations: null,
jobTitle: null,
phone: null,
registeredAdvisorTitle: null,
updatedAt: '2022-01-20T02:28:13.077Z'
},
{
id: '0051N000006sMzcQAE',
email<EMAIL_ADDRESS>first: 'Laura',
last: 'Dern',
photo: null,
Photo: null,
ApprovedPhoto: null,
designations: null,
jobTitle: null,
phone: null,
registeredAdvisorTitle: null,
updatedAt: '2022-01-20T02:28:13.077Z'
},
{
id: '0051N000006sNCiQAM',
email<EMAIL_ADDRESS>first: 'Brad',
last: 'Bean',
photo: null,
Photo: null,
ApprovedPhoto: null,
designations: null,
jobTitle: null,
phone: null,
registeredAdvisorTitle: null,
updatedAt: '2022-01-20T02:28:13.077Z'
}
]

FYI,
on:
_You data seems totally wrong as it's not valid JSON.
The first key-value pair of each object is separated by a semicolon and the second by nothing.
With a valid dataset it works just fine._
My Bad, I typed up an example quickly and simply missed providing the correct format, it is in fact,
{"data":"data","data":"data","data":"data",....}\n
{"data":"data","data":"data","data":"data",....}\n
{"data":"data","data":"data","data":"data",....}\n
{"data":"data","data":"data","data":"data",....}\n
{"data":"data","data":"data","data":"data",....}\n
As you ran in your example, however, I can confirm the output from AWS Firehose failed as noted, when the separator was not '\n' but works when it is set as '\n', which it is currently set as and processing that stream quite well.
Likely it could be transformed to work without requiring separator as '\n' but I simply did not have the time to run that down.

Re:
"and with formatted data also works: ..."
In early testing I didn't run into an issue with the files the Customer was sending, and when the problem came up, looking at the files they were providing I thought they looked different than previously.
So, that begs the thought/suspicion that I think they've changed the process that packages the files to be sent to us to a different formatter or different options with the previous formatter. Your formatted version looks different yet again than what they're providing,
So, I now suspect the issue has to do with formatting options or a Formatter with options that cause the issue.
In any case, we're asking them to not format the files at all. However, this might not fly as they like to have them formatted for spot checking before they send them to us, we'll see how that goes.
Thanks,
KWL

I've attached a renamed file here, and here's an example of the failing formatted content as well,
acmerepsignature_sampleformatted.json.txt
I've attached a renamed file here, and here's an example of the failing formatted content as well,

That file is a simple JSON object.
There are not multiple JSON separated by a separator of any kind.
I tried it with multiple configurations and it works for me.
I also tried chaining the file multiple times separated by one or many \n and it also works file with the empty separator.
So, I'm still unclear on what the problem is.

I now suspect the issue has to do with formatting options or a Formatter with options that cause the issue.
In any case, we're asking them to not format the files at all. However, this might not fly as they like to have them formatted for spot checking before they send them to us, we'll see how that goes.

As long as the formatting is based on whitespaces (whitespaces, tabs, new lines, etc.) it will work just fine.
As said, to help more I need a exact code sample with a exact JSON that produce the error.


Refactor to component approach

Short description of the changes

adds unmounting logic & general error handling (without tests)
refactor to component approach
TODO: Add tests for unmounting/errorhandling

Component approach makes the usage of the plugin easier:
Before: Using Plugin Model Approach
where the tracer is defined:
import Component1 from '<component location>';
import Component2 from '<component location>';

// other OpenTelemetry tracing setup code

const webTracerWithZone = new WebTracerProvider({
        scopeManager: new ZoneScopeManager(),
        plugins: [
             new ReactLoad([]),
        ],
    });

Component definitions:
class export Component1 extend React.Component { ...}

class export Component2 extend React.Component { ...}

Having to import all of the components requiring tracing and then manually adding them to the plugin constructor was considered to be to big of change on the application side for a plugin.
After: Component Approach
where the tracer is defined:
BaseOpenTelemetryComponent.setLogger(logger);
BaseOpenTelemetryComponent.setTracer("name", "version");

Component definitions:
class export Component1 extend BaseOpenTelemetryComponent { ...}

class export Component2 extend BaseOpenTelemetryComponent { ...}

This approach will automatically trace and component extending BaseOpenTelemetryComponent

Codecov Report

:exclamation: No coverage uploaded for pull request base (react-plugin@2258e60). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff               @@
##             react-plugin       #3   +/-   ##
===============================================
  Coverage                ?   93.62%
===============================================
  Files                   ?        3
  Lines                   ?      204
  Branches                ?       31
===============================================
  Hits                    ?      191
  Misses                  ?       13
  Partials                ?        0


⚠ Bofrost.fr has degraded performance

In a7b5154, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 980 ms

Resolved: Bofrost.fr performance has improved in 220a021 after 14 minutes.


Test stratospheric aerosol file (extinction) being uploaded on input4mips FTP

Tagging @durack1 @znichollscr
Strat aerosol test file uploaded for checking
I'm uploading one of my test file to the input4mips FTP for Paul to check as instructed by Zeb, and following instuctions on https://input4mips-validation.readthedocs.io/en/latest/how-to-guides/how-to-upload-to-ftp/. The dry run went well and the file is currently uploading. Let me know if there is any issue.
Outstanding questions
The two main outstanding questions for my datasets are:

Do you want one file per variable? This feels a bit messy but happy to catter to your preference! I have 8 variables in the aerosol optical property dataset, emission depends.

Do you want the emission dataset provided as a gridded (time, lon, lat, height) flux dataset rather than a a list of eruption with emission parameters? There is a limited number of eruptions so I'm unsure this makes sense/how the few modelling group modelling this would prefer the data (I can poll them! With UKESM we work from an eruption list). One other concern is that the core data is a mass of SO2 for each eruption. If I grid that as a flux and people regrid that to their model grid(lat/lon, height, not sure whether there would be a time concern), they should try to conserve the mass for each eruption. But that information would be a lot harder to track from a gridded flux file rather than from an eruption list.

Do you want one file per variable? This feels a bit messy but happy to catter to your preference! I have 8 variables in the aerosol optical property dataset, emission depends.

I'd go one variable per file. It just plays way nicer with the ESGF machinery that way.

2. Do you want the emission dataset provided as a gridded (time, lon, lat, height) flux dataset rather than a a list of eruption with emission parameters?

Trickier. I would do one of two things.
Option a) Make your dimension "emission_number" and then provide time, lat, lon and height as a function of emission number. Also provide "mass_injected" (or whatever you want to call it) as a function of emission_number. Just provide "mass_injected" as total mass, then the modelling groups can work out how they want to spread that over time and their grid. I guess this is relatively clear and easy to work with without making mistakes. My concern is that it would break the ESGF's standard data model, but I could be wrong about what the ESGF requires.
Option b) Use dimensions of ("time", "lat", "lon", "height"). Then you just make "emissions" a function of time, lat, lon and height. There'll be lots of zeros, but that's ok and you can drop out any timesteps with no emissions at all to keep the file size sensible. The emissions would then be a flux. If you do it this way, you have to be very careful with the bounds. You'll have to make sure that the bounds for all variables are sensibly specified (so people know how long the flux should last for i.e. what the total mass to be emitted is, the spatial extent of the flux and the height over which it should be injected). Having written this now, it sounds much more complicated and probably not the approach I would go for. However, if it's the only format that ESGF supports, we might have to take this road.

@thomasaubry which folder did you upload to on the FTP server?

Ok found it ftp://ftp.llnl.gov/incoming/uoe, but it doesn't seem we have any files in there - although we have 3 version subdirs v20240804-v20240806

I'd ping @vnaik60 on the below, balancing ideal single variable per file (makes it far easier for ESGF publication) to the easier to use (multiple variables per file)

Do you want one file per variable?

Do you want the emission dataset provided as a gridded (time, lon, lat, height) flux dataset

Yes, uoe! The data transfer failed this morning, I think because I had bad wifi. Went to campus to relaunch it so it's incoming again. Does the number of thread matter if it's a single file?

linking this discussion with https://github.com/PCMDI/input4MIPs_CVs/issues/9

Does the number of thread matter if it's a single file?

Nope, I'm not that clever :)

Do you want one file per variable?

If it plays well with ESGF, we will go with this. Are you also planning to provide optical properties mapped on to each model's spectral bands? If not, would you be providing some guidance on how this should be done so that there is some consistency across the models?

Do you want the emission dataset provided as a gridded (time, lon, lat, height) flux dataset

option a) that Zeb describes plus information on the vertical extent of the emission would be helpful.

Are you also planning to provide optical properties mapped on to each model's spectral bands? If not, would you be providing some guidance on how this should be done so that there is some consistency across the models?

The plan is to provide a single set of file, but to also provide a piece of codes that modellers can use to convert to their spectral bands. I expect that to be ready in August but I will first ensure that v0 of everything is on ESGF.

option a) that Zeb describes plus information on the vertical extent of the emission would be helpful

Sounds good, on it today so we will see quickly if it works or not.
@znichollscr no matter what network I am on, the upload times out (yesterday afternoon in ended up losing the connection when 78% of the file was uploaded). I'll try again today changing a couple setting on my laptop. Has anyone else run into this problem? the file is not gigantic so I'm surprised.

Has anyone else run into this problem? the file is not gigantic so I'm surprised.

Strange. It's not something I've hit, but I also have small files. How big is the file? If this keeps failing, we might have to go for plan b.

1.4Gb. Definitly means single variable is better too. It should be my biggest file as it's 4D (time, lat, height, wavelength).

@durack1 after some more debugging the emission file is there too (utsvolcemis_input4MIPs_emissions_CMIP_UOEXETER-CMIP-1-1-0_gn_17551020-20211231.nc). The emission file is the only one needed, for optical properties I've only uploaded one file with extinction. Once you have checked/send feedback as needed I will use that to make the other aerosol optical property files and upload.

@thomasaubry great! I am a little confused as to what I am trying to get, there are a couple of copies of files in that subdir, so if you list the filenames I am chasing, I'll be able to pull these directly.
It also might be easier if just the netcdf files are uploaded, trying to pore through these subdirs seems unnecessary - I can impose a directory structure once I have the file and valid metadata

That's probably my bad.
@thomasaubry when you upload, it's best to upload to a new directory every time, so Paul can just grab the entire directory.
For example, for my CR-CMIP-0-3-0 dataset, I did
input4mips-validation --password<EMAIL_ADDRESS>--ftp-dir-rel-to-root "cr-cmip-0-3-0-0"

Then we found the inevitable bug
input4mips-validation --password<EMAIL_ADDRESS>--ftp-dir-rel-to-root "cr-cmip-0-3-0-1"

Then the next bug
input4mips-validation --password<EMAIL_ADDRESS>--ftp-dir-rel-to-root "cr-cmip-0-3-0-2"

etc.
For you, might make sense to do
input4mips-validation --password<EMAIL_ADDRESS>--ftp-dir-rel-to-root "uoe-cmip-0-1-0-0"

input4mips-validation --password<EMAIL_ADDRESS>--ftp-dir-rel-to-root "uoe-cmip-0-1-0-1"

etc.

Thanks @znichollscr ! @durack1 I've put everything in uoexeter-CMIP-1-1-0 yesterday. Two files to look at:
ext_input4MIPs_aerosolOpticalProperties_CMIP_UOEXETER-CMIP-0-1-0_gnz_175001-202312.nc
utsvolcemis_input4MIPs_emissions_CMIP_UOEXETER-CMIP-1-1-0_gn_17551020-20211231.nc
Is it more clear now/do you have access? Please do not upload these to ESGF after. I might have misunderstood the workflow but I thought this was just to check metadata/formatting. Once I have your green light on this I will produce the other aerosol optical property files in the same way and add a couple corrections from feedback already received. This won't take time, I've only been slow getting used to the python codes/formatting practice.
I might also reset my version numbers, I think it makes sense that what goes on esgf first should be 0-0-1.

Is it more clear now/do you have access?

All clear to me

Please do not upload these to ESGF after. I might have misunderstood the workflow but I thought this was just to check metadata/formatting

Yep, I think we're all on the same page about this

I might also reset my version numbers, I think it makes sense that what goes on esgf first should be 0-0-1

To be honest, I'd suggest not doing this. The first version of our GHG files on ESGF is 0-3-0. I did this just in case anyone had heard/somehow been sent our 0-1-0 and 0-2-0 versions by other channels, to ensure there was zero chance of confusion. The version number is pretty arbitrary, so I'd just keep incrementing from whatever you're up to.

@thomasaubry @znichollscr yep, we're on the system - see /p/cscratch/durack1/ThomasAubry-volcEmissions/20240819
@thomasaubry the comments about versions etc that @znichollscr notes above is consistent with my thinking. There are many moving parts, and so my recommendation is that with any change (no matter how big, a unit issue, or small, a typo in netcdf global attributes) we just continue to increment upward and onward. That will mean that we can then keep a track of what changed and when, and then capture this, so when we have 100+ users of these datasets (highly likely in a couple of months), there is a clear source of information about what is the latest version and what issues were resolved in the previous versions. We've already hit this with the SOLARIS-HEPPA-CMIP-4-3 data (4.1 and 4.2 have already been released for folks to use), and we know there are some inconsistencies with metadata in 4.3, so will catch these very minor (and insignificant to users) tweaks with the 4.4 version which at the very minimum will extend the coverage temporally.
Does that make sense?

@durack1 makes perfect sense. Let me know once you've checked and I'll make the other optical properties files and upload the full package :)

see /p/cscratch/durack1/ThomasAubry-volcEmissions/20240819

Thanks. All look good to me

@vnaik60 did you want to take a peek? Or are we full steam ahead for @thomasaubry to mint the "final" prototype versions and get them in the ESGF publication queue?

@durack1 and @thomasaubry march ahead!  Issues, if any, can be captured once the files are available on ESGF for wider testing. Thank you so much for getting us to "v0"!

all uploading currently, accidentally in uoexeter 🤦♂ will go back to uoexeter-CMIP-x-x-x for next uploads. There are 7 files total, one for emission and 6 for optical properties.
I will believe we are at v0 when I see the files on ESGF, despite brilliant help I've been so slow with all the steps after the actual dataset production!

@thomasaubry exciting!
I can see 5 files currently, is that it, or should there be more?

Also we don't currently have an "aerosolOpticalProperties" dataset_category, which currently includes:
[
    "GHGConcentrations",
    "SSTsAndSeaIce",
    "aerosolProperties",
    "atmosphericState",
    "emissions",
    "landState",
    "ozone",
    "radiation",
    "solar",
    "surfaceAir",
    "surfaceFluxes"
]

So not sure this was intended?

Will pull these 5 across to the nimbus (evaluation machine), so that @znichollscr can take a peek directly

Will pull these 5 across to the nimbus (evaluation machine), so that @znichollscr can take a peek directly

@znichollscr's these are live now - ~/durack1/ThomasAubry-volcEmissions/20240822

So not sure this was intended?

Should probably be "aerosolProperties" I think ?

@znichollscr's these are live now - ~/durack1/ThomasAubry-volcEmissions/20240822

They pass the validator (usual caveats about the validators current limits).
@thomasaubry I think there's two outstanding questions:

did some of the files fail to upload? Paul only saw 5 on the server, but you said there should be 7?
should dataset_category be "aerosolProperties" rather than "aerosolOpticalProperties"?

Nearly there!

Hi everyone, awesome!

Missing files: yes, two missing (asy and ext). Currently uploading. Strange that it told me it uploaded all files when 2 were missing.
Dataset category: correct! I've fixed that in my script. Haven't changed it for the two files uploading but I will change it when I reupload everything.
I'm off today so if there are further issues I will be v slow reacting but I will pick them up tomorrow and I can reupload files with the correct dataset category then.

Thanks!

totally exciting and looking forward to seeing on ESGF! @thomasaubry at some point after the dataset goes live, let us know how you would like

to also provide a piece of codes that modellers can use to convert to their spectral bands

via github, zenodo, TT webside etc?

@thomasaubry we're making progress, I can now find 6 files, almost to your target!

@durack1 yes, I'm on holiday and have no good wifi so it's struggling! It says the 7th file (asy_*.nc) now exists in uoexeter but it looks like the transfer aborted after connection lost so to be sure I've reuploaded this file only in uoexeter_asy. I think safer to use this one. Is it all there?
@vnaik60 yes I think this will go on github. I don't have it yet but development ongoing, hope it's for early september. I've outputted an extensive range of wavelength so I hope that in a first instance modellers can run with wavelength closer to there spectral band midpoints. The code will enable to average optical properties over their specific spectral bands with weighting by solar/terrestrial spectrum.
In addition to the code I will have to link documentation and I think the best way will be Zenodo (?).

Hi @thomasaubry, we can't publish anyway until September 1st at the earliest so don't stress too much. It seems like the upload hasn't worked anyway so, a suggestion. When you're back from holiday (not before):

re-process the files with a dataset category of "aerosolProperties" rather than "aerosolOpticalProperties"
re-upload with a stable internet connection into a new folder on the FTP server

Hi @znicholls , all done under uoxeter-CMIP-1-1-2 :) Let me know if you spot any issue!

@thomasaubry excellent - I can see 7 files!

@znicholls these are on the usual place under 20240827.

Thanks mate. All look good to me. I did a test re-write to the DRS too and that all seemed happy so I think we're good for queuing for publication.

Yay that's all of them! Almost there. Looking forward to do some proper science/improvements and writting up manuscripts now.

@thomasaubry wonderful! @sashakames is back from vacation next week, so will drop these as first go in the queue, and hopefully we're live mid-next week!

Noting that #9 can be closed alongside this when these data are live, and our database/HTML pages are updated

Corrected a unit problem so I'm already uploading 1-1-3 on the ftp...

@thomasaubry ok great! I'll await clearance from you that it's uploaded and switchout 1.1.3 for the in place 1.1.2 (which is yet to be published, as we're waiting for @sashakames to be back from vacation next Tuesday)

@thomasaubry it looks like I have 7 new files, but these files are identified as 1-1-2 in the source_id (and filename, I have not checked within file metadata)? Did you want to change that?

Oh gosh, I forgot to change the root folder...I clearly need to call it a day! I'll try to reupload this evening or over the weekend but I suspect connection will be too bad. If so will just do Monday morning UK time. You can delete that 1-1-3 folder...sorry for the fuss!

@thomasaubry all good! I have no control over the files on the ftp.llnl.gov, I can only upload/create or download, not delete. No problem with waiting until Monday, it's a public holiday this side, so if we have data in place Monday night/Tues US PDT, then I can pull these down within a minute and have them back in the publication queue

Ok this time all files should be in "uoexeter-CMIP-1-1-3_v3". Note "v3", v2 had an issue because of partial transfer (and then couldn't overwrite).

And we're live - here woo hoo!

As UOEXETER-CMIP-1-1-3 is done, we need to update the database (and webpages) and then close out this issue - @znichollscr you got this?

Yep will pick up tomorrow in #120

Fixed by https://github.com/PCMDI/input4MIPs_CVs/pull/120, closing


Add a possibility to set a max execution time for cron job

Yesterday I discovered a Chef server that did not converge for a month, but not failures were shown in the report section of Chef Manage. The chef-client hang on the server and I configured that there may only run 1 chef-client process. So newer ones exited right away as they should.
But I had to kill the running chef-client process. After that I started it manually and it converged again perfectly. To make this process more resilient I would like to change my current cron job:
*/30 * * * * root /bin/sleep 526;  /usr/bin/chef-client --run-lock-timeout=0 > /dev/null 2>&1

To:
*/30 * * * * root /bin/sleep 526;  timeout -s 9 3540 /usr/bin/chef-client --run-lock-timeout=0 > /dev/null 2>&1

This way the chef-client run will be killed if it takes 59 minutes (normally it should be finished within a few minutes).
Unfortunately I can only specify daemon_options and no code that comes before the /usr/bin/chef-client command.

Commands can easily be overriden by cron resource. Say if we have an entry such that:
$ crontab -l
# Chef Name: foo
0 5 * * * root /bin/sleep 526;  /usr/bin/chef-client --run-lock-timeout=0 > /dev/null 2>&1

Then core utils(like timeout) or any other options could be added in this manner:
cron 'foo' do
  hour '5'
  minute '0'
  command 'root /bin/sleep 526;  timeout -s 9 3540 /usr/bin/chef-client --run-lock-timeout=0 > /dev/null 2>&1'
end

Chef-client Output:
INFO: cron[foo] updated crontab entry

    - update crontab entry for cron[foo]

Updated crontab entry:
$crontab -l
# Chef Name: foo
0 5 * * * root /bin/sleep 526;  timeout -s 9 3540 /usr/bin/chef-client --run-lock-timeout=0 > /dev/null 2>&1


Project Idea: Relaxer application

I want to contribute a relaxer application project in javascript.
This is a screenshot of the project.

Please assign me this issue under Gssoc'21 @ridsuteri @991rajat @harshgupta20

Similar project already exist


Make sure auct non names get processed correctly

Make sure names that have a non concept authority are treated like this.
For example
Lythrum flexuosum Lag. and Lythrum flexuosum auct. non Lag. are different names. In fact the auct non is a catch all for usages other than the publication by Lag.
The auct/sensu non syntax is often used in synonymies, see gbif/portal-feedback#567 for an apparently badly processed synonym in CoL
Test the name parser to see if we can recognize the sensu/auct part separate from the rest of the authorship

tricky. It is parsed "correctly" in the sense that the auct non part is stripped:
http://api.col.plus/parser/name?name=Lythrum flexuosum auct. non Lag.
The parser places it correctly in ParsedName.sensu as a taxonomic remark. But in the col Name class we do not have that but instead should be transferring it to the Taxon table... Needs more thinking: should we keep taxonomic remarks on the name class or move to the taxon table???

See https://github.com/Sp2000/colplus/issues/38 for a broader discussion about misapplied names and taxonomic remarks in names

Parsed correctly, applying misapplied name status is dealt with in #93


[deliver] fix iPad 12.9" 3rd gen screenshot identifier for App Store Connect API

🔑
Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
Resolves #16975
More context in this comment of mine: https://github.com/fastlane/fastlane/issues/16975#issuecomment-699516304
Description
It appears that since the App Store Connect API, the name of screenshots downloaded from App Store Connect of the iPad 12.9" 3rd gen was changed to %d_APP_IPAD_PRO_3GEN_129_%d.
Since there's no logic to handle that identifier out of the box (i.e. if you download the screenshots using deliver and try to re-upload them, it won't work unless you rename the iPad 12.9" 3rd gen screenshots according to the docs), this PR fixes that, maintaining the old name identifiers, but also adding support to IPAD_PRO_3GEN_129.
This PR also improves some copy and documentation in general, that helps people better understand this caveat of the iPad screenshots.
Testing Steps
This issue could be reproduced by downloading your screenshots from App Store Connect using deliver, and try re-uploading them. fastlane doesn't know that it has to upload those screenshots and assign them to the iPad 12.9" 3rd gen, so it will attempt to upload them as regular iPad 12.9" 2nd gen screenshots, which isn't accepted by Apple during review time (it could potentially get metadata rejected - I've witnessed this).
To test this branch, modify your Gemfile as:
gem "fastlane", :git => "https://github.com/rogerluan/fastlane.git", :branch => "fix-ipad-screenshot-naming"

I can't see what failed the build on CI above ☝ but what's failing locally is something related to the Google Play Store, which is also present in master 😬 not sure how to proceed

Does this request has been merged? I have faced with same issue.

Does this request has been merged? I have faced with same issue.

Not yet, @niyaoyao ! If you're looking for a temporary workaround, check my comment here https://github.com/fastlane/fastlane/issues/16975#issuecomment-699516304 - hope it helps you!
I believe this PR will be merged soon, though. 🙏

🚀

@joshdholtz could you take a look at this one? 🤗 It's been causing new issues to be open e.g. https://github.com/fastlane/fastlane/issues/18045

On it!

Ping @joshdholtz 🤗

Another friendly ping 🙈 @joshdholtz

@rogerluan could you fix the conflicts so we hopefully soon can get this out?

Done @Dinsen 💪

Friendly ping @joshdholtz 🤗

Ping @joshdholtz ? 🙈


Improve error handling for startup crashes

Show Harmony patching errors via a messagebox, so users can report detailed error logs

Implemented in v1.6


buffer names taken on completion

FAQ

[X] I have checked the FAQ and it didn't resolve my problem.

Issues

[X] I have checked existing issues and there are no open or closed issues with the same problem.

Neovim Version
v0.6.1
Minimal reproducible config
base config
Description
After completion menu popup, next few set of buffer names seems to be taken by the completion. So if I open a file (in buffer 1), do some work to trigger the completion, then open another file it will in buffer 3.
Steps to reproduce
Open a file (in buffer 1), do some work to trigger the completion, then open another file.
Expected behavior
Open second file in buffer 2.
Actual behavior
New file opened in buffer 3.
Additional context
No response

I don't think it's a bug. the nuffer number is sequential number. So I think we can't expect next bufnr.
For example, the LSP hover(with floating window) will make the same results, I think.
If I undetstand wrongly, please explain the problem more.

Okay, I was hoping that it's something that I can fix in the config. I usually just work with a few files that I can quickly switch to using alt+n, so this behavior kind of ruined it.


GHA: test Python 3.1, 3.2 and 3.4

Testing Python 3.3 and 3.4 with Ubuntu 18.04 doesn't work: no container is created for Ubuntu 18.04.


US76169 - Hide scroll buttons, and make d2l-scroll-wrapper focusable for keyboard navigation

Scroll to the d2l-scroll-wrapper, and use the arrow keys

Tested with focus-able elements within the table. The table is still able to scroll with the arrow keys as expected. Tested on Chrome.

@omsmith @Surekh?

So you can just scroll with left/right keys?

Yes. If the focus is within or on the d2l-scroll-wrapper, scrolling is possible with left/right keys

LGTM

LGTM!


Update for 0.11 & purs executable

I think we should hold out for PS 0.11.1 before making the npm release, since there are two fairly bad bugs that will be fixed in .1.
I thought I'd open this now to get review feedback, since there's a bunch of stuff changed here for the new purs executable.

Coverage remained the same at 100.0% when pulling fb2112f16b77dfeb3bf6e2a5589127afd9de648c on garyb:update-0.11 into bad9dfe73f31b2fa092da61bbfce48f61b31592e on purescript-contrib:master.

Coverage remained the same at 100.0% when pulling d923066d2e13d1409aa9a112f79e500717d8d4df on garyb:update-0.11 into bad9dfe73f31b2fa092da61bbfce48f61b31592e on purescript-contrib:master.

NOTE: Please do not publish the new version immediately after this PR is merged. A bit work remains to be done.

@shinnn no problem, I'll let you merge it / release when you're ready. We haven't quite made the 0.11.1 release yet anyway.

Coverage remained the same at 100.0% when pulling 921a85c96cf400c76da0dd98f67910bb161cb0bd on garyb:update-0.11 into bad9dfe73f31b2fa092da61bbfce48f61b31592e on purescript-contrib:master.

Coverage remained the same at 100.0% when pulling 921a85c96cf400c76da0dd98f67910bb161cb0bd on garyb:update-0.11 into bad9dfe73f31b2fa092da61bbfce48f61b31592e on purescript-contrib:master.

Coverage remained the same at 100.0% when pulling cb9bf24830639ffae374e0c8ca69046e1c0959cc on garyb:update-0.11 into bad9dfe73f31b2fa092da61bbfce48f61b31592e on purescript-contrib:master.

@shinnn Unless there are other changes you'd like me to make I think this is done now. As mentioned above I'll leave it for you to merge & release, as you mentioned there was some more work to do. Thanks!

Coverage remained the same at 100.0% when pulling 1ad962ba668fc8d7572204682022febc6b2b9673 on garyb:update-0.11 into bad9dfe73f31b2fa092da61bbfce48f61b31592e on purescript-contrib:master.

Published 0.11.1 to npm. Thanks @garyb and all reviewers.

No.

Could you elaborate?

Leaving 0.11.1 marked stable induces a lot of compatibility issue report, for example https://github.com/bodil/pulp/issues/264#issuecomment-290102078 and https://github.com/purescript/purescript/issues/2745#issuecomment-290113677, to the libraries and applications, and can hurry their authors.

Well it means that every user who installs via NPM (which is the recommended approach) can't use Pulp, Pursuit, etc. Yes, it puts pressure on maintainers to update their code, but people operate on different schedules. This is why we note in the release notes that users should exercise caution when updating to the latest compiler. I think we need to at least update the instructions on the website if this is going to be the default.

OK, 0.11.1 is now marked as prerelease. When people just run npm install purescript, it installs 0.10.7.

Thanks, and I'll make a note to update the website instructions.

Thanks for marking it as prerelease.
I'd just like to make sure it's on the record at this point that I really, really dislike any approach which involves breaking things for no reason other than to hurry other maintainers to bring their things up to date. Mainly because I think it's a very poor way of treating the community, to whom, to a large extent, we owe purescript's success. It's inconsiderate not only of new users, who now are faced with problems they are not able to diagnose, but also library and tool maintainers who start receiving avoidable bug reports from confused and annoyed users. I also think it will come across very poorly to library and tool maintainers for us to just assume off the bat that they need to be hurried, that is, that they won't just promptly update things anyway.
Moreover, even if we ignore the moral aspect, I doubt it's an effective approach. I can certainly say, as the principal maintainer of pulp nowadays, that if 0.11.1 had been left as the default version you get when you run npm install purescript it would have made absolutely no difference to the amount of time it would have taken me to get around to looking at updating pulp, and it might even have had the opposite effect.

@shinnn Please let's not delete comments. It's useful to keep discussions around for future reference.
If you disagree with what was said, let's please discuss it (offline if necessary).

The reason I deleted them is not disagreement. Some of them were abusive and annoying.

I'd certainly like to know what you considered abusive or annoying, but maybe we should discuss this offline.

I've closed the 0.11 milestone now. Pulp and Pursuit are up to date, so we should be able to release 0.11.3 as latest, along with Pulp 11.


Hitting secondary GitHub rate limits

When attempting to install to an organization which has a lot of repositories in it you can hit the secondary GitHub rate limits.
In my example I was trying to mass deploy to the JuliaCloud organization which has 17 repositories. Below is what I had ran, and below that the error message returned:
using MassInstallAction

workflow = MassInstallAction.Workflow("BlueStyleFormatter", "blue_style_formatter.yml" => read("formatter2.yml", String))
MassInstallAction.install(workflow, "JuliaCloud"; token=ENV["GITHUB_AUTH"], cc = ["mattBrzezinski"])

ErrorException("Error found in GitHub reponse:\n\tStatus Code: 403\n\tMessage: You have exceeded a secondary rate limit and have been temporarily blocked from content creation. Please retry your request again later.\n\tDocs URL: https://docs.github.com/overview/resources-in-the-rest-api#secondary-rate-limits\n\tErrors: ")

The way to work around this would be to introduce a sleep(1) (much as we have done with the CompatHelper.jl integration tests) and if that continues to arise issues introducing a retry setup.

Yeah, let's start with the sleep, and then we can think about a bounded retry loop with exponential backoff.


♻ refactor test structure

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

#89
#88
#87
#86  👈
#85
main

This stack of pull requests is managed by Graphite.

Merge activity

Feb 24, 9:32 AM EST: @mingi3314 started a stack merge that includes this pull request via Graphite.


intensities_raw in .hdf5 file is different from msms.txt in .zip file

Hello，I downloaded the information you provided .hdf5 file, which has a column of intensities_raw, their values appear to be between 0 and 1, which is similar to the msms.txt in the zip file downloaded from PRIDE website. What is the relationship between intensities in msms.txt file?

The hdf5 files contain the data we used for training/validation/testing. In general, we used the top 3 highest scoring spectra per peptide, modification, charge, mass analyzer and collision energy setting for this. The identification data was extracted from MaxQuant (msms.txt), but the intensities were extracted from the raw data itself, since the msms.txt files contain de-charged and de-isotoped intensities. So there is somewhat of a relationship between the intensities on most, but not all fragment peaks. The hdf5 files contain only straight y- and b-fragment ions (no neutral losses or other things) up to charge 3 - again some difference to the msms.txt as this contains fragment peaks with neutral losses. In addition, we used a score cutoff of 70 - if I recall correctly, so some peptide identifications you will find the msms.txt will not be present in the hdf5 files. The intensities in the hdf5 files are scaled to the most intense fragment, thus 0-1.


[Improvement] Refer to ASF projects by their correct names

What would you like to be improved?
ASF projects need to be referred to as "Apache Foo" not just "Foo", for instance, "kafka" on this create catalog page. Also note that being a name it needs to be capitalised as "Apache Kafka".

How should we improve?
Refer to the projects with the full name were they are first mentioned.

Where are you referring to that needs to be modified? Is it 1 or 2 in the picture below?

My understanding is that any mention of the ASF project name needs to be changed.


Chat tweaks

Chat input is rendered like plain text today. With this change, it becomes Markdown, same as the output except raw HTML tags are escaped. I think this is pretty nice, especially for code, but if we want to be more like ChatGPT we could treat it as plaintext but preserve breaks. (Edit: @wch suggests we make this configurable)
I also made a subtle but important change to the NotifyException, it now chains the underlying exception (raise x from y). We should do a sweep sometime to make sure we do this everywhere; it's so hard to debug without being able to trace back to the original exception.

The failing tests are real, at least some of them. Need to update the tests probably.

Currently on this branch, when rendering a code block from user input, long lines get cut off to the left and there is no way to scroll. For example:

I don't know if this is the best solution, but I found that if I turn off align-self: flex-end, then the code displays more how I would expect:

Thanks @wch, fixed by af655fb

Reviewed in person with Joe.


Add can make certain fields optional

Fields such as phone number could be left as optional as telegram is the preferred means of contacting people.

Yea I agree maybe can remove phone numbers since they are difficult to get hold of and most TAs would probably not have access to them anyways.


Feature/issue55 password security

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Document SmartNICs supported for offload

From the #users channel on Slack:

11:41  Hi all,
11:41  quick question
11:41  on which smart NICs is vRouter offload supported?
11:41  https://tungstenfabric.github.io/website/Tungsten-Fabric-Architecture.html#smartnic-vrouter
12:04  mellanox and neutronome I suppose. both companies worked on supporting theier products. but i don't know exact version where it works

There are two types of offload, full and partial. Docs should cover both. This will probably be in 5.1 and/or 5.1.1.

Migrated to Jira: https://jira.tungsten.io/browse/TFB-1400


upgrade to 0.9.0

Codecov Report

Merging #213 (0c37952) into main (2781bec) will increase coverage by 0.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #213      +/-   ##
==========================================
+ Coverage   59.16%   59.25%   +0.09%
==========================================
  Files          36       36
  Lines        2096     2101       +5
==========================================
+ Hits         1240     1245       +5
  Misses        856      856

Files
Coverage Δ

src/config.rs
94.00% <100.00%> (+0.66%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Data processing code for B1+ efficiency

In this PR, I will write all the processing/plotting code to generate the figures related to B1+ efficiency mentioned in #7.
The current data_processing.ipynb is a work in progress, and only generates for now a graph of the standard deviation of B1+ efficiency, which will soon be replaced by a graph of the inter-quartile range (IQR) across slices.

I just ran the notebook locally. This is a good start. I added an additional figure (which I'll refer to as Fig1 here) which I think is helpful for visual assessment of results:
Fig1

The SC mask does not span the same levels for all subjects, as you can see in the Fig2 above. This means that for the IQR plot (see below) isn't really a fair comparison.
Fig2

We should restrict the analysis to the same SC levels for all subjects. For this we will need a vertebral levels file.

I propose that we deal with the SC labels in another issue/PR and move forward with merging this one.

That PR was merged with the notebook that included the results and the figures. As I mentioned in https://github.com/spinal-cord-7t/coil-qc-code/issues/5, binaries and any notebook outputs should not be added to the git history. It will make cloning of the repository slower, and the repository will increase in size as new binaries are pushed to the repos.

That PR was merged with the notebook that included the results and the figures. As I mentioned in #5, binaries and any notebook outputs should not be added to the git history. It will make cloning of the repository slower, and the repository will increase in size as new binaries are pushed to the repos.

Fixed


Failure to retrieve information regarding successfuly created Account

Hi,
I am new to NEM Techology. For my project I am trying to user your blockchain.
After getting basic ideas clarified with nem2-cli, I have ventured into programming using SDKs.
Setup :
Ubuntu : 18.04
Nem2CLi : 0.12.1
Docker : 18.09.7 Build : 2d0083d
DOcker-Compose : 1.21.2 Build a133471
SDK Repo : Latest
Code-Snippet
const account = Account.createFromPrivateKey(privateKey, NetworkType.MIJIN_TEST);
const password = new Password(user.password);
const wallet = SimpleWallet.create(user.firstName, password, NetworkType.MIJIN_TEST);
const userAccount = wallet.open(password);
 console.log('Your new account address is:', userAccount.address.pretty(), ' its private key', userAccount.privateKey,
        'and its public key', userAccount.publicKey);

This snippet creates a user account and outputs all user information including Address.
Validation
Issue-1
I try to validate this user via nem2-cli giving the user address as argument.
$ nem2-cli account info --address XX_YY_ZZ
Account:        XX_YY_ZZ
Address:	XX_YY_ZZ
at height:	11018
PublicKey:<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
at height:	0
Importance:	0
at height:	0
Mosaics
43c2851137b55a1 90
Issue-2
Also, I check profile list with nem2-cli
$ nem2-cli profile list
Here only those accounts get displayed, which are created/generated by nem2-cli and not the accounts created programmatically by SDKs
Issue-3
When I try to fetch User infromation programtically too, get Public Address 0000.
What may have gone wrong ? Please suggest.
Many thanks for your time.

Expected behaviour, As newly created account doesn't have a 'valid' public key yet. Try transfer some assets see if the public key changed.
cli generated account stored in local env. List all account from server is not supported. expected behaviour

As documented in the Transfer transaction, I executed a transaction with a created mosaic "successfully". But still I do not see any Public Key :(

const mosaicId = '6e527b45794dd414';

        const transferTransaction = TransferTransaction.create(
            Deadline.create(),
            Address.createFromRawAddress(userAccount.address.pretty()),
           // [],
            [new Mosaic(new MosaicId(mosaicId), UInt64.fromUint(10000))],
           // [NetworkCurrencyMosaic.createRelative(10000)],
            PlainMessage.create('enjoy your ticket!'),
            NetworkType.MIJIN_TEST
        );

I am stuck at this situation, without getting public key
Here is the output from nem2-cli
nem2-cli account info --address  SA2FS3-XPLDL2-GKSDJV-IXBZLV-EQQHFM-DCVKKT-264R
Account:        SA2FS3-XPLDL2-GKSDJV-IXBZLV-EQQHFM-DCVKKT-264R
Address:	SA2FS3-XPLDL2-GKSDJV-IXBZLV-EQQHFM-DCVKKT-264R
at height:	11301
PublicKey:<PHONE_NUMBER>000000000000000000000000000000000000000000000000000000
at height:	0
Importance:	0
at height:	0
Mosaics
3a81bd6be96f9b8f:	100000
Here one can see enough assets assigned, but still Public Key is 0

Where are the programtically created/generated account information stored ? how can i validate ?

Thanks in advnace.

Have you tried call the rest endpoint directly. e.g. http://localhost:3000/account/{address} see if the public key is correct there?

Unfortunately not much luck there either
{"meta":{},"account":{"address":"9034596EEF58D7A32A434D5170E575242072B062AA953D7B91","addressHeight":[11301,0],"publicKey":"0000000000000000000000000000000000000000000000000000000000000000","publicKeyHeight":[0,0],"accountType":0,"linkedAccountKey":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=","activityBuckets":[],"mosaics":[{"id":[3916405647,981581163],"amount":[100000,0]}],"importance":[0,0],"importanceHeight":[0,0]}}

When the account first receives a transaction, its address is stored in the network. You can see the block where the address first appeared under addressHeight.
However, at this point you can use the public key as well to query transactions:

http://localhost:3000/account/<public_key>/transactions

When the account first sends a transaction, its public key is stored in the network. You can see the block where the public key first appeared under publicKeyHeight.

I think issue is the publicKey shows 0000....' so you can not even call the  account transaction endpoint. @hulekal123 have you got other accounts created? Are their public keys look the same? Or perhaps call the clear-all` script to refresh your local server (note that this will wipe out everything including your existed nemesis accounts).

Hi  @dgarcia360
For the following setup, i was looking for Public Key
{"meta":{},"account":{"address":"9034596EEF58D7A32A434D5170E575242072B062AA953D7B91","addressHeight":[11301,0],"publicKey":"0000000000000000000000000000000000000000000000000000000000000000","publicKeyHeight":[0,0],"accountType":0,"linkedAccountKey":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=","activityBuckets":[],"mosaics":[{"id":[3916405647,981581163],"amount":[100000,0]}],"importance":[0,0],"importanceHeight":[0,0]}}
and the http://localhost:3000/block/11301   (#11301 is the Addressheight) returns following o/p. Here too i see lot of 00000
{"meta":{"hash":"338C18F834D5FF7C2AEFD7AD0359541AD2036F16BC3D9FC77EAA4220EC74DD30","generationHash":"B93372CB292EDD3CD588EB01E78D73C600F150E38BD0967507F637A01B484AB2","totalFee":[0,0],"subCacheMerkleRoots":["10A8529ABC2B5B76F59408797D1891521A441D9DF4F6528DD7EA2EDFE02CBCC6","F43DE1D5A235696518B4DB39455AD0BC4BF5DAF7EB0DF6CBBADCD90CF9B5CEEB","07803B546544910D4CA59DA8406C5ED0CEFEDE0C5A5362F959F2FA93C5C904C3","0000000000000000000000000000000000000000000000000000000000000000","0000000000000000000000000000000000000000000000000000000000000000","0000000000000000000000000000000000000000000000000000000000000000","0000000000000000000000000000000000000000000000000000000000000000"],"numTransactions":1,"numStatements":1},"block":{"signature":"F1ACE8F3D602E8B06874DF954B8F08F2981F84FA16BF8760F6F05D24C7F48EC069844FB5CB8045EDE47B8747CFA06D1D1D2206F654FC6D4BC90E0043DD955E0D","signer":"269CEFE82A312E3C7A97E070D0408EAF6183A241D6B1210681A9473C6751B6AD","version":36867,"type":33091,"height":[11301,0],"timestamp":[983501284,24],"difficulty":[2606025390,10451],"feeMultiplier":0,"previousBlockHash":"4D9B751D44171BBE0190BD38257F572AC55C5E298133AF38DD23A796EC24399B","blockTransactionsHash":"100824361469624F76FA0A0018DC5D0E65C7966FD47E5123B977A39A81A38DC3","blockReceiptsHash":"5763957600328E45961CB06A49C153FC76BA7DDCD8A73536A164667FD386BB54","stateHash":"630087E5E2BA01C2A4EC6654A13B939891FA9386EB018351E1B0269FA0755CD3","beneficiary":"0000000000000000000000000000000000000000000000000000000000000000"}}

I tried clean-all method too.
UNfortunately I get public key still 0000.
ALso how can i retrieve private key of an account ? is there any reverse look up method ?
THanks

I tried on a clean VM with a fresh setup and with nem2-cli, following tasks to verify @hulekal123 observations
Tasks and Comments

creation of default profile, with generated private key => SUCCESS
generation of another account X => SUCCESS
($ nem2-cli account generate --network MIJIN_TEST -s --url http://localhost:3000 --profile X)
$ nem2-cli profile list  # lists both default and X Accounts with Public and Private keys => SUCCESS
$ nem2-cli account info --address default_ADDRESS  or  $ nem2-cli account info --profile default
=> SUCCESS with dispay of Mosaics, FAIL : but Public Key is 000000...
$ nem2-cli account info --address X_ADDRESS  or  $ nem2-cli account info --profile X => FAIL , Error 404
transfered some mosaics into X account => SUCCESS, generated hash DEF_X_HASH
nem2-cli transaction transfer --recipient X_ADDDRESS --message "payout of 10 xem" --mosaics @cat.currency::10000000
nem2-cli transaction info --hash  DEF_X_HASH  #no useful Address or Keys of either default or X account
$ nem2-cli account info --address X_ADDRESS  or  $ nem2-cli account info --profile X => SUCCESS with dispay of Mosaics, FAIL : but Public Key is 000000...
tested on browser following Endpoints
a) http://localhost:3000/account/{PublicKey}/ or http://localhost:3000/account/{PublicKey}/transactions => Public Key always displayed 0000
b) http://localhost:3000/block/{addressHeight} or http://localhost:3000/block/{publicKeyHeight} => Public Key either not mentioned or always displayed 0000

Conclusion:
So I guess the Public Key is always displayed to be 0000 :-/ However, $ nem2-cli profile list always displays both Public and Private Key
Question
After a successful transfer transaction of Mosaics from Default account to another account X, a Hash and SIgner information are generated. But how can one confirm the Sender and Recipient of transactions ?
Technically how can prove participation of certain account holders without having Public Key or Address displayed ?


Add new console command "cache:clear:all" using shopware CacheClearer component

1. Why is this change necessary?
Add cache:clear:all automating clear all caches and pools
2. What does this change do, exactly?
Use Shopware cache clear to cleaning the caches and pools
5. Checklist

[ ] I have rebased my changes to remove merge conflicts
[ ] I have written tests and verified that they fail without my change
[ ] I have created a changelog file with all necessary information about my changes
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[ ] I have read the contribution requirements and fulfill them.

@AydinHassan sorry but I accidentally closed another PR


Include invocations to retrieve Default TimeZone in jdk-unsafe-1.8 ruleset

While methods that implicitly use the default system timezone are forbidden in the jdk-unsafe-1.8 ruleset (such as java.time.LocalDateTime#now()), it is still possible to explicitly use the default time zone:
LocalDateTime.now(Clock.systemDefaultZone());
LocalDateTime.now(ZoneId.systemDefault());
LocalDateTime.now(TimeZone.getDefault().toZoneId());

In my opinion the following invocations should also be forbidden under the Uses default time zone rules of the jdk-unsafe-1.8 ruleset:
java.time.Clock#systemDefaultZone()
java.time.ZoneId#systemDefault()
java.util.TimeZone#getDefault()

You can also by default use the default charset or default locale. That's explicitely wanted because the usage is explicit.
The idea behind forbiddenapis is to prevent people from accidentally using defaults. If they are explicit, all is fine.

Ah okay then. My assumption was that explicit usage was forbidden too.
In that case I can just create custom signatures to enforce the rules that make sense for our projects.
Thank you for your time.


[WEB] Space-bar on Text-Field in SrollWidget  don't work

When you wrap a test field in a Sroll Widget don't work, but scrolldown instead
Steps to Reproduce

Run flutter create test.
Wrap A text Field in a scroll widget
voilà space bar don't work !!!

Expected results:
Like here in normal textfield we expect that spacebar work
https://user-images.githubusercontent.com/55983266/112535337-da3d1d00-8db4-11eb-807f-5b1c5c409e78.mp4
Actual results:
but when you add a srollwidget to it, it doesn't work like here
https://user-images.githubusercontent.com/55983266/112535918-7ff08c00-8db5-11eb-8515-41d6ac588378.mp4
And when you have multiple item in thats scrollview it scroll down instead, like here in production app
https://user-images.githubusercontent.com/55983266/112537598-8c75e400-8db7-11eb-9b4a-2378815cfde3.mp4

Logs

flutter doctor -v
[✓] Flutter (Channel master, 2.1.0-13.0.pre.243, on Microsoft Windows [Version 10.0.19041.804], locale en-GB)
    • Flutter version 2.1.0-13.0.pre.243 at C:\flutter
    • Framework revision 25571a59b0 (23 hours ago), 2021-03-24 14:02:03 -0700
    • Engine revision d59a01d2d3
    • Dart version 2.13.0 (build 2.13.0-150.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at C:\sdk
    • Platform android-30, build-tools 30.0.0
    • ANDROID_HOME = C:\sdk
    • ANDROID_SDK_ROOT = C:\sdk
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Users\Acer\AppData\Local\Google\Chrome\Application\chrome.exe

[✓] Visual Studio - develop for Windows (Visual Studio Community 2019 16.6.0)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.6.30114.105
    • Windows 10 SDK version 10.0.18362.0

[✓] Android Studio (version 4.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 48.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[✓] VS Code, 32-bit edition (version 1.54.3)
    • VS Code at C:\Program Files (x86)\Microsoft VS Code
    • Flutter extension version 3.20.0

[✓] Connected device (3 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.19041.804]
    • Chrome (web)      • chrome  • web-javascript • Google Chrome 89.0.4389.90
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 89.0.774.45

• No issues found!

another mean: space button works as "scroller" so we can't type " " with space button

Yeah exactly, sorry for my bad non native English

@lucdotdev
Tried a sample app by wrapping TextField in ListView, but was able to use scrollbar properly without any issues. Can you provide a reproducible use case which shows the issue ? Also, what does scroll widget mean ? SingleChildScrollView, ListView or any other scrollable widget ?
Thanks.

@darshankawar probably using SingleChildScrollView and I have this issue too

In SingleChildScrollView an ListView as Well.
And the issues was not with scrollbar but with adding space (" ") in that textfield,
You can repoduce with that code :

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);
  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: SingleChildScrollView(child: TextField()),
      ),
    );
  }
}

And then flutter run -d chrome;
this issues found in both canvasKit and html;

Occasionally the " ", work when you press the spacebar on keyboard quickly after taping some letter. in most time it doesn't work

@lucdotdev
I've not been able to replicate the issue using latest master. Ran your sample code and spacebar from keyboard works properly without any issues.
https://user-images.githubusercontent.com/67046386/112626919-e5488980-8e56-11eb-8c38-d80bc35d0c5d.mov

flutter doctor -v
[✓] Flutter (Channel master, 2.1.0-13.0.pre.287, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.1.0-13.0.pre.287 at
      /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision bc2c7954db (8 hours ago), 2021-03-25 20:55:16 -0700
    • Engine revision b5e15d055d
    • Dart version 2.13.0 (build 2.13.0-162.0.dev)

[!] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin
        code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see
      https://guides.cocoapods.org/using/getting-started.html#installation for
      instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.54.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.20.0

[✓] Connected device (3 available)
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729 • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                               •
      web-javascript • Google Chrome 89.0.4389.90

! Doctor found issues in 1 category.

That’s true i can’t reproduce that’s anymore on the latest version ! Maybe fixed . Thanks for maintaining that great tool i am gona to close this !


Unclean exit when duplicate resource names

If you accidentally create two LWRP's with the same name, you get a cryptic message that is hard to troubleshoot.
It would be nice if the LWRP could detect the duplicate name, and error gracefully.
cron_d "foo" do
  minute '0'
  hour '1'
  command 'echo foo'
end

cron_d "foo" do
  minute '0'
  hour '1'
  command 'echo foo'
end

           ================================================================================
           Error executing action `create` on resource 'template[/etc/cron.d/compresss_kafka_controller_log]'
           ================================================================================

           Chef::Exceptions::ChecksumMismatch
           ----------------------------------
           Checksum on resource (cb551a) does not match checksum on content (9a41a2)

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/cron/providers/d.rb

            36:   t = template "/etc/cron.d/#{new_resource.name}" do
            37:     cookbook new_resource.cookbook
            38:     source 'cron.d.erb'

            40:     variables(
            41:                 :name => new_resource.name,
            42:                 :predefined_value => new_resource.predefined_value,
            43:                 :minute => new_resource.minute,
            44:                 :hour => new_resource.hour,
            45:                 :day => new_resource.day,
            46:                 :month => new_resource.month,
            47:                 :weekday => new_resource.weekday,
            48:                 :command => new_resource.command,
            49:                 :user => new_resource.user,
            50:                 :mailto => new_resource.mailto,
            51:                 :path => new_resource.path,
            52:                 :home => new_resource.home,
            53:                 :shell => new_resource.shell,
            54:                 :comment => new_resource.comment,
            55:                 :environment => new_resource.environment
            56:       )
            57:     action :create
            58:   end
            59:   new_resource.updated_by_last_action(t.updated_by_last_action?)

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cookbooks/cron/providers/d.rb:36:in `block in class_from_file'

           template("/etc/cron.d/compresss_kafka_controller_log") do
             action [:create]
             updated true
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             path "/etc/cron.d/compresss_kafka_controller_log"
             backup 5
             atomic_update true
             source "cron.d.erb"
             cookbook "cron"
             variables {:name=>"compresss_kafka_controller_log", :predefined_value=>nil, :minute=>"6", :hour=>"1", :day=>"*", :month=>"*", :weekday=>"*", :command=>"/usr/bin/find /var/log/kafka -type f -name \"kafka-controller.log.*.tar.gz\" -mtime +30 -exec /bin/rm {} \\;", :user=>"root", :mailto=>nil, :path=>nil, :home=>nil, :shell=>nil, :comment=>nil, :environment=>{}}
             declared_type :template
             cookbook_name :"ND-Kafka"
             mode "0644"
             checksum "cb551a64fc961957b39ad6f2c5fbaabe1bfecb9de8955c95dd9195764058b7fe"
             owner "root"

           end

       Running handlers:
       [2015-06-01T22:52:36+00:00] ERROR: Running exception handlers
       Running handlers complete

       [2015-06-01T22:52:36+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       Chef Client failed. 4 resources updated in 1.078846308 seconds
       [2015-06-01T22:52:36+00:00] ERROR: template[/etc/cron.d/compresss_kafka_controller_log] (/tmp/kitchen/cookbooks/cron/providers/d.rb line 36) had an error: Chef::Exceptions::ChecksumMismatch: Checksum on resource (cb551a) does not match checksum on content (9a41a2)
       [2015-06-01T22:52:36+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

This should no longer be an issue with the latest cookbook now that we enable the inline resources in chef 11+.  Feel free to open it back up if you still see the problem though.


[ListItemText] Update classes type definitions

[x] I have followed (at least) the PR section of the contributing guide.

@davjo664 It's a great first pull request on Material-UI 👌🏻. Thank you for working on it!


Optimize Mastodon profile

DDEV’s Mastodon account could be improved as a consistent presence and marketing tool:

Separate DDEV’s organizational identity from Randy’s personal one.
Use the profile description and links carefully and authoritatively.
Clarify the purpose of the account and use it accordingly.

TODO

[ ] use the attached DDEV logo for the account avatar instead of Randy’s headshot
[ ] update the profile description to remove a redundant leading DDEV and link to GitHub that serves a narrower audience than the homepage, and clarify what kind of tool it is; suggestion:Open-source tool for local PHP and Node.js development environments.

[ ] add a link to https://ddev.com via Preferences → Appearance → Profile metadata

✅ it should appear as verified since the homepage includes a link to the Mastodon profile with rel="me"

[ ] optionally establish a personal account that can follow and promote DDEV, and use the DDEV account strictly for official announcements, following active community members, and liking/boosting directly-relevant toots; let Randy (or others) add personal comments and humanity beyond DDEV’s organizational communication

Thanks, this is important, and this is an important time to do it.

I've done all these, thanks so much for the prod. Please check it out and close when approved.

Also, the password to the ddev account is in shared 1Password.

Looks good! Is there a reason you kept ddev.com at the end there? It seems redundant given the now-prominent link.

No reason. I was going through things in order :) Didn't know the metadata was still to come :)


Pull a "static" image, build the latest code on top

Closes #38
This should have multiple advantages:

We should still run quite quickly. Hopefully less than 10s
Now, if the action runs on git ref X, it means the code underneaath really is the code that ref X points to is used.

I apologise if you were waiting for my approval. Everything looks good to me so far. I think we can merge and then proceed according to the plan in #36.

I apologise if you were waiting for my approval

Nah, don't worry, it's just that some weeks I have a lot of time on my hands, and some weeks I don't :)


"Dist" and "Prob" versions

Reading the paper (section 2.2), I see the two difference functions

"Dist" (Euclidean distance between the embeddings) and
"Prob" (difference in probability of target token)

However, looking at the code, I see the options for the --metric flag are

dist (which subtracts and takes norm to get Euclidean distance), https://github.com/LividWo/Perturbed-Masking/blob/f57859261a7daa24de6310ad6ea8cce977d52fde/dependency/get_matrix_for_dep_probe.py#L78-L79 and
cos (which gets cosine similarity) https://github.com/LividWo/Perturbed-Masking/blob/f57859261a7daa24de6310ad6ea8cce977d52fde/dependency/get_matrix_for_dep_probe.py#L80-L82

Is the metric cos the "Prob" of the paper?

Hi
No, it's not.  But won't be difficult to implement by yourself.

What did you use to get the result in the paper?

right. you need to obtain the probability rather than the embedding.


fix(logs): add correct url to function logs

🎉 Thanks for submitting a pull request! 🎉
Summary
Fixes function log's url
on netlify deploy --build return the wrong url for functions

For us to review and ship your PR efficiently, please perform the following steps:

[ ] Open a bug/issue before writing your code 🧑💻. This ensures we can discuss the changes and get feedback from everyone that should be involved.

No. I asked Hrishikesh in Slack.

[x] Read the contribution guidelines 📖. This ensures your code follows our style guide and
passes our tests.
[ ] Update or add tests (if any source code was changed or added) 🧪
[ ] Update or add documentation (if features were changed or added) 📝
[ ] Make sure the status checks below are successful ✅

A picture of a cute animal (not mandatory, but encouraged)

@zmzlois can you run npm run format:fix:prettier and push it up? Just tried it myself but I can't push to your fork

@zmzlois can you run npm run format:fix:prettier and push it up? Just tried it myself but I can't push to your fork

ran ! @TylerBarnes 🤞🏻


Update redisclient to 3.41

Updates net.debasishg:redisclient from 3.20 to 3.41.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "net.debasishg", artifactId = "redisclient" } ]

labels: library-update

Superseded by #131.


Update ruff config & CODEOWNERS

Description

Found while working on #213 that as CODEOWNERS is currently defined we are required to have approvals from both @conda/conda-dot-org AND @conda/communications teams. We actually want it to be one OR the other.
Also correcting pyproject.toml's ruff config based on my IDE lints.

[ ] If I have added a new page to learn/ or community/, I have added it to the corresponding _sidebar.json file.

Hm actually it looks like specifying CODEOWNERS as:
https://github.com/conda/conda-dot-org/blob/e5d8ab8453cb6014e896a78c92e90e27db61be69/.github/CODEOWNERS#L3-L4
means the latter overwrites the former, so its not even an AND operation


⚠ Bofrost.at has degraded performance

In 1a74c71, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1229 ms

Resolved: Bofrost.at performance has improved in a6fb464.


Unhandled failure: Connection has been closed, response already set (status=200)

version: 2.0.1.RELEASE
[reactor-http-server-epoll-8] [ERROR] [org.apache.commons.logging.impl.Jdk14Logger:99] - Unhandled failure: Connection has been closed, response already set (status=200)
Why is this a problem？？？

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

MESSAGE
Unhandled failure: Connection has been closed, response already set (status=200)
STACKTRACE(most recent call first)
org.springframework.web.server.adapter.HttpWebHandlerAdapter in handleFailure at line 213
org.springframework.web.server.adapter.HttpWebHandlerAdapter in lambda$handle$0 at line 185
reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber in onError at line 88
reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber in onError at line 100
reactor.core.publisher.Operators in error at line 178
reactor.core.publisher.MonoError in subscribe at line 53
reactor.core.publisher.Mono in subscribe at line 3080
reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber in onError at line 97

Can you provide a complete, minimal, verifiable sample that reproduces the problem?  It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

me too!
Unhandled failure: Connection has been closed, response already set (status=200)


Updates to Community Partner Duplication Management

Per Client Feedback:
Say the keyword = “Eastern”  , if we click the  Next Button more than once, the list  is getting appended each time instead of refresh.
Searching a community partner that is already existed, like “AARP” click next button. Now change the value to any name that is not in the community partners data. The search function is not working.
Could we move this list towards the left. It looks like it is centered.
Could we add a table around the partners
Don’t bold the community partners.
Change the text to: The following organizations are already registered as UNO community partners. Please ensure that the community partner that you intent to register is not in this list.
The community partner report link is not working (time out error).
Either way - remove the link and the language: if yes, the community partner already exists! Click on the button below to access it.
Change the language to: If you do not find the community partner in the list, please register the community partner.
Could we change it - in the instance that the search is not providing a list to -
This community organization is currently not listed as a UNO community partner in this database. Please click on ‘Register the Community Partner’ and proceed with the registration.
Change the Register Community Partner button to ‘Register the Community Partner’

@pallavi0902 as QA tester

QA test passed for everything except "remove the link and adding the the language: if yes, the community partner already exists! Click on the button below to access it." because under discussion with clients.
https://drive.google.com/file/d/12VUS0dOIECwaFQ_WPxWVnkApg3PFqfIM/view?usp=sharing


Migrate away from Rawgit for docs

Rawgit seems to be shutting down, as can be seen when you click the "source" button in the documentation. Maybe we could move it under unicredit.github.io/neo/?

I'm a lousy web dev but I'd be happy to give a migration my best shot, if desired.

The problem with the source button was just that the links were incorrect, I have regenerated the documentation (or maybe rawgit rewrites the links to github?).
I'd like to host it elsewhere, but I cannot find a way to handle with github pages both the README and the API page at the same time

I found the html pages are hosted anyway, so I added a link there. But it seems that GH pages are not regenerating for some reason :-(


nextcloud: 13.0.0 -> 13.0.1

Motivation for this change
Update Yeay!
As usual, I updated, tested on my server and now PR.
I usually make a PR against master but it seems to be lagging behind the release, has there  been a change of policy? I'm not very connected.
Cheers
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

We already have this version in master. I backported it to stable as well


Update to Swift 2.0

Hi, I'm getting errors in Xcode 7. Do you have plans to update to Swift 2.0?
Thanks,
Kris

Hi @krishaamer. yes, it'll be updated for 2.0. thanks for the issue.


2 GPU settings for Llama2_7b is not working,  per XPU-SMI device 0 @ 99% and device 1 @ 0% during execution --resolved

I followed the steps from this github link -- https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/GPU/Deepspeed-AutoTP/README.md and attempted to verify 2 GPU inference runs on these Token Combinations

Initial run using default script with sym-int4 and 32 tokens

Note:  This run just used one GPU as the world_size is 1 here.

Also tried with sym int4 , sym int8 , fp8 , fp16 on token sizes 2048x128 , 2048x 256.
The general observation using XPU-SMI is shown below,

Notes:

Top is xpu-smi dump on device 0 and bottom portion is for xpu-smi dump on device 1
Notice device 0 is showing 99% GPU utilization vs device 1 showing close to 0% utilization

Kindly advise if there is any intermediate step to apply to achieve expected 2 GPU processes.
Please note that I am running the inference on this system  -->NF5468-M6 with 8x Intel Flex GPU 170
Full system spec details are available here -> https://wiki.ith.intel.com/display/MediaWiki/Flex-170x8+(Inspur+-+ICX)+Qualification

Synced with @gbertulf offline, this problem is caused by a bash grammar error in the starting script. After the fix, both devices are busy, but there still exists this error:

Issue is resolved. Closing this ticket. Thank you team for your help.