Update version

@lilleyse and @pjcozzi could you look at this when you get a chance? This refactors any glTF 0.8 ->1.0 from gltfDefaults out into a global stage called updateVersion. (This is now in #223) This stage also includes tools for converting from glTF 1.0 -> 2.0.
but lib/Pipeline.js still targets 1.0 by default. This will be changed once glTF 1.1 is fully released.
See https://github.com/KhronosGroup/glTF/issues/605 for the full list of changes from glTF 1.0 -> 1.1
@lexaknyazev, if you have time I'd appreciate your input here as well.

TODO

[x] Test new stage with all glTF sample models

@lasalvavida have you tested this with all the sample models?

@lasalvavida have you tested this with all the sample models?

Not yet.

@lasalvavida have you tested this with all the sample models?
Not yet.

Added to the tasklist at the top of this PR.  Please add any other items there needed before we should merge this.  Updating the entire pipeline to 1.1 can be a separate PR if you prefer.

@lexaknyazev if you are able to look at this, the two files to review are:

updateVersion.js
updateVersionSpec.js - tests

Good start, thanks @lasalvavida!

This is updated from the discussion above.

Great, will aim to review tomorrow morning.

Do we handle this one from https://github.com/KhronosGroup/glTF/issues/605:

Attribute parameters can't have a value defined in the technique or parameter, #563 (comment).

I think we should strip the value and warn in this case.

Is this taken into account?

technique.parameter.count can only be defined when the semantic is JOINTMATRIX or an application-specific semantic is used. It can never be defined for attribute parameters; only uniforms, d2f6945

Is this taken into account?

technique.parameter.semantic is required when the parameter is an attribute, 28e113d

In practice, it is likely to be rare so no need to over invest here.
We could either (1) warning or (2) make a best attempt to map attribute names to semantics and still warn.

I reviewed https://github.com/KhronosGroup/glTF/issues/605 and that is all my comments about how this maps to the 1.1 spec.  Please keep an eye on 1.1 issues in the glTF repo for any last-minute updates.
I'll also take a quick look at this code now.

@lasalvavida just those comments.  What is your plan from here?
Perhaps get this fully into shape, but then do not merge until we have the samples converted and Cesium loader so that we know the full stack works together?

I'll go through all of these as soon as possible.
Currently my plan is to integrate a bundled version of gltf-pipeline that contains parseModelMaterialsCommon, addDefaults, and updateVersion into mainline Cesium with targetVersion set to 1.0. This should make testing for 1.1 support pretty simple (just set targetVersion to 1.1 and see what breaks).

Currently my plan is to merge this into master here and integrate a bundled version of gltf-pipeline that contains parseModelMaterialsCommon, addDefaults, and updateVersion into mainline Cesium with targetVersion set to 1.0. This should make testing for 1.1 support pretty simple (just set targetVersion to 1.1 and see what breaks).

Sounds good.

technique.parameter.count can only be defined when the semantic is JOINTMATRIX or an application-specific semantic is used. It can never be defined for attribute parameters; only uniforms, d2f6945

Attribute parameters can't have a value defined in the technique or parameter, #563 (comment).

These are added.

technique.parameter.semantic is required when the parameter is an attribute, 28e113d

Properties in technique.parameters must be defined in technique.uniforms or technique.attributes,
#563 (comment).

I don't see a reasonable way to resolve these and it isn't an issue I ran into while upgrading the existing sample models. I think they're best left to the validator.
This is ready for review. I've tested this pretty extensively on the Cesium side and it seems to work well.

@pjcozzi

technique.parameter.semantic is required when the parameter is an attribute, 28e113d
Properties in technique.parameters must be defined in technique.uniforms or technique.attributes,
#563 (comment).

I don't see a reasonable way to resolve these and it isn't an issue I ran into while upgrading the existing sample models. I think they're best left to the validator.

Right, gltf-pipeline can't resolve these; however, it should still warn that it knows it is not outputting valid glTF 2.0 because it can't; otherwise, we may get bug reports on how gltf-pipeline output doesn't pass validation and it will be harder to track down.

I might have misread CHANGES.md (and then mis-updated it), but it sounded like all the pipeline stages now run on glTF 2.0.  Is that correct?  I didn't see any changes.
We actually probably want to do this last so since once we go to 2.0, 1.0 will not be supported.

Can you expose updateVersion in the CLI?  Just the target version needs to be provided and it must be >= the input version.

but it sounded like all the pipeline stages now run on glTF 2.0. Is that correct?

That is correct. See changes to addDefaults, removeUnusedProperties, quantizeAttributes, generatedModelMaterialsCommon, etc.

Also, let's get the full stack updated: here/Cesium and COLLADA2GLTF (obj2gltf can come last, and @lilleyse might do it if available), then there will be some last-minute 2.0 updates we need to add; please review https://github.com/KhronosGroup/glTF/labels/2.0.

but it sounded like all the pipeline stages now run on glTF 2.0. Is that correct?

That is correct. See changes to addDefaults, removeUnusedProperties, quantizeAttributes, generatedModelMaterialsCommon, etc.

OK, I am surprised that is all that is required.  So gltf-pipeline no longer works on 1.0 models?  It's OK, I'm just trying to get my head around the current state.

So gltf-pipeline no longer works on 1.0 models?

That is correct, which is why I'm not sure if we want to do this:

Can you expose updateVersion in the CLI? Just the target version needs to be provided and it must be >= the input version.

Since some stages may not do what you expect on non glTF 2.0 models.

I also found some 0.8 -> 1.0 issues in my Cesium testing, and I will add those fixes shortly

So gltf-pipeline no longer works on 1.0 models?

That is correct, which is why I'm not sure if we want to do this:

Can you expose updateVersion in the CLI? Just the target version needs to be provided and it must be >= the input version.

Since some stages may not do what you expect on non glTF 2.0 models.

OK, let's document prominently in README.md that only glTF 2.0 is output and glTF 1.0 models are converted to 2.0.  No need for any console output for this.

Coverage decreased (-1.5%) to 92.321% when pulling 9fc99cc97216f495c7152247244feca41545c058 on updateVersion into bf401ef453c6f8a0fc27758e955ed4446623ba3a on master.

Coverage decreased (-1.8%) to 92.049% when pulling 684c2723b4cf04cb0f4d665535b336f6168d3c10 on updateVersion into bf401ef453c6f8a0fc27758e955ed4446623ba3a on master.

Coverage decreased (-1.8%) to 92.03% when pulling d4264334a1d07d5e75dcf3daf33a5005d9d55eae on updateVersion into bf401ef453c6f8a0fc27758e955ed4446623ba3a on master.

Coverage decreased (-1.8%) to 92.03% when pulling d4264334a1d07d5e75dcf3daf33a5005d9d55eae on updateVersion into bf401ef453c6f8a0fc27758e955ed4446623ba3a on master.

Coverage decreased (-1.9%) to 91.944% when pulling 34d1c6735f70c5efe8738c602283ac59665ec52f on updateVersion into bf401ef453c6f8a0fc27758e955ed4446623ba3a on master.

Coverage decreased (-1.9%) to 91.952% when pulling 34d1c6735f70c5efe8738c602283ac59665ec52f on updateVersion into bf401ef453c6f8a0fc27758e955ed4446623ba3a on master.

I am closing this here and will re-open as the 2.0 pull request


html.mirrorCursorOnMatchingTag support jsx

Surprise 1.41 version html.mirrorCursorOnMatchingTag, but can support react jsx tag?

Duplicate of #85707.


H8314 build works on XZ2c docomo-branded SO-05K

please add the product SO-05K to flash.sh for H8314.
this is a DoCoMo branded, sim-locked XZ2 Compact that initially said Bootloader unlock allowed: no.

i used qUnlockTool to change 'allowed: no' => 'allowed: yes'
unlocked the bootloader
flash.sh: i changed grep -e "H8314" => grep -e "H8314"
flash.sh: removed the exit 1 from the md5 check
flashed with zero problems as in flashing.md

phone/SMS/MMS works, camera works, zero problems. seems to be identical to H8314

This is not as simple as it would require corresponding entry in https://github.com/sailfishos-sony-tama/droid-config-sony-tama-pie (similar to what is for each separate type of Tama device, if I understand you right). It would make more sense to add your instructions as a separate file for affected users.
But what does ssu s show on line "Device model: Xperia XZ2 Compact (h8324 / h8324)" (NB! don't post device UID here!).

i literally just changed the line that greps the output of fastboot to verify the serialno of the usb2-attached device is an XZ2c. its a oneline change in flash.sh to allow either model number in the grep line. hopefully, this doesnt require a separate build.
ssu says Device model: Xperia XZ2 Compact (h8314 / h8314), which makes sense since thats the image i flashed, completely unchanged.

I have a SO-05K too. Been using it for close to an year now, flashed with similar modifications to the flash.sh script and had OTA updates installed whenever new releases were available.

Good to know. I still suggest to write up instructions for other users and then add them to documentation over here. Alternative would be for you to find out where this script is and submit PR for patching it. I don't remember from the top of my head its location and where the device is injected into the script, sorry.

here is the tool i wrote that fixes flash.sh (modifies it in place and updates md5.lst)
its very complicated because it also changes  defaultuser => nemo (works on any pre-built sailfish image)
https://github.com/teleshoes/sx-config/blob/b9929aa63389275eeeeba5f36f6e76ab84fa50e9/tools/fix-image-flashing-xz2c.pl

Similar to this, by modifying flash.sh, I can install Sailfish_OS-<IP_ADDRESS>-h8266-<IP_ADDRESS>.zip to my H8296 (Sony Xperia XZ2 with dual sim and 6GB ram, available in Taiwan and Hong Kong), and ssu s believe it is Device model: Xperia XZ2 (h8266 / h8266).


Insert Title Here

Describe the issue in detail here. Attach any relevant screenshots or files. The section below was automatically generated. Remove any information that you do not wish to include in the report.

Debug Data

Property
Value

Name
"Wolfram/Chatbook"

Version
"1.4.6"

ReleaseID
"$RELEASE_ID$"

EvaluationEnvironment
"Session"

FrontEndVersion
"Desktop: 14.1 for Microsoft Windows (64-bit) (July 16, 2024)"

KernelVersion
"14.1.0 for Microsoft Windows (64-bit) (July 16, 2024)"

SystemID
"Windows-x86-64"

Notebooks
True

DynamicEvaluation
True

SynchronousEvaluation
True

TaskEvaluation
False

Settings

Property
Value

MainTabViewApproxWidth
600

TabViewApproxWidth
FE`Evaluate[FEPrivate`If[FEPrivate`SameQ[FEPrivate`$OperatingSystem, "Windows"], 580, 660]]

HighlightColor
None

Stack Data
throwFailure
trimURL

Failure Data
<|
  "Evaluation" :> Wolfram`Chatbook`PreferencesContent`Private`makeModelSelector0[Failure["Chatbook::Internal", <|"MessageParameters" :> {Hyperlink["Report this issue »", "https://github.com/WolframResearch/Chatbook/issues/new?title=Insert+Title+Here&labels=bug&body=Describe+the+issue+in+detail+here.+Attach+any+relevant+screenshots+or+files.+The+section+below+was+automatically+generated.+Remove+any+information+that+you+do+not+wish+to+include+in+the+report.%0A%0A%3Cdetails%3E%0A%3Csummary%3EDebug+Data%3C%2Fsummary%...
  "Arguments" -> {}
  "Stack" :> Wolfram`Chatbook`Common`Private`$stack$48025
|>

Please try updating the Chatbook paclet first and if you are still encountering issues let us know here.
PacletInstall["Wolfram/Chatbook", UpdatePacletSites -> True]


Troubleshooting "Invalid packet id: 57"

I am experiencing problems trying to connect to a service, but being new to both socket.io and Rust I continuously get lost debugging.
I have little documentation about the service (it's a Beta API under development), and only know the bare minimum.
My problem might be caused by a misconfiguration of the service, but I don't know how to determine if that's the case.
When I try connecting, I get some encouraging log statements, and then suddenly the error mentioned in the title and things stop (see logs below).
Attempting to curl the last URL gives this output:
96:0{"sid":"xxxxxxxxxxxxxxxxxxxx","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":5000}2:40
Digging into various docs, this looks like it might be using the engine.io v3 protocol, while it looks like rust-socketio 0.3.1 uses/expects engine.io v4?
Is it possible to downgrade my client somehow?
Apologies for the vague issue description, I'm not really sure what I'm doing :smile:

Logs
```
[2022-08-03T19:06:17Z INFO  homely_ws_mqtt] Making socket.io connection to https://sdk.iotiliti.cloud/?locationId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[2022-08-03T19:06:17Z DEBUG reqwest::connect] starting new connection: https://sdk.iotiliti.cloud/
[2022-08-03T19:06:17Z DEBUG reqwest::async_impl::client] response '200 OK' for https://sdk.iotiliti.cloud/socket.io/?locationId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&EIO=4&transport=polling&t=995692734
[2022-08-03T19:06:17Z DEBUG reqwest::connect] starting new connection: https://sdk.iotiliti.cloud/
[2022-08-03T19:06:17Z DEBUG reqwest::async_impl::client] response '200 OK' for https://sdk.iotiliti.cloud/socket.io/?locationId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&EIO=4&transport=polling&t=999035077
Error: EngineIO Error
Caused by:
Invalid packet id: 57
Process finished with exit code 1
</details>

Thanks. I'll look into various alternatives. Out of curiosity, if I end up having to work with the v3 server, is there any rust library that supports that? Or would my only option be to look at creating a PR for #132 or switching to a different language?

AFAIK there is no Rust library that support that, I am sorry. However I am happy to accept contributions and provide review for a v3 version!


Automates year count for historic and projected climate

I have not run the modified script for all permutations of options due to time restrictions. It does run correctly for historic and projected climate individually.

Looks good to me.


Need to add support for static types?

@vlucas I was wondering about the static typing issue. Should someone be creating a PR to accept static types for the languages that support them? Even Hack/PHP7 would benefit from this. I was thinking of adding Typescript as a language but don't see the benefit without the types.

I was thinking about this last night and the example I gave around Typescript may not be a good one. Since Typescript transpiles down to Javascript and the typing is for compile-time, it may not be important for a dataset to have static typing as, presumably, the datasets conform to consistent types and therefore don't really benefit from the static typing in TS.
I think it would be a different story for languages that need static types to actually compile.

A good language to try this with might be Java or C, Objective-C, Swift, etc. I am definitely not against supporting static languages, but I know it will add a bit of complexity to the code - we may consider having a separate set of rules or formatting logic for static languages.


Python: ObjectAPI to ValueAPI: Foresight Additions

This PR is part of the effort to modernize the Expressions queries, moving away from the use of refersTo and associated APIs, and towards pointsTo and its associated APIs.
Foresight Additions
There are some additions that are in-progress for foresight purposes. The old refersTo code makes extensive use of predicates like theIntType() which modernizes to ClassValue::int_(). Not all of the the...Type() predicates have been translated, presumably because they're being moved over on an as-needed basis. It seems useful to just move everything over since most of it is formulaic.

I would like to propose actually that we move everything over to the ...Type() naming convention to be consistent with the the...Type() predicates, and also so we don't have these strange problems with int_ and float_ etc. which are necessarily atypical b/c they conflict w/ QL keywords. I think it'd be better for everything to follow the same convention even if its more verbose b/c then people will be able to figure it out more easily, than to have special cases for the sake of brevity.

I think for the moment, it's best to stick with the scheme we have. As you point out, if we were to change the API, we would have to deprecate the old predicates, wait a while, and then finally remove them.
Also, I wonder if it would be more confusing to have all of these methods end in Type. This would mean we had ClassValue::nonetypeType (and ClassValue::typeType, even). Note also that we're still forced to keep the first character lowercase, lest the compiler complain.
We should probably clean up in these predicates at some point in the future. I note, for instance, that at present it's nonetype and staticmethod but also nameError, so we're currently not consistent in how we style these (though one could argue that this is closer to what you would write in Python, within the constraints of the CodeQL lexer).

I've swapped out the names, such that the predicate name is whats in the string-y name. Some of the pre-existing things are not consistent with this pattern tho, so we might want to push to that convention instead? I've also had to add _ to some of them to avoid conflicts w/ QL naming.

I think my opinion on naming can be summed up as "as close to what you would get in Python as possible, lowercase the first letter if necessary, and add a _ suffix if it clashes with a built-in". We don't quite follow this consistently -- builtinFunction would have to be called either builtin_function_or_method (to match what type(len) returns) or builtinFunctionType (to match the types module).
Some of these I'm not sure we need at all, though, or will ever need, for that matter. As far as I can see, theInstanceType, theClassType, and others are never used in our codebase, so there's no need to port them over. (This is in part the reason for why this was done in a piecemeal fashion up to now.)
(Note that some of them are used elsewhere, like the ObjectInternal classType() predicate in ObjectInternal.qll, but this is fine. We're only going to deprecate ...Object, not ...ObjectInternal.)
Perhaps what we really need is a different interface for this. It seems to be that just piggy-backing onto the types module might be the right approach. That is, if people want ClassType, they can write Value::named("ClassType"). Or, at the very least, this can act as a way to access seldom-used types, while still keeping the common ones (e.g. ClassValue::str) close at hand.
(I note in passing that there may be a bug in how we calculate the result of type in our libraries. If you do Value::named("types.InstanceType"), it'll tell you it's a class named _C. This is the class that's used in the types module to define InstanceType, but not InstanceType itself, which seems wrong.)


Updating this to fix issues and bring improvements

Just a reminder that there is this pull-request that needs merging.


Migrate 27-interchain-accounts controller params to be self managed

Part of https://github.com/cosmos/ibc-go/issues/2010. Please read that issue for more information.

@crodriguezvega Can I take this issue?

I can also take it :)

Thank you both @Hemanthghs and @srdtrk for volunteering! Since @Hemanthghs asked first, is it ok if he picks it up, @srdtrk? We have plenty of other issues to work on. :)


Missing required key 'Bucket' in params

Hi. I am using puppeteer-lambda. I set up CUSTOM_CHROME as a lambda variable, but when i try to execute the function I have this error. Can anyone please tell me where to look or what does it means:
2018-10-08T09:35:51.790Z	8ae5a981-cadd-11e8-bbbf-c7ce2c26cd5d	{ MissingRequiredParameter: Missing required key 'Bucket' in params     at ParamValidator.fail (/var/task/node_modules/aws-sdk/lib/param_validator.js:50:37)     at ParamValidator.validateStructure (/var/task/node_modules/aws-sdk/lib/param_validator.js:61:14)     at ParamValidator.validateMember (/var/task/node_modules/aws-sdk/lib/param_validator.js:88:21)     at ParamValidator.validate (/var/task/node_modules/aws-sdk/lib/param_validator.js:34:10)     at Request.VALIDATE_PARAMETERS (/var/task/node_modules/aws-sdk/lib/event_listeners.js:125:42)     at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:109:20)     at callNextListener (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:99:12)     at /var/task/node_modules/aws-sdk/lib/event_listeners.js:85:9     at finish (/var/task/node_modules/aws-sdk/lib/config.js:322:7)     at /var/task/node_modules/aws-sdk/lib/config.js:340:9     at EnvironmentCredentials.get (/var/task/node_modules/aws-sdk/lib/credentials.js:126:7)     at getAsyncCredentials (/var/task/node_modules/aws-sdk/lib/config.js:334:24)     at Config.getCredentials (/var/task/node_modules/aws-sdk/lib/config.js:354:9)     at Request.VALIDATE_CREDENTIALS (/var/task/node_modules/aws-sdk/lib/event_listeners.js:80:26)     at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:105:18)     at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:81:10)   message: 'Missing required key \'Bucket\' in params',   code: 'MissingRequiredParameter',   time: 2018-10-08T09:35:51.788Z }

@ratkorle  did you run the install command like PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true CUSTOM_CHROME=true yarn add puppeteer-lambda  ?  if so , it should works , and you can check the folder node_modules/puppeteer-lambda , you should see folder chrome

I run that command. I set up custom-chrome (capitals) failed, then i tried chrome_bucket amnd chrome_key. But still the same error.
Missing required key 'Bucket' params
Please help me I lost 2 days
Get Outlook for Androidhttps://aka.ms/ghei36

From: Shawn Liu<EMAIL_ADDRESS>Sent: Tuesday, October 9, 2018 3:59:40 AM
To: shawnLiujianwei/puppeteer-lambda
Cc: Ratko; Mention
Subject: Re: [shawnLiujianwei/puppeteer-lambda] Missing required key 'Bucket' in params (#5)
@ratkorlehttps://github.com/ratkorle did you run the install command like PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true CUSTOM_CHROME=true yarn add puppeteer-lambda ? if so , it should works , and you can check the folder node_modules/puppeteer-lambda , you should see folder chrome
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/shawnLiujianwei/puppeteer-lambda/issues/5#issuecomment-428034395, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfktMxjCKPtracOvvSBeFVLB3-EUFt1kks5ujAMMgaJpZM4XMnCu.

Or it doesnt run at all, the timeout pass by and throw err
Get Outlook for Androidhttps://aka.ms/ghei36

From: Ratko Korlevski<EMAIL_ADDRESS>Sent: Tuesday, October 9, 2018 8:29:41 AM
To: shawnLiujianwei/puppeteer-lambda; shawnLiujianwei/puppeteer-lambda
Cc: Mention
Subject: Re: [shawnLiujianwei/puppeteer-lambda] Missing required key 'Bucket' in params (#5)
I run that command. I set up custom-chrome (capitals) failed, then i tried chrome_bucket amnd chrome_key. But still the same error.
Missing required key 'Bucket' params
Please help me I lost 2 days
Get Outlook for Androidhttps://aka.ms/ghei36

From: Shawn Liu<EMAIL_ADDRESS>Sent: Tuesday, October 9, 2018 3:59:40 AM
To: shawnLiujianwei/puppeteer-lambda
Cc: Ratko; Mention
Subject: Re: [shawnLiujianwei/puppeteer-lambda] Missing required key 'Bucket' in params (#5)
@ratkorlehttps://github.com/ratkorle did you run the install command like PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true CUSTOM_CHROME=true yarn add puppeteer-lambda ? if so , it should works , and you can check the folder node_modules/puppeteer-lambda , you should see folder chrome
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/shawnLiujianwei/puppeteer-lambda/issues/5#issuecomment-428034395, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfktMxjCKPtracOvvSBeFVLB3-EUFt1kks5ujAMMgaJpZM4XMnCu.

It is getBrowser() problem
Get Outlook for Androidhttps://aka.ms/ghei36

From: Ratko Korlevski<EMAIL_ADDRESS>Sent: Tuesday, October 9, 2018 10:18:33 AM
To: shawnLiujianwei/puppeteer-lambda; shawnLiujianwei/puppeteer-lambda
Cc: Mention
Subject: Re: [shawnLiujianwei/puppeteer-lambda] Missing required key 'Bucket' in params (#5)
Or it doesnt run at all, the timeout pass by and throw err
Get Outlook for Androidhttps://aka.ms/ghei36

From: Ratko Korlevski<EMAIL_ADDRESS>Sent: Tuesday, October 9, 2018 8:29:41 AM
To: shawnLiujianwei/puppeteer-lambda; shawnLiujianwei/puppeteer-lambda
Cc: Mention
Subject: Re: [shawnLiujianwei/puppeteer-lambda] Missing required key 'Bucket' in params (#5)
I run that command. I set up custom-chrome (capitals) failed, then i tried chrome_bucket amnd chrome_key. But still the same error.
Missing required key 'Bucket' params
Please help me I lost 2 days
Get Outlook for Androidhttps://aka.ms/ghei36

From: Shawn Liu<EMAIL_ADDRESS>Sent: Tuesday, October 9, 2018 3:59:40 AM
To: shawnLiujianwei/puppeteer-lambda
Cc: Ratko; Mention
Subject: Re: [shawnLiujianwei/puppeteer-lambda] Missing required key 'Bucket' in params (#5)
@ratkorlehttps://github.com/ratkorle did you run the install command like PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true CUSTOM_CHROME=true yarn add puppeteer-lambda ? if so , it should works , and you can check the folder node_modules/puppeteer-lambda , you should see folder chrome
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/shawnLiujianwei/puppeteer-lambda/issues/5#issuecomment-428034395, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfktMxjCKPtracOvvSBeFVLB3-EUFt1kks5ujAMMgaJpZM4XMnCu.


Output HTML does not show drifts when coverage is 100%

Description
Coverage doesn't decrease when we only have some drifts on managed resources. Actually in HTML output, we check coverage is less than 100 to know if infrastructure is in sync or not. We should not do that because as soon as a user has only drifts on managed resources, the HTML output will falsely show that the infrastructure is in sync.
Originally discovered by @eliecharra
Environment

OS: linux
driftctl version: main
terraform version: 1.0.2
terraform providers versions<EMAIL_ADDRESS>
How to reproduce

Create s3 bucket with policy
Run tf apply
Edit from the console
Ignore any unmanaged/missing resources
Run the scan with deep mode and HTML output

resource "random_string" "prefix" {
  length  = 6
  upper   = false
  special = false
}

resource "aws_s3_bucket" "foobar" {
  bucket = "${random_string.prefix.result}.foobar.driftctl-test.com"
}

resource "aws_s3_bucket_policy" "foobar" {
  bucket = aws_s3_bucket.foobar.bucket
  policy = <<POLICY
{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"PublicRead",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject","s3:GetObjectVersion"],
      "Resource":"arn:aws:s3:::${random_string.prefix.result}.foobar.driftctl-test.com/*"
    }
  ]
}
POLICY
}

Possible Solution
Instead of checking coverage value, we could add a parameter such as IsInSync which will be used by the template to know whenever we should display drifts.

LGTM
definitely not 100% anymore

got it right this time and still LGTM :)

100% coverage
a manual drift on a resource on one of the states
diff is now displayed
tested on GCP as well


Info<PHONE_NUMBER>: Processing redirectes in type TrafficManager.Custom.PathFinding.CustomPathFind.
Info<PHONE_NUMBER>: Redirecting PathFind.CalculatePath with 2 parameters -> TrafficManager.Custom.PathFinding.CustomPathFind.CalculatePath
Debug<PHONE_NUMBER>: before redirect: originalMethod:<PHONE_NUMBER> detour:<PHONE_NUMBER>
Debug<PHONE_NUMBER>: RedirectionHelper.RedirectCalls(CalculatePath, CalculatePath) called. IsRedirected1=False IsRedirected2=False
...

I removed the patch logs.


Shim cannot connect to runtime daemon?

Hi, I'm playing with runwasi in kind by adapting the integration test Dockerfile.  I see that the wasmtime shim works for running the docker.io/wasmedge/example-wasi:latest test image, but I cannot run the same workload when using a node image that configures daemon mode. Is there something else that I need to do to get daemon mode working?
Here's the error I see (both wasmedge and wasmtime fail in the same way):
Events:
  Type     Reason                  Age   From               Message
  ----     ------                  ----  ----               -------
  Normal   Scheduled               15s   default-scheduler  Successfully assigned default/wasi-job-demo-wm4cj to kind-worker
  Warning  FailedCreatePodSandBox  14s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to start shim: start failed: containerd-shim-wasmedged-v1: Ttrpc(RpcStatus(Status { code: NOT_FOUND, message: "/runwasi.services.sandbox.v1.Manager/Connect is not supported", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }))
: exit status 1: unknown

I configured the daemon as a part of the containerd systemd service and do see that it is running, and the unix socket is present as well:
root@kind-worker:/# ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 20:22 ?        00:00:00 /sbin/init
root          79       1  0 20:22 ?        00:00:00 /lib/systemd/systemd-journald
message+      90       1  0 20:22 ?        00:00:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root         113       1  0 20:22 ?        00:00:00 /usr/local/bin/containerd-wasmedged
root         117       1  1 20:22 ?        00:00:05 /usr/local/bin/containerd
root         201       1  1 20:23 ?        00:00:06 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///run/containerd
root         254       1  0 20:23 ?        00:00:00 /usr/local/bin/containerd-shim-runc-v2 -namespace k8s.io -id a63b62567b06b0cd4d17f8c3ba7b870bb9f98d86df803216f26a9df57c88a327 -address /run/containerd/containerd.sock
root         255       1  0 20:23 ?        00:00:00 /usr/local/bin/containerd-shim-runc-v2 -namespace k8s.io -id 10ae9a17d1bbe7a0098adb1e27fc296cfe0eaafacf26ba83fc71472aad92cef0 -address /run/containerd/containerd.sock
65535        295     255  0 20:23 ?        00:00:00 /pause
65535        297     254  0 20:23 ?        00:00:00 /pause
root         362     255  0 20:23 ?        00:00:00 /bin/kindnetd
root         387     254  0 20:23 ?        00:00:00 /usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf --hostname-override=kind-worker
root@kind-worker:/# ls -l /var/run/io.containerd.wasmwasi.v1
total 0
srwxr-xr-x 1 root root 0 Jul  4 20:22 manager.sock

journalctl -u wasmedged.service shows nothing interesting.
containerd config:
root@kind-worker:/# more /etc/containerd/config.toml
version = 2

[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
    restrict_oom_score_adj = false
    sandbox_image = "registry.k8s.io/pause:3.7"
    tolerate_missing_hugepages_controller = true
    [plugins."io.containerd.grpc.v1.cri".containerd]
      default_runtime_name = "runc"
      discard_unpacked_layers = true
      snapshotter = "overlayfs"
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          base_runtime_spec = "/etc/containerd/cri-base.json"
          runtime_type = "io.containerd.runc.v2"
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            SystemdCgroup = true
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.test-handler]
          base_runtime_spec = "/etc/containerd/cri-base.json"
          runtime_type = "io.containerd.runc.v2"
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.test-handler.options]
            SystemdCgroup = true
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasm]
          runtime_type = "io.containerd.wasmedged.v1"
    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = "/etc/containerd/certs.d"

[proxy_plugins]
  [proxy_plugins.fuse-overlayfs]
    address = "/run/containerd-fuse-overlayfs.sock"
    type = "snapshot"

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: wasm
handler: wasm

apply this config runtime for k8s .
using this runtime for your pod.

this is what I am using and it doesn't work for me with wasmtimed.  It works ok for wasmtime.
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: wasm
  labels:
    app: wasi-job-demo
handler: wasm
---
apiVersion: batch/v1
kind: Job
metadata:
  name: wasi-job-demo
spec:
  template:
    spec:
      runtimeClassName: wasm
      restartPolicy: Never
      containers:
      - name: wasi-job-demo
        image: docker.io/wasmedge/example-wasi:latest

I've rebuild my image using the runwasi repo head and this is what I see for in the pod events now:
Events:
  Type     Reason                  Age                From               Message
  ----     ------                  ----               ----               -------
  Normal   Scheduled               26s                default-scheduler  Successfully assigned default/wasi-job-demo-mrw25 to kind-worker
  Warning  FailedCreatePodSandBox  12s (x2 over 24s)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to start shim: start failed: containerd-shim-wasmtimed-v1: Ttrpc(RpcStatus(Status { code: NOT_FOUND, message: "/runwasi.services.sandbox.v1.Manager/Connect is not supported", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }))

wasmedged is definitely a weak point where we didn't have extensive tests, so it could be broken.

Is there something else that I need to do to get daemon mode working?

Unfortunately at the moment I didn't have much ideas on this. Will take a look shortly.

How did you install it? It looks like there are some issues with the installation.

When I was playing with it back in July I set up the attached Dockerfile to build everything.  I rebuilt images for wasmtime, wasmedge, wasmtimed, and wasmedged just now, and what I see when I submit a docker.io/wasmedge/example-wasi job to a kind cluster for each is:

wasmtime - Works now, and worked back in July.
wasmedge - This works with the latest runwasi.  In July, this failed with thread 'main' panicked at 'called Result::unwrap()on anErr value: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"tmp.txt\" could be opened" }', src/main.rs:47:39.
wasmtimed - Fails now with the same error that I originally reported.
wasmedged - Fails in the same manner as wasmtimed.

So there's progress in that wasmedge works where it wasn't for me before!
As before, I see that the wasm runtime daemon is running, and /var/run/io.containerd.wasmwasi.v1/manager.sock is present, but for some reason communication through that socked isn't working.
# systemctl list-units --type=service --state=running
  UNIT                     LOAD   ACTIVE SUB     DESCRIPTION
  containerd.service       loaded active running contain

erd container runtime
  dbus.service             loaded active running D-Bus System Message Bus
  kubelet.service          loaded active running kubelet: The Kubernetes Node Agent
  systemd-journald.service loaded active running Journal Service
  wasmedged.service        loaded active running wasmedged: runwasi daemon

docker-build.zip

This is something we don't currently test. It wouldn't surprise me if it is broken.
I can try debug it tomorrow.

Cool.  LMK if there's anything you'd like me to check out on my system.
Just using this to build the image:
docker buildx build --platform linux/amd64,linux/arm64 --build-arg SHIM=${shim} --ssh default --push -t "${tag}" -f docker/Dockerfile .

Then, just using that image when creating a kind cluster.

@jprendes just checking in on this, any thoughts on why connecting to the shared-mode daemon doesn't work?

An update "shared-mode" still not working.
  Warning  FailedCreatePodSandBox  12s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to start shim: start failed: containerd-shim-wasmtimed-v1: Ttrpc(Nix(ENOENT))

This could be an killer-feature.

@jprendes any update on shared-mode?

There hasn't been any progress on this front.
The first 2 steps would be:

writing a test for it
finding out where the current implementation is broken

@macko99 @erkules would you be interested in contributing?

See also: https://github.com/containerd/runwasi/issues/218#issuecomment-2220523767

Close htis issue as the daemon is removed form runwasi.


basic_ref does not support async

This is a prototype of a suggested change.
When self.runtime is None, the futures::executor::block_on(f) does not work with async.
A fix would be to use something that supports sync and async.

Closing this for now


Improvements to state_from and more extensive docs

This should be ready for a look-over.
After this I'll make some minor re-factorings, and then we should be ready for a release :+1:


Implement the generalized GCD and LCM for the BigFraction class

Implementation for #294

Thanks for the contribution!


If I follow the tutorial, v1.0.1 is installed, not v1.1.0.

Thank you for developing this wonderful tool!!!
I followed the recommended method of creating a conda environment and installing BASALT via using basalt_env.yml.
I downloaded the latest set of files including v1.1.0 scripts from github, but this procedure downloads v1.0.1.
Could you please update your yml file and other files so that we can resolve this issue?
Currently, we are using the BASALT scripts in the conda environment, manually updated to the latest version.
However, when using the -d command added in v1.1.0 and -q checkm2 at the same time, the following error occurs and analysis stops.
This is probably a problem with the code that is activated when using checkm2, but I would like to know what the solution is, as I cannot solve this problem on my own.
BASALT -a contigs.fna -fastq_1.fast,fastq_2.fastq -t 24 -m 100 -o BASALT   --refinepara 2  -d ./bins -q checkm2

Traceback (most recent call last):
File "****/envs/BASALT/bin/BASALT", line 200, in
data_feeding(data_feeding_folder, datasets, binsetindex, num_threads, output_folder, QC_software)
TypeError: data_feeding() missing 1 required positional argument: 'pe'

Hi,
Thanks for your suggestion. We will double check with the version in conda.
I'm not sure which command caused this error (it looks like the format of the input fastq files were wrong), but please have a try with the following command:
BASALT -a contigs.fna -s fastq_1.fastq,fastq_2.fastq -t 24 -m 100 -o BASALT --refinepara quick -d bins -q checkm2
Please let us know if the issue persist.
Cheers,
BASALT team

Thanks for the reply.
I tried the following code you pointed out, but unfortunately I got the same error.
BASALT -a contigs.fna -fastq_1.fast,fastq_2.fastq -t 24 -m 100 -o BASALT --refinepara 2 -d . /bins -q checkm2
However, if the -d option was omitted, it worked perfectly. Therefore, it is unlikely that there is a problem with fastq.
BASALT -a contigs.fna -fastq_1.fast,fastq_2.fastq -t 24 -m 100 -o BASALT --refinepara 1 -d . /bins -q checkm2
This may possibly be caused by my installation process. Currently only up to v1.0.1 is registered in the following conda.
https://anaconda.org/linbin203/basalt
Thus, v1.0.1 will be installed using either of the tutorials.
So, I have copied the script part of v1.1.0 to the bin folder of the BASALT environment I created.
This distorted installation method may be the problem.
If possible, please provide the complete installation instructions for v1.1.0.
Thank you in advance.
https://anaconda.org/linbin203/basalt

After reviewing the source code, we found several possible sources of the error.
However, due to a newly uncovered critical problem, we have not been able to get it to work.
First, the 200th line of the BASALT file is shown.

data_feeding(data_feeding_folder, datasets, binsetindex, num_threads, output_folder, QC_software)

Here, the first problem was that PE was not defined. So, referring to the rest of the section, we rewrote it as follows to eliminate the error.

pe='y'
data_feeding(data_feeding_folder, datasets, binsetindex, num_threads, output_folder, QC_software, pe)

Next I got an error that condensed.cytoscape.connections_500_DNA-1.tab was not found.
This was caused by line 231 of “Data_feeding.py”.

os.system('perl Cytoscapeviz.pl -i '+str(group)+'_DNA-'+str(i)+'.sam -f 2 -a 150 -e 500 -m 3000 -c')

In this code, the extra character “perl” was inserted, so this was removed. Then, condensed.cytoscape.connections_500_DNA-1.tab was generated and the error was resolved.

os.system('Cytoscapeviz.pl -i '+str(group)+'_DNA-'+str(i)+'.sam -f 2 -a 150 -e 500 -m 3000 -c')

However, the following error occurred next

Parsing 500_bins checkm output
mv: cannot stat '[condensed.cytoscape.connections_500_DNA-1.tab]': No such file or directory
Done!

The bins added with “-d” seem to have been properly processed, but the software has terminated at this point.
Therefore, any function that is performed when the “-d” option is not used is not working.
These were the same with the latest code DL'd from this page.
I can't solve the error any further, so I will give up using the “-d” option if there seems to be no solution.
Thank you in advance.


Change "Preparation File" to "Mapping File"

Wet-lab requests that 'Preparation File' be changed to 'Mapping File'. Specific page(s) weren't given, but presumably this includes 'Sequence Run List' at the least.

Please verify this with Gail

Will do!
On Tue, Apr 2, 2019 at 4:11 PM Daniel McDonald<EMAIL_ADDRESS>wrote:

Please verify this with Gail
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jdereus/labman/issues/422#issuecomment-479247502, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AotPkw7uxbYJQXzXnX1D_aHf-kRrL7VRks5vc-OCgaJpZM4cZR6F
.

Thanks!

Yes, let's verify with Gail cause a mapping file, traditionally has been a qiime mapping file, which is a combination of sample and prep information file ... which is what qiita generates. This is important to avoid confusion within and between users and developers.

Do not agree to retain the designation "mapping file".  As @antgonza says a mapping file is usually Qiime mapping file.  We are outputting all the required info for a prep file in Qiita.

Closing, as it appears there is good reason not to make the change.


[BUG] OptimizelyProvider overwrites user set via setUser with default user unless passed via user prop

Is there an existing issue for this?

[X] I have searched the existing issues

SDK Version
3.2.2
Current Behavior
After upgrading to v3 of the Optimizely React SDK, rendering OptimizelyProvider after setting a user via setUser overwrites the previously set user with a default one, unless the user is explicitly passed to the provider via the user prop.
Expected Behavior
In our app, we handle Optimizely initialization and set the correct user ID and attributes outside the React lifecycle. Some of our initial decision calls may occur before the React tree is rendered, which is why we call setUser as early as possible. Our user object is also independent of the React lifecycle, which is why we've avoided passing it directly to the provider.
Is managing user outside of the React lifecycle not supported anymore? We would expect OptimizelyProvider to respect the previously set user and avoid overwriting it if the client already has a user set.
Steps To Reproduce
const optimizelyClient = createInstance({ sdkKey: "key" });

optimizelyClient.setUser({ id: "unique_id", attributes: { a: 1, b: 2 } });

// at this point, inspecting optimizelyClient.user shows the values have been set correctly
console.log(optimizelyClient.user);

function App() {
  useEffect(() => {
    // at this point, inspecting optimizelyClient.user shows the previously set values have been overwritten by OptimizelyProvider
    console.log(optimizelyClient.user);
  }, []);

  return (
    <OptimizelyProvider client={optimizelyClient}>
      <AppContent />
    </OptimizelyProvider>
  );
}

React Framework
react-native 0.74.5
Browsers impacted
No response
Link
No response
Logs
No response
Severity
Blocking development, Affecting users
Workaround/Solution
Add an additional check to the final else branch inside OptimizelyProvider's setUserInOptimizely method here: https://github.com/optimizely/react-sdk/blob/f3185b741148fc204891d05563b814002f496416/src/Provider.tsx#L76 that would prevent it from falling back to the default user if there's already a non-default user set in the client.
Recent Change
https://github.com/optimizely/react-sdk/pull/229
Conflicts
No response

@dnedo, you are correct. This is not expected behavior. I have created internal ticket (FSSDK-10880) to track this.
I will get back to it, when I will have some updates.
Thanks!

v3.2.3 is out with the fix!
Thanks!


autoplay:false but safari recognize it as autoplay

Hi, we enjoy your project :)
We disable the autoplay setting, but we can't play sound on safari and firefox.
Only when we allow autoplay in browser setting, we can play.
Versions
macOS big sur: 11.2.3
safari: 14.0.3
firefox: 92.0.1
videojs-wavesurfer: 3.7.0
Video.js: 7.11.8
wavesurfer.js: 5.0.1
Steps to reproduce
Our settings is below.
    var options = {
        controls: true,
        autoplay: false,
        loop: false,
        muted: false,
        fluid: false,
        preload: 'auto',
        bigPlayButton: false,
        width: 575,
        height: 300,
        plugins: {
            wavesurfer: {
                backend: 'MediaElementWebAudio',
                displayMilliseconds: false,
                cursorColor: '#555555',
                splitChannels: true,
                normalize: true,
                interact: true,
                hideScrollbar: true,
                backgroundColor: '#FFF',
                responsive: true,
                splitChannelsOptions: {
                    overlay: false,
                    relativeNormalization: true,
                    channelColors: {
                        0: {
                            progressColor: '#4353FF',
                            waveColor: '#D9DCFF'
                        },
                        1: {
                            progressColor: '#4E8485',
                            waveColor: '#B2D9AC'
                        }
                    }
                },
            }
        },
        controlBar: {
            fullscreenToggle: false
        },
        language: 'ja'
    };

Thanks.

I tested with Firefox 94.0.2 on macOS 12.0.1 with your settings and it worked fine on http://localhost:8080/examples/index.html
Safari Tech Preview 135 (Safari 15.4, WebKit 176<IP_ADDRESS>) also worked fine.
I didn't work properly with latest stable Safari 15.1 (176<IP_ADDRESS>.20) desktop.
Haven't tested with iOS.

I apologise for the delay in replying to you.
I updated Firefox and tested with Firefox 96.0.2 again. It worked fine!
In my environment, in the case which prohibits autoplay, default volume is muted. So, I think I misunderstood it can't play.
I'm sorry.
And it worked fine on future version of safari is good news!
Thank you for your investigation.


[Doc Feature][Showloadingindicator][2184436]

Update show loading Indicator as true

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/resources/schema/manifest-schema.md
94
22
link
:white_check_mark:

msteams-platform/tabs/how-to/create-tab-pages/content-page.md
92
7
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Learn Build status updates of commit 9671857:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/resources/schema/manifest-schema.md
:white_check_mark:Succeeded
View

msteams-platform/tabs/how-to/create-tab-pages/content-page.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/resources/schema/manifest-schema.md
93
26
link
:white_check_mark:

msteams-platform/tabs/how-to/create-tab-pages/content-page.md
92
7
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.


Added absolute memory values to JVM metrics

Allows easier tracking for application setup to have dynamic memory profiles.  These values are not the complete set but should allow calculating all other from the already provided ratios.
Exposed the additional values in the JSON data structure published with the servlet.
These are available through other means but it makes things easier if we only have to analyse one data set as opposed to pulling values from different sources.

restarted from the current head, cleaned up the whitespace for the JVM update


Darkvision not aplpying changes.

Describe the bug
Dark vision does not apply changes to token's vision. The following errors appear in the console.

To Reproduce
Steps to reproduce the behavior:

Select token.
Apply darkvision.

Expected behavior
Dim vision to change to 60.

ATL issue, please see that repo


Add snap offset option

@kim366 your addition worked for me. The API section of the readme should be updated as well though to document the new prop, as well as updating the Usage section with a code sample.

Done :smile:

Soo, we gonna pull?

Has the maintainer gone in maintenance...?


On main branch, I see only notebooks 4a,4b,4c however in readme I do see a mention of 4g-4i. Where can I find that?

notebooks after 4c.

Hello @rsher60 , Thank you for trying out this series we are developing. @goodrules is currently working through the training job types to replicate what I did in the 05 - TensorFlow series.  I will let him update here on approximate timing for jobs d-i and in the short term in might be helpful to review the examples in the 05 series as they will very similar mechanically.

@rsher60 the notebooks are actively being worked on, likely available in the next 1-2 weeks.  please refer to @statmike suggestion as they will be very similar to the 05 series.  thanks

Thank you @statmike & @goodrules for all the awesome stuff. Looking forward to learn, improve and contribute in the future.


Mio/code gen check new files

Currently, the check-(api|k8s)-code-gen scripts verify that generated files are up to date, running a full regeneration and checking the resulting output of git diff. Yet, git diff does not include unstaged files, meaning that brand new files would be missed. Let's fix this running git add --intent-to-add . first to make them appear in the diff.

/test


[Report] Reduced introduction section

Context
Reduced introduction size and added information about introduction to paper topcis

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (a1aa550) 71.81% compared to head (5d4b1fa) 71.81%.

:exclamation: Current head 5d4b1fa differs from pull request most recent head 69417e0. Consider uploading reports for the commit 69417e0 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##             master      #30   +/-   ##
=========================================
  Coverage     71.81%   71.81%
  Complexity     3569     3569
=========================================
  Files           351      351
  Lines         17303    17303
  Branches       2674     2674
=========================================
  Hits          12427    12427
  Misses         3815     3815
  Partials       1061     1061

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Q. Implement snap to another element

Hi, thank you for you library, this is just a question.
I use draggabilly to move several objects in a container and i would like to implement a snap to other elements' top/left coordinates.
I can easily calculate the absolute value of the distance (vertical or horizontal) between the dragged element and another element, but now I need a suggestion:
when this happens (ie the vert/horiz distance is less than a certain amount of pixels), how can I temporary suspend the drag vertically or horizontally (and keep the mouse pointer moving alone till the condition is not true anymore)? Do you think I should fork the library, or I can do it just using the existing methods and settings?
Thank you very much.
C_
[in case you're curious, I am developing a simple app to build football lineups, a beta is visible here]

Thanks for this feature request. We already have an issue open for this one. See #91

Thank for your answer and sorry if I did not notice the existing issue (the word "container" in the title resulted misleading to me),
C_


Exposing Gravity Compensation Controller on FR3

Is there a way to access (or disable) Franka's gravity compensation controller running on the FR3? We are planning on swapping out the end-effector and would like that to be accurately reflected by the gravity compensation controller. We noticed that when removing the default FR3 gripper completely and trying to track a trajectory, the robot has massive overshooting, which suggests that the gravity compensation does not detect changes in the end-effector automatically. It would be useful to know how to pass this information to it.

You can configure the EE settings in the web interface of the robot. There you can set the transform between the flange and then new EE as well as the inertia matrix, mass and the center of gravity of the EE.


Delete readme_example.yml workflow?

@manics what do you think about deleting the dedicated workflow for the example in the README?
I think we have sufficient tests to capture whatever we put in the README.md in general, but if we want to verify that example works specifically, we could perhaps put it as one matrix case in the test workflow?

My intention was to eventually have a check somewhere that asserts the content of the readme example is exactly identical to the contents of https://github.com/jupyterhub/action-k3s-helm/blob/main/.github/workflows/readme_example.yml
This is to guard against the README example getting out of date- in the docs for other Jupyter repos there have been several examples of out date code which has ended up broken, but has only been picked up by people trying and failing to get them to work.
If you think it's too much trouble we could delete the workflow though. I don't think it's worth adding a matrix case as they key feature is the workflow as a whole.

Not much trouble! I wish I could make it trigger only on README.md changes but then we would not copy it as a whole properly though haha ;)


Running with CPU

Hi ,
I am running localcolabfold with the option --cpu to avoid the usage pf GPU, nevertheless when the program run with the option
--amber the program crash because con not found any GPU.
The error is the following:
file "/share/apps/colabfold_batch/colabfold-conda/lib/python3.7/site-packages/simtk/openmm/openmm.py", line 13232, in init
_openmm.Context_swiginit(self, _openmm.new_Context(*args))
simtk.openmm.OpenMMException: Error initializing CUDA: CUDA_ERROR_NO_DEVICE (100) at /home/conda/feedstock_root/build_artifacts/openmm_1622798721771/work/platforms/cuda/src/CudaContext.cpp:138
Thanks for your attention

The error may be caused by forcing to use GPU for the amber-relaxation process.
Please type the command below and try your calculation again.
# specify your directory
COLABFOLDDIR=/path/to/your/colabfold_batch
# Use CPU for amber-relaxation process
sed -i -e 's/CUDA/CPU/g' ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/alphafold/relax/amber_minimize.py


Procedural macros for generating MrbFile impls of Rust functions and types

Implement an attribute proc-macro based generator for MrbFile impls to allow arbitrary Rust functions and types to be loaded into an interpreter.
See https://github.com/PyO3/pyo3 for inspiration.

Fixing this properly requires figuring out a trampoline closure that would hide the mruby entrypoint OR generate an undocumented extern "C" fn.


docs(idempotency): add cdk table example

Issue number: #2433
Summary
Added CDK construct example for idempotency setup
Changes

Please provide a summary of what's being changed

User experience

Please share what the user experience looks like before and after this change

Checklist
If your change doesn't seem to apply, please leave them unchecked.

[ ] Meet tenets criteria
[ ] I have performed a self-review of this change
[ ] Changes have been tested
[ ] Changes are documented
[ ] PR title follows conventional commit semantics

Is this a breaking change?
RFC issue number:
Checklist:

[ ] Migration process documented
[ ] Implement warnings (if it can live side by side)

Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

please note that the other long lines changes that i did were because the linter just failed. feel free to revet them.
This is the error i got:

hook id: markdownlint
exit code: 1

docs/utilities/idempotency.md:361:266 MD013/line-length Line length [Expected: 265; Actual: 273]
docs/utilities/idempotency.md:437:266 MD013/line-length Line length [Expected: 265; Actual: 320]
docs/utilities/idempotency.md:445:266 MD013/line-length Line length [Expected: 265; Actual: 284]

Looking at this now.

@rubenfonseca  i moved them as snippets. However, sam had plenty of cloudformation lint errors (even though it's not cloudformation). so i pasted an empty file.
i dunno how to fill the sam part anyways.
BTW, i saw that you guys added a vpc lattice model but no parser variation :(

Thank you for this great addition Ran!


PROPOSAL: Improve experience when retrying MultiPart request with Files

Hello there!
We at Revelo use Retrofit and its generator for all of our http requests. We also use MultiPart a lot and have gotten a significant amount of errors when retrying to send a MultiPart File because the stream has already been listened to.
exception: Bad state: The FormData has already been finalized. This typically means you are using the same FormData in repeated requests.. Error thrown .
I thought of a possible way to make retrying these requests possible and it involves including the [original file path] in the multipart file. At first I thought about inserting it in the file's headers, but I don't know if this could be a security breach or interfere in any other processes. And well, even though it would require us to rebuild the [files] field of the FormData, it's already better than having to implement another class just to deal with the paths of all of the files we're inserting and managing this in local storage.
To make this change, we would ideally add a new parameter in the Part() class, like [keepOriginalPath], which would be a bool. We could then use this bool in the generator to insert the path in the [headers] field's map when generating the code, if this field is given as true.
With this, we could retrieve the path in the RequestOptions object, and would not need to mess with Dio's structures as well.
WDYT? Would this ideia be ok? I was able to make it work properly but the tests got me confused, I would love some help from the team if I could have it and the idea makes sense to you.
Thanks a lot, best regards!

Usage example
Generated Code example

This is a common use case for ourselves (and we think for others as well) due to access token's short duration. As it expires, we automatically renew it with the refresh token, and finally do the original request again, that's why we fall into this problem.

Thanks for the contribution. Please make sure all tests are passed.

Thanks for the proposal. And It should be an issue if you put the file path in the request header?
How about to add a extension function to provider a clone method to get a copy of FormData

Thanks for the proposal. And It should be an issue if you put the file path in the request header?
How about to add a extension function to provider a clone method to get a copy of FormData

@trevorwang thanks for the review. My take on both points:

If we put it in the request header I don't reckon it would be the best practice, since a request may have several files attached to it and it would be difficult to know which path is for each file. I'd rather put it in each file's header if this is okay. WDYT?

About the extension function, do you mean something like this?

import 'package:dio/dio.dart';

extension FormDataExtension on FormData {
  FormData clone() {
    final formData = FormData();
    for (final fieldEntry in fields) {
      formData.fields.add(MapEntry(fieldEntry.key, fieldEntry.value));
    }
    for (final fileEntry in files) {
      MultipartFile newMultipartFile = fileEntry.value;
      final filePath = newMultipartFile.headers?['original_file_path']?.first;
      if (filePath != null) {
        newMultipartFile = MultipartFile.fromFileSync(
          filePath,
          filename: newMultipartFile.filename,
          contentType: newMultipartFile.contentType,
          headers: newMultipartFile.headers,
        );
      }
      formData.files.add(MapEntry(fileEntry.key, newMultipartFile));
    }
    return formData;
  }
}

Also, I feel like I'll need some help understanding the tests. Is there any way I could have some of your time to guide me through it? Or could you guide me async? I've tried some (actually quite a lot) of things but they don't seem to work, even though the code itself is working. Sorry for bugging you with that :(

@trevorwang tests should be working now :) let me know if you want me to include that FormDataExtension in the project 🚀

Thanks for the proposal. And It should be an issue if you put the file path in the request header?
How about to add a extension function to provider a clone method to get a copy of FormData

@trevorwang thanks for the review. My take on both points:

If we put it in the request header I don't reckon it would be the best practice, since a request may have several files attached to it and it would be difficult to know which path is for each file. I'd rather put it in each file's header if this is okay. WDYT?
About the extension function, do you mean something like this?

import 'package:dio/dio.dart';

extension FormDataExtension on FormData {
  FormData clone() {
    final formData = FormData();
    for (final fieldEntry in fields) {
      formData.fields.add(MapEntry(fieldEntry.key, fieldEntry.value));
    }
    for (final fileEntry in files) {
      MultipartFile newMultipartFile = fileEntry.value;
      final filePath = newMultipartFile.headers?['original_file_path']?.first;
      if (filePath != null) {
        newMultipartFile = MultipartFile.fromFileSync(
          filePath,
          filename: newMultipartFile.filename,
          contentType: newMultipartFile.contentType,
          headers: newMultipartFile.headers,
        );
      }
      formData.files.add(MapEntry(fileEntry.key, newMultipartFile));
    }
    return formData;
  }
}

Also, I feel like I'll need some help understanding the tests. Is there any way I could have some of your time to guide me through it? Or could you guide me async? I've tried some (actually quite a lot) of things but they don't seem to work, even though the code itself is working. Sorry for bugging you with that :(

Does it fix your issue with extension or the new header is still needed to be added?

Thanks for the proposal. And It should be an issue if you put the file path in the request header?
How about to add a extension function to provider a clone method to get a copy of FormData

@trevorwang thanks for the review. My take on both points:

If we put it in the request header I don't reckon it would be the best practice, since a request may have several files attached to it and it would be difficult to know which path is for each file. I'd rather put it in each file's header if this is okay. WDYT?
About the extension function, do you mean something like this?

import 'package:dio/dio.dart';

extension FormDataExtension on FormData {
  FormData clone() {
    final formData = FormData();
    for (final fieldEntry in fields) {
      formData.fields.add(MapEntry(fieldEntry.key, fieldEntry.value));
    }
    for (final fileEntry in files) {
      MultipartFile newMultipartFile = fileEntry.value;
      final filePath = newMultipartFile.headers?['original_file_path']?.first;
      if (filePath != null) {
        newMultipartFile = MultipartFile.fromFileSync(
          filePath,
          filename: newMultipartFile.filename,
          contentType: newMultipartFile.contentType,
          headers: newMultipartFile.headers,
        );
      }
      formData.files.add(MapEntry(fileEntry.key, newMultipartFile));
    }
    return formData;
  }
}

Also, I feel like I'll need some help understanding the tests. Is there any way I could have some of your time to guide me through it? Or could you guide me async? I've tried some (actually quite a lot) of things but they don't seem to work, even though the code itself is working. Sorry for bugging you with that :(

Does it fix your issue with extension or the new header is still needed to be added?

It alone won't fix the issue, please notice it uses the header in final filePath = newMultipartFile.headers?['original_file_path']?.first; to get the path and restore the MultipartFile. I believe this is a nice addition to he library, WDYT?

@trevorwang do you have an idea of what's going on with this failing test? I tried everything I could but I can't find a way to make it pass. I thought @kzrnm 's PR would solve it :(

Thanks for the proposal. And It should be an issue if you put the file path in the request header?
How about to add a extension function to provider a clone method to get a copy of FormData

@trevorwang thanks for the review. My take on both points:

If we put it in the request header I don't reckon it would be the best practice, since a request may have several files attached to it and it would be difficult to know which path is for each file. I'd rather put it in each file's header if this is okay. WDYT?
About the extension function, do you mean something like this?

import 'package:dio/dio.dart';

extension FormDataExtension on FormData {
  FormData clone() {
    final formData = FormData();
    for (final fieldEntry in fields) {
      formData.fields.add(MapEntry(fieldEntry.key, fieldEntry.value));
    }
    for (final fileEntry in files) {
      MultipartFile newMultipartFile = fileEntry.value;
      final filePath = newMultipartFile.headers?['original_file_path']?.first;
      if (filePath != null) {
        newMultipartFile = MultipartFile.fromFileSync(
          filePath,
          filename: newMultipartFile.filename,
          contentType: newMultipartFile.contentType,
          headers: newMultipartFile.headers,
        );
      }
      formData.files.add(MapEntry(fileEntry.key, newMultipartFile));
    }
    return formData;
  }
}

Also, I feel like I'll need some help understanding the tests. Is there any way I could have some of your time to guide me through it? Or could you guide me async? I've tried some (actually quite a lot) of things but they don't seem to work, even though the code itself is working. Sorry for bugging you with that :(

Does it fix your issue with extension or the new header is still needed to be added?

It alone won't fix the issue, please notice it uses the header in final filePath = newMultipartFile.headers?['original_file_path']?.first; to get the path and restore the MultipartFile. I believe this is a nice addition to he library, WDYT?

Sorry I misunderstood about above information.  Extension is not necessary here.
How about to handle multiple files?  What if there 3 MultipartFiles in the request

Thanks for the proposal. And It should be an issue if you put the file path in the request header?
How about to add a extension function to provider a clone method to get a copy of FormData

@trevorwang thanks for the review. My take on both points:

If we put it in the request header I don't reckon it would be the best practice, since a request may have several files attached to it and it would be difficult to know which path is for each file. I'd rather put it in each file's header if this is okay. WDYT?
About the extension function, do you mean something like this?

import 'package:dio/dio.dart';

extension FormDataExtension on FormData {
  FormData clone() {
    final formData = FormData();
    for (final fieldEntry in fields) {
      formData.fields.add(MapEntry(fieldEntry.key, fieldEntry.value));
    }
    for (final fileEntry in files) {
      MultipartFile newMultipartFile = fileEntry.value;
      final filePath = newMultipartFile.headers?['original_file_path']?.first;
      if (filePath != null) {
        newMultipartFile = MultipartFile.fromFileSync(
          filePath,
          filename: newMultipartFile.filename,
          contentType: newMultipartFile.contentType,
          headers: newMultipartFile.headers,
        );
      }
      formData.files.add(MapEntry(fileEntry.key, newMultipartFile));
    }
    return formData;
  }
}

Also, I feel like I'll need some help understanding the tests. Is there any way I could have some of your time to guide me through it? Or could you guide me async? I've tried some (actually quite a lot) of things but they don't seem to work, even though the code itself is working. Sorry for bugging you with that :(

Does it fix your issue with extension or the new header is still needed to be added?

It alone won't fix the issue, please notice it uses the header in final filePath = newMultipartFile.headers?['original_file_path']?.first; to get the path and restore the MultipartFile. I believe this is a nice addition to he library, WDYT?

Sorry I misunderstood about above information. Extension is not necessary here. How about to handle multiple files? What if there 3 MultipartFiles in the request

Thanks for pointing this out. I have tried generating from a list of files and it's gotten me an error I'm not sure how to handle, the generating method is different from the individual one. Here's what I did:
1- I set the keepFilePath as true in this test:
@RestApi()
abstract class TestFileList {
  @POST('/')
  Future<void> testFileList(@Part(keepFilePath: true) List<File> files);

  @POST('/')
  Future<void> testOptionalFileList(@Part() List<File>? files);

  @POST('/')
  Future<void> testOptionalFile({@Part() File file});
}

2- I got this error and could not solve it, I believe the generation is not working:
Could not format because the source could not be parsed:

line 17, column 21 of .: Expected an identifier.
   ╷
17 │                     ,
   │                     ^
   ╵
line 18, column 62 of .: Expected to find ','.
   ╷
18 │                     headers: {{original_file_path: [Instance of 'BinaryExpression']}},
   │                                                              ^^
   ╵
line 18, column 65 of .: Expected to find ','.
   ╷
18 │                     headers: {{original_file_path: [Instance of 'BinaryExpression']}},
   │                                                                 ^^^^^^^^^^^^^^^^^^
   ╵
line 32, column 21 of .: Expected an identifier.
   ╷
32 │                     ,
   │                     ^
   ╵
package:dart_style/src/dart_formatter.dart 141:7                DartFormatter.formatSource
package:dart_style/src/dart_formatter.dart 73:12                DartFormatter.format
package:retrofit_generator/src/generator.dart 119:10            RetrofitGenerator._implementClass
package:retrofit_generator/src/generator.dart 76:12             RetrofitGenerator.generateForAnnotatedElement
package:source_gen_test/src/generate_for_element.dart 88:15     generateForElement
package:source_gen_test/src/test_annotated_classes.dart 237:7   AnnotatedTest._generate
package:source_gen_test/src/test_annotated_classes.dart 240:26  AnnotatedTest._shouldGenerateTest

3- I believe the error is somewhere here (lines 1818 to 1838), but could not correct it:
else if (_typeChecker(List).isExactlyType(p.type) ||
            _typeChecker(BuiltList).isExactlyType(p.type)) {
          final innerType = _genericOf(p.type);

          if (_displayString(innerType) == 'List<int>') {
            final fileName = r.peek('fileName')?.stringValue;
            final conType = contentType == null
                ? ''
                : 'contentType: MediaType.parse(${literal(contentType)}),';
            blocks.add(
              refer(dataVar).property('files').property('addAll').call([
                refer('''
                  ${p.displayName}.map((i) => MapEntry(
                '$fieldName',
                MultipartFile.fromBytes(i,
                    filename:${literal(fileName)},
                    $conType,
                    )))
                  ''')
              ]).statement,
            );

Do you have any insight on what I could do here?

You have to use literalList instead of square brackets
  final headers = keepFilePath
                ? literalMap({
                    'original_file_path': [filePath]
                  })
                : '{}';

Hi @gabrielaraujoz
This should be a more ideal way to fix this reuse issue
Future<void> _repeatedlyRequest() async {
  Future<FormData> createFormData() async {
    return FormData.fromMap({
      'name': 'dio',
      'date': DateTime.now().toIso8601String(),
      'file': await MultipartFile.fromFile('./text.txt',filename: 'upload.txt'),
    });
  }

  await dio.post('some-url', data: await createFormData());
}

How about generate this kind of code for Multipart annotation?

Hi @gabrielaraujoz
This should be a more ideal way to fix this reuse issue
Future<void> _repeatedlyRequest() async {
  Future<FormData> createFormData() async {
    return FormData.fromMap({
      'name': 'dio',
      'date': DateTime.now().toIso8601String(),
      'file': await MultipartFile.fromFile('./text.txt',filename: 'upload.txt'),
    });
  }

  await dio.post('some-url', data: await createFormData());
}

How about generate this kind of code for Multipart annotation?

Hey @trevorwang, just fyi, I spent a lot of time trying to figure out the files list generation but was not able to make the tests pass.
Do you mean generate a public method createFormData in the generated file? After we do that, we would have to import almost every generated file with multipart to our error interceptor and then use the method to recreate the request. IDK if this is the ideal approach, since I'd rather have it without the need of the generated files dependency.
In my honest opinion, we could solve it only in the dio package but then we would not support people who use retrofit and make direct http requests without it. That's why I initially thought of the headers parameters to make it easy to regenerate the multipart file.
WDYT? Also, I believe I'm close to figuring out the texts and it's some synthax I don't yet quite understand. Let me know if you have 5 minutes to spare so that we could maybe figure this out synchronously.


Using MU-Migration on the WP VIP local environment

Describe your question
Hello,
I am having difficulties to successfully export a subsite from a local VIP env multisite.
This is how my command looks like:
vip dev-env --slug=my-website --debug @automattic/vip:bin:dev-environment exec -- wp mu-migration export all subsite.zip --blog_id=1
Does it maybe have something to do with the location of the wp-config.php file which is unconventional in the WP VIP sites?

Code of Conduct

[X] I agree to follow this project's Code of Conduct

There were some plugin tables that were not exportable with this plugin. Excluded them during export with --non-default-tables, and it worked.


Support for x-tag-names

I haven't looked through the entire code base, but in the tabs module I found that the checks for tab-heading look for an attribute called tab-heading or data-tab-heading or a tag also called tab-heading or data-tab-heading. The correct prefixed version of the tag name should probably be x-tab-heading (which is what web components will use), not data-tab-heading (which seems to have no basis as a tag name in the current HTML spec).
AngularJS is less strict about the distinction and simply strips x- and data- prefixes from attribute names and tag names alike. AngularUI should probably adopt this behaviour and simply accept x- prefixed attribute names and tag names in addition to the data- prefixed ones it already handles.
Maybe this logic could be factored out into a utility function.

Any examples of real use cases to justify the suggestions?

Angular spec for directive attributes support:
<span ng:bind="a" ng-bind="a" data-ng-bind="a" x-ng-bind="a">

Looking at the angular code, $normalize calls directiveNormalize which calls camelCase:
function camelCase(name) {
  return name.
    replace(SPECIAL_CHARS_REGEXP, function(_, separator, letter, offset) {
      return offset ? letter.toUpperCase() : letter;
    }).
   replace(MOZ_HACK_REGEXP, 'Moz$1');
}

Tab heading is a bit of an oddball in that it is trying to support attribute and tag declaration, relevant code:
function isTabHeading(node) {
  return node.tagName &&  (
    node.hasAttribute('tab-heading') ||
    node.hasAttribute('data-tab-heading') ||
    node.tagName.toLowerCase() === 'tab-heading' ||
    node.tagName.toLowerCase() === 'data-tab-heading'
  );
}

I don't have a dog in this race any more because I have pretty much abandoned AngularJS at this point. If nobody else takes any interest, feel free to close this issue.

I would classify this as a bug since the tabheading should support x- notation according to the Angular spec as @RobJacobs stated.


death's oath

Death's oath aura dmg and self dmg seems not to be present. Gl with 3.0 updates.

I can't add the new aura skill until I get the patch data, unfortunately.

The next update adds support for the Death Aura skill.

add life penalty for Death Aura :)


JSHint problem in Sandcastle

Probably also broken in #3232
Running Sandcastle from master, I noticed is it won't give an error if you're trying to use a variable that hasn't been defined yet.  On the website, it will highlight that line as having an error.
ie, the line var foo = bar should be highlighted because bar isn't defined.
@mramato

Marking this a showstopper.  This was broken with #3210.


[BEAM-13945] add json type support for bigquery connector

(Currently in progress)
BigQuery now natively supports JSON data type [1]. This change adds support and integration tests for using the JSON data type with Java BigQueryIO read and write methods.
[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/json-data#query_json_data

Can I get a review? Would like to know if the input data covers the behavior we want as laid out in the BigQuery JSON documentation.
R: @ihji
R: @chamikaramj

Run PostCommit_Java_Dataflow

Run PostCommit_Java_Dataflow

Run PostCommit_Java_Dataflow

Still figuring out read/write permissions to BigQuery, but could I get a review for the new test data?
R: @chamikaramj

I don't see permissions errors anymore. I see this: https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV1_PR/89/testReport/junit/org.apache.beam.sdk.io.gcp.bigquery/BigQueryIOJsonIT/testExportRead/

@pabloem Thanks for running it and checking. I made a little fix to the table in apache-beam-testing, can you try again? Also, could the permission errors only be coming from my side?

Run PostCommit_Java_Dataflow

Run PostCommit_Java_Dataflow

Run PostCommit_Java_Dataflow

Run PostCommit_Java_Dataflow

retest  this please

Run PostCommit_Java_Dataflow

retest this please

Please update the "draft" status when this is ready to be reviews/merged.
Python changes should be a separate PR.

R: @chamikaramj
Marked ready for review. Will start working on tests for the Python SDK.

Run PostCommit_Java_DataflowV2

Run PostCommit_Java_DataflowV2

retest this please
Run PostCommit_Java_DataflowV2

Run PostCommit_Java_DataflowV2

R: @johnjcasey

Run PostCommit_Java_DataflowV2

Run PostCommit_Java_DataflowV2

Run PostCommit_Java_DataflowV2

R: @chamikaramj
R: @johnjcasey

Thanks for adding the check, looks good

Run PostCommit_Java_DataflowV2

Run PostCommit_Java_DataflowV2

retest this please

Run Java PreCommit

Run Java PreCommit

Run PostCommit_Java_DataflowV2

Hello, this broke the Java postcommit: https://issues.apache.org/jira/browse/BEAM-14356

Hello, this broke the Java postcommit: https://issues.apache.org/jira/browse/BEAM-14356

PR to fix this: https://github.com/apache/beam/pull/17446


#11247 inProgress scope is lost between different loaders.

Adding non-destructive onload and onerror callbacks

What kind of change does this PR introduce?
Changed LoadScriptRuntimeModule onload and onerror to non-destructively bind the onScriptComplete callback. With this change, existing callbacks will be retained in scope and executed in the order they were bound. This also ensures that all the "done" functions get called from the in-scope "inProgress".

Did you add tests for your changes?
No

Does this PR introduce a breaking change?
No

What needs to be documented once your changes are merged?
Nothing

What is a problem?

@evilebottnawi - not sure what the problem is, I'm new to webpack contributions. Should I not be working off the master?

looks like the Travis CI build does not like my reference to window for the global scope ... I should have probably looked a little deeper and used RuntimeGlobals?

@sokra  @evilebottnawi at this point I'm fumbling around to try to fix the test errors. Spent more time trying to get that working than I did identifying the root cause of the bug and purposing a fix.
Some guidance would be extremely helpful at this point.

@sokra  - thank you for your feedback. Between that and some sleep I found an approach that avoids global, while also being non-destructive to callbacks. :)
I updated the test snapshots and was able to run the various "test:*" scripts locally and passing. Travis and Appveyor would not run locally for me though. Not sure what I'm missing and why the checks are not passing here now...
That being said, if you are satisfied with the approach I took in LoadScriptRuntimeModule.js, I can close this PR and create a fresh branch with all the changes in one commit. I also don't like seeing "17 commits" in one PR...
On another note, I did sign the CLA - not sure why that is still saying its not signed.
Hoping I can contribute more smoothly in the future when I come across issues.

@jandvik Please do not mix your work with other commits :smile:

@evilebottnawi how else do you resolve conflicts beside a rebase? Please enlighten me. This is also why I offered to close this PR and move my changes to a fresh branch off webpack's latest to avoid conflicts and rebases. I couldn't exactly find a list of things to do in the process, more being told "dont do that" when I try to figure things out myself. Perhaps there is a more in-depth contribution guide I can read than the one in the repo root folder?

just do rebase, not merge :smile:

It forced me to do both 🤷♂

Would you tell me how to get te rest of the tests running locally so I can resolve the rest of the build issues, I'll move my changes to a fresh checkout and do it all in one clean commit so I can move on to my other work?


ATTACH SQLITE DATABASE: Create a SQLite database which alaSQL can read

Hello alaSQL team,
The main issue is that we cannot create a working example for the "ATTACH SQLITE DATABASE" method.
We would appreciate a working example.
Among others we have tried code below. It uses sql.js but that is not a real requirement for us. Plain javascript + alaSQL would also do.
The main purpose is to have the code working in iOS (iPhone) and Android.
//Create the database
var db = new SQL.Database(sqlTest01.sqlite);
// Run a query without reading the results
db.run("CREATE TABLE test (col1, col2);");
// Insert two rows: (1,111) and (2,222)
db.run("INSERT INTO test VALUES (?,?), (?,?)", [1,111,2,222]);

alasql('ATTACH SQLITE DATABASE sqlTest01("sqlTest01.sqlite");\
    USE sqlTest01; \
    SELECT * FROM test',[],function(res){
        console.log("result:",res.pop());
});

Kind regards,
Mario

@MarioVanDenEijnde
Is sqlTest01 an object with a property  sqlite ?
Should it be var db = new SQL.Database("sqlTest01.sqlite"); instead?

Hello Mathias,
Thanks but still leaves me with "file is encrypted or is not a database"
I thought maybe I have to enter also the path to the database but there is no way I can find out what that path is.
I have tested the following javascript code to write and read a SQLite database with success: https://developer.apple.com/library/safari/documentation/iPhone/Conceptual/SafariJSDatabaseGuide/UsingtheJavascriptDatabase/UsingtheJavascriptDatabase.html#//apple_ref/doc/uid/TP40007256-CH3-SW12
So the "only" thing left is have alaSQL be able to connect to the data(base).
Any help is appreciated.
Kind regards,
Mario

Hello Mathias,
Question: Why actually do you use sql.js and not just "plain" javascript to connect to the data(base)?
Kind regards,
Mario

Im confused - are you asking me why your coding example is as it is?

Hello Mathias,
What we try to achieve is to use alaSQL with SQLite. We tried using code from your wiki but also from a post you replied to. But we cannot succeed. That is why we call in for help.
So the question is can you help us with a working code sample (with or without sql.js).
Kind regards,
Mario

@MarioVanDenEijnde
SO you would like to store a db in a SQLite format?

Hello Mathias,
As I have understood alaSQL works in memory which makes queries and such faster.
But to rely only on in memory storage of data is not an option for us We want to have a persistent storage and on top of that use alaSQL for processing.
Because we also don't want to be limited in storage such as the 50Mb WebSQL (iOS) we want to use SQLite.
As I understand we need the SQLite adapter to use alaSQL on top of SQLite.
Storing data in SQLite format is not the real requirement. We don't need sql.js or even alaSQL for this but can use other javascript.
Does this make sense?
Kind regards,
Mario

Regarding WebSQL - are you thinking browser or an app based on javascript?

I cant test it here - but try this approach:
alasql('ATTACH SQLITE DATABASE mytable ("abc.sqlite")') ;
alasql('USE mytable;') ;
alasql("CREATE TABLE test (col1, col2);");
alasql("INSERT INTO test VALUES (?,?), (?,?)", [1,111,2,222]);
alasql("SELECT * FROM test",[],function(res){
        console.log("result:",res.pop());
});

I am having a very similar problem to @MarioVanDenEijnde. When following the method you describe here and elsewhere, using the Chinook_Sqlite.sqlite database, I get the following:
> var alasql = require( 'alasql' );
undefined
> alasql( 'ATTACH SQLITE DATABASE mytable ("Chinook_Sqlite.sqlite");' );
1
> ReferenceError: SQL is not defined
    at c:\nodejs\node_modules\alasql\dist\alasql.js:17373:31
    at c:\nodejs\node_modules\alasql\dist\alasql.js:3261:17
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)
> alasql( 'USE mytable;' );
1
> alasql( 'SELECT * FROM Genre', [], function( res ) { console.log( "Genres: ", res.pop() ); } );
TypeError: Cannot read property 'columns' of undefined
    at c:\nodejs\node_modules\alasql\dist\alasql.js:9494:13
    at Array.forEach (native)
    at yy.Select.compileDefCols (c:\nodejs\node_modules\alasql\dist\alasql.js:9485:13)
    at yy.Select.compile (c:\nodejs\node_modules\alasql\dist\alasql.js:7179:23)
    at Function.alasql.dexec (c:\nodejs\node_modules\alasql\dist\alasql.js:4225:38)
    at Function.alasql.exec (c:\nodejs\node_modules\alasql\dist\alasql.js:4190:17)
    at alasql (c:\nodejs\node_modules\alasql\dist\alasql.js:121:18)
    at repl:1:1
    at REPLServer.defaultEval (repl.js:164:27)
    at bound (domain.js:250:14)
>

I tried manually including sql.js, and all that happens is that it changes the error message I get following the ATTACH statement:
> TypeError: cb is not a function
    at c:\nodejs\node_modules\alasql\dist\alasql.js:17392:7
    at c:\nodejs\node_modules\alasql\dist\alasql.js:3261:17
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)
>

Thoughts?
Kurt

Hello Kurt,
Because I want to use sqlite for the local mobile database I needed a solution that works.
After some searching I found an excellent solution to work with sqlite:
https://github.com/SchizoDuckie/CreateReadUpdateDelete.js
I don't think it works the same as alasql but I needed to step forward.
I still use some of alasql query features though ;-)
Hpefully in the future alasql has got a working sqlite adapter and we will investigate again.
Kind regards
Mario

@MarioVanDenEijnde Good to know you found a solution that worked for you. AlaSQL has a lot to work - sorry I could not help you right now.

Hello Mathias,
Thanks. Good luck and keep me updated.
Kind regards,
Mario

Thanks, @MarioVanDenEijnde, I am looking at the solution you found as we speak.
To the AlaSQL team, AlaSQL is extremely impressive. Thanks for your hard work. I look forward to seeing how it evolves and gets even better.
Kurt

@kurtkincaid Thank you for taking the time to express your opinion :+1:

Hello Mathias,
Question: We have an sqlite database and we can query the database to deliver a json array.
We would like to use your "extend and merge two arrays containing objects" function.
This function delivers a VAR we can use to query and populate a search box list.
I am not familiar with the "in memory" way to handle this but am I right that it would work as follows:
1.) I check for updates in the sqlite database tables in function A (regularly or as part of a update function)
2.) When there is a table data change I will call the alasql function that updates the VAR
3.) I query VAR to populate the search box list.
And by doing it this way get a performance gain while executing the search query.
Please advise.
Kind regards,
Mario

I would love if you could write something about what you are trying to achieve

Hello Mathias,
We have sqlite data and we want to populate a search result list based on a search box entry filtering the sqlite data while we type. The search will be used multiple times after each other so I thought if we could "cache" the data in memory the search will execute faster.
That is basically the idea.
As I understand the "magic" of using alasql is it executes "in memory".
So I presumed the best way to do this is using alasql to generate and maintain the search data and use alasql queries the filter the result.
Is this more helpfull?
Kind regards,
Mario

We have sqlite data and we want to populate a search result list based on a search box entry filtering the sqlite data while we type.

so... sqlite is located on server and the search autosuggest is on a client somewhere?
How often is the central data updated? How long are the clients having the page open?

Hello Mathias,
No sorry. The data is on the client.
Kind regards,
Mario

@MarioVanDenEijnde Would it be possible for you to post your a working example of "ATTACH SQLITE DATABASE" so @kurtkincaid could get inspired?

No not yet. Because I have this approach only in my head.
But i presume the basic question should be clear by now and I kinda hoped you could answer this.
If I would be more basic the question would be "why did you build alasql?"
Again I thought the "magic" of using alasql is it executes "in memory" and therefore is faster.
Kind regards,
Mario

Hello Mathias,
I am trying to build a working example. Can you modify code below so I also can get columns from hotels in the result?
res = alasql('SELECT id, FIRST(Name) AS Name, COUNT(*) AS [Count] FROM ? facilities \
            JOIN ? hotels ON facilities.id IN hotels.WebFacilities \
            GROUP BY id', [facilities,hotels]);

I think I get it in your example: Changed id into to "FIRST(hotels.id) AS id"
But in my code below I get the message: p.favColl.ID_Food.indexOf is not a function
Can you advise please?
res = alasql('SELECT ID_Food, FIRST(favColl.id) AS id FROM ? foodColl \
JOIN ? favColl ON foodColl.ID_Food IN favColl.ID_Food \
GROUP BY ID_Food', [foodColl,favColl]);

Could it be that not in all favColl records the ID_Food is filled with data?

No I tested with only filled field data  ID_Food but the same error.

I mangeld your code a bit and now I get the same error. It look slike a simple join though.
Can you explain what is wrong?
     var hotels = [
        { id: 8, Name: "Hotel 101" },
        { id: 8, Name: "Hotel 101" },
        { id: 9, Name: "Hotel 101" }
    ];

    var facilities = [
        { id: 8, Name: "Facility 8" },
        { id: 9, Name: "Facility 9" },
        { id: 10, Name: "Facility 10" }
    ];

    var res = alasql('SELECT FIRST(hotels.id) AS id, FIRST(Name) AS Name FROM ? facilities JOIN ? hotels ON facilities.id IN hotels.id GROUP BY id', [facilities,hotels]);

I would really like to help you, but I need to understand your data.
So you have a table of hotelts and a table of facilities
is it an autosuggest you are trying to make?
Are you trying to have a autosuggest / selection of facilities after you have selected a hotel?

Thanks. Can you write the join / aggr for below? the join = id
var arr1 = [{
    id: 1,
    name: 'fred',
    title: 'boss'
}];

var arr2 = [{
    id: 1,
    uid: 1
},
    id: 1,
    uid: 2
}];

And the result should be:
var res= [{
    id: 1,
    name: 'fred',
    title: 'boss',
    uid: 1 (or 2)
}];

SELECT * from arr1 JOIN arr2 ON id Where 1 LIMIT 1

(not really a good answer as you you are not in controle if its uid1 or 2 - but really need more information to help you out)

I hoped you would drop in a Group with an aggregate with FIRST or MAX

@MarioVanDenEijnde I cant help you if you are not providing me the info about what you are doing.

OK. The thing is that the joins I have tested will return 2 objects but I only want 1. I only want the rows of arr1 extended with an item in arr2.
So your 1 LIMIT 1 example gives me exactly that but I want to have a little more playroom.
So If you can give me a group and aggregate I can chhose myself if I want the 2 or the 1.

You hold me in the dark.
So given a uid, you would like to have the data from both arr1 and arr2?

I hoped you would drop in a Group with an aggregate with FIRST or MAX
it does not make sense to make a GROUP if data does not need to agregate. Please provide meaningful data and a question formulated as "Given I have the value XXXX of the field YYYY, how can I get the output to be ZZZZZ"

I am not trying to hold you in the dark. Just want to know how to do it. And from that on I can use it in different ways.
I tried the 1 LIMIT 1 but that is not what I need. It limits the result in only 1 row.
When I update the demo data it would look like below. Does this make more sense?
var arr1 = [{
    id: 1,
    name: 'fred',
    title: 'boss'
},{
    id: 2,
    name: 'max',
    title: 'bossanova'
}];

var arr2 = [{
    id: 1,
    uid: 1
},{
    id: 1,
    uid: 2
},{
    id: 2,
    uid: 3
}];

And the result should be:
var res= [{
    id: 1,
    name: 'fred',
    title: 'boss',
    uid: 1 (or 2) -> MAX or MIN
},{
    id: 2,
    name: 'fred',
    title: 'boss',
    uid: 3
}];

Sorry. The res should be:
var res= [{
    id: 1,
    name: 'fred',
    title: 'boss',
    uid: 1 (or 2) -> MAX or MIN
},{
    id: 2,
    name: 'max',
    title: 'bossanova',
    uid: 3
}];

Please provide meaningful data and a question formulated as "Given I have the value XXXX of the field YYYY, how can I get the output to be ZZZZZ"

I see a XXXX and a ZZZZ - but what is YYYYY?
(are you aware that you can edit your posts in this thread by clicking on the little pen icon at the top right corner of each post?)

Hum. I think I cannot make it any easier but you are missing something.

Would it help if I would ask for a group/aggregate function together with a join?

OK - I think I get it - you are trying to

Find the whole data for the row with some max value in a column per some group identifier

Try this approach
var res = alasql('WITH all AS ( SELECT * from $arr1 JOIN $arr2 ON id ) SELECT a.* FROM all a
LEFT OUTER JOIN all b ON a.id = b.id AND a.uid < b.uid WHERE b.id IS NULL;', {arr1:arr1, arr2:arr2});

If its not working have a look on google for something like "SQL select rows with max value"

Hello Mathias,
It seems the LEFT OUTER JOIN works as a FULL OUTER JOIN.
Can we fix this?
Kind regards,
Mario

Is this a New issue or the same? If its New I would really really livebook you could create a New issue with samplecode of what you experience and what result you expected to support the bug hunt.

Hello Mathias,
Before answering that question I have to excuse myself. The issue is not an issue. I included a field in the select from table B which has got multiple values and thus returns multiple rows.
Thanks for the quick response.
Kind regards,
Mario

No problem