FormData send does not work

the error message states that body.keys() is not a function. Looking at FormData prototype in safari reveals that the only function FormData has implemented is append and keys() is indeed missing.
Thankfully, I found a polyfill that adds it at https://github.com/jimmywarting/FormData
For completeness, you may want to include this in your project though, or at least link to it in the readme.

Good point.  Thanks for the feedback.

The polyfill I linked to has problems when used with this plugin. After trying different combinations, the working solution involved these two steps:
a) once the form is ready, call fd._blob() to get a Blob and pass that as xhr request data
b) take the content type of that blob and set that as the contentType header
Im guessing setting the contentType header to undefined will make it default to the content type set for the blob but I did not test that. If you do decide to integrate that polyfill, do pay attention to this though ;)

Also, on iOS the polyfill _.blob function is currently broken but this is easy to hotfix around by always using the placeholder "File" definition he provides and renaming it to BlobWrapper in his implementation (https://github.com/jimmywarting/FormData/issues/26)

@velochy Just checked in a FormData polyfill.  This is our own impl that works with the xhr polyfill. Please help me verify the fix.

@gvanmat Thank you! I actually had a working workaround but the code was ugly to say the least so if this works, I am really grateful! I will try to find time to check it over the coming week.

@velochy Just pushed in a few fixes to the FormData polyfill.  Missed the forEach method and had a couple more typos.

@gvanmat Sadly, it is still not posting the data properly.
I'm using AngularJS $http (wrapper for XmlHttpRequest) to post (roughly) the following:
formData = new FormData();
formData.append('nonce','xyz');
formData.append('file',Blob(['xyz']),'file.bin')
requestParams = {
method: 'POST',
url: url,
data: formData,
headers: { 'Content-Type': undefined},
withCredentials: true
};
What my server endpoint (Flask in Python) sees is:
HEADERS: [('Cookie', u'session=.eJw1j8FqwzAQRH-l7LmHRtQXQw8uMsaHtbBRK3YvoU3UOLIVipNgSSH_XrXQ2zDwmDc32H4t9jxCeVmu9hG2xz2UN3j4hBJYtwJNH8gNHtMhonydMc2O9XBEN8zc1JFMu5LvXGdQsGFPjp6UxkLJ_YiJJ3TVMzuMnaOk9PtIqY3o69g1tcjcRGIYc07odoH1VLDuM3_YsMmd6NfMBpa0od9NURdK70T2WlXzFrjpBcoqKFm9wD27f9vFf5zs6fL_5nq2y98jKOD-A3VATo0.DOYF8Q.-ceAB5JZx18GeGdF3EcbFqJ4nFU; remember_token=WyI1IiwiZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2UiXQ.DOXrOA.hoto_vO7QbdS7KrrQ1g5AbKlq9E'), ('Content-Length', u'11'), ('User-Agent', u'Achieve%20Music/1.1.6 CFNetwork/887 Darwin/17.0.0'), ('X-Forwarded-Proto', u'https'), ('Host', u'mms.achievemusic.com'), ('Accept', u'application/json, text/plain, /'), ('Accept-Language', u'en-us'), ('Content-Type', u'text/plain;'), ('Accept-Encoding', u'br, gzip, deflate')]
DATA: '{"_map":{}}'
In short, it seems it is not really converting it to the multipart format as it should.
Let me know if I can help with any more debug information.

The FormData is going to send multipart/form data.  The content-type of the request will contain a boundary token the back-end uses to parse the document parts.  So, you will want to remove the content-type header on the request.
However, it looks to me like angularjs $http service is calling JSON.stringify on the formData object versus passing it on the the XMLHttpRequest.send(formData).
What if you tried using the native XMLHttpRequest here just to help isolate the fault. Something like this:
formData = new FormData();
formData.append('nonce','xyz');
formData.append('file',Blob(['xyz']),'file.bin')
var myhttp = new Promise(function (resolve, reject)
{
  var xhr = new XMLHttpRequest();
  xhr.onerror = function (error) {reject(xhr.responseText)});
  xhr.onloadend = function (error) {reject(xhr.responseText)});
  xhr.open("POST", url);
  xhr.send(formData);
})

myhttp.then(function (text)
{
)).catch(function (error)
{
});

Or, it could be the call to identify the body/object passed to the send method is failing to see it as a FormData prototype - FormData.prototype.isPrototypeOf(body).
At any rate, removing the request content-type header will help isolate.  If the FormData prototype check is failing, the server should see a request content-type of "application/json".  It plugins in default request header content-types if not provided.

@gvanmat it seems you are right regarding serialization. https://docs.angularjs.org/api/ng/service/$http "Default transformations" describes exactly that behavior.
Its easy to disable though. will test this tomorrow.
Considering the popularity of angularjs, you may want to make a note about it in the readme if it does work, though

@velochy I hope that's it.  Please let me know.  I suspect angular would have the same issue if you passed in a native javascript FormData.  For example, from a safari console:
var fd = new FormData(); fd.append("foo", "bar"); JSON.stringify(fd);
Would result in "{}".
Since the Blob and File types need the FileReader to convert into a binary string, the conversion is async.  I would have liked to return the multipart body via toString() but needs a promise for the async conversion.

@gvanmat it got us a step forward but there is a new problem. The post is now properly parsed on the server, but the binary file being transmitted comes out broken.
The file going into the FormData object in JS is a binary blob that, in base64 is:
AAAABSoAAAAAAAABEwAAAAcAAAAuAAAAhQAAAAIAAALwAAAAmgAAACYBAAEAAQABAAEAAAABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAgAAACeur6+zs76+wMDNzbm5mJh+fn19dnZzc3Fxbm5zc3V1dnZ2dnd3dXRAuIyaAAAAmggYJEWQ3v/gjDEA/BdjvNaaPggBHGS82aFFDgYZVa7Xp0wQAxNNqdirTxAEDBIRDw8RExQYHR4hNFVsbmRbW2NnXEYxKSclKTEyJRgXHSEiKjAuKSkyPkZOVFFDMiorLiwnIB4nND5LVldPRkE2IxQRGR8jLjg5MCMeISg0QEVDQkRFSUo9Jx0hJSYqLTM3NjMvLi0lGxYYGRgAAAFAKgAAAAEAAAALAAAB/////6WHdcs=
What the server receives is a binary file that is somewhat changed (again, base64):
AAAABSoAAAAAAAABEwAAAAcAAAAuAAAAwoUAAAACAAACw7AAAADCmgAAACYBAAEAAQABAAEAAAABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAgAAACfCrsKvwq/Cs8Kzwr7CvsOAw4DDjcONwrnCucKYwph+fn19dnZzc3Fxbm5zc3V1dnZ2dnd3dXRAwrjCjMKaAAAAwpoIGCRFwpDDnsO/w6DCjDEAw7wXY8K8w5bCmj4IARxkwrzDmcKhRQ4GGVXCrsOXwqdMEAMTTcKpw5jCq08QBAwSEQ8PERMUGB0eITRVbG5kW1tjZ1xGMSknJSkxMiUYFx0hIiowLikpMj5GTlRRQzIqKy4sJyAeJzQ+S1ZXT0ZBNiMUERkfIy44OTAjHiEoNEBFQ0JERUlKPScdISUmKi0zNzYzLy4tJRsWGBkYAAABQCoAAAABAAAACwAAAcO/w7/Dv8O/wqXCh3XDiw==
Since the multipart encoding works now, and other FormData parameters are properly passed, I'm inclined to think this is not related to angular. My best bet would be that the string gets scrambled being sent from javascript to native code.
Regarding AngularJS - it does not have the same issue with native FormData, presumably because it typechecks it somehow. So you may still want to add a line to the readme saying that for AngularJS (and potentially other frameworks) you may need to disable default transformations (for AngularJS, it means passing an extra option transformRequest: function(x) {return x;} to $http calls)

When you create your blob, what context-type are you assigning to the blob?  I'll do some more investigating with different file types.  I'm using the Java EE multipart form servlet in my testing so I know the server side handling is correct and should catch any issues.
If angular is checking the prototype of the data object, it should also work the same since we have replaced the window.FormData with our own instance.

@velochy I'm a fan of angularJS too.  I like the simplicity of creating components and the separation of  controllers with inject-able services.
I was looking in the $http service and you are correct about special handling of the FormData.  I found the isFormData  function and see what I'm doing wrong.
The FormData polyfill toString returns "[object, FormData]" but this code is looking for "[object FormData]".  No comma.  I can fix that up so there is no need for any special code.

@gvanmat regarding blob content type - i do not set it explicitly and it seems to default to binary/octet-stream.
The blob is created pretty directly from base64 data in my own code so you should be able to reproduce with
FD.append ('file',new Blob([btoa ('')]),'file.bin')
Hope it helps.

@gvanmat any news on the blobs getting mangled issue? Is there any other information I could provide?

@velochy I wasn't able to recreate the issue.  I created a test case.  I'm uploading several blobs in the FormData.  The one that is base64 encoded returns the expected result.
What other variables do you have in your usecase outside of the angular http service?

@gvanmat thank you for testing it out. I will ask one of my colleagues for another pair of eyes to see if we can figure out what might be the reason. The fact is that the same code works in desktop browsers and android webviews so it has to be something ios related, but it may well be somewhere in our code. Will keep you posted.

@gvanmat just to make sure: did you test with the same b64 encoded string i sent? The test case you link has a simple ascii string which might not reveal encoding-related issues.

@velochy  I really appreciate your help verifying the FormData polyfill.  I'm pretty sure I have it this time.  Please try the latest again.

I can confirm it does indeed. Thank you for fixing it, @gvanmat. Great plugin and keep up the good work.

Note to any others who land here, this plugin breaks my signed AWS S3 uploads with blobs in FormData. I receive a request denied error from Amazon where something from the FormData signed request / file is lost in transmission. Have not spent much time digging though.
After removing this again, everything works perfectly (and vice versa). If it is of interest or affects others, I can try and set up a test case. Relevant or not, I am using $.ajax to upload the files and my own FormData object (with processData: false).

The plugin has a polyfill for FormData.  Sending the form data will result in the request being a multipart document.  I suspect AWS expects a multipart form.
Could you try using the XMLHttpRequest instead of the jquery wrapper function? It would be good to understand if it's an issue with the jquery integration or the plugin code.  Using the XMLHttpRequest isn't that much more verbose.
In addition, you might also take a couple seconds and enable plugin logging.

Thanks @gvanmat - Quick test shows this to be working fine using the XMLHttpRequest. For some reason I thought I tried this before but must not have!

I suspect the jQuery code looks at the prototype of the FormData or a toString check to identify it over a plain json object and the Pollyfill doesn't pass the check. Thanks for verifying the request object works.


Faster Implementation of Wildcard Matching and added test file for wildcard matching

Issue #, if available:
Description of changes:

Faster Implementation of Wildcard matching
Added test file for wildcard_match function

I have tested this change with the newly added test function.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #178 into master will decrease coverage by 0.07%.
The diff coverage is 96.55%.

@@            Coverage Diff             @@
##           master     #178      +/-   ##
==========================================
- Coverage    83.1%   83.03%   -0.08%
==========================================
  Files          77       77
  Lines        2918     2923       +5
==========================================
+ Hits         2425     2427       +2
- Misses        493      496       +3

Impacted Files
Coverage Δ

aws_xray_sdk/core/utils/search_pattern.py
66.66% <96.55%> (-22.47%)
:arrow_down:

aws_xray_sdk/core/patcher.py
87.2% <0%> (+1.6%)
:arrow_up:

aws_xray_sdk/ext/httplib/patch.py
75.78% <0%> (+3.15%)
:arrow_up:

aws_xray_sdk/core/utils/compat.py
95.23% <0%> (+9.52%)
:arrow_up:

aws_xray_sdk/core/sampling/local/reservoir.py
100% <0%> (+11.11%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 372efaa...0606a15. Read the comment docs.

It looks like code coverage decreased. Can we make sure all code paths are executed in your tests? Additionally, it would be nice if we could get a high level description of the algorithm in the comments for readability purposes.
And can we also confirm that these tests still pass before this change and after the change?

Tests have passed in both the version before this change and after this change. Moreover, I have added description of the algorithm. Initially code coverage was decreased because in the older version we are using simple_wildcard_change which we do not require in this version so I have removed that function.


Prepare for 1.2.1 Release

Hello @majelbstoat, @kylehg, @mikkot,
Please review the following commits I made in branch 'chad-1.2.1-release'.
18509010b4b33daf838b3d2525bf29ecaddd2fa6 (2016-02-16 14:52:22 -0800)
Prepare for 1.2.1 Release
R=@majelbstoat
R=@kylehg
R=@mikkot

Need to close this. The diff is inaccurate.

@majelbstoat, @kylehg, @mikkot: Ready for a review now. The diffs look much better not. Please take a look when you get the chance.

LG


Can't connect to PostgreSQL via SSL

Using version 6.1.1 (latest at time of writing)
I'm trying to connect to a PostgreSQL on Digital Ocean but always get the same error: SSL error: handshake_failure.
I have tried many different variations of the settings but to no avail. For example, I set SSL to true and the mode to require, ignoring the other fields, and I get the error.
I tried using PGAdmin and the connection is successful with the exact same configuration (turning SSL on with mode "require" and not specifying anything else when it comes to certificates and whatnot).
I wanted to use DBeaver but I can't seem to get past this error.
Thanks!
Edit: I have tried setting the SSL factory as mentioned here https://github.com/dbeaver/dbeaver/issues/2133#issuecomment-328644614
I have also tried editing the SSL settings on the driver settings directly, instead of just turning SSL on in the SSL tab.

Same error on windows platform. My postgresql server is 11.4 on linux .

Same error even with the direct connection w/o SSL.
DBeaver Community 6.1.1
Postgres 11.4 on Ubuntu 18.04.2.
PS. Downgrading to 6.1.0 fixed the problem.

got the same error for 6.1.1
downgraded to 6.1.0 made it work again?

Seems to be a result of Java 11 migration..

Actually I can't reproduce the issue on Windows 10.
Could you post error/debug log (https://github.com/dbeaver/dbeaver/wiki/Log-files) with this error?

6.1.1.zip
DBeaver 6.1.1  (clean install, no .dbeaver* or .eclipse folders in home)
Direct PostgreSQL connection - SSH and SSL are disabled
Windows 10 x64 1903
PostgreSQL 11.4
Ubuntu 18.04.2

Hello all.
We checked the SSL connection in version 6.1.1 on our dev environment with self-signed certificate and sslmode verify-ca and it works.
Can you give us more detail about connection mode and cert types (encription strength etc).
Thanks,
Andrew.

I get the error even if I don't have ssl enabled using DBeave4 6.1.1
Client: Windows 10 1903 x64
Server: Ubuntu 18 04 PostgreSQL 11
Get Outlook for Androidhttps://aka.ms/ghei36

From: Andrew<EMAIL_ADDRESS>Sent: Wednesday, June 26, 2019 6:22:46 PM
To: dbeaver/dbeaver
Cc: Nicko Porras; Comment
Subject: Re: [dbeaver/dbeaver] Can't connect to PostgreSQL via SSL (#6148)
Hello all.
We checked the SSL connection in version 6.1.1 on our dev environment with self-signed certificate and sslmode verify-ca and it works.
Can you give us more detail about connection mode and cert types (encription strength etc).
Thanks,
Andrew.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/dbeaver/dbeaver/issues/6148?email_source=notifications&email_token=AMCM52QCM6BTVS6XWXHGGIDP4M7PNA5CNFSM4H3A5RHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYTCERQ#issuecomment-505815622, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMCM52S3GNQYX7BDK4DTOGLP4M7PNANCNFSM4H3A5RHA.

@appsnporras this is because of SSH tunnel issues I think.
Anyway, this looks like this issue: https://support.cloudbees.com/hc/en-us/articles/115003785531-SSL-handshake-failures-due-to-incompatible-Cipher-Suite
Perhaps this happens because we didn't enable Unlimited Stgength Policy in the bundled JDK 11.
Trying to figure out how to fix this..

@NekoKoneko according to the log you have SSL enabled. This is odd as I don't see ssl settings in your datasoures.xml file.
Maybe SSL is required by the server-side.
Anyhow this seems to be the same problem as in other cases - lack of encryption configuration in JDK11.

Please check how it works in Early Access version (https://dbeaver.io/files/ea/).
JDK build was enhanced with extra encryption modules.
Note: only Windows installer was patched.

Here is the log file.

Here is the log file
dbeaver4.zip

Please check how it works in Early Access version (https://dbeaver.io/files/ea/).
JDK build was enhanced with extra encryption modules.
Note: only Windows installer was patched.
Here is the log file.
dbeaver4.zip

Please check how it works in Early Access version (https://dbeaver.io/files/ea/).
JDK build was enhanced with extra encryption modules.
Note: only Windows installer was patched.

Here is the log file.
dbeaver4.zip

Please check how it works in Early Access version (https://dbeaver.io/files/ea/).
JDK build was enhanced with extra encryption modules.
Note: only Windows installer was patched.

6.1.2 EA works well, thanks!

Please check how it works in Early Access version (https://dbeaver.io/files/ea/).

Connecting successfully in EA 6.1.2, thanks!

@flerxu @NekoKoneko thanks for testing!


deflagrations, anthracosilicosis

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[a,d]

/trunk merge


Firmware update segment requests not answered for battery-powered devices

Should be reproducible with Aeotec Multisensor 7

cannot reproduce


Newly added claims are not displayed in the dashboard under Consent Mgt

Steps

Modify and add new claims in the travelocity app
Check Consent Mgt in the dashboard

Issue
The newly added claims will not be displayed unless they are mandatory claims

Can you please elaborate more on this issue ? By just adding a claim to SP configuration doesn't affect any of the consents.

This may related to https://github.com/wso2/product-is/issues/2661.
When requested claims are modified in the SP configurations, it doesn't imply user gives his consent to use newly added attributes. And it doesn't make changes to the users consent reciept which is build on already consented attributes. Hence, no change is reflected in the dashboard.


JET integration enhancement question

Has anyone considered adding support for integrating telemetry data learned from JET?  e.g. RIB API subscription information instead of periodic CLI execution of 'show route ...'?
thanks,
-Colby

HI @jcbarth
That would be great
I did a quick try the other day to ingest events information from MQTT into OpenNTI using Telegraf
it turns out the MQTT client available for Telegraf today was not compatible with the format sent by Junos on MQTT
with a little development it should be possible
Damien


Update Application.php with DIRECTORY_SEPARATOR constants

Fix application paths to use DIRECTORY_SEPARATOR for Windows support.

Whaaaat? This is totally not required!

If you would like to support Windows it is necessary. Please see the discussion at #7632

I have read, I just don't really like this solution. No point in wasting any more time on it.

I just experience this error today after updating on laravel 5.1..
To solve the error you'll have to go to Illuminate\Foundation\Bootstrap\LoadConfiguration
Go to line 72 i think.
And change
$files[$nesting.basename($file->getRealPath(), '.php')] = $file->getRealPath();
To
$files[basename($file->getRealPath(), '.php')] = $file->getRealPath();
Works for me..


Migrate build logic to Kotlin DSL

Closes #186
The diff from GitHub makes it hard to review, but the commit history should be clear.

Fixed - I've also updated README in the last commit as there were a few references (and a link) to settings.gradle. Please check that before merging, thanks!


update README.md

change deprecated provider naming
Details
provider property change name
Related Issue

Motivation and Context

spm update after compile error but README.md is not updated
How Has This Been Tested

Types of changes

[x] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.

Thanks @jun7680 !


Updated ApplicationExecutor with parent-child resource handling

Updated the ApplicationExecutor class to handle parent-child relationships of resources.
A new method SetChildResourceStateAsync(IResource resource, string state) has been added to set the state of a resource's.
Call SetChildResourceStateAsync to ensure that all child resources of a container are updated with the container's state before and after creating a container.

Fixes #2923
Microsoft Reviewers: Open in CodeFlow

/azp run


Won't Fix alerts are still shown

I'm one of the maintainers of the scorecard's project and we integrated with the code scanning a few months ago.
One user https://github.com/ossf/scorecard-action/issues/143 reported that the results keeps showing after the file is updated via dependabot, even though the results were marked as Won't Fix
TL;DR: scorecard creates alerts if the actions defined in a GitHub workflows are not pinned by hash ( https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-third-party-actions). Once in a while, dependabot creates PR to update the hash when a new version of an action becomes available. After accepting a dependabot PR, the code scanning shows previous alerts that were tagged with Won't Fix.
My feature request is that the code scanning dashboard should only show alerts for code that has been changed "meaningfully" by developers, rather than simply updated via dependabot and renovatebot for version updates.

yes this is exactly the behavior. Version updates don't change the code semantic, so one way to improve would simply be to look at previous commit and see if the line was changed by a dependabot/renovatebot PR only. If that's the case, it'd be safe to assume the code has not changed.
Would this be practical? Would you anticipate problems with this approach?

A suggestion - could github/codeql-action/upload-sarif action be expanded to have a exclude option, wherein users can specify ruleIDs they want to ignore? That enables users to completely disable rules/checks they are not interested in.

Or better yet - the UI should have an option to allow users to ignore all failures related to a ruleID.

cc @josepalafox

Hi @laurentsimon I talked with a few folks and received the following overview and suggestion:
To provide some technical details here, Code Scanning uses the SARIF partialFingerprints property to determine whether an alert is the same as a previously seen alert. (Specifically it uses the fingerprint called primaryLocationLineHash.)
When the CodeQL Action is used to upload SARIF files we check if this property is present, and if it is not we calculate a value based on the hash of the surrounding lines in the file. If you make changes to the line containing the alert, the partial fingerprint will change.
If a tool wishes to override this definition of if an alert is "new" or not, they can do this by providing their own value for the partialFingerprints.primaryLocationLineHash which remains stable when certain properties of the line change.

Thank you @josepalafox !
I will take a look.


no-unspported-features support es2021+

it would requires switching from eslint-plugin-es => eslint-plugin-es-x

related:

#26
#12

Any news? Shall I raise a PR to replace eslint-plugin-es?


nocrypto 0.5.4 failed

Hello,
It failed in nocrypto0.5.4.
Please could you tell me ,how can I solve it?

You may want to check the version of topkg. As added here: https://github.com/ocaml/opam-repository/blob/master/packages/nocrypto/nocrypto.0.5.4/opam#L37
nocrypto.0.5.4 works with topkg.0.9.1, and the line
pkg.ml: [ERROR] key --jobs: Unknown key

indicates you have a stale version of topkg

I think it was caused by core-network?
This has been fixed in the latest 0.3.1, you can try with the latest version, hope that'll help.

Thanks you, @sevenEng
May I ask one question,I have forked the all of repositories from me-box because of the version management of Databox.
I forked the Version 0.3.0
Question, when me-box have version upgraded,my forked repositories also influence ?

Please see https://help.github.com/articles/fork-a-repo/
also familiarise yourself with the licensing requirements  http://www.linfo.org/mitlicense.html
Unfortunately we do not have the bandwidth to provide debugging for forked projects.

@sevenEng  I have same error even used the latest version 0.3.1

@haddadi  Thank you for your guidance

@tsh-box I have the same expectation with you that an upgrade should not break a previous version. But the problem is not from Databox per se, but from a upstream package (nocrypto.0.5.4),  it did an online update of the build script for that specific version without rolling out a new version, which may not be the best practice, also the first attempt of doing this actually missed some version constraints (topkg >= 0.9.1), and caused this problem. But this was picked up by our automatic travis build fairly quickly, and  issues were filed, PR was submitted merged. On our side, we also patched up core-network to deal with this in 0.3.1.

I solved it :)


How to refresh federated token and proceed with other AWS calls?

As per https://docs.amplify.aws/sdk/auth/federated-identities/q/platform/ios

federatedSignIn() can be used to obtain federated “Identity ID” using external providers like Google, Facebook or Twitter. If the tokens are expired and new tokens are needed, a notification will be dispatched on the AWSMobileClient listener with the user state signedOutFederationTokensInvalid. You can give the updated tokens via the same federatedSignIn() method.
The API calls to get AWS credentials will be asynchronously blocked until you fetch the social provider’s token and give it to AWSMobileClient. Once you pass the tokens, the AWSMobileClient will fetch AWS Credentials using the new tokens and unblock all waiting calls. It will then use the new credentials.

What I do:

I call Amplify.API.post(request: request) - OK
I catch signedOutFederationTokensInvalid - OK
I catch Amplify.API.post(request: request) API error - OK
I refresh my OpenID token and pass it to federatedSignIn again - OK
On this step my token is refreshed - OK , but all SDK calls during step 4 have been failed - KO
How to proceed with my Amplify.API.post(request: request) from step 1 ?

What does The API calls to get AWS credentials will be asynchronously blocked until you fetch the social provider’s token and give it to AWSMobileClient. Once you pass the tokens, the AWSMobileClient will fetch AWS Credentials using the new tokens and unblock all waiting calls. mean?
Blocking the thread where I get signedOutFederationTokensInvalid doesn't help, Amplify.API.post triggers the error in any case.
Is my understanding correct that there is no an easy way here and we need to create a refresh mechanism with repeating all failed operations ourselves?
I think that it would be good to know the expiration time on the client and refresh a token before doing any AWS calls.
Thank you.

Moved this to the Amplify repo since the final use for the credentials is an Amplify.API call

Hi @ppave, Thanks for opening this issue. What you mentioned is correct that amongst the SDK's (AWSMobileClient, AppSync SDK), the block would not be released until the user signs back in, and in the scenario where the user is unable to sign in, developers can call AWSMobileClient.releaseSignInWait() to unblock the calls. In Amplify, when using API Plugin, the calls will return immediately with a failure. This is the currently by design; developers have to react to the failures immediately, prompt the user to sign in, and depending on the UX, the end user would have to perform the same actions to execute those requests.
Internally, the auth plugin is called releaseSignInWait() when the session expired, so there isn't a situation where it would make sense for the developer to call releaseSignInWait(), just want to call this out since you are going to the escape hatch for the AWSMobileClient instance, and this is also the reason why the API calls are not blocked, they are released immediately after being blocked.
Could you expand on your use case? Is there a difficulty of building the UX to retry the failed operations that you would rather have the calls block and be released upon sign in?

Hi @lawmicha, thank you for your response.
My use case:

User launches the app
On the first app launch, user have to sign in using web-view with OpenID implementation
This web-view returns JWT that is passed to Amplify federatedSignIn
User is not supposed to enter credentials in web-view anymore during the app usage, so all refresh operations should happen in background (the only exception is our OpenID refresh token expiration)
Our OpenID token lives a few mins, Cognito Identity Provider token lives 1hr, so after 1hr we get signedOutFederationTokensInvalid error
We have many calls to AWS resources, so there is a chance that before we get signedOutFederationTokensInvalid and manage to refresh this Cognito token in background, several AWS calls may fail.

Some must-have requirements to our refresh mechanism are:

Seamless background work
Support execution order
Support concurrent execution

This requires some work in application layer, that I would like to have in Amplify layer if possible :)
And if we would need to write a custom refresh mechanism in application layer, in my opinion, it would easier to implement it before this kind of error occurs rather than after. For this case we just need to know when Cognito token expires.
Please let me know if you need a bit more details.

Hi @ppave, thanks for explaining. I found the docs that says the cognito session will expiry in one hour: https://docs.aws.amazon.com/cognito/latest/developerguide/accessing-aws-services.html
To better understand how the work could exist in the Amplify layer, what would the custom refresh mechanism look like in the application layer? Are you going to proactively refresh it before the hour expiry time?

Hi @lawmicha sorry for delay.
It's quite tricky to rely only on duration, would be better to rely on createdAt & duration or expiredAt params that hold ISO 8601 date.
So in this case, before performing any API calls, I think we can do in app layer something like:
let shouldRefreshToken =  currentDate.addingTimeInterval(fiveMinutes) >= expirationDate

Hello,
We have made several improvements to Amplify in v2 with respect to Auth category. Please let us know if this is still a problem.
https://docs.amplify.aws/lib/auth/getting-started/q/platform/ios/
https://github.com/aws-amplify/amplify-swift/releases/tag/2.15.2

See Identity Pool Federation > Token Refresh for more information. Thanks!


Modify spring-cloud-azure-core and spring-cloud-azure-service according to API review

The spring-cloud-azure-core is being reviewed by the architects, this PR is to apply the changes according to comments. https://apiview.dev/Assemblies/Review/eebb74d23c7c462fbdab909e43fb465b#com.azure.spring.core.resource.AbstractAzureStorageProtocolResolver

API changes have been detected in com.azure:azure-core. You can review API changes here

API changes have been detected in com.azure:azure-core-management. You can review API changes here
API changes
-         public static final Region SWEDEN_CENTRAL = new Region("swedencentral", "Sweden Central");

/azp run java - spring - tests

/azp run java - spring - tests


python310Packages.ypy-websocket: 0.8.4 -> 0.9.0

Description of changes
Diff: https://github.com/y-crdt/ypy-websocket/compare/refs/tags/v0.8.4...v0.9.0
Changelog: https://github.com/y-crdt/ypy-websocket/blob/v0.9.0/CHANGELOG.md

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 227908 run on x86_64-linux 1

6 packages failed to build:

manim
manim.dist
python310Packages.jupyterlab-git
python310Packages.jupyterlab-git.dist
python311Packages.jupyterlab-git
python311Packages.jupyterlab-git.dist

24 packages built:

python310Packages.jupyter-server-ydoc
python310Packages.jupyter-server-ydoc.dist
python310Packages.jupyterlab
python310Packages.jupyterlab-lsp
python310Packages.jupyterlab-lsp.dist
python310Packages.jupyterlab.dist
python310Packages.jupytext
python310Packages.jupytext.dist
python310Packages.mkdocs-jupyter
python310Packages.mkdocs-jupyter.dist
python310Packages.ypy-websocket
python310Packages.ypy-websocket.dist
python311Packages.jupyter-server-ydoc
python311Packages.jupyter-server-ydoc.dist
python311Packages.jupyterlab
python311Packages.jupyterlab-lsp
python311Packages.jupyterlab-lsp.dist
python311Packages.jupyterlab.dist
python311Packages.jupytext
python311Packages.jupytext.dist
python311Packages.mkdocs-jupyter
python311Packages.mkdocs-jupyter.dist
python311Packages.ypy-websocket
python311Packages.ypy-websocket.dist

Result of nixpkgs-review pr 227908 run on x86_64-linux 1

30 packages built:

manim
manim.dist
python310Packages.jupyter-server-ydoc
python310Packages.jupyter-server-ydoc.dist
python310Packages.jupyterlab
python310Packages.jupyterlab-git
python310Packages.jupyterlab-git.dist
python310Packages.jupyterlab-lsp
python310Packages.jupyterlab-lsp.dist
python310Packages.jupyterlab.dist
python310Packages.jupytext
python310Packages.jupytext.dist
python310Packages.mkdocs-jupyter
python310Packages.mkdocs-jupyter.dist
python310Packages.ypy-websocket
python310Packages.ypy-websocket.dist
python311Packages.jupyter-server-ydoc
python311Packages.jupyter-server-ydoc.dist
python311Packages.jupyterlab
python311Packages.jupyterlab-git
python311Packages.jupyterlab-git.dist
python311Packages.jupyterlab-lsp
python311Packages.jupyterlab-lsp.dist
python311Packages.jupyterlab.dist
python311Packages.jupytext
python311Packages.jupytext.dist
python311Packages.mkdocs-jupyter
python311Packages.mkdocs-jupyter.dist
python311Packages.ypy-websocket
python311Packages.ypy-websocket.dist


Provide an OAuth example since it will be required starting 1st of May 2020

Since Twitch is going to require OAuth Authentication starting from 1st of May 2020 (More here) it would be great to add an Authentication example in order to speed up initial configuration using this API wrapper.

This is already supported. See the existing docs and examples: https://github.com/nicklaw5/helix/blob/master/docs/authentication_docs.md.

Relates to #42


sqlx::Type for Enum with String repr

Bug Description
Currently, postgres Type would always generate Type with PgTypeInfo::with_name(#ty_name) for enum.
While we are using sqlx::Type derive with Enum and the underline type is expected to be TEXT or VARCHAR, this can result in an error:
error returned from database: type "datatype" does not exist

I suppose we can suppress this generation or use TEXT with some config options.
Minimal Reproduction
#[derive(Debug, Clone, Eq, PartialEq, Serialize, Deserialize, sqlx::Type)]
#[serde(rename_all = "snake_case")]
#[sqlx(rename_all = "snake_case")]
pub enum DataType {
    Int,
    UInt,
    Float,
    Binary,
    String,
    Boolean,
    Interval,
    Timestamp,
    Variant,

    Null,
}

And try to insert into a DB with column data_type TEXT.
Info

SQLx version: 0.8.2
SQLx features enabled: ["postgres", "runtime-tokio-rustls"]
Database server and version: [REQUIRED] Postgres 16.3
Operating system: macOS 13.4
rustc --version: rustc 1.83.0-nightly (363ae4188 2024-09-24)

Straightforward, render type with TEXT if #[sqlx(type_name = "...")]  is not specified, instead of default to ident's name.

Use #[sqlx(type_name = "TEXT")] seems can work it around. But it conflicts with our doc:
With #[repr(_)] the integer representation is used when converting from/to SQL and expects that SQL type (e.g., INT). Without, the names of the variants are used instead and expects a textual SQL type (e.g., VARCHAR, TEXT).

I noticed the "the names of the variants are used" in https://docs.rs/sqlx/latest/sqlx/trait.Type.html#enumeration

Without, the names of the variants are used instead and expects a textual SQL type (e.g., VARCHAR, TEXT).

now.
So this can be a feature request rather than a bug.


How should I apply this plugin without docker layer but just use the containerd ?

Seem the Kubernetes will remove the docker after the version 1.20. How to migrate the gpushare in the new contain runtime?

me too

the gpushare device plugin and scheduler extender themselves don't rely on docker or containerd. what you should do is just update your containerd config to use nvidia-container-runtime, pls refer to https://developer.aliyun.com/article/782035 for more details. as long as you can assign nvidia.com/gpu to your pod with your containerd enabled k8s，you can use gpushare there as well.  BTW, the containerd compatibility has been resolved by default in Alibaba cloud K8s service (aka. ACK).


StealthComms

Describe your project
StealthComms allows users to prove their membership in a group and send messages/signals without revealing their original identity. Built by extending Semaphore's whistleblower contract.
Other info

Name: StealthComms
Tagline: StealthComms allows users to prove their membership in a group and send messages/signals without revealing their original identity.

Links

Website: https://stealthcomms.surge.sh/
Github: https://github.com/atomniketh/zk-app
Twitter: https://twitter.com/AtomNikETH

Hey @atomniketh! Great work and documentation. Thank you!


Simplify CLI usage by deprecating readvars

sfpowerscripts cli supports output variables, which are written to the environment of the host. However to consume this, a source readvars has to be issued before any other command. This makes the scripts verbose and defeat the purpose.
sfpowerscripts cli should store this output variable in its own environment config, and all tasks when provided with a non dereferenced  variable, should check the store and figure the value. It should also print the resolved variables

I propose that we keep the readvars script for now. It may still be useful for power users, who want access to output variables at the terminal layer.

Sure we can keep it, but lets remove it from our main readme and add us a subsection


Optimized away the expensive StringBuilder.Remove(0, 1) in CodeInlineParser

Sorry, the CI on master was broken and it should be fixed now. Could you rebase/merge master to your branch please? Thanks!

Thanks!


Possible websocket issue when bot is in a large guild

When connecting to the gateway the websocket may fail to read (for reasons unknown) and most likely you will get some error that looks like this:
value out of range: -19277
  recv's lead up to read of failed Future:
    Traceback (most recent call last)
    basic_bot.nim(24)        basic_bot
    asyncdispatch.nim(1000)  waitFor
    asyncdispatch.nim(1035)  poll
    asyncdispatch.nim(180)   processPendingCallbacks
    asyncmacro.nim(34)       cb
    asyncmacro.nim(334)      recvLineIter
    asyncfutures.nim(109)    complete
    asyncmacro.nim(34)       cb
    asyncmacro.nim(334)      newAsyncWebsocketIter
    asyncfutures.nim(109)    complete
    asyncmacro.nim(34)       cb
    asyncdispatch.nim(1000)  startSessionIter
    asyncdispatch.nim(1035)  poll
    asyncdispatch.nim(180)   processPendingCallbacks
    asyncmacro.nim(34)       cb
    shared.nim(138)          recvFrameIter
    asyncmacro.nim(387)      recv
    asyncmacro.nim(34)       cb
    asyncnet.nim(367)        recvIter
    system.nim(2662)         sysFatal
  recvFrame's lead up to read of failed Future:
    Traceback (most recent call last)
    basic_bot.nim(24)        basic_bot
    asyncdispatch.nim(1000)  waitFor
    asyncdispatch.nim(1035)  poll
    asyncdispatch.nim(180)   processPendingCallbacks
    asyncmacro.nim(34)       cb
    asyncmacro.nim(334)      recvLineIter
    asyncfutures.nim(109)    complete
    asyncmacro.nim(34)       cb
    asyncmacro.nim(334)      newAsyncWebsocketIter
    asyncfutures.nim(109)    complete
    asyncmacro.nim(34)       cb
    asyncdispatch.nim(1000)  startSessionIter
    asyncdispatch.nim(1035)  poll
    asyncdispatch.nim(180)   processPendingCallbacks
    asyncmacro.nim(34)       cb
    shared.nim(138)          recvFrameIter
    asyncmacro.nim(387)      recv
    asyncmacro.nim(34)       cb
    asyncnet.nim(367)        recvIter
    system.nim(2662)         sysFatal
    [[reraised from:
    basic_bot.nim(24)        basic_bot
    asyncdispatch.nim(1000)  waitFor
    asyncdispatch.nim(1035)  poll
    asyncdispatch.nim(180)   processPendingCallbacks
    asyncmacro.nim(34)       cb
    asyncmacro.nim(334)      recvLineIter
    asyncfutures.nim(109)    complete
    asyncmacro.nim(34)       cb
    asyncmacro.nim(334)      newAsyncWebsocketIter
    asyncfutures.nim(109)    complete
    asyncmacro.nim(34)       cb
    asyncdispatch.nim(1000)  startSessionIter
    asyncdispatch.nim(1035)  poll
    asyncdispatch.nim(180)   processPendingCallbacks
    asyncmacro.nim(34)       cb
    asyncmacro.nim           recvFrameIter
    asyncfutures.nim(220)    read
    ]]

Currently investigating the cause.

The payload recieved appears to be normal however.
data:{"t":"READY","s":1,"op":0,"d":{"v":7,"user_settings":{},"user":{"verified":true,"username":"Test Bot","mfa_enabled":true,"id":"2647435426504376","email":null,"discriminator":"0221","bot":true,"avatar":"6038c8a365d7ceb97f0c2c19755013c2"},"session_id":"f13c684b78ce5ae2a3222f12c","relationships":[],"private_channels":[],"presences":[],"guilds":[{"unavailable":true,"id":"243087636560871424"}],"_trace":["discord-gateway-prd-1-27","discord-sessions-prd-1-8"]}}


Fix: don't need sudo if we're root, other minor changes

Current behavior asks for sudo password via stdin regardless on if we're root or not.
This PR adds an IsRoot boolean to the configuration struct and adjusts the final message appropriately.
Additionally, it implements the D.R.Y principal with the system exec calls in that same pages section, while using sudo only when appropriate.

I also noticed that if we tell people that they can run quickpassthrough itself as root now, we will have to check if the config and backup folders are owned by root and tell them to re-run as root quickpassthrough is not run through sudo.
I originally designed it to be run as the normal user so i never actually tested running it as sudo then normal user 😅

Thanks for the feedback! I was actually just checking in to ping you to request comment, so I'm happy to see you replied! I'll take a look :)

@HikariKnight take a look when you can, I believe I've addressed your comments
but this is a pretty heavy refactor so I definitely need a second pair of eyes

The behavior that actually lead to me starting on this was that I would actually receive silent errors when I ran this command as a user after running it as root. This last commit, https://github.com/HikariKnight/quickpassthrough/pull/28/commits/395f5ab6bca280e06031480270d50ad0afe942d8, finalizes my major changes for now I believe.
Here's a demo of the behavior this commit replaces silent fatal errors with:

this gif looks nice, conveniently i am sick today so i surprisingly has freetime to look at the code once i have the paperwork done

This looks good, will do a test build locally later today and test in VMs to make sure things work properly in ubuntu, pop, arch and fedora since this is a big refactor.

This looks good, will do a test build locally later today and test in VMs to make sure things work properly in ubuntu, pop, arch and fedora since this is a big refactor.

Yes, this definitely needs to be looked at carefully and tested. Thanks for your efforts, and take care of yourself.

^ The code should be easier to review now, as well.

Tested and working on Fedora Workstation (Grubby and Dracut test)

Archlinux tested and works (pure grub and mkinitcpio test)
only need the pop os test to succeed then we should be good @yunginnanet

agreed on all points. adapter function added, complete with unit test


Issue with serverless deploy, requirements.txt not found

I have the following serverless.yml:
# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!

service: awsTest

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
# frameworkVersion: "=X.X.X"
plugins:
  - serverless-python-requirements
custom:
    pythonRequirements:
      invalidateCaches: true
      dockerizePip: true
      dockerImage: lambda-python3.6-with-mysql-build-deps

provider:
  name: aws
  runtime: python3.6
  role: arn:aws:iam::443746630310:role/EMR_DefaultRole

# you can overwrite defaults here
#  stage: dev
#  region: us-east-1

# you can add statements to the Lambda function's IAM Role here
#  iamRoleStatements:
#    - Effect: "Allow"
#      Action:
#        - "s3:ListBucket"
#      Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ]  }
#    - Effect: "Allow"
#      Action:
#        - "s3:PutObject"
#      Resource:
#        Fn::Join:
#          - ""
#          - - "arn:aws:s3:::"
#            - "Ref" : "ServerlessDeploymentBucket"
#            - "/*"

# you can define service wide environment variables here
#  environment:
#    variable1: value1

# you can add packaging information here
#package:
#  include:
#    - include-me.py
#    - include-me-dir/**
#  exclude:
#    - exclude-me.py
#    - exclude-me-dir/**

functions:
  emotion-analysis:
    handler: handler.emotionAnalysis
    events:
       - http:
            path: emotionAnalysis
            method: post

  audio-analysis:
    handler: handler.audioAnalysis
    events:
       - http:
            path: vokaturiAnalysis
            method: post

#    The following are a few example events you can configure
#    NOTE: Please make sure to change your handler code to work with those events
#    Check the event documentation for details
#    events:
#      - http:
#          path: users/create
#          method: get
#      - s3: ${env:BUCKET}
#      - schedule: rate(10 minutes)
#      - sns: greeter-topic
#      - stream: arn:aws:dynamodb:region:XXXXXX:table/foo/stream/1970-01-01T00:00:00.000
#      - alexaSkill
#      - iot:
#          sql: "SELECT * FROM 'some_topic'"
#      - cloudwatchEvent:
#          event:
#            source:
#              - "aws.ec2"
#            detail-type:
#              - "EC2 Instance State-change Notification"
#            detail:
#              state:
#                - pending
#      - cloudwatchLog: '/aws/lambda/hello'
#      - cognitoUserPool:
#          pool: MyUserPool
#          trigger: PreSignUp

#    Define function environment variables here
#    environment:
#      variable2: value2

# you can add CloudFormation resource templates here
#resources:
#  Resources:
#    NewResource:
#      Type: AWS::S3::Bucket
#      Properties:
#        BucketName: my-new-bucket
#  Outputs:
#     NewOutput:
#       Description: "Description for the output"
#       Value: "Some output value"

and the requirements.txt:
cycler==0.10.0
decorator==4.1.2
imutils==0.4.3
Keras==2.1.1
matplotlib==2.1.0
networkx==2.0
numpy==1.13.3
olefile==0.44
opencv-python==<IP_ADDRESS>
pandas==0.21.0
Pillow==4.3.0
pyparsing==2.2.0
python-dateutil==2.6.1
pytz==2017.3
PyWavelets==0.5.2
PyYAML==3.12
scikit-image==0.13.1
scikit-learn==0.19.1
scipy==1.0.0
six==1.11.0
sklearn==0.0
dlib==19.7.0

I am using this Dockerfile to compile dlib and boost:
FROM amazonlinux:latest

RUN touch /var/lib/rpm/*
RUN yum install -y yum-plugin-ovl && cd /usr/src
#RUN  yum check-update
#RUN rpm --rebuilddb
RUN yum history sync
RUN yum install -y wget
RUN yum install -y sudo
RUN yum install -y sudo && sudo yum install -y yum-utils && sudo yum groupinstall -y development
RUN sudo yum install -y https://centos6.iuscommunity.org/ius-release.rpm && sudo yum install -y python36u && yum install -y python36u-pip && yum install -y python36u-devel
#RUN yum install -y grub2

RUN ln -s /usr/include/python3.6m /usr/include/python3.6

RUN wget --no-check-certificate -P /tmp http://flydata-rpm.s3-website-us-east-1.amazonaws.com/patchelf-0.8.tar.gz
RUN    tar xvf /tmp/patchelf-0.8.tar.gz -C /tmp
RUN    cd /tmp/patchelf-0.8 && ./configure && make && sudo make install

RUN  yum install -y blas-devel boost-devel lapack-devel gcc-c++ cmake git
RUN     git clone https://github.com/davisking/dlib.git
RUN     cd dlib/python_examples/
RUN     mkdir build && cd build
RUN     cmake -DPYTHON_INCLUDE_DIR=$(python3.6 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python3.6 -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DUSE_SSE4_INSTRUCTIONS:BOOL=ON dlib/tools/python
RUN     sed -i 's/\/\/all/all/' Makefile && sed -i 's/\/\/preinstall/preinstall/' Makefile
RUN     cmake --build . --config Release --target install
RUN     cd ..

RUN    mkdir ~/dlib
RUN    cp dlib.so ~/dlib/__init__.so
RUN    cp /usr/lib64/libboost_python-mt.so.1.53.0 ~/dlib/
RUN    touch ~/dlib/__init__.py
RUN    patchelf --set-rpath '$ORIGIN' ~/dlib/__init__.so

When I run serverless deploy, I get the following error:
Error --------------------------------------------------

Error: Could not open requirements file: [Errno 2] No such file or directory: '.serverless/requirements.txt'

    at ServerlessPythonRequirements.installRequirements (/Users/manavdutta1/Downloads/awsTest/node_modules/serverless-python-requirements/lib/pip.js:80:11)
From previous event:
    at PluginManager.invoke (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:366:22)
    at PluginManager.spawn (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:384:17)
    at Deploy.BbPromise.bind.then.then (/usr/local/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:120:50)
From previous event:
    at Object.before:deploy:deploy [as hook] (/usr/local/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:110:10)
    at BbPromise.reduce (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:366:55)
From previous event:
    at PluginManager.invoke (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:366:22)
    at PluginManager.run (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:397:17)
    at variables.populateService.then (/usr/local/lib/node_modules/serverless/lib/Serverless.js:104:33)
    at runCallback (timers.js:785:20)
    at tryOnImmediate (timers.js:747:5)
    at processImmediate [as _immediateCallback] (timers.js:718:5)
From previous event:
    at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:91:74)
    at serverless.init.then (/usr/local/lib/node_modules/serverless/bin/serverless:42:50)
    at <anonymous>

I have no idea why this is happening. I have the requirements.txt under .serverless in my local directory and it looks fine. Does anyone know why this is happening?

You should keep your requirements.txt in the root of your service, the plugin creates the file at .serverless/requirements.txt.

Are you running on windows @manav95? I noticed you enabled dockerizePip. See Issue #105

im using mac os x

Check your shared drives in the docker for Mac  settings.

I'm having the exact same issue as @manav95. I'm using default docker image on debian jeesie

Just thought of something it might be.. Add this to your Dockerfile:
RUN mkdir /var/task
WORKDIR /var/task

I'm hitting this too. Runs perfectly locally (OSX) but when using Codeship and the following Dockerfile:
FROM docker:dind

RUN apk add --update \
    nodejs \
    python3 \
    py-pip \
    build-base \
  && pip install virtualenv \
  && rm -rf /var/cache/apk/*

RUN mkdir /var/task
WORKDIR /var/task

COPY . /var/task

RUN npm install -g serverless
RUN sls plugin install -n serverless-python-requirements

Then when I run serverless deploy I get the same error.

I have the same problem.
Also when I am trying to pull image inside the dicker:dindimage:

I got:

(image: codeship_smr-chatbot) (service: smr-chatbot) Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

(image: codeship_smr-chatbot) (service: smr-chatbot) Removing intermediate container 0f5038935f88
(step: deploy to dev) error ✗
(step: deploy to dev) error loading services during run step: failure to build Image{ name: "smr-chatbot", branch: "dev", dockerfile: "/Users/sitin/Documents/Workspace/Chimplie/smr-chatbot/Dockerfile", cache: true }: The command '/bin/sh -c docker pull lambci/lambda:build-python3.6' returned a non-zero code: 1

I am running via jet and have add_docker: true.

I get this error too when I try using the plugin on CircleCI to automate deployment. I don't use any custom Docker images but circleci/python:3.6.4. The plugin configuration I use is as follows:
plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    dockerizePip: true

I've been playing with this a bit more and there's definitely something about running the pip install through docker, from within another docker.
I guess one way to get around this would be to run the pip install command without docker, given we're already within a docker container - as long as the host docker is the right kind to build the package for lambda.
If there was an extended version of https://github.com/lambci/docker-lambda/tree/master/python3.6 that we could use to run serverless deploy from then we could set dockerizePip: false.

@thesmith Yes, this is the current workaround I use. Thank you for posting it – can be useful for other users who are hitting this issue.

RUN npm install

Annoyingly this means you have to flip dockerizePip between deploying via CI and locally.

Ah. yeah I'll ahve to check docker-in-docker out at some point.
Re this @thesmith:

Annoyingly this means you have to flip dockerizePip between deploying via CI and locally.

You an do something like:
custom:
  pythonRequirements:
    dockerizePip: ${self:custom.isCI.${env:CI}, self:custom.isCI.false}
  isCI:
    true: true
    false: non-linux

(this assumes you have a CI env var set to true in CI (CircleCI does this automatically, not sure how standard it is, but it'd be easy to add the var or adapt this technique to your CI provider)

@tgensol FWIW it's Codeship, not CircleCI.
On Codeship Pro, the install / deploy process is itself run in a docker image built using that Dockerfile (with dockerizePip set to false).

We have the same problem. I can package and deploy from Mac but the build fails on AWS.
custom section of our serverless.yml:
custom:
  pkgPyFuncs:
    buildDir: _build
    requirementsFile: requirements.txt
    cleanup: true
    useDocker: false

Error log on AWS:

  Error --------------------------------------------------

  ENOENT: no such file or directory, open 'requirements.txt'

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: ENOENT: no such file or directory, open 'requirements.txt'
    at Object.fs.openSync (fs.js:667:18)
    at Object.fs.readFileSync (fs.js:572:33)
    at generateRequirementsFile (/codebuild/output/src659347776/src/node_modules/serverless-python-requirements/lib/pip.js:163:6)
    at installRequirements (/codebuild/output/src659347776/src/node_modules/serverless-python-requirements/lib/pip.js:34:5)
    at values.filter.map.f (/codebuild/output/src659347776/src/node_modules/serverless-python-requirements/lib/pip.js:215:11)
    at Array.map (<anonymous>)
    at ServerlessPythonRequirements.installAllRequirements (/codebuild/output/src659347776/src/node_modules/serverless-python-requirements/lib/pip.js:210:8)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:372:22)
    at PluginManager.spawn (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:17)
    at Deploy.BbPromise.bind.then.then (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:123:50)
From previous event:
    at Object.before:deploy:deploy [as hook] (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:113:10)
    at BbPromise.reduce (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:372:55)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:372:22)
    at PluginManager.run (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:403:17)
    at variables.populateService.then (/usr/lib/node_modules/serverless/lib/Serverless.js:102:33)
    at runCallback (timers.js:763:18)
    at tryOnImmediate (timers.js:734:5)
    at processImmediate (timers.js:716:5)
    at process.topLevelDomainCallback (domain.js:101:23)
From previous event:
    at Serverless.run (/usr/lib/node_modules/serverless/lib/Serverless.js:89:74)
    at serverless.init.then (/usr/lib/node_modules/serverless/bin/serverless:42:50)
    at <anonymous>

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information -----------------------------
     OS:                     linux
     Node Version:           9.9.0
     Serverless Version:     1.27.2

[Container] 2018/05/14 09:56:20 Command did not exit successfully SLS_DEBUG=* sls deploy --stage $STAGE exit status 1
[Container] 2018/05/14 09:56:20 Phase complete: BUILD Success: false
[Container] 2018/05/14 09:56:20 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: SLS_DEBUG=* sls deploy --stage $STAGE. Reason: exit status 1

Our buildspec.yml:

env:
  variables:
    DOCKER_VERSION: "17.09.1"
    STAGE: "dev"

phases:
  pre_build:
    commands:
      - pip install pip --upgrade
      - npm install -g serverless
      - npm install
      - npm -v
      - sls -v
  build:
    commands:
      - SLS_DEBUG=* sls deploy --stage $STAGE

And our dependencies defined in package.json:
{
  "dependencies": {
    "serverless": "^1.27.2",
    "serverless-package-python-functions": "^0.2.5",
    "serverless-pseudo-parameters": "^1.6.0",
    "serverless-python-requirements": "^4.0.3",
    "serverless-step-functions": "^1.4.1"
  }
}

I also tried appending the following config to the custom entry in the serverless.yml which didn't help:
  pythonRequirements:
    dockerizePip: false

@dschep thanks for the tip! I'll give it a try and if it works without serverless-package-python-functions I'll happily remove it! Less clutter is the better.

I had this issue as well - or a similar one, anyway - running inside Gitlab CI (docker-in-docker) with dockerizePip: true:
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Generated requirements from /builds/group/project/requirements.txt in /builds/group/project/.serverless/requirements.txt...
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/03e86de11d16dfbe8247f02d3d303294_slspyc/requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.6
Serverless: Using download cache directory /root/.cache/serverless-python-requirements/downloadCacheslspyc
Could not open requirements file: [Errno 2] No such file or directory: '/var/task/requirements.txt'

and I accidentally stumbled upon a workaround.
I was already going to start using download and static caching, and I wanted the cache dir to be inside my .serverless directory, so that it would be saved and restored between jobs. So, I ended up with these settings:
custom:
  pythonRequirements:
    dockerizePip: true
    useDownloadCache: true
    useStaticCache: true
    cacheLocation: ./.serverless/.requirements_cache

And, lo and behold, that also fixed the packaging issue.
If you notice above, the plugin is trying to map a requirements.txt file into the container in /var/task/:
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/03e86de11d16dfbe8247f02d3d303294_slspyc/requirements.txt ...

My guess is that the Gitlab CI runner disallows this or interferes with it somehow, because when I set cacheLocation as above, I get this instead:
Serverless: Installing requirements from /builds/group/project/.serverless/.requirements_cache/c1982a9b5b5e665faaa0cd35390c1b60_slspyc/requirements.txt ...

which works perfectly.
This could also be because the /builds/group/project directory is already being mapped into the container as a volume, allowing pip in the container to find the path to requirements.txt. Either way, hopefully this helps someone else with a similar dockerish problem.

@brettdh the cache location shouldn’t be inside of the .serverless folder as that folder is temporary in nature and nothing would ever cache there.  It might work I’ve never actually tested in there but that’s my first gut instinct.  Can you maybe pastebin more of your full serverless config and your gitlabci file and I’ll take a look?

I had this issue as well - or a similar one, anyway - running inside Gitlab CI (docker-in-docker) with dockerizePip: true:
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Generated requirements from /builds/group/project/requirements.txt in /builds/group/project/.serverless/requirements.txt...
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/03e86de11d16dfbe8247f02d3d303294_slspyc/requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.6
Serverless: Using download cache directory /root/.cache/serverless-python-requirements/downloadCacheslspyc
Could not open requirements file: [Errno 2] No such file or directory: '/var/task/requirements.txt'

and I accidentally stumbled upon a workaround.
I was already going to start using download and static caching, and I wanted the cache dir to be inside my .serverless directory, so that it would be saved and restored between jobs. So, I ended up with these settings:
custom:
  pythonRequirements:
    dockerizePip: true
    useDownloadCache: true
    useStaticCache: true
    cacheLocation: ./.serverless/.requirements_cache

And, lo and behold, that also fixed the packaging issue.
If you notice above, the plugin is trying to map a requirements.txt file into the container in /var/task/:
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/03e86de11d16dfbe8247f02d3d303294_slspyc/requirements.txt ...

My guess is that the Gitlab CI runner disallows this or interferes with it somehow, because when I set cacheLocation as above, I get this instead:
Serverless: Installing requirements from /builds/group/project/.serverless/.requirements_cache/c1982a9b5b5e665faaa0cd35390c1b60_slspyc/requirements.txt ...

which works perfectly.
This could also be because the /builds/group/project directory is already being mapped into the container as a volume, allowing pip in the container to find the path to requirements.txt. Either way, hopefully this helps someone else with a similar dockerish problem.

i'm facing the exact same issue and this did not help, any other suggestions ?

@thesmith or @avli
I've got the /var/task problem for a project I'm working on right now. I'm trying to use circleCI and serverless-python-requirements.
Are you setting dockerizePip to false when deploying from the CI or are you using dockerizePip true?
If you are using dockerizePip true did you place in your own docker image for it and then added /var/task folder?
When I try dockerizePip false I run into the lambda limit error, which is not good. Even when I use slimming.
Any clarification would be great here.

Getting the same issue to @chubzor, any news on a fix.

@alexcallow I ended up using this:
custom:
pythonRequirements:
layer: true
slim: true
slimPatterns:
- "**/test_*.py"
strip: false
We abandoned deploying via local dev machines, and this worked for us when deploying with circleCI.
Mind you we have numpy, pandas, scikit-learn in requirements.
I think we are hanging by inside of some size limit so this is not sustainable, but could be helpful for you.

If you have pyproject.toml in your project but you don't use poetry, please remember to set  usePoetry: false. The config will be
custom:
  pythonRequirements:
    dockerizePip: true
    usePoetry: false

Related code:

https://github.com/UnitedIncome/serverless-python-requirements/blob/64e20db2a4acbf95a3d9391797b0c12544234a0c/index.js#L41

https://github.com/UnitedIncome/serverless-python-requirements/blob/master/lib/pip.js#L65

this is not an issue with this plugin, it's circleci that does not allow volume mount in docker executor
https://discuss.circleci.com/t/why-circleci-2-0-does-not-support-mounting-folders/11605

I'm encounter this as well when trying to use CircleCi, my executor is
executor:
  name: node/default
  tag: '10.4'`

which I belive means im doing docker-in-docker.
It seems this command
Running docker run --rm -v /home/circleci/.cache/serverless-python-requirements/2b94ea26f9dceaadc347670525eaa71ffd73487d3460b8428f6a406f834f65af_slspyc\:/var/task\:z -v /home/circleci/.cache/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z sls-py-reqs-custom /bin/sh -c 'chown -R 0\\:0 /var/useDownloadCache && python3.6 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 3434\\:3434 /var/task && cp /usr/lib64/libpq.so.5 /var/task/ && chown -R 3434\\:3434 /var/useDownloadCache'..
is mounting /home/circleci/.cache/serverless-python-requirements/2b94ea26f9dceaadc347670525eaa71ffd73487d3460b8428f6a406f834f65af_slspyc\ to  /var/task\:  but I found this in the CircleCI documentation:
https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-mount-volumes-to-docker-containers-
"It's not possible to use volume mounting with the docker executor, but using the machine executor it's possible to mount local directories to your running Docker containers. "
I switched from docker-in-docker:
jobs:
   build:
     executor:
       name: node/default
       tag: '10.4'
     steps:
        - checkout
        - node/with-cache:
            steps:
              - run: npm install
        - setup_remote_docker
        - run: npx sls package -p ./artifacts/

to Machine:
jobs:
   build:
     machine: true
     steps:
        - checkout
        - node/install-node:
            version: 10.16.3
        - node/install-npm:
            version: 6.12.1
        - node/with-cache:
            steps:
              - run: npm install
        - run: npx sls package -p ./artifacts/

And it ran successfully.
Note: CircleCi warns that machine executors may become a premium feature in the future.

@bxm156 please try my docker in docker support and report back.  See: https://github.com/UnitedIncome/serverless-python-requirements/pull/484

If you have pyproject.toml in your project but you don't use poetry, please remember to set  usePoetry: false. The config will be
custom:
  pythonRequirements:
    dockerizePip: true
    usePoetry: false

Related code:

https://github.com/UnitedIncome/serverless-python-requirements/blob/64e20db2a4acbf95a3d9391797b0c12544234a0c/index.js#L41
https://github.com/UnitedIncome/serverless-python-requirements/blob/master/lib/pip.js#L65

This thing! pyproject.toml...

I bumped to this issue as well with macOS and minikube as docker platform. The problem was that the docker run command executed by the plugin includes volume mounts like mentioned in other comments (those -v switches) and volume mounts don't work out-of-the-box with minikube. I had to run minikube mount command for the folder like this (leave the command running):
minikube mount /path/to/your-lib-folder/Library/Caches/serverless-python-requirements:/path/to/your-lib-folder/Library/Caches/serverless-python-requirements

And after that the plugin was able to generate the requirements and the build worked.


SOT 164: Add Parser

Remember to merge #12 before merging this branch

Need someone to review this ASAP


Use fieldName in mapping instead of name when persisting

This is a bug that happens when the name annotation of an embed document is different of the field name itself.
Related: https://github.com/doctrine/mongodb-odm/commit/85d91b74b64057d9f812d030d6d15d51e9fa278d

Thank you @monteiro! All looks good to me, I'll get to merging today evening or tomorrow

Thanks!

Also backported into 1.0.x branch in https://github.com/doctrine/mongodb-odm/commit/03793a23049afcad53114323529ce9383e598351. Thanks again @monteiro!


Sweep component migration is painful

Copied from https://msdata.visualstudio.com/Vienna/_workitems/edit/2138112
set sweep setting's complex in 2.0 compared with 1.5.
Previously in v1.5, user can leverage a spec.yaml to configure all the hyperparameters: https://msdata.visualstudio.com/Vienna/_git/aml-ds?path=/recipes/signed-components/canary-sweep-component-mnist/spec.yaml
In 2.0, user needs to manually configure everything in the code:

This is painful because:

It requires a lot of coding to migrate, compared to other component types.
This makes collaboration difficult, where people want to use different params.
The python script is not as neat as before.

fyi @jfomhover @clthorre @D-W-

Hi @fuhuifang, thank you for opening an issue! Is this feedback for a particular package in the Azure SDK, or is this regarding DevOps tooling?

Hi @mccoyp, it’s about a component type in SDK V2. Here’s an example https://github.com/Azure/azureml-examples/tree/main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep, but we submit the job via SDK instead of cli + the yaml definition.

Okay, thank you for the details! I'll tag some folks from the ML team who may be able to help -- we'll get back to you as soon as possible. @luigiw @azureml-github

I synced up with Fukui offline. The problem here seems to be the lack of sweep component support.
@wangchao1230, is it possible to allow customers to load a sweep job from a Yaml file and use the sweep job in a pipeline job?

@luigiw @fuhuifang the capability we provide for SDK and CLI is parity.
You can find related yaml sample here: https://github.com/Azure/azureml-examples/blob/main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep/pipeline.yml

Thank you for sharing the example, @cloga. Do you also have a DSL example, where a sweep job object is used in a pipeline dsl function?

@luigiw
I don't think we want customer to use sweep job object in SDK, instead we recommend customer to use builder style on command component command_component.sweep().
Pipeline DSL: https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/pipelines/1c_pipeline_with_hyperparameter_sweep/pipeline_with_hyperparameter_sweep.ipynb
Standalone sweep job: https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/single-step/scikit-learn/train-hyperparameter-tune-deploy-with-sklearn/train-hyperparameter-tune-with-sklearn.ipynb


rand_win.c: check if BCryptGenRandom is present before using it

As mentioned by HerMajestyDrMona it might be useful for some if openssl wouldn't prevent your app from working on XP. At least the code could be guarded with a check to allow delay-loading of bcrypt.

https://github.com/openssl/openssl/pull/8916

PR #8916

which one is your app?
I am on your side, but I need more data.

Pr Author non responsive, closing


Text inside angle brackets does not get printed

Applies To

[X] Notebooks (.ipynb files)
[X] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
In Notebooks and Interactive Windows, text between "<" and ">" does not show up in outputs.
Notebook:

Interactive Window:

I have not found a workaround to this problem. Escaping the angle brackets does not work.

This does not happen with jupyterlabs, or when running the notebook with jupyter-notebook.
Possibly related to https://github.com/microsoft/vscode-jupyter/issues/3149 where the same thing happened in Exception texts. Now exceptions seem to be the only place where it does not occur.

VS Code Version
1.75.0-insider
Jupyter Extension Version
v2023.1.1003661022
Jupyter logs
No response
Coding Language and Runtime Version
Python v3.10.6
Language Extension Version (if applicable)
No response
Anaconda Version (if applicable)
No response
Running Jupyter locally or remotely?
Remote

Duplicate of https://github.com/microsoft/vscode/issues/170263


docker-compose documentation broken

I saw a problem at the following URL: https://docs.docker.com/compose/install/compose-plugin/#installing-compose-on-linux-systems

Issues I found with this page:

A lot of subheaders/tabs missing for easy navigation
Numbering is all over the place
Code block contains instructions (bug?)
Random capitalization ("Install the Plugin")

Also on the landing page there is a spelling error as well: "plugin,s" should be "plugins"
I'm not sure what happened, yesterday morning the page was fine and all of a sudden it's a huge mess that is extremely unclear. I think even if these issues are fixed, the new installations instructions are a step backwards, a lot of clicks and navigation required before finding the installation instruction. Previously you immediately saw the curl command you needed to copy/paste.
Edit: I think this commit is the culprit: https://github.com/docker/docker.github.io/commit/d49af6a4495f653ffa40292fd24972b2df5ac0bc

Hi @mcrapts, you also created a PR (for these same remarks) where we have addressed your concerns. I saw that you left extra comments on the merged PR :). If I don't reach out for a sync mid July, please ping me.


Added support for ".h" and ".m" files.

Added support for ".h" and ".m" files at genstrings.swift file.
Devs will be able to read localised strings at objc files via this updated script.
To be able to use this, there should be ".localised" support at obj-c codes.

Consider review this merge rq which mentioned as a goal of the next release


[Internal] Query: Refactors certain tests to not fail when EnableOptimisticDirectExecution is set to true in 3.35.0-preview package

Pull Request Template
Description
When we enabled Optimistic Direct Execution in Preview Mode for Nuget package 3.35.0-preview, it caused 27 tests to fail. These tests were not related to Optimistic Direct Execution, but they were affected by the new behavior of choosing the Optimistic Direct Execution pipeline. To avoid losing test coverage, I updated these tests to not choose the Optimistic Direct Execution pipeline in Preview Mode.
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Closing issues
To automatically close an issue: closes #IssueNumber

Pull Request Template
Description
When we enabled Optimistic Direct Execution in Preview Mode for Nuget package 3.35.0-preview, it caused 27 tests to fail. These tests were not related to Optimistic Direct Execution, but they were affected by the new behavior of choosing the Optimistic Direct Execution pipeline. To avoid losing test coverage, I updated these tests to not choose the Optimistic Direct Execution pipeline in Preview Mode.
Type of change
Please delete options that are not relevant.

[x]  Bug fix (non-breaking change which fixes an issue)

Closing issues
To automatically close an issue: closes #IssueNumber

I am wondering if its a known gap in the feature thats why these tests are failing. Why are we disabling this feature instead of fixing the gap? This flag is by default "true" in public API (for preview package) does it mean that preview pkg is not in stable state right now?

@sourabh1007 These tests are failing because the new flag behaves differently and optimizes query execution. Because we only have one set of tests and one set of End To End Traces files, when they are run on for the Preview package release they fail because the XML files were not generated with the new feature on. This is exactly like when the Open Telemetry tests were failing on Preview release. Nothing to do with stability.


feat(docs): Firebase Storage docs

Documentation for the $firebaseStorage() service and firebase-src directive.

Coverage remained the same at 93.716% when pulling d4fc74fe0dcebe643e9a8eaa727ebdb149f984a1 on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.

Coverage remained the same at 93.716% when pulling 7c990e71a1d396f703ac212f36c34b5a2eeb2be7 on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.

Coverage remained the same at 93.716% when pulling 8b5244be5238b278d94eebe1da7f8152c6b3d5ca on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.

Coverage remained the same at 93.716% when pulling 5cfa40ed65abbb0908162aebd796dd4e17040b80 on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.

Coverage remained the same at 93.716% when pulling b34edbab969a83701990b84a7e43e406f15ff1dd on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.

Coverage remained the same at 93.716% when pulling 82c3cf1551e137bf98e44017fda5c660fa835f2c on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.

Coverage remained the same at 93.716% when pulling 73f2e777afd7c221f01cae13af41bffd0b9019f3 on storage-docs into 94ca5bc5b9a8df0461b688b7a046c71f729e95f9 on master.


Add log in UI

Add a log window that states when/what/where is written to HD.

Considering this ticket resolved in #39 , even though there's a bug that prevents HD Active from logging properly in the UI. That will be handled in a separate ticket.


Use localizable team.json

We have new version of team.json file now localizable https://github.com/cesko-digital/movapp-data/blob/main/data/team.v1.json
The schema has beed updated https://github.com/cesko-digital/movapp-data/blob/main/data/team.schema.json

Done


Visual studio vb net, all gray toolbox

I'm working on a project on Visual Studio with visual basic vb net language, I build in netcore 8. Basically the bar on the left, where you can add buttons and other things, has become all grey, as if it were disabled. It had happened before, just close and reopen, but now it always remains gray. How come this strange stuff happens?
Configuration
Windows 10

Hi @Slotxa, I could not really tell what is happening with the provided info, please attach a sample app that reproes the issue

Basically it was a problem I had always had but I solved it by restarting visual studio or the pc.
But now I managed to solve it by pressing left on "Reset toolbox". By pressing this button the toolbox reappeared normal, i.e. I can put new label buttons etc...
Thank you


Included bundle version to download bundle directory (#239)

Resolves https://github.com/HumanCellAtlas/dcp-cli/issues/239

Codecov Report

Merging #329 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #329      +/-   ##
==========================================
+ Coverage   84.49%   84.51%   +0.01%
==========================================
  Files          37       37
  Lines        1754     1756       +2
==========================================
+ Hits         1482     1484       +2
  Misses        272      272

Impacted Files
Coverage Δ

hca/dss/__init__.py
90.1% <100%> (+0.06%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a84fae9...a137985. Read the comment docs.


Changing cleaning mode during the start of the job

When I try to start a cleaning with Sweeping and Mopping as the cleaning mode, the vacuum starts preparing, but after a few seconds the cleaning mode changes to Sweeping only and the vacuum does the same. It doesn't matter if I start the cleaning from HA or from the Dreamehome app.
If I disable the integration in HA, this problem does not occur.

Model Name: L20 Ultra
Firmware Version: 4.3.6_1510; v2.0.0b14
Home Assistant Version: 2024.4.4

I just noticed that @s2kta4 already mentioned the issue (#542)


Java version and C++ version predict different probability values.

Hello, when i use  fastText4j , the probability of java version and original C++ version prediction is different.

What is the reason?

Hi,
It would help having additional information such as the model configuration (loss name, number of labels, with or without quantization etc) or an example to reproduce and identify what's going wrong.
Thanks

charMatches method invoke by computeSubwords .  char in java has 2 byte, i think this wil  skip over some  chinese word , because char use unicode characters , include chinese word .  so i think   protected boolean charMatches(char c) {     return (c & 0xC0) == 0x80;   }   should be   protected boolean charMatches(char c) {     return c >=128 && c <=191;   }

charMatches method invoke by computeSubwords . char in java has 2 byte, i think this wil skip over some chinese word , because char use unicode characters , include chinese word . so i think
protected boolean charMatches(char c) {
    return (c & 0xC0) == 0x80;
}

should be
protected boolean charMatches(char c) {
    return c >=128 && c <=191;
}

charMatches method invoke by computeSubwords . char in java has 2 byte, i think this wil skip over some chinese word , because char use unicode characters in java , include chinese word . so i think
protected boolean charMatches(char c) {
    return (c & 0xC0) == 0x80;
}

should be
protected boolean charMatches(char c) {
    return c >=128 && c <=191;
}

Hi,
in my understanding.
if ((word[i] & 0xC0) == 0x80) continue;

The purpose of this code  is to skip over the byte Cannot be a single word.
Because  in the UTF-8 encoding rule, the byte with a beginning of 10 can not be a single word.
So they don't take the feature in computeSubwords.
but in java, this code:
 protected boolean charMatches(char c) {
    return (c & 0xC0) == 0x80;
  }

will  skip over some word like chinese word. cause missing some word feature.

Hi,
in my understanding.
if ((word[i] & 0xC0) == 0x80) continue;

The purpose of this code is to skip over the byte Cannot be a single word.
Because in the UTF-8 encoding rule, the byte with a beginning of 10 can not be a single word.
this type of feature will not be taken in computeSubwords.
But in java, this code:
protected boolean charMatches(char c) {
    return (c & 0xC0) == 0x80;
}
will skip over some word like chinese word. cause missing some word feature.

the different size of char in java/c++(2 bytes and 1 byte) made this
is this fixed?

protected boolean charMatches(char c) {
    return false;
}

That works. The probability difference of  Facebook`s c++ version and fastText4j prediction results is less than 0.05

java8 does not meet such thing


updated links and display name for model tuning notebook

Description of the change

Motivation

Type of change
Choose one: (Bug fix | Feature request | Documentation | Other)
Checklist

[x] I have performed a self-review of my code.
[x] I have added detailed comments to my code where applicable.
[x] I have verified that my change does not break existing code.
[x] My PR is based on the latest changes of the main branch (if unsure, please run git pull --rebase upstream main).
[x] I am familiar with the Google Style Guide for the language I have coded in.
[x] I have read through the Contributing Guide and signed the Contributor License Agreement.

Ah looks like I also did some of this in #378. I'll merge this and rebase my PR


How to deinstall?

How to deinstall? Tried to click right button instead left but doesnt change anything :D

You just press this, finish your game and next game nothing should be there.

You just press this, finish your game and next game nothing should be there.

I did it and yet it still there

You just press this, finish your game and next game nothing should be there.

I did it and yet it still there

The radio box on the left is for hijacking injection, while the radio box on the right is for driver injection. The left button at the bottom is for opening, and the right button is for closing. Opening requires opening in the game hall, and closing requires closing after the game is closed

I clicked the right button two times when there was something wrote about logitech.dll file or whatever the name was lol

Issue not associated with the current project.


React UI: Add Starting Screen to Individual Pages

Currently, when the starting screen finishes it redirects to the home page, even if you didn't start there. Also, because readiness is checked only on the site's initial load, navigating to other pages returns an error as before. This PR adds the starting screen to each individual page, and on completion, the page reloads (to re-fetch data), leaving the user on the view they requested. As it's no longer needed, I removed the /starting page.
https://github.com/prometheus/prometheus/pull/8662#issuecomment-855255393

Hmmm, so now this is doing a request to /-/ready on every route change, rather than only on the initial page load (which would be sufficient). I can see two ways of preventing that: Either wrapping some global loader component / context around everything that conveys the loading state to the children (all pages), or maybe it's even a rare ok use case for a global JS variable that gets set to true when the app is ready and short-circuits subsequent fetches. If we went the context route, we already have a theme context (https://github.com/prometheus/prometheus/blob/d44b4c5f1da2b7e84ad86c73cfde05ca030595c1/web/ui/react-app/src/App.tsx#L68) and a path prefix context (https://github.com/prometheus/prometheus/blob/d44b4c5f1da2b7e84ad86c73cfde05ca030595c1/web/ui/react-app/src/App.tsx#L72) which are wrapped around everything, and I guess we could have a similar StartingContext. Or maybe there's an even better way to solve it?

Hmmm, so now this is doing a request to /-/ready on every route change, rather than only on the initial page load (which would be sufficient).

Ah, going from the initial comment I thought we wanted that behavior removed.
I don't see any clear advantage between a global variable and a context, besides that a global might be simpler. Both could start off true, and then become false if the check on the initial page load determines the server isn't ready, and then the starting screen could set it back to true when it's done.

Ah, going from the initial comment I thought we wanted that behavior removed.

Ah yeah, so I meant that each page should react to the current readiness state separately, but since we're a single-page app with multiple routes that can be navigated to without a full page reload, we can just load the readiness state once initially upon load of the SPA and then consume it from everywhere.

One other thing to consider is React not liking conditional hooks, which means that any time we include one of our useFetch hooks in a component, it has to be run no matter what. This is why /-/ready checks happen after the data is fetched (because of the possibility of returning early), even though if the server isn't ready, the data will never be used, making it an unnecessary fetch. We might just have to live with it though.

Yes, you always have to use the same hooks within one component, but the hooks can internally decide whether they need to do an actual HTTP fetch or not. So if we went for the global variable variant, I would put boolean ready global variable next to the useFetchReady() implementation that is set whenever the first actual fetch() returns that Prometheus is ready, and then whenever useFetchReady() is subsequently called, it sees that ready is already true, in which case it would no longer execute an actual fetch(), but just immediately return that Prometheus is ready. Since that's the smallest change we can do here and seems not too terrible (although I'd always be careful about global variables otherwise), I think that would be good for now. And then later we can still decide on whether it's cleaner to move it to a context.

I tried adding time.Sleep(30 * time.Second) in main.go before localStorage.Set(db, startTimeMargin) to simulate a startup sequence. It correctly shows the starting spinner on individual pages now, but if I navigate to a different page, it already tries to render that page (yielding a "Service Unavailable" error) although the startup hasn't finished yet... if I change the readiness condition check to the one I mentioned in https://github.com/prometheus/prometheus/pull/8909#discussion_r650398272, then it seems to do the right thing though.

Oh whoops, it doesn't. It then keeps reloading the whole page after everything is ready, so need to look a bit more.

Ok, so the issue is that the continuous update to the readiness state is only maintained in the Starting component, so the parent page never knows when to stop rendering the Starting component because no state updates to it are ever triggered. The most react-ish way is probably to make Starting a wrapper with children, like we already have for the loader of individual page contents (withStatusIndicator() wrapper).
Actually I think it would even be best to put all that startup logic into the existing withStatusIndicator() wrapper as well. Then it would render a startup spinner upon TSDB startup, a normal loading spinner after that, and then the actual page content after that. And we would never even need the full page reload anymore, as that is all just handled via React state updates and re-renderings.
We already use withStatusIndicator() for every page except /graph, but it should be easy enough to make it usable for /graph as well I think.

So we'll continue to use the useFetchReady function, and we can just keep ready as a piece of state in withStatusIndicator which, can then be passed to the Starting component that will set it to true when done.

@LeviHarrison Yep, that's exactly what I was thinking!

I just noticed though that withStatusIndicator() is not actually itself responsible for loading anything though, just for displaying an indicator. The loading & state handling is still done by the parent. Gah. We don't actually want to start loading any normal state before the TSDB loading is complete, so probably we can't actually get around adding a full wrapper around all the per-page components :/

So that would mean keeping a separate start-loader component after all, but then it would have to be wrapped around each page like this:
Where previously you had:
export default Status;
...you could have:
export default withStartingIndicator(Status);
And the withStartingIndicator could be similar to withStatusIndicator(), but actually contain the starting state fetching & handling.

I think we're one small test failure away from being done:
Failed to compile.

./src/utils/index.ts
  Line 5:10:  'FetchStateReady' is defined but never used  @typescript-eslint/no-unused-vars

I moved the withStartingIndicator calls to pages/index.ts (feel free to push back on this) and fixed tests.

I was locked in an endless cycle of eslint not wanting a trailing comma and prettier wanting one in pages/index.ts so I ended up added a prettier ignore.

Now there's just another lint check failing:
/home/circleci/project/web/ui/react-app/src/App.tsx
  7:9  error  Replace `·AlertsPage,·ConfigPage,·FlagsPage,·RulesPage,·ServiceDiscoveryPage,·StatusPage,·TargetsPage,·TSDBStatusPage,·PanelListPage·` with `⏎··AlertsPage,⏎··ConfigPage,⏎··FlagsPage,⏎··RulesPage,⏎··ServiceDiscoveryPage,⏎··StatusPage,⏎··TargetsPage,⏎··TSDBStatusPage,⏎··PanelListPage,⏎`  prettier/prettier

✖ 1 problem (1 error, 0 warnings)
  1 error and 0 warnings potentially fixable with the `--fix` option.

Yep, working on it now.

I got one more thing for the starting screen after this is merged, and then I think it's ready for release!


There is error running roslaunch my_localization.launch

Hello,everyone!
When I run roslaunch my_localization.launch,there is a error that IOError: [Errno 2] No such file or directory: u'./src/.config/model/default.urdf',how can I eliminate this error?

I removed urdf file from .config

use URDF file in  vehicle_description package!!


Systemd: invalid /etc/systemd/system/docker.socket generated

Cookbook version
docker cookbook 2.9.5
Chef-client version
Chef Zero 12.13.37
Platform Details
Debian Jessie 8 (under systemd init)
Scenario:
Trying to setup docker daemon using deb package + systemd
failing because of invalid systemd configuration generated
systemctl restart docker
Failed to restart docker.service: Unit docker.socket failed to load: Invalid argument. See system logs and 'systemctl status docker.socket' for details.

Steps to Reproduce:
Here is my code in my wrapper cookbook :
include_recipe "apt-docker"

docker_installation_package 'default' do
  package_version DnDockerHelper.to_package_version(node)
  action :create
end

docker_service_manager_systemd 'default' do
  host ["unix:///var/run/docker.sock", "tcp://<IP_ADDRESS>:2375"]
  graph "/home/docker"
  iptables false
  ipv4_forward true
  ipv6_forward true
  action :start
end

Expected Result:
/etc/systemd/system/docker.socket should contains
ListenStream=/var/run/docker.sock

Actual Result:
/etc/systemd/system/docker.socket should contains
ListenStream=

because it seems that templates/default/systemd/docker.socket-override.erb expect a docker_socket variable which is not provided in libraries/docker_service_manager_systemd.rb

resolved in v2.9.6


Handle OAuth-self-authenticated users

cc @ychiucco
When the associate_by_email action (within a OAuth authentication) fails, a new user is created in the DB. This user doesn't have any of the custom Fractal attributes (e.g. slurm_user, or the cache directory), and then they would not be able to run a workflow on a SLURM cluster.
Apart from that, should we be more strict in what this user can do with Fractal? The current freedom we have has to do with three user attributes:

is_superuser: currently required for a few specific endpoints
is_active: currently required for most (all?) endpoints
is_verified: currently not used, as far as I remember

If we set the self-authenticated user as non-active, they will have essentially no access to any endpoint. In this case, fractal-web should simply show a big warning page that says something on the line of "Please go and ask the admin to activate your account'.
This is a first proposal, and we can then iterate on it.

is_active=False upon self sign-up sounds good to me

How to proceed:

Introduce current_active_verified_user dependency
Use this new dependency in all endpoints under /api/v1/
Make sure that endpoints in /auth (especially GET /auth/current-user/) remain accessible to non-verified users
As of /admin/ endpoints, it's not really important to ask for verification (if you are a superuser it means that another admin granted you this role, and this essentially counts as being verified) -> let's not ask for is_verified=True for superuser-restricted endpoints.
Make sure that a user who self-authenticates via OAuth (and whose email address is not yet present in the DB) is created with is_verified=False

Note: "verified" here corresponds to "an admin verified this user account", rather than the common "I verified my user account by clicking a link my in email I received".

Good question @tcompa
For our target deployments (within given, typically academic institutions, not accessible on the public internet), option 2 seems fair to me. We could think about also limiting task selection (as a user could collect an arbitrary package, not sure if that is at all a security concern?). But them creating projects, datasets, workflows etc. seems unproblematic and indeed a nice thing to already be able to do.

As of this morning call:
Let's only prevent access to apply-workflow and collect-tasks endpoints, for non-verified users.


FR: Configurable syslog tag

I'd like to be able to configure the tag that gopassivedns passes to syslog. It appears it's currently hardcoded to /usr/sbin/gopassivedns[12406]:. Ideally, I'd be able to simply have them set as gopassivedns.

easy enough, I just need to add a config option and put it as the second option on https://github.com/Phillipmartin/gopassivedns/blob/master/log.go#L230.  I'll get to that as soon as I have free time (alternatively, PRs accepted)

https://github.com/Phillipmartin/gopassivedns/pull/28 should add this support.  please test and let me know how it works for you.


test: add nats runtime test

:tada: This PR is included in version 0.23.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket: