Member in a group cannot see the share which has been shared with the group

Description:
When a sharer shares a file with a group, the user in the group cannot find the share in the list of pending shares, or the shares in any state.
Steps to reproduce:

Create users Alice and user Brian
Add user Brian to group grp1
As user Alice, share a file textfile0.txt with group grp1
As user Alice, get all the shares shared by the user as follows:

curl -X GET http://localhost:20080/ocs/v1.php/apps/files_sharing/api/v1/shares -H OCS_APIREQUEST=true -u Alice:123456 | xmllint --format -

<ocs>
  <meta>
    <status>ok</status>
    <statuscode>100</statuscode>
    <message>OK</message>
  </meta>
  <data>
    <element>
      <id>8</id>
      <share_type>1</share_type>
      <uid_owner>Alice</uid_owner>
      <displayname_owner>Alice Hansen</displayname_owner>
<EMAIL_ADDRESS>      <permissions>19</permissions>
      <stime>1624869903</stime>
      <parent/>
      <expiration/>
      <token/>
      <uid_file_owner>Alice</uid_file_owner>
      <displayname_file_owner>Alice Hansen</displayname_file_owner>
<EMAIL_ADDRESS>      <state>0</state>
      <path>/textfile0.txt</path>
      <item_type>file</item_type>
      <mimetype>text/plain</mimetype>
      <storage_id>123e4567-e89b-12d3-a456-426655440000!0cfb1e45-2623-4ccf-be1b-d9f0075ae5d3</storage_id>
      <storage>0</storage>
      <item_source>MTIzZTQ1NjctZTg5Yi0xMmQzLWE0NTYtNDI2NjU1NDQwMDAwOjBjZmIxZTQ1LTI2MjMtNGNjZi1iZTFiLWQ5ZjAwNzVhZTVkMw==</item_source>
      <file_source>MTIzZTQ1NjctZTg5Yi0xMmQzLWE0NTYtNDI2NjU1NDQwMDAwOjBjZmIxZTQ1LTI2MjMtNGNjZi1iZTFiLWQ5ZjAwNzVhZTVkMw==</file_source>
      <file_parent/>
      <file_target>/textfile0.txt</file_target>
      <share_with>grp1</share_with>
      <share_with_displayname>grp1</share_with_displayname>
      <share_with_additional_info/>
      <mail_send>0</mail_send>
      <name/>
    </element>
  </data>
</ocs>

As user Brian, get all the shares shared with the user:

curl -X GET http://localhost:20080/ocs/v1.php/apps/files_sharing/api/v1/shares\?format\=json\&shared_with_me\=true\&state\=all -H OCS_APIREQUEST=true -u Brian:1234 -v;

 HTTP/1.1 200 OK
{"ocs":{"meta":{"status":"ok","statuscode":100,"message":"OK"},"data":[]}}%

This might be related to https://github.com/cs3org/reva/issues/1769

@jasson99 is this expected?
As I'm the owner of the share, I don't see a reason why it should appear in the "Shared with me"?

@jasson99 is this expected?
As I'm the owner of the share, I don't see a reason why it should appear in the "Shared with me"?

It is expected that the share receiver gets the shares in "shared with me". Brian is the expected share receiver in the above case. Alice has shared the file with group grp1 and Brian is in the group grp1

@jasson99 what happens in oC10 core?

@jasson99 what happens in oC10 core?

curl -X GET http://localhost/oc/ocs/v1.php/apps/files_sharing/api/v1/shares\?format\=json\&shared_with_me\=true\&state\=all -u Brian:1234 -v;

< HTTP/1.1 200 OK

{"ocs":{"meta":{"status":"ok","statuscode":100,"message":null,"totalitems":"","itemsperpage":""},"data":[{"id":"95","share_type":1,"uid_owner":"Alice","displayname_owner":"Alice","permissions":19,"stime":1624875397,"parent":null,"expiration":null,"token":null,"uid_file_owner":"Alice","displayname_file_owner":"Alice","additional_info_owner":null,"additional_info_file_owner":null,"state":0,"path":"\/textfile0.txt","mimetype":"text\/plain","storage_id":"shared::\/textfile0.txt","storage":48,"item_type":"file","item_source":2147537570,"file_source":2147537570,"file_parent":2147537577,"file_target":"\/textfile0.txt","share_with":"grp1","share_with_displayname":"grp1","mail_send":0,"attributes":null}]}}%

@jasson99 can you check the logs to see if the group grp1 appears in the list of groups forBrian? Because we have this in prod at the moment and it works as expected

https://github.com/cs3org/reva/issues/1769 is related to the fact that we need to modify the schema for our sharing db to accommodate accepting shares as well. We didn't have the concept of accepting shares previously so there are a few nuances that need to be figured out.

maybe there is an issue in the ldap config. @ishank011 could you have a look at https://github.com/cs3org/reva/blob/master/tests/oc-integration-tests/drone/ldap-users.toml to see if you can spot an issue

My ldapserver has the following objects:
# LDIF Export for dc=owncloud,dc=com
# Server: <IP_ADDRESS> (<IP_ADDRESS>)
# Search Scope: sub
# Search Filter: (objectClass=*)
# Total Entries: 7
#
# Generated by phpLDAPadmin (http://phpldapadmin.sourceforge.net) on June 29, 2021 7:20 am
# Version: 1.2.5

version: 1

# Entry 1: dc=owncloud,dc=com
dn: dc=owncloud,dc=com
dc: owncloud
o: ownCloud
objectclass: top
objectclass: dcObject
objectclass: organization

# Entry 2: cn=admin,dc=owncloud,dc=com
dn: cn=admin,dc=owncloud,dc=com
cn: admin
description: LDAP administrator
objectclass: simpleSecurityObject
objectclass: organizationalRole
userpassword: {SSHA}U6kyFappS9OMXddm+Z4aqE2MtlqKixML

# Entry 3: ou=TestGroups,dc=owncloud,dc=com
dn: ou=TestGroups,dc=owncloud,dc=com
objectclass: top
objectclass: organizationalUnit
ou: TestGroups

# Entry 4: cn=grp1,ou=TestGroups,dc=owncloud,dc=com
dn: cn=grp1,ou=TestGroups,dc=owncloud,dc=com
cn: grp1
gidnumber: 5000
memberuid: Brian
objectclass: posixGroup
objectclass: top

# Entry 5: ou=TestUsers,dc=owncloud,dc=com
dn: ou=TestUsers,dc=owncloud,dc=com
objectclass: top
objectclass: organizationalUnit
ou: TestUsers

# Entry 6: uid=Alice,ou=TestUsers,dc=owncloud,dc=com
dn: uid=Alice,ou=TestUsers,dc=owncloud,dc=com
cn: Alice
displayname: Alice Hansen
gidnumber: 5000
homedirectory: /home/openldap/Alice
mail<EMAIL_ADDRESS>objectclass: posixAccount
objectclass: inetOrgPerson
sn: Alice
uid: Alice
uidnumber: 30000
userpassword: 123456

# Entry 7: uid=Brian,ou=TestUsers,dc=owncloud,dc=com
dn: uid=Brian,ou=TestUsers,dc=owncloud,dc=com
cn: Brian
displayname: Brian Murphy
gidnumber: 5000
homedirectory: /home/openldap/Brian
mail<EMAIL_ADDRESS>objectclass: posixAccount
objectclass: inetOrgPerson
sn: Brian
uid: Brian
uidnumber: 30001
userpassword: 1234

I am running using this config file: https://github.com/cs3org/reva/blob/master/tests/oc-integration-tests/local/ldap-users.toml .
While querying the groups using the api, I get empty result, i.e no groups are returned.
curl -X GET http://localhost:20080/ocs/v1.php/cloud/groups -u Brian:1234 -v;
< HTTP/1.1 200 OK

<ocs><meta><status>error</status><statuscode>998</statuscode><message>Not found</message></meta></ocs>%

curl -X GET http://localhost:20080/ocs/v1.php/cloud/users/Brian/groups -u Brian:1234 -v;
< HTTP/1.1 200 OK
<ocs><meta><status>ok</status><statuscode>100</statuscode><message>OK</message></meta><data><groups></groups></data></ocs>%

I am especially confused in this line as mail has been used for group filter. But group doesnot have mail property in my case.

@jasson99 @individual-it https://github.com/cs3org/reva/blob/master/tests/oc-integration-tests/local/ldap-users.toml#L37 is what needs to be fixed. For a given user, we use this filter to retrieve the list of their groups, and we're searching for posix groups using the user's OpaqueId, which doesn't work.
We need to use the memberof overlay as described here https://github.com/owncloud/ocis/blob/master/storage/pkg/flagset/ldap.go#L66-L67

@jasson99 can you try setting groupfilter (https://github.com/cs3org/reva/blob/master/tests/oc-integration-tests/local/ldap-users.toml#L37) to (&(objectclass=posixGroup)(cn=*)(memberuid={{.Username}}))?

2021-06-30 09:44:52.77 ERR ../../../internal/grpc/interceptors/recovery/recovery.go:50 > template: gf:1:45: executing "gf" at <.Username>: can't evaluate field Username in type *userv1beta1.UserId
error executing group template: userid:idp:"http://localhost:18000" opaque_id:"Alice"
github.com/cs3org/reva/pkg/user/manager/ldap.(*manager).getGroupFilter
	/home/jasminebaral/www/jankaritech-reva/reva/pkg/user/manager/ldap/ldap.go:401
github.com/cs3org/reva/pkg/user/manager/ldap.(*manager).GetUserGroups
	/home/jasminebaral/www/jankaritech-reva/reva/pkg/user/manager/ldap/ldap.go:358
github.com/cs3org/reva/internal/grpc/services/userprovider.(*service).GetUserGroups
	/home/jasminebaral/www/jankaritech-reva/reva/internal/grpc/services/userprovider/userprovider.go:162
github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1._UserAPI_GetUserGroups_Handler.func1
	/home/jasminebaral/go/pkg/mod/github.com/cs3org/go-cs3apis@v0.0.0-20210614143420-5ee2eb1e7887/cs3/identity/user/v1beta1/user_api.pb.go:690
github.com/cs3org/reva/internal/grpc/interceptors/auth.NewUnary.func1
	/home/jasminebaral/www/jankaritech-reva/reva/internal/grpc/interceptors/auth/auth.go:106
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/recovery/interceptors.go:33
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/cs3org/reva/internal/grpc/interceptors/log.NewUnary.func1
	/home/jasminebaral/www/jankaritech-reva/reva/internal/grpc/interceptors/log/log.go:39
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/cs3org/reva/internal/grpc/interceptors/token.NewUnary.func1
	/home/jasminebaral/www/jankaritech-reva/reva/internal/grpc/interceptors/token/token.go:44
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/cs3org/reva/internal/grpc/interceptors/appctx.NewUnary.func1
	/home/jasminebaral/www/jankaritech-reva/reva/internal/grpc/interceptors/appctx/appctx.go:36
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1
	/home/jasminebaral/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34
github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1._UserAPI_GetUserGroups_Handler
	/home/jasminebaral/go/pkg/mod/github.com/cs3org/go-cs3apis@v0.0.0-20210614143420-5ee2eb1e7887/cs3/identity/user/v1beta1/user_api.pb.go:692
google.golang.org/grpc.(*Server).processUnaryRPC
	/home/jasminebaral/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1024
google.golang.org/grpc.(*Server).handleStream
	/home/jasminebaral/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1313
google.golang.org/grpc.(*Server).serveStreams.func1.1
	/home/jasminebaral/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722
runtime.goexit

I get this error in reva logs when I adjust the line 37: https://github.com/cs3org/reva/blob/master/tests/oc-integration-tests/local/ldap-users.toml#L37 to (&(objectclass=posixGroup)(cn=*)(memberuid={{.Username}}))

My bad. The template is applied to userID, not username. (&(objectclass=posixGroup)(cn=*)(memberuid={{.OpaqueId}})) should work. Can you try that @jasson99?

Sure. Thankyou

That works really well. Lots of scenarios start passing in PR #1881 where the CI settings are adjusted.


[BUG] Issue to conversion workflow starts but does not finish.

See the report here.
https://github.com/brainhackorg/global2021/runs/4178661874?check_suite_focus=true
@jhlegarreta pinging you but I can try to start debugging a bit later.

A few notes:

I had already observed this. Cross-referencing https://github.com/brainhackorg/global2021/pull/72#issuecomment-963754572.

As mentioned in the cross-referenced comment, when one tries the script (as provided by the refactoring in PR #72) with the following changes (I had to test on the 2020 repository since there were no projects when I made the changes):

Make the URL point to the 2020 repository.
Change the project repository regex in https://github.com/brainhackorg/global2021/blob/6106d92623711d22b6818e5d67f63b74cf15b9f2/scripts/issues_to_pages.py#L204 to Link to the project to account for the change in the section name of the template
Changes the filter in here https://github.com/brainhackorg/global2021/blob/6106d92623711d22b6818e5d67f63b74cf15b9f2/scripts/issues_to_pages.py#L251 to status:published , and moves the proj_url_re_match.group() block in https://github.com/brainhackorg/global2021/blob/d46594d9d7c62809b07a674d5794ef81b4dfef8e/scripts/issues_to_pages.py#L207 inside the if to avoid accessing the group member in case of Nones,

the script does its job. I know, too many things to make it work with the 2020 repository, but it works. So I thought there was something else that made the script hang.

I did not have a clear idea of what was going on, and assumed the changes in the mentioned PR were not introducing that bug, since I was able to run it successfully with the changes above.

Some testing shows that the script gets stuck precisely in https://github.com/brainhackorg/global2021/blob/6106d92623711d22b6818e5d67f63b74cf15b9f2/scripts/issues_to_pages.py#L204, the issue["body"] contains the issue body as expected, so the regex is having trouble to do its job. Thus,

Given that no project was present prior to the changes in PR #72, the bug was already there before them.
I had tested the regex when doing PR #55 (cross-referencing https://github.com/brainhackorg/global2021/pull/32#issuecomment-932805057), but maybe when testing on the issues in your fork the body of the issue changed slightly to just make the regex work.

Thanks for the detailed explanation @jhlegarreta.
BTW:  I did not mean to imply you had overlooked something.


appSwitch usage

I want to use appSwitch to create three separate switches (for HDMI 3, AV 1, and Live TV) so I'm declaring this in my config
            "appSwitch": [
                "com.webos.app.hdmi3",
                "com.webos.app.livetv",
                "com.webos.app.externalinput.av1"
            ]

Should that work? I only see a single switch in Home app, displaying all three items comma separated. Pressing the switch changes the input once, but does not cycle through.

Yes, this should work. I suspect that you somewhere have a typo and instead of an array a single string is read. Please recheck your config file.

Ah looks like I miss an update! v0.9.2 -> v0.9.3 resolved this issue.


[DOCS]Add images and the Relevant Data of the cabs.

Hey,
I want to add the relevant images and data like the cost of the cab per day. This will help the user to choose between the comfortable rides, and cabs available.
Please assign this task to me, as I want to work on it.

assign me this work @07sumit1002

Hi @07sumit1002 I would like to work on this issue.

assign me this work @07sumit1002


Add exporting to .ics

Temporary fix for #8
(My code is horrible, I know)

what if there were two separate hooks for ics export and gcal export so when the domain stuff is figured out it'll be ready too? could we also combine toICSEvents and toGAPIEvents? seems like two or three small differences

agreed with @Li357 that we want to have two different hooks. biggest issue is the large blocks of code commenting, and putting it on a separate hook would mean we don't have to comment out the old code.
i dont think the shared code is enough to combine them?

maybe not, was assuming that either createEvents from ics or gapi.client.calendar.events.insert would just ignore unknown/additional properties for each event from the other format. then start, end and maybe recurrenceRule would be a conditional


updated links to point to migrated tutorials on docs.ms

Docs Build status updates of commit e27c568:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-autogen/microsoft-cognitiveservices-speech-sdk/index.yml
:warning:Warning
View (azure-node-latest)
Details

docs-ref-autogen/overview/CognitiveServices/BingVisualSearch.yml
:warning:Warning
View (azure-node-latest)
Details

docs-ref-conceptual/index.yml
:white_check_mark:Succeeded
View (azure-node-latest) View (azure-node-preview)

docs-ref-conceptual/toc.yml
:white_check_mark:Succeeded
View (azure-node-latest) View (azure-node-preview)

docs-ref-autogen/microsoft-cognitiveservices-speech-sdk/index.yml

[Warning] Uid(microsoft-cognitiveservices-speech-sdk) has already been defined in docs-ref-autogen/microsoft-cognitiveservices-speech-sdk/index.yml.

docs-ref-autogen/overview/CognitiveServices/BingVisualSearch.yml

[Warning] Uid(azure.nodejs.sdk.landingPage.services.Cognitive.Services.Bing.VisualSearch) has already been defined in docs-ref-autogen/overview/CognitiveServices/CustomVisionTraining.yml.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Feature] changes frontend > colors to design > color tools

Description
Maybe we should create a color tools section on design and send frontend > colors to it, erasing frontend > colors since now we have this separation between design and frontend.
I'm willing to work on this one, if it seems ok.
Screenshots
No response
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (optional)

Hey @GabrielBittar, it sounds nice, do you want to work on it?

Hey @GabrielBittar, it sounds nice, do you want to work on it?

Yes, please :)

@GabrielBittar you're assigned.


Add vectorization info in _interpretation property (in GraphQL and REST)

It would be helpful if the actually parsed string is part of GraphQL's __meta function. So that it is understood what Weaviate used as the actual string to index (not only for classification but for indexation in general).
Can be implemented as part of #1136

Update: Switched from _meta to _interpretation since

there was a conflict with the deprecated meta field (go-swagger internally removes the underscore, then they're called the same)
it's less ambiguous and more descriptive


Add Crick and N4BI samples

See https://deploy-preview-54--euphonious-mochi-664405.netlify.app/?csv=https://raw.githubusercontent.com/ome/ome2024-ngff-challenge/0e14f3376c70a3c524a08bc79d367ec14b549b1a/samples/ngff_samples.csv

Minor issues:

thumbnails (for bf2raw collections?) aren't loading
CSV URLs ending with "/" fail
top-level CSV of the current branch isn't used

But those are all outside of this PR.
cc: @will-moore @constantinpape @folterj @JThoennissen

with #56 merged, getting this in, then we can re-arrange with the hierarchy browser later.

NB: https://radosgw.public.os.wwu.de/n4bi-wp1/challenge/n4bi.csv links to https://radosgw.public.os.wwu.de/n4bi-fzj/ome2024-ngff-challenge/fzj.csv which contains https://radosgw.public.os.wwu.de/n4bi-fzj/ome2024-ngff-challenge/BigBrain.zarr which is Not Found 404.
I'll remove that from our copy of fzj.csv...

Sorry, I didn't manage to get the conversion working in time for this volume ... Thank you for removing!

NB: I made local copies of crick.csv and n4bi.csv in order to add extra columns needed in #53.
See https://github.com/ome/ome2024-ngff-challenge/blob/main/samples/crick.csv and https://github.com/ome/ome2024-ngff-challenge/blob/main/samples/n4bi.csv
The script used to add these extra columns is at https://github.com/ome/ome2024-ngff-challenge/blob/main/samples/load_zarr_stats.py

Ok. I'm deleting my CSVs files from the S3 bucket then to avoid confusion.


Correction of node documentation

Describe the bug
https://taiko.xyz/docs/guides/run-a-node/run-a-taiko-node
Step 3 on this page.
・L2_ENDPOINT_HTTP
・L2_ENDPOINT_WS
If it were written the way it is now, I would mistakenly think that the L3 setting should be the same URL as the L2.
Shouldn't it be noted in the L3 that an INTERNAL PORT is required as a precaution?
L3 Copying and pasting Alchemy's APIKEY as is did not work.
For example
L2_ENDPOINT_HTTP=http://l2_execution_engine:8545/
L2_ENDPOINT_WS=ws://l2_execution_engine:8546 worked fine.
I don't think the explanation of the L3 configuration is sufficient.
Second.
https://taiko.xyz/docs/guides/run-a-node/enable-a-prover
1step
You should write that it is nano .env.l3 nano .env.
env alone is not sufficient.It is hard to understand if you don't write down to nano.
Thank you for your consideration.

Thanks for highlighting this. Will make the changes ser

Glad I was helpful! I would like a POAP

Check out #14400

check done. Thanks for writing the pull request!
can I get a GitPOAP?

I was able to check, and would appreciate it if you could link me to this page.
Can I get a gitpoap?


[SPARK-47435][SQL][3.5] Fix overflow issue of MySQL UNSIGNED TINYINT  caused by SPARK-45561

What changes were proposed in this pull request?

SPARK-45561 mapped java.sql.Types.TINYINT to ByteType in MySQL Dialect, which caused unsigned TINYINT overflow. As regardless of signed or unsigned types, the TINYINT is used for java.sql.Types.
In this PR, we put the signed info into the metadata for mapping TINYINT to short or byte.
Why are the changes needed?

bugfix
Does this PR introduce any user-facing change?

Uses can read MySQL UNSIGNED TINYINT values after this PR like versions before 3.5.0 which has breaked since 3.5.1
How was this patch tested?

new tests
Was this patch authored or co-authored using generative AI tooling?

no

Thank you @dongjoon-hyun

The python-linter is irrelevant.
Thank you @dongjoon-hyun, merged to 3.5.2.


Fix Unicode issues (#58)

This should fix the issues mentioned in #58. I've manually tested the upload of an image file which contains umlauts in the filename, and getting posts by a tag containing umlauts.
Unfortunately, I had problems mocking the open function in the UnitTests. That part was commented, so I guess I wasn't the only one. ;) With open mocked, mimetypes.guess_types hangs infinitely. I could work around that by mocking guess_types as well, though that means mocking away part of what we want to test. It's better than nothing, though.

Thank you for your submission (from years ago), a recent PR that was merged looks like it works with unicode tags and filenames


fix deprecated Iterable import

see https://docs.python.org/3/whatsnew/3.3.html#collections

Pull Request Test Coverage Report for Build 9

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 8:
0.0%

Covered Lines:
57

Relevant Lines:
57

💛  - Coveralls


Feat: Administrative Adjustments and Initiative Agreements

Added backend api views, services, repo, schema for AdminAdjustment and Initiative Agreement
Frontend AddEditViewTransaction component expanded for api data integration and state management
ButtonConfigs for all status' setup
Seeders and model update with a migration

When I'm on analyst role, the transaction history isn't available to view. Is this expected?


aws-sdk-cpp: 1.7.90 -> 1.8.113

Motivation for this change
Regular update.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests): awscli, nix-ssh-serve
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 107623 run on x86_64-linux 1

1 package marked as broken and skipped:

nix-exec

2 packages blacklisted:

tests.nixos-functions.nixos-test
tests.nixos-functions.nixosTest-test

2 packages failed to build:

python37Packages.nixpkgs
python39Packages.nixpkgs

63 packages built:

aws-c-cal
aws-c-common
aws-c-event-stream
aws-c-io
aws-checksums
aws-sdk-cpp
bundix
cabal2nix
cachix
common-updater-scripts
crate2nix
crystal2nix
dep2nix
disnix
disnixos
dydisnix
fusionInventory
go2nix
haskellPackages.cachix
haskellPackages.hercules-ci-agent
haskellPackages.hocker
haskellPackages.nix-paths
haskellPackages.update-nix-fetchgit
hercules-ci-agent
hydra-unstable
libnixxml
lispPackages.quicklisp-to-nix
lispPackages.quicklisp-to-nix-system-info
nix (nixStable)
nix-bundle
nix-direnv
nix-doc
nix-du
nix-index
nix-pin
nix-plugins
nix-prefetch
nix-prefetch-bzr
nix-prefetch-cvs
nix-prefetch-docker
nix-prefetch-git
nix-prefetch-hg
nix-prefetch-scripts
nix-prefetch-svn
nix-serve
nix-update
nix-update-source
nixFlakes (nixUnstable)
nixos-generators
nixos-shell
nixpkgs-review
nixui
python37Packages.nix-kernel
python37Packages.pythonix
python38Packages.nix-kernel
python38Packages.nixpkgs
python38Packages.pythonix
python39Packages.nix-kernel
python39Packages.pythonix
s2n
update-nix-fetchgit
vgo2nix
vulnix

python37Packages.nixpkgs
[pbr] Reusing existing SOURCES.txt
running build_ext
nixpkgs (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: nixpkgs (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: nixpkgs
Traceback (most recent call last):
  File "/nix/store/401hv5vy6jm6wyppmlaalakdlsrzchc6-python3-3.7.9/lib/python3.7/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/nix/store/401hv5vy6jm6wyppmlaalakdlsrzchc6-python3-3.7.9/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/build/nixpkgs-0.2.4/nixpkgs/__init__.py", line 41, in <module>
    import nix
ModuleNotFoundError: No module named 'nix'

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>

I have reported the issue about python37Packages.nixpkgs and python39Packages.nixpkgs which happens independently from this PR at https://github.com/Mic92/pythonix/issues/6

@ofborg eval

@ofborg eval

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 107623 run on x86_64-linux 1

1 package marked as broken and skipped:

nix-exec

2 packages blacklisted:

tests.nixos-functions.nixos-test
tests.nixos-functions.nixosTest-test

6 packages failed to build and are new build failures:

cachix: log was empty
haskellPackages.cachix: log was empty
haskellPackages.hercules-ci-agent: log was empty
hercules-ci-agent: log was empty
python37Packages.nixpkgs: log https://hastebin.com/raw/ivatahitas
python39Packages.nixpkgs: log https://hastebin.com/raw/bayatucidi

59 packages built:

aws-c-cal
aws-c-common
aws-c-event-stream
aws-c-io
aws-checksums
aws-sdk-cpp
bundix
cabal2nix
common-updater-scripts
crate2nix
crystal2nix
dep2nix
disnix
disnixos
dydisnix
fusionInventory
go2nix
haskellPackages.hocker
haskellPackages.nix-paths
haskellPackages.update-nix-fetchgit
hydra-unstable
libnixxml
lispPackages.quicklisp-to-nix
lispPackages.quicklisp-to-nix-system-info
nix (nixStable)
nix-bundle
nix-direnv
nix-doc
nix-du
nix-index
nix-pin
nix-plugins
nix-prefetch
nix-prefetch-bzr
nix-prefetch-cvs
nix-prefetch-docker
nix-prefetch-git
nix-prefetch-hg
nix-prefetch-scripts
nix-prefetch-svn
nix-serve
nix-update
nix-update-source
nixFlakes (nixUnstable)
nixos-generators
nixos-shell
nixpkgs-review
nixui
python37Packages.nix-kernel
python37Packages.pythonix
python38Packages.nix-kernel
python38Packages.nixpkgs
python38Packages.pythonix
python39Packages.nix-kernel
python39Packages.pythonix
s2n
update-nix-fetchgit
vgo2nix
vulnix

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:
python38Packages.nixpkgs:

Zero tests run by pytest got detected: 'Ran 0 tests in 0.000s'
You have the following options to fix this:

Tell pytest(CheckHook) where to find the tests included in the package
Check if the GitHub Repo contains tests but they are not shipped with Pypi. If so please switch to fetchFromGitHub.
If the Packages does not contain any tests add 'doCheck = false;'and apythonImportsCheck`.

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 107623 run on x86_64-linux 1

1 package marked as broken and skipped:

nix-exec

2 packages blacklisted:

tests.nixos-functions.nixos-test
tests.nixos-functions.nixosTest-test

6 packages failed to build and are new build failures:

cachix: log was empty
haskellPackages.cachix: log was empty
haskellPackages.hercules-ci-agent: log was empty
hercules-ci-agent: log was empty
python37Packages.nixpkgs: log https://hastebin.com/raw/ivatahitas
python39Packages.nixpkgs: log https://hastebin.com/raw/bayatucidi

59 packages built:

aws-c-cal
aws-c-common
aws-c-event-stream
aws-c-io
aws-checksums
aws-sdk-cpp
bundix
cabal2nix
common-updater-scripts
crate2nix
crystal2nix
dep2nix
disnix
disnixos
dydisnix
fusionInventory
go2nix
haskellPackages.hocker
haskellPackages.nix-paths
haskellPackages.update-nix-fetchgit
hydra-unstable
libnixxml
lispPackages.quicklisp-to-nix
lispPackages.quicklisp-to-nix-system-info
nix (nixStable)
nix-bundle
nix-direnv
nix-doc
nix-du
nix-index
nix-pin
nix-plugins
nix-prefetch
nix-prefetch-bzr
nix-prefetch-cvs
nix-prefetch-docker
nix-prefetch-git
nix-prefetch-hg
nix-prefetch-scripts
nix-prefetch-svn
nix-serve
nix-update
nix-update-source
nixFlakes (nixUnstable)
nixos-generators
nixos-shell
nixpkgs-review
nixui
python37Packages.nix-kernel
python37Packages.pythonix
python38Packages.nix-kernel
python38Packages.nixpkgs
python38Packages.pythonix
python39Packages.nix-kernel
python39Packages.pythonix
s2n
update-nix-fetchgit
vgo2nix
vulnix

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:
python38Packages.nixpkgs:

Zero tests run by pytest got detected: 'Ran 0 tests in 0.000s'
You have the following options to fix this:

Tell pytest(CheckHook) where to find the tests included in the package
Check if the GitHub Repo contains tests but they are not shipped with Pypi. If so please switch to fetchFromGitHub.
If the Packages does not contain any tests add 'doCheck = false;'and apythonImportsCheck`.

@orivej Can you fix the merge conflict?

@orivej Can you fix the merge conflict?

This broke nix cross compilation: https://github.com/NixOS/nixpkgs/issues/106759#issuecomment-764519504

This broke nix cross compilation: https://github.com/NixOS/nixpkgs/issues/106759#issuecomment-764519504

#110307 should fix it.

#110307 should fix it.


Go 1.5 compatibility (internal/* package visibility)

Attempting to build go-endpoints with 1.5 fails:
=> Fetching dependencies for github.com/GoogleCloudPlatform/go-endpoints/endpoints...
package github.com/GoogleCloudPlatform/go-endpoints/endpoints
imports google.golang.org/appengine/internal: use of internal package not allowed
package github.com/GoogleCloudPlatform/go-endpoints/endpoints
imports google.golang.org/appengine/internal/user: use of internal package not allowed

Since the Go AppEngine Managed VM default runtime is now using Go 1.5, code referencing this library now will no longer build with the default runtime. Can we please get this issue resolved?

+1

+2? :+1:

any news? :)
https://groups.google.com/forum/m/#!topic/google-appengine-go/bY_OqNw2Qzo

This is an issue on App Engine Standard as well since the update to 1.6. Application code will compile and run in dev and production environments. However, a go install or goapp install will fail. Which affects code completion in editors.

Thanks for the report and sorry for the delay, working on this now!


Maintenance work

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (ce54994) 0.00% compared to head (64e35cb) 88.46%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##           main       #5       +/-   ##
=========================================
+ Coverage      0   88.46%   +88.46%
=========================================
  Files         0        1        +1
  Lines         0       26       +26
=========================================
+ Hits          0       23       +23
- Misses        0        3        +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Doc enhancement: Add property order lists in property-sort-order rule doc

close  https://github.com/sasstools/sass-lint/issues/234
These lists are useful for people being refactoring their base code. So a link to exhaustive lists of ordered properties in the documentation could be useful.
DCO 1.1 Signed-off-by: Nicolas Fortin<EMAIL_ADDRESS>

Thanks @zallek


python API missing is_read_compacted in create_reader()

Describe the bug
The python API has is_read_compacted in the Consumer API, but not in the Reader API.
I think this may be an accidental omission.  I checked the Java API docs, both ConsumerBuilder and ReaderBuilder have readCompacted
Similarly, the C++ API has setReadCompacted in ReaderConfiguration.cc
To Reproduce
import pulsar

client = pulsar.Client('pulsar://localhost:6650')

msg_id = pulsar.MessageId.earliest

reader = client.create_reader('my-topic', msg_id, is_read_compacted=True)

while True:
    msg = reader.read_next()
    print("Received message %r id=%s" % (msg.data(), msg.message_id()))

Gives TypeError: create_reader() got an unexpected keyword argument 'is_read_compacted'
Expected behavior
To work the same as the consumer API.  The following works:
consumer = client.subscribe('my-topic', 'my-subscription', is_read_compacted=True)

Desktop (please complete the following information):

OS: Ubuntu 18.04, Pulsar 2.4.1, python 3.6.8, and pulsar-client==2.4.1

will look into it.

@candlerb , have you tried read_compacted? it is defined as this in the code:
    class_<ReaderConfiguration>("ReaderConfiguration")
            .def("message_listener", &ReaderConfiguration_setReaderListener, return_self<>())
            .def("schema", &ReaderConfiguration::getSchema, return_value_policy<copy_const_reference>())
            .def("schema", &ReaderConfiguration::setSchema, return_self<>())
            .def("receiver_queue_size", &ReaderConfiguration::getReceiverQueueSize)
            .def("receiver_queue_size", &ReaderConfiguration::setReceiverQueueSize)
            .def("reader_name", &ReaderConfiguration::getReaderName, return_value_policy<copy_const_reference>())
            .def("reader_name", &ReaderConfiguration::setReaderName)
            .def("subscription_role_prefix", &ReaderConfiguration::getSubscriptionRolePrefix, return_value_policy<copy_const_reference>())
            .def("subscription_role_prefix", &ReaderConfiguration::setSubscriptionRolePrefix)
            .def("read_compacted", &ReaderConfiguration::isReadCompacted)
            .def("read_compacted", &ReaderConfiguration::setReadCompacted)

Yes I had already tried it, and it fails.
Code:
import pulsar

client = pulsar.Client('pulsar://localhost:6650')

msg_id = pulsar.MessageId.earliest

reader = client.create_reader('my-topic', msg_id, read_compacted=True)

while True:
    msg = reader.read_next()
    print("Received message %r id=%s" % (msg.data(), msg.message_id()))

Exception:
$ python3 read.py
Traceback (most recent call last):
  File "read.py", line 7, in <module>
    reader = client.create_reader('my-topic', msg_id, read_compacted=True)
TypeError: create_reader() got an unexpected keyword argument 'read_compacted'

This is with pulsar-client (2.4.1)

I notice in pulsar-client-cpp/python/pulsar/__init__.py that subscribe has this option:
    def subscribe(self, topic, subscription_name,
                  consumer_type=ConsumerType.Exclusive,
                  schema=schema.BytesSchema(),
                  message_listener=None,
                  receiver_queue_size=1000,
                  max_total_receiver_queue_size_across_partitions=50000,
                  consumer_name=None,
                  unacked_messages_timeout_ms=None,
                  broker_consumer_stats_cache_time_ms=30000,
                  negative_ack_redelivery_delay_ms=60000,
                  is_read_compacted=False,   <<<<<
                  properties=None,
                  pattern_auto_discovery_period=60,
                  initial_position=InitialPosition.Latest
                  ):

but create_reader does not:
    def create_reader(self, topic, start_message_id,
                      schema=schema.BytesSchema(),
                      reader_listener=None,
                      receiver_queue_size=1000,
                      reader_name=None,
                      subscription_role_prefix=None
                      ):

But it does create a ReaderConfiguration:
        conf = _pulsar.ReaderConfiguration()

I think it's just missing an argument to pass on.  (Or maybe it should have **kwargs and pass them all?)

I made a fix, it's pretty simple (PR to follow).  However I spotted another problem:
        if reader_listener:
            conf.reader_listener(_listener_wrapper(reader_listener, schema))

In the code you posted before, should
            .def("message_listener", &ReaderConfiguration_setReaderListener, return_self<>())

be changed to
            .def("reader_listener", &ReaderConfiguration_setReaderListener, return_self<>())

?  Or should the python code be changed to use message_listener instead?

Thanks @candlerb for the verify.
And Yes, Your change is needed.
            .def("reader_listener", &ReaderConfiguration_setReaderListener, return_self<>())

👍


minimum presenterm-export version (0.2.0) not met

Getting this on macOS and Windows with latest python.
> python --version
Python 3.12.4
> pip show presenterm-export
Name: presenterm-export
Version: 0.2.1
> presenterm -e .\demo.md
minimum presenterm-export version (0.2.0) not met

Happens still if I downgrade presenterm-export
> pip install --force-reinstall -v "presenterm-export==0.2.0"
> pip show presenterm-export
Name: presenterm-export
Version: 0.2.0
> presenterm -e .\demo.md
minimum presenterm-export version (0.2.0) not met


StandardScaler on returns ? Is this correct ?

Please accept my apologies for the frequent messages. I am writing to you regarding a doubt that has arisen after analyzing your code.
While reviewing your code, I observed that you have used StandardScaler on both features and the return target variable in the function set_scalers(self, df). This approach is perfectly acceptable. However, I have concerns regarding the calculation of captured returns in the custom loss function defined in SharpeLoss class inside deep_momentum_network file.
Specifically, the function call(self, y_true, weights) uses scaled returns in the first argument. The captured returns are then calculated using this scaled return and the position size, captured_returns = weights * y_true. This results in a return that is not accurate since it is based on scaled returns rather than actual returns.
Furthermore, there is no provision to unscale the returns or position in the code as the only function that does that is format_predictions(self, predictions), which does never get called.

Hey @makovez, take a look at how that SharpeLoss.call output is being used. It's not used to report the return of predictions on the test set. Nor is it used to evaluate performance on the validation set. It's being used to determine if the lowest level model fit is improving. For that, all that it needs to "know" is that losses are headed in the right direction and, if so, to continue fitting.
That is why that SharpeLoss calculation is distinct from the SharpeValidationLoss calculation in deep_momentum_network which uses positions * actual returns and for the performance reporting calculations in get_positions() which uses position predictions * actual returns.
Note: when you dig into the SharpeValidationLoss function you'll likely encounter volatility re-scaling calculations used to determine positions. These are used to ensure the super-model is selecting sub-models that appropriately factor-in each asset's risk (volatility). It avoids selecting models merely b/c they have the greatest return in the lookback window.

Hi,
Thanks for your answer.

It's not used to report the return of predictions on the test set.
I never said this. I said the issue is with scaling of returns.

Returns is the target variable, and it's being scaled, before everything else is done. So whether you are looking at  SharpeValidationLoss, get_positions(), SharpeLoss.call every time you see the word "returns" that's the scaled one and not the original real return. Which obviously would lead to a different captured return.
Wherever you are in the code, data is taken from this instance of this class

For example with either: model_features.test_sliding, model_features.test_fixed, model_features.train, model_features.valid
These are being defined here below

As you can see in this snippet of the code:
train, valid, test, test_with_buffer = [
            self.transform_inputs(data)
            for data in [train, valid, test, test_with_buffer]
        ]

For each data, every time, self.transform_inputs(data) is called, which in addition to scaling all the features, also scales the target variable.

Thank you.
Actually I just notice that TARGET is never being scaled. Because it's always being excluded from scaling.

target_returns are scaled, just not with standardScaler in model_inputs.modelFeatures(). It's done in data_prep via classical_strategies.calc_vol_scaled_returns() for the reasons above.


Data consistency issue  in backend API search results and the stored documents.

Describe the bug
Data consistency issue  in backend API search results and the stored documents.
We have been using our own internal parsing engine(with custom tokenization rules),  formatting the results into Odinson JSON documents, and then loading them into an Odinson index.  While everything works well for the most part, occasionally I've been seeing match spans that appear to be off by one or two. I initially assumed an edge case issue in my code, but digging deeper I think there is a conflict between backend and core. After looking at things in more detail, it became apparent that extra tokens are being generated in the query response's "words" field in at least the execute/pattern API call. As I mentioned, we have a custom tokenization which allows for embedded whitespace in some tokens. These tokens (along with some other cases) appear to be getting split when generating the API response for a query. The api's sentence call results show the index contains the un-split tokens. It also appears that the match offsets returned in the query response align correctly with the un-split tokenization as well.
I believe that the Lucene analyser is being applied to the already parsed tokens when the OdinsonController calls the core's DataGatherer.getTokens() call - which is further spliting things.  That seems odd given that you are already dealing with pre-tokenized fields, but it my be necessary for some other aspect of the code base I haven't looked at.
I'm attaching an example scoreDoc entry from a  query result as well as the corresponding sentence request output. You can see that the token. 'Figure 3' becomes two separate tokens in the query response each time it appears in the sentence.
Expected behavior
I would expect Odinson backend API to respect the tokenization of the loaded docs and not re-define it.
ExampleSentenceResponse.txt
ExampleQueryResponse.txt

Hi, @curtkohler .  Thank you for reporting this.  We'll dig in.  Just to confirm: the sentence in ExampleSentenceResponse.txt has the ID 5035?

Yes, that was returned by the API call:  http://XX.XX.XX.XX:8001/api/sentence?sentenceId=5035. Happy to provide the entire original JSON document that was indexed if needed, but I suspect just creating a test JSON doc with the one sentence (less the two additional fields we've added) would be enough to work with.  Also, there are a couple of other Figure references in that particular sentence that we treat as a single token that are getting split up as well.

Hi, @curtkohler. Odinson's IndexWriter currently uses lucene's WhitespaceAnalyzer, which will break any tokens on whitespace, as the name suggests. Currently, the choice of Analyzer isn't configurable. A possible workaround is to extend IndexWriter, overriding the value of writerConfig such that it uses KeywordAnalyzer instead, which preserves whitespace:
import org.apache.lucene.analysis.core.KeywordAnalyzer
import org.apache.lucene.store.Directory
import ai.lum.odinson.digraph.Vocabulary
import ai.lum.odinson.utils.IndexSettings
import ai.lum.odinson.OdinsonIndexWriter

class KeywordOdinsonIndexWriter(
  override val directory: Directory,
  override val vocabulary: Vocabulary,
  override val settings: IndexSettings,
  override val normalizedTokenField: String,
  override val addToNormalizedField: Set[String],
  override val incomingTokenField: String,
  override val outgoingTokenField: String,
  override val maxNumberOfTokensPerSentence: Int,
  override val invalidCharacterReplacement: String,
  override val displayField: String
) extends OdinsonIndexWriter(
  directory,
  vocabulary,
  settings,
  normalizedTokenField,
  addToNormalizedField,
  incomingTokenField,
  outgoingTokenField,
  maxNumberOfTokensPerSentence,
  invalidCharacterReplacement,
  displayField
) {
  val kwAnalyzer = new KeywordAnalyzer()
  override val writerConfig = new IndexWriterConfig(kwAnalyzer)
}

The reason why retrieving the sentence with api/sentence still preserves your custom tokenization is that that endpoint loads your original input JSON from disk rather than loading it from the index, in case there were fields in the input that weren't stored in the index.

Dane, Thanks for the quick reply and suggestion on a workaround.  A couple of observations. I guess would have expected Odinson to leverage the Lucene Keyword Analyzer by default since the input documents are already tokenized. It seems counter-intuitive that the system re-tokenizes previously tokenized content. It is also odd that the match offsets being returned in the API's JSON query response reflect the original source document tokenization strategy and not what was apparently put into the index based on your reply. In my simple query UI, had I made an extra API call to the /sentence endpoint and generated the display from that, the highlighting would have been correct.

@BeckySharp The lucene IndexWriter needs a IndexWriterConfig, which will have some Analyzer in any case (StandardAnalyzer if none is specified IIRC). I'm not as familiar with DataGatherer -- would it be possible to make the choice of Analyzer configurable? Would an Analyzer that keeps things as-is cause problems?
@curtkohler I believe you're right that the KeywordAnalyzer would be more appropriate. Unless I hear otherwise, I'll begin a pull request to make it so.

@danebell I think you can swap it in the DG no problem, but unit tests will tell us for sure.
One thing though @curtkohler the metadata query language won't support the spaces within the tokens, so you'd have to come up with a workaround there.  Again, that's only for the metadata queries.

@BeckySharp - Yes, I would expect the metadata query capability to continue to use one of the other Lucene tokenizers as it doesn't appear to be designed to handle pre-tokenized data (things like title, author names, etc.) and that isn't a problem for us.

Actually, they should be tokenized, but whitespace within a token isn't supported (last I checked, a test could be added).
see, e.g.,:

https://github.com/lum-ai/odinson/blob/master/core/src/test/scala/ai/lum/odinson/metadata/TestMetadataFilter.scala#L44
#329

@danebell has this been worked on?  what needs to be done?

I believe this has now been fully addressed. See this pull request, since merged.


The files in "vendor/bin" point to a path on the host, and therefore cannot be found

Description
If I run composer install from inside Nitro (through a SSH connection from a Windows host) the files in the folder vendor/bin seem to be created in a weird state. They are symlinks but their target is a Windows path. Running ll prints the following (I've only replaced the actual project folder with my-project):
ubuntu@nitro-dev:~/sites/my-project/vendor/bin$ ll
total 20
drwxrwxrwx 1 ubuntu ubuntu  0 Aug 15 21:43 ./
drwxrwxrwx 1 ubuntu ubuntu  0 Aug 15 20:50 ../
lrwxrwxrwx 1 ubuntu ubuntu 83 Aug 15 20:49 composer -> D:/projects/my-project/website/vendor/composer/composer/bin/composer
lrwxrwxrwx 1 ubuntu ubuntu 79 Aug 15 20:49 jsonlint -> D:/projects/my-project/website/vendor/seld/jsonlint/bin/jsonlint
lrwxrwxrwx 1 ubuntu ubuntu 79 Aug 15 20:49 markdown -> D:/projects/my-project/website/vendor/cebe/markdown/bin/markdown
lrwxrwxrwx 1 ubuntu ubuntu 96 Aug 15 20:49 validate-json -> D:/projects/my-project/website/vendor/justinrainbow/json-schema/bin/validate-json
lrwxrwxrwx 1 ubuntu ubuntu 69 Aug 15 20:49 yii -> D:/projects/my-project/website/vendor/yiisoft/yii2/yii

But trying to access those files directly fails:
ubuntu@nitro-dev:~/sites/my-project/vendor/bin$ ll markdown
lrwxrwxrwx 1 ubuntu ubuntu 79 Aug 15 20:49 markdown -> D:/projects/my-project/website/vendor/cebe/markdown/bin/markdown

ubuntu@nitro-dev:~/sites/my-project/vendor/bin$ cat markdown
cat: markdown: No such file or directory

Running composer install also prints the following error (reformatted to make it more readable):
[ErrorException]
file_put_contents(/home/ubuntu/sites/my-project/vendor/bin/markdown): failed to open stream: No such file or directory

Exception trace:
()
  at phar:///usr/local/bin/composer/src/Composer/Installer/BinaryInstaller.php:167

Composer\Util\ErrorHandler::handle()
  at n/a:n/a

file_put_contents()
  at phar:///usr/local/bin/composer/src/Composer/Installer/BinaryInstaller.php:167

Composer\Installer\BinaryInstaller->installUnixyProxyBinaries()
  at phar:///usr/local/bin/composer/src/Composer/Installer/BinaryInstaller.php:161

Composer\Installer\BinaryInstaller->installSymlinkBinaries()
  at phar:///usr/local/bin/composer/src/Composer/Installer/BinaryInstaller.php:88

Composer\Installer\BinaryInstaller->installBinaries()
  at phar:///usr/local/bin/composer/src/Composer/Installer/LibraryInstaller.php:170

Composer\Installer\LibraryInstaller->ensureBinariesPresence()
  at phar:///usr/local/bin/composer/src/Composer/Installer/InstallationManager.php:147

Composer\Installer\InstallationManager->ensureBinariesPresence()
  at phar:///usr/local/bin/composer/src/Composer/Installer.php:313

Composer\Installer->run()
  at phar:///usr/local/bin/composer/src/Composer/Command/InstallCommand.php:122

Composer\Command\InstallCommand->execute()
  at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:245

Symfony\Component\Console\Command\Command->run()
  at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:835

Symfony\Component\Console\Application->doRunCommand()
  at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:185

Symfony\Component\Console\Application->doRun()
  at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:281

Composer\Console\Application->doRun()
  at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:117

Symfony\Component\Console\Application->run()
  at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:113

Composer\Console\Application->run()
  at phar:///usr/local/bin/composer/bin/composer:61

require()
  at /usr/local/bin/composer:24

Ultimately, this prevents some features and functions to work correctly, like updating Craft to a newer version.
Also, if I try to open that file on the host in VS Code I get the following error:
Unable to open 'composer': Unable to read file 'd:\Projects\my-project\website\vendor\bin\composer' (EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, open 'd:\Projects\my-project\website\vendor\bin\composer').

Trying to open that file in Windows Notepad prints (roughly translated): "The syntax of the filename, directory name or volume label is wrong."
Steps to reproduce
On a Windows host:

run nitro ssh
cd into your project folder
run composer install

Additional info

Host: Windows 10 2004
Nitro version: 1.0.1
Multipass version: 1.4.0+win

Small update on this one:
Running composer install from the host system works. But updating Craft from the UI still fails, because it still tries to run Composer inside Nitro and causes the problems mentioned above (probably if it tries to update one of the binaries).

@alinnert out of curiosity, did you set the file mutex as described in the Windows 10 docs? https://craftcms.com/docs/nitro/usage.html#adding-sites

I don't know if I did that correctly. I read the one box below and added IS_WINDOWS=true to my .env. Now that you mention it, I cannot tell if that's a replacement for the entire mutex setting or just for the single hardcoded true value and I still have to add that block but read the value from my .env manually.

Having the same issue

@jasonmccallister I just tried it again (removed and re-added the db, reverted all project files) but with the isWindows option set to true exactly like in the docs. Sadly, it doesn't resolve the problem. composer install only works from the host system, updating Craft fails.

@alinnert same. I did a complete fresh setup, might abandon Nitro until a better fix comes along.

@alinnert I'm going to have to spend some time diving into that issue. I don't use Windows too often but I definitely want to get this resolved.

I've ran into similar issues when running composer install inside the VM while working with Vagrant too recently. I haven't migrated to nitro yet since I have some sites using Vagrant and I need Hyper-V off for some VMs.
Anyway I gave up and just changed the vendor directory path to somewhere outside the shared folder between the vm and the host. I've chosen /usr/local/lib/craft.

Make sure the directory you chose exists and has the necessary permissions.

I've overridden the CRAFT_VENDOR_PATH PHP constant in craft (CLI) and web/index.php to  point to the new location
Added the vendor-dir config options in composer.json, also pointing to the new location

This way I still keep composer.json in version control, but when doing composer install/update all the dependencies will be installed outside the synced vm mount.
I haven't tested this extensively yet, but I was able to run composer install then install craft with no issues.

We just released Nitro 2 where we switched from Multipass to Docker and added a ton of new features. Please see the announcement for more details.
This issue was specific to Multipass so we are going to close this out. Please see the documentation on upgrading from Nitro 1.


static-method-regex should be PascalCase, not UPPERCASE

Rule Suggestion
Is your rule for a general problem or is it specific to your development style?
I have not seen any libraries that use uppercase for static methods, however I believe that PascalCase is quite standard for static methods.
The current regex is:
https://github.com/microsoft/tslint-microsoft-contrib/blob/master/src/functionNameRule.ts#L77
let staticMethodRegex: RegExp = /^[A-Z_\d]+$/;
Honestly I think this is a bug but I'd like to have the discussion.

:wave: @danmarshall, ☠  TSLint is deprecated ☠  and we strongly suggest switching  to https://typescript-eslint.io. tslint-microsoft-contrib is no longer accepting feature requests or bug reports for issues that are not critical security or TypeScript compatibility failures.
Happy linting!

https://github.com/palantir/tslint/issues/4534


chore: test node 19 on CI

Summary
https://nodejs.org/en/blog/announcements/v19-release-announce/
I made some changes in #13467. Hopefully enough (but I don't think so, as mentioned there)

Test plan
Green CI?

Passed 😀


schema text isn't described [phab:T220]

Created by pludemann at 2017-02-22 21:53:50:
Schema doc node has this, but nowhere is text defined (text seems to be used for things other than doc, such as lookup).
//- DocNode.node/kind doc
//- DocNode.text " A function.\n It sums its parameters `[x]` and `[y]`."

fromberger wrote at 2017-02-24 02:59:53:
I think this probably falls under the more general task of documenting the marked-source message. Recategorizing on that basis.

revision #1829

Fixed by #1829.


Releasing cri-tools v0.1

We are about to release first version for kubernetes v1.7.
TODOs

[ ] Bump tools version to 0.1.0
[ ] Add release notes
[ ] Add changelog

/cc @yujuhong @Random-Liu @mrunalp @xlgao-zju

Cool. Will update the version in cri-containerd to v0.1.0 after the release!


Added back_button_adaptive

Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.
Usually when we need to replace the leading button to override the onPressed function in the appbar we need to put again the back icon manually, this can be solved using an adaptive function as other widgets
List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.
If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] I followed the breaking change policy and added Data Driven Fixes where supported.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

I close this PR, I already noticed the existence of the Icon.adaptive icons that already do this function. I will create a medium post to explain this and show about the existence of these icons that are not too common.

You can check my article: https://medium.com/@nestorsgarzonc/ensuring-platform-consistency-with-adaptive-icons-in-flutter-16a8dba373e8


[BUG] VIP is temporarily not accessible when upgrading

Describe the bug
VIP is temporarily not accessible when upgrading.
The VIP will be unreachable when we upgrade with stage apply manifest.
That will cause some services which depend on VIP to go down.
To Reproduce

create a v1.1.2 cluster
upgrade to v1.2.0-rc3
Check the VIP connection, it should be down with stage apply manifest.

Expected behavior
VIP should not be down for more than 30 seconds.
Support bundle
Environment

Harvester ISO version: v1.2.0-rc3
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): 3 VM cluster

Additional context

I can reproduce this issue with these steps:

Create a single node v1.1.2 cluster.
Run this script to bump the Harvester chart to v1.2.0-rc3.

VIP is gone and back, but I can't ping it (even in the node):
4: mgmt-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default q
len 1000
    link/ether 52:54:00:6a:59:7c brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global mgmt-br
       valid_lft forever preferred_lft forever
    inet <IP_ADDRESS>/32 scope global mgmt-br
       valid_lft forever preferred_lft forever

4: mgmt-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default q
len 1000
    link/ether 52:54:00:6a:59:7c brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global mgmt-br
       valid_lft forever preferred_lft forever

4: mgmt-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default q
len 1000
    link/ether 52:54:00:6a:59:7c brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global mgmt-br
       valid_lft forever preferred_lft forever
    inet <IP_ADDRESS>/32 scope global mgmt-br
       valid_lft forever preferred_lft forever

Reason: The Harvester pod adds the annotation kube-vip.io/ignore-service-security: "true" to the service ingress-expose, but the kube-vip can't handle the modification.
Workaround: Delete kube-vip pod.

Test Plans
Single nodes upgrade

[ ] Check can reach the VIP during the upgrading
Test Result $~~$ $\color{black}{\textsf{Ongoing}}$

Multi nodes upgrade

[ ] Check can reach the VIP during the upgrading
Test Result $~~$ $\color{black}{\textsf{Ongoing}}$

Replace Harvester image after restart the kube-vip

[ ] Check after the Harvester updates the service and restart the kube-vip.
Refer to test plan: https://github.com/harvester/harvester/pull/4393#issue-1842412620
Test Result $~~$ $\color{black}{\textsf{Ongoing}}$

Verified fixed on Harvester v1.2-57b26312-head (08/14). Close this issue.
Result
We passed all the test scenarios listed in https://github.com/harvester/harvester/issues/4221#issuecomment-1680103629
The VIP accessibility did not lost during the upgrade service stage (apply manifest job) in:

[x] 1. On single node we can reach the VIP during the upgrading Harvester,

The VIP only stop within 60 seconds then come back, this would be expected since it's single node and restart the kube-vip

[x] 2. On multiple nodes we can always reach the VIP during the upgrading Harvester and the follow up upgrade nodes stage.

Test Information

Test Environment: Three nodes on bare machines and single node on local kvm
Harvester version: v1.2-57b26312-head (08/14)

Verify Steps
 Single node steps

Prepare a single node v1.1.2 Harvester
Upgrade to v1.2-57b26312-head
ssh to management node, and run k9s
Search jobs
Open terminal and ping the VIP
Check can ping VIP during the entire upgrade service process
When the apply manifest job exists (After upgrade service started), check it's status

 Multiple nodes steps

Prepare a three node v1.1.2 Harvester
Upgrade to v1.2-57b26312-head
ssh to management node, and run k9s
Search jobs
Open terminal and ping the VIP
Check can ping VIP during the entire upgrade service process
When the apply manifest job exists (After upgrade service started), check it's status
Check can ping VIP during the follow up upgrade node process


More friendly test coverage report

study go tool cover and trying to build more friendly unit test report

[ ] unit test need include below dimensions

[ ]  package level
[ ]  file level
[ ]  method level

[ ] TBD :  coverage trend report when project is in the version control.

https://excalidraw.com/#json=VFlDPpauXCvEbzilTvGlw,j_aH_12eSJW0jx8q5K9ouQ


[BUG] No registered resource provider found for location 'chinanorth2' and API version '2021-11-01' for type 'namespaces'

Describe the bug
We upgraded azure-resourcemanager from 2.17.0 to a newer version. Then our application failed to start, with the following error. The problem occurs in any version between [2.18.0, 2.22.0]. It occurs in Mooncake, and does NOT occur in public cloud.
After reverting its version to 2.17.0, our application started to work again.
2022-12-30 14:05:27.041 ERROR 53642 --- [           main] o.s.boot.SpringApplication               : Application run failed

...
Caused by: com.azure.core.management.exception.ManagementException: Status code 400, "{"error":{"code":"NoRegisteredProviderFound","message":"No registered resource provider found for location 'chinanorth2' and API version '2021-11-01' for type 'namespaces'. The supported api-versions are '2014-09-01, 2015-08-01, 2017-04-01, 2018-01-01-preview, 2021-01-01-preview, 2021-06-01-preview, 2022-01-01-preview'. The supported locations are 'chinanorth, chinaeast, chinanorth2, chinaeast2, chinanorth3'."}}"
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710)
	at com.azure.core.implementation.http.rest.ResponseExceptionConstructorCache.invoke(ResponseExceptionConstructorCache.java:56)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    ...
	at java.base/java.lang.Thread.run(Thread.java:829)
	Suppressed: java.lang.Exception: #block terminated with an error
		at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
		at reactor.core.publisher.Mono.block(Mono.java:1742)
		at com.azure.resourcemanager.resources.fluentcore.arm.collection.implementation.GroupableResourcesImpl.getById(GroupableResourcesImpl.java:62)
		at com.azure.resourcemanager.resources.fluentcore.arm.collection.implementation.GroupableResourcesImpl.getById(GroupableResourcesImpl.java:28)
        ...

Exception or Stack Trace
Add the exception log and stack trace if available
To Reproduce
Steps to reproduce the behavior:
Code Snippet
        ServiceBusNamespace ns = azureProvider.getAzure(subscriptionId)
            .serviceBusNamespaces()
            .getById(resourceId);

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):

OS: [e.g. iOS]
IDE: [e.g. IntelliJ]
Library/Libraries: [e.g. com.azure:azure-core:1.16.0 (groupId:artifactId:version)]
Java version: 11
App Server/Environment: [e.g. Tomcat, WildFly, Azure Function, Apache Spark, Databricks, IDE plugin or anything special]
Frameworks: Spring Boot

Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[ ] Bug Description Added
[ ] Repro Steps Added
[ ] Setup information Added

Hi @domainname thank you for reporting this issue.  @weidongxu-microsoft could you please follow up?

servicebus is using a newer api-version 2021-11-01 since 2.18.0. Seems like service backend didn't roll out this api version to Mooncake. This needs service attention.
As a workaround before service fix, you can try explicitly specifying to depend on 2.17.0 of azure-resourcemanager.

Hi @Saglodha , @EldertGrootenboer , any updates on this?

I've also mentioned this in other threads as well. But currently the available api-version depends on backend.
@XiaofeiCao One thing we can think/improve on SDK side is whether we can allow customer select an older api-version (there probably some risk and difficulty with this approach, due to breaking changes or potential change from single version to multiple version combined, so we'd likely needs more discussion).

Thanks for bringing this to our attention @domainname
I would check with the team on the roll out status for the China cloud and would circle back with an update soon,

Hi @Saglodha , do you have any updates on this issue?

@domainname, I checked with the team and could know that this has been rolled out. Could you please retry and confirm if you have been facing any issues now?


dark mode is not available in ubantu linux

i installed my onenote app in my ubantu machine after login into my microsoft account the previous notes that i made on my windows machine are note in the same format,and there is no option available to initialize the dark mode

Hi Manthan,
This app is just a wrapper for Microsoft OneNote's web application. As far as I know, there is no dark mode option in the web version of OneNote, which is why this app does not have that option either. I have not modified Microsoft's app in any way, and I haven't been able to force a dark mode without injecting some css into the container. I need some more time to get that working,
Anuj


Update requirements.txt to httpx version compatible with proxies keyword

Trying to run any apps with the current or earlier version of xRx and Groq for STT will cause an error due to a dependency issue.
There was a breaking change in httpx v0.28.0 that impacts Groq library. The proxies parameter was removed in this version, which impacts SDK functionality.
When httpx is v0.28.0 or later, Groq library gives error TypeError: Client.__init__() got an unexpected keyword argument 'proxies'. So running any app gives the following error when trying to use Groq for STT:
Result:
xrx-stt           |     model = self.get_model()  # Get the shared model instance
xrx-stt           |   File "/stt/groq_stt.py", line 22, in get_model
xrx-stt           |     cls._model = Groq(api_key=API_KEY)
xrx-stt           |   File "/usr/local/lib/python3.10/site-packages/groq/_client.py", line 99, in __init__
xrx-stt           |     super().__init__(
xrx-stt           |   File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 824, in __init__
xrx-stt           |     self._client = http_client or SyncHttpxClientWrapper(
xrx-stt           |   File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 722, in __init__
xrx-stt           |     super().__init__(**kwargs)
xrx-stt           | TypeError: Client.__init__() got an unexpected keyword argument 'proxies'```

Fix:
The fix is simple. We install latest version of httpx still compatible with other dependencies in the package. (`httpx==0.27.2`). This was the version likely used when the framework was developed but was not set in the requirements.txt. The issue is resolved with this change.

thanks for getting this @bklieger-groq, looks good

per our offline conversation, we're going to opt to upgrade groq instead of downgrading httpx. closing in favor of #27


Update e2e test

What this PR does / why we need it:
Add the status check of created storage resource in e2e test.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

Coverage remained the same at 58.166% when pulling ebef369d029015f887e11c9d3cf74d7f0e65d218 on leonwanghui:e2e-test into 92561c037673fbc8e45872b212ef629e15fdbf10 on opensds:development.

Coverage remained the same at 58.166% when pulling ebef369d029015f887e11c9d3cf74d7f0e65d218 on leonwanghui:e2e-test into 92561c037673fbc8e45872b212ef629e15fdbf10 on opensds:development.

Coverage remained the same at 58.166% when pulling ebef369d029015f887e11c9d3cf74d7f0e65d218 on leonwanghui:e2e-test into 92561c037673fbc8e45872b212ef629e15fdbf10 on opensds:development.

Coverage remained the same at 58.166% when pulling ebef369d029015f887e11c9d3cf74d7f0e65d218 on leonwanghui:e2e-test into 92561c037673fbc8e45872b212ef629e15fdbf10 on opensds:development.


feat: meaningful error in redirect-based URI handler

Context
This PR supersedes https://github.com/ipfs-shipyard/ipfs-companion/pull/422 and adds a useful error page for rare cases when redirect-based URI handler was unable to process URI.
Note that redirect-based handler is specific to Firefox  as Chrome does not support manifest.json/protocol_handlers.
In my experience this specific type of error is displayed only when someone is playing with protocol handler and enters address by hand to the address bar and makes a typo or puts foobar as a CID.
Before

Opening ipfs://invalid-cid redirected  to https://ipfs.io/#redirect/ipfs%3A%2F%2Finvalid-cid, but because invalid-cid is not a valid IPFS resource, it was not redirected further.
The result was very confusing. Invalid CID just displayed our main page (seen above)
Now

Opening ipfs://invalid-cid redirects  to https://gateway.ipfs.io/ipfs/QmXQY7mKr28B964Uj4ouq3fPgkNLqzaKiajTA7surAiQuD#ipfs%3A%2F%2Finvalid-cid which:

displays meaningful error message and some helpful steps
is a regular IPFS resource, so it does not require HTTP-based hosting and will be redirected to local gateway if IPFS Companion is active

Note: we can't bundle this error page with browser extension, because webRequest/onBeforeRequest with <all_urls>  ignores requests made to moz-extension://* (this seems to be a security feature to block extensions from interacting with each others pages)

gateway.ipfs.io looks better in protocol handler picker

Next
I feel we should not stress too much about this error page due to its rare use.
Design can be improved in a separate PR, but I would wait with that until our visual language matures.

Nice!


Dartium & Dart2js discrepancies

I've found two discrepancies in dart2js and Dartium whilst playing around with my project: https://github.com/jimmyff/cellular_automata
Firstly the Dartium version is significantly faster than the JS version by about a factor of 10 in terms of FPS.
Also the generator function provides different results:
(x, y) => ((x ^ y) + y - x) == 0 || ((x ^ y) + y - x) % y == 0

Correct Dartium version:

Incorrect Dart2js version:

Are x and y negative?
In dart2js we had to limit the range and domain of bitwise operators like '^'.
These operations are expected to be fast, so we cannot do much more than use the underlying JavaScript operations, which are reduce the inputs to signed 32 bit values and produce a signed 32 bit result.
We found that the signed 32 bit values were hard to use and caused some important bit-vector programs to fail, so we decided to force the values to unsigned 32 bit values instead.
With either restriction (signed 32 bit or unsigned 32 bit) there are some programs that fail.
The speed difference is probably due to dart2js being an ahead-of-time compiler and Dart VM in Dartium being ajJust-in-time compiler.  The JIT can see x and y are small integers with a special efficient representation, so after a couple of quick checks to verify this assumption, the rest of generated machine code is similar to what you would get in C.
You can get closer to the VM's great performance in dart2js:

declare x and y to be int:  (int x, int y) => ...
Compile with --trust-type-annotations
Some remaining slowdown is probably due to % (modulo) in ((x ^ y) + y - x) % y == 0.  You could try using remainder instead, which compiles to JavaScript % (remainder). Since the inputs can be negative, you can't generally replace modulo with remainder, but in this case it works because the result is compared to zero.

With regards to the discrepancy with the generator functions: Yes x & y can be negative, 0,0 is the middle of the grid. I still don't understand why dartium vs dart2js shows a different result?  My dart generator function is ported from a javascript function that I know works correctly:
(x, y) => ((x ^ y) + y - x) % y == 0

(I've tried implementing .remainder() instead of modulo but it hasn't altered the output.)
With regards to performance, it's actually the processing of the running cellular automata (eg: conway's game of life) which is highlighting the performance discrepancy. As per your suggestion I've added trust-type-annotations and trust-primitives. A heavily populated 512x512 grid runs at full speed (capped at 20fps) on Dartium however 4fps on chrome -on my macbook. I haven't optimised the processing of the automata which I know will make a difference big difference in terms of performance gains (for both) but I'm surprised that Dartium & JS are so far apart - I guess I'll just have to work around JS performance being much slower that JIT Dart.

As Dartium is soon going away I thought I better quickly do the work to find this discrepancy. I've got the generated values by both Chrome + Dartium for comparison:
ENDLESS_SIERPINSKI
MathematicalGenerators.ENDLESS_SIERPINSKI: (int x, int y) =>
        (x ^ y) + x - y == 0

Chrome results
Darium results
Diff
Example Chrome:
(-64 ^ 64) + -64 - 64 =<PHONE_NUMBER>
(-64 ^ 63) + -64 - 63 =<PHONE_NUMBER>

Example Dartium:
(-64 ^ 64) + -64 - 64 = -256
(-64 ^ 63) + -64 - 63 = -128

SIERPINSKI_MOUNTAINS
MathematicalGenerators.SIERPINSKI_MOUNTAINS: (int x, int y) =>
        ((x ^ y) + y - x) == 0 || ((x ^ y) + y - x).remainder(y) == 0,

Chrome results
Dartium results
Diff
Example Chrome:
((-64 ^ 64) + 64 - -64) =<PHONE_NUMBER>
((-64 ^ 63) + 63 - -64) =<PHONE_NUMBER>

Example Dartium:
((-64 ^ 64) + 64 - -64) = 0
((-64 ^ 63) + 63 - -64) = 126

As explained before, we had to choose between two possible ways of limiting the range.
Since xor does not move (shift) the bit you are expecting to be interpreted as a sign bit, you can obtain the generator function you want by forcing the xor value to be signed:
(x ^ y).toSigned(32)

I think we can close this. We aren't supporting Dartium anymore.


Problem with the Certificate Sign

Hello, we donwloaded the application and built it correctly, we follow every step of the process and we don't see anything wrong with it, but, unfortunedly, we are getting a NSC Notification, is there a way to now a detailed notification about this, because, we already check our Certificate with the one we use on the Enrrollment. By the way, the person taht make the enrrollment is no longer in our institution, and we lost the chance to generate an Administrator end user, is there a way to solve this too?, thanks for further answers to our issues.

You won't get anything that will tell you exactly what the issue is. The two most likely causes are going to be an issue with the keys, some mismatch between what was enrolled and what is being used possibly or that there is an issue with the XML and the signature just wont validate. I haven't seen any problems with the digital signature itself that is created from the application. You can always take the signed Payload file from the program. It should be named _Payload.xml or you can take it from the _Payload.zip file. Both should be available from the processing folder. If you put that in something like Oxygen XML, you can validate the signature and make sure that is not the issue. If it validates there, you should be able to rule that out. If it doesn't and if it is a test file, or you can use a test file, you can send a copy over to<EMAIL_ADDRESS>to have someone look at it and see if there is anything obvious or not so obvious that may be causing it. It may be a little harder to verify a problem with the keys. If you need to update your certificate, you will need to do that through the IDES enrollment site. And if you need to have the admin changed or have problems with the enrollment account, you can contact them through the https://www.ides-support.com/Support page there. So regardless of what you find with the digital signature in the file, you should probably contact them to make sure your enrollment account and administrator setup is taken care of.

Thanks for the answer, we are going to check on what you are advising us, another institution that had a similar error told us that they check the TimeStamp and make it match in every file, Paylod, Metadata and the name of the ZIP itself, is the a issue with this?, because, the TimeStamp on the files is not exactly the same, there's a difference about miliseconds or seconds. Thanks for your support.

No, that shouldn't matter. I used a hardcoded timestamp in the metadata for a long time and it all worked fine. I think if anyone had an issue and did that to try and fix it, that portion would have just been coincidence and the true fix was elsewhere. Having the correct number of digits in the timestamp is very important, but you would see a different error if it was not formatted correctly.

hi again, we already check using Oxigen XML the structure of our XML Payload, and the valide says that is well formed, so, our nest step on the list is to check the Certificate Keys on the IRS Portal, but, unfortunedly, we haven' receive anything from them about the issue we have with the Administrator end user. Hope this can be solved soon. Thanks for the advises.

Hi there!, we already send the file and receive the NIM Notification, but, in the IRS site there's no time specified to get a Notification on when we are going to get a Notificaction or Alert about if there's errors or not within the file, is there a time specified for us to wait?, by the way, we didn't get an Alert by email, only by the idesgateway, we checked the configuration of the alerts and the mail account and everything seems to be in order, but, we don't get the email alet yet. Thanks for all your support, if you could give some advises for the alerts pelase.

I believe the record level processing is schedule for the mid-August time frame right now. If you attend any of the Global IT Forum calls that we put on, they usually discuss some of the different business updates such as testing windows and how things are progressing with that record level processing. But expect it to be later in the summer, around August.
I can see some of the notification alert emails going through for you, the RC021 and RC024. But I don't see the initial IDES submission emails. I would start with a contact at the https://www.ides-support.com/Support page or call them at by phone at 1-800-613-IDES (4337) or<PHONE_NUMBER>. They should be able to double check the account setup and make sure everything looks ok.

mcorralesm, DanielCrouch : What was the problem with the signature of the file? We are getting the NSC notification too. We checked the file with the one included in the IDES TestPacket and they are equal. Even the very IDES TestPacket Payload file does not pass in the CheckSignature method of the program.

I didn't see what the end result of the failure was, but just following the thread it appeared to work itself out after the certificates were checked. I haven't seen any files signed by the application fail the signature check and that is why although the check is in there, it isn't being used. As long as there are no changes made to the Payload file after the signature is applied, it should verify without any issues. I usually use something like Oxygen XML or XML Buddy Validator when I need to check a signature. If it verifies with Oxygen, the only times I have seen it not make it through IDES are when the wrong key pair is being used, for example the private key doesn't match the enrolled key in IDES. This happens quite often during the test windows when someone updates a certificate in production and expects the updated certificate to flow through to the test system and it won't do that after the test window has started. But if you are having signature problems, I would start with Oxygen or XML Buddy and see what they show. If they show a valid signature, I would recommend checking the keys being used.

How can I check if my certificate is on the test system ?

Any checks for certificates should go through the IDES Support help desk. They have a web contact form at https://www.ides-support.com/Support. Their phone number is also listed at if you prefer to contact them by phone: https://ides.desk.com/customer/portal/articles/1791066-support.
Keep in mind that the way the open test windows work is that all production certificates are copied over to the test system shortly before the opening of the test window. So at that time, the same certificate exists in production and in the test system. But if you update production, there is no mechanism to have that flow through to the test system again. That may not be the case, but it is good to be aware of. But either way, if you need to check certificates, that IDES Support desk is where you need to go.

Ok. Thank you. I'll check the file with OxygenXML first.

If you have a test, unencrypted, signed Payload file, we can run it through a validator as well if you don't have one available. If so, you can send it to<EMAIL_ADDRESS>for someone to check. If you send it, just add something about "Github" to the subject line.

I checked with oxygenXml and find the problem, thank you. It was a problem on our program.
For the record :
After we signed and get the signed byte array, we loaded it into a new XmlDocument, instead of using File.WriteAllBytes() directly.
Thank you very much Daniel.

Good to hear, thanks for the update.


Fix links to Glitch

Use new Glitch import from git url format
Fixes: https://github.com/stimulusjs/stimulus/issues/270

Thanks!


Fix derived name mangling in schema

The current mangling schema isn't properly reversible, and we actually rely
on it being so.  Fix this by making sure that
unmangle(mangle(X)) == X always holds regardless of the depth of
mangling.
This creates an incompatibility with earlier dumps, because we carry an
internal name -> id map in them.  Fortunately, it's easy to fix by
retaining the old mangling code and using it to lookup in schema id maps
produced by older servers.
NOTE: the dump tests are failing because the current version derivation is wrong,
and it seemed wrong to work that around in code.

NOTE: the dump tests are failing because the current version derivation is wrong,

Do tests pass locally?

Do tests pass locally?

They do if I munge the local version to say a5.dev...

They do if I munge the local version to say a5.dev...

Not sure I follow, but we need to make sure that the CI and running tests locally works after merging this, obviously.


hi i have dropdown in every row of  the drag handle sortable component . when i set the value in drop down to 5 i should move the current cell to fifth how do i achieve it

Please take the time to put together a clear description of what it is you're trying to accomplish if you want me to help you.


chore(compiler): improve performance for isolatedModules: false

Summary
Improve performance for isolatedModules: false by only creating LanguageService instance with limited amount of initial files.
Because when creating LanguageService instance with fileNames from tsconfig, often this fileNames includes all the files in the project which makes internal TypeScript performs scanning and reading the unnecessary files. This is not optimal because it will create lots of I/O threads which then impact heavily on the performance.
This PR should help #1115 and all related performance issues by somewhere around 30% improvement in speed.
Test plan
Green CI
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information
N.A.

Pull Request Test Coverage Report for Build 4504

48 of 48 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.05%) to 92.893%

Totals

Change from base Build 4502:
0.05%

Covered Lines:
1074

Relevant Lines:
1109

💛  - Coveralls


Quickstart broken?

Following the docs for quick start - it seems to blow up?
npm init @svelteness/kit-docs mydocs

✔ Default KitDocs Theme? (Y/n) · true

[kit-docs]: target directory is empty, creating new SvelteKit app.

create-svelte version 2.3.1

Welcome to SvelteKit!

✔ Which Svelte app template? › Skeleton project
✔ Add type checking with TypeScript? › Yes, using JavaScript with JSDoc comments
✔ Add ESLint for code linting? … No / Yes
✔ Add Prettier for code formatting? … No / Yes
✔ Add Playwright for browser testing? … No / Yes
✔ Add Vitest for unit testing? … No / Yes

Your project is ready!
✔ Type-checked JavaScript
  https://www.typescriptlang.org/tsconfig#checkJs

Install community-maintained integrations:
  https://github.com/svelte-add/svelte-adders

Next steps:
  1: cd mydocs
  2: npm install (or pnpm install, etc)
  3: git init && git add -A && git commit -m "Initial commit" (optional)
  4: npm run dev -- --open

To close the dev server, hit Ctrl-C

Stuck? Visit us at https://svelte.dev/chat
Error: ENOENT: no such file or directory, lstat '/home/bret/Documents/Sites/mydocs/src/routes/index.svelte'
    at Object.lstatSync (node:fs:1574:3)
    at __node_internal_ (node:internal/fs/utils:804:8)
    at rmSync (node:fs:1272:13)
    at main (file:///home/bret/.npm/_npx/cbc22df0399bae1c/node_modules/@svelteness/create-kit-docs/bin/create-kit-docs.js:58:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  errno: -2,
  syscall: 'lstat',
  code: 'ENOENT',
  path: '/home/bret/Documents/Sites/mydocs/src/routes/index.svelte'
}
npm ERR! code 1
npm ERR! path /home/bret/Documents/Sites
npm ERR! command failed
npm ERR! command sh -c -- create-kit-docs mydocs

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/bret/.npm/_logs/2023-01-23T00_12_58_378Z-debug-0.log

It should be now resolved in 1.0, let me know if any issues persists. Thanks!


pass pname in proc params and attach params to proc instance

Pretty much just mooching off of @demmer's work in https://github.com/juttle/juttle/pull/334 and the to-be-reviewed https://github.com/juttle/juttle/pull/364.
Previously, pname was attached to proc instances at this.pname, but that was removed in https://github.com/juttle/juttle/pull/154.
I was using the proc's pname in juttle-flowgraph-viz when monkey-patching emit to do a "join" between the node emitting the points and its entry in the flowgraph JSON.
@dmajda cc @demmer

@go-oleg Do I understand correctly that you just need a unique identifier for each node? If so, I’d set it directly on the instances and call it id in the runtime. That is, I’d do params.$id = pname; in gen_proc and then this.id = params.id; in base.
The idea is to decouple the notion of a unique node identifier from the concept of pname, which I consider an internal compiler-level concept which shouldn’t leak out and be relied upon.

@dmajda it needs to be both a unique identifier and something I can use to lookup the node in the flowgraph stage output (where its referred to as pname).

If I'm using incorrect/unclear terminology w.r.t. "flowgraph stage output", what I mean is
compiler.compile(juttleSource, { stage: 'flowgraph'})

or
./bin/juttle -e juttleSource --mode flowgraph`

For
emit -limit 1 | put v = 1 | view table

its
{
 "built_graph": {
  "nodes": [
   {
    "type": "BuiltinProc",
    "location": {
     "filename": "main",
     "start": {
      "offset": 0,
      "line": 1,
      "column": 1
     },
     "end": {
      "offset": 14,
      "line": 1,
      "column": 15
     }
    },
    "name": "emit",
    "options": [
     {
      "id": "limit",
      "val": {
       "type": "NumericLiteral",
       "value": 1,
       "d": true
      },
      "location": {
       "filename": "main",
       "start": {
        "offset": 4,
        "line": 1,
        "column": 5
       },
       "end": {
        "offset": 13,
        "line": 1,
        "column": 14
       }
      }
     }
    ],
...

I see. OK, let’s keep the concept of pnames for the time being.
+1


Fix listing files inside directory in pack file

When adding a directory path to the inventory of the pack, an empty file name was being added to the file list. That made Directory.get_next() signal end-of-list too early so that files in a subdirectory were missed.
Fixes #15801.
Helps with #16798.

Cherry picked into 3.0.3


Add loading state while generating the story

Hi Tony,
This is Andrii who worked with you on Upwork by fixing the Dropdown selection.
I just added the loading state while generating the story.
By the way, as I remember, you suggest me that have the React/Next.js stuff role.
I send you message on Upwork chat but no response.
Does it still available?
Please leave me a message by commenting this PR.
I will appreciate you let me know about that and I hope to work with you.
Thanks.

Hey thanks @Dante-top!  I don't have any work for you at the moment
However I will pass your contact along for anyone looking for a coder with your skills, and if/when I do have anything you will be the first I contact
Thank you for this PR, my wife was literally asking me to make a loader lol


Catch violations in times(...) expectations

Description
Adds a new quit() method to universe that causes all spawned actors to quit. It also collects actors exit codes that can be used to verify that all actors quit gracefully, which in turn allows to fail the test if mock objects expectations are not met.
Relates to #2754
How was this PR tested?
Added several tests to universe.rs. I also had to modify several times(...) parameters to make tests pass. Somebody who is more familiar with these tests should probably check if these changes make sense or if I inadvertently found and buried actual bugs.

I opened #2768 to fix the license header checker.

One thing that still annoys me with the current implementation is that when a test that uses universe fails for some unrelated reason the logic in universe drop is still triggered. So, you see two panics - first is your test's panic, which is useful, but then you also get the universe's drop panic with irrelevant message "Did you call universe.assert_quit()?". I just implemented it and it confuses me sometimes, I imagine it might also confuse others. Not really sure how to detect it though.

@imotov Yes, if it is too confusing we may have to remove this Drop impl. Anyway, LGTM


Add logging config

#5

Hey Noam! Thanks for the super fast pull request this is great.
My question is more about what is idiomatic in python. I think the idea of creating dynamic loggers on a per-graph might kind of "break" the way that logging is supposed to be used. I think typically libraries provide loggers with static names. That way an informed consumer can grab that logger out of the magical logger namespace, clear out handlers, add their own formatters and handlers, etc.
I would suggest instead just putting the name of the graph in the text of the log message by convention. However I'm relatively new to this python logging stuff so I could be under the wrong impression.

I think you might be right about general usage.
My thinking is along these lines. Let's say I want to write logs to disk for one graph and not for another. Under your suggestion I'd need a specialized handler for that, which is burdensome. Under this pull request, I have the option of per-graph configuration through specifying the name.
On the other hand, if I don't care about graphs with different logging configurations, I leave the name as the default and everything gets logged to the same loggers ('taskmap-worker' and 'taskmap-manager'). So unless I go out of my way to specify a name for the graph, the behavior is defaults to what you mention.
In either case though it's perfectly possible to grab and manipulate the loggers.

Cools sounds good to me. Just wanted to flag as possible issue but it sounds like you've thought about it.
And to clear I applaud your desire to make logging more explicit and configurable on a per-instance basis. The magical global state logging style in python messes with my head a bit.
Should the logging level by configurable in def setup_loggers(config)?
It looks like it is hardcoded to DEBUG right now. This seems like the type of thing that should be in the config dict.

That sounds like a great idea. I'm not going to be able to look at this again for a few days, but feel free to add other logging configuration options to this pull request.

https://github.com/n-s-f/taskmap/pull/8

Thanks!


Milk Quality Prediction

ML-Crate Repository (Proposing new issue)
:red_circle: Project Title : Milk Quality Prediction
:red_circle: Aim : Create a prediction model which will predict the goodness of the milk.
:red_circle: Dataset : https://www.kaggle.com/datasets/cpluzshrijayan/milkquality
:red_circle: Approach : Try to use 3-4 algorithms to implement the models and compare all the algorithms to find out the best fitted algorithm for the model by checking the accuracy scores. Also do not forget to do a exploratory data analysis before creating any model.

📍 Follow the Guidelines to Contribute in the Project :

You need to create a separate folder named as the Project Title.
Inside that folder, there will be four main components.

Images - To store the required images.
Dataset - To store the dataset or, information/source about the dataset.
Model - To store the machine learning model you've created using the dataset.
requirements.txt - This file will contain the required packages/libraries to run the project in other machines.

Inside the Model folder, the README.md file must be filled up properly, with proper visualizations and conclusions.

:red_circle::yellow_circle: Points to Note :

The issues will be assigned on a first come first serve basis, 1 Issue == 1 PR.
"Issue Title" and "PR Title should be the same. Include issue number along with it.
Follow Contributing Guidelines & Code of Conduct before start Contributing.
This issue is only for 'SWOC' and 'JWOC' contributors of 'ML-Crate' project.

:white_check_mark: To be Mentioned while taking the issue :

Full name :
GitHub Profile Link :
Participant ID (If not, then put NA) :
Approach for this Project :
What is your participant role? (Mention the Open Source Program name. Eg. HRSoC, GSSoC, GSOC etc.)

Happy Contributing 🚀
All the best. Enjoy your open source journey ahead. 😎

@abhisheks008 could you please assign me this.

Issue assigned to you @Avik-creator

Please complete the issue before the deadline @Avik-creator

@abhisheks008 Could u please assign me this.

Hello, I am interested to work on this project. Please assign it to me.
Full name : Yashashree Anant Deshmukh
GitHub Profile Link : https://github.com/Yashu812
Participant ID (If not, then put NA) : NA
Approach for this Project : I will perform Data Cleaning and Exploratory Data Analysis on the given dataset. Further perform following five Regression Models and compare their accuracies -
LinearRegression
DecisionTreeRegressor
RandomForestRegressor
KNeighborsRegressor
XGBRegressor
What is your participant role? Contributor

@abhishek-thakre always mention the approach while taking the issue here, and please follow the issue template.
Issue assigned to @shubhammore45

Hey @abhisheks008 ,
I have created a pull request please check and respond.