Performance test prototype

This revision adds Unix50 and EuroSys one liner performance tests, using -m.
As of this writing, memstar.ndr.md is now configured to use the workers/run-perf.sh script, and ci.pash.ndr.md are now listening on port 2048 to manage memstar.ndr.md. Go ahead and run curl ci.pash.ndr.md:2048/stdout to see where the tests are right now. :)
If you wish to run this yourself, check the README in scripts/remote.
Notes:

@angelhof: The Dockerfile has been updated to declare a locale, fixing #106. I ended up needing to do this to get past more and more Python errors when running scripts. Since we discussed it before, can you help me verify that this will not negatively impact artifacts?
I updated all worker scripts to use empty directories for locks, since mkdir is atomic.
The correctness test configuration has been updated to use less code, since the performance test controller needed to share some functionality.
@nvasilakis: Despite what we discussed, I ended up downloading the entire results directory. It turned out to be pretty fast (< 1s) for me, and since it's using rsync I trust that the controller will download only incremental changes from then on. Let me know if this is okay with you.

@angelhof: The Dockerfile has been updated to declare a locale, fixing #106. I ended up needing to do this to get past more and more Python errors when running scripts. Since we discussed it before, can you help me verify that this will not negatively impact artifacts?

Since you were able to run the perf tests it should be fine!

@angelhof: The Dockerfile has been updated to declare a locale, fixing #106. I ended up needing to do this to get past more and more Python errors when running scripts. Since we discussed it before, can you help me verify that this will not negatively impact artifacts?

Since you were able to run the perf tests it should be fine!

@angelhof Requested changes are in. Thanks!

@angelhof Requested changes are in. Thanks!

@nvasilakis: Despite what we discussed, I ended up downloading the entire results directory. It turned out to be pretty fast (< 1s) for me, and since it's using rsync I trust that the controller will download only incremental changes from then on. Let me know if this is okay with you.

Sounds good!

@nvasilakis: Despite what we discussed, I ended up downloading the entire results directory. It turned out to be pretty fast (< 1s) for me, and since it's using rsync I trust that the controller will download only incremental changes from then on. Let me know if this is okay with you.

Sounds good!


[DO NOT MERGE] Remove Java7 samples

Fixes issue #1538
This is ongoing work with the TW team to reorganize and update our repo. The references in the docs need to be removed and updated first.

Nothing specific. Plus, there will be an additional check that nothing is broken when the docs are updated.


Define rule: If a Timing is of type "Fixed reference" then window, windowLower, and windowUpper are all null.

An anchor has no window attributes.

Created as DDF00087 in rule engine.

Moved text from Plain Language Expression to the beginning of Description and removed Criteria from the reference because most attributes are blank (and can't be removed individually).  Added value for Origin and Rule Identifier / Version.
Review complete.

Changed DDF99987 to DDF00025


Forums feature: creation of discussions and leaving comments.

Forums screen provides functionality to create new discussions with interesting topics/questions. In each discussion user can leave comments on the topic.

Leave forums screen as it is.


5106 Alterations - Edit Company Name

Issue #: /bcgov/entity#5106
Description of changes:

Stores an original business snapshot for the purposes of resetting to a base state and for value comparisons.

Parse and display additional Name Request and Business type data for Alterations.

Update Change/Correct labels and indicators to be dynamic based on either Correction or Alteration filijngs.

Added business type check to identify type mismatch. Store conflicts in a property and display Warning modal and UI icon when Name Request type is a mismatch to the current/original business type.

Added some tooltip ovverides to emulate what we've done in Name Request with the Arrows.

Some minor styling updates as per the Invision doc.
https://projects.invisionapp.com/d/main#/console/20562985/436252442/preview

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the bcrs-entities-create-ui license (Apache 2.0).

This was an intentional time to verify the UTC offset, ie, 7:59 UTC is the day before in Pacific (either PST or PDT, not sure).
This is to ensure that code doesn't just truncate the time and assume the date is correct.
As long as this is well tested, I don't really care about the times/dates, but this is an area where we often fall down, so please test thoroughly.

Absolutely, in this scenario we don't do anything in the UI with this date afaik. Just display it.
So if this offset is intentional and consistent from the backend, the ui will reflect the correct date locally.
On that note though, since it is a very specific offset catering to PST, there could be oddities in other time zones.. but really this should be left to the back end 😩

This was an intentional time to verify the UTC offset, ie, 7:59 UTC is the day before in Pacific (either PST or PDT, not sure).
This is to ensure that code doesn't just truncate the time and assume the date is correct.
As long as this is well tested, I don't really care about the times/dates, but this is an area where we often fall down, so please test thoroughly.

Absolutely, in this scenario we don't do anything in the UI with this date afaik. Just display it.
So if this offset is intentional and consistent from the backend, the ui will reflect the correct date locally.
On that note though, since it is a very specific offset catering to PST, there could be oddities in other time zones.. but really this should be left to the back end 😩


gen_csharp_src.bat does not appear to exist anywhere

gen_csharp_src.bat does not appear to exist anywhere.  I am new to protobuf so I now need to find another way to do this.  Can you point me in the right direction and update the README and https://connect.spotware.com/documentation/section/getting-started-your-csharp-app?

Just started looking into those code samples tonight and ran into the same problems.
It seems as if not only the documentation but also the acual proto files provided by Spotware are in need of updating, as the ProtocolBuffer guys seem to have made some changes.
In essence, this is what I can conclude:

The docs link to a package but the link is broken (protobuf-csharp-port-<IP_ADDRESS>1-release-binaries.zip)
Having never seen the promised bat file, I suspect it essentially just calls protoc.exe (provided in the above package) with some command line arguments.
The ProtocolBuffers used in Spotwares examples (and very likely in their active APIs) are of a now outdated version 2.x
Google is now actively working on 3.x
Spotware's documentation link to https://github.com/jskeet/protobuf-csharp-port which is now a "deprecated repo"
This is where things are now happening: https://github.com/google/protobuf
That being said, Jon Skeets repo is still there for legacy reasons (maybe guys like us?) and it does contain a protoc.exe. But I could not get it to generate csharp files for some reason (related to a lack of a csharp code gen).
Trying Google's latest repo instead also caused problems, as the Spotware proto files are of the old syntax (they lack the line syntax = "proto3";, making them proto2) and the latest protoc.exe will only generate C# code for proto3 files. (And no, simply adding that text does not suffice. They did breaking changes to the syntax.)
I found more information here on the problems with Jon Skeets version and the lack of support in Google's latest, indicating that for a very short while there was a version of Google's repo which did contain a protoc.exe which would generate C# code from proto2 files and then it was removed. You will want to get this one. I did manage to get this version to produce C# code!
Based on this, I believe this should be a working command line (where ProtoFiles is a directory with the files and Generated is a directory that will contain the generated cs files): protoc.exe -I=ProtoFiles --csharp_out=Generated ProtoFiles\CommonMessages.proto ProtoFiles\CommonModelMessages.proto ProtoFiles\OpenApiMessages.proto ProtoFiles\OpenApiModelMessages.proto
Assuming you have the Google.ProtocolBuffers NuGet package installed and referenced, the generated code does compile. Haven't had the change to test it yet, though.

So as you can see, this is actually a greater issue:

Spotware should migrate their proto files to comply with proto3 syntax, as that is where all active support is found these days. Sticking with legacy is never a good idea.
They should update their documentation once this has been done.

Until then, I hope the instructions above will help.

Oh, I just realized another thing: this csharp sample code repository already contains pre-generated csharp files, integrated into a complete code example. So you don't even need to be able to generate cs files from the .proto files if you don't want to. (Though the documentaiton certainly wants you to.)
It is a shame that the documentation focuses so much on how to generate the classes yourself, but does so incorrectly causing all this confusion, even though it should not even be necessary at all:
As long as they don't change the API, the already generated classes from the sample project should be all we ever need. Just copy those over to your project and get going.

Just saw this now.  Thanks for the info.  I got the generation working using proto2, and posted my approach to the spotware forum: http://ctdn.com/forum/connect-api-support/10864


Tests leak FDs which leads to test failures on machines with low ulimits.

As was discussed a lot in #1098 many tests open sockets and don't provide a proper clean-up which results in many leaking FDs during a test execution. This leads to problems on machines with low soft ulimits such as Mac OS.
I'll be making an effort to change these tests to not leak FDs such as unclosed connections, pools, etc.

One observation from my testing: I’ve seen some FDs that leak intermittently. This can make it harder to tell if a patch succeeded in cleaning it up, or that was just a lucky run.

I think this issue is mostly finished. Closing for now. Thanks @alexwlchan!


Championship register form assigns cars even if they are already been taken

Step 1: create a new championship with locked entry list, put a certain number of cars in the entry list (we put only 1 Fiat 500 and only 1 Ferrari 458 for this example in the entry list, and also can add other different random cars)
Step 2: apply for the championship. Select the Fiat 500, choose any skin you want. Approve the application.
Step 3: apply for the championship using different credentials.
Problem 1: in the application form, you can still choose the Fiat 500, even if it has been already taken before.
Step 4: approve the second application.
Problem 2: As the application gets approved, the Server Manager will literally kick another random car from the entry list and replace it with a new Fiat 500. Also, if the kicked car was the only one available in the list (the single Ferrari 458 we added before), new applicants won't even be able to choose that one because it has already been kicked out.

Problem 1: This only occurs if admin approval is enabled, the idea is that if somebody wants to apply for a car (even if it is full) then they still can, and the admin is in charge of making decisions/contacting people etc. taking control away from users is something that we try to avoid. I do think this needs some more detailed information though. Maybe the sign up page should tell you how many people have applied/have been accepted for a car. (so in your example it would say Fiat 500 (1/1 approved).
What do you think about that solution?
Problem 2: Good spot, this is definitely a bug, we'll get right on that.


Adf ci cd improvements

I have updated specific language as I honestly found the instructions fairly confusing in terms of translating this to a YAML based CI/CD deployment.
If PG is interested I do have working examples of ADF leveraging YAML for complete CI/CD deployments. It feels that this documentation is fractured as it jumps from the old ADF, new ADF deployments as well as Classic and YAML based builds. Additionally it never demonstrates a YAML pipeline deployment.

@nabhishek
Can you review the proposed changes? Note that the validation warning must be resolved before merge.
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Documentation not available as said on readme.md

Readme.md states: "An extensive documentation is available at https://df2img.dev/."
That link throws a 404 on github, where is the documentation published?
Thanks!

The docs are online again. Thanks for pointing this out to me!


Single Source of Truth

Removes the duplicate rules in favor of the webhook version. This eliminates any need to make the same edit twice.

Unnecessary, the rules have been re-written


feat: add feature_flags parameter to the getSite endpoint

Adds to the swagger documentation the ability to pass in a feature_flags parameter to the getSite api, which will return a list of feature flags and whether or not they are enabled.
Unsure about the values, but went off of this definition: https://github.com/netlify/bitballoon/pull/15495/files#diff-c7977f571d39c09cf8480243e53e44eb1c40acfc7fcb1d76446222e28de96478R100
Also unsure about how to mark as x-internal, as elsewhere in the swagger, only endpoints are set with  tags: [x-internal], not parameters.
Depends on: https://github.com/netlify/bitballoon/pull/15495
Related: https://github.com/netlify/pod-dev-foundations/issues/519

do we really want allow customer to look at the feature flags? iirc we intentionally obfuscate flag names in the frontend to not leak new features etc. if this really needs to be in a openapi doc, consider using the private docs

@mraerino I've updated to the newer version of redoc cli (the version we were using was deprecated) - and now this allows for us to filter out  properties & methods with x-internal:true as described here: https://redocly.com/docs/cli/guides/hide-apis/, while keeping the swagger.yml with all the correct info for the api.


ethash: 0.8.0 -> 0.9.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/ethash/versions.
meta.description for ethash is: "PoW algorithm for Ethereum 1.0 based on Dagger-Hashimoto"
meta.homepage for ethash is: "https://github.com/ethereum/ethash"
meta.changelog for ethash is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 0.9.0 with grep in /nix/store/8bhljyk697miiaav9c3faqhvj56f8aif-ethash-0.9.0

found 0.9.0 in filename of file in /nix/store/8bhljyk697miiaav9c3faqhvj56f8aif-ethash-0.9.0

Rebuild report (if merged into master) (click to expand)

5 total rebuild path(s)

4 package rebuild(s)

4 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
ethash
ethminer
ethminer-cuda
ethminer-free

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/8bhljyk697miiaav9c3faqhvj56f8aif-ethash-0.9.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A ethash https://github.com/r-ryantm/nixpkgs/archive/590bd131fb5b3fd1961624da8d290823db9c9024.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/8bhljyk697miiaav9c3faqhvj56f8aif-ethash-0.9.0
ls -la /nix/store/8bhljyk697miiaav9c3faqhvj56f8aif-ethash-0.9.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

ethash
ethminer (ethminer-cuda)
ethminer-free

Maintainer pings

Result of nixpkgs-review pr 168158 run on x86_64-linux 1

2 packages failed to build:

ethminer (ethminer-cuda)
ethminer-free

1 package built:

ethash

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/chfast/ethash/releases.
meta.description for ethash is: "PoW algorithm for Ethereum 1.0 based on Dagger-Hashimoto"
meta.homepage for ethash is: "https://github.com/ethereum/ethash"
meta.changelog for ethash is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.0.0 with grep in /nix/store/ijm6zcbpx2j3m2ww0kmx0g84v5331pgr-ethash-1.0.0
found 1.0.0 in filename of file in /nix/store/ijm6zcbpx2j3m2ww0kmx0g84v5331pgr-ethash-1.0.0

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
ethash
ethminer
ethminer-cuda
ethminer-free

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/ijm6zcbpx2j3m2ww0kmx0g84v5331pgr-ethash-1.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A ethash https://github.com/r-ryantm/nixpkgs/archive/65023922da01cce9f1aed67c36c083fbfd310415.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/ijm6zcbpx2j3m2ww0kmx0g84v5331pgr-ethash-1.0.0
ls -la /nix/store/ijm6zcbpx2j3m2ww0kmx0g84v5331pgr-ethash-1.0.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages failed to build:

ethminer (ethminer-cuda)
ethminer-free

1 package built:

ethash

Maintainer pings

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/chfast/ethash/releases.
meta.description for ethash is: PoW algorithm for Ethereum 1.0 based on Dagger-Hashimoto
meta.homepage for ethash is: https://github.com/ethereum/ethash
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.0.1 with grep in /nix/store/92y6kcwf8dv48lv30k15srcc8mimg8s1-ethash-1.0.1
found 1.0.1 in filename of file in /nix/store/92y6kcwf8dv48lv30k15srcc8mimg8s1-ethash-1.0.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
ethash

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/92y6kcwf8dv48lv30k15srcc8mimg8s1-ethash-1.0.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A ethash https://github.com/r-ryantm/nixpkgs/archive/7d32fe6dbee7c88c47bd707a769c05686643e255.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/92y6kcwf8dv48lv30k15srcc8mimg8s1-ethash-1.0.1
ls -la /nix/store/92y6kcwf8dv48lv30k15srcc8mimg8s1-ethash-1.0.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

ethash

Maintainer pings


[LiveComponent] Allow binding LiveProp to URL query parameter

Q
A

Bug fix?
no

New feature?
yes

Issues
N/A

License
MIT

Inspired from the Livewire Url attribute, let's bind our LiveProp props to the URL query string!
This is a first proposal, with the minimum feature behavior:

Add a new url constructor parameter to LiveProp
Bound props appear in the URL in a parameter with the same name
When the component is mounted for the first time (not in a live request), its props are initialized with provided query parameters
When the component is rendered on client side, the URL is updated according to the mapping (on first render and further live updates)

Currently supported data types:

Scalar props
Array props
Custom DTO objects with scalar and array properties

#[AsLiveComponent()]
final class MyComponent
{
    #[LiveProp(writable: true, url: true)]
    public ?string $search = null;

    #[LiveProp(writable: ['id', 'name'], url: true)]
    public ?SomeDto $dto = null;

    #[LiveProp(url: true)]
    public array $tags = [];
}

A bound LiveProp doesn't need to be writable, so it could be updated on server side based on a custom action.
BUT for DTOs, for the moment I bind the writable props only. Maybe another config option could be a better solution?
What could be next?

Improve data types support (nested DTOs, enums, specific properties from DTOs...)
Aliasing a prop name to a query parameter name (search => q, dto.id => some_id)
Add a keep option to force presence in the URL even if the prop is empty
Listen to external URL modification to synchronize props
Multiple history strategies (currently the URL is replaced, but we could push new entries instead)
Enable configuration from a getQueryString method in the component instead of attributes

You can also check my demo app to try the feature.

Nice work here! I'm curious how multiple components using this feature would behave. What about multiple of the same component?

Nice work here! I'm curious how multiple components using this feature would behave. What about multiple of the same component?

Hum that's actually a very good question :smile:
I think this is related to the alias feature in some way; and we should be able to configure/override some part of the mapping in the component attributes. Like, maybe
{% component('MyComponent', { query-mapping: {"prop": "alias1}}) %}
{% component('MyComponent', { query-mapping: {"prop": "alias2}}) %}

Actually I'm wondering if the DTO support really worth it. For the moment it's quite limited:

As we should not use dots in query parameters, we use _. This is kind of naming strategy that should be abstracted.
Nested DTO properties are not allowed. This is mainly because I didn't yet implement any config to select some properties of a DTO prop, so it avoids problems such as self recursion etc.

The DTO support, if shipped with this feature, seems close to the Serializer concepts (max depth, projection, sub context, naming strategy, etc.). On the other hand, handling scalar values is really easy. So handling a custom DTO in a custom way, so the developer can eventually manage the scalar properties he/she wants, should be easy too.
That's all to say, I suggest to drop the incomplete support of DTOs, and instead provide an extension point (an interface or some tag injection) so :

The default extractor handles scalars and arrays of scalars
Additional strategies could be added in custom services by the developer

Actually I'm wondering if the DTO support really worth it. For the moment it's quite limited

My immediate reaction when I saw you had DTO support was "wow" but also "that sounds like it may get super complex" :).

That's all to say, I suggest to drop the incomplete support of DTOs, and instead provide an extension point (an interface or some tag injection) so :

Agreed! Though I am also not that concerned immediately about adding the extension point. If you have something obvious in mind, sure, let's add it. But it is not a need.

Actually I'm wondering if the DTO support really worth it. For the moment it's quite limited:

As we should not use dots in query parameters, we use _. This is kind of naming strategy that should be abstracted.
Nested DTO properties are not allowed. This is mainly because I didn't yet implement any config to select some properties of a DTO prop, so it avoids problems such as self recursion etc.

The DTO support, if shipped with this feature, seems close to the Serializer concepts (max depth, projection, sub context, naming strategy, etc.). On the other hand, handling scalar values is really easy. So handling a custom DTO in a custom way, so the developer can eventually manage the scalar properties he/she wants, should be easy too.
That's all to say, I suggest to drop the incomplete support of DTOs, and instead provide an extension point (an interface or some tag injection) so :

The default extractor handles scalars and arrays of scalars
Additional strategies could be added in custom services by the developer

:disappointed:
Live component could set same urls as like submitting a form with GET method where query string is set to parent[field][..] etc. with brackets, I'd expect to work this same way, so I could refactor our live search to REAL live search :)

Live component could set same urls as like submitting a form with GET method where query string is set to parent[field][..] etc. with brackets, I'd expect to work this same way, so I could refactor our live search to REAL live search :)

As I said, I'm interested in getting a rich feature without going crazy on complexities. However, @norkunas may have a point here. We already have a dehydration system that is able to convert a DTO to an array of simple data - e.g. AddressDto dehydrates to ['street' => '404 Page Drive', 'city' => 'London', ...]. So, in theory, if AddressDto has url: true, could we set its "query string mapping" to be address[street]=404%20Page%20Drive&address[city]=London?

Live component could set same urls as like submitting a form with GET method where query string is set to parent[field][..] etc. with brackets, I'd expect to work this same way, so I could refactor our live search to REAL live search :)

As I said, I'm interested in getting a rich feature without going crazy on complexities. However, @norkunas may have a point here. We already have a dehydration system that is able to convert a DTO to an array of simple data - e.g. AddressDto dehydrates to ['street' => '404 Page Drive', 'city' => 'London', ...]. So, in theory, if AddressDto has url: true, could we set its "query string mapping" to be address[street]=404%20Page%20Drive&address[city]=London?

Seems legit, associative arrays should already be supported. That would have been a better solution that the underscores for the current DTO support. I'll take a look :)

We already have a dehydration system that is able to convert a DTO to an array of simple data - e.g. AddressDto dehydrates to ['street' => '404 Page Drive', 'city' => 'London', ...]. So, in theory, if AddressDto has url: true, could we set its "query string mapping" to be address[street]=404%20Page%20Drive&address[city]=London

I agree here, but I think we should do some refactoring on the Hydrator class instead of building new hydration logic. We don't want to maintain two hydrations logic since we want this feature to mirror the rest.

Hi! I made some updates, especially on the URL utils. I ported the code from Livewire as suggested, which brings some improvements:

No need to "expand" prop names anymore, we can set complex parameters directly in the URL. I guess it could also better handle dynamically added nested complex objects, as we don't need to know the data structure before updating query params.
Array values are explicitly indexed in the URL. That's fine but also required additional sort on server side, as InputBag::all() doesn't sort the values by itself.
Parameters order is kept between URL changes (former implementation always put objects and arrays at the end of the query string).

The only main change I made to the original code is to avoid null values in the URL. I found it annoying that setting a single property in a DTO brings all other uninitialized values in the query string. Furthermore, depending on the input type of the prop, nulls may be wrongly casted (got the case with a numeric input filled with 0 because of this).
Also, now the URL is updated once after render. But in Livewire, it seems it is updated on each prop change, because all props may have their own history strategy. Not sure about how we could handle this in our feature?

Also, now the URL is updated once after render. But in Livewire, it seems it is updated on each prop change, because all props may have their own history strategy. Not sure about how we could handle this in our feature?

Hmm, that's a good point. Though, if I have 2 props that both will use history: true, does Livewire actually add two entries into the history (even if both prop changes resulted from a single re-render)? You don't need to actually dig in and answer that, but that looks weird to me. If I change search and mode and then click a button to re-render, that should result in just one history entry.
But what if search is history: true and mode is history: false? Obviously, if I ONLY change mode, then no new history entry should be made. And if I ONLY change search, then a new history entry should be added. But what if I change both at the same time and then re-render? I think the correct answer is that one new history entry should be added. So, by this logic (and please tell me if you disagree), we should update the URL for all props at the same time. And if even one of the changed props has history: true, then we would use pushState() to add a new history entry.

I think the correct answer is that one new history entry should be added. So, by this logic (and please tell me if you disagree), we should update the URL for all props at the same time. And if even one of the changed props has history: true, then we would use pushState() to add a new history entry.

That seems to me the better way to go. Thanks for your advises! I keep this in mind for the future implementation of the history feature.

Nice work here! I'm curious how multiple components using this feature would behave. What about multiple of the same component?

Hello! I made some experimentation on my local project to handle the case of multiple components. I think we should handle an optional attribute to prefix all parameter names in the component (so it acts like a "scope"). Thought about something like:
<twig:MyComponent query-param-prefix='prefix_'>

So the URL looks like ?prefix_prop1=foo.
And maybe we could also leverage the special key attribute as a shortcut for a such attribute?

That seems reasonable. Though, if 2 components are on the same page and are using the url functionality, as long as their parameter names don't collide, won't it work already? If ComponentA is adding a search param and ComponentB is adding a page param, if the URL is currently ?search=hello&page=1 and ComponentA re-renders, will the current implementation "keep" the page param - e.g. update it to ?search=changed&page=1?
Also, I'm super busy this week - but this is an important PR. As far as you know, is this ready to go (other than needing a rebase - thanks!) and is it waiting for a detailed review?

That seems reasonable. Though, if 2 components are on the same page and are using the url functionality, as long as their parameter names don't collide, won't it work already? If ComponentA is adding a search param and ComponentB is adding a page param, if the URL is currently ?search=hello&page=1 and ComponentA re-renders, will the current implementation "keep" the page param - e.g. update it to ?search=changed&page=1?

Yes, I confirm that two different components can share the URL if their parameters don't collide. I added an implementation that uses the special query-param-prefix attribute to prefix parameters, and used key as a shortcut (but it is easily removable).

Also, I'm super busy this week - but this is an important PR. As far as you know, is this ready to go (other than needing a rebase - thanks!) and is it waiting for a detailed review?

Rebase done :wink:
I think the current code runs well and is tested enough. We already discussed the expected behaviors a lot so it should be fine. But I added two more commits which may need a quick check, if you have the time of course!

aff83f2 : Simplify the mapping structure on PHP side, and fix some edge cases in URL data management on JS side
23d4b7c: The prefix attribute implementation mentioned above

Btw, I also worked on the other features (keep, alias, history) and they work pretty nicely.  I can create a new PR when this one gets merged :)

Down to the small stuff. I played with this and it works FANTASTICALLY - super excited to have this!

I'm glad to hear that!
Latest commit handles bc break for TwigComponent and fix empty params removal :)

Sorry for the flood i did not start thinking i'd write many comments :)

@smnandre Updates are coming, I compile all fixes in a single commit ;)

Not sure yet about those things (is there something or ... not at all)
But could you explain me what happens if:

someone adds any query parameters to the _component endpoint ? should they be used ?
someone has a component with a non writable field BUT queryMapping = true  ?
someone has a component with a writable field AND queryMapping = true ?

My intuition would be "it must be writable if a query can change that value" but i'm not sure if some use cases contredict that
On the opposite, i think the "standard" live hydration starts by setting values, before hydrating the "new ones"... Would that not interfere in a way with the query data ?

Sorry - one last change :p. Can you rebase? We have a conflict, I believe, from #1341. Then we'll get this merged!

Sorry - one last change :p. Can you rebase? We have a conflict, I believe, from #1341. Then we'll get this merged!

Should be ok now ;)

Thank you 1 million for this HUGE effort and your constant responsiveness. I can't wait to use this! Cheers @squrious!

Btw, this does still needs some docs. @squrious do you want to make a PR for that? It doesn't need to be perfect - just getting something is 95% of the important part.

I cannot say it better ! I have been really attentive to some details but this PR is really amazing and i’m so glad you worked as much to deliver. Congrats and thank you !

Thanks to you @weaverryan @smnandre  for all the time you invested in the reviews! That was so cool to work on a such feature. I'll make the PR for the documentation ;)


inngest works & implemented bath

Summary by Sourcery
Add bathing functionality to the Tamagotchi game and implement poop mechanics
New Features:

Add bath animation frames showing the Tamagotchi during bathing
Implement bathing interaction that reduces the Tamagotchi's poop level
Add visual poop indicator that shows the current cleanliness state

Enhancements:

Improve state management to handle poop levels with proper validation and bounds (0-10)
Update game rules to clarify poop mechanics and bathing effects on Tamagotchi's state

Chores:

Remove unused start route file
Adjust inngest tick frequency to run every minute instead of every 30 seconds

This is a benchmark review for experiment reviews_for_yhkli.
Run ID: reviews_for_yhkli/benchmark_2024-10-29T11-27-46_v1-23-0-113-g9fada7a5a-dirty.
This pull request was cloned from https://github.com/ykhli/AI-tamago/pull/11. (Note: the URL is not a link to avoid triggering a notification on the original pull request.)

Experiment configuration
review_config:
  # User configuration for the review
  # - benchmark - use the user config from the benchmark reviews
  # - <value> - use the value directly
  user_review_config:
    enable_ai_review: true
    enable_rule_comments: false

    enable_complexity_comments: benchmark
    enable_security_comments: benchmark
    enable_tests_comments: benchmark
    enable_comment_suggestions: benchmark
    enable_functionality_review: benchmark

    enable_pull_request_summary: benchmark
    enable_review_guide: benchmark

    enable_approvals: false

  ai_review_config:
    # The model responses to use for the experiment
    # - benchmark - use the model responses from the benchmark reviews
    # - llm - call the language model to generate responses
    model_responses:
      comments_model: benchmark
      comment_validation_model: benchmark
      comment_suggestion_model: benchmark
      complexity_model: benchmark
      functionality_model: benchmark
      security_model: benchmark
      tests_model: benchmark
      pull_request_summary_model: benchmark
      review_guide_model: benchmark
      overall_comments_model: benchmark

# The pull request dataset to run the experiment on
pull_request_dataset:
- https://github.com/ykhli/AI-tamago/pull/25
- https://github.com/ykhli/AI-tamago/pull/22
- https://github.com/ykhli/AI-tamago/pull/21
- https://github.com/ykhli/AI-tamago/pull/19
- https://github.com/ykhli/AI-tamago/pull/18
- https://github.com/ykhli/AI-tamago/pull/17
- https://github.com/ykhli/AI-tamago/pull/16
- https://github.com/ykhli/AI-tamago/pull/15
- https://github.com/ykhli/AI-tamago/pull/14
- https://github.com/ykhli/AI-tamago/pull/13
- https://github.com/ykhli/AI-tamago/pull/12
- https://github.com/ykhli/AI-tamago/pull/11
- https://github.com/ykhli/AI-tamago/pull/10
- https://github.com/ykhli/AI-tamago/pull/9
- https://github.com/ykhli/AI-tamago/pull/8
- https://github.com/ykhli/AI-tamago/pull/7
- https://github.com/ykhli/AI-tamago/pull/6
- https://github.com/ykhli/AI-tamago/pull/5
- https://github.com/ykhli/AI-tamago/pull/4
- https://github.com/ykhli/AI-tamago/pull/3
- https://github.com/ykhli/AI-tamago/pull/2
- https://github.com/ykhli/AI-tamago/pull/1

# Questions to ask to label the review comments
review_comment_labels: []
# - label: correct
#   question: Is this comment correct?

# Benchmark reviews generated by running
#   python -m scripts.experiment benchmark <experiment_name>
benchmark_reviews:
- dataset_pull_request: https://github.com/sourcery-ai/core/pull/4792
  review_pull_request: https://github.com/sourcery-ai-experiments/core/pull/414
- dataset_pull_request: https://github.com/ykhli/AI-tamago/pull/1
  review_pull_request: https://github.com/sourcery-ai-experiments/AI-tamago/pull/1

Reviewer's Guide by Sourcery
This PR implements a bathing feature for the Tamagotchi and makes several improvements to the poop tracking system. The implementation includes new ASCII art animations for the bathing sequence, state management updates for handling bath interactions, and UI changes to display the poop status.
Sequence diagram for Tamagotchi bath interaction
sequenceDiagram
    actor User
    participant UI as Tamagotchi UI
    participant API as /api/interact
    participant State as StateManager

    User->>UI: Click bath button
    activate UI
    UI->>UI: setIsInteracting(true)
    UI->>UI: setTamaStatus('Bathing...')
    UI->>API: POST /api/interact (BATH)
    activate API
    API->>State: handleBath()
    activate State
    State->>State: Get latest status
    State->>State: Save bath interaction
    State->>State: Update status (decrease poop)
    State-->>API: Updated status
    deactivate State
    API-->>UI: Response
    deactivate API
    UI->>UI: Display bath animation
    Note over UI: Animation plays for 9 seconds
    UI->>UI: Reset to idle state
    UI->>UI: setIsInteracting(false)
    deactivate UI

Class diagram for updated Tamagotchi components
classDiagram
    class StateManager {
        +getLatestStatus()
        +getLastInteractions()
        +saveInteraction()
        +updateTamagotchiStatus()
    }

    class TamagotchiComponent {
        -tamagotchiState
        -isInteracting
        -tamaStatus
        -animation
        +handleBath()
        +handleResponse()
        +displayNeedForBath()
    }

    class InteractionHandler {
        +INTERACTION.BATH
        +handleBath()
    }

    TamagotchiComponent --> StateManager: uses
    TamagotchiComponent --> InteractionHandler: uses

State diagram for Tamagotchi poop management
stateDiagram-v2
    [*] --> Clean: Initial state
    Clean --> NeedsBath: Eating increases poop
    NeedsBath --> MorePoop: Continue eating
    MorePoop --> MaxPoop: Reach poop level 10
    MaxPoop --> NeedsBath: Take bath
    NeedsBath --> Clean: Take bath

    note right of MaxPoop: Poop level cannot exceed 10
    note right of Clean: Poop level = 0
    note right of NeedsBath: Poop level decreases by 1 per bath

File-Level Changes

Change
Details
Files

Added new bath animation frames for the Tamagotchi
Created ASCII art animation sequence showing the Tamagotchi during bathAnimation consists of 10 frames alternating between open and closed eyes
src/components/tamagotchiFrames.ts

Implemented bath interaction functionality
Added handleBath function to process bathing state updatesIntegrated bath interaction type into the API route handlerAdded UI button and handler for triggering bath actionUpdated poop display logic in the UI
src/app/utils/interaction.tssrc/app/api/interact/route.tssrc/components/Tamagotchi.tsx

Enhanced poop tracking system
Changed poop tracking from emoji-based to numeric system (0-10 scale)Updated state management logic to handle poop level changesAdded validation to ensure poop levels never decrease except through bathing
src/app/utils/state.ts

Modified Inngest cron schedule
Changed cron timing from every 30 seconds to every minute
src/inngest/functions.ts

Tips and commands
Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time. You can also use
this command to specify where the summary should be inserted.

Customizing Your Experience
Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.


Authenticating to Hotstar for downloading Disney+HS VIP and Premium videos

Checklist

[x] I'm reporting a feature request
[x] I've verified that I'm running youtube-dl version 2021.04.17
[x] I've searched the bugtracker for similar feature requests including closed ones

Description
I am reporting for a new feature to be introduced or added so that for downloading Disney+ Hotstar VIP and Premium videos by automating login. Since I have a Premium account, I should be able to download the videos after finishing up the login process. Please note that Hotstar now allows login to be OTP based and phone number.

@dstftw It can be done through netrc option or specifiying phone number using --username option


Automatically persist generated password for SQL container

By default today, we auto-generate a password for the SQL Server container when AddSqlServerContainer is called in an AppHost project. A random password on each app start isn't a great experience though as anything created in one session cannot be used in the next if a persistent volume is configured.
We should consider how we might instead automatically manage preservation of the generated password and reuse in subsequent runs, e.g. in user secrets. One challenge of user secrets is it requires initializing the project with an ID (via an MSBuild property in the project file) and modifying the project file while the app is running is probably not a great idea.

@DamianEdwards I've got this PR where I am looking to add secret support to Aspire:
https://github.com/dotnet/aspire/pull/1487
This PR is the first phase which adds the underlying resource types, but in the comments I've added a note on how this could evolve to provide some increased management around managing container password secrets (as an example). The difference here is that we don't store a generated password, instead we get a password from configuration (locally) which could be in user secrets.
I think that this is a reasonable pay for play compromise? Even if we did want to go further and do generation and storage we'd probably layer it on top of this generic secrets mechanism rather than building it bespoke for each resource?

Punting to preview 4 because we won't have time to react to some of the discussions from our app model offsite and it will dictate what we do here.

We discussed this at one of the AppHost syncs. We still want to do this an it's an important building block for container persistence.

Related to https://github.com/dotnet/aspire/issues/2438

Putting this in 8.1. It's important for the volume experience.

Sweet!


chore: CI should test more Go versions when actions/setup-go fixes performance issue

GitHub's ubuntu-latest image recently started to include 4 versions of Go (1.11.x - 1.14.x).  However, actions/setup-go downloads and installs Go if any version != 1.14 is specified.
The preinstalled Go versions can be leveraged by GitHub's setup/go so it won't cause a 10 sec delay during CI for each specified version of Go.
If and when the following setup/go issue is fixed, use the matrix feature of GitHub Workflow Actions to test the latest 3 versions of Go:
https://github.com/actions/setup-go/issues/39

Looks like actions/setup-go#39 got fixed!  🎉


Wrapper-Offline.exe?

This file is nowhere to be found within the folder, is another executive required to open it?

Not an issue - https://github.com/Wrapper-Offline/Wrapper-Offline/releases/tag/v2.0.0


random nrn_timeout error for large simulations (128 cores on GCP)

I'm getting quite frequent and random nrn_timeout in my simulations. This is for an M1 model with 10k multi-compartment multi-channel cells, 7k VecStims, 30M synapses, 5-sec simulations, running on 128 cores (4 nodes x 32 cores; 120GB RAM per node) on Google Cloud CentOS 7 virtual machines, with NEURON7.7.1-37-gd9605cb master (d9605cb), NetPyNE v0.9.4 and Python 3.6.8.
Here's an example error:
nrn_timeout t=4380.62
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 0.

You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

A couple more details which might be relevant:

fixed time step (dt = 0.025)
cvode.cache_efficient(True)
cvode.atol(1e-6)
cvode.use_fast_imem(True)
recording all membrane currents via i_membrane_ to calculate LFP at 0.1ms interval (similar implementation to Allen Brain BMTK or LFPy)
recording somatic voltage from 2 cells at 0.1 ms interval

Below is the output log of a successful simulation, which includes info on the distribution of cells and conns, as well as load balance at the end:

numprocs=128

Start time:  2019-12-22 22:55:08.962242

Reading command line arguments using syntax: python file.py [simConfig=filepath] [netParams=filepath]
Loading file ../data/v56_batch2b/v56_batch2b_0_0_cfg.json ...
Loading simConfig...
Importing netParams from ../data/v56_batch2b/v56_batch2b_netParams.py

Creating network of 22 cell populations on 128 hosts...
  Number of cells on node 9: 134
  Number of cells on node 2: 134

[...]

  Number of connections on node 5: 68670
  Number of synaptic contacts on node 5: 278168

[...]

Running simulation for 5000.0 ms...

  Done; run time = 24927.95 s; real-time ratio: 0.00.

Gathering data...
  Done; gather time = 5.70 s.

Analyzing...

Finished saving!
  Done; saving time = 4.43 s.
Plotting LFP ...
Plotting raster...
Plotting recorded cell traces ... trace
  Done; plotting time = 150.12 s

Total time = 25606.27 s

End time:  2019-12-23 06:01:55.235592
max_comp_time: 24692.92587293384
min_comp_time: 20009.316901733127
avg_comp_time: 21740.200874486483
load_balance: 0.8804222304946103

spike exchange time (run_time-comp_time):<PHONE_NUMBER>06541

Follow debugging approach suggested by @nrnhines : "use diff to find the first spike difference which implies a substantive state difference in that gid and then do a binary search using simulations with different tstop to find the time of the first variable difference in the state [after finitialize add the line pc.prcellstate(27, "t0")]. At that point it is generally obvious why the difference exists."

What one would like for diagnosis is the call stack traces at the timeout for each rank. That would allow one to see why not all ranks have called the same collective and diagnose further with that info in mind.
On some machines the simulation may be run with ddt to directly view the stack traces but I have no familiarity with that. If stack traces are not available then I was thinking that one could place a function at the beginning of each collective wrapper in src/nrnmpi/nrnspike.c . The function would take an argument which would be a number associated with the collective. The function would broadcast that arg from rank 0 to all other ranks and each rank would compare the rank 0 number with the number associated with the collective wrapper it was called from and print a message if different. This isn't a full stack trace but might give enough info to further diagnose the problem. It is unusual for a timeout to happen so late in a simulation (t=4380.62) I wonder if that indicates running out of some resource such as a memory leak or some excessively long Vector...but there was no error message except the timeout. Anyway, I acknowledge the difficulty of diagnosis when it takes 25000s to get to a nondeterministic error.  Does this simulation use callbacks to the interpreter? Does the timeout ever occur when you are NOT calculating LFP?  Does the timeout ever occur when you are NOT recording the two voltage traces.

Presumptively we will find a bug. But I do wonder about a lack of MPI fault tolerance at that time scale on that machine.

Thanks @nrnhines! See inline responses below:
"On some machines the simulation may be run with ddt to directly view the stack traces but I have no familiarity with that."

Not sure what ddt is, but these are CentOS machines, which typically lack a lot of libraries/tools. I'll try to find out if there's any way to view the stack traces in these machines.

"if stack traces are not available then I was thinking that one could place a function at the beginning of each collective wrapper in src/nrnmpi/nrnspike.c . The function would take an argument which would be a number associated with the collective. The function would broadcast that arg from rank 0 to all other ranks and each rank would compare the rank 0 number with the number associated with the collective wrapper it was called from and print a message if different. This isn't a full stack trace but might give enough info to further diagnose the problem."

I have no idea how to go about adding this code; but if it's the only solution I can try if you give me some indications/example.

"It is unusual for a timeout to happen so late in a simulation (t=4380.62) I wonder if that indicates running out of some resource such as a memory leak or some excessively long Vector...but there was no error message except the timeout."

The timeouts happen all across simulation times, from very early (~200 ms) up to over >4sec.

"Anyway, I acknowledge the difficulty of diagnosis when it takes 25000s to get to a nondeterministic error. Does this simulation use callbacks to the interpreter?"

I'm not sure exactly what this means ... there's an h.FInitializeHandler to a python func called at 0.1ms steps to compute the LFP signal, would that be an example of callback to the interpreter?

"Does the timeout ever occur when you are NOT calculating LFP?"

Yes, I tried that a couple days ago.

"Does the timeout ever occur when you are NOT recording the two voltage traces."

I haven't tried this, but will do.

"Presumptively we will find a bug. But I do wonder about a lack of MPI fault tolerance at that time scale on that machine."

My original thought was it might some issue with MPI on the machines; I'm trying to run on a single node with 96 cores to see if that reeduces the timeouts (currently in 4 nodes x 32 cores). Any other suggestions to test this?

git fetch
git pull collective-timeout-debug
Build and run as normal and see if it prints anything from the printf
printf("%d calls %d but 0 calls %d\n", nrnmpi_myid, i, buf);

I installed the new version and rerun 11 of the simulations that had previously failed; 3 of them failed again (at 2.4, 2.7 and 2.7 secs) but weirdly only 1 of them showed the nrn_timeout message; the other 2 didn't (they just stopped). Note these jobs are ran using the SLURM job scheduler, so their output is stored in 2 files (.run and .err).

You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

[ext_salvadordura_gmail_com@g1-login1 v56_batch2]$ tail v56_batch2_0_1.err
/usr/bin/id: cannot find name for group ID<PHONE_NUMBER>
NEURON -- VERSION 7.8.0-59-gd40331e collective-timeout-debug (d40331e) 2019-12-27
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

Additional mechanisms from files
 mod/ar_traub.mod mod/cadad.mod mod/cadyn.mod mod/cagk.mod mod/cal_mh.mod mod/cal_mig.mod mod/cancr.mod mod/canin.mod mod/can_mig.mod mod/catcb.mod mod/cat_mig.mod mod/cat_traub.mod mod/ch_CavL.mod mod/ch_CavN.mod mod/ch_KCaS.mod mod/ch_Kdrfastngf.mod mod/ch_KvAngf.mod mod/ch_KvCaB.mod mod/ch_leak.mod mod/ch_Navngf.mod mod/gabab.mod mod/h_BS.mod mod/HCN1.mod mod/h_harnett.mod mod/hin.mod mod/h_kole.mod mod/h_migliore.mod mod/ican_sidi.mod mod/iccr.mod mod/IC.mod mod/iconc_Ca.mod mod/ikscr.mod mod/IKsin.mod mod/kap_BS.mod mod/kapcb.mod mod/kapin.mod mod/kBK.mod mod/kca.mod mod/kctin.mod mod/kdmc_BS.mod mod/kdr_BS.mod mod/kdrcr.mod mod/kdrin.mod mod/km.mod mod/kv.mod mod/MyExp2SynBB.mod mod/my_exp2syn.mod mod/MyExp2SynNMDABB.mod mod/nafcr.mod mod/nafx.mod mod/nap_sidi.mod mod/nax_BS.mod mod/naz.mod mod/Nca.mod mod/savedist.mod mod/vecstim.mod

[ext_salvadordura_gmail_com@g1-login1 v56_batch2]$ tail v56_batch2_4_4.err
/usr/bin/id: cannot find name for group ID<PHONE_NUMBER>
NEURON -- VERSION 7.8.0-59-gd40331e collective-timeout-debug (d40331e) 2019-12-27
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

Additional mechanisms from files
 mod/ar_traub.mod mod/cadad.mod mod/cadyn.mod mod/cagk.mod mod/cal_mh.mod mod/cal_mig.mod mod/cancr.mod mod/canin.mod mod/can_mig.mod mod/catcb.mod mod/cat_mig.mod mod/cat_traub.mod mod/ch_CavL.mod mod/ch_CavN.mod mod/ch_KCaS.mod mod/ch_Kdrfastngf.mod mod/ch_KvAngf.mod mod/ch_KvCaB.mod mod/ch_leak.mod mod/ch_Navngf.mod mod/gabab.mod mod/h_BS.mod mod/HCN1.mod mod/h_harnett.mod mod/hin.mod mod/h_kole.mod mod/h_migliore.mod mod/ican_sidi.mod mod/iccr.mod mod/IC.mod mod/iconc_Ca.mod mod/ikscr.mod mod/IKsin.mod mod/kap_BS.mod mod/kapcb.mod mod/kapin.mod mod/kBK.mod mod/kca.mod mod/kctin.mod mod/kdmc_BS.mod mod/kdr_BS.mod mod/kdrcr.mod mod/kdrin.mod mod/km.mod mod/kv.mod mod/MyExp2SynBB.mod mod/my_exp2syn.mod mod/MyExp2SynNMDABB.mod mod/nafcr.mod mod/nafx.mod mod/nap_sidi.mod mod/nax_BS.mod mod/naz.mod mod/Nca.mod mod/savedist.mod mod/vecstim.mod

[ext_salvadordura_gmail_com@g1-login1 v56_batch2]$ tail v56_batch2_4_1.err
/usr/bin/id: cannot find name for group ID<PHONE_NUMBER>
NEURON -- VERSION 7.8.0-59-gd40331e collective-timeout-debug (d40331e) 2019-12-27
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

Additional mechanisms from files
 mod/ar_traub.mod mod/cadad.mod mod/cadyn.mod mod/cagk.mod mod/cal_mh.mod mod/cal_mig.mod mod/cancr.mod mod/canin.mod mod/can_mig.mod mod/catcb.mod mod/cat_mig.mod mod/cat_traub.mod mod/ch_CavL.mod mod/ch_CavN.mod mod/ch_KCaS.mod mod/ch_Kdrfastngf.mod mod/ch_KvAngf.mod mod/ch_KvCaB.mod mod/ch_leak.mod mod/ch_Navngf.mod mod/gabab.mod mod/h_BS.mod mod/HCN1.mod mod/h_harnett.mod mod/hin.mod mod/h_kole.mod mod/h_migliore.mod mod/ican_sidi.mod mod/iccr.mod mod/IC.mod mod/iconc_Ca.mod mod/ikscr.mod mod/IKsin.mod mod/kap_BS.mod mod/kapcb.mod mod/kapin.mod mod/kBK.mod mod/kca.mod mod/kctin.mod mod/kdmc_BS.mod mod/kdr_BS.mod mod/kdrcr.mod mod/kdrin.mod mod/km.mod mod/kv.mod mod/MyExp2SynBB.mod mod/my_exp2syn.mod mod/MyExp2SynNMDABB.mod mod/nafcr.mod mod/nafx.mod mod/nap_sidi.mod mod/nax_BS.mod mod/naz.mod mod/Nca.mod mod/savedist.mod mod/vecstim.mod

I'm rerunning the same 11 sims but without recording LFP and voltages.

4/11 jobs with no recording of LFP/voltage crashed with nrn_timeout (I cancelled the rest to save resources). They all showed the nrn_timeout error message but no additional info:
nrn_timeout t=1109.17
nrn_timeout t=1847.95
nrn_timeout t=1313.39
nrn_timeout t=2574.72

I wonder if in addition to turning off the recording of LFP/voltage you can also turn off the printRunTime flag.

If the test above also fails. Please try the following change in order to force output in order to check if
ranks 0 and 1 are entering the check_ function but are not leaving. I would not restrict to rank <2 but
otherwise, I fear the output would be too voluminous
hines@hines-T7500:~/neuron/collective-timeout-debug/src/nrnmpi$ git diff
diff --git a/src/nrnmpi/mpispike.c b/src/nrnmpi/mpispike.c
index 07b1603a..195141e9 100644
--- a/src/nrnmpi/mpispike.c
+++ b/src/nrnmpi/mpispike.c
@@ -99,14 +99,21 @@ static void make_spikebuf_type() {

 static void check_(int i) {
        int buf = i;
+  if (nrnmpi_myid < 2) {
+    extern double t;
+    printf("%d enter check %d  t=%g\n", nrnmpi_myid, i, t);
+  }
        guard(MPI_Bcast(&buf, 1, MPI_INT, 0, nrnmpi_comm));
        if (buf != i) {
                printf("%d calls %d but 0 calls %d\n", nrnmpi_myid, i, buf);
        }
        guard(MPI_Barrier(nrnmpi_comm));
        if (buf != i) {
-               nrnmpi_abort(0);
+               nrnmpi_abort(i);
        }
+  if (nrnmpi_myid < 2) {
+    printf("%d leave check %d\n", nrnmpi_myid, i);
+  }
 }

 int nrnmpi_spike_exchange() {
hines@hines-T7500:~/neuron/collective-timeout-debug/src/nrnmpi$

Ok thanks, will try removing all callbacks, and adding the code above.

It occurred to me that in the call to nrnmpi_abort(i); above, one could encode i, buf, and nrnmpi_myid in to the error message and that could be quite helpful.  How about changeing the i to
nrnmpi_myid10000 + i100 + buf

ok, so I will change it to: nrnmpi_abort(nrnmpi_myid*10000 + i*100 + buf);
thx

That should be good for up to 200k ranks :)

Tried removing all callbacks (LFP, traces and printTime) and 2/11 jobs crashed after ~2.5s; one showed the nrn_timeout error msgs, but the other one didn't. I'm cancelling the rest of jobs now.
Will try the new check_ code tomorrow.

@nrnhines I'm running some sims with the new check_ code -- just wanted to check that the error printf will only show up once there is nrn_timeout, right?

Ok so one of the 12 sims now crashed at 1.2sec and there was no output at all, ie. not even the nrn_timeout error message:
[ext_salvadordura_gmail_com@g1-login1 v56_batch2]$ tail v56_batch2_2_2.run
0.3 s
0.4 s
0.5 s
0.6 s
0.7 s
0.8 s
0.9 s
1 s
1.1 s
1.2 s

[ext_salvadordura_gmail_com@g1-login1 v56_batch2]$ tail v56_batch2_2_2.err
/usr/bin/id: cannot find name for group ID<PHONE_NUMBER>
NEURON -- VERSION 7.8.0-62-g1d75444 collective-timeout-debug (1d75444) 2019-12-30
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2019
See http://neuron.yale.edu/neuron/credits

Additional mechanisms from files
 mod/ar_traub.mod mod/cadad.mod mod/cadyn.mod mod/cagk.mod mod/cal_mh.mod mod/cal_mig.mod mod/cancr.mod mod/canin.mod mod/can_mig.mod mod/catcb.mod mod/cat_mig.mod mod/cat_traub.mod mod/ch_CavL.mod mod/ch_CavN.mod mod/ch_KCaS.mod mod/ch_Kdrfastngf.mod mod/ch_KvAngf.mod mod/ch_KvCaB.mod mod/ch_leak.mod mod/ch_Navngf.mod mod/gabab.mod mod/h_BS.mod mod/HCN1.mod mod/h_harnett.mod mod/hin.mod mod/h_kole.mod mod/h_migliore.mod mod/ican_sidi.mod mod/iccr.mod mod/IC.mod mod/iconc_Ca.mod mod/ikscr.mod mod/IKsin.mod mod/kap_BS.mod mod/kapcb.mod mod/kapin.mod mod/kBK.mod mod/kca.mod mod/kctin.mod mod/kdmc_BS.mod mod/kdr_BS.mod mod/kdrcr.mod mod/kdrin.mod mod/km.mod mod/kv.mod mod/MyExp2SynBB.mod mod/my_exp2syn.mod mod/MyExp2SynNMDABB.mod mod/nafcr.mod mod/nafx.mod mod/nap_sidi.mod mod/nax_BS.mod mod/naz.mod mod/Nca.mod mod/savedist.mod mod/vecstim.mod

This makes no sense. Please send me your diff of the collective-timeout-debug branch. I would have expected a + at the end of the Version hash because of the added printf.

The date (2019-12-30) and commit (1d75444) shown in the version match the last commit in the branch: https://github.com/neuronsimulator/nrn/commit/1d75444e86312bd9faf7f132f22c41e3902261e8
Here's the diff:
[ext_salvadordura_gmail_com@g1-login1 nrn_debug2]$ git diff HEAD^
diff --git a/src/nrnmpi/mpispike.c b/src/nrnmpi/mpispike.c
index 07b1603..d2abfe9 100644
--- a/src/nrnmpi/mpispike.c
+++ b/src/nrnmpi/mpispike.c
@@ -105,7 +105,7 @@ static void check_(int i) {
        }
        guard(MPI_Barrier(nrnmpi_comm));
        if (buf != i) {
-               nrnmpi_abort(0);
+               nrnmpi_abort(nrnmpi_myid*10000 + i*100 + buf);
        }
 }

[ext_salvadordura_gmail_com@g1-login1 nrn_debug2]$ git log -2
commit 1d75444e86312bd9faf7f132f22c41e3902261e8
Author: Michael Hines<EMAIL_ADDRESS>Date:   Mon Dec 30 09:27:00 2019 -0500

    encode rank, etc. into mpi_abort integer.

commit c4c564ba279284841d817a11480032eea7a74cbe
Merge: d40331e d119e45
Author: Michael Hines<EMAIL_ADDRESS>Date:   Sat Dec 28 17:51:14 2019 -0500

I pushed a change to the collective-timeout-debug branch that has the printfs mentioned earlier as well
as the nrnmpi_abort informational arg.

I get the following and "undefined reference to t error" during make (doesn't happen if I try with the previous commit):
make[3]: Entering directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src/nrnmpi'
/bin/sh ../../libtool  --tag=CC   --mode=compile mpicc -DHAVE_CONFIG_H -I. -I../.. -I../../src/nrnpython -I../../src/nrnoc -I../../src/oc -I../../src/parallel -I../../src/nrnjava -I../../src/nrncvode -I../../src/sundials    -I../../src/oc  -I. -g -O2 -MT mpispike.lo -MD -MP -MF .deps/mpispike.Tpo -c -o mpispike.lo mpispike.c
libtool: compile:  mpicc -DHAVE_CONFIG_H -I. -I../.. -I../../src/nrnpython -I../../src/nrnoc -I../../src/oc -I../../src/parallel -I../../src/nrnjava -I../../src/nrncvode -I../../src/sundials -I../../src/oc -I. -g -O2 -MT mpispike.lo -MD -MP -MF .deps/mpispike.Tpo -c mpispike.c  -fPIC -DPIC -o .libs/mpispike.o
mv -f .deps/mpispike.Tpo .deps/mpispike.Plo
/bin/sh ../../libtool  --tag=CC   --mode=link mpicc -I../../src/oc  -I. -g -O2   -o libnrnmpi.la -rpath /usr/local/nrn/x86_64/lib nrnmpi.lo bbsmpipack.lo mpispike.lo nrnrt.lo  -lm -ldl
libtool: link: rm -fr  .libs/libnrnmpi.la .libs/libnrnmpi.lai .libs/libnrnmpi.so .libs/libnrnmpi.so.0 .libs/libnrnmpi.so.0.0.0
libtool: link: mpicc -shared  -fPIC -DPIC  .libs/nrnmpi.o .libs/bbsmpipack.o .libs/mpispike.o .libs/nrnrt.o   -lm -ldl  -O2   -Wl,-soname -Wl,libnrnmpi.so.0 -o .libs/libnrnmpi.so.0.0.0
libtool: link: (cd ".libs" && rm -f "libnrnmpi.so.0" && ln -s "libnrnmpi.so.0.0.0" "libnrnmpi.so.0")
libtool: link: (cd ".libs" && rm -f "libnrnmpi.so" && ln -s "libnrnmpi.so.0.0.0" "libnrnmpi.so")
libtool: link: ( cd ".libs" && rm -f "libnrnmpi.la" && ln -s "../libnrnmpi.la" "libnrnmpi.la" )
make[3]: Leaving directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src/nrnmpi'
Making all in oc
make[3]: Entering directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src/oc'
make  all-am
make[4]: Entering directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src/oc'
/bin/sh ../../libtool  --tag=CC   --mode=link mpicc  -g -O2   -o oc ocmain.o ocnoiv.o nocable.o modlreg.o ockludge.o libocxt.la liboc.la ../nrnmpi/libnrnmpi.la ../memacs/libmemacs.la ../readline/libreadline.la -lncurses   -lm -ldl
libtool: link: mpicc -g -O2 -o .libs/oc ocmain.o ocnoiv.o nocable.o modlreg.o ockludge.o  ./.libs/libocxt.so ./.libs/liboc.so -lX11 ../nrnmpi/.libs/libnrnmpi.so ../memacs/.libs/libmemacs.so ../readline/.libs/libreadline.so -lncurses -lm -ldl -Wl,-rpath -Wl,/usr/local/nrn/x86_64/lib
**../nrnmpi/.libs/libnrnmpi.so: undefined reference to `t'**
collect2: error: ld returned 1 exit status
make[4]: *** [oc] Error 1
make[4]: Leaving directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src/oc'
make[3]: *** [all] Error 2
make[3]: Leaving directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src/oc'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/ext_salvadordura_gmail_com/nrn_debug3/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/ext_salvadordura_gmail_com/nrn_debug3'
make: *** [all] Error 2

Btw, I found this post suggesting using the backtrace() functions to print the stack trace on abort (not really sure if useful):  https://stackoverflow.com/questions/77005/how-to-automatically-generate-a-stacktrace-when-my-program-crashes

I didn't try building with autotools. I used cmake which puts everything into libnrniv.so .  Hence the
extern t was not a problem. If you are restricted to running configure I can work around the issue.
This definitely would not work at present with dynamically loaded mpi.
backtrace sounds like a definite possibility. One puzzle is the variability of abort via timeout vs just quietly quitting. I'll experiment with it a bit and  maybe be able to put it into the branch.  There ought also be a way of autogenerating the  backtrace files or hopefully tiny coredumps  as a consequence of calling abort (called I presume by MPI_Abort)

I read the documentation for MPI_Abort again and for the first time noticed the statement
This routine should not be used from within a signal handler.
Which is exactly what is happening in static void timed_out(int sig) { in src/nrnoc/nrntimeout.c
I've never experienced a problem with this before, though. And never the symptom of printf not getting to stdout and the system exiting without an error message. If there is a problem with MPI_Abort here, I don't see how returning would help as everyone is likely stuck in an MPI collective.
And calling abort(...) would make me worry that not all the mpi processes would exit. Any suggestions @pramodk ?

General comments: whenever I see such timeout issues on cloud environment I first think about where these X number of nodes are allocated. For example, will they be in a single rack or situated across data centres? I am mentioning this because on many cloud environments this is not guaranteed and can cause significant communication issues (saw with some BBP tests on cloud).
We should just check about how cloud nodes are allocated.

On some machines the simulation may be run with ddt to directly view the stack traces but I have no familiarity with that.

DDT requires a license and we won’t have that for google cloud. If needed, we can install other open source tools for diagnosis.

It is unusual for a timeout to happen so late in a simulation (t=4380.62) I wonder if that indicates running out of some resource such as a memory leak or some excessively long Vector...but there was no error message except the timeout.

Anyway, I acknowledge the difficulty of diagnosis when it takes 25000s to get to a nondeterministic error.

If we reduce the model size considerably and keep #cores + #nodes same, doesn’t this error appear?

I'm trying to run on a single node with 96 cores to see if that reeduces the timeouts (currently in 4 nodes x 32 cores). Any other suggestions to test this?

Answer to this would be very helpful! Did you see this issue in single node?

I read the documentation for MPI_Abort again and for the first time noticed the statement. This routine should not be used from within a signal handler.

I also think this won’t be an issue.

Any suggestions @pramodk ?

I wonder if we should run small MPI benchmark at the beginning to see if MPI commutation is stable on allocated nodes.

Thanks @pramodk, those are very good suggestions.
I did try running on 96-core nodes (both regular and with high memory), but in both cases got the following error, which I interpreted as memory-related due to too many processes on the same node:
OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 2514728 max
>>> Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/numpy/core/__init__.py", line 17, in <module>
    from . import multiarray
  File "/usr/local/lib64/python3.6/site-packag>>> Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/numpy/core/__init__.py", line 17, in <module>
    from . import multiarray
  File "/usr/local/lib64/python3.6/site-packages/numpy/core/multiarray.py", line 14, in <module>
    from . import overrides
  File "/usr/local/lib64/python3.6/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: PyCapsule_Import could not import module "datetime"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "init.py", line 15, in <module>
    import matplotlib; matplotlib.use('Agg')  # to avoid graphics error in servers
  File "/usr/local/lib64/python3.6/site-packages/matplotlib/__init__.py", line 138, in <module>
    from . import cbook, rcsetup
  File "/usr/local/lib64/python3.6/site-packages/matplotlib/cbook/__init__.py", line 31, in <module>
    import numpy as np
  File "/usr/local/lib64/python3.6/site-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/usr/local/lib64/python3.6/site-packages/numpy/core/__init__.py", line 47, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
  1. Check that you expected to use Python3.6 from "/usr/bin/python3",
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy version "1.17.2" you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.

Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.

Original error was: PyCapsule_Import could not import module "datetime"

I will try 1) running on a single 64-core node, and 2)  running smaller models on 4 nodes * 32 cores to see if get the same issues.

Fixed the 96-core issue by running:
export OMP_NUM_THREADS=1
export USE_SIMPLE_THREADED_LEVEL3=1

So I now have 10 sims each running on a single 96-core node ... let's see if get timeout here or not...

No timeouts!! So seems like the cause was slow MPI communication between the GCP 32-core nodes. I'm running some more sims to confirm. The only issue with this is that the largest GCP nodes have 96 cores, so that would be the limit for individual simulations. I will bring this up with the Google guys to see if there's a way to guarantee faster communication between nodes.
Thanks so much @nrnhines and @pramodk for your help!

I will bring this up with the Google guys to see if there's a way to guarantee faster communication between nodes.

Considering timeouts happen sporadically at different times during simulation, it seems the network communication performance (specifically latency) is not stable. If they have some suggestions to alleviate this, it will be great (In AWS we often refer to Placement Groups strategy for the same).
Otherwise we can look into disabling timeouts in NEURON. By the way, have you tried increasing timeout using ParallelContext.timeout? Not sure if applicable here but in some of our sims (specific use cases) we have increased this by ~25x i.e. 500sec.

I had already tried to increase parallelcontext.timeout and even disable it (set to 0), but then some jobs just took forever (e.g. job hadn't finished after 28 hrs, even though similar ones took ~10hrs).  So seems like these unstable latencies can be quite persistent.
I ran 20 other jobs in the 96-core node and 2 of them actually timed out (no error msg) after ~2sec (~3 hours). The rest completed ok. So timeouts still occurr but at lower rate than when using multiple cores (~6% vs >25% chance). This small percentage in the single node might be solved by increasing parallelcontext.timeout -- I will try 500sec as you suggested.
I will also ask Google about he Placement Groups strategy. Thanks again!

Given that kind of latency, I don't think we can rule out a bug in NEURON. The puzzle for me has become how it can fail without a timeout error (on the less than 96 core/node jobs). Is it just that printing the message does not make it to the stdout? Does not seem like that is the case as I gather that there is no message from MPI_Abort.
If there was some consistent indication that there really was a timeout, it would make sense to
get some more info with the backtrace. Or perhaps, instead of MPI_Abort, print the backtrace
for every rank, set an error flag,
longjump back to the beginning, check the error flag and abort() (note everyone would abort as we
would change things so that everyone would get a timeout instead of just rank 0).  Given that the 96 core node job did give a timeout whenever it fails, I will put in a backtrace print to the collective-timeout-debug branch. Could be I'm barking up the wrong tree:)

I ran 7 more sims (including the 2 that failed) with timeout set to 500sec, and they completed ok.
@nrnhines I agree it is still worth investigating if there are any NEURON issues. I will try running with the latest changes in timeout-debug branch (I will build with cmake). Thanks.

We found this on the GCP web which sort of supports the latency hypothesis:
"Inter-node latency. For tightly-coupled HPC apps, performance might be sensitive to the inter-node latency between nodes in the cluster. Because Google Cloud provides nodes with sizes up to 64 cores, you can run 64-way parallel jobs without traversing nodes. In most cases, jobs of around 1000 cores or smaller perform reasonably well on non-specialized network hardware."
https://cloud.google.com/solutions/using-clusters-for-large-scale-technical-computing


Move test support classes to wildfly-camel-itests-common

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/wildfly-camel-pullreq/242/
Test PASSed.


edit forms: show more options in dropdown list

with this change we should be able to see 8-10 options (depending on if any wrap on to multiple lines)
before:

after:

@jesicarson @scottofletcher

THIS IS GREAT!! Thanks @ascott !!


install latest docker-compose (not v2.7.0) - use get latest version from Github

already implemented


Cannot use delimiter in migrations

Bug description
When I use delimiter command in migration file, the migration itself fails because of the invalid syntax. However the syntax is valid and when called from other tools, it works. I suspect prisma tries to split commands in the migration file itself and doesn't take delimiter into account.
Although the example uses just simple INSERT, the bug prohibits me to put functions and triggers into the migrations.
How to reproduce

Create a new empty migration file using prisma migrate dev --create-only
Use the following migration

-- This is an empty migration.
DROP TABLE IF EXISTS `TemporaryTable`;
CREATE TABLE `TemporaryTable` (
  `PK` INT AUTO_INCREMENT PRIMARY KEY,
  `id` VARCHAR(36) NOT NULL UNIQUE
);
INSERT INTO `TemporaryTable`(`id`) VALUES (UUID()),(UUID()),(UUID()),(UUID()),(UUID());

DELIMITER @@
INSERT INTO `TemporaryTable`(`id`) VALUES (UUID()) @@
DELIMITER ;

Run the migration using prisma migrate deploy. You will see

Applying migration `20220421130310_`
Error: P3018

A migration failed to apply. New migrations cannot be applied before the error is recovered from. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve

Migration name:<PHONE_NUMBER>0310_

Database error code: 1064

Database error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DELIMITER @@
INSERT INTO `TemporaryTable`(`id`) VALUES (UUID()) @@
DELIMITER' at line 1

Please check the query number 4 from the migration file.

Expected behavior
The migration should complete sucessfully.
Prisma information
Prisma schema is not important for this bug.
Environment & setup

OS: Windows 11
Database: MySQL version 8.0.23
Node.js version: v14.18.1

Prisma Version
prisma                  : 3.12.0
@prisma/client          : 3.12.0
Current platform        : windows
Query Engine (Node-API) : libquery-engine 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules\.pnpm\@prisma+engines<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules\.pnpm\@prisma+engines<EMAIL_ADDRESS>Introspection Engine    : introspection-core 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules\.pnpm\@prisma+engines<EMAIL_ADDRESS>Format Binary           : prisma-fmt 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules\.pnpm\@prisma+engines<EMAIL_ADDRESS>Default Engines Hash    : 22b822189f46ef0dc5c5b503368d1bee01213980
Studio                  : 0.459.0

Yep, it fails.

We have the exact same use case of adding triggers via a migration sql file. This issue is certainly blocking.
Anyone aware of a workaround while this isn't working?

Same issue. I found workaround - each function / trigger / etc. = one migration file.
See: https://github.com/prisma/prisma/discussions/12671#discussioncomment-2564689
Not solution for this issue, but it works ;-)

A workaround looks like be to not bother about the delimiters!
If I take the case of @tandibar linked in the previous comment, which doesn't work with Prisma:
DELIMITER ;;
CREATE FUNCTION `gen_chars`(length INT) RETURNS varchar(255) CHARSET utf8
BEGIN
    DECLARE identifier VARCHAR(255);
    RETURN identifier;
END ;;

CREATE FUNCTION `gen_cuid`() RETURNS char(25) CHARSET utf8
BEGIN
    DECLARE randoms CHAR(8);
    return randoms;
END ;;
DELIMITER ;

This following migration is working, (whereas it don't work elsewhere, if run through phpmyadmin for instance)
CREATE FUNCTION `gen_chars2`(length INT) RETURNS varchar(255) CHARSET utf8
BEGIN
    DECLARE identifier VARCHAR(255);
    RETURN identifier;
END;

CREATE FUNCTION `gen_cuid2`() RETURNS char(25) CHARSET utf8
BEGIN
    DECLARE randoms CHAR(8);
    return randoms;
END;

Credits to Umami for that, see previous mention.

@PatrikValkovic The following code works on my machine with a delimiter set to $$. (Using Prisma v4.6.1)
Can you check if this delimiter works for you too?
CREATE OR REPLACE FUNCTION current_user_id() RETURNS uuid
    LANGUAGE sql STABLE
    AS $$
    SELECT current_setting('userId')::uuid;
$$;

We have the same use case and still not working on our end when adding procedures via migration file using DELIMITER
Environment & setup:
Database: MariaDB 10.5
Node.js version: v18.16.0

Prisma Version:
prisma: 4.14.1
prisma/client: 4.14.1

Still having problems with this? There is no workaround?


[FR] Add more icon sets

Hi @RavenHogWarts, if you want to add more icon sets, not just the Lucide icon set, perhaps the IconicFonts repository would be helpful for your plugin. It has over 55,000 icons from 43 different sets, available as a single font or separate fonts per set.

Hi @RavenHogWarts, if you want to add more icon sets, not just the Lucide icon set, perhaps the IconicFonts repository would be helpful for your plugin. It has over 55,000 icons from 43 different sets, available as a single font or separate fonts per set.

Hi there! Thanks for the suggestion. The IconicFonts repository sounds like a great resource for expanding the icon options available in the plugin. I'll definitely consider integrating it to offer a broader variety of icons. Cheers!


Rework parity_status.md

          Not necessary for this PR, but we probably want to rework this page.

Originally posted by @chrisglein in https://github.com/microsoft/react-native-windows-samples/pull/988#discussion_r1833623142

Actually, looks like we deprecated that page and replaced with something that points to issues on the repo: https://microsoft.github.io/react-native-windows/docs/parity-status
So, we can consider this already done, yay!


Enable sending input to terminal instance by wt.exe

Description of the new feature/enhancement

Some times we need to control windows terminal through command line. For example: switch dir to a given path. Now the only way to do that is to open a new pane or tab through wt.exe. However, open a new pane or tab is very slow when using powershell, it will be faster if we can just send input to a existing terminal, like:
wt.exe -w 0 -S "cd ~/project/MyProj; git pull"

Proposed technical implementation details (optional)

Okay this might seem silly, but wt -w 0 send-input {whatever} seems a lot harder than just typing {whatever} at the commandline in the current window, right?
This sounds a lot like #9368, but I might need more context on your actual use case here

Okay this might seem silly, but wt -w 0 send-input {whatever} seems a lot harder than just typing {whatever} at the commandline in the current window, right?
This sounds a lot like #9368, but I might need more context on your actual use case here

There are many case that we can't type in current window, for example: implement a plugin in vscode to let current terminal cd to current file location.

Okay yea that's on me. Must not have had enough coffee - I read "commandline" and automatically assumed you were already sitting at the Terminal window and wanted to run that commandline in the Terminal window 😝
/dup #9368


[Bug]: Import EC2 Instance with gp2 leads to iops attribute not supported for root_block_device with volume_type gp2

Is there an existing issue for this?

[X] I have searched the existing issues

Affected Resource(s)

ec2.aws.upbound.io/v1beta1 - Instance

Resource MRs required to reproduce the bug
No response
Steps to Reproduce
import ec2 instance with gp2 ebs root block device
apiVersion: ec2.aws.upbound.io/v1beta1
kind: Instance
metadata:
  name: importtest1
  annotations:
    crossplane.io/external-name: i-0418d2932007f7c10
spec:
  forProvider:
    region: us-west-1
  providerConfigRef:
    name: xxx

when import ec2 instances with gp3 ebs root block device everything is working as expected:
kubectl get instance.ec2
NAME          READY   SYNCED   EXTERNAL-NAME         AGE
importtest1   True    True     i-0418d2932007f7c10   84s

What happened?
the import will lead to:
Status:
  At Provider:
[...]
    Root Block Device:
      Delete On Termination:  true
      Device Name:            /dev/sda1
      Encrypted:              false
      Iops:                   100
      Kms Key Id:
      Throughput:             0
      Volume Id:              vol-0ee29cc94ac635ca6
      Volume Size:            8
      Volume Type:            gp2
[...]
  Conditions:
    Last Transition Time:  2024-05-03T07:10:21Z
    Reason:                Creating
    Status:                False
    Type:                  Ready
    Last Transition Time:  2024-05-03T08:07:33Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2024-05-03T07:10:21Z
    Message:               async create failed: failed to create the resource: [{0 collecting instance settings: error creating resource: iops attribute not supported for root_block_device with volume_type gp2  []}]
    Reason:                AsyncCreateFailure
    Status:                False
    Type:                  LastAsyncOperation

Relevant Error Output Snippet
No response
Crossplane Version
1.15.2
Provider Version
1.3.0
Kubernetes Version
No response
Kubernetes Distribution
No response
Additional Info
wonder if the problem is the late init from IOPS field when gp2 is used
TF Provider Code: https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/service/ec2/ec2_instance.go#L2577-L2588

@haarchri, could you please provide us the Instance.ec2 MR example you used initially to reproduce the bug?

this issue will not happen with v1.5.0 again
example:
apiVersion: ec2.aws.upbound.io/v1beta1
kind: Instance
metadata:
  annotations:
    uptest.upbound.io/timeout: "3600"
    meta.upbound.io/example-id: ec2/v1beta1/instance
  labels:
    testing.upbound.io/example-name: sample-instance
  name: sample-instance
spec:
  forProvider:
    region: us-west-1
    ami: ami-07b068f843ec78e72
    instanceType: t2.micro
    networkInterface:
    - deviceIndex: 0
      networkInterfaceIdSelector:
        matchLabels:
          testing.upbound.io/example-name: sample-instance
    blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 100
        volumeType: gp2
    creditSpecification:
    - cpuCredits: unlimited

apiVersion: ec2.aws.upbound.io/v1beta1
kind: Instance
metadata:
  annotations:
    crossplane.io/external-name: i-04f75003c6e2cf24a
  name: import-instance
spec:
  deletionPolicy: Delete
  forProvider:
    region: us-west-1
  managementPolicies:
  - 'Observe'
  providerConfigRef:
    name: default

NAME                                          SYNCED   READY   EXTERNAL-NAME         AGE
instance.ec2.aws.upbound.io/import-instance   True     True    i-04f75003c6e2cf24a   8m54s

NAME                                                  SYNCED   READY   EXTERNAL-NAME           AGE
networkinterface.ec2.aws.upbound.io/sample-instance   True     True    eni-0e5805d46f680354e   26m

NAME                                        SYNCED   READY   EXTERNAL-NAME              AGE
subnet.ec2.aws.upbound.io/sample-instance   True     True    subnet-0a038c9110557b2e6   26m

NAME                                     SYNCED   READY   EXTERNAL-NAME           AGE
vpc.ec2.aws.upbound.io/sample-instance   True     True    vpc-0f610ce590c4df6a5   26m

kubectl describe instance.ec2.aws.upbound.io/import-instance
Name:         import-instance
Namespace:
Labels:       <none>
Annotations:  crossplane.io/external-name: i-04f75003c6e2cf24a
API Version:  ec2.aws.upbound.io/v1beta1
Kind:         Instance
Metadata:
  Creation Timestamp:  2024-06-04T13:39:30Z
  Finalizers:
    finalizer.managedresource.crossplane.io
  Generation:        2
  Resource Version:  3802
  UID:               3b67d2fb-a74f-4d6b-8cb3-06b1930189dc
Spec:
  Deletion Policy:  Delete
  For Provider:
    Region:  us-west-1
  Init Provider:
  Management Policies:
    Observe
  Provider Config Ref:
    Name:  default
Status:
  At Provider:
    Ami:                          ami-07b068f843ec78e72
    Arn:                          arn:aws:ec2:us-west-1:609897127049:instance/i-04f75003c6e2cf24a
    Associate Public Ip Address:  false
    Availability Zone:            us-west-1b
    Capacity Reservation Specification:
      Capacity Reservation Preference:  open
    Cpu Core Count:                     1
    Cpu Options:
      Amd Sev Snp:
      Core Count:          1
      Threads Per Core:    1
    Cpu Threads Per Core:  1
    Credit Specification:
      Cpu Credits:            unlimited
    Disable API Stop:         false
    Disable API Termination:  false
    Ebs Optimized:            false
    Enclave Options:
      Enabled:                             false
    Get Password Data:                     false
    Hibernation:                           false
    Host Id:
    Iam Instance Profile:
    Id:                                    i-04f75003c6e2cf24a
    Instance Initiated Shutdown Behavior:  stop
    Instance Lifecycle:
    Instance State:                        running
    Instance Type:                         t2.micro
    ipv6AddressCount:                      0
    Key Name:
    Maintenance Options:
      Auto Recovery:  default
    Metadata Options:
      Http Endpoint:                enabled
      httpProtocolIpv6:             disabled
      Http Put Response Hop Limit:  1
      Http Tokens:                  optional
      Instance Metadata Tags:       disabled
    Monitoring:                     false
    Outpost Arn:
    Password Data:
    Placement Group:
    Placement Partition Number:     0
    Primary Network Interface Id:   eni-0e5805d46f680354e
    Private Dns:                    ip-172-16-10-100.us-west-1.compute.internal
    Private Dns Name Options:
      Enable Resource Name Dns A Record:     false
      Enable Resource Name Dns Aaaa Record:  false
      Hostname Type:                         ip-name
    Private Ip:                              <IP_ADDRESS>
    Public Dns:
    Public Ip:
    Root Block Device:
      Delete On Termination:   true
      Device Name:             /dev/sda1
      Encrypted:               false
      Iops:                    100
      Kms Key Id:
      Throughput:              0
      Volume Id:               vol-0fc7d734ea20ee925
      Volume Size:             8
      Volume Type:             gp2
    Source Dest Check:         true
    Spot Instance Request Id:
    Subnet Id:                 subnet-0a038c9110557b2e6
    Tags:
      Crossplane - Kind:            instance.ec2.aws.upbound.io
      Crossplane - Name:            sample-instance
      Crossplane - Providerconfig:  default
    Tags All:
      Crossplane - Kind:            instance.ec2.aws.upbound.io
      Crossplane - Name:            sample-instance
      Crossplane - Providerconfig:  default
    Tenancy:                        default
    Vpc Security Group Ids:
      sg-08467904e3109316f
  Conditions:
    Last Transition Time:  2024-06-04T13:39:34Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2024-06-04T13:39:37Z
    Reason:                Available
    Status:                True
    Type:                  Ready
Events:                    <none>


Tooltip does not appear on SubMenu

React/React-dom version: 17.0.2 / 17.0.2
React-Menu version: 3.4.1
Describe the bug
First, thank you for such a great library! The issue is that SubMenu don't display their tooltip (title); in our app, we have a requirement to show a tooltip on menu items as well as submenu items.
To Reproduce

Make sure SubMenu has a title attribute (tooltip)
Click on SubMenu
Tooltip does not display over SubMenu

Expected behavior
Tooltip should display on SubMenu.
Code snippets/CodeSandbox examples/Screenshots

I guess you add title prop to the SubMenu component itself. This will apply title to the ul menu list element. To set title on submenu items, you can use the itemProps.
<SubMenu label="Edit" itemProps={{ title: "edit" }}>
  <MenuItem>Cut</MenuItem>
  <MenuItem>Copy</MenuItem>
  <MenuItem>Paste</MenuItem>
</SubMenu>

That was it! Thanks for the prompt response!!