Consumer group OffsetOutOfRange Error in some containers

Questions?

Check the Troubleshooting / FAQ ?
Search the issues ?

Bug Report
We are using Consumer group to connect to our kafka cluster. We are facing an issue where when the service container is replaced, the new containers starts throwing OffsetOutOfRange continuously.
From logs, It showed that the cases where this error is coming has offset value as doubled. Given below is one of the value received in production-
"highWaterOffset": 7867
"offset": 15732
When debugging this issue I manually called the consumerGroup.scheduleReconnect(0) function and was able to reproduce the same issue where the offset value is coming as a double of its actual value.
This is a serious bug we are facing in production. Any help is appreciated.
Error stack trace:
{"stack":"Error: OffsetOutOfRange\n    at events.push (/usr/src/app/node_modules/kafka-node/lib/protocol/protocol.js:267:25)\n    at /usr/src/app/node_modules/async/dist/async.js:3880:24\n    at replenish (/usr/src/app/node_modules/async/dist/async.js:1011:17)\n    at /usr/src/app/node_modules/async/dist/async.js:1016:9\n    at eachOfLimit (/usr/src/app/node_modules/async/dist/async.js:1041:24)\n    at /usr/src/app/node_modules/async/dist/async.js:1046:16\n    at _parallel (/usr/src/app/node_modules/async/dist/async.js:3879:5)\n    at Object.series (/usr/src/app/node_modules/async/dist/async.js:4735:5)\n    at _decodeFetchResponse (/usr/src/app/node_modules/kafka-node/lib/protocol/protocol.js:244:9)\n    at /usr/src/app/node_modules/kafka-node/lib/protocol/protocol.js:199:12\n    at KafkaClient.Client.invokeResponseCallback (/usr/src/app/node_modules/kafka-node/lib/baseClient.js:634:18)\n    at KafkaClient.Client.handleReceivedData (/usr/src/app/node_modules/kafka-node/lib/baseClient.js:614:10)\n    at KafkaClient.handleReceivedData (/usr/src/app/node_modules/kafka-node/lib/kafkaClient.js:1340:48)\n    at Socket. (/usr/src/app/node_modules/kafka-node/lib/kafkaClient.js:785:10)\n    at emitOne (events.js:96:13)\n    at Socket.emit (events.js:188:7)\n    at readableAddChunk (_stream_readable.js:176:18)\n    at Socket.Readable.push (_stream_readable.js:134:10)\n    at TCP.onread [as _originalOnread] (net.js:548:20)\n    at TCP.onread (/usr/src/app/node_modules/async-listener/glue.js:188:31)","error":{"topic":"review_inserted","maxNum":1,"partition":10,"time":-1,"metadata":"m"},"name":"Error","message":"OffsetOutOfRange"}
Environment

Node version: v6.10.1 and v8.10.0
Kafka-node version: 4.0.2 (also present in 3.0.1)
Kafka version: 2.0.0

For specific cases also provide

Number of Brokers: 3
Number partitions for topic: 40

Include Sample Code to reproduce behavior
Include output with Debug turned on
Thanks for your contribution!

Calling scheduleReconnect(0) will throw an exception and crash your process. Are you sure this is how you reproduced this issue?

I put the scheduleReconnect(0) in the consumer.on('error', function()..). Then I restarted all the brokers and I got the mentioned error. My process didn't crashed at that time, once the brokers were up again, It continued to  consume some events with incorrect offset and then it stopped consuming.
I was not able to reproduce this issue, so I was trying random things. This may not be the case with our production though, because we don't call scheduleReconnect(0) anywhere from our code.
Can this case occur if consumerGroup threw an error during initialization?

What is your retryMinTimeout set to?

retryMinTimeout is set to 5000 ms.
Other Configs for consumer , producer and cluster:
consumerOptions : {
kafkaHost: "localhost:9092",
zk : undefined,
batch: undefined,
ssl: false,
sessionTimeout: 15000,
groupId: '',
protocol: ['roundrobin'],
autoCommit: true,
autoCommitIntervalMs: 5000,
fromOffset: 'latest', // default
commitOffsetsOnFirstJoin: true,
outOfRangeOffset: 'latest',
migrateHLC: false,
migrateRolling: true,
retries: 10000,
retryFactor: 1,
retryMinTimeout: 5000
},
producerConfig : {
requireAcks: 1,
ackTimeoutMs: 300,
partitionerType: 3
},
"clusterConfig" : {
"autoConnect": true,
"requestTimeout": 10000,
"idleConnection": 3600000
}

@hyperlink Can you help me with the next action steps?
Thank you

Didn't look like your configs could cause scheduleReconnect(0) to be called.
I would recommend turning on debug logger to capture more information on what the ConsumerGroup is doing. It can be a bit verbose but those messages may shed some light on what is happening.

Hi @hyperlink ,
Turning debug logger in production is something which is dangerous. I am also facing same issue. I am using container environment on AWS, I started facing the error like :
{"name":"Error","message":"OffsetOutOfRange","stack":"Error: OffsetOutOfRange\n    at events.push (/usr/src/app/node_modules/kafka-node/lib/protocol/protocol.js:267:25)\n    at /usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:3880:24\n    at replenish (/usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:1011:17)\n    at /usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:1016:9\n    at eachOfLimit (/usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:1041:24)\n    at /usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:1046:16\n    at _parallel (/usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:3879:5)\n    at Object.series (/usr/src/app/node_modules/kafka-node/node_modules/async/dist/async.js:4735:5)\n    at _decodeFetchResponse (/usr/src/app/node_modules/kafka-node/lib/protocol/protocol.js:244:9)\n    at /usr/src/app/node_modules/kafka-node/lib/protocol/protocol.js:199:12\n    at KafkaClient.Client.invokeResponseCallback (/usr/src/app/node_modules/kafka-node/lib/baseClient.js:589:18)\n    at KafkaClient.Client.handleReceivedData (/usr/src/app/node_modules/kafka-node/lib/baseClient.js:569:10)\n    at KafkaClient.handleReceivedData (/usr/src/app/node_modules/kafka-node/lib/kafkaClient.js:1367:48)\n    at Socket.<anonymous> (/usr/src/app/node_modules/kafka-node/lib/kafkaClient.js:797:10)\n    at emitOne (events.js:96:13)\n    at Socket.emit (events.js:188:7)\n    at readableAddChunk (_stream_readable.js:176:18)\n    at Socket.Readable.push (_stream_readable.js:134:10)\n    at TCP.onread [as _originalOnread] (net.js:548:20)\n    at TCP.onread (/usr/src/app/node_modules/async-listener/glue.js:188:31)","error":{"topic":"xyz","partition":29,"time":-1,"maxNum":1,"metadata":"m"}}},"level":"error","message":"","timestamp":"2019-04-08T10:02:26.496Z"}
Just after the deployment. During deployment, docker containers are deregistered from receiving traffic and then gets replaced with new containers. Is it possible that library couldn't right the offset properly for the old containers and they got deleted during that time which might have caused the new containers to not able to fetch right offset ? If yes, how can we fix it quickly ?
We have seen few errors in broker like : [GroupCoordinator 1001]: Member kafka-node-client-2a4a6b4c-140f-42c3-ba21-dcac17e90611 in group xxxx has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)
Please help, it's impacting the production system for us.

@hyperlink I think the latest release of version 4.1.0 has some issue which was released around 19 hours ago. Once we rolled back to the older version of 4.0.2, things got fixed. We were initially using latest minor versions of the library, but now we have made it fixed.

Could you provide more details about what issues you are seeing in 4.1.0?

Hi @hyperlink, Looks like your change- 'Fix ConsumerGroup receiving wrong offsets for compressed messages' in 4.1.1 will resolve this issue, right? Or Upgrading to this version may cause some other issues (my ask is based on the comments on this issue- https://github.com/SOHU-Co/kafka-node/issues/1225)?

@shashanksah I believe all issues related to compressed messages and wrong offsets are now resolved in the latest.

@hyperlink Thanks for the response. Can you also confirm that the issue pointed out by mohit3081989 above (that the version 4.1.0 also had some issue which caused the production stack to rollback to previous version) is also resolved?

Sorry, there are lots of different issues being discussed in this thread and I am not clear on what caused @mohit3081989 issue or if it's fixed with the latest. Perhaps he can respond and advise.


Relicensing code for reference use

As the code of clicheio includes good practices, including well-thought out directory structure or templates for libraries (setuptools/cli/alembic, etc.) that could be used as a reference when building other python web projects, I would like to suggest that we repurpose (and relicense) the current code for referential usage without having to license referenced code with AGPL. I would want to ask permissions (and any ideas if wanted) from our contributors. (This wouldn't need any edits to the codebase or manual intervention, it's purely about your consent.)

🆗
How about re-license into MIT?

I don't oppose relicensing. Go for it.

I'm not quite sure if you would want to relicense the project itself though; it can stay as AGPL, but sort of dual-licensing only for referential usage? (Well, technically we would have to define what it means if we want to be exact, but I'm not sure if you want that fine grained details.)

Well, since this project is not maintained after 2015, I would switch the current license to non-AGPL. MIT also fine. This repository is more useful as a code structure than how it behaves.

I feel like it would be somewhat tedious if we would have to put license notice in all the files that we referenced these files from; I have some licenses in mind if you're okay with any of them.

Public Domain
Unlicense
WTFPL
Don't Be A Dick License
Beerware License, without having to retain notice in every file or keep a list of file referenced

I don't have any specific ideas on changing the license. Do as you wish~

+1 for Public Domain

whatever

+1 for public domain or MIT.


latests relases are not packaged to npm registry

the latest release on npm is 2.0.4, since versions 2.1.0 and 2.1.1 are not available.
Consequenly, if we install the backoffice from NPM, we get an old version that is not compatible with Kuzzle V1.

Done.


Errors after updating to Angular 7

When asking general "how to" questions:

Please do not open an issue here!

Instead, ask for help on StackOverflow with Materialize and Angular tags.

Read the CONTRIBUTING document Any issues that are violating the contributing guidelines, will be closed!

Search for duplicate or closed issues and make sure that is not an issue related to Materialize or other library.

Prepare a reduced test case for any bugs.

Provide a StackBlitz that reproduce your issue.

Expected Behavior
I was using ngx-materialize in my project and everything was fine, but after update to angular 7 I'm getting a lot of errors:

Current Behavior
I am getting errors for the sidenav and toast for the moment and nothing is showed.

Possible Solution

Steps to Reproduce (for bugs)

Update to the current version of angular

Context

Your Environment

Version used: 6.1.3
Angular version used: 7
Browser Name and version: Chrome
Operating System and version (desktop or mobile): Windows 10
Link to your project (if appropriate):

Hola, quiero usar esto con un proyecto en angular 7 pero veo que no es compatible y tampoco se espera una solución pronto :S

Hi @dav3ku ,
Hasta ahora lo he usado con Angular 7 sin ningún problema. Sólo tienes que usar las versiones correctas de las librerías y configurarlas adecuadamente en el package.json y angular.json
Until now I've used it with Angular 7 without any problem. You just have to use the correct versions of the libraries and configure them properly in the package.json and angular.json!


Allow loading gzipped data files

Impact

[ ]  Bug fix (non-breaking change which fixes expected existing functionality)
[x]  Enhancement/New feature (adds functionality without impacting existing logic)
[ ]  Breaking change (fix or feature that would cause existing functionality to change)

Description
Allows loading gzipped data files.
The change is similar to https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/resource/PathResource.java#L34
For example, when using:
databaseChangeLog:
  - changeSet:
      id: load data
      author: user1
      changes:
        - loadData:
            file: ./data/CITY.csv.gz

the extension ".gz" switches reading the content as gzipped data.
Things to be aware of

Should an application be using  data files with names ending in ".gz" which are NOT gzipped, the file name has to be changed to a proper extension.

Things to worry about

Nothing

Pre-Review Notes
Change makes sense. Probably good to add a loadData with a .gz file to the common.tests.changelog.xml file to test out the e2e flow through it
Questions I have:

I thought there was maybe gz support before, did that get broken? Or was I just mis-remembering?

Potential risks:

None

What could make the full review difficult:

Nothing

@nvoxland:
You're right that similar functionality exists in liquibase in at least one place, where an entire directory can be marked as containing gzipped resources. There are several other places and ways liquibase loads resources, which imho require a re-factoring to consolidate them into a single place.
In order to remain low impact, this change would improve the situation a lot for the time being.

@mike-seger I agree. Nice to keep this more isolated and consider a larger cleanup later.

Hello @mike-seger ! Thanks for the code, I also added the functional tests to the branch.
But when I added the tests, I noticed that the "resource" being used is of type PathResource that already handles gzip files as from the description link https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/resource/PathResource.java#L34 .
The fix made it into liquibase 4.17.0 , did you try it with this version? As i see your PR is from around the days it was released.

Hello @filipelautert , thanks for adding an integration test.
I checked whether liquibase 4.17.0 would actually resolve this case. Here's what happens:
Caused by: liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset 20220803-210000.yaml::load data::adminux:

     Reason: liquibase.exception.UnexpectedLiquibaseException: CSV file ./data/CITY.csv.gz Line 2 has 1 values defined, Header has 3. Numbers MUST be equal (check for unquoted string with embedded commas)
	at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:126) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Liquibase.lambda$null$0(Liquibase.java:291) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.lambda$child$0(Scope.java:180) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:189) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:179) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:158) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:243) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Liquibase.lambda$update$1(Liquibase.java:290) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.lambda$child$0(Scope.java:180) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:189) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:179) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:158) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Liquibase.runInScope(Liquibase.java:2447) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Liquibase.update(Liquibase.java:236) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Liquibase.update(Liquibase.java:221) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:328) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:283) ~[liquibase-core-4.17.0.jar:na]
	at org.springframework.boot.autoconfigure.liquibase.DataSourceClosingSpringLiquibase.afterPropertiesSet(DataSourceClosingSpringLiquibase.java:46) ~[spring-boot-autoconfigure-2.7.4.jar:2.7.4]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800) ~[spring-beans-5.3.23.jar:5.3.23]
	... 18 common frames omitted
Caused by: liquibase.exception.MigrationFailedException: Migration failed for changeset 20220803-210000.yaml::load data::adminux:
     Reason: liquibase.exception.UnexpectedLiquibaseException: CSV file ./data/CITY.csv.gz Line 2 has 1 values defined, Header has 3. Numbers MUST be equal (check for unquoted string with embedded commas)
	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:714) ~[liquibase-core-4.17.0.jar:na]
Caused by: liquibase.exception.MigrationFailedException: Migration failed for changeset 20220803-210000.yaml::load data::adminux:

	at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:56) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.changelog.ChangeLogIterator$2.lambda$null$0(ChangeLogIterator.java:113) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.lambda$child$0(Scope.java:180) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:189) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:179) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:158) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.changelog.ChangeLogIterator$2.lambda$run$1(ChangeLogIterator.java:112) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.lambda$child$0(Scope.java:180) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:189) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:179) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:158) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:243) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.changelog.ChangeLogIterator$2.run(ChangeLogIterator.java:93) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.lambda$child$0(Scope.java:180) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:189) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:179) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:158) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:243) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.Scope.child(Scope.java:247) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:65) ~[liquibase-core-4.17.0.jar:na]
	... 37 common frames omitted
Caused by: liquibase.exception.UnexpectedLiquibaseException: CSV file ./data/CITY.csv.gz Line 2 has 1 values defined, Header has 3. Numbers MUST be equal (check for unquoted string with embedded commas)
Caused by: liquibase.exception.UnexpectedLiquibaseException: CSV file ./data/CITY.csv.gz Line 2 has 1 values defined, Header has 3. Numbers MUST be equal (check for unquoted string with embedded commas)

	at liquibase.change.core.LoadDataChange.generateStatements(LoadDataChange.java:311) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.database.AbstractJdbcDatabase.executeStatements(AbstractJdbcDatabase.java:1250) ~[liquibase-core-4.17.0.jar:na]
	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:679) ~[liquibase-core-4.17.0.jar:na]
	... 57 common frames omitted

This happens because:
liquibase.change.core.LoadDataChange.getCSVReader().
calls:
ResourceAccessor resourceAccessor = Scope.getCurrentScope().getResourceAccessor();
which does not have any gzip decoding.
Maybe it would make sense to add another bug/cleanup request to consolidate all inputstream handling in one place. It could even be smart and not even rely on the file name, but detect the type of stream (such as gzip) by inspecting the first few bytes using PushbackInputStream.
As this is possibly a major refactoring endeavour, I'm not expecting this to happen soon.
Bottom line, it would be great if this PR could make it into liquibase for the time being.

Hey @mike-seger - I just added another check to make sure it works in other cases, so it may still work for you now. If you think it's fine I'll move it ahead for further code review and integration. Thanks!

@filipelautert The change is still looking good with me


borks when trying to include a package in a temporal shell

Welcome to Nix-on-Droid!
If nothing works, open an issue at https://github.com/nix-community/nix-on-droid/issues or try the rescue shell.
-bash-5.2$ nix-shell -p cmatrix
warning: Nix search path entry '/data/data/com.termux.nix/files/home/.nix-defexpr/channels/nixpkgs/' does not exist, ignoring
error:
       … <borked>

         at «none»:0: (source not available)

       … while calling the 'import' builtin

         at «string»:1:18:

            1| {...}@args: with import <nixpkgs> args; (pkgs.runCommandCC or pkgs.runCommand) "shell" { buildInputs = [ (cmatrix) ]; } ""
             |                  ^

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at «none»:0: (source not available)
-bash-5.2$

is there something i need to preconfig or something?

On August 30, 2024 9:25:20 PM GMT+02:00, Jim @.***> wrote:

Welcome to Nix-on-Droid!
If nothing works, open an issue at https://github.com/nix-community/nix-on-droid/issues or try the rescue shell.
-bash-5.2$ nix-shell -p cmatrix
warning: Nix search path entry '/data/data/com.termux.nix/files/home/.nix-defexpr/channels/nixpkgs/' does not exist, ignoring
error:
      … <borked>

        at «none»:0: (source not available)

      … while calling the 'import' builtin

        at «string»:1:18:

           1|<EMAIL_ADDRESS>with import <nixpkgs> args; (pkgs.runCommandCC or pkgs.runCommand) "shell" { buildInputs = [ (cmatrix) ]; } ""
            |                  ^

      (stack trace truncated; use '--show-trace' to show the full trace)

      error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)

      at «none»:0: (source not available)
-bash-5.2$

is there something i need to preconfig or something?

Have you installed with flakes? If yes, you won't have a nixpkgs channel. Use nix shell nixpkgs#cmatrix instead.

yes, i've just tried flakes recently, sorry fot not pointing that one out
so, how long does it usually take to evaluating derivation 'flake:nixpkgs#cmatrix'?

Depends on the device performance, it's proot+storage-bottlenecked. I'd say, if it's >10 minutes worth power saving disabled and wakelock acquired from the notification, that's too much.

didn't see when it succeeded, so not sure how long, but it worked like a charm, thanks


RELEASE: v0.4.1

Codecov Report
Base: 94.25% // Head: 94.25% // No change to project coverage :thumbsup:

Coverage data is based on head (81d2f9d) compared to base (9e09fbb).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #61   +/-   ##
=======================================
  Coverage   94.25%   94.25%
=======================================
  Files           7        7
  Lines         627      627
=======================================
  Hits          591      591
  Misses         36       36

Flag
Coverage Δ

pytests
94.25% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


ios support

add initial ios support

@mataide Our Realm Flutter preview version already includes limited iOS support. The limitation is one needs to clone realm-dart-samples and use the preconfigured sample project and can not create a new Flutter iOS project from scratch using the flutter tools. If you want to experiment with Realm Flutter for iOS I suggest you clone our preview samples.
We are working on a new FFI version which will include iOS and Android Flutter support.
This PR is a stale version of what needed to be done for the preview version we will be deleting this PR soon.

this is outdated. delete


Frame: Give Referendum SubmitOrigin argument

When determining whether a submission origin should be allowed, it is useful to be able to inspect the track they are submitting for, since it may be desired to have less-privileged origins be valid for some tracks but not others.
This alters the type bound accordingly. To minimise breakage across the ecosystem, all type bounders in Substrate are updated to support the EnsureOriginWithArg trait as well as the EnsureOrigin trait. To facilitate this support both in Substrate and downstream repos, a macro allowing EnsureOrigin implementors to naively support EnsureOriginWithArg is introduced.
🚨 Code Changes
If you are using Referenda pallet and find a build error around the SubmitOrigin (which is probably a custom origin), then simply wrap the type with frame_support::AsEnsureOriginWithArg to get an EnsureOriginWithArg impl. E.g.
impl pallet_referenda::Config for Runtime {
    type SubmitOrigin = MyEnsureOriginImpl;
    // snip
}

Would become:
impl pallet_referenda::Config for Runtime {
    type SubmitOrigin = frame_support::AsEnsureOriginWithArg<MyEnsureOriginImpl>;
    // snip
}

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-check-benches
Logs: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2959545

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: test-linux-stable
Logs: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2968318

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: test-linux-stable
Logs: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2968319


[iOS] onNotificationOpen doesn`t work, if App is in Background

On iOS the onNotificationOpen Event isn`t called, if the app is closed.
The code:
initRemotenotification() {
        let that = this;
        this.firebase.grantPermission();
        this.firebase.getToken().then(token => {
                this.appVersion.getVersionNumber().then(version => {
                    this.authService.getUserInfo().then(user => {
                        this.pushProvider.registerPush(token, this.device.platform, this.device.uuid, version, user);
                    });
                });
            },
            function (err) {

            }
        );

        this.firebase.onTokenRefresh().subscribe(token => {
                this.appVersion.getVersionNumber().then(version => {
                    this.authService.getUserInfo().then(user => {
                        this.pushProvider.registerPush(token, this.device.platform, this.device.uuid, version, user);
                    });
                });
            },
            function (err) {

            }
        );

        this.firebase.onNotificationOpen().subscribe(notification => {
                this.processAdditionalData(notification);
            }, error => {

            }
        );
    }

On Android the onNotificationOpen Event fires, if you clicked on the message in the notification bar. On iOS only the app will be open and the homescreen will be shown. The initRemotenotification) function is called, but the onNotificationOpen Event isnt fired. This work only if the app is in the list of open apps, if you double click on the home button. If the app isnt in this list, the event will not be fire.
How could I get the message data in iOS, if the app isn`t in the list of open apps?
Plugin Version: 0.1.24
ionic Version: 3.9.2

Same problem.

Same problem on android. With latest ionic

Finally, I managed to solve the problem. In my case the problem was due to a incompatibility with cordova-plugin-local-notifications
This plugin captures the event of ondidreceived of ios and disables the normal flow

closing as resolved

@Hanzofm
How to fix the incompatibility with cordova-plugin-local-notifications ?
local-notification (version: 0.9.0-beta2)
cordovan-firebase-plugin (version: 0.1.25)
Thank you.

Hi i am getting this issue and i have not local notification plugin !


Update for Self-Hosted Gateways

It would be great to see this document updated to include new options supported by the self-hosted gateway feature of API Management. Specifically, I'm wondering how you would setup the backend APIs in APIM now that you're gateway is running in-cluster. In that scenario, I should be able to use native Kubernetes DNS entries to address those pods instead of exposing APIs externally.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 64c50ed1-29bd-6ec8-7c57-60a6a76d36be
Version Independent ID: ed403a70-b955-347d-72e6-c875e88ccef7
Content: Use Azure API Management with Microservices Deployed in Azure Kubernetes Service
Content Source: articles/api-management/api-management-kubernetes.md
Service: api-management
GitHub Login: @miaojiang
Microsoft Alias: apimpm

@cwiederspan Thank you for your feedback! We have assigned this issue to the author to review further and take the next course of action.

#reassign:dlepow

GitHub user miaojiang has been removed from the MicrosoftDocs organization, so they were automatically removed as an assignee.

@cwiederspan - Thanks for this suggestions (and apologies for slow response). The article you mention is due for a refresh and I've noted your points which I think would be good to include when we do that. I've schedule work for a future doc sprint. Accordingly, I'll close this issue. Thanks again!
#please-close


Replaced continue with a return.

What's this PR do?
changes the continue inside of dosomething_reportback_mark_reportback_as_excluded  to a  return
How should this be reviewed?
👀
Any background context you want to provide?
causing php upgrade to fail
Relevant tickets
Fixes #6416

👍


chore: flatten eip-7685 requests into a single opaque list

Motivation
The three different request fields have been replaced by a single field.
Requests over engine API will now be in an opaquely encoded format as defined by EIP-7685, and they will be in a single list.
See https://github.com/ethereum/execution-apis/pull/591
Solution
Requests are now a list of opaque Bytes. Additionally, the changes to the engine API also removes the need for ExecutionPayloadV4, so I've removed that (and all variants of it).
PR Checklist

[ ] Added Tests
[ ] Added Documentation
[ ] Breaking changes

After some thinking we likely do not gain much by retaining the type here instead of using blobs, since validation is mostly a CL responsibility now, and system contracts adhering to EIP-7685 return the requests pre-serialized. For user-facing RPC it will matter more, but at that point in time it's ok to pay for the ser/de cost, while we'd likely want to avoid it during execution.

Looks like I need to update ExecutionPayloadBodyV2 as well?

Sorry I meant ExecutionPayloadBodyV4

why?

https://github.com/paradigmxyz/reth/blob/8d32fd788bfa8c2e041bd1b93ecca37093a09c5a/crates/rpc/rpc-types-compat/src/engine/payload.rs#L384-L428


Add Maven support based on existing Gradle support.

Have essentially copy and pasted from existing Gradle support.
Did try to pull common code into an abstract ExecutionEnvironmentProvider class but because MavenExecutionEnvironmentProvider and GradleExecutionEnvironmentProvider don't inherit from a common superclass it just ended up being messy and worse than just having the classes separate.
One thing I noted in testing is that this (and so presumably the Gradle support) works without needing the "Enable experimental features" checkbox enabled.

hi, does this look ok?  Can it be merged? :)

Thanks!

One thing I noted in testing is that this (and so presumably the Gradle support) works without needing the "Enable experimental features" checkbox enabled.

I wish I recalled why I added that... Maybe it's time to drop that option. 🤔


Feat/profile import

Description

Motivation and Context

How Has This Been Tested?

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My code follows the code style and structure of this project.
[ ] I have updated the documentation accordingly.

:tada: This PR is included in version 0.9.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Vods over 24 hours fail to show correct time. Processing fails. End proccess required.

Vods over 24 hours show length starting at 1.xx (maybe meaning 1 day?). The video downloads, however processing never completes. Processing only fails on Vods where the streamer is still live and adding to the vod.
url for vod: https://www.twitch.tv/dansgaming/v/93062533

How can it be that Twitch stores a VOD where the streamer is still live? Normally the COD pops up on the streamers video section, when he is done streaming. In your case I think DansGaming went offline, maybe to start another playlist. So the "live" part has probably nothing to do with the issue unless I missed some new features of Twitch where viewers can already view a VOD while it is not completely processed by Twitch.
And in general I do not know what is the exact purpose of downloading a 38 hour VOD. The processing probably fails because of the size/length (maybe a ffmpeg issue). Why not cropping shorter parts? Like downloading 0-2hrs, then 2-4 and so on? Way more managable on the disk or to share or process with a video software.
As for the issue itself, I cannot do anything without the download log. do you have it? i am not an oracle that can guess what the problem is and be always right, you know :P

Vods aren't published at the end of streams in one go. The Vods are updated every 15 minutes (ish) reflecting an updated time (where a vod would be at 30 minutes it would then show 45 minutes etc). Twitch is processing 15 minute chunks and adding them to the vod untill the streamer stops the stream.
I'm not trying to download the entire vod but a 2 hour part, it still fails the processing ( even if you download a minute of it it still fails to process. (The vod length is now 40 hours now as it's updating.)
This is as far as Twitch Leecher gets

Here is the log:
log.txt

Weird... I can see the DansGameing VOD getting longer, but no other streamer has this! Every streamer in my follow list who's streaming right now does not have a part of the VOD available and neither do i. i only get my latest stream as a VOD when i am finished. Is this some specialty for the really big streamers?

It must be a perk to streamers with 'x' amount of subs I guess. A streamer I follow that currently has 400 viewers has it. Another with 150 viewers doesn't.
I wouldn't worry too much about it as this only seems to happen in the rare circumstance where a VOD is over 24 hours and still being updated. I've used TL for months and never run into any problems, I love it.

WOHOOO, OMG!!! I discovered a major fuckup in the playlist parsing method that is responsible for exactly this problem and it also solves a bunch of other issues i received via the contact mail...
how could this be undiscovered that long? software development is amazing sometimes :D
basically what happens is the following:
whenever the stream quality or settings changes there is a line in the VOD playlist called "#EXT-X-DISCONTINUITY"... due to a parsing bug, this tag gets appended to the end of every playlist that i generate in TL. but only in case the tag really exists in the playlist. VODs that do not change settings do not have this tag in the playlist.
now this tag is responsible for making ffmpeg stop encoding. so every stream containing the tag probably  cannot be downloaded correctly... the encoding stops whenever the tag shows up (can be in the middle of the stream for example).
some people wrote that they wanted to download an entire VOD but it stops after the first half. this is exactly what we are talking about here.
Conclusion: It is time for a new version of TL!

Well I love the enthusiasm! Hopefully it wont be too hard to fix. Good luck!

🎉


Ability to remove files from upload queue prior to upload

Would be cool to have the ability to remove a file from the upload list before the upload starts, e.g. on this screen:

Imagine an X on each file row to remove the file.

#23 resolves this.


Initial draft of MI300A tuning guide, with images, toc, and index page changes

This guide incorporates content from the draft document along with supporting infrastructure changes to fill out the document, which I based on the MI200 guide. The merge will be coordinated with Peter Park, who is reworking the tuning-guide landing page.

I'm going to put my review in here in the form of
Current --> recommended update
I expect the recommendations be considered in the context and not necessarily adopted verbatim.  I'll make them accurate, but embellishment to better bridge to a 'user' would be beneficial.
These settings are critical for performance because the OS is now responsible for memory management across the CPU and GPU accelerators.  --> These settings are critical for performance because the OS on an APU is responsible for memory management across the CPU and GPU accelerators.
Merge System Settings paragraph 2 and 3.
In addition, some settings limit GPU memory allocation. --> In the APU memory model, system settings are available to limit GPU memory allocation.
The System BIOS settings section has 1 sentence fragment.  This section is a carryover from the previous document.  ROCM users are presumably going from one generation to the next.  The APU model is different so while there is not much to do in this section, some embellishment would be beneficial.
Will start again after a more critical review is done.  My original input was just notes... I'm from the system team and there doesn't appear to be continuity from anyone closer to the ROCM user base.

Closing and moving to a new branch


Add picking for 3D objects

Allow objects to be picked. This will form the basis for future UI interactions.

@cdrake , this can be closed too right?


ui-select: Fixing voice reading first item

Background
When users move cursor into the 1st options, there is no voice reading.
Approach
There are 2 main tasks here:

Update the HTML with new aria- for the browsers know the list and option elements
Change the condition inside controller so that it will recognise 0 is a valid value.

Limitation
There is still space for enhancing but it may take more time to achieve:

binding selected value into the component wrapper
new attributes "labelledby" for the ui-select to read out the content with label
optimize the bootstrap theme to use ul instead of div for single-selection type

Testing Instructions
Using NVDA software moving up and down to listen to the value content.

 (3)[Bounty][Accessibility] Dropdown issues

 (3)[Bounty][Accessibility] Dropdown issues


Vernot 2021

Reference
doi or URL: https://science.sciencemag.org/content/early/2021/04/14/science.abf1667
But check if there is any shotgun, there might only be mt/nuc capture which might not be relevant

No shotgun, only capture. Closing.


ext-authz: send SNI in CheckRequest to the authorization server

There isn't any way to pass along SNI used for the underlying connection to the external authorization server.
This patch introduces a new field on the CheckRequest to send TLS session data to the authorization server. Only SNI is sent with this patch, more TLS session-related data can be introduced in the future.

Commit Message: ext-authz: send SNI in CheckRequest to the authorization server
Additional Description:
Risk Level: Low because this is an optional feature in a filter
Work in progress, I will fill the below out if the change seems acceptable from an API perspective.
Testing: added a unit test to ensure that the additional data is correctly populated
Docs Changes: No docs changes are done assuming that the doc string in filter configuration is sufficient
Release Notes: Included
Platform Specific Features: N/A

I tried to read SNI via CheckRequest a few different ways:

First attempt was made to read via dynamic metadata. I was able to apply a patch to envoy to push SNI as dynamic metadata using the tls-inspector plugin. Unfortunately, dynamic metadata is not copied over from listener filters so I hit a roadblock.
Next, I tried to go back and see if it was possible to extract SNI by transforming metadata or logging operators into a HTTP header and then pass the header to the authorization server, I couldn't do that either because SNI isn't emitted by any component within Envoy.

This seems like an invasive approach but it is the best I could come up with. I'm open to alternate approaches.
I will work on the code to make it appropriate for merge if the direction is acceptable.
I have not contributed to Enovy before, please point out mistakes and I will correct them.
Thank you for taking the time to look at this patch.

ping @lizan Are you able to give this a review and let me know if this seems acceptable?

@mattklein123 I have improved the patch and added appropriate tests.
I still have to clean up the commits and add DCO, I'll do so if this patch seems close to what is expected.
Could you please take another look and give me some feedback?
I appreciate your help.

@mattklein123 I've resolved the API comment and added a changelog entry as well.


Cannot install @xmpp/console

When I try to install @xmpp/console I get:
$ npm install -g @xmpp/console

npm ERR! Linux 4.9.8-201.fc25.x86_64
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "install" "-g" "@xmpp/console"
npm ERR! node v6.9.5
npm ERR! npm  v3.10.10
npm ERR! code E404

npm ERR! 404 Not found : @xmpp/console
npm ERR! 404
npm ERR! 404  '@xmpp/console' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
[...]

oops; please try again should be fixed

Works now. Thanks.

@smokku I don't think we have any users for @xmpp/console yet please let me know your thoughts or missing features.

I'm using it while developing my jabber server (based on node-xmpp-server) to test stuff and it is extremely cool and useful. Thanks for the great code in this (node-xmpp) project. Console is great to get a grip what is on the wire. I was using Psi's XML console, but this is much better and I do not need to work around things a full client sends without asking me.
The features I miss so far:

persistent history - after I restart my server and reconnect console I do not have the test stanzas in history anymore and need to copy paste
continuation prompt - as soon as I press Enter stanza is parsed and if it is not complete I get ❌ invalid XML "<..." Error: Incomplete document; and as transmitted stanzas are pretty-printed it is painful to copy-paste stanzas from previous session to send again (see previous point)

xmpp-console does not start session after connecting. Is it by design?

@smokku can you elaborate?

Running xmpp-console "xmpp:user@localhost" "password" ends with:
⮊ OUT
<iq type="set" id="9954011249717829">
  <bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/>
</iq>

but to actually get online I need to manually send:
⮊ OUT
<iq type="set" id="123">
  <session xmlns="urn:ietf:params:xml:ns:xmpp-session"/>
</iq>

This will request session creation and will give me 'online' event.

@smokku <session xmlns="urn:ietf:params:xml:ns:xmpp-session"/> has been removed from the spec, please refer to https://xmpp.org/rfcs/rfc6120.html rather than https://xmpp.org/rfcs/rfc3920.html

So it is by design.
Should I create a bug, that node-xmpp-server still relies on it to bring user session online?
See: https://github.com/node-xmpp/node-xmpp/blob/master/packages/node-xmpp-server/lib/C2S/Session.js#L159 and https://github.com/node-xmpp/node-xmpp/blob/master/packages/node-xmpp-server/lib/C2S/Session.js#L335

@smokku yes please

@smokku thanks for the suggestions

continuation prompt - as soon as I press Enter stanza is parsed and if it is not complete I get ❌ invalid XML "<..." Error: Incomplete document; and as transmitted stanzas are pretty-printed it is painful to copy-paste stanzas from previous session to send again (see previous point)

What solution do you suggest?

continuation prompt - similar to what you have in shell if you type: cat "

@smokku can you open an issue for continuation prompt?

Does it make sense, when the preferred (at least for me now) way of running console is via rlwrap?


The end of the world is coming ?

Hi,
after theme update to 19.20, it seems to be the end of the world ? (my dashboard says it)
Should i worry ?

Btw, is this a joke  or something else ? ^^

New feature...but, not quite running normally yet (it's a way for updates from webmin.com to be displayed to users in the UI, like security updates, etc.). That's the test message, and will go away when the new Webmin is released (it was a joke that wasn't actually meant to be seen by anybody by Ilia for testing, but well...it didn't get changed before 19.20 went out).

😄 🤣 👍

:laughing: Yes. It's new messaging system. The theme is out and if you pulled the latest Webmin too, you would see it. Majority of people are not going to see it.
It's running quite normally, Joe! :smile: You know about the end of the world - means, message delivered.
Besides, I wasn't the one who came up with the message itself, even though I liked the humor!

I had to update webmin from the script without paying attention :sweat_smile: :
Thank you everyone, I understand better now, and I am relieved that the end of the world is not coming. :smile: :laughing:

I will cancel that test message


Custom launch scripts

currently server is launched through executing the command specified in the configuration, that should do in some cases but in many cases the execution of the server require more than a static command like specifying options based on certain conditions. can't LSP provide launch scripts a la VSCode ?

That's what the config property of LanguageHandler is for - if a language server really needs a complicated set of arguments then it can be implemented (and maintained!) outside of this repo.
Many servers don't seem to need it - for example the xml server could just as well default DwatchParentProcess.


'react-native-scripts' command not found on windows 7 os

Done installation as described in installation instructions:

npm i -g create-react-native-app
create-react-native-app my-project
cd my-project
npm start

After

npm start

i get the error "create-react-native scripts" command not found.
I updated node.js cleared als the caches and so on...
I also installed with administration privileges
Propably it is only windows related. Is there a workaround so "react-native-scripts" can be found on windows?

I upgraded (uninstalled and installed) yarn to latest version.
Now it works!

if anyone is willing to use npm only, then run this npm i react-native-scripts --save, then npm start or whatever the command you use


Fix missing import in example

Example does not run without.

Coverage remained the same at 87.0% when pulling 83e9d1a5ec3bf1247b80919fa3f7f6a675fbf996 on jens-k:patch-1 into 686dd2611accb62f823e2a0a14842403ad44e948 on NeuralEnsemble:master.

Thanks for the fast fix!

#ANDA2018 ;)


[Compiler] Add Program#host_compiler

This is an implementation of my proposal in https://github.com/crystal-lang/crystal/pull/9425#issuecomment-727559640
It uses the same compiler configuration for host compiler (used for macro run) as for the target compiler (unless cross-compiling). Without this, macros are compiled with default options only.
The compiler instance is now assigned to Program and serves as a basis for creating a host compiler.
The host compiler is also cached to re-use a single instance every time. This change may be up for debate. Technically, a compiler instance is capable to be re-used. And I don't think macro runs can be executed concurrently. So this should be fine. I'd like to hear some thoughts on that, though.

I guess the CLI options will come in a later PR?

Yes!


updated sidebar

Assignee Tasks

[ ] added unit tests (or none needed)
[ ] written relevant docs (or none needed)
[ ] referenced any relevant issues (or none exist)

Guidelines
Please add a description of this Pull Request's motivation, scope, outstanding issues or potential alternatives, reasoning behind the current solution, and any other relevant information for posterity.

Your PR Notes Here

moved sidebar to components folder


It.IsAny

Hi guys
Is there any way to configure TestKitSilo to return a mock grain when it is called with any id?
I have a case where a grain will generate a new Guid and then activate a grain of another type using that new Guid in order to create a new instance of the target grain.
Thinking in terms of MoqI would use something like It.IsAny() but TestKitSilo requires an actual Id. Since I am Guid.NewGuidding it inside my grain I can't inject a predetermined id without doing a lot of unnecessary factory stuff.
Is there any way to achieve this currently?
Thanks!

The way I achieve it is by storing newly-generated Id in grain state (I need it in grain state anyway).
In the unit test I can fetch the Id from state and access new grain mock.
Another option (I think, not sitting in front of the code now) would be to enumerate all mocks (reflection magic may be needed) and find the one of the type of the new grain.

I think my problem is that I need to know the Id at setup time because I need to configure the mock grain to return certain values at runtime. I have resorted to using an IdFactory in cases where this is needed for me.

I'd be open to a bit of a design discussion around this... I believe we hit it in a couple of projects in the past, and I resorted to TestCluster-based testing for these types of scenarios. I can see some potential for scaffolding a grain without knowing its unique identifier upfront. Do you have an API proposal? Additionally, what if the grain under test creates multiple, new grains of the same type? A bit of an edge case, but if there were an It.IsAny<Id> resolver, would these all resolve to a single instance?

@seniorquico I did some more digging in the code and found something that might work. I have inadvertently refactored away from needing to do this, but there appears to be a way to configure the TestGrainFactory using a factory func instead of an id:
void AddProbe<T>(Func<IGrainIdentity, IMock<T>> factory)

So I guess in theory we could do something like this.
Given this grain:
public interface IMysteryGrain : IGrainWithGuidKey
{
     Task<Guid> WhoAreYou();
     Task<string> WhyAreYou();
}

We could set up a queue to return mocks in a sequence:
var firstMock = new Mock<IMysteryGrain>();
firstMock.Setup(x => x.WhyAreYou()).ReturnsAsync("dunno");

var secondMock = new Mock<IMysteryGrain>();
secondMock.Setup(x => x.WhyAreYou()).ReturnsAsync("becos");

var mocks = new Queue<Mock<IMysteryGrain>>(new[] {firstMock, secondMock});

Then configure the TestKitSilo to return the next mock in sequence and even configure it with some setup based on the grain id it is given:
Silo.AddProbe(id =>
{
    var mock = mocks.Dequeue();

    mock.Setup(x => x.WhoAreYou())
        .ReturnsAsync(id.PrimaryKey);

    return mock;
});

And then confirm that things happened.
firstMock.Verify(x => x.WhoAreYou(), Times.Once);
secondMock.Verify(x => x.WhoAreYou(), Times.Once);

So I'm not sure if this will work or not, but it looks like it should and I will give it a try next time I need this.

Actually, I just had another need for this and it worked, at least in a simiplified scenario:
//Arrange
var newUserDeviceNotificationsGrain = new Mock<IDeviceNotificationsGrain>();
Silo.AddProbe(x => newUserDeviceNotificationsGrain);

//Assert
//blah

//Assert
newUserDeviceNotificationsGrain.Verify(x => //blah );

Very neat, @freever! 🎉
We lack official docs, but we generally try to document how to use the TestKit with unit tests. Do you think you'd have some time to create a new test suite with different cases that show off a few different ways this strategy could be applied? Maybe a simplified case where the grain needs to create just one other grain, and maybe a more complex case where the grain uses your Queue strategy where the grain needs to create multiple other grains?

Will do!
On Tue, 15 Dec 2020 at 18:20, Kyle Dodson<EMAIL_ADDRESS>wrote:

Very neat, @freever https://github.com/freever! 🎉
We lack official docs, but we generally try to document how to use the
TestKit with unit tests. Do you think you'd have some time to create a new
test suite with different cases that show off a few different ways this
strategy could be applied? Maybe a simplified case where the grain needs to
create just one other grain, and maybe a more complex case where the grain
uses your Queue strategy where the grain needs to create multiple other
grains?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OrleansContrib/OrleansTestKit/issues/101#issuecomment-745400111,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AARKDOEQPAZOO6RLH6EY2XDSU6EGFANCNFSM4UWETGFQ
.

Thanks @freever for contributing this example and a new test case!


copr: migrate executor and aggr to be compatible with ChunkedVec

What problem does this PR solve?
Problem Summary
This PR refactors co-processor framework internally. Now all data flowing in aggregator and executor uses the new BytesRef-like interface (or ChunkRef). This laid solid foundation for migrating to ChunkedVec.
Overview
In RFC: Using chunk format in coprocessor framework, we proposed a ChunkedVec struct to support Chunk-based computing. We plan to:

Migrate current co-processor framework to use interface compatible with ChunkedVec.

The detailed plan can be found at Full Chunk-based Computing PR Roadmap.
This PR follows #7979, #8019, #8070 . Now all internal stuff of copr uses NotChunkedVec instead of Vec<Option<T>>. As NotChunkedVec exposes nearly the same interface as our proposed ChunkedVec, we can directly replace this temporary struct with new ChunkedVec in next PR.

Introduce ChunkedVec.

After migrating co-processor to have interface compatible with proposed ChunkedVec, we can introduce ChunkedVec into TiKV. Then, we may further optimize performance by merging null parameters, etc.
What is changed and how it works?
Proposal: RFC: Using chunk format in coprocessor framework
Related Issue: #7724
What's Changed:
The biggest change in this PR is the introduction of NotChunkedVec. NotChunkedVec has nearly the same interface as the proposed ChunkedVec. All data flowing in copr now uses this new struct instead of Vec<Option<T>>.
tidb_query_codegen

rpn_fn now generates code with NotChunkedVec.

tidb_query_datatype

All internal logic now uses new interface.
Some functions that take Json before is now refactored to take JsonRef.
Add map_option_owned in collation, to support some aggregators.
Implement AsMySQLBool on EvaluableRef types. (e.g. &T, BytesRef)
Introduce NotChunkedVec.
ChunkRef now implements a new function called phantom_data, which returns None of the corresponding EvaluableRef type. This is used to let compiler find dynamic dispatch function in aggregators. This will be explained later.
Introduce  UnsafeRefInto trait, that is used to cast reference into 'static ones.
Add to_owned_value and from_owned_value for EvaluableRef.
Remove AsMut for VectorValue.

NotChunkedVec
A vector type that internally stores Option<T>, but only allows access to Option<&T>. It can be constructed from Vec<Option<T>>, or &[Option<T>].
tidb_query_vec_aggr

All data flowing in aggregators are now refactored to use impl ChunkRef. To make this process easier, the generic parameters are all set to 'static.
We ensure there won't be invalid memory access as we don't store reference in any of these aggr structs.
Corresponding traits now have naming scheme like update_balahbalah_unsafe.
Add update, update_concreate, update_vector and update_repeat macros for safely calling these traits.
As update_vector_unsafe now has a parameter with associative type T::ChunkedType, the compiler cannot find corresponding function by dynamic dispatching when AggrFunctionStateUpdatePartial is in a boxed type. In this way, we added a phantom_data of type Option<T> to this function, so as to make it compile. Users using update_vector! macro don't need to supply this phantom parameter, as this has been done in macro.

tidb_query_vec_executors

Refactored to use ChunkRef internally.
The most notable change is in top_n_executor, where an as_slice call is eliminated.

TODO

In fact, the tidb_query_vec_aggr interface can be refactored without using unsafe primitives. This can be done later.
In top_n_executor, we still can't eliminate the overhead of clone.
We should remove as_slice calls from non-test code. This will be done later with a trait only available in tests.
Use to_owned_value instead of .map(|x| x.as_ref()) for code not modified in this PR.

Check List
Tests

Unit test

@TennyZhuang @breeswish PTAL~

/run-all-tests

It seems that tests failed because of timeout, and the bot won't response to run-all-tests any more :(

/run-all-tests

Not sure why test fails... Seems ok on my local PC

/run-all-tests

An example without phantom_data

pub trait AggrFunctionStateUpdatePartial<TT: EvaluableRef<'static>> {
    fn update_vector(
        &mut self,
        // phantom_evaluable: Option<TT>,
        physical_values: TT::ChunkedType,
    ) -> Result<Option<TT>, ()>;
}

impl AggrFunctionStateUpdatePartial<&'static Int> for TestStruct {
    fn update_vector(
        &mut self,
        // phantom_evaluable: Option<&'static Int>,
        physical_values: &'static NotChunkedVecSized<Int>,
    ) -> Result<Option<&'static Int>, ()> {
        Ok(physical_values.get_option_ref(0))
    }
}

impl AggrFunctionStateUpdatePartial<BytesRef<'static>> for TestStruct {
    fn update_vector(
        &mut self,
        // phantom_evaluable: Option<BytesRef<'static>>,
        physical_values: &'static NotChunkedVecBytes,
    ) -> Result<Option<BytesRef<'static>>, ()> {
        Ok(physical_values.get_option_ref(0))
    }
}

pub trait AggrFunction:
    AggrFunctionStateUpdatePartial<BytesRef<'static>> + AggrFunctionStateUpdatePartial<&'static Int>
{
}

impl AggrFunction for TestStruct {}

macro_rules! update_vector {
    ( $state:expr, $physical_values:expr ) => {
        unsafe {
            $state.update_vector(
                // $physical_values.phantom_data().unsafe_into(),
                $physical_values.unsafe_into(),
            )
        }
    };
}

#[cfg(test)]
mod test {
    use super::*;

    #[test]
    fn test_chunk_ref_another() {
        let mut aggr: Box<dyn AggrFunction> = Box::new(TestStruct);
        update_vector!(
            aggr,
            &NotChunkedVecSized::from_slice(&[Some(1), Some(2), Some(3), None, None])
        );
    }
}

error[E0284]: type annotations needed
   --> src/evaluable.rs:144:20
    |
144 |               $state.update_vector(
    |               -      ^^^^^^^^^^^^^ cannot infer type for type parameter `TT`
    |  _____________|
    | |
145 | |                 // $physical_values.phantom_data().unsafe_into(),
146 | |                 $physical_values.unsafe_into(),
147 | |             )
    | |_____________- this method call resolves to `std::result::Result<std::option::Option<TT>, ()>`
...
159 | /         update_vector!(
160 | |             aggr,
161 | |             &NotChunkedVecSized::from_slice(&[Some(1), Some(2), Some(3), None, None])
162 | |         );
    | |__________- in this macro invocation
    |
    = note: cannot resolve `<_ as evaluable::EvaluableRef<'static>>::ChunkedType == &evaluable::NotChunkedVecSized<u64>`

Another way to eliminate this phantom_data is to replace state with <state as AggrPartialFunction<chunked_vec::EvaluableRefType>>.balabala. But this seems much more complex :(

/run-all-tests

@breeswish @TennyZhuang I've made the aggregation functions even safer by applying the following constraints.
Developers should use impl_state_update_partial to automatically implement unsafe parts
impl<T> super::AggrFunctionStateUpdatePartial<T> for AggrFnStateFirst<T>
where
    T: EvaluableRef<'static> + 'static,
    VectorValue: VectorValueExt<T::EvaluableType>,
{
    // ChunkedType has been implemented in AggrFunctionStateUpdatePartial<T1> for AggrFnStateFirst<T2>
    impl_state_update_partial! { T }
}

This macro will implement update_vector_unsafe, etc. which directly calls safe versions of these functions.
Developers should use generic lifetime in "safe" functions, so as to eliminate dangling pointer
    #[inline]
    fn update_vector<'a, TT, CC>(
        &mut self,
        ctx: &mut EvalContext,
        _phantom_data: Option<TT>,
        physical_values: CC,
        logical_rows: &[usize],
    ) -> Result<()>
    where
        TT: EvaluableRef<'a, EvaluableType = T::EvaluableType>,
        CC: ChunkRef<'a, TT>,
    {
        if let Some(physical_index) = logical_rows.first() {
            self.update(ctx, physical_values.get_option_ref(*physical_index))?;
        }
        Ok(())
    }

As we use a generic lifetime parameter, this code MUST be valid for all possible lifetime 'a. In this way, we can provide a "safe" interface to developers.
The only exception is in impl_max_min, where we have to use an unsafe transmute to deal with types. As we ensure T and TT are reference to same type, except that they have different lifetime, this transmute can be proven safe.
    #[inline]
    fn update_concrete<'a, TT>(&mut self, _ctx: &mut EvalContext, value: Option<TT>) -> Result<()>
    where
        TT: EvaluableRef<'a, EvaluableType = T::EvaluableType> + Ord,
    {
        let extreme_ref = self
            .extremum_value
            .as_ref()
            .map(|x| TT::from_owned_value(unsafe { std::mem::transmute(x) }));
        if value.is_some() && (self.extremum_value.is_none() || extreme_ref.cmp(&value) == E::ORD) {
            self.extremum_value = value.map(|x| x.to_owned_value());
        }
        Ok(())
    }

@zhongzc PTAL~ Thanks!

/bench +sysbench +tpcc +tpch

Hi @zhongzc , it seems that tpcc has failed and tpch has been running 14 hrs without a result. Shall we trigger benchmark again?

/bench +tpcc +tpch

/run-all-tests

/merge


fix: avoid issues with pip by making it a core dependency

What I did
Should hopefully take care of a few issues that arise from not having the latest version of pip when installing
How I did it
Added pip to core dependencies
How to verify it
Validate plugin installs
Checklist

[x] All changes are completed
[x] New test cases have been added
[x] Documentation has been updated

Not working ugh
https://github.com/ApeWorX/ape-fantom/actions/runs/1973927487

I think this solution does not meet our needs.
Reason: We need to ensure pip is up-to-date before running the setup.py. Otherwise, I think the install process uses the pip that was installed from the start and does not update in time.

You're right, I don't think this would work correctly.


GDScript 2.0: GDScript compiler issue with super construct

Godot version
4.0.alpha1.official.31a7ddbf8
System information
Windows 7
Issue description
Godot Editor crashes when attempting to invoke a completion query (command "ui_text_completion_query", Ctrl + Space) on an attribute access operator on a super() construct, provided that it must have a return type.
Steps to reproduce
func to_string() -> String:
	return super() # add a dot at the end of "super()" and wait or press 'Ctrl + Space' (command "ui_text_completion_query")

Minimal reproduction project
No response

This also seems to duplicate #57856

Looks like another type detection issue same as #57669

Doesn't seem happening anymore in alpha 7.


Update the character controller to allow walking and jumping

Resolves #127, resolves #60
Player movement
The rough high-level algorithm is as follows:

Check if the character is on the ground
Initiate a jump if appropriate
Accelerate the player based on which keys are held down
Apply gravity, air resistance, and speed cap
Use the computed velocity to move the player, resolving collisions as necessary

Camera orientation
The logic that keeps the player oriented vertically relative to the world is handled client-side.

If the player is almost facing vertically, yaw and potentially pitch the view to reach alignment.
Otherwise, roll the view to reach alignment.

There are several subtleties needed to do the following:

Keep the camera alignment consistent during player movement
Interpret mouse-based pitch and yaw controls
Convert an arbitrary orientation to an aligned orientation (algorithm described above)
Ensure that the player does not get lifted up from the ground by slanted walls

Besides the things I've inevitably missed, there are a few changes that I know are needed before this PR is merged, but limits in my own level of experience and sense of good code structure mean that I will need feedback before I make these changes:

A lot of character_controller's methods have way too many arguments. (Need feedback on which abstractions make sense to combine arguments. Should there be a character state struct? Should functions take fewer mutable arguments in favor of return values?
Client's sim.rs has too much functionality. I believe it would make sense to have client-side-only character controller logic in a separate file (Need feedback on whether you agree and how much of this refactor should be in a separate PR)
There are multiple modules within character_controller, which should probably be separate files. (Need feedback on whether you agree with the idea of having character_controller be a folder within common with the character_controller module and its two submodules)

The character controller is definitely not perfect. The most glaring issue is that walking into an obtuse wall corner will result in some shaking. However, this is expected behavior, and fixing this I believe should be outside the scope of this PR. In addition, the necessary input lag when jumping means that some coyote time would likely be a good idea, but I also consider that out of scope.
I believe the most important aspects of the character controller are as follows:

It should feel relatively fun and intuitive
The code should be readable and malleable enough that improvements can be made by someone other than me.

Most other aspects would likely have a lower priority than adding other essential features to this game, like block manipulation and saving/loading.

To better playtest the character controller, I created a temporary branch on my fork with block placing/breaking: https://github.com/patowen/hypermine/tree/collision-wip-block-placing-temp.
As of the writing of this comment, the character controller code between this branch and that branch are identical, and the only difference is that the collision-wip-block-placing-temp branch has hastily-put-together block manipulation functionality borrowed from the collision demo and adapted to partially work with netcode (but not multiplayer).

If the player is almost facing vertically, [...] Otherwise, [...]

Could we simplify this to always rotating around whatever axis produces the minimum rotation?

I believe it would make sense to have client-side-only character controller logic in a separate file.

Strong agree. In an ideal world where we fully understand how to factor everything, I'd expect sim.rs to contain only the most trivial logic, instead mainly dispatching to other modules.

Need feedback on whether you agree with the idea of having character_controller be a folder within common with the character_controller module and its two submodules

That seems fine.

I believe the most important aspects of the character controller are as follows: [...]

SGTM.

Could we simplify this to always rotating around whatever axis produces the minimum rotation?

I didn't go this route because the math is more complicated, and I don't think it's any better for the player. One case to consider is the fact that a rotation about the look-axis (roll) is less jarring than a rotation about a perpendicular axis, since the object the player is looking at stays fixed. A roll is almost always ideal, and the main exception is when the camera is close enough to vertical that it's no longer obvious which way is right-way up, since then, the correction starts to feel random. That's when yaw and/or pitch becomes nicer.
For a concrete example, if the player is facing forward but is upside-down, I believe the right move is to turn it 180 degrees to make the player right-way-up, but one can reach a valid orientation by having it face straight up or straight down instead, which is only a 90 degree rotation.
All other suggestions seem good to me, and I should get a chance to address them soon.

I am getting some "WARN Unsatisfied existing bound is parallel to new bound. Is the character squeezed between two walls?" messages (a warning I added for an edge case that should ideally be rare). I'm not sure whether my latest changes on July 4th caused them or whether it was the case before then, but when I have more time, I should be able to find the root cause.


Specified line endings in .gitattributes.

I suggest specifing line endings into .gitattributes file, otherwise hundreds of tests don't pass.

https://github.com/composer/composer/blob/master/.gitattributes

?

I have overlooked the link target, I'm sorry! Aren't those lines sufficient?

Currently the control of white characters are checked by fixer itself.
If we want to go with git as well - why not use more advanced checks?

Well, I think those additional options don't make any difference when checking out files. Line ending option does. When I switched EOL to LF via git config, I had to re-checkout all files. Then all tests passed.

it disallows you to add the file with tabs, for example

if we start with .gitattributes I'm :+1: for making it as complete as possible, so adding the other lines as well

@adawolfa
could you update ?

btw. if take
*.yml   text eol=lf whitespace=blank-at-eol,blank-at-eof,space-before-tab,tab-in-indent,tabwidth=2

we should update;
https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/1.12/.travis.yml
https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/1.12/appveyor.yml
as these are indented with 4 and not 2 spaces

-*.yml   text eol=lf whitespace=blank-at-eol,blank-at-eof,space-before-tab,tab-in-indent,tabwidth=2
+*.yml   text eol=lf whitespace=blank-at-eol,blank-at-eof,space-before-tab,tab-in-indent,tabwidth=4

magic ;)

Replaced by #2188


Add missing #include.

VS2019 complained about std::string.

Turns out this was already fixed, but the submodule didn't point to the fixed commit yet.

@wintersteiger Sorry that you encountered the issue with missing include.
@mingweishih will be updating the submodule in openenclave repo as part of his work on ECALL Refactoring.


Match regexes in an array

I'm trying to mock a service that takes an array of active storage urls but the matcher seems to mangle regexes I put in an array:
stub_request(:post, "http://service:8080/")
      .with(
        body: {
          urls: [match(%r{https://example.com/rails/active_storage/disk})],
       }
     )
     .to_return(status: 200, body: network_return)

gets me:
Body diff:
 [["-",
  "urls[0]",
  "https://example.com/rails/active_storage/disk/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaDdDVG9JYTJWNVNTSWhOVGhsWWpoMmMzbGxiVzlxTW1jeWEyazRaMlJqZEdSdFoyZDJkd1k2QmtWVU9oQmthWE53YjNOcGRHbHZia2tpQWNCaGRIUmhZMmh0Wlc1ME95Qm1hV3hsYm1GdFpUMGlkak10Y0d4aGFXNTBaWGgwTFRBd01EQXdNREF3TFRBd01EQXRNREF3TUMwd01EQXdMVEF3TURBd01EQXdNREF3TUMweU1ESXhMVEEwTFRBeFZEQTFMVEF3TG1Wa1oyVmliRzlpSWpzZ1ptbHNaVzVoYldVcVBWVlVSaTA0SnlkMk15MXdiR0ZwYm5SbGVIUXRNREF3TURBd01EQXRNREF3TUMwd01EQXdMVEF3TURBdE1EQXdNREF3TURBd01EQXdMVEl3TWpFdE1EUXRNREZVTURVdE1EQXVaV1JuWldKc2IySUdPd1pVT2hGamIyNTBaVzUwWDNSNWNHVkpJaDFoY0hCc2FXTmhkR2x2Ymk5dlkzUmxkQzF6ZEhKbFlXMEdPd1pVT2hGelpYSjJhV05sWDI1aGJXVTZDWFJsYzNRPSIsImV4cCI6IjIwMjEtMDgtMDNUMTE6MTg6MzAuMTk4WiIsInB1ciI6ImJsb2Jfa2V5In19--c5e1597a9c4e5db029b458c6639e775f26897422/filename"],
 ["+",
  "urls[0]",
  /https:\/\/example.com\/rails\/active_storage\/disk/]]

@saty9 It's not something supported by WebMock, though possibly it will work with RSpec's hash_including.
E.g body: hash_including(urls: [....])


Fixed typo in get parameter in docs [skip ci]

Q
A

Is bugfix?
no

New feature?
no

Breaks BC?
no

Tests pass?
yes

Fixed issues
-

amout should be amount.

Thanks!


Bug on Remove Translation

Describe the bug
When I removed a translation key by right-click on the tree, all the translation content disappear.
I have selected "Auto save" option, and now it's really hard to undo it
To Reproduce

Open new project
Right click on the tree of  translation keys
Select "Remove Translation"
Translation key that I selected has been removed and ALL ANOTHER TRANSLATION CONTENTS REMOVED TOO???

Screenshots

My i18n file after remove translation key:

I will appreciate if you fix it as soon as possible
Many thanks!

@tiendat77 Fixed. Please try it again bro :))

No, Test carefully bro :))

It's work fine. Thanks ❤


Add support for MSVC2015

As the title says.

The main branch for autowiring is now master, go ahead and resubmit this PR targeting master and we should be good.


Bugfix/alpha imports

✨ Codu Pull Request 💻

Pull Request details

Revert removal of additionalDetailsSchema as it broke imports and build
Move to (app) folder

Any Breaking changes

I hope it fixes breaking changes 🙈

@John-Paul-Larkin updated your file locations and reverted the removal of the Schema because it was breaking things. Make sure you pull the latest when this is merged 🦾

@John-Paul-Larkin updated your file locations and reverted the removal of the Schema because it was breaking things. Make sure you pull the latest when this is merged 🦾

Sorry, I missed this. I am not used to parsing the emails.


Update repo for release v0.3.0

New release v0.3.0 with all the bug fixes and improvements.

/assign @jimmidyson

Thanks @hectorj2f. Looks fine to me.
/lgtm


Json serialization for map throwing error

I am trying to compile and run below code:

import { JSON } from "json-as/assembly";

@json
class Inner {
  name: string;

  constructor(name: string) {
    this.name = name;
  }
}

@json
class Outer {
  name: string;
  m: Map<string, Inner>;

  constructor(name: string) {
    this.name = name;
    this.m = new Map();
  }
}

export function init(): void {
  let serializedState = JSON.stringify<Outer>(new Outer("bhavya"));
}

command: npm run asbuild runs fine. When I am running it using the release.js wrapped function, it is throwing below error:
wasm://wasm/0001a6ba:1

RuntimeError: memory access out of bounds
    at wasm://wasm/0001a6ba:wasm-function[25]:0x159f
    at wasm://wasm/0001a6ba:wasm-function[26]:0x5905
    at wasm://wasm/0001a6ba:wasm-function[27]:0x5ce2
    at file:///<BASE-FILE-PATH>/tests/index.js:2:13

. And if I replace Inner with u64 like below:

@json
class Outer {
  name: string;
  m: Map<string, u64>;

  constructor(name: string) {
    this.name = name;
    this.m = new Map();
  }
}

It is working fine.

Thanks, taking a look in a moment

Thanks @JairusSW for the response. Let me know if you need more information about the system on which I am running the example.

Just checking, but your using v0.9.8 (latest) I presume?

No, I am using ^0.8.8 and visitor-as with ^0.11.4

Looks like I didn't support serializing empty maps ({}). Fixed in latest version v0.9.9

Thanks @JairusSW for mentioning this! It's working fine with this version.


Add Lingo language

Description
Adds support for the Lingo programming language, created for Adobe Director.
Note that excluding the biggest contributing organisation, Lingo likely does not currently meet popularity requirements.
Checklist:

[x] I am adding a new language.

[ ] The extension of the new language is used in hundreds of repositories on GitHub.com.

Search results for each extension:

https://github.com/search?type=code&q=NOT+is%3Afork+path%3A*.ls+%2F^on\s\w%2B([\s\S]*%3F)\nend%24%2F

[x] I have included a real-world usage sample for all extensions added in this PR:

Sample source(s):

AutoDismisser.ls
Progress.ls

Sample license(s): MIT

[x] I have included a syntax highlighting grammar: https://github.com/markhughes/vscode-lingo-director
[x] I have added a color

Hex value: #974413
Rationale: Calculated as "average colour" in Adobe Director's logo.

[x] I have updated the heuristics to distinguish my language from others using the same extension.

Lingo? Holy hell, that's a name I've not heard since 2007.

Lingo likely does not currently meet popularity requirements.

Unfortunately, I'm not sure it ever will. GitHub's code search only indexes files that've been modified or published within the last few years, making it that much more difficult to gauge the eligibility of once-popular programming languages. Because Adobe Director was considered an antique even 15 years ago,[^1] it's likely that the majority of Lingo scripts on GitHub are situated in repositories that haven't been touched in a very, very long time.
[^1]: By whom? Well, everybody in my animation class at TAFE, at least. (Hey @dsarajlic, remember Julie's animation class at TAFE? Where we had to animate a scene in a program with a single-level undo history? Good times).

@Alhadis No doubt, I only discovered it because I was archiving / exporting from a 23 year old game 🙂
Just in case it does have some bizarre resurgence (hey, you never know!), I thought the PR would be good practice at Linguist's flow if nothing else!


ModSecurity plugin

Hello.
it's possible to add Modsecurity plugin to Traefik?
https://github.com/acouvreur/traefik-modsecurity-plugin
thank you
best regards

Hi,
To be honest, I don't see the point of this plugin. As you could apply all those rules yourself. Also you'll need owasp/modsecurity-crs running somewhere in the background, so this would create an unwanted dependency. And last but not least, it seems a very unstable, fairly new plugin.
So, no. I'm not going to add this plugin.


[8.x] Http client fix to prevent str_parse for bodies that are not encoded forms

This potentially fixes: https://github.com/laravel/framework/issues/36976
The idea is to parse_str only on content that either has no content type provided or it has explicit www-form-urlencoded type given.
I am not 100% sure about this fix, but it solves my issue and no other tests break.

If we don't feel confident about this fix I would rather not merge it until more people can chime in.

There should be some kind of fix, because currently the parse_str() is run on XML content which produces invalid data in the laravel_data variable. Cleary the code is doing what is should not do. Can we get input from the author of those lines of code - what was the purpose of it?

@briedis I wrote this code to handle the scenarios listed below, specifically the parse_str is to handle where the query is passed as a string or part of the url rather than an array.
Http::get('http://foo.com');
Http::get('http://foo.com?foo=bar');
Http::get('http://foo.com', ['foo' => 'bar']);
Http::get('http://foo.com', 'foo=bar');
Http::get('http://foo.com/get?foo=bar&page=1', ['hello' => 'world']); // Hello world overwrites params
Http::get('http://foo.com', ['foo;bar; space test' => 'laravel']);

While your scenario is somewhat of an edge case we can probably just wrap the parse_str in a try catch to handle this rather than performing a fragile check on the content type. For example:
if (is_string($laravelData)) {
    try {
        parse_str($laravelData, $parsedData);
        $laravelData = is_array($parsedData) ? $parsedData : [];
    } catch (\Throwable $e) {
        $laravelData = [];
    }
}

@dmason30  Please check the attached issue: https://github.com/laravel/framework/issues/36976
parse_str is very dumb, if it sees an "&" it can parse it.
You can see it here:

I would say may case is very far from an edge case. I try to upload an XML with ::withBody($xml) method and it dies with a completely unrelated error.

I don't understand  why the try catch doesn't help? It would prevent the exception from bubbling up?
My proposed solution is the simplest and is most likely to be approved to merge.


[WIP] [SPARK-8426] Enhance Blacklist mechanism for fault-tolerance

What changes were proposed in this pull request?
Update of https://github.com/apache/spark/pull/8760 by @mwws.  The current blacklist mechanism only considers one task a time -- this expands that by considering:

When we determine an executor is bad, we blacklist all tasks from that blacklist, both within the taskset and subsequent task sets.
When many executors on a node appear to be bad, we blacklist the entire node.

How was this patch tested?
Unit tests via jenkins.
Also I ran the additional tests proposed here which include blacklist tests.
TODO:
[ ] performance tests
[ ] more internal comments (in particular on concurrency)
[ ] manual testing on a cluster

Test build #59031 has started for PR 13234 at commit f6bb6de.

For the performance tests, I've collected data here: https://github.com/squito/spark/pull/5 (for lack of a better place).  The brief summary here: the advanced strategy is indeed much slower, but I don't know why yet.

Test build #59031 has finished for PR 13234 at commit f6bb6de.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59031/
Test PASSed.

Test build #59344 has started for PR 13234 at commit 8f2534b.

Test build #59344 has finished for PR 13234 at commit 8f2534b.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59344/
Test FAILed.

(closing till this is in a better state to avoid triggering tests)


Metric Type For UDP Total Session

Existing documentation URL(s)
| cloudflared_tcp_total_sessions | Total number of TCP sessions that have been proxied to any origin. | COUNTER |  |

| cloudflared_udp_total_sessions | Total number of UDP sessions that have been proxied to any origin. | GAUGE |  |
What changes are you suggesting?
Metric type for UDP Total Sessions should be Counter. If this gets reviewed and updated, the Prometheus endpoint of the tunnel will need to be updated as well.
Additional information
File a bug issue over cloudflared
https://github.com/cloudflare/cloudflared/issues/1253

thanks for flagging this issue!


Create bitcoin

/*

ERC20 interface
see https://github.com/ethereum/EIPs/issues/20
*/
contract ERC20 {
uint public totalSupply;
function balanceOf(address who) constant returns (uint);
function transfer(address to, uint value);
function allowance(address owner, address spender) constant returns (uint);

function transferFrom(address from, address to, uint value);
function approve(address spender, uint value);
event Transfer(address indexed from, address indexed to, uint value);
event Approval(address indexed owner, address indexed spender, uint value);

what's this about @btc-kz?

Closing in 24 hours if there is no reply-comment from @btc-kz on this pull request (PR).
If there is no reply-comment on this PR from @btc-kz and if there are future PRs from @btc-kz then that account will be reported to GitHub for possible spam activity. Thank you for understanding @btc-kz.


Multiple context operations in parallel - how to find - call stack in thread pool

I'm getting this exception -
System.InvalidOperationException: 'A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe.'

I understand the reason for this, however I've carefully checked and there's no async / multithreaded code I can see anywhere.  The stack trace is simply:
>	Microsoft.EntityFrameworkCore.dll!Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection()	Unknown	Non-user code. Skipped loading symbols.
 	[Aspect Code]		Annotated Frame
 	System.Private.CoreLib.ni.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown	Non-user code. Skipped loading symbols.
 	System.Private.CoreLib.ni.dll!System.Threading.QueueUserWorkItemCallbackDefaultContext.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()	Unknown	Non-user code. Skipped loading symbols.
 	System.Private.CoreLib.ni.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()	Unknown	Non-user code. Skipped loading symbols.

Steps to reproduce
Big and complex project - can you assist with some tips as to how to track down which query is being executed, etc?  There are no async calls I can see, and no multithreaded code.
Further technical details
EF Core version: 1.1.2
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system:  Windows 10
IDE: VS 2017 15.3 Preview 7

Incidentally, I was getting random errors about connection states and data readers until I added ToList to the end of all my queries.  That's what caused this more informative message to appear - prior to that it was all SingleOrDefault() and no nice message.

(@twistedtwig and I work together) - in the end, there was an issue further up the DI tree which was the ultimate cause.
However we were wondering if there's value in keeping this open to see if -
a) there's a problem where the helpful error message isn't triggered when using SingleOrDefault()
b) there's anything that can be done to give more context about the query - the stack trace/message didn't shed any light about the query/context/etc

Closing as we don't plan to do anything here for now. We can revisit if we get more feedback about this.

Hello,
We are having the same issue in a large app. We can't find the code that produce the error
System.InvalidOperationException: A second operation started on this context before a previous operation completed. This is usually caused by different threads using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
   at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection()
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()

Is there any way we can find the call stack?


Weird issue with LINKLIST consistency

Expected behavior and actual behavior
Got the following error:
com.orientechnologies.orient.core.exception.OValidationException: The field 'LegalEntity.activityPlaces' has been declared as LINKLIST but contains a null record (probably a deleted record?)
    at com.orientechnologies.orient.core.record.impl.ODocument.validateLink(ODocument.java:436)
    at com.orientechnologies.orient.core.record.impl.ODocument.validateLinkCollection(ODocument.java:417)
    at com.orientechnologies.orient.core.record.impl.ODocument.validateField(ODocument.java:287)
    at com.orientechnologies.orient.core.record.impl.ODocument.validate(ODocument.java:2005)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1085)
    at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:604)
    at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:156)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2612)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2580)

The problem is that this field is being modified. Probably link was broken before, but due to this error there is no way to fix the issue.
Steps to reproduce the problem
Hard to say, but probably the following will work:

Create entities A and B
Entity A through property of type LINKLIST should reffer to B
Delete B
Try to modify A

Important Questions
Issue looks quite severe. Why during save db can't just simply remove broken "null" references?
Runninng Mode

[x] Embedded, using PLOCAL access mode
[ ] Embedded, using MEMORY access mode
[ ] Remote

Misc

[ ] I have a distributed setup with multiple servers. How many?
[ ] I'm using the Enterprise Edition

OrientDB Version

[ ] v2.0.x - Please specify last number:
[x] v2.1.x - Please specify last number:  2.1.16
[ ] v2.2.x - Please specify last number:

Operating System

[x] Linux
[ ] MacOSX
[ ] Windows
[ ] Other Unix
[ ] Other, name?

Java Version

[ ] 6
[ ] 7
[x] 8

Hi @PhantomYdn
I tried to reproduce the problem with this code, but it seems to work fine:
  @Test
  public void testODocumentValidation() {
    String url = "plocal:/tmp/ODocumentValidation";
    ODatabaseDocumentTx db = new ODatabaseDocumentTx(url).create();
    try {

      OClass foo = db.getMetadata().getSchema().createClass("Foo");
      foo.createProperty("linklist", OType.LINKLIST);

      OClass bar = db.getMetadata().getSchema().createClass("Bar");

      ODocument bar1 = new ODocument(bar);
      bar1.save();
      ODocument bar2 = new ODocument(bar);
      bar2.save();

      ODocument foo1 = new ODocument(foo);
      List linklist = new ArrayList();
      linklist.add(bar1);
      linklist.add(bar2);
      foo1.field("linklist", linklist);
      foo1.save();

      bar1.delete();
      foo1.field("name", "foo");
      foo1.save();

    } finally {
      db.close();
    }

Can you please suggest a way to reproduce it?
(I also tried via SQL, still no errors...)
Thanks
Luigi

@PhantomYdn closing this issue because no answer. In case, please reopen/comment it.

@lvca , sure - I would be able to reopen, but it seems that the ability to reopen issue by common users is disabled. So only OrientTechnology team can reopen.

I thought if you create an issue you can reopen it. Is there any setting for it? Anyway, in case comment this issue and somebody will reopen it.

It seems there is such settings. Regarding reopening. Please help to reopen this one: #6157 - can't move to 2.2.0 without fixing of those issues.

Hi @PhantomYdn
I just reopened both issues.
Do you have any hint on how to reproduce this one?
Thanks
Luigi

Hi,
I'm closing this now. If you have additional details on how to reproduce this, please let us know.
Many thanks,


weave setup fails in Kubernetes example

I deleted all VMs and re-booted my system and ran vagrant up SSH'ed onto kube-01 and it showed the following:
CoreOS stable (723.3.0)
Update Strategy: No Reboots
Failed Units: 2
  weave.service
  weaveproxy.service

So, then I ran:
weave launch
FATA[0000] Error response from daemon: Cannot start container 18e96982e04f4baf07f78bc276282946fb42efd7c470bac09ab900872683c064: [8] System error: write /sys/fs/cgroup/cpuset/system.slice/docker-18e96982e04f4baf07f78bc276282946fb42efd7c470bac09ab900872683c064.scope/cgroup.procs: no space left on device

seems to be fixed now.


Add 'Export to Google Sheets' button to QT responses

This will call a cloud function or link to a google sheet which holds all entries to a QT
Just needs to be a button on the QT overview page.

@tomlovesgithub do we need this?

@Rebecca-mcknight1991 we implemented an 'export to excel' instead so I suspect this can be put in the ice box, or deleted.


"Moldoveni" is listed in the drop-down menu

Description:
When user writes "Moldoveni" the location appears in the drop-down menu next to the comune and the county witch is a part according to the law
Precondition:
The website is up an running.

Step 1
Write in the search bar "Moldoveni"
Expected results
The location appears in the drop-down menu as "Comuna Moldoveni, ** IALOMIȚA**"


Plugin will only does text replacement in the first sketch window

Sketch Version 3.4.2 (15857)
To reproduce:

Open a new document in Sketch (Untitled)
Create another new document in Sketch (Untitled 2)
Observe that both Untitled and Untitled 2 are available from TitleBar > Window
In Untitled 2, create a rectangle and some text
Get a placeholder image for the rectangle by selecting Plugins > Content Generator Sketch Plugin > Persona > Photos > Male
Observe that the image is replaced correctly
Select the text in Untitled 2 and try to get a placeholder name through Plugins > Content Generator Sketch Plugin > Persona > Names > Male
BUG Notice that nothing happens...

Text replacement actions performed in the window "Untitled" will succeed.
Text replacement actions performed n the window "Untitled 2" will succeed if it is the only (or first) window open.
Expected:

Actions are always performed on the Active sketch window

@cheics Thanks for a great report! Can you please submit as well what do you see in the Console app when the plugin is not running? Thanks
CC @bomberstudios it might be relevant to https://github.com/timuric/Content-generator-sketch-plugin/issues/81

Doesn't seem to be any output in Console.app when the plugin is not running.

@cheics sorry for ambitious message above. What I meant is the Console output when you run the plugin whilst it is broken :)

@timuric FYI the problem @cheics discribes happens with a lot of sketch plugins since the 3.4 update. So probably not directly a problem with your plugin.

Unfortunately, there is absolutely no console output when the bug occurs.
Here's a video of the procedure:
https://youtu.be/OKZRe96zbX0
On 17 November 2015 at 16:49, Peter Klaas<EMAIL_ADDRESS>wrote:

@timuric https://github.com/timuric FYI the problem @cheics
https://github.com/cheics discribes happens with a lot of sketch
plugins since the 3.4 update. So probably not directly a problem with your
plugin.
—
Reply to this email directly or view it on GitHub
https://github.com/timuric/Content-generator-sketch-plugin/issues/82#issuecomment-157519931
.

@cheics in Console.app try selecting system.log in the left pane (apparently this is how it works with Sketch + El Capitan)

Still no output when selecting system.log in Console.app.
On 17 November 2015 at 16:52, Timur Carpeev<EMAIL_ADDRESS>wrote:

@cheics https://github.com/cheics in Console.app try selecting
system.log in the left pane (apparently this is how it works with Sketch

El Capitan)

—
Reply to this email directly or view it on GitHub
https://github.com/timuric/Content-generator-sketch-plugin/issues/82#issuecomment-157520658
.

Again, should be fixed in 3.4.3


Color Customization - HEX Codes

Discussed in https://github.com/ajnart/homarr/discussions/731

Originally posted by MasterKryszn February 23, 2023
Would like to throw out suggestion to implement way to change the theme to very specific user defined colors using HEX Color Codes
For business/public use it would be nice to be able to more specifically match color schemes an organization already uses. The predefined colors in the current option pop out are nice, but not comprehensive

I'm not sure how easy this is to implement. Irrc colors from HEX need to specified using a special function

https://mantine.dev/theming/colors/#primary-color

To clarify on the idea, does not necessarily need to be HEX codes, that's just what I'm personally more familiar with
Some kind of color wheel, RGB options, etc.; i.e., a method for use to really customize and pick more specific colors than a handful of predefined options

Even if its just something more like this (compared to current selection)

We'll use this in combination with a HEX / RGBA input:
https://mantine.dev/core/color-picker/

Y'all are awesome, thanks for being so open to my ideas; really appreciate it and looking forward to next release! Any general ETA of when that may be? I saw the 'disable edit' environment variable made it in (thanks again!)

No, we have no ETA on this.
The next release might take a bit longer than usual, because most maintainers of Homarr don't have time within the next 4 weeks to actively work on Homarr.
But feel free to contribute 😉

I'm learning, but I hope to contribute to projects once I actually become competent lol

Actually, I'll close this issue as this is not planned for the moment.


SGDD buckets are lacking README and manifest

Both cgl-sgdd-reorg and cgl-sgdd-work buckets are lacking manifests and README.
The README should explain the provenance of the data and the directory layout. The manifest should list the checksums of all files.
Let me know if I am missing something here ...

cgl-sgdd-reorg  files have *.md5 files in parallel to the files.
But certainly I can make a manifest file listing all files also with the MD5 checksums too , do you have an example specific format that the manifest should conform to?
I will do the same for cgl-sgdd-work, which is goal of #163

My mistake re MD5's. My preference is using S3 object metadata, then .md5 files, then manifests. So we should be good once the .md5 files show up in cgl-sgdd-work.
So we just the READMEs briefly explaining the directory structure, file naming conventions and provenance of the files.
But let me know if I am not making sense.

Obsoleted by FireCloud disaster.


Unable to decrypt this message.

Lately I'm getting this message on a daily basis:
"Unable to decrypt this message. Check whether you've chosen to only encrypt to trusted devices."
After I send another message it fixes itself.
https://i.imgur.com/eAmlW1s.jpeg

Yeah I can confirm the issue. Almost feels like nio isn't sending keys frequently enough or so, but that's just a gut feeling.

Thanks for feedback.

I'm closing this as a duplicate of #69 which has more info.