Return existing group on duplicate welcome message

Related to https://github.com/xmtp/xmtp-react-native/issues/427

The new repro test passes about 3/10 times.
Before the fix it passed 0/10 times.
When it passes I see that that we are entering the code of the fix, and returning an existing group after failing to process a welcome_id(bc group already exists).
When it fails I see db locking errors. See logs below
here is a log of the new test passing:

logs

warning: Patch `openssl-sys v0.9.92 (https://github.com/xmtp/rust-openssl.git?branch=clone-v0.9.92#00dacb93)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.29s
     Running unittests src/lib.rs (target/debug/deps/xmtpv3-7af2e052c28a47b0)

running 1 test
[INFO][t:27583595]: [libxmtp] Logger initialized
[INFO][t:27583595]: [libxmtp] Creating API client for host: http://localhost:5556, isSecure: false
[INFO][t:27583595]: [libxmtp] Creating message store with path: Some("/var/folders/04/0j7sl4v138zcg6dlvkn8sshr0000gn/T//mLG54pT2VEnHPEWdE4615AEO.db3") and encryption key: false
[INFO][t:27583595]: [libxmtp] Setting up DB connection pool
[INFO][t:27583595]: [libxmtp] Running DB migrations
[INFO][t:27583595]: [libxmtp] Migrations successful
[INFO][t:27583595]: [libxmtp] Creating XMTP client
[INFO][t:27583595]: [libxmtp] Initializing identity
[INFO][t:27583595]: [libxmtp] Getting inbox_ids for account addresses: ["0x0e7c90d3a9cbb4922cff81c1dd6966b797edaf67"]
[INFO][t:27583595]: [libxmtp] Provided Signer: 19fbca7ec102a34f3daa972f46db75b377ab592add2dafe05eb7a4eec990f355
[INFO][t:27583595]: [libxmtp] Missing Signatures: [Address("0x0e7c90d3a9cbb4922cff81c1dd6966b797edaf67"), Installation([25, 251, 202, 126, 193, 2, 163, 79, 61, 170, 151, 47, 70, 219, 117, 179, 119, 171, 89, 42, 221, 45, 175, 224, 94, 183, 164, 238, 201, 144, 243, 85])]
[INFO][t:27583595]: [libxmtp] Created XMTP client for inbox_id: 8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f
[INFO][t:27583595]: [libxmtp] Provided Signer: 0x0e7c90d3a9cbb4922cff81c1dd6966b797edaf67
[INFO][t:27583595]: [libxmtp] Missing Signatures: [Address("0x0e7c90d3a9cbb4922cff81c1dd6966b797edaf67")]
[INFO][t:27583595]: [libxmtp] registering identity
[INFO][t:27583595]: [libxmtp] Creating API client for host: http://localhost:5556, isSecure: false
[INFO][t:27583595]: [libxmtp] Creating message store with path: Some("/var/folders/04/0j7sl4v138zcg6dlvkn8sshr0000gn/T//d9KD7qKHLJqIZkDPLRjRjUFj.db3") and encryption key: false
[INFO][t:27583595]: [libxmtp] Setting up DB connection pool
[INFO][t:27583595]: [libxmtp] Running DB migrations
[INFO][t:27583595]: [libxmtp] Migrations successful
[INFO][t:27583595]: [libxmtp] Creating XMTP client
[INFO][t:27583595]: [libxmtp] Initializing identity
[INFO][t:27583595]: [libxmtp] Getting inbox_ids for account addresses: ["0x1db52df4649ed8714317735ce847099d3f220b09"]
[INFO][t:27583595]: [libxmtp] Provided Signer: 3f2321f3d2cc4a0db4e4cf91dbd1442424aaf27a5fe1efe265bd3ff6158c4bd5
[INFO][t:27583595]: [libxmtp] Missing Signatures: [Address("0x1db52df4649ed8714317735ce847099d3f220b09"), Installation([63, 35, 33, 243, 210, 204, 74, 13, 180, 228, 207, 145, 219, 209, 68, 36, 36, 170, 242, 122, 95, 225, 239, 226, 101, 189, 63, 246, 21, 140, 75, 213])]
[INFO][t:27583595]: [libxmtp] Created XMTP client for inbox_id: 9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f
[INFO][t:27583595]: [libxmtp] Provided Signer: 0x1db52df4649ed8714317735ce847099d3f220b09
[INFO][t:27583595]: [libxmtp] Missing Signatures: [Address("0x1db52df4649ed8714317735ce847099d3f220b09")]
[INFO][t:27583595]: [libxmtp] registering identity
[INFO][t:27583595]: [libxmtp] creating group with account addresses: 0x1db52df4649ed8714317735ce847099d3f220b09
[INFO][t:27583595]: [libxmtp] creating group
[INFO][t:27583595]: [libxmtp] Getting inbox_ids for account addresses: ["0x1db52df4649ed8714317735ce847099d3f220b09"]
[INFO][t:27583595]: [libxmtp] old group membership: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 0}
[INFO][t:27583595]: [libxmtp] updated group membership: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 111, "9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f": 112}
[INFO][t:27583595]: [libxmtp] Getting installation diff. Old: GroupMembership { members: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 0} }. New GroupMembership { members: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 111, "9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f": 112} }
[INFO][t:27583595]: [libxmtp] [8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f] published intent [1] of type [UpdateGroupMembership]
[INFO][t:27583595]: [libxmtp] client [8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f] is  about to process own envelope [195]
[INFO][t:27583595]: [libxmtp] envelope [195] is equal to intent [1]
[INFO][t:27583595]: [libxmtp] [8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f] Validating commit for intent 1. Message timestamp:<PHONE_NUMBER>121174000
[INFO][t:27583595]: [libxmtp] Group context extensions proposal found: GroupMembership { members: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 111, "9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f": 112} }
[INFO][t:27583595]: [libxmtp] Getting installation diff. Old: GroupMembership { members: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 0} }. New GroupMembership { members: {"8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f": 111, "9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f": 112} }
[INFO][t:27583595]: [libxmtp] [8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f] merging pending commit for intent 1
[INFO][t:27583595]: [libxmtp] 8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f: Storing a transcript message with 1 members added and 0 members removed and 0 metadata changes
[INFO][t:27583599]: [libxmtp] Received conversation streaming payload
[INFO][t:27583600]: [libxmtp] Received conversation streaming payload
received new conversation
Found existing group by welcome id
[INFO][t:27583599]: [libxmtp] closing stream
[INFO][t:27583600]: [libxmtp] Received message streaming payload
[INFO][t:27583600]: [libxmtp] client [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f]  is about to process streamed envelope: [195]
[INFO][t:27583600]: [libxmtp] current epoch for [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] in process_stream_entry() is Epoch: [1]
[INFO][t:27583600]: [libxmtp] client [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] is about to process external envelope [195]
[ERROR][t:27583600]: [libxmtp] Wrong Epoch: message.epoch() 0 != 1 self.group_context().epoch()
[INFO][t:27583600]: [libxmtp] [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] syncing group
[INFO][t:27583600]: [libxmtp] current epoch for [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] in sync() is Epoch: [1]
[INFO][t:27583600]: [libxmtp] client [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] is about to process external envelope [195]
[ERROR][t:27583600]: [libxmtp] Wrong Epoch: message.epoch() 0 != 1 self.group_context().epoch()
[ERROR][t:27583600]: [libxmtp] Message processing errors: [OpenMlsProcessMessage(ValidationError(WrongEpoch))]
[ERROR][t:27583600]: [libxmtp] receive error ReceiveErrors([OpenMlsProcessMessage(ValidationError(WrongEpoch))])
[INFO][t:27583600]: [libxmtp] Skipped message streaming payload
[INFO][t:27583595]: [libxmtp] [8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f] published intent [2] of type [SendMessage]
[INFO][t:27583597]: [libxmtp] Received message streaming payload
[INFO][t:27583597]: [libxmtp] client [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f]  is about to process streamed envelope: [196]
[INFO][t:27583597]: [libxmtp] current epoch for [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] in process_stream_entry() is Epoch: [1]
[INFO][t:27583597]: [libxmtp] client [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] is about to process external envelope [196]
[INFO][t:27583597]: [libxmtp] [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] extracted sender inbox id: 8f42c8c702f0c726f0369bae399b9a34a5053905207db201f9c5faffde64878f
[INFO][t:27583597]: [libxmtp] [9a7dfc7c6b26d783901dabf8b050b1edd2b56d195b0c198314506889e7000c3f] decoding application message
Got a message
[INFO][t:27583597]: [libxmtp] Received: hello1
[INFO][t:27583597]: [libxmtp] closing stream
[INFO][t:27583600]: [libxmtp] closing stream
[INFO][t:27583600]: [libxmtp] closing stream
test mls::tests::test_stream_groups_gets_callback_when_streaming_messages ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 21 filtered out; finished in 4.18s

     Running unittests src/bin.rs (target/debug/deps/ffi_uniffi_bindgen-4e4e5566cf2375ca)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/test_generated_bindings.rs (target/debug/deps/test_generated_bindings-b5f934fd814ccdb0)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

here is a log of the new test failing:

logs

warning: Patch `openssl-sys v0.9.92 (https://github.com/xmtp/rust-openssl.git?branch=clone-v0.9.92#00dacb93)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.30s
     Running unittests src/lib.rs (target/debug/deps/xmtpv3-7af2e052c28a47b0)

running 1 test
[INFO][t:27582768]: [libxmtp] Logger initialized
[INFO][t:27582768]: [libxmtp] Creating API client for host: http://localhost:5556, isSecure: false
[INFO][t:27582768]: [libxmtp] Creating message store with path: Some("/var/folders/04/0j7sl4v138zcg6dlvkn8sshr0000gn/T//fi1hLQx0yXwnnmf1GYLYqPRy.db3") and encryption key: false
[INFO][t:27582768]: [libxmtp] Setting up DB connection pool
[INFO][t:27582768]: [libxmtp] Running DB migrations
[INFO][t:27582768]: [libxmtp] Migrations successful
[INFO][t:27582768]: [libxmtp] Creating XMTP client
[INFO][t:27582768]: [libxmtp] Initializing identity
[INFO][t:27582768]: [libxmtp] Getting inbox_ids for account addresses: ["0xc2188424ce1ff549bbfbbc7a738c80ea510b8ef9"]
[INFO][t:27582768]: [libxmtp] Provided Signer: 99015e75245c58b6af13ff8ec0affa579a2c0ce102bfd5ac41d33250416eaf67
[INFO][t:27582768]: [libxmtp] Missing Signatures: [Installation([153, 1, 94, 117, 36, 92, 88, 182, 175, 19, 255, 142, 192, 175, 250, 87, 154, 44, 12, 225, 2, 191, 213, 172, 65, 211, 50, 80, 65, 110, 175, 103]), Address("0xc2188424ce1ff549bbfbbc7a738c80ea510b8ef9")]
[INFO][t:27582768]: [libxmtp] Created XMTP client for inbox_id: 1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7
[INFO][t:27582768]: [libxmtp] Provided Signer: 0xc2188424ce1ff549bbfbbc7a738c80ea510b8ef9
[INFO][t:27582768]: [libxmtp] Missing Signatures: [Address("0xc2188424ce1ff549bbfbbc7a738c80ea510b8ef9")]
[INFO][t:27582768]: [libxmtp] registering identity
[INFO][t:27582768]: [libxmtp] Creating API client for host: http://localhost:5556, isSecure: false
[INFO][t:27582768]: [libxmtp] Creating message store with path: Some("/var/folders/04/0j7sl4v138zcg6dlvkn8sshr0000gn/T//JFydYyWT7yQFAKTPZ23ED2sK.db3") and encryption key: false
[INFO][t:27582768]: [libxmtp] Setting up DB connection pool
[INFO][t:27582768]: [libxmtp] Running DB migrations
[INFO][t:27582768]: [libxmtp] Migrations successful
[INFO][t:27582768]: [libxmtp] Creating XMTP client
[INFO][t:27582768]: [libxmtp] Initializing identity
[INFO][t:27582768]: [libxmtp] Getting inbox_ids for account addresses: ["0x6759216b9fa1e46be86d689882da8f2f0995e63b"]
[INFO][t:27582768]: [libxmtp] Provided Signer:<PHONE_NUMBER>672fd26562756b3ec52104f885accd6a2048b6cc2f7d0ed2ce6149
[INFO][t:27582768]: [libxmtp] Missing Signatures: [Address("0x6759216b9fa1e46be86d689882da8f2f0995e63b"), Installation([33, 49, 37, 72, 81, 103, 47, 210, 101, 98, 117, 107, 62, 197, 33, 4, 248, 133, 172, 205, 106, 32, 72, 182, 204, 47, 125, 14, 210, 206, 97, 73])]
[INFO][t:27582768]: [libxmtp] Created XMTP client for inbox_id: 53bc6da398536a12c7ac4266d38283a9c29a4745addc6c37aee8ace78dc60302
[INFO][t:27582768]: [libxmtp] Provided Signer: 0x6759216b9fa1e46be86d689882da8f2f0995e63b
[INFO][t:27582768]: [libxmtp] Missing Signatures: [Address("0x6759216b9fa1e46be86d689882da8f2f0995e63b")]
[INFO][t:27582768]: [libxmtp] registering identity
[INFO][t:27582768]: [libxmtp] creating group with account addresses: 0x6759216b9fa1e46be86d689882da8f2f0995e63b
[INFO][t:27582768]: [libxmtp] creating group
[INFO][t:27582768]: [libxmtp] Getting inbox_ids for account addresses: ["0x6759216b9fa1e46be86d689882da8f2f0995e63b"]
[INFO][t:27582768]: [libxmtp] old group membership: {"1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 0}
[INFO][t:27582768]: [libxmtp] updated group membership: {"1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 107, "53bc6da398536a12c7ac4266d38283a9c29a4745addc6c37aee8ace78dc60302": 108}
[INFO][t:27582768]: [libxmtp] Getting installation diff. Old: GroupMembership { members: {"1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 0} }. New GroupMembership { members: {"1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 107, "53bc6da398536a12c7ac4266d38283a9c29a4745addc6c37aee8ace78dc60302": 108} }
[INFO][t:27582768]: [libxmtp] [1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7] published intent [1] of type [UpdateGroupMembership]
[INFO][t:27582768]: [libxmtp] client [1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7] is  about to process own envelope [191]
[INFO][t:27582768]: [libxmtp] envelope [191] is equal to intent [1]
[INFO][t:27582768]: [libxmtp] [1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7] Validating commit for intent 1. Message timestamp:<PHONE_NUMBER>206206000
[INFO][t:27582768]: [libxmtp] Group context extensions proposal found: GroupMembership { members: {"53bc6da398536a12c7ac4266d38283a9c29a4745addc6c37aee8ace78dc60302": 108, "1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 107} }
[INFO][t:27582768]: [libxmtp] Getting installation diff. Old: GroupMembership { members: {"1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 0} }. New GroupMembership { members: {"53bc6da398536a12c7ac4266d38283a9c29a4745addc6c37aee8ace78dc60302": 108, "1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7": 107} }
[INFO][t:27582768]: [libxmtp] [1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7] merging pending commit for intent 1
[INFO][t:27582768]: [libxmtp] 1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7: Storing a transcript message with 1 members added and 0 members removed and 0 metadata changes
[INFO][t:27582770]: [libxmtp] Received conversation streaming payload
[INFO][t:27582769]: [libxmtp] Received conversation streaming payload
[ERROR][t:27582771]: [libxmtp] Error processing stream entry: Generic("client: storage error: Diesel result error: database is locked")
received new conversation
[INFO][t:27582768]: [libxmtp] [1aa733c956e5d2a565f3ba2a0a12e6933f875161a70e4575af08da5b5ea7c4c7] published intent [2] of type [SendMessage]
thread 'mls::tests::test_stream_groups_gets_callback_when_streaming_messages' panicked at src/mls.rs:1833:9:
assertion `left == right` failed
  left: 0
 right: 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[INFO][t:27582771]: [libxmtp] closing stream
[INFO][t:27582769]: [libxmtp] closing stream
[INFO][t:27582770]: [libxmtp] closing stream
test mls::tests::test_stream_groups_gets_callback_when_streaming_messages ... FAILED

failures:

failures:
    mls::tests::test_stream_groups_gets_callback_when_streaming_messages

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 21 filtered out; finished in 4.16s

error: test failed, to rerun pass `--lib`

heads up @richardhuaaa, I added [ignore] for the test_can_stream_and_update_name_without_forking_group test in this PR since it seems to show database locked intermittently on PRs as well.


How to use FindInBatches to modify the query result?

Your Question
var results []Ssldomainlist
result := db.Where("status = ?", 0).FindInBatches(&results, 100, func(tx *gorm.DB, batch int) error {
		for i := range results {
                       // batch processing found records
			results[i].Status = 1
		}
		tx.Save(&results)
		fmt.Println(tx.RowsAffected) // number of records in this batch

		fmt.Println(batch) // Batch 1, 2, 3
		if tx.Error != nil {

			fmt.Println(tx.Error)
		}
		// returns error will stop future batches
		return nil
	})
	fmt.Println(result.Error)        // returned error
	fmt.Println(result.RowsAffected) // processed records count in all batches

I have 10000 records want to set status as 1 after some process.
I am curious why these parameter  tx.RowsAffected as 200 and the result.RowsAffected as 5000.
I think tx.RowsAffected and result.RowsAffected should be 100 and 10000.
If i run these code again, the console will show the result.RowsAffected as 2500.
This means that only half of all changes will take effect each time

The document you expected this should be explained
Hope the Sample of FindInBatches can be more complete.
The following sample program is FindInBatches in the Advanced Query page
// batch size 100
result := db.Where("processed = ?", false).FindInBatches(&results, 100, func(tx *gorm.DB, batch int) error {
  for _, result := range results {
    // batch processing found records
  }

  tx.Save(&results)

  tx.RowsAffected // number of records in this batch

  batch // Batch 1, 2, 3

  // returns error will stop future batches
  return nil
})

result.Error // returned error
result.RowsAffected // processed records count in all batches

--

https://gorm.io/docs/advanced_query.html#FindInBatches

Expected answer
Can anyone point out the bugs in my code?

Please submit a playground PR that could reproduce your issue, thank you.

The PR for this issue

Hi @NetLabFox
Fixed, thank you for your report.


Improve character escaping

This method of escaping host IPs is pretty scary:
    // sanitize + remove single quotes
    $host = str_replace('\'', '', filter_var($host, FILTER_SANITIZE_URL));
    // execute command
    $process = proc_open("{$cmd} '{$host}'", $spec, $pipes, null);

Per the PHP docs:
FILTER_SANITIZE_URL -> Remove all characters except letters, digits and $-_.+!*'(),{}|\^~[]`<>#%";/?:@&=.
The majority of those characters have no business being in a domain name/IP address.  I'd suggest checking the host against FILTER_VALIDATE_IP instead... if that doesn't validate, pass it to gethostbyname and use whatever that returns.
Using escapeshellarg instead of just '{$host}' would also be a good move.
I don't see an obvious exploit for this, but that code really stands out to me as suspicious.

If you look at LookingGlass::validate() and it's subsidiary calls, you'll notice it's not that scary. The input is always validated against an IP or URL. The part that you've quoted (FILTER_SANITIZE_URL) is there as a backup.
Using escapeshellarg() isn't necessary, as I've already ensured single quotes are removed (in the section you've quoted). Instead of escaping single quotes via escapeshellarg(), I'm simply removing them.


[Map's button] Spelling mistake in the "Показати на карті" button

Environment: macOS Big Sur (11.1), Google Chrome, 88.0.4324.182
Reproducible: always
Build found: last commit from https://github.com/ita-social-projects/TeachUA/projects/3
Steps to reproduce

Go to 'http://<IP_ADDRESS>:8080/service'
Take a look at words of "Показати на карті" button.

Actual result
Translation mistake in the "Показати на карті" button according to the design.

Expected result
According to the design should be "Показати на мапі".

Additional information
The same defect in Mozilla Firefox (ver. 85).
User story and test case links
E.g.: "User story #116
Labels to be added
"Bug", Priority ("low"), Severity ("text"), Type ("UI").

Fixed.


Inspirations is not compatible with Cosmetic Armor Reworked

Describe the bug
Whenever I have the Inspirations Mod installed, the Cosmetic Armor Reworked Mod's armor icon on the inventory stops working properly and it doesn't allow me to access the "show/hide armor menu." I have tried testing this with disabling certain modules but it doesn't seem to fix the problem. It requires further testing in my opinion.
Versions (please complete the following information):

Minecraft: 1.15.2
Mantle: 1.15.2-1.5.15
Inspirations 1.15.2-1.1.0
Cosmetic Armor Reworked: CosmeticArmorReworked-1.15.2-v3a.jar

Does this happen with just the two mods? And can you post your latest.log so I can check for errors?

Does this happen with just the two mods? And can you post your latest.log so I can check for errors?

The game doesn't crash so I can't show you the latest.log :/
What I reported on the Issue is that the mods don't seem to be compatible.

The latest.log is created whether there is a crash or not. Crash reports only exist when there is a crash. Try looking in your logs folder.

The latest.log is created whether there is a crash or not. Crash reports only exist when there is a crash. Try looking in your logs folder.
And you did not answer my question. Have you tested with only the two mods, and no other mods installed? I know they seem to be incompatible, but there may be another mod causing problems

latest.log
Answer to your question: I went back into my world (the one where I discovered the issue) and the menu from the Cosmetic Armor Reworked seems to be working again. I'm going to try this with only both mods installed to see if it's them or if its another mod that caused the issue.

Your log has a lot of exceptions caused by Optifine. While I have no experience with the 1.15 version, the 1.12 version was known to break much of rendering including UI. Try testing without that as well

Works well for me .

Closing for lack of response, and other users say it work. Personally blaming Optifine as it always breaks even my simplest custom renderers.


[ISSUE# 2325] Use CopyOnWriteArrayList to avoid possible thread safety issues.

What is the purpose of the change
Use CopyOnWriteArrayList to avoid thread safety issues.
Brief changelog
replace Lists.newArrayList() with new CopyOnWriteArrayList<>()
Verifying this change
done
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Coverage decreased (-0.01%) to 51.287% when pulling a19eb6000534c8a3817e88582d2e7b1630d2da3b on medusar:master into 9f95a972e10e0681bc3f2d00e9957aa212e897b5 on apache:master.

@medusar Sorry, I want to help you resolve code conflict when merging to the develop branch, but it can't pass CI now because I forgot to delete a useless import. Could you resubmit a  new PR to develop branch? I will continue to help you track the new PR. I will close this pr firstly.

@RongtongJin I have created a new PR here.


SW-4617 Implement the new query params filter hook in the InventoryTable

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

main

PR #1965

PR #1966

PR #1967

PR #1968

PR #1969   👈

This stack of pull requests is managed by Graphite.

Merge activity

Jan 3, 3:04 PM: @nickgraz started a stack merge that includes this pull request via Graphite.


new API

I think the current API with push and pull and pump and flows is way too complicated. I'd like something different and something simple. Maybe starting from flow abstraction level was a mistake and I should start from high level.
I thought of something like this:
fn = df(fn1).await(fn2)
fn()

which means
fn = function (){
    args = a(arguments)
    cb = function (){
        fn1.apply(this, arguments);
    }
    args.unshift(cb)
    fn2.apply(this, args)
}
fn()

this is similar to async
fn = async.series(fn2, fn1)
fn()

I need to reinforce, that this is not the proper way of treating async code and using async components, ports, etc what I really need.

Actually this is just an API, it does not show the underlying structure, so it could work with the current implementation as well. I worked with the current code 4 months ago, I barely remember why I chose this implementation. :S

What is the main problem by async programming?

         }, y)
     }, x)
})

How to flat this structure?
cb1 = function (x){
    b(cb2, x)
}
cb2 = function (y){
    c(cb3, y)
}
cb3 = function (z){
    ....
}
a(cb1)

What's the problem with this?
It is hard to follow which step comes after the other. By sync code we would do something like this:
x = a()
y = b(x)
z = c(y)
...

By ES7 async functions we can do more or less the same.
How to do something similar by ES5?
df(a)
    .pipe(b)
    .pipe(c)
    .pipe(...)
    .call()

Where pipe automatically generates the callbacks which call the functions and pass the arguments.
This is what the async lib does, and I am sure I don't want another async lib, since it is already implemented.
So let's talk about what we can add to this, which worth creating another lib.

What we have here is a call graph https://en.wikipedia.org/wiki/Call_graph .
a -(x)-> b -(y)-> c -(z)->

we can describe that graph with nodes and links
A = new node(a)
B = new node(b)
C = new node(c)
AB = new link(A, B)
BC = new link(B, C)
...
A.call()

Okay this is not something we want to read in the code, but we can make a graph building API which is simpler:
node(a).link(b).link(c).call()
// node - creating a new node
// link - creating a new node, and linking it from the last added node
// call - calls the first added node and so the whole graph

which is almost the same as the async lib.
Using nodes have the advantage that we can always remove a node or add a new one. This means the existing call graph is totally reusable and can be changed on the fly.
We can add a new node to the graph:
a -(x)-> b -(y)-> c -(z)->
=>
a -(x)-> b -(y)-> q(z) -> c -(p)->

cb1 = function (x){
    b(cb2, x)
}
cb2 = function (y){
    c(cb3, y)
}
cb3 = function (z){
    ....
}
a(cb1)

cbx = cb3
cb3 = function (){
    d(cbx)
}
a(cb1)

or
remove(BC)
add(Q)
add(BQ)
add(QC)

Note that the async lib does not support this feature
This way we can write self modifying code and probably we can modify the running code externally by adding new nodes and links to it. It is possible but that is not an advantage we can exploit in an average code.

Okay, let's talk about another aspect: pull/push.
Pull and push are interesting terms. Pull is the analogy of return/callback and push is the analogy of call. By push we send variables to a component and by pull we receive variables from a component. By push we decide when to send the variables, since we have them. By pull the sender decides when to send us the variables, but we can trigger the variable generating process by sending a pull notification.
push:
a(x) // (a)sync

pull:
y = b() // sync
b(function (y){}) // async

We can describe the call graph:
a -(x)-> b -(y)-> c -(z)->

-push-> a -push(x)-> b -push(y)-> c -push(z)->

But we can describe the call graph in a different way too:
<-pull(z)- c <-pull(y)- b <-pull(x)- a

Why is the important?
This is because async pull is supported by async functions only. So in order to fully support pull we need to add a callback to every function. By node.js this is not a problem, by client side js this is not that usual.
Why is pull better than push?
It is really simple, that's why we use callbacks. By push the function needs to know where to send the results (strong coupling), by pull the functions does not need to know where to send the results (loose coupling). Is this really that loose?
Let's check the code:
cb1 = function (x){
    b(cb2, x)
}
cb2 = function (y){
    c(cb3, y)
}
cb3 = function (z){
    ....
}
a(cb1)

a does not know cb1
cb1 knows about b and cb2
b does not know cb2
cb2 knows about c and cb3
c does not know cb3
So the callback definitions contain the strong coupled part. Now that part must go to a higher abstraction level, since that describes the call graph.

Is there a way to combine the pull/push logic with the node/link logic? How does the node/link logic work?
a = node(function (done, x){
    done(y)
})
b = node(function(done, y){
    done()
})
ab = link(a, b) // a->b
a.call(null, x)
// a(x) -> y, b(y) -> -

node = function (fn){
    links = []
    wrapper = function (){
        done = function (){
            links.forEach(function (link){
                link.target.apply(null, arguments)
            })
        }
        args = array(arguments)
        args.unshift(done)
        fn.apply(this, arguments)
    }
    wrapper.links = links
    return wrapper
}

link = function (source, target){
    link = {source: source, target: target}
    source.links.push(link)
    return link
}

This is the implementation of the push part (a pushes to b).
How to implement the pull part?

What is the pull about?
The pull means that y is a requirement of b and to call b you need to pull in that requirement from a. Let's say that a don't have a param, or b knows the param if it has. That would mean:
function a(x){
    return y
}
function b(){
    y = a(x)
}

but this is not entirely true. Something like this would be nice if we want to pull in arguments:
function a(x){
    return y
}
function b(y = a(x)){
}

This is possible if we don't call b before getting the results of a. How would this look like with the node logic?
a = node(function (done, x){
    done(y)
})
b = node(function (done, y){
    done()
})
link(a, b)
b()

Now before calling b the wrapper calls a to get the y argument. The only problem, that x is unknown in this system, so a cannot run. This can be solved by:
link(a, b, [x])

so if a is pulled, then x is given by the link.
So
df(a).link(b, [x])

can be a good solution, if we are not sure whether b will be called directly, or through a. By direct call b will pull in args from a otherwise a will push args to b.
This is an interesting system, I'll develop this further. I am curious whether it is possible to connect something like this to a stream and pull chunks from the stream or let the stream push them. I'll investigate this later with a small example app.

It is hard to get multiple wrappers from a single statement.
df(a).link(b)

This should return a graph or selection instead of a single node. A selection would be better I think. So df(a) means we created a new selection and a is a member. Then .link(b) means that we added a new node and a new link to the selection. The first item of the selection will be a and the last node will be b. By linking further always the last node will be the source. What if we link multiple nodes e.g. .link(b,c,d). Hmm then all of them will be source by the next link. Hmm by linking we should be able to prev, next, first, last. If the interface would support these, then we would be able to build a complex graph in a single line.

How would I use this in the practice?
var db = df.node(function (done){
    MongoClient.connect("mongodb://localhost:27017/exampleDb", done)
})
var inputName = df.node(function (done){
    done(null, "inf3rno");
})
var findUsersWithName = df.node(function (done, db, name){
    var users = db.collection('users');
    users.find({name: name}).toArray(done);
})

df.link([db, inputName], findUsersWithName)
findUsersWithName(function (err, usersFound){
    console.log(usersFound)
})

This is just a very basic example, but is appears to be more convenient to pull database and input data then trying to inject it somehow. This is similar to require, but that does not work by async requirements afaik.

Is it possible to do the same with objects?
var UserRepository = df.Component.extend({
    prototype: {
        init: function (){
            this.link([this.db, this.inputName], this.findUsersWithName)
        },
        db: function (done){
            MongoClient.connect("mongodb://localhost:27017/exampleDb", done)
        },
        inputName: function (done){
            done(null, "inf3rno");
        },
        findUsersWithName: function (done, db, inputName){
            var users = db.collection('users');
            users.find({name: inputName}).toArray(done);
        }
    }
});

var userRepository = new UserRepository();
userRepository .findUsersWithName(function (err, usersFound){
    console.log(usersFound)
});

It is possible, but this way it is too implicit. I need to find out a more explicit solution for objects.

var UserRepository = df.Component.extend({
    prototype: {
        init: function (){
            this.linkToArgument(this.db, this.findUsersWithName, {arguments: [{index: 0, type: Object}]})
            this.linkToArgument(this.inputName, this.findUsersWithName, {arguments: [{index: 1, type: String}]})
        },
        db: function (){
            MongoClient.connect("mongodb://localhost:27017/exampleDb", this.newCallback())
        },
        inputName: function (){
            this.return("inf3rno");
        },
        findUsersWithName: function (db, inputName){
            var users = db.collection('users');
            users.find({name: inputName}).toArray(this.newCallback());
        }
    }
});

var userRepository = new UserRepository();
userRepository.findUsersWithName(function (err, usersFound){
    console.log(usersFound)
});

This is a little bit more explicit.

readerStream = fs.createReadStream('input.txt');
readerStream.setEncoding('UTF8');
rs = df.node(readerStream)
d = df.node(function (chunk){
    console.log(chunk)
})
b = df.node(function (){
    console.log("begin of stream")
})
e = df.node(function (){
    console.log("end of stream")
})
df.link(rs.data, d)
df.link(rs.begin, b)
df.link(rs.end, e)
d()

Interesting approach to pull streams, but what if we pull the end event and where to put the chunk reading loop?

f = new df.FileStream("input.txt")
fn = new df.AsyncFunction(function (done, chunk){
	console.log(chunk)
	done()
})
fn.pulls(f.data)
fn.notifies(fn)
fn.notify()

This can be an interesting approach for looping without writing the loop into the callback. The only problem that there is an if-else statement which is not in this structure. We need to add a verification.
fn.notifies(fn, function (done){
    done(null, !f.atEnd())
})
fn.notifies(end, function (done){
    done(null, f.atEnd())
})

A little bit complicated for an if-else statement, but it is good for start. Is it possible to make notifications easier? E.g. add a sync if-else statement and a single notification?
fn.notifies(function (done){
    done(null, f.atEnd()?end:fn)
})

so this returns the node/async function instead of a boolean
what if I add event handlers?
fn.on("after:return", function (notify){
    if (f.atEnd())
        end()
    else
        fn()
})

I love this better, but it might be harder to follow, and there is no auto complete for strings. And ofc. it is almost the same as using the default stream with event handlers...
How would this look like with an AsyncClass?
f = new df.FileStream("input.txt")
fn = new df.AsyncFunction(function (done, chunk){
    console.log(chunk)
    done()
})
end = new df.AsyncFunction(function (done){
    done()
})
fn.pulls(f.data)
fn.notifies(function (done){
    done(null, f.atEnd()?end:fn)
})
fn.notify()

How to add all of these to a class?

AsyncFilePrinter= df.AsyncClass.extend({
    prototype: {
        init: function (f){
            this.f = f
            this.async(this.print)
            this.print.pulls(this.f.data)
            this.print.notifies(this.print, negate(f.atEnd))
        },
        print: function (done, chunk){
            console.log(chunk)
            done()
        }
    }
})
p = new AsyncFilePrinter(new df.FileStream("input.txt"))
p.print()

Interesting approach, I am not sure I want to see code like this. It is not that obvious what happens in the background. Maybe after I got used to it.

We can simplyfy it
AsyncFilePrinter= df.AsyncClass.extend({
    prototype: {
        init: function (f){
            this.f = f
            this.pipe(this.f.data, this.print)
            this.pipe(this.print, this.next)
        },
        print: function (done, chunk){
            console.log(chunk)
            done()
        },
        next: function (done){
                if (this.f.atEnd())
                    this.pull(this.print);
                done()
            }
        }
    }
})
p = new AsyncFilePrinter(new df.FileStream("input.txt"))
p.print()

We could make it even better by making it fully compatible with the stream api instead of using a stream wrapper (ofc. we will use one, but in the background).
        init: function (f){
            this.f = f
            this.flow([this.f, "data"])
                .pipe(this.print)
                .pipe(this.next)
        },

f = new AsyncFilePrinter(fs.createReadStream("input.txt"))

So how should this piping work?
flow1.pipe(flow2)

If flow1 can push and flow2 is pushable, then it listens to the flow1 push events and pushes data to flow2.
If flow2 can pull and flow1 is pullable, then waits for the pulls of the flow2 and notifies flow1 about the pulls, so it can send back data.
Note that pull -> notify -> return(data) is the proper way to handle pulls instead of pull -> notify -> push(data). Just as in the current api, push can be paired with wait/read and pull can be paired with sustain/write. I'll use this api, so I need to work on the details now. It must be fully stream compatible without loosing simplicity.

            this.flow([this.f, "data"])
                .pipe(this.print)
                .pipe(this.next)

this means something like
            df.flow([this.f, "data"])
                .pipe(df.flow([this, "print"]))
                .pipe(df.flow([this, "next"]))

This is still simple, especially if we remove the df.flow calls with sugar syntax. Let's stick with this for a while, we can add sugar syntax later again.
So this is a basic tree or graph building api.
flow.pipe(flow).pipe(flow)

Currently it is just the tree, but with some modifications we can build graphs with a single line of code. But the point is now that how will this connect a stream to a callback function in a pullable way?
By streams there is a readable and a writable property. A readable can be pulled while a writable can be pushed. But I am not sure these are the same things. Readable can mean, that it can be read, but the flow controls when the data is available, so it can push only. That means it is not pullable. Ofc. a file read stream is not that kind of stuff, but there can be such data sources, by which we cannot control when they send the data, we are just waiting for it. We could call it pull, but in reality that is not what happens:
source.create()

target.wait()
source.push(data)
...
source.push(end)

Yes, it is similar to
source.create()

target.pull()
source.sustain(data)
...
source.sustain(end)

The difference is in the triggers.
The first one is triggered by the creation and triggered again after the chunk is received or it can wait a few msecs, and don't care whether the target can process the chunks before sending the next.
The second one is triggered by the pulls, so it is controlled by the target, so the target can decide when it is ready to process the next chunk, but it does not care whether the source is ready to send it.
Hmm by streams these features are combined, so they both care whether the other is ready to send or receive the next chunk, at least in ideal cases. We could build a lib, which works the same way by default and add a load balancing lib, if we want something different, for example spawning more targets when all of the existing targets are busy. This can be a strategy in cases when careless pushing or pulling is present. So in many cases probably, since only streams support this behavior, and a lot of other sources or targets doesn't. Hmm sources or sinks maybe. ;-)
Another interesting thing that the current wording of df is not that bad. I might be able to reuse the pump word. I think a pump should represent the loop, which pull or pushes the data.
 flow1  -pump-> -pipe-> flow2 # push
 flow1 -pipe-> -pump-> flow2 # pull

So it depends on which side of the pipe is the pump whether the upstream flow pushes or the downstream flow pulls, or in other words whether the downstream flow is pushed or the upstream flow is pulled. Hmm interesting, how would this look like in practice? I think it is pretty complicated to add pumps everywhere, but maybe not.
flow.pipe(flow2)
flow.pump() // push
flow2.pump() // pull

this is okay, as long as we have 2 flows, but what if we have more?
flow.pipe(flow2).pipe(flow3)
flow.pump() // push - still ok
flow2.pump() // pulling from flow or pushing to flow3? probably both?

Does flow2.pump mean here that we need to pull data from flow in order to sustain data push to flow3? I am not sure. This can mean that only a single pump can be in a closed system like this, which I am not sure is true. E.g. if we add a buffer, then it can have a pump, which works a different frequency. I think the idea to create a separate object for flow sustaining loops is interesting, but it requires a lot more thinking.

Tomorrow has come. :D (after a week)
So the pump is always the loop which drives the flow. Actually this is interesting, since in a complex system only a single pump can be enough.

A pump is just a little more than a circular graph, so I don't think I really need a dedicated class for it, I mean, an a.pipe(a) could solve the problem.

A subgraph can be created by adding a sink to the upper level graph and adding a source to the lower level graph. Between them a simple echo can be enough. So this is a graph separator, which can have a dedicated class.

A source should be both pushing and pullable and a sink should be both pulling and pushable with a single line of code. For example a simple for loop can be something like this:
var s = df.node(function (done, i, l){
    if (i<l)
        done(i+1, l)
})
s.pipe(s)
s.pipe(x) // x is some node which can receive messages

This is okay, because done can call both push or sustain for pull dependent on the situation. The only thing we need here is something which starts the loop. E.g. s.push(0, 10) or x.pull(cb, 0, 10). Can a single pull result in more messages? Can a single push result in more messages. Since the source is piped to itself, it can, but I am not sure whether this should be the default behavior. Another thing, that I need a queue for the pipe, because otherwise the source would get all the indexes from the loop first and only after that would the pipe send them to the receiver. I need some alternation by messages sending to itself and to the receiver, which can be done only by introducing a queue or piping asynchronously, but the latter one is costly.

Hmm I need to add these sentences to a feature description, so it will be easier to develop the exact api later.

Stream api:
source:
  readable - there is data on the stream we can read
  data - chunk is available
  end - no more data available, buffer is empty
  close - related resource is closed
  pipe(sink)/unpipe(sink)
  read(size)
  pause/resume()
sink:
  pipe/unpipe - after a source was un/piped to the sink
  drain - emitted after the write buffer was full, so could not be written, but it became empty again
  finish - when the writing is ended
  close - related resource is closed
  write(chunk)
  end(chunk)
  cork()/uncork()

So it uses a different method to end the stream. Should we do the same or should we raise an exception at the end?

A flow is different from a stream in at least 2 things:

a single flow can connect to many other flows via pipes
a flow is reusable, so either it should be stateless or the begin/end, open/close should not be states of it

What can be common?

I think the drain and write() -> false are okay, so somehow we should be able to indicate that the downstream queue is full and we shall wait before pushing the next message.
The read/write wording is okay, and I might be able to separate them from the push/pull, but I am not sure yet. By streams the write pushes and the read pulls, which is interesting. These features are completely separated by zmq and you normally don't mix them, or probably you cannot mix them.

Let's stick with the loop example.
xxx(function (i, l){
    if (i<l)
        this.resolve(i+1, l)
    else
        this.pause()
})

Hmm this is interesting. Pause is not the word I am looking for. It should be something that notifies the node that the message is handled and there is not output. I am not sure whether it is necessary to notify. It would serve only debug purposes if we must call a callback at the end of every message handling. The problem that message handling does not always mean that we are sending an output message to other nodes.

So another feature that message handling does not always result in an output regardless of having downstream flows.

I think I should add another constraint:

The origin of the message should be unknown, so we cannot filter by upstream flow. This is a need because otherwise we cannot decouple flows. If we need to respond to the upstream flows, then we can give id-s to the messages, and filter the responses by id. The id can be an argument, or we can send DTOs with a message id.

Using pause is in interesting approach. To do that we need to buffer the push and pull events too, so by pause the system will be able to save them and by resume we won't need to resend them.

var s = df.node(function (done, i, l){
    if (i<l)
        done(null, i+1, l)
})
s.pushes(s)
s.inject(0, 10)

The s.pushes(s) part is interesting here, it is implicit, but it means that after every single write s pushes the message to s.
What is really good here, that push or pull is not part of the actual code. What is not so good that we need an implicit rule here: auto push by every output message. We can override this behavior by adding a threshold function.
s.pushes(s, {
    threshold: function (done, queue){
        done(null,  queue.length >= 1)
    }
})`

a.pushes(b).when(function (done, a, b){
    done(null, b.isPushable())
}).otherwise(function (done, a, b){
    a.pause()
})

This might work, but I have to simplify it, and it is not clear when the threshold checking code should run. Normally it runs only when a produces output, but in order to resume it must run when b becomes pushable.

I thought about pumps again. Usually when we are talking about drain, pause, resume, the main problem is load balancing. I think load balancing is a high level problem and we must not mix low level processing code, like message transformation, printing, file reading, etc. with it. It should be on the same level where we connect nodes and describes which node pushes, which pulls, etc. It is interesting that if we manage to merge the common parts of the on.push + pull.cb and on.pull + push.generator code, then in theory we will be able to extract the pumping code, which contains the loops which fuels the message sending. This way it will be much easier to load balance the code, since we could monitor whether the pipes are sustained and slow down, pause the pumps, or add new pipes accordingly. This looks like great in theory, but I am not certain whether it is always possible to make a flow both pushing and pullable or both pulling and pushable. I will start the work on the flows with both traits and maybe later I try to work with exceptional cases, maybe I find a solution for them as well.

Let's start with a constantly pushing and a constantly pulling node.
function a(){
    var i = 0;
    while(true){
        this.write(i);
        i++;
        this.push();
    }
}

function b(){
    while(true){
        this.pull(function (){
            var i = this.read();
            console.log(i);
        }.bind(this))
    }
}

This is the most basic example. Let's make it recursive so it will be async compatible.
var i = 0;
function a(){
    this.write(i);
    i++;
    this.push(a.bind(this));
}

function b(){
    this.pull(function (){
        var i = this.read();
        console.log(i);
        b.call(this);
    }.bind(this))
}

Another variation:
var a = new Flow({
    init: function (){
        this.i = 0;
        this.on("pushReceived", this.cb.bind(this))
    },
    cb: function (){
        this.write(this.i);
        this.i++;
        this.push();
    }
});

var b = new Flow({
    init: function (){
        this.on("pullSustained", this.cb.bind(this));
    },
    cb: function (){
        var i = this.read();
        console.log(i);
        this.pull();
    }
})

We can make them both pushing and pullable and both pulling and pushable this way.
var a = new Flow({
    init: function (){
        this.i = 0;
        this.on("pushReceived", this.cb.bind(this))
        this.on("pulled", this.cb.bind(this));
    },
    cb: function (done){
        this.write(this.i);
        this.i++;
        done(); // done is push by default, but it is sustain by answering pulls
    }
});

var b = new Flow({
    init: function (){
        this.on("pullSustained", this.cb.bind(this));
        this.on("pushed", this.cb.bind(this));
    },
    cb: function (done){
        var i = this.read();
        console.log(i);
        done(); // done is pull by default, but it is receive by receiving pushes
    }
})

So the event handler should get a callback, which calls the next step in the loop.

What happens by an average stream, e.g. by file copy?
a = fs.createReadStream(...);
b = fs.createWriteStream(...);
a.pipe(b)

This means the following in flow terms:
a = new Flow(...);
b = new Flow(...);
a.pipe(b)
a.pushes()
b.pulls() // only if waiting for drain? so this pull restarts the push behavior by empty buffer

Okay, I have a working code with loops:
var a = new Flow({
    i: 0,
    l: 10,
    data: function (done) {
        if (this.i < this.l) {
            this.write(this.i);
            ++this.i;
        }
        done();
    }
});

var b = new Flow({
    data: function (done) {
        var i = this.read();
        console.log(i);
        done();
    }
});

a.pipe(b);
a.pushes();
a.pushDelivered();

The pipe() connects the two flow, the pushes() tells the system that it should create a push loop and the pushDelivered() starts the whole loop. This is almost the same code as above. The Flow class has a medium complexity:
var o3 = require("o3"),
    Queue = require("../util/Queue"),
    Set = require("../util/Set"),
    Pipe = require("./Pipe");

var Flow = o3.Class.extend({
    prototype: {
        init: function (config) {
            this.merge(config);
            this.pipes = new Set();
            this.input = new Queue();
            this.output = new Queue();
        },
        write: function (value) {
            this.output.enqueue(value);
        },
        read: function () {
            return this.input.dequeue();
        },
        pipe: function (destination) {
            var pipe = new Pipe(this, destination);
            this.pipes.add(pipe);
            destination.pipes.add(pipe);
        },
        unpipe: function (flow) {
            var pipe = this.pipes.find(function (pointer) {
                return pointer.destination === flow;
            });
            this.pipes.remove(pipe);
            pipe.destination.pipes.remove(pipe);
        },
        push: function () {
            while (this.output.size()) {
                var message = this.output.dequeue();
                this.pipes.each(function (pipe) {
                    if (pipe.source !== this)
                        return;
                    var destination = pipe.destination;
                    destination.input.enqueue(message);
                }, this);
            }
            this.pipes.each(function (pipe) {
                if (pipe.source !== this)
                    return;
                var destination = pipe.destination;
                destination.pushed(this);
            }, this);
        },
        pushed: function (source) {
            this.data(function () {
                source.pushDelivered();
            });
        },
        pushDelivered: function () {
        },
        pull: function () {
            this.pipes.each(function (pipe) {
                if (pipe.destination !== this)
                    return;
                var source = pipe.source;
                source.pulled(this);
            }, this);
        },
        pulled: function (destination) {
            this.data(function () {
                destination.pullSustained();
            }.bind(this))
        },
        pullSustained: function () {
        },
        data: function (done) {
            done();
        },
        pushes: function () {
            this.pushDelivered = function () {
                this.data(function () {
                    if (this.output.size())
                        this.push();
                }.bind(this));
            }
        },
        pulls: function () {
            this.pullSustained = function () {
                if (this.input.size())
                    this.data(function () {
                        this.pull();
                    }.bind(this))
            };
        }
    }
});

module.exports = Flow;

It would become simpler if I'd use a collection lib I guess. Anyways it is almost a god object compared to the pipe, which is very small:
var o3 = require("o3");

var Pipe = o3.Class.extend({
    prototype: {
        init: function (source, destination) {
            this.source = source;
            this.destination = destination;
        },
        destroy: function () {
            this.source.unpipe(this.destination);
        }
    }
});

module.exports = Pipe;

We should move some part of the code into the pipes, especially the message moving from source.output to destination.input. But that's okay, this is just a proof of concept code. It works.

Let's check the api again:
var a = new Flow({
    i: 0,
    l: 10,
    data: function (done) {
        if (this.i < this.l) {
            this.write(this.i);
            ++this.i;
        }
        done();
    }
});

var b = new Flow({
    data: function (done) {
        var i = this.read();
        console.log(i);
        done();
    }
});

a.pipe(b);
a.pushes();
a.pushDelivered();

The flow creation is straightforward. If the done call is not needed, then I can replace it with return and keep the code sync, but I think it will be needed for example by streams. Only the examples will decide that. I can use factory, for example
var a = df.flow({
    i: 0,
    l: 10,
    data: function (done) {
        if (this.i < this.l) {
            this.write(this.i);
            ++this.i;
        }
        done();
    }
});

and maybe custom factories as well, e.g.
df.shortcut("custom", function (fn){
    return new Flow({data: fn});
});

var b = df.custom(function (done) {
    var i = this.read();
    console.log(i);
    done();
});

Piping is pumping is far from complete.
a.pipe(b);
a.pushes();
a.pushDelivered();

I think it should be much simpler than this. I think the following would be okay:
a.pipe(b);
a.pump();

So the piping makes the connection and the pump makes and starts the loop which delivers the messages. This can be okay by straight lines, for example:
a.pipe(b).pipe(c)
b.pump()
// b.pulls(a)
// b.pushes(c)

But by branches we have some dark spots here, for example:
a.pipe(b)
a.pipe(c)
c.pump()

In this case c pulls the messages from a, but b doesn't. So should b get messages from a, or not?
I don't know the answer. If things like this do not make sense, then it should throw an error. What about
a.pipe(b)
a.pipe(c)
c.pump()
b.pump()

In this case both can get messages from a. If we throw error by the first example, then we should throw error by this as well, since b is not pumping when c is. So need to put the two pumps into a transaction. That's relative easy to do, but it makes the code complicated and hard to follow. I need examples, since now this is just a theoretical problem, and I am not sure whether such systems really exist. If so then the default behavior should support those systems, otherwise the code should throw error.

Let's check the timer example:
Here is the important part with the old (who knows which version) dataflower:
var timerAdapter = new CustomComponent(function () {
	var timer = new Timer({
		events: {
			started: function () {
				this.ports.started.relay(new Message());
			}.bind(this),
			ticked: function (elapsedTime, length) {
				this.ports.ticked.relay(new Message([elapsedTime, length]));
			}.bind(this),
			cleared: function () {
				this.ports.cleared.relay(new Message());
			}.bind(this),
			ended: function () {
				this.ports.ended.relay(new Message());
			}.bind(this)
		}
	});
	this.ports = {
		changeLength: new InputPort({
			callback: function (message) {
				timer.changeLength(message.data);
			},
			component: this
		}),
		start: new InputPort({
			callback: timer.start,
			context: timer,
			component: this
		}),
		started: new OutputPort({
			component: this
		}),
		stop: new InputPort({
			callback: timer.clear,
			context: timer,
			component: this
		}),
		ticked: new OutputPort({
			component: this
		}),
		cleared: new OutputPort({
			component: this
		}),
		ended: new OutputPort({
			component: this
		})
	};
	this.clear = timer.clear.bind(timer);
});

builder.connectAll(
		timerAdapter.ports.changeLength,
		lengthSelect.ports.selected
);
builder.connectAll(
		timerAdapter.ports.start,
		intervalCheckbox.ports.next,
		startButton.ports.clicked
);
builder.connectAll(
		timerAdapter.ports.started,
		startButton.ports.disable,
		stopButton.ports.enable
);
builder.connectAll(
		timerAdapter.ports.stop,
		stopButton.ports.clicked
);
builder.connectAll(
		timerAdapter.ports.ended,
		beepAudio.ports.replay,
		intervalCheckbox.ports.ended
);
builder.connectAll(
		timerAdapter.ports.cleared,
		startButton.ports.enable,
		stopButton.ports.disable,
		display.ports.clear,
		title.ports.clear
);
builder.connectAll(
		timerAdapter.ports.ticked,
		time2Text.ports.stdin
);
builder.connectAll(
		time2Text.ports.stdout,
		display.ports.text,
		title.ports.text
);

So we have a timer object with events, we transform that to a node and we connect that node to view related nodes. How would this look like with the new api?
The event handlers are message sources, so we should transform them into nodes:
df(lengthSelect, "selected").pipe(timer.changeLength);
df(startButton, "clicked").and(intervalCheckbox, "next").pipe(timer.start);
df(timer, "started").pipe(startButton.disable).and(stopButton.enable);
df(stopButton, "clicked").pipe(timer.stop);
df(timer, "ended").and(intervalCheckbox, "ended").pipe(beepAudio.replay);
df(timer, "cleared").pipe(startButton.enable).and(stopButton.disable).and(display.clear).and(title.clear);
df(timer, "ticked").pipe(time2Text).pipe(display.text).and(title.text);

So we don't really have to write adapters manually, it is relative easy to decide when to create a source, node or sink adapter from the arguments. Ofc. we can make this complicated:
var ticked = new Source(timer, "ticked");
var time2TextTransformation = new Node(time2Text);
var displayInTextBox = new Sink(display.text);
var displayInTitle = new Sink(title.text);

ticked.pipe(time2TextTransformation);
time2TextTransformation.pipe(displayInTextBox);
time2TextTransformation.pipe(displayInTitle);

vs
df(timer, "ticked").pipe(time2Text).pipe(display.text).and(title.text);

What about the pump?
Actually in this example we don't need pumps everything reacts on the generated events. In other terms the user pumps the messages manually by hitting a button or selecting a duration. After that the timer's internal async loop serves as pump. Interesting, but in most of the cases we have this kind of pumps.
What about the graph?
It is possible to transform the current code into a graph, but I am not sure whether I really need that feature now. Ofc. later the sub-graph will be needed to hide implementation details and add an interface which can be used by the outside nodes. How to define such an interface?
E.g. the timer could be a sub-graph:
var timerGraph = new Graph({
    changeLength: new Flow(timer.changeLength),
    start: new Flow(timer.start),
    started: new Flow(timer, "started"),
    stop: new Flow(timer.stop),
    ended: new Flow(timer, "ended"),
    cleared: new Flow(timer, "cleared"),
    ticked: new Flow(timer, "ticked")
});

var viewGraph = new Graph({
    lengthChanged: new Flow(lengthSelect, "selected"),
    start: new Flow(startButton, "clicked"),
    stop: new Flow(stopButton, "clicked"),
    startable: new Flow(startButton.enable, stopButton.disable),
    stoppable: new Flow(startButton.disable, stopButton.enable),
    intervalEnded: new Flow(intervalCheckbox, "ended", beepAudio.replay),
    nextInterval: new Flow(intervalCheckbox.next),
    displayNothing: new Flow(display.clear, title.clear),
    displayTime: new Flow(time2Text).pipe(new Flow(display.text, title.text))
});

viewGraph.lengthChanged.pipe(timerGraph.changeLength);
viewGraph.start.pipe(timerGraph.nextInterval, timerGraph.start);
timerGraph.started.pipe(viewGraph.stoppable);
viewGraph.stop.pipe(timerGraph.stop);
timerGraph.stopped.pipe(viewGraph.startable);
timerGraph.ended.pipe(viewGraph.intervalEnded);
timerGraph.cleared.pipe(viewGraph.displayNothing);
timerGraph.ticked.pipe(viewGraph.displayTime);

The connections are interesting here, but it would be great to add them to another graph. So if that graph is deleted, then the two subgraphs will be disconnected. Another possible solution to inject one graph into the other.

Hmm interesting idea, that these connections can act like a graph. I don't think so unless I put the ending points into the graph. So those points will be part of 2 graphs.
Another possible method to make this better is thinking about what we are doing here. We have this many lines long description of how these nodes connect to each other. This is information and to make this simpler we might need to compress this information somehow. So less lines might mean easier understanding. Ofc. this is not always true, but we can try it.
How to make this less?
var timerGraph = new Graph({
    changeLength: new Flow(timer.changeLength),
    start: new Flow(timer.start),
    started: new Flow(timer, "started"),
    stop: new Flow(timer.stop),
    ended: new Flow(timer, "ended"),
    cleared: new Flow(timer, "cleared"),
    ticked: new Flow(timer, "ticked")
});

var viewGraph = new Graph({
    lengthChanged: new Flow(lengthSelect, "selected"),
    start: new Flow(startButton, "clicked"),
    stop: new Flow(stopButton, "clicked"),
    startable: new Flow(startButton.enable, stopButton.disable),
    stoppable: new Flow(startButton.disable, stopButton.enable),
    intervalEnded: new Flow(intervalCheckbox, "ended", beepAudio.replay),
    nextInterval: new Flow(intervalCheckbox.next),
    displayNothing: new Flow(display.clear, title.clear),
    displayTime: new Flow(time2Text).pipe(new Flow(display.text, title.text))
});

viewGraph.lengthChanged.pipe(timerGraph.changeLength);
viewGraph.start.pipe(timerGraph.nextInterval, timerGraph.start);
timerGraph.started.pipe(viewGraph.stoppable);
viewGraph.stop.pipe(timerGraph.stop);
timerGraph.stopped.pipe(viewGraph.startable);
timerGraph.ended.pipe(viewGraph.intervalEnded);
timerGraph.cleared.pipe(viewGraph.displayNothing);
timerGraph.ticked.pipe(viewGraph.displayTime);

There are a lot of repeating things here: timerGraph and viewGraph many times.
var connections = new ConnectionSet({
    @context: [viewGraph, timerGraph],
    connections: [
        lengthChanged: "changeLength",
        nextInterval: "start",
        started: "stoppable",
        stop: "stop", //problem
        stopped: "startable",
        ended: "intervalEnded",
        cleared: "displayNothing",
        ticked: "displayTime",
    ]
})

If the node names are easy to understand without the graph name and they are unique, then this might be a solution. It has at least one drawback, we need to use strings instead of node references. Anyways I would call this the jsonld approach. I am pretty sure I won't use this except if we are talking about REST services or something like that, which have "external" interfaces. Currently we are talking about classes and objects, which have "internal" interfaces. Internal means here that we are not talking about inter-class/object communication in the same process and not about RPC, or something like that.

Something is interesting here, we can describe processes from the continuous parts of these connections. For example

length changing process: timer.changeLength <-> view.lengthChanged, after that we can start thinking about the modifications: timer.changeLength -> timer.lengthChanged -> view.changeLength -> view.lengthChanged + user.interaction -> view.changeLength -> view.lengthChanged -> timer.changeLength -> timer.lengthChanged. It is much more complex this way, but this is the proper description of this data binding.

starting process: view.nextIntervalAlert <- view.start -> view.pending -> timer.start -> timer.started -> view.started -> view.stoppable

etc...
Each one of them describes a feature or at least a scenario. This is interesting, so these routes describe scenarios. So we have a flow control graph, which describes a feature, and we have routes on that graph, which describe a scenario. Each route is directed, so it is relative easy to collect the steps of a scenario if we have the starting point. At least in these examples, maybe different ones are different. Hmm this means that for example we can write something like v.start/v.pending/t.start/t.started/v.started/v.stoppable if we want to describe a route. This means we can use routes to describe connections.
graph.starting = [
    new Route(view.start, view.nextIntervalAlert),
    new Route(view.start, [view, "pending"], timer.start, [timer, "started"], [view, "started"], [view, "stoppable"])
]

We can combine the 2 approaches:
We can define the timer.start, [timer, "started"] subRoute as a node in the graph, since the function results in the event always and the function has input, while the event has output. Interesting approach and does not always apply, but in this case it does.
Another thing that view.start, [view, "pending"], [view, "started"], [view, "stoppable"] can be a single node as view.start, [view, "started"] and the timer starting node is inside this node, so it is some kind of subGraph or subNode? Anyways it would mean that viewStarting(viewPending, timerStarting) -> viewStoppable. This is interesting, so if we use the flattened model, it is less compact and makes less sense than this hierarchical model. Since we are talking about graph construction, it does not really matter which approach we use, but it affects how easily we can find a certain node or a certain connection.

Another possible solution to copy the method names from model to view. Since they always have to be in sync this is special case.
timer.start / started <-> view.start / started

What if we focus on data processing rather than data delivery, wouldn't it be better? This is a bold statement if we are talking about data delivery, but still, we are trying to design an api here, and easy data processing is more important by apis than data delivery. Imho.
What if we have a class with various methods. We somehow get the argument list for each of the methods. And call the methods only when the arguments are available. If the argument is pullable, then it can be pulled, but if not, then the method waits for a push, or throws exception. This is an interesting approach, and it is inpired by the o3 interface draft.

var x = new df.Async({
    a: function (x, y){
        var z = x + y;
        return z;
    },
    b: function (z){
        console.log(z);
    }
});

The only problem here, we don't know that a returns z. So we need some kind of pipe which collects the output of a and ofc. we need some kind of wrapper which waits for x and y when a is called. What if we don't call a? What if a is called automatically when the inputs are present?
var q = new df.Async({
    a: function (x, y){
        var z = x + y;
        this.output({z: z});
    },
    b: function (z){
        console.log(z);
    }
});

q.push({x: 1});
q.push({y: 2});

Okay, what if we have an async output then we send some kind of future or promise?
var q = new df.Async({
    a: function (x, y){
        var p = new df.Promise();
        setTimeout(function (){
            var z = x + y;
            p.fulfill({z: z});
        }, 1);
        return p;
    },
    b: function (z){
        console.log(z);
    }
});

q.push({x: 1}); // nothing happens
q.push({y: 2}); // a(x, y) is called
// wait 1 ms
// promise is fulfilled with z given
// b(z) is called
// console.log 3

I like this, at least this small example. But we need to translate somehow the values and names between nodes, because sooner or later every name will be taken, and we'd need to start namespacing param names which would be burdensome.
Let's try with 2 async nodes.

var x = new df.Async({
    next: function (i){
        var p = new df.Promise();
        setTimeout(function (){
            p.fulfill({j: i+1});
        }, 1);
        return p;
    }
});
var y = new df.Async({
    next: function (j){
        var p = new df.Promise();
        setTimeout(function (){
            p.fulfill({i: j+1});
        }, 1);
        return p;
    }
});
x.push({i: 0});

This is very basic, it would work, but this is not a good way to design nodes. Right now there is a namespace collision. So if we would have a class, for example:
var X = df.Async.extend({
	prototype: {
		next: function (i){
			var p = new df.Promise();
			setTimeout(function (){
				p.fulfill({i: i+1});
			}, 1);
			return p;
		}
	}
});

var x1 = new X();
var x2 = new X();

x1.push({i: 0});

What would we expect?
I would expect that x1 loops itself with i and x2 is never triggered. Anyway this is chaotic, we need to somehow pipe these instances and put some constraints on naming.
var x1 = new X();
var x2 = new X();
x1.pipe(x2);
x2.pipe(x1);
x1.inject({i: 0});

This can be okay, but still the keys in the two objects share the same namespace. So if we would have an internal method in the X class, which is called by the next(), then it would be called in both instances, because the pipes. We need to specify which values are piped.
var x1 = new X();
var x2 = new X();
x1.pipe({i: x2}); // x2 looks for i in x1's ouput (but not in the input)
x2.pipe({i: x1}); // x1 looks for i in x2's output (but no in the input)
x1.inject({i: 0}); // i is added to the input of x1

This is fine, we can define this way which async params are shared and we can communicate via message names. So we don't need a 2 dimensional node/port addressing, which is burdensome if you are programming. Still you need to keep in mind the parameter list, but that's okay. You can do something like
var X = df.Async.extend({
	prototype: {
		expose: {next: true},
		next: function (i){
			var p = new df.Promise();
			setTimeout(function (){
				p.fulfill({i: i+1});
			}, 1);
			return p;
		}
	}
});

so you can hide private non-async methods. Another way to add method prefix, which is acceptable. Probably adding an async prefix to the async methods is fine.
var X = df.Async.extend({
	prototype: {
		asyncNext: function (i){
			var p = new df.Promise();
			setTimeout(function (){
				p.fulfill({i: i+1});
			}, 1);
			return p;
		}
	}
});

So for example a self loop would be
var x = new X();
x.pipe({i: x});
x.inject({i: 0});

I think this code should be in a container, factory or module, since it is about initialization of a graph. So we build the graph structure and inject the first message. The i should be added to the params the graph supports. So whenever something pulls in i on the graph, then it will know that i can be created by the x node, since it is piped. I really like this, but ofc. almost every api I designed so far was nice at the beginning and failed by complex graphs, so I need to test this out.

"async" is a too long word for a prefix. We need to use something shorter which can't be mistyped.
var X = df.Async.extend({
	prototype: {
		asyncNext: function (i){
			var p = new df.Promise();
			setTimeout(function (){
				p.fulfill({i: i+1});
			}, 1);
			return p;
		}
	}
});

asyncNext - too long
asyNext - nope, sound strange
asNext - maybe, but then sNext is the sync?
aNext - funny, "a" and "an" are commonly used, so this is not either
dfNext - nope
iNext - interfaces, nope
yNext - nope
$ - for jquery
_ for private methods and underscore
:S

I'll have an abstract class without this feature implemented and add async as default. So people will be able to extend the abstract base class if they want a custom prefix or a different async method selector. Hmm probably injecting the selector implementation with a factory would be okay as well.

This api more or less solves that async output problem, but I think it is still a problem that I cannot pull properly from the function blocks. I don't want to auto pull by empty input. I checked and ES6 yield is supported by most of the recent browsers, so adding something like:
		asyncNext: function *(i){
			i = yield df.pull(i);
			...
		}

would solve the non sync input problem. so we could call the functions with the future parameter and a yield would wait out while it gets a value from somewhere. this is fine for example by pulls.
waiting out parallel calls is more problematic, since yield has only one output
		asyncNext: function *(i){
			args = yield df.pull(i, j, k);
			...
		}

Using yield might solve the whole ES5 related callback issue as well.
push: f(a,b,c)
pull/await: yield f(a,b,c)
so transactions/message identifiers/stacks are maintained by js and not by the developer, which makes things easier. if we want to mix the results of multiple calls, then we can always add a higher level buffer/queue/array, which collects them.
this might work and it is easy to do and supported by most of the current environments, so I think I'll no longer care about the ES5 compatibility and do it the ES6 way.

Ok. ES6 classes are supported by most of the browsers...

The only way to use async code with yield is yielding something callable, for example a nodejs style function or a promise. So a scheduler can call these and resume the generator when it gets a result. So the generator itself is not async, but the caller of the generator can work in an asnyc way, which is cool. Not a perfect solution for async functions, but much better then breaking up the tasks into many callbacks. Still we have callbacks here by the async part, but it is much better because with generator we can flatten the async code. I think that's all we needed...

ff, chrome, opera, safari  and node supports async functions, while edge doesn't
http://kangax.github.io/compat-table/es2016plus/#test-async_functions

Async functions:
async function asf(a){
    var b = new Promise(function (resolve, reject) {
        setTimeout(function(){
            resolve(2);
        }, 100);
    });
    var c = await b;
    return c;
}

async function run(){
    var c = await asf(1);
    console.log(c);
}

run();

vs generators:
function run(gen, cb){
    var param;
    var callNext = function (param){
        var step = gen.next(param);
        if (step.done)
            cb(step.value);
        else if (step.value instanceof Function)
            step.value.call(null, function (err, value){
                callNext(value);
            });
        else
            callNext(step.value);
    };
    callNext(param);
}

function* gen(a){
    var b = yield function (cb){
        setTimeout(function (){
            cb(null, a);
        }, 100);
    };
    var c = yield b;
    return c;
}

run(gen(1), console.log);

The short version:
async function asf(a){
    var b = await new Promise(function (resolve, reject) {
        setTimeout(function(){
            resolve(2);
        }, 100);
    });
    return b + 1;
}

asf(1).then(console.log);

The interesting thing here, that b is a promise, and b + 1 is converted to a promise too.  Without the await keyword this would not be the case... So there is a huge difference between generators and async functions.

The only problem with generators and async functions, that they are producing promises, which are non-reusable just like streams. I wanted to avoid that by passing only messages instead of futures, but maybe this is a better way, since a future does not need to contain the message id, so we can do the message synchronization with the js engine instead of writing our own code for that...

In #121 I laid down the basis of the new API. It solves the subgraph problem once and for all. This current thread should describe the graph and node definitions, the graph builders and invent some serializable addreses and typed messages. After that I can start to work on the implementation.


Display section data in list

Is there an existing issue for this?

[X] Yes, I have searched the existing issues and it doesn't exist

Feature Description
I have built my resume and it displays all the sections' data in a grid of two columns, I want it to show the data in the list format. I have explored the options but there is no option that would enable the list format.

There's a button to set the number of columns for each section right below the list, as seen in the screenshot here. Setting the number of columns to 1 would display every item as a list, as you intended.


"Țețchea, Hotar, Subpiatră, Telechiu" are villages in "Țețchea" commune, "BIHOR" county

Description
When an user search a location, this appear in the drop-down menu
according to the law
Precondition
The website ( https://peviitor-ro.github.io/ui.orase/ ) is up an running.
Empty the search bar after every step

Steps 1
Write "Țețchea"
Expected result
In the drop down menu you find "Comuna Țețchea, BIHOR
Sat Țețchea, BIHOR(Țețchea)"
Test Data
Țețchea

Steps 2
Write "Hotar"
Expected result
In the drop down menu you find "Sat Hotar, BIHOR(Țețchea)"
Test Data
Hotar

Steps 3
Write "Subpiatră"
Expected result
In the drop down menu you find "Sat Subpiatră, BIHOR(Țețchea)"
Test Data
Subpiatră

Steps 4
Write "Telechiu"
Expected result
In the drop down menu you find "Sat Telechiu, BIHOR(Țețchea)"
Test Data
Telechiu


There is a problem with the installation

I am a rookie. I have the following installation problems when using Windows system

Hi @Adong-1209 ,
From a quick search [link], your problem is related to Microsoft Visual C++. Apart from following the solutions from the issue that I sent you, you can also either:

install a Linux distro and try installing the project there.
or
use the CPU version of the code without installing the CUDA implementation.

Hi @Adong-1209 ,
From a quick search [link], your problem is related to Microsoft Visual C++. Apart from following the solutions from the issue that I sent you, you can also either:

install a Linux distro and try installing the project there.
or
use the CPU version of the code without installing the CUDA implementation.

I tried most of the methods in that link, but I still couldn't solve the problem. What a pity. Even so, thank you for your help.

Hi @Adong-1209 ,
From a quick search [link], your problem is related to Microsoft Visual C++. Apart from following the solutions from the issue that I sent you, you can also either:

install a Linux distro and try installing the project there.
or
use the CPU version of the code without installing the CUDA implementation.

I tried most of the methods in that link, but I still couldn't solve the problem. What a pity. Even so, thank you for your help.

I tried for the last time, and this time it worked. I don't know what happened, maybe because I created a new environment or a project interpreter.

The CUDA implementation is essentially an extension (so it needs to be compiled to run alongside PyTorch). By using the python PyTorch-native code, you can run it in both your CPU and GPU(s) with the only trade-off that it will be slower than the CUDA.
So the installation is only if you want to use the CUDA implementation. You can still use the repo without installing it.
Hope the above helped!

The CUDA implementation is essentially an extension (so it needs to be compiled to run alongside PyTorch). By using the python PyTorch-native code, you can run it in both your CPU and GPU(s) with the only trade-off that it will be slower than the CUDA.
So the installation is only if you want to use the CUDA implementation. You can still use the repo without installing it.
Hope the above helped!

Yes, I do need to use CUDA

The CUDA implementation is essentially an extension (so it needs to be compiled to run alongside PyTorch). By using the python PyTorch-native code, you can run it in both your CPU and GPU(s) with the only trade-off that it will be slower than the CUDA.
So the installation is only if you want to use the CUDA implementation. You can still use the repo without installing it.
Hope the above helped!

Yes, I do need to use CUDA

You can then either: 1) use the PyTorch-native code of the repo without installing it OR  2) try to resolve the compiler error.
Will be closing this issue for now since the simplest solution is to not install the repo and use the pytorch-native implementation. Feel free to open a new issue if other problems occur.
Best,
Alex


Cropped Image issue

Hello,
I'm trying to use the library with a Samsung Galaxy A5 2017 and the cropped image not work correctly (even though the rectangle is recognized correctly):

When I use with a Samsung Galaxy s9 Plus it wok correctly:

Do you have an idea what could cause that ?
Sebastian

@sebacura Thanks for trying out the package. As I'm sure you've tried other packages for solving this issue, most of the packages for image detection and cropping are pretty buggy. I actually rebuilt this one from the ground up while most are just forks based off of a gist from some random android dev (seriously). So while I solved plenty of the bugs there are still some minor nuances that show up and this is one of them.
Android has an interesting api for interacting with the camera. There's a preview screen size and a captured image screen size that you need to set up. These can be different ratios and different resolutions. This package has code to handle transforming the detected area coordinates to map to the captured image size and ratio but every so often I've found it to be a little off in Android and honestly I am not sure why. What I think it is is something to do with the actual hardware of the device and how it is reporting the camera output to the code. This is why it is different from device to device (and why developing for iPhones is so much easier because everything is consistent).
So probably not the answer you are looking for but that is the honest answer.
One thing I tell everyone who has issues with this package is to try to only use the package at 100% screen size (no headers or footers) because sometimes that will skew the images or at least when I tested this on a variety of phones, that is what was the most consistent at working. (I use this package in production!) I did notice that usually issues were isolated to older android phones.
If you have any android dev experience I'd love for you to take a crack at this. I believe I structured the code fairly readable.

you should change some styles:   .    I also met up with this problem.


pull request for e2e test at 02/02/2017 21:26:14

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Data set question about DB15K

Hi, first of all thank you very much for sharing your work. I experimented on FB15K and YAGO15K and achieved the same effect as the paper. But on DB15K, it has achieved extremely poor results. I don’t know if it’s the data set segmentation or something went wrong. Could you please upload your train, valid and test data sets? Thank you very much!
Looking forward to your reply~

HELLO,Could you please ask you some questions about the model? Here is my email<EMAIL_ADDRESS>you！！！！！

Hello, do you have the datasets and pickles the paper used? Here is my email<EMAIL_ADDRESS>

数据集在那个MMKG那个篇论文列你可以去搜一下，代码我准备这两天调试呢。
J1999
@.***

------------------ 原始邮件 ------------------
发件人: @.>;
发送时间: 2023年4月11日(星期二) 下午3:47
收件人: @.>;
抄送: @.>; @.>;
主题: Re: [HestiaSky/IMF-Pytorch] Data set question about DB15K (Issue #2)
Hello, do you have the datasets and pickles the paper used? Here is my email @.***
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

Hello, do you have the datasets and pickles the paper used? Here is my email<EMAIL_ADDRESS>

I also cannot reproduce DB15K results, I only get MRR=44.5<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>I think the results is only close to IMF (w/o DF).

可以加qq交流一下。qq：1131220481
J1999
@.***

------------------ 原始邮件 ------------------
发件人: "Zhiwei @.>;
发送时间: 2023年5月23日(星期二) 晚上9:20
收件人: @.>;
抄送: @.>; @.>;
主题: Re: [HestiaSky/IMF-Pytorch] Data set question about DB15K (Issue #2)
I also cannot reproduce DB15K results, I only get MRR=44.5, @.=38.0, @.=47.5, @.***=57.1, I think the results is only close to IMF (w/o DF).
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>


amplify can't find my appsync after deployment

Describe the bug
I have a website in production with 2 envs, I made a css change on dev branch, commit and push, and since I've the amplify console configured, it deployed correctly my website as usual. However, when I merge to master and push, I've a deployment error it saids:
2019-06-18T19:11:26.680Z [WARNING]: Error updating cloudformation stack
2019-06-18T19:11:26.738Z [INFO]: Following resources failed
2019-06-18T19:11:26.741Z [INFO]: Resource Name: gzqlhcne2vevvegmn7sd47fldaGraphQLSchema (AWS::AppSync::GraphQLSchema)
                                 Event Type: update
                                 Reason: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 2914FFFA231EC77B; S3 Extended Request ID: tTewGguUd4O4+7SUuU8oMQo3pAkKuNdy06i6HQUe2D6KwD/bE1t5zmtLSPU3T+dewUCakw9Z+W4=)
                                 URL: https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/arn%3Aaws%3Acloudformation%3Aus-east-1%3A155161765563%3Astack%2Fportal-20190528145021-apiportal-1J6HJ2ZE3SERG%2Ffffd3bd0-817a-11e9-b938-0aad5d963972/events
                                 init failed
2019-06-18T19:11:26.742Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud
                                    ✖ There was an error initializing your environment.
2019-06-18T19:11:26.744Z [INFO]: { ResourceNotReady: Resource is not in the state stackUpdateComplete
                                 at constructor.setError (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:182:47)
                                 at Request.CHECK_ACCEPTORS (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:44:12)
                                 at Request.callListeners (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
                                 at Request.emit (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
                                 at Request.emit (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:683:14)
                                 at Request.transition (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
                                 at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
                                 at /root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
                                 at Request.<anonymous> (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
                                 at Request.<anonymous> (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:685:12)
                                 message: 'Resource is not in the state stackUpdateComplete',
                                 code: 'ResourceNotReady',
                                 retryable: false,
                                 time: 2019-06-18T19:11:26.679Z,
                                 statusCode: 200,
                                 retryDelay: 30000 }
2019-06-18T19:11:26.759Z [ERROR]: !!! Build failed
2019-06-18T19:11:26.861Z [ERROR]: !!! Non-Zero Exit Code detected
2019-06-18T19:11:26.861Z [INFO]: # Starting environment caching...
2019-06-18T19:11:26.862Z [INFO]: # Uploading environment cache artifact...
2019-06-18T19:11:26.947Z [INFO]: # Environment caching completed

This is happening on my production site, help with this will be be most appreciated!

@damianrr This is a duplicate of your issue out here - #1666
We have a fix for it in this PR - #1668


Help needed, problems with sqlite

I try to use Medoo with sqlite and have some problems.
require_once('vendor/medoo.php');
$db = new Medoo\Medoo([
    'database_type' => 'sqlite',
    'database_file' => ':memory:',
]);

$db->create("users", array(
    'id' => array(
        "INT",
		"NOT NULL",
		"AUTO_INCREMENT",
		"PRIMARY KEY",
    ),
    'name' => array(
        "VARCHAR(30)",
		"NOT NULL",
    )
));

$res = $db->insert('users', array(
    'name' => "Andre",
));

print_r($res);

If I try to need in memory sqlite database with more than one php script maybe database_file need to be :memory::?cache=shared or mode=memory?
How to output tables like show tables?
Why is my output empty? I created a table, inserted a new row and tried to select it?

Medoo 1.7.2
Linux
php-fpm 7.3.6

With debug() / log() I just see the query, but not the error / return value of the executed query?

@catfan
Also tested sqlite db as file, but it is just an empty file, also if I create a table and insert a row.

Checked file permissions and code... DB file is empty.
No error with create table or insert a row, but still empty file.
Tested with plain sqlite works fine.
So looks like a bug
Anyone have the same problems?

It's because the SQLite create table option is different.
There is no varchar type for SQLite. Just using TEXT instead.
Just use INTEGER for INT.
The row with INTEGER PRIMARY KEY will be auto-increased.
For your case, the CREATE statement should be:
$database->create("users", [
	'id' => [
		"INTEGER",
		"PRIMARY KEY",
	],
	'name' => [
		"TEXT",
		"NOT NULL",
	]
])

Read more from: https://www.sqlite.org/lang_createtable.html

Thanks! Should be my problem.
I moved to an orm some days ago, but I'll check it. 👍


Hide Tabs on Event Page without Data

If the event has no result or team data display text saying that no data exists

This was closed in commit 8071e3f


[NEW ICON]: icons from unity Catalog website

Skill Name
Delta ecosystem icons
Why?
I don't know how to create icons, I don't use ilustrator or figma. Please let me know how can I help to make this happen. what do you need?
It will be nice to add the following icons:

Salesforce
Apache Spark
Apache Impala
Clickhouse
Daft
Doris
Presto
Trino
Startree
StarRocks
Ray
PuppyGraph
Pulsar
Presto
Polars
Delta
Pandas
Hive
Glue
Flink
EMR
DuckDB
DLT
Apache DataFusion
Dask
Apache Beam
Airbyte
Apache Arrow
Athena

Reference Image

Svg Availability

[X] I have given the official svg version of the logo in the Reference Image field.

Code of Conduct Acknowledgment

[X] I have read and agree to the Code of Conduct.

SVG version:
Salesforce

Apache Spark

Apache Impala

Clickhouse

Doris

Presto

Trino

StarRocks

Ray

PuppyGraph

Pulsar

Polars

Delta

Pandas

Hive

Glue

Flink

EMR

DuckDB

Dask

Apache Beam

Airbyte

Apache Arrow

Athena

Examples
Salesforce

Apache Impala

Clickhouse

Doris

Presto

Trino

StarRocks

Ray

PuppyGraph

Pulsar

Polars

Glue

EMR

Athena


Handle missing methods

It would be great to have some more explanation of why we need to do this in the commit messages 💭 🎉 ✨


⚠ Bofrost.de has degraded performance

In 41306e4, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1010 ms

Resolved: Bofrost.de performance has improved in a8ebec0 after 9 minutes.


Update Documentation on running multiple ambassadors with ambassador_id

Update Documentation on running multiple ambassadors with ambassador_id
https://www.getambassador.io/reference/running.html
Under the Multiple Ambassadors in One Cluster section the docs should tell the user to include an ambassador_id annotation in their ambassador service.yaml file as well as set the AMBASSADOR_ID env var
Example:
apiVersion: v1
kind: Service
metadata:
  name: ambassador
  namespace: ambassador-1
  labels:
    app: ambassador
  annotations:
    getambassador.io/config: |
      ---
      apiVersion: ambassador/v0
      kind:  Module
      name:  ambassador
      ambassador_id: ambassador-1

Take a look and let me know what you think


Added Types

Initial checklist

[x] I read the support docs
[x] I read the contributing guide
[x] I agree to follow the code of conduct
[x] I searched issues and couldn’t find anything (or linked relevant results below)
[x] If applicable, I’ve added docs and tests

Description of changes

Added Types for TypeScript
Added Source Code Linting with ESLint
Added GitHub Action to lint and test source code
Slightly refactored code for readability
Added function comments for editor quick-previews and quick documentation

Hi @domwebber!
Thank you for the pull request. I've just seen it, I will need to look closer into my notification settings.
I will look closer into the code in following days. In general, I can accept it only partially. Many things are just not needed in the scope of such a small package.

Thank you for your feedback! I appreciate you looking at my code, this was my first proper PR so I appreciate that I may have diverged from the original idea of the repository.
Thank you ☺

@domwebber I've merged your work and published a package. It has version 0.2.1 now. You can check your contribution in the commit history.
If you use the package currently, I would be glad to if you give me feedback how the typing works. I barely tested it on my site.
This is not the best way to work; I am sorry that I close the PR instead of merging it. Thanks a lot for your contribution 👍