very slow "restic-to-s3-over-seaweedfs" initial testing (~10mb/minute ~ 50gb/hour)

Describe the bug
$ ~/go/bin/weed server -dir=/tmp/test-seaweedfs -s3

$ restic --verbose -r s3:http://localhost:8333/rames-example init
enter password for new repository:
enter password again:
created restic repository 939b8cf331 at s3:http://localhost:8333/rames-example

$ restic -r s3:http://localhost:8333/rames-example backup Downloads
enter password for repository:
repository 939b8cf3 opened successfully, password is correct
created new cache in /home/rames/.cache/restic
[4:46] 0.15%  2 files 69.018 MiB, \
   total 48323 files \
   44.744 GiB, 0 errors ETA 52:39:38

...seems "very slow" compared to what I'd expect, but since it's the first time I'm poking at SeaweedFS (and s3/restic), I don't really have a conception of this is a normal-ish speed to expect for local-to-local on SSD's.  As a simple comparision, rsync was running ~150mb/s, whereas seaweed/restic/s3 appears to be ~0.2mb/s (ie: 60mb written in ~5min, ~100x slower??!?).
$ du -sh test-seaweedfs/
56M	test-seaweedfs/

$ time rsync -a --progress  ./Downloads/ /tmp/test-rsync/
sending incremental file list
2019-09-26-raspbian-buster-full.img
  4,582,309,888  67%  170.80MB/s    0:00:12  ^C

System Setup

see above, go install ...@latest
/etc/debian_version:11.8
output of weed version => version 30GB 3.62  linux amd64
model name	: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
MemTotal:       16342216 kB

Expected behavior
...not "unbearably slow" when running on a reasonably performant system with memory + SSD's.  There could be a fair amount of startup costs w/ restic noodling around with ~40gb of files, but it only grew the size of /tmp/test-seaweedfs by ~50mb in ~5minutes, and was predicting ~1hr for the total transfer time, so that's ~1gb/hr which also works out to ~0.2mb/s.
rsync was writing way faster, while seaweed was slow enough (even for restic -r s3:... init) that I thought it was all hung.  Took like a minute or two for it to even return back any sort of feedback and write the "init" directories to disk.
Additional context
I have the linux nature, but am by no means an S3, SeaweedFS, or restic expert. Is this ~100x slowdown "usual"?
Expected behaviour
...I'm presumably doing something "wrong" with seaweedfs, is there some sort of performance issue with $ ~/go/bin/weed server -dir=/tmp/test-seaweedfs -s3?  Is there a better "kick the tires" demo command?  Is there anything to show where the slowness is occuring?  Halp! ...and Thank You! :-)

For comparison:
$ time restic --verbose -r /tmp/restic-rames-example init
enter password for new repository:
enter password again:
created restic repository 48c91a9ae3 at /tmp/restic-rames-example

real	0m5.328s
user	0m2.379s
sys	0m0.033s

$ restic -r /tmp/restic-rames-example backup Downloads
enter password for repository:
repository 48c91a9a opened successfully, password is correct
created new cache in /home/rames/.cache/restic
^Z:08] 3.22%  2 files 1.442 GiB, total 48323 files 44.744 GiB, 0 errors ETA 4:00
[1]+  Stopped                 restic -r /tmp/restic-rames-example backup Downloads

...for comparison, restic init with "just" local drive was ~5s and backup was an ETA of 4minutes instead of ~60 (~10x faster).

This all correlates well with the default settings for one volume server
weed server --help
  -volume.concurrentDownloadLimitMB int
        limit total concurrent download size (default 64)
  -volume.concurrentUploadLimitMB int
        limit total concurrent upload size (default 64)

Wading through:

https://github.com/seaweedfs/seaweedfs/wiki/restic-with-SeaweedFS
https://github.com/seaweedfs/seaweedfs/wiki/s3cmd-with-SeaweedFS
https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#amazon-s3
https://github.com/seaweedfs/seaweedfs/issues/3886#issuecomment-1288192657

Current two clues are:

time restic init -r /tmp/repo --password-command "echo 'test'" => 0.25s
time restic -r s3:http://localhost:8333/resticbucket init --password-command "echo 'test'" => "hangs" (very slow, but might complete after ~5min's or so)
time AWS_ACCESS_KEY_ID=any-key-id AWS_SECRET_ACCESS_KEY=any-access-key restic -r s3:http://localhost:8333/resticbucket init --password-command "echo 'test'" => 2.0s (yay!)

...but it fails with: Fatal: create key in repository at s3:http://localhost:8333/resticbucket failed: client.PutObject: Signed request requires setting up SeaweedFS S3 authentication ... which might be key-mismatch or something where there's another clue:

https://forum.restic.net/t/s3-v2-api-signature/3399/5 => Using the AWS* environment variables always uses auth version 4.

...will eventually try and diagnose more, but it looks like the interminable slowness comes out to be ~7minutes over ~31 files, which is ~13 seconds per file overhead, which very well could be: 31 files * 10s delay == 310 seconds vs. 7minutes * 60 seconds = 420 seconds, which leaves ~110 seconds remaining overhead, which could be another ~10-11 round trips during restic init or restic backup.
...so I'm guessing that it's something derpy going on with timeouts and misconfigured access "stuff".
I'm digging a bit more and will hopefully come up with a resolution (kindof "unlucky operator error"), and try and explain/update the documentation so it's a better "out of the box" experience.

Full repro's here (will run as 3 separate comments in order to provide full scripts along with each run):
$ time ./seaweedfs-with-restic-tmp-raw.sh
>>> This is the scenario of restic writing directly to tmpfs
>>> (as an 'infintely fast baseline')
4.0K	/etc/motd
1
2024-02-19T12:25:26-06:00
Fatal: create repository at /tmp/seaweedfs-restic-tmp-raw failed: config file already exists

real	0m0.025s
user	0m0.017s
sys	0m0.016s
INFO: Sleeping for 5 seconds for contemplation...
2024-02-19T12:25:31-06:00
repository 4bab734a opened successfully, password is correct
found 1 old cache directories in /home/rames/.cache/restic, run `restic cache --cleanup` to remove them

Files:           0 new,     0 changed,     1 unmodified
Dirs:            0 new,     0 changed,     1 unmodified
Added to the repo: 0 B

processed 1 files, 286 B in 0:00
snapshot 41e5db23 saved

real	0m0.792s
user	0m0.592s
sys	0m0.022s
INFO: Done!

real	0m5.835s
user	0m0.614s
sys	0m0.056s

$ cat ./seaweedfs-with-restic-tmp-raw.sh
#!/bin/bash

echo ">>> This is the scenario of restic writing directly to tmpfs "
echo ">>> (as an 'infintely fast baseline')"

TMP_SCENARIO="/tmp/seaweedfs-restic-tmp-raw"
BACKUP_SOURCE="/etc/motd"
RESTIC_TARGET="/tmp/seaweedfs-restic-tmp-raw"
du -sh "$BACKUP_SOURCE" ; \
ls -l "$BACKUP_SOURCE" | wc -l
date --iso-8601=s
time restic -r "$RESTIC_TARGET" init --password-command "echo 'test'"

echo "INFO: Sleeping for 5 seconds for contemplation..."
sleep 5

date --iso-8601=s
time restic -r "$RESTIC_TARGET" backup "$BACKUP_SOURCE" --password-command "echo 'test'"

echo "INFO: Done!"

This one is "with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY" environment variables.  "The usual" lines during startup have been snipped for brevity, feel free to run it locally and get all the gory details.  It takes a minute or so including sleeps, startup, shutdown, etc.  Feels "normal", no complaints w.r.t. the backup speed or anything at all, feels like it's "working the way it should".
A fair portion of this is following the restic + s3 or s3command wiki doc's, but it really really feels like there's some sort of "verybad" interaction w/ restic + seaweedfs whenever there are NO AWS_... variables defined which will come in the next comment.
$ time ./seaweedfs-with-restic-with-s3-aws-key.sh
>>> This is the scenario of restic writing to seaweedfs WITH an S3 AWS key defined
>>> (it appears to run "normally fast", around a minute or so, including server startup/shutdown, etc)
WARNING: If this script is interrupted, it may leave some dangling jobs/processes.  BEWARE!
INFO: Staring 'seaweedfs process'...
2024-02-19T12:28:28-06:00
INFO: Sleeping for 30 seconds to allow time for seaweedfs server to start...
I0219 12:28:28.401347 master.go:269 current: <IP_ADDRESS>:9333 peers:
I0219 12:28:28.401687 file_util.go:27 Folder /tmp/seaweedfs-restic-s3-aws-key Permission: -rwxr-xr-x
I0219 12:28:28.401752 master.go:269 current: <IP_ADDRESS>:9333 peers:<IP_ADDRESS>:9333
...snip....
W0219 12:28:51.438439 auth_credentials.go:128 fail to load config: read S3 config: filer: no entry is found in filer store
I0219 12:28:51.439155 s3api_circuit_breaker.go:35 s3 circuit breaker not configured
I0219 12:28:51.441652 s3.go:354 Start Seaweed S3 API Server 30GB 3.62  at http port 8333
I0219 12:28:51.443283 filer_grpc_server_sub_meta.go:296 +  listener s3@<IP_ADDRESS>:49492 clientId<PHONE_NUMBER> clientEpoch 2
I0219 12:28:51.443305 filer_grpc_server_sub_meta.go:36  s3@<IP_ADDRESS>:49492 starts to subscribe /etc/ from 2024-02-19 12:28:51.440739512 -0600 CST
INFO: configuring AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY
I0219 12:28:58.415774 master_grpc_server.go:349 + client .adminShell@
I0219 12:28:58.415989 masterclient.go:210 master localhost:9333 redirected to leader <IP_ADDRESS>:9333
I0219 12:28:58.416345 master_grpc_server.go:365 - client .adminShell@
I0219 12:28:58.416657 master_grpc_server.go:349 + client .adminShell@
.master: localhost:9333 filers: [<IP_ADDRESS>:8888]
{
  "identities": [
    {
      "name": "iam",
      "credentials": [
        {
          "accessKey": "example-key-28508",
          "secretKey": "example-secret-7365"
        }
      ],
      "actions": [
        "Admin"
      ],
      "account": null
    }
  ],
  "accounts": []
}
I0219 12:28:58.499849 auth_credentials_subscribe.go:63 updated /etc/iam/identity.json
I0219 12:28:58.505710 master_grpc_server.go:365 - client .adminShell@
INFO: Sleeping for 10 seconds to allow time for seaweedfs server to accept configuration...
INFO: Making S3 Bucket...
I0219 12:29:08.638564 auth_credentials_subscribe.go:84 updated bucketMetadata /buckets/name:"resticbucket" is_directory:true attributes:{mtime:1708367348 file_mode:2147484159 uid:1000 gid:1000 crtime:1708367348}
Bucket 's3://resticbucket/' created
INFO: Sleeping for 10 seconds to allow time for s3bucket
INFO: restic init...
4.0K	/etc/motd
1
2024-02-19T12:29:18-06:00
I0219 12:29:21.138945 store.go:166 In dir /tmp/seaweedfs-restic-s3-aws-key adds volume:1 collection:resticbucket replicaPlacement:000 ttl:
I0219 12:29:21.139043 volume_loading.go:142 loading memory index /tmp/seaweedfs-restic-s3-aws-key/resticbucket_1.idx to memory
I0219 12:29:21.142904 store.go:170 add volume 1
I0219 12:29:21.142977 volume_grpc_client_to_master.go:179 volume server <IP_ADDRESS>:8080 adds volume 1
I0219 12:29:21.143090 volume_growth.go:245 Created Volume 1 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:29:21.143120 volume_layout.go:406 Volume 1 becomes writable
I0219 12:29:21.143140 volume_growth.go:258 Registered Volume 1 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:29:21.143312 store.go:166 In dir /tmp/seaweedfs-restic-s3-aws-key adds volume:2 collection:resticbucket replicaPlacement:000 ttl:
I0219 12:29:21.143402 volume_loading.go:142 loading memory index /tmp/seaweedfs-restic-s3-aws-key/resticbucket_2.idx to memory
...snip...
I0219 12:29:21.155106 store.go:166 In dir /tmp/seaweedfs-restic-s3-aws-key adds volume:7 collection:resticbucket replicaPlacement:000 ttl:
I0219 12:29:21.155456 volume_loading.go:142 loading memory index /tmp/seaweedfs-restic-s3-aws-key/resticbucket_7.idx to memory
I0219 12:29:21.157678 store.go:170 add volume 7
I0219 12:29:21.157783 volume_grpc_client_to_master.go:179 volume server <IP_ADDRESS>:8080 adds volume 7
I0219 12:29:21.157946 volume_growth.go:245 Created Volume 7 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:29:21.158011 volume_layout.go:406 Volume 7 becomes writable
I0219 12:29:21.158080 volume_growth.go:258 Registered Volume 7 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
created restic repository 97e9bdf06a at s3:http://localhost:8333/resticbucket

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

real	0m2.698s
user	0m2.463s
sys	0m0.033s
INFO: Sleeping for 10 seconds to allow time for contemplation
I0219 12:29:29.410903 store.go:166 In dir /tmp/seaweedfs-restic-s3-aws-key adds volume:8 collection: replicaPlacement:000 ttl:
I0219 12:29:29.411087 volume_loading.go:142 loading memory index /tmp/seaweedfs-restic-s3-aws-key/8.idx to memory
I0219 12:29:29.413015 store.go:170 add volume 8
I0219 12:29:29.413086 volume_grpc_client_to_master.go:179 volume server <IP_ADDRESS>:8080 adds volume 8
I0219 12:29:29.413132 volume_growth.go:245 Created Volume 8 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:29:29.413146 volume_layout.go:406 Volume 8 becomes writable
I0219 12:29:29.413154 volume_growth.go:258 Registered Volume 8 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:29:29.413178 node.go:79 topo failed to pick 1 from  0 node candidates
I0219 12:29:29.413185 volume_growth.go:99 create 7 volume, created 1: Not enough data nodes found!
INFO: restic backup...
2024-02-19T12:29:31-06:00
repository 97e9bdf0 opened successfully, password is correct
created new cache in /home/rames/.cache/restic
found 1 old cache directories in /home/rames/.cache/restic, run `restic cache --cleanup` to remove them

Files:           1 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repo: 1002 B

processed 1 files, 286 B in 0:00
snapshot 7f2247a1 saved

real	0m0.758s
user	0m0.526s
sys	0m0.037s
INFO: Killing background process (seaweedfs)...
INFO: Sleeping to allow time for shutdown...
I0219 12:29:32.123790 master_grpc_server.go:365 - client .filer@<IP_ADDRESS>:8888
I0219 12:29:32.123786 masterclient.go:225 .master masterClient failed to receive from <IP_ADDRESS>:9333: rpc error: code = Unavailable desc = error reading from server: EOF
W0219 12:29:32.123851 master_grpc_server.go:100 SendHeartbeat.Recv server <IP_ADDRESS>:8080 : rpc error: code = Canceled desc = context canceled
I0219 12:29:32.123875 topology_event_handling.go:87 Removing Volume 1 from the dead volume server <IP_ADDRESS>:8080
volume server has been killed
I0219 12:29:32.123887 volume_layout.go:445 Volume 1 has 0 replica, less than required 1
I0219 12:29:32.123894 volume_layout.go:393 Volume 1 becomes unwritable
I0219 12:29:32.123896 volume_grpc_client_to_master.go:71 heartbeat to <IP_ADDRESS>:9333 error: rpc error: code = Unavailable desc = error reading from server: EOF
I0219 12:29:32.123889 volume_server.go:139 Stopping volume server...

I0219 12:29:32.124041 volume_layout.go:393 Volume 7 becomes unwritable
I0219 12:29:32.124045 topology_event_handling.go:87 Removing Volume 8 from the dead volume server <IP_ADDRESS>:8080
I0219 12:29:32.124052 volume_layout.go:445 Volume 8 has 0 replica, less than required 1
I0219 12:29:32.124057 volume_layout.go:393 Volume 8 becomes unwritable
I0219 12:29:32.124080 node.go:262 topo:DefaultDataCenter:DefaultRack removes <IP_ADDRESS>:8080
I0219 12:29:32.124088 master_grpc_server.go:87 unregister disconnected volume server <IP_ADDRESS>:8080
I0219 12:29:32.124095 master_grpc_server.go:58 remove volume server <IP_ADDRESS>:8080, online volume server: map[]
I0219 12:29:32.149273 volume.go:281 stop send heartbeat and wait 10 seconds until shutdown ...
I0219 12:29:42.149485 volume.go:305 graceful stop cluster http server ...
INFO: Done!

real	1m23.760s
user	0m3.179s
sys	0m0.133s
rames@red-slash:~/Git/research-misc/seaweedfs-repro-scenario$ I0219 12:30:01.907107 volume.go:310 graceful stop gRPC ...
I0219 12:30:01.907662 volume_server.go:149 Shutting down volume server...
I0219 12:30:01.909588 volume_server.go:151 Shut down successfully!

$ cat ./seaweedfs-with-restic-with-s3-aws-key.sh
#!/bin/bash

echo ">>> This is the scenario of restic writing to seaweedfs WITH an S3 AWS key defined"
echo ">>> (it appears to run \"normally fast\", around a minute or so, including server startup/shutdown, etc)"

echo "WARNING: If this script is interrupted, it may leave some dangling jobs/processes.  BEWARE!"

WEED_BINARY="$HOME/go/bin/weed"
TMP_SCENARIO="/tmp/seaweedfs-restic-s3-aws-key"

echo "INFO: Staring 'seaweedfs process'..."
date --iso-8601=s
rm -rf "$TMP_SCENARIO"
mkdir -p "$TMP_SCENARIO"

"$WEED_BINARY" \
  server \
  -dir="$TMP_SCENARIO" \
  -s3 \
  -volume.concurrentDownloadLimitMB 1234 \
  -volume.concurrentUploadLimitMB 1234 \
  &
PID="$!"
echo "INFO: Sleeping for 30 seconds to allow time for seaweedfs server to start..."
sleep 30

AWS_ACCESS_KEY_ID="example-key-$RANDOM"
AWS_SECRET_ACCESS_KEY="example-secret-$RANDOM"
echo "INFO: configuring AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY"
echo "s3.configure -access_key \"$AWS_ACCESS_KEY_ID\" -secret_key \"$AWS_SECRET_ACCESS_KEY\" -user iam -actions Admin -apply" | "$WEED_BINARY" shell
echo "INFO: Sleeping for 10 seconds to allow time for seaweedfs server to accept configuration..."
sleep 10

BACKUP_SOURCE="/etc/motd"
S3CMD_BUCKET="s3://resticbucket"
RESTIC_TARGET="s3:http://localhost:8333/resticbucket"

echo "INFO: Making S3 Bucket..."
s3cmd mb "$S3CMD_BUCKET"\
 --access_key="$AWS_ACCESS_KEY_ID" \
 --secret_key="$AWS_SECRET_ACCESS_KEY"
echo "INFO: Sleeping for 10 seconds to allow time for s3bucket"
sleep 10

echo "INFO: restic init..."
du -sh "$BACKUP_SOURCE" ; \
ls -l "$BACKUP_SOURCE" | wc -l
date --iso-8601=s
time \
  AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" \
  AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" \
  restic -r "$RESTIC_TARGET" \
  init --password-command "echo 'test'"

echo "INFO: Sleeping for 10 seconds to allow time for contemplation"
sleep 10

echo "INFO: restic backup..."
date --iso-8601=s
time \
  AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" \
  AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" \
  restic -r "$RESTIC_TARGET" \
  backup "$BACKUP_SOURCE" \
  --password-command "echo 'test'"

echo "INFO: Killing background process (seaweedfs)..."
kill "$PID"
echo "INFO: Sleeping to allow time for shutdown..."
sleep 20
echo "INFO: Done!"

Many ~bothans~ BogoMIPS died to bring you this message.  Same (near-exact) commands as the one above, however omitting the AWS_... environment variables(!!) brings the overall time from ~1min to ~13min.  I'm snipping a bunch of lines again but keeping attention to "transition time periods" (ie: when logs go from 12:41pm:etc... to 12:43pm:etc..., presumably due to some api call or wakeup or something).
$ time ./seaweedfs-with-restic-no-s3-aws-key.sh
>>> This is the scenario of restic writing (very slowly!) with no AWS keys defined
WARNING: If this script is interrupted, it may leave some dangling jobs/processes.  BEWARE!
INFO: Staring 'seaweedfs process'...
2024-02-19T12:41:00-06:00
INFO: Sleeping for 30 seconds to allow time for seaweedfs server to start...
I0219 12:41:00.889612 master.go:269 current: <IP_ADDRESS>:9333 peers:
I0219 12:41:00.889844 file_util.go:27 Folder /tmp/seaweedfs-restic-no-s3-aws-key Permission: -rwxr-xr-x
I0219 12:41:00.889911 file_util.go:27 Folder /tmp/seaweedfs-restic-no-s3-aws-key Permission: -rwxr-xr-x
I0219 12:41:00.889949 master.go:269 current: <IP_ADDRESS>:9333 peers:<IP_ADDRESS>:9333
I0219 12:41:00.890032 master_server.go:127 Volume Size Limit is 30000 MB
I0219 12:41:00.890172 master.go:150 Start Seaweed Master 30GB 3.62  at <IP_ADDRESS>:9333
...snip...
W0219 12:41:23.926752 auth_credentials.go:128 fail to load config: read S3 config: filer: no entry is found in filer store
I0219 12:41:23.928120 s3api_circuit_breaker.go:35 s3 circuit breaker not configured
I0219 12:41:23.932909 s3.go:354 Start Seaweed S3 API Server 30GB 3.62  at http port 8333
I0219 12:41:23.933657 filer_grpc_server_sub_meta.go:296 +  listener s3@<IP_ADDRESS>:59720 clientId -382047910 clientEpoch 2
I0219 12:41:23.933702 filer_grpc_server_sub_meta.go:36  s3@<IP_ADDRESS>:59720 starts to subscribe /etc/ from 2024-02-19 12:41:23.931056408 -0600 CST

INFO: Making S3 Bucket...
Bucket 's3://resticbucket/' created
I0219 12:41:30.960881 auth_credentials_subscribe.go:84 updated bucketMetadata /buckets/name:"resticbucket" is_directory:true attributes:{mtime:1708368090 file_mode:2147484159 uid:1000 gid:1000 crtime:1708368090}
INFO: Sleeping for 10 seconds to allow time for s3bucket

INFO: restic init...
4.0K	/etc/motd
1
2024-02-19T12:41:40-06:00
I0219 12:42:01.899477 store.go:166 In dir /tmp/seaweedfs-restic-no-s3-aws-key adds volume:1 collection: replicaPlacement:000 ttl:
I0219 12:42:01.899865 volume_loading.go:142 loading memory index /tmp/seaweedfs-restic-no-s3-aws-key/1.idx to memory
I0219 12:42:01.902680 store.go:170 add volume 1
I0219 12:42:01.902846 volume_grpc_client_to_master.go:179 volume server <IP_ADDRESS>:8080 adds volume 1
I0219 12:42:01.903063 volume_growth.go:245 Created Volume 1 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:42:01.903136 volume_layout.go:406 Volume 1 becomes writable
I0219 12:42:01.903179 volume_growth.go:258 Registered Volume 1 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
...snip...
I0219 12:42:01.923278 store.go:170 add volume 7
I0219 12:42:01.923368 volume_grpc_client_to_master.go:179 volume server <IP_ADDRESS>:8080 adds volume 7
I0219 12:42:01.923565 volume_growth.go:245 Created Volume 7 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:42:01.923586 volume_layout.go:406 Volume 7 becomes writable
I0219 12:42:01.923665 volume_growth.go:258 Registered Volume 7 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:44:43.493063 store.go:166 In dir /tmp/seaweedfs-restic-no-s3-aws-key adds volume:8 collection:resticbucket replicaPlacement:000 ttl:
I0219 12:44:43.493400 volume_loading.go:142 loading memory index /tmp/seaweedfs-restic-no-s3-aws-key/resticbucket_8.idx to memory
I0219 12:44:43.497127 store.go:170 add volume 8
I0219 12:44:43.497233 volume_grpc_client_to_master.go:179 volume server <IP_ADDRESS>:8080 adds volume 8
I0219 12:44:43.497419 volume_growth.go:245 Created Volume 8 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:44:43.497493 volume_layout.go:406 Volume 8 becomes writable
I0219 12:44:43.497538 volume_growth.go:258 Registered Volume 8 on topo:DefaultDataCenter:DefaultRack:<IP_ADDRESS>:8080
I0219 12:44:43.497579 node.go:79 topo failed to pick 1 from  0 node candidates
I0219 12:44:43.497608 volume_growth.go:99 create 7 volume, created 1: Not enough data nodes found!
created restic repository eb8ee75d7c at s3:http://localhost:8333/resticbucket

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

real	3m32.729s
user	0m2.491s
sys	0m0.047s
INFO: Sleeping for 10 seconds for contemplation...
INFO: restic backup...
2024-02-19T12:45:23-06:00
repository eb8ee75d opened successfully, password is correct
created new cache in /home/rames/.cache/restic
found 1 old cache directories in /home/rames/.cache/restic, run `restic cache --cleanup` to remove them

Files:           1 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repo: 1002 B

processed 1 files, 286 B in 4:30
snapshot 86993cc8 saved

real	8m30.786s
user	0m0.639s
sys	0m0.292s
INFO: Killing background process (seaweedfs)...
INFO: Sleeping to allow time for shutdown...
I0219 12:53:54.496193 masterclient.go:225 .master masterClient failed to receive from <IP_ADDRESS>:9333: rpc error: code = Unavailable desc = error reading from server: EOF
I0219 12:53:54.496204 master_grpc_server.go:365 - client .master@<IP_ADDRESS>:9333
I0219 12:53:54.496244 volume_grpc_client_to_master.go:71 heartbeat to <IP_ADDRESS>:9333 error: rpc error: code = Unavailable desc = error reading from server: EOF
I0219 12:53:54.496221 master_grpc_server.go:304 => client .filer@<IP_ADDRESS>:8888: cluster_node_update:{node_type:"master" address:"<IP_ADDRESS>:9333"}
...snip...
I0219 12:53:54.496476 master_grpc_server.go:87 unregister disconnected volume server <IP_ADDRESS>:8080
I0219 12:53:54.496483 master_grpc_server.go:58 remove volume server <IP_ADDRESS>:8080, online volume server: map[]
I0219 12:53:54.501152 volume.go:281 stop send heartbeat and wait 10 seconds until shutdown ...
............................I0219 12:54:04.501505 volume.go:305 graceful stop cluster http server ...
I0219 12:54:04.501643 volume.go:310 graceful stop gRPC ...
I0219 12:54:04.502010 volume_server.go:149 Shutting down volume server...

INFO: Done!

real	13m8.648s
user	0m4.512s
sys	0m1.137s

$ cat ./seaweedfs-with-restic-no-s3-aws-key.sh
#!/bin/bash

echo ">>> This is the scenario of restic writing (very slowly!) with no AWS keys defined"
echo "WARNING: If this script is interrupted, it may leave some dangling jobs/processes.  BEWARE!"

WEED_BINARY="$HOME/go/bin/weed"
TMP_SCENARIO="/tmp/seaweedfs-restic-no-s3-aws-key"

echo "INFO: Staring 'seaweedfs process'..."
date --iso-8601=s
rm -rf "$TMP_SCENARIO"
mkdir -p "$TMP_SCENARIO"

"$WEED_BINARY" \
  server \
  -dir="$TMP_SCENARIO" \
  -s3 \
  -volume.concurrentDownloadLimitMB 1234 \
  -volume.concurrentUploadLimitMB 1234 \
  &
PID="$!"

echo "INFO: Sleeping for 30 seconds to allow time for seaweedfs server to start..."
sleep 30

BACKUP_SOURCE="/etc/motd"
S3CMD_BUCKET="s3://resticbucket"
RESTIC_TARGET="s3:http://localhost:8333/resticbucket"

echo "INFO: Making S3 Bucket..."
s3cmd mb "$S3CMD_BUCKET"
echo "INFO: Sleeping for 10 seconds to allow time for s3bucket"
sleep 10

echo "INFO: restic init..."
du -sh "$BACKUP_SOURCE"
ls -l "$BACKUP_SOURCE" | wc -l
date --iso-8601=s
time \
  restic -r "$RESTIC_TARGET" \
  init --password-command "echo 'test'"

echo "INFO: Sleeping for 10 seconds for contemplation..."
sleep 10

echo "INFO: restic backup..."
date --iso-8601=s
time \
  restic -r "$RESTIC_TARGET" \
  backup "$BACKUP_SOURCE" \
  --password-command "echo 'test'"

echo "INFO: Killing background process (seaweedfs)..."
kill "$PID"
echo "INFO: Sleeping to allow time for shutdown..."
sleep 15
echo "INFO: Done!"

...so... in the original bug report I'd written:
Expected behaviour
...I'm presumably doing something "wrong" with seaweedfs, is there some sort of performance issue with $ ~/go/bin/weed server -dir=/tmp/test-seaweedfs -s3? Is there a better "kick the tires" demo command? Is there anything to show where the slowness is occuring? Halp! ...and Thank You! :-)

...I kindof stand by that statement / assumption.
Looking at the wiki: https://github.com/seaweedfs/seaweedfs/wiki/restic-with-SeaweedFS, there is a reference to the AWS_ACCESS_KEY... variables, but it's not clear that they are required (especially in the "no auth, test/demo/trial") case.
There's no mention of the echo "s3.configure ..." | weed shell technique, and some of the bits around s3cmd and the setup around that.
It's also VERY strange that apparently the "very slow" backup (no AWS_...) appears to be interfacing "correctly" with weed server -s3, however I would call it a stretch that 13 minutes is "correct".
The wiki/docs frequently state AWS_CREDS=...doesn't_matter... which is why I thought that omitting them would be totally fine (and it kindof is?!?) except that it's infinitely slower than not having the variables set??

I'd love to get some feedback on this, and don't mind kindof going through a sweep of the docs examples and making sure they "behave better" or are a little more clear, but before bombing in on the documentation, I'd really love to know a bit more about your (dev/maintainer) P.O.V. on "no-AWS_...-env-var == very very slow performance, but actually appears to work totally fine".
I'm having a tough time imagining why/how I got into this situation when this all "looked like" a very simple and easy demo to get set up?  Again... I could understand if it was slow and didn't work, but I can't understand why it is so very slow but does work... I'll be poking over at the restic people/code and see if I can figure out if there's something they're doing really funky (probably) involving timeouts and S3-creds / env. variables.
Thanks for your patience (if you've read this far).

...and it's restic:
$ time ./restic -r "s3:http://localhost:8333/resticbucket" init --password-command "echo 'test'"
created restic repository 257309f3a7 at s3:http://localhost:8333/resticbucket

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

real	0m2.612s
user	0m2.393s
sys	0m0.029s

$ time ./restic -r "s3:http://localhost:8333/resticbucket" backup /etc/motd --password-command "echo 'test'"
repository 257309f3 opened (version 2, compression level auto)
created new cache in /home/rames/.cache/restic
found 1 old cache directories in /home/rames/.cache/restic, run `restic cache --cleanup` to remove them
no parent snapshot found, will read all files
[0:00]          0 index files loaded

Files:           1 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repository: 1002 B (885 B stored)

processed 1 files, 286 B in 0:00
snapshot 5fa31ace saved

real	0m0.696s
user	0m0.481s
sys	0m0.033s

$ git diff
diff --git a/internal/backend/s3/s3.go b/internal/backend/s3/s3.go
index d41f4479d..3e0887942 100644
--- a/internal/backend/s3/s3.go
+++ b/internal/backend/s3/s3.go
@@ -117,11 +117,6 @@ func getCredentials(cfg Config) (*credentials.Credentials, error) {
 		&credentials.EnvMinio{},
 		&credentials.FileAWSCredentials{},
 		&credentials.FileMinioClient{},
-		&credentials.IAM{
-			Client: &http.Client{
-				Transport: http.DefaultTransport,
-			},
-		},
 	})

 	c, err := creds.Get()

...kindof more actually minio-go, which dawdles around a bunch in https://github.com/minio/minio-go/blob/master/pkg/credentials/iam_aws.go#L337, presumably not caching "invalid/non-existent" credentials calls. :-/

Closing issue with resolution of:

Restic uses minio with a NewCredentialsChain containing &credentials.IAM{...}
minio credentials.IAM{...} deals very poorly when not running inside on EC2 instance
...eventually restic (via minio) flunks each item in the getCredentials, leading to successful anonymous access
...but it slows everything down to unusable speeds.

Workaround is: MUST configure and provide AWS_... environment variables so that restic successfully auths and never reaches the IAM{...} provider.

Updated wiki with summary of info: https://github.com/seaweedfs/seaweedfs/wiki/restic-with-SeaweedFS/_compare/b161b9eca2112116aab6305d2f3cc4c4acafdfde...f4d5d942928ad0d929356059a40ab5dac0690cf1

@ramses0 big thanks for your investigation, ran into this issue too


And lint, test, build on commit

https://blog.testproject.io/2021/02/01/using-github-actions-to-run-automated-tests/

Done enough in release.yml


Missing license

Greetings
I would like to use this project (or a fork that is more up to date respectively) in an ongoing open-source project of mine.
However, this project is missing a LICENSE.md file, thus making it copyright protected with no permissions whatsoever granted (which I assume, is not the intended purpose).
Could you clarify this, or add a LICENSE.md that grants permission (e.g. MIT, CC, GPL) or explicitly probhits use?

Apologies, I usually license all my work under MIT, but this one slipped that radar. I'll fix this now.


A11y - OU: ARIAdialog and alert dialog nodes should have an accessible name

Ensure every ARIA dialog and alert dialog node has an accessible name. [+more informationLink opens in a new window+|https://dequeuniversity.com/rules/axe/4.10/aria-dialog-name?application=AxeChrome]
h3. Element Location:
{noformat}#b2b-organization-unit-confirm-modal{noformat}
h3. To solve this problem, you need to fix at least (1) of the following:

aria-label attribute does not exist or is empty
aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
Element has no title attribute

!Screen Shot 2024-11-13 at 16.51.43 PM.png|width=2052,height=1818,alt="Screen Shot 2024-11-13 at 16.51.43 PM.png"!

[public] Shopware automated response: This issue is linked to the internal issue: NEXT-39576
[created from NEXT-39576, comment 512410]


Support for Techcrunch.com

Hi guys,
I was wanting to download a few videos from http://techcrunch.com/. They are a leading news website focused on information technology companies. Unfortunately, youtube-dl wasnt able to extract their videos.
youtube-dl -v -o "/media/%(title)s.mp4"  http://techcrunch.com/video/facebook-creates-on-this-day-crunch-report/518726732/                                     [debug] System config: []o/facebook-creates-on-this-day-crunch-report/518726732/                                                                                [debug] User config: []
[debug] Command-line args: [u'-v', u'-o', u'/media/%(title)s.mp4', u'http://techcrunch.com/video/facebook-creates-on-this-day-crunch-report/518726732/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.03.24
[debug] Python version 2.7.6 - Linux-3.13.0-35-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: ffmpeg git-2014-12-27-d4fd3f2, ffprobe git-2014-12-27-d4fd3f2
[debug] Proxy map: {}
[generic] 518726732: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 518726732: Downloading webpage
[generic] 518726732: Extracting information
ERROR: Unsupported URL: http://techcrunch.com/video/facebook-creates-on-this-day-crunch-report/518726732/
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 825, in _real_extract
    doc = parse_xml(webpage)
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1528, in parse_xml
    tree = xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: undefined entity: line 39, column 2
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 651, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 275, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1313, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://techcrunch.com/video/facebook-creates-on-this-day-crunch-report/518726732/

Thank you for the report. 5min embeds, which are used on techcrunch.com, should be supported in the next version of youtube-dl. Until it is released, you may download techcrunch videos using their IDs and the 5min shorthand, for example:
youtube-dl '5min:518726732'


Added in the semantic properties docs

Adds in docs for https://github.com/CommunityToolkit/Maui.Markup/pull/103

Docs Build status updates of commit 559f5ca:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/maui/markup/extensions/automation-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/semantic-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/markup.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit fc45a47:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/maui/markup/extensions/automation-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/semantic-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/visual-element-extensions.md
:white_check_mark:Succeeded
View

docs/maui/markup/markup.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 213ffa5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/maui/markup/extensions/automation-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/semantic-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/visual-element-extensions.md
:white_check_mark:Succeeded
View

docs/maui/markup/markup.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 6074ff1:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/maui/markup/extensions/automation-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/semantic-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/visual-element-extensions.md
:white_check_mark:Succeeded
View

docs/maui/markup/markup.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 5146029:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/maui/markup/extensions/automation-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/semantic-properties.md
:white_check_mark:Succeeded
View

docs/maui/markup/extensions/visual-element-extensions.md
:white_check_mark:Succeeded
View

docs/maui/markup/markup.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


symplify / config-transformer Feature suggestion

When using the switch-format  command the tool is unable to read directories.
In Loader.php line 71:

  Cannot load resource "folder_name/".

It would be a useful feature for it to convert all yaml files to php regardless of directory depth.

Hey @Kerrialn , I'm not sure what you mean.
Could you provide some specific example?


Icon submission: Avast

[x] I have read and followed CONTRIBUTING document.
URL of the site or app (required): https://www.avast.com/
Where did you took the BG color(s) (required, unless it's generic icon): https://static3.avast.com/20001312/web/i/v2/components/logos/avast-logo-default.svg, https://press.avast.com/hubfs/media-materials/logos-guidelines/Avast/avast-logo.zip

Uploaded files:
Avast.zip

Added to the repo 👍


[0.10.12-insider] Debug adapter client doesn't send configurationDone when no breakpoints are registered

It appears that the new insider release has changed the startup behavior of the debug adapter client which causes the PowerShell debug adapter to never start executing the script.  We've determined that this is happening because the configurationDone request isn't being send when no breakpoints have been created in the Debug workspace.
Was this intentional?  Should we change the expectations for this request in our code?
See the following bug for more details: https://github.com/PowerShell/PowerShellEditorServices/issues/196

VSCode Version: 0.10.12-insider
OS Version: Windows 10

Steps to Reproduce:

Install PowerShell extension 0.5.0
Open folder `c:\Users<username>.vscode-insiders\extension\ms-vscode.PowerShell-0.5.0\examples'
Select the DebugTest.ps1 file
Switch to Debug workspace and start the debugger
Observe that no output is written and you cannot pause the debugger.  This is because the PowerShell debug adapter never starts executing the script because it didn't receive the configurationDone request.

I apologize, I didn't realize Keith had already filed a bug :)  Let's use his instead: https://github.com/Microsoft/vscode/issues/4567


[multi-stage] allow configurable timeout

this enables configuration for timeout on a

server config level via broker timeout or query runner timeout

either config key is fine, one is in nano second one is in millis

or via query options

TODO
add test to test query option override.

Codecov Report

Merging #9571 (741565a) into master (03121cf) will decrease coverage by 44.69%.
The diff coverage is 44.44%.

@@              Coverage Diff              @@
##             master    #9571       +/-   ##
=============================================
- Coverage     60.34%   15.65%   -44.70%
+ Complexity     5072      174     -4898
=============================================
  Files          1919     1878       -41
  Lines        102764   100784     -1980
  Branches      15616    15381      -235
=============================================
- Hits          62017    15774    -46243
- Misses        36089    83824    +47735
+ Partials       4658     1186     -3472

Flag
Coverage Δ

integration2
?

unittests1
?

unittests2
15.65% <44.44%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...requesthandler/MultiStageBrokerRequestHandler.java
0.00% <0.00%> (ø)

...rg/apache/pinot/query/service/QueryDispatcher.java
84.15% <75.00%> (+0.15%)
:arrow_up:

...ot/query/runtime/executor/WorkerQueryExecutor.java
95.45% <100.00%> (+0.06%)
:arrow_up:

...query/runtime/operator/MailboxReceiveOperator.java
79.31% <100.00%> (ø)

...src/main/java/org/apache/pinot/sql/FilterKind.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ain/java/org/apache/pinot/core/data/table/Key.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...in/java/org/apache/pinot/spi/utils/BytesUtils.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...n/java/org/apache/pinot/core/data/table/Table.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../java/org/apache/pinot/core/data/table/Record.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../java/org/apache/pinot/core/util/GroupByUtils.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 1631 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Correct me if I am wrong, the queryTimeoutMs in queryOption only applies to query server. And this one applies to broker?
So the query one can be tuned per query and this one is tuned per table config? I feel this is a little weird..

@61yao unfortunately there's no direct equivalence.
the currently query timeout logic is:

query timeout can be overwritten in this order

user config inside the query payload
table config
cluster config
cluster default (10sec)

query timeout is being used since the beginning of the query planning

only the remainder part of the timeout (e.g. total - plan finished duration ) is send to server as the server timeout

from a user perspective I think the user-config override should still apply as the top level (as from user's view, a query runs for at maximum X many seconds)
however

there's no direct equivalence of table config override

consider a join of 2 tables, which table's override should we apply?
do we apply on per stage or per query?

if user is specifying for the entire query, how do we split between stages

here is my proposal

user-config override should always be considered as the total time (maybe minus planning time, i will explain later)

with this in mind, all the current timeout configuration during query should also be considered as total time

all stages have a fixed timeout at an absolutely ts value = sys.currentTimeNano() + desiredTimeoutNano

if any stage hits this timeout, they will all agree to terminate.

@61yao unfortunately there's no direct equivalence. the currently query timeout logic is:

query timeout can be overwritten in this order

user config inside the query payload
table config
cluster config
cluster default (10sec)

query timeout is being used since the beginning of the query planning

only the remainder part of the timeout (e.g. total - plan finished duration ) is send to server as the server timeout

from a user perspective I think the user-config override should still apply as the top level (as from user's view, a query runs for at maximum X many seconds) however

there's no direct equivalence of table config override

consider a join of 2 tables, which table's override should we apply?
do we apply on per stage or per query?

if user is specifying for the entire query, how do we split between stages

here is my proposal

user-config override should always be considered as the total time (maybe minus planning time, i will explain later)

with this in mind, all the current timeout configuration during query should also be considered as total time

all stages have a fixed timeout at an absolutely ts value = sys.currentTimeNano() + desiredTimeoutNano

if any stage hits this timeout, they will all agree to terminate.

Maybe after the proposal is done, we can document this somewhere? Let me file an issue to track this.

Do we agree that this is the end state ?
User specifies a query timeOut T which from user's perspective is the end to end query timeOut.
If Pinot does not finish processing the query within T, it should terminate execution (and ideally not leave any lingering execution resources)
In the current engine, T can be specified at Instance level / cluster level (broker config) or query option level or table level. The last one probably does not make sense for a multi table query in multi-stage engine.
Broker uses T depending on where it is coming from, spends some time T' in query planning etc and sends the remaining (T - T') to worker nodes when dispatching the plan. This is what is done in the current engine.
For the multi-stage engine as well, regardless of how many stages the plan is going to be executed in, X = (T - T') is the max time subsequent stages have.
If for example leaf stage itself takes more time than X, then it should terminate / interrupt threads and downsteam stages (parents) should detect that timeOut has expired, mailbox should not expect anything.
So, the entire operator pipeline / chain gets terminated if X gets exhausted anywhere

+1 on @siddharthteotia 's comment.
let's do the following

ignore the table level config for multi-stage (at least for now)
the rest is exactly the same, make sure T is the e2e timeout by setting an absolute "must-finish-or-timeout" epoch and all process should check again

readjusted the PR. PTAL


Copy-on-click

Given that there's no action when you click on a row, it would be cool if clicking automatically copied the token (this is what the Google Auth app does).
That way, I'd not have to press-and-hold and then select Copy

I think this is a good idea. I did try this at one point but it interfered with the "advance counter" button for counter based tokens. I think we might be able to make it a bit simpler by allowing one to tap to bring up the context-menu instead of tap-and-hold.


Changing Mirage's default linkage data behavior in 0.4 to better conform to JSON:API's semantics

I wanted to write a quick post detailing a change that happened in Mirage's 0.4.0 release that I originally didn't signal as a breaking change, but that affected more apps than I realized.
I have since added the following note about it to the [0.3 -> 0.4 upgrade guide]({{ site.baseurl }}{% link docs/v0.4.x/upgrading.md %})
, and I'm reposting it here for more visibility. I think it's worth understanding, as the issue sheds light on the semantics around some of JSON:API's design decisions.
Apologies if you upgraded to 0.4 and had to debug this yourself!

There is one primary change in 0.4 that could break your 0.3 app.
In 0.3.x, Mirage's JSONAPISerializer included all related foreign keys whenever serializing a model or collection, even if those relationships were not being included in the payload.
This actually goes against JSON:API's design. Foreign keys in the payload are known as Resource Linkage and are intended to be used by API clients to link together all resources in a JSON:API compound document. In fact, most server-side JSON:API libraries do not automatically serialize all related foreign keys, and only return linkage data for related resources when they are being included in the current document.
By including linkage data for every relationship in 0.3, it was easy to develop Ember apps that would work with Mirage but would behave differently when hooked up to a standard JSON:API server. Since Mirage always included linkage data, an Ember app might automatically be able to fetch related resources using the ids from that linkage data plus its knowledge about the API. For example, if a post came back like this:
// GET /posts/1
{
  data: {
    type: 'posts',
    id: '1',
    attributes: { ... },
    relationships: {
      author: {
        data: {
          type: 'users',
          id: '1'
        }
      }
    }
  }
}

and you forgot to ?include=author in your GET request, Ember Data would potentially use the user:1 foreign key and lazily fetch the author by making a request to GET /authors/1. This is problematic because

This is not how foreign keys are intended to be used
It'd be better to see no data and fix the problem by going back up to your data-loading code and add ?include=author to your GET request, or
If you do want your interface to lazily load the author, use resource links instead of the resource linkage data:

// GET /posts/1
{
  data: {
    type: 'posts',
    id: '1',
    attributes: { ... },
    relationships: {
      author: {
        links: {
          related: '/api/users/1'
        }
      }
    }
  }
}

Resource links can be defined on Mirage serializers using the links method (though including is likely the far more simpler and common approach to fetching related data).
So, Mirage 0.4 changed this behavior and by default, the JSONAPISerializer only includes linkage data for relationships that are being included in the current payload (i.e. within the same compound document).
This behavior is configurable via the alwaysIncludeLinkageData key on your JSONAPISerializers. It is set to false by default, but if you want to opt-in to 0.3 behavior and always include linkage data, set it to true:
// mirage/serializers/application.js
import { JSONAPISerializer } from 'ember-cli-mirage';

export default JSONAPISerializer.extend({
  alwaysIncludeLinkageData: true
});

If you do this, I would recommend looking closely at how your real server behaves when serializing resources' relationships and whether it uses resource links or resource linkage data, and to update your Mirage code accordingly to give you the most faithful representation of your server.

@samselikoff I wouldn't say this better conforms with the JSONAPI spec. I believe this may actually be a slight step backward.
While the JSONAPI spec itself is fairly ambiguous about whether the server may omit some (or all) relationships from a resource object's relationships object when the client hasn't requested any particular sparse fieldset (see https://github.com/json-api/json-api/issues/1388), this change, at a minimum, goes against JSONAPI's recommendations:

The base specification is agnostic about including links with a resource response. However, it is recommended that the following links be included within response documents:

Top-level links like a self-link (for the whole response) as well as relative pagination links (if appropriate).
Resource-level links like a self-link for each resource (which differs from the top-level, if the resource is part of a collection).
Relationship links for all available relationships of a resource.

(From the following recommendation: https://jsonapi.org/recommendations/#including-links)
I wish Mirage would better align with JSONAPI serializer libs throughout which a common behavior is to:

Have all attributes and relationships be represented in the resource object's attributes object and relationships object respectively, unless otherwise requested by a sparse fieldset.
Have all relationship objects contain a links object by default (unless otherwise specified by what these libraries commonly refer to as "default includes", in which case resource linkage would be included in addition to the links object).
When the client supplies an explicit list of includes, then the relationships corresponding to these includes will be "expanded" in that their relationship object will contain resource linkage in addition to their links object while relationships that were not explicitly requested will still be represented and their relationship object will contain a links object.

To summarize, if I have a model defined as:
// models/my-resource.js
export default Model.extend({
    foo: attr(...),
    bar: attr(...),
    baz: belongsTo(...),
    qux: hasMany(...),
});

I would expect a request to GET /my-resources/123?include=baz to generate the following response:
{
    "data": {
        "type": "my-resources",
        "id": 123,
        "attributes": {
            "foo": ...,
            "bar": ...
        },
        "relationships": {
            "baz": {
                "links": { ... },
                "data": { ... }
            },
            "qux": {
                "links": { ... }
            }
        }
    },
    "included": [
        ...
    ]
}

Looking at your example, I think the only thing Mirage doesn't do out of the box is define links on the relationships hash, and that's because Mirage has no higher-level routing concept baked into it. (You always only define single routes at a time.)
However, there is a links API on Serializers which can be used for this purpose:
import { JSONAPISerializer } from 'ember-cli-mirage';

export default JSONAPISerializer.extend({

  links(myResource) {
    return {
      baz: {
        links: {
          self: `/my-resources/${myResource.id}/relationships/baz`,
          related: `/my-resources/${myResource.id}/baz`
        }
      }
      qux: {
        links: {
          self: `/my-resources/${myResource.id}/relationships/qux`,
          related: `/my-resources/${myResource.id}/qux`
        }
      }
    };
  }

});

Obviously this would be cumbersome to implement for every relationship on every resource. In the past when this is important to me, I've added a bit of dynamic code to solve this. Something like the following (pseudocode):
import { JSONAPISerializer } from 'ember-cli-mirage';

export default JSONAPISerializer.extend({

  links(resource) {
    let links = {};

    resource.associationKeys.forEach(key => {
      links[key] = {
        links: {
          self: `/${pluralize(resource.modelName)}/${resource.id}/relationships/${key}`,
          related: `/${pluralize(resource.modelName)}/${resource.id}/${key}`
        }
      }
    });

    return links;
  }

});

This would require the related endpoints be defined, which could also be done somewhat dynamically:
// mirage/config.js
this.get('my-resources/:id/relationships/:relationship_name', (schema, request) => {
  let myResource = schema.myResources.find(request.params.id);

  return myResource[request.params.relationship_name];
});

The reason we haven't baked any of this in yet is because the links strategy for backends differ - there is nothing in the spec that indicates what the link relationship should look like. However, we absolutely could and should provide some routing abstraction that makes this much easier.
The main reason this post claimed that the change related to linkage data (as opposed to links) better conformed to the spec, is because linkage data is intended to link resources within a single document, rather than be used to fetch remote resources by ID.

@samselikoff Thank you for the quick reply! That makes sense. I definitely think Mirage should ship with some of the code you included in your comment as this is something people are likely to have to reinvent over and over again.
Perhaps the JSONAPISerializer could expose a hook that given a model instance and a relationship key would return the self and/or related link(s) with a reasonable default implementation.


Rearrange Equation to find unknown variable

Would it be possible to rearrange an equation to report unknown variable?
For example:
Phi = 0.9
Mn = 100 kip.ft
Fy = 50 ksi
PhiMn = FyZx
Zx = ?
This could be helpful for longer equations.
Please point me to the post if this already a feature.
Thanks for your time.

Hi!
If you mean to solve for the variable Zx symbolically and get Zx = Phi*Mn/Fy, Calcpad cannot do that. It is a numerical package and does not include symbolic evaluations. It can only calculate the formulas you enter in the code.
We do not plan to develop a symbolic engine. It is a huge task and there are many symbolic CAS's already available.

Thanks Ned


Doesn't remove instance of Editor

I am using this tinymce editor directive inside ng-repeat. When I try to remove specific instance from the list then it always trigger the $destroy on last element in the list. For example, if I have a list of 4 elements and uses ng-repeat to display a list with editor and remove button; then when I click the remove button on element 2, it will trigger $destroy on last element 4 and removes that editor.

If the editor all has the same id attribute value in the ng-repeat, then it may not garbage collect properly - what happens if you do something like id="foo-{{$index}}" in the ng-repeat?

I believe this should be fixed in master. Be sure to remove the id attribute if you are using it, as I made an update to the API of the directive in order to correctly manage situations with ng-repeat and ng-if.


Datastore rounding recipient country percentages

Describe the bug
The datastore is rounding recipient country percentages:
https://iatidatastore.iatistandard.org/search/activity?q=(iati_identifier:"NO-BRC-980997278-970000")&fl=recipient_country_*&wt=xslt&tr=activity-xml.xsl&rows=1
Expected behavior
Would expect that the datastore would provide the exact percentages as published:

This obviously creates an issue when you are then applying these percentages to transaction values to determine the value by country. It means that the sum of the transaction values by country will not be equal to the original transaction value.

It's important that the datastore reflects the original xml. This needs to be fixed.

@PetyaKangalova  you can check here. There is no rounding of decimal value https://test-datastore.iatistandard.org/search/activity?q=(iati_identifier:"NO-BRC-980997278-970000")&fl=recipient_country_*&wt=xslt&tr=activity-xml.xsl&rows=1

Thanks @luminhan ! This looks correct in the file you shared!
I wanted to verify with one of SIDA's activities but all percentages are 0 or 100, so not a good one to test!


[Deepcast] Cannot translate

Extension
https://www.raycast.com/mooxl/deepcast
Description
I've created an account for DeepL and registered for the free API, got the Key and pasted it into the extensions settings.
Now, when I wan't to translate something via the extension, I just cannot because "something went wrong."
I checked the API and used 0/500,000 characters. The key is correct, i copied it multiple times, typed it manually in, created a new key, nothing worked.
Steps To Reproduce

Use the "translate" command of Deepcast via Raycast
Enter some text and let it translate

Current Behaviour
When trying to translate it just says "something went wrong. Check your internet connection, API key, or you've maxed out the API."
Expected Behaviour
To get the translated text.

Hi!
Seems to be a duplicate of #6105 and #5882.
I'd have to get some more details on the error and on how to reproduce it. No matter what I do, Deepcast works on my machine. I don't have another one to test it out, unfortunately.
For now I can suggest you doing this test to see whether the API is accessible from your Mac.
Error reporting system in Deepcast is currently not informative at all, but I already have an open PR #6118 which aims to drastically improve error handling.

Hi @MonsterDeveloper
yes, seems to be duplicate, sorry for that!
I did the test and received the expected return of:
{"translations":[{"detected_source_language":"EN","text":"Hallo, Welt!"}]}

Hi once again!
My PR #6118 has just been merged, which includes improved error handling. Could you please update the extension and make a screenshot of the error?

Thanks for the info! I just tried and got a translation, so no error so far!


Hai Yorokonde by This Kento

Song Name
Hai Yorokonde
Artist Name
This Kento
Source
Youtube
Youtube Link
jzi6RNVEOtA
Direct File Link
No response
Song ID
123
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 107252586
Downloaded: No

accept

Accept


iOS: Fix an image loader crash

Fixes #10433
The code didn't account for the fact that cancelLoad is set by two different threads. It gets set on the URL request queue when the request completes successfully and it gets set on the UI queue during cancelation. This oversight lead to a couple of different kinds of crashes.

Attempt to invoke a nil function -- We check that cancelLoad is non-nil and on the next line we call it. However, cancelLoad could have been set to nil by the other thread between the time it was verified to be non-nil and the time it was called. Consequently, the program will attempt to call nil and crash.

Block deallocated while it's executing -- Suppose cancelLoad points to a block, it is verified to be non-nil, and it is successfully invoked. In the middle of executing the block, cancelLoad, the last reference to the block, is set to nil. This causes the block to be deallocated and its captured values to be freed. However, the block continues executing and the next time it attempts to use a captured value, the program will crash because the captured value has already been freed. This explains how the program can crash on OSAtomicOr32Barrier(1, &cancelled);: cancelled is a captured value that may already have been freed.

Test plan (required)
I built a test app which consistently repros this bug: https://github.com/rigdern/RNImageRepro (see the repro steps in the README). After this change, the test app no longer repros the crash.
Notes
While working on this code, some of the design intentions were not clear to me.
For example, are the cancelation handlers always supposed to be run on the UI thread? If not and they can be run on multiple threads, I believe it's possible for a cancelation handler to run multiple times (it's not clear to me if that's a bad thing).
Is the scenario allowed where both a completion handler and a cancelation handler run for a singe request? I believe this is possible in the current implementation but I wasn't sure if this was intended.

cc @javache

You're completely right that there's a lack of threading guarantees here. RCTNetworking is mostly meant to run its own queue, but everything image-related blows a big hole in that assumption. I do think the underlying layers should be prepared to accept cancel to be invoked multiple times. RCTNetworkTask already protects against this internally.
Thank you so much for fixing this!

@facebook-github-bot shipit


scss() always publishes to public even though other path specified

So in the docs, sass is like mix.sass('resources/assets/sass/app.scss', 'public/css');
I changed public/css to resources/build/css. The css file ends up in public/resources/build/css/app.css. I would expect it to go to resources/build/css/app.css instead.

Add mix.setPublicPath('resources')

@JeffreyWay but that limits me to one directory.
I want to first build scss to css in resource folder. Then use scripts() to combine it that file to public folder.
This wouldn't work either:
mix.setPublicPath('resources')
    .sass('resources/assets/sass/app.scss', 'build/css');

mix
    .setPublicPath('public')
    .styles([
        'resources/build/css/app.css',
        'resources/semantic/dist/semantic.css'
    ], 'public/css/all.css');

Also, if public path is already public why would the argument public/css not go to public/public/css?

It's assumed that your Sass should compile directly to the public directory. You can't change that right now.

Also, if public path is already public why would the argument public/css not go to public/public/css?

"public" in your output path is optional. So if you include it, we strip it off to avoid confusion.

It's assumed that your Sass should compile directly to the public directory. You can't change that right now.

Even then, there doesn't seem to be a way to queue the tasks so that the combine method scripts() uses the compiled sass.

When you say "queue the task," do you mean combine the stylesheets only after the Sass has compiled?
This works for me:
mix.sass('resources/assets/sass/app.scss', 'build/css')
    .styles([
        'public/build/css/app.css',
        'resources/semantic/dist/semantic.css'
    ], 'public/css/all.css');

Okay that worked.
Must have been confused by the stripping. :smile:
I think the auto-stripping adds more confusion since setPublicPath is set by default unbeknownst to the developer.
It would make more sense to have it optional but pre-populate the file with setRelativePath('public') so that the second parameter is relative to root by default. Less confusing this way. (Also setPublicPath() is not mentioned in Laravel docs)
// default webpack file
mix.setRelativePath('public')
    .sass('resources/assets/sass/app.scss', 'css');

@JeffreyWay Also, I tried your code with version(). It stopped working -- I think because the build intermediate file is also versioned.

When you add versioning, it no longer creates resources/build/css/app.css. It creates something like resources/build/css/app.13241212512.css. So compensate for that by using a star.
mix.sass('resources/assets/sass/app.scss', 'build/css')
    .styles([
        'public/build/css/app.*.css',
        'resources/semantic/dist/semantic.css'
    ], 'public/css/all.css');


Node 16

@microsoft-github-policy-service agree [company="{your company}"]

@microsoft-github-policy-service agree

/azp run

@Mmdixon opened this new PR after dropping the package-lock.json changes as you suggested at
https://github.com/microsoft/PhoneticMatching/pull/49
Build still failing, could you help me identifying the problem? Thanks!


Creating hashes used in the objects takes a lot of time

This issue has been resolved in ac9cb2c9c1654219fdd627f861b70c48e1059408 where index-based accessors are used instead of hash.


fix: factory tests

Current dependencies on/for this PR:

main

PR #142

PR #133

PR #149

PR #158   👈
PR #98

PR #155

PR #156
PR #145

PR #157

PR #144

PR #152

PR #154

PR #128

PR #151
PR #150
PR #138

PR #135
PR #134

PR #147

PR #148

PR #140
PR #139
PR #132

PR #143
PR #137

This comment was auto-generated by Graphite.


predictor feeding shape error

hello, after the program recognize a face, i have problem in feeding the predictor.
ValueError: Cannot feed value of shape (53, 53) for Tensor 'DecodeJpeg:0', which has shape '(?, ?, 3)'
what i have to do ?

@linaashaji Same error, have you solved this?

hello, after the program recognize a face, i have problem in feeding the predictor.
ValueError: Cannot feed value of shape (53, 53) for Tensor 'DecodeJpeg:0', which has shape '(?, ?, 3)'
what i have to do ?

Hey! Please share the solution if you have solved it orany resources might help....Thank You


kytos code improvement

[ ] In the static method named get_type_port_speed, Instead of doing a linear lookup, you could have values as a dictionary with multiple keys mapping to the same value.
The dictionary-based approach is generally more efficient than the original list-based approach for lookups, especially when the number of entries increases, as dictionary lookups have an average time complexity of O(1) compared to the linear search in lists with time complexity of O(n).
[ ] In the get_port method and others, Start adding type hints in convert_topology.py
As you did for the port, have a function to encapsulate the transformation from a switch to a node_urn, like get_node_urn().
[ ] In the listen_event method, Use pool="dynamic_single" like Pathfinder does, making this a single thread exclusive handler.
[ ] Def test_kytos_topology(self): This method isn't used anywhere

The task has been completed successfully and the code has been improved.


fix (curriculum): changed the index to 0 for the fibonacci sequence challenge

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #38325

@hassaanp @jeremylt shouldn't fibonacci(3) be equal to 1 and so on?

The argument of the original issue was that the mathematical form of the Fibonacci sequence starts from base 0 and that should be reflected in the solution expected as well. So instead of starting from index 1, we start with fibonacci(0) = 0, fibonacci(1) = 1, fibonacci(2) = 1, fibonacci(3) = 2 and so on.

Ah, I see. That makes sense


Version 1.10 Maven Central probabily broken?

Hi @kennycason ,
just recognized, that you released 1.10.
However, I think, that this release on mvn-central is broken.
I cannot find the "father-pom" for 1.9 / 1.10: http://repo1.maven.org/maven2/com/kennycason/kumo/
Modules are available in 1.9 and 1.10: http://repo1.maven.org/maven2/com/kennycason/
Maybe you can take a look at it :) - for now, it is not useable via mvn central and one have to install it locally.

oh crap. Thanks! I'll get this fixed asap.

@rzo1 Appears to be updated. I checked both directories you linked to.

seems to be available now, but i will check in 1-2 days until the mvn repos are synced

ok, i integrated 1.11 which worked via Maven Central.