cvat-cli create cannot create task from local media and manifest.jsonl

Actions before raising this issue

[X] I searched the existing issues and did not find anything similar.
[X] I read/searched the docs

https://opencv.github.io/cvat/docs/manual/advanced/data_on_fly/
https://opencv.github.io/cvat/docs/api_sdk/cli/
Steps to Reproduce
labels.json
[
    {
        "name": "foo",
        "attributes": []
    }
]

manifest.jsonl
{"version": "1.0"}
{"type": "images"}
{"name": "dataset/image", "extension": ".png", "width": 100, "height": 100 , "meta": {"related_images": []}, "checksum": "c2bc34648c583e6b9959c60bf51a4eff"}

Recreate dataset
mkdir dataset
wget -O dataset/image.png https://en.wikipedia.org/static/images/icons/wikipedia.png

cvat-cli   --server-host "http://<IP_ADDRESS>"  --server-port 8080 create task --labels labels.json --use_cache --filename_pattern "*" local manifest.jsonl

Expected Behavior
I expected cvat-cli to

Create a task in my locally running cvat instance
Upload the image.png file to the instance and attach it to the task

Only 1) is carried out (I see the task created in the CVAT GUI), but the image is not, because I get the following error:
cvat-cli --server-host "http://<IP_ADDRESS>" --server-port 8080 create task --labels labels.json --use_cache --filename_pattern "*" local manifest.jsonl
Password:
[2024-02-04 20:17:52] INFO: Created task ID: 108 NAME: task
Uploading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 196/196 [00:00<00:00, 2.40kB/s]
[2024-02-04 20:17:52] INFO: Awaiting for task 108 creation...
[2024-02-04 20:17:54] INFO: Task 108 creation status: Failed (message=Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/rq/worker.py", line 1428, in perform_job
    rv = job.perform()
  File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 1278, in perform
    self._result = self._execute()
  File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 1315, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/django/cvat/apps/engine/task.py", line 651, in _create_thread
    media, task_mode = _validate_data(media, manifest_files)
  File "/home/django/cvat/apps/engine/task.py", line 265, in _validate_data
    raise ValueError('No media data found')
ValueError: No media data found)
[2024-02-04 20:17:54] CRITICAL: Status Code: 200
Reason: OK
HTTP response headers: HTTPHeaderDict({'Allow': 'GET, HEAD, OPTIONS', 'Content-Length': '846', 'Content-Type': 'application/vnd.cvat+json', 'Cross-Origin-Opener-Policy': 'same-origin', 'Date': 'Sun, 04 Feb 2024 20:17:54 GMT', 'Referrer-Policy': 'same-origin', 'Server': 'nginx/1.18.0 (Ubuntu)', 'Vary': 'Accept, Accept-Encoding, Origin', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'DENY, deny', 'X-Request-Id': 'e5858ebf-e54a-4cb8-b0c0-89d7f05f5c62'})
HTTP response body: b'{"state":"Failed","message":"Traceback (most recent call last):\\n  File \\"/opt/venv/lib/python3.10/site-packages/rq/worker.py\\", line 1428, in perform_job\\n    rv = job.perform()\\n  File \\"/opt/venv/lib/python3.10/site-packages/rq/job.py\\", line 1278, in perform\\n    self._result = self._execute()\\n  File \\"/opt/venv/lib/python3.10/site-packages/rq/job.py\\", line 1315, in _execute\\n    result = self.func(*self.args, **self.kwargs)\\n  File \\"/usr/lib/python3.10/contextlib.py\\", line 79, in inner\\n    return func(*args, **kwds)\\n  File \\"/home/django/cvat/apps/engine/task.py\\", line 651, in _create_thread\\n    media, task_mode = _validate_data(media, manifest_files)\\n  File \\"/home/django/cvat/apps/engine/task.py\\", line 265, in _validate_data\\n    raise ValueError(\'No media data found\')\\nValueError: No media data found","progress":0.0}'

I really don't get it, since the image is clearly there
tree  dataset/
dataset/
└── image.png

0 directories, 1 file

Possible Solution
My workaround for now is to specify the images in manifest.jsonl separately i.e. to run
cvat-cli --server-host "http://<IP_ADDRESS>"  --server-port 8080 create task --labels labels.json local dataset/image.png #  dataset/image2.png dataset/image3.png # if there are more images

EDIT
Actually, it is possible to pass the list of images using find + xargs
find dataset/ -regex .*png | xargs cvat-cli --server-host "http://<IP_ADDRESS>"  --server-port 8080 create task --labels labels.json local

But I was wondering if there is a better way. Also, I noticed cvat limits the number of frames per task to a few hundred. Can this limit be controlled?
Context
I need to upload multiple (thousands) images at once, and I am afraid just adding them all to the command line may not be an option, because I may get a Too many arguments error or something like that. Besides, it should be possible to just upload a folder of images, just like one can upload a video (which is a container of images).
Environment
I am using cvat with commit a33f7f57088744bab61f18e8a8cf6528a0c22fd2  (tag: v2.10.1) on Docker

Client:
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.1
 Git commit:        20.10.21-0ubuntu1~22.04.3
 Built:             Thu Apr 27 05:57:17 2023
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.1
  Git commit:       20.10.21-0ubuntu1~22.04.3
  Built:            Thu Apr 27 05:37:25 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.12-0ubuntu1~22.04.3
  GitCommit:
 runc:
  Version:          1.1.7-0ubuntu1~22.04.2
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:

Environment:
Python 3.10.11 (mamba environment)
cvat_cli 2.10.1
cvat_sdk 2.10.1
OS:
Ubuntu 22.04.2 LTS
docker logs cvat_ui:
cvat_server                      | 2024-02-04 20:27:35,462 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "POST /api/auth/login HTTP/1.0" 200 OK
cvat_server                      |
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":50,"DownstreamStatus":200,"Duration":146339063,"OriginContentSize":50,"OriginDuration":146276161,"OriginStatus":200,"Overhead":62902,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":52,"RequestCount":277,"RequestHost":"<IP_ADDRESS>","RequestMethod":"POST","RequestPath":"/api/auth/login","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:35.316800436Z","StartUTC":"2024-02-04T20:27:35.316800436Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:35Z"}
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":450,"DownstreamStatus":200,"Duration":15345823,"OriginContentSize":450,"OriginDuration":15299043,"OriginStatus":200,"Overhead":46780,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":0,"RequestCount":278,"RequestHost":"<IP_ADDRESS>","RequestMethod":"GET","RequestPath":"/api/server/about","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:35.464562219Z","StartUTC":"2024-02-04T20:27:35.464562219Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:35Z"}
cvat_server                      | 2024-02-04 20:27:35,480 DEBG 'uvicorn-0' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "GET /api/server/about HTTP/1.0" 200 OK
cvat_server                      |
cvat_server                      | 2024-02-04 20:27:35,521 DEBG 'uvicorn-0' stderr output:
cvat_server                      | [2024-02-04 20:27:35,521] INFO cvat.apps.engine.serializers: [Task.id=113] label:create Label id:405 for spec:OrderedDict([('name', 'foo'), ('color', '#ddfb36')]) with sublabels:[], parent_label:None
cvat_server                      |
cvat_server                      | 2024-02-04 20:27:35,544 DEBG 'uvicorn-0' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "POST /api/tasks HTTP/1.0" 201 Created
cvat_server                      |
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":652,"DownstreamStatus":201,"Duration":63032638,"OriginContentSize":652,"OriginDuration":63006598,"OriginStatus":201,"Overhead":26040,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":63,"RequestCount":279,"RequestHost":"<IP_ADDRESS>","RequestMethod":"POST","RequestPath":"/api/tasks","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:35.48123049Z","StartUTC":"2024-02-04T20:27:35.48123049Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:35Z"}
cvat_server                      | 2024-02-04 20:27:35,595 DEBG 'uvicorn-0' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "POST /api/tasks/113/data/ HTTP/1.0" 202 Accepted
cvat_server                      |
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":202,"Duration":48416031,"OriginContentSize":0,"OriginDuration":48382407,"OriginStatus":202,"Overhead":33624,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":0,"RequestCount":280,"RequestHost":"<IP_ADDRESS>","RequestMethod":"POST","RequestPath":"/api/tasks/113/data/","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:35.54712414Z","StartUTC":"2024-02-04T20:27:35.54712414Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:35Z"}
cvat_server                      | 2024-02-04 20:27:35,634 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "POST /api/tasks/113/data/ HTTP/1.0" 200 OK
cvat_server                      |
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":200,"Duration":36320719,"OriginContentSize":0,"OriginDuration":36284740,"OriginStatus":200,"Overhead":35979,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":614,"RequestCount":281,"RequestHost":"<IP_ADDRESS>","RequestMethod":"POST","RequestPath":"/api/tasks/113/data/","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:35.59875809Z","StartUTC":"2024-02-04T20:27:35.59875809Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:35Z"}
cvat_worker_import               | 2024-02-04 20:27:35,686 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,686] DEBUG rq.worker: Dequeued job create:task.id113 from import
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,687 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,686] INFO rq.worker: import: cvat.apps.engine.task._create_thread(113, {'chunk_size': None, 'size': 0, 'image_quality': 70, 'start_frame': 0, 'sto...) (create:task.id113)
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,687 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,687] DEBUG rq.worker: Sent heartbeat to prevent worker timeout. Next one should arrive in 480 seconds.
cvat_worker_import               |
cvat_server                      | 2024-02-04 20:27:35,688 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "POST /api/tasks/113/data/ HTTP/1.0" 202 Accepted
cvat_server                      |
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":426,"DownstreamStatus":202,"Duration":52363274,"OriginContentSize":426,"OriginDuration":52289322,"OriginStatus":202,"Overhead":73952,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":147,"RequestCount":282,"RequestHost":"<IP_ADDRESS>","RequestMethod":"POST","RequestPath":"/api/tasks/113/data/","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:35.636223294Z","StartUTC":"2024-02-04T20:27:35.636223294Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:35Z"}
cvat_worker_import               | 2024-02-04 20:27:35,703 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,703] DEBUG rq.worker: Started Job Registry set.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,703 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,703] DEBUG rq.worker: Preparing for execution of Job ID create:task.id113
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,704 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,704] DEBUG rq.worker: Sent heartbeat to prevent worker timeout. Next one should arrive in 90 seconds.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,706 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,706] DEBUG rq.worker: Job preparation finished.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,706 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,706] DEBUG rq.worker: Performing Job...
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,718 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,718] INFO cvat.apps.engine.task: create task #113
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,720 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,720] DEBUG rq.worker: Job create:task.id113 raised an exception.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,721 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,721] DEBUG rq.worker: Handling failed execution of job create:task.id113
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,723 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,723] DEBUG rq.worker: Handling exception for create:task.id113.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,723 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,723] ERROR rq.worker: [Job create:task.id113]: exception raised while executing (cvat.apps.engine.task._create_thread)
cvat_worker_import               | Traceback (most recent call last):
cvat_worker_import               |   File "/opt/venv/lib/python3.10/site-packages/rq/worker.py", line 1428, in perform_job
cvat_worker_import               |     rv = job.perform()
cvat_worker_import               |   File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 1278, in perform
cvat_worker_import               |     self._result = self._execute()
cvat_worker_import               |   File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 1315, in _execute
cvat_worker_import               |     result = self.func(*self.args, **self.kwargs)
cvat_worker_import               |   File "/usr/lib/python3.10/contextlib.py", line 79, in inner
cvat_worker_import               |     return func(*args, **kwds)
cvat_worker_import               |   File "/home/django/cvat/apps/engine/task.py", line 651, in _create_thread
cvat_worker_import               |     media, task_mode = _validate_data(media, manifest_files)
cvat_worker_import               |   File "/home/django/cvat/apps/engine/task.py", line 265, in _validate_data
cvat_worker_import               |     raise ValueError('No media data found')
cvat_worker_import               | ValueError: No media data found
cvat_worker_import               |
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,724 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,723] DEBUG rq.worker: Invoking exception handler <function rq_exception_handler at 0x7f4764402560>
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,724 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,724] DEBUG rq.worker: Invoking exception handler <function handle_rq_exception at 0x7f48492cf6d0>
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,744 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,744] DEBUG rq.worker: Sent heartbeat to prevent worker timeout. Next one should arrive in 480 seconds.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,744 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,744] DEBUG rq.worker: Sent heartbeat to prevent worker timeout. Next one should arrive in 480 seconds.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,745 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,745] DEBUG rq.worker: *** Listening on import...
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,745 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,745] DEBUG rq.worker: Sent heartbeat to prevent worker timeout. Next one should arrive in 480 seconds.
cvat_worker_import               |
cvat_worker_import               | 2024-02-04 20:27:35,745 DEBG 'rqworker-import-0' stderr output:
cvat_worker_import               | [2024-02-04 20:27:35,745] DEBUG rq.worker: Dequeueing jobs on queues import and timeout 405
cvat_worker_import               |
cvat_redis_inmem                 | 1:M 04 Feb 2024 20:27:35.764 * 100 changes in 60 seconds. Saving...
cvat_redis_inmem                 | 1:M 04 Feb 2024 20:27:35.764 * Background saving started by pid 41
cvat_redis_inmem                 | 41:C 04 Feb 2024 20:27:35.769 * DB saved on disk
cvat_redis_inmem                 | 41:C 04 Feb 2024 20:27:35.769 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
cvat_redis_inmem                 | 1:M 04 Feb 2024 20:27:35.865 * Background saving terminated with success
cvat_server                      | 2024-02-04 20:27:37,725 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "GET /api/tasks/113/status HTTP/1.0" 200 OK
cvat_server                      |
traefik                          | {"ClientAddr":"<IP_ADDRESS>:36248","ClientHost":"<IP_ADDRESS>","ClientPort":"36248","ClientUsername":"-","DownstreamContentSize":846,"DownstreamStatus":200,"Duration":33580691,"OriginContentSize":846,"OriginDuration":33531176,"OriginStatus":200,"Overhead":49515,"RequestAddr":"<IP_ADDRESS>:8080","RequestContentSize":0,"RequestCount":283,"RequestHost":"<IP_ADDRESS>","RequestMethod":"GET","RequestPath":"/api/tasks/113/status","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"<IP_ADDRESS>:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"<IP_ADDRESS>:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2024-02-04T20:27:37.692386328Z","StartUTC":"2024-02-04T20:27:37.692386328Z","entryPointName":"web","level":"info","msg":"","time":"2024-02-04T20:27:37Z"}
cvat_server                      | 2024-02-04 20:27:44,743 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server                      |
cvat_server                      | 2024-02-04 20:27:55,633 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server                      |
cvat_server                      | 2024-02-04 20:28:08,369 DEBG 'uvicorn-1' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server                      |
cvat_db                          | 2024-02-04 20:28:09.155 UTC [22] LOG:  checkpoint starting: time
cvat_db                          | 2024-02-04 20:28:13.981 UTC [22] LOG:  checkpoint complete: wrote 52 buffers (0.3%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.814 s, sync=0.005 s, total=4.826 s; sync files=47, longest=0.003 s, average=0.001 s; distance=100 kB, estimate=211 kB
cvat_grafana                     | logger=cleanup t=2024-02-04T20:28:14.994597339Z level=info msg="Completed cleanup jobs" duration=19.019678ms
cvat_grafana                     | logger=grafana.update.checker t=2024-02-04T20:28:15.01149483Z level=info msg="Update check succeeded" duration=12.737237ms
cvat_grafana                     | logger=plugins.update.checker t=2024-02-04T20:28:15.066506865Z level=info msg="Update check succeeded" duration=37.658164ms
cvat_server                      | 2024-02-04 20:28:16,383 DEBG 'uvicorn-0' stdout output:
cvat_server                      | INFO:     <IP_ADDRESS>:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server                      |

Hello,
From local you may only upload flat list of files.
If you need hierarchy structure, zip this structure and upload ZIP archive.


stream discard

Howdy,
could you add https://ffmpeg.org/doxygen/7.0/structAVStream.html#a492fcecc45dbbd8da51edd0124e9dd30 i dont want process streams that i dont need =)

It was quick so I've added it in master. Can you confirm it fixes your issue?

Thanks for the kind words ❤
Yeah please open a new issue 👍


Enum field is causing @AllArgsConstructor to not create a constructor.

For example,
@AllArgsConstructor
public class Foo {
    EnumValue value;
}

Causes this error
java: constructor Foo in class Foo cannot be applied to given types;
    required: noarguments
    found: EnumValue
    reason: actual and formal argument lists differ in length

However when manually creating the constructor it works fine.
I'm using version 0.11.16 of the lombok plugin and version 1.16.8 of lombok.

Turns out I didn't have annotation processing enabled in IntelliJ.


Remove rain.re-frame/event macro

This can be removed by enhancing the Hiccup parser on the server to ignore functions as attribute values.

Fixed in v0.1.5


Bug when run test_embedding_construction.py to initialize vocab_model

import torch

from ...modules.graph_construction.embedding_construction import EmbeddingConstruction
from ...modules.utils.padding_utils import pad_2d_vals_no_size
from ...modules.utils.vocab_utils import VocabModel

if __name__ == "__main__":
    raw_text_data = [["I like nlp.", "Same here!"], ["I like graph.", "Same here!"]]

    vocab_model = VocabModel(
        raw_text_data, max_word_vocab_size=None, min_word_vocab_freq=1, word_emb_size=300
    )

    src_text_seq = list(zip(*raw_text_data))[0]
    src_idx_seq = [vocab_model.word_vocab.to_index_sequence(each) for each in src_text_seq]
    src_len = torch.LongTensor([len(each) for each in src_idx_seq])
    num_seq = torch.LongTensor([len(src_len)])
    input_tensor = torch.LongTensor(pad_2d_vals_no_size(src_idx_seq))
    print("input_tensor: {}".format(input_tensor.shape))

    emb_constructor = EmbeddingConstruction(vocab_model.word_vocab, "w2v", "bilstm", "bilstm", 128)
    emb = emb_constructor(input_tensor, src_len, num_seq)
    print("emb: {}".format(emb.shape))

When run the above codes, a bug occurs. "AttributeError: 'list' object has no 'extract' attribute".  It seems that when initialize a VocabMode class, the init function run the instance.extract() where the instance is a list of str as illustrated in your comment. However, the list has not the 'extract' attribute. Could you please fix the bug, or please provide another example to initialize a VocabMode with several simple sentecnes.

I apologize for the late reply since we have been busy during the past two months.
This is due to the inconsistency of the doc. The variable data_set should be the list of DataItem. We will fix it ASAP in the next coming release.


[Feature Request] Convert rem|em values to pixel values

I have a use case for a function that transform relative (em, rem) units into pixel values. Basically the opposite of what the em and rem functions do.
The reason behind this is that I use rem units throughout my styles for consistent UI component design, and I came across an issue where SVG elements sized with rem units don't work across all browsers.
As an example, Chrome renders ok with rem units:

But Firefox doesn't :

They do however seem to work with em units, but since that's dependent on whatever context the component is used in, it does not seem solid for my use case.
The API could look something like this:
import { px } from 'polished'

const valueInPx = px('3rem')

If it seems like a good idea, I could give a try to implementing it. Suggestions for naming and API are welcome !

Why was this closed? I just hit a bug where rem values aren't working for height and width attributes so I need to convert the rem value to px to solve. I looked at the polished docs for that helper and saw the rem helper but no px helper and ended up here.

I have similar needs

I have similar needs

Reopening for evaluation...

Reopening for evaluation...

We've released v4.1.0 which includes a remToPx helper for converting rem values to px. This was the primary use case we saw the most support for.
We will continue to monitor future feature requests for a helper that covers em or other values. We will be closing this issue permanently, so please create a new request if this doesn't cover your use case.

We've released v4.1.0 which includes a remToPx helper for converting rem values to px. This was the primary use case we saw the most support for.
We will continue to monitor future feature requests for a helper that covers em or other values. We will be closing this issue permanently, so please create a new request if this doesn't cover your use case.


Compute the necessarry inversions of the witness in batch after it's creation

Follow-up of https://github.com/o1-labs/proof-systems/pull/2047
but we proceed differently.
The scratch state now contains field element which are marked to be inversed or not, using a variant.
The Batch inversion is then done when creating the ProofInput.
We first create an intermediary type NotInversedProofInput, containing the same variant as the scratch state, and implement a conversion to ProofInput  which does the batch inversion

Can we close this @marcbeunardeau88?


Format MDX blocks in markdown files

Formatting ocaml code blocks in .md files. This PR requires https://github.com/ocaml-ppx/ocamlformat/pull/1941

Replaced by https://github.com/gpetiot/ocamlformat-mdx-blocks/pull/1


Can Atom have a custom right-click context menu?

When right clicking in Atom, I can't help to notice that the popup is my system's attempt at a context menu.
I think it would be interesting if we used a custom right-click context menu in the document, or even a frameless BrowserWindow.  This way its more accessible as a piece of the software.  For things like themes and packages to customize and use.

+1

+1 . If someone point me where to start, I can try develop this feature.

If someone point me where to start, I can try develop this feature.

First you'll probably need to see whether or not Electron supports custom context menus.  If it does, then it's just a matter of using those APIs within Atom.


Fix several issues under disagg mode

What problem does this PR solve?
Issue Number: ref https://github.com/pingcap/tiflash/issues/6827
Problem Summary:
Fix some bugs introduced by

https://github.com/pingcap/tiflash/pull/7047
https://github.com/pingcap/tiflash/pull/7050
If some S3 clients are release after Aws::ShutdownAPI(aws_options); they will make crash. So don't keep long live S3 clients as class member

Workaround a tiup bug

https://github.com/pingcap/tiup/issues/2139

What is changed and how it works?

Config default values changed:

Inferr storage.format_version to be 5 when S3 is enabled and user don't set it explicitly
Add check for storage.format_version must be 5 when explicitly set when S3 is enabled
Change flash.use_autoscaler default to be false

Use 1 PiB as store capacity in PD when S3 is enabled
Make BackgroundInitStores exception safe
Fix lifecycle rule settings, report warning when the rule is not defined

Creating a new rule by AWS SDK is always fail, don't know why

Make AWS SDK info logging as debug level
Disagg Compute node still need to init ps_write because it needs to start proxy
Disable RegionPersister and its background task on disagg compute node
Disable ReadIndexRunner on disagg compute node
Remove long-living S3 clients, they will cause crashes if they are released later than S3::ClientFactory ends
Reduce some verbose logging in ProxyFFIStatusService SchemaSyncService

Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/build

/run-integration-test

/merge

/hold

/unhold

/merge

/run-integration-test

waiting for https://ci.pingcap.net/blue/organizations/jenkins/tiflash-ghpr-integration-tests/detail/tiflash-ghpr-integration-tests/11807/pipeline/85


bootstrap.sh report error

./bootstrap.sh
./bootstrap.sh: line 199: ./tools/build/src/engine/build.sh: No such file or directory
Building B2 engine..
./bootstrap.sh: line 229: ./tools/build/src/engine/build.sh: No such file or directory
Failed to build B2 build engine
mac.

I just looked in the 1.77.0 release.  That file is present.
I checked the 1.76.0 release.  It's there too.
Where did you get your boost installation? (and what version, etc).

I reproduced the same issue.
Cloned repository.
➜  boost git:(master) git describe
boost-1.71.0.beta1-1637-g164ae2115
./bootstrap.sh: line 199: ./tools/build/src/engine/build.sh: No such file or directory
Building B2 engine..
./bootstrap.sh: line 229: ./tools/build/src/engine/build.sh: No such file or directory
Failed to build B2 build engine

➜  boost git:(master) sw_vers
ProductName:	macOS
ProductVersion:	11.5.2
BuildVersion:	20G95
If I sort it out, I will post back here.

The solution for me was to read the instructions :(
It's possible we did the same thing.
You need to recursive clone before running the bootstrap.
Found "Install from Git" instructions here:
https://github.com/boostorg/wiki/wiki/Getting-Started%3A-Overview
@runner365 , give it a go and close the issue if this solves the problem. I may be a good idea to check for this missing file and recommend the --recursive flag on the clone, or a printout which points to the instructions for those of us who never read instructions to begin with :D

This seems to happen on the releases provided on GitHub (for example https://github.com/boostorg/boost/archive/refs/tags/boost-1.77.0.tar.gz ), but is not an issue for the releases on JFrog, for example https://boostorg.jfrog.io/ui/native/main/release/1.77.0/source/

Links from https://www.boost.org/users/history/ now point to https://boostorg.jfrog.io/artifactory/main/release/1.77.1/source. For example: https://boostorg.jfrog.io/artifactory/main/release/1.77.0/source/boost_1_77_0.tar.gz


[integration] Botkube Integration

Tool / Product Name
Botkube
Goal
Hi all! The Botkube team has prepared a Botkube x Keptn Plug-in that uses web hook service. A step by step walk through can be seen here: https://botkube.io/blog/implementing-your-own-botkube-plugin-a-real-life-use-case @huseyinbabal

#help-integrations on Slack

[X] Yes

No Existing Integrations

[X] Yes

Assisted
Yes
Any other info?
Keptn plug in is already made

Since Keptn V1 has reached EOL on December 22nd, 2023, this issue has been closed.


Eliminate cred helper log spam

This picks up https://github.com/google/go-containerregistry/pull/1284 to eliminate pkg/v1/google.Keychain spam, and https://github.com/awslabs/amazon-ecr-credential-helper/pull/309 (not yet merged) to allow callers to eliminate ecr-login spam, which ko now does.

docker: Error response from daemon: hcsshim::CreateComputeSystem b0e1e7d0da21f466bb9430d7e75566c3792e551b324ae8c8d1f1e45bc739b813: The container operating system does not match the host operating system.

docker: Error response from daemon: hcsshim::CreateComputeSystem b0e1e7d0da21f466bb9430d7e75566c3792e551b324ae8c8d1f1e45bc739b813: The container operating system does not match the host operating system.

Fixed by https://github.com/google/ko/pull/599, I'll rebase


Add support for "classifier" dependency substitution

Add support for dependency substitution with/without classifiers.

@bigdaz I'd be curious to read your sentiment about this and the approach I'm using.
@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

OK, I've already cancelled the old build and triggered a new ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

Release notes will be added once branching for 6.5 happens.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.


More than one file was found with OS independent path libflutter.so'

Use case
In my project we have ios and android native project. But after heard about flutter we decide that new functionality will develop into flutter module ( flutter create -t module project_name ) and finish work on flutter, we export .aar file from flutter project.
Now go to my android project and import flutter exported .aar file in android project as module or lib. for reference  see the image

After import we analyze .aar just like .apk file and here are image for reference

How can we call dart route page from Android app
`
@Override
protected void onCreate(Bundle savedInstanceState){
Flutter.startInitialization(getApplicationContext());
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
DisplayMetrics displayMetrics=new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(displayMetrics);
height=displayMetrics.heightPixels;
width=displayMetrics.widthPixels;
    View flutterView=Flutter.createView(
    MainActivity.this,
    getLifecycle(),
    "/"
    );
    FrameLayout.LayoutParams layout=new FrameLayout.LayoutParams(width,height);

    addContentView(flutterView,layout);
    }`

Call this code from my android activity ..
and we are successfully able to load flutter view on android activity
and for android we export app.framwork and flutter.framwork , ios app also able to load same view as like android(save time of development same thing in different lang or on difference platform )
issue
Now we create other module in flutter and import same as last time. during the import .aar we can change name of module using android studio. Here are we attaching image reference.

Now you can see the .so file in second .aar file with the same name of last one .aar file..
when we run this it gives error
More than one file was found with OS independent path libflutter.so'
Proposal

Please suggest me way that we can resolve this and make it easy

@blasten

The Flutter tool (v1.12.13 now builds AARs out of the box. If you run flutter build aar in the module directory, you will get the .aar artifacts. You can read more on https://flutter.dev/docs/development/add-to-app/android/project-setup#option-a---depend-on-the-android-archive-aar.
Feel free to file a new issue if you encounter a problem.


Admin Command issues

The issue tracker is ONLY used for bug report and feature request. Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our mailing lists. We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
BUG REPORT

Please describe the issue you observed:

What did you do (The steps to reproduce)?
Trying to prepare open source user guide, Run clusterRT/checkMsgSendRT/updateBrokerConfig found issue.

What did you expect to see?
clusterRT/checkMsgSendRT should see RT when amount is 1
updateBrokerConfig should see update messageDelayLevel normally

What did you see instead?
clusterRT/checkMsgSendRT when amount is 1 return NaN
updateBrokerConfig messageDelayLevel change to "1s" ,the value after first blank is missing

Please tell us about your environment:
Linux Centos

Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):

The first problem "clusterRT/checkMsgSendRT when amount is 1 return NaN" is indeed a bug, when amount is 1. I suggest you can propose a PR to fix it.
But i can't understand your second question, can you provide more details?

ps. I think -n ip:port only support one namesrv addr, ";" in Linux Command Line is the end of command, the description should update

ps. I read the code, the reason to drop the first request is reasonable, not to calculate connection time, but I will try a better way.

I tried command ./mqadmin updateBrokerConfig -k messageDelayLevel -v "1s 2s", found it can work normally.

@ymwneu you are right, my version is 4.2.0


[FEATURE REQ] [Identity] Support cancel in device code login

We are using identity in intellij plugin, with cancelable button in device code dialog, in current implementation, the only way to shutdown then endless loop inside device code workflow is calling shutdownNow to the executorService, but will result in a not friendly system error:

2021-04-15 08:48:00,763 [ 155502]  ERROR - actor.core.publisher.Operators - Operator called default onErrorDropped
reactor.core.Exceptions$ErrorCallbackNotImplemented: com.azure.core.exception.ClientAuthenticationException: Failed to acquire token with device code
Caused by: com.azure.core.exception.ClientAuthenticationException: Failed to acquire token with device code
at com.azure.identity.implementation.IdentityClient.lambda$authenticateWithDeviceCode$23(IdentityClient.java:606)
at reactor.core.publisher.Mono.lambda$onErrorMap$30(Mono.java:3384)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
at reactor.core.publisher.MonoCompletionStage.lambda$subscribe$0(MonoCompletionStage.java:76)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.InterruptedException: sleep interrupted
at java.base/java.lang.Thread.sleep(Native Method)
at java.base/java.lang.Thread.sleep(Thread.java:339)
at java.base/java.util.concurrent.TimeUnit.sleep(TimeUnit.java:446)
at com.microsoft.aad.msal4j.AcquireTokenByDeviceCodeFlowSupplier.acquireTokenWithDeviceCode(AcquireTokenByDeviceCodeFlowSupplier.java:62)
at com.microsoft.aad.msal4j.AcquireTokenByDeviceCodeFlowSupplier.execute(AcquireTokenByDeviceCodeFlowSupplier.java:27)
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:59)
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:17)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
... 3 more

@andxu
To cancel the authentication you can use this flow:

// Subscribe asynchronously on the getToken output.
        Disposable deviceCodeSubscription = deviceCodeCredential.getToken(new TokenRequestContext().addScopes("scopes"))
                .subscribe(accessToken -> {
                    //logic to process accessToken;
                    System.out.println("accessToken");
                });

// When the user cancels the operation, you can dispose of the subscription to cancel the operation.
        deviceCodeSubscription.dispose();
```java

Let us know if the issue still persists or if further assistance is needed.


The downloaded file won't be truncated to correct size after sdk/storage/azblob:v1.2.0

Bug Report

import path of package in question, e.g. github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
SDK version e.g. v1.2.0

Specify the exact commit if possible: https://github.com/Azure/azure-sdk-for-go/pull/21259

output of go version: 1.20

What happened?
The downloaded file will not be truncated after method DownloadFile under azure-sdk-for-go/sdk/storage/azblob/blockblob/client.go is called. Therefore the md5 and size of the downloaded file does not match the original blob in container.
What did you expect or want to happen?
The file size and md5 should be the same between the downloaded file and the original one in the container.
How can we reproduce it?
Call the method DownloadFile and compare the sha/md5 and the size of the file.

Can you provide exact test case that you are doing. Why do you expect downloaded file to be truncated? downloaded file shall match exactly the one in container by default.

Can you also share the file-size in local and on download.

Can you provide exact test case that you are doing.

Test environment: Azure Storage Account with a test container.

Upload a dummy file, with size 8241066B to the container via Azure Portal
Download the file via Azure Portal to compare_directory
rename the downloaded file origin
Write a simple go script, which uses sdk/storage/azblob v1.1.0 to download that blob to compare_directory, by calling method blockblob.NewClientWithSharedKeyCredential(...).DownloadFile(...)
rename the downloaded file 1_1_0
do ls -la <compare_directory>

-rw-r--r--   1 user dialout 8241066 Nov 20 07:54 1_1_0
-rw-r--r--   1 user dialout 8241066 Nov 20 07:50 origin

They have the same size, everything works as expected.
7. Then use the same go script, but update the sdk/storage/azblob to v1.2.0, then call method blockblob.NewClientWithSharedKeyCredential(...).DownloadFile(...) to download the same blob
8. rename the downloaded file 1_2_0
9. do ls -la <compare_directory>
-rw-r--r--   1 user dialout 8241066 Nov 20 07:54 1_1_0
-rw-r--r--   1 user dialout 8388608 Nov 20 08:06 1_2_0
-rw-r--r--   1 user dialout 8241066 Nov 20 07:50 origin

As you can see the file downloaded by v1.2.0 has a different size.

Why do you expect downloaded file to be truncated? downloaded file shall match exactly the one in container by default.
As I have seen in the algorithm:

In the comment of method DownloadFile inblockblob/client.go, it is announced that The file would be truncated if the size doesn't match.
In DownloadFileOptions struct in blob/models.go, there is one option called BlockSize. My assumption is that the algorithm will first allocate blocks and then download the content into blocks in parallel. Since the file size normally will not match num_of_blocks times size_of_block, in the end the algorithm needs to truncate the file to the original size.
There seems to be a truncate algorithm in blob/client.go, but does not work in the new version v1.2.0.

@Malsourie, thanks for reporting this issue. The fix has been merged and will be part of the next release.


My  index page doesn't look like your demo How can I change it

My index page doesn't look like your demo
How can I change it
http://hexotest.computerlab.io/

Also with the same problem(Help...)


Fix CLI invocations in first_profile.md

This PR fixes two minor invocation errors in the documentation.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Adolfo Garcia Veytia (puerco) seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


the performance of  v0.2.2 is very very very terrible

http://fatidol.com/phy-benchmark/planck-test.html
it's worse than  the old version

Sorry to hear that. Do you know which version you used before v0.2.2?

@zOadT https://github.com/shakiba/planck.js/blob/master/lib/collision/AABB.js#L93-L115
Need to optimize

I got it, I will fix it.

I reverted that change, can you please check it out again.

It's still slow than old version( v0.2.1 ).

@finscn Thanks for sharing your performance reviews. I used you example code with both v0.1 and v0.2.3, I can't see the difference.
Also please note that drawing shapes on canvas is an expensive process. I use stage.js in testbed, it draws shapes to textures and then reuse textures to avoid redrawing shapes in each frame.

@finscn Can I add your test to examples in this project, or would you like to make PR for that?

It seems that minified file of version 0.2+ is actually slow!


Changing base architecture for SSD300

Hi,
I am trying to use a different architecture than VGG16. I am planning to use Mobilenets as base architecture. Do I have to add the "Extra Feature Layer" on top of Mobilenets as the SSD paper does for VGG16 or how does one proceed by changing the base architecture?
Any suggestions will be greatly appreciated.
Thanks!

If by "extra feature layers" you mean what is conv6 to conv9 in SSD300, then no, you don't have to add those to MobileNet. You can add extra convolutional layers though. It's your design choice, really. The point is that you choose your base network, then you decide from which layers of the base network you want to make predictions, and then you add predictor layers as described in keras_ssd7.py on top of those respective layers in the base network.
Whether or not you want to add additional layers on top of MobileNet also depends on what spatial resolution you want for your last predictor layer(s). For example, the extra feature layers on top of the VGG16 in SSD300 result in the output feature map of conv9_2 having spatial dimensions exactly 1x1, so the last predictor layer predicts only one large box spatially. You may or may not want to create the same design for your MobileNet SSD. As I said, that's your design choice.

Thank you!. That clarified my question.
About the "extra feature layers" , yes, I meant the convolutional layers conv6 to conv11. I will add the predictor layers for some of the base network layers.
I have two more questions though.
1.) Do I also have to use pre-trained weights for Mobilenets too as you do for VGG16?
2.) How can I compute the mean average precision for the detection problem?  Is there any keras function or similar? Say that I want to compute mAP for the examples that you have on your tutorial.

I mean, is there any reason not to use pre-trained MobileNet weights? They are right there in keras.Applications, you only need to download them. If MobileNet can be trained end to end from scratch (i.e without any pre-trained weights), then a MobileNet SSD can probably be trained end to end from scratch, too. If not, then probably not, although it cannot be said with certainty, you would just have to try.
There is no universal definition of the mAP. It's a much more complex metric than something like classification accuracy and there are several variations of it. The authors of the Pascal VOC competitions used one formula for their 2007 competition, and then a slightly different formula for their 2012 competition. In the MS COCO competitions, yet another formula is used. They are all similar, but not identical. So if you want to compute the mAP, you have to decide what version of it you want to compute first. If you want to compute the MS COCO-style mAP, then you can use the pycoco tools that come with COCO. If you want to compute the Pascal VOC-style mAP, then you can either use their official Matlab scripts or find some Python implementation somewhere if you can. Or write your own, of course. In any case you can always use the evaluation notebooks in this repo to produce the prediction output in the format that is required by the aforementioned mAP calculators.


Issues reading vtu files generated by amrvac

I ran into two issues while trying to read vtu files generated by amrvac.

With the vtu file vaclogo0000.vtu.zip, if I run

meshio-info vaclogo0000.vtu

There will be an error:
Traceback (most recent call last):
  File "/home/xlu/.local/lib/python3.7/site-packages/meshio/vtu/_vtu.py", line 290, in __init__
    tree = ET.parse(str(filename), parser)
  File "/usr/local/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
    tree.parse(source, parser)
  File "/usr/local/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 604, in parse
    parser.feed(data)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 461, column 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xlu/.local/bin/meshio-info", line 8, in <module>
    sys.exit(info())
  File "/home/xlu/.local/lib/python3.7/site-packages/meshio/_cli/_info.py", line 15, in info
    mesh = read(args.infile, file_format=args.input_format)
  File "/home/xlu/.local/lib/python3.7/site-packages/meshio/_helpers.py", line 69, in read
    return reader_map[file_format](filename)
  File "/home/xlu/.local/lib/python3.7/site-packages/meshio/vtu/_vtu.py", line 554, in read
    reader = VtuReader(filename)
  File "/home/xlu/.local/lib/python3.7/site-packages/meshio/vtu/_vtu.py", line 293, in __init__
    root = _parse_raw_binary(str(filename))
  File "/home/xlu/.local/lib/python3.7/site-packages/meshio/vtu/_vtu.py", line 252, in _parse_raw_binary
    da_tag.set("offset", "%d" % len(arrays))
AttributeError: 'NoneType' object has no attribute 'set'

I am able to trace the issue to this line:
https://github.com/nschloe/meshio/blob/e7629defb191979925305df3fdf2ba6c19485448/src/meshio/vtu/_vtu.py#L251
The problem is that the generated vtu file has spaces in offset attribute:
<DataArray type="Float32" Name="rho" format="appended" offset="               0">

Removing the spaces will fix the error.
So, is this format standard? Should either side change their code, meshio or amrvac?

After removing the spaces that cause the first error, if I run

meshio-info vaclogo0000.nospaces.vtu

There will be another error:
Traceback (most recent call last):
  File "/home/oliver/.local/bin/meshio-info", line 8, in <module>
    sys.exit(info())
  File "/home/oliver/.local/lib/python3.9/site-packages/meshio/_cli/_info.py", line 15, in info
    mesh = read(args.infile, file_format=args.input_format)
  File "/home/oliver/.local/lib/python3.9/site-packages/meshio/_helpers.py", line 69, in read
    return reader_map[file_format](filename)
  File "/home/oliver/.local/lib/python3.9/site-packages/meshio/vtu/_vtu.py", line 555, in read
    reader = VtuReader(filename)
  File "/home/oliver/.local/lib/python3.9/site-packages/meshio/vtu/_vtu.py", line 369, in __init__
    pts = self.read_data(data_array)
  File "/home/oliver/.local/lib/python3.9/site-packages/meshio/vtu/_vtu.py", line 550, in read_data
    data = data.reshape(-1, int(c.attrib["NumberOfComponents"]))
ValueError: cannot reshape array of size 100 into shape (3)

I added a print command print(data, data.shape, c.attrib) before the reshape command (line 550) and it prints the following just before the error
[0.5 0.5 0.5 2.  2.  2.  2.  2.  2.  2.  0.5 0.5 0.5 2.  2.  2.  2.  2.
 2.  2.  0.5 0.5 2.  2.  2.  2.  2.  2.  2.  2.  0.5 0.5 2.  2.  2.  2.
 2.  2.  2.  2.  0.5 0.5 2.  2.  2.  2.  2.  2.  2.  2.  0.5 0.5 2.  2.
 2.  2.  2.  2.  2.  2.  0.5 2.  2.  2.  2.  2.  2.  2.  2.  2.  0.5 2.
 2.  2.  2.  2.  2.  2.  2.  2.  0.5 2.  2.  2.  2.  2.  2.  2.  2.  2.
 2.  2.  2.  2.  2.  2.  2.  2.  2.  2. ] (100,) {'type': 'Float32', 'NumberOfComponents': '3', 'format': 'appended', 'offset': '427932'}

Everything before that does not have any errors.
Is this a parsing error on the meshio side, or a data exporting error of amrvac?

Interesting. I added better error messages in #1136, but the spacing issue cannot easily be resolved due to a shortcoming in Python's own XML parser. I haven't had time to look at the other error yet.

It's all right. I'll look more into it, and ask the devs of amrvac as well.

Hello, I'm having the same issue (ValueError: cannot reshape array of size 106 into shape (3)) when reading some vtu files created by elmerfem. Oddly enough it only happens occasionally, The file opens in paraview so I belive that the error is in meshio.
Here are two files, on that work and on that doesn't (with slightly different mesh sizes), both exported from elmerfem:
meshio_vtu_error.zip

@brizzer The file is in fact corrupt; the Number Of Components in strain_yy is wrong, and the data for displacement is wrong, too. I've added a better warning/error message.

@OliverLew Have the amrvac devs fix this yet?

Probably not, I'll try again at some time. Also, I was only using these for class, now I have not paid attention for a while.
Revisiting the posts, I think the problem should be on amrvac's side, e.g. the export format and data array length. If you were cleaning up issues, maybe this one can be closed (should you want to).


update runc binary to 1.2.0; apply runc PR  4473

https://github.com/opencontainers/runc/releases/tag/v1.2.0

CI is failing for Fedora 40 and Rocky 9
    default: === RUN   TestPodUserNS
    default: === RUN   TestPodUserNS/userns_uid_mapping
    default:     pod_userns_linux_test.go:246: Create a sandbox with userns
    default: E1022 10:38:44.240499   45870 remote_runtime.go:132] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to start sandbox "ed8348b9215a10dba3ef48191f37dfa41c7a4648bbdf7fba9365fdf8a4c1ed4e": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/ed8348b9215a10dba3ef48191f37dfa41c7a4648bbdf7fba9365fdf8a4c1ed4e/rootfs/dev/mqueue: operation not permitted
    default:     pod_userns_linux_test.go:251: Unexpected RunPodSandbox error: rpc error: code = Unknown desc = failed to start sandbox "ed8348b9215a10dba3ef48191f37dfa41c7a4648bbdf7fba9365fdf8a4c1ed4e": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/ed8348b9215a10dba3ef48191f37dfa41c7a4648bbdf7fba9365fdf8a4c1ed4e/rootfs/dev/mqueue: operation not permitted
    default: === RUN   TestPodUserNS/userns_gid_mapping
    default:     pod_userns_linux_test.go:246: Create a sandbox with userns
    default:     pod_userns_linux_test.go:251: Unexpected RunPodSandbox error: rpc error: code = Unknown desc = failed to start sandbox "d89053afdbbc20f3b11b2eae107e3d70213b21f473707dd5baa762d1a317aa3c": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/d89053afdbbc20f3b11b2eae107e3d70213b21f473707dd5baa762d1a317aa3c/rootfs/dev/mqueue: operation not permitted
    default: === RUN   TestPodUserNS/rootfs_permissions
    default:     pod_userns_linux_test.go:246: Create a sandbox with userns
    default: E1022 10:38:44.623562   45870 remote_runtime.go:132] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to start sandbox "d89053afdbbc20f3b11b2eae107e3d70213b21f473707dd5baa762d1a317aa3c": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/d89053afdbbc20f3b11b2eae107e3d70213b21f473707dd5baa762d1a317aa3c/rootfs/dev/mqueue: operation not permitted
    default:     pod_userns_linux_test.go:251: Unexpected RunPodSandbox error: rpc error: code = Unknown desc = failed to start sandbox "83bda990b49619f5e98b41dd6fa5c6178264677bd3a2735debb66fb114ce0859": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/83bda990b49619f5e98b41dd6fa5c6178264677bd3a2735debb66fb114ce0859/rootfs/dev/mqueue: operation not permitted
    default: === RUN   TestPodUserNS/volumes_permissions
    default: E1022 10:38:44.971328   45870 remote_runtime.go:132] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to start sandbox "83bda990b49619f5e98b41dd6fa5c6178264677bd3a2735debb66fb114ce0859": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/83bda990b49619f5e98b41dd6fa5c6178264677bd3a2735debb66fb114ce0859/rootfs/dev/mqueue: operation not permitted
    default:     pod_userns_linux_test.go:246: Create a sandbox with userns
    default:     pod_userns_linux_test.go:251: Unexpected RunPodSandbox error: rpc error: code = Unknown desc = failed to start sandbox "e579723f7f6ece7cc7e6c5294fa73308777f15baacd3f0f317225c0911b9c01b": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/e579723f7f6ece7cc7e6c5294fa73308777f15baacd3f0f317225c0911b9c01b/rootfs/dev/mqueue: operation not permitted
    default: === RUN   TestPodUserNS/fails_with_several_mappings
    default:     pod_userns_linux_test.go:246: Create a sandbox with userns
    default: E1022 10:38:45.379638   45870 remote_runtime.go:132] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to start sandbox "e579723f7f6ece7cc7e6c5294fa73308777f15baacd3f0f317225c0911b9c01b": failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "mqueue" to rootfs at "/dev/mqueue": setxattr /run/containerd-test/io.containerd.runtime.v2.task/k8s.io/e579723f7f6ece7cc7e6c5294fa73308777f15baacd3f0f317225c0911b9c01b/rootfs/dev/mqueue: operation not permitted
    default: E1022 10:38:45.401499   45870 remote_runtime.go:132] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to create network namespace for sandbox "461d0d64ea29a2c2b36262ad005d0ebaed8f1ea1d969f6944b575165caebc8a2": required only one uid mapping, but got 2 uid mapping(s)
    default: --- FAIL: TestPodUserNS (1.51s)
    default:     --- FAIL: TestPodUserNS/userns_uid_mapping (0.35s)
    default:     --- FAIL: TestPodUserNS/userns_gid_mapping (0.38s)
    default:     --- FAIL: TestPodUserNS/rootfs_permissions (0.35s)
    default:     --- FAIL: TestPodUserNS/volumes_permissions (0.41s)
    default:     --- PASS: TestPodUserNS/fails_with_several_mappings (0.02s)

https://github.com/containerd/containerd/actions/runs/11457221604/job/31880030218?pr=10877

@AkihiroSuda It might be a regression from this PR: #10607. See this issue, which is already fixed, that was a regression too: https://github.com/containerd/containerd/issues/10847.
Maybe you can try to revert #10741 and #10607 in this PR, just to see if that fixes the issue on CI?

/retest

Given that the CI is passing for crun, it could be also argued that this is an issue on runc's side?

It looks like there is bug in runc side...

It looks like there is bug in runc side...

Opened an issue in runc repo:

https://github.com/opencontainers/runc/issues/4466

runc creates the userns and then the other namespaces to avoid a bug with selinux and mqueue in the kernel. See here: https://github.com/opencontainers/runc/blob/e37371ebd791114e4f23444ab1fbcae87ac6b928/libcontainer/nsenter/nsexec.c#L866-L871:
		 * A specific case of this is that the SELinux label of the
		 * internal kern-mount that mqueue uses will be incorrect if the
		 * UTS namespace is cloned before the USER namespace is mapped.
		 * I've also heard of similar problems with the network namespace
		 * in some scenarios. This also mirrors how LXC deals with this
		 * problem.

It seems creating the namespaces in two steps, as runc is doing it, might be a way forward.
@AkihiroSuda are you sure that crun is tested on vagrant here in containerd? Because only vagrant is failing in this PR, as far as I can see
@fuweid any thoughts? :)

are you sure that crun is tested on vagrant here in containerd? B

Sorry, I was wrong 😅

assumming the issue is what I mentioned in the last comment, it doesn't seem simple to fix. It is quite limited what we can do in go. Currently we create a process with the userns and the netns here: https://github.com/containerd/containerd/blob/main/pkg/sys/unshare_linux.go#L48-L63
We can't easily run something inside that process, because we are using /proc/self/exe as the command to exec (but with ptrace so we don't really exec it). So doing first the unshare for userns and then for the netns doesn't seem like a 1 line fix, at least.
@fuweid any thoughts or ideas? It would be great to first validate my theory of what is the bug, though.

runc creates the userns and then the other namespaces to avoid a bug with selinux and mqueue in the kernel, it seems to be what we hit here. See here: https://github.com/opencontainers/runc/blob/e37371ebd791114e4f23444ab1fbcae87ac6b928/libcontainer/nsenter/nsexec.c#L866-L871:
		 * A specific case of this is that the SELinux label of the
		 * internal kern-mount that mqueue uses will be incorrect if the
		 * UTS namespace is cloned before the USER namespace is mapped.
		 * I've also heard of similar problems with the network namespace
		 * in some scenarios. This also mirrors how LXC deals with this
		 * problem.

It seems creating the namespaces in two steps, as runc is doing it, might be a way forward.
@AkihiroSuda are you sure that crun is tested on vagrant here in containerd? Because only vagrant is failing in this PR, as far as I can see
@fuweid any thoughts? :)

Thanks for the information. I will take a look in this week. We can rolllback first and create a tracking issue for this.
I want to keep ptrace function because of https://github.com/golang/go/blob/69234ded30614a471c35cef5d87b0e0d3c136cd9/src/runtime/proc.go#L4837

I actually ran the test with crun + SELinux, and it seems green.
https://github.com/containerd/containerd/pull/10883

Testing this runc PR in CI:

https://github.com/opencontainers/runc/pull/4473

Thanks @lifubang

/ok-to-test

Failure on ARM
=== FAIL: integration/client TestContainerPids (0.16s)
    log_hook.go:47: time="2024-10-25T06:44:04.760946146Z" level=debug msg="remote introspection plugin filters" func="proxy.(*introspectionRemote).Plugins" file="/home/runner/work/containerd/containerd/core/introspection/proxy/remote.go:59" filters="[type==io.containerd.snapshotter.v1, id==overlayfs]" testcase=TestContainerPids
    container_test.go:598: expected 2 process but received 1

Probably unrelated

Failure on ARM
=== FAIL: integration/client TestContainerPids (0.16s)
    log_hook.go:47: time="2024-10-25T06:44:04.760946146Z" level=debug msg="remote introspection plugin filters" func="proxy.(*introspectionRemote).Plugins" file="/home/runner/work/containerd/containerd/core/introspection/proxy/remote.go:59" filters="[type==io.containerd.snapshotter.v1, id==overlayfs]" testcase=TestContainerPids
    container_test.go:598: expected 2 process but received 1

Probably unrelated

I think maybe there is a race in the test case here, when we list the pids in the container, the sleep process has not been started yet. Looks like this is similar to https://github.com/opencontainers/runc/issues/4437

/test pull-containerd-build
/test pull-containerd-node-e2e

The runc PR is now merged and also backported to runc 1.2.x branch, so I guess we need to wait for 1.2.1 to release and then we can update

Updated to v1.2.1

The TestContainerPids race is happening pretty frequently, are the changes in runc making the issue show up more?

/cherrypick release/1.7

/cherrypick release/1.6


Builder & mmap

A major motivation for using mmap'ed files is whe the data-set does not fit in the memory. Currently, the builder uses a lot of memory.
It would be great to have a Builder that will work under these conditions.

@alecthomas - sorry it is not so clear to me what you mean.  I have the same problem as @raichu where I run out of memory.

Would be nice to see how to build an index for the 35 GB of the Have I been Pwned database.


Adding dummy.txt to storage dirs

Fixing https://github.com/graphite-project/carbon/issues/711
Also, upgrade manual for next release should include instructions to remove all /opt/graphite/storage entries from /opt/graphite/lib/carbon-x.x.x-pyx.x.egg-info/installed-files.txt file before the upgrade.

Codecov Report

Merging #753 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #753   +/-   ##
=======================================
  Coverage   49.16%   49.16%
=======================================
  Files          36       36
  Lines        3305     3305
  Branches      471      471
=======================================
  Hits         1625     1625
  Misses       1580     1580
  Partials      100      100

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4c5a4cf...4147dfc. Read the comment docs.


fix(template) Proper parse exception for inconsistant variables

Signed-off-by: Jerome Simeon<EMAIL_ADDRESS>Changes

Traps low-level error for inconsistent variable in parser to return them as ParseException
Also passes ast node to template exception (unused for the time being, but consistent with the API)

Coverage increased (+0.04%) to 91.712% when pulling 430ed38cd5160cbf867e3bd6c136473ba43bd417 on jeromesimeon:js-fix-error-handling into d1bac653aab39967a149112690309aa5458183d4 on accordproject:master.


Valid connections are killed when testing idle connections

Pool ends up being empty after eviction (not counting busy connections), and valid connections are not re-used

Yeah , I was afraid that might be the case when you showed the problem to me yesterday. Merged. Thanks for investigation.


"Add a bottom app bar to the floating action button motion demo" regressed missed_frame_build_budget_count

"Add a bottom app bar to the floating action button motion demo" (https://github.com/flutter/flutter/pull/16196) regressed missed_frame_build_budget_count.
Please investigate the cause and resolve this issue. Thanks!
(Splitting this off from https://github.com/flutter/flutter/issues/16234)

I ran an experiment to open demo pages 10x after starting the app and recording the GPU ms/frame after doing this.  This was done in profile mode with the performance overlay turned on using a Nexus 6p.
Some results:
as a control, the bottom navigation demo got 17.5 ms/frame
The bab demo as-is got 36.6 ms/frame
The bab demo with bottom bar and fab removed got 24.3 ms/frame
The bab demo with just the bottom bar removed got 25.3 ms/frame
The bab demo with the bab in but its AnimatedCrossFade child removed got 25.8 ms/frame
It appears that the AnimatedCrossFade widget in the bottom app bar is what's causing the slowdown in this demo.
The simple fix to this problem is to just build one set of menu icons and not change it.  I produced that as part of my testing and it's available at #16389
The full fix would involve making the AnimatedCrossFade faster when it's rebuilding but not animating.

Wow, great research, thanks!
What on earth is AnimatedCrossFade doing that's so expensive?
(25ms is still a lot. 17ms is a lot. Ideally we'd be below 8ms, so that we get 60Hz with minimal latency, but at least below 16ms, so we get 60Hz with only one frame extra latency.)

From looking at the AnimatedCrossFade source, it looks like it is greedily creating multiple AnimationControllers and setting them up to run every time it's created.  Both build and initState on it are very busy methods.  The perf tests involve creating and destroying the widget's state very quickly, so we hit the worst case performance.
I think if the crossfade widget were to lazily initialize its internal animations, it would perform better.

Reopening so that we don't lose the information you discovered above — the point of the benchmark is to find this kind of problem, not to just paper over them by writing the gallery to work around these issues.

Unassigning this because I have other work to do and want to leave the cookie untouched.

Looks like #16389 didn't actually fix this regression.

Close this as this is too old and we're no longer tracking missed_frame_build_budget_count for our performance benchmarks. (Increasing missed_frame_build_budget_count could actually mean an improvement instead of a regression: https://github.com/flutter/flutter/issues/18727.)


HDDS-6459. EC: [Refactor-1] Check isFullCell inside handleDataWrite

What changes were proposed in this pull request?
To make review easier, let's break down HDDS-6400 (#3151).
This is the 1st patch of the series.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6459
How was this patch tested?
Existing CI should cover.

@umamaheswararao, please take a look, thanks.

Thanks @umamaheswararao for the review.


wasm-bindgen-futures: Use single-threaded implementation when multi-threading is impossible

I encountered this when using the atomics target feature while running in a shared worker in Firefox, where cross-origin isolation is impossible and therefor sharing a SharedArrayBuffer with a dedicated worker is not available, which was the workaround used for Firefox not implementing Atomics.waitAsync().
The solution is to use the single-threaded implementation of wasm-bindgen-futures when we detect that multi-threading is impossible.
Note: I somehow managed to accidentally push this commit onto the main branch ... which I promptly reverted and added a branch protection so this doesn't happen again.

This causes a problem because the single-threaded runtime always runs tasks on the current thread, meaning that if the task started on another 'thread' it's now technically been sent to a different thread despite not necessarily being Send, which is unsound.

How could you sent it to another thread though? I mean its true that in this scenario its using the same SharedArrayBuffer, but how is access from another achieved in such a case?

At first I thought this wasn't an issue in practice because the wrapper JS only lets you instantiate the wasm module once, except that you can create more than one copy of the JS wrapper using e.g. the ?i=1 trick and then instantiate both of them with the same SharedArrayBuffer.

That sounds terribly unsound to me in the first place ... because it means that the initialization of the module is run twice on the same piece of memory?

This causes a problem because the single-threaded runtime always runs tasks on the current thread, meaning that if the task started on another 'thread' it's now technically been sent to a different thread despite not necessarily being Send, which is unsound.

How could you send it to another" thread" though? I mean its true that in this scenario its using the same SharedArrayBuffer, but how is access from another "thread" achieved in such a case?

By sending the task's Waker to another thread the same way you would normally, then calling Waker::wake. When using the single-threaded implementation, wake always polls the task on the thread where it was called, so calling wake on a task from another thread accidentally moves it to that thread. For example:
use std::sync::Mutex;
use std::task::Waker;
use std::{future, thread};

use wasm_bindgen::prelude::*;

static WAKER: Mutex<Option<Waker>> = Mutex::new(None);

#[wasm_bindgen]
pub fn spawn_task() {
    wasm_bindgen_futures::spawn_local(future::poll_fn(|cx| {
        web_sys::console::log_1(&JsValue::from_str(&format!("{:?}", thread::current())));
        *WAKER.lock().unwrap() = Some(cx.waker().clone());
        std::task::Poll::Pending
    }))
}

#[wasm_bindgen]
pub fn wake_task() {
    if let Some(waker) = WAKER.lock().unwrap().take() {
        waker.wake();
    }
}

import init1, * as mod1 from "./pkg/wasm_bindgen_playground.js?i=1";
import init2, * as mod2 from "./pkg/wasm_bindgen_playground.js?i=2";

// this should probably be using `wasm_bindgen::memory()` but it doesn't really matter here
const memory = new WebAssembly.Memory({ initial: 18, maximum: 16384, shared: true });
await init1("./pkg/wasm_bindgen_playground_bg.wasm", memory);
await init2("./pkg/wasm_bindgen_playground_bg.wasm", memory);

mod1.spawn_task();
// let the task get polled so that waking it doesn't do nothing
await Promise.resolve();
mod2.wake_task();

Without this PR, this prints:
Thread { id: ThreadId(1), name: None, .. }
Thread { id: ThreadId(1), name: None, .. }

polling the task on the same thread both times like it should.
With this PR, it prints:
Thread { id: ThreadId(1), name: None, .. }
Thread { id: ThreadId(2), name: None, .. }

polling it on a different thread the second time.

Thanks @Liamolucko, I didn't understand exactly what you meant with its "moved to the other thread", but after trying your example and studying the wasm-bindgen-futures implementation I understood the issue.
Unfortunately I didn't not find a way to solve it. Initially I thought I could make a Promise and call resolveFunc when waking it, which would be similar to the multi-threaded example. But the issue is that I can't actually store and retrieve resolveFunc (a js_sys::Function, because well, its another thread.
Let me know if you have another solution in mind!


Upsert pgvector records

Current implementation deletes all of chunks before creating to keep idenpotency. But upsert operation is better in terms of performance especially for large dataset.
Primary key may need to be changed as: s3_uri + ':' + chunk_num

I support this as we augment our postgres entries externally and this causes us issues.


Unable to delete overlay

For some reason i'm unable to completely delete an overlay from a map using the worldmap node
The readme describes to do it by sending msg.payload.command.clear = "name of your layer/overlay to remove" to the worldmap node. For a layer this works but not for an overlay.
The flow attached shows the issue. Do i miss something obvious?
(NR 19.4,  Worldmap 1.4.5, Env: Windows, Browsers Chrome/Edge)
delete_overlay_test.txt

Confirmed to solve my issue! Thanks a lot !!


Add http-get check

This adds a http-get check and modifies the README to reflect this.

Fixed language in help text. Merging...


chore: upgrade graphql-config, language server

TODO

[x] upgrade graphql-config and graphql-language-service-server
[x] load graphql config/schema using @3 API
[x] update readme
[x] enable jsx/tsx/ts language features (before only highlighting was enabled, except for .js)
[x] headers and env
[x] get operations working with new endpoints extension (default to schema url if provided)
[x] improve error handling for requests
[x] variables validation for requests
[x] remove requirement for typescript LSP extension

resolves #124 #145 #171 #165 #116 #184
requires:
https://github.com/graphql/graphiql/pull/1616

oops this was supposed to be a draft haha

Any update to this PR? I'm waiting for days...

@matt-huh  whole days, huh? welcome to OSS! :laughing:
still working on several required PRs to the LSP server that this depends on. will need to publish about 24 new NPM package versions before I can even finish this PR.  this branch is what I used for reference while developing the LSP changes.

@acao Thanks for sharing the status. I'm always grateful for all the great contributions made - no intention to push!

good news folks! this is very close. close enough to test very easily
prisma folks and I are still working out a few details, and deciding whether/how to pre-release this, or whether to just issue a full stable release. I still need to finish updating the prisma graphql config extension that this depends on.
testing instructions:

clone this repo, check out this branch chore/graphql-config-3
npm install
open in vscode
open debugger tab, run "Run (Extension)"
open a project with a graphql-config@3 config (schemaPath -> schema, includes -> documents, use .js and process.env instead of funky old  yaml env template replacement, etc

I think the documentation could use more information, and that it would be nice to start a changelog with this PR

good news!
you can install from the marketplace now:
https://github.com/prisma-labs/vscode-graphql/issues/181#issuecomment-672337523

I am getting a lot of errors, what is the best way to provide feedback? Like this for example:

@huv1k let's provide feedback here #181. try 0.3.5 update and a reload!


docs (mongodb): add a note for default _id type

No changes in code, just some note in the documentation to highlight the default _id treated by Mongo as an ObjectId type.

Thanks @oscerd , I think this is not yet good to go as I realized the converter is not provided out of the box. In my test it was working because I was using some Spring converter implicitly. I will log a Jira in order to work on the provisioning of a Converter and re-issue a PR with updated documentation.


Clean up GitHub Actions jobs

Currently the GitHub Actions CI workflow is... pretty ugly.
We copy-paste the Node+PNPM installer pretty much everywhere.
To attempt to fix this, I tried to make a reusable workflow which didn't go so well.
I'd really appreciate help from someone who actually knows how to use GitHub Actions.
https://github.com/freshgum-bubbles/typedi/blob/c1d294acd5aa1a226465269651098a574ca5cb03/.github/workflows/continuous-integration-workflow.yml#L17-L38

Hi,
Is your idea to optimize the installation phase of Node and PNPM and make it reusable in the other GH Actions?
I'll fork the project and take a look at it, if I solve it I'll open a PR for you.

Is your idea to optimize the installation phase of Node and PNPM and make it reusable in the other GH Actions?

Yeah, pretty much :-)  I tried to do it myself (something something "Reusable workflows") but ended up baffled by the whole thing.  It'd be really nice if there was a way to test Actions locally :/
#22, #15 and quite a few commits to the Workflows was as far as I got before the current, copy-paste situation.
If you're interested, there's another problem with Actions here which I don't have the wherewithall to fix: each one seems to run twice for pull requests (see https://github.com/freshgum-bubbles/typedi/pull/114 as an example).  I'm sure that's burned through quite a lot of my minutes already, but alas :(


Unable to build hello_world intro example for ESP32-C6

I'm try following the Embedded Rust (no_std), target to ESP32-C6.
First I try keep  everything as it is and run cargo check. Everything checked successfully.
Then, I try to change the target to ESP32-C6 and try to check again. Unfortunately, it failed.
Here is the output
error[E0599]: no method named `compare_exchange` found for struct `AtomicU8` in the current scope
  --> C:\Users\mymon\.cargo\registry\src\index.crates.io-6f17d22bba15001f\esp-hal-0.16.1\src\soc\mod.rs:43:14
   |
42 |           if MAC_OVERRIDE_STATE
   |  ____________-
43 | |             .compare_exchange(0, 1, Ordering::Relaxed, Ordering::Relaxed)
   | |             -^^^^^^^^^^^^^^^^ method not found in `AtomicU8`
   | |_____________|
   |

For more information about this error, try `rustc --explain E0599`.
error: could not compile `esp-hal` (lib) due to previous error

Here is the Cargo.toml file:
[package]
name = "hello_world"
version = "0.1.0"
authors = ["Sergio Gasquez<EMAIL_ADDRESS>edition = "2021"
license = "MIT OR Apache-2.0"

[dependencies]
esp-hal = { version = "0.16.1", features = ["esp32c6"] }
esp-backtrace = { version = "0.11.0", features = [
    "esp32c6",
    "panic-handler",
    "exception-handler",
    "println",
] }
esp-println = { version = "0.9.0", features = ["esp32c6", "uart"] }

And rust-toolchain.toml file:
[toolchain]
channel = "nightly-2023-11-14"
components = ["rust-src"]
targets = ["riscv32imac-unknown-none-elf"]

Please check the target in .cargo/config.toml is riscv32imac-unknown-none-elf

Thank you @bjoernQ. I was missing that. It's  working now.


locked ProxyAccessor.Configure till end of his work

fix for https://github.com/epam/epam-graphql/issues/95
[feel free to close without explanation if it does not work]

Codecov Report

Merging #96 (c9e28f7) into develop (c2545bf) will decrease coverage by 0.01%.
The diff coverage is 94.87%.

@@             Coverage Diff             @@
##           develop      #96      +/-   ##
===========================================
- Coverage    88.55%   88.54%   -0.02%
===========================================
  Files          260      260
  Lines        17045    17052       +7
  Branches      1226     1227       +1
===========================================
+ Hits         15095    15099       +4
- Misses        1668     1670       +2
- Partials       282      283       +1

Impacted Files
Coverage Δ

...hQL/Configuration/Implementations/ProxyAccessor.cs
98.24% <94.87%> (-0.87%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

fixed here in https://github.com/epam/epam-graphql/pull/97


NameError: name 'yf' is not defined

Hi Everyone
I'm new to this and trying to use yfinance in python (anaconda/Spyder5). The code bellow gives me an error (bellow that). Could someone please help me out with this issue. It's probably an easy fix for you, but I'm getting hung up on it.
Thanks
Lines of code
!pip install yfinance
!import pandas as pd
!import yfinance as yf
msft = yf.Ticker("MSFT")
print(msft)
msft.info
Error
File "/Users/daviddowney/untitled4.py", line 5, in
msft = yf.Ticker("MSFT")
NameError: name 'yf' is not defined

You should only have exclamation marks before shell commands not python code (and it only if you are working in a jupyter notebook like environment).

Thank you for your response. I removed the ! and got a different error. I am using Spyder from the Anaconda python package.
The new code is
pip install yfinance
import pandas as pd
import yfinance as yf
msft = yf.Ticker("MSFT")
print(msft)
msft.info
The new error is
File "/Users/daviddowney/untitled4.py", line 1
pip install yfinance
^
SyntaxError: invalid syntax
I get this right away (it fails before I even get to the yf part issue.
Once again than you for your help.

You did not use the code I added to my comment that works


try fetch token again when token timeouts

when max_concurrent_download is specified, get requests are not sent immediately for all layers. Token might timeout and we need to try to fetch it again.
eg. max_concurrent_download = 3, fooimg:latest has 4 blob layers, network is poor and first 3 layers took 2hr to finish downloading. when 4th layer begins, it tries to reused the token in host(which has already expired), invalid_token err is thrown.

not sure if this change is appropriate, ptal @fuweid @dmcgowan

closed by https://github.com/containerd/containerd/pull/8735


Design Issue iOS

I am integrating it to ios devices. But facing an issue in the screen resolution of iPhone 5 and 6. The polar chart is going out of the screen and cutting the titles around the chart. Please check the attached screenshots and help me fix this
Thanks

It is easy for me,I will fix it tomorrow😊

Update AAChartKit Demo and do something as follow

Configure AAOptions instance object

    AAChartModel *aaChartModel= AAObject(AAChartModel)
    .chartTypeSet(AAChartTypeColumn)//图表类型
    .titleSet(@"")//图表主标题
    .subtitleSet(@"")//图表副标题
    .colorsThemeSet(@[@"#fe117c",@"#ffc069",@"#06caf4",@"#7dffc0"])//设置主体颜色数组
    .yAxisTitleSet(@"")//设置 Y 轴标题
    .polarSet(true)
    .categoriesSet(@[@"Java", @"Swift", @"Python", @"Ruby", @"PHP", @"Go", @"C", @"C#", @"C++", @"Perl", @"R", @"MATLAB", @"SQL"])
    .seriesSet(@[
                 AAObject(AASeriesElement)
                 .nameSet(@"2017")
                 .dataSet<EMAIL_ADDRESS>@6.9, @9.5, @14.5, @18.2, @21.5, @25.2, @26.5, @23.3, @18.3, @13.9, @9.6]),
                 ]
               );

    AAOptions *aaOptions = [AAOptionsConstructor configureChartOptionsWithAAChartModel:aaChartModel];
    aaOptions.chart.marginLeft = @70;
    aaOptions.chart.marginRight = @70;

Draw chart with AAOptions instance object

    [aaChartView aa_drawChartWithOptions:aaOptions];

The key content is this
 aaOptions.chart.marginLeft = @70;//configure the left margin of chart
 aaOptions.chart.marginRight = @70;//configure the right margin of chart

Hi, Thanks for the update
I am writing code in swift and unable to alloc      AAOptions *aaOptions = [AAOptionsConstructor configureChartOptionsWithAAChartModel:aaChartModel];
Please check and let me know if you can help in this
` viewChart = AAChartView.init(frame: CGRect(x:0, y:0, width:piechartBackView.frame.size.width, height:piechartBackView.frame.size.height))
piechartBackView.addSubview(viewChart)
viewChart.scrollEnabled = false
piechartBackView.bringSubview(toFront: addbtn)
    let aaChartModel = AAChartModel()
    aaChartModel.chartType = AAChartTypeColumn
    aaChartModel.title = ""//The chart title
    aaChartModel.subtitle = ""//The chart title
    aaChartModel.dataLabelEnabled = false //Enable or disable the data labels. Defaults to false
    aaChartModel.polar = true
    aaChartModel.yAxisVisible = false
    aaChartModel.legendEnabled = false
    aaChartModel.borderRadius = 80
    aaChartModel.tooltipValueSuffix = ""//the value suffix of the chart tooltip
    let catArr = Array(pieChartData.keys)
    var valArr = [Float]()
    var nameLblArr = [String]()
    for item in catArr {
        nameLblArr.append(item.capitalized.replacingOccurrences(of: "_", with: " "))
        valArr.append(pieChartData[item] as? Float ?? 0.0)
    }
    aaChartModel.categories = nameLblArr

    aaChartModel.colorsTheme = ["#00b8a2"/*,"#ffc069","#06caf4","#7dffc0","#fe227c","#1dc6ad","#06ffac","#c94848"*/]

    let element1 = AASeriesElement() //"Value"
    element1.name = "Value"
    element1.data = valArr
    aaChartModel.series = [element1/*,element2,element3,element4,element5,element6,element7,element8*/]

    let aaOptions = AAOptions()
    aaOptions.chart.marginLeft = 70
    aaOptions.chart.marginRight = 70

//        viewChart.aa_drawChart(with: aaChartModel)
viewChart.aa_drawChart(with: aaOptions)`

Are you using AAChartKit or AAChartKit-Swift in your project?

AAChartKit

Sorry,I have no experience on Swift and OC Hybrid programming,Maybe you should use AAChartKit-Swift in your project and change a little source code of AAChartKit-Swift for your project

I wrote it in README and the font is bold !!! 😊

Okay, I will convert it, But Did you update these margin parameters in AAChartKit-Swift too?
I am trying to update this in your demo AAChartKit-Swift code and unable to access AAOptionsConstructor in Swift code too. Any lead on this  😔

I will fix it later ,there is many difference between AAChartKit and AAChartKit-Swift

Can you tell me when you will update AAChartKit-Swift ? I need to integrate it to my project on urgent bases, your support will be appreciable.
Thanks

This week's weekend.😊
Speaking off topic, did you specifically register GitHub to raise this issue?

Yeah, They didn't allow me to post issue without an account. I have an account on bitbucket, not on github.

Can we give different colors to every sector of the polar chart?

I will give you a different colors polar chart demo,So do not worry about this:)
But the annular style  string  around the polar chart is impossible for me on the screenshot!!!!

Could you tell me where did you get the screenshot photo?

annular style string around the polar chart is impossible for me

That's fine, I don't want annular style string, straight is fine for me. But can you save them from overlapping the polar outer ring?

Could you tell me where did you get the screenshot photo?

That's from some other library. Here's the link https://www.anychart.com

In my opinion, if you just want to draw single type chart in your project ,you should use AnyChart.js rather than AAChartKit!!!

I am finding your library better and easy to integrate, So If you can help me change the colors of sectors that will be fantastic. ALong will those minor fixes of text overlap and out margins.

Hey, Any update??

The polar chart is going out of the screen and cutting the titles around the chart.
To resolve this problem
You can configure the AAChartModel instance object as follow

            aaChartModel = AAChartModel()
                .chartType(AAChartType.Column)
                .polar(true)
                .dataLabelEnabled(false)
                .categories(["January", "February", "March", "April", "May", "June",
                             "July", "August", "September", "October", "November", "December"])
                .marginLeft(50)//left margin of chart
                .marginRight(30)//right margin of chart
                .series(
                    [
                        AASeriesElement()
                            .name("Tokyo")
                            .data([7.0, 6.9, 9.5, 14.5, 18.2, 21.5, 25.2, 26.5, 23.3, 18.3, 13.9, 9.6])
                            .toDic()!,
                        ]
            )

I am unable to get these properties, marginLeft and marginRight

Update the AAInfographic Demo,then you can configure the AAChartModel instance object as follow
        aaChartModel = AAChartModel()
                .chartType(AAChartType.Column)
                .polar(true)
                .dataLabelEnabled(false)
                .categories(["January", "February", "March", "April", "May", "June",
                             "July", "August", "September", "October", "November", "December"])
                .marginLeft(50)
                .marginRight(30)
                .series(
                    [
                        ["name":"2018",
                         "data":[7.0, 6.9, 9.5, 14.5, 18.2, 21.5, 25.2, 26.5, 23.3, 18.3, 13.9, 9.6],
                         "colorByPoint":true //🌈🌈let chart be colorful🌈🌈
]
                        ]
            )

text overlap and out margins.

I donnot fix this yet!!!o(╯□╰)o

Can you give mein commit id from which to update

The AAInfographics(AAChartKit-Swift) was updated,not the AAChartKit

Okay, and can you help me with different colors of each sector?

You can not get colorful polar chart yet?

I did, thanks, I didn't check the colortype key in the code. It's colorful now.
But Marginleft and right doesn't fix the outer outgoings of labels

Hey
Its fixed and it working all fine now. Thanks, mate for your support.
Beers for you are due on me till we meet! ;) 🍻


feat(#26): incorrect-version stronger

In this pull I made incorrect-version lint stronger by adding support for version tags after version, added more test cases.
closes #26
History:

feat(#26): reproduce
feat(#26): move to existing pack
*feat(#26): ignore .DS_Store
feat(#26): more
feat(#26): more test cases
feat(#26): passes
feat(#26): passes on more cases
feat(#26): reorder

@h1alexbel thanks!


[Improvements]: Enable -Wextra warning

Description

enable -Wextra
fix unused parameter
fix sign compare

first step of #2387
How to test
run C++ unit tests
Types of changes

Improvement

Checklist

[x] Create pull request as draft initially, unless its complete.
[x] Add tests to cover changes as needed.
[x] Update documentation as needed.
[x] If there is a related Issue, mention it in the description.

Wow, a monster changeset!  Looks good, but I did not dare I don't feel confident enough yet to approve

Looks good to me - I actually looked through everything :)
But what's the benefit of the change? Looks like maintaining those [[maybe_used]] everywhere would be a small pain

I think that's actually the opposite, by using a c++ attribute it make it easier to track the unused parameter for a futur refactoring, you can grep in command line and have a full overview. Later with a bigger refactoring we will be able to do more modern interface that have optional parameters defaulted to std::nullopt

can we ignore those warnings for tests folder?

I think it's not a good idea. Most of the fixes in the tests folder concern integer usage. It's better to use the intended integer when checking size of a container etc

can we ignore those warnings for tests folder?

I think it's not a good idea. Most of the fixes in the tests folder concern integer usage. It's better to use the intended integer when checking size of a container etc

I understand the point, trying to make our life easier when writing tests


Allow URL bookmarking

Best would be to add state management across page refresh (with cookies)
Short-term solution might be to remember the initial URL and redirect there after re-login

Bookmarking works when sci-viz is deployed in loginless mode.


Doesn't work in groups with topics

Bot does not respond in groups with topics. I created a topic for games in my group, but @unurobot doesn't respond. Other game bots work correctly

hm, that's weird. I have implemented topics support already (and just tested in a group). Can you try giving admin rights to the bot (can be just admin title, no permissions are required)?

hm, that's weird. I have implemented topics support already (and just tested in a group). Can you try giving admin rights to the bot (can be just admin title, no permissions are required)?

True. Making the bot admin without permissions didn't solve it. But when giving permission to manage threads it started to work. Thanks! The bot needs to have permission to manage topics.The bot needs to have permission to manage topics.