Error: Request failed with status code 500. "OSError: [Errno 5] Input/output error\n".

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

I add an external device for increasing the disk image size. But when I want to export the dataset as YOLO format, this error "OSError: [Errno 5] Input/output error\n" shows up. Could someone help me figure out what is going on here?
 Docker Swarm

Operating System and version (e.g. Linux, Windows, MacOS): MacOS

Code example or link to GitHub repo or gist to reproduce problem:

Other diagnostic information / logs:

Logs from `cvat` container

Next steps
You may join our Gitter channel for community support.

Hi, could you share server logs? They can be obtained with docker logs cvat.

@zhiltsov-max Sure. But this time I met this error.

Could not export dataset for the task 1
Error: Request failed with status code 500. "Work-horse was terminated unexpectedly (waitpid returned None)".

Here are the logs.
2020-10-26 14:50:09,203 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 14:50:11,486 DEBG 'runserver' stderr output:
[Mon Oct 26 14:50:11.454102 2020] [wsgi:error] [pid 243:tid<PHONE_NUMBER>74464] [remote <IP_ADDRESS>:50082] [2020-10-26 14:50:11,350] ERROR django.request: Internal Server Error: /api/v1/tasks/1/dataset
2020-10-26 14:50:11,497 DEBG 'runserver' stderr output:
[Mon Oct 26 14:50:11.497484 2020] [wsgi:error] [pid 243:tid<PHONE_NUMBER>74464] [remote <IP_ADDRESS>:50082] ERROR:django.request:Internal Server Error: /api/v1/tasks/1/dataset
2020-10-26 14:50:20,498 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 14:56:50,568 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 14:56:54,322 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 14:57:05,538 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 15:03:35,595 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 15:03:39,399 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 15:03:50,544 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

Could you try to reproduce the first problem?

@zhiltsov-max I will try to reproduce the first problem. BTW, why did it take me a long time to export the dataset? I have around 91,000 frames and set the image quality as 70%.

@zhiltsov-max I have reproduced the first problem. Here are the logs.
2020-10-26 21:28:55,389 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:29:25,356 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:29:55,325 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:30:25,292 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:30:55,262 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:31:25,229 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:31:55,195 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:32:25,165 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:32:55,131 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:33:25,100 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:33:48,830 DEBG 'rqworker_default_1' stderr output:
[2020-10-26 21:33:48,743] ERROR cvat.server.task_3: [cvat.apps.dataset_manager.views @ export_task]: exception occurred
Traceback (most recent call last):
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1776, in write
shutil.copyfileobj(src, dest, 1024*8)
File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj
fdst_write(buf)
File "/usr/lib/python3.8/zipfile.py", line 1141, in write
self._fileobj.write(data)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
task.export_task(task_id, temp_file, dst_format,
File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
task.export(f, exporter, host=server_url, save_images=save_images)
File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
exporter(dst_file, task_data, **options)
File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in call
f_or_cls(*args, **kwargs)
File "/home/django/cvat/apps/dataset_manager/formats/yolo.py", line 29, in _export
make_zip_archive(temp_dir, dst_file)
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1312, in exit
self.close()
File "/usr/lib/python3.8/zipfile.py", line 1839, in close
self._write_end_record()
File "/usr/lib/python3.8/zipfile.py", line 1904, in _write_end_record
self.fp.write(centdir)
OSError: [Errno 5] Input/output error
2020-10-26 21:33:48,855 DEBG 'rqworker_default_1' stderr output:
ERROR:cvat.server.task_3:[cvat.apps.dataset_manager.views @ export_task]: exception occurred
Traceback (most recent call last):
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1776, in write
shutil.copyfileobj(src, dest, 1024*8)
File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj
fdst_write(buf)
File "/usr/lib/python3.8/zipfile.py", line 1141, in write
self._fileobj.write(data)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
task.export_task(task_id, temp_file, dst_format,
File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
task.export(f, exporter, host=server_url, save_images=save_images)
File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
exporter(dst_file, task_data, **options)
File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in call
f_or_cls(*args, **kwargs)
File "/home/django/cvat/apps/dataset_manager/formats/yolo.py", line 29, in _export
make_zip_archive(temp_dir, dst_file)
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1312, in exit
self.close()
File "/usr/lib/python3.8/zipfile.py", line 1839, in close
self._write_end_record()
File "/usr/lib/python3.8/zipfile.py", line 1904, in _write_end_record
self.fp.write(centdir)
OSError: [Errno 5] Input/output error
2020-10-26 21:33:48,875 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Handling failed execution of job /api/v1/tasks/3/dataset/YOLO 1.1
2020-10-26 21:33:49,032 DEBG 'rqworker_default_1' stderr output:
ERROR:rq.worker:Traceback (most recent call last):
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1776, in write
shutil.copyfileobj(src, dest, 1024*8)
File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj
fdst_write(buf)
File "/usr/lib/python3.8/zipfile.py", line 1141, in write
self._fileobj.write(data)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 936, in perform_job
rv = job.perform()
File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 684, in perform
self._result = self._execute()
File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 690, in _execute
return self.func(*self.args, **self.kwargs)
File "/home/django/cvat/apps/dataset_manager/views.py", line 84, in export_task_as_dataset
return export_task(task_id, dst_format, server_url=server_url, save_images=True)
File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
task.export_task(task_id, temp_file, dst_format,
File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
task.export(f, exporter, host=server_url, save_images=save_images)
File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
exporter(dst_file, task_data, **options)
File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in call
f_or_cls(args, **kwargs)
File "/home/django/cvat/apps/dataset_manager/formats/yolo.py", line 29, in _export
make_zip_archive(temp_dir, dst_file)
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1312, in exit
self.close()
File "/usr/lib/python3.8/zipfile.py", line 1839, in close
self._write_end_record()
File "/usr/lib/python3.8/zipfile.py", line 1904, in _write_end_record
self.fp.write(centdir)
OSError: [Errno 5] Input/output error
Traceback (most recent call last):
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1776, in write
shutil.copyfileobj(src, dest, 10248)
File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj
fdst_write(buf)
File "/usr/lib/python3.8/zipfile.py", line 1141, in write
self._fileobj.write(data)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 936, in perform_job
rv = job.perform()
File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 684, in perform
self._result = self._execute()
File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 690, in _execute
return self.func(*self.args, **self.kwargs)
File "/home/django/cvat/apps/dataset_manager/views.py", line 84, in export_task_as_dataset
return export_task(task_id, dst_format, server_url=server_url, save_images=True)
File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
task.export_task(task_id, temp_file, dst_format,
File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
task.export(f, exporter, host=server_url, save_images=save_images)
File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
exporter(dst_file, task_data, **options)
File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in call
f_or_cls(*args, **kwargs)
File "/home/django/cvat/apps/dataset_manager/formats/yolo.py", line 29, in _export
make_zip_archive(temp_dir, dst_file)
File "/home/django/cvat/apps/dataset_manager/util.py", line 20, in make_zip_archive
archive.write(path, osp.relpath(path, src_path))
File "/usr/lib/python3.8/zipfile.py", line 1312, in exit
self.close()
File "/usr/lib/python3.8/zipfile.py", line 1839, in close
self._write_end_record()
File "/usr/lib/python3.8/zipfile.py", line 1904, in _write_end_record
self.fp.write(centdir)
OSError: [Errno 5] Input/output error
2020-10-26 21:33:49,131 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Invoking exception handler <function rq_handler at 0x7fa3e415fee0>
2020-10-26 21:33:50,611 DEBG 'runserver' stderr output:
[Mon Oct 26 21:33:50.526909 2020] [wsgi:error] [pid 243:tid<PHONE_NUMBER>81760] [remote <IP_ADDRESS>:51026] [2020-10-26 21:33:50,523] ERROR django.request: Internal Server Error: /api/v1/tasks/3/dataset
2020-10-26 21:33:50,633 DEBG 'runserver' stderr output:
[Mon Oct 26 21:33:50.632877 2020] [wsgi:error] [pid 243:tid<PHONE_NUMBER>81760] [remote <IP_ADDRESS>:51026] ERROR:django.request:Internal Server Error: /api/v1/tasks/3/dataset
2020-10-26 21:33:55,112 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:34:25,101 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
2020-10-26 21:34:44,742 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:34:44,743 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
INFO:rq.worker:Cleaning registries for queue: default
DEBUG:rq.worker:*** Listening on default...
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:35:12,283 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:35:12,286 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-26 21:35:29,109 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:41:29,512 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:41:57,327 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:42:14,195 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:48:14,616 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:48:42,367 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:48:59,212 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:54:59,699 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:55:27,436 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 21:55:44,250 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:01:44,734 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:01:44,735 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-26 22:02:12,489 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:02:12,490 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-26 22:02:29,306 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:08:29,799 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:08:57,565 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:09:14,383 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:15:14,804 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:15:42,579 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:15:59,446 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:21:59,836 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:22:27,590 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:22:44,467 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:28:44,910 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:28:44,912 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-26 22:29:12,646 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:29:12,647 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-26 22:29:29,529 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:35:29,995 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:35:57,705 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:36:14,553 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:42:15,047 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:42:42,757 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:42:59,596 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:49:00,136 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:49:27,826 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:49:44,621 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:55:45,229 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:55:45,232 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-26 22:56:12,848 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 22:56:12,850 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-26 22:56:29,641 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:02:30,329 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:02:57,893 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:03:14,679 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:09:15,348 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:09:42,958 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:09:59,691 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:16:00,451 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:16:28,012 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:16:44,705 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:22:45,489 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:22:45,492 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-26 23:23:13,074 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:23:13,076 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-26 23:23:29,724 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:29:30,502 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:29:58,110 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:30:14,813 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:36:15,548 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:36:43,209 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:36:59,826 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:43:00,590 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:43:28,268 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:43:44,852 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:49:45,669 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:49:45,670 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-26 23:50:13,323 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:50:13,325 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-26 23:50:29,888 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:56:30,751 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:56:58,383 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-26 23:57:14,912 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:03:15,822 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:03:43,410 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:03:59,923 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:10:00,935 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:10:28,460 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:10:44,980 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:16:45,997 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:16:45,999 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-27 00:17:13,539 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:17:13,540 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-27 00:17:30,023 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:23:31,010 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:23:58,640 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:24:15,074 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:30:16,025 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:30:43,698 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:31:00,149 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:37:01,080 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:37:28,710 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:37:45,196 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:43:46,105 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:43:46,107 DEBG 'rqworker_default_1' stderr output:
INFO:rq.worker:Cleaning registries for queue: default
2020-10-27 00:44:13,796 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:44:13,797 DEBG 'rqworker_low' stderr output:
INFO:rq.worker:Cleaning registries for queue: low
2020-10-27 00:44:30,236 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:50:31,153 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:50:58,889 DEBG 'rqworker_low' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2020-10-27 00:51:15,325 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

It looks like there is not enough memory for writing the results. Does the system has enough space?
If you have a video and trying to export it with images, they will be recoded from video frames, it can take a while (a recent patch speeds up the process a bit, #2229).

I set the disk image size to 200GB. If there is no memory, I guess this error #2348 will occur.

how can we fix memory issue ?

@Diaislam, which memory issue do you mean exactly?

@Diaislam, which memory issue do you mean exactly?

when trying to download annotations with format segmentation mask 1.1 i receive this error
Error: Request failed with status code 500. "Work-horse was terminated unexpectedly (waitpid returned 9)".
cvat logs :
DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:*** Listening on default...
2021-03-15 10:43:10,800 DEBG 'rqworker_default_0' stderr output:
DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
2021-03-15 10:43:10,853 DEBG 'runserver' stderr output:
[Mon Mar 15 10:43:10] [wsgi:error] [pid 229:tid<PHONE_NUMBER>42176] [remote ...:49050] [2021-03-15 ] ERROR django.request: Internal Server Error: /api/v1/tasks/241/annotations
2021-03-15 10:43:10,854 DEBG 'runserver' stderr output:
[Mon Mar 15 10:43:10.853588 2021] [wsgi:error] [pid 229:tid<PHONE_NUMBER>42176] [remote ...:49050] ERROR:django.request:Internal Server Error: /api/v1/tasks/241/annotations

@Diaislam, the problem you described looks like Out of Memory error, please check dmesg -HTw for out of memory errors. The issues https://github.com/openvinotoolkit/cvat/issues/2264, https://github.com/openvinotoolkit/cvat/issues/2939#event-4457084774 look similar to what you're describing.

@Diaislam, the problem you described looks like Out of Memory error, please check dmesg -HTw for out of memory errors. The issues #2264, #2939 (comment) look similar to what you're describing.

it seems using the ui to download masks 2gb of freey memory isn't enough.
while using command cli works fine

@Diaislam , it looks like it is a problem with your CVAT instance. Need enough memory to do some operations. We are constantly improving CVAT to take less resources. I will close the issue. Don't hesitate to reopen if you still see the issue.

@nmanovic thanks you, I upgraded cvat and my machine, awesome work you are doing guys I really appreciate how it can handle big amounts of data and users activity, one thing I'm using cvat before it wasn't written with react. the old one is still faster in loading images.
I notice that projects tasks need pagination sometimes a project with 4000 task doesn't load I get an error because of the huge amount of tasks.
I'm having the latest version,if it happens that I update my cvat I have 0 risks of losing tasks data and kibana analytics data?


Histograms from multiple sources

As brought up by @kratsg:
There are applications where a template histogram needs to be filled with data from various sources, with possibly different selection requirements / weight per source. An example for this is a histogram of a Monte Carlo sample, containing an expected distribution of data across multiple years. Across those years, the luminosity might differ (so different weights are needed per year), as might the selections to be applied (due to e.g. a change in trigger threshold). Native support for this would be good.

While there is no native support for this yet, #78 implements a method to achieve this manually in the meantime. A user-defined function can be provided to cabinetry to handle this.


Suggestion to Executive Summary messaging

Dear wonderful Repo contributer:
in this issue I'm suggesting the following enhancements to the execute summary provided after running the tools:
No Validation errors found:

Validation completed with no issues found.
Number of operations validated.

Errors found:

Validation completed with X issues found
Number of operations validated.
Issues summary:
-- issue 1
-- issue 2
-- issue 3
.
.
.

Completed in v0.1 dogfooding by @amarzavery


Revert messages are missing

Component
Anvil
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.2.0 (3fc1491 2022-06-06T00:14:30.023454Z)
What command(s) is the bug in?
anvil
Operating System
macOS (Intel)
Describe the bug
Transactions will revert messages are missing revert messages.
I have a method and modifier like:
    modifier onlyOwner() {
        require(msg.sender == owner, "!authorized");
        _;
    }
    function getSecret() public onlyOwner view returns(uint256 secret) {
        return 123;
    }

The response data I am getting looks like:
{'jsonrpc': '2.0', 'id': 23, 'error': {'code': -32003, 'message': 'execution reverted: ', 'data': '0x'}}

which is maybe what I would expect had I not included a revert message.
!unathorized is missing from the response.
I believe this is a regression, as this was working a month or so ago.

I am using web.py to facilitate requests but see this response data via breakpoints

@mattsse are we correctly returning reverts from anvil?

+1 to this bug. Was able to reproduce this by deploying a contract with a simple revert("revert string"); call.
Using anvil in forked mode and using Ethers.js to make the call.
Anvil version: anvil 0.1.0 (1616380 2022-06-05T00:17:15.393047Z)

When deploying to a forked network, this bug is still present I think

Ah, okay I am getting this error on estimating gas.
Can we also include the revert message when estimating gas?

Also, web.py is not triggering exceptions from my reverts when I specify gas in
self.web3.eth.get_transaction(txn_hash).keys()

thanks for providing this link
will have a look what's up with eth_estimate and reverts

I tried the PR and web3.py still is not catching the reverts for some reason. hmmmmmmm.

haven't tested this in python because I never get it working
but curious why this condition is not met:
https://github.com/ethereum/web3.py/blob/c741e55b8ac95fed72a7135e7bb0e9f0d4a40431/web3/_utils/method_formatters.py#L628-L629
seems like this should for:
JsonRpcError {
                    code: 3,
                    message: "execution reverted: Not enough Ether provided.",
                    data: Some(
                        String(
                            "0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a4e6f7420656e6f7567682045746865722070726f76696465642e000000000000",
                        ),
                    ),
                },

I am debugging this.. One thing I noticed is that web3.py expects the key "error" key instead of "result" when errors occur... Am I reading this correctly?

So this might be related, I don't know:
My eth_estimateGas call works on Hardhat but fails on Anvil with exception:
{
    "jsonrpc": "2.0",
    "id": 0,
    "error": {
        "code": -32602,
        "message": "unknown field `id`, expected one of `from`, `to`, `gasPrice`, `maxFeePerGas`, `maxPriorityFeePerGas`, `gas`, `value`, `data`, `nonce`, `accessList`, `type`"
    }
}

oh so somewhere in eth_estimateGas we're missing a proper conversion

Ok never mind the part about the ID field... I am in the midst of debugging and just confused

Ok I am working on a more bare-bones reproduction script to try and debug stuff.
I got this:
from solcx import compile_source
from web3 import HTTPProvider, Web3
from web3.exceptions import ContractLogicError

# Mnemonic       : "test test test test test test test test test test test junk"
# Derivation path: "m/44'/60'/0'/{acct-index}"
ACCOUNT = "0x1e59ce931B4CFea3fe4B875411e280e173cB7A9C"
PKEY = "0xdd23ca549a97cb330b011aebb674730df8b14acaee42d211ab45692699ab8ba5"

SOURCE_CODE = """
pragma solidity ^0.8.2;

contract NumberStorage {
    uint256 public number;

    function setNumber(uint256 num) public {
        require(num != 0, "RevertStringFooBar");
        number = num;
    }
}
""".strip()

def main():
    # Connect to node (either Hardhat node or Anvil)
    web3 = Web3(HTTPProvider("http://<IP_ADDRESS>:8545/"))
    web3.eth.default_account = ACCOUNT

    # Compile simple contract
    compiled_sol = compile_source(SOURCE_CODE, output_values=["abi", "bin"])
    _, contract_interface = compiled_sol.popitem()
    bytecode = contract_interface["bin"]
    abi = contract_interface["abi"]
    contract =  web3.eth.contract(abi=abi, bytecode=bytecode)

    # Deploy contract and create instance
    params = {"maxFeePerGas":<PHONE_NUMBER>352, "maxPriorityFeePerGas":<PHONE_NUMBER>}
    tx_hash = contract.constructor().transact(params)
    tx_receipt = web3.eth.wait_for_transaction_receipt(tx_hash)
    instance = web3.eth.contract(address=tx_receipt.contractAddress, abi=abi)

    # Transact in a way that should succeed
    tx_hash = instance.functions.setNumber(5).transact(params)
    tx_receipt = web3.eth.wait_for_transaction_receipt(tx_hash)
    print("Success")

    # Transact in a way that should fail
    try:
        tx_hash = instance.functions.setNumber(0).transact()
    except ContractLogicError as err:
        print(f"Failed: {err}")

if __name__ == "__main__":
    main()

When I use Hardhat, this is the output of the script:
Success
Failed: execution reverted: Error: VM Exception while processing transaction: reverted with reason string 'RevertStringFooBar'

When I connect to Anvil however, this is the output:
Success
Failed: execution reverted:

For some reason, the revert message is not there!

@mattsse should we introduce an extra decoding step - per the above analysis?

yeh, need to debug this.

Anvil is hitting the Geth condition now as a result to this PR
However - the "Geth" case seems to assume the error message is already decoded as a string.

this is not clear to me, what's the "error message" here? the expected revert reason or the 'message' object
in what RPC call does this happen @unparalleled-js?


Compatibility issue with Swiper latest version while testing with Jest

Check that this is really a bug

[X] I confirm

Reproduction link
_blank
Bug description
SyntaxError: Cannot use import statement outside a module
web:test:
web:test:       1 | import React, { memo } from 'react'
web:test:     > 2 | import { Swiper, SwiperSlide } from 'swiper/react'
web:test:         | ^
web:test:       3 | import 'swiper/css'
web:test:       4 | import { Autoplay, FreeMode } from 'swiper'
web:test:       5 |
Expected Behavior
No response
Actual Behavior
No response
Swiper version
9.2.3
Platform/Target and Browser Versions
macOS
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
[X] Make sure this is a Swiper issue and not a framework-specific issue

Would you like to open a PR for this bug?

[ ] I'm willing to open a PR

Do you want to ask a question? Are you looking for support? Stack Overflow or Swiper Discussions is the best places for getting support

Please, don't use GitHub issues for questions


[AIRFLOW-3580] Adds tests for HiveToMySqlTransfer

Make sure you have checked all steps below.
Jira

[x] My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-3580
In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

This PR:

adds tests for the HiveToMySqlTransfer operator
refactors the tested class HiveToMySqlTransfer

Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

It adds tests for all features:

general transfer
transfer with mysql pre-operator
transfer with mysql post-operator
transfer via bulk load

Commits

[x] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters (not including Jira issue reference)
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Documentation

[x] In case of new functionality, my PR adds documentation that describes how to use it.

When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
All the public functions and the classes in the PR contain docstrings that explain what it does

Code Quality

[x] Passes flake8

thanks for adding the tests.


Compatibility with hecs-0.10.3

This mostly:

moves QueryItem to Q::Item
moves get and get_mut to get::<&[mut]>
adds lifetimes to match up with hecs additions

Looks good, let me know if you're ready to merge.

Great - yep, sure!

I am not working that much on this library as I am currently focusing more on my own ECS experiment flax.
Let me know if there is any feature you'd like to have in hecs-schedule for me to work on, and I am more than happy for PRs.


Add relative reminder messaging field to Campaign object

[ ] Add string field to campaign object
[ ] Expose field in the campaigns API

Quicksilver will pull this information when sending relative reminders

 Add 'reminder messaging' field to Gambit


Analysis results extra setter behavior

As reported by Erick, after the dataframes refactor, MitigatedTomographyAnalysis isn't able to set the extra flag so the user is unable to easily distinguish whether a result is mitigated or not:
https://github.com/Qiskit-Extensions/qiskit-experiments/blob/8cc20dfed97afb1320004b4757b5006b41840777/qiskit_experiments/library/tomography/mit_tomography_analysis.py#L110-L122

With the current version of main, I couldn't recreate the error. @coruscating, could you tag Erick with so I will get a code snippet that recreate the error?
The code I ran was the following:

from qiskit_aer import AerSimulator
from qiskit.providers.fake_provider import FakePerth

backend = AerSimulator.from_backend(FakePerth())

import qiskit
from qiskit_experiments.library import MitigatedStateTomography

# GHZ State preparation circuit
nq = 2
qc_ghz = qiskit.QuantumCircuit(nq)
qc_ghz.h(0)
qc_ghz.s(0)
for i in range(1, nq):
    qc_ghz.cx(0, i)

# QST Experiment
qstexp1 = MitigatedStateTomography(qc_ghz)
qstdata1 = qstexp1.run(backend, seed_simulation=100).block_for_results()

# Print results
for result in mqstdata1.analysis_results():
    print(result)

state_result = mqstdata1.analysis_results("state")
print(state_result.value)

@ItamarGoldman I think @nkanazawa1989 is already working on this issue in #1344.


Schema mismatch for _Session.expiresAt; expected date but got string

After migration, attempting to log a user in results in a status=400. Used the provided parse-server-example code.
[Error]: schema mismatch for _Session.expiresAt; expected date but got string (Code: 111, Version: 1.12.0)

Manual Fix is changing _SCHEMA session.date to a string.

This should also be fixed in 2.0.4 of parse-server.  Thanks.


Jest - TypeError: Cannot read property 'Commands' of undefined

Do you want to request a feature or report a bug?
bug - related to jest
What is the current behavior?
The error TypeError: Cannot read property 'Commands' of undefined is thrown when i trigger a function that references a <ListView/> and calls the scrollTo method of the <ListView/> like below:
// This is a react-native app
// MyComponentTest.js
it('test scrollTo', () => {
  const tree = renderer.create(<MyComponent/>).toJSON()
  tree.children[0].props.onPress()
})

// MyComponent.js
class MyComponent extends Component {
  _scrollToTop = () => {
    this.refs.listView.scrollTo({x:0,y:0})
  }

  render() {
    return (
      <View>
        <TouchableHighlight onPress={this._scrollToTop}/>
        <ListView
          ref="listView"
          {...moreProps}
        />
      </View>
    )
  }
}

When i run the tests, the stack trace ends at 'Component.scrollResponderScrollTo (node_modules/react-native/Libraries/Components/ScrollResponder.js:380:24)'
The error is caused by the line 'this.refs.listView.scrollTo({x:0,y:0})'...
Package Versions
I run the react-native app with the following package configuration:
"react": "15.4.1"
"react-native": "0.39.2"
"babel-jest": "17.0.2"
"jest": "17.0.3"
"react-test-renderer": "15.4.1"
"babel-preset-react-native": "1.9.0"

I have the same issue as @tobiasMeinhardt

A Workaround to fix the Problem with srcollTo:
`jest.mock('ListView', () => {
const RealComponent = require.requireActual('ListView')
class ListView extends RealComponent {
constructor(props){
super(props)
}
scrollTo = () => {
}

}
return ListView
})`

@fasibio thanks for the workaround!
The workaround also work for ScrollView:
jest.mock('ScrollView', () => {
  const RealComponent = require.requireActual('ScrollView')
  class ScrolLView extends RealComponent {
    scrollTo = () => {}
  }
  return ScrollView
})

Or
jest.mock('ListView', () => {
  const RealComponent = require.requireActual('ListView');
  class ListView extends RealComponent {
    constructor(props) {
      super(props);
      this.scrollTo = () => {};
    }
  }
  return ListView;
});

for the ones that use the airbnb eslint configuration.

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with Canny! Check out its new home: https://react-native.canny.io/feature-requests/p/jest---typeerror-cannot-read-property-commands-of-undefined

Im working on snapshot testing for my react native app, Im facing the same problem when testing the home page. I've used react-native-navigation from wix, and have made an single page application.
$ jest
 FAIL  src/screens/__tests__/Home-test.js
  ✕ Home renders correctly (19ms)

  ● Home renders correctly

    TypeError: Cannot read property 'setOnNavigatorEvent' of undefined

      10 |             pinCode: ''
      11 |         };
    > 12 |         this.props.navigator.setOnNavigatorEvent(this.onNavigatorEvent.bind(this));
      13 |
      14 |     }
      15 |     onNavigatorEvent(event) {

+1 same issue


Replace slack with discord in help center

Pull Request
Related issue
Closes #361
What does this PR do?

Replace Slack with Discord
Remove the now-useless Slack logos

Screenshots

PR checklist
Please check if your PR fulfills the following requirements:

[x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
[x] Have you read the contributing guidelines?
[x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

Hello @mdubus,
I think the current discord invitation link is outdated. Indeed, when trying to join the discord, I get the error: "Invite invalid".
Could you please share/update the link to get access to the discord server? Thanks a lot!
Best,
Adel

Hello @Adel-Moumen !
We fixed the invitation link, it should be ok now. Thanks for reporting it 🙏


Change access identifiers for ImageMarkDownItem and TextAttachment

Fixes #38

Thanks! I'll try to make a new official version later today


[11.0] ISPN-12507 Upgrade to XStream 1.4.14

https://issues.redhat.com/browse/ISPN-12507

integrated! thanks @tristantarrant !


[Bug] Could not find a declaration file for module 'tailwind-scrollbar'.

Could not find a declaration file for module 'tailwind-scrollbar'.

Hi! Can you help me with this error?
I'm using Next.JS with Tailwind CSS.

I have this problem using React.JS and Tailwind v2.01
./src/index.css (./node_modules/css-loader/dist/cjs.js??ref--5-oneOf-4-1!./node_modules/postcss-loader/src??postcss!./src/index.css)
TypeError: require(...) is not a function

Any suggestion?

I have this problem using React.JS and Tailwind v2.01
./src/index.css (./node_modules/css-loader/dist/cjs.js??ref--5-oneOf-4-1!./node_modules/postcss-loader/src??postcss!./src/index.css)
TypeError: require(...) is not a function

Any suggestion?

Just came here for this same reason. Any ideas, I'm using the latest version of Next.js

I have this problem using React.JS and Tailwind v2.01
./src/index.css (./node_modules/css-loader/dist/cjs.js??ref--5-oneOf-4-1!./node_modules/postcss-loader/src??postcss!./src/index.css)
TypeError: require(...) is not a function

Any suggestion?

Just came here for this same reason. Any ideas, I'm using the latest version of Next.js

For now I'm using a class in the tailwind.css file.
.disable-scrollbar {
    scrollbar-width: none; /* Firefox */
    -ms-overflow-style: none;  /* IE 10+ */
}
.disable-scrollbar::-webkit-scrollbar {
    width: 0px;
    background: transparent; /* Chrome/Safari/Webkit */
}

@sudo-kuro @atlanteavila that looks like an unrelated issue: it looks like I added an extra () to the README when I shouldn't have. (Where I've used this in my own projects, I haven't included it, so I'm not sure what I was thinking in putting that into the README.) Please see 37919c40f5cae6949bbd1a7a7d08edc7b06778e1 for the correct installation instructions.
Sorry about that!

@leandroatallah This is a TypeScript error. I'm not much of a TS user, and I'm having trouble replicating the issue; if you have an example repo that demonstrates the issue, I'd be willing to take a crack at making the plugin TS compliant, though. In the meantime, Google's suggesting that there are some solutions you can use on your end (e.g. this).


Form with nested taggable models don't update *_list attributes

I'm seeing this problem in an app I'm working on. I have the following setup:
class Parent < ActiveRecord::Base
  has_many :children
  accepts_nested_attributes_for :children, allow_destroy: true
  attr_accessible :children_attributes
end

class Child < ActiveRecord::Base
  belongs_to :parent
  acts_as_taggable_on :categories, :filters, :keywords
  attr_accessible :title, :category_list, :filter_list, :keyword_list
end

Now, I have a form for Parent which has fields for Child which include text inputs for category_list, filter_list, and keyword_list.
If I have a form specifically for Child with these fields, I can create and edit them just fine.
In the nested form, however, I can create them, but editing them doesn't ever work.
At first I thought it was related to the dirty module on multiple contexts, but fixing that doesn't change anything. I finally wrote some feature specs to test out the behavior, and I'm finding that even if I change something that isn't from acts_as_taggable_on changes with the list fields, ensuring that the child object would be dirty and thus updated, the tests still fail. The title attribute changes successfully, but the *_list attributes will not change from a nested form.

I just ran into this as well... did you find a fix? From what I can tell, the Child is not being marked as being dirty... I tried to mess around with the gem, but found this was just as simple...
class Bread < ActiveRecord::Base
  acts_as_taggable
  acts_as_taggable_on :descriptors

  def descriptor_list=(values)
    set_values = super(values)
    updated_at_will_change! if descriptor_list_changed?
    set_values
  end
end

You can change the update_at to say, an id or something else... I just used updated_at because nearly all of our records use a timestamp of some nature.

descriptor_list_changed? is always false for me in my case. Any tips?

I haven't ran into the descriptor_list not being marked dirty, are you sure it is being set? That seems odd to me that it wouldn't be working.
We currently use #723 in our gemfile which includes the fix. I'm not sure where acts-as-taggable-on stands in their master branch.

Problem still persists today. Please merge #723.


when I use gevent patch_all, the jaeger won't have span record

any solution?

This package is no longer being maintained, users should migrate to the opentelemetry-api package.
Closing.


Create OWNERS for ar localization

Adding OWNERS to fix the issue with approve for arabic team.
Slack discussion :  https://kubernetes.slack.com/archives/CP9FKRD51/p1710921248918939

/assign @a-mccarthy

Thanks for opening this! I think you also need to add sig-docs-ar-reviews to the https://github.com/kubernetes/website/blob/dev-1.29-ar.1/OWNERS_ALIASES file. Right now you just have sig-docs-ar-approvers in that file but both list sig-docs-ar-reviews and sig-docs-ar-approvers in ins PR adding content/ar/OWNERS
Are you also willing to be a reviewer for AR? If so, i'd add your name on to the reviews list as well.

Thanks for opening this! I think you also need to add sig-docs-ar-reviews to the https://github.com/kubernetes/website/blob/dev-1.29-ar.1/OWNERS_ALIASES file. Right now you just have sig-docs-ar-approvers in that file but both list sig-docs-ar-reviews and sig-docs-ar-approvers in ins PR adding content/ar/OWNERS
Are you also willing to be a reviewer for AR? If so, i'd add your name on to the reviews list as well.

done

/lgtm
thanks!

/approve


ProjectServiceTests Fails

https://github.com/gmarland/GhostRunner/blob/master/GhostRunner.Tests/SL/ProjectServiceTests.cs#L73
Test doesn't match DeleteProject signature of public Boolean DeleteProject(String projectId, String packageCacheLocation)

Thanks for finding this. I will investigate.

Fixed. All tests should now be passing


UI improvement for course details page

UI improvement for course details page
Change the UI to a more beautiful UI
**Steps to reproduce **

Design a mockup with Figma
Implement the mockup on android studio

Screenshots

Would you like to work on the issue?
Yes

I guess we have a good UI here. You can add a Join/Leave Course button in the bottom of the course home fragment.

Check the below URL:  https://outreachdashboard.wmflabs.org/courses/Consumer_Reports/Wikipedia_Volunteer_Corps_Course_(winter_2015)


Version 0.1.0

Readme
Active Storage Abstract Class

Great suggestion, adding active_storage as a dependency. Building on that, I working on building out v0.2 as a rails plugin instead of a standalone gem. That should make it easy to leverage the rails environment.

I was at a loss at how best to go about the specs. Since the class basically forwards communication between active_storage and cloudinary, I try to assuming the two collaborating parties are well tested in themselves and avoid the trap of repeating tests for them.
Instead, I aimed to test that the communication happens between them, hence the heavy reliance on expect(xyz).to have_received ...
Do you think there're possible flaws to this approach?


3D point with improper GPS coordinate

Dear Sir or Madam,
I successfully reconstructed with openSFM, however the output GPS coordinate differs with the imput exif data(the pic above is the oringal one).
Is there any method to aglin point clound xyz  to the origin GPS coordinate in reconstruction process?

Hi @zxdragon21 ,
The gps position is reported relative to the reference (the values are in the same json).
Just take a look at TopocentricConverter in geo.py.

Dear Fabianschenk,
I used geo.py to generate the file reconstruction.geojson, and I found the result coordinate is the one I want. However, the counterpart in reconstruction.geojson is not. Therefore, is there any method to add GPS constraint of the coordinates in triangulation or global bundle adjustment? So that no need to transfrom the 3D points coordinates in an additional process.
Thank you!

Hi @zxdragon21 ,
The GPS coordinates are added to triangulation and BA but in a local reference frame with local coordinates. That's also the reason why this conversion from local to global coordinates is required for the export.
Maybe setting the reference to [0,0,0] directly in the code or maybe in the initial .json could achieve what you want.
Best,
Fabian

Hi,
If you want lat/lon coordinates, you can use the export_gps or export_geojson binaries : https://github.com/mapillary/OpenSfM/blob/main/bin/export_geojson and https://github.com/mapillary/OpenSfM/blob/main/bin/export_gps
Yann

That sounds good!
Thank you
----- 原始邮件 -----
发件人："Yann N." @.>
收件人：mapillary/OpenSfM @.>
抄送人：zxdragon21 @.>,  Mention @.>
主题：Re: [mapillary/OpenSfM] 3D point with improper GPS coordinate (Issue #901)
日期：2022年06月21日 17点15分
Hi,
If you want lat/lon coordinates, you can use the export_gps or export_geojson binaries : https://github.com/mapillary/OpenSfM/blob/main/bin/export_geojson and https://github.com/mapillary/OpenSfM/blob/main/bin/export_gps
Yann
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


added denoland

I copied from https://github.com/kawarimidoll/deno-cli-tools/blob/30107adad1dd7d00da8be08c76f843bbfd88b455/denosay/mod.ts#L64-L73
because MIT License
https://github.com/kawarimidoll/deno-cli-tools/blob/main/LICENSE

$ fortune | cowsay -f deno
 __________________________________________
/   It was a fine, sweet night, the nicest \
| since my divorce, maybe the nicest since |
|    the middle of my marriage.  There was |
| energy, softness, grace and laughter.    In my circle, |
|       that means class.               -- |
|      Andrew Bergman "The Big Kiss-off of |
\                                    1944" /
 ------------------------------------------
     \
      \  _
        ( ･ヽ
          \ \
           ⎞ \
           |  ｀ヽ
           ⎩      ﾄ､
            u¯u︶u


model.evaluate_generator(validation_generator) error with TensorFlow 1.11.0

Hi there! I met an unexpected problem in model.evaluate_generator(validation_generator) with TensorFlow 1.11.0 as backend. My environment is Windows 2016 Server, Python 3.6.5 and Keras 2.2.4. The same code works fine with TensorFlow 1.10.0 but not good in TensorFlow 1.11.0. Traceback is as followed:
Traceback (most recent call last):
  File "C:/CNN/AlexNet.py", line 166, in <module>
    Val_acc = 100 * model.evaluate_generator(validation_generator)[1]
  File "C:\Users\Saber\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Saber\Anaconda3\lib\site-packages\keras\engine\training.py", line 1472, in evaluate_generator
    verbose=verbose)
  File "C:\Users\Saber\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 301, in evaluate_generator
    raise ValueError('`steps=None` is only valid for a generator'
ValueError: `steps=None` is only valid for a generator based on the `keras.utils.Sequence` class. Please specify `steps` or use the `keras.utils.Sequence` class.

Any idea about this? Thanks!

Since you are giving a generator, and a generator should yield data infinitly, now there is a steps argument so that the function know when to stop. Now idea about why there is a difference of behavior with TF 1.10. Could you provide a script to reproduce?

Might be connected with the issue #11452 I submitted today. I suspect this is an inheritance mix-up because both Tensorflow and Keras provide the Sequence class.

We'll track this issue in #11452 .


I Might just be dumb but I cant find the libfile to inject into mac

I have csgo-run i just need the libfile

This is just the source code, you need to download XCode and compile this

well now I i have to learn how to do that. thank you for the help

I need help i have been trying for 2 hours and I still don't know how to compile this


add: wsysmon-bin

Does this program have an icon?

This project does not have an icon, but I found a similar project with an icon that could be used. Is it worth it to do so?

https://github.com/KrispyCamel4u/SysMonTask/blob/master/icons/SysMonTask.png

Does this program have an icon?

This project does not have an icon, but I found a similar project with an icon that could be used. Is it worth it to do so?
 https://github.com/KrispyCamel4u/SysMonTask/blob/master/icons/SysMonTask.png

If you want, sure. However I'd recommend not using other projects icons.


WebSite  : LocalizationWebSite

Hi,
The website example contains errors, in the method GetStringSafely of the class TestStringLocalizer, the search is into the directories "Resources\bin" which not exist, and the extension file that use is resx and not .resources.
After making changes, the ResourceManager class don't load the strings, the count is zero.

@kirthik Can you please take a look?

Dnx does not support creating .resources files or satellite assemblies yet. I am creating .resources as part of localization test.
@branciat, you could run the code in https://github.com/aspnet/Mvc/blob/dev/test/Microsoft.AspNet.Mvc.FunctionalTests/LocalizationTest.cs#L140 passing resource file name for creating .resources as a temporary workaround.
Once dnx supports adding .resources file, this workaround would not be needed.

Thanks i look that today.

@kirthik  It's all right, the files are converted and the localization also.
I close the issue.
Thank


Why don't update qunit from 1.11 to 1.16

the current version of qunit ship with tested is 1.11, which don't have feature like
assert.async()
I just want to know the reason why hesitated to upgrade

If you would like to upgrade it, please send in a pull request and I will test and merge.

#460 I did my best, hopefully don't break anything


[Question]: Planned Integration with Ragflow in Infinity 0.4

Describe your problem
It is anticipated that Infinity 0.4 will include integration with Ragflow. Could you provide clarification on how this integration will be implemented? Will Elasticsearch be replaced outright, or will users have the flexibility to choose their preferred search engine?
Additionally, for those already using an alternative search engine (e.g., Typesense), will there be support for integrating existing solutions? My goal is to experiment with various databases and search methods to identify the most suitable option for our use case.

Infinity will be used an alternative at first. And Elasticsearch will still exist while user could choose which one to use. For some advanced features such as tensor based reranking which is required by some future features, ES based RAGFlow wouldn't provide due to being lack of those functionalities.
Regarding to other search engines, it does not satisfy the requirements, we won't integrate.
For example, typesense is an in-memory solution, it does not support "WAND" for full text search, the hybrid search functionality is much weaker,...,etc.   Other search engines also have similar problems.  RAGFlow aim's to guarantee the eventual effectiveness of RAG, it's not an LLMOps platform, so we'll only integrate the ones that deserve to be integrated, not the ones that are likely to be used


Use SASS

Stylesheets can be written much easier with SASS.

https://github.com/blue-jay/blueprint/commit/22f8ecb5fb971b6c88d119d35d84c87c6b0d2b5d


chore: lock semantic release version to v19 in workflow

Locks semantic release to v19 in workflow as latest version requires node 18

:tada: This PR is included in version 3.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Look for multiple hosts holding the same VIP

This indicates keepalived issues / split brain, which might lead to
hosts trying to reach the API in servers aren't even running the
kube-apiserer
Example: https://issues.redhat.com/browse/AITRIAGE-8553?focusedId=23179977&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-23179977

/lgtm


Add hello world integration tests to demonstrate interaction between test files

This is a small example showing how the base integration test, post deploy test, and test variable files work together.
Submission Checklist

[x] Have you installed and run this change against pre-commit? (pre-commit install)
[x] Are all tests passing? (make tests)
[ ] Have you written unit tests to cover this change?
[x] Is unit test coverage still above 80%?
[x] Have you updated all applicable documentation?
[x] Have you followed the guidelines in our Contributing document?

add a line or two describing this hello-world example.

Done.


The tool doesn't detect a Rename Method refactoring in a case when a method's body is empty

The original tool also doesn't detect this kind of refactoring in this case.


Consider downgrading App Service Plan for TRE services.

The App Service Plan used by default in the TRE in the base, base-adf and airlock-notifier workspace templates is P1v3, which is expensive about £192/month.
The other options in the pulldown menu in the TRE are P1v2 and S1, which are cheaper, but are also deprecated, which isn't great.
It would be nice to experiment with the basic offerings, but the TRE is configured to log everything about the App, which requires the Premium plan.
In particular, the AppServiceFileAuditLogs and AppServiceAntivirusScanAuditLogs log types require premium. However, the AppServiceFileAuditLogs isn't yet available for Apps running in Linux containers (which is all of them in the TRE), and the AppServiceAntivirusScanAuditLogs is still in preview, so not generally available.
Describe the solution you'd like
I'd like to see if we can downgrade to a non-premium tier. This is a per-workspace cost, so with 100+ workspaces running, saving £1/day becomes significant.

I've managed to downgrade my own workspace to a B1, manually - not sure how, I can't reproduce it on other workspaces, they all object to becoming non-premium.
Note also that the S1 doesn't support those two log types, which is interesting, I'd expect Terraform to throw an error at that point, but it doesn't.


SparkFun ESP32 Thing

I got it working with the Adafruit HUZZAH32, now I am trying the SparkFun ESP32 Thing.
After flashing I only get garbled characters from the serial connection. An idea what the problem could be?

I am sorry I don't have any idea.

I use nodemcu

The sparkfun board uses a 26 MHz crystal in stead of the more usual 40MHz. Be sure to set the the option in your ESP dev environment with make menuconfig under esp32 specific options. I have used this code on both the sparkfun esp thing and a nodemcu, and needed to make this change.

It can be set in "make menuconfig" under "Component Config" -> ESP32-Specific -> Main XTAL frequency.
Choices are:
40 MHZ
26 MHz
Autodetect

@streicherlouw thank you it works
@younghyunjo I didn't know they exist, who is the manufacturer of your nodemcu?

@yene
I bought it from here.


📝 Add FastAPI Medium Article: Deploy a dockerized FastAPI application to AWS

Added FastAPI related article to the external links section

Awesome, thanks @vjanz ! :rocket: :memo:
And thanks for the help @ycd and @ArcLightSlavik :coffee:


Fixed issue with creating certs for root domains.

Removed call causing issues with root domains and also updated docs as this is no longer an issue.

I haven't tried it yet, but I have a question.
The current flow is:

Create an order for the cert for a domain
Create a CNAME in Azure DNS to validate the ownership of that domain
Obtain the cert
Install it on App Service

Your code "fixes" the issues with point 4 for root domains, but how do you manage point 2?
Since it is not possible to create in Azure DNS a CNAME for a root domain, in theory the process stops at point 2 so having the code handling point 4 is not useful.
Am I missing something?

I thought for step 2 it actually created a TXT record, not a CNAME?

I thought for step 2 it actually created a TXT record, not a CNAME?

You are right, sorry my fault.
I was referring to an early implementation that is not there anymore.
That's the problem on working on too many things on early morning 😁

Hi, how do you feel about accepting this PR? Have you had a chance to test it?

Hi, how do you feel about accepting this PR? Have you had a chance to test it?

Didn't have much time to test unfortunately, but I'm reaching out to the Fluent SDK owners to understand what that piece of code actually does...
I think we are probablu "missing something" :)

Alright, first of all thanks for your patience.
I've been able to talk with one of the owners of the Fluent SDK so now I have a much clearer picture.
It works this way:

If you already have a hostname bound to your AppService and you just need to associate the SSL cert, then you can remove that part
If instead you need to both create the binding and then associate the cert, you need either the “WithThirdPartyHostnameBinding” or the “WithManagedHostnameBinding”

So now it is down to how we want to implement this. I'd say that, since this is a piece of software that aims to simplify the SSL cert assignment/creation/renewal and nothing more, it would be sufficient letting people know that they need to have a hostname already bound to their AppService.
If you agree, then I will accept the PR and add a comment in the README to point that out.

I agree that this is about SSL creation, so I think creating hostname bindings is outside of that scope.


contributors.svg doesn't work

Hi, my url is https://opencollective.com/hexo-theme-stun/contributors.svg and it returns "Not found".
I've been waiting for many days.

Fixed


Make preview window follow cursor when scrolling

This commit makes the hunk preview window follow the cursor when scrolling.
To see this behavior, open a preview window using :Gitsigns preview_hunk, then use Ctrl-e and Ctrl-y to scroll the buffer. Without this patch, the window will stay where it was opened. With this patch, the window will change position to be follow the cursor as the buffer scrolls.

Looks good. Thanks!


Fix paths

Closes #19
Saved you some time :)
Tested on both Mac and Win

Ack.  I just realized this will break the glob matching in the chokidar watching.    So I'll leave this open so you see my idea, but it won't really work unless you make a watchGlobs or something and that starts to feel messy.  I'll be interested to see how you approach this.

Last commit works cross-platform, tested with HMR on Mac and Windows.  But its not the cleanest.  Feel free to use or discard; no offense here either way :)

@JohnCampionJr I was too careless as I thought my wip commit in on another branch. Thanks a lot for the fix!


make impact comparable

As the tracker priority/severity depends on the highest impacts of all the related affects and flaws so we would like to be easily able to do things like max with it.

@costaconrado this is another small step towards the trackers

@Elkasitu I hope I incorporated your suggestions so please have one more look


Policies/global-privacy-practices.md: fix typo from #143

Fix typo reported in #143

Thanks for your feedback! This typo has been addressed in the Global Privacy Practices update.


Training the model over a networkx dataset

Hi, I have a dataset of networkx objects with subgraph samples and a dataset of egonets in networkx format too. And I trying founds subgraphs for similarity in each egonet. Reading yours paper the problem seems to me a very similar and I'm try test your model for my context.  Can I use a dataset with networkx format for it?
Thanks and congrats for the amazing work 😃

Yes, to use your own data with SubGNN, you just need an edge list for your base graph and a text file that describes the subgraphs and their labels. See the updated README for instructions on how to format the data.
I'm closing the issue, but feel free to post another if you have any additional questions.


Test upgrade Live to 1.5.6

Describe the Issue
We have a new SEA release, And we need to test this update before June
Additional Context
SEA 1.5.6
(https://github.com/aws-samples/aws-secure-environment-accelerator/releases/tag/v1.5.6)
Acceptance Criteria
Execute in Live LZ the regression test plan on the ASEA after Upgrade to 1.5.6

closing as complete


[FR] Support AWS IAM Role Profile

Is your feature request related to a problem? Please describe.
We are attempting to remove the use of access keys and secrets from all AWS Service interactions.  In doing so we would require the use of an IAM Role Profile to be leveraged when using the service broker instead of AWS Access Key and Secret.  This might require the broker to run on a VM instead of an application on cf when running on cf-for-vms.  However, when running on cf-for-k8s EKS does support the ability to leverage a k8s service accounts.
Describe the solution you'd like
Allow for the configuration of IAM Role Profile on an EC2 instance via Instance Profile.
Describe alternatives you've considered
Continue to use AWS Access Key and Secrets and work with our security team to get an exception.
Additional Context
This has come up as apart of our HITRUST compliance audit.  Still waiting to hear if its an absolute necessity or a delighter.
Priority
Medium unless it causes us to fail the audit then High.
Priority Context
Yes this would ensure that the platform continues to use instance and role profiles for obtaining temporary use access keys and secrets which would prevent the possibility of a developer or operator leaking them.
Platform
AWS
Applicable Services
Provisioning of AWS Managed Services would be interesting to see how this would work for brokered services like Dynamo or AWS ElasticSearch Service which could use AWS IAM credentials instead of basic auth for authentication.  I believe that would be scoped to a different issue.  This should strictly be on the provisioning of resources.  We have teams that have done similarly with terraform recently when running on a VM I believe.

Hey Bryan! Will bring this up at standup... may require some further questions/a spike...
Initial take on this is that we probably won't prioritize this until after GA (EOM Sep)

Just to add more context: This is a hard requirement generally for many security-conscious organizations like Banks and Government. Additionally, in some government AWS regions it is impossible to use an Access Key and Secret for longer than 1 hour, so using an IAM instance profile is the preferred option for long-lived credentials.

I can see three potential options for implementing this on cf-for-vms:
1. Deploy the service broker to a VM instead of a CF app.
(+) This would allow users to assign an IAM instance profile to that one VM.
(-) Higher resource usage in the form of an additional VM
(-) Requires supporting a bosh-release install method in addition to / instead of the cf push installation method
2. Use an isolation segment to split 'trusted' service broker apps from normal workloads
You would create an isolation segment to be used only for AWS service broker(s). You would then assign an IAM instance profile to all Diego cells within this isolation segment.
(+) This would allow users to assign an IAM instance profile to just the Diego cells in that isolation segment
(+) The existing cf-push-based deployment should work, no need to create a new bosh release version
(-) Still higher resource usage since you need one or more Diego cells just for the broker
3. Use application security groups to control access to the metadata service
Basically, you would create an application security group that allows traffic on TCP port 80 to the IP <IP_ADDRESS>. AWS IAM instance profiles work by making a series of HTTP requests to the metadata service at <IP_ADDRESS>. The default application security group will deny traffic to that IP. As long as the operator is careful to assign the new ASG only to the service broker, no credentials should be accessible by other apps on the platform.
(+) This would not require any additional compute capacity beyond the existing Diego cells in a deployment
(+) Does not require supporting a bosh release in addition to the cf push installation method
(-) This would require assigning the IAM instance profile to all Diego cells in your deployment (or within an isolation segment if you also use those).
(?) I imagine that some risk assessment teams may not like this approach since it relies on the logical isolation provided by ASGs instead of the 'physical' isolation of a VM, but I could be wrong!
@xyloman Do you have any thoughts on how these options would go with your audit? I haven't had experience with HITRUST yet.
@omerbensaadon Does your team have any input on the technical complexity/long term maintainability of these options?

@Logiraptor woah thank you for the thorough response... ENG will follow up but curious to hear what @xyloman's thoughts are on this.

Seems to me that #2 is the cleanest. @xyloman Any concerns?

@erniebilling I believe that all of them have their pros and cons but none of them I have concerns with. After internal conversations, I believe we are leaning towards the isolation segment given we might have more than just this service broker that could benefit from having a role assumed.  I assume that the cloud service broker would allow configuring what role to assume in case of spinning up a database in a different account?
Also, what thoughts have been given to workloads that might make use of this same flow but to communicate with the backing service.  For instance, ElasticSearch Service supports AWS IAM for communication, and instead of generating an access key and secret would an application be able to have the role dynamically created by the service broker and then assume the role?

@xyloman If the isolation segment that the broker was running in had multiple roles, then it seems like it would be necessary for the broker to be directed which role to assume. We'll look in to that.
For the apps, there is no support in OSBAPI/CCAPI to directly support this as its designed to very explicitly pass credentials around. But it may be possible with isolation segments again, though I'm not sure how that would work if the IAM roles would need to be dynamic.

This is why brokers in the higher security environments while better are not able to meet the rigorous regulatory requirements in regards to IAM roles. The answer might simply be in cf for vms we leverage access keys generated by the broker. However, for higher security environments we might only be able to accomplish this on kubernetes because it can pass a role to a pod. In personally have only read about this and have not used it in practice.

Here's what I've done at this point:
Setup

Installed Tanzu Application Service for VMs 2.10.4
Added an isolation segment
Assigned an IAM instance profile to all diego cells in that isolation segment
Created an application security group to allow traffic to the instance metadata service
Created an org bound to the isolation segment
Bound the ASG from (4) to a space in the org from (5)
Installed the cloud service broker into the space from (5) by following the steps here: https://github.com/pivotal/cloud-service-broker/blob/master/docs/aws-installation.md. I did not configure AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY.

Usage
For this first spike, I am just aiming to get an RDS instance created and bound to an app.
Running the following gives an error.
cf create-service csb-aws-mysql small NAME -c '{"aws_vpc_id": "vpc-0de4b7e9566e1593e", "region": "us-east-1"}'

Creating service instance access-key-mysql in org isolated-apps / space apps as admin...
Service broker error: 1 error(s) occurred: couldn't compute the value for "aws_secret_access_key", template: "${config(\"aws.secret_access_key\")}", config: Missing config value aws.secret_access_key
FAILED

Running the same thing with blank credentials actually works.
cf create-service csb-aws-mysql small NAME -c '{"aws_vpc_id": "vpc-0de4b7e9566e1593e", "region": "us-east-1", "aws_access_key_id": "", "aws_secret_access_key": ""}'```

In other words, the basic functionality is already there, presumably because terraform already supports IAM instance profiles.

I've only tried MySQL, but looking at the terraform templates I don't see why Postgres and Elastic search wouldn't work. S3 requires hitting the AWS API in order to use the bucket, so that feels out of scope for my test right now. We would need to place the end-user application in an isolation segment, which is potentially not a viable option.

# Still TODO

* [ ] I haven't checked, but I assume the credentials are marked required in the tile. They should be optional.
* [ ] Allow specifying a role for terraform to assume besides the one bound to the instance. We can use the `assume_role` configuration in terraform for this: https://registry.terraform.io/providers/hashicorp/aws/latest/docs#assume-role. This is useful if you want multiple AWS brokers to co-exist in the same isolation segment. It will allow each one to assume a separate role at runtime.
* [ ] Decide if there is anything to be done with the s3 service. In my customer's case, we are not able to automate IAM users at all, so we would probably skip using the s3 broker at this time. For other customers, you could create the bucket via an IAM instance profile and get an exception for the application to reach s3.

@Logiraptor  ironically I am working on other tasks that are making use of AWS IAM for applications running on CF that make use of terraform for ACME provider.  I can confirm the the assume role works and would also work for cross account roles.  Assuming that the role that you are assuming into has been granted to the role that the Diego cell is operating with.  This is very exciting.
Curious what is the backend config for the terraform in this case?  For my use case, I have an iam role that is used for backend config and then I was dabbling with another IAM role for the actual operations that terraform would be performing.
We have the same desire to leverage S3 with the broker and we too are hung up with the in ability to manage IAM directly with terraform because of bucket policies.  This is a bummer because I fear there will be other services like S3 that will use IAM polices for managing access.  I think I have seen DynamoDB work this same way.

Awesome that completely makes sense.  Question, will it be possible for the assumption of roles.  We are finding it easier to grant the role assumed by the instance profile a role that has the permissions to assume other roles.  This could be beneficial if we desire to have the services run in a different account than say the one that service broker is running with in.

Curious what is the backend config for the terraform in this case?

I took a look in the code and it appears that there is no explicit backend config. So in other words, the terraform state is written to disk inside the app container, and then that state is persisted to the DB by the broker.
Looks like there is code here to manage that process (write state from DB to disk before running terraform, then save new state to DB after).
I'm not on the core CSB team, but I imagine that decision was made to keep it cloud-agnostic rather than requiring s3 access for example.

This is correct, terraform state is serialized and kept in the metadatadb to keep the broker as deployment agnostic as possible.

Here's what I've done at this point:
Setup
1. Installed Tanzu Application Service for VMs 2.10.4

2. Added an isolation segment

3. Assigned an IAM instance profile to all diego cells in that isolation segment

4. Created an application security group to allow traffic to the instance metadata service

5. Created an org bound to the isolation segment

6. Bound the ASG from (4) to a space in the org from (5)

7. Installed the cloud service broker into the space from (5) by following the steps here: https://github.com/pivotal/cloud-service-broker/blob/master/docs/aws-installation.md. **I did not configure AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY**.

Usage
For this first spike, I am just aiming to get an RDS instance created and bound to an app.
Running the following gives an error.
cf create-service csb-aws-mysql small NAME -c '{"aws_vpc_id": "vpc-0de4b7e9566e1593e", "region": "us-east-1"}'

Creating service instance access-key-mysql in org isolated-apps / space apps as admin...
Service broker error: 1 error(s) occurred: couldn't compute the value for "aws_secret_access_key", template: "${config(\"aws.secret_access_key\")}", config: Missing config value aws.secret_access_key
FAILED

Running the same thing with blank credentials actually works.
cf create-service csb-aws-mysql small NAME -c '{"aws_vpc_id": "vpc-0de4b7e9566e1593e", "region": "us-east-1", "aws_access_key_id": "", "aws_secret_access_key": ""}'

In other words, the basic functionality is already there, presumably because terraform already supports IAM instance profiles.
I've only tried MySQL, but looking at the terraform templates I don't see why Postgres and Elastic search wouldn't work. S3 requires hitting the AWS API in order to use the bucket, so that feels out of scope for my test right now. We would need to place the end-user application in an isolation segment, which is potentially not a viable option.
Still TODO
* [ ]  Allow specifying a role for terraform to assume besides the one bound to the instance. We can use the `assume_role` configuration in terraform for this: https://registry.terraform.io/providers/hashicorp/aws/latest/docs#assume-role. This is useful if you want multiple AWS brokers to co-exist in the same isolation segment. It will allow each one to assume a separate role at runtime.

* [ ]  I haven't checked, but I assume the credentials are marked required in the tile. They should be optional.

* [ ]  Decide if there is anything to be done with the s3 service. In my customer's case, we are not able to automate IAM users at all, so we would probably skip using the s3 broker at this time. For other customers, you could create the bucket via an IAM instance profile and get an exception for the application to reach s3.

This is great information @Logiraptor, thanks for sharing.

hey @xyloman  we have put a spike in the backlog to understand how we can implement IAM access and service key access side-by-side without creating a new brokerpak...
thank you to @Logiraptor for his work here. We'll let you know soon!

Agreed that s3 is going to be the hardest one to integrate with.  The reason it exclusively uses IAM and IAM policies.  Therefore, if you create a bucket with the broker and cannot create a user with an access key and secret then you are unable to access the bucket unless you assume into a role profile and use ec2 instance credentials.
One option to explore in this area was to leverage IAM Tags: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_iam-tags.html these could provide additional restrictions but I am unfamiliar with how these works.
Basically we need a least privileged way to a user that the application will use that would get destroyed when the binding to the application is destroyed.  The lifecycle management of the access key and secret would actually pass what our security team is looking to accomplish in regards to ensure keys are not long lived.  However, the hang up now is granting the broker the ability to dynamically create the user, s3 bucket and role.  I have not had an opportunity to test the s3 bucket workflow but if this is implemented today and could be managed by assuming a role I would like to present this to my stakeholders and see what they think.

Was also requested by the development team at JPMC

This could be beneficial if we desire to have the services run in a different account than say the one that service broker is running with in.

We need all services to run/be created in a different account to the broker. We currently do this with the AWS service broker and is a hard requirement for us to switch to using CSB and TAS4K8's. Also, we have a requirement from our security teams to only use roles going forward, no access keys for apps/services. (SNSW)

This could be beneficial if we desire to have the services run in a different account than say the one that service broker is running with in.

We need all services to run/be created in a different account to the broker. We currently do this with the AWS service broker and is a hard requirement for us to switch to using CSB and TAS4K8's. Also, we have a requirement from our security teams to only use roles going forward, no access keys for apps/services. (SNSW)


Remix Icon is breaking QuickAccessItems into Header

Context
I just migrated 2 libs in my project :

react-dsfr : 0.54.0 => 0.57.4
tss-react : 4.5.2 => 4.7.4

Then I discovered these regression
Before

After

Here is the code
<Header
        brandTop={brandTop}
        serviceTitle={title}
        homeLinkProps={homeLinkProps}
        quickAccessItems={[
          {
            iconId: "fr-icon-account-circle-line",
            linkProps: {
              href: "#",
              className: classes.disabled,
            },
            text: "Me connecter",
          },
          {
            iconId: "ri-chat-3-line",
            linkProps: {
              href: feedbackLink || "#",
              target: "_self",
            },
            text: "Soumettre mes retours",
          },
          {
            iconId: "ri-mail-line",
            linkProps: {
              href: `mailto:${contactMail}`,
            },
            text: "Nous contacter",
          },
        ]}
        serviceTagline="Prototype - Version1 - 2023"
      />

Solution path ?
I replaced Remix icons by dsfr icons, and it works backs.

With this code
{
  iconId: "fr-icon-account-circle-line",
  linkProps: {
    href: feedbackLink || "#",
    target: "_self",
}

Thank you, I can reproduce.

By the way, this is something the visual regression testing should have cached 😞

Fixed in latest version, thanks for reporting!


Added OAuth user data

issue https://github.com/TryGhost/Team/issues/614

Looks good, are these column sizes definitely what we want? Will they work for all/most providers?

@allouis good point, I've increased the size of the access and refresh tokens to match the maximum length specced by google:

Tokens can vary in size, up to the following limits:

Authorization codes: 256 bytes
Access tokens: 2048 bytes
Refresh tokens: 512 bytes

Changes made:

access token: from 191 to 65535 chars
refresh token: from 191 to 2000 chars

That is huge and should fit most OAuth providers.

I'm creating another PR to reduce the users row size because I can't add these oauth fields

Hey all, after understanding that I wouldn't be able to modify the users table (because we're not yet releasing a major 😅 ), I've changed this PR to create another table linked to users.
It should be a 1:1 relation in the upcoming implementation, but I prefer to keep the door open to a user that has multiple OAuth providers configured, which is why the foreign key is from the oauth table to users.

@tpatel Reminder that we need to add an entry to https://github.com/TryGhost/Ghost/blob/main/core/server/data/exporter/index.js#L13-L47

@tpatel We'll need to update this to 4.11 now 🙂

@tpatel We'll need to update this to 4.11 now 🙂

Right! it's done ✅


Introduction to Remote Procudure Call

Introduction to Remote Procudure Call
Remote procedure call refers to a method of constructing distributed systems. In essence, it enables a software on one system to call a function on another machine without the latter being aware that it is remote.This technique is used for communication between  the client and the server.
Key takeaways
What is a remote procedure call?
Events involved in the remote procedure call.
Diagramatic implementation of a remote procedure call.
Program to implement a remote procedure call in both the server and the client.
Benefits of remote procedure call.
Drawbacks of the remote procedure call.
Article quality
This article is unique as it covers the topic in-depth with additional information about remote procedure calls and implementation of a remote procedure call in both architecture and client-server programs.
References
N/A

Thank you for your response and we thank you for submitting your topic. After some careful consideration it struck us that this topic may be a bit over saturated throughout other blog sites and official documentations such as these ones


Change 6V_REG soft-start cap to 0.1uf

6V_REG works on cape but only starts when no load is present on Blue. Cause is likely the 1nF capacitor C21 on Blue. Should change to 0.1uF to match cape.

Updated schematics and BOM to change the value of C21.

Can we modify a board to confirm?

I'll have a go tomorrow.
On Tue, Nov 1, 2016 at 7:38 PM, Jason Kridner<EMAIL_ADDRESS>wrote:

Can we modify a board to confirm?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jadonk/beaglebone-blue/issues/35#issuecomment-257756610,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF9HoOTT6GEIHhHzuXbyKYap_OcCZ2ANks5q5_eYgaJpZM4Kdz95
.

confirmed, the capacitor change fixes soft-start
On Wed, Nov 2, 2016 at 1:11 AM, James Strawson<EMAIL_ADDRESS>wrote:

I'll have a go tomorrow.
On Tue, Nov 1, 2016 at 7:38 PM, Jason Kridner<EMAIL_ADDRESS>wrote:

Can we modify a board to confirm?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jadonk/beaglebone-blue/issues/35#issuecomment-257756610,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF9HoOTT6GEIHhHzuXbyKYap_OcCZ2ANks5q5_eYgaJpZM4Kdz95
.

@StrawsonDesign Thanks!!


Using uuid lib to avoid conflicts

The UUID used before was a function that just increments a variable. Just to avoid conflicts as a precaution I made this PR that uses a small lib to generate a UUID value using RFC documentation.

I was thinking about doing the same thing. Thanks! 👍

@lelandrichardson You're welcome. Thanks for merge.


Training on custom dataset

i want train the model for detection and extraction of tables. I am not sure on how the data should be labeled and the training procedure can you help me with that

Hi, thank you for your question.
Maybe you can have a look in the discussion section: There is a chat about how to setup a custom dataset and how to transform data into the library framework.
Feel free to ask any further questions that haven't been covered so far.

Hi, thank you for your question.
Maybe you can have a look in the discussion section: There is a chat about how to setup a custom dataset and how to transform data into the library framework.
Feel free to ask any further questions that haven't been covered so far.

the table extractor is trained on pubtabnet, i downloaded the dataset. and checking the labels in jsonl file. Is that how i am supposed to label my dataset. If yes can you tell me which tool they are using for labelling