CG-50 Content structured data markup

Added getFirstImageUrl and stDataMarkup in ContentPresenter for JSON-LD Structured Data Markup

Coverage decreased (-1.8%) to 47.184% when pulling 7863745a2b80f5626d56c8f22242d9ff1b6924d6 on CG-50_content_structured_data_markup into 8df7bbcbdb97bacc3b43a4b07445ce131dbdbbbe on master.

Coverage decreased (-1.8%) to 47.184% when pulling 788059e427cfe9b4febee7453620f31bc0bac3ff on CG-50_content_structured_data_markup into 8df7bbcbdb97bacc3b43a4b07445ce131dbdbbbe on master.

:+1:


ModuleNotFoundError: No module named 'graphgpt'

ModuleNotFoundError: No module named 'graphgpt' when runing sh ./scripts/tune_script/graphgpt_stage1.sh
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
    from graphgpt.train.llama_flash_attn_monkey_patch import (
ModuleNotFoundError:     No module named 'graphgpt'from graphgpt.train.llama_flash_attn_monkey_patch import (

ModuleNotFoundError: No module named 'graphgpt'
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
    from graphgpt.train.llama_flash_attn_monkey_patch import (
ModuleNotFoundError: No module named 'graphgpt'
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
    from graphgpt.train.llama_flash_attn_monkey_patch import (
ModuleNotFoundError: No module named 'graphgpt'
[2024-02-13 15:35:41,869] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 1123051) of binary: /afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/bin/python
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/run.py", line 810, in <module>
    main()
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main
    run(args)
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
graphgpt/train/train_mem.py FAILED
------------------------------------------------------------
Failures:
[1]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 1 (local_rank: 1)
  exitcode  : 1 (pid: 1123052)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 2 (local_rank: 2)
  exitcode  : 1 (pid: 1123053)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[3]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 3 (local_rank: 3)
  exitcode  : 1 (pid: 1123054)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 1123051)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

I meet the same problem. Has it been resolved?

ModuleNotFoundError: No module named 'graphgpt' when runing sh ./scripts/tune_script/graphgpt_stage1.sh
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
    from graphgpt.train.llama_flash_attn_monkey_patch import (
ModuleNotFoundError:     No module named 'graphgpt'from graphgpt.train.llama_flash_attn_monkey_patch import (

ModuleNotFoundError: No module named 'graphgpt'
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
    from graphgpt.train.llama_flash_attn_monkey_patch import (
ModuleNotFoundError: No module named 'graphgpt'
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/DIAL-Lab/GraphGPT/graphgpt/train/train_mem.py", line 4, in <module>
    from graphgpt.train.llama_flash_attn_monkey_patch import (
ModuleNotFoundError: No module named 'graphgpt'
[2024-02-13 15:35:41,869] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 1123051) of binary: /afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/bin/python
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/run.py", line 810, in <module>
    main()
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main
    run(args)
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/afs/crc.nd.edu/user/k/kle3/.conda/envs/GraphGPT/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
graphgpt/train/train_mem.py FAILED
------------------------------------------------------------
Failures:
[1]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 1 (local_rank: 1)
  exitcode  : 1 (pid: 1123052)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 2 (local_rank: 2)
  exitcode  : 1 (pid: 1123053)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[3]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 3 (local_rank: 3)
  exitcode  : 1 (pid: 1123054)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-02-13_15:35:41
  host      : qa-a100-002.crc.nd.edu
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 1123051)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

Hello, all shell files need to be run directly under the ./GraphGPT directory. Specifically, before running, please first run cd /path/to/GraphGPT


E2E for MXJob is the flaky test

/kind bug
E2E for MXJob is the flaky test. We should fix the test.
INFO     root:utils.py:36

MXJob is running
INFO     root:utils.py:47 mxjob-mnist-ci-test            Running              2023-01-25 19:02:12+00:00
INFO     root:utils.py:47 mxjob-mnist-ci-test            Running              2023-01-25 19:02:12+00:00
INFO     root:utils.py:47 mxjob-mnist-ci-test            Running              2023-01-25 19:02:12+00:00
INFO     root:utils.py:47 mxjob-mnist-ci-test            Running              2023-01-25 19:02:12+00:00
INFO     root:utils.py:47 mxjob-mnist-ci-test            Running              2023-01-25 19:02:12+00:00
INFO     root:utils.py:47 mxjob-mnist-ci-test            Running              2023-01-25 19:02:12+00:00
INFO     root:utils.py:47 mxjob-mnist-ci-test            Failed               2023-01-25 19:03:34+00:00
=========================== short test summary info ============================
FAILED sdk/python/test/e2e/test_e2e_mxjob.py::test_sdk_e2e - RuntimeError: MXJob default/mxjob-mnist-ci-test is Failed. MXJob conditions: [{'last_transition_time': datetime.datetime(2023, 1, 25, 19, 1, 57, tzinfo=tzlocal()),
 'last_update_time': datetime.datetime(2023, 1, 25, 19, 1, 57, tzinfo=tzlocal()),
 'message': 'MXJob mxjob-mnist-ci-test is created.',
 'reason': 'MXJobCreated',
 'status': 'True',
 'type': 'Created'}, {'last_transition_time': datetime.datetime(2023, 1, 25, 19, 2, 12, tzinfo=tzlocal()),
 'last_update_time': datetime.datetime(2023, 1, 25, 19, 2, 12, tzinfo=tzlocal()),
 'message': 'MXJob mxjob-mnist-ci-test is running.',
 'reason': 'MXJobRunning',
 'status': 'False',
 'type': 'Running'}, {'last_transition_time': datetime.datetime(2023, 1, 25, 19, 3, 34, tzinfo=tzlocal()),
 'last_update_time': datetime.datetime(2023, 1, 25, 19, 3, 34, tzinfo=tzlocal()),
 'message': 'mxjob mxjob-mnist-ci-test is failed because 1 Worker replica(s) '
            'failed.',
 'reason': 'MXJobFailed',
 'status': 'True',
 'type': 'Failed'}]
============= 1 failed, 32 passed, 6 skipped in 692.86s (0:11:32) ==============
Error: Process completed with exit code 1.

https://github.com/kubeflow/training-operator/actions/runs/4009038764/jobs/6883927326
/cc @johnugeorge

/kind cancel

/kind e2e-test-failure

Yeah. This is the cause of failure in most integration tests. Need to root cause the problem
Thanks @tenzen-y for creating this.

It seems to face connection errors. Maybe, we need to fix the sample codes.
mxjob-mnist-ci-test-worker-0 mxnet INFO:root:start with arguments Namespace(add_stn=False, batch_size=64, disp_batches=100, dtype='float32', gc_threshold=0.5, gc_type='none', gpus=None, image_shape='1, 28, 28', initializer='default', kv_store='dist_sync', load_epoch=None, loss='', lr=0.05, lr_factor=0.1, lr_step_epochs='10', macrobatch_size=0, model_prefix=None, mom=0.9, monitor=0, network='mlp', num_classes=10, num_epochs=1, num_examples=1000, num_layers=None, optimizer='sgd', profile_server_suffix='', profile_worker_suffix='', save_period=1, test_io=0, top_k=0, use_imagenet_data_augmentation=0, warmup_epochs=5, warmup_strategy='linear', wd=0.0001)
mxjob-mnist-ci-test-worker-0 mxnet DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): web.archive.org:443
mxjob-mnist-ci-test-worker-0 mxnet DEBUG:urllib3.connectionpool:https://web.archive.org:443 "GET /web/20160828233817/http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz HTTP/1.1" 302 0
mxjob-mnist-ci-test-worker-0 mxnet DEBUG:urllib3.connectionpool:https://web.archive.org:443 "GET /web/20161113155520/http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz HTTP/1.1" 200 28881
mxjob-mnist-ci-test-worker-0 mxnet incubator-mxnet/example/image-classification/train_mnist.py:38: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
mxjob-mnist-ci-test-worker-0 mxnet   label = np.fromstring(flbl.read(), dtype=np.int8)
mxjob-mnist-ci-test-worker-0 mxnet DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): web.archive.org:443
mxjob-mnist-ci-test-worker-0 mxnet Traceback (most recent call last):
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 175, in _new_conn
mxjob-mnist-ci-test-worker-0 mxnet     (self._dns_host, self.port), self.timeout, **extra_kw
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/util/connection.py", line 95, in create_connection
mxjob-mnist-ci-test-worker-0 mxnet     raise err
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/util/connection.py", line 85, in create_connection
mxjob-mnist-ci-test-worker-0 mxnet     sock.connect(sa)
mxjob-mnist-ci-test-worker-0 mxnet ConnectionRefusedError: [Errno 111] Connection refused
mxjob-mnist-ci-test-worker-0 mxnet
mxjob-mnist-ci-test-worker-0 mxnet During handling of the above exception, another exception occurred:
mxjob-mnist-ci-test-worker-0 mxnet
mxjob-mnist-ci-test-worker-0 mxnet Traceback (most recent call last):
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 710, in urlopen
mxjob-mnist-ci-test-worker-0 mxnet     chunked=chunked,
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 386, in _make_request
mxjob-mnist-ci-test-worker-0 mxnet     self._validate_conn(conn)
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
mxjob-mnist-ci-test-worker-0 mxnet     conn.connect()
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 358, in connect
mxjob-mnist-ci-test-worker-0 mxnet     self.sock = conn = self._new_conn()
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 187, in _new_conn
mxjob-mnist-ci-test-worker-0 mxnet     self, "Failed to establish a new connection: %s" % e
mxjob-mnist-ci-test-worker-0 mxnet urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x404f5c92d0>: Failed to establish a new connection: [Errno 111] Connection refused
mxjob-mnist-ci-test-worker-0 mxnet
mxjob-mnist-ci-test-worker-0 mxnet During handling of the above exception, another exception occurred:
mxjob-mnist-ci-test-worker-0 mxnet
mxjob-mnist-ci-test-worker-0 mxnet Traceback (most recent call last):
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/requests/adapters.py", line 450, in send
mxjob-mnist-ci-test-worker-0 mxnet     timeout=timeout
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 786, in urlopen
mxjob-mnist-ci-test-worker-0 mxnet     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/urllib3/util/retry.py", line 592, in increment
mxjob-mnist-ci-test-worker-0 mxnet     raise MaxRetryError(_pool, url, error or ResponseError(cause))
mxjob-mnist-ci-test-worker-0 mxnet urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='web.archive.org', port=443): Max retries exceeded with url: /web/20160828233817/http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x404f5c92d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
mxjob-mnist-ci-test-worker-0 mxnet
mxjob-mnist-ci-test-worker-0 mxnet During handling of the above exception, another exception occurred:
mxjob-mnist-ci-test-worker-0 mxnet
mxjob-mnist-ci-test-worker-0 mxnet Traceback (most recent call last):
mxjob-mnist-ci-test-worker-0 mxnet   File "incubator-mxnet/example/image-classification/train_mnist.py", line 97, in <module>
mxjob-mnist-ci-test-worker-0 mxnet     fit.fit(args, sym, get_mnist_iter)
mxjob-mnist-ci-test-worker-0 mxnet   File "/mxnet/incubator-mxnet/example/image-classification/common/fit.py", line 182, in fit
mxjob-mnist-ci-test-worker-0 mxnet     (train, val) = data_loader(args, kv)
mxjob-mnist-ci-test-worker-0 mxnet   File "incubator-mxnet/example/image-classification/train_mnist.py", line 56, in get_mnist_iter
mxjob-mnist-ci-test-worker-0 mxnet     'train-labels-idx1-ubyte.gz', 'train-images-idx3-ubyte.gz')
mxjob-mnist-ci-test-worker-0 mxnet   File "incubator-mxnet/example/image-classification/train_mnist.py", line 39, in read_data
mxjob-mnist-ci-test-worker-0 mxnet     with gzip.open(download_file(base_url+image, os.path.join('data',image)), 'rb') as fimg:
mxjob-mnist-ci-test-worker-0 mxnet   File "/mxnet/incubator-mxnet/example/image-classification/common/util.py", line 42, in download_file
mxjob-mnist-ci-test-worker-0 mxnet     r = requests.get(url, stream=True)
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/requests/api.py", line 75, in get
mxjob-mnist-ci-test-worker-0 mxnet     return request('get', url, params=params, **kwargs)
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/requests/api.py", line 61, in request
mxjob-mnist-ci-test-worker-0 mxnet     return session.request(method=method, url=url, **kwargs)
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/requests/sessions.py", line 529, in request
mxjob-mnist-ci-test-worker-0 mxnet     resp = self.send(prep, **send_kwargs)
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/requests/sessions.py", line 645, in send
mxjob-mnist-ci-test-worker-0 mxnet     r = adapter.send(request, **kwargs)
mxjob-mnist-ci-test-worker-0 mxnet   File "/usr/local/lib/python3.7/dist-packages/requests/adapters.py", line 519, in send
mxjob-mnist-ci-test-worker-0 mxnet     raise ConnectionError(e, request=request)
mxjob-mnist-ci-test-worker-0 mxnet requests.exceptions.ConnectionError: HTTPSConnectionPool(host='web.archive.org', port=443): Max retries exceeded with url: /web/20160828233817/http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x404f5c92d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

This flaky test seems to be fixed by #1754.

/close


python311Packages.rapidfuzz: 3.6.1 -> 3.6.2

Description of changes
Diff: https://github.com/rapidfuzz/RapidFuzz/compare/refs/tags/v3.6.1...v3.6.2
Changelog: https://github.com/maxbachmann/RapidFuzz/blob/refs/tags/v3.6.2/CHANGELOG.rst

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

closes https://github.com/NixOS/nixpkgs/pull/294327

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 293822 run on x86_64-linux 1

2 packages marked as broken and skipped:

python311Packages.pynitrokey
python311Packages.pynitrokey.dist

4 packages failed to build:

python312Packages.autofaiss
python312Packages.autofaiss.dist
python312Packages.pdf2docx
python312Packages.pdf2docx.dist

92 packages built:

adl
anime-downloader
anime-downloader.dist
apostrophe
atomic-operator
atomic-operator.dist
creds
creds.dist
emacsPackages.lsp-bridge
ibus-engines.uniemoji
instawow
instawow.dist
knowsmore
knowsmore.dist
ledger-autosync
ledger-autosync.dist
mealie
mealie.dist
nitrokey-app2
nitrokey-app2.dist
osc-cli
osc-cli.dist
paperless-ngx
poetry
poetry.dist
poetryPlugins.poetry-audit-plugin
poetryPlugins.poetry-audit-plugin.dist
poetryPlugins.poetry-plugin-up
poetryPlugins.poetry-plugin-up.dist
puddletag
puddletag.dist
pynitrokey
pynitrokey.dist
python311Packages.autofaiss
python311Packages.autofaiss.dist
python311Packages.cleo
python311Packages.cleo.dist
python311Packages.fire
python311Packages.fire.dist
python311Packages.fuzzywuzzy
python311Packages.fuzzywuzzy.dist
python311Packages.jarowinkler
python311Packages.jarowinkler.dist
python311Packages.jiwer
python311Packages.jiwer.dist
python311Packages.levenshtein
python311Packages.levenshtein.dist
python311Packages.nkdfu
python311Packages.nkdfu.dist
python311Packages.pdf2docx
python311Packages.pdf2docx.dist
python311Packages.rapidfuzz
python311Packages.rapidfuzz.dist
python311Packages.syncedlyrics
python311Packages.syncedlyrics.dist
python311Packages.thefuzz
python311Packages.thefuzz.dist
trytond (python311Packages.trytond)
trytond.dist (python311Packages.trytond.dist)
python311Packages.videocr
python311Packages.videocr.dist
python312Packages.cleo
python312Packages.cleo.dist
python312Packages.fire
python312Packages.fire.dist
python312Packages.fuzzywuzzy
python312Packages.fuzzywuzzy.dist
python312Packages.jarowinkler
python312Packages.jarowinkler.dist
python312Packages.jiwer
python312Packages.jiwer.dist
python312Packages.levenshtein
python312Packages.levenshtein.dist
python312Packages.nkdfu
python312Packages.nkdfu.dist
python312Packages.rapidfuzz
python312Packages.rapidfuzz.dist
python312Packages.syncedlyrics
python312Packages.syncedlyrics.dist
python312Packages.thefuzz
python312Packages.thefuzz.dist
python312Packages.trytond
python312Packages.trytond.dist
python312Packages.videocr
python312Packages.videocr.dist
rapidfuzz-cpp
spotdl
spotdl.dist
sublime-music
sublime-music.dist
ulauncher
ulauncher.dist


Remove isolatedModules

@keybase/react-hackers
Without this flag our update time in tsc watch is < 1s for small changes.

For now making a new tsconfig and yarn command yarn tsc-fast. If you get an error from CI or a normal yarn tsc but not in yarn tsc-fast, let me know!
Note: You must delete your old cache rm -rf .tsOut before using this or you will get wrong errors.


Add support for eliot:OnOffLightEliotComponent

Describe the bug
hi.
I have test your integration.
I have in my tahoma box a legrand light which is linked from legrand account.
In the tahoma application the state of my light works good. When i turn on the light in my legrand application, the state in the tahoma application refresh and show me on. But in ha, i can't see the state on of my light. The state is always off and i can't change manually the state.
To Reproduce
Steps to reproduce the behavior:

Go to service in developer tools
Launch light.turn_on in the light entity
The light is on
The state of the light is always off

Expected behavior
When I turn on the light, the state of the entity should be "on" and no "off"
Environment (please complete the following information):

ha-tahoma version: 2.0.1
Home Assistant version: 0.113.3
Platform: light

Device: interrupteur legrand celiane with netatmo (with neutral) linked in tahoma application

Model: StatefulOnOffLight
Type: eliot:OnOffLightEliotComponent

Additional context
{
	"commands": [{
		"commandName": "off",
		"nparams": 0
	}, {
		"commandName": "on",
		"nparams": 0
	}, {
		"commandName": "onWithTimer",
		"nparams": 1
	}, {
		"commandName": "refreshState",
		"nparams": 0
	}, {
		"commandName": "setOnOff",
		"nparams": 1
	}, {
		"commandName": "refreshAllLightStates",
		"nparams": 0
	}],
	"states": [{
		"type": "DataState",
		"qualifiedName": "core:FirmwareRevisionState"
	}, {
		"values": ["off", "on"],
		"type": "DiscreteState",
		"qualifiedName": "core:OnOffState"
	}],
	"dataProperties": [],
	"widgetName": "StatefulOnOffLight",
	"uiClass": "Light",
	"qualifiedName": "eliot:OnOffLightEliotComponent",
	"type": "ACTUATOR"
}

Thanks @HaSHi93110! I haven't seen an eliot: device (legrand) before. It looks related to #167, since there is a command available called refreshAllLightStates.

It would be interesting to dump the tahoma website to see where in their code they use this command. @iMicknl What do you think?

When i turn on the light with tahoma app, thé light appear on in 5 6 seconds in the app, and she is also on in ha.
But when i turn on the entity in ha, the buttons return in off position after 2 3 seconds and the state of the light won't change

When i change the state to on from off by ha woth the service light.turn_off, the light is immediatly off, she appears off about in 2 3 seconds in the tahoma app and 1 2 minutes in ha

After tests, it appears that when i go in tahoma app, thé state off the light is update and ha show the New state. But if i don't go in the app, the state doesn't change

@HaSHi93110 thanks! This is exactly the same as #167. Apparently for some devices, the state is only refreshed when you open the TaHoma app/website or when you execute a command via the TaHoma app/website. We are still figuring this out, so would be great to retrieve a few more details.
The suggestion from @tetienne to make a dump of your tahomalink.com environment would be great. They do a lot of stuff in their website, thus in their JavaScript files. He can provide you with instructions how to make a dump.

OK.
How can i give this ?

On the Tahoma website, just type ctrl+s (or cmd+s if on mac). Select a folder and confirm. You can then zip this file and share it.

@HaSHi93110 could you give https://github.com/iMicknl/ha-tahoma/archive/feature/add_refresh_states_service.zip a try? You can unzip this file and place the tahoma folder in your custom_components folder. Best is to first remove the old one, to make sure you don't have any conflicts.
This will add a new service call named tahoma.refresh_states. Calling this service should trigger an update within 30 seconds (or less, based on your configured update interval). Could you give it a try? We will investigate better solutions in the future in #167.

I will close this issue for now, since we will track this feature in https://github.com/iMicknl/ha-tahoma/issues/167. If you subscribe to that issue, you will get notifications of updates and when it is ready :-).
Please give the zip in my post above a try, would be great to know if this service works for you.


Resolving ElegantPages fails

Hi, I'm trying to add ElegantCalendar as a Swift Package, but it keeps failing when it tries to resolve "Elegant Pages"

I'm experiencing the same problem here. Would definitely like to implement it into my project!

Make sure that you are using Xcode 11 > and ensure that your project configuration supports this library


Getting access to raw audio data

Just wondering if there is anyway to intercept the audio data as a raw data stream of sorts. Need raw audio data so that some signal processing (silence removal) can be done. Thanks!

Yeah, it's definitely possible
I've did that for recording to file
In Android part check out recorder and audio interceptor classes

Ah, alright thanks, I'll close the issue then!.


Add JavaxSecurityServletFilterConfiguration

If spring-security is enabled logbooks autoconfiguration fails. An additional Autoconfiguration when javax.servlet is on the classpath fixes the issue
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.

:+1:

👍


Page and static prefix support proposal for koenbok/Cactus#177

Add a 'page-prefix' and 'static-prefix' for configuration
Manager prefixes in site._get_url_for_static() and site._get_url_for_page()

A few notes (sorry, I have a lot on the side, so not able to be as reactive as I wish to be):

This might break for external URLs (which are returned absolute and would still get the prefix).
There's a bit of duplicate logic here, would it make sense to move it to _get_url? This would also let you solve the problem of external URLs more easily.
Can we get some tests added? Both in cases where page-prefix and static-prefix are set?
We might also want some docs. Historically we've been bad at this, but no one will find this feature unless it's at least somewhere in the README.
Can you briefly explain the use case for this? Not that I don't believe it's useful, but I just want to make sure there aren't other changes we should be making.

Thanks!

Can you briefly explain the use case for this? Not that I don't believe it's useful, but I just want to make sure there aren't other changes we should be making.

I want to drop my files in http://www.mysite.com/subdir/
There is still a problem: sitemap is generated based on site-url, but only uses the main domain name and does not include the prefix.

This might break for external URLs (which are returned absolute and would still get the prefix).

Ya, that a problem. One (weak) solution is to use normal URL instead of static and url tags.

There's a bit of duplicate logic here, would it make sense to move it to _get_url? This would also let you solve the problem of external URLs more easily.
Can we get some tests added? Both in cases where page-prefix and static-prefix are set?
We might also want some docs. Historically we've been bad at this, but no one will find this feature unless it's at least somewhere in the README.

The final submission will be factorized / optimized, I'm just want feedbacks first to see if mu proposal is interesting and correct.

Tests are working, your turn.

@afrigeri the '/static' pattern is used as-is in the Cactus source, see https://github.com/koenbok/Cactus/commit/c759eee87a24fb337f15df027ab5e979d0453d66#L28 . My proposal is just to add a prefix to it, hence having something like '/my-prefix/static'.

I opened a separate issue, but maybe it's better as a comment here. I used this patch because I had the same use-case. And it works fine. However, I can't do testing any more, because the files in .build all have the site-path encoded in whereas for local testing '/static/...' urls suffice. It's almost as if one needs a 'deploy' option like with S3, but for this purpose. Or have a way to only do the rewrite for the final build instead of testing?

largely outdated


Remove flashing of callout

For offline geocoding sample while reverse geocoding on tap-hold-and-move.

Merged


Start langflow container without postgres container

Feature Request
I seem to be able to run langflow locally without having a postgres db next to it but I cannot find where the persistance is saved. there must be a local db like sqlite or anything.
Can't anything in the doc about it
Motivation
since I run it locally for smal tests. I tend to try to make a very simple deployment to limit the number of containers I need
Your Contribution
No response

@dosu maybe you can help me here

so technically where that ./langflow.db ? in /app of the containers? @dosu


change ngl_position from vec4 to vec3

Change ngl_position from vec4 to vec3, so that it matches the declaration in the node graph

Unfortunately I don't think you can split the commits here because the intermediate states will break: this is an API change

sure I can merge the commits.  I mainly split the commits to simplify rebasing and rewriting history, etc.

sure I can merge the commits. I mainly split the commits to simplify rebasing and rewriting history, etc.

every commit must be functional, be it only for git bisect to work as expected. When in doubt, use git rebase master --exec "make test"

sure I can merge the commits. I mainly split the commits to simplify rebasing and rewriting history, etc.

every commit must be functional, be it only for git bisect to work as expected. When in doubt, use git rebase master --exec "make tests"

updated

I've updated the commit by adding some documentation adjustments and rebased it on top of master.


ColorToHex(Color).toString() dont returns the hex code

String textColor = ColorToHex(Colors.teal).toString();

expected: textColor='#460000'
result: textcolor='Color(0xff460000)'

Not sure what ColorToHex is, its not part of this package.

sorry, my bad. typed in wrong window


Cannot past content from system clipboard

Hello, I checked out the code. It doesn't look like pasting content from system clipboard is supported. Is there a plan to add this feature in the near future?

I was able to copy from excel direct into the demo site, what problem did you see ?

The issue with multiple cells copy/paste.
I can copy-paste one cell but not many

多个单元格复制/粘贴的问题。 我可以复制粘贴一个单元格，但不能复制，解决了么，我只在火狐浏览器终于到这个问题


ergocub: remove sign in calibration 12 and update pitch roll neck gains

This PR fixes an important aspect in the calibration of the head joints of ergoCub, by removing the minus sign. This is because calibration type 12 needs the absolute value of the encoder.
Additionally, the PID values of the neck pitch and roll are updated to the latest autotuning results.

Why still draft?

Opened in draft accidentally, it's not wip.


how to docker login

I'm running docker-compose action after docker login but I get error:
Error: compose up failed {"exitCode":18,"err":"time=\"2023-08-01T14:32:39Z\" level=warning msg=\"The \\\"PWD\\\" variable is not set. Defaulting to a blank string.\"\n node_server Pulling \n cr_server_database Pulling \n node_server Error \n cr_server_database Error \nError response from daemon: pull access denied for cr-server-node_server, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\n","out":""}
The same docker-login works for standard build and push. How can I fix it?
These are the steps I'm running:
- name: Login against registry
  uses: docker/login-action@v2
  with:
    registry: ${{ env.REGISTRY }}
    username: ${{ github.actor }}
    password: ${{ secrets.GITHUB_TOKEN }}

- name: Run docker compose
  uses<EMAIL_ADDRESS>  env:
    IMAGE_TAG: ${{ needs.publish.outputs.image_tag }}
  with:
    compose-file: "./docker-compose.yml"
    up-flags: "--no-build"
    down-flags: "--volumes"

Do you know why permission solved the error? It is ok to update README.md, but I hope to make it clear why permission is required.

Permissions is required in my case, cause I'm trying to allow docker-compose to pull from registry (by using permissions.packages: read) the docker image I just built, since I just built and pushed the image in the preceding job (which I omitted in the code I posted).
I guess permissions.contents: read is not really needed here. By the way the permissions keyword is not really required due to isbang/compose-action but to the way github works in case you want to interact with things like github-packages. That's why I think it's worth it to mention in the README.
More info here: https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
Here the complete workflow with build and push image:
name: Build Push and Test

on:
  push:
    paths-ignore:
      - '.github/workflows/clean-containers.yml'
    branches:
      - 'master'
      - 'ft**'
    tags: [ 'v*.*.*' ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

env:
  # Use docker.io for Docker Hub if empty
  REGISTRY: ghcr.io
  # github.repository as <account>/<repo>
  IMAGE_NAME: ${{ github.repository }}

jobs:
  build-and-test:
    name: Build and test
    runs-on: ubuntu-latest
    timeout-minutes: 15

    strategy:
      matrix:
        node-version: [20.x] # can have multiple parallel build (e.g. [18.x, 20.x])

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}

      - name: Install dependencies
        run: |
          npm clean-install

      - name: Build
        run: |
          npm run build

      # - name: Test
      #   run: |
      #     npm run test

  publish:
    name: Publish to registry
    needs: build-and-test
    runs-on: ubuntu-latest
    timeout-minutes: 15

    permissions:
      contents: read
      packages: write

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Setup Docker buildx
        uses: docker/setup-buildx-action@v2

      # Login against a Docker registry
      # https://github.com/docker/login-action
      - name: Login against registry
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      # Extract metadata (tags, labels) for Docker
      # https://github.com/docker/metadata-action
      - name: Extract Docker metadata
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=schedule
            type=ref,event=branch
            type=ref,event=tag
            type=ref,event=pr
            type=sha

      # Build and push Docker image with Buildx (don't push on PR)
      # https://github.com/docker/build-push-action
      - name: Build and push Docker image
        # don't change this is, because it will be referenced in the next step
        id: build-and-push
        uses: docker/build-push-action@v3
        with:
          context: .
          target: production
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

    # define the outputs of this job to be referenced in other jobs
    outputs:
      image_tag: ${{ steps.meta.outputs.version }}

  run-compose-and-test:
    name: Run compose and test
    needs: publish
    runs-on: ubuntu-latest

    permissions:
      contents: read
      packages: read

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      #   - name: Execute Docker container at "ghcr.io/${{ github.repository }}:${{ needs.publish.outputs.image_tag }}"
      #     run: |
      #       echo "Running"

      # https://github.com/docker/login-action
      - name: Login against registry
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      # https://github.com/marketplace/actions/docker-compose-action
      # https://docs.docker.com/engine/reference/commandline/compose_up/
      # TODO testing ideas (at the moment we are just testing compose works without testing the app):
      #   https://medium.com/swlh/the-ultimate-guide-to-integration-testing-with-docker-compose-and-sql-f288f05032c9
      #   https://dev.to/sahanonp/setup-docker-for-integration-testing-in-github-action-39fn
      - name: Run docker compose
        uses<EMAIL_ADDRESS>        timeout-minutes: 5
        env:
          IMAGE_TAG: ${{ needs.publish.outputs.image_tag }} # required by docker-compose to pull the right image tag
        with:
          compose-file: "./docker-compose.yml"
          # --no-build cause we want to pull the IMAGE_TAG we just built
          up-flags: "--detach --no-build" # --abort-on-container-exit (but in the abort case you have to omit '--detach')
          down-flags: "--volumes"

docker-compose.yaml example where I make use of IMAGE_TAG variable defined within the workflow:
version: "3.8"

services:

  cr_server_database:
    image: mysql:8
    command: --default-authentication-plugin=mysql_native_password
    volumes:
      - database_cr_server:/var/lib/mysql
      - ${PWD}/scripts:/docker-entrypoint-initdb.d
    restart: unless-stopped
    ports:
      - "3306:3306"
    healthcheck:
      test: mysqladmin -u${DB_USER} -p${DB_PASS} ping -h <IP_ADDRESS>
    environment:
      MYSQL_ROOT_PASSWORD: ${DB_PASS}
      MYSQL_DATABASE: ${DB_NAME}
      MYSQL_USER: ${DB_USER}
      MYSQL_PASSWORD: ${DB_PASS}

  node_server:
    image: ghcr.io/my-image:${IMAGE_TAG:-dev}
    build:
      context: .
      target: development
    environment:
      DB_HOST: cr_server_database
    ports:
      - "3000:3000"
    depends_on:
      cr_server_database:
        condition: service_healthy

volumes:
  database_cr_server:


iOS landscape orientation error

Describe the bug
When rotating the device the camera does not rotate.
To Reproduce
Call FlutterBarcodeScanner.scanBarcode() and rotate the device.
Expected behavior
That the camera rotate with the device.
Screenshots

Smartphone (please complete the following information):

Device: iPhone X
OS: iOS13.1.3

Also, if you change orientation while scanning, overlay is not re-centered.

This issue has been resolved and will be available in the next release.

V1.0.0 has been released
There is a minor overlay issue persist and it will be resolved in the next release.
Closing this issue as of now, feel free to reopen


The AnimatedSprite node could have the signal finished ( )

It would be possible for the AnimatedSprite node to have the signal finished ( )
And so it would be easier to handle certain actions from From the code

How about animations with loop?

@Geequlim I guess it should work the same as AnimationPlayer (i.e. finished on each completed iteration AFAIK)


Curly brace positioned incorrectly in new angular.json file

🐞 Bug report
Command (mark with an x)

[x] new
[ ] build
[ ] serve
[ ] test
[ ] e2e
[ ] generate
[ ] add
[ ] update
[ ] lint
[ ] xi18n
[ ] run
[ ] config
[ ] help
[ ] version
[ ] doc

Is this a regression?
Perhaps, but not recent.
Description
When creating a new project, the very end of angular.json has 2 curly braces on the same line next to each other. The second one should be indented on the next line.
🔬 Minimal Reproduction
Create a new project with ng new, scroll to the very bottom of the created angular.json file, and notice the 2 curly braces right above defaultProject.
🌍 Your Environment
Angular CLI: 10.1.4
Node: 14.13.0
OS: win32 x64

Angular:
...
Ivy Workspace:

Package                      Version
------------------------------------------------------
@angular-devkit/architect    0.1001.4 (cli-only)
@angular-devkit/core         10.1.4 (cli-only)
@angular-devkit/schematics   10.1.4 (cli-only)
@schematics/angular          10.1.4 (cli-only)
@schematics/update           0.1001.4 (cli-only)
Anything else relevant?
How it looks now:

How it should look:

Should be fixed in version 11 via https://github.com/angular/angular-cli/pull/18708


Add documentation on CF API concurrency rate limit

CLA Seems to be broken ?

@FloThinksPi : It seems if you wait a little longer, signing the CLA should work, see cla-assistant/cla-assistant#874. Could you please try again?

Thank you for your contribution @FloThinksPi, the documentation will be updated with the next release.


Use golang/dep

Codecov Report

Merging #2 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #2   +/-   ##
=======================================
  Coverage   94.36%   94.36%
=======================================
  Files           3        3
  Lines          71       71
=======================================
  Hits           67       67
  Misses          2        2
  Partials        2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38773da...e1aaee6. Read the comment docs.


As a customer i want to see the cake menu so that i can analyze which one I will buy

[] As a business owner I want to register the cakes I sell so that my clients can see them on the menu
[] As a business owner I want to control how many cakes I have left so that I would not offer cakes that are not in the stock
[] As a customer I want to visualize the available cakes so that i can choose which one I will buy


[CELEBORN-1298][FOLLOWUP] Support Spark2.4 with Scala2.12

What changes were proposed in this pull request?
Support Spark2.4 with Scala2.12 in sbt.md. Meanwhile, the CI workflow adds the test for Spark2.4 and Scala2.12.
Follow up #2344.
Why are the changes needed?
Spark2.4 with Scala2.12 is supported.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
CI.

cc @waitinfuture, @cfmcgrady.

Thanks, merging to main(v0.5.0)/branch-0.4(v0.4.1).


Global leaderboard add_posts_to_main_feed

TODO:

[ ] Migrate all private projects to have add_to_main_feed=False
[x] Ensure global leaderboard only includes questions with default PUBLIC project

related #1657

related #1621


ci: add SonarCloud config file

Description

This PR adds the config file for SonarCloud so we can add / enable it there to check the code for code smells and other issues.

Types of changes

[ ] Documentation
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (anything that would change an existing functionality)
[x] Maintenance (refactor, code cleaning, development tools...)

Checklist

[x] I have read and follow the CONTRIBUTING.md document.
[x] The pull request title and template are correctly filled.
[x] The pull request targets the right branch (develop or develop-v...).
[x] My commits are correctly titled and contain all relevant information.
[ ] I have updated the documentation accordingly to my changes (if relevant).
[ ] I have added tests to cover my changes (if relevant).

@DanielRuf who's gonna create the necessary account and when is the analysis supposed to be triggered?

doesn't make much sense to merge this PR until the config contains some content

doesn't make much sense to merge this PR until the config contains some content

The config can be empty by default which sets the default values.

@DanielRuf who's gonna create the necessary account and when is the analysis supposed to be triggered?

Someone with the proper rights (owner, admin) to add the application from the marketplace.

I started work on getting this configured.


Timeout instead of error when using put file with recursive flag in the wrong location

What happened?:
pachctl put file -r gvcf_temp@master:/test_2 -f s3://concordance-studies/temp-vcf/test_2
could not connect to pachd at "pachd.rnd.info:650": context deadline exceeded

What you expected to happen?:
I was supposed to put the recursive flag after repo@branch.
pachctl put file gvcf_temp@master:/test_2 -r -f s3://concordance-studies/temp-vcf/test_2

I guess I would have expected this to throw an error instead of hinting at a connection problem.
Environment?:

Kubernetes version (use kubectl version):
1.14.6
Pachyderm CLI and pachd server version (use pachctl version):
1.9.6
Cloud provider (e.g. aws, azure, gke) or local deployment (e.g. minikube vs dockerized k8s):
aws ec2

Ha. I think this is actually a connection problem. Oops


Make quotes sequences of graphemes, proper fromLeft, count

Also wrote a small HTML generator/DSL which uses all that. Maybe move it into tests later?
TODO for this PR:

Test all available quote words. Just has to be done.
Make some benchmarks vs. old quote implementation & publish them here.

I've already decided to merge no matter what the benchmarks say, but I still want to do them, simply out of curiosity.
There are some TODOs in the example I mentioned which I'll also need to have a look at.

The ASCII-only sample string is the contents of zigcodegen.cpp, a 10 000 line long (and roughly 500 Kb in size), ASCII-only file (taken from Zig stage1 as you may have guessed).
The clusters sample string is the contents of a file filled with a pattern of grapheme clusters followed by a few ASCII characters, and repeated a few thousand times. This makes the clusters sample string 260 Kb in size.

Here are the benchmark results I got. Note that it is impossible to benchmark grapheme clusters using OLD quote implementation, because it doesn't support them.
OLD: create                            675.96M (  1.48ns) (± 7.54%)    0.0B/op         1.01× slower
OLD: count                             678.18M (  1.47ns) (± 7.42%)    0.0B/op         1.00× slower
OLD: empty?                            677.80M (  1.48ns) (± 5.13%)    0.0B/op         1.00× slower
OLD: stitch to itself                  12.75k  ( 78.46µs) (± 5.73%)   969kB/op     53323.17× slower
OLD: stitch                            25.51k  ( 39.20µs) (± 7.33%)   484kB/op     26637.78× slower
OLD: at? 0                             64.85M  ( 15.42ns) (±17.01%)   16.0B/op        10.48× slower
OLD: at? 1000                          61.83M  ( 16.17ns) (±16.21%)   16.0B/op        11.11× slower
OLD: at? count - 1 (worst case)        61.60M  ( 16.23ns) (±16.19%)   16.0B/op        11.15× slower
OLD: slice_at? 0                      399.26M  (  2.50ns) (± 3.84%)    0.0B/op         1.72× slower
OLD: slice_at? count - 1 (worst case)  25.52k ( 39.18µs) (± 5.89%)   484kB/op      26919.09× slower
OLD: slice_at? count                  254.02M  (  3.94ns) (± 3.39%)    0.0B/op         2.70× slower

NEW ASCII ONLY: create                              38.82 ( 25.76ms) (± 0.89%)    0.0B/op  17696152.06× slower
NEW ASCII ONLY: count                             678.21M (  1.47ns) (± 7.26%)    0.0B/op         1.01× slower
NEW ASCII ONLY: empty?                            678.66M (  1.47ns) (± 7.32%)    0.0B/op         1.01× slower
NEW ASCII ONLY: stitch to itself                    8.08k (123.83µs) (± 5.01%)   969kB/op     85068.14× slower
NEW ASCII ONLY: stitch                             21.44k ( 46.64µs) (± 5.29%)   484kB/op     32043.29× slower
NEW ASCII ONLY: at? 0                             301.43M (  3.32ns) (± 3.16%)    0.0B/op         2.28× slower
NEW ASCII ONLY: at? 1000                          301.33M (  3.32ns) (± 3.21%)    0.0B/op         2.28× slower
NEW ASCII ONLY: at? count - 1 (worst case)        223.95M (  4.47ns) (± 2.83%)    0.0B/op         3.07× slower
NEW ASCII ONLY: slice_at? 0                       678.90M (  1.47ns) (± 4.57%)    0.0B/op         1.01× slower
NEW ASCII ONLY: slice_at? count - 1 (worst case)   20.19k ( 49.52µs) (± 5.43%)   484kB/op     34022.74× slower
NEW ASCII ONLY: slice_at? count                    78.58M ( 12.73ns) (± 2.28%)    0.0B/op         8.74× slower

NEW CLUSTERS: create                             175.17 (  5.71ms) (± 0.50%)    0.0B/op   3921810.24× slower
NEW CLUSTERS: count                             679.60M (  1.47ns) (± 4.72%)    0.0B/op         1.01× slower
NEW CLUSTERS: empty?                            679.24M (  1.47ns) (± 4.72%)    0.0B/op         1.01× slower
NEW CLUSTERS: stitch to itself                   20.61k ( 48.53µs) (± 6.21%)   511kB/op     33337.98× slower
NEW CLUSTERS: stitch                             37.35k ( 26.77µs) (± 6.33%)   255kB/op     18394.39× slower
NEW CLUSTERS: at? 0                               4.10M (244.05ns) (±10.03%)    112B/op       167.66× slower
NEW CLUSTERS: at? 1000                            7.48k (133.67µs) (± 2.43%)  10.5kB/op     91832.21× slower
NEW CLUSTERS: at? count - 1 (worst case)         157.49 (  6.35ms) (± 2.28%)   496kB/op   4362139.13× slower
NEW CLUSTERS: slice_at? 0                       687.00M (  1.46ns) (± 6.03%)    0.0B/op              fastest
NEW CLUSTERS: slice_at? count - 1 (worst case)   135.06 (  7.40ms) (± 3.54%)  1.22MB/op   5086589.70× slower
NEW CLUSTERS: slice_at? count                    79.14M ( 12.64ns) (± 1.51%)    0.0B/op         8.68× slower

We can see that in case of an ASCII-only string, most time is spent on the creation of Quote. Creating a Quote from the outside is almost always an O(2N) (= O(N)) operation, unless you give it a few hints (like the amount of characters in the quote, and whether it is ASCII-only). Implementing quote creation as one-pass (O(N)) instead of two-pass should increase the speed somewhat.
For better performance, you should avoid creating a lot of quotes at runtime (e.g., by keeping one in a definition and/or using it in a read-only matter for as long as possible), because quotes cache the results of those two passes, and their descendant quotes respect the cache via some computation, meaning they won't do the two passes anymore. Thanks to the latter, quote creation should be faster than that presented in the benchmark results for Novika quote literals, because they are created only once (at slurp-time), and then their descendants are used.
In general, the slowest parts of the old quote implementation and the new quote implementation when the string is ASCII-only are the same. In general, the slow down in these parts is about 30% (judging by how many times the new implementation is slower than the old one). In the new implementation, slice_at? count - 1 has the same speed as the old implementation (at least in this particular case).
Clusters create and stitch faster because the sample string size is smaller. If we were to benchmark two equivalent strings, we would see that the cluster implementation is slower. The general solution for optimizing quotes is to avoid grapheme computations  for as long as possible.
at 1000 case is slower in clusters because it's always O(N) where N = 1000 in this case, speaking in terms of loop iterations only (there is a lot of other stuff happening besides that). I believe that byte_at (which is used in the ASCII-only case) is O(1), being just a pointer + offset dereference (which I suppose it is).
In clusters, the slowest cases are the worst cases slice_at? count - 1 and at? count - 1. Again, this is expected, because both are O(N) + extra work.

The general conclusions are that a) we must avoid creating a lot of quotes, b) as language implementors, we should avoid grapheme computations for as long as possible, even at the cost of a single expensive computation (a single O(N) vs. a thousand of them makes a difference in the real world, even though it's still O(N)), and c) further optimization efforts are required.
Note that my comments, conclusions, and benchmark results must be taken with a grain of salt.


[QA] shared file is visible, but cannot be accessed

Tested with

ocis-1.0.0-rc1 via https://github.com/owncloud-docker/compose-playground/blob/master/examples/hetzner-deploy/make_ocis_eos_compose_test.sh
firefox

Version details ocis-1.0.0-rc1

OCIS_VERSION:         v1.0.0-rc1
ocis --version:       ocis version 199267c^M
git log:              commit 199267c13e916aa8de027a81dbf6733af820369f (HEAD, tag: refs/tags/v1.0.0-rc1)
eos --version:        EOS 4.6.5 (2019)^M
xrootd -v:            v4.11.0^M
bin/ocis contains:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

Reproducer

use two differnet browser session. log in marie and einstein simultaneously.
marie uploads a photo.
marie shares the photo with einstein
einstein visits 'shared with me' and accepts the photo
einstein visits his "Shares" folder, and sees the photo.
einstein attempts media viewer. it fails.
einstein attempts download, nothing happens.
einstein attempts to copy the photo elsewhere, an error appears.

Expected behaviour: einsteins should have at least read access.

 Receiver cannot view.

Desktop client also fails to access the received files

These kind of things are all likely the same problem as issue https://github.com/owncloud/product/issues/208
The failing API test scenario does this sequence:

Alice shares a file with Brian
Brian accepts the share
Brian tries to download the file

The download in that scenario gets a 404 and the body content is therefore empty.
Fixing that will likely fix a lot of sharing stuff. It is being worked on in https://github.com/cs3org/reva/pull/1170

@butonic This was marked a release blocker in RC1, any updates here? Related to https://github.com/owncloud/product/issues/208 ?

Duplicate of cs3org/reva#1170

Known limitiation on EOS, other storage drivers work fine


new design system pt 3: electric boogaloo

we back

I definitely read through this and it looks good to me

😅😅😅


added movie-sample-card

You can also checkout our Idea-Arca, the project idea bank for any languages, for open-source contributions.
Happy Contributing season!
Keep up the good work!


Quotation in Vivan Grey not found? Alternate edition?

I've seen two reference locating the words "the reign of Poesy is over, at least for half a century" in Disraeli's Vivian Grey.  It's in the 1888 edition, but it is not present in the copy in the database. (1888 see https://books.google.com/books?id=h_BFAQAAMAAJ&pg=PA67&dq="at+least+for+half+a+century"&hl=en&sa=X&ved=0CC8Q6AEwA2oVChMI94HZlaifxwIVRxc-Ch0Dbg6t#v=onepage&q="at least for half a century"&f=false)

The edition in the database appears to be abridged.


Add ability to log multiple metrics in train and add history state

Instead of having evaluate only provide the scalar loss value, have it provide a dict containing at least the loss but can also contain other things like accuracy which are logged (printed to console) during training. Also add a history state dict to the superclass which keeps track of the values on a per-epoch basis. That way, a user can call exp.history['loss'] can get the loss for each epoch.

Here is basic try at this:
class MeanMetric:
  """
  Scalar metric designed to use on a per-epoch basis
  and updated on per-batch basis. For getting average across
  the epoch.
  """

  def __init__(self):
    self.vals = []

  def update(self, new_val):
    if torch.is_tensor(new_val):
        new_val = new_val.item()
    self.vals.append(new_val)

  def reset(self):
    self.vals = []

  def get(self):
    mean_value = sum(self.vals)/len(self.vals)
    return mean_value

class Experiment(ABC):
    """
    Base experiment superclass. All other experiments
    should inherit from this class. Each sub-experiment
    must provide an implementation of the "evaluate" abstract
    method. A sub-experiment has full autonomy to override
    the basic components such as the training loop "train".
    """
    def __init__(self, model, optimizer, trainloader, wandb_logger=None, verbose=False):
        """
        Experiment superclass initializer. Each subclass must provide
        a model, optimizer, and trainloader at the very least.
        Arguments:
        -----------
        - model: torch nn.module
        - optimizer: torch optimizer
        - trainloader: torch Dataloader to be used for training
        """
        self.model = model
        self.optimizer = optimizer
        self.trainloader = trainloader
        self.wandb_logger = wandb_logger
        self.verbose = verbose
        self.first_batch = True

    def train(self, num_epochs):
        """
        Training loop.
        """
        self.model.train()
        epoch_num = 0

        metrics = {}

        for epoch in range(num_epochs):  # loop over the dataset num_epochs times
            epoch_num += 1

            tqdm_loader = tqdm(self.trainloader)
            for batch in tqdm_loader:
                tqdm_loader.set_description(f"Epoch {epoch_num}")
                self.optimizer.zero_grad()
                evals = self.evaluate(batch)
                if self.first_batch:
                    assert type(evals) is dict
                    assert "loss" in evals.keys(), "evaluate must return a 'loss' value"
                    # register metrics
                    for key in evals:
                        curr_metric = MeanMetric()
                        metrics[key] = curr_metric
                    self.first_batch = False
                loss = evals["loss"]
                loss.backward()
                self.optimizer.step()
                # update metrics
                for key in metrics:
                    metrics[key].update(evals[key])

            if self.verbose:
                for key in metrics:
                    print(f"{key}: {metrics[key].get()}")
            if self.wandb_logger is not None:
                for key in metrics:
                    self.wandb_logger.log({str(key): metrics[key].get()})

        self.model.eval()
        print('Finished Training!')

but the problem is that this only works for mean metrics on a per-epoch basis (i.e., logging accuracy wouldn't work). And so it seems that this is a bit harder of a problem than it looks. One viable option would be to force the user to provide the metric classes but this goes against the simplicity of the package. But of course, we could provide common metric classes (i.e., accuracy, IoU, etc.).

I think it might actually make more sense to just provide do_this_on_batch_end and do_this_on_epoch_end functions for which users can just keep track of the state themselves. Also provide MeanMetric class in utils.py

Update: I think I will push this change sometime soon and have evaluate enforce that a dictionary is returned rather than a numerical loss. But the dict must contain the loss.
With regards to:

but the problem is that this only works for mean metrics on a per-epoch basis (i.e., logging accuracy wouldn't work).
I disagree now... accuracy would still work.

Also, check out torchmetrics which basically follows the (get, reset, update) class structure,

In addition, provide metrics.py:
class MeanMetric:
  """
  Scalar metric designed to use on a per-epoch basis
  and updated on per-batch basis. For getting average across
  the epoch.
  """

  def __init__(self):
    self.vals = []

  def update(self, new_val):
    self.vals.append(new_val)

  def reset(self):
    self.vals = []

  def get(self):
    mean_value = sum(self.vals)/len(self.vals)
    return mean_value

class MeanMetricCustom(ABC):
  """
  Abstract scalar metric. Must provide calculation given preds and y.
  """

  def __init__(self):
    self.vals = []

  @abstractmethod
  def calculate(self, logits, y):
    # returns a value
    pass

  def update(self, logits, y):
    self.vals.append(self.calculate(logits,y))

  def reset(self):
    self.vals = []

  def get(self):
    mean_value = sum(self.vals)/len(self.vals)
    return mean_value

# then provide some common metrics such

Implemented in version 0.0.7.


nessus: make parser more robust on missing fields

Ensure the parser doesn't fail when any of the following fields are missing:

Synopsis
Solution
Description
See Also

I've updated the PR title + description to make it more informative.

@whitespots whitespots edited 5 days ago

It's impossible now to parse the usual nessus report. Also it does not show any valuable error. So, anyway, this change would not be bad

The above was your original description. Can you explain what problem you are trying to solve? Is it that 'str' will fall on a None value? Or something else?

It had some problems with the rules engine. I just tried to make some "refactoring" :)
It is impossible to import the nessus report with any rule enabled

I suppose if code was failing on Nones, it is better to hand how they are used.
So,  on line 70 replace
find.description += row.get('Plugin Output')

by
find.description = (find.description or '') + row['Plugin Output']

and on file 73 replace
description = description + str(row.get('Plugin Output'))

by
description = (description or '') + str(row['Plugin Output'])


Need help with this SQLliteError

"use strict";
require('dotenv').config()          // Don't forget to set up environment variables
const fs = require("fs");           // fs module grants file system access
const db = require("../Models/db"); // We need our db connection

// Now read the schema.sql file into a string
// It is ok to do this synchronously since this script executes before loading the server
const schemaString = fs.readFileSync(__dirname + "/schema.sql", "utf-8");

// Now just run the sql file
db.exec(schemaString);

/Users/jonpi/node_modules/better-sqlite3/lib/methods/wrappers.js:9
this[cppdb].exec(sql);
^
SqliteError: near ")": syntax error
at Database.exec (/Users/jonpi/node_modules/better-sqlite3/lib/methods/wrappers.js:9:14)
at Object. (/Users/jonpi/Database/init-db.js:11:4)
at Module._compile (node:internal/modules/cjs/loader:1101:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:17:47 {
code: 'SQLITE_ERROR'
}

I recommend posting these type of question on StackOverflow as they appear unrelated to better-sqlite3. Your schema.sql seems to contain invalid SQL, that's what the error says.


Fix for #3888: Define method_name in Chef::Cookbook::Metadata.validate_json if undefined

It looks like method_name could never be properly scoped here?
It's inclusion makes any cookbook with a dependency and no metadata.rb in a cookbook fail sharing to both public and private supermarkets.
Line 626 throws an exception if method_name is undefined.
For now I've assigned an empty string if its undefined. Open for input @chef/client-core
closes #3888

uhm....
whatever all this method_name code is doing its clearly terrible.  is there some closure somewhere that makes this actually defined sometimes?  on the surface it looks like some kind of bad copypasta error that is always broken...
anyway, i'd really like to see a more aggressive patch that did more than just apply a bit of duct tape...

So far I can't find any way that method_name would actually be defined, so I think any cookbook that has any version constraint and only a metadata.json will fail to upload via knife cookbook site share or knife supermarket site share.
I wanted a 2nd set of eyes on this before I tried something more verbose, I'll take a deeper look at what we're trying to validate and how we might do it tomorrow.

I've updated this to behave in what appears to be the same way as when we grab the metadata from a metadata.rb.
I did a quick test, killing my JSON name attribute and I receive the expected
ERROR: Chef::Exceptions::MetadataNotValid: Cookbook loaded at path(s) [/Users/andrewdufour/Development/cookbooks/push-jobs] has invalid metadata: The name' attribute is required in cookbook metadata`
We should probably have some tests surrounding this. Also I believe we can remove that validate_json method if this is a good approach.

Isn't the whole #validate_json approach doing a lot of work to solve a problem indirectly, and we can just construct the object and let it fail with a NoMethodError if the json isn't valid?  Then if we want to we can rescue the NoMethodError and give the user a 'nicer' looking error message and/or re-raise with Chef::Exceptions::MetadataNotValid that suggests to the user that their json has bad keys in it...

I tried to just pass this down to Chef::Cookbook::Metadata.from_json as a json string, I think because it isn't eval'd like from_file is it doesn't do any version constraint checks. So a version constraint in the json of 'something': '>= a.b.c' was passing through validation.
Since we probably don't want to let people bork their version constraints I think validating the json's constraints makes sense, and I've added a reworked validate_json method to the PR (it's not really validating json, it's validating the json's constraints), and fixed the failing tests.

Alright so after spending too much time on this the last couple of days I'll explain how I arrived where we are.
When you read in a metadata.rb we use Chef::Mixin::FromFile.from_file which evals from the context of the Chef::Cookbook::Metadata object. When you execute methods like 'depends' it runs validation against the version constraints in the metadata.rb.
We were correct in assuming we'd throw an exception in the case of an invalid JSON object. But when you read in a json file and run Chef::Cookbook:Metadata.from_json you never validate version constraints like you do with from_file.
The result is you can have invalid version constraints in your valid json - e.g., {"dependencies":{"runit":">> a.b.c"}} and this will be happily processed as valid metadata, whereas we throw an exception if the same dependency is listed in a metadata.rb.
So, this validate_json method is really validating the json's constraints. I tried a few approaches like adding this to run_validation in Chef::Cookbook::Metadata but it causes a bunch of errors from Pedant running against chef-zero, that will take a bit of effort to fix.
As it stands this seems to me to be the simplest solution that isn't a complete hack that catches both invalid JSON and invalid constraints. If we want to go deeper to patch this I'll likely need some guidance and assistance.

@chef/client-core could I get another look at this? I know it's not ideal in its current state, but it puts our share commands in a better spot.
If we want to improve this more I am willing to do the work but may need some guidance.

actually even after all that, the shape of the API doesn't make a lot of sense, since your debug message states that you're loading the metadata, but it looks like you're trying to call validate_json to load the metadata, but it both doesn't do that, and that's an odd side-effect for a 'validate' method to have.

Just curious why this issue from 2015 is still not addressed?

time

what is the status of the above PR?

I was finally able to pick this back up and look at it today, and noticed that @mrigeshpriyadarshi in https://github.com/chef/chef/issues/5201 was able to glean the original developers intent, that I missed.
This is a simplified solution, that, although not pretty, does work without reworking the validation methods of the class. I was also able to add a spec test to ensure we actually test some invalid JSON to make sure we're parsing things.
@lamont-granquist if you want me to take this further and rewrite the validation methods I can give it another stab later this week. It would be great if we could merge this as a fix for anyone with only a metadata.json though.

I'm working on rewriting all the specs to be less stubby in a branch right now


Is this possible?

When the html page opens up, and I click an image, then the popup is working as expected.
But If I click the text displayed below the image inside the popup I activated, then a js function "clAndOpen" (see below), where I try to
a) close the current popup, and
b) open another popup based on the id of another popup I want to open.
is not working as expected. I'm just unable  to open another popup based on the content in the js function. If someone can give me som instructions on how I can make this work, that would be nice.
As you can see in code below I'm initializing the magnific popup with type = inline.
I really like this pupup. If I'm to use it in my project then I have to make this work one way or another.
The reason I want this behaviour is because some of the images displayed on main page has related images on the page. When clicking on an image which has other related images, I want in the popup to show a link to all related images for the clicked image. Example. I click an image with magnific id = "#magpop-1", then #magpop-2 and #magpop-3 might be related to the clicked image. So in the pupup I will in the end show three thumb-nails belw the enlarged image in the current popup, and when I click one of the other two related images then I will make it so the current popup or a new popup (after closing current) show the correct image. Here I'm testing to first close current popup, and then opening an new popup with the image clicked in the newly closed popup - using the id sent to the function clAndOpen to create the correct id to send to the magnific popup open.
function clAndOpen(openId){
   // closing is working
   $.magnificPopup.close();
   var openIdText = "magpop-" + openId;

   // THIS DOES NOT WORK
   $('.open-popup-link').magnificPopup('open', openIdText);
}

Here is the structure of the divs with two images and related info.

            Some text here...

            Open image 5

            Some text here...

            Open image 1

Here is how I initialize the magnificent popup:
$('.open-popup-link').magnificPopup({
    type:'inline',
    alignTop: true,
     midClick: true,
     closeBtnInside:true,
});

I figured how to make this work. Even if I have set the removalDelay option to zero, it seems to me that this line:
$.magnificPopup.close();
is not getting finished before this line is run:
$('.open-popup-link').magnificPopup('open', openIdText);
To make it work I had to put the open line above in a function "tryPopup", and use this after the close line:
setTimeout(function () { tryPopup(openId); }, 100);
I do not know why it has to be like this, but it works.
NOTE: this is done because some of the images displayed (enlarged) in the popup have related images. And, if so I wanted to show thumb images below the enlarged in the popup, with a link that when clicked opens the enlarged thumb in another popup. But before that could happen, I had to close current popup first.
Anyway... this magnific popup is truly magnificent.


Custom cluster reg command enhancement: Banner The cluster needs to have at least one node with each role. can be changed when registering windows nodes as windows node is only for worker nodes for custom clusters

Rancher version: v2.6-head 2e55e84
Installation option (Docker install/Helm Chart): Docker install
Version: local k8s, v1.21.3+k3s1
Proxy/Cert Details: self-signed
Information about the Cluster
Kubernetes version: v1.21.4-rancher1-1
Cluster Type (Local/Downstream): Downstream custom cluster rke2
Result:
While registering a windows cluster, a banner is displayed The cluster needs to have at least one node with each role.
Expected result:
This banner can be changed that says only worker role can be selected for windows nodes.
Screenshot:

@anupama2501 can you clarify what the banner should say? Your screenshot appears to be the Linux blurb which can allow all three, but the Windows one below that can only be worker.


Documents missing after full sync thru SyncGateway

We've had a strange behaviour when testing sync process. I must warn that this problem seems to be related to another one on the SyncGateway side (see: https://github.com/couchbase/sync_gateway/issues/870).
The context is a SyncGateway that doesn't deliver all documents that should be delivered on the _changes url, 2 documents are missing at the end of the result (9950 and 9951 on a total of 10000 docs that are all the same with different id for testing purpose) but following documents from 9952 to 10000 are returned (so, just a gap of 2 documents that should be here). The strange thing is that the sync process on android successfully pulls all docs before the two missing, and then seems to stop pulling documents after the missing ones and consider itself up to date. I was expecting to get all documents except the 2 missing in the _changes url result, but all documents after the missing ones are never synced.
I realize that fixing the problem on the SyncGateway side should fix the problem on android too, but I think there may be a bug because there is no reason for this behaviour. If the SyncGateway returns x documents, the client should get x documents, whether there a some missing or not in the result ... And morevover, the android client should not even be aware that 2 documents are missing if they are not returned by the SyncGateway.
Let me know if you want more details on the context of this bug.
Thanks

Hi @popy,
Thank  you for your report. From current information, it seems SG has problem with /_changes. Currently SG team is working for couchbase/sync_gateway#870. We should retest this after 870 will be fixed.

Hi @hideki
I commented on the related issue on the Sync Gateway project. After restarting the couchbase server (sync gateway was restarted multiple times but not couchbase server), the problem disappeared and we haven't been able to reproduce it since.


links2: 2.10 -> 2.11

CC @viric, the only working GitHub handle.
Changes: http://links.twibright.com/download/ChangeLog.

I am going to merge this. There seem to be no breaking changes, if there are problems we can still revert.

Thanks, I'll be less afraid of stepping on toes (after a few days) next time.


Remove this from VGridSkeleton props default

Fixes

Fixes #3268 by @Ashhar-24
Description

As  mentioned in the issue #3268 that VGridSkeleton uses this in the props' default function and this causes a build error in Vue 3.
So I have moved default element count to the setup function and the numElems is replaced with elementCount in the template.
Note: if this PR is ready after #3322 is merged, this PR needs to remove the TODO from packages/eslint-plugin/src/configs/vue.ts:
    // TODO: disable after https://github.com/WordPress/openverse/issues/3268 is fixed
    "vue/no-deprecated-props-default-this": "off",

Testing Instructions

Checklist

[x] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[ ] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible errors.
[ ] I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

You will need to run just lint and commit the changes.

You will need to run just lint and commit the changes.

This is error message that i'm receiveing.

Thank you @obulat for your constant support.
Was nice working with the issue.


R3F snapshots, editable types, and derived properties

We're working out how to handle a use case that doesn't seem easy to solve under the current release (theatre/r3f v0.4.0), but might be improved by #139.
Our target user experience calls for the position/rotation of certain objects to be derived from, but not fully specified by, a theatre.js sequence. For example, the path of a PerspectiveCamera might be animated from pose A to pose B, but the viewer has some constrained influence to rotate the camera during that transition, creating an experience that's more interactive than video:
https://user-images.githubusercontent.com/1848368/166966283-3301c19e-0c7c-4f0f-9ab6-57bedb35da79.mp4
Under 0.4.0-dev.23 this requires us to avoid using editable for the camera, because editable will override the camera's position. That would be OK, except that then the camera is lost from the snapshot view, which is a really great piece of the 3D workflow.
The solution I'd been leaning toward requesting (prior to seeing #139) was this — could the snapshot view and the editable types be independent? I'm hoping that the snapshot view could become an additional <View/> or canvas for the original three.js/r3f scene, rendered in the same animation loop. This would remove the need for manually cloning the scene and refreshing the view, and would ensure that changes to position/rotation are reflected in the snapshot, even if those changes were not made by editable.
I could imagine #139 might offer other options here — perhaps a user-defined schema for PerspectiveCamera that removes the position/rotation properties and re-defines them manually. Wondering if you have thoughts on how users, or the theatre/r3f extension, should approach this? I'd understand if (as hinted in #141?) the editable types are pretty core to the theatre/r3f extension and it doesn't make sense to decouple them from the viewport, I think I'm more trying to understand the direction of the plugin than to necessarily influence it at this point.
Thanks!

Hey @donmccurdy, I've long been bothered by this too, but wasn't sure how exactly to address it, since, as you mentioned, there are multiple options. The schema API would seem a bit cumbersome to me though for this (although it is very flexible and would allow for this possibility). My thought was that there should also be an easy-to-use API for ad-hoc "interception" of theatre values before they'd be applied to the object. In particular it is interesting you mention the camera rotation, because I had the exact same problem, which at the time I solved by wrapping the camera in an extra group, which, as you said is not pretty. At the time my thought was that instead of passing an initial value to editable objects' props, like <e.mesh position={[0,0,0]} />, you could instead pass a function like <e.mesh position={(position /* received from theatre */) => { /* do whatever */ return myPosition }} />, but I'm not sure if there is a better alternative. This would be super easy to use thought IMO. Any thoughts? This is something I see a lot of value in.

instead of passing an initial value to editable objects' props ... you could instead pass a function

As an isolated API change, yes — I see no problem with this and it may be useful.
But to be honest, I'm having to fight against the abstraction in theatre/r3f now, and I'm not sure adding more abstraction would be the right direction for my application. Currently, for each object in the application, properties are divided into two groups:

fully managed by theatre/r3f
attached to the objRef by my application

Properties in group (1) often work as intended, but when they don't it's very hard to work around — see camera controls described above, or changing parameterization of rotations like in #86. Properties in group (2) require a bit of imperative glue code to create and update, but it's not really an obstacle.
So I'm mostly finding that I prefer (2) more than (1). That might be specific to my application — users are interacting with theatre.js but are not modifying R3F/JSX or three.js code for the most part, so we want to be more selective about how the API is exposed. Directly exposing three.js and R3F properties isn't quite right for us, and mapping values through callbacks is more complex than managing the properties directly.
The most obvious workaround I'm seeing right now is using theatre/studio to manually generate the property bindings for our objects, porting the theatre/r3f snapshot view as a standalone extension that displays an R3F or three.js scene, and having that panel commit transactions when objects are moved by transform controls — without letting the panel manage editable types. Especially with general theatre extension UI/API improvements like #141 this is very doable.
On the other hand – if I were rapidly prototyping an R3F application and switching between coding with R3F and animating with theatre/r3f, the current editable types probably save a lot of time in that process. So I do understand why they're there – the editable types perhaps only present more of an obstacle when the property UI is itself part of an non-coder-facing surface that requires more UI/UX customization to the task at hand.


Untyped JS module is not assignable to type even though it satisfies it

🔎 Search Terms
Untyped JS module is not assignable to type even though it satisfies it, "Object literal may only specify known properties"
⏯ Playground Link
https://github.com/bradzacher/bug-repros/tree/ts-js-issue-huh
💻 Code
import eslintReact from 'eslint-plugin-react/configs/recommended.js';

(eslintReact satisfies {
  plugins: {
      react: {
          rules: any;
      };
  };
});

🙁 Actual behavior
There should be no errors because the type
{ plugins: { react: { deprecatedRules: any; rules: any; }; }; }
is assignable to
{ plugins: { react: { rules: any; }; }; }
🙂 Expected behavior
Type '{ plugins: { react: { deprecatedRules: any; rules: any; }; }; rules: any; readonly languageOptions: any; } & { languageOptions: any; rules: { 'react/display-name': number; 'react/jsx-key': number; 'react/jsx-no-comment-textnodes': number; ... 18 more ...; 'react/require-render-return': number; }; }' does not satisfy the expected type '{ plugins: { react: { rules: any; }; }; }'.
  The types of 'plugins.react' are incompatible between these types.
    Object literal may only specify known properties, and 'deprecatedRules' does not exist in type '{ rules: any; }'

Additional information about the issue
This issue is particularly weird because it only occurs in a JS file!
For more context this is the original bug report https://github.com/typescript-eslint/typescript-eslint/issues/8522
And the original repro from the issue (the usecase we really want to fix): https://github.com/printfn/tseslint-react-repro
We have built support for ESLint flat configs via our typescript-eslint package and it exports a function to allow typechecking of the config array. So the repo looks like this
// @filename eslint.config.js

// @ts-check

import tseslint from 'typescript-eslint';
import eslintReact from 'eslint-plugin-react/configs/recommended.js';

export default tseslint.config(
	eslintReact, // type error is on this line
);

Best I can tell it looks like TS is treating the eslintReact variable the same way it would as an inline object expression - hence it's erroring saying "too many properties".

This is a private repo or something: https://github.com/bradzacher/bug-repros/tree/ts-js-issue-huh

oops sorry made it public

the repro doesn't contain a concrete TS version and a tsconfig.json :P

Yeah I know. The latter one is intentional.
Users don't setup an eslint.config.js and cover it with a tsconfig - it just uses the vscode defaults
As for the TS version - I doubt it matters? I was just using whatever is installed in the latest vscode version.
In the issue I linked a bigger repro if you want some more stuff. I was just trying to be as isolated as possible.

There is some difference between my default and your default setups then because I don't get this error but rather this at the import statement:

Module '"/repros/bug-repros/foo"' has no default export.ts(1192)

We have to use different module resolutions for some reason.

sod it, sorry @Andarist - I forgot to push a fix.
The import in foo.ts should be from eslint-plugin-react/configs/recommended.js (I just updated the repo)

I still get a different error:

Module '"/repros/bug-repros/node_modules/eslint-plugin-react/configs/recommended"' can only be default-imported using the 'esModuleInterop' flagts(1259)
recommended.js(5, 1): This module is declared with 'export =', and can only be used with a default import when using the 'esModuleInterop' flag.

Maybe there's some vscode settings that impact this?
Have you changed any of the defaults?
I spent a while playing around with a tsconfig.json but I couldn't make it repro with any combo of tsconfig settings I tried.
Can you give this repro a go? https://github.com/printfn/tseslint-react-repro
Just install and open the eslint.config.js.
If it doesn't repro there then you defs have tweaked some settings.

No luck either even after disabling extensions. I downloaded the insiders version of VS Code and I managed to repro it there

This should fix this incorrectly reported error: https://github.com/microsoft/TypeScript/pull/57467 . However, it still won't typecheck in your case because this property from the source isn't compatible with your target: https://github.com/jsx-eslint/eslint-plugin-react/blob/36e791de784e7fa1edd5b6d232fbcaf2dd7cf9bf/configs/all.js#L28
Its type gets computed as { [x: string]: number; [x: number]: number; [x: symbol]: number; }. See the error reported here: TS playground

Its type gets computed as { [x: string]: number; [x: number]: number; [x: symbol]: number; }.

If I locally modify my node_modules and comment out the line that's currently erroring then I get no more errors in my IDE (i.e. I can't repro that wide signature).
Unless I'm misunderstanding and you're saying that after your fix the record is inferred as that wide type?

If I locally modify my node_modules and comment out the line that's currently erroring then I get no more errors in my IDE (i.e. I can't repro that wide signature).

When doing just that (without applying my patch in tsserver, just removing that single line with deprecatedRules property) I get the same error as the one that I get with my tsserver patch applied (the one showcased on the TS playground above).
Could you try hovering over activeRulesConfig here? What type is displayed by the quick info? Is it { [x: string]: number; [x: number]: number; [x: symbol]: number; }?

Note: filed https://github.com/jsx-eslint/eslint-plugin-react/pull/3694 to add some minimal JSDoc annotations which will short-circuit the TS inference and fix things.

Could you try hovering over activeRulesConfig here? What type is displayed by the quick info? Is it { [x: string]: number; [x: number]: number; [x: symbol]: number; }?

It just shows any for me.

😅

Could you try applying my patch to see what happens with it?

still any 😨
the error disappeared though so your patch definitely was applied.
I think you're getting hit by automatic type acquisition and TS is automatically pulling in @types/object.fromentries.
If I manually install that then I get the same behaviour.
Essentially without @types/object.fromentries the object.fromentries package is too complex for TS to infer types for -- so the fromEntries call returns any.
With it TS understands the return type and thus propagates that up the call stack and infers that final type.
The JSDoc annotations I added in https://github.com/jsx-eslint/eslint-plugin-react/pull/3695 will short-circuit TS's inference here.

I think you're getting hit by automatic type acquisition and TS is automatically pulling in @types/object.fromentries.

Yeah, that's probably it. Thanks for looking into this.

The JSDoc annotations I added in https://github.com/jsx-eslint/eslint-plugin-react/pull/3695 will short-circuit TS's inference here so that ATA won't change the signature

IIUC ATA might impact activeRulesConfig's type and you have left that untyped so it won't fix your issue completely. The added annotations there might help with the bug fixed by https://github.com/microsoft/TypeScript/pull/57467 though 👍

IIUC ATA might impact activeRulesConfig's type and you have left that untyped so it won't fix your issue completely

const activeRulesConfig = configureAsError(activeRules); -- I typed configureAsError so the type flows into activeRulesConfig so it should fix things!

"typescript.disableAutomaticTypeAcquisition": true
I disabled ATA a long time ago because I kept getting bit by it auto-grabbing @types/eslint when I didn't want it to.

const activeRulesConfig = configureAsError(activeRules)

ah cool, I somehow missed that 👍


Add tests

Hej @mikaelbr,
I would like to add a few test.
Do you mind if I use the following stack?

mocha
chai (using bdd 'expect' style)
blanket.js for coverage

Or to you prefer another testing framework, assertion library or coverage tool?

That sounds perfect for me. Pretty much the exact same stack I use. :+1:

Update, I just Istanbul for coverage because blanket drove me mad lately :angry:

Hehe, Istanbul works just OK (nothing fancy or special), but hopefully it won't drive you as mad.

Hehe, it seems that I’ll be fine with Istanbul :blush:
The HTML reports look a bit outfashioned compared to blanket but that thing does what is supposed to do and I’m not missing any feature of blanket.


Removing creator migration code

Removed it from GenericWork Controller and it is no longer referenced.

Duplicate of #1496?

Coverage decreased (-36.7%) to 55.303% when pulling 0e1a996823845a07badca80adde1c7df47d872f8 on i#1443.8 into 540b944f301c994273e890c1b27c0836bc7e3ab3 on develop.

@awead you are correct this is a duplicate


Tests and benches for extension fields

C++ side tests and benchmarks refactored using templates;
Extension field tests and benchmarks added;
README added for benches;
Some small additions to extension field API for convenience;

One issue that I'm having is the following error during cargo test:

Invalid argument : by: NTT at: icicle/icicle/src/ntt/ntt.cu:624 error: NTT size=16 is too large for the domain. Consider generating your domain with a higher order root of unity.

Though it's also happening in the V2 branch for me, so maybe it was introduced by changes around release domain or EC NTT? Might be related to https://github.com/ingonyama-zk/icicle/pull/469

@DmytroTym the issue you had was caused by the release_domain test, #469 fixes it by assuring that the domain release test is running last
Not sure why CI tests are failing here


Hardware Target not shown in VS 2019

Hardware Target fields are not shown in Visual Studio 2019. On the very same PC they are indeed shown in VS 2017. I do run VS 2019 Enterprise Version 16.0.2

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2c8fe303-324f-149e-d664-95b3f335bbb0
Version Independent ID: 98bfc861-2906-8225-455c-573035403312
Content: Target hardware dependencies
Content Source: AzureSphere/articles/app-development/manage-hardware-dependencies.md
Product: azure-sphere
GitHub Login: @pennyo
Microsoft Alias: azurespheredocs

@Thoemmeli Thanks for the feedback! We are currently investigating and will update you shortly.

@Thoemmeli would you please try to update Visual Studio to VS 2019 16.04


New localization file encoding to UTF-8

Please check if the pull request fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)
[ ] CHANGELOG.md has been updated to describe this change

To update the documentation on yarnspinner.dev, please visit the documentation repository.

What kind of change does this pull request introduce?

[X] Bug Fix
[ ] Feature
[ ] Something else

What is the current behavior? (You can also link to an open issue here)
When creating a new localization csv file, its encoding becomes automatically us-ascii. This meant I couldn't add spanish characters to such file because it kept getting rejected by the system since it couldn't recognize them. I had to manually convert it to utf-8 in order to work with it.

What is the new behavior (if this is a feature change)?
The created file gets automatically encoded to utf-8 to avoid such errors.

Does this pull request introduce a breaking change? (What changes might users need to make in their application due to this PR?)
No.

Other information:
When I overwrote the us-ascii file with spanish characters in Visual Studio or Visual Studio Code, it also automatically got converted to iso-8859-1, but this did not solve the issue. My computer language and keyboard layout were both in Spanish.

This is a little strange, because the documentation for both File.CreateText and File.WriteAllText indicate that they work with UTF-8 by default. I'm happy to update the code to specify UTF8 (and merge this PR), but I'm a little surprised that you were seeing this behaviour.


Thanks!

No issue, just wanted to say thank you, great job!
I was googling for a solution on Stack Overflow or something, and here you are, providing an npm package that does just what I need 👍

Very cool! I appreciate the kind words!!!