add 8 new packages

add

libmongoc-1.0-0
libmongoc-dev
libcap2
libcap-ng0
libcap-ng-dev
libprocps6
libprocps-dev
libdpkg-dev

removed libcap2 from gentoo
removed libprocps6 from gentoo

Package
Debian
Gentoo
Ubuntu

libmongoc-1.0-0
libmongoc-1.0-0

libmongoc-1.0-0

libmongoc-dev
libmongoc-dev
dev-libs/mongo-c-driver
libmongoc-dev

libcap2
libcap2

libcap2

libcap-ng0
libcap-ng0
sys-libs/libcap-ng
libcap-ng0

libcap-ng-dev
libcap-ng-dev

libcap-ng-dev

libprocps6
libprocps6

libprocps6

libprocps-dev
libprocps-dev

libprocps-dev

libdpkg-dev
libdpkg-dev

libdpkg-dev

hellö,
Is there any update on review?

@clalancette I'm checking the linter script locally, it seems to be not properly functioning for detecting empty lines, but it correctly identifies trailing whitespaces.
Can't detect this:
a:
  b: precious

Can detect this (if you try to mark the right side of the 'b' you'll see the whitespace:
a:
  b: value

I'm working on another PR for check_rosdep.py script for detecting empty lines which may save mare mortals like me from embarrassing situations like this :D.

Adding this will do.
def no_empty_lines(buf):
    clean = True
    for i, l in enumerate(buf.split('\n')[:-1]):
        if re.match(r'^\s*$', l):
            print_err("empty line %u" % (i+1))
            clean = False
    return clean

Results:
checking for trailing spaces...
checking for empty lines...
  ERR: empty line 6225
checking for incorrect indentation...
checking for non-bracket package lists...
checking for item order...
building yaml dict...
there were errors, please correct the file

I also checked for a case where an empty line also has trailing whitespaces which is kinda rare. If that happens script returns this:
$ ./check_rosdep.py ../rosdep/base.yaml
checking for trailing spaces...
  ERR: trailing space line 6225
checking for empty lines...
  ERR: empty line 6225
checking for incorrect indentation...
  ERR: line 6224:
Traceback (most recent call last):
  File "./check_rosdep.py", line 223, in <module>
    if not main(args.infile):
  File "./check_rosdep.py", line 181, in main
    my_assert(correct_indent(buf))
  File "./check_rosdep.py", line 116, in correct_indent
    return generic_parser(buf, fun)
  File "./check_rosdep.py", line 82, in generic_parser
    s = re.search(r'(?!' + indent_atom + ')[^\s]', l).start()
AttributeError: 'NoneType' object has no attribute 'start'
$ echo $?
1

I didn't want to commit my local change, since it will automatically be a part of this PR which labelled as rosdep. If its ok, I can commit it, if its not I'll wait for this PR to be merged and create a new PR.

If its ok, I can commit it, if its not I'll wait for this PR to be merged and create a new PR.

Please submit a separate PR for this change, since it isn't directly tied to these rosdep rules.  And thanks for doing that!


Return gradient 0 in mixed_precison setting

System information
OS Platform and Distribution: linux
TensorFlow version (use command below): tf2.4.1
Reproducing code

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras import mixed_precision
import numpy as np
from tqdm import tqdm

gpus = tf.config.experimental.list_physical_devices('GPU')

for gpu in gpus:
    tf.config.experimental.set_memory_growth(gpu, True)

mixed_precision.set_global_policy('mixed_float16')

def forward_conv(x, filters, kernels, name='forward', padding='same'):
    i = 0
    for flt, kernel in zip(filters, kernels):
        x = layers.Conv3D(flt, kernel, activation='relu', padding=padding, dilation_rate=(1, 1, 1),
                          use_bias=False, name=str(i) + '_' + name)(x)
        x = layers.BatchNormalization(name=str(i) + '_bn_' + name)(x)
        i += 1
    return x

def part_one(ipt):
    l1 = forward_conv(ipt, (4, 4), (3, 3), name='enc1')
    d2 = layers.MaxPool3D(pool_size=(2, 2, 2))(l1)
    l2 = forward_conv(d2, (4, 4), (3, 3), name='enc2')
    return l1, l2

def part_inner(ipt1, ipt2):
    l1 = forward_conv(ipt1, (4, 4), (3, 3), name='enc1')
    l2 = forward_conv(ipt2, (4, 4), (3, 3), name='enc2')
    return l1, l2

def part_two(ipt1, ipt2):
    l2 = forward_conv(ipt2, (4, 4), (3, 3), name='dec2')
    u1 = layers.UpSampling3D(size=(2, 2, 2))(l2)
    r1 = forward_conv(ipt1 + u1, (4, 4), (3, 3), name='dec1')
    return r1

initial = tf.ones([1, 256, 368, 368, 1], dtype=tf.float16)

tf.random.set_seed(1)

with tf.GradientTape() as g:
    g.watch(initial)
    l1_, l2_ = part_one(initial)
    for _ in range(2):
        l1_, l2_ = part_inner(l1_, l2_)
    opt_ = part_two(l1_, l2_)
    loss = tf.reduce_mean(l1_) + tf.reduce_mean(opt_)
    gd = g.gradient(loss, initial)
    print('-' * 100)
    print(f'loss is {loss} and grad is {np.sum(gd)} with ckpt= {ckpt}')

Behavior description
When using tf.float32 setting, the result of gradient is reasonable with value around 0.6, however, when shift to tf.float16 with mixed_precision, the gradient is constantly 0. Should we expect that the computed gradient is so different between normal float32 mode and mixed_precision float16 mode? Thank you!

I have met the same issue with TF 2.4.1.  It prevents the use of newer features of new hardware.

Was able to replicate the issue in TF nightly-2.6.0.dev20210603 ,please find the gist here..Thanks !

@WingsOfPanda I ran your code and face OOM error. Can you please check and reduce size to run the code successfully? Thanks!

@jvishnuvardhan oh sure, u may reduce the initial size
initial = tf.ones([1, 256, 368, 368, 1], dtype=tf.float16)
to
initial = tf.ones([1, 64, 128, 128, 1], dtype=tf.float16)
in my code. or even smaller if you need

btw, I checked with the latest build tf2.5, the result is the same

@jvishnuvardhan Hi. Any update? Have you able to run my code after reduce the input size? Thank you!

@WingsOfPanda Now it is throwing the following error. Can you try installing the version mentioned in the error trace below and then verify whether the issue was resolved or not.
NotFoundError: No algorithm worked! Please try upgrading to cuDNN 8.0.5. You are using cuDNN 8.0.4, which has a bug causing this error. [Op:Conv3DBackpropFilterV2]
Please check the gist here. Thanks!

@jvishnuvardhan hmm, I think I have the correct cuDNN version. on my side I didn't see this error. One more thing, I tried tf2.6 but still facing this problem. Any further suggestions?


[Build] Improving debug and build-dir options

Adding debug option to task_build to allow debug version of TVM to be built easily.
Adding --build-dir option to bash.sh to specify which directory holds the built tvm to be patched through to docker.

cc @Liam-Sturge

Thanks @neildhickey!


prospector is a wrapper

prospector for Python is really just a wrapper around various static analyzers. It is not an analyzer itself.

:+1:


added node example

Added a node example check it out!

How I test in console?

To test you will need node installed. Then you can type node hello_world.js


[BUG] python openai_api.py后报错'FieldInfo' object has no attribute 'required'

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior
No response
期望行为 | Expected Behavior
No response
复现方法 | Steps To Reproduce
python openai_api.py --server-name <IP_ADDRESS> --server-port 7905 -c /home/wyxx/warBackup/ner/qwen/model后遇到的问题
运行环境 | Environment
- OS:Ubuntu 16.04
- Python:3.9
- Transformers:4.32.0
- PyTorch:2.1.0
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):11.6
- GPU v100

备注 | Anything else?
具体报错
/data/software/anaconda3/lib/python3.9/site-packages/torch/cuda/init.py:138: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 11060). Please update your GPU driverby downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your versionof the CUDA driver. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
return torch._C._cuda_getDeviceCount() > 0
/data/software/anaconda3/lib/python3.9/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
warn("The installed version of bitsandbytes was compiled without GPU support. "
/data/software/anaconda3/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cadam32bit_grad_fp32
[2023-11-14 13:38:57,861] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda(auto detect)
/data/software/anaconda3/lib/python3.9/site-packages/pydantic/_internal/_config.py:317: UserWarning: Valid config keys have changed in V2:

'allow_population_by_field_name' has been renamed to 'populate_by_name'
'validate_all' has been renamed to 'validate_default'
warnings.warn(message, UserWarning)
/data/software/anaconda3/lib/python3.9/site-packages/pydantic/_internal/fields.py:128: UserWarning: Field "model_persistence_threshold" has conflict with protected namespace "model".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = ().
warnings.warn(
Traceback (most recent call last):
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1130, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/data/software/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/generation/utils.py", line 27, in
from ..deepspeed import is_deepspeed_zero3_enabled
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/deepspeed.py", line 37, in
from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig
File "/data/software/anaconda3/lib/python3.9/site-packages/accelerate/init.py", line 3, in
from .accelerator import Accelerator
File "/data/software/anaconda3/lib/python3.9/site-packages/accelerate/accelerator.py", line 35, in
from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
File "/data/software/anaconda3/lib/python3.9/site-packages/accelerate/checkpointing.py", line 24, in
from .utils import (
File "/data/software/anaconda3/lib/python3.9/site-packages/accelerate/utils/init.py", line 133, in
from .launch import (
File "/data/software/anaconda3/lib/python3.9/site-packages/accelerate/utils/launch.py", line 33, in
from ..utils.other import is_port_in_use, merge_dicts
File "/data/software/anaconda3/lib/python3.9/site-packages/accelerate/utils/other.py", line 30, in
from deepspeed import DeepSpeedEngine
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/init.py", line 16, in
from . import module_inject
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/module_inject/init.py", line 6, in
from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/module_inject/replace_module.py", line 792, in
from ..pipe import PipelineModule
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/pipe/init.py", line 6, in
from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/pipe/init.py", line 6,in
from .module import PipelineModule, LayerSpec, TiedLayerSpec
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/pipe/module.py", line 19, in
from ..activation_checkpointing import checkpointing
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 25, in
from deepspeed.runtime.config import DeepSpeedConfig
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/config.py", line 29, in
from .zero.config import get_zero_config, ZeroStageEnum
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/zero/init.py", line 6,in
from .partition_parameters import ZeroParamType
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 616, in
class Init(InsertPostInitMethodToModuleSubClasses):
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 618, in Init
param_persistence_threshold = get_config_default(DeepSpeedZeroConfig, "param_persistence_threshold")
File "/data/software/anaconda3/lib/python3.9/site-packages/deepspeed/runtime/config_utils.py", line 115, in get_config_default
assert not config.fields.get(
AttributeError: 'FieldInfo' object has no attribute 'required'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/wyxx/warBackup/ner/qwen/openai_api.py", line 512, in
model = AutoModelForCausalLM.from_pretrained(
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 503, in from_pretrained
model_class = get_class_from_dynamic_module(
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/dynamic_module_utils.py", line 485, in get_class_from_dynamic_module
final_module = get_cached_module_file(
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/dynamic_module_utils.py", line 313, in get_cached_module_file
modules_needed = check_imports(resolved_module_file)
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/dynamic_module_utils.py", line 174, in check_imports
importlib.import_module(imp)
File "/data/software/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers_stream_generator/init.py", line 1, in
from .main import init_stream_support
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers_stream_generator/main.py", line1, in
from transformers import (
File "", line 1055, in _handle_fromlist
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1121, in getattr
value = getattr(module, name)
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1120, in getattr
module = self._get_module(self._class_to_module[name])
File "/data/software/anaconda3/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1132, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'

@jklj077 我看log里有deepspeed，这个是用户finetune过的ckpt吗？我不太了解deepspeed finetune这块。猜测与 pydantic 冲突相关。deepspeed要求v1，目前的open_api.py要求v2 —— 不过可以简单把open_api.py里一些地方改成兼容v1。

@jklj077 https://github.com/QwenLM/Qwen/pull/632/files 帮忙double check并merge下这个PR（让 openai_api.py兼容pydantic v1）
然后，楼主可以尝试用 https://github.com/QwenLM/Qwen/pull/632/files 这个PR里的 openai_api.py，并安装与deepspeed兼容的pydantic 版本（可能是 pip install "pydantic<2.0.0"，不太确定)

@qihangf 我们观察到您加载了DeepSpeed ZeRO-3微调后的checkpoint并可能使用bitsandbytes库。

DeepSpeed与本repo中openai_api.py对pydantic的版本要求有冲突，该问题已在主分支fix，请尝试回退pydantic版本小于2.0.0并使用更新后的openai_api.py
目前本repo中量化相关均依赖AutoGPTQ，bitsandbytes目前我不确定是否可用。

谢谢大佬们，果然是pydantic版本问题

请问一下，最后用什么版本的pydantic才好用啊

请问一下，最后用什么版本的pydantic才好用啊

我最后改用了1.9.0版本没有问题了


Add last_endline optional parameter in bag.to_textfiles (#3743)

Implements #3743

LGTM

Care to merge @martindurant ?
On Thu, Jul 12, 2018 at 3:45 PM, Martin Durant<EMAIL_ADDRESS>wrote:

LGTM
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/3745#issuecomment-404644870, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszJoH08fwAqEnc3-mHr_yHRZY-qQ5ks5uF7VVgaJpZM4VKmNI
.


Cannot install in Mac Catalina

hi al.
I am trying to install this lib but got failed on Mac Catalina
this is error log i got. Can you take a look.
2019-12-28T15:35:26,718 Collecting secp256k1
2019-12-28T15:35:26,718   Created temporary directory: /private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-unpack-8b97l6sg
2019-12-28T15:35:26,782   Using cached https://files.pythonhosted.org/packages/52/62/d7bf3829e126e517e253d2e22a63511c54bbaac34d7ddea316cde040fc49/secp256k1-0.13.2.tar.gz
2019-12-28T15:35:26,842   Added secp256k1 from https://files.pythonhosted.org/packages/52/62/d7bf3829e126e517e253d2e22a63511c54bbaac34d7ddea316cde040fc49/secp256k1-0.13.2.tar.gz#sha256=a3b43e02d321c09eafa769a6fc2c156f555cab3a7db62175ef2fd21e16cdf20c to build tracker '/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-req-tracker-andiakrd'
2019-12-28T15:35:26,847     Running setup.py (path:/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-install-asprpmg1/secp256k1/setup.py) egg_info for package secp256k1
2019-12-28T15:35:26,847     Running command python setup.py egg_info
2019-12-28T15:35:27,126     0.29.2
2019-12-28T15:35:27,147     WARNING: The wheel package is not available.
2019-12-28T15:35:28,680       ERROR: Command errored out with exit status 1:
2019-12-28T15:35:28,681        command: /Users/antory/.pyenv/versions/3.6.9/envs/p36env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-wheel-y2eg77wj/pytest-runner/setup.py'"'"'; __file__='"'"'/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-wheel-y2eg77wj/pytest-runner/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-wheel-cwga1twi
2019-12-28T15:35:28,681            cwd: /private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-wheel-y2eg77wj/pytest-runner/
2019-12-28T15:35:28,681       Complete output (7 lines):
2019-12-28T15:35:28,681       WARNING: The wheel package is not available.
2019-12-28T15:35:28,681       usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
2019-12-28T15:35:28,681          or: setup.py --help [cmd1 cmd2 ...]
2019-12-28T15:35:28,681          or: setup.py --help-commands
2019-12-28T15:35:28,681          or: setup.py cmd --help

2019-12-28T15:35:28,681       error: invalid command 'bdist_wheel'
2019-12-28T15:35:28,681       ----------------------------------------
2019-12-28T15:35:28,681       ERROR: Failed building wheel for pytest-runner
2019-12-28T15:35:28,903     ERROR: Failed to build one or more wheels
2019-12-28T15:35:28,948     Traceback (most recent call last):
2019-12-28T15:35:28,948       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/setuptools/installer.py", line 128, in fetch_build_egg
2019-12-28T15:35:28,949         subprocess.check_call(cmd)
2019-12-28T15:35:28,949       File "/Users/antory/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 311, in check_call
2019-12-28T15:35:28,949         raise CalledProcessError(retcode, cmd)
2019-12-28T15:35:28,949     subprocess.CalledProcessError: Command '['/Users/antory/.pyenv/versions/3.6.9/envs/p36env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/tmpkysqesjx', '--quiet', 'pytest-runner==2.6.2']' returned non-zero exit status 1.

2019-12-28T15:35:28,949     During handling of the above exception, another exception occurred:

2019-12-28T15:35:28,949     Traceback (most recent call last):
2019-12-28T15:35:28,949       File "<string>", line 1, in <module>
2019-12-28T15:35:28,949       File "/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-install-asprpmg1/secp256k1/setup.py", line 295, in <module>
2019-12-28T15:35:28,949         "Topic :: Security :: Cryptography"
2019-12-28T15:35:28,949       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
2019-12-28T15:35:28,949         _install_setup_requires(attrs)
2019-12-28T15:35:28,949       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
2019-12-28T15:35:28,949         dist.fetch_build_eggs(dist.setup_requires)
2019-12-28T15:35:28,950       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/setuptools/dist.py", line 721, in fetch_build_eggs
2019-12-28T15:35:28,950         replace_conflicting=True,
2019-12-28T15:35:28,950       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
2019-12-28T15:35:28,950         replace_conflicting=replace_conflicting
2019-12-28T15:35:28,950       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1065, in best_match
2019-12-28T15:35:28,950         return self.obtain(req, installer)
2019-12-28T15:35:28,950       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1077, in obtain
2019-12-28T15:35:28,950         return installer(requirement)
2019-12-28T15:35:28,950       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
2019-12-28T15:35:28,950         return fetch_build_egg(self, req)
2019-12-28T15:35:28,950       File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/setuptools/installer.py", line 130, in fetch_build_egg
2019-12-28T15:35:28,950         raise DistutilsError(str(e))
2019-12-28T15:35:28,950     distutils.errors.DistutilsError: Command '['/Users/antory/.pyenv/versions/3.6.9/envs/p36env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/tmpkysqesjx', '--quiet', 'pytest-runner==2.6.2']' returned non-zero exit status 1.
2019-12-28T15:35:28,969 Cleaning up...
2019-12-28T15:35:28,970   Removing source in /private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-install-asprpmg1/secp256k1
2019-12-28T15:35:28,983 Removed secp256k1 from https://files.pythonhosted.org/packages/52/62/d7bf3829e126e517e253d2e22a63511c54bbaac34d7ddea316cde040fc49/secp256k1-0.13.2.tar.gz#sha256=a3b43e02d321c09eafa769a6fc2c156f555cab3a7db62175ef2fd21e16cdf20c from build tracker '/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-req-tracker-andiakrd'
2019-12-28T15:35:28,983 Removed build tracker '/private/var/folders/md/xlm2b58s5wv6sm85x0rdz_w80000gn/T/pip-req-tracker-andiakrd'
2019-12-28T15:35:28,983 ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
2019-12-28T15:35:28,983 Exception information:
2019-12-28T15:35:28,983 Traceback (most recent call last):
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 153, in _main
2019-12-28T15:35:28,983     status = self.run(options, args)
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 382, in run
2019-12-28T15:35:28,983     resolver.resolve(requirement_set)
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 201, in resolve
2019-12-28T15:35:28,983     self._resolve_one(requirement_set, req)
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 365, in _resolve_one
2019-12-28T15:35:28,983     abstract_dist = self._get_abstract_dist_for(req_to_install)
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 313, in _get_abstract_dist_for
2019-12-28T15:35:28,983     req, self.session, self.finder, self.require_hashes
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 224, in prepare_linked_requirement
2019-12-28T15:35:28,983     req, self.req_tracker, finder, self.build_isolation,
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 49, in _get_prepared_distribution
2019-12-28T15:35:28,983     abstract_dist.prepare_distribution_metadata(finder, build_isolation)
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/distributions/source/legacy.py", line 39, in prepare_distribution_metadata
2019-12-28T15:35:28,983     self.req.prepare_metadata()
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 563, in prepare_metadata
2019-12-28T15:35:28,983     self.metadata_directory = metadata_generator(self)
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/operations/generate_metadata.py", line 124, in _generate_metadata_legacy
2019-12-28T15:35:28,983     command_desc='python setup.py egg_info',
2019-12-28T15:35:28,983   File "/Users/antory/.pyenv/versions/3.6.9/envs/p36env/lib/python3.6/site-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess
2019-12-28T15:35:28,983     raise InstallationError(exc_msg)
2019-12-28T15:35:28,983 pip._internal.exceptions.InstallationError: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

pip install wheel


Backend implementation

@lrebscher could you list below here in the epic in a comment high level all the tasks required and estimates with it?
I will take care of creating cards based on the list you will compile here, thanks!

Create data model with its entities and relations (16h)
Create dummy data for each entity and upload obj files (4h)
Test queries to verify that we can query all required information efficiently (8h)
Export dummy data so that we can import it across stages (8h)
Administration (2h)

Configure users
Configure roles

Configure endpoint permissions for each entity (4h)
Ingestion

Implement strapi CSV import plugin & custom import logic (16h)
Implement connector to google-drive where raw data is located (8h)

Preprocessing: write obj to draco converter & verify results (8h)
Generate frontend-client based on openapi specification, if possible (4h)


ipc-shm-us-lib: ipc_os_init(): Can't open /dev/uio0 device

I've tried a couple of builds and I keep getting this error. There really isn't a device /dev/uio0. I'm using s32g274ardb2 and latest fsl-auto-yocto-bsp, building with fsl-image-auto.

NXP recommends using the dedicated NXP support page for questions and support for NXP products.
Please reach out to https://www.nxp.com/support/support:SUPPORTHOME where you can find different methods for requesting support: community, tickets or live chat.


Google drive folder is empty

Hello, I was unable to locate the model checkpoint weights in the Google Drive folder you shared. Could you please upload the weights? Unfortunately, I couldn't access the Baidu Drive link. Thanks a lot.

I confirm that the problem is still relevant.

Sorry for this.  I have updated the Google Drive folder with the model checkpoint weights. You can access them here:
https://drive.google.com/drive/folders/1ArEgk-VNKZnXd5Fjkf6tiNA4YvBDZCCF?usp=sharing


shellvar doesn't work properly for export

Want to edit below line in puppet
/opt/dtp/etc/dtp.conf
export CHO='NON-PROD'

Used shellvar resource, please find the snippet below
shellvar { 'CHO':
  ensure => exported,
  target => '/opt/dtp/etc/dtp_config',
  value  => 'NON-PROD',
}

when I run first time it's removing the word "export", got o/p like javascriptDTP="NON-PROD"
If I run second time, getting below o/p
javascript"Notice: /Stage[main]/Example::Dtp/Shellvar[DTP]/ensure: ensure changed 'present' to 'exported'
and the o/p is   export DTP="NON-PROD
please refer the below url
http://stackoverflow.com/questions/33168034/unable-to-print-augeas-for-own-configuration

This is probably a duplicate of https://github.com/hercules-team/augeasproviders_shellvar/pull/10

I will merge #10 and you can test if that fixes your issue (but I'm pretty sure it will).

Fixed in #10. Will release asap.

Released as 2.2.1


Fix"sort by" when clicking deposited label

several deposited vaults (crvCOIMP crvBBTC) do not move to the top when clicking the "deposited" label. Others such as crvSETH and crvSTETH work as expected.

I believe this is fixed now


return cluster instance

when we create a geodemographic cluster result, we're appending the labels to the input gdf without giving the user access to the cluster instance. this means there's a lot of functionality lost in the exchange and prevents the user from, e.g. calculating silhouette scores or other fit stats.
The original intent was to keep things dead simple and just append a new column to the gdf to avoid having users make the merge and such, but i'm thinking that might be abstracting away too much...
it would be simple to add the cluster model as an attribute on the Community instance, but a single Community could be used as cluster input several times (e.g. to add labels from different clustering methods to the same gdf) so i'm not sure how to provide access to several cluster models

maybe something like
return Community if attach==True else ClusterInstance

?

resolved by #121


Create fresh-air.md

Hello,
I am attachinng a pull request, for Fresh Air.
Have a nice day

Hi, @m3m3m3m3. Thanks for the PR. A couple of things. All commits to this repository require that you use the -s option to sign off the commit.
Also, it is required that you fill in the Initial Committers and Sponsor sections.

Also please fill out the Pre-Existing repository.
It might also help to rework the various sections removing some of "manifesto" style remarks and focusing on Engineering. Especially if your aim is to attract people to your lab there needs to be clarity of thought and an achievable goal clearly stated; I can help with this if we can have a conversation. I would focus on the aspects of "space" and "Position" after re-iterating the "time" aspect in the TimeStamp (or chronological ordering) service which is what the Blockchain really is (as you state).

So, why did you chose to close this?

Dear Arnaud,
It has been not clear for new user how to simply register the project.
First. The hyperledger instruction says: fork, then as I was using github via web I pressed fork button,
buy hyperledged answered that I should have not used it, because then I was working on a local copy.
then guessing what button to press then was edit, create a .md, and pull it.
then I received the commentaries with some command line commit.
looking in google it says that it is not possible to do this github command line on web browser.
then looking for the command I installed github for desktop window. and git
but it says no valid fork.
so, for a fresh refresh then I delete it.
And tried installed again with another name.
but when publishing it, now github says permission error.
so, it was like walking blind in try-error.
best
Looking In google it said the command requested to run can not be
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On 2 March 2018 2:01 PM, Arnaud J Le Hors<EMAIL_ADDRESS>wrote:

So, why did you chose to close this?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.


Stricter pinning on libtiff

Comment:
I would like to pin libtiff a littler harder than it currently is.
While the ABI is compatible between builds, it seems that the returned values from the libraries often change by a very small amount to make it so that downstream projects need to patch before they become compatible with libtiff.
This is, for example, what is happening with pillow at the moment with:

https://github.com/conda-forge/libtiff-feedstock/pull/76
https://github.com/conda-forge/pillow-feedstock/pull/119

I'm OK with that. We should updated the run_exports here and the global pin. What to you suggest x.x or x.x.x?

x.x

the problem is, that without a widespreading repodata patch, old packages that are build will be "chosen" over new packages that are claimed to be incompatible with a new release.

the problem is, that without a widespreading repodata patch, old packages that are build will be "chosen" over new packages that are claimed to be incompatible with a new release.

Indeed :-(

For now, I did a "widespread" patch for pillow, but you know, this is less than ideal.

an other one bites the dust: https://github.com/conda-forge/libtiff-feedstock/issues/78

libtiff 4.5.0 broke abi compatibility with 4.4. I'm going to pin harder today
@conda-forge-admin please rerender

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I rerendered the recipe in conda-forge/libtiff-feedstock#86.


Dynamically added list items are expandables too without reinitializing

This attaches one single event handler to an expandable list, but sets
the scope to the collapsible header. This way you attach one event
handler instead of potentially hundreds and it'll make sure any new
dynamically added list item is able to be collapsed without
reinitializing the component.
It also plays nicely with frameworks (Vuejs for example). Due to the
nature of Javascript, reinitializing the component causes the last item
to not work, because the items get cached before the last item is
rendered by the framework.
Here's a quick example of what I changed: https://jsfiddle.net/nyy085de/
Note: this only changes how the expandables work, not collapsibles. I
don't have the time to have a look at the rest, but I have the feeling
that this kind of thing happens on other parts of the project; applying
event handlers to ONLY a cached element/list of things.

Thanks!


Address ethics better

The standards need to do a better job addressing research ethics. Please use this issue thread to make suggestions.

One might incorporate statements in the Essential section of the General Standard along these lines:
•	discusses the research ethics issues arising in undertaking the work, or if no issues are deemed to have arisen, discusses what issues were considered potentially relevant and why these were dismissed.
•	where formal ethics approval has been obtained, this must be stated including any reference numbers.
I included the ‘no issues’ clause to differentiate absence of discussion from determination of there being no ethical issues to consider.  Stating these in the general standard would then provide a basis against which discussion of things like inability to share data owing to ethical limits (referred to elsewhere in the standard) could rest.

We now have ethics supplements for research with human participants and research with secondary data. We still need an ethics supplement for engineering research (i.e. research that proposes tools, technologies or other functional artifacts)


Overpass generator string now includes version number

Recently, a version number was added to the generator string, e.g. <osm version="0.6" generator="Overpass API <IP_ADDRESS> 054bb0bb">. Unfortunately, this change breaks Achavi. The following patch should fix it:
diff --git a/js/FormatRegistry.js b/js/FormatRegistry.js
index 59c350c..1c534b2 100644
--- a/js/FormatRegistry.js
+++ b/js/FormatRegistry.js
@@ -36,7 +36,7 @@ FormatRegistry.prototype.getFormatType = function(doc) {
     // special cases with common root node name but different content
     if (type === 'osm') {
         var generatorAttribute = doc.documentElement.getAttribute('generator');
-        if (generatorAttribute && generatorAttribute === 'Overpass API') {
+        if (generatorAttribute && generatorAttribute.indexOf('Overpass API') == 0) {
             type = 'osmDiff';
         } else {
             var node = doc.documentElement.firstChild

startsWith does not work with Internet Explorer. So i would suggest indexOf

@HolgerJeromin : good point, patch updated accordingly.


Understanding relay modern match and module directives

Hello maintainers,

[ ] Received Feedback that API is still iterated upon, I will refresh the PR as the API stabilizes. cc: @jstejada. Updating to latest relay version has broken this branch.

This PR is to help gather feedback to merge https://github.com/relayjs/relay-examples/pull/96, I have modified the existing App for easy review and comparison.
The app is modified to showcase a working example with the usage of @match and @module directive
The existing simple example app for todo is great for initial adopters to understand the basics before diving into these advanced constructs, that was the rationale for adding another example to this repo via https://github.com/relayjs/relay-examples/pull/96, that can showcase the usage of @match and @module directive.

Task status

[x] database changes

[x] Add CRUD support for Todo variants PLAIN_TODO and BOLD_TODO

[x] server changes

[x] schema changes to support usage of @match and @module

[x] JSDependency scalar
[x] support union FormattedContent = PlainContent | BoldContent

[x] support content(supported: [String!]!): FormattedContent field in Todo type to support @match
[x] PlainContent and BoldContent types with js field to use with @module

[x] Update mutations add/update/remove Todos
[x] Update Todo type to support js field to use with @module

[x] frontend changes

[x] add support for dynamic import with @babel/plugin-syntax-dynamic-import

[x] flow-typed changes
[x] .flowconfig changes

[x] webpack changes
[x] Update mutations add/update/remove Todos to handle Todo types with optimistic updates
[x] Implement support for relay Environment::operationLoader get/load
[x] Modify TodoTextInput component to capture both TodoTypes
[x] Add component to render/rename PlainTodo
[x] Add component to render/rename BoldTodo
[x] Modify TodoList component to list both TodoTypes
[x] Showcase @match and @module usage using Todolist
[x] Showcase standalone @module usage with Todo component
[x] Showcase sample SuspenseMatchContainer to async load matched modules

[ ] Pending stuff worth doing?

[ ] Top level README.md?
[ ] Showcase how operationLoader.get gets triggered on GC
[ ] Showcase how supported arg can be used in the server

The Todo type is modified to support two simple content types, plain text and bolded text.

Demo gif

Note: I think we could do the minimal webpack / babel magic that is required to get this showcased and leave the rest as an exercise to the adopters, so far able to code split the operations and the renderer components with/without @match scope

Usage snippet
    fragment Todo_todo on Todo {
      complete
      id
      content @match {
        ...PlainTodoRenderer_value @module(name: "PlainTodoRenderer")
        ...BoldTodoRenderer_value @module(name: "BoldTodoRenderer")
      }
    }

compiled snippet
fragment Todo_todo on Todo {
  complete
  id
  content(supported: ["PlainContent", "BoldContent"]) {
    __typename
    ... on PlainContent {
      ...PlainTodoRenderer_value
      __module_operation: js(module: "PlainTodoRenderer_value$normalization.graphql")
      __module_component: js(module: "PlainTodoRenderer")
    }
    ... on BoldContent {
      ...BoldTodoRenderer_value
      __module_operation: js(module: "BoldTodoRenderer_value$normalization.graphql")
      __module_component: js(module: "BoldTodoRenderer")
    }
  }
}

Prerequisite 1: Define the JSDependency scalar in the schema.
The shape of the scalar is not important to relay.
Relay will hand over the value returned by server in this scalar via Environment::operationLoader
https://github.com/facebook/relay/blob/f6f6723849c1b7dbd7b72a4a0f6fd18a521f8d44/packages/relay-runtime/store/RelayModernStore.js#L72
methods get(reference: JSDependency)
https://github.com/facebook/relay/blob/f6f6723849c1b7dbd7b72a4a0f6fd18a521f8d44/packages/relay-runtime/store/RelayStoreTypes.js#L452
and load(reference: JSDependency)
The implementors have to decode the value to perform the required operations specific to their project setup so that the operationLoader.get / operationLoader.load resolves to the correct module operation that is generated by the relay compiler. In the current implementation, it is mapped to echo the value of module arg in the js field send by the client.
Prerequisite 2: @match directive can be specified only on fields of type Union or Interface in format yourFieldName(supported: [String!]!): YourUnionOrInterfaceType
Example
union FormattedContent = PlainContent | BoldContent

type Todo implements Node {
  id: ID!
  content(supported: [String!]!): FormattedContent
  complete: Boolean!
}

At runtime relay will pass the matched supported Union type names to the graphql server like this
         content(supported: ["PlainContent", "BoldContent"]) {

Note: I do not yet know how this can be leveraged at server side, add more details as things become more clear.
Prerequisite 3: @module directive can be specified only on fields that resolves to types that have a js(module: String): JSDependency field
In order to use the @module directive
Examples
type PlainContent {
  data: PlainContentData
  js(module: String!, id: String): JSDependency # <-- special field
}

relay compiler aliases this field to store the module operation and module component details as you can observe in the snippet from the compiled .graphql files. Your graphql server can potentially enrich this to return information that can be used by the operation loader
  __module_operation: js(module: "PlainTodoRenderer_value$normalization.graphql")
  __module_component: js(module: "PlainTodoRenderer")

@module directive can be used independent of @match directive
see how the Todo module is code splitted using the same

Hi @soneymathew! Thank you very much for this example - very helpful. Could you please elaborate on valid use cases of this feature as well as motivation behind it?
From my understanding (correct me if I am wrong please) it seems like this allows you to do code-splitting of components rendering union types while previously, it was necessary to require everything in the final bundle (even though it was not used). Some use-cases I can imagine:

A layout which renders different variants of some component based on the defined union (one-way trip, roundtrip, multicity trip, ...). I could lazy-load only the components that are actually necessary to be rendered based on the response.
Server-driven rendering where GraphQL returns union with many types and client renders (and lazy-loads) only what's supported.

Do I understand it correctly? Are there any other "bigger picture" use-cases I am missing?

@mrtnzlml most of what you have described is matching my understanding as well, additionally, we could leverage this to do more than that based on insights from the discussions
These directives primarily facilitate data-driven code splitting, one of the 3 strategies discussed in
https://developers.facebook.com/videos/2019/building-the-new-facebookcom-with-react-graphql-and-relay/ @ 18:30

it's not restricted just to Union types.
I have updated the example to use it outside the @match boundaries
see: https://github.com/relayjs/relay-examples/pull/95#discussion_r280618641
you can test the impact by reloading the page after clearing all the todos, you will notice that Todolist does not initiate a request for fetching Todo module any more
@module name  is specified by the client, which gives us the possibility that the same type can be represented differently at different places.
for example, the module that is used to render a specific type in mixed search results that contain multiple types in a drop down will differ from how the same type can be represented in a full page  search results
some other examples where these can be used as I can imagine

visual representations that differ based on platform and form factor of the device

lightweight module when screen real estate is less
lightweight module when the device doesn't need polyfills

feature flagging modules for A/B experiments using in combination with @include / @skip directive

@josephsavona / @jstejada should be able to explain more possibilities based on their current usages.

At runtime relay will pass the matched supported Union type names to the graphql server like this ... Note: I do not yet know how this can be leveraged at server side, add more details as things become more clear.

To use the example here of plain vs bolded text, imagine that an earlier version of the todo app only supports plain text todos. Later, on the server support is added for bold. If a user creates bold todos in a new version of the app and views them on an older version, ideally they would still be able to see those todos, just rendered as plaintext instead of bold. The supported argument tells the client what types the particular client version knows about so that it can attempt to do things like fall back to an  alternative rendering strategy for some content  (ie falling back to plaintext instead of bold).

This PR is no longer required.
Please refer to https://github.com/relayjs/relay-examples/tree/main/data-driven-dependencies for working example of 3D


Adding Don't Recommend Channel & Not Interested Endpoints

Describe your suggestion
YouTube on the mobile app and website hides a function to say not interested and not recommend channel for videos when you click on the ellipsis of a video to get more actions. I didn't see any way of doing this in the library's docs or while looking through the library myself.
Other details
No response
Checklist

[X] I am running the latest version.
[X] I checked the documentation and found no answer.
[X] I have searched the existing issues and made sure this is not a duplicate.
[X] I have provided sufficient information.

It's already possible. If you see a button or menu on YouTube's website then its node is most likely buried somewhere in the API response. Once you find it, you can call its endpoint to execute the action.
Example:
import { Innertube, UniversalCache, YTNodes } from 'youtubei.js';

const yt = await Innertube.create({
  cache: new UniversalCache(true),
  generate_session_locally: true,
});

yt.session.on('auth-pending', (data) => {
  console.log(`Go to ${data.verification_url} in your browser and enter code ${data.user_code} to authenticate.`);
});

yt.session.on('auth', ({ credentials }) => {
  console.log('Sign in successful:', credentials);
});

yt.session.on('update-credentials', async ({ credentials }) => {
  console.log('Credentials updated:', credentials);
  await yt.session.oauth.cacheCredentials();
});

await yt.session.signIn();
await yt.session.oauth.cacheCredentials();

const home = await yt.getHomeFeed();

const firstVideo = home.videos.first().as(YTNodes.Video);
const menu = firstVideo.menu;

if (menu) {
  const notInterestedMenuItem = menu.items.get({ icon_type: 'NOT_INTERESTED' }).as(YTNodes.MenuServiceItem);
  const endpoint = notInterestedMenuItem.endpoint;

  endpoint.payload = {
    feedbackTokens: [endpoint.payload.feedbackToken],
    isFeedbackTokenUnencrypted: false,
    shouldMerge: false
  };

  const response = await endpoint.call(yt.actions, { parse: false });
  console.log(response.data.feedbackResponses);
}


Refactor

dcm2mha complete, please review and will continue on to mha2nnunet
WILL NOT MERGE, REVIEW ONLY

Check out resolve_duplicates, I have rewritten it to be a whole lot more readable.

Looking into resolve_duplicates now, will provide a proposed refactor, which hopefully simplifies it even further

Please see https://github.com/DIAGNijmegen/picai_prep/pull/9

ISSUE: ./imagesTr/10356_204666144756126301976935564941758369725_0_0000.nii.gz images are saved in dataset_json as ./imagesTr/10356_204666144756126301976935564941758369725_0.nii.gz


Misc issues

I reviewed everything but "Guided Learning" and "Extensions".

https://doc.babylonjs.com/divingDeeper/developWithBjs/howToStart

http://localhost:1338/sandbox/index-local.html should be http://localhost:1338/sandbox/public/index-local.html

https://new-doc-page.vercel.app/divingDeeper/developWithBjs/approvedNamingConventions

"documentation" => points to the old doc repo

https://new-doc-page.vercel.app/divingDeeper/developWithBjs/usingThePlaygroundForBJSDev

https://github.com/BabylonJS/Babylon.js/blob/master/src/Math/babylon.math.ts => 404

https://new-doc-page.vercel.app/divingDeeper/developWithBjs/contributeToDocs

https://github.com/BabylonJS/Documentation/tree/master/content => points to the old doc repo
"Documentation" => points to the old doc repo
"Documentation/public/html/img/" => points to the old doc repo

https://new-doc-page.vercel.app/divingDeeper/gui

"dat.GUI" => wrong link

https://new-doc-page.vercel.app/divingDeeper/gui/selector

Fig 1 has no picture / text

https://new-doc-page.vercel.app/divingDeeper/gui/gui3D

when clicking on "Babylon 3D GUI" in the left menu, the "The Babylon GUI" entry gets also selected

https://new-doc-page.vercel.app/divingDeeper/importers/stl

"how to load from any file type" => 404

https://new-doc-page.vercel.app/divingDeeper/lights/shadows_csm

the links 2 and 3 at the bottom of the page are not displayed correctly => only the text should be visible and behave like a link

https://doc.babylonjs.com/divingDeeper/materials/using/introToPBR

"Ashkimin Shirley Microfacets" => 404

https://doc.babylonjs.com/divingDeeper/materials/using/ktx2Compression

"Working with compressed texture formats" paragraph => bad layout of the text + link not as a link

https://doc.babylonjs.com/divingDeeper/materials/advanced/logarithmicDepthBuffer

http://www.gamasutra.com/blogs/BranoKemen/20090812/85207/Logarithmic_Depth_Buffer.php => broken

https://doc.babylonjs.com/divingDeeper/mesh/creation/set/tiled_plane

"This image..." => there's no image displayed

https://doc.babylonjs.com/divingDeeper/mesh/creation/param/ribbon_extra

when clicking on "Ribbon in Detail" in the left menu, "Ribbon" entry gets also selected

https://doc.babylonjs.com/divingDeeper/mesh/creation/param/ribbon_maths

when clicking on "Ribbon with Maths" in the left menu, "Ribbon" entry gets also selected

https://doc.babylonjs.com/divingDeeper/mesh/transforms/parent_pivot/pivots

when clicking on "Pivots" in the left menu, "Rotate Around Axis" entry gets also selected

https://doc.babylonjs.com/divingDeeper/mesh/decals

link to "Decal" picture broken

https://doc.babylonjs.com/divingDeeper/postProcesses/usePostProcesses

"this" in "Read this if you want to know how to fix it." => 404

https://doc.babylonjs.com/divingDeeper/scene/optimizeOctrees

https://www.babylonjs.com/?OCTREE does not link to the right page
https://www.babylonjs.com/?INSTANCES does not link to the right page

https://doc.babylonjs.com/divingDeeper/scene/renderToPNG

text too big for the "hint:" paragraph

Sprites => Sprite Map

maybe make a sub-directory below Sprite Map to put all the sprite map related pages?

https://doc.babylonjs.com/divingDeeper/tags

two broken picture links

https://doc.babylonjs.com/divingDeeper/webXR/webXRInputControllerSupport

"WebXR Input Source" => 404

https://doc.babylonjs.com/advanced_topics/introToShaders/shaderCodeInBjs

"BABYLON.ShaderMaterial" => 404

https://doc.babylonjs.com/toolsAndResources/tools/playground

"indexStable.html" => produces errors in the console log

https://doc.babylonjs.com/toolsAndResources/tools/inspector

"Introduction to the inspector - video series" => 404

https://doc.babylonjs.com/toolsAndResources/tools/inspector/customizingInspector

"How to Start Contributing to Babylon.js" => 404
"this tutorial" => 404
"here" in "here in Github" => 404

https://doc.babylonjs.com/toolsAndResources/tools/particleEditor

"The Inspector" => 404

https://doc.babylonjs.com/toolsAndResources/tools/spriteEditor

"The Inspector" => 404

https://doc.babylonjs.com/toolsAndResources/tools/skeletonViewer

"The Inspector" => 404
"Skeleton Viewer Documentation" => 404

First 4 pages + "Shadow Only" + "Cell Shading" below Tools and Resources => Available Asset Libraries => The Materials Library

links not displayed as links / 404

https://doc.babylonjs.com/toolsAndResources/assetLibraries/materialsLibrary/skyMat

"A Practical Analytic Model for Daylight" and "Simon Wallner" => 404

https://doc.babylonjs.com/toolsAndResources/assetLibraries/postProcessLibrary/asciiArtPP

https://github.com/BabylonJS/Babylon.js/blob/master/dist/preview => 404
https://github.com/BabylonJS/Babylon.js/blob/master/dist/preview => 404

https://doc.babylonjs.com/toolsAndResources/assetLibraries/postProcessLibrary/digitalRainPP

https://github.com/BabylonJS/Babylon.js/blob/master/dist/preview => 404
https://github.com/BabylonJS/Babylon.js/blob/master/dist/preview => 404

https://doc.babylonjs.com/toolsAndResources/utilities/Minimise_Vertices

"Materials and Facets" => 404

https://doc.babylonjs.com/toolsAndResources/offsite

"PR in Github" => points to the old doc repo

Also:

bad layout when multiple PGs:

eg: https://doc.babylonjs.com/divingDeeper/mesh/creation/param/custom_extrude

legend beneath a picture is too far below
eg: https://doc.babylonjs.com/divingDeeper/mesh/transforms/parent_pivot/pivots

small display problem when showing code in a bullet point list (https://doc.babylonjs.com/divingDeeper/particles/solid_particle_system/sps_vertices)

Image alignment is now correct,
Text code blocks now have a top margin to separate between them. I leave the rest to @PirateJC  :-)

Fixed all links apart from these:
https://www.babylonjs.com/?OCTREE from https://doc.babylonjs.com/divingDeeper/scene/optimizeOctrees
https://www.babylonjs.com/?INSTANCES from https://doc.babylonjs.com/divingDeeper/scene/optimizeOctrees
https://www.babylonjs-playground.com/indexStable.html#6F0LKI#2 from https://doc.babylonjs.com/toolsAndResources/tools/playground

Fixed Octree doc demos.
Leaving indexStable as is for now.


Fixed an issue in which proposals concluded as 'needs revision' aren'…

…t closed properly

Derp

Big whoops. But good on you for fixing it so quickly!


win,test: fix shared library build

Ifdef out the tests that rely on internal symbols when making a shared build.
R=@piscisaureus

Hold, I think I got a thing wrong here.

Now it's ready for review. CI: https://jenkins-iojs.nodesource.com/view/libuv/job/libuv+any-pr+multi/121/ (the CI doesn't do shared builds, but it serves as regression testing).
Also /cc @bnoordhuis maybe you can also help review :-)

LGTM, I suppose.

Thanks Ben! FTR, what the patch does is define USING_UV_SHARED when building the run-tests and run-benchmarks targets, iff libuv was built as a shared library. That define is already in use by us: https://github.com/libuv/libuv/blob/v1.x/include/uv.h#L35 and by projects using libuv as a shared library: https://github.com/nodejs/io.js/blob/master/src/node.h#L18
So I ifdef-ed out the tests that use internal symbols, using that define.

I tested this and works fine, so in it went! :-)


Update cats-effect-laws, cats-effect-std to 3.3.8

Updates

org.typelevel:cats-effect-laws
org.typelevel:cats-effect-std

from 3.3.6 to 3.3.8.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.typelevel" } ]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #459.


Futhark: add missing tokens

Adds recognition of 'def'/'type^'/'type~' keywords and adds some
missing characters to the regexes for identifiers and punctuation.

Merged, thanks!


Write a wiki page how to set settings for numerical identifiers

Description of the issue
With the change to numerical identifiers we need a documentation how to change Protégé user settings.
Ideas of solution

Write a wiki how-to page
Link in CONTRIBUTE.md

Workflow checklist

[ ] I am aware of the workflow for this repository
[ ] I added this issue to the Project 'Issues'. If suitable, I add it to further Projects.

I'll create the page shortly before we change to avoid confusion. The content will be:
File -> Preferences -> new entities
Select:
Entity IRI:

Active ontology IRI
/
Auto-generated ID

Entity Label:

Same as label renderer

Auto-generated ID:

Numeric (iterative)
prefix: OEO_[your user id] e.g. OEO_0001 if you user ID is 1
Digit count: 4

Please include there also the info that OEO_0000 is reserved for the existing classes.

Specified IRI: http://openenergy-platform.org/ontology/oeo/

this is solved


PTDF matrix does not use lookup dictionary when retrieving a row of the matrix

I found that the PTDF matrix does not recognize the lookup dictionary when trying to access the vector of line data corresponding to a given bus. For example, in the code below, I get an error on the last line because the number for the bus "Cobb" (321) is greater than that dimension of the matrix (73). This is using Julia 1.9 with PSY v3.3.0, PNM v0.10.0, and PSCB v1.2.5.

using PowerNetworkMatrices, PowerSystems, PowerSystemCaseBuilder
rts_sys = build_system(PSITestSystems, "test_RTS_GMLC_sys")

ptdf = PTDF(rts_sys)
bus = get_component(Bus, rts_sys, "Cobb")

vector = ptdf[:, get_number(bus)]

This error came up because I was using the code from PowerSimulations.jl here to compute the LMPs of a Sienna simulation. This was fairly easy to work around (just mapping the values in ptdf.axes[1] to the order they appear) but wasn't sure if this was expected behavior. Wondering if it would make sense to have ptdf.axes[1] be the bus names instead of the bus numbers to avoid this issue. For example, if I instead call ptdf["A1", :], I do get the expected behavior where it gives me the column corresponding to Line "A1".
Also, as a related note, it was a little confusing to me that the PTDF matrix expects the indexes to be passed as column then row, rather than row then column (e.g., ptdf["A1", 101] rather than ptdf[101, "A1"]) since the dimensions in ptdf.axes are listed with the bus number as dimension 1 and line names as dimension 2. Just an observation as a first time user. Overall though, thank you for making and supporting the Sienna suite. It has been very helpful.

I think this is a bug in the indexing.

I looked into this issue and it seems that we default to interpreting int indexes the the bus location and not the number. if you pass the bus object then it works fine. We could make this error more informative.

this should work
using PowerNetworkMatrices, PowerSystems, PowerSystemCaseBuilder
rts_sys = build_system(PSITestSystems, "test_RTS_GMLC_sys")

ptdf = PTDF(rts_sys)
bus = get_component(Bus, rts_sys, "Cobb")

vector = ptdf[:, bus]


#677 implemented for Post button.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: mrdevms sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Test Fluent/Organizations

[ ] Test Fluent/Organizations
[ ] Test Fluent/Organization

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@806c98e). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #5   +/-   ##
=========================================
  Coverage          ?   43.94%
  Complexity        ?      262
=========================================
  Files             ?       20
  Lines             ?      817
  Branches          ?        0
=========================================
  Hits              ?      359
  Misses            ?      458
  Partials          ?        0

Flag
Coverage Δ
Complexity Δ

#phpunit
43.94% <0.00%> (?)
262.00% <0.00%> (?%)

#unit
43.94% <0.00%> (?)
262.00% <0.00%> (?%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 806c98e...55c50cc. Read the comment docs.


exit status doesn't seem to propagate when starting container

This code runs a busybox container executing /bin/false and prints exit status: https://gist.github.com/nresare/c31d4a42edae2e8efd58
I would expect exit status to be 1 and not 0 though, as the equivalent execution on the command line results in exit status 1:
docker run busybox /bin/false
docker inspect --format "{{.State.ExitCode}}" $(docker ps -l --format="{{.ID}}")

What is up with that?

For completeness, the java code uses the following dependency snippet in pom.xml:
    <dependency>
      <groupId>com.spotify</groupId>
      <artifactId>docker-client</artifactId>
      <version>3.5.0</version>
    </dependency>

This is probably the problem in the linked gist :smile: :
    final ContainerConfig containerConfig = ContainerConfig.builder()
        .image("busybox").cmd("/bin/true").build();

Also the container in the gist is created but never started, so /bin/false never runs.
With this code:
public static void main(String... args) throws Exception {
    try (DefaultDockerClient client = createClient()) {
      final ContainerConfig containerConfig = ContainerConfig.builder()
          .image("busybox").cmd("/bin/false").build();
      final ContainerCreation creation = client.createContainer(containerConfig);
      final String id = creation.id();

      client.startContainer(id);
      final ContainerExit containerExit = client.waitContainer(id);
      final ContainerInfo containerInfo = client.inspectContainer(id);

      System.out.printf("docker inspect .State.ExitCode for container %s: %d\n",
                        id,
                        containerInfo.state().exitCode());

      System.out.printf("ContainerExit.statusCode for container %s: %d\n",
                        id,
                        containerExit.statusCode());
    }
    System.exit(0);
  }

I get this output:
docker inspect .State.ExitCode for container 58913b4ff121b00fff3391e994db785fd0e50330711bca595335725ae30b4973: 1
ContainerExit.statusCode for container 58913b4ff121b00fff3391e994db785fd0e50330711bca595335725ae30b4973: 1


Fixes for a SPA written in TypeScript

Fixed on page editing through the PreviewController (Missing ClientResources)
Added TypeScript Definition for the FormRendering service
Fixed 404 reply on the ContentDeliveryAPI when performing JSON requests against the web URLs

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Remko Jantzen seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


[FEATURE] OPDS v2.0 support

Describe the solution you'd like
Stump should expose a v2.0-compatible OPDS API
Additional context
https://drafts.opds.io/opds-2.0.html

Experimental support was added (to the experimental docker image) by #367. I am closing this issue. If (when) issues arise relating to OPDS V2, a separate issue should be created to outline it specifically.


Added "aarch64" for M1 Monterey

Added "aarch64", this is needed on M1 mac's running Mac OS Monterey

This is not enough to get it working on M1 macs, it complains about:
[WARNING] No driver contexts detected for the current operating system, no maven properties set!
[22:27:30,628 INFO][TID 20][c.l.s.c.DriverFactory.instantiateWebDriver] Local Operating System: MAC OS X
[22:27:30,631 INFO][TID 20][c.l.s.c.DriverFactory.instantiateWebDriver] Local Architecture: aarch64
[22:27:30,631 INFO][TID 20][c.l.s.c.DriverFactory.instantiateWebDriver] Selected Browser: chrome


cannot find the acl2020-commonsense-reasoning repository

When I tried to clone from acl2020-commonsense-reasoning repository, I found that the repository cannot found

Thanks for spotting this. I should be the link of the current repository. "https://github.com/SAP-samples/emnlp2021-attention-contrastive-learning".


Welcome modal fixes

:nut_and_bolt: Description: What code changed, and why?
These commits fix issues where

welcome modal would display again on page refresh after dismissing it on first viewing
telemetry checkbox toggle would not display on first viewing of welcome modal

:chains: Related Resources
https://github.com/chef/automate/issues/1655
:+1: Definition of Done

Welcome modal only automatically displays on first login (unless option to always display is selected)
Welcome modal provides option to toggle telemetry on first login

:athletic_shoe: How to Build and Test the Change

Start new, empty browser instance (incognito)
Log in
See welcome modal is displayed w/ correct content and checkbox options
Dismiss modal and refresh page
See modal doesn't reappear (unless option to always show on startup is selected)

:white_check_mark: Checklist

[x] Tests added/updated?
[ ] Docs added/updated?

we hit some issues with this, we will come back to it later when theres time.


Update submitted applications page

Description
Updated submitted applications permit page with new card designs, wording consistencies, and utilizing new StatusPill component.
PADS-359
Types of changes

New feature (non-breaking change which adds functionality)

Checklist

[X] I have read the CONTRIBUTING doc
[X] I have checked that unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

Looking good....2 issues I see:

The Latest Updates note is not working and also it is showing some weird code instead of a note...if there is no note it should be empty and if there is one it should show.
Required Permits that show should no longer be hyperlinked (or appear to be a hyperlink)
Under Required Permits, it says "We are investigating...." when empty but what if there is nothing left to investigate?  (Perhaps a UX question)

Looking good....2 issues I see:

The Latest Updates note is not working and also it is showing some weird code instead of a note...if there is no note it should be empty and if there is one it should show.
Required Permits that show should no longer be hyperlinked (or appear to be a hyperlink)
Under Required Permits, it says "We are investigating...." when empty but what if there is nothing left to investigate?  (Perhaps a UX question)

Note fixed. Some temporary placeholder slipped though.
Appearance of link removed
Will need UX to chime in on this one. Not sure how to proceed.

Looking good! I'll confirm with Navigators on what to display when there's nothing to investigate and get back to you in a bit. Here are a few other issues:

When there's no updates, the Latest updates in the card should display "No updates at the moment"
The spacing between permit cards is missing. It should be 16 px for both the Required Permits section and the Submitted Applications section.
Please remove the hover effects from the permits in the Required Permits section, so that it doesn't look interactive.
Please change the "Ask a Navigator" button to "Ask my Navigator", and change the style to the secondary button style. Please let me know if you need the css.

Re: re-ordering the dropdown on Nav side.

For Application stage, please follow this order: Pre-submission, Application submission, Technical review, Pending decision, and Post-decision.
For Application status, please switch "Pending client action" and "In progress".

Looking good! I'll confirm with Navigators on what to display when there's nothing to investigate and get back to you in a bit. Here are a few other issues:

When there's no updates, the Latest updates in the card should display "No updates at the moment"
The spacing between permit cards is missing. It should be 16 px for both the Required Permits section and the Submitted Applications section.
Please remove the hover effects from the permits in the Required Permits section, so that it doesn't look interactive.
Please change the "Ask a Navigator" button to "Ask my Navigator", and change the style to the secondary button style. Please let me know if you need the css.

Re: re-ordering the dropdown on Nav side.

For Application stage, please follow this order: Pre-submission, Application submission, Technical review, Pending decision, and Post-decision.
For Application status, please switch "Pending client action" and "In progress". The rest looks good to me!

Changes made, now awaiting final designs of the "Required Permits" section.

Re: the empty state messaging for the Require Permits section.

Replace the current message with "We will update the necessary permits here as the project progresses. You may see this message while we are investigating or if no application is needed at this time."
Show this message when NO permits are in "Pre-submission", OR when NONE of the "Pre-submission" permits' Needed field is "Yes".

Please make this additional change. Change the empty state message for the Submitted Applications section from "You will see your permit applications here once submitted" to "We will update your submitted applications here".
This change will mitigate user confusion. Captured this problem while reviewing another ticket. The old message doesn't consider the situation when Props submit an intake form, they may have already submitted applications.

Can be merged once sign off given by UX.


Latency between Unity marker stream recorded in Lab Streaming Layer and Unity marker stream recorded in Emotiv

I am currently building a Unity based experiment with Emotiv Flex, saline.
I set up a simple oddball task with 1 sec display of an object and a isi of 500 ms.
I recorded the marker stream sent from Unity both on LabRecorder (LSL) and received the marker stream on EmotivPro for visualization purpose.
However, when I checked the data, there is a 60ms to whooping 200ms latency between Unity marker stream (LSL) and Marker stream recorded with the EEG data. There are inconsistent jitters also, which makes it almost impossible to correct them in the preprocessing stage.
This is a serious issue as EEG experiments are very time sensitive and 200ms latency ruins the sanity of the data.
Which marker timestamp is correct? or are they both wrong? I have included the marker plot for illustration.
In conclusion, the timestamp values on the Unity marker stream is different depending on which program (LSL, EmotivPro) records them.
PS. There are also loss of Markers in EmotivPro, too (see the marker between 6 and 7, which only exists on Unity marker)

Hi @gomsing
Thank you for your valuable feedback. EEG Timestamp and Inject markers are one of our key features, and we handle them with utmost care.
Regarding the reported issue, there are a few aspects to consider:

Marker Transmission Delay: Check the time delay when a marker is sent from Unity to EmotivPro.

Consider sending the marker time along with the marker value to ensure it is unaffected by the transmission method.
Explore our API https://emotiv.gitbook.io/cortex-api/headset/syncwithheadsetclock for syncing with the headset clock. This API offers a high-performance solution, leveraging steady clock time to overcome the instability of computer lock time.
(I will confirm with the development team whether this API is supported in the Unity Plugin.)

Marker Timestamp Method:

Examine how the marker timestamps are set.
Marker Creation Performance

Thank you for your prompt response.
The marker time was recorded on the LSL, along with the marker value.
In Unity, a marker is sent during trial onset and offset.
But the LSL time (timestamp) value does not match among two methods, which is weird considering they were recorded at the same time on the same LSL clock.
I merged the two dataframes along the 'timestamp' with the latency value (dif) to clarify.

I will try to record the time on Unity and compare it with the LSL time and Emotiv time.

@gomsing .

The EEG timestamp processing is independent of other processes; it is not delayed by any application process.
A marker is incorporated into an EEG stream based on its timestamp.
If a marker's timestamp is added before being sent via LSL, there is no impact from the transmission layer thereafter.
Therefore, we should ensure that the timestamp is added to the marker as soon as possible after its created.

@hoangphamemotiv
Thank you for a prompt response.
I think I may have phased the question in a confusing way, so let me clarify and start over.
So this is my setup, the markers are sent from Unity via LSL4Unity.

The marker stream was recorded both on LabRecorder (see bottom left) and via Emotiv Inlet.
We streamed the data via inlet only for visualizing purpose, but checked it anyway to see whether we can just use one data stream instead of two.
However, when I use the LSL timestamp to align the two data, there is a mismatch, with  markers recored via Emotiv Inlet being 60~100ms behind.
So based on this and your response I have a few questions.
When you say "a marker's timestamp is added before being sent via LSL" does this mean I should include the timestamp value from Unity in the marker stream? will it be incorporated via that value (and what clock is the Emotiv timestamp based on?)
Also, if that is the case, why is there as delay when it is incorporated into Emotiv when it was based on LSL timestamp value? Could the latency due to the process of Unity sending the marker stream to LSL -> LSL records the Unity marker stream -> LSL sends the received Unity marker stream to Emotiv?
Moreover,  I noticed the 'transition time' in the inlet stream page. Could you please give me more information on this?

Thank you for your time.

@gomsing
"Does this mean I should include the timestamp value from Unity in the marker stream?" - Yes, that's correct.
From your image, I see that the "number of channels" is set to 1, which means it only includes the marker value and doesn't have the marker timestamp.
It should be set to 3. You can refer to this link: (https://github.com/Emotiv/labstreaminglayer/tree/master/examples/unity#how-to-send-marker-from-unity-to-emotiv-lsl-inlet ) to include the timestamp of the marker and the time point when it's sent via LSL.
The timestamp should be in Epoch Time format,  (https://www.epochconverter.com/). Then, you can use it for markers.
The "transition time" represents the transmission time from when the marker is sent via LSL to when it's received by EmotivPro. I noticed that it's very small at 0.1 ms, so the issue likely doesn't originate from there.

@hoangphamemotiv
AH! thank you so much
I thought the LSL will take care of the timestamp, but apparently I was wrong.
Thank you for a prompt response :)

@gomsing
With "number of channels" set to 1 (only marker value), EmotivPro will assign a Timestamp to the marker as soon as it's received. It will then subtract the transmission time. However, EmotivPro doesn't have information about when the marker was created. That's why Option 2, which involves adding a timestamp to the marker stream, is very useful.
I hope this clarifies the process.

I tried it again with the Epoch time. My marker sender code is the following.
`using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using LSL;
using System;
public class MarkerSender : MonoBehaviour
{
private LSL.StreamInfo markerStreamInfo;
private LSL.StreamOutlet markerOutlet;
public int lslChannelCount = 3;

private const LSL.channel_format_t lslChannelFormat = LSL.channel_format_t.cf_double64;

private double[] sample;

public long epochNow;

void Start()
{

    // Create a marker stream info
    markerStreamInfo = new LSL.StreamInfo("ExpMarkerStream", "Markers", lslChannelCount, 0,
                                            lslChannelFormat, "UnityLSL001");

    LSL.XMLElement chns = markerStreamInfo.desc().append_child("channels");
        string[] channels = {"MarkerTime", "MarkerValue", "CurrentTime"};
        foreach(string chanName in channels) {
            chns.append_child("channel").append_child_value("label", chanName).append_child_value("type", "Marker");
        }

    // Create a marker stream outlet
    markerOutlet = new LSL.StreamOutlet(markerStreamInfo);
    Debug.Log("Marker stream created.");
}

public void SendMarker(int markerNum, long epochNow)
{
    sample = new double[lslChannelCount];

    if (markerOutlet != null)
    {
        // Send the marker using the outlet

        epochNow = (long)(DateTime.Now - new DateTime(1970, 1, 1)).TotalSeconds;
        sample[0] = epochNow;
        sample[1] = markerNum;
        sample[2] = epochNow;
        markerOutlet.push_sample(sample);
        Debug.Log(string.Join(", ", sample));
    }
    else
    {
        Debug.LogWarning("Marker outlet is not initialized.");
    }
}

}
`
However, the marker value received from Emotiv Inlet and received from LSL via Unity Outlet still has latency.
Now it is about 200 ms, which is not good for any EEG research.
Can you help me with why this is happening?

@gomsing
can you store timestamp you added for marker in : public void SendMarker(int markerNum, long epochNow) ?
then check if the marker add into correct EEG with closest timestamp.
so the delay = Marker_Timestamp - EEG_Timestamp which added marker.

@hoangphamemotiv
So I merged the marker stream data and eeg data based on LSL timestamp, and then compared the recorded Epoch time.

The marker recorded on inlet has some data losses, and it has a consistent latency of 32400 epoch time.

@gomsing
I don't be clear your test yet, maybe we are talking with different terms.
what are "LSL timestamp", "the recorded Epoch time"?
How did you draw your table? i see MarkerTime some row is NaN? and it doesn't show timestamp exactly to ms. (it is e+09). so it is hard for me to check.
can you send me some delay values from this:
the delay = Marker_Timestamp - EEG_Timestamp which added marker.

and i see you are using: epochNow = (long)(DateTime.Now - new DateTime(1970, 1, 1)).TotalSeconds;
seem marker timestamp is second. but it should be miliseconds

@hoangphamemotiv
So what I mean by LSL timestamp and recorded Epochtime is that I merged the two data streams (marker and eeg) according to the LabStreamingLayer clock(LSL timestamp) and compared the recorded epochtime between the markers recorded in unity and markers recorded (received) in emotiv.
If the marker was aligned correctly, the epochtime should have the same values as it was aligned with the EEG data within the EmotivPro.
The delay value is the 'dif' in the data table, and the NaN value is there because the marker timing does not match (which is the problem here)
I will try again with the miliseconds, but in the reference you gave me, it used TotalSeconds.
https://github.com/Emotiv/labstreaminglayer/blob/master/examples/unity/Assets/LSL4Unity/Scripts/LSLMarkerStream.cs

I changed the epochNow to miliseconds, and the difference between the timestamp in EmotivPro and the timestamp recorded in Unity is too big, the EmotivPro seems to run on total seconds?

Timestamp in Emotivpro is seconds. (exact to ms)
marker timestamp from LSL is miliseconds .
=> to compare time. it need to be: /1000 to convert to seconds

These are the recorded delay values

Both are recorded through LabRecorder (https://github.com/labstreaminglayer/App-LabRecorder)
So the 'timestamp' is the LSL timestamp, and "Timestamp" is the emotiv/unity epochtime.
Here are the two recording files.
EEG.csv
Marker.csv

is that correct files? i see timestamp much different.

The left is the raw EEG data with the markers, and the right is the Marker data with only the event marker values and the timestamp.
The data with only the markers is here. There are many marker drops (as you can see on the plot), so the data size may not match.
markerEEG.csv

@gomsing .
so sorry, but i haven't had what i need for the checking.
your marker file had timestamp very different with EEG timesamp. can you explain about that?

Our QA tested the feature again. And it works fine.

@hoangphamemotiv
I will look into the timestamp, I did not notice that.
The feature worked fine for us, too, but the marker received from inlet was just recorded at a different time from the data recorded from the Unity outlet.

Thanks @gomsing  and waiting for the marker file with correct timestamps.
I'm trying to check step by step on your flow. the first step is ensure a marker is added correct into EEG stream with correct timestamp.
Yes, timestamp is double to store a big number of Epoch time.

@hoangphamemotiv
I checked my original files, and it seems that the all the timestamps are recorded wrong, which caused the latency (I think). I will check my codes, try again, and get back to you. Thank you.

@hoangphamemotiv Whatever I try, the Epochtime recorded in Unity and the Epochtime recorded in Emotiv seems to be off.
That is why it wont align correctly in EmotivPro. The attatched files are the most recently recorded ones, but they still have vastly different timestamps although they are recorded at the same time.
But I'm lost why this would happen?? I've attatched my unity scripts that I use to send the markers to LSL.
Marker.csv
EEG.csv
Scripts.zip

@hoangphamemotiv
So I checked the Epoch timestamps, and the Emotiv timestamps are not the current time (about 9 hour difference) whileas the Epochtime stamps in Unity is based on current(KST) time.

@hoangphamemotiv
P011_EPOCFLEX_201215_2023.11.14T15.10.00+09.00.md.bp.csv
P011_EPOCFLEX_201215_2023.11.14T15.10.00+09.00_intervalMarker.csv
Here are the files exported from Emotiv Pro

Thanks @gomsing . so from the file name, we can see it handled the your timezone +09. and it recorded : 15.10.00 today.
so, i guess, your time now is about 3 PM.
There are some suggestions:

As i noted above, when you created a marker from software/apps with timestamp = A. it will be added to an EEG sample EXACTLY just sampled in Emotiv headset, not  add to a sample run to software that time point. -> we need to wait x (ms) more to received the correct EEG sample. -> you will get correct EEG sample at: A + x. BUT with timestamp = A. x can be affected and changed by manything. but A never changed.
If you want to add markers to EEG stream yourself, you should store Marker timestamp, and find the closest EEG sample manually to add.

@hoangphamemotiv
One question, why aren't the timezone reflected in the timestamps when the system clearly receives it?? This is a big problem as the alignment, as you said, is done via Epoch time. This clearly defeats the purpose of sending Epochtime in the marker stream for proper marker alignment in EmotivPro.
I guess I can just use the LSL time and the marker timestamps recorded with Unity LSL WITHOUT sending it to EmotivPro, if the EEG samples have little to no latency of sending the data to LSL (as you said before).
If I want to add the markers to EEG stream, should I add +9 hours to my Unity Epochtime? because the closest timestamp is currently non-existnet.

@gomsing
i think EEG Timestamp is: epoch time with time zone +9.
Timestamp went with Markers and send to EmotivPro via LSL also Epoch time with timestamp.
because, if marker time and EEG time was not close, EmotivPro couldn't map them.
I'm not sure about timesamp in your Marker.csv file.

@hoangphamemotiv
Ahhhhhhh, I now see why this happended.
The EEG timestamps are epoch time with GMT +9, and the Markertime recorded KST+9.
But this is weird because I did not touch anything in the Unity code and just used
epochNow = (double)(DateTime.Now - new DateTime(1970, 1, 1)).TotalMilliseconds / 1000;
To record the marker timestamps. maybe I shouldnt use DateTime.Now?

@hoangphamemotiv
I changed it to epochNow = (double)(DateTime.UtcNow - new DateTime(1970, 1, 1)).TotalMilliseconds / 1000;, and now I gives me the expected values.
Still have the issue of latency in the LSL recorded data, but I think that is not the problem regarding Emotiv.

@hoangphamemotiv
I think I found the source of the problem.
In plot 1, the red line plot is the EEG data exported from EmotivPro (with the Unix Timestamps as X axis values) and the blue line plot is the same EEG data that was pushed to LSL and recorded on LabRecorder.
The plot below is the LSL pushed data with LSL timestamps as the X axis.
You can see that the Marker value now aligned up, but the EEG signal data is vastly different EVEN on unix time.
This is a serious issue regarding EEG experiments, because whileas the marker epoch timestamps are the SAME in emotiv pro data and LSL data, the corresponding EEG data are not. This means that marker was not the problem, the EEG stream was.


✅ Add a11y test for Radio

Resolves #2017

Maybe one test with aria-label instead of label as well?


Upgrade to Symfony v5.4 minimum

Pull Request
Related issue
Fixes #263 (Symfony part)
What does this PR do?

Upgrade to Symfony v5.4 as LTS

command definition via https://symfony.com/doc/5.4/console.html#creating-a-command

Leverage Rector to ease and automate upgrade (removed as per comments)

@norkunas should rector be kept (for later usage?)
PR is ready nonetheless

@norkunas should rector be kept (for later usage?)

@brunoocasali should decide, but in case we need it very rarely if at all, maybe we shouldn't introduce as dev dependency to save some resources

By the way Symfony 6.3 should be added to matrix

Oki will do indeed the new 6.3
And remove rector for now as project is small so lets keep it simple

@brunoocasali should decide, but in case we need it very rarely if at all, maybe we shouldn't introduce as dev dependency to save some resources

I'm not opposed to keeping it, but my go-to opinion about this is: not to add new dependencies, even dev ones. So if we can remove it let's do it! :)

@brunoocasali @norkunas friendly ping, PR ready for reviews :)
cheers

@brunoocasali @norkunas friendly ping, PR ready for reviews :) cheers

lgtm

Can you check the CI error @94noni? 😇

i will have a look at it indeed, hopeful soon :)

i don't know why checks are not launched :s
alos for reviewers, as this PR will need a major release, perhaps it could be great to also review https://github.com/meilisearch/meilisearch-symfony/pull/266 so we can remove such old services names prior to v1
thx

Hi @94noni, it is because you never got a PR merged. As soon you merge your first one, the CI will automatically run :) it is just a way to avoid spam (I think so)!

6.1.* fails, I suggest to override public static function getDefaultName() and public static function getDefaultDescription() so they will work on all versions

can you try git commit --amend --no-edit && git push -f to retrigger CI? not sure why integration-tests (PHP 8.2) (Symfony 6.2.*) does not run

bors try

We should merge #266 before

Yeah, I was trying to force this CI action to trigger but had no success, either! Let's fix the conflicts then after approved we can see what is happen.

as https://github.com/meilisearch/meilisearch-symfony/pull/266 will hopefully be merged shortly, I will rework on this right after
thx

I just merged the #266 @94noni thanks a lot for your help :)

bors merge

Hello @94noni can you rebase your branch please? 😊
There are git conflicts so we cannot merge

@curquiza thx for the ping, I've tried resolving on GitHub.com ui, hope it is good
i will not have time available to finish soon this PR and thus also the following one https://github.com/meilisearch/meilisearch-symfony/pull/275
@norkunas please feel free to continue and proceed with PRs

@94noni Thanks for your work. Do you have some time to finish this PR? After that PR is merged, Symfony 7 support will be easier to implement.

@Chris53897 not so much sadly at the moment
If anyone wants to take over and finish it i will be more than happy to review etc

closing in favor of https://github.com/meilisearch/meilisearch-symfony/pull/301 which takes over
thanks for original reviewers and @Chris53897 for continuing :)


Failing tests under Windows

If I run the test suite (all_tests) locally under Windows (Python 2.7), some of the tests fail due to path separator mismatch.
I'm not sure if it is a problem of the tests or the code, or I do something wrong. Do you have any idea?
Here is the relevant output from a test run under Windows:
ERROR: test_shutil (fake_filesystem_unittest_test.TestPyfakefsUnittest)
Fake shutil module is bound
Traceback (most recent call last):
File "..\pyfakefs\fake_filesystem_unittest_test.py", line 92, in test_shutil
File "..\pyfakefs\pyfakefs\fake_filesystem_shutil.py", line 79, in rmtree
File "\pyfakefs\pyfakefs\fake_filesystem.py", line 902, in RemoveObject
IOError: [Errno 2] No such file or directory in the fake filesystem: '/test/dir1'
======================================================================
ERROR: test_rm_tree (example_test.TestExample)
Test example.rm_tree()
Traceback (most recent call last):
File "..\pyfakefs\example_test.py", line 133, in test_rm_tree
File "..\pyfakefs\example.py", line 121, in rm_tree
File "..\pyfakefs\pyfakefs\fake_filesystem_shutil.py", line 79, in rmtree
File "..\pyfakefs\pyfakefs\fake_filesystem.py", line 902, in RemoveObject
IOError: [Errno 2] No such file or directory in the fake filesystem: '/test/dir1'
======================================================================
FAIL: testExpandUser (fake_filesystem_test.FakePathModuleTest)
Traceback (most recent call last):
File "..\pyfakefs\fake_filesystem_test.py", line 1897, in testExpandUser
self.os.environ['USERPROFILE'].replace('\', '/'))
AssertionError: 'C:\Users\username' != 'C:/Users/username'
======================================================================
FAIL: test_glob (fake_filesystem_unittest_test.TestPyfakefsUnittest)
Fake glob module is bound
Traceback (most recent call last):
File "..\pyfakefs\fake_filesystem_unittest_test.py", line 80, in test_glob
AssertionError: Lists differ: ['/test/dir1\dir2a'] != ['/test/dir1/dir2a']
======================================================================
FAIL: test_get_globs (example_test.TestExample)
Test example.get_glob()
Traceback (most recent call last):
File "..\pyfakefs\example_test.py", line 118, in test_get_globs
AssertionError: Lists differ: ['/test/dir1\dir2a', '/test/d... != ['/test/dir1/dir2a', '/test/di...
======================================================================
FAIL: get_glob (example)
Doctest: example.get_glob
Traceback (most recent call last):
File "C:\Python27\lib\doctest.py", line 2226, in runTest
AssertionError: Failed doctest test for example.get_glob
File "..\pyfakefs\example.py", line 106, in get_glob

File "..\pyfakefs\example.py", line 114, in example.get_glob
Failed example:
get_glob('/test/file*.txt')
Expected:
['/test/file1.txt', '/test/file2.txt']
Got:
['/test\file1.txt', '/test\file2.txt']

Ran 413 tests in 0.273s
FAILED (failures=4, errors=2, skipped=22)

I would have thought that your pull request #77 addresses this.  Do you get these results before or after applying #77?

These are still there after applying the pull request. I will check if I can fix at least some of them - I guess you don't have a Windows test system.
One question (more may follow): in testExpandUser() the original os.path.expanduser() is tested - to fix this, it may be replaced in the fake file system (probably by calling the original version and adapting the path afterwards). Is this the way you would handle this? If yes, I would give it a try, likewise for the other problems.

Ok, I fixed most of the problems in #77, except the 2 failing doc tests. As their outcome is path separator-specific, they could be adapted something like TestPyfakefsUnittest.test_glob(), but this would be quite ugly in a doc string.
Any idea how to fix this more nicely or at least just disable it under Windows?

That's great progress.
Which doctest is this that is failing?

The last one (File "..\pyfakefs\example.py", line 114, in example.get_glob).
The problem here is that the expected output is different in Windows (due to the different path separator).

I see.  example.py is my file so I'll fix it.

Ok, thanks. You could also just make it a normal comment instead of a doc test, as there is already a similar unit test.
I'm done with pull request #77 in this case - if you decide to merge it, it shall also fix issue #28.

I fixed example.py with what I hope is a cross-platform doctest.
Could you try out pull request #82?  Thank you.

Still does not work correctly. The one that works for me (in Python 2.7) is the following:
    >>> # Get the expected file names in the platform's native style
    >>> (file_names == ['/test/file1.txt', '/test/file2.txt']   # UNIX-like
    ...      or file_names == [r'\\test\\file1.txt', r'\\test\\file2.txt']) # Windows
    True

First, the logic was incorrect (the result of the or was always the first string), but the more ugly problem is the escaping in doc strings - it seems that you need to add an escape due to the text parsing.
I have no experience in doc strings, so maybe you have a better idea...

Please give it another go.

Thanks  - as far as I can see, this should almost be ok, apart from the missing extra escape for the backslashes  (as mentioned above). I may check it again tomorrow - too late for today, time to sleep...

Here is the output under Windows:
File "...\pyfakefs\example.py", line 116, in example.get_glob
Failed example:
    file_names == (['/test/file1.txt', '/test/file2.txt']   # UNIX-like
                       or
                   [r'      estile1.txt', r'       estile2.txt']) # Windows
Expected:
    True
Got:
    False

It works if I change it to:
    >>> file_names == (['/test/file1.txt', '/test/file2.txt']   # UNIX-like
    ...                    or
    ...                [r'\\test\\file1.txt', r'\\test\\file2.txt']) # Windows
    True

(e.g. insert extra escapes before the backslashes), or if not If not using raw strings:
   ...                ['\\\\test\\\\file1.txt', '\\\\test\\\\file2.txt']) # Windows

After the latest update I get a new failed test:
Traceback (most recent call last):
  File "..pyfakefs\fake_filesystem_test.py", line 2158, in testGetattrForwardToRealOsPath
    'Get a real os.path function not implemented in fake os.path')
AssertionError: Get a real os.path function not implemented in fake os.path

Can _joinrealpath() be platform specific?

Whoops - I have to apologize! Your previous commit for the doc test (before you added my "fix") was completely ok. The mysterious behavior with the additional escape needed was due to an "r" I somehow managed to put before the docstring itself (and completely failed to see afterwards).
So if you revert your last commit and merge PR #85 this issue shall be fixed. Sorry for the inconvenience...

Backslashes removed.

All tests passing now  - closing.


Reverse geocoder doesn't find major roads

If you click on a major street, unless there is a housenumber, or shop mapped nearby, the reverse geocoder will find a random minor street instead.
Example (random big road in london):

Is this project getting any support? This is the inly desktop alternative to google maps that exists, would be great if some development could go into it.


chore: bdd-spring-1582015467 to 0.0.1

chore: Promote bdd-spring-1582015467 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Update EmailService.java

Add support for updating triggers

@publinchi thanks for your contribution. The purpose of this method is to just update the JobDescriptor and not the Triggers associated with it.
There are plans to create a method that updates individual triggers of a JobDescriptor and add or remove additional triggers.
The reasoning behind this is to update a JobDescriptor independently of it's trigger(s). This way a user can list triggers of a JobDescriptor and optionally modify the triggers.

Got it, doesnt apply... :(


uploading files

"Hello, after deploying on the Vercel platform and setting up the OpenAI API key, I'm encountering an error when uploading PDF file. Could you please help me understand the reason and how to resolve it?"

an error when uploading txt file


using the dataframe HTML styler changes representation of NaNs

The regular output of a dataframe with NaN values displays them as "NaN",
when using the HTML styler, they are displayed as "nan"
I'm not sure if this breaks anything but it is an inconsistency.

I don't think there is currently any way to actually specify how these display, and they are default rendered. You could do:
.format(lambda x: 'NaN' if pd.isnull(x) else x) I think
cc @TomAugspurger

Yeah, the default should changed. For .style it's defined here but should probably just be reusing pieces from https://github.com/pydata/pandas/blob/master/pandas/core/format.py