[Bug] When using the diffuers compiler, neither oneflow nor nexport can run properly

Your current environment information
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 62, in proxy_class
out = transform_mgr.transform_cls(cls)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 84, in transform_cls
mock_cls = self._transform_entity(mock_full_cls_name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 61, in _transform_entity
result = self.mocker.mock_entity(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 95, in mock_entity
return self.load_entity_with_mock(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 133, in load_entity_with_mock
mock_pkg = getattr(mock_pkg, name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 185, in getattr
obj_entity = self._get_module(name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 172, in _get_module
raise RuntimeError(
RuntimeError: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
Exception: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
cuda 12.2
nvidia:a6000 ada
onediff:1.9.1
No matter what method I use to specify the graphics card or set the number of graphics cards, this CUDA error is always reported
🐛 Describe the bug
(sd3) (base) ubuntu@kemove-Z690-GAMING-X-DDR4-V2:~/16T/lsm/ComfyUI_bf/onediff1/onediff_diffusers_extensions/examples/kolors$ CUDA_VISIBLE_DEVICES=0 python text_to_image_kolors.py --compiler oneflow
/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/xformers/ops/fmha/flash.py:211: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
@torch.library.impl_abstract("xformers_flash::flash_fwd")
/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/xformers/ops/fmha/flash.py:344: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
@torch.library.impl_abstract("xformers_flash::flash_bwd")
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 15.01it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 11.53it/s]
oneflow backend compile...
Could not load bitsandbytes native library: _get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
Traceback (most recent call last):
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 109, in
lib = get_native_library()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 88, in get_native_library
cuda_specs = get_cuda_specs()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/cuda_specs.py", line 38, in get_cuda_specs
highest_compute_capability=(get_compute_capabilities()[-1]),
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/cuda_specs.py", line 19, in get_compute_capabilities
return sorted(torch.cuda.get_device_capability(torch.cuda.device(i)) for i in range(torch.cuda.device_count()))
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/cuda_specs.py", line 19, in
return sorted(torch.cuda.get_device_capability(torch.cuda.device(i)) for i in range(torch.cuda.device_count()))
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 76, in get_device_capability
device_prop = get_device_properties(device)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 56, in get_device_properties
return flow._oneflow_internal._get_device_properties(device)
TypeError: _get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
CUDA Setup failed despite CUDA being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
WARNING [2024-12-02 16:26:22] /home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/custom_transform.py:48 - Failed to import register_diffusers from /home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/infer_compiler_registry/register_diffusers. e=RuntimeError("Failed to import diffusers.models.unets because of the following error (look up to see its traceback):\n_get_device_properties(): incompatible function arguments. The following argument types are supported:\n    1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties\n\nInvoked with: 'cuda:0'")
Starting warmup...
0%|                                                                                | 0/50 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 170, in _get_module
return importlib.import_module(module_name)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/unets/init.py", line 5, in
from .unet_1d import UNet1DModel
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/unets/unet_1d.py", line 24, in
from ..modeling_utils import ModelMixin
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 35, in
from ..quantizers import DiffusersAutoQuantizer, DiffusersQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/init.py", line 15, in
from .auto import DiffusersAutoQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/auto.py", line 21, in
from .bitsandbytes import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/bitsandbytes/init.py", line 2, in
from .utils import dequantize_and_replace, dequantize_bnb_weight, replace_with_bnb_linear
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/bitsandbytes/utils.py", line 32, in
import bitsandbytes as bnb
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
from . import research, utils
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 2, in
from .autograd._functions import (
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/research/autograd/_functions.py", line 8, in
from bitsandbytes.autograd._functions import GlobalOutlierPooler, MatmulLtState
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/init.py", line 1, in
from ._functions import get_inverse_transform_indices, undo_layout
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 244, in
class MatmulLtState:
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 266, in MatmulLtState
formatB = F.get_special_format_str()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/functional.py", line 413, in get_special_format_str
major, _minor = torch.cuda.get_device_capability()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 76, in get_device_capability
device_prop = get_device_properties(device)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 56, in get_device_properties
return flow._oneflow_internal._get_device_properties(device)
TypeError: _get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 62, in proxy_class
out = transform_mgr.transform_cls(cls)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 84, in transform_cls
mock_cls = self._transform_entity(mock_full_cls_name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 61, in _transform_entity
result = self.mocker.mock_entity(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 95, in mock_entity
return self.load_entity_with_mock(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 133, in load_entity_with_mock
mock_pkg = getattr(mock_pkg, name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 185, in getattr
obj_entity = self._get_module(name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 172, in _get_module
raise RuntimeError(
RuntimeError: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 41, in wrapper
result = dispatcher(first_param, *args, **kwargs)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/functools.py", line 889, in wrapper
return dispatch(args[0].class)(*args, **kw)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 199, in _
new_md_cls = proxy_class(type(mod))
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 66, in proxy_class
raise RuntimeError(
RuntimeError: An exception occurred during class transformation:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 170, in _get_module
return importlib.import_module(module_name)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/unets/init.py", line 5, in
from .unet_1d import UNet1DModel
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/unets/unet_1d.py", line 24, in
from ..modeling_utils import ModelMixin
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 35, in
from ..quantizers import DiffusersAutoQuantizer, DiffusersQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/init.py", line 15, in
from .auto import DiffusersAutoQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/auto.py", line 21, in
from .bitsandbytes import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/bitsandbytes/init.py", line 2, in
from .utils import dequantize_and_replace, dequantize_bnb_weight, replace_with_bnb_linear
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/bitsandbytes/utils.py", line 32, in
import bitsandbytes as bnb
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
from . import research, utils
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 2, in
from .autograd._functions import (
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/research/autograd/_functions.py", line 8, in
from bitsandbytes.autograd._functions import GlobalOutlierPooler, MatmulLtState
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/init.py", line 1, in
from ._functions import get_inverse_transform_indices, undo_layout
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 244, in
class MatmulLtState:
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 266, in MatmulLtState
formatB = F.get_special_format_str()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/functional.py", line 413, in get_special_format_str
major, _minor = torch.cuda.get_device_capability()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 76, in get_device_capability
device_prop = get_device_properties(device)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 56, in get_device_properties
return flow._oneflow_internal._get_device_properties(device)
TypeError: _get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 62, in proxy_class
out = transform_mgr.transform_cls(cls)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 84, in transform_cls
mock_cls = self._transform_entity(mock_full_cls_name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 61, in _transform_entity
result = self.mocker.mock_entity(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 95, in mock_entity
return self.load_entity_with_mock(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 133, in load_entity_with_mock
mock_pkg = getattr(mock_pkg, name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 185, in getattr
obj_entity = self._get_module(name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 172, in _get_module
raise RuntimeError(
RuntimeError: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
Exception: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/onediff_diffusers_extensions/examples/kolors/text_to_image_kolors.py", line 204, in
main()
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/onediff_diffusers_extensions/examples/kolors/text_to_image_kolors.py", line 177, in main
kolors.warmup(gen_args, args.warmup_iterations)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/onediff_diffusers_extensions/examples/kolors/text_to_image_kolors.py", line 119, in warmup
self.pipe(**warmup_args)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/pipelines/kolors/pipeline_kolors.py", line 993, in call
noise_pred = self.unet(
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1603, in _call_impl
result = forward_call(*args, **kwargs)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/deployable_module.py", line 38, in wrapper
return func(self, *args, **kwargs)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/online_quantization_utils.py", line 65, in wrapper
output = func(self, *args, **kwargs)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/graph_management_utils.py", line 123, in wrapper
ret = func(self, *args, **kwargs)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/args_tree_util.py", line 72, in wrapper
graph_file_management(func)(self, *mapped_args, **mapped_kwargs)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/graph_management_utils.py", line 123, in wrapper
ret = func(self, *args, **kwargs)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/deployable_module.py", line 142, in forward
dpl_graph = self.get_graph()
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/deployable_module.py", line 108, in get_graph
self._deployable_module_model.oneflow_module,
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/dual_module.py", line 30, in oneflow_module
self._oneflow_module = torch2oflow(self._torch_module)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 43, in wrapper
raise NotImplementedError(f"Transform failed of {type(first_param)}: {e}")
NotImplementedError: Transform failed of <class 'diffusers.models.unets.unet_2d_condition.UNet2DConditionModel'>: An exception occurred during class transformation:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 170, in _get_module
return importlib.import_module(module_name)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/unets/init.py", line 5, in
from .unet_1d import UNet1DModel
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/unets/unet_1d.py", line 24, in
from ..modeling_utils import ModelMixin
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 35, in
from ..quantizers import DiffusersAutoQuantizer, DiffusersQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/init.py", line 15, in
from .auto import DiffusersAutoQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/auto.py", line 21, in
from .bitsandbytes import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/bitsandbytes/init.py", line 2, in
from .utils import dequantize_and_replace, dequantize_bnb_weight, replace_with_bnb_linear
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/diffusers/quantizers/bitsandbytes/utils.py", line 32, in
import bitsandbytes as bnb
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
from . import research, utils
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 2, in
from .autograd._functions import (
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/research/autograd/_functions.py", line 8, in
from bitsandbytes.autograd._functions import GlobalOutlierPooler, MatmulLtState
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/init.py", line 1, in
from ._functions import get_inverse_transform_indices, undo_layout
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 244, in
class MatmulLtState:
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 266, in MatmulLtState
formatB = F.get_special_format_str()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/bitsandbytes/functional.py", line 413, in get_special_format_str
major, _minor = torch.cuda.get_device_capability()
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 76, in get_device_capability
device_prop = get_device_properties(device)
File "/home/ubuntu/miniconda3/envs/sd3/lib/python3.10/site-packages/oneflow/cuda/init.py", line 56, in get_device_properties
return flow._oneflow_internal._get_device_properties(device)
TypeError: _get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/builtin_transform.py", line 62, in proxy_class
out = transform_mgr.transform_cls(cls)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 84, in transform_cls
mock_cls = self._transform_entity(mock_full_cls_name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/transform/manager.py", line 61, in _transform_entity
result = self.mocker.mock_entity(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 95, in mock_entity
return self.load_entity_with_mock(entity)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/importer.py", line 133, in load_entity_with_mock
mock_pkg = getattr(mock_pkg, name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 185, in getattr
obj_entity = self._get_module(name)
File "/home/ubuntu/16T/lsm/ComfyUI_bf/onediff1/src/onediff/infer_compiler/backends/oneflow/import_tools/dyn_mock_mod.py", line 172, in _get_module
raise RuntimeError(
RuntimeError: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'
Exception: Failed to import diffusers.models.unets because of the following error (look up to see its traceback):
_get_device_properties(): incompatible function arguments. The following argument types are supported:
1. (arg0: int) -> oneflow._oneflow_internal._CudaDeviceProperties
Invoked with: 'cuda:0'

Getting same error


Does this support GPU?

I saw this article (https://medium.com/google-cloud/jupyter-tensorflow-nvidia-gpu-docker-google-compute-engine-4a146f085f17) that run Docker w/ GPU on GCP.
Is this possible for dsub and Pipeline API?

Not available in dsub yet. We are awaiting support from the Pipelines API, which is in progress.

GPU support added with #91 (release 0.1.3)


Why does DNSRecords require DNSRecord?

I am not sure why DNSRecord is requried as an argument of DNSRecords , because the zone identifier seems to be enough to specify DNS records which you want to fetch and the use-cases are not documented.
So I always have to pass an empty DNSRecord to acquire DNSRecords of a zone. Could you please let me know why does DNSRecords require DNSRecord? or it can be removed?

Hi @5t111111,
This is because the Cloudflare API allows you to filter the results returned: https://api.cloudflare.com/#dns-records-for-a-zone-list-dns-records
When you pass a DNSRecord object, you can populate just some of the fields, e.g. Type to just return records of that type, or Name to only return records of that name.
Does that makes sense? I agree the documentation needs updating to make this clearer.

Ah I see. Fair enougn information, thanks!

FWIW I updated the documentation for this in #115 - hopefully it's clearer now. There are examples too, if you read it via godoc.


Why is hideKeyboardAccessoryBar true by default?

This completely breaks the functionality of dropdown menus on both iOS and Android and makes apps with them pretty much unusable.

This directive fixed it for me:
.directive('form', function () {
      return function () {
        if (window.cordova && window.cordova.plugins.Keyboard && navigator.userAgent.match(/(iPod|iPhone|iPad)/)) {
          cordova.plugins.Keyboard.hideKeyboardAccessoryBar(false);
        }
      };
    }
  );

hideKeyboardAccessoryBar(false) doesnt work in iOS 9 for some reason, it has been completely ignore and keyboard was shown without "<", ">" and "done" button, which make user unable to hide their keyboard after input.

I've accomplished a different workaround on this.
angular.module('app')
  .run(function($rootScope, $ionicPlatform, $ionicScrollDelegate){

      $ionicPlatform.ready(function () {
          if (window.cordova && window.cordova.plugins.Keyboard){
              cordova.plugins.Keyboard.disableScroll(true); // This will prevent the view to bounce when inputs are on focus
          }
      });

      $rootScope.$on('$ionicView.loaded', function () {
          if (window.cordova && window.cordova.plugins.Keyboard) {
              cordova.plugins.Keyboard.hideKeyboardAccessoryBar(false); // This makes the accessory bar visible and it only works when the view is loaded and DOM ready
          }
      });

      window.addEventListener('native.keyboardshow', function () {
          $ionicScrollDelegate.scrollBy(0, 1); //This will return focus to the current input once the keyboard slides-up in the view
      });

};)

@alejandro-heredia-coderoad-com This fix does not work for me. the hideKeyboardAccessoryBar(false) is stil being ignored.

For us, hideKeyboardAccessoryBar(false) works for IOS 9 and IOS 10 for the KEYBOARD.
But for IOS 10, the accessory bar is NOT visible for the SELECT pane: it is not possible to select any value, or to leave the select pane.

We have also this problem. In IOS 10 keyboard accessories did not show.

For iOS10 I am unable to get any inputs to show the keyboard accessory bar.
Any ideas?

I'm using Ionic 2.0.0-rc.1 and this: Keyboard.hideKeyboardAccessoryBar(false); works fine in the iOS simulator when running on 9.3.  However, when running on a device in the simulator with iOS 10, I only see the arrows and done button with NO KEYBOARD.  Appears to be related to the plugin itself.


Make Pagination Functional

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
Hello mentor, I want to make the Newer, 1, 2, ... 15, Older Pagination feature functional! Could you kindly assign the issue to me and label the level? Thank you, mentor!
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

We Didn't created lot content yet cause it's a new one so the pagination is useless in this case because we haven't created more content if you're willing to add both we'll assign to you.

We Didn't created lot content yet cause it's a new one so the pagination is useless in this case because we haven't created more content if you're willing to add both we'll assign to you.

Okay mentor, I would add content and make the pagination functional! If you may assign and label the level of the same. Thank you!

@codewithinfiver Kindly share your contact details.

@codewithinfiver Kindly share your contact details.

Hello mentor, you can contact me at<EMAIL_ADDRESS>Thank you, mentor.

@codewithinfiver Did you start working on the issue?

@codewithinfiver Did you start working on the issue?

Yes mentor, I have forked the repository and started working on the same locally. I will open a pull request shortly. Thank you, mentor!

@codewithinfiver Did you start working on the issue?

Mentor I am also working on the content to be added, if any issues crop up, I will mail you. Could you provide your contact details for the same?

@codewithinfiver Text here +91 79956 65539

@codewithinfiver Text here +91 79956 65539

Thank you, mentor!


uiua: refactor to support multiple versions, uiua-unstable: init at 0.14.0-dev.6

Creates uiua-unstable by moving the current builder to a generic.nix and parameterizing it with a function, also changes the updater script for uiua to only get stable versions.
This PR is closely modeled after https://github.com/NixOS/nixpkgs/pull/331187
Currently the only minor annoyance I see is with having to make a bash script for the uiua-unstable updater when nix-update-script would suffice, but to do that would require changing the current updater so uiua doesn't update alongside the vscode extension or using another callPackage for only the unstable version.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Opened a different PR that uses a different pattern to support multiversion: https://github.com/NixOS/nixpkgs/pull/362951
I personally like that pattern more, since it keeps the main logic in package.nix and only has one entrypoint (package.nix).

I'm closing this in favor of  #362951 since that was just merged. Thanks for getting the ball rolling on this however! Perhaps you wanna add yourself to the maintainers list for uiua?


EMG data

hello can i use specific EMG data to play a sound or present a picture
can you help me

Yes you can, but the specifics are not within the scope of this library.
This more of a  general development question that is more suited for stackoverflow.


keyboard on dialogSheet

When the dialog opened and the virtual keyboard is called, the dialog below the keyboard continues
This is scandal

This issue has already been fixed. It will be available in the next release.

Solved in version 2.0.8.


[core] Mac display rectangle size. bug

I have been playing around the examples, I tried to apply the post-processing sample code on another example I have been working on and realized that the texture that the post-processing effects are rendered on is 1/4th of the actual screen size.
After @JeffM2501 instructions I tried using DrawTexturePro, which gave the exact results, after hacking it a bit and doubling the sizes it produced the expected results.
Before with both methods and normal screen sizes.

After doubling the dest rect sizes in DrawTexturePro

raylib -> 3.5 version
system info -> MacBook Pro (16-inch, 2019), i7, AMD Radeon Pro 5500M 8 GB
code -> https://gist.github.com/geofl/92ae036a79e7a4de5172ffbadf0b6d5c

Please, could you confirm this issue is not a duplicate of #1510 and #1086?


Add swagger ui routing example

One of the most important thing I want when using this plugin is to serve a swagger UI. However this is missing from the README and requires effort to find it.
     route("swagger") {
            swaggerUI("/api.json")
        }

Hi,  thanks for the hint. Example in the readme is fixed now :+1:


metadata config

some minor refactoring and parse [workspace.metadata.cross]
fixes #1158

this pr could maybe also implement #1281


sla的报告通知可以关闭吗

Background & X Problem
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Proposal and Expectation
A clear and concise description of what you want to happen.
Solutions & Y Problems
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Yes, you can set the SLA schedule to "none" using the following configuration:
settings:
  sla:
    schedule: "none"


Unit Test SearchDescriptor

Due to lack of documentation I cannot unit test my SearchDescriptor.
Simple :-)

This should help you for now.

Cool thanks. Woked a treat. Is there anything different planned for the future??
For future ref:
var query = ((ISearchRequest) searchDescriptor).Query.Match.Query;

No :)
Being able to cast to interfaces is the way to go if you want to unit test your descriptors


Added test case for commit ref with percent encoding

Fixes https://github.com/google/go-github/issues/1101
Added test case for Repositories.GetCommitSHA1, when it was called with the branch name which includes the non-alphabet character.

Thanks, @vaibhavsingh97.  First, please don't use force-push for commits on PRs for this repo, as we "Squash and merge" them anyway.
Second, you may be aware of this already, but the Travis CI build is currently failing with the most recent commit, with this error:
--- FAIL: TestRepositoriesService_NonAlphabetCharacter_GetCommitSHA1 (0.00s)
    repos_commits_test.go:240: Repositories.GetCommitSHA1 returned error: GET http://<IP_ADDRESS>:46687/api-v3/repos/o/r/commits/master%2520hash: 404  []
    repos_commits_test.go:245: Repositories.GetCommitSHA1 = , want 01234abcde

@gmlewis Sorry for late update. I had updated my PR, please review

Hmmm... I'm not sure why GitHub is showing us such a mess in the diffs.
Why we are seeing v23 or v24 at all is a mystery to me. Normally, we see diffs against the master branch which already has v24 in it, so these diffs make no sense to me.
I'll have to study this again later tonight and figure out what is going on.

@gmlewis I had fixed this but it's needed a force push, please let me know if I needed to do the same

OK that's odd.  If you can fix this with force pushes, then please go ahead despite what I said earlier.  :smile:

@gmlewis Please review 😄


Better handling of sql session

If an error occurs during a sql transaction, there is no rollback and the database can be locked.
We should use context manager to better handling sql session: commit on success and rollback on error.
Possible implementation: https://stackoverflow.com/a/29805305/7497291

From sqlachemy version 1.4 onwards, it seems that we can avoid a lot of verbosity:
https://docs.sqlalchemy.org/en/14/orm/session_api.html#sqlalchemy.orm.sessionmaker
and
https://docs.sqlalchemy.org/en/14/orm/session_basics.html#session-getting


Performance enhancement of backend connect with database by NIO in sharding-proxy

I have researched currently program in sharding-proxy where implements backend connection with database by NIO, and I have some advisement for performance enhancement.
To show my idea clearly, I draw up design plan.
Legend

Current case

Frontend and backend use different worker eventloop group(this means use different IO thread pool):

Application's request reaches worker eventloop group of frontend;
Then worker eventloop group writes data to user eventloop group, and the later processes protocol package; (Switch thread)
Then user eventloop group writes data to worker eventloop group of backend, and the later executes SQL(this means send protocol package to DB); (Switch thread)
Then query result from DB returned to work eventloop group of backend;
Then work eventloop group of backend writes data to user eventloop group; (Switch thread)
Then user eventloop group processes query result by blocking;
Then user eventloop group writes the processed result to work eventloop group; (Switch thread)
Then work eventloop group writes back data to application.

Total 4 times switch thread in a query's life time.
Case 1

Frontend and backend use same worker eventloop group(this means use same IO thread pool):

Application's request reaches worker eventloop group;
Then worker eventloop group writes data to user eventloop group, and the later processes protocol package; (Switch thread)
Then user eventloop group writes data to worker eventloop group, and the later executes SQL(this means send protocol package to DB); (Switch thread)
Then query result from DB returned to work eventloop group;
Then work eventloop group of backend writes data to user eventloop group; (Switch thread)
Then user eventloop group processes query result by blocking;
Then user eventloop group writes the processed result to work eventloop group; (Switch thread)
Then work eventloop group writes back data to application.

Advantages: Reduce a thread pool.
Maybe only a little performance enhancement, but still 4 times switch thread in a query's life time.
Case 2

Frontend and backend use same worker eventloop group(this means use same IO thread pool), and every eventloop in the group establish connection with DB by channel or channel pool:

Application's request reaches worker eventloop group;
Worker eventloop group processes protocol package and sends package to DB;(Considering processing protocol package in memory is very fast and no-blocking, so do this operation in IO thread to avoid switch thread)
Then query result from DB returned to work eventloop group;
If the query results need to be merged, switch thread and merge them in user eventloop group by blocking. If not, process them in IO thread pool to avoid switch thread;
If process the query results in user eventloop group, then user eventloop group writes data to work eventloop group(Switch thread);
Then work eventloop group writes back data to application.

Advantages:

Reduce a thread pool.
Reduce times of switch thread from 4 to 1 or 2.

Disadvantages:

Every IO thread establishes connection with DB, if the count of IO thread in worker eventloop group is 4, then the count of connection is 4 times as the count of current case.

Could anyone give me some suggestions? Thank you very much!

related with #686

Wonderful, all details are brief and clear

Closed. reason just ref #1893


How to get to 10^6 accounts: A guide for the perplexed

We should figure out a plan for scaling the ledger.
@nholland94 to write up
Potential blockers

right now we bin_io the whole epoch ledger

@nholland94 has determined the work we need to do


Create YARA rules from SaneSecurity's repository

Convert ndb files to yara rules (using .yar extension): http://debian.man.ac.uk/.f/pub/misc/malwaresigs/sanesecurity/
Using https://www.aldeid.com/wiki/Yara

clamav_to_yara.py taken from malware analysts cookbook in .txt
clamav_to_yara.txt

all those .ndb which ran successfully from http://debian.man.ac.uk/.f/pub/misc/malwaresigs/sanesecurity/ found within
likeTakingYaraFromClams.zip

thanks! I will add them tonight


feat: new filenames structure

Codecov Report

Merging #3789 into dev will decrease coverage by 0.4%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##              dev    #3789      +/-   ##
==========================================
- Coverage   97.84%   97.43%   -0.41%
==========================================
  Files          18       18
  Lines        1159     1170      +11
  Branches      312      320       +8
==========================================
+ Hits         1134     1140       +6
- Misses         24       27       +3
- Partials        1        3       +2

Impacted Files
Coverage Δ

lib/builder/webpack/base.js
90.9% <100%> (-7.46%)
:arrow_down:

lib/common/nuxt.config.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22559a8...67cfb93. Read the comment docs.

@clarkdo Is it included in the 2.0 docs yet? :thinking:

@manniL https://github.com/nuxt/docs/commit/1eca53bf7974cd74115485be4ada2f7ab59e2f22

@clarkdo Thank you :relaxed:


[ENHANCEMENT] add_seasonality in prophet model

Is your proposed enhancement related to a problem? Please describe.
In the non-hts version of prophet, we use external variables in both the external_regressor way and in the 'add_seasonality' way, since the performance of using one or the other differs per variable. Currently (as far as I can tell), this option is not yet available here.
Describe the solution you'd like
Ideally we would be able to make the distinction between adding exogenous variables as regressors and as seasonality.
Describe alternatives you've considered
One option would be to add an extra parameter to the hierarchyTree besides exogenous (f.e. named seasonality)
another would be to be able to specificy the distinction in the exogenous dictionary.
f.e. instead of filling exogenous with:
'total': ['general_variable1','general_variable2']
you would also be able to fill it with:
'total': {name:'general_variable1', type: 'external_regressor'}, {name:'general_variable2', type: 'seasonality'}
Additional context
Once we agree on what would be the best solution here, we might also be able to pick up (part of) the implementation depending on the complexity.

Sorry for the late reply, but I'd be happy to support that feature.
'total': {name:'general_variable1', type: 'external_regressor'}, {name:'general_variable2', type: 'seasonality'}`

this pattern seems to fit well the current implementation. Let me know if you are still interested in contributing.

Am I correct in understanding this issue/enhancement is in regards to adding external regressors (e.g. holidays) to the hts model? I want to make sure I don’t open a duplicate issue if so.


Retweet text truncated

Should the text column include the full text of the original retweet instead of the truncated version?

Yes, this can be done. We have the referenced tweets. I was thinking of keeping as close as possible to the json but I think this just creates more problems and confusion. I'll add that along with the other referenced tweets changes

Yes, I definitely understand wanting to keep it like the JSON. But I think people converting the JSON to CSV will exepct to see the full text of the retweet.

I was going to report exactly this, truncated text in the text field i nretweets makes it more difficult to work with them when using CSV. Earlier @congosto mentioned that to me.

I haven't forgotten about this - just didn't get around to it yet. Will do in the next few days though.

Another related issue is with hashtags as lists, but this would be a separate thing: https://twittercommunity.com/t/problems-getting-the-correct-data-structure-and-full-text/159459/2

After a few attempts, i've a new version, but i'm not sure anymore if simply replacing the retweet text is a good idea, as this breaks all the other metadata for the retweet, and replacing the retweet metadata with the retweeted one seems pointless, when including the referenced_tweets inline as before gives you the data you need.. So alternatively we can replace the retweets entirely. Either way, this means treating retweets differently. Quoted tweets and replies should be treated separately too, and definitely should not be replacing tweets. I don't think we should "merge" quoted tweets in the same way at all.
Maybe this is something that should be better documented because it's more of a property of twitter data, not of a CSV representation.
For example:
This is a retweet:
{
  "text": "RT @quocleix: Primer: Searching for Efficient Transformers for Language Modeling\n\nhttps://t.co/RJ7IfdBvKo\n\nWe use evolution to design a new…",
  "entities": {
    "mentions": [
      {
        "start": 3,
        "end": 12,
        "username": "quocleix",
        "id": "989251872107085824"
      }
    ],
    "urls": [
      {
        "start": 82,
        "end": 105,
        "url": "https://t.co/RJ7IfdBvKo",
        "expanded_url": "http://arxiv.org/abs/2109.08668",
        "display_url": "arxiv.org/abs/2109.08668"
      }
    ]
  },
  "author_id": "495430242",
  "id": "1440297242162651139",
  "reply_settings": "everyone",
  "created_at": "2021-09-21T12:49:56.000Z",
  "public_metrics": {
    "retweet_count": 121,
    "reply_count": 0,
    "like_count": 0,
    "quote_count": 0
  },
  "source": "Twitter Web App",
  "referenced_tweets": [
    {
      "type": "retweeted",
      "id": "1440024037896245248"
    }
  ],
  "conversation_id": "1440297242162651139",
  "possibly_sensitive": false,
  "lang": "en"
}

and the corresponding referenced_tweet (the original tweet):
{
  "text": "Primer: Searching for Efficient Transformers for Language Modeling\n\nhttps://t.co/RJ7IfdBvKo\n\nWe use evolution to design a new Transformer variant, called Primer. Primer has a better scaling law, and is 3X to 4X faster for training than Transformer for language modeling. https://t.co/daoHnuQEjS",
  "author_id": "989251872107085824",
  "id": "1440024037896245248",
  "reply_settings": "everyone",
  "attachments": {
    "media_keys": [
      "3_1440021169722167300"
    ]
  },
  "created_at": "2021-09-20T18:44:19.000Z",
  "public_metrics": {
    "retweet_count": 121,
    "reply_count": 2,
    "like_count": 566,
    "quote_count": 22
  },
  "source": "Twitter Web App",
  "conversation_id": "1440024037896245248",
  "possibly_sensitive": false,
  "entities": {
    "urls": [
      {
        "start": 68,
        "end": 91,
        "url": "https://t.co/RJ7IfdBvKo",
        "expanded_url": "http://arxiv.org/abs/2109.08668",
        "display_url": "arxiv.org/abs/2109.08668"
      },
      {
        "start": 271,
        "end": 294,
        "url": "https://t.co/daoHnuQEjS",
        "expanded_url": "https://twitter.com/quocleix/status/1440024037896245248/photo/1",
        "display_url": "pic.twitter.com/daoHnuQEjS"
      }
    ]
  },
  "lang": "en"
}

Note the differences between public_metrics and entities in the retweet and original tweet. The original tweet has 2 URLs, a link and an Image, the retweet has the link but is missing the image, and if we replace the retweet text
"RT @quocleix: Primer: Searching for Efficient Transformers for Language Modeling\n\nhttps://t.co/RJ7IfdBvKo\n\nWe use evolution to design a new…"
with the original
"Primer: Searching for Efficient Transformers for Language Modeling\n\nhttps://t.co/RJ7IfdBvKo\n\nWe use evolution to design a new Transformer variant, called Primer. Primer has a better scaling law, and is 3X to 4X faster for training than Transformer for language modeling. https://t.co/daoHnuQEjS"
Then we are missing the metadata for the https://t.co/daoHnuQEjS link - and the public_metrics (retweets only show retweet_count number ) and the retweet entities no longer match.
In order for things to work, the entire tweet has to be copied over, and only the author and id and created_at changed.
So the choices are:

Do nothing: Keep just the result output, with truncated retweet text, and no referenced tweets in the CSV (this is the worst choice i think, right now it's like the current one with --no-inline-referenced-tweets this is what the main branch is on now)

Inject the original tweet just before the retweet (the previous behavior --inline-referenced-tweets preserves all the data but confuses people - maybe do this only for retweets?)

Replace entire retweet: drop all the retweet metadata and replace it with the original tweet metadata, apart from the author and id.

I'm thinking of 3: replacing entire tweet if it's a retweet, with an extra command line switch --merge-retweets / --no-merge-retweets
I'll post some examples of commands and outputs later to make this more clear.


Ionide does not support paket when installed as a global tool

Install paket as a global tool eg dotnet tool install -g paket and then run any Paket command. Ionide complains it cannot find paket even though it is available straight from the CLI.

Fixed in 2.0


Mnemonic security changes for private network tutorial

The private network tutorial suggested cURL as an option for adding keys to the keystore of a node, but this is potentially insecure because the mnemonic will be stored in the user's bash history.
This addresses #21

I am going to merge this for now so that we can address the immediate concern and get this PR taken care of. Leaving #21 open for additional discussion :+1:


Python3 for 3.11

Python3 compatibility back-ported from 4.0 to 3.11.

I do not think this will ever be merged. It lacks a JIRA ticket too. Feel free to file a ticket and discuss it there if necessary and we might reopen.

The JIRA ticket is CASSANDRA-16403


Vickel: Watch fologrow(?!.com)

Vickel requests the watch of the watch_keyword fologrow(?!\.com). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

fologrow(?!\.com) has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Columns flexget status are not described

When running
$  flexget status

It is not clear what Produced in the table means, the task was generated correctly but not accepted?
It would be ok a description for every column.

Entries can remain Undecided.
Produced is the total number of entries in the task, including Accepted, Rejected and Undecided which doesn't need its own column since you can just subtract the other two to find its value.

I guess it might be more descriptive if "produced" was renamed to "entries". Longer texts are not really feasible.


Add “.nvmrc” file for Node version management

Node Version Manager (nvm) simplifies Node.js version management. This pull request adds a .nvmrc file. Used version 0.10 based on Node.js version specified in .travis.yml.

Closing pull request after two years without activity.


STORM-2832: misuse of Utils.readStormConfig() and ConfigUtils.readSto…

…rmConfig()

@Ethanlm could you have a have at the changes, not sure if it fully covres

@revans2 ty :)


esy scriptName

What do people think about the following feature:
esy scriptName

Could call the script named scriptName, stored in the "scripts" section of package.json. This would be similar to npm run scriptName, but it would construct an environment from esy (as if you were building that package using script name scriptName).
Yarn also has a similar feature - the only difference being that we would want to construct the environment according to esy's approach.
The way I think about "scripts", in this case, is that they are like named binaries placed in the path of esy commands executed in that sandbox. I imagine if for every script name, there was an executable by that same name created implicitly in the directory that you execute esy commands.

I think that the current esy CMD behaviour is a better building block for more complex workflows. It allows ad-hoc command execution easily while "named binaries" can be implemented by just placing a script which calls into esy with a given command:
#!/bin/bash
exec esy some-executable

That said I'm totally biased against npm scripts approach as I think managing code inside a JSON file is just plain wrong. I think current esy CMD is superior.

Okay, that's a good point, I'll try out your suggested approach for a while.

Sounds good.


Custom shields documentation

Hello, thanks for your amazing work, keep it up!
I think is very important to make custom shield generator more noticeable in shields.io.
INSTALL.md points custom shields to img.shields.io which redirect to shields.io.
Also in README.md there is no specification to generate your own shield.

Thanks for your suggestion!


feature/sqrt

Codecov Report
Base: 100.00% // Head: 100.00% // No change to project coverage :thumbsup:

Coverage data is based on head (bb3f1d1) compared to base (a2777d6).
Patch coverage: 100.00% of modified lines in pull request are covered.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff            @@
##            master       #28   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           22        22
  Lines          522       557   +35
  Branches        95       103    +8
=========================================
+ Hits           522       557   +35

Impacted Files
Coverage Δ

contracts/libs/math/DSMath.sol
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Please update both versions to 2.0.3


Stream of debug messages breaks alternative UI

The new constant stream of debug messages to the temrinal that launched Moony is interfering with the alternative UI feature.

You have checked-out at an unlucky commit, e.g. the synthpod UI writing out loads of debug messages and thus slowing down everything else.
Choose a more recent commit, e.g. 307af11cc7df6c9284b0c77cb1669e6ec542a3ef


Get message for all errors and exceptions

Dart core library: dart:core
Dart VM version: 2.8.4 (stable) (Wed Jun 3 12:26:04 2020 +0200) on "windows_x64"

Why doesn't Dart SDK allow to get value of message field from Exception class? The Error class even doesn't have message field.
Currently, we must specify child Exception and Error to get the message.
try {
   //...
} on AssertionError catch(e) {
  e.message; //OK
}

How can I get the message for all errors and exceptions?
try {
   //...
} catch(e) {
  e.message; // There is no getter for message field??? Error class even doesn't have message field???
}

I know that there is e.toString(), but it usually contains other information that we don't want sometime.

e is caught as an Object, apparently, which does not have a message field. I think Effective Dart has some good advice and explanations on the subject. The Error class is a parent type to many errors, but it also doesn't have message. toString is as good as you're going to get on an unknown error type. Why do you sometimes not want the information provided there?

Neither the Error class nor the Exception class have a message field.
Not all errors and exceptions have messages to expose, because they don't need them.
Exceptions are intended to be caught and handled programmatically, not printed and ignored. They represent the message by their fields, which may contain actual data. That can be anything. There is not even a promise that toString does anything useful.
The exception class constructor is not recommended for actual code. It's available so you can wrap up a quick exception object during development, but you should not be using it in your production code. It's a useless exception because it doesn't tell you what went wrong. A real exception object would contain domain specific data about the exceptional case.
Errors are intended to be printed, but not "handled". There is nothing to handle, your program might be in a bad state afterwards, and the safest thing you can do is to crash the entire thing. The toString is the intended API. Some errors have a message field because they can carry a text to put into the toString, but for example the RangeError doesn't really need a message field. It inherits it one ArgumentError, but it will only contain a small part of the entire toString message because the remaining is also contained in the min, max and value fields.
If you are catching anything, then also remember that Dart can throw any (non-null) object. It may not be just Errors or classes implementing Exception. The only API you can trust is toString.

Thank you guys for your useful reference and nice explanation!
I totally agree with your comments.
So my next question is, how could I know if I am missing any exception and/or error?
How can the IDE, Dart Analysis or any tool somehow tell/warn me?

Dart does not include exceptions in the type signatures, so you have to read the method documentation (or possibly class or library documentation, if they say that "all foo methods throw FooException if foo happens").
It's technically possible to scan function DartDoc for paragraps of the form:
/// Throws (a) [ExceptionType] if/when ....

and warn you if a call to the function doesn't catch ExceptionType. We don't currently do that.
(That's the format that we intend to use for documenting exceptions in the SDK code, but a lot of code was written before we came up with that.)

It's technically possible to scan function DartDoc for paragraps of the form:
/// Throws (a) [ExceptionType] if/when ....

and warn you if a call to the function doesn't catch ExceptionType.

If it happens, I'm not sure if it could be a performance problem, because exceptions are usually in nested DartDoc and could be at a very deep level of the chain.

We don't currently do that.

Is that feature in the plan of Dart team?

No current plan that I'm aware of. The structured way to write DartDoc was never really embraced, so we haven't done anything with it.


Fix missing array declaration in a unique_ptr

Which was causing a wrong deletion of the array

And by the way, thanks for your library ! :)


Volume 1 Security Considerations gather the AuditEvent profiles

For ease of finding the AuditEvent profiles, the Volume 1 Security Considerations section could itemize each of the AuditEvent profiles and point to their transaction specific security considerations definition.

I don't understand the recommendation. Can you provide an example ?


Intermittent failures to create K8s resources by Sparkoperator due to suddenly missing K8s API permissions

Hi,
we have been running Spark applications via the Spark operator for a while now. We recently updated from K82 v1.25 to v1.27 (AWS EKS, to be precise) and on one of our prod systems (the one making the most use of Spark applications) we are now seeing intermittent problems of the Spark operator to create K8s resources for a new SparkApplication CR, e.g.
failed to create UI service for SparkApplication myns/xyzabc-mysparkapp-k93nb0: services is forbidden: User "system:serviceaccount:myns-spark:myns-spark-spark-operator" cannot create resource "services" in API group "" in the namespace "myns"

or
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://<IP_ADDRESS>/api/v1/namespaces/myns/pods. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods is forbidden: User "system:serviceaccount:myns-spark:myns-spark-spark-operator" cannot create resource "pods" in API group "" in the namespace "myns".

Attached, find a (sanitized) Spark operator log snippet with the full logs during such a situation
This usually happens for all K8s resource creation attempts during a short time interval and then it starts working again.
Currently, our best guess that this might be related to the change in service account token lifetime, which originally was unlimited, but starting with K8s 1.21 (?) this was changed to one hour. (but on EKS clusters, the life time of these tokens has been (and probably still is, there is no clear info on that) extended to 90 days to allow a smoother transition).
We guess that the tokens expire now more rapidly (or now even hourly), are eventually refreshed by the Spark operator's K8s client, but with some delay, during which requests fail.
We currently still use 1.3.3, I was looking for changes that might fix this in the newer versions but couldn't find any release notes (or any new entries in this project's releases for the operator, only for the Helm chart).
Any hints appreciated.
J
Log_Sanitized.txt

We actually made some progress on the root cause of this behavior. We deploy our app and also the Spark Operator itself with ArgoCD. ArgoCD maps the Helm hooks that are used for Sparkoperator's service account, RBAC and the webhook init job to its own hooks. In particular
"helm.sh/hook-delete-policy": (hook-failed, ) before-hook-creation
is mapped to ArgoCD's
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation

and
 "helm.sh/hook": pre-install, pre-upgrade

is mapped to
argocd.argoproj.io/hook: PreSync

This appears to result in every ArgoCD sync (and this also seems to include syncs that are effectively a no-op) ArgoCD will first delete all these "hook-created" resources and then to recreates them.
So there is a short time where the serviceaccount, clusterrole or CRB might not be there during a sync, which would explain the behavior.
I assume the reason for having the spark operator's RBAC resources (SA, CR, CRB) created by the hook is to be able to use them also from the webhook-init-job (which IIRC mainly creates a bunch of secrets with keys and CA certs which are then used for TLS in the sparkl operator's webhook endpoint and which are also passed to K8s with the creation of the mutatingadmission webhook resource to make K8s trust the webhook endpoint) and later the same SA+CR+CRB are reused for "runtime use" by the spark operator.
Since Argo does not seem to offer a way to run the hooks only on "non-no-op" syncs, I guess the proper solution to this problem would be to have dedicated SA+CR+CRB for the webhook init job (which would probably require fewer permissions) and "runtime" SA+CR+CRB for the sparkoperator which would be created as regular resources, and thus would not be victim of the ArgoCD hook-delete-policy.
Thoughts?

any progress on this?
we have the same problem when we moved spark operator to be managed by argo

I'm hitting this with Rancher Fleet as well. I have to do some combination of forcing another helm no-op run and then manually deleting the operator pod to restart it (the logs show TLS certificate errors until I do) and then the webhook starts working again.

I also met this issue with spark-operator + argoCD, I have to restart the spark-operator pod manually

Disabling serviceAccount and RBAC creation in the argoCD app solves this issue:
kind: Application
metadata:
  name: spark-operator
  namespace: argocd
spec:
  project: default
  source:
    chart: spark-operator
    repoURL: https://kubeflow.github.io/spark-operator
    targetRevision: 1.3.2
    helm:
      releaseName: spark-operator
      valuesObject:
        webhook:
          enable: true
        sparkJobNamespaces: {{ .Values.spec.sparkJobNamespaces }}
        # FIXME: Creating rbac and sa in argocd prevent correct startup of spark-operator
        serviceAccounts:
          spark:
            create: false
            name: spark
          sparkoperator:
            create: false
            name: spark-operator
        rbac:
          createClusterRole: false
  destination:
    server: {{ .Values.spec.destination.server }}
    namespace: spark-operator
  syncPolicy:
    syncOptions:
      - CreateNamespace=true


Group name is not set after reordering a tab

Describe the bug
Hello,
When you have a tab in a group and you reorder it, the group is not set after drag off.
To Reproduce
Steps to reproduce the behavior:

Create a group
Go to the page of the matched group
Reorder your tab
See error. You need to refresh the tab to see the group

Expected behavior
When I reorder the tab, I need to keep the group
Extension version: 1.0.5

Related PR : https://github.com/furybee/chrome-tab-modifier/pull/313


There seems to be a new bug in version 2.2.1 where the copy and paste button doesn't work and I'm back to the old version

There seems to be a new bug in version 2.2.1 where the copy and paste button doesn't work and I'm back to the old version！

There seems to be a new bug in version 2.2.1 where the copy and paste button doesn't work and I'm back to the old version！

Please check the copy and paste functionality in the latest version.

2.2.1 版中似乎有一个新错误，复制和粘贴按钮不起作用，我又回到了旧版本！

请检查最新版本中的复制和粘贴功能。

The latest version of the copy and paste function is normal, but the search tag function is missing. I think you can add a search button on the right side so that you can search whether there is a specified tag

The latest version of the copy and paste function is normal, but the search tag function is missing. I think you can add a search button on the right side so that you can search whether there is a specified tag

There is a tag search, you just need to start typing text in the list of all tags. You can also use the filtering of the list of all tags. Or do you mean something else?


Marqo Vector Store Examples & Type Hints

This PR improves the example notebook for the Marqo vectorstore implementation by adding a new RetrievalQAWithSourcesChain example. The embedding parameter in from_documents has its type updated to Union[Embeddings, None] and a default parameter of None because this is ignored in Marqo.
This PR also upgrades the Marqo version to 0.11.0 to remove the device parameter after a breaking change to the API.
Related to #7068 @tomhamer @hwchase17

thanks @OwenPendrighElliott!


Different input feature lengths

❓ Questions & Help

hi, i'm trying to create a model that takes in 2 different types of features, both of which have different feature lengths. additionally, it's a bipartite graph, so the neighbors of one would be of the other feature type. effectively, i'd like to learn an embedding (ie. put a dense layer ontop of each feature) and propagate the messages through something already pre-defined like GCN. any suggestion on how to do this?

perhaps the above image would help. features of nodes in U (blue nodes) have a length of [1, 933], while in V (green nodes) they have a length of [1,950]. my original idea would be to pass the features first through their own dense layers to get a transformed output of maybe [1, 512], stack these together, recompute their edge_index (as they now are in one matrix), then pass them through like a normal GCN

Your solution sounds good to me. If you only want to exchange messages in one direction, you can encode your bipartite graph as an assignment matrix in edge_index and make use of directional message passing, e.g.:
edge_index = ... # Encodes an assignment matrix of dense shape [N_blue, N_green]
x_blue = torch.randn(N_blue, 933)
x_green = torch.randn(N_green, 950)
x_blue = self.lin1(x_blue) # Map node features to same dimensionality
x_green = self.lin2(x_green)
conv1 = SAGEConv(512, 512, normalize=False)
out = conv1((x_blue, x_green), edge_index, size=(N_blue, N_green))
# out now has shape [N_green, 512]

Ah, that's interesting. I was wondering how to get all the data points working with the DataLoader class, as I thought the edge index should be given by [2, num_edges] in shape. Additionally, I have 2 separate data frames for x here (x_blue and x_green). When you talk about directional message passing, does that mean my edges can't be undirected edges?

They can be undirected, but overall, PyG only knows about directed graphs. That's why for undirected edges, we need to hold both directions in edge_index.

I see. Would the to_undirected function help with that then? I guess it takes an edge_index and converts it to twice the length such that it looks like there's 2 way?

Yes, the to_undirected method is meant for this :)

I see alright. So i guess when I construct the Data object, I'd have something like x= [N_blue, feat_len1], edge_index=[N_blue, N_green], x2=[N_green, feat_len2] ?

Well, no :) your edge_index is still [2, num_edges], it just encodes dense matrix of shape [N_blue, N_green]. You either need to convert it to shape [N_blue + N_green, N_blue + N_green] as you suggested, or need to pass messages twice, from blue to green and from green to blue.

Ah okay, I sort of get it. Basically, we can create an edge index of size [2, N_blue+N_green], and use that to index the X matrix. However, I'm a bit confused on how to stack the X_blue and X_green together, as the node feature length is different.

You need to transform them to the same node feature length via separate MLPs.

Yup, I understand that part. But will the Data object accept that? From my understanding, Dataset class constructs a data set whereby we pass in a list of Data objects. These objects should have things like the node features and the edge_index. These Data objects are also what we can use to create mini-batches. Hence, I was wondering how to batch X_blue and X_green such that during the training phase, I can map them via separate MLPs, stack them back, then pass into the GNN.

They get concatenated independently, hence, there is no mismatch in shapes. So if you initialize your data as Data(x_blue=..., x_green, edge_index=...) everything should work as expected. However, you need to ensure that edge_index gets increased as expected in a mini-batch scenario. You want edge_index[0] to be increased by the number of nodes in x_blue, and edge_index[1] to be increased by the number of nodes in x_green. This is best achieved by overriding the __inc__ method in the Data class:
class MyBipartiteData(Data):
    def __inc__(self, key, value):
        if key == 'edge_index`
            return torch.tensor([self.x_blue.size(0), self.x_green.size(0)])
        return 0

Hmm ok! This sounds like a good idea. Thanks for all the follow-ups! I'll reach out if I need more help.


feat hidden annotations

It includes

isVisible property on annotation (Implementation based on locking feature)

Shouldn't we use isVisible like we did in Cornerstone Legacy, rather than the inverse "hidden"/"isHidden"? As such the module would then be the visibility module. Annotations should have a visible option that is added as true by default?
As for the CS3D perspective:

On the one hand, we use visible for labelmaps, so should use the same nomencleture for annotations?
On the other hand, hidden behaves more like locked, so its a good flag to have?

I haven't done a full review for this reason, as I would be suggesting e.g. ANNOTATION_HIDE_CHANGE => ANNOTATION_VISIBILITY_CHANGE etc. I can do a full review once we have agreement on the direction.
Otherwise I agree with Alireza's points.

@sedghi  what do you think.
It totally agree with James and his perspectives capture exactly my questions. I followed approach 2, hidden act as locking, as it allows incrementally adding this feature with none (or last say few) regression possibility.

Shouldn't we use isVisible like we did in Cornerstone Legacy, rather than the inverse "hidden"/"isHidden"? As such the module would then be the visibility module. Annotations should have a visible option that is added as true by default?
As for the CS3D perspective:

On the one hand, we use visible for labelmaps, so should use the same nomencleture for annotations?
On the other hand, hidden behaves more like locked, so its a good flag to have?

I haven't done a full review for this reason, as I would be suggesting e.g. ANNOTATION_HIDE_CHANGE => ANNOTATION_VISIBILITY_CHANGE etc. I can do a full review once we have agreement on the direction.
Otherwise I agree with Alireza's points.

@sedghi what do you think? It totally agree with James and his perspectives capture exactly my questions. I followed approach 2, hidden act as locking, as it allows incrementally adding this feature with none (or lets say few) regression possibilities.

I think visible default to true makes a lot of sense to me (as per visibility controller for segmentation too)

@sedghi @JamesAPetts  I did the changes proposed, can you take a look again?

Branch name is not accurate. closing it


2x PortMaster entries

Hi
In ArkOS Emulation Station menu I had one entry in the Options menu with the name "PortMaster".
Then I auto-updated PortMaster in ArkOS, and now the Options menu has 2x "PortMaster" entries.
PORTMASTER
_ CONFIG
_ _ PORTMASTER
PORTMASTER
Christian Haitian from ArkOS says this is the correct place to report this.
Thanks

I followed your instructions:
I install your PortMaster, I restart Emulation Station, and it seems to be fixed -> Only one PortMaster menu entry in Emulation Station. Good!
Then,
I start the PortMaster, it detects a new version, downloads, I restart Emulation Station, and the double PortMaster menu returns.
Not good.
Sorry for my basic English.

Hi @lauren7ino it turns out there was a bug in the Install.PortMaster.sh script generation. I have just released a new one which if you run will fix your issues.
https://github.com/PortsMaster/PortMaster-GUI/releases/download/2024.03.20-1013/Install.PortMaster.sh
Sorry for any inconvenience 😄

Hi! It works.
This version doesn't detect new updates, so let's see if it's still fixed next time it updates PortMaster.
It dumps a lot of (debugging?) messages when changing themes/colors. Otherwise seems fine.
Thanks

Yeah it will be fine next time it updates. 🤞
One day i will clean up the debugging info, but for now it's not hurting anyone. 😄
Glad this could be sorted!


For discussion: MSR's CHERI+MTE composition

As mentioned in the meeting yesterday, Microsoft has spent some time considering the composition of CHERI and MTE, specifically as part of our work on CHERI heap temporal safety.  Our design is geared towards supporting Cornucopia Reloaded's sweeping revocation while offering, among other things,

decreased rate of quarantine buildup (inversely proportional to the number of MTE tag values),
stronger security (closing the UAF/UAR distinction, as we did in CHERIoT), and
lower software complexity (permitting safe in-band allocator metadata, also as on CHERIoT).

The key bits of this have been presented publicly before, but it'd be good to have it all here, too.  We consider these proposed architectural semantics to be Capability Essential IP.
Please find attached some slideware (pptx with animations and extensive slide notes, or pdf without animations) with most of the details and (hopefully pretty) pictures.  But, in quick summary:

Memory metadata tags expand to fit both CHERI and MTE tags.  Any positive natural number of MTE tag bits provides all the security benefits; additional MTE tag bits "just" improve software performance.

In-capability MTE tags are protected, dedicated fields, not merely part of the address field.  New accessor instructions are added. [0]

Capabilities are differentiated into "polychromatic" (or "rainbow") and "monochromatic" by reserving one MTE tag value for the former state (and all the rest the latter).

Polychromatic capabilities can be attenuated to monochromatic progeny with the setter instruction.
Monochromatic capabilities' tags are fixed; attempting modification clears the tag.
Memory capability roots are polychromatic.

Memory metadata tags may be manipulated only with polychromatic authority.  There are new accessor instructions to manipulate memory metadata tags. An exciting atomic instruction on memory metadata tags is also included to allow lockless fast paths in allocators like snmalloc to detect attempts at double-free.

The rule for dereference is "the authority is polychromatic or the authority's and memory's tags match".

As a consequence, "polychromatic" memory is accessible only to polychromatic authorities, making it perfect for in-band allocator state.

Mismatching loads precisely trap.  This is not a significant microarchitectural burden: the load instruction cannot retire without its data, and the memory metadata tags can come along for the ride.

Mismatching stores "fizzle" rather than trap: the store instruction may commit prior to knowing the memory metadata tag value, and mismatch results in the store being silently dropped without altering memory.

The rule for capability revocation is that any capability whose MTE tag mismatches the memory metadata tag of its base is subject to tag-clearing at any point.

[0] MTE, and TBI-like schemes in general, might strongly differentiate "the address field" from "the bottom half of the capability".  See https://github.com/CTSRD-CHERI/cheri-specification/wiki/Tracking-discussion-for-CGetAddr-vs-CToPtr-vs-as-integer-alias-vs-CGetLow for details.  It might be nice if the base Zcheri specification took this possible division into account and reserved the relevant opcodes, even if most CHERI capability encodings will have identical behavior for some pairs.  It's also worth pointing people at CHERIoT-sail: v2? Permuted capability encoding optimized for 33-bit busses for another reason we might want to support prying these things apart in the base spec.

Mismatching stores "fizzle" rather than trap: the store instruction may commit prior to knowing the memory metadata tag value, and mismatch results in the store being silently dropped without altering memory.

I think that we proposed this as an optional extension.  Ideally, a core would support both stores-fizzle and stores-trap mode and would expose a counter of the number of fizzled stores.  Running in stores-trap mode would come with a noticeable performance penalty (possibly as high as 20%), so would typically not be the default, but if a store has fizzled then it means that you have a store-after-free bug and probably want to fix it at some point.  Typically, you'd want to use stores-trap mode during debugging.  Systems with good telemetry infrastructure would want to run stores-trap as a sampling mode after detecting some threshold number of fizzled stores (could be one).  If an app has store-after-free bugs, the OS would turn on the trapping mode, log stack traces (but still fizzle the stores by skipping the instruction prior to mret) to telemetry, and then resume.


[doc] Reference util/container/Dockerfile in the OT starter guide

Dockerfile is great for creating a reproducible development environment. It is especially useful for developers who use a non-standard linux distribution and newcomers.
At the moment there are no references to this container in:
https://docs.opentitan.org/doc/ug/install_instructions/
As a consequence it is probably not used as much as it would be otherwise.

@mundaym @luismarques  what do you guys think?

@mundaym @luismarques  what do you guys think?

The current state of affairs is intentional, as we don't support (or test) the docker image. That can be changed, of course, however it will require a bit more effort.

The current state of affairs is intentional, as we don't support (or test) the docker image. That can be changed, of course, however it will require a bit more effort.

The current state of affairs is intentional, as we don't support (or test) the docker image. That can be changed, of course, however it will require a bit more effort.

Makes sense

The current state of affairs is intentional, as we don't support (or test) the docker image. That can be changed, of course, however it will require a bit more effort.

Makes sense


Suspicious commit in master and release/v4.0.1

fb220795ed1c1628c6ea5becee3423cc268a495c, in both master and release/v4.0.1, doesn't look good. I think it reverts much more that it supposed to do.
@serban-nicusor-catena @rok-cesnovar @bbbales2

What particulary do you have in mind in terms of "reverts much more"?  Can you give an example?

What particulary do you have in mind in terms of "reverts much more"? Can you give an example?

For example, do you really want to revert lib/boost update from 1.72.0 to 1.75.0 in v4.0.1?
Just wanted to bring it to your attention, but it's ok if all is intended.

At least, it should be revered with merging the patches in master into develop.

At least, it should be reverted with merging the patches in master into develop.

At least, it should be reverted with merging the patches from master into develop.

update from 1.72.0 to 1.75.0 in v4.0.1

No we do not want that. We accidentally merged develop to master and then reverted. Which is why all this mess. Develop has 1.75, master has 1.72.
Looking at https://github.com/stan-dev/math/tree/master/lib its 1.72.0 which is correct.
https://github.com/stan-dev/math/tree/release/v4.0.1/lib has 1.72.0 which is also correct
and develop https://github.com/stan-dev/math/tree/develop/lib has 1.75.0 which is also correct.
Thanks for looking into this, these things get tricky so more eyes is good.
We will merge the changes in master to develop as soon as we triple check everything is fine.

@rok-cesnovar Sounds good. Closing this issue!


metamorph: use and combination of reset, flushWith and sameEntity attributes

since some time I'm struggling with the appropriate use of the reset, flushWith and sameEntity attributes on the various collector artefacts but also on the entity tag of Metamorph
Mostly after some try and error definitions I got a result which was the one I expected but I had difficulties to explain the clear differences between the examples
Hopefully I can now provide a distinguished example which might be helpful to discuss the mechanisms behind the implementation
This directory contains the examples.
flushWith and reset=true is used in the MorphScript_case_1 whereas sameEntity=true and reset=false  delivers the correct result in this Morph script_case_2
the output files are for case 1 and case 2
it seems combine uses the sameEntity=true attribute for collecting the various values of an Entity in combination with an If Statement (and no inside collectors) while concat needs the flushWith Attribute in combination with IF and combine with reset=true attribute as the outer collector
During our get Together meeting in HH I already mentioned my troubles with this and a helpful voice in the audience (I couldn't see as Skype participant... (edit: me,@dr0i)) was optimistic for being able to give more background infomation
By the way these examples are part of my attempt to transform our XSLT based transformations for Solr SearchDocuments into a  Metafacture/Metamorph implementations. My main motivation for this is the question: Where are the limits and differences of Metamorph compared to traditional XSLT procedures. If you are interested how it looks like as XSLT you can find it here   Hints are welcome if you know more elegant possibilities to express match ranges matches(@code, '[b-su-z8]') in XSLT compared to the way I have done this

Thanks for any replies!!
Günter

Hints are welcome if you know more elegant possibilities to express match ranges matches(@code, '[b-su-z8]') in XSLT compared to the way I have

Not exactly sure but if you meant to abbreviate  this:
<data source="811??.b|811??.c|811??.d" />

you can use character classes:
  <data source="811??.[bcd]" />

Re: case1 vs. case2:
A bit complex, this. I admit to not totally forsee the result of a complex  combination of attributes and their parameters myself! So at that point I just modify&run&test until the result is satisfactory.
I don't understand what the problem is:
Since the result of the two cases are also very different (case1: every matched field of all entities are entity-indistinguishable put into a set, while case2 puts one concatenated string per entity into the set) you evidently use two different approaches.
Nevertheless here some remarks:
In case1 you use <data source="950??.e"/> in the concat section while flushing with (in principle) every 950-subfield (I think). As 950??.e happens to be the last entry all 950-entities and must be concatenated before the concatenated string is flushed ("saturation" - I think this is so, not exactly sure) the "flushing" coincides with what would be better declared as ... flushWith="950??.e" > (because of the clearness of it).
It's sometimes hard to explain and understand complex data transformations, I know. I would find it easier if you would have a question like "I have thos data and want this outcome - how to do it?". But I gather you want more of the deeper understanding which I find hard for myself.
Maybe you haven't read http://b3e.net/metamorph-book/latest/datamodel.html yet? I haven't yet - seems to be interesting.

Closing this. Note that some of the problems described here may be solved by newer documentation at https://github.com/metafacture/metafacture-core/wiki/Metamorph-Cookbook.


[#27] Add support for building universal lua-https modules.

What

Add support for building universal lua-https modules on macOS (see #27)

How

Since leafo's Github actions does not build universal LuaJIT modules, we have to build LuaJIT ourselves like done on Windows.

Attached is a sample LÖVE app using the produced artifact to fetch the LÖVE website and print the response to the screen. (The app extension is named weird because GitHub won't allow uploading a .love file but obviously will run fine in LÖVE for macOS).
https.love.zip
Result:

Awesome, thanks for doing this!

The library extensions just got my attention.

LuaJIT is built as libluajit.so which is incorrect (correct extension for shared libraries on macOS is dylib), so it's later corrected by renaming it to libluajit.dylib.
lua-https is built as https.so on macOS. This seems correct, given that this is a "bundle" (a dynamically loaded module)  rather than shared library.

LuaJIT happily loads https.so on macOS if it is in cpath (https://github.com/luajit/luajit/issues/76). One possible upside of having it named https.dylib is that this name wouldn't collide with Linux library, allowing to have https.dll, https.so and https.dylib to be stored in the same directory, so LuaJIT would load the one that matches the platform that it was built for.
Can anyone comment on this?

What are the actual differences between the MH_DYLIB and MH_BUNDLE in the context of Lua? As I understand from the SO answer linked in my previous message, it's a common convention to use .dylib and .so extensions respectively for these file types.
I can see that you are building https://github.com/erinmaus/devi as a dynamic library rather than a bundle for macOS. Is there a specific reason for this? Should the CMake target be switched to SHARED for lua-https as well?

One possible upside of having it named https.dylib is that this name wouldn't collide with Linux library, allowing to have https.dll, https.so and https.dylib to be stored in the same directory, so LuaJIT would load the one that matches the platform that it was built for.

Can anyone comment on this?

Unfortunately stock Lua made a decision to only look for the .so extension for dynamic libraries on macOS in its C module loader used for require, even though the proper extension should be .dylib. LuaJIT is just following the standard set by stock Lua.
LÖVE adds its own C module loader that goes through love.filesystem, and that looks for .so for compatibility, as well as .dylib for correctness. Using .dylib for this library would still stop it from working outside of LÖVE with the default require setup though, so I'd like to avoid that.
All that being said, within a single Linux or Windows platform you can have multiple dll/.so files for different architectures (x86 versus x64 etc), so putting everything in one folder and just relying on extensions wouldn't work with that regardless.

I think that Lua devs are correct in using .so extension on macOS. I read a bit about the difference between shared libraries and modules, and it seems that lua-https should be built as a module (and this is how it's done now).
Shared libraries can be linked against and then loaded at runtime. Modules can not be linked against, only loaded at runtime. So in the context of Lua, building native libraries as a modules makes more sense, they are not linked against and loaded dynamically (however nothing stops one from loading a shared library at runtime without linking against it).
In the context of macOS, shared libraries have the .dylib extension, and modules (or "bundles" in macOS terminlogy) can have any extension. Apple recommends using .bundle, but most ported software is using .so (and this is also what CMake is using by default, overridable by setting CMAKE_SHARED_MODULE_SUFFIX). If I interpret the term "any" correctly, then it can have the .dylib extension as well (other platforms are using .so for both, and it doesn't seem to be a source of confusion).
To conclude, on macOS:

libluajit.so is incorrect as it's a shared library; libluajit.dylib is correct.
https.so is correct as it's a module. If set(CMAKE_SHARED_MODULE_SUFFIX ".dylib") is added to CMakeLists.txt, then https.dylib would be built, and it would've been correct as well ("any"), but it wouldn't work with current Lua interpreters without modifying cpath.
If we change the library type in CMakeLists.txt from MODULE to SHARED, then libhttps.dylib would be built (unless CMAKE_SHARED_LIBRARY_PREFIX and CMAKE_SHARED_LIBRARY_SUFFIX are set to something else), and it would also be correct, but as this library is not supposed to be linked against, this is unnecessary (except for iOS case maybe?)

As always, if you feel like I made a mistake, feel free to correct me.


neil did some reverting stuff

tictactoe environment added with tests

Disregard the travis CI stuff, I was just messing around with it.


Remove NuGet.targets from .nuget

It is not required anymore since newer NuGet versions recommend Automatic Package Restore instead of using the .targets file.
This should make Travis CI green.

Perfect, merged !!

It succeed, with the changes in #59 and #107 the lib is working great on Mono
@akoeplinger : about NuGet deploy, I must use portable libraries or MonoDevelop direct consume the 4.0 version or 4.5 of the assembly
Thanks again !!

Sorry, I don't understand what you're asking about NuGet deploy, but a recent MonoDevelop works fine here.

I was asking if Mono can consume NuGet packages targeting .net 4.0 or 4.5 or is better to use a special folder inside NuGet with the binaries compiled with mono itself.
But I realized that works with the first scenario
Sorry

Yes, it can consume both 4.0 and 4.5 DLLs :)


feat/chore: send a new event to try to trigger a new failover_path to be provisioned in certain cases

This is a feat/chore optimization to try to anticipate failover_path to be provisioned, but only it's safe to do so, the EVC try_setup_failover_path(), no changes in this method, but in certain cases such as:

Gets created
Gets updated (and needs to be redeployed)
Gets redeployed

There's opportunity to try to trigger try_setup_failover_path(), and then if there hasn't been any self.affected_by_link_at recent updates it'll take the advantage to try to setup sooner than expecting the next periodic execution.
cc'ing @italovalcy (I went ahead and mapped this in a new issue just so we can refer to it in the future when the time comes)

Subscription to "kytos/mef_eline.(redeployed_link_(up|down)|deployed)" has been added again:
https://github.com/kytos-ng/mef_eline/blob/master/main.py#L931-L941
so, it's already trying to call evc.try_setup_failover_path() to trigger a new failover computation if there hasn't been any link change. I'll go ahead and close this issue. But, let's keep it in mind if we change again the event subscriptions https://github.com/kytos-ng/mef_eline/issues/445


Why the selectpicker's display shows none?

Hi,
I following the getting start including CDN and download locally with the very simple code as example. However, nothing shows there!
When I dived into the code, I found the attribute to "selectpicker" class was "Display: none"? I am wondering what wrong with it or there is any misunderstanding? Would you kindly advise?

It sounds like you're only loading the CSS and not including bootstrap-select.js.

CDN seems method seems not working. Then I install to locally and still has error in javascript.
bootstrap-select.js:350 Uncaught TypeError: Cannot read property 'valHooks' of undefined
at bootstrap-select.js:350
at bootstrap-select.js:3152
at bootstrap-select.js:21
at bootstrap-select.js:23

Are you loading jQuery before Bootstrap and bootstrap-select?

I am having the same issue, console shows no errors. using latest version of bootstrap-select with bootstrap 4. All my bootstrap dependencies are loaded as they are used across our whole site, however when using bootstrap select all my selects end up being hidden

When loading the select (whether inside a modal or on the main page), insert this call after the HTML:
$('.selectpicker').selectpicker('render');
This way it will render the selects again.