Unable to connect to GCS at <IP_ADDRESS>:9008. Check that (1) Ray GCS with matching version started successfully at the specified address, and (2) there is no firewall setting preventing access.

Issue Type
Bug
Source
binary
Secretflow Version
0.8.0b1
OS Platform and Distribution
linux
Python version
3.8
Bazel version
No response
GCC/Compiler version
No response
What happend and What you expected to happen.
测试环境（使用的生产模式）  直接在服务安装anaconda 创建虚拟环境 安装secretflow
在两台服务器同事运行一样的代码   其中一台 报错如下：
运行代码：
#!/usr/bin/env python
# coding: utf-8

import secretflow as sf
from secretflow.device.driver import wait, reveal
from secretflow.data import FedNdarray, PartitionWay
import pandas as pd

# SPU settings
cluster_config ={
    'parties': {
        'np1': {
            # replace with alice's real address.
            'address': '<IP_ADDRESS>:9009',
            'listen_addr': '<IP_ADDRESS>:9009'
        },
        'np2': {
            # replace with bob's real address.
            'address': '<IP_ADDRESS>:9008',
            'listen_addr': '<IP_ADDRESS>:9008'
        },
    },
    'self_party': 'np2'
}

# init all nodes in local Standalone Mode.
sf.init(address="<IP_ADDRESS>:9008", cluster_config=cluster_config)

# 定义设备
np1 = sf.PYU('np1')
np2 = sf.PYU('np2')

# 数据读取函数
def get_data(path,train):
    # x：iloc[:,1:-1]   y：iloc[:,-1:]
    from sklearn.preprocessing import StandardScaler
    ##  二分类变量 不进行归一
    import numpy as np
    data = pd.read_csv(path, encoding='utf-8')
    if train:
        data = data.iloc[:,1:-1]
        scaler = StandardScaler()
        x = scaler.fit_transform(data)
        return x
    else:
        data = data.iloc[:,-1:]
        y = np.array(data)
        return y

# 使用FedNdarray函数把数据读取并合并
v_data = FedNdarray(###  数据合并  HORIZONTAL 水平
    partitions={
        np1: np1(get_data)(path='/root/softs/train1.csv',train = True),
        np2: np2(get_data)(path='/root/sf_code/train2.csv',train = True),
    },
    partition_way=PartitionWay.HORIZONTAL,
)

v_data.shape[1]

label_data = FedNdarray(###  y数据合并
    partitions={
        np1: np1(get_data)(path='/root/softs/train1.csv',train = False),
        np2: np2(get_data)(path='/root/sf_code/train2.csv',train = False),
    },
    partition_way=PartitionWay.HORIZONTAL,
)

wait([p.data for p in v_data.partitions.values()])
wait([p.data for p in label_data.partitions.values()])

import tensorflow as tf

# In[23]: 定义模型

def create_conv_model():
    def create_model():
        from tensorflow import keras
        from tensorflow.keras import layers
        # Create model
#         model = tf.keras.Sequential()  # 顺序处理模型
        model = tf.keras.Sequential(
            [
                tf.keras.layers.Dense(1,input_shape=(v_data.shape[1],),activation=tf.nn.sigmoid)
            ])

        model.compile(
            optimizer='adam',
            loss='binary_crossentropy',  # 逻辑回归使用 binary_crossentropy 作为损失函数
            metrics=['accuracy'])
        return model

    return create_model

from secretflow.security.aggregation import SPUAggregator, SecureAggregator
from secretflow.ml.nn import FLModel

model = create_conv_model()

device_list = [np1, np2]

secure_aggregator = SecureAggregator(np1, [np1, np2]) # 安全聚合
spu_aggregator = SPUAggregator(spud) # 设备

fed_model = FLModel(server=np1,
                    device_list=device_list,
                    model=model,
                    aggregator=secure_aggregator,
                    strategy="fed_avg_w",
                    backend = "tensorflow")

history = fed_model.fit(v_data,
                        label_data,
                        epochs=10,
                        sampler_method="batch",
                        batch_size=128,
                        aggregate_freq=1)

global_metric = fed_model.evaluate(x_test, y_test, batch_size=128)
print(global_metric)

Reproduction code to reproduce the issue.
INFO:root:Run secretflow in production mode.
2023-04-24 11:12:06,589 INFO worker.py:1352 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:9008...
2023-04-24 11:14:11,602 WARNING utils.py:1414 -- Unable to connect to GCS at <IP_ADDRESS>:9008. Check that (1) Ray GCS with matching version started successfully at the specified address, and (2) there is no firewall setting preventing access.

hi，@nainai123
辛苦贴下两台机器分别启动Ray节点的步骤
另外：
如果在生产模式下，需要参与方都同时跑代码，这里代码是指所有的代码。当中的'self_party'与sf.init中的address要换成运行当事方的self_party 和ip:port。

hi，@nainai123 辛苦贴下两台机器分别启动Ray节点的步骤 另外： 如果在生产模式下，需要参与方都同时跑代码，这里代码是指所有的代码。当中的'self_party'与sf.init中的address要换成运行当事方的self_party 和ip:port。
1、两台机器的启动步骤
np1:启动
ray start --head --node-ip-address="<IP_ADDRESS>" --port="9009" --include-dashboard=False --disable-usage-stats
np2：启动
ray start --head --node-ip-address="<IP_ADDRESS>" --port="9008" --include-dashboard=False --disable-usage-stats

2、两台机器的代码
np1  SPU settings
SPU settings
cluster_config ={
'parties': {
'np1': {
# replace with alice's real address.
'address': '<IP_ADDRESS>:9009',
'listen_addr': '<IP_ADDRESS>:9009'
},
'np2': {
# replace with bob's real address.
'address': '<IP_ADDRESS>:9008',
'listen_addr': '<IP_ADDRESS>:9008'
},
},
'self_party': 'np1'
}
init all nodes in local Standalone Mode.
sf.init(address="<IP_ADDRESS>:9009", cluster_config=cluster_config)
np2：# SPU settings
SPU settings
cluster_config ={
'parties': {
'np1': {
# replace with alice's real address.
'address': '<IP_ADDRESS>:9009',
'listen_addr': '<IP_ADDRESS>:9009'
},
'np2': {
# replace with bob's real address.
'address': '<IP_ADDRESS>:9008',
'listen_addr': '<IP_ADDRESS>:9008'
},
},
'self_party': 'np2'
}
init all nodes in local Standalone Mode.
sf.init(address="<IP_ADDRESS>:9008", cluster_config=cluster_config)

INFO:root:Run secretflow in production mode.
2023-04-24 13:50:43,469	INFO worker.py:1352 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:9009...
2023-04-24 13:50:43,495	INFO worker.py:1538 -- Connected to Ray cluster.
2023-04-24 13:50:43 INFO api.py:202 [np1] --  Started rayfed with {'CLUSTER_ADDRESSES': {'np1': {'address': '<IP_ADDRESS>:9009', 'listen_addr': '<IP_ADDRESS>:9009'}, 'np2': {'address': '<IP_ADDRESS>:9008', 'listen_addr': '<IP_ADDRESS>:9008'}}, 'CURRENT_PARTY_NAME': 'np1', 'TLS_CONFIG': {}, 'CROSS_SILO_SERIALIZING_ALLOWED_LIST': None, 'CROSS_SILO_MESSAGES_MAX_SIZE_IN_BYTES': None, 'CROSS_SILO_TIMEOUT_IN_SECONDS': 60}
2023-04-24 13:50:44 INFO barriers.py:353 [np1] --  RecverProxy was successfully created.
2023-04-24 13:50:44,045	ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::RecverProxyActor.run_grpc_server() (pid=15397, ip=<IP_ADDRESS>, repr=<fed.barriers.RecverProxyActor object at 0x7f2104469a30>)
File "/root/miniconda3/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/root/miniconda3/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/fed/barriers.py", line 283, in run_grpc_server
return await _run_grpc_server(
File "/root/miniconda3/lib/python3.8/site-packages/fed/barriers.py", line 111, in _run_grpc_server
server.add_insecure_port(f'[::]:{port}')
File "/root/miniconda3/lib/python3.8/site-packages/grpc/aio/_server.py", line 84, in add_insecure_port
return _common.validate_port_binding_result(
File "/root/miniconda3/lib/python3.8/site-packages/grpc/_common.py", line 166, in validate_port_binding_result
raise RuntimeError(_ERROR_MESSAGE_PORT_BINDING_FAILED % address)
RuntimeError: Failed to bind to address [::]:9009; set GRPC_VERBOSITY=debug environment variable to see detailed error message.
(RecverProxyActor pid=15397) E0424 13:50:44.041055426   15437 chttp2_server.cc:1053]      UNKNOWN:No address added out of total 1 resolved for '[::]:9009' {created_time:"2023-04-24T13:50:44.040955214+08:00", children:[UNKNOWN:Failed to add any wildcard listeners {created_time:"2023-04-24T13:50:44.040939377+08:00", children:[UNKNOWN:Unable to configure socket {created_time:"2023-04-24T13:50:44.040902579+08:00", fd:38, children:[UNKNOWN:Address already in use {created_time:"2023-04-24T13:50:44.040895264+08:00", errno:98, os_error:"Address already in use", syscall:"bind"}]}, UNKNOWN:Unable to configure socket {created_time:"2023-04-24T13:50:44.040935329+08:00", fd:38, children:[UNKNOWN:Address already in use {syscall:"bind", os_error:"Address already in use", errno:98, created_time:"2023-04-24T13:50:44.040932342+08:00"}]}]}]}
2023-04-24 13:50:44 INFO barriers.py:388 [np1] --  SendProxy was successfully created.
2023-04-24 13:50:44 INFO cleanup.py:85 [np1] --  Start check sending thread.
2023-04-24 13:50:44 INFO cleanup.py:91 [np1] --  Start check sending monitor thread.
Traceback (most recent call last):
File "tf_lr.py", line 76, in
v_data.shape[1]
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/data/ndarray.py", line 85, in shape
shapes = self.partition_shape()
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/device/driver.py", line 130, in wrapper
return reveal(func_or_object(*arg, **kwargs))
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/device/driver.py", line 153, in reveal
all_object = sfd.get(all_object_refs)
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/distributed/primitive.py", line 47, in get
return fed.get(object_refs)
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/fed/api.py", line 402, in get
received_ray_object_ref = recv(
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/fed/barriers.py", line 410, in recv
receiver_proxy = ray.get_actor(f"RecverProxyActor-{party}")
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/ray/_private/worker.py", line 2538, in get_actor
return worker.core_worker.get_named_actor_handle(name, namespace or "")
File "python/ray/_raylet.pyx", line 2260, in ray._raylet.CoreWorker.get_named_actor_handle
File "python/ray/_raylet.pyx", line 199, in ray._raylet.check_status
ValueError: Failed to look up actor with name 'RecverProxyActor-np1'. This could because 1. You are trying to look up a named actor you didn't create. 2. The named actor died. 3. You did not use a namespace matching the namespace of the actor.
2023-04-24 13:50:44 INFO cleanup.py:103 [np1] --  Notify check sending thread to exit.
2023-04-24 13:50:45 WARNING cleanup.py:62 [np1] --  Signal self to exit.
*** SIGTERM received at time=1682315445 on cpu 9 ***
2023-04-24 13:50:45 INFO cleanup.py:66 [np1] --  Check sending thread was exited.
PC: @     0x7f285f4dcb3b  (unknown)  do_futex_wait.constprop.1
@     0x7f285f4de630  (unknown)  (unknown)
@ ... and at least 1 more frames
[2023-04-24 13:50:45,777 E 15336 15336] logging.cc:361: *** SIGTERM received at time=1682315445 on cpu 9 ***
[2023-04-24 13:50:45,777 E 15336 15336] logging.cc:361: PC: @     0x7f285f4dcb3b  (unknown)  do_futex_wait.constprop.1
[2023-04-24 13:50:45,777 E 15336 15336] logging.cc:361:     @     0x7f285f4de630  (unknown)  (unknown)
[2023-04-24 13:50:45,777 E 15336 15336] logging.cc:361:     @ ... and at least 1 more frames
Exception ignored in: <module 'threading' from '/root/miniconda3/envs/sf/lib/python3.8/threading.py'>
Traceback (most recent call last):
File "/root/miniconda3/envs/sf/lib/python3.8/threading.py", line 1388, in _shutdown
lock.acquire()
File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/ray/_private/worker.py", line 1640, in sigterm_handler
sys.exit(signum)
SystemExit: 15
(SendProxyActor pid=15443) 2023-04-24 13:50:45 ERROR barriers.py:240 [np1] --  Failed to send data to seq_id 5 of np2 from {upstream_seq_id}, error: <AioRpcError of RPC that terminated with:
(SendProxyActor pid=15443) 	status = StatusCode.UNIMPLEMENTED
(SendProxyActor pid=15443) 	details = ""
(SendProxyActor pid=15443) 	debug_error_string = "UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:9008 {created_time:"2023-04-24T13:50:45.766118424+08:00", grpc_status:12, grpc_message:""}"
(SendProxyActor pid=15443) >

INFO:root:Run secretflow in production mode.
2023-04-25 13:36:21,247	INFO worker.py:1352 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:9009...
2023-04-25 13:36:21,254	INFO worker.py:1538 -- Connected to Ray cluster.
2023-04-25 13:36:21 INFO api.py:202 [np1] --  Started rayfed with {'CLUSTER_ADDRESSES': {'np1': {'address': '<IP_ADDRESS>:9090', 'listen_addr': '<IP_ADDRESS>: 9090'}, 'np2': {'address': '<IP_ADDRESS>:9080', 'listen_addr': '<IP_ADDRESS>: 9080'}}, 'CURRENT_PARTY_NAME': 'np1', 'TLS_CONFIG': {}, 'CROSS_SILO_SERIALIZING_ALLOWED_LIST': None, 'CROSS_SILO_MESSAGES_MAX_SIZE_IN_BYTES': None, 'CROSS_SILO_TIMEOUT_IN_SECONDS': 60}
2023-04-25 13:36:21 INFO barriers.py:353 [np1] --  RecverProxy was successfully created.
(RecverProxyActor pid=8525) 2023-04-25 13:36:21 INFO barriers.py:114 [np1] --  Successfully start Grpc service without credentials.
2023-04-25 13:36:22 INFO barriers.py:388 [np1] --  SendProxy was successfully created.
2023-04-25 13:36:22.551574: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
2023-04-25 13:36:22.578678: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-04-25 13:36:23.106839: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
2023-04-25 13:36:23.106928: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
2023-04-25 13:36:23.106944: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
(scheduler +2s) Tip: use ray status to view detailed cluster status. To disable these messages, set RAY_SCHEDULER_EVENTS=0.
(scheduler +2s) Error: No available node types can fulfill resource request {'np2': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +2s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'np1': 1.0}. Add suitable node types to this cluster to resolve this issue.
(SPURuntime pid=8612) 2023-04-25 13:36:24,074	ERROR worker.py:763 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::SPURuntime.init() (pid=8612, ip=<IP_ADDRESS>, repr=<secretflow.device.device.spu.SPURuntime object at 0x7fb9a984abe0>)
(SPURuntime pid=8612)   File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/device/device/spu.py", line 275, in init
(SPURuntime pid=8612)     fill_link_desc_attrs(link_desc=link_desc, desc=desc)
(SPURuntime pid=8612)   File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/device/device/spu.py", line 71, in fill_link_desc_attrs
(SPURuntime pid=8612)     raise InvalidArgumentError(
(SPURuntime pid=8612) secretflow.utils.errors.InvalidArgumentError: Unsupported param connect_retry_times in link desc, ['connect_retry_interval_ms', 'recv_timeout_ms', 'http_max_payload_size', 'http_timeout_ms', 'throttle_window_size', 'brpc_channel_protocol', 'brpc_channel_connection_type'] are now available only.
2023-04-25 13:36:24 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.security.aggregation.secure_aggregator.Masker'> with party np1.
2023-04-25 13:36:24 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.security.aggregation.secure_aggregator.Masker'> with party np2.
2023-04-25 13:36:24 INFO cleanup.py:85 [np1] --  Start check sending thread.
2023-04-25 13:36:24 INFO cleanup.py:91 [np1] --  Start check sending monitor thread.
2023-04-25 13:36:37 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.ml.nn.fl.backend.tensorflow.strategy.fed_avg_w.PYUFedAvgW'> with party np1.
2023-04-25 13:36:37 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.ml.nn.fl.backend.tensorflow.strategy.fed_avg_w.PYUFedAvgW'> with party np2.
2023-04-25 13:36:37 INFO fl_model.py:317 [np1] --  FL Train Params: {'self': <secretflow.ml.nn.fl.fl_model.FLModel object at 0x7f5fc4d3de20>, 'x': FedNdarray(partitions={np1: <secretflow.device.device.pyu.PYUObject object at 0x7f633c71ef70>, np2: <secretflow.device.device.pyu.PYUObject object at 0x7f633c6a73a0>}, partition_way=<PartitionWay.HORIZONTAL: 'horizontal'>), 'y': FedNdarray(partitions={np1: <secretflow.device.device.pyu.PYUObject object at 0x7f633c6a7820>, np2: <secretflow.device.device.pyu.PYUObject object at 0x7f633c6a7940>}, partition_way=<PartitionWay.HORIZONTAL: 'horizontal'>), 'batch_size': 128, 'batch_sampling_rate': None, 'epochs': 10, 'verbose': 1, 'callbacks': None, 'validation_data': None, 'shuffle': False, 'class_weight': None, 'sample_weight': None, 'validation_freq': 1, 'aggregate_freq': 1, 'label_decoder': None, 'max_batch_size': 20000, 'prefetch_buffer_size': None, 'sampler_method': 'batch', 'random_seed': 37692, 'dp_spent_step_freq': None, 'audit_log_dir': None}
(pid=8796) 2023-04-25 13:36:38.908261: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
(pid=8796) 2023-04-25 13:36:38.933950: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
(pid=8796) 2023-04-25 13:36:39.408846: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
(pid=8796) 2023-04-25 13:36:39.408925: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
(pid=8796) 2023-04-25 13:36:39.408934: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
0%|                                                                                | 0/3 [00:00<?, ?it/s](scheduler +37s) Error: No available node types can fulfill resource request {'np2': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +37s) Error: No available node types can fulfill resource request {'np1': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +1m12s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'np2': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +1m12s) Error: No available node types can fulfill resource request {'np1': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.

在按照严格的生产环境安装流程 调试各种配置之后  发现报错如下  ：需要怎么处理
INFO:root:Run secretflow in production mode.
2023-04-25 13:36:21,247	INFO worker.py:1352 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:9009...
2023-04-25 13:36:21,254	INFO worker.py:1538 -- Connected to Ray cluster.
2023-04-25 13:36:21 INFO api.py:202 [np1] --  Started rayfed with {'CLUSTER_ADDRESSES': {'np1': {'address': '<IP_ADDRESS>:9090', 'listen_addr': '<IP_ADDRESS>: 9090'}, 'np2': {'address': '<IP_ADDRESS>:9080', 'listen_addr': '<IP_ADDRESS>: 9080'}}, 'CURRENT_PARTY_NAME': 'np1', 'TLS_CONFIG': {}, 'CROSS_SILO_SERIALIZING_ALLOWED_LIST': None, 'CROSS_SILO_MESSAGES_MAX_SIZE_IN_BYTES': None, 'CROSS_SILO_TIMEOUT_IN_SECONDS': 60}
2023-04-25 13:36:21 INFO barriers.py:353 [np1] --  RecverProxy was successfully created.
(RecverProxyActor pid=8525) 2023-04-25 13:36:21 INFO barriers.py:114 [np1] --  Successfully start Grpc service without credentials.
2023-04-25 13:36:22 INFO barriers.py:388 [np1] --  SendProxy was successfully created.
2023-04-25 13:36:22.551574: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
2023-04-25 13:36:22.578678: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-04-25 13:36:23.106839: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
2023-04-25 13:36:23.106928: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
2023-04-25 13:36:23.106944: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
(scheduler +2s) Tip: use ray status to view detailed cluster status. To disable these messages, set RAY_SCHEDULER_EVENTS=0.
(scheduler +2s) Error: No available node types can fulfill resource request {'np2': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +2s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'np1': 1.0}. Add suitable node types to this cluster to resolve this issue.
(SPURuntime pid=8612) 2023-04-25 13:36:24,074	ERROR worker.py:763 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::SPURuntime.init() (pid=8612, ip=<IP_ADDRESS>, repr=<secretflow.device.device.spu.SPURuntime object at 0x7fb9a984abe0>)
(SPURuntime pid=8612)   File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/device/device/spu.py", line 275, in init
(SPURuntime pid=8612)     fill_link_desc_attrs(link_desc=link_desc, desc=desc)
(SPURuntime pid=8612)   File "/root/miniconda3/envs/sf/lib/python3.8/site-packages/secretflow/device/device/spu.py", line 71, in fill_link_desc_attrs
(SPURuntime pid=8612)     raise InvalidArgumentError(
(SPURuntime pid=8612) secretflow.utils.errors.InvalidArgumentError: Unsupported param connect_retry_times in link desc, ['connect_retry_interval_ms', 'recv_timeout_ms', 'http_max_payload_size', 'http_timeout_ms', 'throttle_window_size', 'brpc_channel_protocol', 'brpc_channel_connection_type'] are now available only.
2023-04-25 13:36:24 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.security.aggregation.secure_aggregator.Masker'> with party np1.
2023-04-25 13:36:24 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.security.aggregation.secure_aggregator.Masker'> with party np2.
2023-04-25 13:36:24 INFO cleanup.py:85 [np1] --  Start check sending thread.
2023-04-25 13:36:24 INFO cleanup.py:91 [np1] --  Start check sending monitor thread.
2023-04-25 13:36:37 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.ml.nn.fl.backend.tensorflow.strategy.fed_avg_w.PYUFedAvgW'> with party np1.
2023-04-25 13:36:37 INFO proxy.py:180 [np1] --  Create proxy actor <class 'secretflow.ml.nn.fl.backend.tensorflow.strategy.fed_avg_w.PYUFedAvgW'> with party np2.
2023-04-25 13:36:37 INFO fl_model.py:317 [np1] --  FL Train Params: {'self': <secretflow.ml.nn.fl.fl_model.FLModel object at 0x7f5fc4d3de20>, 'x': FedNdarray(partitions={np1: <secretflow.device.device.pyu.PYUObject object at 0x7f633c71ef70>, np2: <secretflow.device.device.pyu.PYUObject object at 0x7f633c6a73a0>}, partition_way=<PartitionWay.HORIZONTAL: 'horizontal'>), 'y': FedNdarray(partitions={np1: <secretflow.device.device.pyu.PYUObject object at 0x7f633c6a7820>, np2: <secretflow.device.device.pyu.PYUObject object at 0x7f633c6a7940>}, partition_way=<PartitionWay.HORIZONTAL: 'horizontal'>), 'batch_size': 128, 'batch_sampling_rate': None, 'epochs': 10, 'verbose': 1, 'callbacks': None, 'validation_data': None, 'shuffle': False, 'class_weight': None, 'sample_weight': None, 'validation_freq': 1, 'aggregate_freq': 1, 'label_decoder': None, 'max_batch_size': 20000, 'prefetch_buffer_size': None, 'sampler_method': 'batch', 'random_seed': 37692, 'dp_spent_step_freq': None, 'audit_log_dir': None}
(pid=8796) 2023-04-25 13:36:38.908261: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
(pid=8796) 2023-04-25 13:36:38.933950: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
(pid=8796) 2023-04-25 13:36:39.408846: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
(pid=8796) 2023-04-25 13:36:39.408925: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/oracle/instantclient_11_2::/usr/local/cuda-10.1/lib64:/usr/lib64
(pid=8796) 2023-04-25 13:36:39.408934: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
0%|                                                                                | 0/3 [00:00<?, ?it/s](scheduler +37s) Error: No available node types can fulfill resource request {'np2': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +37s) Error: No available node types can fulfill resource request {'np1': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +1m12s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'np2': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +1m12s) Error: No available node types can fulfill resource request {'np1': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.

@nainai123 请问一下你这个问题解决了吗？怎么解决的？


Doctrine provider

Hi
As discussed a while ago here's an implementation of a Doctrine provider for qpush.  The idea is to have a table 'uecode_qpush_message' that holds all the messages for all the queues for the provider.  Each message has a queue_name and a boolean to mark it as delivered or not.  The message gets serialised by doctrine and then stored/retrieved from the table.  Messages are not deleted from the table apart from when the destroy method is called.  So the message table has a history of all the message sent along with created and updated times.
To use - examples
config.yml

uecode_qpush:

providers:
    doctrine_provider:
        driver: doctrine
queues:
    modified_products:
        provider: doctrine_provider

You can optionally pass in an entity manager for doctrine.  Otherwise it defaults to doctrine.orm.default_entity_manager
entity_manager:  my.entity.manager
Add this to composer.json
    "doctrine/orm":           "^2.4.8",
    "stof/doctrine-extensions-bundle": "^1.2"

To create the table use  console doctrine:schema:update
Other than that usage is as per the other providers.

A new inspection was created.

The inspection completed: 3 new issues, 29 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 28 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 28 updated code elements

A new inspection was created.

The inspection completed: 3 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: No new issues

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

Hey @talisman-innovations! I added a couple minor CS type comments, otherwise, this is looking pretty good.

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

Hi @kmfk - no problem - I think I got them all, plus added an implementation of the create function for good measure.  Regards Steve

For a use case we have I'm thinking of implementing a function to allow for redelivery of a message.  As the messages sit in the DB it's trivial to implement.  Any issues with adding that?
Tt could go in the AbstractProvider if other providers offer that feature.
Any thoughts on naming - maybe republish(id)?

A new inspection was created.

The inspection completed: 2 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 4 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 4 new issues, 30 updated code elements

A new inspection was created.

The inspection completed: 9 new issues, 31 updated code elements

A new inspection was created.

The inspection completed: 5 new issues, 31 updated code elements

@talisman-innovations I have no memory of why this would still be open for so long.  Looking it over, this looks good to me, but wanted to confirm with you before merging.

Hi - yes feel free to merge.  Thanks.

Will do.  Currently need to fix an issue with PHPUnit and Symfony ^3.2 to fix the failing builds.  Will get this in afterward.

This provider is not documented.
In production environment, even if this provider is NOT used, cache:clear triggers:
In AnnotationException.php line 54:
  [Semantical Error] The annotation "@Gedmo\Mapping\Annotation\Timestampable" in property Uecode\Bundle\QPushBundle\Entity\DoctrineMessage::$created does not exist, or could not be auto-loaded.


Add whereLabel* functions

This PR adds a few functions from the whereLabel* family:

whereLabelExists
whereLabelDoesNotExist
whereLabelValueMatches

This should help with situations like those mentioned in https://github.com/jwilder/docker-gen/pull/82#issuecomment-158323192
Right now, these functions are strictly for []*RuntimeContainer, but they could be adjusted to work in other situations if need be. I could also see adding a hasLabel function that takes *RuntimeContainer or exposing the regexp.MatchString function used by whereLabelValueMatches, but I wanted to keep this PR simple and focused.

An example use case (as I mentioned in https://github.com/jwilder/docker-gen/pull/82#issuecomment-165362470) would be:
whereLabelValueMatches . "com.docker.compose.service" "^web$"

Thanks @md5!

:+1:


The SST-2 dataset for explanation is different from the SST-2 for training.

I note that you used the sst-2 parsing tree from torchtext.datasets.SST at explanation stage. However, I check the text file in "./.data/sst/trees/train.txt"  which is gotten from torchtext. This data is different from the sst-2 used in your training stage, which is the official dataset. For example, there are 1101 sentences in the dataset from torchtext but we only have 872 in the official dataset. Maybe the dataset in torchtext is too old.
I'm not sure if some data appears in both training set at the training stage and test set at the explanation stage. Please check it!

sorry, it should be "1101 in the torchtext's dev set and 872 in the official dev set".


Ability to background performance tests

Provide users with the ability to background performance tests, so that they can use other aspects of Meshery while tests are running.

@leecalcote, is the capability that was incorporated today enough to consider this taken care of?


Large parallel uploads (> 100 GB) failing due to timeout?

Question
I am using tusd with tus-js-client to upload large files (> 100 GB) using parallel uploads (10 uploads). I see uploads failing sometimes, with the final file not being concatenated. The browser receives a timeout during the final HTTP call (Upload-Concat), which is likely caused by tusd taking too long to concatenate the file. I suspect that tusd stops the concatenation once the timeout has been reached.
Even if I set the tusd timeout to 0 (infinite) the browser will still timeout at some point - I believe it is 5 minutes in chrome.
Have you seen this behaviour with large parallel uploads and is there a solution to it? Can I make tusd send the response to the final Upload-Concat before the file is completely concatenated? I was also thinking about using onChunkComplete in tus-js-client and simply consider the upload done / resolve once all bytes have been received.
Setup details
Operating System: Linux (Ubuntu 20.04)
Used tusd data storage: disk store via NFS mount
Used tusd configuration: -upload-dir=/data/incoming -host=<IP_ADDRESS> -behind-proxy -expose-metrics=false -hooks-dir=/data/tushooks
Used tus client library: tus-js-client using parallel uploads (10 uploads)

I haven't seen this behavior before. What timeout are you exactly seeing? Is it a timeout sent from the tusd server or is it a timeout from the browser because it didn't receive the response in 5 minutes? Also, do the logs from tusd mention something?
If you are concatenation a 100GB file, you have to know that once all individual uploads are finished, tusd will read each individual upload and write the entire 100GB to disk again. Depending on your NFS setup, these 100GB might take too long to read and write, causing the long wait time. Maybe you also want to evaluate the performance of your NFS system.

Have you seen this behaviour with large parallel uploads and is there a solution to it?

No, I have never seen that before.

Can I make tusd send the response to the final Upload-Concat before the file is completely concatenated?

No, that is not possible and I don't think it would be a good solution.

I was also thinking about using onChunkComplete in tus-js-client and simply consider the upload done / resolve once all bytes have been received.

I would not recommend this approach since this will swallow errors with can occur during concatenation.

Closing this issue due to a lack of information. Feel free to leave a comment if you want to continue the discussion :)


fix(helm): get/get-helm-3 whitespace support in runAsRoot

When get/get-helm-3 is run with a HELM_INSTALL_DIR containing spaces, the installation fails.
This PR fixes the issues, by wrapping the parameters correctly.
Closes #9346

Close due to issues (DCO checks) after changing the merge target to overcome update issue of github. Will open a new PR.

@jdolitsky reopened in #9349


Fixed #25807 -- Added support for LazyObject serialization in migrations.

Thanks to Trac alias mrgaolei for the report.

Comments addressed.

Do you want to add this to the docs/release notes similar to what was done in 537818af8783a4f9f239ed6277b1f810c1666839?

Sure, I'll do it tomorrow.

Release note added.

How about documentation in topics/migrations too? I'd combine the release note with the existing one about new serialization support.

Relase nototes combined and topics/migrations doc added.


Enhancement: some way to disallow including the bundle

Using zero for basically 100% server side rendering it would be great if I could stop the bundled JS from being included on the page as it dramatically increases page weight for zero benefit.
idea for a zero-config approach for this:
import {NoBundle} from "zero";

<NoBundle/>

// normal JSX/MDX/etc code goes here

Thoughts?

I think that's a good idea. How about this:
export const config = {
  noBundling: true
}

export default ()=>(

Zero bundle

Do you think it's better this way? Rather than importing a JSX component just so we avoid having config.

That solution wouldn’t work for MDX though so... would still need a solution for that.
Effectively the imported component could do just that tho.
Maybe, to generalize:
import {Overrides} from “zero”;

<Overrides bundle={} someotherflag={} />

Overrides could effectively be a stateless function like so:
export const Overrides = (props)=>{
      // set higher scope stuff here from props
    return;
};

That way it could also be used in regular JS as:
Overrides({bundle: false});

Zero config, compatibility with all formats, flexibility for the future!

Actually it works with MDX too:
# Heading
This is markdown

export const config = {
  noBundling: true
}

I understand the benefits that you listed of it being a component like <Helmet />. I will research pros and cons of both and keep this issue updated.

Ah that's interesting. Didn't realize you could add JS exports to MDX.
Still not a huge fan of adding JS semantics to the files when not required however...


Add extend cursorline option

I'd like to add an option not extend the cursorline in certain modes behind the line number and sign column. Currently, the cursorline is not extended when in Normal mode but is extended for other modes. Perhaps after this option is added we may want to extend the cursorline in Normal mode when extend is true.

Can this be achieved by configuring the (undocumented) Modes{Mode}CursorLineNr/ Modes{Mode}CursorLineSign highlight groups?


Agora - Check enough public IP available

Added function Get-AzAvailablePublicIpAddress to check available public ip
Update output

For azd combined VM and Public IP capacity check.
For others updated document.


chore: add canonical URL to head template

Fix #260

@davlgd thank you

I am sorry, but it is a good idea to provide the possibility to specify the canonical URL if any. This is useful to content writers referring to another resource as original.
On our website https://leipapa.com we use the following structure:
Added canonical to front matter and ...
{{ with .Params.canonical }}
  <link rel="canonical" href="{{ . }}" itemprop="url" />
{{ else }}
  <link rel="canonical" href="{{ .Permalink }}" itemprop="url" />
{{ end }}

I am sorry, but it is a good idea to provide the possibility to specify the canonical URL if any. This is useful to content writers referring to another resource as original.
On our website https://leipapa.com we use the following structure:
Added canonical to front matter and ...

{{ with .Params.canonical }}

  <link rel="canonical" href="{{ . }}" itemprop="url" />

{{ else }}

  <link rel="canonical" href="{{ .Permalink }}" itemprop="url" />

{{ end }}

@mafendi hey, can you create a PR for this?


Service Accounts named 'default' are hard to distinguish when using the dashboard UI search

Is your feature request related to a problem? Please describe.
I can not see which namespace a Service Account belongs to while searching for it on the UI using the 'Kubernetes Service Account Detail' dashboard.
I came across this issue when trying to find the default Service Account for a namespace.
Describe the solution you'd like
Search shows some additional context like which cluster it's in, and I'd like it to have the namespace info too.
Additional context
Here's a screenshot of searching for the default namespace:

The related code, I think, can be found here:
https://github.com/turbot/steampipe-mod-kubernetes-insights/blob/91c5d24b4b480af75ca4d97790f2a062fec5103c/dashboards/serviceaccount/service_account_detail.sp#L182-L195

@oguzhan-yilmaz Apologies for the inconvenience.
We have raised a PR in which we added namespace to the service account title for unique identification.
Please let us know if this works for you.
Thanks.

Hey @khushboo9024, I think it'd be better to add the namespace info as a tag like the cluster-name.
And it could be added to other dashboards as well. Thank you!

@oguzhan-yilmaz  We have updated the PR with namespace info as a tag to the service account detail page for unique identification . Please let us know if this works for you.
Thanks

Thank you this is what I've been looking for!


New ipywidgets release seems to break things in bqplot

If i update ipywidgets and bqplot, bqplot no longer works.

After consulting with @SylvainCorlay , it seems to be an issue created by the resizing.


fix: set nginx imagePullPolicy from values

Simple fix to use the chart's nginx.imagePullPolicy value, based on some less simple debugging in our dev cluster where we're using a latest tag 😅

I like the "based on some less simple debugging in our dev cluster" ;-) That was an interesting one to track.

Looks good to me I'll merge this and then bump the charts


stuck in update loop during installation

don't really know what to make of it or how to explain, something about an error with acpi. just constantly cycles through what you see in the picture until manually stopped with ctrl+c. gets through the archvault setup but can't complete the install

Arch is a rolling release distro and there's always a possibility upstream pkg updates botch downstream installers like Archvault. For better or worse, I encounter these sorts of issues several times per year.
However, in this case, upstream has not updated the acpi pkg since 2020-05-06. I last tested Archvault on 2020-05-19 in VirtualBox, where it succeeded without encountering this error. Hence I don't see how our friend acpi here would be causing this issue by itself.
Did you bootstrap Archvault on a clean system? shred -fvz -n 0 /dev/sdX -- you will lose all your data -- could be of help. On physical hardware, if for whatever reason I cancel out of the Archvault bootstrap, I always reboot and shred the hard drive before re-attempting. Or equivalently, I delete the VM I'm testing on and then bring one up fresh.
Normally, I would spin up a VM to test Archvault right now, but I can't right now due to bandwidth considerations. Sorry for the inconvenience.

That was a good suggestion. Thankfully I have nothing important on this laptop and can continuously wipe it and reinstall :) I now receive a different style error (still involving acpi for some reason). I'll give it a go in a VM at some point to see where that leads me

lsblk behavior changed upstream in latest util-linux release, botching partition indexing. Fixed in https://github.com/atweiden/archvault/commit/70403259c1eb35dda0c39a38034fea5d109c5c2a

works no problem now! thanks!


Stop using COVID Tracking Project

They posted today that they will stop updating on March 7th. For now the easiest thing would be to start using the NYTimes data that's already in the repo.

Please assign #3 to me when you've made the change in the charting.

Commit b3e5f3c fixes this. Now using NY Times data only. Graphs look the same to the eye, except for the NJ/Bergen Jan 4 change in reporting spike


Update remove_host to delete IPAM blocks for host

Help fix projectcalico/calico-docker#519: calicoctl node stop does not release IPAM allocation for host
Use BGP Path format for IPAM Path constant

On its own, it isn't sufficient to just delete the host IPAM tree as the IPs associated with that host will still be assigned - so this will result in inconsistent IPAM data.
I'm planning to have a think about the broader implications of node stop, so I'll revisit this soon.

Also, this code should really be in the ipam module rather than the datastore.
Probably need a delete_host or similar method in ipam to delete the IP assignments, remove any block affinities, and delete the host tree.
Don't update this PR at the moment as I'm looking at a reworked design for this.

Should we close this PR if you are working on an alternate, more robust design?

Yup.  Closing now - just wanted to get my thoughts down first.


I don't even know.

Description
I recently installed ALVR to try with my Quest 2, and at first, everything launched fine but the visible framerate on the headset was like 0.5 fps, while the diagnostics page said it was 60 (what I had it set to). There were weird artifacts in the view (SteamVR Home) although audio was perfect. I tried both video encoders and H264 was even worse. It just showed a mess of green pink splotches and other colored lines; I could not see what I was supposed to at all. The SteamVR view looked as expected, so I'm assuming it's some sort of connection/networking issue. After taking the headset off, this monster of an error screen appeared and I had to use Ctrl+Alt+Del to open task manager and shut down the program. My mouse was glitching like crazy whilst this was happening and if I could I would've taken a screen recording.

Environment
Hardware
CPU: Intel i7-1165G7 @ 2.80GHz
GPU: NVIDIA GeForce MX450
RAM: DDR4 16GB
Installation
ALVR Version: v19.0.0
SteamVR Version: v1.24.7
Install Type:

[x] Packaged (exe)

OS Name and Version: Windows 11 Home 22H2
OS Build Number: 22621.1105

GeForce MX450 doesn't have hardware video encoder, so it falls back to software encoding on CPU (which is quite intensive task). What you experience is a bug with software encoding and you should try nightly where it is working https://github.com/alvr-org/ALVR-nightly.


NAS oneshot (stage 2) - Supernet modules

it would be better to provide simple tutorials about how to use these operators, sampling, and differentiables.

Sure. Will do.


ISPN-7979 Add licenses to distributions

Added server distribution

https://issues.jboss.org/browse/ISPN-7979

Closing for now, have to do some more changes

add in console licenses.xml
move files to doc/licenses
ensure licenses match names exactly


Fix the warning message about which version of rubocop-performance to upgrade to

We have received a couple of questions about this in https://github.com/rubocop-hq/rubocop/pull/7249. If there is a plan to release a minor or revision update, we may want to include this.
rubocop-performance hasn't created their 1.5.0 release yet. Maybe it would be easier for that version to jump to 1.15.0 🤣

Thanks!


Fix duplicate frames and respect original frame rate

A couple of nuances of how ffmpeg works are fixed here:

The default ffmpeg frame rate is 25 fps, but many security cameras are set at 5-15 fps.  The new fps command (source_fps) sets the sampling rate to the original video's rate.
when decoding videos to images, ffmpeg will duplicate frames to match the output files to the input file.  This resulted in many duplicate frames.  We can set fps_mode to passthrough to tell ffmpeg that we don't want it to create duplicate frames to fill in the gaps.  That's why we used select!  We want to dump some frames!

Side note:  For debugging ffmpeg, it's useful to add another parameter to the command:  "2> /config/www/llmvision/out.txt" so we can see what it's doing.
This fixes https://github.com/valentinfrlch/ha-llmvision/issues/118 and also results in nice, smooth frames and drastically improves the descriptions the LLM returns.

Also note this is extracting every other frame. If the intent was different, let me know and I can adjust the command to meet the intent

Thanks so much! It will probably take me a while to test and merge this, but I definitely will!

Can I ask why you closed this? Sorry it takes this long, I'm very busy until next week. I know its frustrating when it takes so long to get a PR merged but I think your contribution would be a great addition!

Oh sorry. I merged the branches and deleted them in my repo without thinking about it. I'll restore the branches

I also have a version that I'm using now that allows a parameter to pull out evenly spaced frames from the video if you want that

Spent some more time with this.  The version in the PR extracts every interval'th frame (e.g. every second frame) then uses the motion filter to pick the top n up to max_frames.
I have two other versions on my own fork, each controlled by params.  Let me know if you want PRs for them:

An option that uses evenly spaced frames.  Takes a video of any length and extracts max_Frames, evenly spaced throughout the video.  Works for all frame rates.  We always keep the first and last frame.  For example, if the video is 100 frames and you max_frames is 10, you get frames 1, 12, 23, 34, 45, 56, 67, 78, 89, 100.  Note that ffmpeg only works per-frame so we either have to pre-process to determine the number of frames, or post-process and remove frames.  I opted for the latter so ffmpeg extracts all frames to the tmp directory, then we just drop all the frames that we don't want.  This takes a little more processing and disk space, but immaterial compared to the other processing for llm vision.
An option that captures one frame every nth seconds.  This one is pretty straightforward, but I haven't tested it as much.  Here's the ffmpeg command:

                        # Extract frames from video every frame_per_interval seconds
                        ffmpeg_cmd = [
                            "ffmpeg",
                            "-i", video_path,
                            "-vf", f"fps=fps='source_fps',select='bitor(gte(t-prev_selected_t\\,{frame_per_interval})\\,isnan(prev_selected_t))'",
                            "-fps_mode", "passthrough",
                            os.path.join(tmp_frames_dir, "frame%04d.jpg")
                        ]

Let me know if you want PRs for any of that and happy to send them upstream

Thanks again for your work!


s2n-quick-rustls should using ring on windows

Resolved issues:
resolves #ISSUE-NUMBER1, resolves #ISSUE-NUMBER2, etc.
Description of changes:

Call-outs:

Testing:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Thank you for the PR. As of version 1.9, aws-lc-rs builds on Windows with no additional build dependencies. Therefore, we will continue to use aws-lc on all platforms.


Inline class on the fieldset causing issues with the requirement for stacked items inside

I have found in our service we have a requirement to have stacked radio options within an inline group of options of Guilty / Not guilty
Essentially placing a fieldset inside a fieldset which is totally fine. The problem is that the second fieldset block labels pick up the inline styling regardless of having no inline class associated with the second fieldset.
Do you have something in place to cater for this that I am not aware of?
I think this is a little too generic
.inline .block-label
I have worked around the problem by adding a new class with attributes to override the inherited attributes.
.stacked .block-label

Can you move the inline class further down your tree? For example, you could have a <fieldset class="inline"> that wraps just the guilty / not guilty questions and doesn’t include the further questions.

Hi Rob. Yes I could do that. Will give it a try this morning and see how that goes.

Did that help? Will close this issue if it did.

Hi @trevorsaint, did @robinwhittleton's suggestion fix your issue? Please let us know so we can close this issue, thanks!

Hi @trevorsaint, I'm hoping @robinwhittleton's suggestion fixed your issue. Please re-open if not.


Allow additional namespaceSelector matchExpressions to be configured on ValidatingWebHook

What this PR does / why we need it: Azure Kubernetes Service force-injects an additional matchExpression on the namespaceSelector of this webhook... one that, if you don't template it out, causes continuous synchronization failures using most Continuous Delivery tools, namely ArgoCD in the parent issue.  It may also be the case that a user wishes to configure additional namespaceSelectors to their liking for their given cluster or cloud provider: not just AKS.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #4114
Special notes for your reviewer: none needed really.
Release note:

Allow additional namespaceSelector matchExpressions to be configured on ValidatingWebHook

/assign @munnerz

Thanks for re-working this PR! :)
Do we need a similar configuration option for the mutating webhook as well?
/ok-to-test

/kind feature

@munnerz I don't think so for this context.  The use case I have doesn't at least.

Anything else keeping this from proceeding?


Protobuf compiler in j2objc v2.1.1 generates code with references to enum values missing from jar

With input protobuf definition sample.proto:
syntax = "proto2";

package protobuf;

option optimize_for = LITE_RUNTIME;

message Handshake {
    required string version = 1;
    optional string android_app_version = 101;
    optional string ios_app_version = 102;
}

I run the protobuf compiler using $J2OBJC/j2objc_protoc --java_out=java --j2objc_out=objc sample.proto
The generated Java code includes references to enum values GET_MEMOIZED_IS_INITIALIZED and SET_MEMOIZED_IS_INITIALIZED for type com.google.protobuf.GeneratedMessageLite.MethodToInvoke that are missing in $J2OBJC/lib/protobuf_runtime.jar (verified missing using JD-GUI). This causes errors when using j2objc to translate Java source files with references to the generated protobuf Java code.

The "optimize for LITE_RUNTIME" option no longer supports reflective methods in the Java protobuf runtime. That support is now available in a separate libprotobuf-lite.jar, but there isn't a j2objc equivalent to that runtime. I recommend you remove that option from your proto file.

Thanks for looking into this.


Update the structure of my-yoga-server

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks a lot for catching this and sorry for the delay @Ferdous-Al-Imran 🙌


Michael Minella's coverage gap observations (Bug 4706, 4712, 4713, 4799)

Towards the end of the 1.0 FR cycle, Michael Minella wrote up a number of detailed "coverage gaps" in the TCK.
We closed all the bugzilla bugs around 1.0 FR, but I'm guessing some of these observations are still valid.
Some of these may have already been addressed (some you see  are already addressed in the bug comments).. some we might not want to do .
Just a starting point:
See:
https://java.net/bugzilla/show_bug.cgi?id=4706
https://java.net/bugzilla/show_bug.cgi?id=4712
https://java.net/bugzilla/show_bug.cgi?id=4713
https://java.net/bugzilla/show_bug.cgi?id=4799

8c4e36c covered bug 4706


[v2] Comparison var o = {}; o.constructor === Object; wrongly evaluates to false

When using Miniflare v2.0.0-rc.2, the following condition inside my worker script evaluates to false:
const obj = {};
obj.constructor === Object // false, but should be true
obj.constructor.name === 'Object' // true

This problem does not occur with v1.4.1

Wooah! thanks @mrbbot for those details. That makes sense.
Regarding your last question, maybe this particular case doesn't deserve a new configuration flag. But in the future, if you get more feedback about it, a new config param might be worth it.
In our case, we have a function to check if a given variable is a "plain" object (with Object prototype) or not, that looks like this:
function isObject(obj) {
  return obj !== null && typeof obj === 'object' && obj.constructor === Object;
}

but it is working fine now, just by changing the last part to obj.constructor.name === 'Object'
Thanks again for the quick response , and feel free to close the ticket.


AEROGEAR-8027 - send client info when audit logging enabled

With this PR, GraphQL requests now have the client info (client id, app info, device info) in the extensions part of a request. That's only done when auditLogging is enabled.
The code that's collecting the client info is made reusable.
Verification instructions:
NOTE: Device/app/client info can't be collected in the browser. You need to verify using Cordova.
1. Voyager server
Get https://github.com/aerogear/apollo-voyager-server/pull/28
Execute
npm install
npm run bootstrap
npm run compile
node examples/auditLogging/server.js

This will start the audit logging example where some audit logs are written to stdout with every request.
2. JS SDK
Gotta build and link the JS SDK.
 Sample app
Get https://github.com/aliok/sync-ionic-example-auditing
Edit src/app/tab1/tab1.page.ts and change the ip with your local ip.
const uri = 'http://<IP_ADDRESS>:4000/graphql';

Execute
npm link @aerogear/datasync-js
ionic build

# in my machine ionic cordova run android doesn't work. so I do it manually
ionic cordova build android
adb install platforms/android/app/build/outputs/apk/debug/app-debug.apk

4. Final steps
When the Cordova app starts, you will see a button called "Voyager" at the top.
Click on that one.
That button will initiate a Graphql request. And in this sample app, auditLogging=true is passed to JS SDK.
You will see some logs in stdout in Voyager server similar to this one (important part is the clientInfo field):
{
  "level": 30,
  "time":<PHONE_NUMBER>476,
  "pid": 11889,
  "hostname": "localhost.localdomain",
  "tag": "AUDIT",
  "msg": "",
  "operationType": "query",
  "fieldName": "hello",
  "parentTypeName": "Query",
  "path": "hello",
  "success": true,
  "arguments": {},
  "clientInfo": {
    "clientId": "848d2a10-0505-11e9-888f-8d166149101a",
    "timestamp":<PHONE_NUMBER>843,
    "data": {
      "app": {
        "appId": "io.ionic.starter",
        "appVersion": "0.0.1",
        "sdkVersion": "2.0.0",
        "framework": "cordova"
      },
      "device": {
        "platform": "android",
        "platformVersion": "9",
        "device": "Android SDK built for x86_64"
      }
    }
  },
  "v": 1
}

Ready for code review and verification guys.
@danielpassos @wtrocki @darahayes

Just wondering if we should add a default value for auditLogging in SyncConfig

Just wondering if we should add a default value for auditLogging in SyncConfig

Sure, doing it now. Default false ;)

I got this warning but we can fix it later
```WARNING in Circular dependency detected:
../../Development/Projects/AeroGear/aerogear-js-sdk/packages/core/dist/Core.js -> ../../Development/Projects/AeroGear/aerogear-js-sdk/packages/core/dist/metrics/index.js -> ../../Development/Projects/AeroGear/aerogear-js-sdk/packages/core/dist/metrics/MetricsService.js -> ../../Development/Projects/AeroGear/aerogear-js-sdk/packages/core/dist/Core.js```

PS: We should add this on apollo-voyager-ionic-example


Revert "Added some bot responses and snowpack dev"

Reverts kanha-43/Add-a-javascipt-bot#11

Which issue you are solving, mention it first

fixes #10


Hard dependency on "sweetalert2" package

https://github.com/zenflow/sweetalert2-react/blob/c23ec32540cf13d0d650cd75327f070bf71efefc/src/SweetAlert.js#L3
https://github.com/zenflow/sweetalert2-react/blob/c23ec32540cf13d0d650cd75327f070bf71efefc/package.json#L64-L70
More explanation later if needed

It can be a peer dep with sweetalert2 marked as an "external" dependency in Webpack, so it's not included in the library build. Is that the problem here?

No, there is no library build here.
Issue is that swal is always require('sweetalert2'), and in order to use this with sweetalert2-react-content we need to be able to redefine swal as require('sweetalert2-react-content')(require('sweetalert2'))
Should be an easy change to support optionally passing in your own swal instance.

@toverux Sorry, yes, sweetalert2 being a regular dependency instead of a peer dependency is part of the problem.


Update README.md

Scan submitted to Checkmarx

Checkmarx SAST - Scan Summary & Details
Cx-SAST Summary
Total of 19 vulnerabilities 15 High 4 Medium 0 Low 0 Info
Violation Summary
No policy violation found


Gemnasium Broken

Gemnasium badges are all returning "invalid" at the moment. I'm not sure if this is you guys or gemnasium, just figured I'd make it known! :sparkling_heart:

Thanks for pointing it out! It should be fixed, thanks to this https://github.com/badges/shields/pull/346

Awesome, thanks guys for the great work!


Add sources tab to lavalink API documentation.

Add sources tab to lavalinks API documentation pages.

feel free to reopen if you feel like working on this again


Signal Sound for Terminal Command Completed

Today we only have signal sounds for the following terminal events.

Terminal Bell
Terminal Quick Fix
Terminal Command Failed

Just like we have signal sounds for task failed and completed. Is it possible to also add signal sound for terminal command completed?

/duplicate #178989


fix(AMBER-222): fix PriceRange input allowing for 0-0 price range [BLOCKED]

The type of this PR is: Fix

This PR solves AMBER-222
Blocked by https://github.com/artsy/force/pull/12894 due to Palette versions
Description
Pulls in @anandaroop's fix for the PriceRange component in palette: https://github.com/artsy/palette/pull/1313

https://github.com/artsy/force/assets/1389948/dff3ed93-2eba-430f-85c9-562139d7af37

Oh hey I already merged the bot PR that included this Palette bump. It looked safe to me, but lmk if you think otherwise.

oh thanks! yeah i think so too after looking at https://github.com/artsy/force/pull/12894, but didn't follow up on grepping for that Pill variant, thanks for that, closing!


refs #2092; port copySessionDescription to all relevant examples; tin…

…y fixes codestyle in api.go file;
Description
I've updated all relevant jsfiddles in the example directory.
Tiny fixes codestyle in api.go. Move default initialize before executing option callbacks.
refs #2092 ticket;

Great work @r-novel, awesome job!
Mind sending over the With.. as a separate change. Will approve and merge right away :) glad to make the code more consistent and cleaner.
Would you mind also updating the README.md of the examples you updated?

Hello, @Sean-Der ! Thank you so much for your code review. I've added unit tests and updated README files.

Great job @r-novel merging!
Some of the examples didn't have log defined so I switched to console.log. I also updated all the jsfiddle links with your new code, thanks for doing this :)


Add origins:scoreboard and origins:command conditions

Ideas for:

Added origins:scoreboard and origins:command conditions
Set the default permission level to 4
Added != comparison (as it is useful in the case of origins:scoreboard)

Could you post examples of how one would use these conditions? :)

Well the idea is to more strongly connect functions and powers together, and to massively increase the potential of conditions overall.
First, the scoreboard condition. Let's say you have a power defined with this condition:
{
    "condition": "origins:scoreboard",
    "objective": "obj",
    "comparison": ">",
    "compare_to": 3
}

and this scoreboard objective:
/scoreboard objectives add obj dummy
this condition would only be true if the player's scoreboard objective was greater than 3, which can be modified in many different ways (unlike resources, which can only be modified in one way, and are untouchable by functions):
/scoreboard players set @s obj 4
/scoreboard players add @s obj 4
/scoreboard players remove @s obj 4
/scoreboard players operation @s obj <operation> <Player> <otherObjective>
If the player does not have the scoreboard objective (that is, if /scoreboard objectives add obj dummy has been run and none of the above commands were run), this condition would always return false (even if != is run). You can then use the != comparison in combination with the == comparison to test if the player does not have this objective set (for example if a player has newly joined a world or had their objectives reset).
Second, the command condition. It can do everything the scoreboard condition can do (though having the scoreboard objective is better because it can run instantly) and more. Every command in minecraft returns an integer; 1 on success and 0 on failure. Some commands even return numerical values. Take for example the /execute if subcommand. You can use this both to test for entites, blocks, storages, custom predicates and also count how many things exist in the world. Another example is the /clear @s <item> 0 command, which counts how many items a player has.
Take for example this command:
/execute if entity @e[distance=..3,type=creeper,nbt={powered:1b}]
This returns the number of supercharged creepers within a three block radius of the player. So lets define a condition like this:
{
    "condition": "origins:command",
    "command": "execute if entity @e[distance=..3,type=creeper,nbt={powered:1b}]",
    "comparison": ">",
    "compare_to": 1
}

This condition would only be true if there is more than 1 supercharged creeper near the player.
Another thing to remember is that these should support modded commands too. Imagine using something like the /script command from scarpet, which can do almost anything. The power of these conditions ultimately rests upon the already in-place powerful minecraft command system; this one condition introduces so many other conditions (including the objective one) it's crazy.

Merged this manually into feature branch: https://github.com/apace100/origins-fabric/commit/62150ce54835a6fc1e5151e87352890d3316cae0


Cant receive DMX : timer is not running (esp-idf v5.0.2)

I have upgraded my project from esp-idf v4.4.4 upgraded  to v5.0.2 so that I could use some new networking featrues I require. IN doing so I have changed from esp_dmx V2.0.2 to V3.0.3-beta as I needed compatability with esp-idf V5 and above.
However when I run a loop to recieve DMX I get repeated errors about the timer not running:
They look to be running about DMX_TIMEOUT_TICK interval. If I actually connect a DMX source then I get the same error message repeated for what looks like every byte received. This is quickly followed by the watch dog timer error and the ESP halts.
I'm building with PlatformIO under VSCode, its my own custom PCB with an ESP-32-WROVER
[env:esp-wrover-kit]
platform = espressif32
board = esp-wrover-kit
framework = espidf

build_type = debug
board_upload.flash_size = 16MB
board_upload.maximum_ram_size = 532480
board_upload.maximum_size = 16777216

board_build.partitions = partitions.csv

monitor_filters = esp32_exception_decoder
monitor_speed = 115200
monitor_rts = 0
monitor_dtr = 0

PACKAGES:
 - framework-espidf @ 3.50002.230601 (5.0.2)
 - tool-cmake @ 3.16.4
 - tool-esptoolpy @ 1.40501.0 (4.5.1)
 - tool-mkfatfs @ 2.0.1
 - tool-mklittlefs @ 1.203.210628 (2.3)
 - tool-mkspiffs @ 2.230.0 (2.30)
 - tool-ninja @ 1.9.0
 - toolchain-esp32ulp @ 1.23500.220830 (2.35.0)
 - toolchain-xtensa-esp32 @ 11.2.0+2022r1

I have built a small test project to make sure there is nothing else in my main project code thats causing an issue. Pretty much a direct copy of the sample code from the documentation.
#include "esp_dmx.h"
#include "stddef.h"
#include "log.h"

#define DMX_RX_UART DMX_NUM_1
#define DMX_IN_TX 5
#define DMX_IN_RX 19
#define DMX_IN_RTS 18

extern "C" void app_main()
{
 LOG_I("Starting Main");

 uint8_t rx_buffer[DMX_PACKET_SIZE];

 dmx_set_pin(DMX_RX_UART, DMX_IN_TX, DMX_IN_RX, DMX_IN_RTS);
 dmx_driver_install(DMX_RX_UART, DMX_DEFAULT_INTR_FLAGS);

 dmx_packet_t packet;

 LOG_I("Installed Driver");

 while(1) {
  if (dmx_receive(DMX_RX_UART, &packet, DMX_TIMEOUT_TICK)) {

   // Check that no errors occurred.
   if (packet.err == ESP_OK) {
    dmx_read(DMX_RX_UART, rx_buffer, packet.size);
   } else {
    LOG_I("An error occurred receiving DMX!");
   }

  } else {
   LOG_I("Timed out waiting for DMX.");
  }

 }
}

I (29) boot: ESP-IDF 5.0.2 2nd stage bootloader
I (29) boot: compile time 07:55:20
I (29) boot: chip revision: v3.0
I (32) boot.esp32: SPI Speed      : 40MHz
I (37) boot.esp32: SPI Mode       : DIO
I (41) boot.esp32: SPI Flash Size : 16MB
I (46) boot: Enabling RNG early entropy source...
I (51) boot: Partition Table:
I (55) boot: ## Label            Usage          Type ST Offset   Length
I (62) boot:  0 nvs              WiFi data        01 02 00009000 00015000
I (70) boot:  1 otadata          OTA data         01 00 0001e000 00002000
I (77) boot:  2 ota_0            OTA app          00 10 00020000 005f0000
I (85) boot:  3 ota_1            OTA app          00 11 00610000 005f0000
I (92) boot:  4 spiffs           Unknown data     01 82 00c00000 00400000
I (100) boot: End of partition table
I (104) esp_image: segment 0: paddr=00020020 vaddr=3f400020 size=0a244h ( 41540) map
I (127) esp_image: segment 1: paddr=0002a26c vaddr=3ffb0000 size=01efch (  7932) load
I (131) esp_image: segment 2: paddr=0002c170 vaddr=40080000 size=03ea8h ( 16040) load
I (140) esp_image: segment 3: paddr=00030020 vaddr=400d0020 size=18eb4h (102068) map
I (179) esp_image: segment 4: paddr=00048edc vaddr=40083ea8 size=08ac8h ( 35528) load
I (200) boot: Loaded app from partition at offset 0x20000
I (200) boot: Disabling RNG early entropy source...
I (211) cpu_start: Pro cpu up.
I (211) cpu_start: Starting app cpu, entry point is 0x40081c60
I (0) cpu_start: App cpu up.
I (226) cpu_start: Pro cpu start user code
I (226) cpu_start: cpu freq: 160000000 Hz
I (226) cpu_start: Application information:
I (230) cpu_start: Project name:     test
I (235) cpu_start: App version:      1
I (240) cpu_start: Compile time:     Jun 16 2023 07:54:49
I (246) cpu_start: ELF file SHA256:  299e75ec48a7c4c6...
I (252) cpu_start: ESP-IDF:          5.0.2
I (256) cpu_start: Min chip rev:     v0.0
I (261) cpu_start: Max chip rev:     v3.99
I (266) cpu_start: Chip rev:         v3.0
I (271) heap_init: Initializing. RAM available for dynamic allocation:
I (278) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (284) heap_init: At 3FFB2808 len 0002D7F8 (181 KiB): DRAM
I (290) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (297) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (303) heap_init: At 4008C970 len 00013690 (77 KiB): IRAM
I (311) spi_flash: detected chip: generic
I (314) spi_flash: flash io: dio
I (319) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (329) main.cpp: Line 13: app_main(): Starting Main
I (339) main.cpp: Line 22: app_main(): Installed Driver
E (1589) gptimer: gptimer_stop(389): timer is not running
I (1589) main.cpp: Line 35: app_main(): Timed out waiting for DMX.
E (2839) gptimer: gptimer_stop(389): timer is not running
I (2839) main.cpp: Line 35: app_main(): Timed out waiting for DMX.
E (4089) gptimer: gptimer_stop(389): timer is not running
I (4089) main.cpp: Line 35: app_main(): Timed out waiting for DMX.
E (5339) gptimer: gptimer_stop(389): timer is not running
I (5339) main.cpp: Line 35: app_main(): Timed out waiting for DMX.
E (6589) gptimer: gptimer_stop(389): timer is not running
I (6589) main.cpp: Line 35: app_main(): Timed out waiting for DMX.
E (7839) gptimer: gptimer_stop(389): timer is not running

Thanks for reaching out and thank you for your patience on this. I'll try to recreate this issue. I'm guessing it has something to do with the v5.0.2 changes because I believe I am still running v5.0.0!

Indeed, it appears that this is a v5.0.0 vs. v5.0.2 issue. I was able to replicate the problem. The GPTimer is a hardware timer that is used for various purposes including sending and receiving DMX and RDM packets. In IDF v5.0.0, it appears we were able to call the function to stop the GPTimer even if the timer wasn't running. In v5.0.2, a log message is now generated when stopping an already stopped timer.
I have implemented a fix which adds a timer_is_running flag to the DMX driver. Before stopping the timer, the flag is checked. This fix appears to be working. I will hopefully have a patch soon!

This is fixed in the dev branch. Let me know if you have any other issues!

Many thanks for a quick investigation and fix.
I shall download the dev branch and test.


HiroSystems.Clarinet version 0.15.0

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Uhm wut, not sure what happened here, can you delete /manifests/h/HiroSystems/Clarinet/1.15.0 which i submitted with the wrong version?


Notebook for cdc-sql

Ingest data from MONGODB® to SingleStore using SQL commands.

Rather than all of the USE commands, you should show an "Action Required" box that tells the user to select the database in the drop-down at the top of the notebook (see https://github.com/singlestore-labs/spaces-notebooks/blob/master/notebooks/getting-started-with-notebooks/notebook.ipynb for an example).
Also, you should look over the SQL commands in the notebook and make the keyword casing more consistent (all upper-case). It starts off that way, but later in the notebook the casing is not consistent.

@kesmit13 I knowingly added "use" in all commands so that user doesnt run into the error, will this force into user selecting database before proceeding

The coding style that's been established in all of the other notebooks is to select the database at the top of the notebook. I think we should stick to it so they are consistent. The error won't happen if they select the database at the top. It also ensures that any connections make directly with the Python client, SQLAlchemy, or Ibis also work correctly.

The coding style that's been established in all of the other notebooks is to select the database at the top of the notebook. I think we should stick to it so they are consistent. The error won't happen if they select the database at the top. It also ensures that any connections make directly with the Python client, SQLAlchemy, or Ibis also work correctly.

on it make changes

@sudo-savitaj I'll be here tomorrow, and all this week. So I can still approve when it comes through.

@sudo-savitaj I'll be here tomorrow, and all this week. So I can still approve when it comes through.

cool thank you so much.. Ill push changes post testing once cluster creation issue is resolved

@kesmit13 pushed suggested changes, please review PR.

Everything looks good as far as content goes. It looks like the pre-commit checks are failing though. Can you install and run those (see https://github.com/singlestore-labs/spaces-notebooks/blob/master/CONTRIBUTING.md)?


ValvePosition attribute of EveCluster Cluster

Add ValvePosition attribute of EveCluster Cluster for Eve Thermo.

Could you add a PR Label?


refractor input parsing to be modular and assert correct inputs

@Askaholic discussed in other issue tickets to remove duplication on shared input arguments and their syntax between different programs.
In addition we discussed capturing driver errors early one.  So, should we also implement a verification that input arguments are supplied correctly, directly after parsing?

e.g. assumption on date order for seasonality?
e.g. verifying dates and time are in correct and recognizable format

Lets discuss below.

I think a lot of the input validation can be done in the parsing stage directly by supplying parsing functions through the type parameter.

Would you recommend a single location for this for the parser and checks? https://github.com/dbekaert/RAiDER/blob/a3548e9f23d6280629f14bd2a15b75bcb5263f35/tools/RAiDER/statsPlot.py#L23
Few examples for discussion stimulation:

This does not enforce the date formatting to be checked in the parser: https://github.com/dbekaert/RAiDER/blob/a3548e9f23d6280629f14bd2a15b75bcb5263f35/tools/RAiDER/downloadGNSSDelays.py#L72

The checking of the bounding box is done in the code and not in the parser. Parser is requiring a string. @Askaholic i assume you want to do this as part of the parser? https://github.com/dbekaert/RAiDER/blob/a3548e9f23d6280629f14bd2a15b75bcb5263f35/tools/RAiDER/downloadGNSSDelays.py#L326-L337

this does not check the order of the dates or the formating as part of the parser: https://github.com/dbekaert/RAiDER/blob/a3548e9f23d6280629f14bd2a15b75bcb5263f35/tools/RAiDER/statsPlot.py#L70

Yea I'm thinking some directory structure like this:
RAiDER/
    cli/
        __init__.py
        validators.py
        parser.py

We would put the parameter parsing functions in validators.py. I think you're pretty much free to do as much input validation as you want in the parsing functions even checking that files exist etc: https://docs.python.org/3/library/argparse.html#type

@dbekaert do you want to close this issue re #82 or leave it open for further work? @Askaholic notes that more could be done.

Don't close it, it's far from being done.


What is the default QUIC version

Hello, I was running the http_client script "./http_client -H www.example.com -s <IP_ADDRESS>:4433 -p /". I just want to know what the default QUIC version is because I am able to toggle between the different Google QUIC versions but I am not switch between the different IETF versions.
Also what is the command for the IETF versions? version=ID-32?

The default version is the latest non-experimental version (see the header file).
To select h3-29, use -o version=h3-29

The default version is the latest non-experimental version (see the header file).
To select h3-29, use -o version=h3-29

Thank you for the quick reply. That worked but I have a follow up question. When using Google QUIC, the client side is in a connected loop and the server displays this
21:12:33.000 [NOTICE] Document root is not set: start in Interop Mode
21:12:38.634 [WARN] mini-conn: enc hist IKObj; User-Agent: lsquic/2.26.1
21:12:38.634 [WARN] [QUIC:A01ECB088B45DAA3] mini-conn: destroyed.  Diagnostics: conn flags: 0x420008, mc flags: 0xD0, incoming-history (trunc: 0) ����U, received: 1, sent: 0, lost: 0, deferred: 0, still-deferred: 0, dropped: 0, in-flight: FFFFFFFFFFFFFFFF, acked: 0, error_code: 0x0, ticks: 1, pack size: 1370, lifetime: 578 usec, version: Q050, mc hist: CSKHe
Is there a way to disable the loop?

Thank you for the quick reply. That worked but I have a follow up question. When using Google QUIC, the client side is in a connected loop and the server displays this
21:12:33.000 [NOTICE] Document root is not set: start in Interop Mode
21:12:38.634 [WARN] mini-conn: enc hist IKObj; User-Agent: lsquic/2.26.1
21:12:38.634 [WARN] [QUIC:A01ECB088B45DAA3] mini-conn: destroyed.  Diagnostics: conn flags: 0x420008, mc flags: 0xD0, incoming-history (trunc: 0) ����U, received: 1, sent: 0, lost: 0, deferred: 0, still-deferred: 0, dropped: 0, in-flight: FFFFFFFFFFFFFFFF, acked: 0, error_code: 0x0, ticks: 1, pack size: 1370, lifetime: 578 usec, version: Q050, mc hist: CSKHe
Is there a way to disable the loop?

I cannot reproduce this.  How do you run the client?  (And what do you mean by the "connected loop?")

I cannot reproduce this.  How do you run the client?  (And what do you mean by the "connected loop?")

Here is the example I ran:
With success:
After starting the server, it says:
09:37:04.575 [NOTICE] Document root is not set: start in Interop Mode
Client:
Script: ./http_client -H www.example.com -s <IP_ADDRESS>:4433 -p / -o version=h3-29
and returns
HTTP/1.1 200 OK
server: lsquic/2.26.1
content-type: text/html
content-length: 940

LiteSpeed IETF QUIC Server Index Page

LiteSpeed IETF QUIC Server Index Page
Hello!  Welcome to the interop.  Available services:

POST to /cgi-bin/md5sum.cgi.  This will return
                   MD5 checksum of the request body.
           GET /123K or GET /file-123K.  This will return
                   requested number of payload in the form of repeating text
                   by Jerome K. Jerome.  The size specification must match
                   (\d+)[KMG]? and the total size request must not exceed
                   2 gigabytes.  Then, you will get back that many bytes
                   of the beloved classic.

Time Taken:
0.018651962280273438
With google QUIC
Server returns
09:37:04.575 [NOTICE] Document root is not set: start in Interop Mode
09:39:53.645 [WARN] mini-conn: enc hist IKObj; User-Agent: lsquic/2.26.1
09:39:53.645 [WARN] [QUIC:F1D4EB0795C58B95] mini-conn: destroyed.  Diagnostics: conn flags: 0x420008, mc flags: 0xD0, incoming-history (trunc: 0) ؊�grU, received: 1, sent: 0, lost: 0, deferred: 0, still-deferred: 0, dropped: 0, in-flight: FFFFFFFFFFFFFFFF, acked: 0, error_code: 0x0, ticks: 1, pack size: 1370, lifetime: 604 usec, version: Q050, mc hist: CSKHe
client
Script: ./http_client -H www.example.com -s <IP_ADDRESS>:4433 -p / -o version=Q050
returns
Time Taken:
10.005310535430908
Reason I am asking this is because I want to be able to measure time for the two different variants and the Google QUIC version is taking a lot longer than it should.

Here is the example I ran:
With success:
After starting the server, it says:
09:37:04.575 [NOTICE] Document root is not set: start in Interop Mode
Client:
Script: ./http_client -H www.example.com -s <IP_ADDRESS>:4433 -p / -o version=h3-29
and returns
HTTP/1.1 200 OK
server: lsquic/2.26.1
content-type: text/html
content-length: 940

LiteSpeed IETF QUIC Server Index Page

LiteSpeed IETF QUIC Server Index Page
Hello!  Welcome to the interop.  Available services:

POST to /cgi-bin/md5sum.cgi.  This will return
                   MD5 checksum of the request body.
           GET /123K or GET /file-123K.  This will return
                   requested number of payload in the form of repeating text
                   by Jerome K. Jerome.  The size specification must match
                   (\d+)[KMG]? and the total size request must not exceed
                   2 gigabytes.  Then, you will get back that many bytes
                   of the beloved classic.

Time Taken:
0.018651962280273438
With google QUIC
Server returns
09:37:04.575 [NOTICE] Document root is not set: start in Interop Mode
09:39:53.645 [WARN] mini-conn: enc hist IKObj; User-Agent: lsquic/2.26.1
09:39:53.645 [WARN] [QUIC:F1D4EB0795C58B95] mini-conn: destroyed.  Diagnostics: conn flags: 0x420008, mc flags: 0xD0, incoming-history (trunc: 0) ؊�grU, received: 1, sent: 0, lost: 0, deferred: 0, still-deferred: 0, dropped: 0, in-flight: FFFFFFFFFFFFFFFF, acked: 0, error_code: 0x0, ticks: 1, pack size: 1370, lifetime: 604 usec, version: Q050, mc hist: CSKHe
client
Script: ./http_client -H www.example.com -s <IP_ADDRESS>:4433 -p / -o version=Q050
returns
Time Taken:
10.005310535430908
Reason I am asking this is because I want to be able to measure time for the two different variants and the Google QUIC version is taking a lot longer than it should.

Are you using Linux?  I wonder how you can specify -s <IP_ADDRESS>:4433 to the client -- I wouldn't expect it to work.

Are you using Linux?  I wonder how you can specify -s <IP_ADDRESS>:4433 to the client -- I wouldn't expect it to work.

I am using Linux. It is not <IP_ADDRESS> it is an actual IP. Sorry, I know it is an internal IP I guess I am just paranoid.

I am using Linux. It is not <IP_ADDRESS> it is an actual IP. Sorry, I know it is an internal IP I guess I am just paranoid.

Well, I cannot reproduce what you are seeing.  Could you please specify exact steps?

Well, I cannot reproduce what you are seeing.  Could you please specify exact steps?


python310Packages.tornado_5: 5.1.1 -> 6.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.tornado_5 is: "A web framework and asynchronous networking library"
meta.homepage for python310Packages.tornado_5 is: "https://www.tornadoweb.org/"
meta.changelog for python310Packages.tornado_5 is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 6.1 with grep in /nix/store/fmddpmdm699cfkfn944nlxwb80460byl-python3.10-tornado-6.1

found 6.1 in filename of file in /nix/store/fmddpmdm699cfkfn944nlxwb80460byl-python3.10-tornado-6.1

Rebuild report (if merged into master) (click to expand)

8 total rebuild path(s)

4 package rebuild(s)

4 x86_64-linux rebuild(s)
4 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
python310Packages.graphite_beacon
python310Packages.tornado_5
python39Packages.graphite_beacon
python39Packages.tornado_5

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/fmddpmdm699cfkfn944nlxwb80460byl-python3.10-tornado-6.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.tornado_5 https://github.com/r-ryantm/nixpkgs/archive/13b2c958ccaeb9e114a8016979076ad0e09557f0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/fmddpmdm699cfkfn944nlxwb80460byl-python3.10-tornado-6.1
ls -la /nix/store/fmddpmdm699cfkfn944nlxwb80460byl-python3.10-tornado-6.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages failed to build:

python310Packages.graphite_beacon
python39Packages.graphite_beacon

2 packages built:

python310Packages.tornado_5
python39Packages.tornado_5

Maintainer pings

@r-ryantm same for this package

@SuperSandro2000 torando should be skipped now https://github.com/ryantm/nixpkgs-update/commit/58c5426a5b122d7bc6a5bf456cd9e9baf43eacd7


kojiapi: logs for a compose

Currently there is no way to get the logs for a compose. As long as we can't stream logs, the easiest would probably to just have a compose/id/logs with a json object containing the overall log and the logs for the individual images.
Or we could pretend to support streaming of Logs already, and combine that API with the wait-for-compose to finish API and stream json-line/json-records that could contain log updates.

The first solution sounds the simpler with what we currently have, and I also think the most relevant. Streaming logs for this isn't all that important, is it? I would be happy to add that too once we have the infrastructure though.
For now, let's add compose/id/logs as you proposed, which I think would be good to have even if we one day get streaming logs, no?

Yes, totally agree, sounds all good to me!


Add rush to the list of services that fail on startup if cassandra ca…

…n't be reached.
@tomaslin please review.

@tomaslin PTAL.


Rails 5 compatiblity

The stable release is limited to Rails 3 and the release candidate to Rails 4. Now that Rails 5 is the stable version, this gem cannot be used anymore.
Can you please fix this @lwe ?

Hey Guillaume, currently my time for working on entypo-rails is a bit
limited, sorry. However I'd happily merge a pull-request that provides
Rails 5 compatibility.
On Sun, Nov 27, 2016 at 1:32 PM Guillaume Hain<EMAIL_ADDRESS>wrote:

The stable release is limited to Rails 3 and the release candidate to
Rails 4. Now that Rails 5 is the stable version, this gem cannot be used
anymore.
Can you please fix this @lwe https://github.com/lwe ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lwe/entypo-rails/issues/19, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFGzI3kwljr7AfpY7cTj1GcZDoIDkMHks5rCXhxgaJpZM4K9ItL
.

I believe it's just a simple change in the gemspec.  https://github.com/lwe/entypo-rails/pull/20 fixed the issue for me.

You are right @piedoom, but actually not specifying the upper version of Rails doesn't sound good to me (while it will work). Usually we set lower to 6.0 or lower or equal to 5.1. (Just to mention it :))

And thank you by the way :)

Thanks @zedtux - I'm admittedly not a pro when it comes to Rubygems.  Is setting the version < 6.0 to avoid conflicts with a new (potentially breaking) version of Railties?

Good idea, will add this 👍 release will follow next week
On Fri, 23 Dec 2016 at 15:02 Alexander Lozada<EMAIL_ADDRESS>wrote:

Thanks @zedtux https://github.com/zedtux - I'm admittedly not a pro
when it comes to Rubygems. Is setting the version < 6.0 to avoid conflicts
with a new (potentially breaking) version of Railties?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lwe/entypo-rails/issues/19#issuecomment-268994214,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFGzKudFHc-kMbB3TBuvWFGrjxpE9CMks5rK9RbgaJpZM4K9ItL
.

Yes @piedoom, it's just to avoid having people installing the current version with Rails 6 (when it will be out) while this version is maybe not compatible with.

Thanks for all the work :tada:, just pushed version 3.0.0 to rubygems.
https://rubygems.org/gems/entypo-rails


multipart: proposal for concurrent API

About this change - What it does

New API to perform multipart uploads allowing concurrent & non-monotonic upload of chunks.
Why this way

Currently we do use multipart uploads to upload big files, but this is done serially by uploading one part at a time in monotonically increasing order. This limits concurrency.
In order to allow the clients to upload multiple parts concurrently we need to somehow expose the concept of a concurrent upload.
The API proposed here still hides a little bit of details.
limits
As far as I can tell the limits for the different providers are:

AWS S3: 10k chunks of at most 5 GiB
GCP: 10k chunks of at most 5 GiB
Azure: 50k chunks of at most 4GiB
Swift: 1k+ chunks (depending on configuration) of at most 5GiB

I'm not sure if it makes much sense to do smart stuff to avoid the limits, since they are quite close and I'd expect the clients to be able to be configured for reducing the chunks size...
I'd enforce by default at most 10k chunks and 4GiB per chunk since this should work for AWS/GCP/Azure... swift is configurable so doesn't make much sense to enforce that limit.

Codecov Report
Patch coverage: 85.29% and project coverage change: +1.55 :tada:

Comparison is base (5f65d18) 65.25% compared to head (0a3a225) 66.81%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #128      +/-   ##
==========================================
+ Coverage   65.25%   66.81%   +1.55%
==========================================
  Files          31       31
  Lines        3606     3896     +290
==========================================
+ Hits         2353     2603     +250
- Misses       1253     1293      +40

Impacted Files
Coverage Δ

rohmu/object_storage/base.py
74.59% <61.11%> (-1.60%)
:arrow_down:

rohmu/object_storage/s3.py
69.41% <83.87%> (+5.28%)
:arrow_up:

rohmu/object_storage/local.py
87.95% <86.56%> (-1.58%)
:arrow_down:

rohmu/object_storage/azure.py
36.28% <100.00%> (ø)

rohmu/object_storage/google.py
40.71% <100.00%> (ø)

rohmu/util.py
68.57% <100.00%> (+19.73%)
:arrow_up:

... and 2 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Converted to draft since, after discussions, there is an alternative proposal which should probably be preferred.https://github.com/aiven/rohmu/pull/130

Gone with the other approach


UI Fixes

Fixed #244 #245
Fix for horizontal margin in the content starting side based on the direction of the OBS resource
Fix for snack - sometimes snack bar shows below some of the components ( hiding behind )

@vipinpaul  this PR can be merged. all the changes are reflected.

Snack bar loading in front of all the resource pane and the audio tool bar
Right side of the margin has been increased to the OBS resources in the right and left for both RTL and LTR. and it is looking neet.


Blocking beginTransaction for second write transaction

I'm currently implementing node-lmdb in a web API setting where one request transports data that will be streamed one of many dbis in one environment.
As each request opens a dbi and begins a transaction to stream its data there may be a situation where another requests tries to write its data while the streaming of an earlier request is not finished.
Therefore the second requests attempts to open a write transaction while the first one is still active.
According to the lmdb docs and even the README.md in this very repository, the call to env.beginTxn should block until the first write transaction is committed or aborted.
The current behaviour is that an error is thrown as of #88
This contradicts the documentation and makes it difficult to use node-lmdb for multiple async write approaches.
Would it make sense to make beginTransaction return a Promise that resolves with a write transaction as soon as it is valid to create one?
If so I would attempt to implement this and send a PR.
If not, what is the correct solution to this problem?

Make all write attempts synchronious so that there cannot be a second simoultaneous attemp? That is not feasable in an Web API, at least not on the same node process.

Manage the write transaction in a singleton service and provide it to each writer? That would mean to keep it alive indefinetly. How to handle commiting/renweing then?

@braydonf You reported #88 and wrote about a async queue to handle this problem here
Could you post how you solved this, or if the promise approach I suggested would solve your use case as well?
Thanks

What error are you getting? Are you running the other transaction in a different process (or thread)? A write beginTransaction has always blocked for me when there is another ongoing write transaction in another process. And this is the implicit assumption of the documentation as well, it is self-defeating to block a write transaction in the same thread/process as the ongoing transaction since it would block any further execution that would permit the first to finish (and I think lmdb does detect and throw an error for that). And since, normally all JS runs in same thread, you need a second process (or thread, but I don't think node-lmdb supports node worker threads yet) to observe this.
If you want the transaction to run in a separate thread from your javascript, this is exactly why the new batchWrite method was introduced:
https://github.com/Venemo/node-lmdb#asynchronous-batched-operations
This will run your transactions in separate threads, and if you use this as the primary way to write data, any blocking will be a separate thread and won't block your JS.
If you are interested in the best strategy for managing different writes and using these separate threaded transactions, I wrote a description of how lmdb-store uses batched writes (with promises) here https://dev.doctorevidence.com/lmdb-in-node-29af907aad6e.

I agree with what Kris said. Also would be useful to see what error you are getting, and maybe give us a small example to reproduce the problem.

Sorry I linked the wrong PR, I meant to link this commit
The error message is "You have already opened a write transaction in the current process, can't open a second one."
I expected that beginTransaction would block while another WriteTransaction is active, but of course this only works when called on another thread or process
Thanks for your explanation @kriszyp
I am trying to stream binary data from multiple streams to different dbis in the same environment at the same time. Using one transaction for each chunk is very imperformant but its very complicated to batch chunks of different streams together.
If will have a closer look at https://github.com/DoctorEvidence/lmdb-store

Just another idea:
would it make sense to have a beginTransactionAsync  method that returns a promise which will either resolve immediatelly if no other write transaction is currently active, or resolve as soon as the current write transaction is unset ?

Pull requests are welcome in that area, if you wish to improve it.