In topi schedules, for OpenCL target, num_thread == 512 is too big for AMD GPU

Hi, I noticed that some schedules in topi.cuda use thread block of size 512 in one dimension.
For example, here and here.
512 threads in one dimension is too big for AMD OpenCL, because their driver restricts the maximum  blockDim.x to be 256. According to the discussion here, this restriction is due to their implementation of driver, not their hardware itself.
I understand that schedules in topi/cuda are optimized for CUDA, but if I change the 512 threads to 256 threads, all tests in https://github.com/dmlc/tvm/tree/master/topi/tests/python pass for my AMD gpu as well. I was also able to run tutorials and resnet benchmark from the latest nnvm, on AMD gpu.
So, unless there is a strong reason to use thread block of size 512, is it possible to change 512 to 256?
Of course, if you are planning to implement dedicated schedules for opencl, that's better :)

I agree that we should have dedicated schedules for devices(e.g. AMD). We will do a upgrade of the topi interface to make plugin of device-specific schedule easy

On firefly3399, I can pass all unittests except test_topi_conv2d_nchw.py which complains running out of resources. I probably need to reduce # of threads to a smaller number.
Just curious, are you able to run NNVM benchmark with OpenCL on amd platform?
NNVM benchmark has 2 benchmarks, one is for rasp and the other one is for cuda. There is no Opencl benchmark
Thanks,

@kaishijeng yes, I modified cuda bench to use opencl target and context. I also modified nnvm internal to enable opencl target.
Proof:
$python cuda_imagenet_bench.py --model resnet
[09:38:49] src/runtime/opencl/opencl_device_api.cc:195: Initialize OpenCL platform 'AMD Accelerated Parallel Processing'
[09:38:49] src/runtime/opencl/opencl_device_api.cc:215: opencl(0)='gfx803' cl_device_id=0x7f85d4dc46f0
benchmark args: Namespace(model='resnet', num_iter=100, opt_level=3)
ProfileResult(mean=0.01073681044)
ProfileResult(mean=0.01073411568)
ProfileResult(mean=0.01073584366)
For test_topi_conv2d_nchw, you can use my modified schedule here

To enable opencl target in nnvm, I changed
if target == "cuda":
to
if target == "cuda" or target == "opencl":
in
python/nnvm/top/nn.py
python/nnvm/top/reduction.py
python/nnvm/top/tensor.py

With your modified schedule, I still got out of resource error below. I
guess firefly3399 has smaller GPU than your amd.
Do you mind to share your modifed cuda_imagenet_bench.py and how you modify
nnvm internal to enable opencl target?
TVM: Initializing cython mode...
Use memoize
topi.tests.test_topi_conv2d.verify_con2d_nchw.get_ref_data.pkl(5, (1, 3,
224, 224), 'float32', 2, 3, (64, 3, 7, 7))
Skip because cuda is not enabled
[17:55:55] src/runtime/opencl/opencl_device_api.cc:195: Initialize OpenCL
platform 'ARM Platform'
[17:55:55] src/runtime/opencl/opencl_device_api.cc:215:
opencl(0)='Mali-T860' cl_device_id=0x7f93fa8dd8
[17:55:55]
/home/firefly/2TB/src/firefly/tvm/dmlc-core/include/dmlc/logging.h:308:
[17:55:55] src/runtime/opencl/opencl_module.cc:223: Check failed: e ==
CL_SUCCESS OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
Stack trace returned 10 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN4dmlc15LogMessageFatalD1Ev+0x44)
[0x7f944bc6e4]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZNK3tvm7runtime17OpenCLWrappedFuncclENS0_7TVMArgsEPNS0_11TVMRetValueEPPv+0xbac)
[0x7f94854b9c]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS1_6detail17PackFuncVoidAddr_ILi4ENS1_17OpenCLWrappedFuncEEENS1_10PackedFuncET0_RKSt6vectorINS6_14ArgConvertCodeESaISC_EEEUlS2_S4_E_E9_M_invokeERKSt9_Any_dataOS2_OS4+0xd0)
[0x7f94854f10]
[bt] (3)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (4) [0x7f9676a9dc]
[bt] (5) [0x7f9676a2ac]
[bt] (6)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZZN3tvm7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS_7runtime10ModuleNodeEEENKUlNSB_7TVMArgsEPNSB_11TVMRetValueEE0_clESG_SI+0x3c)
[0x7f946f2a3c]
[bt] (7)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS0_7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS1_10ModuleNodeEEEUlS2_S4_E0_E9_M_invokeERKSt9_Any_dataOS2_OS4+0x2c)
[0x7f946f2cfc]
[bt] (8)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (9)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/_ffi/_cy2/core.so(+0x180f8)
[0x7f8a5280f8]
[17:55:55]
/home/firefly/2TB/src/firefly/tvm/dmlc-core/include/dmlc/logging.h:308:
[17:55:55] src/codegen/llvm/llvm_module.cc:59: Check failed: ret == 0 (-1
vs. 0) [17:55:55] src/runtime/opencl/opencl_module.cc:223: Check failed: e
== CL_SUCCESS OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
Stack trace returned 10 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN4dmlc15LogMessageFatalD1Ev+0x44)
[0x7f944bc6e4]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZNK3tvm7runtime17OpenCLWrappedFuncclENS0_7TVMArgsEPNS0_11TVMRetValueEPPv+0xbac)
[0x7f94854b9c]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS1_6detail17PackFuncVoidAddr_ILi4ENS1_17OpenCLWrappedFuncEEENS1_10PackedFuncET0_RKSt6vectorINS6_14ArgConvertCodeESaISC_EEEUlS2_S4_E_E9_M_invokeERKSt9_Any_dataOS2_OS4+0xd0)
[0x7f94854f10]
[bt] (3)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (4) [0x7f9676a9dc]
[bt] (5) [0x7f9676a2ac]
[bt] (6)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZZN3tvm7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS_7runtime10ModuleNodeEEENKUlNSB_7TVMArgsEPNSB_11TVMRetValueEE0_clESG_SI+0x3c)
[0x7f946f2a3c]
[bt] (7)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS0_7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS1_10ModuleNodeEEEUlS2_S4_E0_E9_M_invokeERKSt9_Any_dataOS2_OS4+0x2c)
[0x7f946f2cfc]
[bt] (8)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (9)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/_ffi/_cy2/core.so(+0x180f8)
[0x7f8a5280f8]
Stack trace returned 6 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN4dmlc15LogMessageFatalD1Ev+0x44)
[0x7f944bc6e4]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZZN3tvm7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS_7runtime10ModuleNodeEEENKUlNSB_7TVMArgsEPNSB_11TVMRetValueEE0_clESG_SI+0x1f4)
[0x7f946f2bf4]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS0_7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS1_10ModuleNodeEEEUlS2_S4_E0_E9_M_invokeERKSt9_Any_dataOS2_OS4+0x2c)
[0x7f946f2cfc]
[bt] (3)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (4)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/_ffi/_cy2/core.so(+0x180f8)
[0x7f8a5280f8]
[bt] (5) python2(PyObject_Call+0x50) [0x458ee8]
Traceback (most recent call last):
File "./test_topi_conv2d_nchw.py", line 72, in
test_conv2d_nchw()
File "./test_topi_conv2d_nchw.py", line 58, in test_conv2d_nchw
verify_conv2d_nchw(1, 3, 224, 64, 7, 3, 2)
File "./test_topi_conv2d_nchw.py", line 54, in verify_conv2d_nchw
check_device(device)
File "./test_topi_conv2d_nchw.py", line 48, in check_device
func1(a, w, b)
File
"/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/_ffi/function.py",
line 128, in call
return f(*args)
File "tvm/_ffi/_cython/./function.pxi", line 264, in
core.FunctionBase.call
File "tvm/_ffi/_cython/./function.pxi", line 213, in core.FuncCall
File "tvm/_ffi/_cython/./function.pxi", line 205, in core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 131, in core.CALL
tvm._ffi.base.TVMError: [17:55:55] src/codegen/llvm/llvm_module.cc:59:
Check failed: ret == 0 (-1 vs. 0) [17:55:55]
src/runtime/opencl/opencl_module.cc:223: Check failed: e == CL_SUCCESS
OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
Stack trace returned 10 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN4dmlc15LogMessageFatalD1Ev+0x44)
[0x7f944bc6e4]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZNK3tvm7runtime17OpenCLWrappedFuncclENS0_7TVMArgsEPNS0_11TVMRetValueEPPv+0xbac)
[0x7f94854b9c]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS1_6detail17PackFuncVoidAddr_ILi4ENS1_17OpenCLWrappedFuncEEENS1_10PackedFuncET0_RKSt6vectorINS6_14ArgConvertCodeESaISC_EEEUlS2_S4_E_E9_M_invokeERKSt9_Any_dataOS2_OS4+0xd0)
[0x7f94854f10]
[bt] (3)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (4) [0x7f9676a9dc]
[bt] (5) [0x7f9676a2ac]
[bt] (6)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZZN3tvm7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS_7runtime10ModuleNodeEEENKUlNSB_7TVMArgsEPNSB_11TVMRetValueEE0_clESG_SI+0x3c)
[0x7f946f2a3c]
[bt] (7)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS0_7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS1_10ModuleNodeEEEUlS2_S4_E0_E9_M_invokeERKSt9_Any_dataOS2_OS4+0x2c)
[0x7f946f2cfc]
[bt] (8)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (9)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/_ffi/_cy2/core.so(+0x180f8)
[0x7f8a5280f8]
Stack trace returned 6 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN4dmlc15LogMessageFatalD1Ev+0x44)
[0x7f944bc6e4]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZZN3tvm7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS_7runtime10ModuleNodeEEENKUlNSB_7TVMArgsEPNSB_11TVMRetValueEE0_clESG_SI+0x1f4)
[0x7f946f2bf4]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS0_7codegen14LLVMModuleNode11GetFunctionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt10shared_ptrINS1_10ModuleNodeEEEUlS2_S4_E0_E9_M_invokeERKSt9_Any_dataOS2_OS4+0x2c)
[0x7f946f2cfc]
[bt] (3)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(TVMFuncCall+0x74)
[0x7f9482951c]
[bt] (4)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/_ffi/_cy2/core.so(+0x180f8)
[0x7f8a5280f8]
[bt] (5) python2(PyObject_Call+0x50) [0x458ee8]
[17:55:55]
/home/firefly/2TB/src/firefly/tvm/dmlc-core/include/dmlc/logging.h:308:
[17:55:55] src/runtime/workspace_pool.cc:92: Check failed:
allocated_.size() == 1 (3 vs. 1)
Stack trace returned 4 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN3tvm7runtime13WorkspacePool4Pool7ReleaseE9DLContextPNS0_9DeviceAPIE+0x488)
[0x7f94838fc0]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN3tvm7runtime13WorkspacePoolD1Ev+0x48)
[0x7f94837710]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN3tvm7runtime2cl17OpenCLThreadEntryD1Ev+0x18)
[0x7f9484c818]
[bt] (3) /lib/aarch64-linux-gnu/libc.so.6(__call_tls_dtors+0x44)
[0x7f96985474]
terminate called after throwing an instance of 'dmlc::Error'
what():  [17:55:55] src/runtime/workspace_pool.cc:92: Check failed:
allocated_.size() == 1 (3 vs. 1)
Stack trace returned 4 entries:
[bt] (0)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN3tvm7runtime13WorkspacePool4Pool7ReleaseE9DLContextPNS0_9DeviceAPIE+0x488)
[0x7f94838fc0]
[bt] (1)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN3tvm7runtime13WorkspacePoolD1Ev+0x48)
[0x7f94837710]
[bt] (2)
/usr/local/lib/python2.7/dist-packages/tvm-0.1.0-py2.7-linux-aarch64.egg/tvm/libtvm.so(_ZN3tvm7runtime2cl17OpenCLThreadEntryD1Ev+0x18)
[0x7f9484c818]
[bt] (3) /lib/aarch64-linux-gnu/libc.so.6(__call_tls_dtors+0x44)
[0x7f96985474]
Aborted (core dumped)
On Tue, Oct 10, 2017 at 5:46 PM, masahi<EMAIL_ADDRESS>wrote:

@kaishijeng https://github.com/kaishijeng yes, I modified cuda bench to
use opencl target and context. I also modified nnvm internal to enable
opencl target.
Proof:
$python cuda_imagenet_bench.py --model resnet
[09:38:49] src/runtime/opencl/opencl_device_api.cc:195: Initialize OpenCL
platform 'AMD Accelerated Parallel Processing' [09:38:49]
src/runtime/opencl/opencl_device_api.cc:215: opencl(0)='gfx803'
cl_device_id=0x7f85d4dc46f0 benchmark args: Namespace(model='resnet',
num_iter=100, opt_level=3) ProfileResult(mean=0.01073681044)
ProfileResult(mean=0.01073411568) ProfileResult(mean=0.01073584366)
For test_topi_conv2d_nchw, you can use my modified schedule here
https://gist.github.com/masahi/18b4ed1fb8fa1a049ffdba35ed807509
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335648649, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3kpr7vIftAYXZ7FlNEcyTzrgMd_Qks5srA_9gaJpZM4PzdCV
.

This is mainly because AMD card and ARM GPU have different resources and requires different kind of schedules

I change num_thread from your 256 to 32, still got out of resources which
looks strange to me.
I also see num_thread_x = 64 in the code, do I need to chage this variable
too?
Thanks,
On Tue, Oct 10, 2017 at 6:01 PM, Tianqi Chen<EMAIL_ADDRESS>wrote:

This is mainly because AMD card and ARM GPU have different resources and
requires different kind of schedules
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335650776, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3v0WfHr-F5Hx3fDgxcFWPist5RDSks5srBN4gaJpZM4PzdCV
.

@kaishijeng Sorry I don't know anything about ARM gpus. Maybe you can try smaller inputs?
Say,  verify_conv2d_nchw(1, 3, 64, 64, 3, 1, 0) ?
I can push my repo, after I clean it up.

Yes, it works OK with verify_conv2d_nchw(1, 3, 64, 64, 3, 1, 0)
Thanks,
On Tue, Oct 10, 2017 at 6:37 PM, masahi<EMAIL_ADDRESS>wrote:

@kaishijeng https://github.com/kaishijeng Sorry I don't know anything
about ARM gpus. Maybe you can try smaller inputs?
Say, verify_conv2d_nchw(1, 3, 64, 64, 3, 1, 0) ?
I can push my repo, after I clean it up.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335655803, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3rzJGs8Uq8nmRtLKl9f_D8_uTtiwks5srBvDgaJpZM4PzdCV
.

Glad to hear that.
So your CL_OUT_OF_RESOURCES is due to insufficient memory, not the amount of threads.

Can you share your modified cuda_imagenet_bench.py to enable OpenCL?
Thanks,
On Tue, Oct 10, 2017 at 7:04 PM, kaishi Jeng<EMAIL_ADDRESS>wrote:

Yes, it works OK with verify_conv2d_nchw(1, 3, 64, 64, 3, 1, 0)
Thanks,
On Tue, Oct 10, 2017 at 6:37 PM, masahi<EMAIL_ADDRESS>wrote:

@kaishijeng https://github.com/kaishijeng Sorry I don't know anything
about ARM gpus. Maybe you can try smaller inputs?
Say, verify_conv2d_nchw(1, 3, 64, 64, 3, 1, 0) ?
I can push my repo, after I clean it up.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335655803, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3rzJGs8Uq8nmRtLKl9f_D8_uTtiwks5srBvDgaJpZM4PzdCV
.

Sure it's here
Note that I all changed is just
target = "opencl"
ctx = tvm.cl(0)
With the same change you can also run tutorial/from_onnx.py and tutorial/from_mxnet.py etc.
You can also use
python/nnvm/top/nn.py
python/nnvm/top/reduction.py
python/nnvm/top/tensor.py
This should be enough to run resnet bench and tutorials.

I encountered CL_OUT_RESOURCES again with your changes.
tvm._ffi.base.TVMError: [19:44:47] src/codegen/llvm/llvm_module.cc:59:
Check failed: ret == 0 (-1 vs. 0) [19:44:47]
src/runtime/opencl/opencl_module.cc:223: Check failed: e == CL_SUCCESS
OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
On Tue, Oct 10, 2017 at 7:26 PM, masahi<EMAIL_ADDRESS>wrote:

Sure it's here
https://gist.github.com/masahi/de658ec98b1e275de6331a4cf4c873b3
Note that I all changed is just
target = "opencl" ctx = tvm.cl http://tvm.cl(0)
With the same change you can also run tutorial/from_onnx.py and
tutorial/from_mxnet.py etc.
You can also use
python/nnvm/top/nn.py
https://gist.github.com/masahi/a9b67fe27b24b783334ef8fc6f877ef9
python/nnvm/top/reduction.py
https://gist.github.com/masahi/706fd19cb4c589bc7ea59d68cde99a8a
python/nnvm/top/tensor.py
https://gist.github.com/masahi/1d503f9e7839636dc8086d3737523d64
This should be enough to run resnet bench and tutorials.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335662720, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3rLt4GKrsFxLsDkZ7bU_HeNNS6abks5srCdZgaJpZM4PzdCV
.

Are you running resnet bench? If so, try mobilenet bench first.
It requires less memory I think.
2017/10/11 午前11:47 "kaishijeng"<EMAIL_ADDRESS>
I encountered CL_OUT_RESOURCES again with your changes.
tvm._ffi.base.TVMError: [19:44:47] src/codegen/llvm/llvm_module.cc:59:
Check failed: ret == 0 (-1 vs. 0) [19:44:47]
src/runtime/opencl/opencl_module.cc:223: Check failed: e == CL_SUCCESS
OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
On Tue, Oct 10, 2017 at 7:26 PM, masahi<EMAIL_ADDRESS>wrote:

Sure it's here
https://gist.github.com/masahi/de658ec98b1e275de6331a4cf4c873b3
Note that I all changed is just
target = "opencl" ctx = tvm.cl http://tvm.cl(0)
With the same change you can also run tutorial/from_onnx.py and
tutorial/from_mxnet.py etc.
You can also use
python/nnvm/top/nn.py
https://gist.github.com/masahi/a9b67fe27b24b783334ef8fc6f877ef9
python/nnvm/top/reduction.py
https://gist.github.com/masahi/706fd19cb4c589bc7ea59d68cde99a8a
python/nnvm/top/tensor.py
https://gist.github.com/masahi/1d503f9e7839636dc8086d3737523d64
This should be enough to run resnet bench and tutorials.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335662720, or mute
the thread
<https://github.com/notifications/unsubscribe-
auth/AMGg3rLt4GKrsFxLsDkZ7bU_HeNNS6abks5srCdZgaJpZM4PzdCV>
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335666043, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABsbE1POGJTHEVVfNe-2fQ8nVIAyfVBVks5srCwwgaJpZM4PzdCV
.

No luck with mobilenet.
It is probably related to how tvm/nnvm allocates GPU resources in various
layers because OpenCL functions in ARM ComputeLibrary
is running ok on Firefly3399
Thanks,
On Tue, Oct 10, 2017 at 7:50 PM, masahi<EMAIL_ADDRESS>wrote:

Are you running resnet bench? If so, try mobilenet bench first.
It requires less memory I think.
2017/10/11 午前11:47 "kaishijeng"<EMAIL_ADDRESS>
I encountered CL_OUT_RESOURCES again with your changes.
tvm._ffi.base.TVMError: [19:44:47] src/codegen/llvm/llvm_module.cc:59:
Check failed: ret == 0 (-1 vs. 0) [19:44:47]
src/runtime/opencl/opencl_module.cc:223: Check failed: e == CL_SUCCESS
OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
On Tue, Oct 10, 2017 at 7:26 PM, masahi<EMAIL_ADDRESS>wrote:

Sure it's here
https://gist.github.com/masahi/de658ec98b1e275de6331a4cf4c873b3
Note that I all changed is just
target = "opencl" ctx = tvm.cl http://tvm.cl(0)
With the same change you can also run tutorial/from_onnx.py and
tutorial/from_mxnet.py etc.
You can also use
python/nnvm/top/nn.py
https://gist.github.com/masahi/a9b67fe27b24b783334ef8fc6f877ef9
python/nnvm/top/reduction.py
https://gist.github.com/masahi/706fd19cb4c589bc7ea59d68cde99a8a
python/nnvm/top/tensor.py
https://gist.github.com/masahi/1d503f9e7839636dc8086d3737523d64
This should be enough to run resnet bench and tutorials.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335662720, or
mute
the thread
<https://github.com/notifications/unsubscribe-
auth/AMGg3rLt4GKrsFxLsDkZ7bU_HeNNS6abks5srCdZgaJpZM4PzdCV>
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335666043, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABsbE1POGJTHEVVfNe-
2fQ8nVIAyfVBVks5srCwwgaJpZM4PzdCV>

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335666416, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3psKYLJ6VQRek8IY7MJQ47HLedFmks5srCzZgaJpZM4PzdCV
.

@tqchen Halide has auto scheduling work merged in master. Do you think auto scheduling in tvm is possible? That will be pretty cool.

yes, we are looking into it. The auto schedule pattern for GPU and multi-core CPU could go beyond what pattern we can use for image processing works and we are gaining some experience on what new changes need to be done here

That's great, looking forward to learning the techniques.

Why is resource required in  conv2d_nchw.py  much biigger than conv2d_hwnc?
I still try to figure out why there is not enough resource
in  conv2d_nchw.py, but conv2d_hwnc.py is OK on unittest
Thanks
On Wed, Oct 11, 2017 at 4:17 PM, masahi<EMAIL_ADDRESS>wrote:

That's great, looking forward to learning the techniques.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-335975462, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3jUrLUfkqemOnJeLuCYeapJ79M9mks5srUykgaJpZM4PzdCV
.

The conv2d_nchw is quite specially tuned for K80 card, which will launch quite a lot more threads with shared memory resources. They are not as good schedule for the mobile GPUs.

Can I always use conv2d_hwnc even though it is not optimized for a
particular processor/platform?
How do I know whch one , ie conv2d_nchw or conv2d_hwnc is used in the
cuda_imagenet_bench.py
https://github.com/dmlc/nnvm/blob/master/examples/benchmark/cuda_imagenet_bench.py
?

Thanks,
On Thu, Oct 12, 2017 at 10:23 AM, Tianqi Chen<EMAIL_ADDRESS>wrote:

The conv2d_nchw is quite specially tuned for K80 card, which will launch
quite a lot more threads with shared memory resources. They are not as good
schedule for the mobile GPUs.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-336207221, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3n-DLJxIH3Iw6_6jt6xK8UD19JdFks5srksRgaJpZM4PzdCV
.

The imagenet bench always use the nchw one, the hnwc one is designed for the batch kernel in cuda. I think we can try to make the schedule more adjustable so it can adapt to more platforms as a future goal

Will wait for your new design.
I think if NNVM/TVM can take advantage of mobile GPU to speed up deep
learning, it will be enable more deep learning applications on mobile
devices.
Thanks,
On Thu, Oct 12, 2017 at 3:52 PM, Tianqi Chen<EMAIL_ADDRESS>wrote:

The imagenet bench always use the nchw one, the hnwc one is designed for
the batch kernel in cuda. I think we can try to make the schedule more
adjustable so it can adapt to more platforms as a future goal
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-336301104, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3jY291ZSSjLacwyrpD4r2vrdcZr1ks5srpgUgaJpZM4PzdCV
.

Solved by #556

Masahi,
How do #556 solve this issue?
With the latest code, I got different error of test_topi_conv2d_nchw.py
below. The error of previous code is an insufficient resource of CL
allocation.
TVM: Initializing cython mode...
Use memoize
topi.tests.test_topi_conv2d.verify_con2d_nchw.get_ref_data.pkl(5, (1, 3,
224, 224), 'float32', 2, 3, (64, 3, 7, 7))
Skip because cuda is not enabled
[10:56:06] src/runtime/opencl/opencl_device_api.cc:195: Initialize OpenCL
platform 'ARM Platform'
[10:56:06] src/runtime/opencl/opencl_device_api.cc:215:
opencl(0)='Mali-T860' cl_device_id=0x7f75be3dd8
[10:56:06]
/home/firefly/2TB/src/firefly/tvm/dmlc-core/include/dmlc/logging.h:308:
[10:56:06] src/runtime/opencl/opencl_module.cc:223: Check failed: e ==
CL_SUCCESS OpenCL Error, code=-55: CL_INVALID_WORK_ITEM_SIZE
On Sat, Oct 14, 2017 at 4:55 PM, masahi<EMAIL_ADDRESS>wrote:

Closed #530 https://github.com/dmlc/tvm/issues/530.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#event-1293574506, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AMGg3j_Ea_PtPZxSj4n-PtcyY_EdIEp9ks5ssUnfgaJpZM4PzdCV
.

To enable Mali GPUs, we need to further mobile GPU options here https://github.com/dmlc/tvm/blob/master/python/tvm/target.py#L63
Note that most of the current cuda schedules need tweaks for the mali target. So additional registered function from topi is needed

I add one addiitonal target, "opencl" to set self.max_num_threads = 8  in
the target.py as below:
    elif target_name in ("opencl",):
        # For now assume rocm schedule for opencl
        self.keys += ("rocm", "gpu")
        self.max_num_threads = 8

However, I still got the same error is  Error, code=-55:
CL_INVALID_WORK_ITEM_SIZE.
The error above seems due to other issues than available GPU resources.
On Sun, Oct 15, 2017 at 11:51 AM, Tianqi Chen<EMAIL_ADDRESS>wrote:

To enable Mali GPUs, we need to further mobile GPU options here
https://github.com/dmlc/tvm/blob/master/python/tvm/target.py#L63
Note that most of the current cuda schedules need tweaks for the mali
target. So additional registered function from topi is needed
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-336732599, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3r3iNZAc1P7Fl2yw0E1qNa6AwthTks5sslQcgaJpZM4PzdCV
.

Have you reinstalled tvm?

@kaishijeng We don't have Mali GPUs to test. #556 was confirmed to work for AMD device.
You might want to open a separate issue for your case.

Yes, I did reinstall tvm by doing python ./setup.py install
Thanks,
On Sun, Oct 15, 2017 at 12:44 PM, masahi<EMAIL_ADDRESS>wrote:

@kaishijeng https://github.com/kaishijeng We don't have Mali GPUs to
test. #556 https://github.com/dmlc/tvm/pull/556 was confirmed to work
for AMD device.
You might want to open a separate issue for your case.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-336736288, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3g-Atnaqlz0NswUWPJfnaTQlBOzWks5ssmC7gaJpZM4PzdCV
.

masahi,
I still see large value of  num_threads in topi/python/topi/cuda/conv2d_nchw.py.
How does max_num_threads in target.py relate to  num_threads in topi/python/topi/cuda/conv2d_nchw.py?
Thanks,

Thanks, it looks like not all schedules under topi.cuda have been updated to use max_num_threads. I need to discuss this with @tqchen

Tiangi/masahi
Just curious, why is there no opencl directory in tvm/topi/python/topi?
There are rasp and cuda., but no opencl.
On Mon, Oct 16, 2017 at 2:39 AM, masahi<EMAIL_ADDRESS>wrote:

Thanks, it looks like not all schedules under topi.cuda have been updated
to use max_num_threads. I need to discuss this with @tqchen
https://github.com/tqchen
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-336792370, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3mxaHZH9RHjd6GQtU_0x78wfI-Gyks5ssvf3gaJpZM4PzdCV
.

This is mainly due to fact that there is not yet effort to optimize schedule for opencl devices. We do expect that there should be topi/rocm(for amd gpus) and topi/mali (for mobile gpus)

masahi
Does the latest code have proper  max_num_threads enforced?

If yes, what changes do I need to do to reduce max_num_threads for firefly3399?
Thanks,

@kaishijeng yes, at least it solves my AMD opencl problem. Not sure if it will work for you, though.
If max_num_threads = 256 is still too big for your card, you can set a smaller number here.

Masahi,
WIll try different values if 256 is too high. Just curious, I saw the
change of max_num_threads is under target_name in ("rocm", "opencl"). Will
firefly 3399 use the same target name?
Thanks,
On Sun, Oct 22, 2017 at 7:28 PM, masahi<EMAIL_ADDRESS>wrote:

@kaishijeng https://github.com/kaishijeng yes, at least it solves my
AMD opencl problem. Not sure if it will work for you, though.
If max_num_threads = 256 is still too big for your card, you can set a
smaller number here
https://github.com/dmlc/tvm/blob/122183583663ae11a5389e46d3ef6c92e2b82d05/python/tvm/target.py#L113
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-338531763, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3tQ8a_q9SQmmAZVjZdCS1FIvXKYLks5su_mtgaJpZM4PzdCV
.

Yes, what matters is the keyword  "gpu" in self.keys += ("rocm", "gpu")
This will cause schedules registered under "gpu" to be called by opencl backend.

I have tried different values of  self.max_num_threads in target.py and
even down to 8,still got
Check failed: e == CL_SUCCESS OpenCL Error, code=-5: CL_OUT_OF_RESOURCES
when  doing
test_conv2d_nchw
Thanks,
On Sun, Oct 22, 2017 at 11:12 PM, masahi<EMAIL_ADDRESS>wrote:

Yes, what matters is the keyword "gpu" in self.keys += ("rocm", "gpu")
This will cause schedules registered under "gpu" to be called by opencl
backend.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-338557189, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3lpV7hB1Y7P3_gwLQTBC6LB08OWrks5svC4zgaJpZM4PzdCV
.

Seems your card doesn't have enough memory to run test_conv2d_nchw
You should try smaller inputs, as I told you earlier.

But it is not useful to have a smaller input because most of nets have
input size of (224,224, 3).
Thanks,
On Sun, Oct 22, 2017 at 11:24 PM, masahi<EMAIL_ADDRESS>wrote:

Seems your card doesn't have enough memory to run test_conv2d_nchw
You should try smaller inputs, as I told you earlier.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/530#issuecomment-338559078, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMGg3hn17G0KiMLMJEIJ45njMDbQCmXQks5svDEWgaJpZM4PzdCV
.

you can try reducing output channels or smaller filter size.

The existing schedule for AMD/CUDA GPUs do not work well on the ARM GPUs, and we will need independent schedules for them. For example, it is usually not helpful to take use of shared memory


User/Password authentication for GraphSpace

We would like spras to automatically post graphs to GraphSpace. The graphspace-python repo provides a Python client to post graphs using a REST API.  However, the catch is that username/password would be in plain text in the config file for it to work as implemented.  This issue outlines some initial steps towards resolving this.
Short-Term Suggestions

Output JSON files for upload  through the GraphSpace web interface. Pros: having output files are in line with snakemake. Doesn't require username/password in spras. Cons: requires user to take extra steps to upload their graph to Cytoscape or GraphSpace.
Prompt the user for their username and password at the terminal.  I would suggest doing this in the parse_config() function. Pros: simple, good for prototyping, will post graphs at the end of the run. Cons: waiting for user input is not great. We would want to implement a check to make sure that username/password combo is valid upon user entry.

Longer-Term Suggestions
This has to have been addressed by others. We want to keep ease of use in mind, especially for non-computational folks.

Implement password encryption/authentication within the graphspace-python client. GraphSpace should support looking up username and password from an encrypted file or a keychain or something similar. There is a question about whether graphspace-python should support it at the level of the REST API or the Python client, but that's a matter for the repo. We would have to be make sure that the choice works with snakemake and config files.
Allow secrets in the yaml files that are not shared. Something like HomeAssistant allows users to have a secrets.yaml file that removes private information from config files.
Encrypt the passwords in the yaml files. Take a look at PipelineWise's documentation.
Use environment variables in the yaml files to point to access tokens or passwords. This is also how users could receive emails upon completion of tasks. Take a look at Environment Variables in Snakemake documentation. This seems straightforward too.

Thanks for researching this @annaritz.  For the very short term, I like the idea of outputting JSON files for upload to GraphSpace via the web interface.  That requires an extra step for the user, but we're short on development time.
The longer term ideas make sense to me.  I don't have a strong preference about which to pursue but agree we should emphasize usability.


Fix building on MacOS + arm64

Quick fix that allows for arm64 dir as an option + checks for existence of temp /GameMakerStudio2 first, as otherwise calling mkdirSync with a missing parent hangs.
MacOS .apps are folders and can't be called, so the option was either directly calling the internal executable, or appending open, which seems potentially more troublesome(?)
I could have a go cleaning up the code involving this stuff as well, but this was mainly so I could use it on my own system quickly.

Cheers! Hopefully this won't cause any side effects for x64 Mac

Cheers! Hopefully this won't cause any side effects for x64 Mac

Turns out this did cause one tiny side effect which I have submitted a PR to address. LMK if this looks ok to you.


[Bug]: exec-if not clearing in new update if condition is not met

Checklist

[X] I have read the appropriate section in the contributing guidelines
[X] I believe this issue is a problem with polybar itself and not a misconfiguration on my part
[X] I have searched for other open and closed issues that may have already reported this problem
[X] I have checked the known issues page for this problem.
[X] I have followed the debugging guide to narrow down the problem to a minimal config.

Steps to reproduce
Use an exec-if function in your config and stop meeting the requirement.
Minimal config
[module/mpv]
type = custom/script
exec = playerctl --player=mpv metadata | awk '/title/{print}' | awk '{for (i=3; i<NF; i++) printf $i " "; print $NF}' | awk '{sub(".mp3","")}1'
exec-if = playerctl --player=mpv status | grep Playing
interval = 0.5
format-underline = #8800FF

click-left = playerctl --player=mpv play-pause
click-middle = killall mpv
click-right = i3-msg "[class="music"] scratchpad show move position 1280 21"
label-active-font = 2

[module/mpv_paused]
type = custom/script
exec = playerctl --player=mpv metadata | awk '/title/{print}' | awk '{for (i=3; i<NF; i++) printf $i " "; print $NF}' | awk '{sub(".mp3","")}1'
exec-if = playerctl --player=mpv status | grep Paused
interval = 0.5
format-foreground = #666
format-underline = #8800FF

click-left = playerctl --player=mpv play-pause
click-middle = killall mpv
click-right = i3-msg "[class="music"] scratchpad show move position 1280 21"
label-active-font = 2

Polybar log
notice: Parsing config file: /home/jimbo/.config/polybar/config.ini
notice: Listening for IPC messages (PID: 586996)
notice: Loading module 'i3' of type 'internal/i3'
notice: Loading module 'xwindow' of type 'internal/xwindow'
notice: Loading module 'mpv' of type 'custom/script'
notice: Loading module 'mpv_paused' of type 'custom/script'
notice: Loading module 'pulseaudio' of type 'internal/pulseaudio'
notice: pulseaudio: using default sink alsa_output.pci-0000_00_1b.0.analog-stereo
notice: Loading module 'updates' of type 'custom/script'
notice: Loading module 'backlight' of type 'custom/script'
notice: Loading module 'weather' of type 'custom/script'
notice: Loading module 'cpu' of type 'internal/cpu'
notice: Loading module 'memory' of type 'internal/memory'
notice: Loading module 'battery' of type 'internal/battery'
notice: Loading module 'date' of type 'internal/date'
notice: Loaded 12 modules
notice: Loaded font "Ubuntu:fontformat=truetype:size=12:antialias=true" (name=Ubuntu, offset=3, file=/usr/share/fonts/ubuntu/Ubuntu-R.ttf)
notice: Loaded font "UbuntuMono Nerd Font:fontformat=truetype:size=12:antialias=true" (name=UbuntuMono Nerd Font, offset=3, file=/usr/share/fonts/TTF/Ubuntu Mono Nerd Font Complete.ttf)
notice: Loaded font "Source Han Sans JP:size=11" (name=Source Han Sans JP, offset=2, file=/usr/share/fonts/adobe-source-han-sans/SourceHanSansJP-Regular.otf)
notice: Loaded font "Source Han Sans KR:size=11" (name=Source Han Sans KR, offset=2, file=/usr/share/fonts/adobe-source-han-sans/SourceHanSansKR-Regular.otf)
notice: Loaded font "pango:Ubuntu 10" (name=Noto Sans, offset=0, file=/usr/share/fonts/noto/NotoSans-Regular.ttf)
warn: Ignoring restack of i3 window (not needed when `override-redirect = false`)
error: tray: Failed to put tray above 0x3a00002 in the stack (XCB_MATCH (8))
notice: Received signal(2): Interrupt
notice: Termination signal received, shutting down...
warn: module/i3: Attempting to reconnect socket (reason: Unexpected EOF while reading header)

Expected behavior
When the exec-if function does not meet the expected result, nothing will be displayed and the module will hide itself.
Actual behavior
Module does not hide itself, and is instead left taking up room on the bar (duplicated in the context of my config).
Window Manager and Version
i3-gaps 4.20.1
Linux Distribution
Arch Linux
Polybar version
polybar 3.6.2

Features: +alsa +curl +i3 +mpd +network(libnl) +pulseaudio +xkeyboard

X extensions: +randr (+monitors) +composite +xkb +xrm +xcursor

Build type: Release

Compiler: /sbin/c++

Compiler flags: -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -O3 -DNDEBUG -Wall -Wextra -Wpedantic -Wsuggest-override

Linker flags: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -Wall -Wextra -Wpedantic -Wsuggest-override -Wall -Wextra -Wpedantic -Wsuggest-override

Additional Context / Screenshots
I have this set up as a script to hook into and show me the current song playing using playerctl. When playing, the first module is executed and called. When paused, it hides the first and shows me the second module. When neither condition is met, they both hide. This behavior works in 3.6.1

In 3.6.2, both are able to execute but neither hide when the condition ceases to be met, leaving both modules on screen, even when playerctl gives a blank result to both modules.

Tested now on two devices.

Thanks for reporting :)
@indev29 I think we broke something. When calling m_runner.clear_output(); in the script module, it doesn't trigger an update. I'll try to fix it later today, unless you want to go first ;)

Thanks for reporting :)
@indev29 I think we broke something. When calling m_runner.clear_output(); in the script module, it doesn't trigger an update. I'll try to fix it later today, unless you want to go first ;)
Already fixing it, you can assing me :)
Also, I think you've accidentally merged #2668 that was supposed to be released with 3.7.0. Should we roll it back or just add changelog notes now?

That was fast, thanks :)
#2668 was merged correctly. Everything merged to master will only appear in the next minor release (X.Y.0) and not in any hotfix releases.
If you're interested in how we do releases in polybar, you can read about it in the docs.

Fixed in #2675


Add option to set allowance during mint

Multi-contract systems involving snip20's could benefit from having an optional field in the mint function to set an allowance for another wallet address.

what's the use case here? the minter is usually not the recipient of the funds, and if it is you can send a multi-message tx.

gotcha, sounds interesting


fix: warnings during build

🐛 The bug
Simply run pnpm build in the given reproduction.
The warnings start point is during prerender
When running in local I get the below:
ℹ Initializing prerenderer

[9:03:34 PM]  WARN  Export "useNitroApp" of module "node_modules/.pnpm/nitropack<EMAIL_ADDRESS>was reexported through module "virtual:#imports" while both modules are dependencies of each other and will end up in different chunks by current Rollup settings. This scenario is not well supported at the moment as it will produce a circular dependency between chunks and will likely lead to broken execution order.
Either change the import in "node_modules/.pnpm/@nuxtjs+sitemap@6.0.0-beta.1_h3@1.12.0_magicast@0.3.4_rollup@4.20.0_vite@5.4.0_@types+node<EMAIL_ADDRESS>to point directly to the exporting module or reconfigure "output.manualChunks" to ensure these modules end up in the same chunk.

Further warnings
[9:04:27 PM]  WARN  Export "useNitroApp" of module "node_modules/.pnpm/nitropack<EMAIL_ADDRESS>was reexported through module "virtual:#imports" while both modules are dependencies of each other and will end up in different chunks by current Rollup settings. This scenario is not well supported at the moment as it will produce a circular dependency between chunks and will likely lead to broken execution order.
Either change the import in "node_modules/.pnpm/@nuxtjs+sitemap@6.0.0-beta.1_h3@1.12.0_magicast@0.3.4_rollup@4.20.0_vite@5.4.0_@types+node<EMAIL_ADDRESS>to point directly to the exporting module or reconfigure "output.manualChunks" to ensure these modules end up in the same chunk.

ℹ Prerendered 73 routes in 66.009 seconds
✔ Generated public .output/public
ℹ Building Nuxt Nitro server (preset: node-server)
Export "useNitroApp" of module "node_modules/.pnpm/nitropack<EMAIL_ADDRESS>was reexported through module "virtual:#imports" while both modules are dependencies of each other and will end up in different chunks by current Rollup settings. This scenario is not well supported at the moment as it will produce a circular dependency between chunks and will likely lead to broken execution order.
Either change the import in "node_modules/.pnpm/@nuxtjs+sitemap@6.0.0-beta.1_h3@1.12.0_magicast@0.3.4_rollup@4.20.0_vite@5.4.0_@types+node<EMAIL_ADDRESS>to point directly to the exporting module or reconfigure "output.manualChunks" to ensure these modules end up in the same chunk.

🛠 To reproduce
https://stackblitz.com/edit/nuxt-starter-7mnzr1?file=package.json
🌈 Expected behavior
Build successfully without errors or warnings.
ℹ Additional context
No response

I'm getting the same during build.
Export "useNitroApp" of module "node_modules/nitropack/dist/runtime/app.mjs" was reexported through module "virtual:#imports" while both modules are dependencies of each other and will end up in different chunks by current Rollup settings. This scenario is not well supported at the moment as it will produce a circular dependency between chunks and will likely lead to broken execution order.
Either change the import in<EMAIL_ADDRESS>to point directly to the exporting module or reconfigure "output.manualChunks" to ensure these modules end up in the same chunk.

This issue may be related? https://github.com/nuxt/nuxt/issues/18789#issuecomment-1420047699

@harlan-zw Are you sure this is fixed? Running into this problem myself, so I tried the reproduction above but with 2.0.0-rc.23 and the same warning showed up again.


FR: Use Jetty's native session variables, instead of our own.

bookshelf/6-gce/src/main/java/com/example/getstarted/util/DatastoreSessionFilter.java
+

datastore = DatastoreOptions.getDefaultInstance().getService();
keyFactory = datastore.newKeyFactory().setKind("SessionVariable");
// Delete all sessions unmodified for over two days
@tswast
tswast 3 days ago Google Cloud Platform member
Ok, but I wish we didn't have to do this on filter initialization. I guess we don't have cron/task queues yet on env: flex. :-(

bookshelf/6-gce/src/main/java/com/example/getstarted/util/DatastoreSessionFilter.java

// identifier.
String sessionId = getCookieValue(req, "bookshelfSessionId");
if (sessionId.equals("")) {

 String sessionNum = new BigInteger(130, new SecureRandom()).toString(32);
@tswast

tswast 3 days ago Google Cloud Platform member
Any reason we didn't use UUID? It's 128 bits and 36 characters (4 of which are hyphens). It appears it does use SecureRandom internally http://stackoverflow.com/a/7532965/101923

Bookshelf is being replaced.  UUID would have been a good idea.


Automatically provision custom hostnames on popular DNS providers

Currently, support for using one's own domain requires either a CNAME to the provisioned flynnhub.com hostname, or manually adding A records for each of the cluster's hosts. Ideally, we could provision a hostname for the cluster and for each app within that cluster, and manage those hostnames as the cluster members change.
Off the top of my head I'd see support for Amazon Route 53, Google DNS, DNSimple, DNSmadeeasy, and Dyn.

Flynn is unmaintained and our infrastructure will shut down on June 1, 2021. See the README for details.


fix(autoware_occupancy_grid_map_outlier_filter): fix bugprone-incorrect-roundings

Description
This is a fix based on clang-tidy bugprone-incorrect-roundings error.
/home/emb4/autoware/autoware/src/universe/autoware.universe/perception/autoware_occupancy_grid_map_outlier_filter/src/occupancy_grid_map_outlier_filter_node.cpp:143:16: error: casting (double + 0.5) to integer leads to incorrect rounding; consider using lround (#include <cmath>) instead [bugprone-incorrect-roundings,-warnings-as-errors]
      std::max(static_cast<int>(min_points_and_distance_ratio_ / distance + 0.5f), min_points_),
               ^
/home/emb4/autoware/autoware/src/universe/autoware.universe/perception/autoware_occupancy_grid_map_outlier_filter/src/occupancy_grid_map_outlier_filter_node.cpp:143:33: error: casting (double + 0.5) to integer leads to incorrect rounding; consider using lround (#include <cmath>) instead [bugprone-incorrect-roundings,-warnings-as-errors]
      std::max(static_cast<int>(min_points_and_distance_ratio_ / distance + 0.5f), min_points_),
                                ^
/home/emb4/autoware/autoware/src/universe/autoware.universe/perception/autoware_occupancy_grid_map_outlier_filter/src/occupancy_grid_map_outlier_filter_node.cpp:203:16: error: casting (double + 0.5) to integer leads to incorrect rounding; consider using lround (#include <cmath>) instead [bugprone-incorrect-roundings,-warnings-as-errors]
      std::max(static_cast<int>(min_points_and_distance_ratio_ / distance + 0.5f), min_points_),
               ^
/home/emb4/autoware/autoware/src/universe/autoware.universe/perception/autoware_occupancy_grid_map_outlier_filter/src/occupancy_grid_map_outlier_filter_node.cpp:203:33: error: casting (double + 0.5) to integer leads to incorrect rounding; consider using lround (#include <cmath>) instead [bugprone-incorrect-roundings,-warnings-as-errors]
      std::max(static_cast<int>(min_points_and_distance_ratio_ / distance + 0.5f), min_points_),

Related links
Parent Issue:

Link

How was this PR tested?
Confirm that the value does not change by actually operating the system.
#include <iostream>
#include <cmath>
#include <algorithm>

int calculateMinPointsThreshold(double min_points_and_distance_ratio_, double distance, int min_points_, int max_points_) {
    return std::min(
        std::max(static_cast<int>(std::lround(min_points_and_distance_ratio_ / distance)), min_points_),
        max_points_
    );
}

int calculateMinPointsThreshold_old(double min_points_and_distance_ratio_, double distance, int min_points_, int max_points_) {
    return std::min(
      std::max(static_cast<int>(min_points_and_distance_ratio_ / distance + 0.5f), min_points_),
      max_points_);
}

int main() {
    double min_points_and_distance_ratio = 30;
    double distance = 3.9;
    int min_points = 5;
    int max_points = 15;

    int min_points_threshold = calculateMinPointsThreshold(min_points_and_distance_ratio, distance, min_points, max_points);
    int min_points_threshold_old = calculateMinPointsThreshold_old(min_points_and_distance_ratio, distance, min_points, max_points);

    std::cout << "Calculated min_points_threshold: " << min_points_threshold << std::endl;
    std::cout << "Calculated min_points_threshold_old: " << min_points_threshold_old << std::endl;

    return 0;
}

emb4@emb4-E5-165:~/autoware_cppcheck/autoware.universe$ ./test
Calculated min_points_threshold: 8
Calculated min_points_threshold_old: 8

Notes for reviewers
None.
Interface changes
None.

Effects on system behavior
None.

@amc-nu @YoshiRi @yukkysaito
Please confirm the PR. Thank you very much.

@amc-nu @YoshiRi @yukkysaito
Please confirm the PR. Thank you very much.

@kobayu858 Thank you. I believe std::lround might increase the computational cost, so could you measure the latency before and after implementing it?

@yukkysaito
double min_points_and_distance_ratio = 400.0;
int min_points = 4;
int max_points = 70;

This is the average latency of 1,000,000 calculations with varying DISTANCE using this parameter.
New is the modified implementation and Old is the existing implementation.
Depending on the value of distance, the maximum increase will be about 4ns.
distance [0.1, 10.0]
calculateMinPointsThreshold (New) - Average: 20.2532 ns
calculateMinPointsThreshold_old (Old) - Average: 16.7661 ns

distance [10.0, 100.0]
calculateMinPointsThreshold (New) - Average: 17.2914 ns
calculateMinPointsThreshold_old (Old) - Average: 16.9614 ns

distance [100.0, 1000.0]
calculateMinPointsThreshold (New) - Average: 17.7293 ns
calculateMinPointsThreshold_old (Old) - Average: 17.2558 ns

Restored because it was closed by mistake


⚠ Bofrost.es has degraded performance

In cafa8cb, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1694 ms

Resolved: Bofrost.es performance has improved in 24e5101 after 12 minutes.


Unclear operation in code ID:02

In the scripts n_tap_fir.v and n_tap_complex_fir.v, the operations inside the state FIR_MAIN are quite unclear and the comments as to why we do what we do are unclear. Maybe consider improving this through an image explanation or detailed explanation in the code.

The issue has been solved with the branch "Fir filter documentation id 02".


[spike+] [bug] unit testing a recursive CTE fails

Is this a new bug in dbt-bigquery?

[X] I believe this is a new bug in dbt-bigquery
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
I'm trying to unit test a model with a recursive CTE (a use case specifically called out in the docs), but it's failing with this error in dbt Cloud:

WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.

Examining the generated code, I can confirm that it starts with select * from (WITH RECURSIVE.
Expected Behavior
I expect to be able to use unit tests with recursive CTEs on BigQuery.
Steps To Reproduce

Add the attached 3 files to a BigQuery dbt environment with version 1.8. (I used dbt Cloud.)
dbt build --select +recursive_cte+

Relevant log output
13:32:41 Began running node unit_test.edsights.recursive_cte.test_recursive_cte
13:32:41 2 of 3 START unit_test recursive_cte::test_recursive_cte ....................... [RUN]
13:32:41 Acquiring new bigquery connection 'unit_test.edsights.recursive_cte.test_recursive_cte'
13:32:41 Began compiling node unit_test.edsights.recursive_cte.test_recursive_cte
13:32:41 Timing info for unit_test.edsights.recursive_cte.test_recursive_cte (compile): 13:32:41.383097 => 13:32:41.383103
13:32:41 Began executing node unit_test.edsights.recursive_cte.test_recursive_cte
13:32:41 Opening a new connection, currently in state init
13:32:41 Writing injected SQL for node "unit_test.edsights.recursive_cte.test_recursive_cte"
13:32:41 Writing injected SQL for node "unit_test.edsights.recursive_cte.test_recursive_cte"
13:32:41 On unit_test.edsights.recursive_cte.test_recursive_cte: /* {"app": "dbt", "dbt_version": "2024.0.0", "profile_name": "user", "target_name": "default", "node_id": "unit_test.edsights.recursive_cte.test_recursive_cte"} */

    create or replace table `production-one-405017`.`dbt_harlan`.`test_recursive_cte__dbt_tmp`

    OPTIONS(
      expiration_timestamp=TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL 12 hour)
    )
    as (
      select * from (
        WITH RECURSIVE
 __dbt__cte__recursive_cte_given as (

-- Fixture for recursive_cte_given
select safe_cast('1' as INT64) as x
), asdf AS (
    SELECT *
    FROM __dbt__cte__recursive_cte_given
)

SELECT *
FROM asdf
    ) as __dbt_sbq
    where false and current_timestamp() = current_timestamp()
    limit 0

    );

13:32:41 BigQuery adapter: https://console.cloud.google.com/bigquery?project=production-one-405017&j=bq:us-central1:a993367b-a621-4a41-8bf7-23d7275b7e5a&page=queryresults
13:32:42 BigQuery adapter: Retry attempt 1 of 1 after error: BadRequest('WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.; reason: invalidQuery, location: query, message: WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.')
13:32:43 BigQuery adapter: https://console.cloud.google.com/bigquery?project=production-one-405017&j=bq:us-central1:16d300ec-588c-4414-aafd-9583fc051629&page=queryresults
13:32:43 BigQuery adapter: https://console.cloud.google.com/bigquery?project=production-one-405017&j=bq:us-central1:16d300ec-588c-4414-aafd-9583fc051629&page=queryresults
13:32:43 Timing info for unit_test.edsights.recursive_cte.test_recursive_cte (execute): 13:32:41.384037 => 13:32:43.609843
13:32:43 Runtime Error in unit_test test_recursive_cte (models/intermediate/recursive_cte.yml)
  An error occurred during execution of unit test 'test_recursive_cte'. There may be an error in the unit test definition: check the data types.
   Database Error
    WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.
13:32:43 2 of 3 ERROR recursive_cte::test_recursive_cte ................................. [ERROR in 2.23s]
13:32:43 Finished running node unit_test.edsights.recursive_cte.test_recursive_cte
13:32:43   Runtime Error in unit_test test_recursive_cte (models/intermediate/recursive_cte.yml)
  An error occurred during execution of unit test 'test_recursive_cte'. There may be an error in the unit test definition: check the data types.
   Database Error
    WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.

Environment
dbt Cloud with version 1.8

Additional Context
See also dbt Labs support ticket 64761.

recursive_cte.zip
ZIP file includes two SQL files and a YML file.

Hey - thanks so much for opening! Since this is currently broken, we're going to remove that callout from our docs site. But I will sync with our engineers to see how we can fix this.

Just adding the reproducible example Harlan shared.
I have a model I want to unit test:
# models/recursive_cte.sql

WITH RECURSIVE
asdf AS (
    SELECT *
    FROM {{ ref('recursive_cte_given') }}
)

SELECT *
FROM asdf

I add a unit test:
unit_tests:
  - name: test_recursive_cte
    model: recursive_cte
    given:
      - input: ref('recursive_cte_given')
        format: csv
        rows: |
          x
          1
    expect:
      format: csv
      rows: |
        x
        1

It looks like WITH RECURSIVE is supported for most of the dbt Labs supported adapters:

:white_check_mark: BigQuery: * https://cloud.google.com/bigquery/docs/recursive-ctes
:white_check_mark: Snowflake: https://docs.snowflake.com/en/sql-reference/constructs/with#syntax
:x: Spark:

can't find any documented support
this databricks forum indicates no support?

:white_check_mark: Postgres: https://www.postgresql.org/docs/current/queries-with.html#QUERIES-WITH-RECURSIVE
:white_check_mark: Redshift: https://docs.aws.amazon.com/redshift/latest/dg/r_WITH_clause.html

For implementation:

It looks like the extraneous select * is coming from the unit materialization, where we create a temp empty table of the model being tested using get_create_table_as_sql(True, temp_relation, get_empty_subquery_sql(sql))
Then, the get_empty_subquery_sql wrapper adds the select * and where false limit 0 wrapper around the user-provided sql.

I'm not sure if it's possible to remove the select * but maintain the where false limit 0 filter. If we don't have the where false limit 0 filter, obtaining the column schema of the tested model isn't possible without actually running the query which could be costly...

Additionally, we also expect to be able to wrap the user-provided sql in a subquery when constructing the statement that unions actual and expected results for comparison here.
It seems that being able to wrap the user-provided SQL in a subquery is an assumption held by the unit testing framework in multiple places currently.

We're going to call this out as a known limitation for the 1.8 release. But this is something we will revisit for 1.9 as an outcome of https://github.com/dbt-labs/dbt-core/issues/8499


Add experimental @ArrayContentBased

First step of implementing #1: This annotation will be used to mark array properties so that Poko uses their content in its generated functions, rather than the default (useless) Array.equals.
Also adds @ExperimentalArrayContentSupport opt-in annotation, as this is a work in progress. I'll decide later whether to keep this in the first release with this feature or drop it.

I chose the more verbose @ArrayContentBased rather than just @ContentBased because I want it to be very obvious that this only affects the handling of array properties (especially when a property may be Any).


Incorrect value informed for array.filter() result

Request type

[ ] Please close this issue, I accidentally submitted it without adding any details
[ ] New documentation
[X] Correction or update

Details

In the page:
https://developer.mozilla.org/pt-BR/docs/Web/JavaScript/Reference/Global_Objects/Array/filtro#Exemplo_Filtrando_entradas_inválidas_para_JSON
In the example "Exemplo: Filtrando entradas inválidas para JSON", the number of entries on the new array should be 4 (four), instead of 5 (five).
...
console.log('Number of Invalid Entries = ', invalidEntries);
// Number of Invalid Entries = 5

This last "number 5" informed is wrong. The .filter() on the informed array returns a new array with 4 values (length of 4, so 4 entries...).
Thanks.

I've tested it and ... yes, you are right!
I've updated it now; thanks for reporting it.


remove unused code in util file

What problem does this PR solve?
Remove unused code
What is changed and how does it work?
Check List
Tests

Unit test

Code changes

Has Go code change

Side effects
n/a
Related changes
Does this PR introduce a user-facing change?:

NONE

/merge

Codecov Report

Merging #397 into master will increase coverage by 0.50%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #397      +/-   ##
==========================================
+ Coverage   58.16%   58.66%   +0.50%
==========================================
  Files          60       60
  Lines        3650     3624      -26
==========================================
+ Hits         2123     2126       +3
+ Misses       1355     1330      -25
+ Partials      172      168       -4

Impacted Files
Coverage Δ

pkg/chaosdaemon/util.go
81.66% <ø> (+1.33%)
:arrow_up:

controllers/timechaos_controller.go
0.00% <0.00%> (ø)

api/v1alpha1/timechaos_types.go
4.34% <0.00%> (+0.18%)
:arrow_up:

api/v1alpha1/kernelchaos_types.go
4.65% <0.00%> (+0.20%)
:arrow_up:

api/v1alpha1/networkchaos_types.go
18.93% <0.00%> (+0.28%)
:arrow_up:

api/v1alpha1/iochaos_types.go
32.55% <0.00%> (+1.44%)
:arrow_up:

api/v1alpha1/podchaos_types.go
32.55% <0.00%> (+1.44%)
:arrow_up:

controllers/iochaos_controller.go
90.47% <0.00%> (+4.76%)
:arrow_up:

controllers/podchaos_controller.go
90.47% <0.00%> (+4.76%)
:arrow_up:

controllers/networkchaos_controller.go
90.47% <0.00%> (+4.76%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bdb15d3...b6e02e0. Read the comment docs.


bug(MatSelectionList): Non multiple selection-list returns Array

Reproduction
Use StackBlitz to reproduce your issue:

Components: https://stackblitz.com/edit/angular-8izayv

Steps to reproduce:

Open https://stackblitz.com/edit/angular-8izayv
Select an option;
Check the FormControl.value.

Expected Behavior
What behavior were you expecting to see?
I'd like to have the FormControl.value as a single value (not as Array), since I passed [multiple]="false".
Actual Behavior
What behavior did you actually see?
FormControl.value is an Array even with [multiple]="false".
Environment

Angular: 11
CDK/Material: 11
Browser(s):
Operating System (e.g. Windows, macOS, Ubuntu):

It's worth keeping in mind that if we decided to change it now, it would be a breaking change.


add: Community Page

Fixes Issue

closes #20
closes #22
Changes proposed

Screenshots

Note to reviewers

Code of Conduct

[ ] By submitting this pull request, I confirm I've read and complied with the CoC 🖖

Check List (Check all the applicable boxes)

[ ] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[ ] All new and existing tests passed.
[ ] This PR does not contain plagiarized content.
[ ] The title of my pull request is a short description of the requested changes.

You can also join our Discord community.
Feel free to check out other cool repositories of the Thecyberworld.
Join the Thecyberworld GitHub Organisation by raising an issue (you will be sent an invitation).

Hey @kabir0x23 can you assign settings page issue (feature) to me?


Intermittent error running terrascan - rego_parse_error: unexpected assign token: non-terminated set

terrascan version: unsure because we're calling it from super-linter
Operating System: linux

Description
We run terrascan as part of github's super-linter https://github.com/github/super-linter
Today we've experienced an intermittent error where terrascan failed with this error:
2022-03-03 10:15:09 [ERROR]   Found errors in [terrascan] linter!
2022-03-03 10:15:09 [ERROR]   Error code: 1. Command output:
------
2022-03-03T10:15:09.312Z	error	opa/engine.go:231	error compiling rego files{rule 15 0 allUsersReadWriteAccess <nil>} {raw rego 15 0 package accurics

allUsersReadWriteAccess[retVal] {
    bucket := input.aws_s3_bucket[_]
    bucket.config.acl == "public-read-write"
    traverse = "acl"
    retVal := { "Id": bucket.id, "ReplaceType": "edit", "CodeType": "attribute", "Traverse": traverse, "Attribute": "acl", "AttributeDataType": "string", "Expected": "private", "Actual": bucket.config.acl }
}

allUsersReadWriteAccess<no value>[retVal] {
    bucket := input.aws_s3_bucket_acl[_]
    bucket.config.acl == "public-read-write"
    traverse = "acl"
    retVal := { "Id": bucket.id, "ReplaceType": "edit", "CodeType": "attribute", "Traverse": traverse, "Attribute": "acl", "AttributeDataType": "string", "Expected": "private", "Actual": bucket.config.acl }
} <nil>} {error 26 0  1 error occurred: allUsersReadWriteAccess:11: rego_parse_error: unexpected assign token: non-terminated set
	    bucket := input.aws_s3_bucket_acl[_]
	           ^}
2022-03-03T10:15:09.312Z	error	opa/engine.go:267	error compiling rego files{policy path 15 0 /github/home/.terrascan/pkg/policies/opa/rego/aws <nil>} {error 26 0  1 error occurred: allUsersReadWriteAccess:11: rego_parse_error: unexpected assign token: non-terminated set
	    bucket := input.aws_s3_bucket_acl[_]
	           ^}
2022-03-03T10:15:09.312Z	error	runtime/executor.go:198	failed to initialize policy engine for path /github/home/.terrascan/pkg/policies/opa/rego/aws, error: failed to initialize OPA policy engine
2022-03-03T10:15:09.312Z	error	runtime/executor.go:199	perform 'terrascan init' command and then try running the scan command again
2022-03-03T10:15:09.312Z	error	cli/run.go:135	scan run failed{error 26 0  failed to initialize OPA policy engine}

This occurred in two separate github actions workflow runs, at

2022-03-03 10:15 GMT
2022-03-03 10:57 GMT

We have had subsequent runs which have gotten past this point of failure so it does seem like an intermittent error. I thought it was worth reporting so that you folks are aware and in case you might want to build in some mitigation.
As I say subsequent runs have succeeded so there is no issue for us.
What I Did
We don't explicitly run the command, we run it from super-linter. Here is our super-linter configuration.
  super-lint:
    name: Lint Code Base
    runs-on: [self-hosted, ubuntu, dataplatform]
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Run Super-Linter
        uses: github/super-linter/slim@v4
        env:
          DEFAULT_BRANCH: main
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          VALIDATE_ALL_CODEBASE: false
          VALIDATE_PYTHON_ISORT: false
          VALIDATE_DOCKERFILE: false
          VALIDATE_DOCKERFILE_HADOLINT: false
          VALIDATE_GITHUB_ACTIONS: false # disabling until https://github.com/rhysd/actionlint/issues/114 is fixed
          VALIDATE_SQLFLUFF: false

Unfortunately I can't provide a link to the failing job because its in a private repository.
Hope this is useful.

@jamiet-msm thanks for reporting this issue. We just fixed it, and have a thread related to this.


Propagate residual as part of error bars to DOS

In mp.backward.sqe2dos.

A plot of DOS with errorbar from residual included. C_ms was set to 1.0 to exaggerate the residual.


Install dependencies in the first pass

Fixes #56. @GnaphronG Please confirm me this PR is ok and I'll apply the fix to other variants.

The bug:
~/p/ffmpeg> docker run -it ffmpeg-fixed-4
root@ec751a3d68b4:/# ffmpeg
ffmpeg: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory
root@ec751a3d68b4:/# ldd /usr/local/bin/ffmpeg
	linux-vdso.so.1 =>  (0x00007ffe5f38f000)
	libavdevice.so.57 => /usr/local/lib/libavdevice.so.57 (0x00007fe43bc53000)
	libavfilter.so.6 => /usr/local/lib/libavfilter.so.6 (0x00007fe43b858000)
	libavformat.so.57 => /usr/local/lib/libavformat.so.57 (0x00007fe43b49b000)
	libavcodec.so.57 => /usr/local/lib/libavcodec.so.57 (0x00007fe43a1a0000)
	libavresample.so.3 => /usr/local/lib/libavresample.so.3 (0x00007fe439f83000)
	libpostproc.so.54 => /usr/local/lib/libpostproc.so.54 (0x00007fe439d6f000)
	libswresample.so.2 => /usr/local/lib/libswresample.so.2 (0x00007fe439b55000)
	libswscale.so.4 => /usr/local/lib/libswscale.so.4 (0x00007fe4398e0000)
	libavutil.so.55 => /usr/local/lib/libavutil.so.55 (0x00007fe43968b000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe439382000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe439165000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe438d9b000)
	libvidstab.so.1.1 => /usr/local/lib/libvidstab.so.1.1 (0x00007fe438b86000)
	libfreetype.so.6 => /usr/local/lib/libfreetype.so.6 (0x00007fe4388f0000)
	libass.so.9 => /usr/local/lib/libass.so.9 (0x00007fe4386c1000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe4384a7000)
	libxvidcore.so.4 => /usr/local/lib/libxvidcore.so.4 (0x00007fe438192000)
	libx265.so.110 => /usr/local/lib/libx265.so.110 (0x00007fe43711f000)
	libx264.so.148 => /usr/local/lib/libx264.so.148 (0x00007fe436d79000)
	libvpx.so.4 => /usr/local/lib/libvpx.so.4 (0x00007fe43694a000)
	libvorbisenc.so.2 => /usr/local/lib/libvorbisenc.so.2 (0x00007fe43669d000)
	libvorbis.so.0 => /usr/local/lib/libvorbis.so.0 (0x00007fe436464000)
	libtheoraenc.so.1 => /usr/local/lib/libtheoraenc.so.1 (0x00007fe436220000)
	libtheoradec.so.1 => /usr/local/lib/libtheoradec.so.1 (0x00007fe436004000)
	libopus.so.0 => /usr/local/lib/libopus.so.0 (0x00007fe435db3000)
	libopenjp2.so.7 => /usr/local/lib/libopenjp2.so.7 (0x00007fe435b63000)
	libopencore-amrwb.so.0 => /usr/local/lib/libopencore-amrwb.so.0 (0x00007fe43594f000)
	libopencore-amrnb.so.0 => /usr/local/lib/libopencore-amrnb.so.0 (0x00007fe435725000)
	libmp3lame.so.0 => /usr/local/lib/libmp3lame.so.0 (0x00007fe435468000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe435264000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe43be61000)
	libgomp.so.1 => not found
	libfribidi.so.0 => /usr/local/lib/libfribidi.so.0 (0x00007fe43504b000)
	libfontconfig.so.1 => /usr/local/lib/libfontconfig.so.1 (0x00007fe434e07000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fe434a85000)
	libogg.so.0 => /usr/local/lib/libogg.so.0 (0x00007fe43487e000)
	libexpat.so.1 => not found
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe434668000)

Both not-found where previously installed in the first RUN apt-get install.
While running a bash command within the container and issuing a apt-get install -y libgomp1 libexpat1-dev make ffmpeg work.
So my best bet was to put back these two in the first RUN of apt-get install.
Unfortunatly, I have no explanation as of why this is the solution... @GnaphronG Any reason expat was moved to buildDeps and libgomp1 removed from the requirements? Could ARG LD_LIBRARY_PATH=/opt/ffmpeg/lib causes a problem for subsequent library installs?

This PR is a duplicate of #58 from @GnaphronG. Closing.


NODE_F207ZG not found, and having motors move

I supplied the expansion board for the stepper motor drivers with 24 V.
I tried running the example program on a Nucleo-144 F767ZU and an Arduino Mega2560.
Here is the same error message for both attempts:
Arduino: 1.8.9 (Windows 7), Board: "Nucleo-144, Nucleo F207ZG, Mass Storage, Enabled (generic 'Serial'), None, Low/Full Speed, Smallest (-Os default), Newlib Nano (default)"

A subdirectory or file sketch already exists.

Sketch uses 34916 bytes (3%) of program storage space. Maximum is 1048576 bytes.
Global variables use 1348 bytes (1%) of dynamic memory, leaving 129724 bytes for local variables. Maximum is 131072 bytes.
NODE_F207ZG not found. Please ensure the device is correctly connected.
An error occurred while uploading the sketch

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

Hi @adamaero
you have to select the right "Board part number" in the Tools Menu.
https://github.com/stm32duino/wiki/wiki/Getting-Started

Thank you. It seems to upload to the F767ZI board.
I'm used to Arduino programs automatically running.
The stepper motors are not moving. Is there an additional step I'm missing?

@cparata any though on this?

Hi,
I guess the issue is due to the fact that on F767ZI in D3 (PE13) we do not have the SPI1 clock. So you are obliged to use D13 instead D3 as SPI clock. In order to do that you need to unsolder SB34 and solder SB12 on X-NUCLEO-IHM02A1 and then initialize the SPI instance in the sketch in this way:
dev_spi = new SPIClass(D11, D12, D13);
I hope that this tip can help you to solve your issue.
Best Regards,
Carlo

You could try to wire manually before rework the SB in order to test.

Success. Thank you both very much!
Now I just need to figure out how to add a third motor to the program (using a second X-NUCLEO-IHM02A1), and integrate that program with the encoders.


Correct branches for GitFlow workflow created - includes master, develop, and release branches

Added master, develop and release branches to repo

Develop and Release will be pushed with latest code changes

okay sure ill update


Type issues when training weights are matrices

First of all, thanks for publishing this very interesting package!
I am trying to use this to do some backprop through an ODE where my parameters are matrices rather than a collection of scalars, as in the example. I am having trouble wrangling the type system into getting this to work.
If I try the approach given by the code below
p = [a, b]
prob = ODEProblem(deriv, u0, (0.0, T), p)
# sol  = solve(prob, SSPRK83(), dt=dt, progress=true, abstol=1e-6, reltol=1e-6, progress_steps=100);
params = Flux.Params([param(a), param(b)]);

function predict_rd()
  Array(diffeq_rd(param([a, b]), prob, SSPRK83(), dt=dt, progress=true, abstol=1e-6, reltol=1e-6, progress_steps=100))
end

loss_rd() = 1/sum(abs2, x for x in predict_rd()[400, 70, 1, :])

cb =  display(loss_rd())

I get the error
ERROR: MethodError: no method matching zero(::Type{Array{Float64,2}})
Closest candidates are:
  zero(::Type{LibGit2.GitHash}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/LibGit2/src/oid.jl:220
  zero(::Type{Pkg.Resolve.VersionWeights.VersionWeight}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/resolve/VersionWeights.jl:19
  zero(::Type{Pkg.Resolve.MaxSum.FieldValues.FieldValue}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.1/Pkg/src/resolve/FieldValues.jl:44
  ...
Stacktrace:
 [1] zero(::Array{Array{Float64,2},1}) at ./abstractarray.jl:849
 [2] TrackedArray(::Array{Array{Float64,2},1}) at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:32
 [3] param(::Array{Array{Float64,2},1}) at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/Tracker.jl:103
 [4] predict_rd() at ./none:2
 [5] loss_rd() at ./none:1
 [6] top-level scope at none:0

I think this must have something to do with my wrapping of param().

Trying the following
function predict_rd()
  Array(diffeq_rd([param(a), param(b)], prob, SSPRK83(), dt=dt, progress=true, abstol=1e-6, reltol=1e-6, progress_steps=100))
end

gives
ERROR: MethodError: Cannot `convert` an object of type Float64 to an object of type TrackedArray{…,Array{Float64,2}}
Closest candidates are:
  convert(::Type{T<:TrackedArray}, ::S<:T<:TrackedArray) where {T<:TrackedArray, S<:T<:TrackedArray} at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:36
  convert(::Type{#s21} where #s21<:TrackedArray, ::TrackedArray) at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:38
  convert(::Type{#s21} where #s21<:TrackedArray{T,N,A}, ::AbstractArray) where {T, N, A} at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:41
  ...
Stacktrace:
 [1] _broadcast_getindex at ./broadcast.jl:578 [inlined]
 [2] getindex at ./broadcast.jl:511 [inlined]
 [3] copy at ./broadcast.jl:787 [inlined]
 [4] materialize at ./broadcast.jl:753 [inlined]
 [5] #diffeq_rd#1(::Array{Float64,3}, ::Base.Iterators.Pairs{Symbol,Real,NTuple{5,Symbol},NamedTuple{(:dt, :progress, :abstol, :reltol, :progress_steps),Tuple{Float64,Bool,Float64,Float64,Int64}}}, ::Function, ::Array{TrackedArray{…,Array{Float64,2}},1}, ::ODEProblem{Array{Float64,3},Tuple{Float64,Float64},true,Array{Array{Float64,2},1},ODEFunction{true,wave_equation,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing,DiffEqBase.StandardODEProblem}, ::SSPRK83{typeof(OrdinaryDiffEq.trivial_limiter!),typeof(OrdinaryDiffEq.trivial_limiter!)}) at /Users/iwill/.julia/packages/DiffEqFlux/1w1tX/src/Flux/layers.jl:8
 [6] (::getfield(DiffEqFlux, Symbol("#kw##diffeq_rd")))(::NamedTuple{(:dt, :progress, :abstol, :reltol, :progress_steps),Tuple{Float64,Bool,Float64,Float64,Int64}}, ::typeof(diffeq_rd), ::Array{TrackedArray{…,Array{Float64,2}},1}, ::ODEProblem{Array{Float64,3},Tuple{Float64,Float64},true,Array{Array{Float64,2},1},ODEFunction{true,wave_equation,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing,DiffEqBase.StandardODEProblem}, ::SSPRK83{typeof(OrdinaryDiffEq.trivial_limiter!),typeof(OrdinaryDiffEq.trivial_limiter!)}) at ./none:0
 [7] predict_rd() at ./none:2
 [8] loss_rd() at ./none:1
 [9] top-level scope at none:0

Flux internally makes the weights of its neural network be Float32 matrices. That's why when using neural_ode we set u0 to be Float32. If you have all Float64 parameters, you should set u0 to be Float64. Basically, you just want everything to match types. I'll need a working example if you'd like me to run it.

I thought my types were consistent, but I changed everything to Float32 just in case. It seems like this error is coming down the line 3 of diffeq_rd():
_prob = remake(prob,u0=convert.(eltype(p),u0),p=p)

where it's trying to convert u0, which is now Float32 into the element type of p which is a TrackedArray{…,Array{Float32,2}}. The error thrown at the above line of diffeq_rd() is
ERROR: MethodError: Cannot `convert` an object of type Float32 to an object of type TrackedArray{…,Array{Float32,2}}
Closest candidates are:
  convert(::Type{T<:TrackedArray}, ::S<:T<:TrackedArray) where {T<:TrackedArray, S<:T<:TrackedArray} at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:36
  convert(::Type{#s21} where #s21<:TrackedArray, ::TrackedArray) at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:38
  convert(::Type{#s21} where #s21<:TrackedArray{T,N,A}, ::AbstractArray) where {T, N, A} at /Users/iwill/.julia/packages/Flux/GBuXv/src/tracker/lib/array.jl:41

I've assigned p as
a = ones(Float32, Nx, Ny)
b = ones(Float32, Nx, Ny)
p = [a, b]

and passed it to diffeq_rd as
diffeq_rd( param.(p), ...)

The difference from the example is that a and b are matrices rather than scalars, so you end up with an array of arrays situation, which is what is confusing diffeq_rd I think. Is this the correct way to pass them to diffeq_rd?

yeah, that should be using a recursive element type. Do you have an example code I can work with?

I just added you to the private repo that this code is in. Feel free to take a look at it whenever you get a chance. You can feel free to follow up via email, which is on my website linked in my profile

We have encounted a similar problem here. We could sent you an example code as well.
Thanks a lot!

Your example now works with DiffEqFlux.jl's current master, in a sense. If you make that Array{TrackedArray} a vector and just use views in your PDE discretization, the diffeq_adjoint just works. I know, not the nicest interface, but that'll always do, and I don't see that changing soon since writing an adjoint ODE on something where length(p) isn't the true length you want is hard!
For diffeq_rd, https://github.com/JuliaDiffEq/DiffEqFlux.jl/commit/daaac9aac9d93e92526ec06663a5a4dbe1af6593 fixes the type issue. So any ODE that is properly setup will work with it now! However, in your example code there was still an issue because you made a lot of assumptions that your parameters were going to be Float64 instead of TrackedReal{Float64} or TrackedArray. For more information on how to handle this, see the DiffEq FAQ. For all of your delta functions, just remove the type restrictions. Then in your delta buffer, you need to allow that to be an array of Tracked values as well. This example has some details: http://docs.juliadiffeq.org/latest/basics/faq.html#I-get-Dual-number-errors-when-I-solve-my-ODE-with-Rosenbrock-or-SDIRK-methods...?-1 . It only took about 10-15 lines of changes to your PDE's MOL discretization to make that work, but let me know if you have any questions. So kudos and thanks for the report!


Fix application output

This fixes the finished message not appearing when in use with a custom server.
This looks like it was introduced by a bad git merge. 🤔
Fixes #7259
And https://github.com/zeit/next.js/issues/4016#issuecomment-484525764

I am experiencing this issue on the latest version ^8.1.0. Has this fix been promoted to the master branch or is this on in canary? If it is only in canary then do you want us to update to the canary branch?

I got the same issue, and I tried to do a next build, and I got the error that in css file a path to image was wrong, changed the path, did the build, and then run the server again, all good, strange but it worked.

@Valnexus Same in my case, wrong path in css.

 Pffffffffff

@kozborn @Valnexus @0m15 it's probably a good idea to read through this: https://twitter.com/timneutkens/status/1154351043049926656
It's not really useful to post "Same issue" on merged PRs, closed issues etc. Instead it is helpful if you provide a clear and concise reproduction of the issue you're having so that we can have a look and try to help you. Thanks 🙏


gtk4-macros: Blueprint sources should not include the using Gtk 4.0 declaration by default

Bug description
The current implementation of Blueprint templates automatically prepends the declaration using Gtk 4.0;. This is part of Blueprint's syntax, and omitting it in a Blueprint file causes the language server to error out.
Fixing it is a simple one-line change in gtk4-macros/blueprint.rs, specifically deleting line 17 here:
https://github.com/gtk-rs/gtk4-rs/blob/3a996287385051ea5c8d213d6696d28eaae66ede/gtk4-macros/src/blueprint.rs#L16-L20
Obviously the tests need to be modified, but that's not a big problem.
Backtrace
N/A

Seems like this was fixed, I'll close this issue for now.


Typo in fr/alpes_maritimes dataset

Hello,
First thanks for your great work !
I have an issue with the dataset I downloaded here: http://results.openaddresses.io/sources/fr/alpes_maritimes
There is a typo in an address:
all the addresses like this:
chemin de peidesalle
should be like this:
chemin de peidessalle
There should be 2 s's.
You can find it on openstreetmap:
https://www.openstreetmap.org/search?query=230 Chemin De Peidessalle#map=19/43.64001/7.02979
When searching on https://openaddresses.io/ map search engine I can also find the correct address, but it is not on the dataset.
Should I parse it and submit the correct address ?
Thanks for your help !

Looking thru the data provided by the French government that OpenAddresses sources, I see that there are two possible fields for street name: nom_voie and nom_afnor.  Here is the header row and a sample with the street name you mentioned:
ADRNIVX_0000000261692859;chemin de peidesalle;0242;152;B;06152;06560;"";"";CHEMIN DE PEIDESSALLE;VALBONNE;1025240.3;6290288.8;7.03113256592113;43.6369108193955;Valbonne

I don't know French well enough to say for certain, but it appears that nom_afnor is more accurate than nom_voie in this instance.  The metadata document says that nom_afnor means (translated) "Name standardized according to postal standard".  It also says that nom_voie means (again, translated): "Track name in lowercase
Accented with alternative names
potential".  In light of this information, I think we can safely switch to nom_afnor if you think this is the correct course of action.  Please advise.

Hello,
Sorry to respond this late.
Indeed the nom_afnor is the normalized name use by the postal services. I think we should use this one.
But I think the nom_voie is not the correct one (it should be the same as nom_afnor in lower case but with accents if there are any).
Could you tell me the contact email address so I can send them a notice to change the nom_voie.
Thanks.

OA pulls the data from https://adresse.data.gouv.fr/ which is run by the French goverment.  We don't have a relationship with them directly, but this is the contact email address on the site is<EMAIL_ADDRESS>

Thanks I'll contact them.

Wait, looking at the data file now, it contains many incorrect normalized names like "RUE DE L EGLISE" instead of "rue de l'église".
Elisions and accents are not super easy to add back in once they've been removed. The elisions alone affect > 18k addresses in exchange for fixing one spelling mistake (or maybe a handful). I think it's better to contact the government and ask them to correct that mistake than to use normalizations in place of data.

So this should be switched back and we'll just lose the SS but retain the elisions and accents?

Yeah, spot checked a few cases and it looks like nom_voie is as close to the real street name as is possible from this file. nom_afnor is sort of like using a USPS address validation response.

K, sorry @kei42, we'll have to switch it back for more accurate data in the vast majority of cases.  Hopefully an email to the addressing authority can get the SS into nom_voie.

Indeed, I haven't considered the apostrophe and that the accent could be a problem, it's my bad.
thanks @thatdatabaseguy to pointing it out.


Project Closeout Report

Closing this in favor of the dedicated repo for all closeout reports:
https://github.com/Anastasia-Labs/project-close-out-reports


[ldd-clipper] Update orbit_number description

Issue Type
This is an enhancement for the ldd-clipper mission dictionary.
Describe the issue identified (if applicable)
The current description for orbit_number is generic.
Describe the solution you'd like
I suggest the a more Clipper-specific description as follows:

The orbit_number attribute identifies the number of the orbital revolution of the spacecraft around Jupiter. Prior to JOI (i.e., from launch through cruise including gravity assists), the orbit value will be 0. After JOI, starting with the first apojove, the value will increase by 1 with each revolution around Jupiter incrementing at each subsequent apojove.

Describe alternatives you've considered
N/A
LDD Dictionary Version
N/A
PDS4 IM Version
N/A
Need-by Date
Summer 2023
Additional context
This value is captured within the mission as a 9-bit unsigned integer meaning it has a valid value range of 0 to 511. I am not sure how common this is within other mission dictionaries, but we could put a value range restriction on this element, if desired.

We can set a minimum_value = 0 and a maximum_value=511. Seems like this could be limiting for any extended mission. Maybe set a maximum_value=999 or round up to 1000...?

Please review: Please review PR: https://github.com/pds-data-dictionaries/ldd-clipper/pull/6

The orbit is limited by its definition as a 9-bit integer, so 511 is the true maximum value. There are going to be 83 orbits in the prime mission, so I don't believe there are any issues overrunning the maximum value in an extended mission.

Okay - set down to 511, PR rebuilding schemas now.


Refactored to fix multiple issues.

As it's nearly Christmas 🎄 I thought I'd share this gift 🎁
Tagging those who have expressed interest: @charettes @sarahboyce

Includes the following changes (in no particular order):

Removed outdated configuration for Azure Pipelines
Removed outdated configuration for Travis CI
Removed obsolete PIP_CACHE_VOLUME from .env
Removed obsolete entries from .gitignore
Removed obsolete support for unofficial Oracle images
Removed obsolete docker_push.sh script
Added workflow for linting in GitHub Actions
Added pre-commit, hadolint, ruff, yamllint, and zizmor for linting
Defaulted DJANGO_PATH to ../django, i.e. assume adjacent checkouts
Removed reference to ${PWD} which is not available on Windows
Updated Dockerfile to use modern syntax, including:

Using here documents with RUN to avoid backslash escaping
Using --mount=type=cache,... with RUN to improve build speed

Updated apt-get and pip installation to be more optimal:

Enabled caching of downloads for both commands
Extracted packages to external files
Combined requirements files to optimise dependency resolution
Switched to using uv for installation of Python dependencies

Included set -o errexit -o nounset -o pipefail -o xtrace for RUN
Added forced color output, etc. to Dockerfile for use in CI
Tided up the location of injected and generated files in the image
Added preliminary support for alternative Python interpreters, specifically PyPy
Updated docker-compose.yml to use modern features:

Dropped the non-required version property
Used additional_contexts to source files from multiple places
Used deploy: mode: global for databases, caches, etc.
Reverted to using healthcheck in place of wait-for-it.sh

Added configuration to speed up database services
Added support for Oracle Database Free 23ai
Added GIS support for MariaDB and Oracle
Added a container for Redis to enable testing the Redis cache backend
Added Edge support for testing with Selenium
Fixed building of documentation using Sphinx
Renamed Dockerfile to Containerfile
Renamed docker-compose.yml to compose.yml
Merged settings files into a single dynamic file to reduce maintenance
Updated README to document all of the changes and features
Added a LICENSE file using the same license as Django

Fixes #4, closes #5, resolves #8, fixes #42, closes #47.

That's quite the  🎄 gift indeed Nick!!!
I should be able to give it a run during the holidays but assuming this is based on your work in the Oracle issue branch I can confirm that I've been using with success for Oracle, MySQL, and Postgres for about 6 months now.


Add run name to tensorboardx patch

wandb, version 0.8.5
Python 3.7.3
Linux
Please change the "run name" to the name provided name by SummaryWriter through tensorboardX.

Hey dream flasher, do you mean the name of the tfevent file?   Would you want the machine and timestamp included as well?

I mean this writer = SummaryWriter(experiment_unique_name)
I'd appreciate if this name shows up as the run name.
Currently the run names are "clear-sunset-60" and thus meaningless.

Thanks for pointing out that I can call wandb.init(name=unique_name, sync_tensorboard=True) to specifiy the name. That works for me, but I think it would still be nice to have that synced right away.

Hey @dreamflasher
I think due to the fact that https://pytorch.org/docs/stable/_modules/torch/utils/tensorboard/writer.html#SummaryWriter takes the log_dir as the directory in which the tf event files would log, wandb would not have any way to figuring out the name of the directory. It would be beneficial to use either the init as you mentioned, or using filename as a variable and using it in the init for naming the run as you want.
CC: @cvphelps


Date field: field setting »Display time« should be honored in index view too

When I have a date field only used to show dates without time that works fine in the edit view. In the index list I get arbitrary times that might confuse my customer :)

Oups!

@animaux Can you please confirm that a942068 fixes the problem ?
@brendo @michael-e can you review please ?

Seems to work! Thanks for the quick fix. 2.6 feels like lovely so far :)

My pleasure! We hope it is!

@animaux Can you please test my latest commit ? I've refactored the thing. Thanks!

Looks good!

Awesome thanks.

No no no. Thank you!
:)

:smile:

Looks good, thanks @nitriques

My pleasure.


Improve transaction modeling and authorization hashing.

This is a prelude to being able to do a useful custody / signing protocol.
As mentioned in #810:

In order to have a useful signing/custody protocol, we need to classify transaction data
into authorizing data (which describes the actions that signatures authorize)
and effecting data (which effects those actions, but is not necessary to
authorize, e.g., proof data). Then we'll want to have a protocol that allows
us to bundle up only the authorizing data and send it to a signer, who signs an
authorization hash (a renamed sighash) of the authorization data. However,
this means that we want to pull apart the proto structures into chunks, where
each chunk has only authorizing /or/ effecting data. To do /that/, it'd be
useful to be able to describe an OutputBody, which the NotePayload isn't
really.

This PR makes two substantive changes:
First, it changes the definition of the transaction structures and protos to
split out "authorizing data", data which describes what actions the transaction
does, which should be fed into an authorization hash for signing, and
"effecting data", which effects the actions described by the authorizing data,
but does not itself describe them, and which the authorization does not need to
be bound to.
This distinction is now consistently mapped to the Foo/FooBody distinction:
FooBody contains the authorizing data for Foo, and Foo contains both a
FooBody as well as any effecting data.
In particular, we move the anchor and the ZK proofs out of the Body
structures and consider the effecting data.  This allows a custodian to
authorize a transaction without restricting when the prover can create the
transaction proof.
Otherwise, if the anchor were included in the authorization hash, the prover
would be forced to use whatever anchor was available at the point that they
requested signing, and this creates a side channel that allows a third party to
learn information about what kind of custody was used for transaction
authorization.  For instance, threshold signing with offline keys necessarily
has high latency, which would force a prover to use an anchor that would be
hours or days out of date, distinguishing it from other transactions that used
recent anchors.
Second, it removes the old sighash code, and replaces with a new AuthHash
implementation. This serves the same purpose, but:

has a better name: it's the hash of the authorizing data (as opposed to
effecting data);

is specified explicitly rather than implicitly, so that we don't require a
signer to have the exact same protobuf implementation we use.

These changes lay the groundwork for a custody protocol, as mentioned in #587.

I think this design is basically good, and requested review from everyone primarily to flag the design changes mentioned in the PR description rather than to request careful, line-by-line auditing, but if I missed something feel free to flag it.


feat add stream operation for StreamReader

What this PR does / why we need it?
feat add stream operation for StreamReader
Summary of your change
Please indicate you've done the following:

[ ] Made sure tests are passing and test coverage is added if needed.
[ ] Made sure commit message follow the rule of Conventional Commits specification.
[ ] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.

Codecov Report
Patch coverage: 78.57% and project coverage change: -1.97 :warning:

Comparison is base (83746b3) 75.97% compared to head (bf9b9d2) 74.01%.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1550      +/-   ##
============================================
- Coverage     75.97%   74.01%   -1.97%
+ Complexity    20535    19777     -758
============================================
  Files           629      619      -10
  Lines         80941    80705     -236
  Branches      17697    17648      -49
============================================
- Hits          61493    59730    -1763
- Misses        12149    13505    +1356
- Partials       7299     7470     +171

Impacted Files
Coverage Δ

...com/alibaba/fastjson2/stream/JSONStreamReader.java
75.00% <ø> (ø)

...libaba/fastjson2/stream/JSONStreamReaderUTF16.java
52.63% <ø> (ø)

...alibaba/fastjson2/stream/JSONStreamReaderUTF8.java
66.33% <ø> (ø)

...a/com/alibaba/fastjson2/support/csv/CSVReader.java
78.12% <ø> (-1.43%)
:arrow_down:

...ava/com/alibaba/fastjson2/stream/StreamReader.java
64.21% <78.57%> (-16.84%)
:arrow_down:

... and 75 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.