arc64 local variable corruption in z_init_static_threads

Describe the bug
When running the sample.portability.cmsis_rtos_v2.timer_synchronization test on arc64 under qemu using picolibc, z_init_static_threads goes into an infinite loop in the second instance of _FOREACH_STATIC_THREAD where it's calling schedule_new_thread.
Debugging this, I discover that the register holding the termination value has been trashed and holds a completely bogus value (0x4275de2 instead of 0x80407594). Perturbing the code above the loop in essentially any way at all (I added __asm__("") just after k_sched_lock()) makes this issue go away. The difference in the assembly code is trivial -- it moves the load of __static_thread_data_list_start after the call to k_sched_lock
Please also mention any information which could help others to understand
the problem you're facing:

What target platform are you using?

qemu_arc_hs5x
SDK version zephyr-sdk-0.16.2-rc1-5-g8d9b4fe
picolibc instead of minimal C library.

What have you tried to diagnose or workaround this issue?

Modifying the code in even trivial ways appears to work around this issue. I applied this diff:
diff --git a/kernel/thread.c b/kernel/thread.c
index fb578eb48d..5721023929 100644
--- a/kernel/thread.c
+++ b/kernel/thread.c
@@ -783,6 +783,7 @@ void z_init_static_threads(void)
         * delay of K_FOREVER.
         */
        k_sched_lock();
+       __asm__("");
        _FOREACH_STATIC_THREAD(thread_data) {
                if (thread_data->init_delay != K_TICKS_FOREVER) {
                        schedule_new_thread(thread_data->init_thread,

This generated the following change in the generated code:
--- broken.S	2023-07-05 23:27:30.536430480 -0700
+++ working.S	2023-07-05 23:27:18.483894200 -0700
@@ -1073,8 +1073,8 @@
 	add2	r14,r14,11
 .L106:
 	blo_s	.L107
-	mov_s	r14,@__static_thread_data_list_start
 	bl	@k_sched_lock
+	mov_s	r14,@__static_thread_data_list_start
 .L108:
 	mov_s	r0,@__static_thread_data_list_end
 	cmp_s	r14,r0

setting a break point on the cmp_s instruction here also makes things work. That makes me suspect that qemu might be at fault somehow?

Is this a regression? If yes, have you been able to "git bisect" it to a
specific commit?

I haven't tried bisecting.

To Reproduce
Steps to reproduce the behavior:

./scripts/twister -O arc64-bug -xCONFIG_PICOLIBC=y -p qemu_arc_hs5x --test samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization

Expected behavior
Test completes successfully
Impact
This will delay the switch to using picolibc as the default C library.
Environment (please complete the following information):

OS: Debian unstable
Toolchain: SDK version 0.16.2-rc1-5-g8d9b4fe
Commit SHA or Version used: 8560fb319249e3fad3122f8541df4c4beef6e5e2 (main as of this writing)

Thanks Keith for the initial investigations, I'll assign to @evgeniy-paltsev.

Hi @keith-packard
as I can see you are not using release SDK version. I'm wondering if I can download this 0.16.2-rc1-5-g8d9b4fe SDK from somewhere? Or I need to build it myself?
Thanks!

sorry about the random SDK build. Turns out the regular 0.16.2-rc1 also shows the problem, get that from here:
https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.16.2-rc1/zephyr-sdk-0.16.2-rc1_linux-x86_64.tar.xz
Note that the error I get is this:
ASSERTION FAIL [!sys_dnode_is_linked(&to->node)] @ ZEPHYR_BASE/kernel/timeout.c:110

@keith-packard I'm wondering it this issue reproduces every time or not?
I've run twister 50 times in a loop - and I'm not able to reproduce it.
I was using v0.16.2-rc1 SDK (exact version as you've proposed last time), exact same twister command and exact same zephyr commit (https://github.com/zephyrproject-rtos/zephyr/commit/8560fb319249e3fad3122f8541df4c4beef6e5e2).

Note that the error I get is this:
ASSERTION FAIL [!sys_dnode_is_linked(&to->node)] @ ZEPHYR_BASE/kernel/timeout.c:110

May I also ask you to send full error log?

oof, I was hoping this would be easy to reproduce...
I think the only bits that should affect this are the Zephyr version and the SDK version, right?
$ arc64-zephyr-elf-gcc --version
arc64-zephyr-elf-gcc (Zephyr SDK 0.16.2-rc1) 12.2.0
$ git describe --abbrev=12 --always HEAD
zephyr-v3.4.0-1074-g18636af6d665

I made sure to run 'west update' in the zephyr repo before building. Here's the commands and their output; you can see that it's just hanging today, not generating an error message.
$ west update
=== updating acpica (modules/lib/acpica):
HEAD is now at f16a0b4d0 Update version to 20230628
=== updating canopennode (modules/lib/canopennode):
HEAD is now at dec12fa stack: map CANopenNode crc16_ccitt() to Zephyr crc16_itu_t()
=== updating chre (modules/lib/chre):
HEAD is now at b7955c27 Merge pull request #7 from stephanosio/require_full_libcpp
=== updating cmsis (modules/hal/cmsis):
HEAD is now at 74981bf Update DSP library for additional platforms
=== updating edtt (tools/edtt):
HEAD is now at 64e5105 edttt_bsim: Avoid infinite exception loop
=== updating fatfs (modules/fs/fatfs):
HEAD is now at 427159b zephyr: fix ffconf.h override use of STRINGIZE
=== updating hal_altera (modules/hal/altera):
HEAD is now at 0d225dd zephyr: update include paths to use <zephyr/...>
=== updating hal_atmel (modules/hal/atmel):
HEAD is now at 5ab4300 Remove LITTLE_ENDIAN and __LITTLE_ENDIAN defines
=== updating hal_espressif (modules/hal/espressif):
HEAD is now at 9531c1c178 hal: flash: avoid lock in ISR
=== updating hal_ethos_u (modules/hal/ethos_u):
HEAD is now at 90ada2e Clarify README regarding CPU choice
=== updating hal_gigadevice (modules/hal/gigadevice):
HEAD is now at 2994b7d README.md: add pllmf and pack exception for gd32a50x
=== updating hal_infineon (modules/hal/infineon):
HEAD is now at 0bebc14 hal: infineon: Add blob support for BLESS
=== updating hal_microchip (modules/hal/microchip):
HEAD is now at 5d079f1 mpfs:CMakeLists.txt: Adding mss_gpio directory to build
=== updating hal_nordic (modules/hal/nordic):
HEAD is now at 20ae0a7 nrfx: Update to version 3.1.0
=== updating hal_nuvoton (modules/hal/nuvoton):
HEAD is now at 8a2b5de hal: NuMaker M46x: fix warnings of M467 StdDriver
=== updating hal_nxp (modules/hal/nxp):
HEAD is now at e0116b8e s32k3: remove CONFIG_NXP_S32_PM
=== updating hal_openisa (modules/hal/openisa):
HEAD is now at d1e61c0 vega_sdk_riscv: Ignore GCC warning about potential uninitialized variable
=== updating hal_quicklogic (modules/hal/quicklogic):
HEAD is now at b3a66fe Remove references to FPGA IP core drivers
=== updating hal_renesas (modules/hal/renesas):
HEAD is now at f2d791d Add da1469x_hal
=== updating hal_rpi_pico (modules/hal/rpi_pico):
HEAD is now at b7801e4 Merge pull request #4 from soburi/update-1.5.0
=== updating hal_silabs (modules/hal/silabs):
HEAD is now at a143f03 gecko: add pa-conversions for efr32xg27
=== updating hal_st (modules/hal/st):
HEAD is now at 5948f7b3 sensor/stmemsc: Align stmemsc i/f to v2.02
=== updating hal_stm32 (modules/hal/stm32):
HEAD is now at 65b64134 dts: st: wba: generate stm32wba*-pinctrl.dtsi wba addition
=== updating hal_telink (modules/hal/telink):
HEAD is now at 38573af B91 ADC driver support
=== updating hal_ti (modules/hal/ti):
HEAD is now at ae1db23 zephyr: update SYS_INIT calls
=== updating hal_wurthelektronik (modules/hal/wurthelektronik):
HEAD is now at 24ca987 Initialized WE HAL module repository.
=== updating hal_xtensa (modules/hal/xtensa):
HEAD is now at 41a631d cmake: enable using SOC_TOOLCHAIN_NAME for SoC overlay
=== updating libmetal (modules/hal/libmetal):
HEAD is now at b91611a lib: update libmetal to release v2023.04.0
=== updating liblc3 (modules/lib/liblc3):
HEAD is now at 448f3de Merge pull request #8 from carlescufi/rename
=== updating littlefs (modules/fs/littlefs):
HEAD is now at ca583fd Merge tag 'v2.5.0' of https://github.com/littlefs-project/littlefs into zephyr
=== updating loramac-node (modules/lib/loramac-node):
HEAD is now at ce57712f Don't memset the TXRX buffer when radio is set to receive mode (#1345)
=== updating lvgl (modules/lib/gui/lvgl):
HEAD is now at f7cf01d4 Merge pull request #41 from faxe1008/update_8.3.7
=== updating lz4 (modules/lib/lz4):
HEAD is now at 8e303c2 zephyr: make this repo a zephyr module
=== updating mbedtls (modules/crypto/mbedtls):
HEAD is now at 6e7841e5a Move the implication of MBEDTLS_PSA_CRYPTO_CLIENT where it belongs
=== updating mcuboot (bootloader/mcuboot):
HEAD is now at 6a8746d7 boot_serial: fix image number handle in image upload request
=== updating mipi-sys-t (modules/debug/mipi-sys-t):
HEAD is now at a819419 library: allow building without wchar support
=== updating nanopb (modules/lib/nanopb):
HEAD is now at 42fa8b2 Merge pull request #2 from pdgendt/zephyr
=== updating net-tools (tools/net-tools):
HEAD is now at e0828aa net-setup: Add instructions for NAT and routing
=== updating nrf_hw_models (modules/bsim_hw_models/nrf_hw_models):
HEAD is now at 33b999d GPIO: Fix bug during initialization
=== updating open-amp (modules/lib/open-amp):
HEAD is now at c904a01 lib: update open-amp lib to release v2023.04.0
=== updating openthread (modules/lib/openthread):
HEAD is now at 37fb77098 [cli] fix `childsupervision` commands documentation (#9201)
=== updating picolibc (modules/lib/picolibc):
HEAD is now at d07c38ff0 Version 1.8.2
=== updating segger (modules/debug/segger):
HEAD is now at 5792675 segger: update to 3.40
=== updating sof (modules/audio/sof):
HEAD is now at ffbf9c2a6 Merge pull request #27 from nashif/cache_api
=== updating tflite-micro (modules/lib/tflite-micro):
HEAD is now at 9156d05 tflite-micro: fix module name
=== updating tinycrypt (modules/crypto/tinycrypt):
HEAD is now at 3e9a49d cmake: Fix conditional in root CMakeLists.txt
=== updating TraceRecorderSource (modules/debug/TraceRecorder):
HEAD is now at bc839bf zephyr: update SYS_INIT calls
=== updating trusted-firmware-m (modules/tee/tf-m/trusted-firmware-m):
HEAD is now at 79a6115d3 Build: Install crypto_platform.h for Crypto service
=== updating trusted-firmware-a (modules/tee/tf-a/trusted-firmware-a):
HEAD is now at 421dc0502 Merge pull request #3 from SgrrZhf/topics/huizha01/2.9.0-updates
=== updating tf-m-tests (modules/tee/tf-m/tf-m-tests):
HEAD is now at 0f80a65 Merge pull request #6 from microbuilder/tfm17_limit_attestation
=== updating psa-arch-tests (modules/tee/tf-m/psa-arch-tests):
HEAD is now at 6a17330 dev-apis: platform: Increase timeout for nRF devs
=== updating uoscore-uedhoc (modules/lib/uoscore-uedhoc):
HEAD is now at e892019 Merge pull request #11 from rlubos/zephyr-integration
=== updating zcbor (modules/lib/zcbor):
HEAD is now at 67fd8bb Update version from 0.6.99 to 0.7.0
=== updating zscilib (modules/lib/zscilib):
HEAD is now at 34c3432 Merge pull request #50 from microbuilder/ztest_fixes
=== updating thrift (modules/lib/thrift):
HEAD is now at 10023645a Merge pull request #1 from zephyrproject-rtos/add-zephyr-module-dir
$ ./scripts/twister -O arc64-bug -xCONFIG_PICOLIBC=y -p qemu_arc_hs5x --test samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization
Install tabulate python module with pip to use --device-testing option.
Renaming output directory to arc64-bug.6
INFO    - Using Ninja..
INFO    - Zephyr version: zephyr-v3.4.0-1074-g18636af6d665
INFO    - Using 'zephyr' toolchain.
INFO    - Building initial testsuite list...
INFO    - Writing JSON report arc64-bug/testplan.json
INFO    - JOBS: 64
INFO    - Adding tasks to the queue...
INFO    - Added initial list of jobs to queue
ERROR   - qemu_arc_hs5x             samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization  FAILED : Timeout
ERROR   - see: /build/keithp/zephyrproject/zephyr/arc64-bug/qemu_arc_hs5x/samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization/handler.log
INFO    - Total complete:    1/   1  100%  skipped:    0, failed:    1, error:    0
INFO    - 1 test scenarios (1 test instances) selected, 0 configurations skipped (0 by static filter, 0 at runtime).
INFO    - 0 of 1 test configurations passed (0.00%), 1 failed, 0 errored, 0 skipped with 0 warnings in 66.81 seconds
INFO    - In total 1 test cases were executed, 0 skipped on 1 out of total 584 platforms (0.17%)
INFO    - 1 test configurations executed on platforms, 0 test configurations were only built.
INFO    - Saving reports...
INFO    - Writing JSON report /build/keithp/zephyrproject/zephyr/arc64-bug/twister.json
INFO    - Writing xunit report /build/keithp/zephyrproject/zephyr/arc64-bug/twister.xml...
INFO    - Writing xunit report /build/keithp/zephyrproject/zephyr/arc64-bug/twister_report.xml...
INFO    - -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
INFO    - The following issues were found (showing the top 10 items):
INFO    - 1) samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization on qemu_arc_hs5x failed (Timeout)
INFO    -
INFO    - To rerun the tests, call twister using the following commandline:
INFO    - west twister -p <PLATFORM> -s <TEST ID>, for example:
INFO    -
INFO    - west twister -p qemu_arc_hs5x -s samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization
INFO    - or with west:
INFO    - west build -p -b qemu_arc_hs5x -T samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization
INFO    - -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
INFO    - Run completed
$ cat /build/keithp/zephyrproject/zephyr/arc64-bug/qemu_arc_hs5x/samples/subsys/portability/cmsis_rtos_v2/timer_synchronization/sample.portability.cmsis_rtos_v2.timer_synchronization/handler.log
*** Booting Zephyr OS build zephyr-v3.4.0-1074-g18636af6d665 ***

Did a bit more debugging today. With the code as above, the fault happens when vprintk is called from boot_banner.
(gdb) x/20i vprintk
   0x80400214 <vprintk>:	st.aw	r15,[sp,-4]
   0x80400218 <vprintk+4>:	st.aw	r14,[sp,-4]
   0x8040021c <vprintk+8>:	st.aw	blink,[sp,-4]
   0x80400220 <vprintk+12>:	sub3	sp,sp,0x2
   0x80400224 <vprintk+16>:	mov_s	r14,r0
   0x80400226 <vprintk+18>:	mov_s	r15,r1
   0x80400228 <vprintk+20>:	mov_s	r2,0x10
   0x8040022a <vprintk+22>:	mov_s	r1,0
   0x8040022c <vprintk+24>:	bl.d	27444	;0x80406d60 <memset>
   0x80400230 <vprintk+28>:	mov_s	r0,sp
   0x80400232 <vprintk+30>:	stb	2,[sp,2]
   0x80400236 <vprintk+34>:	mov_s	r0,0x804001fc
   0x8040023c <vprintk+40>:	mov_s	r2,r15
   0x8040023e <vprintk+42>:	st.as	r0,[sp,1]
   0x80400242 <vprintk+46>:	mov_s	r1,r14
   0x80400244 <vprintk+48>:	bl.d	27708	;0x80406e80 <vfprintf>
   0x80400248 <vprintk+52>:	mov_s	r0,sp
   0x8040024a <vprintk+54>:	add3	sp,sp,0x2
   0x8040024e <vprintk+58>:	ld.ab	blink,[sp,4]
   0x80400252 <vprintk+62>:	ld.ab	r14,[sp,4]

When I place a break point at 0x8040024a (right after vfprintf returns), I notice a difference in register contents depending upon whether I also place a break point at 0x80400242 (right before vprintf is called). When the system stops at 242 and is then continued, the application works correctly. When there is no break point at 242, the application hangs, having branched to address 0x0.
The register contents at 0x8040024a when the application works looks like this:
r0             0x41                65
r1             0x804087c9          -2143254583
r2             0x80408980          -2143254144
r3             0x80409c4c          -2143249332
r4             0xffffffff          -1
r5             0x0                 0
r6             0x80408a80          -2143253888
r7             0xffffffff          -1
r8             0x0                 0
r9             0x0                 0
r10            0x0                 0
r11            0x0                 0
r12            0x0                 0
r13            0xaaaaaaaa          -1431655766
r14            0x80408788          -2143254648
r15            0x80409c5c          -2143249316
r16            0x0                 0
r17            0x0                 0
r18            0x80409c9c          -2143249252
r19            0xaaaaaaaa          -1431655766
r20            0xaaaaaaaa          -1431655766
r21            0xaaaaaaaa          -1431655766
r22            0xaaaaaaaa          -1431655766
r23            0xaaaaaaaa          -1431655766
r24            0xaaaaaaaa          -1431655766
r25            0xaaaaaaaa          -1431655766
r26            0xaaaaaaaa          -1431655766
fp             0xaaaaaaaa          -1431655766
sp             0x80409c3c          -2143249348
ilink          0x80400244          -2143288764
gp             0x80409c94          -2143249260
blink          0x8040024a          -2143288758
acc0           0xaaaaaaaa          -1431655766
pcl            0x80400248          -2143288760
pc             0x8040024a          0x8040024a <vprintk+54>
status32       0x7ffe8008081c      [ E=14 Z RB=0 AD IE ]
bta            0x8040024a          0x8040024a <vprintk+54>
eret           0x0                 0x0
timer_build    0x7ffe00000304      [ version=4 t0 t1 p0=0 p1=0 ]
irq_build      0xf141901           [ version=1 IRQs=25 exts=20 p=15 ]
vecbase_build  0x10                [ version=4 addr=0 ]
isa_config     0x12e47402          [ version=2 va_size=4 pa_size=4 a=3 n m=2 c=0 d=1 ]
timer_count0   0x7fffcaaf<PHONE_NUMBER>
timer_ctrl0    0x3                 [ ie nh ]
timer_limit0   0x7fffcaaf<PHONE_NUMBER>
timer_count1   0x80000927          -2147481305
timer_ctrl1    0x0                 [ ]
timer_limit1   0xffffff            16777215
erstatus       0x0                 [ e=0 rb=0 ]
erbta          0x0                 0x0
ecr            0x0                 [ parameter=0 causecode=0 vectornumber=0 ]
efa            0x7ffe00000000<PHONE_NUMBER>20736
icause         0x0                 0
aux_irq_ctrl   0x207               [ nr=7 b ]
aux_irq_act    0x0                 [ active=0 ]
irq_priority_pending 0x0           0
aux_irq_hint   0x0                 0
irq_select     0x10                16
irq_enable     0x1                 true
irq_trigger    0x0                 false
irq_status     0x7ffe00000010      [ p=0 e ]
irq_pulse_cancel 0x0               false
irq_priority   0x0                 0
mmu_ctrl       {en = 0x0, ku = 0x0, wx = 0x0, tce = 0x0} {en = false, ku = false, wx = false, tce = false}
mmu_rtp0       {addr = 0x0, ASID = 0x0} {addr = 0x0, ASID = 0x0}
mmu_rtp1       {addr = 0x0, ASID = 0x0} {addr = 0x0, ASID = 0x0}

the register contents at 0x8040024a when the application fails looks like this:
r0             0x41                65
r1             0x804087c9          -2143254583
r2             0x80408980          -2143254144
r3             0x80409c4c          -2143249332
r4             0xffffffff          -1
r5             0x0                 0
r6             0x80408a80          -2143253888
r7             0xffffffff          -1
r8             0x0                 0
r9             0x0                 0
r10            0x0                 0
r11            0x0                 0
r12            0x0                 0
r13            0xaaaaaaaa          -1431655766
r14            0x80408788          -2143254648
r15            0x80409c5c          -2143249316
r16            0x0                 0
r17            0x0                 0
r18            0x80409c9c          -2143249252
r19            0xaaaaaaaa          -1431655766
r20            0xaaaaaaaa          -1431655766
r21            0xaaaaaaaa          -1431655766
r22            0xaaaaaaaa          -1431655766
r23            0xaaaaaaaa          -1431655766
r24            0xaaaaaaaa          -1431655766
r25            0xaaaaaaaa          -1431655766
r26            0xaaaaaaaa          -1431655766
fp             0xaaaaaaaa          -1431655766
sp             0x80409c38          -2143249352
ilink          0x8040024a          -2143288758
gp             0x80409c94          -2143249260
blink          0x8040024a          -2143288758
acc0           0xaaaaaaaa          -1431655766
pcl            0x8040024a          -2143288758
pc             0x8040024a          0x8040024a <vprintk+54>
status32       0x7ffc8008081c      [ E=14 Z RB=0 AD IE ]
bta            0x80401570          0x80401570 <_rirq_exit+72>
eret           0x0                 0x0
timer_build    0x7ffc00000304      [ version=4 t0 t1 p0=0 p1=0 ]
irq_build      0xf141901           [ version=1 IRQs=25 exts=20 p=15 ]
vecbase_build  0x10                [ version=4 addr=0 ]
isa_config     0x12e47402          [ version=2 va_size=4 pa_size=4 a=3 n m=2 c=0 d=1 ]
timer_count0   0x7fffcaaf<PHONE_NUMBER>
timer_ctrl0    0x3                 [ ie nh ]
timer_limit0   0x7fffcaaf<PHONE_NUMBER>
timer_count1   0x80000927          -2147481305
timer_ctrl1    0x0                 [ ]
timer_limit1   0xffffff            16777215
erstatus       0x0                 [ e=0 rb=0 ]
erbta          0x0                 0x0
ecr            0x0                 [ parameter=0 causecode=0 vectornumber=0 ]
efa            0x7ffc00000000<PHONE_NUMBER>86144
icause         0x0                 0
aux_irq_ctrl   0x207               [ nr=7 b ]
aux_irq_act    0x0                 [ active=0 ]
irq_priority_pending 0x0           0
aux_irq_hint   0x0                 0
irq_select     0x10                16
irq_enable     0x1                 true
irq_trigger    0x0                 false
irq_status     0x7ffc00000010      [ p=0 e ]
irq_pulse_cancel 0x0               false
irq_priority   0x0                 0
mmu_ctrl       {en = 0x0, ku = 0x0, wx = 0x0, tce = 0x0} {en = false, ku = false, wx = false, tce = false}
mmu_rtp0       {addr = 0x0, ASID = 0x0} {addr = 0x0, ASID = 0x0}
mmu_rtp1       {addr = 0x0, ASID = 0x0} {addr = 0x0, ASID = 0x0}

Checking the differences, we see:
--- reg-working	2023-07-13 11:54:43.495966032 -0700
+++ reg-broken	2023-07-13 11:26:44.252615552 -0700
@@ -27,17 +27,17 @@
 r25            0xaaaaaaaa          -1431655766
 r26            0xaaaaaaaa          -1431655766
 fp             0xaaaaaaaa          -1431655766
-sp             0x80409c3c          -2143249348
-ilink          0x80400244          -2143288764
+sp             0x80409c38          -2143249352
+ilink          0x8040024a          -2143288758
 gp             0x80409c94          -2143249260
 blink          0x8040024a          -2143288758
 acc0           0xaaaaaaaa          -1431655766
-pcl            0x80400248          -2143288760
+pcl            0x8040024a          -2143288758
 pc             0x8040024a          0x8040024a <vprintk+54>
-status32       0x7ffe8008081c      [ E=14 Z RB=0 AD IE ]
-bta            0x8040024a          0x8040024a <vprintk+54>
+status32       0x7ffc8008081c      [ E=14 Z RB=0 AD IE ]
+bta            0x80401570          0x80401570 <_rirq_exit+72>
 eret           0x0                 0x0
-timer_build    0x7ffe00000304      [ version=4 t0 t1 p0=0 p1=0 ]
+timer_build    0x7ffc00000304      [ version=4 t0 t1 p0=0 p1=0 ]
 irq_build      0xf141901           [ version=1 IRQs=25 exts=20 p=15 ]
 vecbase_build  0x10                [ version=4 addr=0 ]
 isa_config     0x12e47402          [ version=2 va_size=4 pa_size=4 a=3 n m=2 c=0 d=1 ]
@@ -50,7 +50,7 @@
 erstatus       0x0                 [ e=0 rb=0 ]
 erbta          0x0                 0x0
 ecr            0x0                 [ parameter=0 causecode=0 vectornumber=0 ]
-efa            0x7ffe00000000<PHONE_NUMBER>20736
+efa            0x7ffc00000000<PHONE_NUMBER>86144
 icause         0x0                 0
 aux_irq_ctrl   0x207               [ nr=7 b ]
 aux_irq_act    0x0                 [ active=0 ]
@@ -59,7 +59,7 @@
 irq_select     0x10                16
 irq_enable     0x1                 true
 irq_trigger    0x0                 false
-irq_status     0x7ffe00000010      [ p=0 e ]
+irq_status     0x7ffc00000010      [ p=0 e ]
 irq_pulse_cancel 0x0               false
 irq_priority   0x0                 0
 mmu_ctrl       {en = 0x0, ku = 0x0, wx = 0x0, tce = 0x0} {en = false, ku = false, wx = false, tce = false}

Critically important here is that sp is different. In the failing case, when blink is loaded from the stack, the value is 0, causing the system to branch to 0x0.
If I reset sp and ilink to the correct values at this point, then the application works correctly.
Ok, after even more debugging, I think I've discovered a bug in qemu?
Here's the last two instructions in vfprintf:
   0x80406f10 <vfprintf+136>:	j_s.d	[blink]
   0x80406f12 <vfprintf+138>:	ld.ab	r25,[sp,4]

When stopped at the first one, blink points at the return address in vprintk:
   0x8040023e <vprintk+42>:	st.as	r0,[sp,1]
   0x80400242 <vprintk+46>:	mov_s	r1,r14
   0x80400244 <vprintk+48>:	bl.d	27716	;0x80406e88 <vfprintf>
   0x80400248 <vprintk+52>:	mov_s	r0,sp
=> 0x8040024a <vprintk+54>:	add3	sp,sp,0x2
   0x8040024e <vprintk+58>:	ld.ab	blink,[sp,4]
   0x80400252 <vprintk+62>:	ld.ab	r14,[sp,4]
   0x80400256 <vprintk+66>:	j_s.d	[blink]
   0x80400258 <vprintk+68>:	ld.ab	r15,[sp,4]

I stick a break point on both addresses along with _isr_wrapper, stop at the first to wait for a timer interrupt to pend. At this point, sp is  0x80409c40. Continue, and the timer interrupt fires causing the processor to handle the interrupt. At this point, sp is 0x80409c00. That means it moved the stack by 16 values, but there are 17 values to store here -- 14 registers, blink, pc and status32. Looking at memory at the old stack location we find that qemu has actually smashed that value, storing status32 there.
When the rtie instruction executes, it pops all 17 values from the stack, leaving sp at 0x80409c44, which doesn't match the value from before the interrupt was handled.

ok, I'm pretty sure this is a bug in qemu caused by combining an interrupt and a jump with a delay slot. In this bug, the timer fires exactly as vfprintf is in the middle of returning, in this sequence:
   0x80406f08 <vfprintf+128>:	ld.ab	r23,[sp,4]
   0x80406f0c <vfprintf+132>:	ld.ab	r24,[sp,4]
   0x80406f10 <vfprintf+136>:	j_s.d	[blink]
   0x80406f12 <vfprintf+138>:	ld.ab	r25,[sp,4]

The instruction in the delay slot increments the stack pointer. If the timer fires right on top of the j_s.d [blink] instruction, then the stack pointer gets mangled for the next few instructions. If you place a break point before that happens, even a single instruction, then the timer will expire while the breakpoint is pending and fire immediately after you continue execution, so the interrupt occurs before the branch and things are just fine.
I'm thinking we should just disable branch delay slots in the qemu targets; they're certainly not an optimization for this platform and it seems that qemu doesn't have them quite right. When I do that, everything is just fine.

Ok, looks like this qemu bug is fixed upstream and we just need a newer version in the SDK to get this to work.

Reopen as we haven't switched to new SDK yet (but the qequired fixes are merged to SDK repo)

Re-opening as a reminder to revert the workaround when the new SDK with qemu fix is used in CI.


.Net: Bug: .NET Microsoft.SemanticKernel.Connectors.Onnx missing package release for CUDA and DirectML

Describe the bug
Look like Microsoft.SemanticKernel.Connectors.Onnx is missing the NuGet package release for CUDA and DirectML.
The project file already contains condition build configuration for:

Microsoft.ML.OnnxRuntimeGenAI.Cuda
Microsoft.ML.OnnxRuntimeGenAI.DirectML

https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Connectors/Connectors.Onnx/Connectors.Onnx.csproj

This is an integration design problem that will be addressed by the ONNX GenAI library repository in the future. To avoid relying on extra packages that might eventually be deprecated, we are holding off on this update until one ONNX library can work seamlessly with CPU, CUDA, and DirectML interfaces, ensuring long-term compatibility

Sounds logical, especially since the ONNX library is planned to work with CUDA, CPU, and DML directly. Switching between these three is not optimal. I know the reason behind it, but still.
Do you know when this update is expected—Q1 2025 or rather Q4 2025?

@quicksln , I don't have a specific time frame, hopefully Q1 2025, as soon this is in place will update the connector.


Feature request: optimised asm support

Hi,
can we have optimised assembler versions for important architectures?

z80, vax, and m68k are a must

I think the following is enough:
mov x0, #0
mov x8, #1
svc #0

What about handling bad true values and shared library handling?

m68k version :
move.b #0, d0
rts

z80:
ld a,0
ret

premature optimization, you should worry about adoption first!

I think we need to profile libtrue on all architectures before making premature optimizations.


fix: Prefer “libarpack” over “libparpack” in FindARPACK

When using libparpack installed with Homebrew, undefined symbols were encountered. Use of libarpack fixed it.

I'd suggest to remove the parpack name from the file completely as that is the parallel version which has p prefixed symbols so they are unusable from MIRTK which uses the serial version which don't have the prefix. So removing parpack from the list of names avoids using that if it where found anywhere because those libraries are not interchangeable but fundamentally different.

Thanks, that's good to know. So I guess the Find module should actually look for two different libraries, if it was intended for general use (not MIRTK specific). Happy to remove the name.


Decouple refill from capacity

https://help.shopify.com/api/getting-started/api-call-limit

The bucket size is 40 calls (which cannot be exceeded at any given time), with a "leak rate" of 2 calls per second that continually empties the bucket. If your app averages 2 calls per second, it will never trip a 429 error ("bucket overflow").

Other APIs have similar burstable capacity. Other Java "leaky bucket" implementations support burstable capacity, but the fact that bucket4j supports distributed tokens through JCache is very interesting.

The token-bucket library is good example how to this issue can be done.
With token-library you can achieve the requirements above:
TokenBucket bucket = TokenBuckets.builder()
       .withCapacity(40)
      .withFixedIntervalRefillStrategy(1, 500, TimeUnit.MILLISECONDS)
      .build();

These requirements can not be solved by Bucket4j because of refill is not independent entity, speed of refill is tightly coupled with bandwidth capacity.
But it is very trivial job to decouple refill from capacity. I suppose to use following API for case when refill and capacity should be independent:
Bucket bucket = Bucket4j.builder()
      withLimitedBandwidth(40, Refill.fixed(2, Duration.ofSeconds(1)))
      .build();

The old builder API should stay unchanged, because from my experience the situations when refill and capacity should be coupled are more frequent.
@efenderbosch do you have any concerns according this design?

Looks good, I like that builder API.

The coding and testing is done. Functionality will be released in few days in scope of release 1.3(after #16 will be completed).
Example of usage can be found there.


Upgrade to 21 alpha9

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Change default color to light ?

Hi, thanks for your great dashboard.
I try to change the default color to the light one, but i can't find where to do that.
Can you point me in the right direction ?
Thanks. :)

Solved, to close. :)

Just for future reference, add white-content to the body tag.
<body class="menu-on-right white-content">


Fatal: Couldn't mount '/sys/firmware/efi/efivars'.

mount: unknown filesystem type 'efivarfs'
Fatal: Couldn't mount '/sys/firmware/efi/efivars'.
Using Linux (Debian Wheezy)
It worked before I used the patch apple_set_os.efi which enabled the iGP.
Now it throws this error when I try to execute:
./gpu-switch -i
./gpu-switch -d
Any possible solution?

Hi,
Can you check if CONFIG_EFIVAR_FS=y is enabled in your kernel config? You can get the config from /proc/config.gz.
Could you check if it starts working again if you remove apple_set_os.efi?

Hi!
I think you didn't boot Linux in EFI mode.
Can you check if efivar -l give you any output?

ah-: I see no file like config.gz in /proc/, see below
0xbb:
$ efivar -l
bash: efivar: command not found
I am using TAILS os.
Using grub.cfg from project here on GIT (I see your activity in it)
which gives grub this configuration:

= Added according to your instructions on apple_set_os.efi

set timeout=30
set default=0
menuentry "TAILS live" {
*chainloader /EFI/BOOT/custom/apple_set_os.efi
*boot #added by me
fakebios
search --set -f /live/vmlinuz
linux /live/vmlinuz askmethod boot=live config live-media=removable nopersistent noprompt timezone=Etc/UTC block.events_dfl_poll_msecs=1000 splash nox11autologin module=Tails quiet
initrd /live/initrd.img
}
Both graphic proccessors are now powered and active. However I see no way to disable discrete one (enable only integrated).

Sorry for confusing post.
using grub.cfg from TAILS-OSX (here on git)

Seems /proc/config.gz doesn't exist in Debian/TAILS, can you have a look at ls /boot/config-$(uname -r)?
And maybe put your dmesg up as a gist?

I gave it a try and was successful with booting the amd64 version of Tails (vmlinuz2) and removing fakebios from the grub.cfg:
menuentry "TAILS live" {
        chainloader /EFI/BOOT/custom/apple_set_os.efi
        boot
        search --set -f /live/vmlinuz2
        linux /live/vmlinuz2 askmethod boot=live config live-media=removable nopersistent noprompt timezone=Etc/UTC block.events_dfl_poll_msecs=1000 splash nox11autologin module=Tails  quiet
        initrd /live/initrd2.img
}

Thanks. That did the trick to be able to apply ./gpu-switch (-i/-d)
However, it takes no effect. Since I need to reboot to apply the changes, and Tails is running in RAM (no connection between running instances.)
Would it be possible to make the ./gpu-switch -i/-d somehow permament?
Because right now there are 2 graphic processors running.
[lspci -v | grep VGA]
00:02.0 VGA compatible controller: Intel Corporation Crystal Well Integrated Graphics Controller (rev 08) (prog-if 00 [VGA controller])
01:00.0 VGA compatible controller: NVIDIA Corporation Device 0fe9 (rev a1) (prog-if 00 [VGA controller])

Or I was thinking to prompt it at the very boot time. (Hardcore it somehow into the Tails os.) So that we could simulate the needed reboot to take effect.

It should be permanent, gpu-switch only writes a firmware variable that sets the primary gpu from the next boot on. This doesn't power down anything yet, it just selects whether your xorg runs with the Intel or Nvidia gpu.
Can you check whether your xorg runs with the Intel gpu after a gpu-switch -i and then a reboot?
Once that works you can have a look at turning the Nvidia GPU off using vga_switcheroo: echo OFF > /sys/kernel/debug/vgaswitcheroo/switch

I see. The issue is there isn't vgaswitcheroo in Tails OS.
I am looking for a way to permanently power down Nvidia GPU.
I am sorry to misunderstood the real purpose of the GPU-Switch tool.

I thought the debian kernel included vgaswitcheroo? What does "grep -i switcheroo /boot/config-uname -r" say?
Maybe it's just not initializing properly, if you post your dmesg I'll have a look.
Yeah, the situation is a bit complicated at the moment, there were promising patches for proper switching on the dri-devel mailing list just a few days ago, maybe it improves soon. For now you can switch to Intel with gpu-switch, but to turn off the Nvidia gpu you need vgaswitcheroo.

I did a small test and I succeeded to turn off the Nvidia GPU!
It turned out that Tails is just not mounting the debugfs.
I used this commands to turn the discrete GPU off:
mount -t debugfs none /sys/kernel/debug
echo OFF > /sys/kernel/debug/vgaswitcheroo/switch

From now on you are the [HERO] number [ONE] for me.
I was trying to figure this out for more than a month and a half.
You bring all this so flawlessly.
Thank you a milion. Really. Believe that this will and may help even more people.
Time to bring it to the light.

I am out guys. Gonna sleep on the laptop tonight. Because it will last that long.
No more baked hotdogs on the case, no more extra chargers to make it stand.

0xbb saved this day.
You man. You just broke biggest stone I had above my head in the past time.
In few days and few replies.
Thank you so much.
Never gonna forget You and Your work.
Live a great life.
Really.

Haha, thank you :). Don't forget ah-, he discovered most of these tricks!

This issue is closed. May be used for other as a resource how to:
ENABLE IGP (INTEL IRIS PRO) ON MACBOOK PRO (RETINA)
Tails:
Get patch from the guy(s?) above called apple_set_os.efi
Get grub.cfg
[example configuration above, may depend slightly whether you want to enable apparmor as well]
Copy both of the files into:
cp apple_set_os.efi /EFI/BOOT/custom/
cp grub.cfg /EFI/BOOT/
[on your Tails stick. Not running one.]
Boot.
Enjoy your integrated graphic card.

Other linux distro:
may depend, but probably same procedure with grub and apple_set_os.efi
DISABLE DISCRETE NVIDIA GPU
mount -t debugfs none /sys/kernel/debug [IF USING TAILS]
echo OFF > /sys/kernel/debug/vgaswitcheroo/switch

ALL THAT THANKS TO THE GUYS ABOVE. 0XBB & AH-. PRAISE THEM.


Sort out the pip install to optionally install pytorch

Hopefully this wont be too hard. Essentially I'd like the default setting to install bare minumum, and an -all type flag to install full optional dependencies. Don't know the standard for that but need to sort it for ease.

Yes, love this! And hopefully the ability to go from default to --all easily with pip too.

Weirdly the pip install no longer works for me at all!
(mdtk) [charles01]s0816700: pip install -e ./
Obtaining file:///mnt/cdtds_cluster_home/s0816700/git/midi_degradation_toolkit
    ERROR: Command errored out with exit status 1:
     command: /home/s0816700/miniconda3/envs/mdtk/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/mnt/cdtds_cluster_home/s0816700/git/midi_degradation_toolkit/setup.py'"'"'; __file__='"'"'/mnt/cdtds_cluster_home/s0816700/git/midi_degradation_toolkit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
         cwd: /mnt/cdtds_cluster_home/s0816700/git/midi_degradation_toolkit/
    Complete output (5 lines):
    running egg_info
    writing mdtk.egg-info/PKG-INFO
    /home/s0816700/miniconda3/envs/mdtk/lib/python3.7/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '2019.06rc1' to '2019.6rc1'
      normalized_version,
    error: [Errno 22] Invalid argument
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

BUT running  python setup.py install does 🤷♂

Also - add mir_eval to optional deps

This is fixed by https://github.com/JamesOwers/midi_degradation_toolkit/commit/bc64f2fd4c348a29f51340af9e8bd5d1b20a49c2. I have tested on Linux and Unix, but not windows. @apmcleod - do you have ability to test on a windows machine? All that's required is trying to run the following:
git clone https://github.com/JamesOwers/midi_degradation_toolkit
cd midi_degradation_toolkit
conda create -y -n mdtk python=3.7
conda activate mdtk
pip install .
./make_dataset.py

If that runs with no problem then we're good.

I'll try. There's no cd command, for example, but I can write some instructions if I get it to work.

Ah sure, that's dir right?! TBH there's a limit to how much support I
think is reasonable to provide (I would kind of expect windows users to
quickly be able to google that one!).
On Fri, 15 Nov 2019 at 10:51, Andrew McLeod<EMAIL_ADDRESS>wrote:

I'll try. There's no cd command, for example, but I can write some
instructions if I get it to work.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/JamesOwers/midi_degradation_toolkit/issues/65?email_source=notifications&email_token=AA24Q47C3TUBENYUJVDIOS3QTZ5LXA5CNFSM4I72A3KKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEFCJVA#issuecomment-554312916,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA24Q4YO6WY4BXIGXYPCVUDQTZ5LXANCNFSM4I72A3KA
.

Fully agree


x509: certificate is valid for {domain}, not {domain} during cloud-cloud transfers

Is there a flag or setting that I should use when copying from one cloud provider to another? I'm getting a certificate error when copying from onedrive to amazon cloud drive.
What is your rclone version (eg output from rclone -V)
rclone v1.29
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux 64 bit
Which cloud storage system are you using? (eg Google Drive)
OneDrive, Amazon Cloud Drive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy 1drv_remote:dir acd_remote:dir
A log from the command with the -v flag (eg output from rclone -v copy /tmp remote:tmp)
x509: certificate is valid for {domain}, not public-dm2pap001.files.1drv.com

I don't think this is anything particularly to do with copying from one cloud store to another.
It looks like this is a problem with onedrive.
Does it work if you include --no-check-certificate ?
I suspect it is a mis-configuration by the onedrive team which they will fix quickly - so does it still do it?

Is this working now?

I'm going to assume this is fixed now - if not please re-open!


Update kiali to 1.38

https://github.com/kiali/kiali/pull/4108 is required to use it with Istio 1.11

@howardjohn Should we classify this as an RB since we'll need it for 1.11?

I put it as P0 because technically we don't ship Kiali. So release blocker feels a bit extreme. That being said, Kiali plans to release on the 30th so should be no issue

Fixed by https://github.com/istio/istio/pull/34462#event-5096861760


Added glossary term for Container environment variable

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For 1.10 Features: set Milestone to 1.10 and Base Branch to release-1.10
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
NOTE: After opening the PR, please un-check and re-check the "Allow edits from maintainers" box so that maintainers can work on your patch and speed up the review process. This is a temporary workaround to address a known issue with GitHub.>
Please delete this note before submitting the pull request.

This change is

/lgtm

/hold

This is a difficult definition. I'll put some thought into it. I agree that it's not about the container runtime. And I think the definition needs some further adjustment. For example, I don't think the word "important" belongs in the definition; an environment variable can hold anything, important or not.

/lgtm
The revised version looks good tome.

/hold cancel

/approve


Split back end logic into a library project.

Now that the main logic is split into the library project, the library can be utilised within other solutions.
I've also added a second constructor within the KinectSource as to pass in the KinectSensor object, in case it's being used elsewhere.

Sorry, I seem to have missed actually removing the files from the main project here, Resulting in duplicated files.
Here's the clean up,. :)
https://github.com/Spillmas/DumpKinectSkeleton/commit/ff435e4a0482368d84674391fd43423e325f3839

Yeah I should have check that before merging... Can you create a pull-request for this ?
Thanks

Sure thing.


internal: Get rid of some console.error spam in test

Codecov Report

Merging #1416 (bc92943) into master (55b3889) will increase coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1416      +/-   ##
==========================================
+ Coverage   98.56%   98.62%   +0.05%
==========================================
  Files          93       93
  Lines        1747     1747
  Branches      346      346
==========================================
+ Hits         1722     1723       +1
  Misses          5        5
+ Partials       20       19       -1

Impacted Files
Coverage Δ

packages/core/src/state/NetworkManager.ts
97.75% <0.00%> (+1.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55b3889...bc92943. Read the comment docs.


onDownloadfinished called directly after connectionlost on lolipop 5.0

hi , thx for this great library but i am facing issue , in lolipop 5.0 ondownloadFinished Called Directly after connectionlost , but on other apis it works fine even on 5.1 , this issue appear only in 5.0
i noticed that an error raised
07-31 16:50:06.941    1670-1804/com.filmov.itech.movies E/AndroidRuntime﹕ FATAL EXCEPTION: Thread-229
Process: com.filmov.itech.movies, PID: 1670
java.lang.NullPointerException: Attempt to invoke virtual method 'int java.io.FileInputStream.read(byte[])' on a null object reference
at tools.majid.core.chunkWorker.Rebuilder.run(Rebuilder.java:52)

I have the same problem when I pause a resumable task:
dm.pause(token);
it called the sequence :
onpause → rebuild → rebuild finish → complete
so I can`t resume the download any more

this happened only with android lolipop 5.0.1
in earlier versions it works perfectly..

Did you check that on handheld device or on an emulator?

@majidgolshadi  thanks for the fast response,
I am working on real device.

i tested it on real device and emulator and both the same result , please fix this issue your library is very great and i only faced this issue on it

after days of hardworking i have found the source of the error , in Asyncworker :
if (!this.isInterrupted()) {
observer.rebuild(chunk);
Log.i("isInterrupted()","yes");
}
for some reason this if statment become true after we call pausedownload , and for this reason observer call rebuild to finish the download , i saw this issue only on 5.0 , i compared this on other apis , the if statement was false when we call pausedownload
sorry bad english

FIXED :
I fixed this issue by adding flag inside AsyncWorker , i name it as "isInturrputed" , the default value of this flag is false and became true when trying to inturrput every thread chunk , the code now became like this
AsyncWorker Class :
package tools.majid.core.chunkWorker;
import android.os.Build;
import android.util.Log;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.SocketTimeoutException;
import java.net.URL;
import tools.majid.Utils.helper.FileUtils;
import tools.majid.database.elements.Chunk;
import tools.majid.database.elements.Task;
/**

Created by Majid Golshadi on 4/14/2014.
*/
public class AsyncWorker extends Thread{
private final int BUFFER_SIZE = 1024;
private final Task task;
private final Chunk chunk;
private final Moderator observer;
private byte[] buffer;
private ConnectionWatchDog watchDog;
private Boolean isInterrupted=false;
public boolean stop = false;
public AsyncWorker(Task task, Chunk chunk, Moderator moderator){
buffer = new byte[BUFFER_SIZE];
 this.task = task;
 this.chunk = chunk;
 this.observer = moderator;

}
public void setInterrupt()
{
isInterrupted=true;
}
@Override
public void run() {
try {
     URL url = new URL(task.url);
     HttpURLConnection connection = (HttpURLConnection) url.openConnection();
     connection.setConnectTimeout(50000);
     connection.setReadTimeout(50000);
     if (chunk.end != 0) // support unresumable links
         connection.setRequestProperty("Range", "bytes=" + chunk.begin + "-" + chunk.end);

     connection.connect();

     File cf = new File(FileUtils.address(task.save_address, String.valueOf(chunk.id)));
     InputStream remoteFileIn = connection.getInputStream();
     FileOutputStream chunkFile = new FileOutputStream(cf, true);

     int len = 0;
     // set watchDoger to stop thread after 1sec if no connection lost
     watchDog = new ConnectionWatchDog(5000, this);
     watchDog.start();

         while (!this.isInterrupted() &&
                 (len = remoteFileIn.read(buffer)) > 0) {

             watchDog.reset();
             chunkFile.write(buffer, 0, len);
             process(len);
         }

     chunkFile.flush();
     chunkFile.close();
     watchDog.interrupt();
     connection.disconnect();

     if(Build.VERSION.SDK_INT==21)
     {

         if(!isInterrupted)
         {
             observer.rebuild(chunk);
             Log.i("isInterrupted()","yes");
         }
     }
     else
     {
         if (!this.isInterrupted()) {
             observer.rebuild(chunk);
             Log.i("isInterrupted()","yes");
         }

     }

 }catch (SocketTimeoutException e) {
 	e.printStackTrace();

 	observer.connectionLost(task.id);
 	puaseRelatedTask();

 } catch (MalformedURLException e) {
     e.printStackTrace();
 } catch (IOException e) {
     e.printStackTrace();
 }

 return;

}
private void process(int read) {
observer.process(chunk.task_id, read);
}
private void puaseRelatedTask()	{
observer.pause(task.id);
}
private boolean flag = true;
public void connectionTimeOut(){
if (flag) {
watchDog.interrupt();
flag = false;
observer.connectionLost(task.id);
puaseRelatedTask();
}
}

}
and the block of code that handle inturrputing every thread chunk in Moderator Class became like this :
List taskChunks =
chunksDataSource.chunksRelatedTask(task.id);
for (Chunk chunk : taskChunks) {
Thread worker = workerList.get(chunk.id);
if (worker != null) {
worker.interrupt();
if(Build.VERSION.SDK_INT==21)
((AsyncWorker)worker).setInterrupt();
workerList.remove(chunk.id);
Log.i("workerListremove",String.valueOf(chunk.id));
}
}

Send your bug fix with pr

so many thanks @majidgolshadi  ..

I had a similar bug reported from crashlytics.  But I couldn't reproduce this. How can I encounter this with some steps?


UI: Add more info to Stage details page

Currently, at the top level Stage View, we only show Subscriptions. We also have tabs for Promotions, Verifications, and the Live Manifest.
There's a lot of information in the Live Manifest which would be useful to surface at the top level too, to prevent the need to parse the large and hard to read manifest.
In addition, I think there's a better way to present Promos and Verifications, perhaps withinin the same tab (there's a lot of wasted space on virtually useless info, for example the Verification ID)

I'm going to close this because I feel the Stage view has improved since this was opened and I think we can open new issues if there are specific items we discover are missing or would be helpful.


Update test-file

Test

Test

test

test

test

Testing

Hello

Testing

hello

Test

Test

a

Hello bot!

Testing

/echo hello

/echo hello

/echo test

/test testing123

/echo foo

/echo test

/echo foo

/echo 123

/echo testing

/echo testing

/test hello!

/echo hello!

/echo hello hello hello

/echo is anyone there

/echo HELLO

  /echo testing with leading and trailing spaces

/echo is this thing on

/echo echo

/echo /echo

/echo

/echo

/echo

/echo


Missing troubleshooting information

Hi all,
I ran into an issue recently regarding installing GPU functionality on Mac - which should be covered by the info in the Readme in the troubleshooting section. However, this did not work for that particular device (maybe newer Mac devices require a different wrapper for opencl functionality?)
Anyway, installing this package fixed it.

Correct! See also installation instructions here


Fix up exception shutdown

Define the exceptions in pijuice handler so that we're only shutting down on certain exceptions.
Possibly put a wait into it so that server can still be logged in to while failing.

This shouldn't be a problem too much any more with longer wait time before program start.


CLOUDSTACK-8308:Adding automation testcases for Delta Snapshot testpath

Ping, any updates?

@pritisarap12 Any update on this PR?
Please make just one commit, and get rid of the merge commit. You can force push to this PR. Ping us if you need help.

See #720 -> Resolved comments.

@remibergsma can you please comment on this PR, if it was merged ?  thanks,

It was not merged, see also #720.

@pritisarap12 please rebase against latest master and share status of your PR

@pritisarap12 @remibergsma referenced PR #720 which appears to implement a similar set of tests.  How do the tests in this PR differ?
it also appears that the Travis build is failing.  Could you please fix the failure and trigger Travis to rebuild?  Finally, is there a JIRA ticket associated with this change?

opening #720 in favour of this one and will assign to me


⚠ Bofrost.at has degraded performance

In 65af63c, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1593 ms

Resolved: Bofrost.at performance has improved in 50e028b.


Support precommit transaction interface

Enhancement

Add more background about this interface.
I don't know the background. And why do you add this interface


Script.before_process() wrongly triggered at the 'txt2img' mode

Environment
Version

Stable Diffusion
v1.6.0-2-g4afaaf8a

A Person Mask Generator
commit: 04cad56

At the img2img mode, it works perfectly. But when clicking Generate button at the txt2img tab, it will trigger an error:
*** Error running before_process: \path\to\StableDiffusion\stable-diffusion-webui\extensions\a-person-mask-generator\scripts\a_person_mask_generator.py
    Traceback (most recent call last):
      File "\path\to\StableDiffusion\stable-diffusion-webui\modules\scripts.py", line 611, in before_process
        script.before_process(p, *script_args)
    TypeError: Script.before_process() missing 2 required positional arguments: 'enabled' and 'mask_targets'

It seems that the bug is caused by no arguments to submit at the txt2img mode while before_process() is wrongly triggered to require two non-exist arguments. May fix it by adding a tab/mode check.

@LIU-Yinyi this has been fixed.  Thanks for creating the issue!


slurm configuraiton for snakemake

template configuration file:
https://github.com/Snakemake-Profiles/slurm/tree/master/%7B%7Bcookiecutter.profile_name%7D%7D

@rahulbshrestha @xinyuejohn ,
in case you are working on this, we can discuss it in this issue.

https://www.sichong.site/workflow/2021/11/08/how-to-manage-workflow-with-resource-constraint.html


ValueError: Padding is incorrect.

Current temp is 0c.
It sends commands just throws every time this error. Does not update status.
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 279, in async_update_ha_state
await self.async_device_update()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 470, in async_device_update
await self.async_update()
File "/config/custom_components/midea_ac/climate.py", line 104, in async_update
await self.hass.async_add_executor_job(self._device.apply)
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/msmart/device.py", line 205, in apply
data = self._lan_service.appliance_transparent_send(data)
File "/usr/local/lib/python3.7/site-packages/msmart/lan.py", line 78, in appliance_transparent_send
reply = self.decode(self.security.aes_decrypt(response[40:88]))
File "/usr/local/lib/python3.7/site-packages/msmart/security.py", line 26, in aes_decrypt
decrypted = unpad(decrypted, self.blockSize)
File "/usr/local/lib/python3.7/site-packages/Crypto/Util/Padding.py", line 90, in unpad
raise ValueError("Padding is incorrect.")
ValueError: Padding is incorrect.
Versions (版本信息)

Home Assistant version:
Midea msmart version:

Downgraded to msmart==0.1.15 and it works.

update to 0.1.18

thank you!


Formats curly braces

Just formatted some curly braces to match coding style.

gofmt is the norm :D


code covarage to low

[x] should be at least 60%
[ ] lets remove unused code

core should have cc close to 100%

CC is 81%


Example in docs doesn't compile

Hey,
I'm following 5.0.1 Connection pooling, where in the 'Basic Usage' example you write:
RedisClient client = RedisClient.create(RedisURI.create(host, port));
GenericObjectPool<StatefulRedisConnection<String, String>> pool = ConnectionPoolSupport
               .createGenericObjectPool(() -> client.connect(), new GenericObjectPoolConfig());

// executing work
try (StatefulRedisConnection<String, String> connection = pool.borrowObject()) {
    connection.multi();
}

This code does not compile, and the error was fixed in 5.1.0 docs, but it should be:
try (StatefulRedisConnection<String, String> connection = pool.borrowObject()) {

    RedisCommands<String, String> commands = connection.sync();
    commands.multi();

Thanks for the report. I'm not sure I follow. We fixed the docs for 5.1.0. We do not re-deploy documentation for already released versions because a fair amount of docs lives in the Wiki.
Is that the issue that you meant?

Thanks for the report. I'm not sure I follow. We fixed the docs for 5.1.0. We do not re-deploy documentation for already released versions because a fair amount of docs lives in the Wiki.
Is that the issue that you meant?

There is an error in the docs of 5.0.1 which does not appear in 5.1.0, I'm reporting the error of 5.0.1 and noting that it was fixed in 5.1.0.

Thanks for the clarification. The main issue with building and deploying documentation for older versions is that the Wiki isn't tied to the source code version and building older documentation would pull the most recent wiki contents and so it would reflect a different state.
That's why we keep old documentation stable. Prior to lettuce.io, the Wiki was the sole place for documentation.
Closing this ticket as there's nothing we can do here.


Loosing getInitialProps in HOC

Hey :)
I have really simple styled-components HOC provider.
import React from 'react'
import { ThemeProvider } from 'styled-components'
import { theme } from 'common/globalStyles'

const withTheme = WrappedComponent => props => (
  <ThemeProvider theme={theme}>
    <WrappedComponent {...props} />
  </ThemeProvider>
)

export default withTheme

Then I have custom _error.js page component wrapped in withThemeProvider hoc.
import React from 'react'
import { withThemeProvider } from 'hocs'

class Error extends React.Component {
  static getInitialProps({ res, err }) {
    const statusCode = res ? res.statusCode : err ? err.statusCode : null;
    return { statusCode }
  }

  render() {
    return (
      <p>
        {this.props.statusCode
          ? `An error ${this.props.statusCode} occurred on server`
          : 'An error occurred on client'}
      </p>
    )
  }
}

export default withThemeProvider(Error)

getInitialProps is not getting called. Can you please help me fix it? :)

Try using the hoistNonReactStatics package https://github.com/mridgway/hoist-non-react-statics

Or provide getInitialProps in the HOC 👍

@shane805 Thank for your answer. That library is perfect. :) I did not want to create anything on my own because I was almost sure there must be perfect solution already available.
This is my withThemeProvider now (I will extract the hoistStatics in the future):
import React from 'react'
import { ThemeProvider } from 'styled-components'
import { theme } from 'common/globalStyles'
import hoistNonReactStatics from 'hoist-non-react-statics';

const withTheme = WrappedComponent => props => (
  <ThemeProvider theme={theme}>
    <WrappedComponent {...props} />
  </ThemeProvider>
)

const hoistStatics = higherOrderComponent => BaseComponent => {
  const NewComponent = higherOrderComponent(BaseComponent)
  hoistNonReactStatics(NewComponent, BaseComponent)
  return NewComponent
}

export default hoistStatics(withTheme)

Note that it's much better to not use hoist-non-react-statics, it's huge for what it does<EMAIL_ADDRESS>You can manually copy the getInitialProps, which is what you generally want anyway:
import React from 'react'
import { ThemeProvider } from 'styled-components'
import { theme } from 'common/globalStyles'

function withTheme(WrappedComponent) {
 function Theme (props) {
   return (
      <ThemeProvider theme={theme}>
        <WrappedComponent {...props} />
      </ThemeProvider>
    )
  }

  if(WrappedComponent.getInitialProps) {
    Theme.getInitialProps = WrappedComponent.getInitialProps
  }

  return Theme
}


MINOR: Check null keys in KTableSource

Review by @enothereska @mjsax @dguy

LGTM

+1

Jenkins failure are transient and tracked in other JIRAs.

Merged to both trunk and 0.10.0.


AttributeError: 'JointBilstmCrfModel' object has no attribute '_trainable_weights'

Describe the bug
A clear and concise description of what the bug is.
in tf1.12, run python main.py --cmd train --config ../egs/atis2/nlu_joint/v1/config/nlu_joint.yml ,AttributeError: 'JointBilstmCrfModel' object has no attribute '_trainable_weights'
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error
AttributeError: 'JointBilstmCrfModel' object has no attribute '_trainable_weights'
Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS] ubuntu 18.04
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Does test under tf1.13 or tf1.14?

sorry,I only installed CUDA 9

If nvidia driver support CUDA10, you can try with this https://github.com/didi/delta/blob/master/docs/installation/manual_setup.md#pip-install.


Document how to produce a "json feed" containing all content as single JSON file

📚 Documentation
Similar to Hugo's index.json.  I would like to know how to make a JSON output representing all site content and metadata.  Like this:
https://www.screencast.com/t/a3spUYaU1J0
This supports a variety of downstream transformations of site content, such as building a custom Algolia index, transforming to other output formats, etc.
Could someone document or suggest how to do this?
Thanks!
Stu
Have you read the Contributing Guidelines on issues?
yes

We don't have a good way to do that at the moment as we don't aggregate the data model during the bootstrap process into one single data structure, unlike Gatsby. Contents and data fetched within each plugin is scoped to that plugin. The only overall site-wide data we have is the routes, which is used by docusaurus-plugin-sitemap to create sitemap.xml.
But what you have proposed is a step in the right direction. In future, we would try to have similar APIs to  build data models in plugins which can be processed downstream in subsequent plugins.

The contentLoaded attribute now accept a "allContent" attribute that permits to access the content loaded by all the plugins.
 async contentLoaded({actions: {createData, addRoute}, allContent}) {
Take a look at the official debug plugin for inspiration, as it access the data of all the plugins: https://docusaurus.io/__docusaurus/debug/content

Migrated from v2 to v3 ... and there is no longer allContent available :(

@kesor this is something we never documented because I wasn't sure it was a good idea.
We later refactored it to another lifecycle method you can try allContentLoaded. Again, still undocumented, we might not make it a public API surface and may need to refactor it if we find a better API design. But the "feature" should stay because we need it ourselves for the debug plugin: https://docusaurus.io/__docusaurus/debug/content


enhance pretrained-embeddings example to set trainable=False for  the pre-trained embeddings

This PR enhances the existing entertainment-with-pretrained-embeddings.ipynb notebook in the following ways:

users now can set the trainable param in the Embeddings function
users can use InputBlockV2 and feed to the model

rerun tests

Should the unittest include testing that the embedding weights did not change?

We might want to add a unittest to check, if the weights were freezed (comparing the embeddings after training with the original numpy array)

we do have a unit test here if that' what you meant.

rerun tests

rerun tests


How to adjust raw data to this format

Is there a tutorial to use own raw data correctly

Hi Geoffrey, thanks for making such a nice tutorials. It would indeed be super useful if you could include the preprocessing steps in the tutorial. By any chance could you share an anndata object with the raw counts?


add secrets to values.yaml

Before creating a pull request make sure that:

[ ] commit messages are meaningful and follow good commit message guidelines
[ ] README and other documentation has been updated / added (if needed)
[ ] tests have been updated / new tests has been added (if needed)

Please remove this line and everything above and fill the following sections:
JIRA link (if applicable)
Change description
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[ ] No

I will merge it and test it. and remove it later.


move typescript files into separate ts/ dir

It seems that tslint wasn't being run on the test files. This commit
also fixes that.
This supercedes #141. This ensures that the output js files remain in their original location as before this PR avoiding an unnecessary semver major.

I think it is worse to have input and output files commingled. @JustinBeckwith would you be happier if I called it ts/ instead of src/?

Preemptively moved stuff to ts/. @JustinBeckwith PTAL.

Coverage remained the same at 93.455% when pulling 8ff0684605d2822097f9c481b3753b032905870f on ofrobots:srcdir into 2582beed82b81c4052f47f311de8692602c3004e on google:master.

Coverage remained the same at 93.455% when pulling 8ff0684605d2822097f9c481b3753b032905870f on ofrobots:srcdir into 2582beed82b81c4052f47f311de8692602c3004e on google:master.

Coverage remained the same at 93.455% when pulling 8ff0684605d2822097f9c481b3753b032905870f on ofrobots:srcdir into 2582beed82b81c4052f47f311de8692602c3004e on google:master.

To elaborate on the challenge/constraints here:

The output JavaScript files must be in lib/ because external modules depend upon internal files.
The directory structure of input and output directory must be comparable. That is, tests should be able to do require('../lib/file').
We do not want input and output files to be commingled. We cannot put output files in a toplevel build/ directory because of the previous bullet. Specifically, going forward I want to start using a package.json files section rather than a .npmignore (i.e. more tightly control what actually gets shipped in the tarball).

Another possibility would be to have a lib.ts directory that is transpiled to lib and a test.ts directory that is transpiled to test.

@DominicKramer I am not sure that would work. Example:
import GoogleAuth from '../lib/auth/googleauth';

This will have to be changed to ../lib.ts/auth/googleauth to make it compile, but the javascript transpiled version will no longer work.
Perhaps there is way to work around this with tsconfig.json config, but I don't know it.

@ofrobots Ok.  That makes sense.


Fix encoding empty containers

Changes
Allowed encoding empty containers
Previously encoding empty set of chunks into a container would fail with a nasty Prelude.head error.
It was due to the following line:
objSchema    = getSchema (P.head (P.head xss))

Current version allows encoding empty chunks because the schema is now being passed into an internal encoding function (Avro.Encode.encodeContainer and Avro.Encode.encodeContainerWithSync).
Their public wrapper API Data.Avro.encodeContainer now takes the schema from a provided ToAvro a and passes it through, so there is no need to take an element from a possibly empty list(s) anymore.
Updated Data.Avro.decodeContainer signature.
To match decode/decodeWithSchema I have updated the signature for decodeContainer.
Now there are two functions:
This one is renamed from decodeContainer:
decodeContainerWithSchema :: FromAvro a => Schema -> ByteString -> [[a]]

It ignores the schema that is attached to a via FromAvro a and uses the one that is being passed explicitly (mirroring decodeWithSchema behaviour).
This one is a new decodeContainer:
decode ::  FromAvro a => ByteString -> Result a

It uses the schema that is attached to a via FromAvro a as a reader schema (mirroring decode).
@TikhonJelvis I think you may use containers, please have a look and let me know if this rename is not OK with you.

@AlexeyRaga container rename looks good. Recently @TikhonJelvis and I got bitten by encode/decode though. We assumed encode/decode were container based function (we overlooked the container based). Maybe the name should be more explicit for them?

Don't know, we have never used containers until very recently so encode/decode for individual values looked just logical to us :)
Anyway, we can think of renaming them as a separate change, don't want to do it now because it would be a breaking change perhaps not only for us two.


Update for google-datastore.js

@saivittalb Thanks for you PR, but I think if you run a linter on the code, it would be same again as before as per the mentioned standard.
You can look up to adding more services of GCP to nodecloud-gcp-plugin

No, I was just browsing through and did some changes randomly on seeing the code.... Nothing more than that! You can close this PR if you want! :)

FYI :) you can't just randomly edit code and hope it'll follow the standards or work properly.

FYI :) you can't just randomly edit code and hope it'll follow the standards or work properly.

Thanks for info but I think so this code just works fine though I randomly found it! Tbh, I didn't even try to make a change to this PR but I still didn't even know why I did it! I did that when I joined GitHub..... I was a noob back 20 days on GitHub! That's all!


Support lazy loading for cupyx.scipy.* submodules

Lazy load submodules using __getattr__ as SciPy does: https://github.com/scipy/scipy/pull/15230
c.f. #7390, https://github.com/conda-forge/cupy-feedstock/issues/187

Hello @emcastillo and @kmaehashi , can i work on this issue ?

Hi, can I work on this issue? Also could you elaborate a bit more on the issue? @kmaehashi @emcastillo I am a beginner

The PR is already ongoing in #8706


Update to CI 3.0

Even better if we can update whole project to CI 3

Indeed! Actually I am working on a new repository from here:
https://github.com/waifung0207/ci_bootstrap_3
The reason for separating from old repo is there are quite a lot of structure changes. While the new repo will have much enhancements (e.g. classes like models and helpers can be shared among different sites), I am going to take this chance to refactor some codings so developers can be easier to customize for their sites.
The new repo is still in progress but I am going to push it to more stable state within this month, with documentation about how to make use of it, so please stay tuned on that :)


Adding configurable cpu requests/limit for the debug daemonset

This is to reserve CPU and memory for the daemonset in case resources are scarce

/dci-rerun

/dci-rerun

/dci-rerun

/dci-rerun

/dci-rerun


Add getter for LAPIC IPI delivery status

This PR adds LocalApic::get_ipi_delivery_status to allow accessing bit 12 of ICR, which should be clear before the CPU sends another IPI.
(The formatting changes in mod.rs were caused by cargo fmt, I hope this is okay.)
Thank you for this crate! I've been using it for my own kernel and it has been very useful, but being unable to access this bit made sending IPIs properly difficult. I hope the PR is useful.

Thanks @aurelilia! This slipped under my radar, but your contribution is much appreciated.


Container may not have stat

Currently, the stat command is used before extracting files from the container (see: https://github.com/cloud66/habitus/blob/master/build/builder.go#L340). Certain containers (such asscratch`) will not contain this method and will fail to extract the file. In those cases it may make sense to fallback to not copying over the file permissions.

There is another option rather than trying to exec stat on the container. You could also just copy/stat it from the host where docker is mounting the volumes. This obviously doesn't work if you aren't on the same machine as the docker daemon and would likely require root.

I ran into some problems trying to use alpine based images in Habitus. The image didn't have bash installed but that could be fixed by adding apk add bash to the Dockerfile. However the stat command seems to work differently on alpine; it throws an error because the "format" option is not recognized.
This is when I gave up on using the alpine image (19 MB) and switched to an Ubuntu based image (258 MB) instead.
I think the build pipeline should be a bit more resilient to different OS in images. Maybe the docker cp command could be used to get the artifacts out instead? This should work with any docker image.

hi @michaelhue.  thanks for the feedback. the stats command is to get the permission right. I assigned to a new milestone to dive deeper in this matter

Hey @kamerblauwlicht, thanks for the response!
Regarding permissions with docker cp:

The cp command behaves like the Unix cp -a command in that directories are copied recursively with permissions preserved if possible. Ownership is set to the user and primary group at the destination. For example, files copied to a container are created with UID:GID of the root user. Files copied to the local machine are created with the UID:GID of the user which invoked the docker cp command. If you specify the -L option, docker cp follows any symbolic link in the SRC_PATH. docker cp does not create parent directories for DEST_PATH if they do not exist.

Sounds reasonable but I'm not sure if this is the behavior you need.

@michaelhue thanks. will work on this for the next release. let me check if I can make this part of Habitus more resilient.

I added coreutils to alpine and it worked for me.

stat can cause problems using stat, breaking build. looking for an alternative.

https://docs.docker.com/engine/reference/commandline/cp/

we are using docker cp in the background, don't need stat. removing it. docker cp trying to keep the   premissions


Add params for KVS overlay and sagemaker metrics

Do we need a parameter for Evaluation as well?

Do you think it's necessary to have separate control over the KVS overlay for eval + training?
If so then I could split them.


Brainstorm possibilities for variation visualization

possibilities from a brainstorming session, prioritizing a simple static visualization we can submit as part of an application

I've put up the first spin at a d3-based visual for comparing witnesses: http://fv-dev.library.cmu.edu/d3/single_synoptic/
n.b. this is still based off of old p1 collation data from December, so may be quite out of date, but gives an idea of what we might do


[Refactor] Adopt prototype ProbabilisticTensorDictModule and ProbabilisticTensorDictSequential

Description
This PR adopts the prototype refactored ProbabilisticTensorDictModule and ProbabilisticTensorDictSequential classes introduced in pytorch-labs/tensordict#104
Currently WIP while I work through the failing tests, but new usage should be clear. Most important changes are in:

torchrl/modules/tensordict_module/probabilistic.py - refactors SafeProbabilisticModule and introduces the new SafeProbabilisticSequential class
torchrl/modules/tensordict_module/actors.py - note the change to ActorValueOperator in particular

Most other changes are just updating usage.
This PR supercedes #719, though I will leave that open for now until it's clear we want to move ahead with the approach laid out in this PR and pytorch-labs/tensordict#104
TODO:

[x] Update tutorials

Regarding the failing tests, I tried to put this after the creation of the ddpg model:
    proc = mp.Process(target=dummyfun, args=(X,))
    proc.start()
    proc.join()

where
def dummyfun(model):
    print(model)

It fails when X=model[0] but runs when X=model[0][0] or X=model[0][1]
Hence the problem seem to be related to ProbabilisticActor. I'll keep investigating


Add Portkey Custom Endpoints Docs

This PR introduces new documentation explaining how to use Portkey's custom endpoints with LibreChat-AI. Key additions include:

How to update librechat.yaml file to integrate with Portkey

@danny-avila

@fuegovic can you take a look at this PR?

Thanks!


Request for model from Università Politecnica delle Marche, thanks!!

Hi,
thanks for your great work.
I'm Andrea Generosi from the Università Politecnica delle Marche, you should have received an email from my tutor prof. Maura Mengoni. I'd be greatfull if you could send me the high precision emotion model.
email<EMAIL_ADDRESS>Thank you very much!
Best regards

Your email is not sent successfully, can you change another email?

<EMAIL_ADDRESS>Tnx!

Hi,
thanks for your great work.
email<EMAIL_ADDRESS>Thank you very much!
Best regards

Your email is not sent successfully, can you change another email?

Please, send to me also! My contacts are<EMAIL_ADDRESS><EMAIL_ADDRESS>Thanks a lot!


Question regarding overlapping

First of all, Thank you for sharing your excellent work!
I have a question about overlapping (pingpong design). From my understanding:

With FP8 precision and a head dimension of 128, the exponential function seems to take the same amount of time as GEMM0 + GEMM1. This is because matmul is 512 times faster than the exponential function, and there are 512 times more FLOPs in matmul than the exponential operation.

The softmax function involves two MUFU: one for the exponential function and another for floating-point division. This indicates that softmax should be about twice as slow as GEMM0+GEMM1.

However, the figure provided shows that softmax takes only half as much time as GEMM0 + GEMM1.
If softmax takes twice more time than GEMMs (GEMM0 + GEMM1), GEMM operations might be idle for half of the time.

So my questions are:

Is my understanding of the time consumption for GEMM and softmax correct?
Specifically, is the time consumption really bound by MUFU for FP8 precision and a head dimension of 128?
If my understanding is correct, does this mean that GEMM operations are only executing for half the time while the other half of the time, they remain idle?
Thank you!

The figure is not drawn to scale, it's just an illustration.
The way we do it, softmax only has 1 MUFU (exponential). There's no floating point division. Division is done at the very end.


Fix GLES3 crash with Mesh surface with exactly 65536 vertices

One line fix for bug in GLES3 driver that can crash when a mesh has exactly 65536 vertices, because the wireframe generation code doesn't use the right buffer index size in unpacking, causing an out-of-bounds read.
See https://github.com/godotengine/godot/issues/95837 for minimal example project.
Bugsquad edit:

Fixes #95837

Thanks! And congrats for your first merged Godot contribution :tada:

Cherry-picked for 4.3.1.


Questions about LSTR's performance.

I just tested the performance，and the result seems not that fast，just about 100fps. Is there something wrong with the testing method？Here's the testing code below.

speed_simple
import torch
import time
import numpy as np
from nnet.py_factory import NetworkFactory
from config import system_configs
import os
import json

cfg_file = os.path.join(system_configs.config_dir, "LSTR.json")
print("cfg_file: {}".format(cfg_file))
with open(cfg_file, "r") as f:
    configs = json.load(f)

configs["system"]["snapshot_name"] = "LSTR"
system_configs.update_config(configs["system"])

torch.cuda.set_device(0)
torch.backends.cudnn.benchmark = True
nnet = NetworkFactory()
nnet.load_params(500000)
nnet.cuda()
nnet.eval_mode()

input_size = [360, 640]
images = torch.zeros((1, 3, input_size[0], input_size[1]), dtype=torch.float32).cuda() + 1
masks = torch.ones((1, 1, input_size[0], input_size[1]), dtype=torch.float32).cuda() + 2

for i in range(10):
    outputs, weights = nnet.test([images, masks])

t_all = []
for i in range(200):
    torch.cuda.synchronize(0)
    t1 = time.time()
    outputs, weights = nnet.test([images, masks])
    torch.cuda.synchronize(0)
    t2 = time.time()
    t_all.append(t2 - t1)

print('average time:', np.mean(t_all) / 1)
print('average fps:', 1 / np.mean(t_all))

print('fastest time:', min(t_all) / 1)
print('fastest fps:', 1 / min(t_all))

print('slowest time:', max(t_all) / 1)
print('slowest fps:', 1 / max(t_all))

Please read the README.md carefully.
The MACs and the number of parameters are more important than an FPS evaluated on graphics cards. Speed is just a matter of implementation.
If you test the time consumption of each module you will find that the speed is mainly consumed by the Transformer. However, there are many fast implementation versions of the Transformer, like Faster Transformer, which could speed up LSTR again but  is not our concern currently.
Besides, for research and algorithm landing purposes, the computation complexity is more compelling than the speed on the graphics card since no matter how fast you run the algorithm on the graphics card, the computational complexity of the algorithm and the number of parameters are independent of the implementation.

I agree with the importance of MACs(0.574G) and  the number of parameters(0.77M) of your model.
But it can't explain the fast speed(420fps) in the paper. However the speed is the one of core points in your paper.
As you described, you tested TuSimple dataset without bells and whistles.
So I wonder how do you test your model's speed?

:slightly_smiling_face: What you are seeking lies in the README.

emmm……
According to my opinion, FPS has nothing to do with batch. It's calculated by one inference, one image, one time.
The number of images the model can predicts every time just depends on the video memory.
You can read the speed testing code below for reference.
https://github.com/cfzd/Ultra-Fast-Lane-Detection/blob/master/speed_simple.py

FPS has nothing to do with batch? WOW!
Video memory? So you think a video application MUST process video frames one by one? WOW!
The UltraFast measures one frame speed only because their memory cost is very large.
Processing 1 3x288x800 image uses 1190MiB GPU memory usage, but processing 16 3x360x640 images only use 877MiB by our method.
Also, the number of parameters is 71, 333, 800, but ours is only 765, 787.
More importantly, why do not you test the batch performance of UltraFast?

Now on my platform, using the python script you linked, iterations=1000, when batch=1, UltraFast runs 202 fps and uses1189MiB GPU memory usage; when batch=16, UltraFast runs 299 FPS and uses 5653MiB GPU memory usage.  You can list the performance of LSTR evaluated by you own and you will get the idea.

Model serving is another thing. It depends on the real scenario.
It may process one frame one time in real time scenario, or process batch frames one time in non-real time scenario.
According your opinion,  FPS has relationship with batch, more batch corresponds to higher FPS theoretically, so how to define FPS.
If the GPU memory is unlimited, FPS is infinitely great?
DL papers about fps has never mentioned corresponding batch parameter.
And I run the python script directly, here's the result below.

How a GPU memory can be unlimited? What’s your point? I never want the gpu memory to be unlimited.
I list the FPS performance and the memory usage, why you always ignore the cost of gpu memory of different methods when discussing?
 Do you ever just think a little why this happens?

DL papers care about computation complexity, and FPS is just a matter of implementation. If you do not accept that idea, please do not disturb me and I will keep this issue open.

I said I agree with the importance of MACs(0.574G) and the number of parameters(0.77M) of your model.
I also care about computation complexity.
And we argue about if FPS has relationship to do with batch. The FPS here is just standard criterion, not the practical model serving result or implementation result.
If it does, how do you get your 420fps? batch=16? It seems unfair to compare with others whose batch=1. Or at least you should mark out batch=16 in your paper.
If it does not, well you seems disagree with this.

You said, "The FPS here is just standard criterion, not the practical model serving result or implementation result",  but did you know the convolution operation has been accelerated very fast through years of effort by many accelerating methods (you need CUDA to run any PyTorch codes, don't you ever think why?), but the PyTorch implementation of Transformer seems to have no such treatment. I have repeatedly asked you to focus on computation complexity because it is the ONLY criterion that has nothing to do with implementation, not the FPS you think ideally (standard criterion??? how standard??? which criterion??? are you living in your own world???).
How to measure speed has been written on the README since the beginning of this repo.
I don't know who you are but I still respect your passion, so could you please:

think that why other methods do not increase FPS significantly after increasing parallel data throughput by 16 times;
find a method that achieves ~96.18 TuSimple accuracy and runs faster than LSTR in any case on any of your own platform.

Then I will be willing to discuss with you. Otherwise, please do not waste our time and public resources.
By the way, the other issue you raise also makes me confused #20. Normally I would help anyone who raises issues, but I've never been able to get your point. I think you just live in your own world.

I don't want to discuss with you any more.
Lightweight model doesn't mean fewer cost time.
Lane detection belongs to driverless field! Do you think how to apply this? predict batch after driving a long time or predict one by one in real time? No matter how high fps can get, it's meaningless if the cost time can't be low.
Maybe you just research for research's sake, regardless of the actual application scenario. Real time is its premise!
If using batch prediction, fps can be higher more or less, why other models don't calculate fps in this way? Maybe you should think about it carefully.
With all the power of irony, how horrible and arrogant!
Well, you are right about everything, do not waste our time and public resources, just close it!

Our company has applied a faster version Transformer to accelerate LSTR (weak up! you are just a rookie), which achieves an incredible speed FPS number you'd never imagine on a real mobile chip.
If you are very happy to use BIG models to detect lanes on GPU in 300 FPS, WOW, you are so clever and the smartest genius in the world. Yes, you are the best and the most polite questioner. I am horrible and arrogant!!!:smiling_imp::smiling_imp::smiling_imp: So what?  However, it's good that you're having fun on your own.

Well, it seems modesty and politeness being laughted and rudeness being advocated, you put the cart before a horse! The moral degeneration of the world is getting worse day by day.
If you think my question is low, you can just ignore it or close it. Mock、sarcasm、persiflage and personal attack, is it necessary?  Can't even imagine it's a PHD's words.
You are all right about everything, OK? Don't comment any more. When did it get so hard to be polite?

Please look at how other guys raise issues and how I answer them. Out of enthusiasm,  I discuss issues with every person.
However, when did Ph.D. status become a kind of moral kidnapping?
No matter what my status is, I am only targeting you for mock, sarcasm, persiflage, and personal attack. :smiling_imp:

Not moral kidnapping, but the basic morality of being a man. Like don't swear first, don't hit first.

Maybe you are right, education has nothing to do with morality.
As I said, If you think my question is low, you can just ignore it or close it. Sarcasm may not be the appropriate way.
When one person disagrees with you or can't understand you, mock, sarcasm, persiflage, and personal attack are your first action? You are so mean? When did it get so hard to seek common points while reserving difference.
Well, as you said, it's good that you're having fun on your own.

I never feel that there is a high or low level of questions, however, from all other issues that you raised  #20 and talengu/Wox.Plugin.BookmarkSearcher/issues/9 (other authors do not even want to talk to you), I think you should mind your manners first.
Again, please look at how other guys raise issues and how I answer them.
 I think your self-presentation is very appropriate.

So polite man should always blindly self-surrender? Can't he fight back?
It's you who reacts in these manners first(mock、sarcasm、persiflage and personal attack), and you admin it in the reply!(No matter what my status is, I am only targeting you for mock, sarcasm, persiflage, and personal attack.)
 Maybe the pictures don't upload successfully.
How mean you are!  So you can find mudslinging to humiliate me? Do you have psychological shadows?
And as you say, other authors don't answer me, it's OK for me and everything's fine. Can't you learn it?

I just repeat the words you say to me, if you think that was an admin (admit I guess), sorry I overestimated your emotional and intellectual quotient.
When did GitHub's contribution activity is kind of personal privacy?
So I just click your GitHub profile then you think I have psychological shadows?
Hey, calm down, will you?

‘I am only targeting you for mock, sarcasm, persiflage, and personal attack‘
How clear it is？And you can't even admit your words! Being a man, can you be of some responsibility?
 Is there any reason you attack me first?
Maybe you have strong research ability, but the morality, emmm, you don't derserve it!

我佛了

Just can't help myself to say something after spotting this issue. Not to take anyone's side, I believe it is only proper to add a disclaimer in the arxiv version to support the batching choice, by saying the model is actually too small to show superiority on Teraflops GPUs due to a low utility rate (or possibly I/O constrains). Although we all admit LSTR is ultra fast and probably still the fastest, batching it in the FPS test does not appear to be common practice and should've been stated on the record for future reference.
p.s. I wonder if it is possible for LSTR to achieve real-time on CPU/Jetson. Listing CPU inference speed for fast algorithms is also good choice to show great performance.

Hi, thanks for you understanding.
I will update the arxiv to clarify batch=1's FPS and batch=16's FPS on the Table.

@liuruijin17 That's great to hear! Recently I've been thinking that maybe instead of changing batch-size, changing the input resolution is a more logical choice to profiling different methods. We might get round to that later here.

I just tested the performance，and the result seems not that fast，just about 100fps. Is there something wrong with the testing method？Here's the testing code below.
speed_simple

Hello, I use your code speed.py , and then the following error appears, sincerely ask what is the reason?
Traceback (most recent call last):
File "speed.py", line 19, in
nnet = NetworkFactory()
File "/home/cfy/TuSimple/LSTR/nnet/py_factory.py", line 67, in init
macs, params, = profile(self.model, inputs=(input_test, input_mask), verbose=False)
File "/home/cfy/miniconda3/envs/lane/lib/python3.8/site-packages/thop/profile.py", line 188, in profile
model(*inputs)
File "/home/cfy/miniconda3/envs/lane/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/cfy/TuSimple/LSTR/nnet/py_factory.py", line 39, in forward
return self.module(*xs, **kwargs)
File "/home/cfy/miniconda3/envs/lane/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/cfy/TuSimple/LSTR/models/py_utils/kp.py", line 213, in forward
return self._test(*xs, **kwargs)
File "/home/cfy/TuSimple/LSTR/models/py_utils/kp.py", line 208, in _test
return self._train(*xs, **kwargs)
File "/home/cfy/TuSimple/LSTR/models/py_utils/kp.py", line 185, in _train
p = self.conv1(images)  # B 16 180 320
File "/home/cfy/miniconda3/envs/lane/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/cfy/miniconda3/envs/lane/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 423, in forward
return self._conv_forward(input, self.weight)
File "/home/cfy/miniconda3/envs/lane/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 419, in _conv_forward
return F.conv2d(input, weight, self.bias, self.stride,
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Hi, thanks for you understanding.
I will update the arxiv to clarify batch=1's FPS and batch=16's FPS on the Table.

hello, have you updated it ?


Import _make_reasons from firedrake

Codecov Report

Merging #11 (27caf5f) into master (46128e6) will decrease coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #11      +/-   ##
==========================================
- Coverage   70.87%   70.80%   -0.08%
==========================================
  Files           3        3
  Lines         388      387       -1
  Branches       65       64       -1
==========================================
- Hits          275      274       -1
  Misses         81       81
  Partials       32       32

Impacted Files
Coverage Δ

firedrake_ts/solving_utils.py
64.89% <100.00%> (-0.15%)
:arrow_down:


Recreate inputs

What's this PR do?
Base recreation of inputs.  Auto Ties them together
What are the important parts of the code?
I think the most intersting part is how I tied them together with ID's
How should this be tested by the reviewer?
standard npm start
Is any other information necessary to understand this?
Not really, context from react.....
What are the relevant tickets? (Please add closes, refs, etc)
Closes #14
Screenshots (if appropriate)

This is my second submission of TextField.  I used git push -f because I needed to rebase.  I am using context; however, context seems to be geared for situations such as these.

Need to remove context and use cloneElement instead in a bunch of places.  id should probably be specified on propTypes too.

I didn't realize context was going to be that big of an issue.  I'll work on this later.

I updated everything you wanted me to.  There are very obscure references with cloneelement, I think I got the gist of it here.  I also set the ID up to randomly generate at the beginning as a fallback using our pattern we had established with the other toggle items.
I think I got a good pattern with the primary element.

@samrocksc This PR is really shaping up,  just a few more kinks to work out.

Cool, I got all the changes put in there was a little bit of conflicting feedback on being able to customize the ID on the <TextArea> component.  I went ahead and left the ability to assign the ID as a vestigial link in case someone needed access to that variable in the future.
I also made sure that the most pertinent element TextFieldLabel to using the id name is properly denoted and passed as htmlFor, otherwise it defaults to passing as id.
I can change those up if needed; however it seemed like a pretty good compromise and still leaves full customizability to the user.

this was the last one needed before a version bump i do believe

changes made

mop


[PARTNER-48] Upgrade OpenAPI plugin

This fixes schema descriptions not being rendered

Preview is available here:http://developers-pr137.previews.kube001.services.stellar-ops.com


refactor(logger): standardize how logger is used throughout rps

Codecov Report

Merging #288 (217ff01) into master (512804f) will increase coverage by 0.19%.
The diff coverage is 39.81%.

@@            Coverage Diff             @@
##           master     #288      +/-   ##
==========================================
+ Coverage   44.90%   45.10%   +0.19%
==========================================
  Files          50       50
  Lines        3042     3053      +11
  Branches      533      533
==========================================
+ Hits         1366     1377      +11
  Misses       1499     1499
  Partials      177      177

Impacted Files
Coverage Δ

src/utils/SecretManagerService.ts
15.78% <13.63%> (+1.50%)
:arrow_up:

src/WSManProcessor.ts
25.90% <18.18%> (+0.44%)
:arrow_up:

src/Validator.ts
42.28% <23.52%> (+0.38%)
:arrow_up:

src/actions/CIRAConfigurator.ts
20.49% <25.00%> (+0.49%)
:arrow_up:

src/WebSocketListener.ts
31.48% <28.57%> (+1.29%)
:arrow_up:

src/actions/NetworkConfigurator.ts
31.42% <33.33%> (+0.99%)
:arrow_up:

...repositories/factories/AMTDeviceVaultRepository.ts
21.42% <37.50%> (+1.91%)
:arrow_up:

src/ProfileManager.ts
53.21% <40.00%> (+0.43%)
:arrow_up:

src/DataProcessor.ts
38.02% <42.85%> (ø)

src/actions/Activator.ts
55.24% <50.00%> (+0.24%)
:arrow_up:

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 512804f...217ff01. Read the comment docs.