tests/lib/c_lib/thrd/ fails on multiple platforms

Describe the bug
This test fails on a several platforms with different symptoms.
CAVS 25
tests/lib/c_lib/thrd/libraries.libc.c11_threads.minimal  ( Failed )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc   ( Timeout )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc.module  ( Timeout )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc.notls  ( Failed )

ACE 15 MTL
tests/lib/c_lib/thrd/libraries.libc.c11_threads.minimal ( Failed )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc ( Failed )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc.module ( Failed )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc.notls ( Failed )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.newlib ( Failed )

UP SQUARED
tests/lib/c_lib/thrd/libraries.libc.c11_threads.minimal ( Failed )
tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc.notls ( Failed )

To Reproduce
The different steps for different platforms:
In general:
 twister --device-testing  --hardware-map <path_to>/map.yml -p <platform> -T tests/lib/c_lib/thrd -vv
Logs and console output
/twister-out/up_squared/tests/lib/c_lib/thrd/libraries.libc.c11_threads.minimal/handler.log
***** delaying boot 500ms (per build configuration) *****
*** Booting Zephyr OS build zephyr-v3.5.0-2033-gde63da0081dd (delayed boot 500ms) ***
Running TESTSUITE libc_cnd
===================================================================
START - test_cnd_broadcast_wait
PASS - test_cnd_broadcast_wait in 0.101 seconds
===================================================================
START - test_cnd_errors
PASS - test_cnd_errors in 0.001 seconds
===================================================================
START - test_cnd_init_destroy
PASS - test_cnd_init_destroy in 0.001 seconds
===================================================================
START - test_cnd_signal_timedwait
PASS - test_cnd_signal_timedwait in 0.051 seconds
===================================================================
START - test_cnd_signal_wait
PASS - test_cnd_signal_wait in 0.051 seconds
===================================================================
START - test_cnd_timedwait_timeout
E: Timeout waiting on cond 0x128140
PASS - test_cnd_timedwait_timeout in 0.204 seconds
===================================================================
TESTSUITE libc_cnd succeeded
Running TESTSUITE libc_mtx
===================================================================
START - test_mtx_destroy
PASS - test_mtx_destroy in 0.001 seconds
===================================================================
START - test_mtx_init
PASS - test_mtx_init in 0.001 seconds
===================================================================
START - test_mtx_lock
PASS - test_mtx_lock in 0.001 seconds
===================================================================
START - test_mtx_timedlock
Expecting timedlock with timeout of 200 ms to fail
E: Timeout locking mutex 0x128220
Expecting timedlock with timeout of 200 ms to succeed after 100ms
PASS - test_mtx_timedlock in 0.316 seconds
===================================================================
START - test_mtx_trylock
E: k_mutex_unlock() failed: -16
PASS - test_mtx_trylock in 0.004 seconds
===================================================================
START - test_mtx_unlock
E: Mutex is invalid (b105f00d)
PASS - test_mtx_unlock in 0.004 seconds
===================================================================
TESTSUITE libc_mtx succeeded
Running TESTSUITE libc_once
===================================================================
START - test_call_once
PASS - test_call_once in 0.001 seconds
===================================================================
TESTSUITE libc_once succeeded
Running TESTSUITE libc_thrd
===================================================================
START - test_thrd_create_join
PASS - test_thrd_create_join in 0.001 seconds
===================================================================
START - test_thrd_current_equal
PASS - test_thrd_current_equal in 0.001 seconds
===================================================================
START - test_thrd_detach
E: Pthread 0x1286c0 is not a joinable
E: Pthread claims to be initialized (80000001)
PASS - test_thrd_detach in 0.109 seconds
===================================================================
START - test_thrd_exit
PASS - test_thrd_exit in 0.001 seconds
===================================================================
START - test_thrd_reuse
PASS - test_thrd_reuse in 0.002 seconds
===================================================================
START - test_thrd_sleep
sleeping 0 ms
sleeping 100 ms
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/thrd.c:37: libc_thrd_test_thrd_sleep: end - start >= delay_ms[i] is false
FAIL - test_thrd_sleep in 0.019 seconds
===================================================================
START - test_thrd_yield
PASS - test_thrd_yield in 0.001 seconds
===================================================================
TESTSUITE libc_thrd failed.
Running TESTSUITE libc_tss
===================================================================
START - test_tss_create_delete
PASS - test_tss_create_delete in 0.001 seconds
===================================================================
START - test_tss_get_set
E: Invalid pthread (80000006)
E: Invalid pthread (80000006)
E: Invalid pthread (80000006)
PASS - test_tss_get_set in 0.010 seconds
===================================================================
TESTSUITE libc_tss succeeded
------ TESTSUITE SUMMARY START ------
SUITE PASS - 100.00% [libc_cnd]: pass = 6, fail = 0, skip = 0, total = 6 duration = 0.409 seconds
- PASS - [libc_cnd.test_cnd_broadcast_wait] duration = 0.101 seconds
- PASS - [libc_cnd.test_cnd_errors] duration = 0.001 seconds
- PASS - [libc_cnd.test_cnd_init_destroy] duration = 0.001 seconds
- PASS - [libc_cnd.test_cnd_signal_timedwait] duration = 0.051 seconds
- PASS - [libc_cnd.test_cnd_signal_wait] duration = 0.051 seconds
- PASS - [libc_cnd.test_cnd_timedwait_timeout] duration = 0.204 seconds
SUITE PASS - 100.00% [libc_mtx]: pass = 6, fail = 0, skip = 0, total = 6 duration = 0.327 seconds
- PASS - [libc_mtx.test_mtx_destroy] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_init] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_lock] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_timedlock] duration = 0.316 seconds
- PASS - [libc_mtx.test_mtx_trylock] duration = 0.004 seconds
- PASS - [libc_mtx.test_mtx_unlock] duration = 0.004 seconds
SUITE PASS - 100.00% [libc_once]: pass = 1, fail = 0, skip = 0, total = 1 duration = 0.001 seconds
- PASS - [libc_once.test_call_once] duration = 0.001 seconds
SUITE FAIL -  85.71% [libc_thrd]: pass = 6, fail = 1, skip = 0, total = 7 duration = 0.134 seconds
- PASS - [libc_thrd.test_thrd_create_join] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_current_equal] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_detach] duration = 0.109 seconds
- PASS - [libc_thrd.test_thrd_exit] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_reuse] duration = 0.002 seconds
- FAIL - [libc_thrd.test_thrd_sleep] duration = 0.019 seconds
- PASS - [libc_thrd.test_thrd_yield] duration = 0.001 seconds
SUITE PASS - 100.00% [libc_tss]: pass = 2, fail = 0, skip = 0, total = 2 duration = 0.011 seconds
- PASS - [libc_tss.test_tss_create_delete] duration = 0.001 seconds
- PASS - [libc_tss.test_tss_get_set] duration = 0.010 seconds
------ TESTSUITE SUMMARY END ------
===================================================================
RunID: 8c65313215d86c1a1da52a982b123bdf
PROJECT EXECUTION FAILED

/twister-out/intel_adsp_cavs25/tests/lib/c_lib/thrd/libraries.libc.c11_threads.newlib/handler.log
***** delaying boot 500ms (per build configuration) *****
*** Booting Zephyr OS build zephyr-v3.5.0-2033-gde63da0081dd (delayed boot 500ms) ***
Running TESTSUITE libc_cnd
===================================================================
START - test_cnd_broadcast_wait
PASS - test_cnd_broadcast_wait in 0.101 seconds
===================================================================
START - test_cnd_errors
PASS - test_cnd_errors in 0.001 seconds
===================================================================
START - test_cnd_init_destroy
PASS - test_cnd_init_destroy in 0.001 seconds
===================================================================
START - test_cnd_signal_timedwait
PASS - test_cnd_signal_timedwait in 0.051 seconds
===================================================================
START - test_cnd_signal_wait
PASS - test_cnd_signal_wait in 0.051 seconds
===================================================================
START - test_cnd_timedwait_timeout
E: Timeout waiting on cond 0x9e028b74
PASS - test_cnd_timedwait_timeout in 0.201 seconds
===================================================================
TESTSUITE libc_cnd succeeded
Running TESTSUITE libc_mtx
===================================================================
START - test_mtx_destroy
PASS - test_mtx_destroy in 0.001 seconds
===================================================================
START - test_mtx_init
PASS - test_mtx_init in 0.001 seconds
===================================================================
START - test_mtx_lock
PASS - test_mtx_lock in 0.001 seconds
===================================================================
START - test_mtx_timedlock
Expecting timedlock with timeout of 200 ms to fail
E: Mutex is invalid (be012d48)
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/mtx.c:124: libc_mtx_test_mtx_timedlock: thrd_timedout not equal to ret
FAIL - test_mtx_timedlock in 0.001 seconds
===================================================================
START - test_mtx_trylock
E: k_mutex_unlock() failed: -16
PASS - test_mtx_trylock in 0.001 seconds
===================================================================
START - test_mtx_unlock
E: Mutex is invalid (b105f00d)
PASS - test_mtx_unlock in 0.001 seconds
===================================================================
TESTSUITE libc_mtx failed.
Running TESTSUITE libc_once
===================================================================
START - test_call_once
PASS - test_call_once in 0.001 seconds
===================================================================
TESTSUITE libc_once succeeded
Running TESTSUITE libc_thrd
===================================================================
START - test_thrd_create_join
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/thrd.c:74: libc_thrd_test_thrd_create_join: (BIOS_FOOD not equal to x)
expected: -1325010931 actual: 0
FAIL - test_thrd_create_join in 0.001 seconds
===================================================================
START - test_thrd_current_equal
PASS - test_thrd_current_equal in 0.001 seconds
===================================================================
START - test_thrd_detach
E: Pthread 0x9e028040 is not a joinable
E: Pthread claims to be initialized (80000000)
PASS - test_thrd_detach in 0.101 seconds
===================================================================
START - test_thrd_exit
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/thrd.c:99: libc_thrd_test_thrd_exit: BIOS_FOOD not equal to x
FAIL - test_thrd_exit in 0.001 seconds
===================================================================
START - test_thrd_reuse
PASS - test_thrd_reuse in 0.004 seconds
===================================================================
START - test_thrd_sleep
sleeping 0 ms
sleeping 100 ms
sleeping 200 ms
sleeping 400 ms
PASS - test_thrd_sleep in 3.854 seconds
===================================================================
START - test_thrd_yield
PASS - test_thrd_yield in 0.001 seconds
===================================================================
TESTSUITE libc_thrd failed.
Running TESTSUITE libc_tss
===================================================================
START - test_tss_create_delete
PASS - test_tss_create_delete in 0.001 seconds
===================================================================
START - test_tss_get_set
E: Invalid pthread (80000005)
E: Invalid pthread (80000005)
E: Invalid pthread (80000005)
PASS - test_tss_get_set in 0.001 seconds
===================================================================
TESTSUITE libc_tss succeeded
------ TESTSUITE SUMMARY START ------
SUITE PASS - 100.00% [libc_cnd]: pass = 6, fail = 0, skip = 0, total = 6 duration = 0.406 seconds
- PASS - [libc_cnd.test_cnd_broadcast_wait] duration = 0.101 seconds
- PASS - [libc_cnd.test_cnd_errors] duration = 0.001 seconds
- PASS - [libc_cnd.test_cnd_init_destroy] duration = 0.001 seconds
- PASS - [libc_cnd.test_cnd_signal_timedwait] duration = 0.051 seconds
- PASS - [libc_cnd.test_cnd_signal_wait] duration = 0.051 seconds
- PASS - [libc_cnd.test_cnd_timedwait_timeout] duration = 0.201 seconds
SUITE FAIL -  83.33% [libc_mtx]: pass = 5, fail = 1, skip = 0, total = 6 duration = 0.006 seconds
- PASS - [libc_mtx.test_mtx_destroy] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_init] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_lock] duration = 0.001 seconds
- FAIL - [libc_mtx.test_mtx_timedlock] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_trylock] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_unlock] duration = 0.001 seconds
SUITE PASS - 100.00% [libc_once]: pass = 1, fail = 0, skip = 0, total = 1 duration = 0.001 seconds
- PASS - [libc_once.test_call_once] duration = 0.001 seconds
SUITE FAIL -  71.43% [libc_thrd]: pass = 5, fail = 2, skip = 0, total = 7 duration = 3.963 seconds
- FAIL - [libc_thrd.test_thrd_create_join] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_current_equal] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_detach] duration = 0.101 seconds
- FAIL - [libc_thrd.test_thrd_exit] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_reuse] duration = 0.004 seconds
- PASS - [libc_thrd.test_thrd_sleep] duration = 3.854 seconds
- PASS - [libc_thrd.test_thrd_yield] duration = 0.001 seconds
SUITE PASS - 100.00% [libc_tss]: pass = 2, fail = 0, skip = 0, total = 2 duration = 0.002 seconds
- PASS - [libc_tss.test_tss_create_delete] duration = 0.001 seconds
- PASS - [libc_tss.test_tss_get_set] duration = 0.001 seconds
------ TESTSUITE SUMMARY END ------
===================================================================
RunID: 632b6d0ba6e8b65f618c8df94ee23093
PROJECT EXECUTION FAILED

/twister-out/intel_adsp_ace15_mtpm/tests/lib/c_lib/thrd/libraries.libc.c11_threads.picolibc.notls/handler.log
*** Booting Zephyr OS build zephyr-v3.5.0-2033-gde63da0081dd (delayed boot 500ms) ***
Running TESTSUITE libc_cnd
===================================================================
START - test_cnd_broadcast_wait
PASS - test_cnd_broadcast_wait in 0.101 seconds
===================================================================
START - test_cnd_errors
PASS - test_cnd_errors in 0.001 seconds
===================================================================
START - test_cnd_init_destroy
PASS - test_cnd_init_destroy in 0.001 seconds
===================================================================
START - test_cnd_signal_timedwait
PASS - test_cnd_signal_timedwait in 0.051 seconds
===================================================================
START - test_cnd_signal_wait
PASS - test_cnd_signal_wait in 0.051 seconds
===================================================================
START - test_cnd_timedwait_timeout
E: Timeout waiting on cond 0x40047c84
PASS - test_cnd_timedwait_timeout in 0.201 seconds
===================================================================
TESTSUITE libc_cnd succeeded
Running TESTSUITE libc_mtx
===================================================================
START - test_mtx_destroy
PASS - test_mtx_destroy in 0.001 seconds
===================================================================
START - test_mtx_init
PASS - test_mtx_init in 0.001 seconds
===================================================================
START - test_mtx_lock
PASS - test_mtx_lock in 0.001 seconds
===================================================================
START - test_mtx_timedlock
Expecting timedlock with timeout of 200 ms to fail
E: Mutex is invalid (a0033c58)
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/mtx.c:124: libc_mtx_test_mtx_timedlock: thrd_timedout not equal to ret
FAIL - test_mtx_timedlock in 0.001 seconds
===================================================================
START - test_mtx_trylock
E: Mutex is invalid (a0033c52)
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/mtx.c:159: libc_mtx_test_mtx_trylock: thrd_busy not equal to ret
FAIL - test_mtx_trylock in 0.001 seconds
===================================================================
START - test_mtx_unlock
E: Mutex is invalid (b105f00d)
PASS - test_mtx_unlock in 0.001 seconds
===================================================================
TESTSUITE libc_mtx failed.
Running TESTSUITE libc_once
===================================================================
START - test_call_once
PASS - test_call_once in 0.001 seconds
===================================================================
TESTSUITE libc_once succeeded
Running TESTSUITE libc_thrd
===================================================================
START - test_thrd_create_join
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/thrd.c:74: libc_thrd_test_thrd_create_join: (BIOS_FOOD not equal to x)
expected: -1325010931 actual: 0
FAIL - test_thrd_create_join in 0.001 seconds
===================================================================
START - test_thrd_current_equal
PASS - test_thrd_current_equal in 0.001 seconds
===================================================================
START - test_thrd_detach
E: Pthread 0x40047040 is not a joinable
E: Pthread claims to be initialized (80000000)
PASS - test_thrd_detach in 0.101 seconds
===================================================================
START - test_thrd_exit
Assertion failed at WEST_TOPDIR/os.rtos.zephyr.zephyr/tests/lib/c_lib/thrd/src/thrd.c:99: libc_thrd_test_thrd_exit: BIOS_FOOD not equal to x
FAIL - test_thrd_exit in 0.001 seconds
===================================================================
START - test_thrd_reuse
PASS - test_thrd_reuse in 0.003 seconds
===================================================================
START - test_thrd_sleep
sleeping 0 ms
sleeping 100 ms
sleeping 200 ms
sleeping 400 ms
PASS - test_thrd_sleep in 0.877 seconds
===================================================================
START - test_thrd_yield
PASS - test_thrd_yield in 0.001 seconds
===================================================================
TESTSUITE libc_thrd failed.
Running TESTSUITE libc_tss
===================================================================
START - test_tss_create_delete
PASS - test_tss_create_delete in 0.001 seconds
===================================================================
START - test_tss_get_set
E: Invalid pthread (80000005)
E: Invalid pthread (80000005)
E: Invalid pthread (80000005)
PASS - test_tss_get_set in 0.001 seconds
===================================================================
TESTSUITE libc_tss succeeded
------ TESTSUITE SUMMARY START ------
SUITE PASS - 100.00% [libc_cnd]: pass = 6, fail = 0, skip = 0, total = 6 duration = 0.406 seconds
- PASS - [libc_cnd.test_cnd_broadcast_wait] duration = 0.101 seconds
- PASS - [libc_cnd.test_cnd_errors] duration = 0.001 seconds
- PASS - [libc_cnd.test_cnd_init_destroy] duration = 0.001 seconds
- PASS - [libc_cnd.test_cnd_signal_timedwait] duration = 0.051 seconds
- PASS - [libc_cnd.test_cnd_signal_wait] duration = 0.051 seconds
- PASS - [libc_cnd.test_cnd_timedwait_timeout] duration = 0.201 seconds
SUITE FAIL -  66.67% [libc_mtx]: pass = 4, fail = 2, skip = 0, total = 6 duration = 0.006 seconds
- PASS - [libc_mtx.test_mtx_destroy] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_init] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_lock] duration = 0.001 seconds
- FAIL - [libc_mtx.test_mtx_timedlock] duration = 0.001 seconds
- FAIL - [libc_mtx.test_mtx_trylock] duration = 0.001 seconds
- PASS - [libc_mtx.test_mtx_unlock] duration = 0.001 seconds
SUITE PASS - 100.00% [libc_once]: pass = 1, fail = 0, skip = 0, total = 1 duration = 0.001 seconds
- PASS - [libc_once.test_call_once] duration = 0.001 seconds
SUITE FAIL -  71.43% [libc_thrd]: pass = 5, fail = 2, skip = 0, total = 7 duration = 0.985 seconds
- FAIL - [libc_thrd.test_thrd_create_join] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_current_equal] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_detach] duration = 0.101 seconds
- FAIL - [libc_thrd.test_thrd_exit] duration = 0.001 seconds
- PASS - [libc_thrd.test_thrd_reuse] duration = 0.003 seconds
- PASS - [libc_thrd.test_thrd_sleep] duration = 0.877 seconds
- PASS - [libc_thrd.test_thrd_yield] duration = 0.001 seconds
SUITE PASS - 100.00% [libc_tss]: pass = 2, fail = 0, skip = 0, total = 2 duration = 0.002 seconds
- PASS - [libc_tss.test_tss_create_delete] duration = 0.001 seconds
- PASS - [libc_tss.test_tss_get_set] duration = 0.001 seconds
------ TESTSUITE SUMMARY END ------
===================================================================
RunID: ae16559e6dc6c32cd229b2b0caa26312
PROJECT EXECUTION FAILED

Environment (please complete the following information):

OS: Linux
Toolchain Zephyr SDK 0.16.3

This is suspected to be SMP related, i.e. it passes on some of those platforms with CONFIG_SMP=n

@peter-mitsis  FYI

FWIW, I have reproduced the failures on the intel_adsp_ace15_mtpm. Those, failures could be SMP related, however the output is raising an eyebrow ...
E: Timeout locking mutex 0xnnnnnnnn  # Passing case
vs
E: Mutex is invalid (nnnnnnnn)           # Failing case

Why would we be getting an invalid mutex message? I could be wrong, but my spidey-sense suspects that an incoherent cache is coming into play--that is, one CPU is attempting to access the stack of a thread running on another CPU. (That problem does not exist on uniprocessor xtensa systems).
Digging into this ....

The xtensa cache thing is coming into play again. Take test_mtx_timedlock() for example. It creates a mutex on the stack of one thread, and then accesses it from another. Since on an Xtensa SMP system, that is likely to be on another CPU we have the not-so-good scenario of a thread on one CPU accessing stack memory of a thread running on another CPU (not so good because that memory is guaranteed to be kept in sync in that circumstance).
To get around this, either we would need to fix the tests so that the objects being tested are global objects (and thus not on the stack), or prevent this set of tests from being executed on xtensa platforms (which would arguably leave a testing hole).
As for the up_squared failure ... I have not delved into that one.

To get around this, either we would need to fix the tests so that the objects being tested are global objects (and thus not on the stack), or

That's exactly what I was thinking. I kind of suspected it was something like that. Not an extensive xtensa user unfortunately.
@peter-mitsis - Do you want to go for it, or should I?

Closing.


Update help window

Codecov Report
Base: 72.15% // Head: 72.10% // Decreases project coverage by -0.05% :warning:

Coverage data is based on head (781de0d) compared to base (85bdf07).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master      #64      +/-   ##
============================================
- Coverage     72.15%   72.10%   -0.06%
  Complexity      399      399
============================================
  Files            70       70
  Lines          1232     1233       +1
  Branches        125      125
============================================
  Hits            889      889
- Misses          311      312       +1
  Partials         32       32

Impacted Files
Coverage Δ

src/main/java/seedu/address/ui/HelpWindow.java
0.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


alveo u280 error during the performance test for face detection

Hi,
I am following quick start for alveo for running the example models and when I try to run the performance test for face detection, I get the following error:
I0528 07:56:52.808877   910 benchmark.hpp:176] writing report to <STDOUT>
I0528 07:57:04.140395   910 benchmark.hpp:203] waiting for 0/60 seconds, 8 threads running
I0528 07:57:14.140538   910 benchmark.hpp:203] waiting for 10/60 seconds, 8 threads running
terminate called after throwing an instance of 'std::runtime_error'
  what():  Error: CU timeout 1
Aborted (core dumped)

You can check  outputs before the error and xbutil query during the process in case it is useful.
I would be glad if anyone helps.

Hi @berankilic ,
Please refer to Issue-405. The densebox model was found some issues on DPUCAHX8L, so please try another sample instead of face detection.


Update Wandsworth Christmas visit slots

Unbookable - Christmas Day, 28th Dec, New Year's Day

Unable to test in preprod - too far ahead.

Tested locally


Schema options after rules

Why this feature is required (specific use-cases will be appreciated)?
Most AdonisJS users happen to even add trim option when create string schema validator, if passing rules to this string validator. It is not comfortable!
example:

It would be great if the options were after the rules, then all users wouldn't have to passing trim option for string validator every time.
As a suggestion, make a escape option to false a default, because, i think, users do not use this a lot.
If users need to pass true to the escape parameter, they will simply pass it after the rules.
You can also make a vote is it worth it to do so, suddenly I'm wrong.
Have you tried any other work arounds?
No, I haven't found any other work arounds.
Are you willing to work on it with little guidance?
I'm ready to do whatever is necessary to achieve this goal.

Going to be a breaking change and cannot be shipped until AdonisJS 6.0

This commit makes the first argument to accept both the options or an array of rules.
After the next release, you can simplify your code as follows.
schema.string([
  rules.unique(),
  rules.minLength()
])


revert -g avoidance

I will merge this.

@paciorek FYI in case you hadn't seen it, there is a git revert command that sometimes works. In this case we may have been able to simply
git checkout devel
git checkout -b revert-g-flag
git revert 7f3553fa


Tableau Connector : Unify Data Models

Is your feature request related to a problem? Please describe.
When ingesting Data Models in tableau, multiple datamodels are displayed for the same data source.
This explodes the number of total data sources, even though unique, and makes discovery and lineage more complicated.

Describe the solution you'd like
Today OMD relies on the nodes segment of Tableau metadata to create the data model.
  embeddedDatasourcesConnection(first: {first}, offset: {offset} ) {{
    nodes {{
      id
      name
      fields {{
        id
        name
        upstreamColumns{{
          id
          name
          remoteType
        }}

But perhaps a better way would be to create the data model based on the root data model, since these share the same ID across the models

@harshach I wanted to chime in on this conversation. At my organisation, we're ingesting a large Tableau instance and we've also noticed this behavior where there are multiple versions of the same datasource. What we found, is that a workbook (dashboard) can have it's own embedded datasource, which is a workbook unique version of an upstream datasource that it connects to (usually one that exists on Tableau server). The reason for this, seems to be, that a workbook can connect to a datasource, then change field names, add calculated fields and do various other things to have it's own version of the connected datasource.
Ideally what we would like (and @jsampaiog please jump in if you disagree); is for published datasources to be ingested into OpenMetadata as well as the embedded datasources (would be nice to have a different icon to differentiated the different datamodels).
I think it's important to keep both the published and embedded datasources, because that way we can see what transformations have occurred at the workbook level and compare it to the published server model.
Here's a screenshot of what it might look like:

Hi @chillerno1, thanks for chiming in. Indeed your depicted behavior would be the best target scenario!

Thanks @jsampaiog, I agree with that!

@pmbrull are you still planning to include this in 1.4.0 release? I see that was removed :(

hi we had to reprioritize certain topics and ran out of time to handle this, so 1.4.1 - 1.5 would be the new ETA.
My 2 cents on the conversation above is to keep things simple. Aiming to keep the Published DataModel IMO would be the way to go to reduce complexity

@pmbrull thanks for the context on the timelines.
I believe that "Published DataModel" should do the job even in case of "Dashboard" with embedded data models. We just need to be sure that we don't introduce a regression, where data models are totally missing.

Thanks @OnkarVO7 , for the this thread.
We also have similar problem of having duplicate Models rather a combined model for all workbooks down the stream.
Since currently OMD use this query
query  {
embeddedDatasourcesConnection(filter: {name: "Tech Data Model"}) {
nodes{
id
name
workbook {
id
name
}
}
totalCount
}
}
We checked with Tableau team (spent a lot of time with Tableau support team to get information in right way) and they proposed to use below query
query  {
publishedDatasources(filter: {name: "Tech Data Model"}) {
id
name
hasExtracts
downstreamWorkbooks{
id
luid
name
}
}
}

Hi,
Sorry for commenting in this thread, we are facing the same situation: the sources are duplicated for each workbook (dashboard in OM) that we ingested.

The dashboard datamodel exists only once on Tableau:

In addition, we have it separated by different services, each service is a tableau folder, since this allows us to assign owner by folder, perhaps, if in the ingestion the folder (tableau) is ingested as the database service would allow us to maintain that hierarchy that also allows us to filter by folder:
DB Ingestion->Schema->table
Ingestion Tableau->Folder->Workbook & datamodels
thanks,
Carlos

Do I get this right, that using the query provided in here https://github.com/open-metadata/OpenMetadata/issues/15218#issuecomment-2084583097 might be enough to cover this issue?

Hi! A quick question. Was it finally fix this issue in version 1.5? I'd like to understand if yes, so start adding the new version


Support for request logging in kafka emitter.

This is an initial version of #5700 fix. Works as expected, rich request logs are emitted to dedicated kafka topic as complex JSON structure. But the overall design of the solution is ugly, kafka emitter needs to know too many details about emitted events to serialize them correctly.
For ServiceMetricEvent and AlertEvents kafka emitter serializes map like structure of the event. But for ReqestLogEvent it does not work due to complex query and queryStats fields. So for the complex object like RequestLogEvent Jakson databind is used for serialization instead of Event.toMap method.
In my opinion emitters should depend fully on Jakson databind for serialization instead of Event.toMap method. But it requires changes in ServiceMetricEvent and AlertEvents and I'm not sure about impact on another emitter implementations.
@dkhwangbo, @jihoonson What do you think?

@mkuthan I got a little bit misunderstanding, so please describe more detail about your question with example. Is it not work with it?
It's another question: It's mandatory to use? I think it's proper using optionally because default value of druid.request.logging.type is noop. (docs)

@dkhwangbo  As long as EmittingRequestLogger.toMap returns empty map, kafka emitter emits only empty JSON structure instead of query payload with stats.
EmittingRequestLogger.toMap method is not implemented because there is no way to map complex RequestLogEvent structure into flat map. There is polymorphic query object and map like query stats in the RequestLogEvent class.

@mkuthan sorry for late response. I'm reviewing this PR.

Further work is continued in #6493.


Can't Install Defiads, Error Related to C++

I tried to compile defiads today. When I got to cargo build --release it began compiling various things and then broke at this point:
-------- Begin code --------
Compiling aho-corasick v0.7.6
Compiling futures-util-preview v0.3.0-alpha.18
Compiling bstr v0.2.8
error: failed to run custom build command for bitcoinconsensus v0.17.1
Caused by:
process didn't exit successfully: /home/[blocked-for-privacy]/defiads/target/release/build/bitcoinconsensus-92955efa0baa0f53/build-script-build (exit code: 1)
--- stdout
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
CRATE_CC_NO_DEFAULTS = None
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
running: "c++" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "bitcoin/src" "-I" "bitcoin/src/secp256k1/include" "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-D__STDC_FORMAT_MACROS" "-o" "/home/[blocked-for-privacy]/defiads/target/release/build/bitcoinconsensus-64a41d0281aadc9e/out/bitcoin/src/utilstrencodings.o" "-c" "bitcoin/src/utilstrencodings.cpp"
--- stderr
error occurred: Failed to find tool. Is c++ installed?
warning: build failed, waiting for other jobs to finish...
error: build failed
-------- End of code --------
I have g++ installed. Am I supposed to have something else installed to make c++ work?

Most distros will alias c++ to g++ if installed, or maybe you need to run update-alternatives and select for c++ what g++ will be, what is your distro?

My distro is debian stretch. The problem was that I was wrong -- I did not have g++ installed. This issue can be closed.


added command issuemorefrom

Hi, I just added command issuemorefrom.

The checked failed because PHPUnit_Framework_TestCase was not found.
PHP Fatal error:  Class 'PHPUnit_Framework_TestCase' not found in /home/travis/build/Kunstmaan/libphp-multichain/tests/MultichainClientTest.php on line 6


Don't show debug information when imported

Expected Behavior
iohook should be imported without creating any noise in the terminal.
Current Behavior
It is logging this:
refresh_locale_list [476]: GetKeyboardLayoutList(0, NULL) found 2 layouts.
refresh_locale_list [493]: Received 2 locales.
load_input_helper [866]: refresh_locale_list() found 2 locale(s).

Possible Solution
Silently perform this in the background and only alert if failed.
Steps to Reproduce (for bugs)
require("iohook")

Context
I want to be able to silently import iohook without it spitting information in the terminal.
Your Environment

Version used: 0.6.5
Environment name and version (e.g. Chrome 39, node.js 5.4): Node.js 13.7.0
Operating System and version (desktop or mobile): Windows 10

cc @Djiit

@Djiit Ping!

@Djiit Friendly ping!

Hey there :-)
I'm sorry I don't have much free time these days.
Maybe you could open a PR with a proposed solution ?

Sure! I was going to do that initially except I've been getting dabbed on by the build requirements.

The code is pretty old as we speak. I'm more able to maintain the JS aspects, as I'm not the original author :), so yeah, any help is appreciated ! Thanks!

After some investigating, it appears that this line is the culprit:
https://github.com/wilix-team/iohook/blob/master/libuiohook/src/logger.c#L42

Somewhere in the native interface for this library, you can do something like this:
bool logger_proc(unsigned int level, const char *format, ...) {
    bool status = false;

    va_list args;
    switch (level) {
        #ifdef USE_DEBUG
        case LOG_LEVEL_DEBUG:
        case LOG_LEVEL_INFO:
            va_start(args, format);
            status = vfprintf(stdout, format, args) >= 0;
            va_end(args);
            break;
        #endif

        case LOG_LEVEL_WARN:
        case LOG_LEVEL_ERROR:
            va_start(args, format);
            status = vfprintf(stderr, format, args) >= 0;
            va_end(args);
            break;
    }

    return status;
}

...

hook_set_logger_proc(&logger_proc);

I am not sure of node has a logging facility, but if it does, logger_proc should probably interface with it instead of std output.

See: https://github.com/wilix-team/iohook/blob/cd1c5ae74c46d00e8739bee1dec66f58fb63e612/libuiohook/src/logger.c

You shouldn't need to monkey around with libuiohook, only call logger_proc with a new function from iohook.cc.

@kwhat I have no idea how to program in C++ but if we can somehow callback to the main script then it would be much easier to do logging.

@Richienb just console.log?

@kwhat console.log in C++?

Sorry, I am a bit of a novice at node outside of just using NPM to get and build stuff.  It looks like iohook.cc already has a logger function setup, so the only thing that would need to be done would be to change that function to call a javascript function that does logging.  This is effectively what is happening in the [JNI](https://github.com/kwhat/jnativehook/blob/2.1/src/jni/jni_Logger.c#L97 example I provided earlier for the Java side of things.  This library provides no javascript logging facility, so I am trying to figure out what we would call back to in logger_proc which is why I was asking about console.log.  This function would just need to be updated to call out to the javascripts console methods to log instead of going out to stdout/stderr as it currently does.  As I previously stated, I don't know much about node, but this seems to be the most basic method of logging in node.  There are at least another 3 NPM logging libraries that do more advanced logging and the logger_proc method could easily interface with any number of those by either using some type of configuration or some yet to be created iohook javascript logging facility.  This logging facility would need to have a javascript method that is called from the native code directly (logger_proc) and it would need to call a callback function to do the logging.  There should also be a default callback function and a method of setting the current callback function.  There maybe a way to do this from the native code directly and just skip the javascript part, but it would probably be easier and cleaner to just do it in javascript.  I will have to do some more digging.

@kwhat The easiest way to do this is to use a Nan Callback to pass on all the arguments logger_proc is provided with.

To stop the logs coming from iohook you need simply pass false in the start function
const ioHook = require('iohook');
ioHook.start(false);

Will make sure this is in docs.

Added to doc.


Use tuple as function arguments

🚀 Feature request
Current Behavior
If you want to pass a tuple as function arguments you have to deconstruct the tuple, but miss point free syntax.
declare const f: (a: A, b: B) => C;

pipe(
  tuple(a, b),
  x => f(...x),
);

Desired Behavior
Use function apT to resolve this
declare const f: (a: A, b: B) => C;

// apT will apply the tuple as function arguments
const fT: (t: [A, B]) => C = apT(f);

pipe(
  tuple(a, b),
  fT,
);

Suggested Solution
This function could be implemented as
const apT: <A extends Array<unknown>, B>(f: (...a: A) => B) => (a: A) => B =
    f => a => f(...a);

Your environment
Every OS

Software
Version(s)

fp-ts
2.2.0

TypeScript
3.7.2

Is unary the right name for this functionality? It clashes with all other references and definitions of unary function or operator I have seen.

@gkamperis we can change the name, no problem. How can we name a combinator that accepts a function (a: A, b: B) => C and returns a function (x: [A, B]) => C?

Ideally should be a <name> such that

<name>: (f: (a: A, b: B) => C) => (t: [A, B]) => C
un<name>: (f: (t: [A, B]) => C) => (a: A, b: B) => C)

(like the pair "curry" / "uncurry")

apT / unapT?
unary / ununary?

It's a tough one... But personally I would exclude unary.
Take the below with kindness... :)
tupleArg
untupleArg

structureArg
destructureArg

In Scala there's tupled

@gcanti better than unary in my opinion

Thanks for the feature! :+1:
Are you considering adding unTupled yet?
export function unTupled<A extends Array<unknown>, B>(f: (a: A) => B): (...a: A) => B {
  return (...a) => f(a)
}

@frndmg https://github.com/gcanti/fp-ts/pull/1063/commits/57417ff970c1b5dddc8dafcb666389f4aba4a5a9

Great!


New data table from TestByAdam

Data table added by TestByAdam at 2015-05-18T12_25_06.287873

Will merge this with correct username


Better dry view integration

Update to dry-view 0.2.x, provide a formal roda plugin for easy rendering of views, and generate a "welcome" view in new apps.

In this PR I'm also removing the specs, which were very out of date, and very minimal. This'll let the build go green and allow me to merge this.
Next I plan to work on an improved way to test this: generate an app on the fly, modify files within it, add some extra files to it as needed, and then test on that generated app. This'll ensure we're always testing the actual shape/structure of a freshly generated dry-web-roda app.


Use requirements.txt to install python packages

Just found your tool - cool. Figured would add this to make installing packages easier.

Thanks alot for the pull request! :+1:


Getting error when starting Webpack

Hi Sayar,
I followed the procedure of installing webpack and npm modules for this project but I am getting below error when giving command npm run start-webpack-server.

Please help.

Hello, the repo and the videos are fairly out of date. We're working on a new version to be released soon.

@pankaj-arunsingh I updated the webpack scripts throughout. You should no longer experience that error.

Thanks a lot @sayar . This really helped.


Medical imaging tutorial

I created a medical imaging tutorial that shows how to work with X-ray DICOM data. For this matter, I updated the load_image function in vision.core to be able to load DICOM files using pydicom. For the tutorial, I created a small subset of the SIIM-ACR Pneumothorax Segmentation dataset with 250 DICOM files and a .csv file with labels. I informed the SIIM and they gave consent to create this subset and put it online. I'm currently hosting the .tgz file on a server of mine, but I think it would be a good idea to put it on a CDN together with the other datasets. I can forward to you the email with the permission statement from the SIIM to use their data if needed.

I would very much like vision.core to stay independent from pydicom to avoid the dependency on the main library (ultimately, fastai.medical will be a real submodule). In medical.imaging, you have a patch to Path objects to implement a dcmread method for opening them that could be useful.
We can host the tgz with our other datasets if the SIIM is okay with that.

Hi Sylvain, yes I totally agree, it's much cleaner to keep it separately. I'd really like to use the vision functionality (data augmentation etc.), though. Do you have a recommendation how to integrate the DICOM loading in the most efficient way and still being able to use the vision functionality?

I see you're using cls=PILImageBW in your data block. Why not create your own subclass of it, named  DicomImageBW. You can have the code you used to load inside the .create method and it should work as is since you would still have the behavior of PILImageBW everywhere.

Thanks a lot for your input, that was very helpful! I now created a subclass PILDicom based on PILBase. Additionally, I modified the dcmread function so that it can be used to in the PILDicom.create function. I made the according changes in the tutorial. How do you want to proceed with the small SIIM dataset? Will you download it from http://files.vedavimedical.com/siim_small.tgz and put it on your cdn?

Thanks a lot! Two things: I think we can hide one output in your notebook. And the second is I'm not sure you should change the dcmread command as PIL does not deal with int16 images AFAIK. All of this should be in your PILDicom only.

I just found this closed issue https://github.com/pytorch/vision/issues/105 that states that PIL handles int16 grayscale images. I now reverted the dcmread function to the old version and use the PIL conversion in the PILDicom.

Thanks for making the changes. This is looking pretty great :)

One follow up: the image you put as an attachment in the notebook did not arrive on GitHub. Can you put it in the images directory in another PR? Thanks!
I'll deal with the dataset today and adjust the url when I've put it on out server.

Oh it is there properly, just not working withour doc building. Sorry, no need to do anything, will fix manually :)


Project dead ?

Looking to implement this for personal use and probably in production environment but not sure if it is still maintained/updated because the last commit is almost 2 years old.

Unfortunately I think it is, I was looking to use this to replace Pushbullet but don't think I can switch :disappointed:
For some reason the Android client was updated a lot more recently though? Wish I had the skills to maintain a fork..

I would be interested in helping to maintain a fork. But I would need some help. Does anybody have interest in helping?

100% interested! Let's see if we can get some more people on board.

I would be really interested too!

I use pushjet for notifications from lab experiments, really useful and have recently tried to find alternatives, but failed. I don't know java for the android client, but would be very interested in helping maintain the python components

I have create a slack group we can use to collaborate on this:
https://join.slack.com/t/pushjetfork/shared_invite/enQtNDMxNzk4MjU2OTk0LTZlMjZmOWRlNmJjYWE4MmExOWE0OGI1ZmVkZTE4ZjA3NDEyOTBiYjI3ODgyMzVjYzE3OTVmMjg1ZmZjZmMzMTk
Feel free to join and move the discussion to slack.
If you don't like slack. Please, feel free to suggest any other alternative.

@4oo4 @ScheissSchiesser @aminehmida @weatherhead99 I'm currently working on updating Pushjet to use Firebase Cloud Messaging as opposed to Google Cloud Messaging. I know enough Java/Go/Python to understand @Mechazawa code base. Just have to work through understanding Android Studio and creating apps. If any of you have started on a fork, please let me know! I'll be happy to contribute.

@jsonvillanueva We are currently working on a free-and-open-source fork and discussing path forward regarding encryption. Please join our slack group: fosspush slack

gotify/server is also an alternative, it's a little different as Gotify has a user management, but is has the same premise.


Add Bootstrap Extension Documentation

Basic documentation to link to the Bootstrap Extension plugin.

Once again, should the plugin be moved over to your GitHub to keep it all in the same place?

Let's move it then, since Bootstrap Extension is very tightly bound to Bootstrap, infotexture/dita-bootstrap.extension is the most logical place to host  an extension to infotexture/dita-bootstrap`.

@jason-fox This should be good to go now.

Shall we merge this and package up the initial release of dita-bootstrap.extension for the registry?
Shall we tag that as plug-in version 1.0.0, or change the initial version to 5.3.3 to indicate the DITA Bootstrap version it was designed for?
Anything else you'd like to add or change before DITA-OT Day?

Shall we merge this and package up the initial release of dita-bootstrap.extension for the registry?

That sounds like a good plan. . :shipit:

Shall we tag that as plug-in version 1.0.0 or 5.3.3 to indicate the DITA Bootstrap version it was designed for or 5.3.0 to indicate the version of @slackero/bootstrap-extension it bundles?

I would opt for 5.3.0 - the extension is unlikely to be updated frequently, and appears to only be updated for Bootstrap Minor releases, not every build. Actually as more of the functions get bundled into Bootstrap proper (as has been the case in the past), then Bootstrap extension becomes less and less likely to be required.  The dependency of Extension would need to be DITA Bootstrap 5.3.4 since it relies on changes made since the last  DITA Bootstrap release.

Anything else you'd like to add or change before DITA-OT Day?

DITA Bootstrap itself needs a new release, as does the Lunr plugin. Only bug fixes, but its important to get everything working on Windows.

OK, packaged up the new releases for the Lunr, Sass, & Extensions and will now merge this, tag the 5.3.4 release here, and submit the whole lot to the registry.


Allow custom http client for authorization

This library relies on the JDK HttpClient to connect to the authorization server. This api offers a rich configuration, for example the connect timeout or the used ssl context for validating the tls connection.
A lib consumer is forced to use the default http client, thus disallowing self signed certifcates or using a proxy. It is desirable to extend the API to allow an already preconfigured http client for authorization.

Fixed with https://github.com/coffeelibs/tiny-oauth2-client/commit/c5c4d2553ef19a3612fc062a7b9cae83b48c02e3


Understanding c++ templates

Be sure to visit our Resources Page for tools, resources, and example articles to go over.
NOTE: We tend to stray away or tend not to publish reviews/comparisons of commercial product offerings.
Proposed title of article
Understanding c++ templates
Introduction paragraph(s):
A template is a method of writing a single function or class for a family of related functions or classes. In plain terminology, you can declare a single function or a class to work with different data types using templates.  This helps the developer avoid redundant code in a program. In the broader codebase, we use templates for code reusability and software flexibility. This article will take a look at templates and how to use them for generic programming in c++.
Key takeaways:

How templates work and when to use them
Class and function templates
Variadic function templates
How to create a class and function template using intuitive programs
Advantages of templates

References:
N/A
Templates to use as guides

How To Guide Template
Software Review Template
Tutorial Template

good topic @Dawe-7 - approved


Visual Studio needed to install??

@Crafter-san took a while to respond to this issue. nodejs-snowflake v2.0.0 fixes the above issue and significantly improves (roughly 120% increase).

Kindly close the issue if it is working for you now...


Add support for Amplitude clickstream/tracking

Description
Add support for Admin to specify Amplitude apikey for tracking/analytics
Review Time Estimate

[ ] Immediately
[x] Within one week
[ ] When possible

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the CHANGELOG.md.
[ ] I have signed the CLA
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

will this PR eventually contain the actual tracking code, or will that come in a different PR?

Question asked and answered 😆
Supplementary PR that contains the actual event code, which may require a bit more discussion: https://github.com/clowder-framework/clowder/pull/133
I'm hoping that this one will be a quicker merge while we discuss the actual events themselves

Question. Would it make sense to have a small javascript file that gets included on the main page. This javascript file will have 3 pieces:

initalize the code based on what api keys are set (google, amplitude, others), will be always loaded
page load event, send a page load to each service enabled, will be always loaded
logevent(code, parameters), an explicit function that can call all of the serviecs. right now it will just implement the clickstream.

The thinking is that if we add additional trackers, we can just update this one javascript file and it will support a new service.

Question. Would it make sense to have a small javascript file that gets included on the main page. This javascript file will have 3 pieces:

initalize the code based on what api keys are set (google, amplitude, others), will be always loaded
page load event, send a page load to each service enabled, will be always loaded
logevent(code, parameters), an explicit function that can call all of the serviecs. right now it will just implement the clickstream.

The thinking is that if we add additional trackers, we can just update this one javascript file and it will support a new service.

In the developer meeting today we discussed doing this at a later date


data processing

Your job is great, I want to know how you handle the . csv files.
Thanks

Hi, Did you get any text  input text sample in the .csv file ?


Does not work with node.js

The dependency on module jkroso-type has some ES6 usage of Map and Set, which node.js does not support.
If Map and Set have not been polyfilled, an error occurs:
ReferenceError: Map is not defined
at type (jkroso-type/index.js:20:22)

The latest node has both those. I guess I should update the "engine" metadata

Ahh you're right.  Thanks for the quick reply.  Closing issue.


Adding sensor interface and LM75 i2c sensor support

A new sensor interface is implemented, that reads sensors in the specified interval in a seperated thread. New sensors may easily be implemented by using the base class Sensor(GenericSensor) and implementing its methods. YAML config schema was extended for sensor interface. LM75 i2c temperatur sensor is implemented. This will be a basic step for #40.

Thanks for your review and the good comments.

I'm not in a position to test this out right now, so I take it it all works for you? Do you use the IO modules as well as the sensor ones?

I just tested the commented version on my raspberry with gpio modules only, sensor modules only and both gpio and sensor modules. It's just working fine. The version without your comments is running since 14 days in my home automation 24/7.

From what I understand, and just to confirm, this doesn't break backwards compatibility with existing configurations, or add any dependencies does it?

From what I understand, and just to confirm, this doesn't break backwards compatibility with existing configurations, or add any dependencies does it?

Thats correct. It still works fine with your original config.example.yml, that has no sensors configured. If someone notices a bug/defect, I'll fix it, of course. ;-)

Sorry for the delay merging this. I'll try to remember to do it this weekend.


Deployment part of portfolio

https://wils-fourr-speed.github.io/My-portfolio/

Hi @Wils-FOURR-speed ,

Great work, your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉
Remember: Every finish line 🏁  is the beginning of a new race 🐎
Highlights

[ ] Project is deployed ✔
[ ] Deployment is up & running ✔

Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.


PNG compress and resize

At the moment I am manually resizing and compressing the PNGs inside the animations.
Would you consider adding these functions as part of the export?

I did some experimenting with this, but it's complicated because it slows down rendering too much.
I still need to experiment some more and see if I can at least use a node module for compressing or exporting jpegs.


Host IP not showing up in container details page of LB Agent

v0.29.0-rc2
Steps To Reproduce:

Start a Load Balancer
Click on LBAgent Container

Result: Host IP is missing.
Expected Result: Since the agent is running on a host, the host IP should be shown.

Verified Fixed v0.30.0-rc4


#147 Split test for resultMatch and resultNoMatch into separate blocks

...for consistency and better readability, as recommended.

This closes #147


Fix color detection

Update dependencies and restore and chalk for improved color support detection. This should fix issues with Windows terminals multiplying or otherwise messing up the output.

Looks like chalk includes supports-color as an inline vendor module and also exports it, so we don't need to install both:
https://github.com/chalk/chalk/blob/a370f468a43999e4397094ff5c3d17aadcc4860e/source/index.js#L220-L223

Ping @okonet should we merge this?

Lgtm!

Looks like semantic-release also messed up by publishing 13.1.3 and 13.1.4 instead of 13.1.3-next.1 etc, so we will have to skip those two versions from the latest tag. 🙈

Should we switch to changesets? I feel it is much more reliable and controllable.

It might be good, yes... I'm up for it but would need to spend some on it, unless you want to implement.
For now maybe I just create two empty fix: version bump only Pull Requests so we get 13.1.5 released?

Actually, a single feat: version bump only fixes this since semantic-release didn't even get to creating the versions in the repo.

I can implement when I find some time. No pressure ;)


FStar backend usize literals support

We're hitting this:
error[HAX0001]: (FStar backend) something is not implemented yet.
                usize literals (fail if pattern maching, otherwise TODO)
   --> src/riot-rs-runqueue/src/runqueue.rs:143:17
    |
143 |                 0 => i.is_none(),
    |                 ^

This is a known limitation. It is better to cast the usize into u8 in the pattern match:
match (x as u8) {.
  0 => ...
} ```

I'm closing that issue in favor of https://github.com/hacspec/hax/issues/464, and PR #602 changes the error to mention both Karthik's workaround and issue #464.

Thanks!


feat($compile): support omitting required controller name if same as the local name

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Feature
What is the current behavior? (You can also link to an open issue here)
You have to explicitly specify the required controller's name, even if it's the same with the local name used to bind to the requiring controller (unlike how scope: {...} works, for example).
What is the new behavior (if this is a feature change)?
require: {someDir: '?^someDir'} can be shortened as require: {someDir: '?^'}.
Does this PR introduce a breaking change?
No
Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

LGTM


[USMP] Adding U4 usecase

This commit adds a new usecase (U4) for
the USMP that enables the user to place I/O
tensors within the workspace and not
providing them seperately.
This allows the space for I/O tensors to
be utilized by the inference.

cc : @Mousius @leandron @areusch

thanks @manupa-arm , just one point to clarify now

will let @Mousius see if he has any feedback and you guys can merge whenever

Thanks @Mousius for catching that! now fixed!

A friendly ping for review/merge...

@Mousius could you take another look when you get a minute?


Remove version checks for Job and CronJob task pages

Currently the pages about Job & CronJob instruct readers to check their cluster version, without saying what version is needed.
Job has been in Kubernetes for a good while and CronJob has been beta since v1.8
With that in mind, drop the version check from the relevant Task pages.
/kind cleanup

/assign

/lgtm
/approve


Add byte[] taking variant(s) in ByteBufferUtils for Base64 handling

Since some use cases (esp. ones to do with gRPC messages, see #1402) need to work on byte[], not ByteBuffer; and since ByteBufferUtils already internally converts things to byte[] as necessary, it'd make sense to expose that functionality.
In addition, use of Stream operations for character checking, a la:
if (base64.chars().anyMatch(c -> c == '/' || c == '+' || c == ' ')) {

is rather inefficient (see https://cowtowncoder.medium.com/measuring-string-indexofany-string-performance-java-fecb9eb473fa ), it'd make sense to optimize this case.

Based on @dougwettlaufer's comments on PR, looks like this special handling no longer needed, so changed PR to instead remove it. The best outcome possible!


valid_url() in example.py

valid_url() was not working correctly. This change fixed it for me:
return all([p['scheme'], p['authority'], p['path']])
all() has to get an iterable object, so I put the three elements in a list...

Sounds good. I'll put this change in.


TensorFlow Samples Do Not Run - Can't get past documented 16358, 17393 regardless version used

Installed without error tensorflow 1.10 as instructed on the tensorflow.org site(https://www.tensorflow.org/install/install_windows)
Executed without error the following sample code from tensorflow.org site (https://www.tensorflow.org/tutorials/keras/basic_text_classification)
import tensorflow as tf
from tensorflow import keras
import numpy as np
print(tf.version)
imdb = keras.datasets.imdb
When the next sample line was executed it generated the errors documented under "import tensorflow failed, "ImportError: DLL load failed". Even after install visual studio 2015, Microsoft Visual C++ 2015 Redistributable Update 3.  #17393" displayed:
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
I tried uninstalling & installing every version of tensorflow from 1.5 to present version. The error occurred for each version.
In attempting to get this short, simple tensorflow example, I have encountered not only this DLL issue but also:
AttributeError: module 'numpy' has no attribute 'version_'
ModuleNotFoundError: No module named 'keras'
(Request for updating keras/datasets files to r1.5  #16358)
Does not appear the documented tensorflow site sample(s) works.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce
Mobile device

OS Platform and Distribution: Windows 8
TensorFlow installed from: tensor flow.org
TensorFlow version: tried 1.5 to most recent 1.10
Bazel version: N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce:
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
Mobile device: N/A
ERRORS:
ImportError: DLL load failed
(As reported in #17393)
AttributeError: module 'numpy' has no attribute 'version_'
(As reported in multiple postings)
ModuleNotFoundError: No module named 'keras'
(As reported in #16358)

The problem statement from the tensor flow sample program is:
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
Initially, the noted statement generated a DLL error on this statement after successfully running the following statements from the posted tensorflow sample:
import tensorflow as tf
from tensorflow import keras
import numpy as np
print(tf.version)
imdb = keras.datasets.imdb
I installed the recommended C++ Redistributable as advised by the error displayed (17393).
The problem statement then began to generate the following error.
AttributeError: module 'numpy' has no attribute
I removed tensor flow 1.10 and installed tensorflow 1.5 as advised by another posting.
The problem statement then began to generate the following error.
ModuleNotFoundError: No module named 'eras' (#16358)
I then tried different versions of tensor flow on the basis that various postings hinted this was a version issue that was reportedly fixed in the next release.
So far, none of the postings on this topic resolve this issue.

Nagging Assignee @martinwicke: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @martinwicke: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @martinwicke: It has been 44 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Regarding error 2: indices = np.arrange(len(x_train)) appears to be a typo, but in the current version of imdb.py this has been fixed (on Aug 1).
I cannot reproduce the library problem.
Importing directly from tensorflow.python is definitely not a good idea. Here's a workaround (though this style of import should really work). I have tested this on 1.8, so it's not a new thing:
>>> import tensorflow as tf
>>> imdb = tf.keras.datasets.imdb
>>> imdb
<module 'tensorflow.tools.api.generator.api.keras.datasets.imdb' from '/Users/wicke/Library/Python/2.7/lib/python/site-packages/tensorflow/tools/api/generator/api/keras/datasets/imdb/__init__.pyc'>
>>> dir(imdb)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'get_word_index', 'load_data']
>>>

Nagging Assignee @martinwicke: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @martinwicke: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @martinwicke: It has been 44 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.


Brew - download link is broken

Hi,
The download URL is not correct:
haithem@dbt-ultimile % brew install quarylabs/quary/sqruff
==> Fetching quarylabs/quary/sqruff
==> Downloading https://github.com/quarylabs/quary/releases/download/0.0.6/sqruff-darwin-aarch64.tar.gz
curl: (22) The requested URL returned error: 404

Hey @haithem-souala
Sorry, you've had that experience. Thank you though for filing the issue. We don't quite understand how that has happened but we're pushing a fix:
https://github.com/quarylabs/homebrew-quary/pull/7
We'll also investigate it to prevent it happening again. I'll let you know when it's been merged.

https://github.com/quarylabs/homebrew-quary/pull/7 was merged ;)
Let me know if that works for you.

Hey @haithem-souala,
It looks like you are trying to run sqruff on top of a dbt project. We don't yet support parsing the dbt syntax, in fact we only support ANSI. My guess is that is why the parser is breaking.

Understood, thank you!

I would definitely recommend https://sqlfluff.com/ in the meantime.


[Bug] [org/apache/seatunnel/shade/com/typesafe/config/impl/ConfigParser.java] java: 不兼容的类型: com.typesafe.config.ConfigIncluder无法转换为org.apache.seatunnel.shade.com.typesafe.config.ConfigIncluder

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
I download the source code of dev. find something problem that I can`t solve，I guess that a bug of org.apache.seatunnel.shade.com.typesafe.config.Config.（https://github.com/apache/seatunnel/discussions/1043）
The following are my questions：

I have the confuse of the class:  org/apache/seatunnel/shade/com/typesafe/config/impl/ConfigParser.java
thats no problem ？  I remember Have a hot fix of the class(org/apache/seatunnel/shade/com/typesafe/config/impl/ConfigParser.java)  but, why i cant run my code in my PC？
it`s my configuration.

I try it：mvn clean install, but  can`t solve it.
SeaTunnel Version
our seatunnel version is 2.3.8
SeaTunnel Config
in my question of sreenshot

Running Command
no

Error Exception
java: 不兼容的类型: com.typesafe.config.ConfigIncluder无法转换为org.apache.seatunnel.shade.com.typesafe.config.ConfigIncluder

Zeta or Flink or Spark Version
No response
Java or Scala Version
1.8
Screenshots

Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

i guess something of  BUG in branch of dev. I try again：sh ./mvnw clean install -DskipTests -Dskip.spotless=true

I try 2.3.8 and 2.3.5 , I can`t run it in my pc ,always have the problem.
com.typesafe.config.ConfigIncluder无法转换为org.apache.seatunnel.shade.com.typesafe.config.ConfigIncluder.

try ./mvnw install -Dmaven.test.skip

try ./mvnw install -Dmaven.test.skip

oh,my big bro , that look no use。  our seatunnel source code can work well in local PC?  OR that a bug for run seatunnel source code in local PC? I try it again,again,again, but it look so upset.

Our company is looking forward to use seatunnel in production environment，could you help me?  big bro~

try ./mvnw install -Dmaven.test.skip

if you in china, may be we can exchange wehat and discuss in depth the question.xues snacks NEED YOU. Its my we chat:18770170039

thanks for bigbro to solving the proble. why we meet the question? because window enviroment will cut out maven command.
在 Windows 环境下，命令行解析器（如 CMD 或 PowerShell）对命令的解析可能导致参数被截断或错误解释。具体到你的问题中，-D 后面的内容可能由于解析器对符号 . 的处理不当而被误解析。
错误根本原因：
Maven将.test.skip 解释为i一个生命周期阶段（lfiecycle phase）,由于命令解释器错误地分割-D后面的内容,导致它变成了无效参数.
解决方案：
step1: 将拉取下来的代码回滚 rollback（可以全部回滚的，亲测）
step2:
./mvnw install  -Dmaven.test.skip=true
更改为：
./mvnw install  '-Dmaven.test.skip=true'
并在在控制台输入.


Preview md files even though 'Enable Markdown' option is unchecked

Microsoft PowerToys version
0.33.1
Running as admin
No
Area(s) with issue?
MD Preview
Other Software
No response
Steps to reproduce

Start PowerToys as admin
Enable Markdown preview
Verify that md files are shown in the preview
Disable Markdown preview
Verify that md files are not shown in the preview
Restart PowerToys as user
Verify that md files are not shown
Restart PowerToys as administrator
Verify that md files are shown even though Markdown preview is not enbaled

✔ Expected Behavior
Markdown preview does not work
❌ Actual Behavior
Markdown preview works

Resolved with the release of v0.45.


docs(book): include PRQL changelog in book

Close #2344

One thing we want to avoid here is:

Someone adds an entry to the Changelog with a prql block; the code doesn't compile
This is only picked up when building the book, not in the book tests, since the book tests only look at the .md files in the book
Then build-web fails next time someone makes a small change to the web, and we have to unravel what caused it, and we're reverting big PR because of a small changelog problem.

We could run build-web on changes to the changelog; I think that would solve it

I added no-eval tag to skip evaluate the code block in 9e89bbbd3d323714e498c65b7bccf9db79b61bed.
This should work the same way as the elm on the book, so it can be used as a perfect replacement for the elm code block.
(Of course, we could continue to use elm.)

Perfect!
Should we also add a https://pre-commit.com/#pygrep so no one puts a prql$ tag in the Changelog?

Should we also add a pre-commit.com/#pygrep so no one puts a prql$ tag in the Changelog?

I added that in 8347d17ddedb7b08d4885f4e9636f355f199f586 and it seems working fine de5c5ff5c56710918f09b59444b29bb08769cd89

It would be great if we could automatically link to issue numbers and GitHub user accounts like pkgdown does (https://pkgdown.r-lib.org/news/index.html), but my skills are not currently up to the task of implementing this...
https://github.com/r-lib/pkgdown/blob/c354aa7e5ea1f9936692494c28c89e5bdd31fc68/R/repo.R#L39-L46

It would be great if we could automatically link to issue numbers and GitHub user accounts like pkgdown does (https://pkgdown.r-lib.org/news/index.html), but my skills are not currently up to the task of implementing this... https://github.com/r-lib/pkgdown/blob/c354aa7e5ea1f9936692494c28c89e5bdd31fc68/R/repo.R#L39-L46

Yes totally. A challenge for the future (...generations...)!

I hit the button to get this in for 0.7.0, hope that's OK!

In! Thanks @eitsupi !

Thanks!


Rename metric response to responseMsec

To maintain consistency with vts

looks good, could you fix the conflicts? thanks

Conflicts fixed, @hnlq715! :smiley:


⚠ AGEG website (redirect) has degraded performance

In ee23661, AGEG website (redirect) (https://ageg.ca/) experienced degraded performance:

HTTP code: 200
Response time: 2171 ms

Resolved: AGEG website (redirect) performance has improved in ec57341.


Use spec-compliant Ed25519 key derivation

Adapts the canister code to use dfinity/ic's ic-crypto-ed25519 utility crate because this crate implements the Ed25519 hierarchical key derivation according to the Ed25519 key derivation spec of the management canister API draft specification.
This change is mainly relevant for users who want to perform the Ed25519 key derivation outside of the canister, e.g., with the Rust ic-crypto-ed25519 utility crate, which follows the spec, or with their own implementation of the spec (e.g., in other languages).

A little late, but LGTM!


B 18522 i 12556 fix loadtest prime sim

I-12556

Warnings

:warning:
This PR does not include changes to unit tests, even though it affects app code.

  Generated by :no_entry_sign: dangerJS against 10b44c2d8c894a973b1dba6eb5f1705675105f2a


How to Use PTTL

How to return to the rest of the time, thank you.
redis> PEXPIRE key 10086
(integer) 1

redis> PTTL key
(integer) 6179

redis.pttl(key, (e, a) => console.log(a));
its ok


"Cannot read property 'dispose' of undefined" when exiting VC

Seen when exiting VC (to lobby):
componentDidMount @ bundle.js:120488
bundle.js:77201 Uncaught TypeError: Cannot read property 'dispose' of undefined
    at disposeMaterial (https://<IP_ADDRESS>:9090/bundle.js:77201:12)
    at NewComponent.remove (https://<IP_ADDRESS>:9090/bundle.js:77155:6)
    at HTMLElement.value (https://<IP_ADDRESS>:9090/bundle.js:81521:18)
    at bound (https://<IP_ADDRESS>:9090/bundle.js:86696:18)
    at Array.forEach (native)
    at HTMLElement.value (https://<IP_ADDRESS>:9090/bundle.js:81200:37)
    at removeChild (https://<IP_ADDRESS>:9090/bundle.js:106900:15)
    at Object.processUpdates (https://<IP_ADDRESS>:9090/bundle.js:107044:12)
    at Object.dangerouslyProcessChildrenUpdates [as processChildrenUpdates] (https://<IP_ADDRESS>:9090/bundle.js:107935:28)
    at processQueue (https://<IP_ADDRESS>:9090/bundle.js:111478:30)
bundle.js:77201 Uncaught TypeError: Cannot read property 'dispose' of undefined
    at disposeMaterial (https://<IP_ADDRESS>:9090/bundle.js:77201:12)
    at NewComponent.remove (https://<IP_ADDRESS>:9090/bundle.js:77155:6)
    at HTMLElement.value (https://<IP_ADDRESS>:9090/bundle.js:81521:18)
    at bound (https://<IP_ADDRESS>:9090/bundle.js:86696:18)
    at Array.forEach (native)
    at HTMLElement.value (https://<IP_ADDRESS>:9090/bundle.js:81200:37)
    at removeChild (https://<IP_ADDRESS>:9090/bundle.js:106900:15)
    at Object.processUpdates (https://<IP_ADDRESS>:9090/bundle.js:107044:12)
    at Object.dangerouslyProcessChildrenUpdates [as processChildrenUpdates] (https://<IP_ADDRESS>:9090/bundle.js:107935:28)
    at processQueue (https://<IP_ADDRESS>:9090/bundle.js:111478:30)

fixed by @rxl881 somewhere :)


Ec2RouteTable support

Description:
At the moment Cartography doesn't give detailed information about traditionally routed environments (ie. environments not using Transit Gateway) in AWS. The aim is to develop the required code to import this information as well.
I identified the following relations so far
(Ec2RouteTable) <- [ASSOCIATED] - (EC2Subnet)
(Ec2RouteTable) - [ROUTES] -> (EC2Route)
(AWSVpc) - [RESOURCE] -> (Ec2RouteTable)
(AWSAccount) - [RESOURCE] -> (Ec2RouteTable)
(EC2Route) - [CIDR] -> (AWSCidrBlock)
(EC2Route) - [PREFIXLIST] -> (AWSPrefixList)
(AWSPrefixList) - [CIDR] -> (AWSCidrBlock)
(EC2Route) - [GATEWAY] -> (InternetGateway|VPCPeering (!)|NATGateway) (Relation doesn't exist for local prefixes.)
There is already one issue that needs to be clarified: VPC_PEERING is already existing as relation between AWSCidrBlocks, this has to be refactored.
Relevant Links:
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeRouteTables.html

Appreciate you taking the time to put this together. I think follow-up PRs should include just one new relationship each to make them easier to review.

There is already one issue that needs to be clarified: VPC_PEERING is already existing as relation between AWSCidrBlocks, this has to be refactored.

The reasoning for this schema choice is that VPCs are peered through their CIDR blocks. The path looks something like
(i) VpcA-[:BLOCK_ASSOCIATION]-CidrBlockA-[:VPC_PEERING]-CidrBlockB-[:BLOCK_ASSOCIATION]-VpcB

In this case, an instance connected to BlockA can directly talk to BlockB, and we wanted to make that as clear as possible.
What is your proposed refactoring? I think you're proposing to add a new "shortcut" relationship equivalent to (i):
VpcA-[:NAME_TO_BE_DECIDED]-VpcB

Appreciate you taking the time to put this together. I think follow-up PRs should include just one new relationship each to make them easier to review.

No problem. Separating PRs is agreed.

There is already one issue that needs to be clarified: VPC_PEERING is already existing as relation between AWSCidrBlocks, this has to be refactored.

The reasoning for this schema choice is that VPCs are peered through their CIDR blocks. The path looks something like
(i) VpcA-[:BLOCK_ASSOCIATION]-CidrBlockA-[:VPC_PEERING]-CidrBlockB-[:BLOCK_ASSOCIATION]-VpcB

In this case, an instance connected to BlockA can directly talk to BlockB, and we wanted to make that as clear as possible.
What is your proposed refactoring? I think you're proposing to add a new "shortcut" relationship equivalent to (i):

My idea is to turn the relation/edge into a node, so that we can connect a VPC_PEERING with EC2_ROUTEs.
So the current scheme could turn into
VpcA-[:BLOCK_ASSOCIATION]-CidrBlockA-[:PEERING]-VPC_PEERING-[:PEERING]-CidrBlockB-[:BLOCK_ASSOCIATION]-VpcB
But the VPC_PEERING would appear in actual routes as well:
(EC2Subnet) - [ASSOCIATED] - (Ec2RouteTable) -  [ROUTES] - (EC2Route) - [GATEWAY] - (VPC_PEERING)-  [GATEWAY] - (EC2Route) - [ROUTES] - (Ec2RouteTable) - ASSOCIATED] - (EC2Subnet)
This way we could use the database to check routing between IP endpoints. (VPC peerings are tricky, in order to have connectivity, the peering needs to be there, routing needs to be there and the security groups as well.) On complex networks with multiple dozens of VPC peerings across multiple AWS accounts, it can be a tedious work to troubleshoot connectivity issues.
On the other side Neo4j is flexible enough to keep both ways of maintaining the relation between the CidrBlocks through a VPC peering. This can be a temporary as well, to get users used to the new path without breaking actual reports. (Adding a 'deprecated' property to the VPC_PEERING relation/edge, can be the preparation for the transition. -- but maybe who cares if both paths remain.

My idea is to turn the relation/edge into a node, so that we can connect a VPC_PEERING with EC2_ROUTEs.

Ah, this makes sense.

This way we could use the database to check routing between IP endpoints.

I like this a lot. I'm supportive of the change.
@marco-lancini @SecPrez you have any reports that would break from this?
In general our schema change process has been informal and lightweight, and I think it's fine that way so far. I think it's sufficient to socialize the major schema-breaking stuff on Slack and if no one complains, ship it.

I might have some queries (https://github.com/marco-lancini/cartography-queries/blob/master/queries/queries.json) impacted by this, but it won't be difficult to update them

FYI started to work on it in https://github.com/krisek/cartography/tree/%23410-Ec2RouteTable-support
As we discussed, the push will happen in stages, first the re-implementation of peering connections, and then the routing tables and then eventually other things like nat gateway, etc.

@achantavy @marco-lancini can we agree that we keep the cleanup jobs for the 'old style' VPC_PEERING relations? This way these would be deleted in favour for the new node based representation when first running the new sync logic. Alternative: keep the old VPC_PEERING relations and let the users delete them manually (Once they realize that these relations doesn't get updated anymore.)
I'm in favour for the first option.

BTW sneak preview... this data model really follows now AWS

The respective pull request is pending for quite a while... can somebody please review?

Reviewing now.
@sachafaust: you might find this interesting since you built the original VPC peering module. I'll cc you in the PR as well.

Hi @krisek, any updates regarding the support for Route Table?
Thanks

Hi @krisek, any updates regarding the support for Route Table?
Thanks

Hi @aashishgaba-cn, I had some other priorities in the last few weeks, so no much progress. Hopefully I can pick it up soon... first step is to get PeeringConnections rework merged. #487

Hi @aashishgaba-cn, I had some other priorities in the last few weeks, so no much progress. Hopefully I can pick it up soon... first step is to get PeeringConnections rework merged. #487

Hi @krisek. sounds good.
In case you haven't started working on the Route Table and if you don't mind, I can pick it up, and the support for the same. Later on, when you're done with PeeringConnections work, you may add it :)
A route can have these parameters
- DestinationCidrBlock [Y]
- DestinationIpv6CidrBlock [Y]
- DestinationPrefixListId [Y]
- EgressOnlyInternetGatewayId
- GatewayId
- InstanceId [Y]
- NatGatewayId
- TransitGatewayId [Y]
- NetworkInterfaceId [Y]
- VpcPeeringConnectionId
How about I add the relationships for the ones that the cartography currently supports(marked with [Y]) and later we can keep on adding for the missing ones?
What do you guys(you and @achantavy) think about this?
Pardon me if marked [Y] to anything that can't be added as of now, we can definitely discuss and select them.

Hi @krisek. sounds good.
In case you haven't started working on the Route Table and if you don't mind, I can pick it up, and the support for the same. Later on, when you're done with PeeringConnections work, you may add it :)
A route can have these parameters
- DestinationCidrBlock [Y]
- DestinationIpv6CidrBlock [Y]
- DestinationPrefixListId [Y]
- EgressOnlyInternetGatewayId
- GatewayId
- InstanceId [Y]
- NatGatewayId
- TransitGatewayId [Y]
- NetworkInterfaceId [Y]
- VpcPeeringConnectionId
How about I add the relationships for the ones that the cartography currently supports(marked with [Y]) and later we can keep on adding for the missing ones?
What do you guys(you and @achantavy) think about this?
Pardon me if marked [Y] to anything that can't be added as of now, we can definitely discuss and select them.

Sure, please go ahead, much appreciated... I have two open pull requests, I didn't want to increase the entropy.
I assume the resulting code will be the same anyway nevetheless who writes it :)

Sure, please go ahead, much appreciated... I have two open pull requests, I didn't want to increase the entropy.
I assume the resulting code will be the same anyway nevetheless who writes it :)

Started working on this.


Set credentials implementation

implement credential set command for DockerHub, GCR and basic auth
optionally set default image prefix when setting credentials

Add Stdin to the CommandTable to make content available to be read.
Refs #1254

I have a general concern about "riff credentials set" - it sounds like a noun and is a bit confusing. Also, it's not clear to me why image prefix has anything at all to do with credentials.

Please could you supply the missing help text as that will actually help me review the code and tests.

Which command/flag is vague? Perhaps we can find names that are more self descriptive.

I have a general concern about "riff credentials set" - it sounds like a noun and is a bit confusing.

Do you have another suggestion for a command that will create/update a credential? While we can, I tried to avoid creating separate insert and update commands since the update is nearly identical to a delete and create.
This command was originally defined in #1254

Also, it's not clear to me why image prefix has anything at all to do with credentials.

The credentials are for an image registry. For DockerHub and GCR we are able to infer the image prefix based solely on the credentials. It's a connivence for the user. The default image prefix was originally specified as part of namespace init being derived from the credentials.
I considered adding an explicit --default-image-prefix to allow a user to set a custom value, but that made more sense as part of a riff config command (which doesn't currently exist).

Do you have another suggestion for a command that will create/update a credential? While we can, I tried to avoid creating separate insert and update commands since the update is nearly identical to a delete and create.

If we have explicit "create" and "update" then the former would fail if a credential with that name already exists and the latter would fail if it doesn't yet exist. Maybe that's a good thing since, for example, "set" might inadvertently apply an update?

@glyn the last updated specs for the command were: https://github.com/projectriff/riff/issues/817#issuecomment-489078485.

If we have explicit "create" and "update" then the former would fail if a credential with that name already exists and the latter would fail if it doesn't yet exist. Maybe that's a good thing since, for example, "set" might inadvertently apply an update?

I tried splitting out set into independent create and update commands. It doesn't end up simplifying either command and now there are two commands where changes need to be kept in sync (I kept as much code in common as practical). If we really think the UX around create and update vs set is what we want, then we can do this, but I'd rather not.
Splitting create and update is useful when there are semantic differences between the two commands. For example, some options may only be available during create. Update is used to mutate an existing resource, preserving some core essence of its original self. This is not true for credentials. The same flags are available for both commands and they are applied the same way. The resulting secret data from the create command and the update command are identical, no matter the preexisting state.
Another way to think about this, imagine I took away kubectl apply and forced you to use kubectl create and kubectl update instead. There are times when k create and k updated carry meaningful semantics, but most of the time those difference don't matter and forcing the user to think about which to use is a cognitive burden, especially when the result is identical.

There are times when k create and k updated carry meaningful semantics, but most of the time those difference don't matter and forcing the user to think about which to use is a cognitive burden, especially when the result is identical.

wouldn't that same logic apply to the other commands like "function" and "application"?

Merging so we can use the command with riff-system. Happy to continue iterating in issues and PRs


chore(main): release 1.27.1

:robot: I have created a release beep boop
1.27.1 (2024-08-30)
Bug Fixes

use ginkgo label matching to match on storage filter (#487) (7f36e7d)
use unique test keys (#488) (f8fbd5e)

Miscellaneous

remove publisher id from simple topics example and use only one polling function at a time (#489) (d97e718)
set topics resubscribe delay to 500ms (#490) (8d70251)
update go topics examples (#483) (ff0332f)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/momentohq/client-sdk-go/releases/tag/v1.27.1 :sunflower:


feat: indent guides

I'm sorry for the delay. There wasn't enough time to finish.

Thanks for making those changes. While testing ti out I realized there is one major item we forgot about:

We add a config option for a special component called "indent_renderer" which follows the interface of a component but is at the global config level.

I think it needs to be changed to this method of configuration to make it easier to configure and prevent breaking changes. With this strategy, the indent_renderer would be at the top level of the config and be implicitly put at the beginning of all renderers.
If you are short on time I can make these changes before merging, just let me know.

I have added a global configuration implementation. Now the user can globally configure any component. If the user manually changed the component, the configuration will be overridden (local configuration is more significant than the global one).
For example, user can configure icons:
require("neo-tree").setup({
      icon_renderer = {
          folder_closed = "-",
      },
     -- other configurations
})

I really like the idea of having a global config and we should definitely do that.
What this still doesn't address is the fact that it is a breaking change to take an implicit component and turn it into an explicit one. Everyone that has applied any customization to their renderers will suddenly lose indents with this update,
To prevent that, we have to either:

Remove it from the individual renderers completely and just pull the settings from global config instead. This is what happens now.
In setup(), check for the existence of the "indent" component in each config, and inject it only if it doesn't already exist.

Number two fits better with what you have now. It can probably happen in the add_global_components_config function. I would say test for the existence of "indent" in the loop and inject it in the first position only if it wasn't seen. It is is possible someone may have it but not in the first position because they want to have something like diagnostics or git_status all the way on the left before the indent.

Hey @danilshvalov, I'm going to finish this up and merge it. The only outstanding issue was ensuring the component is always implicitly added even if the users have a custom renderer without an indent component.

Oops, didn't mean to close it.

Content moved to #131 so I could squash and rebase.


Update dependencies

Coverage remained the same at 97.986% when pulling ae550a7e1cde4b3e086750bb65460a5c834f3a94 on update-deps into 5c349c6eb8b07f9860f0127c9b1aa5e391380059 on v3.0.0.


Extend aad app set to add certificate to already available App

Create the option of adding the certificate to already available Azure AD App using the command aad app set.
Following would be the expected options for the command.

Option
Description

--certificateFile [certificateFile]
Path to the file with certificate private key. Specify either certificateFile or certificateBase64Encoded

--certificateBase64Encoded [certificateBase64Encoded]
Base64-encoded string with certificate private key. Specify either certificateFile or certificateBase64Encoded

Hi guys, I like this one, you can assign me here!

Thanks @martinlingstuyl for your help.
All yours!!

Hi @arjunumenon,
would it be an idea to add an extra option: --certificateDisplayName?
Using the displayName property on the Microsoft Graph AAD app update call is optional. If not filled in, the displayName will be set to the certificate subject.
In my case it became something like: CN=Test Certificate for CLI

Makes perfect sense @martinlingstuyl. Great suggestion! 👏

Wonderful suggestion @martinlingstuyl 👍. I have updated the spec for the same.
I have also updated the spec for #3115 which also deals with the similar enhancement

A small thing. The specs speak about a private key, but I assume you mean public key...

Hi guys, is it acceptable if I close #3115 and #3116 in a single PR?

Hi guys, is it acceptable if I close #3115 and #3116 in a single PR?

Yes, but please keep the commits separate so that it's clear which changes apply where.

By the way, I'm not replacing the certificate. I'm adding it to the list of available certificates. Seems to me the best route, as you would want to have some form of fallback to use the older certificates if they are referenced somewhere.

That's definitely the right approach: let's not replace whatever is in there but rather add another certificate

This can be closed now


More info

I'd be helpful to state which Minecraft versions this plugin works on. Also, build instructions would be appreciated too

https://modrinth.com/plugin/uwulogs says here

Ok, and what about the building instructs?

You build UwULogs the same as any Maven plugin. You can open a terminal, and assuming you have Maven installed, you can run mvn package to build the plugin. I'll edit the README to include some instructions though.

README has been updated.

Thanks.


Problem intercepting messages sent with StreamBridge

I have a global interceptor configured for all my output channels to log messages:
@Slf4j
@RequiredArgsConstructor
public class OutputChannelInterceptor implements ChannelInterceptor {
  private final BindingServiceProperties bindingServiceProperties;

  @Override
  public Message<?> preSend(Message<?> message, MessageChannel channel) {
    InterceptorProperties properties =
        new InterceptorProperties(message, channel, bindingServiceProperties);
    log.debug(
        "Outbound channel {} received message in topic {}",
        properties.getChannelName(),
        properties.getTopicName());
    log.trace("Full message is {}", message);
    return message
  }

  @Override
  public void postSend(Message<?> message, MessageChannel channel, boolean sent) {
    InterceptorProperties properties =
        new InterceptorProperties(message, channel, bindingServiceProperties);
    log.debug(
        "Outbound channel {} processed message in topic {} and message {} sent",
        properties.getChannelName(),
        properties.getTopicName(),
        sent ? "was" : "was not");
    log.trace("Full message is {}", message);
  }

  @Data
  static class InterceptorProperties {
    private final String channelName;
    private final String topicName;

    public InterceptorProperties(
        Message<?> message,
        MessageChannel channel,
        BindingServiceProperties bindingServiceProperties) {
      if (channel instanceof DirectWithAttributesChannel directChannel) {
        channelName = directChannel.getFullChannelName();
        topicName =
            directChannel.getComponentName() != null
                ? Optional.ofNullable(
                        bindingServiceProperties.getBindingProperties(
                            directChannel.getComponentName()))
                    .map(BindingProperties::getDestination)
                    .orElse("")
                : "";
      } else {
        channelName = "";
        topicName = "";
      }
    }
  }
}

I use bindingServiceProperties mostly to determine the kafka topic that the message is being sent to.
This works fine when i use message processing like
@Bean
public Function<String, String> interceptor() {
  return message -> "interceptorMessage";
}

However, when i need to send a message on demand (e.g., via a REST API call), I'm using the StreamBridge approach
@Slf4j
@Component
@RequiredArgsConstructor
static class StreamProcessor {
  @Autowired public StreamBridge streamBridge;

  public void testProduce(String key, String message) {
    streamBridge.send(key, "message");
  }
}

For this case, I can't seem to extract the channel name from the channel in the interceptor. I navigated in stream bridge code up to the point where the channel is created, and it does not seem to have a name at all when it is created this way. Is there any way that I can have at least the channel name in the interceptor, or is there an alternative way to produce the message without stream bridge?
Version of the framework 2023.0.0

@tigermarques, Could you put the code you provided above as a runnable sample application? This way, the triaging becomes easier.

Do you mean a repo with the issue in a reproducible format? Or just the classes?

I meant a standalone app in a repo.

Hi,
I managed to get the code in a minimal repo so that you can see the issue.
https://github.com/tigermarques/test-stream-bridge
There is one test which is failing because the interceptor cannot get the channel name when the message is sent via the StreamBridge.

@tigermarques Thanks for the sample; that helped. It turned out to be a bug. I pushed a change to address the issue. Please take a look at the commit above. However, you need a change in your test. I believe that the interceptor-test-application in lines from 28 - 35 needs to be updated to use streambridge-test-application from here.
We are having some issues with our builds right now. Because of this, a snapshot of the new changes is currently unavailable. In the meantime, you can check out the main branch locally, build the spring-cloud-stream module individually, and then use 4.1.1-SNAPSHOT in your pom.xml for spring-cloud-stream. to verify the changes. The fix will be available in the next point release (4.1.1). Thanks!

Thanks a lot! You're right, the test needs to be updated, I had copied it from a different app and forgot to change the application name in the test.
Do you know in which version of spring cloud dependencies this may be available?

I just backported to 4.0.x of Spring Cloud Stream, so this will be available via spring-cloud 2023.0.1 and 2022.0.5 when they are released.

@sobychacko Can i get some insights on this thread:
https://stackoverflow.com/questions/78694499/spring-cloud-stream-kafka-binder-consumer-interceptor
I am not sure if i can post this here, but just thought of checking if i can
get some thoughts on intercepting the messages.


Cleanup relnotes some more

@bot-gradle test and merge

OK, I've already triggered a build for you.