Segfault when signing certificate using CA with oid_section

Hi All,
I'm receiving a SIGSEGV on openssl 1.1.0.f (Arch Linux amd64).  The codepath in question does not seem to be affected by Arch-specific modifications.
System:
Arch Linux amd64
CPU: Intel Core i7-7700K @ 8x 4.8GHz
Kernel: x86_64 Linux 4.10.13-1-vfio
Packages:
openssl 1.1.0.f-1
glibc 2.25-4
GDB Output:
gdb ~/dev/opensource/abs/packages/openssl/trunk/pkg/openssl/usr/bin/openssl
GNU gdb (GDB) 8.0
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/randombk/dev/opensource/abs/packages/openssl/trunk/pkg/openssl/usr/bin/openssl...done.
(gdb) set environment LD_PRELOAD=/home/randombk/dev/opensource/abs/packages/glibc/trunk/pkg/glibc/usr/lib/libc.so.6 /home/randombk/dev/opensource/abs/packages/openssl/trunk/pkg/openssl/usr/lib/libcrypto.so
(gdb) run ca -config etc/dlp-trusted-device-ca.conf -in certs/rdlp2/rdlp2.csr -out certs/rdlp2/rdlp2.crt
Starting program: /storage/randombk/dev/opensource/abs/packages/openssl/trunk/pkg/openssl/usr/bin/openssl ca -config etc/dlp-trusted-device-ca.conf -in certs/rdlp2/rdlp2.csr -out certs/rdlp2/rdlp2.crt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Using configuration from etc/dlp-trusted-device-ca.conf

Program received signal SIGSEGV, Segmentation fault.
__strcmp_sse2_unaligned () at ../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:31
31		movdqu	(%rdi), %xmm1
(gdb) bt
#0  __strcmp_sse2_unaligned () at ../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:31
#1  0x00007ffff770c6e9 in OBJ_bsearch_ex_ (key=0x7fffffffd128, base_=0x7ffff77ad780 <ln_objs>, num=<optimized out>, size=4, cmp=0x7ffff770bf40 <ln_cmp_BSEARCH_CMP_FN>, flags=0) at crypto/objects/obj_dat.c:602
#2  0x00007ffff770cdb2 in OBJ_bsearch_ln (num=1052, base=0x7ffff77ad780 <ln_objs>, key=0x7fffffffd128) at crypto/objects/obj_dat.c:53
#3  OBJ_ln2nid (s=s@entry=0x0) at crypto/objects/obj_dat.c:553
#4  0x00007ffff770d066 in OBJ_create (oid=oid@entry=0x55555580c8aa "<IP_ADDRESS>.4.1.432<IP_ADDRESS>", sn=sn@entry=0x55555580c830 "DLPSECA", ln=ln@entry=0x0) at crypto/objects/obj_dat.c:688
#5  0x00007ffff762a7a5 in do_create (name=0x55555580c830 "DLPSECA", value=0x55555580c880 "DLP Networks Security Level A (DLPSEC-A), <IP_ADDRESS>.4.1.432<IP_ADDRESS>") at crypto/asn1/asn_moid.c:79
#6  oid_module_init (md=<optimized out>, cnf=<optimized out>) at crypto/asn1/asn_moid.c:37
#7  0x00007ffff7674312 in module_init (cnf=0x555555807f10, value=0x55555580c740 "dlp-network-oids", name=0x55555580c6f0 "oid_section", pmod=0x5555557f4350) at crypto/conf/conf_mod.c:307
#8  module_run (flags=0, value=<optimized out>, name=0x55555580c6f0 "oid_section", cnf=0x555555807f10) at crypto/conf/conf_mod.c:168
#9  CONF_modules_load (cnf=0x555555807f10, appname=appname@entry=0x0, flags=flags@entry=0) at crypto/conf/conf_mod.c:102
#10 0x00005555555824b4 in app_load_modules (config=<optimized out>, config@entry=0x555555807f10) at apps/apps.c:470
#11 0x00005555555872e6 in ca_main (argc=0, argv=0x7fffffffdde8) at apps/ca.c:435
#12 0x000055555558054d in do_cmd (prog=0x555555806a70, argc=7, argv=0x7fffffffddb0) at apps/openssl.c:468
#13 0x0000555555580a0e in do_cmd (argv=0x7fffffffddb0, argc=7, prog=<optimized out>) at apps/openssl.c:452
#14 main (argc=7, argv=0x7fffffffddb0) at apps/openssl.c:176
(gdb) i r rdi xmm1
rdi            0x0	0
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}

The function parameters look alright, so I am unable to tell if this is an error in openssl or glibc. The relevant code from glibc is as follows:
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  23) ENTRY ( __strcmp_sse2_unaligned)
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  24)      movl    %edi, %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  25)      xorl    %edx, %edx
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  26)      pxor    %xmm7, %xmm7
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  27)      orl     %esi, %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  28)      andl    $4095, %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  29)      cmpl    $4032, %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  30)      jg      L(cross_page)
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  31)      movdqu  (%rdi), %xmm1
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  32)      movdqu  (%rsi), %xmm0
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  33)      pcmpeqb %xmm1, %xmm0
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  34)      pminub  %xmm1, %xmm0
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  35)      pxor    %xmm1, %xmm1
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  36)      pcmpeqb %xmm1, %xmm0
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  37)      pmovmskb        %xmm0, %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  38)      testq   %rax, %rax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  39)      je      L(next_48_bytes)
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  40) L(return):
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  41)      bsfq    %rax, %rdx
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  42)      movzbl  (%rdi, %rdx), %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  43)      movzbl  (%rsi, %rdx), %edx
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  44)      subl    %edx, %eax
8f02859f17d01 (Ondřej Bílka 2013-09-03 16:21:38 +0200  45)      ret

Quick update:
I can reproduce the error on Debian 9 (Stretch), using:
OpenSSL 1.1.0f  25 May 2017
built on: reproducible build, date unspecified
platform: debian-amd64
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/x86_64-linux-gnu/engines-1.1\""
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"

The error dates back to at least OpenSSL 1.1.0d.
The error also occurs on the master branch, compiled using no-asm.  The stack trace is:
GNU gdb (GDB) 8.0
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /ram/openssl/apps/openssl...done.
(gdb) (gdb) Starting program: /ram/openssl/apps/openssl ca -config etc/dlp-trusted-device-ca.conf -in certs/rdlp2/rdlp2.csr -out certs/rdlp2/rdlp2.crt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Using configuration from etc/dlp-trusted-device-ca.conf

Program received signal SIGSEGV, Segmentation fault.
__strcmp_sse2_unaligned () at ../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:31
31		movdqu	(%rdi), %xmm1
(gdb) #0  __strcmp_sse2_unaligned () at ../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:31
#1  0x00007ffff76f21a7 in ln_cmp (a=0x7fffffffc208, b=0x7ffff77ae858 <ln_objs+2168>) at crypto/objects/obj_dat.c:50
#2  0x00007ffff76f21ed in ln_cmp_BSEARCH_CMP_FN (a_=0x7fffffffc208, b_=0x7ffff77ae858 <ln_objs+2168>) at crypto/objects/obj_dat.c:53
#3  0x00007ffff76f38ed in OBJ_bsearch_ex_ (key=0x7fffffffc208, base_=0x7ffff77adfe0 <ln_objs>, num=1085, size=4, cmp=0x7ffff76f21a9 <ln_cmp_BSEARCH_CMP_FN>, flags=0) at crypto/objects/obj_dat.c:602
#4  0x00007ffff76f3842 in OBJ_bsearch_ (key=0x7fffffffc208, base=0x7ffff77adfe0 <ln_objs>, num=1085, size=4, cmp=0x7ffff76f21a9 <ln_cmp_BSEARCH_CMP_FN>) at crypto/objects/obj_dat.c:583
#5  0x00007ffff76f2232 in OBJ_bsearch_ln (key=0x7fffffffc208, base=0x7ffff77adfe0 <ln_objs>, num=1085) at crypto/objects/obj_dat.c:53
#6  0x00007ffff76f36c1 in OBJ_ln2nid (s=0x0) at crypto/objects/obj_dat.c:553
#7  0x00007ffff76f3cd9 in OBJ_create (oid=0x555555830d9a "<IP_ADDRESS>.4.1.432<IP_ADDRESS>", sn=0x555555830d20 "DLPSECA", ln=0x0) at crypto/objects/obj_dat.c:687
#8  0x00007ffff76067c4 in do_create (value=0x555555830d70 "DLP Networks Security Level A (DLPSEC-A), <IP_ADDRESS>.4.1.432<IP_ADDRESS>", name=0x555555830d20 "DLPSECA") at crypto/asn1/asn_moid.c:79
#9  0x00007ffff7606680 in oid_module_init (md=0x555555821450, cnf=0x555555821430) at crypto/asn1/asn_moid.c:37
#10 0x00007ffff7673658 in module_init (pmod=0x555555816360, name=0x555555830be0 "oid_section", value=0x555555830c30 "dlp-network-oids", cnf=0x555555821430) at crypto/conf/conf_mod.c:307
#11 0x00007ffff767311b in module_run (cnf=0x555555821430, name=0x555555830be0 "oid_section", value=0x555555830c30 "dlp-network-oids", flags=0) at crypto/conf/conf_mod.c:168
#12 0x00007ffff7672ef1 in CONF_modules_load (cnf=0x555555821430, appname=0x0, flags=0) at crypto/conf/conf_mod.c:102
#13 0x0000555555581a37 in app_load_modules (config=0x555555821430) at apps/apps.c:495
#14 0x0000555555587e51 in ca_main (argc=0, argv=0x7fffffffde88) at apps/ca.c:435
#15 0x00005555555a14b1 in do_cmd (prog=0x55555581ff80, argc=7, argv=0x7fffffffde50) at apps/openssl.c:487
#16 0x00005555555a0a53 in main (argc=7, argv=0x7fffffffde50) at apps/openssl.c:166

This was compiled using the following gcc:
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/7.1.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --disable-multilib --disable-werror --enable-checking=release
Thread model: posix
gcc version 7.1.1 20170516 (GCC)

The error is also reproducible when compiled in 32-bit mode using setarch i386 ./config -m32.

From what you pasted, it seems this base_=0x7ffff77adfe0 <ln_objs>, num=1085, size=4 might be the reason of crash. Both num and size are incorrect for ln_objs to do a binary search. (1085 and 4 are for sn_objs).

Ignore previous "dumb" comment...

It seems the crash happened at the first round of binary search, is it related to OBJ_ln2nid (s=0x0) at crypto/objects/obj_dat.c:553 here?

It seems not all strcmp can accept NULL as input parameter. Passing ln = NULL to OBJ_create will lead to call to strcmp(NULL, "xxxxxx") and crash there... Seems __strcmp_sse2_unaligned implementation of strcmp doesn't tolerate to NULL pointers.
So I think this should be fixed, what's your idea @mattcaswell ?

I can reproduce this by modifying the default openssl.cnf file as follows:
diff --git a/apps/openssl.cnf b/apps/openssl.cnf
index b3e7444e5f..05789cc41a 100644
--- a/apps/openssl.cnf
+++ b/apps/openssl.cnf
@@ -8,6 +8,10 @@
 HOME                   = .
 RANDFILE               = $ENV::HOME/.rnd

+openssl_conf = openssl_conf_section
+
+[openssl_conf_section]
+
 # Extra OBJECT IDENTIFIER info:
 #oid_file              = $ENV::HOME/.oid
 oid_section            = new_oids
@@ -28,7 +32,7 @@ oid_section           = new_oids
 # testoid2=${testoid1}.5.6

 # Policies used by the TSA examples.
-tsa_policy1 = <IP_ADDRESS>.1
+tsa_policy1 = <IP_ADDRESS>.1,blah
 tsa_policy2 = <IP_ADDRESS>.5.6
 tsa_policy3 = <IP_ADDRESS>.5.7

And then:
openssl ca -config openssl.cnf

Yes, with a comma, ln will be NULL.

Passing in NULL as the “ln” parameter is wrong, the code that does this should be fixed.  If the program had not crashed, we would not have found this bug ☺

The bug is easy to fix. The problem is in do_create() in asn_moid.c (PR coming up). I am interested though that this bug has only been found now. That code has existed in that state since 2004. This suggests to me that the built in oid module is never normally used in practice. Looking at the apps code none of it seems to expect the oid module to be used...where OIDs need to be loaded it is done directly be the apps themselves rather than using the built in module for it.
In order to actually activate the built-in OID module I had to add the "openssl_conf" bit to the config file above. Does anyone ever actually use this in practice? Without it the "oid_section = new_oids" line never gets activated. Is this an error in the default config file?
Perhaps @snhenson can shed some light.

Without it the "oid_section = new_oids" line never gets activated. Is this an error in the default config file?

Ah, that line does get used when the apps (ts, ca, req) explicitly load OIDs. They just don't do it through the built-in OID module.

It turns out that there was a little more to it that I first thought. I had assumed that this bug had been there since forever, but it turns out that it was not a problem in 1.0.2 and below. The reason is that in 1.0.2 OBJ_create() allows a NULL sn or ln to be passed. Commit 528832e47 broke that for 1.1.0. This had the knock on effect of breaking the built-in OID module which relied on that capability.
Fix in #3753


Added offline version of test_decompress_compressed_header

Added test_decompress_compressed_header_offline()
Added stream_decompress() and decompress() in streams.py to mimic the online test behavior
Marked test_decompress_compressed_header() as live_test_only

hey @zhzhh97 I pushed a few commits to your PR to see if we can get this merged. :)

/azp run python - anomalydetector - ci

/azp run python - storage - ci

/azp run python - autorest - pr

/check-enforcer override


Display error on mobile device

When accessed via a mobile device, disable the app and show a friendly error message.

Code looks good to me. We should confirm with the team the exact copy to display here though.

Code looks good to me. We should confirm with the team the exact copy to display here though.

Thanks for the review! Yes, the copy is still pending, and will not merge this until is confirmed. I wanted to quickly get your review on the code.


How to remove anything after the first space

Hello,
I am trying to truncate one word from a two word sentence.
Although I am using the parameter word="1" it is not able to remove the first word.
Let's say the sentence is
Hello World
I only want to display Hello

Did you try with the "words" instead of "word"? I believe that may be your issue.


mapViewDidFinishRenderingFrameFullyRendered not called in MLNMapView for iOS since 6.x.x update

Description
When interacting with the map using gestures like panning or flinging, the mapViewDidFinishRenderingFrameFullyRendered should be called in MLNMapView in order to adapt to the updated view. The NavigationMapView of maplibre-navigation-ios inherits from MLNMapView and overrides the function to update the puck position for certain cases. Since the 6.0.0 update, the function is not called, which causes massive problems with the behavior of the puck movement during active navigation.
To Reproduce

Run any iOS application with a MLNMapView
Add a breakpoint to the mapViewDidFinishRenderingFrameFullyRendered
pan or fling the map

Expected behavior
mapViewDidFinishRenderingFrameFullyRendered is called with each update caused by the gesture.
Actual behavior
mapViewDidFinishRenderingFrameFullyRendered is not called.
Screenrecording
https://github.com/maplibre/maplibre-native/assets/895046/dd59b70e-9475-4ec4-8273-4be5b8e8b913
Platform information :

OS: iOS 16.7.4 onwards
Device:  Simulator and physical

Additional context
The related issue for maplibre-navigation-ios: https://github.com/maplibre/maplibre-navigation-ios/issues/27

I think this was fixed in https://github.com/maplibre/maplibre-navigation-ios/pull/53
The title might be misleading, because this was actually in the context of NavigationMapView in maplibre-navigation-ios which is a subclass of MLNMapView.
The method signature for MLNMapView.mapViewDidFinishRenderingFrameFullyRendered changed in
https://github.com/maplibre/maplibre-native/commit/76469a0c2227927f03a15fb95b3a20f34becc78c and then again in
https://github.com/maplibre/maplibre-native/commit/8ad5f9c1fd58c60b93d26c2819637fd8f2144fd1
The reason this occurred without anyone noticing, is that this is a private method. maplibre-navigation-ios was only able to call it because we've copied in part of the private header. When the method signature change, our copied header became stale, and our "overridden" method was no longer being called.
An immediate fix would be to copy in an updated version of the private header like I did in https://github.com/maplibre/maplibre-navigation-ios/pull/53
Longer term, I'm not really sure what the right solution is. I think this is a frequent point of tension trying to reconcile inheritance and delegation.
Some ideas:

We could make this method public and use the public header instead of the private one.
We could utilize mapView.delegate in the NavigationMapView implementation. This seems tricky though, since we also want downstream users to be able to implement their own MLNMapViewDelegate for the NavigationMapView.

Thanks for the clarification. Private methods do not have a stable API of course.
Feel free to create a new issue or design proposal for the needed API changes.


Dash is not one of top ten coins in the CMC list and it is an issue for attracting investors

This is a chicken and egg problem. Unfortunately the legitimacy of coins is somewhat perceived based on their position in the CMC list, though we know that claimed MCs may not necessarily have anything to do with the network size and usage. Currently the fact that Dash is not even in the top 30 and constantly going lower is a huge barrier in attracting new investors.

This topic came up in the Dash discord #mno here.

IMHO being in the Top Ten on MC should never be a priority of DCG or any other team. We should focus on being the first project to be used globally for common payments, which is the reason most of us were attracted to Bitcoin/Crypto in the first place. With time price will be a function of usability and not popularity.
It is a fact that raising awareness is difficult without the popularity, but it is also true that Dash Evolution has been in the works for more than 5 years and people get tired of waiting. We should all be patient. The strategy is ok.
"Timing, perseverance, and ten years of trying will eventually make you look like an overnight success."

I agree on what's important is usability and I think being one of top 10 coins increases adoption, which helps usability because then it is much better supported by merchants, exchanges and other major player in the market. This also brings more funds to the network. My concern is that if we lose too much time other projects, who are in a much better financial position, will copy Dash features or new projects replace it.


[#292] Update TF version and GH actions versions

Close #292

What happened 👀

Upgrade TF to 1.8.3 version
Upgrade GH actions to latest versions
Install TF and node through asdf instead of manual installation

Proof Of Work 📹
CI is green.

It is a cleaner way to install dependencies. I will borrow it to other projects I'm working on 🤩

Yep, I like it as well. It was introduced by @nvminhtue in his PR: https://github.com/nimblehq/infrastructure-templates/pull/262


Revert "SSL HTTPS webapp"

Reverts GDoesCode/COM619DevOpsGroupWork#10
Should have dealt with keys and added instructions for https before merging

Reverting merge


branch swapping does not always work

I created a tree including all 22 Vibrionales genomes, and spotted the following issues

searching for the order Vibrionales hits two orders that contain "vibrionales" in their name (Bdellovibrionales and Desulfovibionales), but not the order Vibrionales itself
swapping branches of the tree does modify the similarity matrix as expected, but the tree is not modified simultaneously; I have not observed this previously for other groups of organisms, but it seems to be consistent for the Vibrionales tree
lower half of the tree branch for the most similar pair of Vibrio cholerae strains is not drawn

Note that the strain Vibrio cholerae O395 has been sequences twice, so maybe that's causing the latter two issues (I would hope that internally the bioproject IDs are used, and not the names). If I remove one of the Vibrio cholerae O395 sequences from the list, however, the last two issues seem to be resolved.
Original issue by @pdawyndt on Sat Jun 14 2014 at 13:01.
Closed by @bmesuere on Sat Jun 14 2014 at 19:37.

Same problem with tree for the genus Mycobacterium, where M. smegmatis str. MC2 155 has been sequenced twice.
Original comment by @pdawyndt on Sat Jun 14 2014 at 13:45.

Same problem with tree for the genus Burkholderia, where B. multivorans ATCC 17616 has been sequenced twice.
Original comment by @pdawyndt on Sat Jun 14 2014 at 15:46.

First issue is due to case sensitiveness of the search. Created issue #377 for this.
Original comment by @bmesuere on Sat Jun 14 2014 at 15:49.


Cannot remap c-axis in vanilla sm64js on controller

Whenever I try to change the c-axis for my gamepad, in vanilla sm64js, it doesn't change in the game, even if I click save controls. Remapping all the other buttons works, just not the c-axis.

Sticks are bound to the left and right sticks and a way to change them doesn't make sense to me and won't be worked on

It's just that the controller I use (usb switch controller) uses axis 3 and 6 for the right stick. I haven't tested it in a while, so I don't know if it has already been fixed.


SPLITED PARSER FOR OKTA AND OKTAV2

Required items, please complete
Change(s):

created new two files for oktaV2 for both Asim AND Vim and added oktaV2 parser in it and removed oktaV2 parser in oktaSSO parser in Asim and Vim and added these newly created names in AsimAuthentication and imAuthentication

Reason for Change(s):

To Split OktaSSO and OktaV2 parsers

Version Updated:
-Yes
Testing Completed:

yes

Checked that the validations are passing and have addressed any issues that are present:

yes

Hi @Alekhya0824,
Please check for failure validations. Thanks

Hi @Alekhya0824,
Please check open comments from @vakohl and act on them. Thanks

Hi @Alekhya0824,
Please check open comments and also KQL validation failure. Thanks

Hi @Alekhya0824,
Please check open comments from @vakohl and also failure validations. Thanks

Hi @Alekhya0824,
Please update on above. Thanks

Hi @Alekhya0824,
Validations failing please check. Thanks

Hi @Alekhya0824,
Please confirm if you have addressed all open comments. Thanks

Hi @Alekhya0824,
Please respond on above comment. Thanks

Hi @Alekhya0824,
Waiting for your response. Thanks

Hi @Alekhya0824,
I am still seeing a few open comments, please respond on them. Thanks

Hi @Alekhya0824,
We wanted to check on the status of PR #10499. PR is pending for more than 30 days. Please let us know if you need any assistance to review this PR. Per our standard operating procedures if no response is received in the next 7 business days, we will close this PR. Thank you for your cooperation.

@microsoft-github-policy-service agree company="Microsoft"


Time wrong for Pythia community meeting

The meeting time for the Pythia community meeting is incorrectly listed as 1pm ET. It should be 12pm ET. Thanks!
Tagging @brian-rose for awareness

@jmunroe will try to get this fixed.

Thanks @maxrjones !


Add support for default values inside parameters

Converts function allTheOptionalFormsWithDefaults(quuz = 'Nope Nope Nope') {} into
allTheOptionalFormsWithDefaults(quuz: ?string = 'Nope Nope Nope') {}

Other than that, LGTM, thank you!

I want this to land soon, so I may take this PR over.

Hi sorry will have a lookt today/tomorrow ;)

LGTM - Thank you very much!


draft: Otel filter

Initial cut of a filter to allow the proxy to start injecting OpenTelemetry traces into kafka message headers.

This could be really interesting in a number of ways, but we need to be clear what the objective is.
1 Are we trying to "just" add spans so we can observe messages being produced into and consumed from a Kafka cluster (without the client itself supporting otel)? (i.e. observability of a cluster as a black box).
2 How much value is there in being able to observe what the individual filters themselves are doing (i.e. observability into the proxy itself)
3 How much value is there in being able to observe a Kafka cluster as a white box? (i.e. if we were proxying the interbroker requests we could add spans to the follower Fetch requests, that could potentially allow you observe things like increased latencies affecting individual messages due to leader change).
I think you're aiming at 1. with this PR, right?

This was a Day of learning branch, so wasn't really setup with a specific goal. That said 1 is what it tries to do mostly as a way of bootstrapping things.

Is my personal interest but I'm quite happy to leave this PR to 1 if and when I find the time to get back to it.

This is still something I want to progress but this PR is stale and rotting further. I need to come back and revisit it properly so  keeping it open is just noise.


Adding 0.5 to Affine Transformation for Images causes misalignment

The shift (0.5, 0.5) mentioned on line 740 being applied to the image but not the rest of the components causes alignment issues. Adding this shift to the rest of the components corrects these issues.
Without fix:

With fix:

Turns out it was because matplotlib has coordinates centered at -0.5, -0.5.


type of variable passed into json decode changes

I tried to use functions to create the right structures for json decoding, this constructor function has a func() interface{} signature. After calling json.Decode with a pointer pointing to a struct created by this function, the type of the variable will be map[string]interface{} and not the given struct anymore.
Here's a complete example to reproduce this:
https://play.golang.org/p/PQwC4tu71E

For asking questions about learning the language, see one of our forums: https://golang.org/wiki/Questions
We only use this issue tracker for tracking bugs. People in the forums can explain what is happening here.


Enhancement: Different level of cell comment

I have a suggestion. Like in emacs' org mode, if u write * it means root level, ** means *'s children namely the second level, **** means the third etc.  Can we have a feature like this:
#%% Outer loop
#%%% inner loop1
    do something

#%%% inner loop2

    #%%%% inner inner loop3
        do something

    do something

When you select the parent level, all its descendants level will be activated. this feature would be really helpful when developing.
Also, it would be nice if the comment in different level have different color like in emacs

It seems too complex for my taste, and not easy to make it work right.

@ccordoba12 Never mind. But would u plz tell me in which file u implement the cell comment function? May be I can do something fit my own taste:P

For reference, we had a discussion about a similar thing in #2384.

I am :-1: on this

Anyway not interested. I don't use spyder anymore.
On Sun, Dec 6, 2015 at 1:54 PM, Gonzalo Peña-Castellanos <
<EMAIL_ADDRESS>wrote:

I am [image: :-1:] on this
—
Reply to this email directly or view it on GitHub
https://github.com/spyder-ide/spyder/issues/2719#issuecomment-162265338.


daemon: allow tmpfs to trump over VOLUME(s)

Much like binds override image's defined VOLUME(s) - tmpfs should do the same:
docker run --rm -v /tmp:/run test true <--- this pass
docker run --rm --tmpfs /run test true <--- this should pass as well

Right now, the second case fails with "duplicate mounts"
ping @cpuguy83
Signed-off-by: Antonio Murdaca<EMAIL_ADDRESS>

Makes sense to support this not sure about the implementation, but haven't looked much at how tmpfs is implemented.

Design makes sense to me, will move to code review.

LGTM

@cpuguy83 fixed the test

ping @cpuguy83 is this good to go now?

LGTM

@cpuguy83 updated not to clutter volume.MountPoint

integration tests hang for some reason not related to this PR

@cpuguy83 added check for duplicate mounts

LGTM assuming janky is happy!
Thanks @runcom, sorry for the nitpick.

@cpuguy83 no worry - understood all the nitpick points, I moved this to for-merge since we have 3 LGTM and just tests running left

janky and experimental passed (TP5 is probably burnt):
15:11:32 OK: 1327 passed, 33 skipped
15:11:32 PASS

15:14:32 OK: 1338 passed, 47 skipped
15:14:32 PASS


Necessary polyfills?

Could you tell me which polyfills this component depends on?
I'm using it in a environment, where I can't afford including the complete Babel polyfill. I'd like to include only the polyfills that are necessary.
When I use require('core-js/fn/symbol');, I get the following error in IE11:
Object doesn't support property or method 'Symbol(Symbol.iterator)_6.dcz7vm7xaw5'

I also tried another Symbol polyfill with require('es6-symbol/implement');. IE11 says:
Object expected

Are there other polyfill dependencies apart from Symbol?
I'd need to get the component working in IE10 and IE11. I'm using version 0.3.2 of the component, as I still have to use React 0.13.3. I've also uploaded a repo demonstrating the issue here.

I had a quick look, it seems like you need require these modules to run :)
require('core-js/es6/object');
require('core-js/es6/map');

Works great, thanks!
On Tue, Dec 8, 2015 at 9:56 PM David Chin<EMAIL_ADDRESS>wrote:

I had a quick look, it seems like you need require these modules to run :)
require('core-js/es6/object');require('core-js/es6/map');
—
Reply to this email directly or view it on GitHub
https://github.com/davidchin/react-input-range/issues/8#issuecomment-163015385
.

add this to the list
import 'core-js/es6/weak-map';

It is documented somewhere what the prerequisites are for each es6 module?

If it's still not working for you guys, try changing for (let foo in bar) to for (let foo in Array.from(bar)) (make sure you polyfill Array.from)


Starting a normal draft can start a burn draft

Start a glimpse/burn draft
Stop it
Start a normal draft
Another glimpse/burn draft is started because the burn cards option (set when starting a glimpse/burn draft) is preserved

Probably a side effect of how things work and the "shortcut" implemented in the Other Game Modes menu.

Probably a side effect of how things work and the "shortcut" implemented in the Other game modes menu.

That's exactly it. I knew about the limitation but am kind of hoping that no one will hit this case in normal use :D

As you are aware of it... and it should only happen in rare occasions... :)
Feel free to close this as a no fix for now to keep the tracker more clear.

Hmm, as I realized one can not start the more special game modes like "Rochester" or "Grid" draft with bots.
In comparison, Glimpse/Burn draft is allowed to start with bots.
However, bots only pick cards and do not burn cards... so it's basically broken with bots and makes no sense.
I'm posting this here as Glimpse/Burn might need its own treatment in order to avoid these problems.
As glimpse/burn draft is handled as regular draft, they also offer to replace dropped players with bots which makes no sense if they are not supported. Rochester and Grid do not offer that for example.

What about this... if a draft is done, the "number of cards to burn from each booster" setting is reset to defaults. That way the Draft button has it's normal expected behavior again and the additional Glimpse/Burn button still sets the desired custom value on start.


Is there any tip on super multiclass classifying?

Could you provide some suggestions on the classification of super multiple categories (such as 3000 categories)?
An attempt was made to use a 3000-word dictionary, but the model did not converge after 300000 iterations. [cry]

what is your config for train?


No way to change/specify the Server Headers

The base server headers list:

Status_Code
Content Length
Content Type

But I would like to be able to provide:

Date
Server Type
Last Modified
Accepted Ranges

I mainly want to disguise Japronto as another web-service...

Hmmmm....
https://github.com/squeaky-pl/japronto/blob/master/tutorial/5_response.md
  # And of course you can provide custom `headers`.
  def headers(request):
      return request.Response(
          text='headers',
          headers={'X-Header': 'Value',
                   'Refresh': '5; url=https://xkcd.com/353/'})

@aphillipo  Wow. Guess I overlooked that in the examples. I started searching through source code to try and find this. Thanks.


Vickel: Watch bigmoney.vip

Vickel requests the watch of the watch_keyword bigmoney\.vip. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

bigmoney\.vip has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Single GPU

Hello, I want to use single gpu to train the net. But I find that if I choose to use gpu-ids=1 or 2, the following error will appear。

Fortunately， if I choose gpu-ids=0 the code will work successful.
But why? Can I use single gpu with other gpu-ids not 0?

Hi,
You can set CUDA_VISIBLE_DEVICES=x, x is the gpu id you want to use. Also you should set --gpu-ids 0 when using single gpu.

@jfzhang95  Thank you so much!
What's more, if I want to use two gpus can I choose any two different ids such as 1,2 or 1,3?
At that time how should I set the --gpu-ids?

If you use gpu 2 and 3, you should run:
CUDA_VISIBLE_DEVICES=2,3 python train.py --backbone resnet --lr 0.007 --workers 4 --epochs 50 --batch-size 8 --gpu-ids 0,1 --dataset pascal

hello!
I have two GPUs on my device. I want to use them separately. When I use GPU1, I find that the memory is not occupied, and the utilization rate is very low. Are they all running on GPU0?
If i want use GPU1 separately,How do I set it ?


[Bug] Bar chart breaks when y-axis is time and inverse is true

Version
5.5.0
Link to Minimal Reproduction
https://echarts.apache.org/examples/en/editor.html?c=area-time-axis&version=5.5.0&code=PYBwLglsB2AEC8sDeAoWsAeBBDEDOAXMmurAERgCeIApmUWQG4CGANgK50A0JAvj-ko58RVKXJVa9CRAC23EujIRojGgCc8dImHWdYAegOwIAM1im2WribCwA7hFatYAE3whWzSrADGwdXUaXzBWSj4BWC11CBpCWABtRWJxcWVXaTIAYS88PAhfWCzmTTJI1PJoZnlMnOY8gqKSvDLktNdmMGZpJIqK3r7BgGYANhHywdSARgAGABYpoYB2AE4AJhnNzba-gF0JwYHJ1Lm5tYPj2FmltZWpgFYNrZmdiv3X1KPL9AeVi-OprM5vdliNni9vrsPu9BhRqNpyAAjEqtWF4Lq-ADWmWAiJaE34yTEfXSmQAaiougBzGhNUr_SrVBFkCnQam04r0j5kDpdHofcRfb5rNZTBmDIGLVZPLYC0gwy5Cy5rAAcc3FfWutweMu2kI16CVAJm5zlFUB8xBSzBzzNsChgwVFThUgYyPUqJJ6OYWJxeM96EJ4mJzogGQYAFlgGAAr5KL5WHEA6kyFUapHo7H44n8dzed0iEbPnbxLM_iXSJLlutwXanZMixKRuqKz8Zjc7o9a_q7Y2-vcVQbpkCrTbZZDoQyXcz3cm0t7fQxcbnxISobwANxAA
Steps to Reproduce
Check the provided link. It's using a bar chart with a y-axis of type time and inverse equals true.
Current Behavior
The bars are overflowing and the y-axis values are not rendering correctly. If you set inverse to false, everything works as expected.
Expected Behavior
The chart is displayed correctly, independently of the inverse value.
Environment
- OS: macOS Sonoma 14.5
- Browser: Chrome 126.0.6478.115

Any additional comments?
No response

@villebro


Update manage script to automate set-up

The manage script has updated to automate the steps to start-up vc-authn in various modes:

the user will be prompted to select whether they want to run in single-tenant or multi-tenant/traction mode
the user will be prompted to choose whether to use ngrok to expose the services or not

Both curl and jq are being executed in a docker container, removing the dependency for separate binaries on the path.
Ngrok containers have been moved to a separate compose project file: they need to be started before all of the other services in order to retrieve the public URLs and use set environment variables using their values.
I have not been able to test whether this works with traction or not as I cannot start the project in its current state - @Jsyro please let me know if you can run the code and it works.
Resolves #210

I am still trying to upgrade the ngrok containers to use the official, more regent image. Other than that the changes should be good to go.

It looks like there is an issue with the newer ngrok images, I will wait until that is resolved before I attempt to upgrade again the configuration. See https://github.com/ngrok/docker-ngrok/issues/25

Added some tweaks that should make quality of life better when restarting the project while keeping the same settings, and refreshing ngrok endpoints if necessary. If this gets us close enough to where we want to be I'd lean towards merging/closing and logging more specific issues about what needs tweaking as we find out.


POD Crashes when deploying cosmos DB binding

I have a producer deployment (yaml is attached) which is posting data to Azure service bus queue binding in DAPR. This producer works fine with Azure Service BUS binding. I created a new Cosmos DB Binding (attached is the binding Yaml). As soon as the binding is deployed in default namespace the producer pod crashes. I increased the request and limit memory but still the issue continues. If I delete the binding component for Azure Cosmos DB , POD starts running. I am able to shell into the POD and post data at 3500 port for Azure Service bus . I created a state store for COSMOS DB and it works. I am able to save the data in COSMOS DB using statestore. But for bindings it is failing. I have the POD logs and DAPR container logs attached with yaml file. Appreciate if you can provide some input on why bindings in cosmos DB is causing the POD to crash,
daprdeploy.zip

Given the errors in your pod logs:
time="2021-08-17T14:55:32.504974844Z" level=debug msg="loading component. name: ukbcosmos, type: bindings.azure.cosmosdb/v1" app_id=producer instance=producer-66f7578fd-qz6r6 scope=dapr.runtime type=log ver=1.3.0
time="2021-08-17T14:55:32.615879781Z" level=error msg="failed to init output binding ukbcosmos (bindings.azure.cosmosdb/v1): , " app_id=producer instance=producer-66f7578fd-qz6r6 scope=dapr.runtime type=log ver=1.3.0
time="2021-08-17T14:55:32.616122885Z" level=error msg="failed to init output bindings: , " app_id=producer instance=producer-66f7578fd-qz6r6 scope=dapr.runtime type=log ver=1.3.0
time="2021-08-17T14:55:32.616299288Z" level=warning msg="process component error daprd process will exited, gracefully to stop" app_id=producer instance=producer-66f7578fd-qz6r6 scope=dapr.runtime type=log ver=1.3.0

This suggests an issue in the configuration of your daprcosmosbindings, which are blanked out in the logs you've provided. I'd test that your cosmosdb configuration works independently first (e.g. maybe by replacing the kafka in the bindings quickstart and verifying that the producer sample can start locally).

Yes that would ne helpful. If you need further details , pls ping me.

@ubhanja I think you misunderstood me: I've already tried this with using cosmosdb binding in a minimal environment like the quickstart, and the cosmosdb component inits successfully. I'm suggesting that you test your cosmosdb configuration similarly in an isolated setting to help debug your failure, because the failure is very early in initializing the output binding, which usually indicates an incorrect component configuration.

@CodeMonkeyLeet  can you please share your sample cosmosdb binding

@CodeMonkeyLeet  I think I got the issue. CosmosDB with DocumentDB is working fine. When I use Mongo client the binding is crashing the POD. i.e if it is  https://.documents.azure.com:443/ it works fine. but for  https://.mongo.azure.com:443/ it fails . Can you please confirm that COSMOS DB Binding only works for Cosmos DB Document DB

@CodeMonkeyLeet I think I got the issue. CosmosDB with DocumentDB is working fine. When I use Mongo client the binding is crashing the POD. i.e if it is https://.documents.azure.com:443/ it works fine. but for https://.mongo.azure.com:443/ it fails . Can you please confirm that COSMOS DB Binding only works for Cosmos Document DB

Yes, I can confirm that.
You can try using the Dapr MongoDB state store component.

/assign


desc all failed on es external table

Steps to reproduce the behavior (Required)
CREATE external TABLE partsupp (
              ps_partkey int,
              ps_suppkey int,
              ps_availqty int,
              ps_supplycost double,
              ps_comment varchar(199)
              ) ENGINE=ELASTICSEARCH
              properties (
              "hosts" = "%s",
              "user" = "root",
              "password" = "root",
              "index" = "partsupp_10g",
              "type" = "_doc");
desc all partsupp;

Expected behavior (Required)
query ok
Real behavior (Required)
ERROR 1064 (HY000): Unknown storage engine 'ELASTICSEARCH'
StarRocks version (Required)

You can get the StarRocks version by executing SQL select current_version()

mysql> select current_version();
+----------------------------+
| current_version()          |
+----------------------------+
| BRANCH-2.3-RELEASE 7ef2250 |
+----------------------------+
1 row in set (0.07 sec)

I will process this later


Fix Dump tools clamping

https://forum.babylonjs.com/t/maybe-bug-about-dumptools-ts/40685/2

Snapshot stored with reference name:
refs/pull/13850/merge
Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/13850/merge/index.html
To test a playground add it to the URL, for example:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/13850/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/13850/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/13850/merge
https://gui.babylonjs.com/?snapshot=refs/pull/13850/merge
https://nme.babylonjs.com/?snapshot=refs/pull/13850/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/13850/merge#BCU1XR#0


updating from from 2.13 to 2.15: many problems with older zugferd documents

We're using the lib for validation and pdf rendering.  All docs working with 2.13. But many of them now have problems with 2.15.
Some examples of errors

Failed to parse PDF: java.lang.NullPointerException: Cannot invoke "java.math.BigDecimal.multiply(java.math.BigDecimal)" because the return value of "org.mustangproject.ZUGFeRD.IZUGFeRDExportableItem.getQuantity()" is null (mainly zugferd 1)
Cannot extract invoiceorg.mustangproject.Exceptions.ArithmetricException: Could not reproduce the invoice, this could mean that it could not be read properly (urn:xoev-de:kosit:standard:xrechnung_2.0)

Is this a known restriction? can't we use 2.15 for the older ones?

we are updateing from 2.14.2 an facting the same issue.
Looks very similar to #569.

Cannot extract invoiceorg.mustangproject.Exceptions.ArithmetricException: Could not reproduce the invoice, this could mean that it could not be read properly (urn:xoev-de:kosit:standard:xrechnung_2.0)

happens with some default files from ZUGFeRD directly:

EXTENDED_Fremdwaehrung.pdf
EXTENDED_Kostenrechnung.pdf
EXTENDED_Warenrechnung.pdf

the other files from 2.3 are fine, and do not raise any exceptions
can be downloaded here: ZugFerd 2.3 Download on ferd-net.de
Stracktrace (removed parts that are not related to the zugferd libary:
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.setDocument(ZUGFeRDInvoiceImporter.java:271)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.setRawXML(ZUGFeRDInvoiceImporter.java:240)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.setRawXML(ZUGFeRDInvoiceImporter.java:254)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.extractFiles(ZUGFeRDInvoiceImporter.java:215)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.extractLowLevel(ZUGFeRDInvoiceImporter.java:150)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.setInputStream(ZUGFeRDInvoiceImporter.java:95)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.<init>(ZUGFeRDInvoiceImporter.java:81)
org.mustangproject.ZUGFeRD.ZUGFeRDImporter.<init>(ZUGFeRDImporter.java:43)
org.mustangproject.validator.PDFValidator.validate(PDFValidator.java:125)
org.mustangproject.validator.ZUGFeRDValidator.internalValidate(ZUGFeRDValidator.java:126)
org.mustangproject.validator.ZUGFeRDValidator.validate(ZUGFeRDValidator.java:254)

Caused by: org.mustangproject.Exceptions.ArithmetricException: Could not reproduce the invoice, this could mean that it could not be read properly
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.extractInto(ZUGFeRDInvoiceImporter.java:745)
org.mustangproject.ZUGFeRD.ZUGFeRDInvoiceImporter.setDocument(ZUGFeRDInvoiceImporter.java:267)
... 45 more


empty: add simplify Zero and Empty compared to []bytes converted into string

https://github.com/search?q=language%3Ago+"assert.Empty(t%2C+string("&type=code
https://github.com/search?q=language%3Ago+"assert.Zero(t%2C+string("&type=code
Here are some examples
https://github.com/jroehl/go-suitesync/blob/01764321396a29047cc96ba2c0680c5291993c8c/lib/helper_test.go#L195
        assert.Zero(t, string(res))

https://github.com/testmeifyoucan/schreder/blob/fd29194cd8aacd59e99e2ee91fe36305a7e06fa1/testrunner.go#L203
        return assert.Empty(t, string(responseBody), "expected empty response")

They should be replaced by calling empty

Duplicated by #119


get_deletable_objects needs to check relationships of children of Tree objects

Proposed Changes
nautobot.utilities.testing.utils.get_deletable_objects (in 1.5+) filters a queryset on protected fields fieldname__isnull=True but also needs to check any children of a tree object. If a child's fieldname is not null, you cannot delete the parent.

Justification
Reuse testing code for testing object deletion

I ran into an issue where a factory-created object might have one relationship but not another, so get_deletable_objects always returns an empty queryset. Certainly this is a problem for the creation of the random objects, but having these considerations on the retrieval side should help as well.

I think it would be best to limit deletable objects to leaf nodes. If we delete nodes with children the tests will all need to be updated to handle tree objects. If you delete a node with children you may be deleting more objects than you expect to be deleted. See DeleteObjectViewTestCase for an example.

Proposed solution:
        if getattr(field, "on_delete", None) is PROTECT:

I like it except for this line since is tests identity. Better to do an equality test here (==). Reasoning is that since models.PROTECT is actually a function, there could be a scenario where the hashing may not line up for is to always be deterministic.

On some deliberation I think is works here! :)

The above suggestion breaks on InventoryItem because the children reverse relationship name is child_items instead of children.

@HanlinMiao try this
from tree_queries.models import TreeNodeForeignKey
from mptt.models import TreeForeignKey

def get_deletable_objects(model, queryset):
    """
    Returns a queryset of objects in the supplied queryset that have no protected relationships that would prevent deletion.
    """
    q = Q()
    for field in model._meta.get_fields(include_parents=True):
        if getattr(field, "on_delete", None) is PROTECT:
            q &= Q(**{f"{field.name}__isnull": True})
        # Only delete leaf nodes of trees to reduce complexity
        if isinstance(field, (TreeForeignKey, TreeNodeForeignKey)):
            q &= Q(**{f"{field.related_query_name()}__isnull": True})
    return queryset.filter(q)


Abstract out S3 (& SQS)

We should not be calling directly into S3 (or SQS). Abstractions should be put into place that can be shared with other Conveyal libraries (if needed) for accessing files.
There was previously a desire to cache files locally on the machine instead of repeatedly fetching them from S3, but if we're running on EC2 instances then I believe that bandwidth is free and the download speeds are usually ridiculously fast. So not sure if the local cache is needed when accessing files from S3.
We've also discussed potentially just eliminating our usage of SQS, in that case an abstraction would not be needed.

We have already eliminated usage of SQS, but the issue of abstracting out storage is still current (see PR #162).
I believe on EC2 now the "local" storage we're using is actually network attached (EBS) anyway, so perhaps the same speed as reading from S3.
I was going to say on R5 we're still going to need the multi-layered cacheing system, but on second thought I'm not entirely sure that's the case. The thinking was based on the assumption "the local filesystem" (assumed to be an attached disk) is much faster and cheaper than "network storage", assumptions which may not hold on EC2. S3 is cheaper and possibly fast enough.
We should audit it to see if there are places in R5 we want to persist something to S3 but need repeated access. But if S3 is fast enough, even in cases of heavy repeated usage we want to hit S3 every time and operate on the incoming data as a stream.
Once created, I believe all our files become immutable. We usually just send them to the client and don't need random access. S3 should be fine for this.
https://stackoverflow.com/a/47249072/778449


Improper display of Google ads when site is whitelisted

Description
Whitelisting a website displaying Google AdSense ads results in said ads being displayed improperly. They only display an external frame and a "report this ad" icon, but the ad contents are gone.
Steps to reproduce

Whitelist any website containing Google AdSense ads
Ads will be displayed improperly

Submission & filtering log details
See ticket 2273678 for screenshots & filtering log details.

This is an old issue with nested iframes -- AdGuard cannot detect that the request from the whitelisted website


fix: slots not working in template components

Changes
Fixes an issue with slots not working in template components (i.e. components that have a <template> tag at their root).
Fixes #31.
Notes
It's probable that this isn't an adequate fix. I don't know exactly when it's fine to skip out of raw mode after processing a template and I don't know exactly whether it's fine to avoid escaping template content if it's the child of a document fragment. Both changes were necessary to get my fix working while not breaking any existing tests.
Also, this PR assumes #31 to actually be a bug. If it isn't, this PR's content can safely be ignored.

test.macro!!!!!! what is this magic

Can you have a look? I pushed some changes to your PR per my comments in #31 and I’d love it if you could have a look at see if that fixes the issues you noted!

@zachleat Yup, that's pretty much the same as what I did locally as well. I've added another commit based on that which also reverts an unnecessary change to how escaped content is processed (no longer needed).

Merging, thank you!


Status Effects entry for Exhaustion doesn't update reference to phb when enabled

While CONFIG.DND5E.conditionTypes.exhaustion.reference is set accordingly, the entry for exhaustion under CONFIG.statusEffects isn't. This means that the effect description for exhaustion is still always referencing the SRD rules entry, which doesn't present an issue for the system normally as far as I can tell, but if using Visual Active Effects or DFreds Effects Panel (or some other module that exposes the effect itself), it'll have a description that doesn't necessarily match with the condition tooltip.

In retrospect this should be an issue on the PHB module, not the system, if someone would be kind enough to move it.

Glad you figured out. I was just about to mention that it looks fine to me when I don't have the PHB loaded the references point to the same thing.


app.allocations.fetchAvailable(1, true); gives me "404: The requested resource could not be found on the server." - Uncaught PteroAPIError Error

ERROR:
(node:20352) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
index.js:279
Uncaught PteroAPIError Error:
- 404: The requested resource could not be found on the server.
    at handleError (file:///C:/Users/Admin/node_modules/@devnote-dev/pterojs/dist/index.js:1568:11)
    at <anonymous> (file:///C:/Users/Admin/node_modules/@devnote-dev/pterojs/dist/index.js:1517:62)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
Process exited with code 1

CODE (pretty much the code in the example):
const { PteroApp } = require('@devnote-dev/pterojs');

const app = new PteroApp(
    'https://mc.abysl.dev/', // <-- this is the panel, the node is at http://mc.abysl.dev:8080/
    'REDACTED'
);

(async () => {
    const alloc = await app.allocations.fetchAvailable(1, true);
    const server = await app.servers.create({
        name: 'Welcome',
        user: 1,
        egg: 5,
        dockerImage: 'ghcr.io/pterodactyl/yolks:java_18',
        startup:
            'java -Xms128M -XX:MaxRAMPercentage=95.0 -Dterminal.jline=false -Dterminal.ansi=true -jar {{SERVER_JARFILE}}',
        environment: {
            MINECRAFT_VERSION: 'latest',
            SERVER_JARFILE: 'server.jar',
            DL_PATH: '',
            BUILD_NUMBER: 'latest'
        },
        limits: {
            cpu: 1000,
            memory: 8192,
            swap: 0,
            disk: 4096,
            io: 200
        },
        featureLimits: {
            allocations: 0,
            backups: 1,
            databases: 0,
        },
        allocation: {
            default: alloc.id,
        },
    });

    console.log(server);
})();

Anything obvious I'm missing? I'm not sure why it's not working, is there something I need to install on the server? When I try the wing/node, it says unauthorized, with use and application keys. When I try the panel, it says not found, for user and application keys.
EDIT: If I type a port directly instead of using allocation, I do get "422: The selected allocation.default is invalid.". If I understand correctly (although I am a little sleep deprived), this means it's a problem with getting the ports. Since the server can respond with that, but when I try to get an allocation, it says 404 - another edit: confirmed, this. I can't get the allocations. It returns 404. Is there another way to choose one maybe? Works perfecty if I use an id from 3000 to 3999 and add 1 to it

The warning NODE_TLS_REJECT_UNAUTHORIZED is unrelated to the project but I'd advise fixing it as it might affect requests.

const alloc = await app.allocations.fetchAvailable(1, true);

As per the docs for the fetchAvailable method, the first argument is the ID of the target node, so the 404 means a node with that ID does not exist.


Bitrise builds failing with suggested setup from documentation

I was following suggested documentation with bitrise setup:

maestro-cloud-upload@1:
inputs:

app_file: "$BITRISE_APP_PATH"
api_key: "$MDEV_API_KEY"

But I get this error for the step:

The only thing I found mismatching is that these paths are not the same:
export PATH="$PATH":"$HOME/.maestro/bin"

Command is searching it in /Users/vagrant/git/.maestro rather than where it is exported /Users/vagrant/.maestro/bin


github的主题与demo显示的不一样？

git clone this theme, but it's different from the demo?
How can I get the theme as the demo?
Thanks.

Maybe I didn't set tag so much and didn't fill in much info as your demo...
I will try a again.


Standardize formatting of manifests/x/xushengfeng/eSearch/1.9.6

Microsoft Reviewers: Open in CodeFlow

@wingetbot run


Reusing Variables from Host Rule Regex

Perhaps this is already possible, but I didn't found it documented.
Assume a Host Rule like this (docker-compose style):
    labels:
      - "traefik.frontend.rule=HostRegexp:{portal:.+}.example.org"

Now I want to forward this regex'd variable to the app via HTTP Header like this:
      - "traefik.frontend.headers.customRequestHeaders=portal:$portal"

Is that anyhow possible?
Edit: Clarification: The $portal variable obviously does not work and was just an example to point out what I want to do.

Hi! I'm Træfiker :robot: the bot in charge of communication regulation.
Thanks for your interest in Traefik!
We dedicate the issue tracker to bug reports and feature requests only. My advanced AI tells me this one is neither of them.
I encourage your to seek community support: join our Community Forum and reach out to us on the Traefik section.
You can of course double check Traefik's documentation :sweat_smile:

Hi @port22 , you might want to use the header X-Forwarded-Header to provide this information to your backend information, what do you think?

Hi @dduportal ,
maybe you got me wrong - I want to extract a part of the domain and re-use it at another place.
If it's the X-Forwarded-Header or a custom HTTP Header I don't actually care, but how's it gonna be written?
In nginx you do this:
server_name ~^(?<portal>.+?)\.example\.intra$;
proxy_set_header PORTAL "$portal";

portal is just an example to reflect any arbitrary part of the domain.


DS UI filtering on Page Type for Navigation buggy

I've noticed that when filtering on a Page Type, it only works for root subjects, e.g.
Root page /foo has Page Type "Foo". Its child page /foo/bar has an empty or other Page Type.  When using "Foo" as the filter, the child page will be included in the XML no matter its Page Type.
<navigation>
    <page handle="foo" id="1">
        <name>Foo</name>
        <types>
            <type>Foo</type>
        </types>
        <page handle="bar" id="2">
            <name>Bar</name>
        </page>
    </page>
</navigation>

If I set a secondary filter of '/foo' on Parent Page, of the two pages in this example the  will be empty when the child page does not match the filter.
Suppose I filter on '/foo' for Parent Page which has three child pages, only two of which have the Page Type of "Foo", if I filter on that, too, only two of the child pages are output in <navigation>.  This seems to be the desired result, but only because they are now output as the first level.
Feature or bug? :)

Feature or bug? :)

@michael-e @brendo ? I would say bug.... but I am not sure. All pages should be filtered ?

I don't really understand the description of the issue, and I am afraid that we need a very systematical approach here, probably combining testing and reverse engineering.

Meaning if @cylkee is right this is what he gets when filtering on page types: First level are properly filtered but not their children. All children are always returned.

When using "Foo" as the filter, the child page will be included in the XML no matter its Page Type.

First level are properly filtered but not their children. All children are always returned.

Correct.  I came across this while trying to create a Navigation data source in a Symphony 2.6.11 installation in which I have created multiple Pages of two main Types and none, with both main Types of Pages having child Pages.


Update RELEASING.md

What was changed? Why is this necessary?
Resolves #39
Add a note about the need to generate a personal access token to use for releases.
How was it tested?
N/A
How to test
This is bare minimum acceptable testing

[ ] mvn clean install -U

Reviewers

[ ] John Leacox
[ ] Sundeep Paruvu
[ ] Nimesh Subramanian
[ ] Supriya Lal
[ ] Brian van de Boogaard

+1

+1

+1


Support for .srt subtitles

.srt subtitles are not appearing in player.

This will depend on if your client supports them. Many only support WebVTT

This would probably not be too difficult to support at least simple conversion from one format to another, depending on what is supported. There are plenty of libs out the that do an ok job, even if the features of the various formats aren't exactly the same.


Error adding Microsoft.AspNetCore.SpaServices.1.0.0-beta-000007 to net451 project

Looking at the issues and nugets it seems like I should be able to use some of features on an existing
asp.net mvc app targeting net451?
But adding the SpaServices packages fails:
Attempting to gather dependency information for package 'Microsoft.AspNetCore.SpaServices.1.0.0-beta-000007' with respect to project 'WebApplication1', targeting '.NETFramework,Version=v4.5.1' Attempting to resolve dependencies for package 'Microsoft.AspNetCore.SpaServices.1.0.0-beta-000007' with DependencyBehavior 'Lowest' Resolving actions to install package 'Microsoft.AspNetCore.SpaServices.1.0.0-beta-000007' Resolved actions to install package 'Microsoft.AspNetCore.SpaServices.1.0.0-beta-000007' Removed package 'Newtonsoft.Json.6.0.4' from 'packages.config' Error loading type library/DLL. (Exception from HRESULT: 0x80029C4A (TYPE_E_CANTLOADLIBRARY)) Successfully uninstalled 'Newtonsoft.Json.6.0.4' from WebApplication1 Adding package 'Microsoft.AspNetCore.Cryptography.Internal.1.0.0' to folder 'C:\foo\WebApplication1\packages' Added package 'Microsoft.AspNetCore.Cryptography.Internal.1.0.0' to folder 'C:\foo\WebApplication1\packages' Install failed. Rolling back... Package 'Microsoft.AspNetCore.Cryptography.Internal.1.0.0' does not exist in project 'WebApplication1' Package 'Newtonsoft.Json.6.0.4' already exists in folder 'C:\foo\WebApplication1\packages' Removing package 'Microsoft.AspNetCore.Cryptography.Internal.1.0.0' from folder 'C:\foo\WebApplication1\packages' Removed package 'Microsoft.AspNetCore.Cryptography.Internal.1.0.0' from folder 'C:\foo\WebApplication1\packages' Failed to add reference to 'Microsoft.AspNetCore.Cryptography.Internal'.
I've tried with and existing 4.6.1 mvc app and a new empty mvc 451 app - all in vs2015 update 3
Should this work?

Hi Richard
When you say "an existing MVC app", do you mean MVC 5 or less, or do you mean the new Microsoft.AspNetCore.Mvc framework running on the .NET 4.6.1 runtime?
The SpaServices package has a dependency on Microsoft.AspNetCore.Mvc, so it's only going to work with ASP.NET Core (though that can run on .NET 4.6.1). It doesn't support MVC 5 or earlier.

thanks Steve - as I expected - my scenario is an MVC5 app - got excited when I saw 451 as the target.
The reason its an MVC5 app is because it makes use of OData and SignalR - I think these are some way off for ASP.NET core?
Do you think its feasible to back port the spa services features to MVC5 ?

Do you think its feasible to back port the spa services features to MVC5 ?

I don't see any reason why it couldn't be made to work - there's nothing we do in SpaServices that fundamentally requires anything that's different in ASP.NET Core. So if you want to have a go at porting it, I'm sure you can succeed (even if it's a fair bit of work).
If this becomes a common request, we might even consider factoring out as much of Microsoft.AspNetCore.SpaServices as we could into some new package (e.g., Microsoft.AspNetCore.SpaServices.Implementation), and have the .Implementation package not take a dependency on ASP.NET Core, but just have .SpaServices take that dependency and act as an adapter on to the ASP.NET Core APIs. Then there could be a different package (e.g., Microsoft.AspNetCore.SpaServices.Mvc5) that also provides an adapter to the ASP.NET MVC 5 APIs.
I can't promise that we will do this, because I don't know that either (a) there'd be enough demand for it, or (b) that it would be clean and simple enough not to add maintenance costs in the future. It's just a thought in case lots of people ask for this.
I'll close this because there's no action for us to take at this moment, but please let us know if you choose to pursue any sort of porting or refactoring on your own. Would love to hear how easy or hard that turns out to be.


Open_basedir restriction

Hi there, thanks for the great plugin. I'm getting this issue on some hosting (not every):
exception 'ErrorException' with message 'file_exists(): open_basedir restriction in effect. File(/tmp//log.htm) is not within the allowed path(s): (/usr/local/www/data/:/var/tmp)' in /usr/local/www/data/mysite.cz/www/plugins/renatio/dynamicpdf/vendor/dompdf/dompdf/src/Dompdf.php:699
In my User export PDF plugin I fixed that by explicit setting of logOutputFile: https://github.com/vojtasvoboda/oc-userexportpdf-plugin/commit/34e07dd0dddddd8ff1b3afadcfc873f2e8d8a5c3
But when I'm using only your plugin separately, I'm having same error when I click on PDF template > Preview.
So I just want to let you know.

Thanks for heads up. I added this to the plugin code.
https://github.com/mplodowski/DynamicPDF/commit/9645cc099320bf88445fd3701b0505b2cbca531b


Unable to set password on first time login to Netlify CMS

The problem
I used the Deploy to Netlify button to configure Netlify with git-gateway and user by invitation, but cannot login after receiving the invitation email. Clicking the link in the email opens the home page, not the CMS page with the Login by Netlify Identity button.
The repo was originally based on the Gatsby demo https://www.gatsbyjs.org/tutorial/blog-netlify-cms-tutorial/.
I tried to fix this with Snippet Injection as suggested in the Netlify docs, but did not work. It seemed overkill to put a snippet on every page when it was only needed on the home page; and the only login needed is for the CMS.
Just before doing all above with my repo, I did the simple starter tutorial described in https://www.gatsbyjs.org/docs/sourcing-from-netlify-cms/. This worked fine for accessing the CMS.  In both, all steps of https://www.netlifycms.org/docs/add-to-your-site/#authentication were followed.
To reproduce
Click on link "Accept the invite" from the email below:-
You have been invited to create a user on http://friendly-goodall-661b1b.netlify.com. Follow this link to accept the invite:
Accept the invite
The public repo is here
Expected behavior
Clicking the link should go to the /admin/ page where a popup form lets me set my password.
Applicable Versions:
gatsby-plugin-netlify-cms version: 4.1.26
Git provider: GitHub
OS: Windows
Browser Opera Version:66.0.3515.72
Node.JS version: 12.13.0
Other context
config.yml includes:-
backend:
name: git-gateway
repo: tristpc/seedrez11
4 plugins are set:-
"gatsby-plugin-netlify-cms": "^4.0.0",
"gatsby-plugin-netlify-identity": "0.0.3"
"netlify-cms-app": "^2.9.1",
"react-netlify-identity-widget": "^0.2.5",

Hi @zzrez, are you using this plugin: https://github.com/tristpc/seedrez11/blob/82852216febaf1bf2dd3e13a73b661c05a3ea1fa/gatsby-config.js#L20
Looks like removing it solves it.

Hi @erezrokah , great removing that plugin really did solve the problem. Thanks, much appreciated.


incorrect names/description in mesh.Plane

@param {int} segmentsX - The number ox x segments
@param {int} segmentsY - The number of y segments

these are actually the number of vertices.  not segments..

Thanks for sharing, fancy putting a PR together? 👍

👍


The MOT benchmark test haven't labels, how compute the loss?

Hello, i'm studing adversarial sample in multi-object tracking. In your paper, using loss function to generate adversarial video, but the test lable can't get, so how do you did it? Hope to get your help.

Thank you for your attention. As mentioned in paper's Sec. 4.1, we call a successful attack when the detected objects of the attack trajectory are no longer associated with the original tracklet predicted by the target model after the attack.


Overwrites report file in umbrella app

Hi.
In my setup junit-formatter is running fine if I run individual test suites in individual apps.
However, when I run "mix test" in an umbrella app containing them all, it simply writes all report files to the same location which means the later test suites overwrite results from the earlier ones.
Maybe it could make subdirectories?

PS - this even happens when I give one of the apps in the umbrella its own config.exs with a different file name.

Thanks for reporting @DerKastellan and sorry it took me sometime to reply.
I will look into it (I confess I expected it to work with umbrella...)!

I've published version 1.1.0 with support for umbrella projects. It now generates the report file in Mix.Project.app_path instead of build_path. This will allow for several projects to have each their own report.
Please, check it out! I will leave this one open until you confirm it solves your issue.

Thank you, works for me. Writes results to ./_build/test/lib//test-junit-report.xml :smile:

Can I close this one @DerKastellan ?

Yes, please close this.


Libdoc: Create xsd schema for spec files

Libdoc spec files don't have official schema definition. Now that they've got new attributes deprecated (#3498) as well as source and lineno (#3507) that creating a schema gets even more important.

@Snooz82 reported several problems with the schema file in #3525. I closed that issue and re-open this one instead.

Enhanced schema was provided by @Snooz82 in PR #3526, more precisely 683a994caee554eaa5cf0ac822616cb73f0f2e6b. I'll add some more comments to the schema still but otherwise this is ready to be re-closed.

Comments were added in 88fa2cdc7cc754d769f3877ba5b1ac350f3a39d1. I decided to add tests for the schema and this issue can stay open until they are ready.


⚠ Adacta-fintech.com has degraded performance

In 88c8026, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1650 ms

Resolved: Adacta-fintech.com performance has improved in f508e0d after 7 minutes.


Support roles like for example user, system, assistant

When sending messages directly to the Mistral API (similar to OpenAI API), then one can send messages with different roles, whereas see

https://platform.openai.com/docs/guides/text-generation/chat-completions-api
https://cookbook.openai.com/examples/azure/chat_with_your_own_data#code

Is this possible somehow with ollama4j respectively Ollama itself?

Ollama wise this is possible, for example:
curl http://localhost:11434/api/chat -d '{
  "model": "orca-mini",
  "messages": [
    { "role": "system", "content": "You are a sailor that answers every question in a very piraty way" },
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

would override the system-prompt given in the modeldefinition of ollama.
Thus it should be possible to implement this. If @amithkoujalgi is okay with this, I'd like to give it a try to extend the RequestModel to support this.
Cheers!

@michaelwechner Thank you for the info. I'd very much like to support this from Ollama4j, but unfortunately, I'm currently occupied with a ton of work and unable to dedicate time to Ollama4j.
@AgentSchmecker I'd very much appreciate it if you could give it a try!

In addition to the /api/chat endpoint, the system prompt parameter (and other parameters that override model behaviours) can also be provided to requests at /api/generate.
Thus these should also be made available for the ollama4j API.


Support for setting table properties as part of a model configuration

resolves #33
Description
Supports to set tblproperties when creating tables and views.
In the models config, set tblproperties as follows.
{{ config(
  materialized='incremental',
  tblproperties={
    'delta.autoOptimize.optimizeWrite' : 'true',
    'delta.autoOptimize.autoCompact' : 'true'
  }
) }}

Thanks! merging.

@ueshin @superdupershant
Hey! I discovered that though properties are applied for the table creation step they are not applied incrementally if you add the properties later. Is this expected for this change, or I had something wrong with my set up?


Data: Telemetry: Ability Requiring Target used but no target shown

{ "time": "2017-02-18T12:55:24+0000", "type": "UseAbility", "payload": { "Team": "Left", "Actor": "*Hero016*", "Ability": "HERO_ABILITY_RONA_B_NAME", "Position": [ -18.59, 0.01, 41.43 ] } }

Upon deeper research into this i discovered that DealDamage does not appear when you are attacking a minion or jungle monster therefore this is possibly attacking a minion. However, this really should show which minion was targeted. Foesplitter requires a target be selected and then has a dash towards that target as well.
The next time Rona is shown is:
{ "time": "2017-02-18T12:55:28+0000", "type": "EarnXP", "payload": { "Team": "Left", "Actor": "*Hero016*", "Source": "*Neutral_JungleMinion_DefaultSmall*", "Amount": 5, "Shared With": 1 } }

Can we also please add on whether an ability timed out. For example Foesplitter, has 2 activations. The second ability times out when not using it after a little bit. It is impossible to determine currently.

I discovered that her second activation has:
{ "time": "2017-02-18T13:09:29+0000", "type": "DealDamage", "payload": { "Team": "Left", "Actor": "*Hero016*", "Target": "*VainCrystalAway*", "Source": "Ability__Rona__BAttackOverride", "Damage": 251, "Delt":  251, "IsHero": 1, "TargetIsHero": 0 } }

So this can be used to determine it but it is not the best source to determine it. It should should the stats increase as well as when the stats decrease and the ability dies out.

Targets were added


Learning about point tasks in a process

Using global variables and the usual thread synchronization, it is easy to take process-local notes as point tasks execute.  However, the distribution of point tasks over (Unix) processes is in general known only by the relevant mapper instances, and it could be expensive to communicate it to other processes where the point tasks end up running.  As such, no process knows when all its assigned point tasks have executed (so as to finalize/transmit the collected notes).  (As a particular example, a process might end up with 0 point tasks such that there is no hint as to when the launch took place at all.)
Could we have a means of learning from Legion (which already has to arrange for the right number of calls to take place) either how many such point tasks from a particular (index) launch have or will run in each process or that, again per-launch, no more point tasks will be started in each process?  One form that this interface might take would be a callback function (with access at least to the TaskArgument) called once per Legion::Runtime per task launch.  It would either provide the number as an argument or (following the CPS idea) be invoked after all relevant point tasks.  In the usual (non-debugging) situation of one Runtime per process, this produces one call on each process as desired, independent of sharding patterns.
A toy example might be to collect per-process statistics about a field for lightweight in-situ analysis:
std::atomic<unsigned> max;
void bunny_task(...) {
  unsigned mine=0;
  for(i...) {
    // update various fields, including bunny_field
    mine=std::max(mine,bunny_field[i]);
  }
  max.fetch_max(mine);
}
void callback(Legion::Task *) {
  if(max.exchange(0)>75) std::cerr << getpid() << ": too many bunnies\n";
}
void caller_task() {
  run->execute_index_space(ctx,bunny_task_launcher);
}
void near_main() {
  Runtime::set_top_level_task_id(...);
  Runtime::register_launch_complete_callback(callback);
  Runtime::start(...);
}

Obviously as written this works only if all the launches of bunny_task are serialized (though their point tasks can run in parallel!).  Passing the Legion::Task* to the callback allows a more sophisticated (read: realistic) callback to perform separate bookkeeping for separate launches.
A very different CPS-like approach (which would be sufficient for our use cases) might be to equip Legion::Future with a then (and Legion::FutureMap with an all_then) that performs a callback (in the process that registered the callback, or else in all processes that collectively registered one) when the future is (all the futures are) ready without requiring a separate task launch or blocking.

One interesting special case here is a task configured with set_idempotent: there the total number of point tasks is not known a priori, although it would seem that the same interface would suffice.

A very different CPS-like approach (which would be sufficient for our use cases) might be to equip Legion::Future with a then (and Legion::FutureMap with an all_then) that performs a callback (in the process that registered the callback, or else in all processes that collectively registered one) when the future is (all the futures are) ready without requiring a separate task launch or blocking.

How is that different than launching a task which only depends on that future? Tasks are already callbacks that you get when dependences are satisfied. If you don't want to send the task all the way through the pipeline you can even make it a local_function_task since it has no region requirements, but that is an optimization and not required.
https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/legion.h?ref_type=heads#L1619-1627

A local_function_task might be most of what we want here, but I'm not sure:

We would want to be able to have it proceed only when an entire FutureMap was complete (ideally more efficiently than by adding every FutureMap::get_future result to TaskLauncher::futures).
Does "on a local processor where the parent task is executing" mean that the local function task runs on every such processor when the parent task is control-replicated, or just on an unspecified one of them?
What exactly does "pure function with no side effects" mean here?  Without any region requirements, of course such a task cannot alter any field values, so is this a statement about making other calls into Legion (e.g., executing further tasks) or what?

We would want to be able to have it proceed only when an entire FutureMap was complete (ideally more efficiently than by adding every FutureMap::get_future result to TaskLauncher::futures).

You can't do this at the moment, but it probably wouldn't be too hard to add. Do you really want it to wait on every future in the future map (all point tasks), or are just certain futures enough?

Does "on a local processor where the parent task is executing" mean that the local function task runs on every such processor when the parent task is control-replicated, or just on an unspecified one of them?

It will actually run on exactly the same processor as the parent task (when the parent is preempted and the futures have all completed).

What exactly does "pure function with no side effects" mean here? Without any region requirements, of course such a task cannot alter any field values, so is this a statement about making other calls into Legion (e.g., executing further tasks) or what?

Mostly this means that you don't have any region requirements. You can have side effects, but like calling printf but it's up to you to manage them in that case. I think you can have local function tasks launch further sub-tasks, but since they can't have any region arguments so there is no way to pass privileges.

You can't do this at the moment, but it probably wouldn't be too hard to add. Do you really want it to wait on every future in the future map (all point tasks), or are just certain futures enough?

Well, for the particular use case we have in mind, it would be sufficient to wait on all the futures corresponding to point tasks dispatched by the same Runtime or by the same shard.  I said just "all" because it seemed more likely to be generically useful.

It will actually run on exactly the same processor as the parent task (when the parent is preempted and the futures have all completed).

So, to be sure, if the parent task is control-replicated and (as one would expect) all of its shards post the same local function task, that task runs once each on every processor that has been running (one shard of) the parent task?

Mostly this means that you don't have any region requirements. You can have side effects, but like calling printf but it's up to you to manage them in that case. I think you can have local function tasks launch further sub-tasks, but since they can't have any region arguments so there is no way to pass privileges.

I suppose you could create entirely new logical regions?

Well, for the particular use case we have in mind, it would be sufficient to wait on all the futures corresponding to point tasks dispatched by the same Runtime or by the same shard. I said just "all" because it seemed more likely to be generically useful.

Right, that is an optimization, but you would probably want to express it as a dependence on the whole future map.

So, to be sure, if the parent task is control-replicated and (as one would expect) all of its shards post the same local function task, that task runs once each on every processor that has been running (one shard of) the parent task?

Yes, we implicitly replicate local function tasks so we run a copy on every shard if the parent is control replicated. This is easy to do since the only inputs and outputs are futures and not regions.

I suppose you could create entirely new logical regions?

You could I suppose, but I think we might enforce that the selected variant be a leaf task variant, so you could make the new logical regions but then not populate it with any data.

So it sounds like TaskLauncher::local_function_task with a new TaskLauncher::future_maps or so would satisfy our requirements, so long as set_idempotent doesn't spoil it.  Can we have that?

So it sounds like TaskLauncher::local_function_task with a new TaskLauncher::future_maps or so would satisfy our requirements, so long as set_idempotent doesn't spoil it. Can we have that?

What do you mean by "so long as set_idempotent doesn't spoil it?" What are you afraid that setting it idempotent would do?

I can imagine that idempotent task launches, since the runtime has permission to run them extra times, might complicate the accounting: might a Future(Map) returned from an idempotent task launch become ready (in the wait_all_results sense as appropriate) before all point tasks have completed if the value is known from those that have?

might a Future(Map) returned from an idempotent task launch become ready (in the wait_all_results sense as appropriate) before all point tasks have completed if the value is known from those that have?

No, wait_all_results can only return once all the futures are ready. If one point task has to restart for resilience reasons then it won't be considered complete and therefore not be counted towards wait_all_results. Similarly a Future for a particular point task is not considered complete until that point is done running. If the point fails and is restarted for resilience reasons (why you might mark it idempotent) then the future will only make it to the complete stage once we know we no longer need to rollback and rerun the task.

I was under the impression that idempotence might be used not for resilience but just to substitute computation for communication, but for the purpose of this issue that doesn't matter: if the FutureMap becomes ready only when all the point tasks (idempotent extras or no) are complete, then the TaskLauncher::future_maps idea is good enough.  Is that something we can have soon?

Just thinking about what we've got already, can you use Runtime::reduce_future_map to reduce the future map down to a a future that you then pass as a precondition to the TaskLauncher? I'm not just suggesting this for convenience; I also suspect that it will be better for performance as well because it will do a butterfly all-reduce between the shards for communicating when all the futures are done which is more scalable than what you'll get with the FutureMap::wait_all_results call today.

We would need to use reduce_future_map on the user tasks' FutureMap objects for which we don't necessarily have any reductions defined (most commonly because the task returns void).  Is there some dummy ReductionOpID that could be used to not actually reduce anything?  Would the original values (when not void) still be available via the FutureMap?

You can't have function parameters of type void, nor can you ever have references to void, so I don't see how to do this.

I've always found that a bit ridiculous. Rust's type system has the nice bottom type/value ().

Does FutureMap::get_void_result work even when the actual type isn't void?

Yes, it works regardless of the return type.

Could we avoid this circumlocution by having Legion provide a LEGION_REDOP_VOID that does nothing, or would that be just as hard (though in Legion)?

Let me think about this. In general, Legion's philosophy is that dependences should only exist if they are data dependences. This would go against that philosophy because we'd only be propagating a control dependence and not any actual data.

Also, the question about the destructiveness or not of the reduction remains.

I've lost track of this: what is the question about destructiveness?

I've always found that a bit ridiculous. Rust's type system has the nice bottom type/value ().

It has of course been proposed to rectify that, but obviously compatibility is often an insurmountable challenge.  (But it's not the usual bottom, but unit.)

Could we avoid this circumlocution by having Legion provide a LEGION_REDOP_VOID that does nothing, or would that be just as hard (though in Legion)?

Let me think about this. In general, Legion's philosophy is that dependences should only exist if they are data dependences. This would go against that philosophy because we'd only be propagating a control dependence and not any actual data.

I'm not sure that's a coherent position.  Trivially, we already have get_void_result, and obviously that's supposed to establish a synchronization without a data dependency.  At a slightly more complicated level, it suggests that as soon as one bool is known to be true in a LEGION_REDOP_OR_BOOL reduction, we can launch tasks that depend on the result, denying an obvious place a user might expect synchronization.  (We could say we do want that, but then what about INT_MAX with LEGION_REDOP_MAX_INT32?  Is that special enough that it should defeat synchronization?)
Moreover, "true" data dependencies are very hard to define, and appealing to the fact that Legion (not being a compiler) can't conduct any of the proofs required to make that definition interesting is just saying that void (construed as a unit type) is special just because we happen to know its value statically.  Would we then extend that special treatment to std::nullptr_t?  To std::monostate?  To any type satisfying std::is_empty_v?  To std::tuple<std::nullptr_t,std::nullptr_t>?

Also, the question about the destructiveness or not of the reduction remains.

I've lost track of this: what is the question about destructiveness?

From a few posts above:

Would the original values (when not void) still be available via the FutureMap?

I'm not sure that's a coherent position. Trivially, we already have get_void_result, and obviously that's supposed to establish a synchronization without a data dependency.

That's a synchronization and implies blocking behavior. It's not an asynchronous dependence. Legion supports the blocking behavior because it has to, but there are no ways to express asynchronous control dependences in Legion.

At a slightly more complicated level, it suggests that as soon as one bool is known to be true in a LEGION_REDOP_OR_BOOL reduction, we can launch tasks that depend on the result, denying an obvious place a user might expect synchronization. (We could say we do want that, but then what about INT_MAX with LEGION_REDOP_MAX_INT32? Is that special enough that it should defeat synchronization?)

We don't do short-circuiting with reduction operators because it leads to races with the distributed reduction protocols. We always run the full protocol even if technically we could short-circuit a computation. This also ensures that reduction operators can be treated like black boxes and Legion doesn't need to know the semantics of them. Legion doesn't explicitly know about the default reduction operators which are really only there as a library of convenience for users.

Moreover, "true" data dependencies are very hard to define,

It may be hard to define for multi-thread code with a relaxed memory model, but not for "single-threaded" programs with sequential semantics. As with other (apparently) sequential programs, all Legion programs can trivially define true data dependences: every read has to see the result of the most recent write (and the accumulation of any reductions on top of that most recent write) as defined by the program order of the tasks. Memory coherence and memory consistency models are both trivial in such circumstances and true data dependences can be precisely defined (unless you use relaxed coherence modes in which case it is not trivial but still well defined).

Would we then extend that special treatment to std::nullptr_t? To std::monostate? To any type satisfying std::is_empty_v? To std::tuplestd::nullptr_t,std::nullptr_t?

No, those are different than void because they are types with non-empty storage and information can be passed through those types. You should have no trouble defining reduction operators using those types and Legion will abide by them. As long as there are bits where information can flow then Legion has to respect that as a true data dependence.

Would the original values (when not void) still be available via the FutureMap?

Yes, future maps are always immutable and are not consumed by an all-reduce operation.

That's a synchronization and implies blocking behavior. It's not an asynchronous dependence. Legion supports the blocking behavior because it has to, but there are no ways to express asynchronous control dependences in Legion.

I'm not sure I know what "asynchronous control dependence" means.  Are you saying that a void Legion::Future can't delay the launch of a task via TaskLauncher.futures (which I think I remember discussing at some point)?  If so, sure, but then we just get into std::nullptr_t and so forth again and we wonder why void is special.

It may be hard to define for multi-thread code with a relaxed memory model, but not for "single-threaded" programs with sequential semantics.

I'm not sure that's the right way to read that discussion.  The effects of data dependencies on the analysis of relaxed-memory-model code are certainly subtle, but the definition doesn't ever seem easy to me.  Does y depend on x in y=a*x+b?  What if a is 0—for a particular evaluation, or throughout a particular task, or via a template parameter, or via a macro, or as a literal?  What if a isn't 0, but is a sufficiently large power of 2 that the multiplication with x is shifted out of the smaller range of y and is lost in the conversion?  What if b was previously assigned c-a*x such that it's just y=c?

No, those are different than void because they are types with non-empty storage and information can be passed through those types. You should have no trouble defining reduction operators using those types and Legion will abide by them. As long as there are bits where information can flow then Legion has to respect that as a true data dependence.

I don't know what sort of information you mean if it can be passed through std::nullptr_t: "reading" such an object doesn't even touch memory ([conv.lval]/3.1).  It's true that its sizeof is non-zero, so you can talk about its object representation as a byte buffer, but it sounds strange to use a C++ type not for its values but for its memory, but since sizeof(void) doesn't exist at all we might as well pretend that it's 1 (with all padding bits) like compilers do for their void* arithmetic extensions as say that it uniquely is 0.
Or are you saying that Legion is interested in the object representation as a byte buffer?  I suppose that works even with the reduction interface, since everything is done by reference, but it seems more than a little fragile to have x=y break your data because it doesn't actually copy anything.

Would the original values (when not void) still be available via the FutureMap?

Yes, future maps are always immutable and are not consumed by an all-reduce operation.

Great.  I think that means the question about void is the only thing open now.  (If Legion decides not to support it, I suppose my second choice would be to use an internal empty enumeration type, so that we could simply map it back to void for our users; I think that's easy to implement.)

Are you saying that a void Legion::Future can't delay the launch of a task via TaskLauncher.futures (which I think I remember discussing at some point)?

Yes.

If so, sure, but then we just get into std::nullptr_t and so forth again and we wonder why void is special.

Legion doesn't understand types. Futures that carry no data (have zero-sized buffers, aka void futures) are special because no information (in the information theoretic sense) can be passed by value through them (although I suppose you could abuse them by delaying setting them and communicate information in a side-channel way through how long it takes for your task to finish and set the future, but that is an egregious abuse of the programming model).

Does y depend on x in y=ax+b? What if a is 0—for a particular evaluation, or throughout a particular task, or via a template parameter, or via a macro, or as a literal? What if a isn't 0, but is a sufficiently large power of 2 that the multiplication with x is shifted out of the smaller range of y and is lost in the conversion? What if b was previously assigned c-ax such that it's just y=c?

Legion doesn't care. If there are bits that are passed from one computation (aka task) to another then there is a data dependence; no bits means no data dependence. The interpretation of those bits is up to the application. Legion won't optimize away for things that C++ might consider undefined behavior (overflow, etc). Moving bits -> data dependence.

Or are you saying that Legion is interested in the object representation as a byte buffer?

Precisely.

I suppose that works even with the reduction interface, since everything is done by reference, but it seems more than a little fragile to have x=y break your data because it doesn't actually copy anything.

Not sure what you mean by this.

Are you saying that a void Legion::Future can't delay the launch of a task via TaskLauncher.futures (which I think I remember discussing at some point)?

Yes.

I guess a significant part of what (little) remains to discuss is why Legion bothers to notice this case: if what Legion cares about is byte buffers, then void is special only in that the buffer is empty (rather than being special in the type system), and I can't immediately think of a reason that a client would express a dependency graph in terms of void futures and be happy that Legion ignored those dependencies.

Legion doesn't care. If there are bits that are passed from one computation (aka task) to another then there is a data dependence; no bits means no data dependence. The interpretation of those bits is up to the application. Legion won't optimize away for things that C++ might consider undefined behavior (overflow, etc). Moving bits -> data dependence.

I guess I would call this a "potential data dependence", since it's really a statement about the limited knowledge Legion has about the application.

I suppose that works even with the reduction interface, since everything is done by reference, but it seems more than a little fragile to have x=y break your data because it doesn't actually copy anything.

Not sure what you mean by this.

In
std::nullptr_t x,y;
std::memset(&x,1,1);
y=x;
assert(std::memchr(&y,1,1)==&y);

the assert can fail because the byte stored in the "buffer" x is not copied to y by the assignment.  That's the cost of summarily ignoring the type system, which might or might not be acceptable.

We now have the mapper-based approach working with tracing (by using select_task_options and ignoring our own results when running a trace other than the first time), but I realize that that brings back the potential for trouble from idempotence.  How does restarting (or, if it happens, duplication) of point tasks work there?  Surely our wrapper code would run again, which is bad unless we can learn that it's happening.

Documenting our discussion from the meeting today: currently marking a task as idempotent does not have any impact on whether a task can be re-run or not in Legion. It is an aspirational feature only at the moment. In the future, it will be used to know that a task could potentially be able to be rerun either for resilience purposes or because of mis-speculation during speculative execution. Both of those features will be opt-in features though so applications and mappers will know when they are being used, and the runtime won't just be suddenly rerunning tasks out of thin air. This means situations involving idempotent tasks being rerun are safe to ignore for now and will only become relevant if, and only if, you actually start using future resilience and/or speculative execution features.