The speed command did not support config option

Version:  OpenSSL 3.1.0-dev
OS: linux-x86_64
Step:
1 Build
./Configure enable-fips  -g
make
2 execute the speed command by configured provider
./apps/openssl speed -provider-path=./providers/ -provider fips -evp aes-256-gcm
Error information:
the selftest parameter get from core with core_get_params,but there is no configration for  SSL_PROV_FIPS_PARAM_INSTALL_MAC
speed: unable to load provider fips
Hint: use -provider-path option or OPENSSL_MODULES environment variable.
C0C1F7231E7F0000:error:1C8000D5:Provider routines:SELF_TEST_post:missing config data:providers/fips/self_test.c:289:
C0C1F7231E7F0000:error:1C8000E0:Provider routines:ossl_set_error_state:fips module entering error state:providers/fips/self_test.c:387:
C0C1F7231E7F0000:error:1C8000D8:Provider routines:OSSL_provider_init_int:self test post failure:providers/fips/fipsprov.c:706:
C0C1F7231E7F0000:error:078C0105:common libcrypto routines:provider_init:init fail:crypto/provider_core.c:903:name=fips
Analysis:
the -provider fips failed load the fips library as the fips configure file did not setted by environment variables. and the speed command did not support setted by command. This result provider option could not working without  environment variables.
Support config option will make the speed working without special environment variables.

Read the FIPS instructions.

Read the FIPS instructions.

Thanks for the response, the FIPS instructions  works for the config file generation but not for the load

You need to tell OpenSSL about the config.  You most likely want to create your own config file.

You need to tell OpenSSL about the config. You most likely want to create your own config file.

Sure, the config was configured with OPENSSL_CONF and successed loaded when init, the issue was for reported when configure provider in speed command. it seems that macdata was not loaded in the parameters

Did the config file include the FIPS specific configuration?  I.e. have the .include fipsmodule.cnf like this (or the various parameters included):
openssl_conf = openssl_init

# Comment out the next line to ignore configuration errors
config_diagnostics = 1

.include fipsmodule.cnf

[openssl_init]
providers = provider_sect

[provider_sect]
fips = fips_sect
base = base_sect

[base_sect]
activate = 1

Did the config file include the FIPS specific configuration? I.e. have the .include fipsmodule.cnf like this (or the various parameters included):
openssl_conf = openssl_init

# Comment out the next line to ignore configuration errors
config_diagnostics = 1

.include fipsmodule.cnf

[openssl_init]
providers = provider_sect

[provider_sect]
fips = fips_sect
base = base_sect

[base_sect]
activate = 1

openssl.cnf:
openssl_conf = openssl_init
config_diagnostics = 1
.include /absolute path/fipsmodule.cnf
[openssl_init]
providers = provider_sect
[provider_sect]
default = default_sect
fips = fips_sect
[default_sect]
activate = 1
fipsmodule.cnf:
[fips_sect]
activate = 1
conditional-errors = 1
security-checks = 1
module-mac = 13:4A:AE:5E:8B:EC:D0:63:0D:42:55:14:A5:15:D6:A1:0C:60:37:3A:2D:EC:62:45:A0:5A:0C:39:A9:9A:46:86

That looks okay.  A blank line before the .include might be required.

That looks okay. A blank line before the .include might be required.

Sure, the blank line present.
the openssl.cnf and fipsmodule.cnf have loaded, it seems that the module macdata was not setted to the providers.
OSSL_PROV_PARAM_CORE_MODULE_FILENAME was defined,
but OSSL_PROV_PARAM_CORE_MODULE_MAC_DATA was not defined.

OSSL_PROV_FIPS_PARAM_MODULE_MAC?  There is no OSSL_PROV_PARAM_CORE_MODULE_MAC_DATA.  There isn't mean to be.
OSSL_PROV_FIPS_PARAM_MODULE_MAC is module-mac in the config file.

OSSL_PROV_FIPS_PARAM_MODULE_MAC

Got it ,I regarded this as the MAC Algrithm,I will check further, thanks.

OSSL_PROV_FIPS_PARAM_MODULE_MAC

Got it ,I regarded this as the MAC Algrithm,I will check further, thanks.

this have been checked , the openssl.cnf and the included fipsmodule.cnf was loaded in CONF_modules_load_file, but the params in speed command was get in core_get_params , these bellows to different libctx, so the speed command could not get the configured macdata in fipsmodule.cnf defaultly.
the callstack in CONF_modules_load_file:
#0  provider_conf_params (prov=0x70e230, provinfo=0x0, name=0x0, value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:77
#1  0x00007ffff74808d9 in provider_conf_load (libctx=0x0, name=0x7089d0 "default", value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:219
#2  0x00007ffff7480bf8 in provider_conf_init (md=0x70e0c0, cnf=0x705420) at crypto/provider_conf.c:293
#3  0x00007ffff737f699 in module_init (pmod=0x7055c0, name=0x708890 "providers", value=0x7088e0 "provider_sect", cnf=0x705420) at crypto/conf/conf_mod.c:375
#4  0x00007ffff737f1d6 in module_run (cnf=0x705420, name=0x708890 "providers", value=0x7088e0 "provider_sect", flags=32) at crypto/conf/conf_mod.c:239
#5  0x00007ffff737eefa in CONF_modules_load (cnf=0x705420, appname=0x0, flags=32) at crypto/conf/conf_mod.c:138
#6  0x00007ffff737f01b in CONF_modules_load_file_ex (libctx=0x0, filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:181
#7  0x00007ffff737f0bb in CONF_modules_load_file (filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:203
#8  0x00007ffff737fe7d in ossl_config_int (settings=0x0) at crypto/conf/conf_sap.c:64
#9  0x00007ffff74749f1 in ossl_init_config () at crypto/init.c:242
#10 0x00007ffff74749d6 in ossl_init_config_ossl_ () at crypto/init.c:240
#11 0x00007ffff7016a80 in pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:103
#12 0x00007ffff7485c1a in CRYPTO_THREAD_run_once (once=0x7ffff7906cfc , init=0x7ffff74749cd <ossl_init_config_ossl_>) at crypto/threads_pthread.c:144
#13 0x00007ffff7474ffd in OPENSSL_init_crypto (opts=30284, settings=0x0) at crypto/init.c:584
#14 0x00007ffff79403db in OPENSSL_init_ssl (opts=30284, settings=0x0) at ssl/ssl_init.c:127
#15 0x000000000044c2ca in apps_startup () at apps/openssl.c:66
#16 0x000000000044c3b8 in main (argc=8, argv=0x7fffffffe4d8) at apps/openssl.c:265
the callstack for params in spped command:
#0  core_get_params (handle=0x70d4e0, params=0x7fffffffdbe0) at crypto/provider_core.c:1825
#1  0x00007ffff5c566d7 in fips_get_params_from_core (fgbl=0x70d090) at providers/fips/fipsprov.c:156
#2  0x00007ffff5c57523 in OSSL_provider_init_int (handle=0x70d4e0, in=0x7ffff78eeff0 <core_dispatch_+752>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fipsprov.c:686
#3  0x00007ffff5c560e5 in OSSL_provider_init (handle=0x70d4e0, in=0x7ffff78eed00 <core_dispatch_>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fips_entry.c:18
#4  0x00007ffff748298d in provider_init (prov=0x70d4e0) at crypto/provider_core.c:911
#5  0x00007ffff7482e8e in provider_activate (prov=0x70d4e0, lock=0, upcalls=1) at crypto/provider_core.c:1100
#6  0x00007ffff74830ef in ossl_provider_activate (prov=0x70d4e0, upcalls=1, aschild=0) at crypto/provider_core.c:1177
#7  0x00007ffff747f5a0 in OSSL_PROVIDER_try_load (libctx=0x0, name=0x7fffffffe7b4 "fips", retain_fallbacks=0) at crypto/provider.c:31
#8  0x00007ffff747f63c in OSSL_PROVIDER_load (libctx=0x0, name=0x7fffffffe7b4 "fips") at crypto/provider.c:50
#9  0x00000000004876cf in app_provider_load (libctx=0x0, provider_name=0x7fffffffe7b4 "fips") at apps/lib/app_provider.c:37
#10 0x000000000048783b in opt_provider (opt=1601) at apps/lib/app_provider.c:78
#11 0x0000000000473f89 in speed_main (argc=7, argv=0x7fffffffe4e0) at apps/speed.c:1589
#12 0x000000000044c95b in do_cmd (prog=0x707e40, argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:418
#13 0x000000000044c574 in main (argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:298

OSSL_PROV_FIPS_PARAM_MODULE_MAC

Got it ,I regarded this as the MAC Algrithm,I will check further, thanks.

this have been checked , the openssl.cnf and the included fipsmodule.cnf was loaded in CONF_modules_load_file, but the params in speed command was get in core_get_params , these bellows to different libctx, so the speed command could not get the configured macdata in fipsmodule.cnf defaultly.
the callstack in CONF_modules_load_file: #0 provider_conf_params (prov=0x70e230, provinfo=0x0, name=0x0, value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:77 #1 0x00007ffff74808d9 in provider_conf_load (libctx=0x0, name=0x7089d0 "default", value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:219 #2 0x00007ffff7480bf8 in provider_conf_init (md=0x70e0c0, cnf=0x705420) at crypto/provider_conf.c:293 #3 0x00007ffff737f699 in module_init (pmod=0x7055c0, name=0x708890 "providers", value=0x7088e0 "provider_sect", cnf=0x705420) at crypto/conf/conf_mod.c:375 #4 0x00007ffff737f1d6 in module_run (cnf=0x705420, name=0x708890 "providers", value=0x7088e0 "provider_sect", flags=32) at crypto/conf/conf_mod.c:239 #5 0x00007ffff737eefa in CONF_modules_load (cnf=0x705420, appname=0x0, flags=32) at crypto/conf/conf_mod.c:138 #6 0x00007ffff737f01b in CONF_modules_load_file_ex (libctx=0x0, filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:181 #7 0x00007ffff737f0bb in CONF_modules_load_file (filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:203 #8 0x00007ffff737fe7d in ossl_config_int (settings=0x0) at crypto/conf/conf_sap.c:64 #9 0x00007ffff74749f1 in ossl_init_config () at crypto/init.c:242 #10 0x00007ffff74749d6 in ossl_init_config_ossl_ () at crypto/init.c:240 #11 0x00007ffff7016a80 in pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:103 #12 0x00007ffff7485c1a in CRYPTO_THREAD_run_once (once=0x7ffff7906cfc , init=0x7ffff74749cd <ossl_init_config_ossl_>) at crypto/threads_pthread.c:144 #13 0x00007ffff7474ffd in OPENSSL_init_crypto (opts=30284, settings=0x0) at crypto/init.c:584 #14 0x00007ffff79403db in OPENSSL_init_ssl (opts=30284, settings=0x0) at ssl/ssl_init.c:127 #15 0x000000000044c2ca in apps_startup () at apps/openssl.c:66 #16 0x000000000044c3b8 in main (argc=8, argv=0x7fffffffe4d8) at apps/openssl.c:265
the callstack for params in spped command: #0 core_get_params (handle=0x70d4e0, params=0x7fffffffdbe0) at crypto/provider_core.c:1825 #1 0x00007ffff5c566d7 in fips_get_params_from_core (fgbl=0x70d090) at providers/fips/fipsprov.c:156 #2 0x00007ffff5c57523 in OSSL_provider_init_int (handle=0x70d4e0, in=0x7ffff78eeff0 <core_dispatch_+752>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fipsprov.c:686 #3 0x00007ffff5c560e5 in OSSL_provider_init (handle=0x70d4e0, in=0x7ffff78eed00 <core_dispatch_>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fips_entry.c:18 #4 0x00007ffff748298d in provider_init (prov=0x70d4e0) at crypto/provider_core.c:911 #5 0x00007ffff7482e8e in provider_activate (prov=0x70d4e0, lock=0, upcalls=1) at crypto/provider_core.c:1100 #6 0x00007ffff74830ef in ossl_provider_activate (prov=0x70d4e0, upcalls=1, aschild=0) at crypto/provider_core.c:1177 #7 0x00007ffff747f5a0 in OSSL_PROVIDER_try_load (libctx=0x0, name=0x7fffffffe7b4 "fips", retain_fallbacks=0) at crypto/provider.c:31 #8 0x00007ffff747f63c in OSSL_PROVIDER_load (libctx=0x0, name=0x7fffffffe7b4 "fips") at crypto/provider.c:50 #9 0x00000000004876cf in app_provider_load (libctx=0x0, provider_name=0x7fffffffe7b4 "fips") at apps/lib/app_provider.c:37 #10 0x000000000048783b in opt_provider (opt=1601) at apps/lib/app_provider.c:78 #11 0x0000000000473f89 in speed_main (argc=7, argv=0x7fffffffe4e0) at apps/speed.c:1589 #12 0x000000000044c95b in do_cmd (prog=0x707e40, argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:418 #13 0x000000000044c574 in main (argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:298

this result from provider_activate failed, maybe the store failed to set in case of fips provider,anyway, I will check further

OSSL_PROV_FIPS_PARAM_MODULE_MAC

Got it ,I regarded this as the MAC Algrithm,I will check further, thanks.

this have been checked , the openssl.cnf and the included fipsmodule.cnf was loaded in CONF_modules_load_file, but the params in speed command was get in core_get_params , these bellows to different libctx, so the speed command could not get the configured macdata in fipsmodule.cnf defaultly.
the callstack in CONF_modules_load_file: #0 provider_conf_params (prov=0x70e230, provinfo=0x0, name=0x0, value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:77 #1 0x00007ffff74808d9 in provider_conf_load (libctx=0x0, name=0x7089d0 "default", value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:219 #2 0x00007ffff7480bf8 in provider_conf_init (md=0x70e0c0, cnf=0x705420) at crypto/provider_conf.c:293 #3 0x00007ffff737f699 in module_init (pmod=0x7055c0, name=0x708890 "providers", value=0x7088e0 "provider_sect", cnf=0x705420) at crypto/conf/conf_mod.c:375 #4 0x00007ffff737f1d6 in module_run (cnf=0x705420, name=0x708890 "providers", value=0x7088e0 "provider_sect", flags=32) at crypto/conf/conf_mod.c:239 #5 0x00007ffff737eefa in CONF_modules_load (cnf=0x705420, appname=0x0, flags=32) at crypto/conf/conf_mod.c:138 #6 0x00007ffff737f01b in CONF_modules_load_file_ex (libctx=0x0, filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:181 #7 0x00007ffff737f0bb in CONF_modules_load_file (filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:203 #8 0x00007ffff737fe7d in ossl_config_int (settings=0x0) at crypto/conf/conf_sap.c:64 #9 0x00007ffff74749f1 in ossl_init_config () at crypto/init.c:242 #10 0x00007ffff74749d6 in ossl_init_config_ossl_ () at crypto/init.c:240 #11 0x00007ffff7016a80 in pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:103 #12 0x00007ffff7485c1a in CRYPTO_THREAD_run_once (once=0x7ffff7906cfc , init=0x7ffff74749cd <ossl_init_config_ossl_>) at crypto/threads_pthread.c:144 #13 0x00007ffff7474ffd in OPENSSL_init_crypto (opts=30284, settings=0x0) at crypto/init.c:584 #14 0x00007ffff79403db in OPENSSL_init_ssl (opts=30284, settings=0x0) at ssl/ssl_init.c:127 #15 0x000000000044c2ca in apps_startup () at apps/openssl.c:66 #16 0x000000000044c3b8 in main (argc=8, argv=0x7fffffffe4d8) at apps/openssl.c:265
the callstack for params in spped command: #0 core_get_params (handle=0x70d4e0, params=0x7fffffffdbe0) at crypto/provider_core.c:1825 #1 0x00007ffff5c566d7 in fips_get_params_from_core (fgbl=0x70d090) at providers/fips/fipsprov.c:156 #2 0x00007ffff5c57523 in OSSL_provider_init_int (handle=0x70d4e0, in=0x7ffff78eeff0 <core_dispatch_+752>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fipsprov.c:686 #3 0x00007ffff5c560e5 in OSSL_provider_init (handle=0x70d4e0, in=0x7ffff78eed00 <core_dispatch_>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fips_entry.c:18 #4 0x00007ffff748298d in provider_init (prov=0x70d4e0) at crypto/provider_core.c:911 #5 0x00007ffff7482e8e in provider_activate (prov=0x70d4e0, lock=0, upcalls=1) at crypto/provider_core.c:1100 #6 0x00007ffff74830ef in ossl_provider_activate (prov=0x70d4e0, upcalls=1, aschild=0) at crypto/provider_core.c:1177 #7 0x00007ffff747f5a0 in OSSL_PROVIDER_try_load (libctx=0x0, name=0x7fffffffe7b4 "fips", retain_fallbacks=0) at crypto/provider.c:31 #8 0x00007ffff747f63c in OSSL_PROVIDER_load (libctx=0x0, name=0x7fffffffe7b4 "fips") at crypto/provider.c:50 #9 0x00000000004876cf in app_provider_load (libctx=0x0, provider_name=0x7fffffffe7b4 "fips") at apps/lib/app_provider.c:37 #10 0x000000000048783b in opt_provider (opt=1601) at apps/lib/app_provider.c:78 #11 0x0000000000473f89 in speed_main (argc=7, argv=0x7fffffffe4e0) at apps/speed.c:1589 #12 0x000000000044c95b in do_cmd (prog=0x707e40, argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:418 #13 0x000000000044c574 in main (argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:298

this result from provider_activate failed, maybe the store failed to set in case of fips provider,anyway, I will check further

the init_function was not setted，so failed in case of “prov->init_function == NULL”

OSSL_PROV_FIPS_PARAM_MODULE_MAC

Got it ,I regarded this as the MAC Algrithm,I will check further, thanks.

this have been checked , the openssl.cnf and the included fipsmodule.cnf was loaded in CONF_modules_load_file, but the params in speed command was get in core_get_params , these bellows to different libctx, so the speed command could not get the configured macdata in fipsmodule.cnf defaultly.
the callstack in CONF_modules_load_file: #0 provider_conf_params (prov=0x70e230, provinfo=0x0, name=0x0, value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:77 #1 0x00007ffff74808d9 in provider_conf_load (libctx=0x0, name=0x7089d0 "default", value=0x708a20 "default_sect", cnf=0x705420) at crypto/provider_conf.c:219 #2 0x00007ffff7480bf8 in provider_conf_init (md=0x70e0c0, cnf=0x705420) at crypto/provider_conf.c:293 #3 0x00007ffff737f699 in module_init (pmod=0x7055c0, name=0x708890 "providers", value=0x7088e0 "provider_sect", cnf=0x705420) at crypto/conf/conf_mod.c:375 #4 0x00007ffff737f1d6 in module_run (cnf=0x705420, name=0x708890 "providers", value=0x7088e0 "provider_sect", flags=32) at crypto/conf/conf_mod.c:239 #5 0x00007ffff737eefa in CONF_modules_load (cnf=0x705420, appname=0x0, flags=32) at crypto/conf/conf_mod.c:138 #6 0x00007ffff737f01b in CONF_modules_load_file_ex (libctx=0x0, filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:181 #7 0x00007ffff737f0bb in CONF_modules_load_file (filename=0x0, appname=0x0, flags=50) at crypto/conf/conf_mod.c:203 #8 0x00007ffff737fe7d in ossl_config_int (settings=0x0) at crypto/conf/conf_sap.c:64 #9 0x00007ffff74749f1 in ossl_init_config () at crypto/init.c:242 #10 0x00007ffff74749d6 in ossl_init_config_ossl_ () at crypto/init.c:240 #11 0x00007ffff7016a80 in pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:103 #12 0x00007ffff7485c1a in CRYPTO_THREAD_run_once (once=0x7ffff7906cfc , init=0x7ffff74749cd <ossl_init_config_ossl_>) at crypto/threads_pthread.c:144 #13 0x00007ffff7474ffd in OPENSSL_init_crypto (opts=30284, settings=0x0) at crypto/init.c:584 #14 0x00007ffff79403db in OPENSSL_init_ssl (opts=30284, settings=0x0) at ssl/ssl_init.c:127 #15 0x000000000044c2ca in apps_startup () at apps/openssl.c:66 #16 0x000000000044c3b8 in main (argc=8, argv=0x7fffffffe4d8) at apps/openssl.c:265
the callstack for params in spped command: #0 core_get_params (handle=0x70d4e0, params=0x7fffffffdbe0) at crypto/provider_core.c:1825 #1 0x00007ffff5c566d7 in fips_get_params_from_core (fgbl=0x70d090) at providers/fips/fipsprov.c:156 #2 0x00007ffff5c57523 in OSSL_provider_init_int (handle=0x70d4e0, in=0x7ffff78eeff0 <core_dispatch_+752>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fipsprov.c:686 #3 0x00007ffff5c560e5 in OSSL_provider_init (handle=0x70d4e0, in=0x7ffff78eed00 <core_dispatch_>, out=0x7fffffffde50, provctx=0x7fffffffde58) at providers/fips/fips_entry.c:18 #4 0x00007ffff748298d in provider_init (prov=0x70d4e0) at crypto/provider_core.c:911 #5 0x00007ffff7482e8e in provider_activate (prov=0x70d4e0, lock=0, upcalls=1) at crypto/provider_core.c:1100 #6 0x00007ffff74830ef in ossl_provider_activate (prov=0x70d4e0, upcalls=1, aschild=0) at crypto/provider_core.c:1177 #7 0x00007ffff747f5a0 in OSSL_PROVIDER_try_load (libctx=0x0, name=0x7fffffffe7b4 "fips", retain_fallbacks=0) at crypto/provider.c:31 #8 0x00007ffff747f63c in OSSL_PROVIDER_load (libctx=0x0, name=0x7fffffffe7b4 "fips") at crypto/provider.c:50 #9 0x00000000004876cf in app_provider_load (libctx=0x0, provider_name=0x7fffffffe7b4 "fips") at apps/lib/app_provider.c:37 #10 0x000000000048783b in opt_provider (opt=1601) at apps/lib/app_provider.c:78 #11 0x0000000000473f89 in speed_main (argc=7, argv=0x7fffffffe4e0) at apps/speed.c:1589 #12 0x000000000044c95b in do_cmd (prog=0x707e40, argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:418 #13 0x000000000044c574 in main (argc=7, argv=0x7fffffffe4e0) at apps/openssl.c:298

this result from provider_activate failed, maybe the store failed to set in case of fips provider,anyway, I will check further

the init_function was not setted，so failed in case of “prov->init_function == NULL”

the build macro FIPS_MODULE was not active for provider_predefined.c
gcc  -I. -Iinclude -Iproviders/common/include -Iproviders/implementations/include  -DAES_ASM -DBSAES_ASM -DCMLL_ASM -DECP_NISTZ256_ASM -DGHASH_ASM -DKECCAK1600_ASM -DMD5_ASM -DOPENSSL_BN_ASM_GF2m -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DPOLY1305_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DX25519_ASM -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O0 -g -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSLDIR=""/usr/local/ssl"" -DENGINESDIR=""/usr/local/lib64/engines-3"" -DMODULESDIR=""/usr/local/lib64/ossl-modules"" -DOPENSSL_BUILDING_OPENSSL -DNDEBUG  -MMD -MF crypto/libcrypto-lib-provider_predefined.d.tmp -MT crypto/libcrypto-lib-provider_predefined.o -c -o crypto/libcrypto-lib-provider_predefined.o crypto/provider_predefined.c

Configure with enable-fips should add that macro.  After building run /util/opensslwrap.sh fipsinstall -out f.cnf -module providers/fips.so and it will tell you if the module can be used (it runs the installation self tests using the module).

Configure with enable-fips should add that macro. After building run /util/opensslwrap.sh fipsinstall -out f.cnf -module providers/fips.so and it will tell you if the module can be used (it runs the installation self tests using the module).

Yes, enable-fips will add FIPS_MODULE in the fips providers,but not in the provider_predefined.c which define the default provider init, this result the fips failed to active in libctx, so the default configure could not work for the speed command.
I will check further with the Configure file.

Lets go back a step.  After building does the following work:
OPENSSL_CONF=test/fips.cnf OPENSSL_CONF_INCLUDE=providers ./util/opensslwrap.sh speed -provider-path providers -provider fips -evp aes-256-gcm
You can confirm that the FIPS provider is being used by changing aes-256-gcm to md5 which should immediate produce an error.

Lets go back a step. After building does the following work:
OPENSSL_CONF=test/fips.cnf OPENSSL_CONF_INCLUDE=providers ./util/opensslwrap.sh speed -provider-path providers -provider fips -evp aes-256-gcm
You can confirm that the FIPS provider is being used by changing aes-256-gcm to md5 which should immediate produce an error.

this working, thanks. opensslwrap.sh seems a new shell file in 3.0.0 ,  I will check what have done by opensslwrap.sh.

It's a script that sets up some environment variables (to locate shared libraries, providers etc in the build tree).  Better might be looking at util/wrap.pl which does the work.
It's not new but it does more now.

It's a script that sets up some environment variables (to locate shared libraries, providers etc in the build tree). Better might be looking at util/wrap.pl which does the work.
It's not new but it does more now.

Got it, I will check further, thanks

Running: util/wrap.pl env | egrep 'OSSL|OPENSSL' might be informative.

util/wrap.pl env | egrep 'OSSL|OPENSSL'
It shows that 4 environment variables setted , It working fine for "./util/opensslwrap.sh speed -provider-path providers -provider fips -evp aes-256-gcm", but the "missing config data" still reported with "./apps/openssl speed -evp aes-256-gcm -provider-path ./providers/ -provider fips"

environment variables:
OPENSSL_CONF_INCLUDE=providers
OPENSSL_ENGINES=util/../engines
OPENSSL_CONF=test/fips.cnf
OPENSSL_MODULES=util/../providers

util/wrap.pl env | egrep 'OSSL|OPENSSL'
It shows that 4 environment variables setted , It working fine for "./util/opensslwrap.sh speed -provider-path providers -provider fips -evp aes-256-gcm", but the "missing config data" still reported with "./apps/openssl speed -evp aes-256-gcm -provider-path ./providers/ -provider fips"

environment variables: OPENSSL_CONF_INCLUDE=providers OPENSSL_ENGINES=util/../engines OPENSSL_CONF=test/fips.cnf OPENSSL_MODULES=util/../providers

@paulidale After 4 environment variables setted, the result still  different between opensslwrap.sh and openssl, what result this difference?

I've no idea -- it will be something in your individual setup I suspect.

I've no idea -- it will be something in your individual setup I suspect.

Okay, I will check, thanks

the -provider fips failed load the fips library as the fips configure file did not setted by environment variables. and the speed command did not support setted by command. This result provider option could not working without environment variables.
Support config option will make the speed working without special environment variables.
@paulidale


ENH: Expose displacement mechanisms for dimer

i.e. be able to run single saddle point jobs without first going through the server and an AKMC job.

This was dropped somewhere along the line, although the Epicenters bit is in C++ (needs a rewrite); the problem is..
  if (params->saddleDisplaceType == EpiCenters::DISP_LOAD) {
    // displacement was passed from the server
    saddle->con2matter(displacementFilename);
  } else {
    // displacement and mode will be made on the client
    // in saddleSearch->initialize(...)
    *saddle = *initial;
  }

with similar lines in ProcessSearch as well; except there's no valid saddleSearch->initialize(...) to do this.

On the basis that this was probably supposed to work this is being upgraded to a BUG.


docs: Update README.md Angular version 17 -> 19

By the way, technically the photo is outdated since it shows "Using Angular v.17.0.1". But I don't think anyone will notice lol.

Thank you @michael-small !


fix: Prevent  CFunctionInfo/CTypeInfo leak in OpCtx

We now explicitly call drop_in_place on the allocations, which will invoke the C++ '_DELETE' method in the latest rusty_v8

I've rebased my pr now that #699 has been merged

I've rebased my pr now that #699 has been merged


Route bindings returning NULL (Laravel 5.4)

Hi there,
I'm trying to generate the doc for my api and I'm not receiving any route parameters in my code.
This is the command I'm using:
php artisan api:generate --routePrefix="api/v1/*" --noPostmanCollection --noResponseCalls --useMiddlewares --bindings="sub_account,3|account,2|customer,2"

When in my code I try to get the route parameters like: $request->route('customer') it returns NULL.
BUT, if I remove --noResponseCalls from the command, when it makes the requests the bindings work, but still it does not work in the second part of the generator and does not inject them
Am I doing something Wrong?

I've the same issue.
@cesuarez do you have solution?

@cesuarez @siarheipashkevich Do you still have this issue? We're trying to clean out any old bugs.


Update macOS SDK to 14

@SuperQ previously stated that we need to update the xcode stuff in golang-builder image:
https://github.com/prometheus/node_exporter/issues/2539#issuecomment-1868903208
So, I've attempted to do that. More below:
What I've done

[x] Read & understood Xcode license terms
[x] Extracted and used macOS SDK 14
[x] As I don't have access to PROM_OSX_SDK_URL secret, I used my local copy of macOS SDK (Not committed to repo) >> This will fail the CircleCI pipeline
[x] Bumped JOBS to 4 to make compilation faster
[x] Created new golang-builder image with updated macOS SDK, didn't change tagging (resulting tag was quay.io/prometheus/golang-builder:1.21.5-main-master) (Not yet committed to quay.io!)

Questions

Why don't we commit the SDK tar files to the repo?
How can I test the new image with CircleCI?
What are your thoughts about the license terms? (see my notes below)

Notes
Xcode license agreement says
You may use the macOS SDKs to test and develop application and other software;

and
Except as otherwise expressly set forth in Section 2.2.B., You may not distribute any Applications
developed using the Apple SDKs (excluding the macOS SDK) absent entering into a separate written
agreement with Apple

My interpretation is that we are able to develop, and distribute using macOS SDK.
But, the catch was right at the start:
THIS SOFTWARE IS AUTHORIZED ONLY FOR EXECUTION ON AN APPLE-BRANDED PRODUCT
RUNNING MACOS. ANY OTHER DOWNLOAD OR USE OF THIS SOFTWARE IS NOT AUTHORIZED
AND IS IN BREACH OF THIS AGREEMENT

In this case, I feel like we should stay in the safe side and build it on macOS machine, as CircleCI already offers those.
To do (My plan)

Feel free to comment!

[x] With help of maintainer, commit the SDK tar under the secret PROM_OSX_SDK_URL, or change the way SDK is injected into the docker image
[x] With help of maintainer, get CircleCI stuff sorted, maybe add a mac runner to be on the safe side
[x] Test the new image on CircleCI
[x] Clean up (Put JOBS back to what it was, and put PROM_OSX_SDK_URL stuff back to what it was if we won't change the injection method)

Feel free to add more comments and info as you see fit.
Thanks!

I think the issue is that we can't build a cross-compiling package that will work in our docker image without uploading and using the SDK as we're currently using it.

Indeed, so I was curious:

Is the current SDK package a CircleCI secret and is injected like that?

We'd need to upload the newer version with someone who has write access.

Updated 1.20 too and squashed.

I tested this manually on my system and it seems to build and run fine. I don't have an arm64 mac to test with.


Bound data value does not update as expected

Version
2.5.16
Reproduction link
https://codepen.io/experiencehq/pen/yEKMwv
Steps to reproduce

Create a new component, which expects a single property (in the example, color)
Create a data property, based on the bound property (data-color)
Create a computed property, based on the bound property (computed-color)
Note that the data-color and computed-color values are correct when the component is first loaded
Update the bound property value
Note that the computed-color value updates as expected, but the data-color value does not update

What is expected?
The data-color value should change when the bound property upon which it is based changes, as described in the documentation:

...every time the parent component is updated, all props in the child component will be refreshed with the latest value.

What is actually happening?
The data-color value does not change.

'data-color' will not be synchronized with 'prop-color'

The values specified in data is initial data, so it is expected the value will not be automatically updated. You should always use computed or manually update with watching props value.

@ktsn Thank you for the clarification. It may be worth updating the documentation to be more explicit regarding this limitation.


[gql][consistent-reads][6/n] Implement consistent reads for dynamic fields

Description
As title
Test Plan
dynamic_fields.move

If your changes are not user-facing and do not break anything, you can skip the following section. Otherwise, please briefly describe what has changed under the Release Notes section.
Type of Change (Check all that apply)

[ ] protocol change
[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes

oops .. must've messed up a rebase


[Suggestion] Overlay for recently updated chunks

This would be incredibly useful when debugging chunk update issues, you could have it render with the standard boundary view or have it separate, with the former simply adding a number to the yellow center line as to how many updates in the past 5 seconds would suffice.

I didn't find a way go monitor chunk updates. Someone an idea how I can monitor chunk updates, like an event or something?

Pretty sure forge has a chunk update event in the render pipe, but I haven't checked recently...


Custom Header X-Auth-Token not sent with IE11

I create POST requests with a custom header field X-Auth-Token as follows:
const post = (url) => {
  return request
    .post(url)
    .set('Content-Type', 'application/json')
    .set('Accept', 'application/json')
    .set('X-Auth-Token', _token)
    .timeout(TIMEOUT);
};

With Firefox, Edge, Chrome, ... the X-Auth-Token header is included in the request:
POST myurl HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0
Accept: application/json
Content-Type: application/json
X-Auth-Token:
[.....]

With IE11, the X-Auth-Token header is not included:
POST myurl HTTP/1.1
Content-Type: application/json
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; ASU2JS; rv:11.0) like Gecko
 ---- NO X-Auth-Token here

Try allowing that header with CORS:
https://developer.mozilla.org/en/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Headers

This was not enough. I included Access-Control-Allow-Headers="*" or the value of Access-Control-Request-Headers + ",X-Auth-Token" but it failed:

With "*", an error is shown in the console: content-type is not included in the response,
With the other (as shown below), the custom X-Auth-Token is simply not included in the request.

`OPTIONS /myurl HTTP/1.1
Accept: /
Origin: (myorigin)
Access-Control-Request-Method: POST
Access-Control-Request-Headers: content-type, accept
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; ASU2JS; rv:11.0) like Gecko
Host: (myhost)
Content-Length: 0
Connection: Keep-Alive
Cache-Control: no-cache
HTTP/1.1 200 OK
Date: Fri, 16 Jun 2017 17:14:30 GMT
Server: Apache
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: content-type, accept,X-Auth-Token
Allow: GET, HEAD, POST, PUT, DELETE, TRACE, OPTIONS, PATCH
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
POST /myurl HTTP/1.1
Content-Type: application/json
Accept: application/json
Referer: (myreferer)
Accept-Language: fr-FR
Origin: (myorigin)
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; ASU2JS; rv:11.0) like Gecko
Host: (myhost)
Content-Length: 48
Connection: Keep-Alive
Cache-Control: no-cache`

Responding with a full set of CORS headers, including Access-Control-Allow-Credentials (as described in https://www.webdavsystem.com/ajax/programming/cross_origin_requests) worked better but was not correct for all navigators.
I stopped investigating this point, as I resorted to proxying the back-end requests (avoiding the CORS issue), so I close this issue.


bugfix qtplot png representation

I haven't looked into the details, but this fixes the png representation on my machine. Probably a change in pyqtgraph?
@jenshnielsen @WilliamHPNielsen

@dpfranke, two questions: what problem does this PR solve? What versions of packages are you using?
When I play around with
PyQt5 (5.9)
pyqtgraph (0.10.0)
QtAwesome (0.4.4)
qtconsole (4.3.1)
QtPy (1.3.1)

I get a UnicodeDecodeError when using this PR, but a (seemingly) nice bytestring when using the current master version.

Ok, I get
AttributeError: 'QByteArray' object has no attribute '_getValue'
with the master version. This seems to be connected to the remote argument, working / failing example for IPython:
from qcodes.plots.pyqtgraph import QtPlot
plot = QtPlot(remote=False)
plot

@dpfranke, I see, thanks for clarifying. I think the way to go is then indeed as you suggest to only apply the fix from this PR if remote == False. That would at least be the quickfix.

couldn't find that option, now using hasattr
Could you check whether that work around works for you, @WilliamHPNielsen

do i understand correctly that this png/plotting functionality is not and cannot be automatically tested, hence the complaints about code coverage? If so, then let's merge, because I see that the solution works for both @dpfranke and @WilliamHPNielsen ?

@WilliamHPNielsen we use it as part of our (qt-based) data viewer / browser


Consistent suffix within context

Description
For all templates the suffix only contains the path of the URL. For Cloudflare it is the complete URL which is not consistent
Related Issue
N/A
Motivation and Context
Having the same functions deployed everywhere and the context is consistent.
How Has This Been Tested?
local deployment & testing
Screenshots (if appropriate):
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Looks good, but could you change the commit history so that at least one commit reads fix(cloudflare): suffix should not include full URL
We generate the release notes from the commit messages, so this is more than cosmetic.


Trying to upload while another upload is incomplete gives a blank page with "The page was not displayed because there was a conflict."

Tab A go to https://preview.nuget.org/packages/upload
Tab B go to https://preview.nuget.org/packages/upload
Tab A, upload a package, don't verify/cancel
Tab B, Upload a package

Blank page with text :
"The page was not displayed because there was a conflict."
Low pri but the blank page is not so nice, could just redirect to the first package to finish.

I think this is solved - at least I now receive a clear information, that a upload can be canceled or submitted or that I can upload a new package.

Yeah, this is a prehistoric issue from 2011...the UI has changed a loooot since then and now properly handles uploading a different package before you finish an upload.


Use ES6 in webpack prod config

It is confusing to see the dev config in ES6, like all the code and then the prod config in ES5.
Any particular reason for that?

Yes: I couldn't find a way to pass to the webpack bin (used by the npm run build script) a config written in ES6. On dev we don't use the bin so we require the config in a node.js script, after babel/register hook has been installed. Anyway it's a good point, going to ask on the webpack's gitter chat...

What about doing something similar to here?
https://github.com/gpbl/isomorphic500/blob/31e91c650c4b99d91424ecbef6e13531ab50e9a1/src/config.js
and load babel/register in this config?

Yes I was thinking to that option, but perhaps keeping the config (and /webpack/utils) in ES5 would be a better compromise.

Yeah. Maybe just less tricky and it will be more consistent that having one
config in ES6 and another in ES5.
On 26 May 2015 at 21:33, Giampaolo Bellavite<EMAIL_ADDRESS>wrote:

Yes I was thinking to that option, but perhaps keeping the config (and
/webpack/utils) in ES5 would be a better compromise.
—
Reply to this email directly or view it on GitHub
https://github.com/gpbl/isomorphic500/issues/49#issuecomment-105656758.


[question] EvalCallback using MPI

When running a training loop using MPI, the EvalCallback doesn't seem to make use of the parallelisation:
for example, in this train function:
https://github.com/hardmaru/slimevolleygym/blob/master/training_scripts/train_ppo_mpi.py
it seems that the EvalCallback will be called once per instance, after a combined total of eval_freq timesteps across all of the instances. This appears to be problematic if you want to use the callback decide whether to save out a new best model, as there will be multiple attempts at calculating the average reward and therefore the best_model file will be overwritten potentially several times on the same update. The best score will be naturally inflated the more instances you have, due to some reward calculations being slightly higher than average due to favourable random fluctuations.
Is this correct?
If so, is there a way to instead split the n_eval_episodes across the workers and aggregate into a single score?

Hello,

When running a training loop using MPI, the EvalCallback doesn't seem to make use of the parallelisation:

yes, the EvalCallback does not support MPI parallelization.
I would recommend you to try to switch to the VecEnv version of PPO (PPO2) if this is possible.
And even to switch to Stable-Baselines3 ;) : https://github.com/DLR-RM/stable-baselines3

If so, is there a way to instead split the n_eval_episodes across the workers and aggregate into a single score?

this is possible but non-trivial and would require you to define a custom callback (cf doc).

Hello,

When running a training loop using MPI, the EvalCallback doesn't seem to make use of the parallelisation:

yes, the EvalCallback does not support MPI parallelization.
I would recommend you to try to switch to the VecEnv version of PPO (PPO2) if this is possible.
And even to switch to Stable-Baselines3 ;) : https://github.com/DLR-RM/stable-baselines3

If so, is there a way to instead split the n_eval_episodes across the workers and aggregate into a single score?

this is possible but non-trivial and would require you to define a custom callback (cf doc).

Hi @araffin - thanks for the reply!
Does VecEnv parallelise the gradient computation, or just the env part?
https://twitter.com/hardmaru/status/1260852988475658242
I've got PPO + MPI working really well on a multicore machine with a custom callback to handle the parallelisation of the evaluation. I'm also hesitant to switch to SB3 as it doesn't support Tensorflow, which is a shame.
Thanks for your help!

Hi @araffin - thanks for the reply!
Does VecEnv parallelise the gradient computation, or just the env part?
https://twitter.com/hardmaru/status/1260852988475658242
I've got PPO + MPI working really well on a multicore machine with a custom callback to handle the parallelisation of the evaluation. I'm also hesitant to switch to SB3 as it doesn't support Tensorflow, which is a shame.
Thanks for your help!

Does VecEnv parallelise the gradient computation, or just the env part?

Just the env part.

I've got PPO + MPI working really well on a multicore machine with a custom callback to handle the parallelisation of the evaluation.

SB3 do not support MPI by default, but we would be happy to have an implementation of PPO MPI in our contrib repo ;)
See https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/issues/11

I'm also hesitant to switch to SB3 as it doesn't support Tensorflow, which is a shame.

The decision to move to PyTorch and drop MPI (for the default install) was not arbitrary, see https://github.com/hill-a/stable-baselines/issues/733 and https://github.com/hill-a/stable-baselines/issues/366 ;)

Does VecEnv parallelise the gradient computation, or just the env part?

Just the env part.

I've got PPO + MPI working really well on a multicore machine with a custom callback to handle the parallelisation of the evaluation.

SB3 do not support MPI by default, but we would be happy to have an implementation of PPO MPI in our contrib repo ;)
See https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/issues/11

I'm also hesitant to switch to SB3 as it doesn't support Tensorflow, which is a shame.

The decision to move to PyTorch and drop MPI (for the default install) was not arbitrary, see https://github.com/hill-a/stable-baselines/issues/733 and https://github.com/hill-a/stable-baselines/issues/366 ;)


Bridge gets stuck using ph4 (at least) in GPU mode

Describe the bug
Using Bridge to connect ph4 and Fi.  When ph4 is in CPU mode, everything runs.  When ph4 is in GPU mode, the code stops on or after the first evolution of the individual codes that the bridge is connecting.
To Reproduce
I've attached gravhydrotest.py.  Change mode="gpu" to mode="cpu" in the ph4 instantiation, and the code runs through to the end.  With mode="gpu", the code hangs up after the first kick/drift (see gravhydrotest_gpu.log).
Expected behavior
I expect the bridge to function when ph4 is in GPU mode just as it does when ph4 is in CPU mode.
Logs
I've attached two files.  gravhydrotest_cpu.log is stdout with mode="cpu" on ph4, and gravhydrotest_gpu.log is stdout with mode="gpu" on ph4.  The cpu version ran in a second or so.  I interrupted the gpu version after about a minute.
Environment (please complete the following information):

OS and version: Devuan GNU/Linux ascii
Compiler: gcc 6.3, python 3.5

Additional context
GPU is GeForce GTX TITAN Z
gravhydrotest.py.zip
gravhydrotest_cpu.log
gravhydrotest_gpu.log

@rknop are you still experiencing this?
Unfortunately I don't have a Sapporo-compatible GPU to test this with myself...


⚠ Samsung has degraded performance

In e951426, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2921 ms

Resolved: Samsung performance has improved in ec34ae0.


azure-sdk: build broken after exposing vendor/ directory

This was observed after exposing the vendor/ directory in v26.3.0 via b99be020a281cae39761deadb8672d22f44b89eb
Problem
Apparently due to dep problems, the vendor directory has been exposed in b99be020a281cae39761deadb8672d22f44b89eb
This causes problems with builds that import github.com/Azure/go-autorest/autorest/... manually, since the github.com/Azure/azure-sdk-for-go/ will use its vendored go-autorest package, whereas
the calling code pulls in the non-vendored one.
The result are build errors like:
sh> make
...
# azureaction
src/azureaction/session.go:100:23: cannot use &client.BaseClient.Client (type *"github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest".Client) as type *"github.com/Azure/go-autorest/autorest".Client in argument to s.populateBaseClient
src/azureaction/session.go:108:23: cannot use &client.BaseClient.Client (type *"github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest".Client) as type *"github.com/Azure/go-autorest/autorest".Client in argument to s.populateBaseClient
src/azureaction/session.go:117:23: cannot use &client.BaseClient.Client (type *"github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest".Client) as type *"github.com/Azure/go-autorest/autorest".Client in argument to s.populateBaseClient
...

Our import statement has this use case of having to refer to autorest separately:
import (
        "aggregates/site"
        "net/http"
        // ...

        "github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2018-04-01/compute"     // KEEP THIS VERSION
        "github.com/Azure/azure-sdk-for-go/services/network/mgmt/2018-01-01/network"     // KEEP THIS VERSION
        "github.com/Azure/azure-sdk-for-go/services/resources/mgmt/2016-09-01/locks"     // KEEP THIS VERSION
        "github.com/Azure/azure-sdk-for-go/services/resources/mgmt/2018-05-01/resources" // KEEP THIS VERSION
        "github.com/Azure/azure-sdk-for-go/services/storage/mgmt/2017-10-01/storage"     // KEEP THIS VERSION
        "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest"
        "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest/adal"
        "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest/azure"
        "github.com/pkg/errors"
)

(The KEEP THIS VERSION comment is internal, these API versions have worked best for us.). We are not (yet) using dep or a vendoring tool. Trusting upstream and writing issues like this one when things break :-)
There is a catch-22 here: when I try to outsmart go build/get by changing the import paths like this:
import (
    // ...
   "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest"
   "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest/adal"
   "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest/azure"
)

I get build errors like that:
package github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest: must be imported as github.com/Azure/go-autorest/autorest
package github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest/adal: must be imported as github.com/Azure/go-autorest/autorest/adal
package github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest/azure: must be imported as github.com/Azure/go-autorest/autorest/azure

How to reproduce
Take the example program shown on the azure-sdk github page and try to compile it using go build ./test_program.go.
It results in:
sh> go build ./c.go
# command-line-arguments
./c.go:18:25: cannot use authorizer (type "github.com/Azure/go-autorest/autorest".Authorizer) as type "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest".Authorizer in assignment:
        "github.com/Azure/go-autorest/autorest".Authorizer does not implement "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest".Authorizer (wrong type for WithAuthorization method)
                have WithAuthorization() "github.com/Azure/go-autorest/autorest".PrepareDecorator
                want WithAuthorization() "github.com/Azure/azure-sdk-for-go/vendor/github.com/Azure/go-autorest/autorest".PrepareDecorator

Dupe of https://github.com/Azure/azure-sdk-for-go/issues/4262.  Should have a fix later today, sorry for the hassle.


[BEAM-8807] Add integration test for SnsIO.write (Sdk v1 & v2)

Add integration test for SnsIO.write (Sdk v1 & v2)
Also fixes a bug in BasicSnsProvider (Sdk v1) which didn't set the service endpoint.
For clarity, obsolete implementations to create CloudWatch clients are deprecated where not needed.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Java PostCommit

@aromanenko-dev ✅ should be good


Discrepancy in output of torchvision.io.read_image vs PIL.Image

🐛 Describe the bug
Some images from the imagenetv2 dataset (downloadable here) contain nonzero differences when loaded using torchvision.io.read_image, with some images containing large differences in pixel values.
import torch
import torchvision.io
import numpy as np
from PIL import Image

def loadimage_pil(path):
    return torch.tensor(np.array(Image.open(path).convert("RGB"))).permute(2, 0, 1)

def loadimage_torchio(path):
    return torchvision.io.read_image(path, torchvision.io.ImageReadMode.RGB)

# assuming archive is unpacked in the same folder as script - change accordingly.
filepath = "./imagenetv2-matched-frequency-format-val/455/aaaf43c110a10aabce09700a6a3cfb2622b4847a.jpeg"
print(f"loading '{filepath}'")
img_pil = loadimage_pil(filepath)
img_tio = loadimage_torchio(filepath)
difference = img_pil.to(float) - img_tio.to(float)

error = torch.sqrt(torch.mean(torch.square(difference)))
print(error)
# > tensor(6.3985, dtype=torch.float64)

When loading the file used in the example imagenetv2-matched-frequency-format-val/455/aaaf43c110a10aabce09700a6a3cfb2622b4847a.jpeg the printed error value is 6.3985.
Versions
Collecting environment information...
PyTorch version: 2.1.0
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Fedora Linux 36 (Thirty Six) (x86_64)
GCC version: (GCC) 12.2.1 20220819 (Red Hat 12.2.1-2)
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:26:04) [GCC 10.4.0] (64-bit runtime)
Python platform: Linux-6.0.8-200.fc36.x86_64-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
<snip>

Nvidia driver version: 520.56.06
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==1.2.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.4
[pip3] torch==2.1.0
[pip3] torch-tb-profiler==0.4.3
[pip3] torchaudio==2.1.0
[pip3] torchinfo==1.8.0
[pip3] torchvision==0.16.0
[pip3] triton==2.1.0
[conda] blas                      1.0                         mkl    conda-forge
[conda] libblas                   3.9.0            16_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            16_linux64_mkl    conda-forge
[conda] libjpeg-turbo             2.0.0                h9bf148f_0    pytorch
[conda] liblapack                 3.9.0            16_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            16_linux64_mkl    conda-forge
[conda] mkl                       2022.1.0           h84fe81f_915    conda-forge
[conda] numpy                     1.24.4                   pypi_0    pypi
[conda] pytorch                   2.1.0           py3.10_cuda11.8_cudnn8.7.0_0    pytorch
[conda] pytorch-cuda              11.8                 h7e8668a_5    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torch-tb-profiler         0.4.3                    pypi_0    pypi
[conda] torchaudio                2.1.0               py310_cu118    pytorch
[conda] torchinfo                 1.8.0              pyhd8ed1ab_0    conda-forge
[conda] torchtriton               2.1.0                     py310    pytorch
[conda] torchvision               0.16.0              py310_cu118    pytorch

Hi @8uurg , I guess these come from diverging version of libjpeg[-turbo].
Could you please share the PIL version, as well as the output of ldd _imaging.so (from https://stackoverflow.com/a/24397115)
Could you also please share the output of
import torch
import torchvision

print(f"{torch.ops.image._jpeg_version() = }")
print(f"{torch.ops.image._is_compiled_against_turbo() = }")

Hi @8uurg , I guess these come from diverging version of libjpeg[-turbo].
Could you please share the PIL version, as well as the output of ldd _imaging.so (from https://stackoverflow.com/a/24397115)

PIL.__version__ = '9.3.0'

ldd <path-to-virtualenv>/lib/python3.10/site-packages/PIL/_imaging.cpython-310-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007fff6d3eb000)
        libjpeg.so.9 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../../libjpeg.so.9 (0x00007f1b34cad000)
        libz.so.1 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../../libz.so.1 (0x00007f1b34c93000)
        libtiff.so.5 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../../libtiff.so.5 (0x00007f1b34c06000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f1b34800000)
        libwebp.so.7 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../.././libwebp.so.7 (0x00007f1b34b71000)
        libzstd.so.1 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../.././libzstd.so.1 (0x00007f1b34a9f000)
        liblzma.so.5 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../.././liblzma.so.5 (0x00007f1b34a76000)
        libLerc.so => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../.././libLerc.so (0x00007f1b34764000)
        libdeflate.so.0 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../.././libdeflate.so.0 (0x00007f1b34a66000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f1b34686000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1b34d6d000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1b34a5f000)
        libstdc++.so.6 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../../././libstdc++.so.6 (0x00007f1b344d2000)
        libgcc_s.so.1 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../../././libgcc_s.so.1 (0x00007f1b34a46000)

Could you also please share the output of
import torch
import torchvision

print(f"{torch.ops.image._jpeg_version() = }")
print(f"{torch.ops.image._is_compiled_against_turbo() = }")

torch.ops.image._jpeg_version() = 80
torch.ops.image._is_compiled_against_turbo() = True

Thanks for the output

    libjpeg.so.9 => <path-to-virtualenv>/lib/python3.10/site-packages/PIL/../../../libjpeg.so.9 (0x00007f1b34cad000)

I think that's it: PIL is relying on libjpeg while torchvision is relying on libjpeg-turbo. They're both jpeg-compliant and, from past experiments, models aren't sensitive to these decoding differences. I think that if you installed PIL 10, you'd get turbo for PIL as well, and have results that are closer to the torchvision ones.

Thanks for the information!
I was investigating this because there was a small change in the validation performance of a model after changing how the images were loaded. The difference was not too big (single sample got changed, I think), but I was expecting the result to be identical. For those following in my footsteps: when installed through conda in my environment, Pillow 10 doesn't seem to link libjpeg-turbo.
When I install Pillow via pip, I can confirm that it is indeed a difference between libjpeg and libjpeg-turbo


Fixed odd "bump" on closing sidebar menu

Fixed odd "bump" on closing sidebar menu
Example slowed down to 10%:

Hmm sorry I'm getting to this so late @promatik . Thanks a lot for taking the time to put together the PR - much appreciated. But 😄 I remember we've already fixed this - most likely without having seen your PR, I'm sorry:

You can also see it's fixed in the online demo. So I'm going to close the PR without merging it. Sorry I didn't see this, it must have slipped through the cracks.
Cheers!

Oups - no "the bump" wasn't fixed, sorry. It was the container width. Similar, but not the same thing.
In order to have this fixed I've had to:

[x] make the change in the Backstrap theme, inside the src/scss/_custom.scss file (but src, not dist);
[x] build & tag a new version of Backstrap;
[x] require that new version of Backstrap inside Backpack;
[x] use Laravel Mix to update and build the CSS bundle; in the process, of course NPM has some fucking weird error - it has one every fucking week/month; I had to reinstall node-gyp; to do that, I had to download and reinstall XCode; which took 15-20 minutes; this gave me plenty of time to write this in anger;
[x] tag a new release of Backpack;

Which stands to show how un-fucking-necessarily complicated our lives have become, as web developers, because of the "modern front-end".  Line. Incredible.
Anyway. I've done all that. The bug is fixed on Backstrap.net. And it's fixed in Backpack's online demo.
To get the fix, one has to do:
composer update backpack/crud
php artisan vendor:publish --provider="Backpack\CRUD\BackpackServiceProvider" --tag="public"

New installs get it by default.
Thanks again @promatik for raising this, and providing the fix. Since it needed to be done inside a different repo, it'll look like closed, but it's actually "merged". Sorry for the rant - it just stands to show that our decision was a good one, to shield developers from having to do all this. That's it's actually a good thing the Backpack does not require developers to use NPM at all.
Cheers!

Hi @tabacitu!
I'm glad this is fixed despite the work you had 😅 well done!
I noticed the change without the publishing part, since I created a symlink from public\packages to vendor\backpack\crud\src\public\packages. I always have the new stuff...
New backpack is great by the way 👌


yahoo api issues

per a bunch of threads on the yahoo forums, the yahoo api is having a lot of issues.  commenters suggest using OAuth 2, but I believe that I'm already hitting that endpoint?

OrderedDict([(u'yahoo:error', OrderedDict([(u'@xmlns:yahoo', u'http://yahooapis.com/v1/base.rng'), (u'@xml:lang', u'en-US'), (u'yahoo:description', u'Internal server error')]))])

Ok, so the issue here was that Yahoo killed support for OAuth 1.0.  This script used OAuth 1 flow (the 'v2' was yahoo's 2nd implementation, not OAuth2).
Moved everything over to OAuth2 using @josuebrunel's [yahoo_oauth] library and nuked like 50 lines of code.  Thanks, Josue!
Also had to move all of the urls to https.

Ok, so the issue here was that Yahoo killed support for OAuth 1.0. This script used OAuth 1 flow (the 'v2' was yahoo's 2nd implementation, not OAuth2).

That's good to know.

Moved everything over to OAuth2 using @josuebrunel's [yahoo_oauth] library and nuked like 50 lines of code. Thanks, Josue!

@almartin82 you're welcome buddy.


German translation

a new try for pr the translation as i messed it up last time.

Thank you for your contribution.
I will update the description on the web stores later.


Feat request: Submenu in contextmenu

Add support for adding submenu in contextmenu

This is underway. It should be available soon.

That is available in the most recent version. We are working to update the documentation now.

The documentation will be released later today, including those features.


detekt config looks for a  detekt.yml in root always.

Describe what happened
detekt config looks for a detekt.yml in root always.  While publishing the sdk, do we really need detekt? And if we do need it, can we add an option to specify an alternate path for the config file, if the detekt.yml is not maintained in the root of the consumer project?
Steps to reproduce the issue:
Any project using the sdk where the detekt.yml is in any location other than project root will throw an error while running detekt check.
Describe what you expected:
An option to provide a custom path to the config file or remove the detekt before publishing.
Additional context
This is the code from v1.1.5

detekt {
  input = files("$projectDir/src/main/kotlin")
  config = files("${project.rootDir}/detekt.yml")

  reports {
    xml {
      enabled = true
      destination = file("build/reports/detekt.xml")
    }
  }
}

Hi @zsfVishnu, thanks for reaching out!
I agree that this could be improved, and even if most RN projects don't use detekt, we could give more flexibility for projects using it.
To help us understand better how to solve this, could you specify your use case?
Why do you have a detect.yml in your React Native project and where is it located?
Thanks!

Hey @louiszawadzki , our project has native code across different modules inside the android folder of our react native project. For similar projects, the detekt.yml is usually kept in the root of the android folder which gets picked up when detekt is run. For various purposes including saving time on our build, we run detekt on specifc modules and provide their respective detekt config files within that module and not the root. I do agree that this might not be a regular use case and so i was thinking we could add a capability to optionally run datadog:detekt or provide an alternate path.

I am also facing the same issue on RN 0.68.2

@karankalsi can you confirm whether your detekt.yml is in the project root or not?

@zsfVishnu I implemented a fix using our detekt config if none is detected at the root of the project - ensuring it will always pass (see link just above)
Would this work for you?

@louiszawadzki This looks good .

an you confirm whether your detekt.yml is in the project root or not?

No detekt.yml is not there in the project root.


Cannot See Monitor Settings or add to Energy Dashboard

Configuration
Hi there,
New GEM owner here with a new unit, just installed. First thanks for the integrations, really appreciate how well everything works together. I started with the native integration before seeing this one on HACS, so I let it migrate my YAML config and then that portion from my configuration.yaml. So far so good, the monitors came up and data was coming in. I enabled all sensors and am excluding them all from Recorder, so they just hit Influx.
Describe the bug
I noticed two issues but wonder if they're related:

none of the kWH sensors are listed when I try to add them to Grid Consumption on the energy dashboard
When I click "configure" and "edit per-monitor options" I cannot enter the serial number of my GEM and there is nothing listed in the drop down. See attached screenshot.

Also was wondering if maybe the migration from the built-in integration had a hiccup - so would removing and reconfiguring from scratch be worthwhile? Happy to help test whatever I can help with.
Debug log

I turned on debugging but didn't see anything of interest in the logs thus far:

2023-06-06 16:42:52.247 INFO (MainThread) [greeneye.monitor] Closing server on ('<IP_ADDRESS>', 7997)
2023-06-06 16:42:52.251 DEBUG (MainThread) [greeneye.monitor] queue consumer is getting canceled
2023-06-06 16:42:52.253 DEBUG (MainThread) [greeneye.monitor] Packet processor started
2023-06-06 16:42:52.280 INFO (MainThread) [greeneye.monitor] Server started on <IP_ADDRESS>:7998
2023-06-06 16:42:57.353 INFO (MainThread) [greeneye.monitor] Discovered new monitor: 1234567
2023-06-06 16:42:57.523 INFO (MainThread) [greeneye.monitor] Configured 1234567 from settings API call.
2023-06-06 16:42:57.531 INFO (MainThread) [custom_components.greeneye_monitor.sensor] Set up sensors for new monitor 1234567
2023-06-06 16:42:57.532 INFO (MainThread) [custom_components.greeneye_monitor.number] Added configuration entities for new monitor 1234567

Thanks for the report!
Hmmm, yeah, that is weird. The serial numbers for that dropdown are pulled directly from the config entry for the integration, which would suggest that it didn't get populated correctly. Yet, the logs you show look like everything is working normally. Is that dropdown still empty if you go to it now? (FWIW, the only per-monitor options have to do with the time unit used by pulse counters to report their rate.)
For troubleshooting that part, it would be interesting to see both your old YAML config and the contents of .storage/core.config_entries, specifically the entry with "domain": "greeneye_monitor".
For the energy dashboard thing, are they still not showing up? It can take a little while for Home Assistant to recognize a new statistics-enabled entity.  If they're still not appearing as options in the energy dashboard config, a look at their entries in .storage/entity_registry would be interesting.
Setting up from scratch with the HACS integration is worth a shot if things haven't sorted themselves out by now, though I'd love to see the YAML and the config entry that it created first, in case this is an issue with the migration code.

Ah, that may explain it. I don't have any pulse counters so nothing was enabled in my legacy yaml config, though it did create them. So not sure what I should be seeing here. I don't see anything odd in the files you mentioned but admittedly I'm very new to home assistant and python in general and still consider myself a beginner.
Given all that the only thing that gave me pause was in my legacy yaml config my serial number was entered with a leading zero but the integration created items do not, but I'm not sure that's germane as it might be just stripped out.
I still cannot add anything from the integration to the Energy Dashboard, they still are not showing up.
Here's the info you requested, thanks again. If nothing strikes you as being conclusive I can do a uninstall and reconfig, that seems like the next thing to try.
Original YAML
'''yaml
# GEM
greeneye_monitor:
port: 7997
monitors:
- serial_number: "01234567"
channels:
- number: 1
name: gem_total_power
net_metering: false
#        - number: 2
#          name: CHANNEL2
#          net_metering: false
- number: 3
name: gem_radiant_floor
net_metering: false
- number: 4
name: gem_kitchen_inside_wall
net_metering: false
- number: 5
name: gem_air_conditioner
net_metering: false
- number: 6
name: gem_furnace
net_metering: false
- number: 7
name: gem_whirlpool_tub
net_metering: false
- number: 8
name: gem_garage_north
net_metering: false
- number: 9
name: gem_washer_servers
net_metering: false
- number: 10
name: gem_kitchen_side_back
net_metering: false
- number: 11
name: gem_central_fan_bath_sonos_hall
net_metering: false
- number: 12
name: gem_guest_room_attic
net_metering: false
- number: 13
name: gem_dishwasher_garbage_disposal
net_metering: false
- number: 14
name: gem_family_room
net_metering: false
- number: 15
name: gem_garage_240_20
net_metering: false
- number: 16
name: gem_front_path_3D_printer
net_metering: false
- number: 17
name: gem_garage_240_50
net_metering: false
- number: 18
name: gem_kitchen_under_stairs
net_metering: false
- number: 19
name: gem_garage_south_deck
net_metering: false
- number: 20
name: gem_garage_outlets
net_metering: false
- number: 21
name: gem_basement_cold_storage
net_metering: false
- number: 22
name: gem_range
net_metering: false
- number: 23
name: gem_water_heater
net_metering: false
- number: 24
name: gem_dryer
net_metering: false
- number: 25
name: gem_sump_pump_kitchen_fan
net_metering: false
- number: 26
name: gem_main_bedroom_upstairs_baths
net_metering: false
- number: 27
name: gem_kitchen_dining_lights
net_metering: false
#        - number: 28
#          name:
#          net_metering: false
#        - number: 29
#          name:
#          net_metering: false
#        - number: 30
#          name:
#          net_metering: false
#        - number: 31
#          name:
#          net_metering: false
#        - number: 32
#          name:
#          net_metering: false
\      #pulse_counters:
\      #  - number: 1
\      #    name: sprinklers_water_usage
\      #    counted_quantity: "gal"
\      #    counted_quantity_per_pulse: 1
\      #    time_unit: "min"
\     # temperature_sensors:
\     #   temperature_unit: "C"
\     #   sensors:
\     #     - number: 1
\     #       name: back_porch_temperature
\      voltage:
\        - number: 1
\          name: gem_house_volts
'''
I've attached the relevant sections from the other two files you mentioned, hoping there's something there.
Config_Entries_GEM.txt
Entity_Registry_GEM.txt

Okay yeah, the dropdown is empty because there are no pulse counters configured. I filed #24 to clean that up. I think that may be a quirk of the YAML migration, but by itself that's not a reason to re-setup from scratch.
Your config entries and registry both look fine, so I'm a little mystified as to what's going on with the energy dashboard. Are you seeing any sensors at all in the list? Note that the names you had in your YAML will only apply to the power sensors; the energy sensors will have names like sensor.gem_1234567_channel_1_energy until you go in and rename them in the UI.

I have about seven z-wave plug-in modules from Aeotec (Smart Switch 7 B plugs) that I see as available inputs in the energy dashboard for grid consumption and individual devices. I am tracking water consumption from a Moen Flo that appears to be working, so from what I can see it appears everything is copacetic on that front.

I now have a drop-down with my GEM serial number when I do edit per-monitor options, even though I don't have any pulse counters configured I can still pull them up. So it looks like the reconfiguration did help with something.
I'll give it a few hours to see if I can add channels to the Energy Dashboard.

Did the energy dashboard start working after this?

After leaving it alone for a few days, no change. None of the sensors are listed as choices to add as grid or individual consumption.Multiple Z-Wave Plugs do show up however, along with a virtual meter on a v3 SmartThings Hub tied in through the native integration. I will try to pull new logs today when I time if that will be useful.
Dropdown for per monitor is still working fine even without any pulse counters so the from scratch reload did seem to take care of that piece.

Here are the files, had a few other projects call for attention. Let me know if there's anything else I can provide, from what I can tell other than the nothing on the GEM showing up as available consumption on the energy dashboard everything else appears to be working great. Happy to assist though I'm very much still learning python.
core.entity_registry_gem_19Jun2023.txt
core.config_entries_gem_19Jun2023.txt

I had a thought about this. You started with the YAML setup, right? Did you also do the part about setting up InfluxDB and excluding GEM sensors from recorder? If so, you'll want to un-exclude the energy sensors that you want to use with the energy dashboard. E.g. here's what my recorder config looks like in the YAML; I exclude all of the sensors by default, and then add back the ones I care about:
# Use MariaDB instead of SQLite for basic state logging (much faster)
recorder:
  db_url: !secret mariadb_url
  exclude:
    domains:
      - sensor
  include:
    entities:
      - sensor.total_current_energy
      - sensor.solar_panels_current_energy
      - sensor.sprinklers_water_usage_count

I've just pushed up a new version that checks for energy monitor entities being excluded from logbook/recorder and displays a repair in Settings. Let me know if that warning appears for your case!

Thanks, been a bit tied up here with some home repairs but I appreciate the follow-up, and I think that's the issue. I did have the sensors disabled per the old integration. I just turned them on, I'll give it a few hours to filter in, I do see history now for the GEM devices so that's looking good. I also see the new version, I will also update and restart. After it has some time to settle I'll report back, but I'm feeling like you have gotten to the root of it, thanks.

Data (and energy meters) are now flowing into the Dashboard, thanks again for the insight, and of course the integration.


ValueError: There is no such driver by url

Hi, i just fork the repo and runned on github, but got this error:
ValueError: There is no such driver by url https://chromedriver.storage.googleapis.com/105.0.5177/chromedriver_linux64.zip

Ya please solve it we don't know whats the actual issue

Hooray, it's now fixed!
The issue was that Ubuntu comes with a version of Chromium – 105.0.5177 – that doesn't have a Selenium ChromeDriver. I edited scrape.yml and added the following lines:
- name: Remove default Chromium
   run: sudo apt purge chromium-browser
- name: Install a new Chromium
   run: sudo apt install -y chromium-browser

By uninstalling the default Chronium and installing a new one it somehow magically works! I don't know if this is the best long-term solution, but it seems fine for now.


[🐛] app crashes when calling launchImageLibrary on android

Description
hey there, I'm using the package on android ,the issue is
the package nam in PackageList.java file is undefined.ImagePickerPackage in version 5.6.0
so I downgraded to 5.4.2 but after picking image from image library, app crashes with no error
I guess it's something with newer sdk and android versions, could someone give me a workarounnd?
my emulator android version is 11
and react native version is 0.66.4
How to repeat issue and example
try to select image from library with the launchImageLibrary({mediaType: 'photo'}) function
Solution
it was okay before, not sure why this happens
Additional Information

Image Picker version: 5.4.0 and 5.4.2
React Native version: 0.66.4
Platform: Android
Development Operating System: macOS
Dev tools: VSCode

If android app crashes with no error:

open Android Studio
open Logcat
select your device in Logcat
reproduce the error
find error log in Logcat

If logs in Logcat won't help you much, post error here and maybe I could help you out

thanks I cheked the logcat and found out it was something with onActivityResult that was modified and caused the crash, I commented the added codes and it fixed the problem

closed as the problem fixed

onActivityResult

can you share how to fix it?


Update Protocol Parameters Fix

Changelog
- description: |
    `cardano-cli conway governance action create-protocol-parameters-update` was writing the update directly to disk instead of putting it in a `Proposal`
# uncomment types applicable to the change:
  type:
  - feature        # introduces a new feature
  # - breaking       # the API has changed in a breaking way
  - compatible     # the API has changed but is non-breaking
  # - optimisation   # measurable performance improvements
  # - improvement    # QoL changes e.g. refactoring
  # - bugfix         # fixes a defect
  # - test           # fixes/modifies tests
  # - maintenance    # not directly related to the code
  # - release        # related to a new release preparation
  # - documentation  # change in code docs, haddocks...

Context
Additional context for the PR goes here.
If the PR fixes a particular issue please provide a
link
to the issue.
Checklist

[ ] Commit sequence broadly makes sense and commits have useful messages
[ ] The change log section in the PR description has been filled in
[ ] New tests are added if needed and existing tests are updated.  These may include:

golden tests
property tests
round trip tests
integration tests
See Running tests for more details

[ ] The version bounds in .cabal files are updated
[ ] CI passes. See note on CI.  The following CI checks are required:

[ ] Code is linted with hlint.  See .github/workflows/check-hlint.yml to get the hlint version
[ ] Code is formatted with stylish-haskell.  See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
[ ] Code builds on Linux, MacOS and Windows for ghc-8.10.7 and ghc-9.2.7

[ ] Self-reviewed the diff

It would make sense to remove these options for a protocol parameter change:
--protocol-major-version  \
--protocol-minor-version \

major version change is used when triggering a hardfork and not part of the regular parameter changes.
From CIP1694

Separation of Hard Fork Initiation from Standard Protocol Parameter Changes
In contrast to other protocol parameter updates, hard forks (or, more correctly, changes to the protocol's major version number) require much more attention. ...
Hence, this proposal, unlike the Shelley scheme, promotes hard fork initiations as a standalone governance action, distinct from protocol parameter updates.

This, however can be part of a separate PR as using the options like in
~/src/cli-pparams/cardano-cli conway governance action create-protocol-parameters-update \
--testnet \
--governance-action-deposit 0 \
--stake-verification-key-file "${UTXO_DIR}/stake1.vkey" \
--proposal-anchor-url https://tinyurl.com/3wrwb2as \
--proposal-anchor-metadata-file "${TRANSACTIONS_DIR}/govActionJustification.txt" \
--protocol-major-version 10  \
--protocol-minor-version 0 \
--out-file "${TRANSACTIONS_DIR}/pparams.action"

produces a proposal with an empty parameter change [0, null, {}],
[0, h'E0E04D56D1CC733F26906F79327CF88FB831F905307A08C3338C1CC03F', [0, null, {}], ["https://tinyurl.com/3wrwb2as", h'52E69500A92D80F2126C836A4903DC582006709F004CF7A28ED648F732DFF8D2']].

and even when voted "yes" by relevant actors it has absolutely no consequences on the system.
+ cardano-cli conway governance query gov-state --testnet-magic 42
+ jq -r .ratify.pgaPParamUpdate
null


Add support for DigitalOcean Spaces

Add support to enable syncing of backups to DigitalOcean Spaces.
Similar to the current Amazon S3 support as described at: https://github.com/nystudio107/craft-scripts#sync_backups_to_s3sh.
More info at: https://www.digitalocean.com/community/tutorials/how-to-manage-digitalocean-spaces-with-s3cmd

Looks neat. I don't have a DO box to play around on tho unfortunately. I'd like to add it tho.


Problem with current working directory when sharing a folder

I have a minor problem with Live share when sharing a folder on my remote Linux server. On the host instance, the current working directory is set to the share folder, which is fine. But, on all guest instances, the current folder is always set to my home folder on that server. Can you please help me to fix this problem? Thanks,

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 62d2dbef-9f65-dfc7-3d5d-b74b7d88c399
Version Independent ID: 09d0d40c-987f-112a-de83-8a3db354b2b9
Content: Troubleshooting - Visual Studio Live Share
Content Source: docs/troubleshooting.md
Product: visual-studio-family
Technology: live-share
GitHub Login: @Chuxel
Microsoft Alias: clantz

fixed since the python version 2021.2.xxxx7922


fix(ci): increment package number

Disabled branch protection, which should allow the npm publish to run

@dependabot rebase


Some comments about this documentation page

Hi,
Just executed a firmware update of my Surface docking, here are my comments on this documentation page, which could use some rewriting if you ask me.

You need to download an .MSI from the Microsoft Surface Dock Firmware Update page. The MSI installs and asks for a reboot? Later I read that a reboot is not really necessary.
The documentation states:"After you connect Surface Dock to your Surface device, the tool checks the firmware status while running in the background." - does that mean that the MSI must be installed while the Surface Dock is disconnected? or do we mean that we need to disconnect and connect the Surface Dock in order for "something" to start? - and why is there no visual window explaining what is going on?

So, apparently the firmware was updated, but again no visual indication of what happened. or will it only be completed once the power disconnects and it boots again?
About Manual Installation:
Reconnect your Surface Dock for 2 minutes and then disconnect it from your device. The DisplayPort firmware update will be installed while the hardware is disconnected. The LED in the Ethernet port of the dock will blink while the update is in progress. Please wait until the LED stops blinking before you unplug your Surface Dock from power.
a) what is the difference here compared to the first installation...was that the automatic installation or something? and let me see if I understand this completely - I need to reconnect my surface dock for 2 minutes (so, disconnect if already connected and then connect), within those 2 minutes "something" happens. Then I need to disconnect for some installation to take place, and then when the Ethernet LED stops blinking I need to unplug the Surface Dock from power?
Manually installing the MSI file may prompt you to restart Surface; however, restarting is optional and not required. -< then remove this from the MSI, its confusing and unnecessary
You will need to disconnect and reconnect the dock twice before the update fully completes. <- this was neither mentioned in the first part and manual installation part
Network Deployment: Msiexec.exe /i /quiet /norestart will do nothing, you must specify the .MSI file after the /i parameter. when people execute this command line nothing will happen.
it says "A log file is not created by default. In order to create a log file, you will need to append "/l*v [path]" two times? why?
For Intune deployment, please specify how the detection method must be created, later the registry locations are explained but nowhere that these could be used as a detection method as well, also here the .msi file must be specified after /i
The Notes section of the "Surface Dock Firmware Update log" metions Earlier versions of this tool wrote events to Applications and Services Logs\Microsoft Surface Dock Updater.? What does the Surface Dock Firmware Update Log have to do with the EventLog? Is this the logfile you create with the /L*V parameter for msiexec? or another logfile..
The events are logged into the "Application" event log, and I can see the event ID mentioned in Table 2, but all the eventlog messages look something like:

The description for Event ID 2009 from source SurfaceDockFwUpdate cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
surfacedockfwupdate
Dock MCU firmware version does not support telemetry. hr=0 DriverTelementry EventCode = 2009
The request is not supported

Come on Microsoft, you can do better than these vague messages? please make sure that valid eventlog messages are written to the event log which are readable and not raise questions on whether the installation failed or was successful.
Troubleshooting Tips:
Uninstall any current Surface Dock Firmware Update and then install the latest version.  How ?????

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2a7b356e-3dbd-7f59-845e-a92e05865a52
Version Independent ID: 711c5497-373f-662c-7fae-ba60f046e50c
Content: Microsoft Surface Dock Firmware Update - Surface
Content Source: devices/surface/surface-dock-firmware-update.md
Product: w10
Technology: windows
GitHub Login: @Dansimp
Microsoft Alias: dansimp

@illfated . Please review the PR

(Ref. #5759 )

@illfated. I think PR#5759 still not merged with article.

Yes, I know. I simply referenced this ticket back to the PR, to make it more visible that you have not added any comment to automatically close this ticket when the PR becomes merged.

@illfated. I have some screenshots which shows the proofs of some articlesa and it's compulsory needed , so I want to know how to add screenshots3 photos to the articles.
Could you tell me steps or give instructions how to add

Well, as soon as you have started working on a new PR branch, you should be able to upload images (make sure to use lowercase file names) to the GitHub folder where the image is meant to be, usually in a separate folder called "images" inside the folder where the article is located.

@illfated. Thanks, i will try  to upload in next PR

Example image:

in your case, the button in my image saying [ Tree: 24a24bb523 ] should show the name of your PR branch ( e.g. "patch-123").
Notice also the 2nd red frame around the [ Upload files ] button.

(ref. #5832 )

(ref. #5214 )

I thought I had a reply from the product team, but I'm still waiting for some details.  I wish I could test this myself and just update the topic based on the behavior, but I don't have a surface dock.

Well wadda you know I suddenly have a surface dock :)  I'll see what I can do to follow the instructions in the guide and clarify them.

I was able to get some information that I'll be adding to the topic.
Here's how the update process works:
Note: before starting you might want to open eventvwr.msc and click on Windows Logs\Application then click Filter Current Log in the right hand pane, and enter a source of SurfaceDockFwUpdate. This will help to notify when the process is started and when it is finished.
Also review the following registry node:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\WUDF\Services\SurfaceDockFwUpdate\Parameters
There are several REG_DWORD items here that provide status of the update. The most important ones are:
Component10CurrentFwVersion
Component20CurrentFwVersion
When the update is complete, you will see the following values for these two DWORDs:
Component10CurrentFwVersion 0x04ac3970 (78395760)
Component20CurrentFwVersion 0x04915a70 (76634736)
...anyway here's what to do:

Download and install the .msi as described in the article.
You will notice some events in the Application log, but the firmware will not update until you disconnect from the dock and then reconnect.
Disconnect the magnetic power cord from the side of the Surface device and wait a couple minutes.  Then connect it again.
When you look in the Application log again, you should see these events:
ID 2001 update started
ID 2004 get version (10)
ID 2004 get version (20)

ID 2007 update finished
ID 2008 telemetry start
ID 2011 telemetry end
(I gave a general description of the event above in parenthesis - this isn't the precise wording in the event)

Refresh the registry keys again and now you should have the updated versions.

I still have a couple more questions to get answered, but I should be able to update this topic on Monday with better procedures.

FYI you can use this query from the command line:
Reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\WUDF\Services\SurfaceDockFwUpdate\Parameters"

Good updates. Nice to see that there is progress, particularly for issues important to quite a lot of users.

Documentation improvements are good but can't the firmware updater be updated so it actually tells you what it is doing and if you need the update. Would be easier than having to check the registry and event viewer. Seems rather strange it doesn't do this as all bios updaters have this basic functionality.

Looks like a quite useful feedback to the firmware updater developers. I agree, based on other firmware updating experience.

I have heard that this particular tool is not meant to be a final solution, and that it won't be developed further, but rather replaced by something with better functionality.

I have heard that this particular tool is not meant to be a final solution, and that it won't be developed further, but rather replaced by something with better functionality.

I sure hope so. The current version seems to have made a lot of users unhappy. Let's keep the hope up.

OK I have updated the article to hopefully make it clearer.
There was a lot of information at the end of the article that was helpful to understand the beginning of the article, but you'd never see it if you just read it top to bottom because you would be confused before you got there. I've linked those sections now and also explained the process a little simpler.  I'll close this issue tomorrow assuming I don't hear that more is needed.  The changes I just made will appear this afternoon, in about 3-4 hours.

I see the GitHub page  is updated,

GitHub page: https://github.com/MicrosoftDocs/windows-itpro-docs/blob/public/devices/surface/surface-dock-firmware-update.md
commits: https://github.com/MicrosoftDocs/windows-itpro-docs/commits/public/devices/surface/surface-dock-firmware-update.md

but it will still take a couple of days for the changes to migrate to the docs.microsoft.com page.
I presume the involved users will be interested: @DennisGaida, @termnews, @StuBeck32, @MJBu, @goxia

The updates are very good. Thank you @greg-lindsay and congrats on your new Surface Dock 😉
There is still the error of this original issue in the doc: The line Msiexec.exe /i /quiet /norestart will do nothing as the MSI needs to be specified. Same goes further down for Intune Deployment.
Given that the update doc is pretty technical (event log, using reg query & registry) I don't think any layperson (i.e. regular consumer) understands how to update the Surface Dock. I hope this is intentional.

Hi Dannis, thanks for pointing out these issues!  I should have seen that, but I was focused on reorganizing all the topics. I'll get in and fix that right now.
Indeed the page is meant for IT pros, not the general public.  However we can't prevent the general public from hitting this page when they run a search.

I've pushed this latest update out. It should appear later this afternoon. I'm closing the issue now. Let me know if more needs to be done here. Thanks again for helping to improve the article so much.

Is there a way to set this so the issue isn’t closed until either a week has passed or someone external has okayed the changes? Closing it before we can even see the changes makes it seem like you don’t want us to review the changes and simply create a new issue.
Stuart

From: Greg Lindsay<EMAIL_ADDRESS>Sent: Tuesday, February 11, 2020 2:47:24 PM
To: MicrosoftDocs/windows-itpro-docs<EMAIL_ADDRESS>Cc: StuBeck32<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/windows-itpro-docs] Some comments about this documentation page (#5756)
I've pushed this latest update out. It should appear later this afternoon. I'm closing the issue now. Let me know if more needs to be done here. Thanks again for helping to improve the article so much.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fwindows-itpro-docs%2Fissues%2F5756%3Femail_source%3Dnotifications%26email_token%3DAK5FPN6MY3GMBQJOTNNE3VTRCL6EZA5CNFSM4KAM6WF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELNZY7I%23issuecomment-584817789&data=02|01||45efbc887c4248a6f4b108d7af2b37c4|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637170472458624970&sdata=HtO2VFG29e7RzohGRQJYHvPLFTo099Td9YBgAPm7K5U%3D&reserved=0, or unsubscribehttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAK5FPN2G76YEZEGPNW2KPO3RCL6EZANCNFSM4KAM6WFQ&data=02|01||45efbc887c4248a6f4b108d7af2b37c4|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637170472458624970&sdata=GB9Kh6L73OJM3%2BPIB7JgaLDJykggswLKBpStSYY0g6k%3D&reserved=0.

The changes were very minor. All I did was escape some hidden characters. I reviewed the PR and verified that the characters are displaying correctly. If there's still a problem I can fix it without having to open another issue. I need to move on though, this topic is just one of many hundreds I'm looking at.

Understand, and also of course you have a ton of work to do like all of us. This seems to be a bigger problem with how github sends notices either way as I was responding more to your "I'm closing this" and not seeing the other notes you had done. MS has a history here of closing cases without actually doing anything.

I see, yes the notifications aren't ideal. In fact it makes it difficult for me to even see issues unless they are specifically assigned to me and that doesn't happen for a while in some cases.


Fixed roster model

longflatroster was not connected to roster and hence the rest-time constraints were disconnected from the rest of the problem. I fixed this bug and, moreover, I fixed the explanation of the "Too much rest" constraint.

Thanks, looks good!


Add custom backend support

I wish to store binary objects in db but JSON doesn't support binary encoding. And I guess CBOR is more sufficient for me. I can create it myself but there is no custom datastore API as I see.

Not planned but you can always encode/decode your blob in base64. This will take a bit more space (1.33x the size of your array) but in most usecases that's not an issue.
Do not use utf8 to do so or you're in for some hard to debug trouble, see http://haacked.com/archive/2012/01/30/hazards-of-converting-binary-data-to-a-string.aspx/


Unable to install on Ubuntu Xenial (16.04) with pip- 8.1.1

Seeing this error when I try to install this package using system default pip (8.1.1) on Ubuntu Xenial (16.04):
Collecting redis (from -r requirements/main.txt (line 2))
  Using cached redis-2.10.5-py2.py3-none-any.whl
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 209, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 328, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 748, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 360, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 448, in _prepare_file
    req_to_install, finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 387, in _check_skip_installed
    req_to_install.check_if_exists()
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 1011, in check_if_exists
    self.req.project_name
AttributeError: 'Requirement' object has no attribute 'project_name'

Spoke too soon. Sorry. Another package is causing the error.


staticmethod in module not properly exposed / accessible from outside of module

Acton Version
0.21.0
Steps to Reproduce and Observed Behavior
I want to call the staticmethod from_json on the TestInfo class in testing.TestInfo. I get an error. My current work around is to do from testing import TestInfo but this should not be necessary, see
https://github.com/actonlang/acton/pull/1757/files#diff-ad702e1f2fe61135b8b43f1c0063a44f36a80aa12414d3d118066b39a7cb7d9bR6
To reproduce, try changing https://github.com/actonlang/acton/pull/1757/files#diff-8912c4341e78ba2504029fcf3fcc1b37ee141d14b1833932f3a3541fffd54509R744 to access testing.TestInfo.from_json instead!
Expected Behavior
It should work!

@nordlander here's that bug we looked at the other day, I hope the reproduction via my soon-to-be-merged code is simple enough?

Yep! And I already know where the error is :)


Add ability to rotate image

It is now possible to add a 'rotate' value to the addImage() function

Thanks @alabaki !


Hide side spinner

I want to hide the small semi-circle rotating in top-right corner. How can I do that ?
The reason to hide that is that it is not in best position and it just looks ugly

Simply set the showSpinner={false}
<NextTopLoader showSpinner={false} />


Refactor BaseDataset --> PythonDataset

Depends on the development plan of CXFlow: If and only if we plan to implement support for other languages anytime soon is this issue relevant.
If we plan to implement other languages soon, then current BaseDataset is a base dataset just for python and therefore should be renamed to PythonDataset or BasePythonDataset

Support of additional languages is already implemented. You can implement your dataset in any language you want, the only thing is that you have to compile it so that python can load it.
Note that other languages will not inherit from our datasets (instead, they will implement the provided interface of the abstract dataset).
Hence, the base dataset is actually really a base dataset for python datasets. Note that you can implement python dataset without using the base dataset as long as it implements the interface given by abstract dataset.
Any comments? @FloopCZ @blazekadam

I agree.