VerneMQ crashes under load with ~30k Websocket connections

Environment

VerneMQ Version: 1.9.2
OS: Amazon Linux - EKS / Kubernetes
VerneMQ configuration (vernemq.conf) or the changes from the default:

allow_register_during_netsplit = off
allow_publish_during_netsplit = off
allow_subscribe_during_netsplit = off
allow_unsubscribe_during_netsplit = off
allow_multiple_sessions = off
coordinate_registrations = on
max_inflight_messages = 20
max_online_messages = 1000
max_offline_messages = 1000
max_message_size = 0
upgrade_outgoing_qos = off
listener.max_connections = 10000
listener.nr_of_acceptors = 10
listener.tcp.default = <IP_ADDRESS>:1883
listener.vmq.clustering = <IP_ADDRESS>:44053
listener.http.default = <IP_ADDRESS>:8888
systree_enabled = on
systree_interval = 20000
graphite_enabled = off
graphite_host = localhost
graphite_port = 2003
graphite_interval = 20000
shared_subscription_policy = prefer_local
plugins.vmq_passwd = on
plugins.vmq_acl = on
plugins.vmq_diversity = off
plugins.vmq_webhooks = off
plugins.vmq_bridge = off
metadata_plugin = vmq_plumtree
vmq_acl.acl_file = ./etc/vmq.acl
vmq_acl.acl_reload_interval = 10
vmq_passwd.password_file = ./etc/vmq.passwd
vmq_passwd.password_reload_interval = 10
vmq_diversity.script_dir = ./share/lua
vmq_diversity.auth_postgres.enabled = off
vmq_diversity.postgres.ssl = off
vmq_diversity.postgres.password_hash_method = crypt
vmq_diversity.auth_cockroachdb.enabled = off
vmq_diversity.cockroachdb.ssl = on
vmq_diversity.cockroachdb.password_hash_method = bcrypt
vmq_diversity.auth_mysql.enabled = off
vmq_diversity.mysql.password_hash_method = password
vmq_diversity.auth_mongodb.enabled = off
vmq_diversity.mongodb.ssl = off
vmq_diversity.auth_redis.enabled = off
vmq_bcrypt.pool_size = 1
log.console = file
log.console.level = info
log.console.file = ./log/console.log
log.error.file = ./log/error.log
log.syslog = off
log.crash = on
log.crash.file = ./log/crash.log
log.crash.maximum_message_size = 64KB
log.crash.size = 10MB
log.crash.rotation = $D0
log.crash.rotation.keep = 5
nodename =<EMAIL_ADDRESS>distributed_cookie = vmq
erlang.async_threads = 64
erlang.max_ports = 262144
leveldb.maximum_memory.percent = 70
leveldb.maximum_memory.percent=40
erlang.max_ports=1000000
listener.tcp.allowed_protocol_versions=3,4,5
listener.nr_of_acceptors=100
plugins.vmq_diversity=on
erlang.process_limit=1000000
allow_subscribe_during_netsplit=on
listener.max_connections=1000000
vmq_diversity.custom_auth_postgres.file=/vernemq/share/lua/custom-scripts/custom_auth_postgres.lua
allow_publish_during_netsplit=on
allow_anonymous=off
listener.ws.allowed_protocol_versions=3,4,5
allow_unsubscribe_during_netsplit=on
plugins.vmq_passwd=off
metadata_plugin=vmq_swc
plugins.vmq_acl=off
log.console=console
allow_register_during_netsplit=on
listener.tcp.localhost=<IP_ADDRESS>:1883
erlang.distribution.port_range.minimum = 9100
erlang.distribution.port_range.maximum = 9109
listener.tcp.default = <IP_ADDRESS>:1883
listener.ws.default = <IP_ADDRESS>:8080
listener.vmq.clustering = <IP_ADDRESS>:44053
listener.http.metrics = <IP_ADDRESS>:8888

Cluster size/standalone: 1 node.

Expected behavior
VerneMQ not crashing.
Actual behaviour
VerneMQ crashes with the following errors logged:
10:06:03.831 [error] Supervisor vmq_swc_store_sup_meta1 had child {vmq_swc_store,meta1} started with vmq_swc_store:start_link({swc_config,'VerneMQ@vernemq-production-0.vernemq-production-headless.vernemq.svc.cluster.local',...}) at <0.208.0> exit with reason no case clause matching {ok,{915,4046764476968596145877828492065898424682979057122137404917064403273609207443373092353759448397119992303400902952133191298813534506630275096688480596528496784428238547082218745982149919165111430523779904007281222679483072061544800444562913290800260729761872442899043132866618947434843829193803113185105842968222380115950629785455588470768623516543833222595616104063702386903178835614672583522555396990906982463122296785894372111531842568203321152453153551187632832056116330005458849450797692616969311188416143896655899313937800401319106294904355497629235982970013014403246230210969852018292949080615645615373404484542730770554574685378145278120240069536763923308661201788358101081847861684615779054721508649347473287973448889598273196798768434805442712423133420800014124981128000940230840654480227637654050162732455149118340409373668736633926599673746635639267336838302861412960033262471822151131277009752823501855645221510818878195037419487143829927463849634932982300890840633229559433719266050842232188372870236274437146286444865592079917011387574393520954185611299972460377227170720336121904416388796891447414521964910360697264396431817393485834838746445666351540026478094448438769147189027644897756682087891753956881063754406034942641634204446102466881399906929803975618788268930650987957427601221545315555186060728283620933653581999547354692830142982168672251501315596161546077006518487939453267373266409852330922960547876744260840194082684688636697818628035268536129277429207417143333214366645489178218581025279446713102094345756529628229663981952160571977892057466364606599858066764932764265871063023991586485412715757983189297460676030150895521978034917361096611930987001327555594926439697643908296062073634288601510655728675591065327211367114388429226644896814115434223017224589499606041597462249090466230404804942673843684232855509466030804237271425156... in context child_terminated
10:06:04.224 [error] CRASH REPORT Process vmq_swc_store_batcher_meta1 with 0 neighbours exited with reason: {{{case_clause,{ok,{915,

...

10:06:04.225 [error] Error in process <0.373.0> on node<EMAIL_ADDRESS>with exit value:
{badarg,[{vmq_swc_store_batcher,add_to_batch,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store_batcher.erl"},{line,28}]},{vmq_swc_store,enqueue_op_sync,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl"},{line,672}]},{vmq_swc,maybe_resolve,5,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc.erl"},{line,114}]},{vmq_swc,'-fold/5-fun-1-',8,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc.erl"},{line,105}]},{vmq_swc_db_leveldb,iterate,6,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_db_leveldb.erl"},{line,137}]},{vmq_swc_db_leveldb,'-handle_call/3-fun-1-',6,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_db_leveldb.erl"},{line,109}]}]}
10:06:04.225 [error] Error in process <0.465.0> on node<EMAIL_ADDRESS>with exit value:
{badarg,[{vmq_swc_store_batcher,add_to_batch,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store_batcher.erl"},{line,28}]},{vmq_swc_store,enqueue_op_sync,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl"},{line,672}]},{vmq_swc,maybe_resolve,5,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc.erl"},{line,114}]},{vmq_swc,'-fold/5-fun-1-',8,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc.erl"},{line,105}]},{vmq_swc_db_leveldb,iterate,6,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_db_leveldb.erl"},{line,137}]},{vmq_swc_db_leveldb,'-handle_call/3-fun-1-',6,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_db_leveldb.erl"},{line,109}]}]}
10:06:04.225 [error] Supervisor vmq_ranch_sup had child vmq_ranch_config started with vmq_ranch_config:start_link() at undefined exit with reason bad argument in call to ets:lookup(vmq_swc_store_r_o_w_meta1, <<16,0,0,0,2,16,0,0,0,2,12,187,91,110,32,8,12,177,219,237,214,107,77,156,8,16,0,0,0,2,12,187,91,...>>) in vmq_swc_store:maybe_get_cached_object/2 line 482 in context start_error
10:06:04.225 [error] Supervisor vmq_server_sup had child vmq_ranch_sup started with vmq_ranch_sup:start_link() at undefined exit with reason {shutdown,{failed_to_start_child,vmq_ranch_config,{badarg,[{ets,lookup,[vmq_swc_store_r_o_w_meta1,<<16,0,0,0,2,16,0,0,0,2,12,187,91,110,32,8,12,177,219,237,214,107,77,156,8,16,0,0,0,2,12,187,91,110,53,251,157,150,229,118,178,220,128,8,12,179,220,172,55,11,69,166,233,101,175,220,46,86,91,53,166,240,8>>],[]},{vmq_swc_store,maybe_get_cached_object,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl"},{line,482}]},{vmq_swc_store,read,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_sto..."},...]},...]}}} in context start_error
10:06:04.226 [error] CRASH REPORT Process <0.492.0> with 0 neighbours crashed with reason: bad argument in call to ets:lookup(vmq_swc_store_r_o_w_meta1, <<16,0,0,0,2,16,0,0,0,2,12,187,91,110,32,8,12,177,219,237,214,107,77,156,8,16,0,0,0,2,12,187,91,...>>) in vmq_swc_store:maybe_get_cached_object/2 line 482
10:06:04.619 [error] Supervisor vmq_swc_store_sup_meta1 had child {vmq_swc_store_batcher,meta1} started with vmq_swc_store_batcher:start_link({swc_config,'VerneMQ@vernemq-production-0.vernemq-production-headless.vernemq.svc.cluster.local',...}) at <0.207.0> exit with reason {{{case_clause,{ok,{915,40467644769685961458778284920658984246829790571221374049170644032736092074433730923537594483971199923034009029521331912988135345066302750966884805965284967844282385470822187459821499191651114305237799040072812226794830720615448004445629132908002607297618724428990431328666189474348438291938031131851058429682223801159506297854555884707686235165438332225956161040637023869031788356146725835225553969909069824631222967858943721115318425682033211524531535511876328320561163300054588494507976926169693111884161438966558993139378004013191062949043554976292359829700130144032462302109698520182929490806156456153734044845427307705545746853781452781202400695367639233086612017883581010818478616846157790547215086493474732879734488895982731967987684348054427124231334208000141249811280009402308406544802276376540501627324551491183404093736687366339265996737466356392673368383028614129600332624718221511312770097528235018556452215108188781950374194871438299274638496349329823008908406332295594337192660508422321883728702362744371462864448655920799170113875743935209541856112999724603772271707203361219044163887968914474145219649103606972643964318173934858348387464456663515400264780944484387691471890276448977566820878917539568810637544060349426416342044461024668813999069298039756187882689306509879574276012215453155551860607282836209336535819995473546928301429821686722515013155961615460770065184879394532673732664098523309229605478767442608401940826846886366978186280352685361292774292074171433332143666454891782185810252794467131020943457565296282296639819521605719778920574663646065998580667649327642658710630239915864854127157579831892974606760301508955219780349173610966119309870013275555949264396976439082960620736342886015106557286755910653272113671143884292266448968141154342230172245894996060415974622490904662304048049426738436842328555094660308042372714251568... in context child_terminated
10:06:04.619 [error] gen_server vmq_systree terminated with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:04.619 [error] CRASH REPORT Process vmq_systree with 0 neighbours crashed with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:04.619 [error] Supervisor vmq_metrics_sup had child vmq_systree started with vmq_systree:start_link() at <0.317.0> exit with reason no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76 in context child_terminated
10:06:04.619 [error] gen_server vmq_systree terminated with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:04.619 [error] CRASH REPORT Process vmq_systree with 0 neighbours crashed with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:04.619 [error] Supervisor vmq_metrics_sup had child vmq_systree started with vmq_systree:start_link() at <0.495.0> exit with reason no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76 in context child_terminated
10:06:05.468 [error] gen_server vmq_systree terminated with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:05.468 [error] CRASH REPORT Process vmq_systree with 0 neighbours crashed with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:05.468 [error] Supervisor vmq_metrics_sup had child vmq_systree started with vmq_systree:start_link() at <0.496.0> exit with reason no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76 in context child_terminated
10:06:06.469 [error] gen_server vmq_systree terminated with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:06.469 [error] CRASH REPORT Process vmq_systree with 0 neighbours crashed with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:06.469 [error] Supervisor vmq_metrics_sup had child vmq_systree started with vmq_systree:start_link() at <0.497.0> exit with reason no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76 in context child_terminated
10:06:07.469 [error] gen_server vmq_systree terminated with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:07.470 [error] CRASH REPORT Process vmq_systree with 0 neighbours crashed with reason: no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76
10:06:07.470 [error] Supervisor vmq_metrics_sup had child vmq_systree started with vmq_systree:start_link() at <0.498.0> exit with reason no match of right hand value [] in vmq_cluster:netsplit_statistics/0 line 76 in context child_terminated
10:06:07.530 [error] Supervisor vmq_server_sup had child vmq_cluster_node_sup started with vmq_cluster_node_sup:start_link() at <0.486.0> exit with reason killed in context shutdown_error
10:06:07.533 [error] CRASH REPORT Process <0.188.0> with 0 neighbours exited with reason: {{shutdown,{failed_to_start_child,vmq_ranch_sup,{shutdown,{failed_to_start_child,vmq_ranch_config,{badarg,[{ets,lookup,[vmq_swc_store_r_o_w_meta1,<<16,0,0,0,2,16,0,0,0,2,12,187,91,110,32,8,12,177,219,237,214,107,77,156,8,16,0,0,0,2,12,187,91,110,53,251,157,150,229,118,178,220,128,8,12,179,220,172,55,11,69,166,233,101,175,220,46,86,91,53,166,240,8>>],[]},{vmq_swc_store,maybe_get_cached_object,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl"},{line,482}]},{vmq_swc_store,read,2,[{...},...]},...]}}}}},...} in application_master:init/4 line 138
10:06:07.534 [info] Application vmq_server exited with reason: {{shutdown,{failed_to_start_child,vmq_ranch_sup,{shutdown,{failed_to_start_child,vmq_ranch_config,{badarg,[{ets,lookup,[vmq_swc_store_r_o_w_meta1,<<16,0,0,0,2,16,0,0,0,2,12,187,91,110,32,8,12,177,219,237,214,107,77,156,8,16,0,0,0,2,12,187,91,110,53,251,157,150,229,118,178,220,128,8,12,179,220,172,55,11,69,166,233,101,175,220,46,86,91,53,166,240,8>>],[]},{vmq_swc_store,maybe_get_cached_object,2,[{file,"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl"},{line,482}]},{vmq_swc_store,read,2,[{...},...]},...]}}}}},...}
[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
{"Kernel pid terminated",application_controller,"{application_start_failure,vmq_server,{{shutdown,{failed_to_start_child,vmq_ranch_sup,{shutdown,{failed_to_start_child,vmq_ranch_config,{badarg,[{ets,lookup,[vmq_swc_store_r_o_w_meta1,<<16,0,0,0,2,16,0,0,0,2,12,187,91,110,32,8,12,177,219,237,214,107,77,156,8,16,0,0,0,2,12,187,91,110,53,251,157,150,229,118,178,220,128,8,12,179,220,172,55,11,69,166,233,101,175,220,46,86,91,53,166,240,8>>],[]},{vmq_swc_store,maybe_get_cached_object,2,[{file,\"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl\"},{line,482}]},{vmq_swc_store,read,2,[{file,\"/vernemq-build/apps/vmq_swc/src/vmq_swc_store.erl\"},{line,103}]},{vmq_swc,get,4,[{file,\"/vernemq-build/apps/vmq_swc/src/vmq_swc.erl\"},{line,58}]},{vmq_swc_metrics,timed_measurement,4,[{file,\"/vernemq-build/apps/vmq_swc/src/vmq_swc_metrics.erl\"},{line,102}]},{vmq_swc_plugin,metadata_get,3,[{file,\"/vernemq-build/apps/vmq_swc/src/vmq_swc_plugin.erl\"},{line,107}]},{vmq_config,get_env,4,[{file,\"/vernemq-build/apps/vmq_server/src/vmq_config.erl\"},{line,93}]},{vmq_config,'-get_all_env/1-fun-0-',4,[{file,\"/vernemq-build/apps/vmq_server/src/vmq_config.erl\"},{line,115}]}]}}}}},{vmq_server_app,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,vmq_server,{{shutdown,{failed_to_start_child,vmq_ranch_sup,{shutdown,{failed_to_start_child,vmq_ranch_config,{badarg,[{ets,lo

Crash dump is being written to: /erl_crash.dump...```
- Please include minimal steps to reproduce the issue, if possible.

@olesku thanks... can you share more information on the test scenario?
Just so there's some context stored for this issue here.

@olesku thanks... can you share more information on the test scenario?
Just so there's some context stored for this issue here.

Hi!
This is a test with real production traffic from web-browser clients connecting to VerneMQ through the websocket endpoint using mqtt.js. Mostly to the same topics.
It is a fan-out scenario where we are pushing events to all the clients/consumers from a single producer. We are pushing events to each client at random intervals as they come in. Messages are not large in terms of size (below 1kb for the most part).
We did some successful synthetic testing with MZBench towards the same setup before trying with real production traffic and was able to reach approx 100k clients per node with that. But when running in production with real traffic it crashes intermittently when there is connection spikes.

Incidently, did you also run the same production traffic on a single node VerneMQ using Plumtree?
Also, the Broadcast is not partitioned, right? I.e. Subscribers all subscribe to the same topic?

@olesku's peer here

No we didn't use Plumtree at all, we went straight for swc

exactly, all subscribers subscribed to the same topic. I know it may not sound realistic but for our cases it actually does

good :)
it's a totally legit scenario! only question is whether it has to be optimsed/adapted for.
BTW did you observe anything CPU related? it wasn't maxed out or anything?

@dergraf quite the SWC puzzle here!

What are the machine specs for those nodes?

@ioolkos
we tested on c5.xlarge AWS ECs so 4 vCPU, 8 GB Memory
regarding CPU usage it looks like around time when it crashed usage went super high - 97.1 % in just 20 minutes. But actually at that time I think we didn't put any significant traffic on the cluster, I

OS config question: for how many file descriptors was the system configured for?

OS config question: for how many file descriptors was the system configured for?

It was configured for 1048000 connections.
The following sysctl settings was tuned:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_fin_timeout=15
fs.nr_open=1048000
fs.file-max=1048000
net.core.somaxconn=1000000
net.ipv4.tcp_max_syn_backlog=1000000
net.core.netdev_max_backlog=1000000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.core.optmem_max=16777216
net.ipv4.tcp_rmem=1024 4096 16777216
net.ipv4.tcp_wmem=1024 4096 16777216
net.netfilter.nf_conntrack_buckets=128000
net.nf_conntrack_max=1000000
net.netfilter.nf_conntrack_tcp_timeout_time_wait=30
net.ipv4.tcp_max_tw_buckets=1048576
net.ipv4.tcp_fin_timeout=15

and the ephemeral port range?

and the ephemeral port range?

net.ipv4.ip_local_port_range = 32768	60999
This might be a bit low actually. Im not quite sure how AWS NLB's connect to their backends, but I would imagine they split thr requests accross multiple ip's to allow for more connections than the ephemeral port range ?


GetExpression's Return Expectation - why an AST? :-)

Hello,
Based on https://github.com/microsoft/DataConnectors/issues/318#issuecomment-674159531, for a view to support Direct Query, part of what’s needed is for it to provide a GetExpression handler that returns an AST which more or less represents a Value.NativeQuery(MyConnector.MyDataSourceFunction) call.
Normally in the M paradigm, if one piece of code needs access to something from elsewhere, either the value of interest is passed directly (e.g. the result from invoking MyConnector.MyDataSourceFunction() is passed), a function reference is passed which then can be invoked at the recipient’s pleasure to produce the needed value (e.g. pass MyConnector.MyDataSourceFunction without invoking it) or a “delegate” function is passed which returns the needed value when invoked (e.g. () => MyConnector.MyDataSourceFunction()).
However, in the case of DQ support/GetExpression, none of these approaches are used; instead, an AST is expected (which presumably is then evaluated by some other Power Query component to get the value returned by my data source function).
Sometimes (often?) in development, understanding the “why?” behind an ask helps with doing a good job at implementing it.
In that spirit, would you be willing to shed some light on why an AST is expected here vs. a more traditional passing of value, reference or delegate?
Thanks,
Ben

Based on the research I’ve been doing, I’m wondering?!?!?! if an AST is used because possibly two separate instances of Power Query are at play:

One that evaluates the initial M expression that defins the table in order to get an initial table view.
Another that is more tightly associated with the DQ engine. When the DQ engine receives a DAX expression, it first translates it to SQL, then asks this PQ instance to translate the SQL into M then evaluate that M.

So, if a table is defined by the following query:
let
    Source = MyDataSource.Connect(),
    Data = Source{[Name="TableA"]}[Data]
in
    Data

The first PQ instance evaluates this (calls the data source function to get its navigation table, filters that table down to a single row, then gets the Data field of that row’s record) to ultimately end up with a table view for TableA.
The second PQ instance wants a table view for it to execute its SQL-to-M translation against. A view from the first PQ instance can’t be passed as a normal M value to this second instance, as it is a separate. Instead, to bridge the divide, GetExpression is called on the view from the first instance. GetExpression returns an AST describing how to call the connector to get a brand-new view for TableA. This AST is passed to the second PQ instance.
The second PQ instance follows the AST’s instructions to get its own view instance, then works with that view the same as the view would be worked with in import mode—e.g. On* handlers, GetType and GetRows are called, as appropriate, to configure query folding and get data. (Handler GetExpression isn’t relevant to this second PQ instance.)
Is this anywhere close to what is going on and why an AST is used? 😊
Thank you,
Ben

/CC: @CurtHagenlocher

Conceptually, all M values are associated with expressions. This is why, for instance, we're able to do query folding: we can recover some version of the expression used to generate a function in order to do processing on it (such as converting it to SQL for evaluation on a server). More recently, this ability has also been used to try to show a query plan for query diagnostics. So expressions really are a fundamental part of the implementation, even if they're not currently very available to M code.
As for their use in Direct Query, this is perhaps best explained as a consequence of the original Direct Query implementations -- for SQL Server, Oracle and Teradata -- which work much differently than newer ones. For those implementations, the AS engine gets the expression for the folded table and extracts the native SQL from the Value.NativeQuery implementation. It then uses that in conjunction with a "cartridge" in order to do Direct query, cutting us out of the loop entirely. Newer implementations (such as BigQuery or Snowflake) are evaluated through us, but the contract with AS largely remains the same. And that's why Value.NativeQuery needs to be part of the returned expression.

Thank you, @CurtHagenlocher!
So, if I were to explain this to someone else in summary form, would it be fair to say that the reason an AST is needed here is for historic reasons?

Hey @bgribaudo
I'm going through all the open issues on the repo trying to provide a resolution to them or address some of them in the new VS Code SDK that we're creating but moving forward we'll be addressing questions like this one around the internals of Power Query through the official Power Query forum inside the Power BI Community site. You can find more information about this from the last section of the readme file of this repo.
Thanks!


dependabot stuck on Checking now for more than 12 hours

Is there an existing issue for this?

[X] I have searched the existing issues

Package ecosystem
Bundler
Package manager version
2.3.14
Language version
Ruby 2.7.6
Manifest location and content before the Dependabot update
/Gemfile
dependabot.yml content
version: 2
updates:

package-ecosystem: "npm"
directory: "/"
schedule:
interval: "weekly"
target-branch: "develop"
package-ecosystem: "bundler"
directory: "/"
schedule:
interval: "weekly"
target-branch: "develop"

Updated dependency
No response
What you expected to see, versus what you actually saw
Expected: pr created for outdated gems
Actual: dependabot stuck on Checking now for more than 12 hours
Native package manager behavior
bundle install successfully
Images of the diff or a link to the PR, issue, or logs
No response
Smallest manifest that reproduces the issue
https://github.com/monacohq/monaco-rails/network/updates
it just stuck there, and we don't have a way to collect other information

Given the ! sign, it's likely hitting some kind of error.
Does it succeed if you run with the dry-run script? https://github.com/dependabot/dependabot-core#dry-run-script
Unfortunately, without more information, it's not really actionable for us here on dependabot-core... it's just as likely going to be a database flake (not something for this issue tracker) as a possible code bug (which belongs in this issue tracker). Feel free to file a support request, but I suggest first trying the dry run script and see if that surfaces any clues.

sorry about the late response, it's still stuck on "checking now"
let me figure out what do you mean by "the dry-run script", will feedback later, thanks for your time 🙏
btw, could you pls clarify how to "file a support request" ? I haven't done this before

having fun debugging with bin/dry-run.rb  :D
thanks

hi,  it took hours to finish checking 195 updates, and everything looks good
any suggestion on how to restart the dependabot checking job in Dependency graph ?

One more thing to mention:  in our Gemfile, we lock the bundler version to 2.3.22, but it seems dependabot is using 2.3.14 , which causes the checking failed
I'm not sure whether this is an issue for dependabot, in my opinion bundler version shouldn't be a checking blocker
Any thoughts/comments on this ? 🙇

@junwei-lii I'm confused... you mention previously that this is stuck on Checking now but then you now show an error... are these unrelated issues or did the Checking now go away and this error show up instead?
Regardless, ☝  should probably be on a separate issue from the one stuck in Checking now... and this ticket resolved if Checking now is gone...
Also, before you file a new ticket, can you please click the Check for updates button again? I literally just deployed #5509 and #5513 and so that may fix what you're seeing here.

but then you now show an error
This is an error happened before(we made a workaround by lock bundler version to 2.3.14) , sorry about the misleading

Also, before you file a new ticket, can you please click the Check for updates button again?

this is the issue, I don't have any button to click on

@junwei-lii I'm confused... you mention previously that this is stuck on Checking now but then you now show an error... are these unrelated issues or did the Checking now go away and this error show up instead?
Regardless, ☝ should probably be on a separate issue from the one stuck in Checking now... and this ticket resolved if Checking now is gone...
Also, before you file a new ticket, can you please click the Check for updates button again? I literally just deployed #5509 and #5513 and likely that affects what you're seeing here, although unclear if that resolved the error or is resulting in the error. But re-running to confirm before you file a new ticket will save us both time.

tested with pet project, bundler version restriction should be resolved now
pending issue is "how to reset the checking"

@junwei-lii I suspect this is a case where the Bundler run takes a long time... you mentioned hours for example. However the job should have timed out, but clearly didn't, so we can try clearing it for you. You didn't post the repo link, and I suspect it's a private repo, so please file a support ticket about your account and include the actual repo URL. If it's a private repo, then don't file a community issue--those are public to the world. More info on filing tickets here: https://docs.github.com/en/support/contacting-github-support/creating-a-support-ticket
I'm going to close as there's nothing we can do here from the dependabot-core perspective, this is a one-off support issue.

so we can try clearing it for you

This is what I need 🙏

You didn't post the repo link, and I suspect it's a private repo

Yes, it's a private repo;
sorry about the bothering, I expect this issue should be handled by some kind of official organization, just didn't take time to figure out how to do it, because I find firing an issue is the most easy one to do
Thank you again for the bundler fix and the bin/dry-run.rb mode guide, they're pretty helpful 🙇
And I'll read more carefully before firing an issue

@junwei-lii BTW, thanks for being persistent on this. As someone may have commented on the internal ticket you filed, once you provided the proper info about which repo it was, we were able to identify a bug that was affecting thousands of  jobs from timing out properly... we got that fixed and a cleanup job is slowly clearing out the backlog... so your persistence and willingness to send the right info is making things better for a lot more repos than just yours. ❤


feat(Resource Controller): add service to project

PR summary
This PR adds the Resource Controller V2 service to the project.
PR Checklist
Please make sure that your PR fulfills the following requirements:

[x] The commit message follows the Angular Commit Message Guidelines.
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] New tests
[ ] Build/CI related changes
[ ] Documentation content changes
[ ] Other (please describe)

What is the current behavior?

What is the new behavior?

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

:tada: This PR is included in version 0.9.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add automated Tests in GitHub Action

To reduce the manual overhead for package maintenance, I want to add automated (smoke) testing to the CI Pipeline.

:tada: This issue has been resolved in version 2.4.7-rc.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 2.4.7 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Through association id datetime (try 2).

Second try to resolve Issue  #1265.
I've refactored the part like this:
...
if (record instanceof Date) {

      var item = new Date(record);
      _.extend(item.__proto__ , record.__proto__);

      this.object[association] = item;

      // _.isObject() does not match null, so we're good here.
    }
...

The question is do we need _.extend(item.__proto__ , record.__proto__); or it has no sense here, since no one extends Date with custom methods?
@particlebanana
I'll fix and rewrite anything you say.
Thanks for your patience and care.
 We'll get it figured out and merged in. I'll try and take a look at this later today. Thanks for your work!

@particlebanana thanks a lot!
You guys are wonderful!

@particlebanana thank you vary much! You saved our day!
You're totally right about date as PK :)

@particlebanana thank you vary much! You saved our day!
You're totally right about date as PK :)


Micro-metrics-datadog only sends counts to datadog

In DatadogMetricsConfigurer we have the following:
    EnumSet<DatadogReporter.Expansion> expansions = EnumSet.of(COUNT);

This should be:
    EnumSet<DatadogReporter.Expansion> expansions = EnumSet.of(ALL);

This means only count metrics are being sent to datadog daemon

Fixed in https://github.com/aol/micro-server/issues/266


fix: Fixed application segment match_style validation

fix: Fixed application segment match_style validation

Codecov Report
Attention: Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 75.92%. Comparing base (a74ddce) to head (5d3d3b3).

Files
Patch %
Lines

zpa/resource_zpa_application_segment.go
66.66%
1 Missing and 1 partial :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #476      +/-   ##
==========================================
- Coverage   76.19%   75.92%   -0.27%
==========================================
  Files          92       92
  Lines       16818    16817       -1
==========================================
- Hits        12814    12769      -45
- Misses       3145     3187      +42
- Partials      859      861       +2

Files
Coverage Δ

...resource_zpa_application_segment_browser_access.go
59.92% <ø> (-2.29%)
:arrow_down:

zpa/resource_zpa_application_segment_inspection.go
64.98% <ø> (-0.28%)
:arrow_down:

zpa/resource_zpa_application_segment_pra.go
57.89% <ø> (-5.49%)
:arrow_down:

zpa/resource_zpa_application_segment.go
66.80% <66.66%> (-0.15%)
:arrow_down:

... and 3 files with indirect coverage changes


Fail if a message in a non-primary language isn't present in the primary

Checking that all non-'en' messages match an 'en' message could be useful, though this should be off by default possibly, or at least off for WMF config given that TWN likes to remove them itself.

Fixed in 0.1.0.


Add auditory feedback

currently the application is completely silent, these should be some auditory feedback for the user
this is not a necessity

Closed by 9243c78f90927fc93f2ffe4b18b1192cffddb829


[SwiftASTContext] Change GetByteStride() to return an Optional.

Before, we didn't have a way to notify whether the function
errored out or not. NFCI, for now.
rdar://problem/54004017

@lorentey as he wrote the formatters original [he didn't write this bug though :)]

@swift-ci test


how to use epoch for this code?

In other codes I see the number of iteration is often defined by epoch.
I wonder why you use a user-defined iterations in this project.
:)

Using iterations allows for a more precise control over setting the LR decay, fine-tuning, with varying batch-sizes. On the other hand, you don't always want this to be governed by how many times are you going over the whole dataset.
It wouldn't be wrong to say that in many projects, epochs are converted internally to iterations.

If  I want to reproduce the training process in someone else's paper.
For example, if the training set has n pics and the author has trained for 20 epoch.
I shall set the iteration  = n * 20 and val-iters = n.
Is this setting right?

An iteration is an update of one batch. So to get iterations from data size and epochs you need to do:
iterations = (data size * epochs) / batch size

You could of course use the inverse of this to print out your epoch number in train.py. Something like this:
epoch_no = iterations * batch_size / len(data_loader)
I'll close this issue, but please reopen if you have a problem implementing either of these solutions.

@james-nvidia Hello!
And where can I find the data_loader in the code?


Update Copyright Year & Remove reference line of sample template

Remove line "Blog posts, websites, academic papers, or books that mention the metric and provide more background." from references

Thanks for taking a look at these @TejasMate -- we'll review in this week's DEI meeting

Sorry @germonprez, I want to commit for #431 issue mistakenly use same branch :(

Thanks @TejasMate. Could you break this out in to 3-4 different PRs -- one per document change?

Yeah, sure. I will follow these while PRs

Closing without merging. We have captured the recommendations for each of the respective documents. We will issue PRs against the single documents where appropriate. Thanks!


Skip only faulty entries

Inspired by https://tex.stackexchange.com/q/497829/35864 I am wondering if Biber could be slightly more tolerant about malformed .bib entries.
Currently a malformed .bib entry like in the following example completely aborts the Biber run and leaves an empty .bbl file.
\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{csquotes}

\usepackage[style=authoryear, backend=biber]{biblatex}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@book{appleby,
  author  = {Humphrey Appleby},
  title   = {On the Importance of the Civil Service},
  date    = {1980},
}
@book{bppleby,
  author  = {Humphrey Bppleby}
  title   = {On the Importance of the Civil Service},
  date    = {1981},
}
@book{cppleby,
  author  = {Humphrey Cppleby},
  title   = {On the Importance of the Civil Service},
  date    = {1982},
}
\end{filecontents}

\addbibresource{\jobname.bib}

\begin{document}
\cite{appleby,bppleby,cppleby}
\printbibliography
\end{document}

The error message by Biber
ERROR - BibTeX subsystem: <file>, line 12,
        syntax error: found "title", expected end of entry ("}" or ")") (skipping to next "@")

seems to suggest that only the faulty entry is skipped. I suspect the message comes from btparse, so I assume that btparse would have the ability to just skip the faulty entry and press on, that's why I'm asking if this would also be possible for Biber.
Also related https://github.com/plk/biber/issues/261

Fix in DEV. Note that this error means that it skips from where it finds the error to the next entry, leaving an entry with incomplete data. It doesn't skip the entry and there is no way to do that. btparse doesn't do that and due to the old-fashioned way btparse handles STDERR, it can't be caught in real-time so that biber could skip the whole entry.

Ah, it would have been a nice added bonus if had been possible to completely remove the entry as that would have been more visible for the user (surprisingly few people read the .blg or Biber output due to their editor setup), but if it is not possible to do that we at least emulate BibTeX in that behaviour.
As in #261 I think it would be much nicer to display the
[341] Utils.pm:300> WARN - BibTeX subsystem: C:\Users\Moritz\AppData\Local\Temp\34LxXYaFt1\wuljdhsjsh.bib_12600.utf8, line 12, syntax error: found "title", expected end of entry ("}" or ")") (skipping to next "@")

in the log as a real ERROR. Is that possible? (It seems that the only change here was biber_error, to biber_warn, so I assume some work would be needed for an error that doesn't abort processing...)

It's a pain that bibtex/btparse don't report the entries with errors, just the line numbers in the file because they are basically just streaming the file.
Can you try DEV again now for the error?

Thank you. The example works as I expected now.
It would be great if there was some way to identify the problematic entry, but BibTeX is not better in that regard.
Previously I found it more useful to plough on despite the error, but I can definitely see the appeal in completely aborting the Biber run and producing an empty .bbl to tell the user that something is off (a half-processed entry is much less visible). What do you think?

Well, there is already the biber option --nodieonerror so people could choose. Currently, the fix overrides this option but I can easily make it obey that option - the default is always to die on errors.

Ah yes, given all that I guess I'll change my mind and vote for making Biber die on these kind of errors by default again and have the user override that explicitly with --nodieonerror.

Fixed in latest 2.13 release.


Weekly Scrum - March 31, 2022

Last Week:

#13

This Week:

#28

Blockers:

None

Last Week:

#14

This Week:

#30

Blockers:

None

Last Week:

#16
This Week
-#27
Blockers:
None

Last Week:
https://github.com/James-LG/SYSC4806-Term-Project/issues/15
This Week:
https://github.com/James-LG/SYSC4806-Term-Project/issues/29
Blockers:
None


Automatic logout

Hi @bersace!
Do you know how i can automatic logout a user from postgres when this user is removed from a group in active directory?
Because the ldap2pg remove all permission from user when they are removed from the group, but they still connected in the database and only when they make logout, they cant login in the database. I would like to know if is possible to do automatic logout by the ldap2pg.

Hi @diegodblr Good point. The idea is to kill backends with targetted user before dropping it ?

The idea is just logout the user at same time that the user is removed from the active directory group.
Example: i have two groups in active directory - database1 and database2. My user is in group database1 and i have owner permissions in the database1 on postgresql. I am loged in to this database1. But i resolve to remove my user from database1 group in active directory. Now i am in the database2 group. But i am still loged in the database1 and only if i logout and try to login to database1 i will be denied.  But for security reason, it is better if the ldap2pg when detect the user is out of the group, automatic force logout of this user from the database1.

Maybe we can implement a more general approach like executing a query after CREATE ROLE, ALTER ROLE and DROP ROLE? A default for dropping roles could be the query to kill the session, but other queries like refreshing views could be configured, too.

Maybe we can implement a more general approach like executing a query after CREATE ROLE, ALTER ROLE and DROP ROLE? A default for dropping roles could be the query to kill the session, but other queries like refreshing views could be configured, too.

Yep. What do you think of https://github.com/dalibo/ldap2pg/issues/171#issuecomment-491708103 ?

Thanks for referencing. Think this would solve the feature request, too.


Documentation

Hi,
I was just wondering if you have some basic documentation about this. Interested in using it but just wondering if its the right fit.

Hi...the best documentation exists in the README.md file, but nothing else really exists, my apologies.
This project was really started as an experiment / Proof of Concept, which I believe worked quite well.  Unfortunately, I don't have much use for it in my day-job, as we're not using JPA, but hopefully that may change in the near-future.  Also, this means that I don't get a lot of time to dedicate to this project.
However, I'm extremely open to collaboration, and accept pull-requests quite frequently.  Please, feel free to make this project better for the community, and contribute!


WIP: [python-package] [ci] use setup.cfg and pyproject.toml for package-building configuration

Contributes to #5061 .
This PR proposes moving some of the configuration for how to build the Python package out of setup.py and into the two configuration files mentioned in PEP 517 and PEP 518.

setup.cfg: package metadata like license file, requirements, trove classifiers, etc.
pyproject.toml: description of which "build backend" to use and what is required to build sdists and wheels

This shouldn't break anything about LightGBM's existing Python package source distributions or wheels. But it gets the project a bit closer to supporting the package-building standards described in PEP 517 and PEP 518.
Shouldn't cmake be a build dependency?
Probably, but I'm proposing deferring that to a later PR. Including cmake as a build dependency in pyproject.toml causes installation of the cmake from PyPI, regardless of whether or not you had separately installed cmake another way.
If you're on a system that PyPI doesn't have a cmake wheel for, that means building lightgbm from source will also require building cmake from source. See for example the failed CUDA CI jobs from an earlier state of this PR (build link).

Failed to build cmake
ERROR: Could not build wheels for cmake, which is required to install pyproject.toml-based projects
error: subprocess-exited-with-error

How I tested this
References

PEP 517: "PEP 517 – A build-system independent format for source trees"
PEP 518: "Specifying Minimum Build System Requirements for Python Projects"
"Dependencies management in setuptools" (link to setuptools docs)
PyPA docs on pyproject.toml (link)

I've learned a lot since first opening this.I'm going to close it, lock it, and open a new PR with a different proposal in the next few days.


Namespaced fields that start with "r" incorrectly processed by BGE

Details
This bug is specific to this area of code and was discovered during development - https://github.com/SalesforceFoundation/NPSP/blob/4d80ddc64569bc1cbf82154525638512e7244ce8/src/classes/BGE_BatchGiftEntry_UTIL.cls#L296
The intent of the code above is to take a lookup field that ends in __r and replace that __r with a __c
If a field with the name npsp__Random_Name__c is mapped and used during entering Gift Entry, the lines of code above will assume that the field is a lookup field and excessively shorten it to fieldName == 'npsp__c' after lines 296 and 297 run.
Fix
Change 296 to use .endsWith instead of .contains.
Change 297 to use .substringBeforeLast instead of .substringBefore

W-9987343

This issue has been migrated to the Known Issues site: https://trailblazer.salesforce.com/issues_view?id=a1p4V0000029kRnQAI&title=gift-entry-namespaced-fields-that-start-with-r-are-incorrectly-processed-by-gift-entry
Please follow the status of the issue there and select "This Issue Affects me" if you are affected by the issue.


[WIP] User Agent now reports the OS platform

This pr fixes #4913, I have not written any tests, If these are required then please notify me of this and Ill add them

Minor Nit: I suggest you change the title of the PR to something like 'User Agent now reports the OS platform'.  When I first read the title, I was expecting to find a behavioral change that is dependent on the OS platform.

@LDSpits Thanks for the update. Can you please add corresponding tests? You can use https://github.com/PowerShell/PowerShell/blob/master/test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1#L456 as an example to get user-agent.

@LDSpits One more thing, when you push new commits, please make sure the message of the last commit contains [Feature] in it so that it will trigger a full test run. Web cmdlets tests by default are not included in CI runs, so we need a full test run to validate changes related to them. Thanks!

@iSazonov @daxian-dbw @markekraus Is it really nessecary to move the tests to a new file? shouldn't we add the test to WebCmdlets.tests.ps1 since most of the infrastructure required for the tests is already inside of there?

@LDSpits My preference would be for the tests to remain in WebCmdlets.tests.ps1 along with the other User-Agent test. They should be new It blocks in the same file preferably right after the current ones. Here is the one for Invoke-WebRequest:
https://github.com/PowerShell/PowerShell/blob/2e901e53c3c1bfc3eebd66c3ae3223cd3ca6a499/test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1#L456-L467

@iSazonov @daxian-dbw @markekraus can I ask a few questions about tests? I was in the process of adding tests for execute-webrequest. I have copied over most of the code from the Invoke-WebRequest returns User-Agent test with only changing the regex to '.*\(Windows NT \d+\.\d*;.*\) PowerShell\/\d+\.\d+\.\d+.*'. this test failed however. so I tested the regex using powershell (to ensure i was using the same regex parsing engine as pester). However this returned true. So now im puzzled as to why it failed. Could you maybe give some pointers?
powershell code used to test regex:
PS C:\Users\spits\Desktop\Development\Powershell\PowerShell> $bingdings =[System.Reflection.BindingFlags]::NonPublic -b
xor [System.Reflection.BindingFlags]::Static
PS C:\Users\spits\Desktop\Development\Powershell\PowerShell> [Microsoft.PowerShell.Commands.PSUserAgent].GetProperty('Us
erAgent',$bingdings).GetValue($null,$null) -match '.*\(Windows NT \d+\.\d*;.*\) PowerShell\/\d+\.\d+\.\d+.*'
True

Current (windows) test:
It "Invoke-WebRequest returns Correct User-Agent on Windows" {

        $uri = Get-WebListenerUrl -Test 'Get'
        $command = "Invoke-WebRequest -Uri '$uri' -TimeoutSec 5"

        $result = ExecuteWebCommand -command $command
        ValidateResponse -response $result

        # Validate response content
        $jsonContent = $result.Output.Content | ConvertFrom-Json
        $jsonContent.headers.'User-Agent' | Should MatchExactly '.*\(Windows NT \d+\.\d*;.*\) PowerShell\/\d+\.\d+\.\d+.*'
    }

pester test message:
Describing Invoke-WebRequest tests
 [-] Invoke-WebRequest returns Correct User-Agent on MacOSX 6.87s
   Expected: {Mozilla/5.0 (Windows NT; Microsoft Windows 10.0.16296 ; nl-NL) PowerShell/6.0.0} to exactly match the expression {.*\(Macintosh;.*\) PowerShell\/\d+\.\d+\.\d+.*}
   at line: 466 in C:\Users\spits\Desktop\Development\Powershell\PowerShell\test\powershell\Modules\Microsoft.PowerShell
.Utility\WebCmdlets.Tests.ps1
   466:         $jsonContent.headers.'User-Agent' | Should MatchExactly '.*\(Macintosh;.*\) PowerShell\/\d+\.\d+\.\d+.*'
 [-] Invoke-WebRequest returns Correct User-Agent on Linux 1.14s
   Expected: {Mozilla/5.0 (Windows NT; Microsoft Windows 10.0.16296 ; nl-NL) PowerShell/6.0.0} to exactly match the expr
ession {.*\(Linux;.*\) PowerShell\/\d+\.\d+\.\d+.*}
   at line: 479 in C:\Users\spits\Desktop\Development\Powershell\PowerShell\test\powershell\Modules\Microsoft.PowerShell
.Utility\WebCmdlets.Tests.ps1
   479:         $jsonContent.headers.'User-Agent' | Should MatchExactly '.*\(Linux;.*\) PowerShell\/\d+\.\d+\.\d+.*'
 [-] Invoke-WebRequest returns Correct User-Agent on Windows 1.08s
   Expected: {Mozilla/5.0 (Windows NT; Microsoft Windows 10.0.16296 ; nl-NL) PowerShell/6.0.0} to exactly match the expression {.*\(Windows NT \d+\.\d*;.*\) PowerShell\/\d+\.\d+\.\d+.*}
   at line: 492 in C:\Users\spits\Desktop\Development\Powershell\PowerShell\test\powershell\Modules\Microsoft.PowerShell
.Utility\WebCmdlets.Tests.ps1
   492:         $jsonContent.headers.'User-Agent' | Should MatchExactly '.*\(Windows NT \d+\.\d*;.*\) PowerShell\/\d+\.\
d+\.\d+.*'

@LDSpits It looks like you are not using a version of PowerShell Compiled with your changes. The pester output shows Mozilla/5.0 (Windows NT; Microsoft Windows 10.0.16296 ; nl-NL) PowerShell/6.0.0 as the user-agent. not that the <major>.<minor> is missing after Windows NT

@iSazonov @daxian-dbw @markekraus the Travis build is erroring? had a problem with the previous build on the http proxy tests...

@LDSpits Its related to #5005. We need to fix the other delay tests. I'll create an issue for it.

#5035 has been merged and I restarted the CI builds. Hopefully the CI will pass this time.

Let's hope they'll pass this time 😄

@LDSpits Thanks for your contribution!
@markekraus @iSazonov Thank you for the thorough review!

@LDSpits Thanks for doing this!

@daxian-dbw @markekraus No problem, expect to see me around 😄


Issue with k8s.io/docs/tasks/configure-pod-container/pull-image-private-registry/

This tutorial has a subtle but important ambiguity: Should the base64-encoded registry secret be the YAML format of the registry secret, or JSON format? The tutorial mentions both, but I'm not sure if the JSON portion was just talking about user-facing files, or if the tutorial wants the user to use one or the other, or perhaps it doesn't even matter? Please have the tutorial focus on only the formats that should actually be used for this kind of configuration.

Hi @erictune 👋 Could you please take a look at this issue and provide necessary recommendation to fix https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
Thanx

Evidentally the base64-encoded portion should be in JSON format, while the overall secret is often in YAML.


TTL / Job

There should be a server side (redis scripts?) TTL mechanism which prevents stuck/stale active jobs.
client node.js worker processors should be aware of job's TTL being exceeded if they are live

http://stackoverflow.com/questions/11810020/how-to-handle-session-expire-basing-redis/11815594#11815594
https://groups.google.com/forum/#!topic/redis-db/rXXMCLNkNSs

So, is this getting removed? :(

Semantically yes, this is split into two features

stalled jobs: the timeout when a worker is crashed/restarted and it's job stays active
active job expiry: the timeout when a worker is alive, but the user's app is not calling done in time


[TF-935] Replaces '-Xllvm -run-jvp-generation' by '-enable-experiment…

…al-forward-mode-differentiation'.
-enable-experimental-forward-mode-differentiation is a frontend option that is clearer on it's intent and more
accessible. It defaults to false.
Resolves TF-935.

thank you for the review @dan-zheng, I addressed your comments on 6527605

@swift-ci please test tensorflow

@swift-ci please test tensorflow


C# union types additions

A few additions to your current PR.
Did a few C# refactorings with resharper and the 2space indentation.
Also replaced yammer assembly reference in the typeconverter class with Assembly.GetExecutingAssembly.
Regards and thanks for the inital PR!

Have there been any updates on the PR for support for union types in C#?  I see this is still only existing as a separate branch.

This PR lacks attention from write access folks and gets outdated from time to time introducing merge conflicts.


Relax the configuration file format

When getting started with CmpRaComponent, even a basic "hello world" example can be a challenge, because it requires the user to fill out some boilerplate structures in the configuration file. This is especially problematic if the user is not yet familiar with the meaning of every option.
An example is when empty structures need to be created:
DownstreamConfiguration:
  - VerificationContext:

A simpler and more compact config could let the user omit such entries altogether. In the current situation, these lines are only here to help the parser (at the expense of extra work for the user).
Another example is to consider a more compact notation like ForceRaVerifyOnUpstream: true  instead of the current:
ForceRaVerifyOnUpstream:
  - value: true

My understanding is that these are effects of using https://github.com/eclipse-ee4j/jaxb-ri for automatically processing YAML files into Java objects. Perhaps some improvement can be made by adjusting parameters of jaxbi, or maybe there are other types of low-hanging fruits.

Configuration file parsing is done in https://github.com/siemens/LightweightCmpRa/tree/main/src/main/java/com/siemens/pki/lightweightcmpra/configuration  , this CmpRaComponent uses only  interface implementations from https://github.com/siemens/cmp-ra-component/tree/main/src/main/java/com/siemens/pki/cmpracomponent/configuration


Integration of collection entries for heatadapt data

HAUC1_land_surface_temperature_analysis HAUC2_ipcc_scenarios_cities HAUC3_sealing_impact_cities

Will not be merged in this form. Pending decision. Converting to draft


AC Iptc4xmpExt:CVterm guidance may be inconsistent with IPTC spec

About http://terms.tdwg.org/wiki/Audubon_Core_Term_List#Iptc4xmpExt:CVterm

In private email about an example he constructed, Steve Baskauf wrote:
"Probably not using Iptc4xmpExt:CVterm correctly.  I'm using it with a
URI-value but I'm not sure if it should be a literal.  I'm linking to the
standard view URIs I've defined, but I don't think that's right either. "

I replied approximately:

I think the AC guidance for this term is somewhere between misleading and
wrong. We don't specify  which of the several supported implementations in [1]
we mean to be applicable.  My feeling is that the original MRTG intent for IPTC
terms was highly motivated by xmp usage.  For that, [1] is unambiguous: the
value is a bag URI. (I find no definition of "bag" in [1], but that is a minor
point.) In summary, I think the suggestion in the AC spec that literals can be
used is simply contradicted by the IPTC spec.  Not to mention that we know what
problems arise in RDF if a predicate is used with both a URI and a literal. As
a further aside, [1] happens to specify
   (a) "Required CV Any Controlled Vocabulary registered with the IPTC for this field. "
   (b) "Implementation Note(s) This property is part of the Photo Metadata 2008 specifications, but should not released to the public on the standard Adobe Custom Panels for IPTC metadata or other user
interfaces unless agreed by the IPTC."

[1]
http://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata-201007_1.
pdf

Original issue reported on code.google.com by morris.bob on 15 Nov 2013 at 6:36

I believe that from [1] (?) we can conclude that  there is always(?) a  Iptc URI that has the form of an http URI but is not resolvable.  This implies (?) that such each such thing is a literal.

Closing this as it is now included in Issue #115


⚠ Bofrost.fr has degraded performance

In b04c4b7, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1035 ms

Resolved: Bofrost.fr performance has improved in 41901ab after 7 minutes.


Refactored HypixelRankPlusColor to HypixelColors and added documentation

The HypixelColors class can be used for more general purposes than just rank plus colors, such as pet colors, guild tag colors and rank colors.

Looks good


Remove online autocomplete provider

Dear *,
with this change, the auto-completion provider doesn't go online anymore to search for pulldown-content. It immediately takes the content from the html file provided in the resources-root of the jar file.
Furthermore, I updated this HTML file.
This might be related to the bug reported here: https://github.com/fiji/fiji/issues/217
Cheers,
Robert

Great!


nixos/*: more options md conversion

Description of changes
this time around we'll convert most manpage references properly. to allow lossless conversion they'll have to converted to not contain any unnecessary whitespace, which makes the docs a little bit less readable for the moment.
nix-doc-munge has also learned to edit modules that produce warnings (eg because an option doesn't have a type) and to edit modules inside the nixpkgs tree (which is very slow, but doesn't break dependencies for modules that include stuff relative to themselves).
724 option descriptions to go!
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

merging since the first commit produces only encoded-whitespace changes to options.json, and the second commit produces no changes to options.xml (from which everything else is derived).


connect() hangs with no info why on Cube Orange

I'm running dronekit on NVIDIA Jetson Nano connecting through UART with Pixhawk Cube Orange.
dronekit version = 2.9.2
pymavlink version = 2.4.9
python version = 3.6.9
After running mavproxy.py --master=/dev/ttyTHS1 --baudrate=115200
I'm getting:
Connect /dev/ttyTHS1 source_system=255
Log Directory:
Telemetry log: mav.tlog
Waiting for heartbeat from /dev/ttyTHS1
MAV> online system 1
STABILIZE> Mode STABILIZE

but after running
>>> import dronekit
>>> v = dronekit.connect('/dev/ttyTHS1', baud=115200)

It hangs without showing anything more (not even heartbeat timeout), so it seems like heartbeat is present.
I can see the some incoming messages every second after running screen /dev/ttyTHS1. I was also trying to set flag wait_ready to False, but without success.

Try a different baud rate? Check what the Pixhawk is expecting in Mission planner and use that rate, but default is 57600 I think.

I've changed baudrate in mission planner to 115200 and protocol to Mavlink2. I'm also using the same baudrate in mavproxy.py run

Hi @siadajpan , do you find solution? I have the same issue but on Jetson Xavier. When connecting using mavproxy.py on Xavier, it shows only:
mavproxy.py --master=/dev/ttyTHS0 --baudrate 921600
WARNING: You should uninstall ModemManager as it conflicts with APM and Pixhawk
Connect /dev/ttyTHS0 source_system=255
Log Directory:
Telemetry log: mav.tlog
Waiting for heartbeat from /dev/ttyTHS0
MAV> online system 1
ALT_HOLD> Mode ALT_HOLD
While connecting using mavproxy.py on Ground Computer, it shows more information:
mavproxy.py --master=/dev/cu.usbserial-AD0K5Q02
Connect /dev/cu.usbserial-AD0K5Q02 source_system=255
Log Directory:
Telemetry log: mav.tlog
Waiting for heartbeat from /dev/cu.usbserial-AD0K5Q02
MAV> GPS lock at 0 meters
AP: GCS Failsafe Cleared
online system 1
ALT_HOLD> Mode ALT_HOLD
AP: PreArm: Radio failsafe on
AP: ArduCopter V4.0.4 (40502bd9)
AP: ChibiOS: d4fce84e
AP: CubeOrange 0026003A 34385111 31373439
AP: RCOut: PWM:1-12
AP: Frame: QUAD
In addition, when I change mode from Ground Computer, it works but it show nothing after typing the syntax on Xavier.
When I change mode from Ground Computer, the new mode still be synced on Xavier console.
Anyone has a clue?
Thank a lot,

I have the same issue. Did you find a solution?


[MM-20377] Convert persist_gate to TypeScript

Summary
This pull request converts the persist_gate component in the webapp to TypeScript. To do this I had to add a new initialization variable definition to the redux storage type.
Ticket Link
Fixes mattermost/mattermost-server#16730
https://mattermost.atlassian.net/browse/MM-20377

Hello @LauSam09,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

Hello @LauSam09,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

/update-branch

Thanks again for the PR! This'll be on Mattermost Cloud in a few weeks and then Mattermost 5.33 a couple weeks after that.


feat(functions): ✨ Trim ending spaces off name, no error throw.

Make it so that you may enter a name with spaces at the end. The code will trim it for you and won't throw an error. This works for both splash and join and does still trim all the ending spaces off.

In addition to trimming before passing to the validator you should trim before passing to the backend


"docs" and "examples" folders are missing from the actual plugin in Godot

Whether downloading from the Github as a zip, or from the Asset Library in Godot or on the site, the resulting plugin misses the documentation folder.
Furthermore, I don't think I see an option to download them separately on Github either, due to them missing from the generated zip file...

You have to clone the repository to get the examples and docs folders. They are stripped out of the release downloads automatically.


Background color

The app displays background as white, but after saving the image it's black (if the resulting file is opened by another app). In fact, the background is transparent which results in different filling in different apps.
The color must be explicit in the code, it should be possible to change it using #6 as well. Transparent should be one of options for the color.

Is it still open to work

Implemented in #41


bug: esbuild-repl is creating way too many workers for hljs

That's true. I will have to refactor it again since I'm still a newbie at svelte.

Cool


add scrape annotation to service template

We want to scrape our admission controller. I am not sure if this is everything we need. Does anyone know if there is more to it? Do we have to do any fancy thing in g8s-prometheus or prometheus-config-controller?

I guess we need to expose the default metrics, which is not yet in place: https://prometheus.io/docs/guides/go-application/#how-go-exposition-works

So far all microservices had this builtin because we used microkit. I am not sure why this one here should be different.

Then let's followup on this. I will try to create a new PR with microkit. Not blocking you here

I think this should already work. It could be that we also have to define the metrics port but if it's the default that shouldn't be needed.

The project does not use the microkit package and it does also not use the prometheus client packages. There is no metrics endpoint defined. I am not sure how this should work as of now. I will merge this PR and then we need to follow up on this still. I tried running the binary locally but couldn't because of all kinds of missing flags and expectations the microservice apparently has about its process environment.


Expose dataset profile logging

We should expose dataset profile logging directly.  This is very useful.  Users should be able to override dataset metadata when logging as well (such as dataset name).

@andy-dng What do you know, @ibackus called this a few days ago before my urgent plea.
This has been completed with merge #38.


Enhancement Suggestion - Text filters or sanitization rules

Any plans on adding text filters or a similar way of sanitization?
Sometimes you repeatedly have to do some same kind of sanitization or formatting  on the received input. For example, trimming the input, making it lowercase etc.
What if we enhance the validator to accept filters to apply i.e. something like
$rules = [
    'email': 'required|filter:lower,trim',
    'name': 'required|filter:strip_tags',
    'attempts': 'int',
    'amount': 'double'
];

Or we may implement something different altogether.
A different service may be e.g. if we pass it the request object, it will mutate the data in the request itself.
$sanitizer = Sanitizer::make($request, [
     'email': 'lower|trim',
     'username': 'strip_tags',
     'attempts': 'int',
     'amount': 'double'
]);
$sanitizer->sanitize();

// `$request` will now have sanitized/filtered data set in place.

If array is passed, items will be mutated and returned i.e.
$sanitizer = Sanitizer::make($input, [
     'email': 'lower|trim',
     'username': 'strip_tags',
     'attempts': 'int',
     'amount': 'double'
]);
$input = $sanitizer->sanitize();

// `$input` will now have sanitized/filtered data set in place.

I am in the last phases of releasing a Laravel package for this (using second method), thought I should ask if I may create a PR with this.

Thanks for getting in touch @kamranahmedse. Ping @taylorotwell.

I discussed a similar proposal with @taylorotwell yesterday. My specific example was in a Request class, but it could be added as an argument to a normal Validator if required.
My proposal looks like the following (inside a Request class):
public function filters()
{
    return [

        'pre' => [
            'first_name' => ['trim', 'strip_spaces'],
            'reference_number' => ['trim'],
        ],

        'post' => [
            'reference_number' => ['uppercase'],
        ]

    ];
}

In the above example, before validation, first_name would be trimmed and have spaces removed, and reference_number would be trimmed.
After validation passes, the data available on the Request would have the reference_number uppercased.
One thing to mention is whether or not the pre rules apply to the data that is set on the Request after validation is successful. Personally, I think I would prefer to only have the result of the post filters applied to the original input. This gives the opportunity for different processing to happen before validation than to the final result which might be useful in a case where you need to trim to do some comparison, but you actually want to store the original data the user entered.
I think this separation of these filters makes it obvious what they do, and separates them out from the $rules array to keep that clean.

Hmm... I like the general idea of filters. However, I don't believe validation should modify the output. Now, the pre concept mentioned by @mdavis1982 is interesting. However, I don't quite see the point, as all that's doing is modifying the input for validation, in which case you might as well just use another validation rule, no? I do however like the idea of post (and especially ensure that the filter is applied after validation).

I like the idea of a laravel-y way of adding sanitizers (probably integrated into Request objects), but I don't think they belong in validators. Sanitizing and validating are different concepts and should be kept apart.
I agree with having the separate Sanitizer class that looks similar to a validator, but returns mutated data instead of validated data, and have it be built-in to form requests.
From a UX perspective, I'm a heavy proponent of sanitizing + sensible defaults over validation, so I'd love to see this pass.

Agreed with @phroggyy and @rizqidjamaluddin. Validation and sanitation are separate concepts and should have separate APIs. But having a sanitation option for input would be quite awesome (especially if you can extend it like you can the validators so you could write your own re-usable sanitizers).

I agree, too... I didn't know if there was an API parity requirement between the Validator and the Request classes (like how you specify rules() in the Request class, but pass $rules to the Validator).
I think this is an ideal opportunity to apply filters to data - but I would like to see it 'baked in' to the Request class so that you can just specify a filters() method and return the filters the same way you do validation rules and messages.

Funny enough, I'm not usually one to enjoy shortcuts (and I certainly am not a fan of pseudo-magic strings) - but I think a laravel-style sanitizer fits right into many workflows and I do find myself doing a lot of triming and intvaling user input - often a good chunk of a controller is just this. Having a sanitizing shortcut would mean I can more comfortably do $request->only() without having a bunch of pre-processing scripts.
One thing to carefully note is that this practice mutates data before it gets validated, which means validation error messages may not match the provided input from the user.

@rizqidjamaluddin ^^What I had in my mind was passing the request object. And then it will automatically mutate the data in the request object.
But yeah, it could be made as to mutate both ways i.e. if the request object is passed, params will be mutated and set in the request. Or if it is an array mutate and return the array.

I think binding sanitizing to the request mechanism is too far interlocked - it's not uncommon to want to sanitize other data, like incoming information from an API request, parsing an inbound email webhook or just people who still do $request->all. I'd suggest that having a $sanitize property in a FormRequest should suffice for people who want the automagicky parts.

@rizqidjamaluddin What do you think about using a middleware in case of filtering?

At first view, middleware might be a thing.

Hey everyone,
Anyone willing to make a PR for this?

@themsaid I'll look into it this weekend!

@phroggyy I'm going to close this issue since it's a feature request, but looking forward to your PR when you have time :)

@themsaid yep, started on it this morning!
On Tue, 27 Sep 2016 at 16:56, Mohamed Said<EMAIL_ADDRESS>wrote:

@phroggyy https://github.com/phroggyy I'm going to close this issue
since it's a feature request, but looking forward to your PR when you have
time :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/laravel/framework/issues/14467#issuecomment-249883240,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AG65g7IJpNwMFHt5mPO1hsPZno-xrdpKks5quSrwgaJpZM4JT2Lz
.


Tasks for other users using EWS Java API

With the EWS JAVA API I can only create tasks for the signed in user and also the REST APIs don't support tasks.
I couldn't find a way to create tasks for other users.
Is there any way to create tasks for other users using EWS Java API?
I have seen one workaround over internet for EWS .net Managed API,  "You would need to create the tasks in the "assigned" user's Task folder. That would require that the signed-in user have delegate access { Delegate access and EWS in Exchange : https://msdn.microsoft.com/EN-US/library/office/dn641957(v=exchg.150).aspx }
to the assigned user's Task folder, or you'd need to consider using an app-only token to get access to all users' mailboxes. "  , if this workaround works for java api , can you provide sample .
Thanks,
Showkath.

Hi ,
I could able to solve problem by writing below code .
Mailbox mbAssignee =new Mailbox("showkath.naseem@p365microsoft.com");
task.save(new FolderId(WellKnownFolderName.Tasks, mbAssignee));
Hope this helps.
Regards,
Showkath.

Hi ,
I could able to solve problem by writing below code .
                              FolderId parentFolderId = null;
			List<String> assignees = new ArrayList<String>();
			assignees.add("showkath@sap.onmicrosoft.com");
			assignees.add("srimathi@sap.onmicrosoft.com");
                            assignees.add("nitin@sap.onmicrosoft.com");
			for (int index = 0; index < assignees.size(); index++) {
				Mailbox mailbox = new Mailbox(assignees.get(index));
				parentFolderId = new FolderId(WellKnownFolderName.Tasks, mailbox);
				task.save(parentFolderId);
				//To be verify - for each user - how is response

			}

Hope this helps.
Regards,
Showkath.

I think above approach misleads to self-task creation  in their inbox as I have used impersonating users inbox , so this is not right approach for  "How can one user assigning task to other user in O365 "


[ODS-6165] Update Actions Workflows for main-6x

The build failures we see here  are expected, since we haven't finished updating everything to .NET 8. If the code in the pull request looks fine and is approved through code review, then I will apply an administrative override to allow the squash & merge.


Flix scans

Source information
Flix scans 1.4.3
Source language
English
Source new URL
https://flixscans.net
Other details
No response
Acknowledgements

[X] I have updated all installed extensions.
[X] I have opened WebView and checked that the source URL is not updated yet.
[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I will fill out all of the requested information in this form.

The way the API works has changed, changing the URL + API URL is not enough rn


sap_swpm: The variable 'sap_swpm_tmpdir' is undefined when using the tag 'sap_swpm_generate_inifile'

Without this variable being defined, the following error is displayed:
TASK [community.sap_install.sap_swpm : SAP SWPM Pre Install, create inifile - Process SWPM inifile template for creating 'inifile.params'] ***
fatal: [host01]: FAILED! =>
  msg: |-
    The task includes an option with an undefined variable. The error was: 'sap_swpm_tmpdir' is undefined. 'sap_swpm_tmpdir' is undefined

A solution is to add the tag sap_swpm_generate_inifile to the following task:
https://github.com/sap-linuxlab/community.sap_install/blob/ce6c49904024d7b85797445fcd462631eb046262/roles/sap_swpm/tasks/pre_install/swpm_prepare.yml#L4

Solved in https://github.com/sap-linuxlab/community.sap_install/pull/918.


Luckperms 1.12

Is luckperms 4..0.60 Compatible with Spongeforge 1.12? if not is there a version i could get from somewhere?

Yes, all versions are compatible with Sponge API 5-7.
You can download the latest version from here: https://ci.lucko.me/job/LuckPerms/


No app.UseAuthentication() in webapp with individual identity

Here is the generated code by visual studio:
Even though the app does not have app.UseAuthentication(), it has no problem.
using Microsoft.AspNetCore.Identity;
using Microsoft.EntityFrameworkCore;
using WebAppTest.Data;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
var connectionString = builder.Configuration.GetConnectionString("DefaultConnection") ?? throw new InvalidOperationException("Connection string 'DefaultConnection' not found.");
builder.Services.AddDbContext<ApplicationDbContext>(options =>
    options.UseSqlServer(connectionString));
builder.Services.AddDatabaseDeveloperPageExceptionFilter();

builder.Services.AddDefaultIdentity<IdentityUser>(options => options.SignIn.RequireConfirmedAccount = true)
    .AddEntityFrameworkStores<ApplicationDbContext>();
builder.Services.AddRazorPages();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseMigrationsEndPoint();
}
else
{
    app.UseExceptionHandler("/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}

app.UseHttpsRedirection();
app.UseStaticFiles();

app.UseRouting();

app.UseAuthorization();

app.MapRazorPages();

app.Run();

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 94bfd895-d2af-390b-0801-428b36a27ff4
Version Independent ID: 0e0d870e-e49a-e16c-c259-68d58e701b8f
Content: Introduction to Identity on ASP.NET Core
Content Source: aspnetcore/security/authentication/identity.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Thanks for reporting this. Code needs to be updated for ASP.NET Core 6 and later.

Dup of #27219


Add Adyen field to PaymentGateway

Hello,
I am using your library for SiftScience and I like it a lot!
Since I need to use it for Adyen payments, it would be really cool to do it in this library since it is already supported by SiftScience ( https://siftscience.com/resources/references/events-api#complex-fields-payment ) .
Please let me know if you have any question and thank you a lot for your job.

Awesome, thanks for your help


Revert "Breadcrumbs should not be duplicated if there is no mechaism on Android"

Figure out if tests are broken because of this PR
https://github.com/getsentry/sentry-dart/pull/938

Codecov Report

Merging #940 (392f858) into main (46626ac) will increase coverage by 5.61%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #940      +/-   ##
==========================================
+ Coverage   89.94%   95.55%   +5.61%
==========================================
  Files           9        2       -7
  Lines         169       45     -124
==========================================
- Hits          152       43     -109
+ Misses         17        2      -15

Impacted Files
Coverage Δ

dio/lib/src/tracing_client_adapter.dart

dio/lib/src/dio_event_processor.dart

dio/lib/src/sentry_dio_extension.dart

dio/lib/src/failed_request_interceptor.dart

dio/lib/src/breadcrumb_client_adapter.dart

dio/lib/src/sentry_dio_client_adapter.dart

dio/lib/src/sentry_transformer.dart

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46626ac...392f858. Read the comment docs.


Datasets Upload Logic Needs A Change

As of now, if someone uploads a dataset by mistake with the same name in the JSON file and some other dataset has it as well, the CLI will throw error after the upload and all is complete? whereas the logical thing to do is to check for the dataset unique name even before the upload began IMO.
Imagine someone uploading 20-30 gigs and has to redo because of this error.

The way they are storing the information in the backend might play a role here. Because I guess when they search for the exact title, it's not done whether the user who has requested the upload already has a similar dataset with that title or not. They are doing it at a global level,  hence the experience isn't "personalized" to every user kinda analogy.


chrono CVE-2020-26235

Hi! Currently, tiberius depends on chrono  which seems to have a security issue that isn't fixed for a long while already: CVE-2020-26235
Which is raised in the bug tracker as well (see also the last comments there): chronotope/chrono#499
on refinery I changed the dependency to time directly.
Thanks!

Oh god.
This is going to be a massive effort in Prisma. I'm pushing it forward now but I can't give you a proper timeline when I can do any work on Tiberius.
We should maybe have it additionally next to chrono first with a feature flag, then make it the default feature in the next preview release.

This is going to be a massive effort in Prisma. I'm pushing it forward now but I can't give you a proper timeline when I can do any work on Tiberius.

no, of course, I got that feeling when searching in the issue tracker if this was already reported and saw multiple chrono related issues.

We should maybe have it additionally next to chrono first with a feature flag, then make it the default feature in the next major release.

yeah makes sense, I would follow the same approach.
Thanks for your interest!

It's probably not too hard to do. I isolated all chrono functionality to here: https://github.com/prisma/tiberius/blob/main/src/tds/time/chrono.rs
Otherwise the crate should be playing with the date and time formats returned from TDS. If you want to give it a try, I'd add a new module next to the chrono.rs and try to implement the conversions there. I'd love to do this too, but the issue tracker is having everything else in front of this until we can prioritize again and allocate some time for a dev to take a look.
The Tiberius part might be easy, triggering the change in all our org a bit more work.

@jxs there's a PR now open to support time crate. Do you want to check it out and test it? Merging to main soonish, cutting a new version when one other PR gets merged.

Hi @pimeys how are you? Seems I came too late sorry. Thanks for addressing this!


hivemq-platform-operator: issue with enabling prometheus extension

Hello,
I use the recent hivemq-platform-operator and hivemq-platform charts.
I wanted to enabled prometheus metrics for hivemq and therefore tried to enable the related extension in plaform chart:
extensions:
  - name: hivemq-prometheus-extension
    enabled: true

but after that, I'm getting the following error in hivemq-platform-operator:
04:01:44.936 [pool-13-thread-24] WARN  c.h.p.o.d.StatefulSetResource - [hivemq4] Cannot reconcile the StatefulSet: The custom resource spec is invalid
java.lang.IllegalArgumentException: The extension 'hivemq-prometheus-extension' is fully managed by the HiveMQ Platform Operator and cannot be customized via an extension configuration
	at com.hivemq.platform.operator.v1.HiveMQPlatformValidator.lambda$validateExtensions$4(HiveMQPlatformValidator.java:246)
	at java.base/java.util.ArrayList.forEach(Unknown Source)
	at com.hivemq.platform.operator.v1.HiveMQPlatformValidator.validateExtensions(HiveMQPlatformValidator.java:238)
	at com.hivemq.platform.operator.v1.HiveMQPlatformValidator.validateSpec(HiveMQPlatformValidator.java:92)
	at com.hivemq.platform.operator.dependants.StatefulSetResource.reconcile(StatefulSetResource.java:108)
	at com.hivemq.platform.operator.dependants.StatefulSetResource.reconcile(StatefulSetResource.java:55)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowReconcileExecutor$NodeReconcileExecutor.doRun(WorkflowReconcileExecutor.java:123)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.NodeExecutor.run(NodeExecutor.java:22)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

I also set this configuraion for the platform chart:
# Monitoring settings used by the HiveMQ Prometheus extension.
metrics:
  # Set to disable or enable the HiveMQ Prometheus extension.
  enabled: true
  # Defines the port to be used by the HiveMQ Prometheus extension.
  port: 9399
  # Defines the URL path to be used by HiveMQ Prometheus extension.
  path: /

but it made no effect.
So how do I get metrics then?
I looked up in /opt/hivemq/extensions/hivemq-prometheus-extension on the pod and there is "DISABLED" file in this folder, so the init script skips this extension. How to properly enable it then?

Hello @ArtemZ!
Thanks for reaching out.
As you can see from the error log, the HiveMQ Prometheus extension is already bundled with the HiveMQ Platform Operator
The extension 'hivemq-prometheus-extension' is fully managed by the HiveMQ Platform Operator and cannot be customized via an extension configuration

Out of the box, when installing the HiveMQ Platform you will automatically get a a metric service available for you, so you don't really need to do anything else to setup the Prometheus extension to get the HiveMQ metrics.
Of course, you can still configure it by using some of the Helm HiveMQ Platform chart values for the same as documented here: https://docs.hivemq.com/hivemq-platform-operator/configuration.html#metrics-options
Hope it helps.


Rename deque to devec

Hi, me and @al-humea would like to work on this if possible

Hi, me and @al-humea would like to work on this if possible. Should the other related repositories have it renamed as well?

Where are you referring to other repo? Is it from other folders?

Hi, me and @al-humea would like to work on this if possible. Should the other related repositories have it renamed as well?

Oh? Are you referring to those users who have already started using it? Indeed, they need to make changes because the core API is still in an unstable stage, and users need to bear the risk of these consistency issues. If you're talking about something else, you can add it.

I meant the entirety of the moonbitlang project, including the course, documentation and such. So as to have consistent naming across all files.
And thank you for letting us work on this!

I meant the entirety of the moonbitlang project, including the course, documentation and such. So as to have consistent naming across all files.
And thank you for letting us work on this!

Oh... as far as I know, this part may not have been used by those yet, so I should only focus on the core part.

You're right, thank you.
I've noticed a deque is often called "dq", would it make sense to change the variable name into "dv" ?

You're right, thank you.
I've noticed a deque is often called "dq", would it make sense to change the variable name into "dv" (to reference devec)?

just do it.

It is now done and we've opened a pull request. We made sure to make separate commits for each type of rename just in case. :)

done in #357.


Wrong Configuration to enable login data publisher

Is your suggestion related to a missing or misleading document? Please describe.

The configuration given at https://is.docs.wso2.com/en/latest/learn/prerequisites-to-publish-statistics/#step-02-enable-analytics-in-wso2-is to enable login data publisher in wrong.

Describe the improvement

The correct configuration should be as,
[identity_mgt.analytics_login_data_publisher_v110]
enable=true

Optional Fields
Additional context

Related Issues:

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


FIX: Allow filtering to work with auto_select

Background:
https://github.com/seawolf/tty-prompt/blob/master/lib/tty/prompt/list.rb#L223-L225
The List#choices method only returns a Choices object if the list is original and unfiltered (ie. all choices are enabled) - otherwise it returns an array of choices.  So the safest way to determine that there is only one choice available is to check the size property on the return value directly.
Describe the change
What does this Pull Request do?
Why are we doing this?
Any related context as to why is this is a desirable change.
Benefits
How will the library improve?
Drawbacks
Possible drawbacks applying this change.
Requirements

[ ] Tests written & passing locally?
[ ] Code style checked?
[ ] Rebased with master branch?
[ ] Documentation updated?
[ ] Changelog updated?

Looks like a good improvement, thanks!

There is also a bug in the auto select code which means that if you turn on filtering in the menu (useful for many projects), it crashes because it returns an array rather than the choices object.
[Something elsewhere] enables filtering (requires this to be merged to avoid crash) which makes finding answers easier.

I can't remember the details and cases of it, so would like a play just to make sure I wasn't specifically wanting just a single enabled option to be auto-selected.

Does this test need updating?  piotrmurach/tty-prompt@76363c7 (#157)

Not sure about the tests - in terms of behaviour, it was unexpected but worked fine IMO to activate the selection as soon as there's only a single option left.


Update trace4cats-core, trace4cats-fs2, ... to 0.14.2

Updates

io.janstenpickle:trace4cats-core
io.janstenpickle:trace4cats-fs2
io.janstenpickle:trace4cats-tail-sampling

from 0.14.0 to 0.14.2.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (0.14.0).
You might want to review and update them manually.
docs/components.md
docs/examples.md
docs/migrating.md
docs/sampling.md
project/Dependencies.scala

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.janstenpickle" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "io.janstenpickle" }
}]

labels: library-update, early-semver-minor, semver-spec-patch, version-scheme:early-semver, old-version-remains, commit-count:1

Superseded by #206.


Why 'Dart_NotifyLowMemory' was not called when memory warning

Steps to Reproduce

It's easier get OOM after Flutter Add to exists App
I found there is a API call 'Dart_NotifyLowMemory' , why not call it when memory warning
Any other way to avoid OOM ?
thanks

Logs

Fixed by https://github.com/flutter/engine/pull/14517


It seems that AcrylicMessageBox is missed in Nuget package!

Was trying to implement this class from latest Nuget,  but VS couldn't resolve its name (although it resolved the rest acrylic classes),  so I assume it is private or just dropped from it.
Of course I ended up in using the source code directly that contains it and working fine.
And thanks for the great work  ;)

Hi @TMS-Namespace
AcrylicMessageBox is currently preview state, that is only included in preview package.
FluentWPF 0.10.0-alpha001
I'll release AcrylicMessageBox with stable FluentWPF 0.10.0 packages. Please wait for a while

I'll close this issue.
If you have any problems, please reopen this issue.


Addition of sidebar search filter for SideBarLink items

I have updated Sidebar.js to include a search box which can quickly filter the SideBarLink items. I choose to make this addition directly in the Sidebar component as it is very closely tied with the children prop of Sidebar.js.
Additionally, I made a small change to the Examples page such that all direct children of the Sidebar component are SidebarLink components to ensure full compatibility with the new search feature.
Please test the search and let me know what you think, I hope you find it useful!

Issue: https://github.com/sButtons/sbuttons/issues/1446

Thank you for your work!
@all-contributors please add @jazibobs for code


Skip inference on fill_with_nans! unless testing consistency

This PR adds a type, TestDycoreConsistency, so that we can skip inference on fill_with_nans! when we're not running consistency tests. This should be a fine work-around to close #1728.

bors r+


dummy commit for rerendering

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Based on #14, trying to sort linker error

@conda-forge-admin, please rerender

Will reopen in a moment!


Error: 10 ABORTED: Too much contention on these documents. Please try again.

Environment details

OS: Docker Container, linux/arm64
Node.js version: 12.16.1
npm version:
@google-cloud/firestore version: 3.7.1

Steps to reproduce
I'm using a Promise.all to ADD around 6,000 new documents. I am not updating any of these, so I am a bit confused by the contention issues... my understanding is this error could happen if trying to read/update a single document more than once per second.
await Promise.all(
        objects.map(record => {
          return new Promise(resolve =>
            db
              .collection("newCollection")
              .add(record)
              .then(() => {
                resolve();
              })
          );
        })
      );

Error:
Error: 10 ABORTED: Too much contention on these documents. Please try again.
    at Object.callErrorFromStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Http2CallStream.<anonymous> (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:96:33)
    at Http2CallStream.emit (events.js:323:22)
    at /usr/src/app/node_modules/@grpc/grpc-js/build/src/call-stream.js:100:22
    at processTicksAndRejections (internal/process/task_queues.js:79:11)

this can probably close - made an issue about this earlier today https://github.com/googleapis/nodejs-firestore/issues/1020

this can probably close - made an issue about this earlier today #1020

Oh yea, probably related... I think it may have been fixed but not sure if it's been released or not.

@matrix4123 inserting 6000 documents in parallel I think is likely to step on resource limits at the API level (@schmidt-sebastian can correct me if I'm wrong, I'm not an expert on this API).
An approach I might recommend is something like this:
const work  = [] // an array with all the work you want to do.
const WORK_SIZE = 16;
while (work.length) {
  const chunkOfWork = work.splice(0, WORK_SIZE);
  await Promise.all(
        chunkOfWork.map(record => {
          return new Promise(resolve =>
            db
              .collection("newCollection")
              .add(record)
              .then(() => {
                resolve();
              })
          );
        })
      );
}

☝ an approach like this would help control the amount of concurrency.

Hi @bcoe! I have not seen any resource limits specifically around inserting. I was following the starred note from this section:
https://firebase.google.com/docs/firestore/manage-data/transactions#batched-writes

Note: For bulk data entry, use a server client library with parallelized individual writes. Batched writes perform better than serialized writes but not better than parallel writes. You should use a server client library for bulk data operations and not a mobile/web SDK.

I just saw this https://cloud.google.com/firestore/docs/best-practices#high_read_write_and_delete_rates_to_a_narrow_document_range
Couple things to add here:

Each time this runs it is a brand new collection
Not setting the document ID, letting firestore generate it
Not doing any kind of ramp up at the moment

I'll try your recommendation for now.

@matrix4123 we've released as recently as 6 days ago, so #827 I believe should be out the door.
It might be worth waiting for @schmidt-sebastian's advice, who I believe is back soon.

Hello! I have some good news and some bad news: #827 is only meant to address contention that occurs within a transaction. It unfortunately doesn't address the contention that you might be seeing here.
The good news that we are currently working a new API to make bulk data entry easier. This API should ship within the next couple weeks and will implement throttling, which I believe is the one thing that is missing from your code.
In the meantime, I would advise to lease take a look here and see if either the write limits or the suggestion to ramp up traffic gradually apply to your use case: https://firebase.google.com/docs/firestore/best-practices#ramping_up_traffic

@matrix4123 Since there is likely nothing we can in this repro at this time, do you mind filing a support request for this issue? https://cloud.google.com/support-hub
You can also send me your project ID and timestamps for the errors directly and I can follow up on a discussion we already have internally (mrschmidt (at) google.com).

Internal bug: b/154364083

@schmidt-sebastian if i can wait a couple of weeks on this, is there a need to still worry about implementing what you mentioned here. Or should i be fine using the new bulk data entry once it comes out

The new Bulk Writer interface will implement throttling. If that is what you are after, you can wait for it.