Query causes cluster crash

CrateDB version: 3.2.2
Environment description:
12 Nodes, each with 16 CPUs, 64GB RAM, 2GB HDD
When running certain types of queries on our cluster, we experience some sort of a crash.  Generally, the cluster becomes unresponsive within a few seconds of executing the query.  After a few minutes of the query running, the cluster reports job killed, then the web UI shows multiple nodes missing, lots of undereplicated data and shards.  The servers then appear to come back up and and it takes 15-30 minutes re-replicate the data.  We often end up with some tables with "missing shards".  Sometimes I have to manually restart crate on a server if it doesn't come back.
Query can be as "simple" as:
select field1, count(field2) from table1 group by field1, field2.
The tables we're querying contain a 1-3 billion rows, 50+ fields.  They are partitioned by day with 10 shards per partition and 1 replica each.  The servers only consume around 1-5% CPU while ingesting data.  If a query such as above is issued, the CPUs go to 100% on multiple servers and stay there until crash.

Thank you for the report, could you provide some additional information:

Could you post your crate.yml - were there any changes to the circuit breaker settings?
How did you configure the heap size (CRATE_HEAP_SIZE)?
Are there any errors in the logs?
How is the cardinality of the columns you're grouping on?

All circuit breakers are left at default.
I will check the logs to see if anything is still there and get back to you.
Cardinality is fairly low.  Column 1 has roughly 188,000 possible values repeating over 2+ billion rows.  Column 2 has roughly 8,000 possible values repeating over those same 2+ billion rows.  The  resulting data set from grouping on the two columns should be less than 100 million rows.  We are actually trying to insert the result into another table (so no limit is used for the select queries.)  But even if running the select alone  with limit 100, it results in a long running process and crash.

Crate.yml:
license.enterprise: false
cluster:
  name: xxxxxxxx

node.name: "yyyyyyyy"

bootstrap.memory_lock: true

network.bind_host: _eth0_
network.publish_host: _eth0_

discovery.zen.minimum_master_nodes: 7
discovery.zen.ping.unicast.hosts:
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300
  - 10.x.x.x:4300

gateway:
  recover_after_nodes: 12
  recover_after_time: 5m
  expected_nodes: 12

path.data: /cratedb/data

Memory configuration:
/etc/security/limits.conf
crate soft memlock unlimited
crate hard memlock unlimited
root soft memlock unlimited
root hard memlock unlimited

/etc/sysconfig/crate
MAX_LOCKED_MEMORY=unlimited
CRATE_HEAP_SIZE=30500m

Currently the total number of shards on the cluster is around 4000-5000 x 2 (1 replica). Each table is about 300 shards currently + 1 replica.

That's likely too many shards. Cluster stability would likely improve with a less aggressive partitioning / sharding. Going for a weekly partitioning scheme with a bit more shards per partition is often a better tradeoff, as you can overall have less shards while having more shards for the active partitions. (Which in many use cases is queried more often)

I will check the logs to see if anything is still there and get back to you.

Any update here?
 It might help to reduce the heap size further, but it's hard to tell without more data.

We are seeing a ton of these:
[gc][63285] overhead, spent [351ms] collecting in the last [1s]
And also, various timeouts:
[crateserverp2] timed out waiting for all nodes to process published state [37553] (timeout [30s], pending nodes:
[crateserverp2] Received response for a request that has timed out, sent [46643ms] ago, timed out [16808ms] ago, action [internal:discovery/zen/fd/ping]
Caused by: org.elasticsearch.env.ShardLockObtainFailedException: [schema.table][0]: obtaining shard lock timed out after 5000ms
And many shard errors like these:
[ERROR][o.e.c.a.s.ShardStateAction] [crateserverp2] [schema.table][3] unexpected failure while failing shard
org.elasticsearch.cluster.action.shard.ShardStateAction$NoLongerPrimaryShardException: primary term [14] did not match current primary term [15]
crate: java.io.IOException: failed to obtain in-memory shard lock
I increased the discovery.zen.fd.ping_timeout value and the cluster no longer "crashes" on these queries, but it still gets bogged down and becomes unresponsive for a very long time.  I believe the complex queries we issue along with its high data ingest rate makes the servers too busy to communicate  properly (at least that's what it seems like to me).  We will try rebuilding the tables with different partitioning/sharding scheme as you suggested.  Perhaps that could reduce the "chattyness".  Thanks for the suggestion.

You might also want to split out 3 dedicated master nodes (they usually can be lower specced, especially in regards to heap/memory).
This would help as expensive queries would no longer have a negative inpact on the cluster state processing.

Great idea.  If I understand correctly, we would just set node:data=false on the dedicated masters, and node:master:false on the rest of the nodes?

Hello again.
So I've reduced heap to 26G on each node (from 30.5), added 3 dedicated master nodes, and changed partitioning/sharding scheme on high ingest tables to 1 partition/week with 5 shards.  I actually recreated all the tables and copied the data.  We're still experiencing the same issue, maybe it's even worse.  Issuing a query with compound grouping crashes the cluster every time.  On some nodes, crate won't even restart, we have to reboot the server.
I noticed all the data nodes have 100% memory utilization at all times (I assume it's normal because Lucene takes up the rest of the memory?).  We are also seeing some very concerning GC logs.  Here's en example:
2019-05-31T17:01:38.477920+00:00 crate1p6 crate: [2019-05-31T17:01:38,461][WARN ][o.e.m.j.JvmGcMonitorService] [crate1p6] [gc][old][6398][20] duration [1.8m], collections [1]/[1.9m], total [1.8m]/[31.4m], memory [24.7gb]->[24.7gb]/[25.2gb], all_pools {[young] [318.6mb]->[357.6mb]/[798.7mb]}{[survivor] [0b]->[0b]/[99.8mb]}{[old] [24.4gb]->[24.4gb]/[24.4gb]}
2019-05-31T17:03:09.863615+00:00 crate1p6 crate: [2019-05-31T17:03:09,825][WARN ][o.e.m.j.JvmGcMonitorService] [crate1p6] [gc][old][6399][21] duration [1.5m], collections [1]/[1.5m], total [1.5m]/[32.9m], memory [24.7gb]->[24.8gb]/[25.2gb], all_pools {[young] [357.6mb]->[490.3mb]/[798.7mb]}{[survivor] [0b]->[0b]/[99.8mb]}{[old] [24.4gb]->[24.4gb]/[24.4gb]}
2019-05-31T17:05:04.260245+00:00 crate1p6 crate: [2019-05-31T17:05:04,255][WARN ][o.e.m.j.JvmGcMonitorService] [crate1p6] [gc][old][6400][22] duration [1.8m], collections [1]/[1.9m], total [1.8m]/[34.8m], memory [24.8gb]->[24.8gb]/[25.2gb], all_pools {[young] [490.3mb]->[480.4mb]/[798.7mb]}{[survivor] [0b]->[0b]/[99.8mb]}{[old] [24.4gb]->[24.4gb]/[24.4gb]}
2019-05-31T17:06:35.219905+00:00 crate1p6 crate: [2019-05-31T17:06:35,118][WARN ][o.e.m.j.JvmGcMonitorService] [crate1p6] [gc][old][6401][23] duration [1.5m], collections [1]/[1.5m], total [1.5m]/[36.3m], memory [24.8gb]->[24.9gb]/[25.2gb], all_pools {[young] [480.4mb]->[526.2mb]/[798.7mb]}{[survivor] [0b]->[0b]/[99.8mb]}{[old] [24.4gb]->[24.4gb]/[24.4gb]}
2019-05-31T17:08:29.099484+00:00 crate1p6 crate: [2019-05-31T17:08:29,098][WARN ][o.e.m.j.JvmGcMonitorService] [crate1p6] [gc][old][6402][24] duration [1.8m], collections [1]/[1.8m], total [1.8m]/[38.2m], memory [24.9gb]->[24.9gb]/[25.2gb], all_pools {[young] [526.2mb]->[545.9mb]/[798.7mb]}{[survivor] [0b]->[0b]/[99.8mb]}{[old] [24.4gb]->[24.4gb]/[24.4gb]}
2019-05-31T17:11:54.759804+00:00 crate1p6 crate: [2019-05-31T17:11:54,637][WARN ][o.e.m.j.JvmGcMonitorService] [crate1p6] [gc][old][6403][26] duration [3.4m], collections [2]/[3.4m], total [3.4m]/[41.6m], memory [24.9gb]->[25.2gb]/[25.2gb], all_pools {[young] [545.9mb]->[798.7mb]/[798.7mb]}{[survivor] [0b]->[15.1mb]/[99.8mb]}{[old] [24.4gb]->[24.4gb]/[24.4gb]}
Note the sizes and durations on the above.  We also continue to see timeouts and failed shard locks.  The master eventually fails all nodes but they don't ever come back.  I have to restart crate on all of them.
I am starting to thing that we are either missing something critical in our configuration, or perhaps this is some sort of a bug?  Or maybe we're running crate beyond its limitations?  We can throw more servers at this but we've already gone from 5 servers to 12, now 15 and the issues continue.  We're currently at over 200 CPUs and almost a 1TB of RAM.
Any additional help would be most welcome.

I wanted to add that the ingest continues to work phenomenally and takes up very few resources.  Simple select queries also run well.  The behavior that I am describing only happens on very “labor intensive” queries such as select x,y group by x,y queries.

I did some more research and it appears that this behavior is not uncommon with ES and compound aggregations.  ES has a setting called search.max_buckets which seems to mitigate the crashing issue with these killer queries.  It should allow the cluster to gracefully kill the query if too many buckets are being created before it causes a crash.  Does this setting exist in Crate?

Does this setting exist in Crate?

No it doesn't. Our engine for group by execution is quite different from how ES' works.
We've recently made some other changes that should help terminate such expensive queries earlier on high memory load.
If you'd be okay with trading result accuracy you might want to have a look at the downsampling approach explained here

So I've reduced heap to 26G on each node (from 30.5)

Judging from the GC log entries it seems that the workload does actually require quite a lot of live heap memory. I suspect 30.5 might have been the better choice.

Or maybe we're running crate beyond its limitations? We can throw more servers at this but we've already gone from 5 servers to 12, now 15 and the issues continue

Did you notice an improvement going from 5 to 12?
GROUP BY queries are a bit tricky to scale. They're being processed distribued, so adding more nodes generally helps - but the final merge phase must always happen on a single node. So there is a point where that phase might end up taking the majority of the time, especially if the cardinality is high (and I wouldn't consider yours as fairly low).

Thanks for your continual engagement!
Ok we bumped the heap back to 30.5G and cluster stability has indeed improved.  It still  occasionally becomes unresponsive but generally allows us to manually kill the bad query.
When we had 5 nodes, we were experiencing very high CPU load due to high ingest and 1 sec refresh on all tables so it wouldn't be a fair comparison.
The columns we're grouping by are string columns, some of which can be quite big.  In the query that crashes every time, one column is 7 characters exactly, the other column is anywhere from 10 to 2,000 characters (our ingest code limits the field to 2,000) with high cardinality per your assessment.
The queries always run well when we limit the amount of data we are processing (e.g. last 7 days only, pre-filter with a subquery or a view, etc).  So the issue seems to be definitely related to the number of total rows in the table being queried.  When dealing with hundreds of millions of rows, everything is smooth.   The bad behavior starts when we get into the billion+ realm.
Most of our aggregations are in the form of:
select count(a), b from table group by a, b
or
select a, b from table group by a, b
Sometimes we add in max(c), max(d), avg(d), arbitrary(f) etc.  (Side note, ARBITRARY does not seem to perform better than MAX/MIN which seems odd).  Adding these additional computations increases runtime  of the query, but never causes issues.  In other words, if the "group by a, b" works, then it works with other max/min/avg and however many other fields I throw at it.
We will try to add more nodes and see if things improve.  We have some good metrics now for comparisons.  But it sounds like from your post that it may be the 30.5g limit per node that is the bottleneck.  We will let you know.
Currently we are getting around the issue by having our code populate tables with subsets of aggregations, then filtering and joining the tables to get the final results.  It's a lot of extra work but it gets the job done for now.

Update. Over the weekend, 1 of our nodes failed with java heap space exception.  Unfortunately the log got rolled over so we don't have the logs immediately prior.  Here's the last few logs prior to failure from systemctl:
Jun 08 19:41:39 crate1pj crate[14171]: at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.jav...Final]
Jun 08 19:41:39 crate1pj crate[14171]: at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:502...Final]
Jun 08 19:41:39 crate1pj crate[14171]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) [netty-transpo...Final]
Jun 08 19:41:39 crate1pj crate[14171]: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventE...Final]
Jun 08 19:41:39 crate1pj crate[14171]: at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
Jun 08 19:41:39 crate1pj crate[14171]: [2019-06-08T19:41:39,437][ERROR][o.e.i.e.Engine           ] [crate1pj] [wkst...failed
Jun 08 19:41:39 crate1pj crate[14171]: java.lang.OutOfMemoryError: Java heap space
This is the first time we're seeing this error.  The only new change is the addition of these automated "micro-aggregations" that I described above (last paragraph).  They run periodically.
I verified that the jvm is starting with the appropriate -Xms30500m -Xmx30500m flags.

Side note, ARBITRARY does not seem to perform better than MAX/MIN which seems odd

Yes that is a known issue. We currently have a eager value lookup, so even if an aggregation wouldn't require the value we still load it. It's something we would like to fix eventually but we didn't focus on it so far.

This is the first time we're seeing this error.

Unfortunately the circuit breaker that aborts queries if there is not enough memory left is not fully bullet proof. We recently made some improvements (in 4.0) so that it should work better.

Currently we are getting around the issue by having our code populate tables with subsets of aggregations, then filtering and joining the tables to get the final results. It's a lot of extra work but it gets the job done for now.

Could you elaborate a bit on how that looks like? The amount of optimizations we can do to improve the raw select count(a), b from table group by a, b execution are limited. So we're also looking into the direction of making pre-aggregations easier. Knowing more about existing patterns users use will help us to get it right.

We've seen cases where queries are looking over hundreds of millions of records and it causes crate to go into a yellow state where it begins to move shards around (due to the high load)...We're at 30.5gb heap on our nodes (don't run into memory issues at this time)
The queries are relatively simple too (examples below) and there's gotta be some way for us to give (for example) 80% of our CPU to data processing and leave enough for crate to respond to internal communications (which I assume, it doesn't respond within 5s, that it begins moving shards)
Queries:
select count(*), month from mytable where partitionkey >= ? and partitionkey <= ? and date >= ? and date <= ? group by month

select count(*), month, distinct(somecol) as num_cols from mytable where partitionkey >= ? and partitionkey <= ? and date >= ? and date <= ? group by month, num_cols

select count(*), month, anothercol from mytable where partitionkey >= ? and partitionkey <= ? and date >= ? and date <= ?
 group by mnth, anothercol

Essentially, we have dedicated query, master, ingest, and data nodes
We partition by week with 4 shards and our dates are milliseconds

@bputt
Thank you for the inputs, that's very valuable.
Is the distinct in
select count(*), month, distinct(somecol) as num_cols from mytable where partitionkey >= ? and partitionkey <= ? and date >= ? and date <= ? group by month, num_cols

A count(distinct )  ?
Had you considered using the hyperloglog_distinct aggregation instead? That would use less memory for high cardinality columns.

@mfussenegger in addition to that query, yes, we do a count(distinct somecol) we don't have enterprise, so we can't use the hyperloglog_distinct function :(


getHeader and submitBlindBlock latencies

Not sure if this is the best place to track this. Feel free to close this and let me know a more appropriate place, we can move the issue somewhere else.
I've been observing high latencies with both getStatus, getHeader and submitBlindBlock calls using the Kiln relay.
getStatus, between 100-500ms

getHeader, this gotten worse in the last 24 hours, many calls exceeded 1-2s

submitBlindBlock, mostly around 200ms

This is a prysm beacon node getting a beacon block for validator to sign. We are seeing 10-25x latency increases

I understand that many moving parts are currently under optimized, but I still want to open an issue for tracking

That's great, thank you for reporting. Well investigate today and update here.

getHeader has gotten much better 🎉

submitBlindBlock and getStatus are pretty much the same as before. 95% percentile still around ~200-500ms.
We won't be calling getStatus anymore. Only once during startup so that's not going to be an issue for us. Let me know if you want me to close this and just track submitBlindBlock separately

Last 24h for us on Ropsten.
Focus on the pies, we had 6 slow responses (between 1s and 2s) for GetHeader and 3 slow (between 300 and 500 ms) for GetPayload

registration response time improved significantly

@tbenr @terencechain do you have any current charts for the last days/week?

Last 2 days in sepolia:


Suggestion: Dropping pamplets

Dropping pamplets over a sector within a set time before the sector is captured would yield 8% better relation to the civil faction instead of 5%

An interesting idea.

And it could also made them leave the town for some time or ultil you attack it so you dont have to worry about killing civilians


feat(server-asset): custom node loader for .wasm and .bin

todo

[x] .bin

use inlined base64 for vercel edge

[x] .wasm

should we limit to .wasm?module since .wasm esm integration is different?
well, forget about it. if time comes, cloudflare would probably change the convention.

[x] add test
[x] test it on https://github.com/hi-ogawa/next-app-router-playground/pull/8

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#622  👈
#619 : 1 other dependent PR (#621 )
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @hi-ogawa and the rest of your teammates on  Graphite

Merge activity

Aug 21, 6:10 AM EDT: @hi-ogawa started a stack merge that includes this pull request via Graphite.


App devs can debug enclaves using LLDB

Closely related to #435. Part of making the statement of supporting Clang for use with Open Enclave should also include its related tools such as the LLDB debugger. The current oe-gdb implementation takes GCC-specific dependencies today to make GDB work in enclaves (i.e. overloading of the __morestack function for stack stitching)
This requires further investigation as to the amount of work needed to:

[ ] Design an LLDB extension that supports debugging (with stack stitching) an enclave
[ ] Converge oe-gdb implementation where appropriate (and possibly with the events needed for windbg as well)

Hello, I've rewrote existing GDB extension using LLDB Python API and made the PR #997

We're not going to do this (for now).


Taxi dataset unavailable

Running the first cell results in:
Py4JJavaError: An error occurred while calling o194.load.
: Operation failed: "This request is not authorized to perform this operation using this permission.", 403, HEAD, https://contosolake.dfs.core.windows.net/users/NYCTripSmall.parquet?upn=false&action=getStatus&timeout=90

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 65331fde-51d0-365e-e182-2c3b01de8f99
Version Independent ID: 79b4f02b-27a5-019a-1b36-4e80a04f539f
Content: Quickstart: Get started analyzing with Spark - Azure Synapse Analytics
Content Source: articles/synapse-analytics/get-started-analyze-spark.md
Service: synapse-analytics
Sub-service: spark
GitHub Login: @saveenr
Microsoft Alias: saveenr

Hi @parasin Thank you for reaching out. At this time we are reviewing the ask and will provide an update as appropriate

Hi @parasin - Just wanted to double check, if you have placed the sample data ( NYC Taxi data ) in the primary storage account you created for the workspace.
If not, I would highly recommend to complete the other tutorials which are the prerequisites for this tutorial.

You can follow this section to Place sample data into the primary storage account

Please do follow the order and do let us know if you encounter any further issues.
Thank you

Ah, looks like it was on my end. You can close this issue
From: Kranthi Pakala @.>
Sent: Thursday, July 1, 2021 2:47 AM
To: MicrosoftDocs/azure-docs @.>
Cc: Param Singh @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] Taxi dataset unavailable (#77635)
Hi @parasinhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fparasin&data=04|01|Param.Singh%40microsoft.com|0c56cc51e24e49dc1a2008d93c5c09cc|72f988bf86f141af91ab2d7cd011db47|1|0|637607188274467629|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=tu4MiGhH%2B1zujnjWyHxYHAkiBij4IkcjyOoWKyj7PcM%3D&reserved=0 - Just wanted to double check, if you have placed the sample data ( NYC Taxi data ) in the primary storage account you created for the workspace.
If not, I would highly recommend to complete the other tutorials which are the prerequisites for this tutorial.
[image]https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F50185738%2F124078839-368b8a80-d9fd-11eb-9347-bbd93a7502d9.png&data=04|01|Param.Singh%40microsoft.com|0c56cc51e24e49dc1a2008d93c5c09cc|72f988bf86f141af91ab2d7cd011db47|1|0|637607188274477620|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=3Q2yoIOjc%2B2EboaZ9FRxgungDDT3bDXP%2FG2dS678P%2Fk%3D&reserved=0
You can follow this section to Place sample data into the primary storage accounthttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fsynapse-analytics%2Fget-started-create-workspace%23place-sample-data-into-the-primary-storage-account&data=04|01|Param.Singh%40microsoft.com|0c56cc51e24e49dc1a2008d93c5c09cc|72f988bf86f141af91ab2d7cd011db47|1|0|637607188274477620|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=JcI2tXH3eR%2B8%2BrIYTJ66xfldDi3Tp6RvIpk2zRhvJOk%3D&reserved=0
[image]https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F50185738%2F124078947-5de25780-d9fd-11eb-86f0-f449589a0309.png&data=04|01|Param.Singh%40microsoft.com|0c56cc51e24e49dc1a2008d93c5c09cc|72f988bf86f141af91ab2d7cd011db47|1|0|637607188274487615|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=gI7Q6Lc44lWUpJHTuPDJU4FoGdN%2FOSUkfGqc6SgdIAE%3D&reserved=0
Please do follow the order and do let us know if you encounter any further issues.
Thank you

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F77635%23issuecomment-871972219&data=04|01|Param.Singh%40microsoft.com|0c56cc51e24e49dc1a2008d93c5c09cc|72f988bf86f141af91ab2d7cd011db47|1|0|637607188274487615|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=kAVaHa8pS6ijHPdAJhyzOoAnVcXnfSazc6bZ80Dj9CE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FASL7S4D5PWZKE7U4HVJGNSDTVQFORANCNFSM47RAELAQ&data=04|01|Param.Singh%40microsoft.com|0c56cc51e24e49dc1a2008d93c5c09cc|72f988bf86f141af91ab2d7cd011db47|1|0|637607188274497610|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=rb5fgv4Ha7X7%2F5TXhSRlRGUBpirHOZ%2FV6mdFMAo7mKE%3D&reserved=0.

Thanks for confirming @parasin. We will now proceed to close this issue.


Use elasticsearch1 instead of elasticsearch

Using elasticsearch1 allows the projects to use pyelasticsearch in
parallel of a recent version elasticsearch-py with ealsticsearch-dsl

I cannot believe that they used the same namespace for both :face_with_head_bandage:


How do I connect to Log Analytics thru PowerShell?

How do I configure Backup Reports' Diagnostics settings to connect a vault to a Log Analytics workspace thru PowerShell?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4f1ffca1-98ba-0b91-1c6f-a92dde893b30
Version Independent ID: 8e14882f-c7b2-3f27-554f-973a41e1d380
Content: Configure reports for Azure Backup
Content Source: articles/backup/backup-azure-configure-reports.md
Service: backup
GitHub Login: @adigan
Microsoft Alias: adigan

@Ayanmullick Thanks for the Comment. We are actively investigating and will get back to you soon.

The below runs successfully but doesn't update anything for the Vault diagnostics settings.
Set-AzureRmDiagnosticSetting -ResourceId /subscriptions/<SubID>/resourceGroups/<RGName>/providers/Microsoft.RecoveryServices/vaults/<Vaultname> -StorageAccountId /subscriptions/<SubID>/resourceGroups/<RGName>/providers/Microsoft.Storage/storageAccounts/<StorageAccountName> -WorkspaceId /subscriptions/<SubID>c/resourcegroups/<RGName>/providers/microsoft.operationalinsights/workspaces/<WorkspaceName> -Enabled $true -Verbose

@Ayanmullick AFAIK this is currently not supported. Here's a list of supported Log Analytics mgmt. operations via PowerShell.

We will now close this issue. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.


FEAT Add TrustAIRLab/forbidden_question_set Dataset

Name: TrustAIRLab/forbidden_question_set
Link: https://github.com/verazuo/jailbreak_llms/blob/main/data/forbidden_question/forbidden_question_set.csv
Relevant Columns: "content_policy_name","question"

Originally posted by @divyaamin9825 in #429

Describe the solution you'd like
This dataset should be available within PyRIT: https://huggingface.co/datasets/TrustAIRLab/forbidden_question_set
Also available here: https://github.com/verazuo/jailbreak_llms/blob/main/data/forbidden_question/forbidden_question_set.csv
Associated paper: https://arxiv.org/html/2308.03825v2
Additional context
There are examples of how PyRIT interacts with other datasets here: https://github.com/search?q=repo%3AAzure%2FPyRIT %23 The dataset sources can be found at%3A&type=code
[[Content Warning: Prompts are aimed at provoking the model, and may contain offensive content.]]
Additional Disclaimer: Given the content of these prompts, keep in mind that you may want to check with your relevant legal department before trying them against LLMs.

Hi, I am new to contributing but have been looking into this feature and can open a pull request for what I have worked on so far. Can I contribute?

Yes please! Thank you @ritikakumar0204


Mobile/VPN support

Android and iOS both have system-wide VPN configuration support, which seems like the logical way for uProxy to integrate with those platforms.   Problems to solve:

IP->TCP stream reassembly
UDP forwarding to enable DNS (or UDP-DNS emulation using DNS-TCP? What about queries to bad DNS servers?)
Platform VPN API usage

Examples to follow:

Shadowsocks-android
http://darkk.net.ru/redsocks/
http://transocks.sourceforge.net/
http://oss.tiggerswelt.net/transocks_ev/README

@bemasc brought up an interesting point wrt to using the VpnService API in Android.
You must call "protect" on any socket that should bypass the VPN:
https://developer.android.com/reference/android/net/VpnService.html#protect(java.net.DatagramSocket)
In the case of uproxy-lib, this would mean we'd need to intercept any UDP sockets made.
In the cordova build, we can fork the cca implementation of chrome.sockets to take in a VpnService and call protect appropriately on any UDP sockets.

Implemented a Cordova plugin to expose VPN functionality to Android.
Integrated to uProxy.


Release Google.Cloud.ErrorReporting.V1Beta1 version 3.0.0-beta02

Changes in this release:
New features

Enable REST transport in C# (commit 31e55cd)

Created release for Google.Cloud.ErrorReporting.V1Beta1-3.0.0-beta02


Decompose large simdblockwrite to smaller simdblockwrites

Similar to https://github.com/intel/intel-xpu-backend-for-triton/pull/2193, but for simdblockwrite.
Decompose simdblockwrite of vector size > 8 to a number of simdblockwrites of vector size 8.
e.g., <64xi16> simdblockwrite is not supported by OpenCL C builtins, decompose it to 8 x <8xi16>.
Restrict TritonGEN::SIMDBlockWriteOp to only accept vector types that are allowed by OpenCL C builtins.

CIs:

This PR + TRITON_INTEL_ENABLE_FIRST_LOAD_TO_SLM: https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/10842020746


ERROR  'default' is not exported

Nuxt 3 (RC-8.0 )code did not build or generate and in development it did not loading the Lottie animations
 ERROR  'default' is not exported by node_modules/lottie-web/build/player/lottie.js, imported by node_modules/vue3-lottie/dist/vue3-lottie.es.js                         17:33:50
file: /Users/thinkindragon/Development/TService/966rtl/apps/userRTL/node_modules/vue3-lottie/dist/vue3-lottie.es.js:2:0
1: import { defineComponent, ref, computed, watch, onMounted, openBlock, createElementBlock, normalizeStyle } from "vue";
2: import Lottie from "lottie-web";
   ^
3: var vue3Lottie_vue_vue_type_style_index_0_lang = "";
4: var _export_sfc = (sfc, props) => {

Instructions have been updated for Nuxt. You can find them here: https://vue3-lottie.vercel.app/guide.html#nuxt-3

I am not using Nuxt, yet this error is still present in my project:
RollupError: "default" is not exported by<EMAIL_ADDRESS>imported by "node_modules/.pnpm/vue3-lottie<EMAIL_ADDRESS>10:58:54.318

<br class="Apple-interchange-newline">

I am using Vite + Vue

Same issue here. I'm using Nuxt 3 along with Vite. I have followed the installation guide for Nuxt 3 provided by the plugin's documentation.

Same issue here. I'm using Nuxt 3 along with Vite. I have followed the installation guide for Nuxt 3 provided by the plugin's documentation.

the issue is already resolved
make sure you follow the instruction
and keep the component in  tag


✨contributor page has been added✨

@SurajPratap10 can you -please review this update.
#1265 solved.
https://github.com/user-attachments/assets/f03f24d7-e8c8-485d-a87a-b53fd95cfd15

Hey @SurajPratap10, @T-Rahul-prabhu-38 has apparently stolen the entire design of @ tarunkumar2005 (not tagging him intentionally)
This is the design made by Tarun 3 weeks ago tarun design
Tarun alleges that Rahul has taken his entire code and only done minor changes ( such as website name ) and submitted the PR.
@MastanSayyad pinging you incase needed.


[feature suggestion] self speculative decoding

Good morning(or afternoon/ evening)!
There is a methodology called self speculative decoding among the techniques to enhance the speed of LLM inference. Would it be possible to implement this feature in Luminal? If it aligns with Luminal's philosophy, I believe this type of work could greatly contribute to speed improvement! Even though it's not included in the v0.3 roadmap, I'd like to start this task slowly if it's alright.

Summary of abstract
This paper introduces self-speculative decoding, a novel inference scheme designed to accelerate Large Language Models (LLMs) without relying on auxiliary models. It operates in two stages: drafting, which quickly generates draft tokens by selectively skipping intermediate layers, and verification, which validates draft output using the original LLM in a single forward pass. The approach maintains output quality identical to that of unaltered LLMs, without requiring additional neural network training or extra memory, offering a plug-and-play and cost-effective solution for inference acceleration, with benchmarks showing speedups of up to 1.73× on LLaMA-2 and its fine-tuned models.

self speculative decoding paper
code(python implementation)

Thanks! @jafioti
I'm gonna start with two graphs as you suggested.
btw, what do you think is the right place for this feature to be merged?
creating generation.rs either in luminal/crates/luminal_nn/src/ or luminal/crates/luminal_nn/src/transformer seems possible, I guess.

I would suggest for now just making it an example (copy llama and rename it llama_speculative) and work off that until it takes shape. Once it works we can see how it fits into the whole ecosystem.

Good morning! @jafioti
According to original implementation of the paper(draft & verify:Lossless Large Language Model Acceleration via Self-Speculative Decoding), the skipped layers are chosen in advance.
It seems like the Bayesian optimization discussed in the paper is implemented using the Python library "bayes_opt". Once the skipped layers are determined, it doesn't seem like they change during the draft pass. However, in my opinion, depending on the type of LLM model and the prompt, the skipped layers could potentially change. So, I'm thinking of implementing a function that takes skipped layers as input for now(maybe const generic). Does that sound like a good idea?

Sure sounds good👍

Hey @NewBornRustacean , how's it been going with speculative decoding? If you need any help, feel free to reach out. Happy to talk if you're stuck on anything.

Thanks for your comment! Actually, I've been so busy with work for the past few weeks that I haven't been able to get much done. This week is a holiday in Korea, so I think I'll finally have some time! If I run into any difficulties, I'll ask for help right away.
Btw, the slides you shared(at disocrd) were very helpful to understand the concepts.
I've been reading the Mirage paper you shared during my commute, and although it's difficult, I find it interesting. I want to learn more about this topic so I can contribute more to Luminal!


Implement initial Teams Functional Test Plan

💡 Summary
Create test plan for teams to support automated functional testing.
Motivation and context
Supports automated testing of products
This would be useful because it reduces human touch labor.
Implementation notes
Please provide details for implementation, such as:

Create yanl test plan

Acceptance criteria
How do we know when this work is done?

[ ] Test plan executes without any undocumented errors

Closed with PR #582


Support Additional Languages

Is your feature request related to a problem? Please describe.
A twitter user requested Spanish support and is willing to help with translations. We could support additional languages, and make it easier for people to add new translations.
Describe the solution you'd like

Support additional spoken languages in the Nomination form as fields
Support additional locales for the site

update the locale switcher to be a selector rather than a toggle
decide which languages to focus on first -- Should it be translator driven? Should we look at popularity? See https://en.wikipedia.org/wiki/Languages_of_Japan

Chinese
Korean
Portuguese
Hindi
Vietnamese
Thai
others??

Hi there, I love the work you're doing with this project 💯
As a native Spanish speaker myself, I would also love to see more language representation even if just in the spoken languages section
However, the truth is that, at least in my experience in tech events in Japan, most events only want talks in English or Japanese, so adding this support might bloat the spec in a maybe not-so-desirable way 😞
So, I think you should all carefully consider how valuable adding this is to the project scope as it's quite a big feature
Again, thanks a lot for the work you're doing with this 👍

Yeah, the infrastructure is exactly what I was concerned about, getting that right can take quite a bit of time or money 😵 💸
However, if you do get it up running, I would be willing to collaborate with Spanish translations so ping me if you ever happen to do it 👍
I do think that maybe allowing for at least more variety in the spoken languages section could be great, better even if you could use those as a search filter so that this database can be helpful even for people running international online events

Added additional languages to Airtable Languages table last night.
I selected languages based on which nationalities are most represented in Japan, then added the top languages spoken in those areas. We can always add more upon requests, and the form allows custom entries, though at least one use had difficulty with the current UX.
Here's a helpful guide for listing languages in their native language. I cross-referenced with what Google uses (checking under chrome://settings, searching for Languages, then examining the search terms)


Define Model with Stored Procedure

Can I use the results of a Stored Procedure to define a Model with Objection.js?

Each model must point to a table. The table name cannot be arbitrary SQL. Maybe you could create a view and make the model point to that?

My DBA's mandate is that I must use Stored Procedures to interact with the DB.
What if I could guarantee you that the Stored Procedure would return the the appropriate representation of the "Model/Table" each and every time?
OR --- are does the use of Stored Procedures sort of negate ORMs?

You don't need to guarantee me anything  😄. It would be extremely difficult to make this work and figure out how to pass arguments to the stored procedures. Also you are the first one I've ever met that wants to use only stored procedures to interact with the DB.
Considering the amount of people this feature would be useful for (very few), and how difficult it would be to implement, support is just not going to be added.
I'd say you'll have very difficult time getting any ORM to work like this. My suggestion is to use SQL or a query builder like knex.


create a new gnomad AF annotation

current liftover misses quite many variants e.g.
https://userresults.finngen.fi/variant/2:40409040-G-T

@anastasia-shch this is previous ticket for the annotation task on wrike. Close this when merging PR please.

Closed as requested.


Backlog

Hey @bd3dowling - i'm happy to contribute some code in here. Is there a rough backlog that you had in mind?
If there is, it'd be great if you could spin up some issues at some point 🙏

Hey James, yeah I have a rough roadmap in mind - will issue up this afternoon


QT wallet - grammar, typos, UI stuff

"SELECTED" needs to be shifted a little to the left

Fix to "supported"
Make the link clickable
Fix to "Any questions?"
*if you click anywhere in the text, this text cursor thing appears - I would change it so the text is not "clickable"

I suggest to fix these and change to:

of a hardware
might lose
might lose
on a separate drive like a USB stick
locking or refunding
before a redeem
to finish the redeem or make a refund
refund your
finish redeeming a transaction
which is optional
Please note that (no ",")
Pause the trade before the lock step, until the trade data is backed up.

However, consult this with a native speaker just to make sure it's 100% correct.

1  - fixed.
2 - 'Make the link clickable'  unable to fix, Qt doesn't support that.
'*if you click anywhere in the text, this text cursor thing appears - I would change it so the text is not "clickable"'   - unable to fix,  we want users to be able to copy any text. Cursor is needed for that.
3  - fixed

Verified. Good job, I'm closing this issue.


Hide all horizontal lines but keep the first line?

Hi guys,
I'd like to hide all horizontal lines but still keep the first line of the chart (xAxis), I use withHorizontalLines={false} and all of horizontal lines has been disappeared.
Is there any solutions?
Thanks.

Hi try this code   withOuterLines={true}

@DanielSuarezDev it still does not work

Do you want to do something like this if I understood correctly?

For that you must add the following line of code withInnerLines={false}. You can also combine it with withVerticalLines={false} to remove the first line of yAxis.
<LineChart
    data={data}
    width={Metrics.screenWidth}
    height={Metrics.screenWidth * 0.7}
    withInnerLines={false} // add this line
    /* withVerticalLines={false} */
    chartConfig={chartConfig}
  />


calculate_test_positivity takes Dataset

This PR
This PR is part of https://trello.com/c/6ZPv7k9T/477-clean-up-redundant-test-positivity-calculation-code-api-vs-combined-dataset

Changes calculate_test_positivity to run the smoothing itself and take a OneRegionTimeseriesDataset instead of raw DataFrame.
Adds test helpers build_dataset and build_one_region_dataset to simplify making test data

Tested
Ran python ./run.py api generate-test-positivity in master and this branch. Compared results/output/test-positivity-all.csv and results/output/test-positivity-output.csv from each. They were identical.

This all looks good to me - It would be helpful if you could add some hints as to the patterns we want in the future, not sure I can deduce the direction from just the code as it is

I'm working towards making functions and take and return OneRegionTimeseriesDataset / MultiRegionDataset objects with more use of provenance for tracking how data flowed so we can more easily retire unused parts.


TextField cursor tap position issue

Steps to Reproduce

Run Gallery application and select "Text fields" item.
Tap on the TextField and write any text.
Unfocus TextField with written text.
Tap at the vertical center on the TextField and see that cursor shows up at the position of the tap and its draggable indicator is visible too.
Unfocus TextField once again.
Tap a little bit below the bottom line/divider of the TextField with written text.
See that cursor shows up at the beginning of the written text without indicator.

Expected: IMO in both cases indicator should behave the same way and be visible. Tapping out of the TextField area shouldn't focus the field or should set its cursor position at the horizontal position of the tap.
Flutter Doctor
[✓] Flutter (on Linux, locale en_US.UTF-8, channel alpha)
• Flutter at /home/kamil/Flutter
• Framework revision e8aa40eddd (4 weeks ago), 2017-10-17 15:42:40 -0700
• Engine revision 7c4142808c
• Tools Dart version 1.25.0-dev.11.0
[✓] Android toolchain - develop for Android devices (Android SDK 26.0.2)
• Android SDK at /home/kamil/Programs/Android/
• Platform android-26, build-tools 26.0.2
• ANDROID_HOME = /home/kamil/Programs/Android/
• Java binary at: /home/kamil/Programs/android-studio/jre/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_112-release-b06)
[✓] Android Studio (version 2.3)
• Android Studio at /home/kamil/Programs/android-studio
• Java version OpenJDK Runtime Environment (build 1.8.0_112-release-b06)
[✓] Connected devices
• Android SDK built for x86 • emulator-5554 • android-x86 • Android 8.0.0 (API 26) (emulator)

The TextField widget wraps its entire contents (including the InputDecoration) in a GestureDetector that sets focus to the text field on a tap event.
This has some nice properties (e.g. the field will gain focus if you tap on the decoration's icon).  But if you tap outside the bounds of the EditableText, then the Editable will not receive the tap event and will default to positioning the cursor at the start of the text.

I agree but, can't imagine that someone who already set text in textfield clicking anywhere at the end and outside the bounds would like to have cursor at the beginning. At least android doesn't have such a feature so maybe is not needed in flutter too ;)

This "place cursor nearest to tap" is something that our Blink engine has a bunch of code to know how to do.  It's used on the web for instance when text fields have large padding, if you tap in the padding it places the cursor nearest to your tap.
I suspect we're not even forwarding the out-of-bounds tap to the Text in these cases.

Yes it's been a very long time. Sorry about that.
The current behavior of TextField is as @eseidel mentioned. Tapping on a TextField gives it the focus and places the insertion point as close as possible to the tap. This is consistent with Android.
I don't think there's bug here (anymore).

Without additional information, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now. Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.


68 feature implement traffic light control

Description
This PR adds the implementation of the traffic light detection. It enables to classify traffic lights into the states green, yellow, red, backside.
Unfortunately it is not possible for me to implement DVC till tomorrow, so there are "some" images added to git ^^
Time invested
Marco 0h (time already counted in old PR)
Type of change
Please delete options that are not relevant.

New feature (non-breaking change which adds functionality)

Does this PR introduce a breaking change?
No, since the traffic light classification is not used yet.
Most important changes
Most important changes are the traffic_light_training.py and classification_model.py
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works (might be obsolete with CI later on)
[x] New and existing unit tests pass locally with my changes (might be obsolete with CI later on)

Thank you for your very good work, nice and clean implemenation of the traffic light model.
I think one thing that would be very helpful for reference is to have an overview of how we can use this model. Like a simple example of loading a traffic light image into it and getting a label back? Or do you think it is already clear from looking at the validation code?

You’re right, that’s a good idea. It should be clear for us from the validation code, but it is maybe hard to find for others. I’ll implement a inference class for loading a model and images and classify them 👍

PS: Don't forget to track your additional time on this PR ;)

PS: Don't forget to track your additional time on this PR ;)

Thanks :D Done :)


WIP: Separate sender and receiver sides in OOB protocol

Signed-off-by: Miroslav Kovar<EMAIL_ADDRESS>

Codecov Report

Merging #350 (9a2f9f5) into feature/out-of-band-wrapper (bb4d62f) will decrease coverage by 4.64%.
The diff coverage is 0.00%.

:exclamation: Current head 9a2f9f5 differs from pull request most recent head dc96288. Consider uploading reports for the commit dc96288 to get more accurate results

@@                       Coverage Diff                       @@
##           feature/out-of-band-wrapper     #350      +/-   ##
===============================================================
- Coverage                        48.56%   43.91%   -4.65%
===============================================================
  Files                              165      105      -60
  Lines                            17984     8639    -9345
  Branches                          4042     2183    -1859
===============================================================
- Hits                              8734     3794    -4940
+ Misses                            6498     3451    -3047
+ Partials                          2752     1394    -1358

Flag
Coverage Δ

integration
?

unittests
43.91% <0.00%> (-0.03%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb4d62f...dc96288. Read the comment docs.

@mirgee some conflicts popped up here

After merging the original oob wrapper, this will need some update

I just realized fixing some of the conflicts will be a bit more complicated than expected, will have to revert some of the commits


Reloading apache2 gracefully using MDMessageCmd

I'm trying to auto reload apache2 gracefully after a certificate is renewed using the trigger provided by mod_md via MDMessageCmd, but somehow it's giving me a hard time. So two questions about this. First, is this actually supported? I can imagine that reloading the config (graceful or not) while mod_md is doing it's thing might intervene. Either way, here is my current setup:
               LogLevel md:debug
               ServerName mydomain
                Protocols h2 http/1.1 acme-tls/1
                MDCertificateAuthority https://acme-staging-v02.api.letsencrypt.org/directory
                MDCertificateAgreement accepted
                MDPortMap http:-
                MDMessageCmd /etc/apache2/reload.sh

                <MDomain mydomain>
                  MDRenewMode auto
                  MDCAChallenges tls-alpn-01
                  MDRenewWindow 99%
                </MDomain>

Reload script:
#!/bin/bash
now=$(date)

echo $now $1 $2 >> /tmp/command.log

if [ $1 = "renewed" ]; then
 #/usr/sbin/apache2ctl -k graceful
 echo "$now Reloaded apache2 because of md trigger" >> /tmp/reload.log
fi
exit 0

Secondly, MDRenewWindow cannot be set higher than 99%. This means that after a change I'll have to wait ~24h to see if this is working. Is there a way to trigger mod_md to do the renewal faster/on-demand? I tried fiddling with the config files in md/mydomain but this is not working. This is a fairly specific case as I really need a 'renewal' (so deleting the certificate and installing it again is another 'flow' and not really simulatiing my case).

The MDMessageCmd is invoked from a running child process that has, on most installations, the user www-data while the server's parent process runs as root. This means the script has insufficient rights to restart your server.
I use a daily graceful restart via systemd, since the traffic on my server is low. If you want to avoid that, your message script could create a file restart.asap somewhere where your cron job picks it up.
As to the waiting part: the easiest is probably to just delete the .pem files in md/domains/<your-domain>/*pem and reload the server. When you need a lot of this, it's probably best to use the Let's Encrypt staging servers, because the real ones to rate-limiting.
Hope this helps.

I was under the impression that MDMessageCmd always executes as root, but after checking the manual again, root is only mentioned with installed and not renewed, so fair enough. This also explains why it sometimes worked and sometimes not during testing.
'renewed' means that a new certificate has been obtained and is stored in the 'staging' area in the MD store. It will be activated on the next server restart/reload.

'installed' is triggered when a new certificate has been transferred from staging into the domains location in MD store. This happens at server startup/reload. Different to all other invocations, MDMessageCmd is run with root permissions (on *nix systems) and has access to the certificate files (and keys). Certificates needed for other applications or in different formats can be processed on this event.

Your suggestion working via 'messages' is also a good one, but currently I'm trying with sudo rights for the www-data user for this specific script and having an async delay via at, thinking that security will not really be impacted. Any thoughts on this?


Missing properties after upgrade 1.21.0 to 1.21.1

Upgrade from 1.21.0 to 1.21.1 breaks request body properties
for swagger file where allOf is used
1.21.0

1.21.1

{
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "swagger": "2.0",
  "info": {
    "description": "",
    "title": "",
    "version": "2018-02-23"
  },
  "basePath": "/v1",
  "paths": {
    "/users": {
      "post": {
        "summary": "",
        "parameters": [
          {
            "name": "data",
            "in": "body",
            "required": true,
            "schema": {
              "type": "object",
              "allOf": [
                {
                  "$ref": "#/definitions/AuthUser"
                },
                {
                  "required": [
                    "password"
                  ],
                  "properties": {
                    "password": {
                      "description": "The user's password",
                      "type": "string",
                      "minLength": 8
                    }
                  }
                }
              ]
            }
          }
        ],
        "responses": {
          "200": {
            "description": "user created",
            "schema": {
              "$ref": "#/definitions/AuthUser"
            }
          }
        }
      }
    }
  },
  "definitions": {
    "AuthUser": {
      "type": "object",
      "allOf": [
        {
          "$ref": "#/definitions/Entity"
        },
        {
          "required": [
            "username"
          ],
          "properties": {
            "username": {
              "type": "string"
            }
          }
        }
      ]
    },
    "Entity": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string",
          "example": "1234"
        }
      }
    }
  },
  "parameters": {}
}

consumes:
  - application/json
produces:
  - application/json
swagger: '2.0'
info:
  description: ''
  title: ''
  version: '2018-02-23'
basePath: /v1
paths:
  /users:
    post:
      summary: ''
      parameters:
        - name: data
          in: body
          required: true
          schema:
            type: object
            allOf:
              - $ref: '#/definitions/AuthUser'
              - required:
                  - password
                properties:
                  password:
                    description: The user's password
                    type: string
                    minLength: 8
      responses:
        '200':
          description: user created
          schema:
            $ref: '#/definitions/AuthUser'
definitions:
  AuthUser:
    type: object
    allOf:
      - $ref: '#/definitions/Entity'
      - required:
          - username
        properties:
          username:
            type: string
  Entity:
    type: object
    properties:
      id:
        type: string
        example: '1234'
parameters: {}

@frodeaa thanks for reporting and for detailed test-case!
release v1.21.2 has just been released


Loss?

Hello,
I see in your example photo that the amount of the loss is being presented. However, in your shared code script there is no Loss...
How can I check the loss value?

how can I check the progress of the loss?


Add start and stop measurement events for storing the absolute start and stop times

The EV cannot be linked to a DG block so start and stop times cannot ne stored in an event block


doc: updated build instructions for mac os X

Edit to the platform specific setup for mac os X. The installation of llvm does not accept the options --with-clang and --with-asan anymore.

Modern homebrew does not allow these options. I think homebrew's llvm has clang and asan by default anyway, but haven't managed to confirm as I'm travelling with my antique laptop.


Chore/UIPEditor: simplify editor

@AlexanderBazukevich @PolinaVolchetskaya @nattallius @ala-n
Need some time to research if we can get rid of js_beautify to (which takes about 200mb)

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add support for playback controller command issued events

Add support for playback controller command issued events

Coverage increased (+0.004%) to 98.966% when pulling 43ae3abd62895e4e3837193f20f64659deabac3c on rmberrios:dev into 4be6f6c81b25d9e216f7efccbc9bc477bc607d49 on mediarain:dev.

Coverage remained the same at 98.962% when pulling 7e45ba333e8abbd1e074c48807f5e91a825c3a60 on rmberrios:dev into 4be6f6c81b25d9e216f7efccbc9bc477bc607d49 on mediarain:dev.

Coverage increased (+0.004%) to 98.966% when pulling 4a450db80b0b189a20e139d5e4953c5947961526 on rmberrios:dev into 4be6f6c81b25d9e216f7efccbc9bc477bc607d49 on mediarain:dev.


chore: Update go to 1.20.3

This updates the go version to 1.20.3

Validated against servers and queries as expected.


Revert "Change  "for" attribute to match the right checkbox"

linguist/vendor/README.md

Hey @igat259, the label looks like it is already correct as it is.


Set a max nr of running evaluations

Fixes #130

@jnywong, the ❌  above is because you enabled the RTD PR check right 😅?
If we're not going with RTD and will host the docs somewhere else, can you please remove it? I am not a maintainer on RTD and don't think I can without that 🤔


Copy always successful, also when permission denied

Description
When I copy a dll file in a folder to replace the already existing dll but the dll is used by a exe, i also get the result process successful, but the dll is not replaced
Steps To Reproduce
Start an .exe with a implemented dll
replace the dll
=> the DLL is still the same like before
Expected behavior
I should get the result "error while replacing"
Files Version
Version: <IP_ADDRESS> OS Version: 10.0.22000.527
Windows Version
Windows 11 Pro 21H2 22000.527
Relevant Assets/Logs
debug_fulltrust.log
debug.log

Merging with #7947


[ADD]: WEB APPLICATION VULNERABILITY SCANNER SCRIPT 👾

Aim

The script will scan for vulnerable 3rd-party web applications.
While working on a network security assessment (internal, external, redteam gigs etc.), we often come across vulnerable 3rd-party web applications or web front-ends that allow us to compromise the remote server by exploiting publicly known vulnerabilities. Some of the common & favorite applications are Apache Tomcat administrative interface, JBoss jmx-console, Hudson Jenkins and so on.
If you search through Exploit-db, there are over 10,000 remotely exploitable vulnerabilities that exist in tons of web applications/front-ends and could allow an attacker to completely compromise the back-end server. These vulnerabilities range from RCE to malicious file uploads to SQL injection to RFI/LFI etc.

Details

This script is built to quickly scan the network for such vulnerable applications thus serving pwnable targets on a silver platter.

Do I want to work on this:

[x]  Yes
[ ] No

Please make sure to assign this issue to me @avinashkranjan, @vanshitaverma.


Conditional snippets excluding DataClassification / ApplicationArea ?

@waldo1001  Not sure if issues are actively monitored here, but wanted to get your opinion on the below ;)
Since AL runtime 10 some properties are no longer mandatory.

DataClassification on table fields is no longer needed, if specified on table level.
ApplicationArea on page controls is no longer needed, if specified on page level.

Some of the page field / table field related snippets contain default code for setting the DataClassification / ApplicationArea properties.
It would be great if - based on the AL runtime version - these settings could be removed / no longer suggested as part of the snippet.
One of the complexities is that these properties are only redundant on table / page objects and NOT on table extension / page extension objects.

PS: We're using following rules to discover / cleanup redundant properties.
https://github.com/StefanMaron/BusinessCentral.LinterCop/wiki/LC0019
https://github.com/StefanMaron/BusinessCentral.LinterCop/wiki/LC0020

Valid!
I could do this, but I can' t do this based on runtime - it's "just" a snippet.
So - doing it "now" - would that make sense?


[BUG] In Spark 3.2.0 and above dynamic partition pruning and AQE are not mutually exclusive

The existing tests for this failed initially because they are testing that when DPP happens AQE is disabled.  That is no longer the case and we need to look at that situation and see if we are doing the right thing.  We should also add in tests to verify that we are doing the right thing.

I started looking at this and I think it will be simple enough to add new tests but I am going to wait until https://github.com/NVIDIA/spark-rapids/pull/2073 is merged since branch-0.5 does not currently compile with Spark 3.2.0

Removing from the sprint since we are not supporting Spark 3.2.0 in the 0.5 release.

So the tests that are failing now for AdaptiveQueryExecSuite  are all failing in Spark code, specifically in AQEShuffleReadExec.scala:86 where it has a hard coded set of partitioning that is allowed.
      // For coalesced shuffle read, the data distribution is not changed, only the number of
      // partitions is changed.
      child.outputPartitioning match {
        case h: HashPartitioning =>
          CurrentOrigin.withOrigin(h.origin)(h.copy(numPartitions = partitionSpecs.length))
        case r: RangePartitioning =>
          CurrentOrigin.withOrigin(r.origin)(r.copy(numPartitions = partitionSpecs.length))
        // This can only happen for `REBALANCE_PARTITIONS_BY_NONE`, which uses
        // `RoundRobinPartitioning` but we don't need to retain the number of partitions.
        case r: RoundRobinPartitioning =>
          r.copy(numPartitions = partitionSpecs.length)
        case other => throw new IllegalStateException(
          "Unexpected partitioning for coalesced shuffle read: " + other)
      }

This includes the one DPP test Join partitioned tables DPP fallback so I don't know what the next steps for this are. Do we need to get something into Spark before 3.2.0 ships or if there is a way for us to work around this.

As always Andy is on top of this.
https://issues.apache.org/jira/browse/SPARK-36666
That didn't make it into 3.2.0-RC2 but should be in the next one.

I think I can just close this. If I manually update things to test on 3.2.1-SNAPHSOT (which includes the above fix) there are no failures in the AQE tests


network: optimize update procedure for listen_addrs and external_addrs

Motivation
Remove the need of updating listen_addresses and external_addresses on every single call to NetworkWorker::next_action.
This is very inefficient, as it involves creating new vectors from hashsets on every call, even if nothing changed in the address set. It turns out to be time consuming when testing in Versi, under a heavy load - about 5-8% of the total polkadot CPU utilisation.
Discussion about motivation here: https://github.com/libp2p/rust-libp2p/discussions/3840#discussioncomment-6575512
Fix
This is a tricky problem to solve, since this data needs to be accessible from a NetworkService instance, which does not have direct access to the Swarm.

Changed the underlying data store from a Vec to a HashSet, as it's more efficient for inserts/removes (which is also what libp2p uses for this data). It's converted to a vector only when needed (rarely).

For listen_addresses, we now simply alter the HashSet as a reaction to FromSwarm::NewListenAddr and FromSwarm::ExpiredListenAddr, which is returned by Swarm::poll_next_event(), whenever there is a new listen address.

For external_addresses it's a bit different, because they are actually updated either from a network Behaviour or from the NetworkWorker itself. This PR introduces an ExternalAddress struct, that is defined on the PeerInfo network behaviour. It reacts to external address swarm events and updates the shared view of the addresses whenever needed.

Checklist

[ ] My PR includes a detailed description as outlined in the "Description" section above
[ ] My PR follows the labeling requirements of this project (at minimum one label for each A, B, C and D required)
[ ] I have made corresponding changes to the documentation (if applicable)
[ ] I have added tests that prove my fix is effective or that my feature works (if applicable)
[ ] If this PR alters any external APIs or interfaces used by Polkadot, the corresponding Polkadot PR is ready as well as the corresponding Cumulus PR (optional)

You can remove the "Checklist" section once all have been checked. Thank you for your contribution!

bot merge


Permission for uploading page-decoration plugin

Description
Add permission to upload page-decoration-plugin.

link to plugin/component Git repository in description above
link to resolved HOSTING issue in description above

Submitter checklist for changing permissions
Always

[X] Add link to plugin/component Git repository in description above

For a newly hosted plugin only

[X] Add link to resolved HOSTING issue in description above

For a new permissions file only

[X] Make sure the file is created in permissions/ directory
[X] artifactId (pom.xml) is used for name (permissions YAML file).
[X] groupId / artifactId (pom.xml) are correctly represented in path (permissions YAML file)
[X] Check that the file is named plugin-${artifactId}.yml for plugins

When adding new uploaders (this includes newly created permissions files)

[X] Use the Jenkins community (LDAP) account name in the YAML file, not the GitHub account name
[X] All newly added users have logged in to Artifactory at least once

Not opposed, but would a less generic (and more expression) name not be better? It's basically the name of the extension point you're implementing…

For some reason, there's no email address stored for your account in Artifactory. Could you check to make sure there's a correct email address specified in https://accounts.jenkins.io?

I realize the name is uninspired, but this is really all I wanted out of a plugin and I didn't want to come up with anything that would imply additional functionality.  I am open to suggestions on both fronts, but all I was looking for was a way to show deploy information from Ansible in the page footer.
As for my email, the https://accounts.jenkins.io just points me to the wiki or Jira.  Jira has the correct email, but I don't see anything on https://repo.jenkins-ci.org in my profile except Bintray settings.

@tvon

As for my email, the https://accounts.jenkins.io just points me to the wiki or Jira. Jira has the correct email

Log in there, then select Profile in the menu and check your email address. it is currently tvon (I don't know how…). Set it to a real one, that'll also make password resets much more pleasant (i.e. possible).

I didn't want to come up with anything that would imply additional functionality.

Could be as simple as being more specific – what differentiates this from Simple Theme Plugin, for example, is that it takes its input from environment variables. Sooo…

page-decoration-from-environment?
environment-variable-page-decoration?

Ah, I missed the profile button.  Email updated.
I'm fine with a name change, let's say "environment-variable-page-decoration", what is the process for that?

@tvon Done -- all you need to do is change your POM and sources accordingly.

Thank you, rename in-code is done.


#6 Port to python 3, #19 MI calculation issue, #15 HELP!! All-NaN slice encountered

…e encountered
Port to Python3 based on "https://github.com/mavillan" code - Port to Python3 #6
Also, solved MI calculation issue #19 and HELP!! All-NaN slice encountered #15

Thanks for that. It works nicely with Python3.

Thanks so much and sorry for being so slow with this!


Invalid TabMon options

I can't start the tabmon service on a 2019.2 server.  I get the error "[5] FATAL TabMon.TabMonAgent - Invalid TabMon options specified!
Aborting.."
I'm guessing it's in the Process Types but I don't see any issues.
TabMon.zip

any thoughts on this?  maybe a working config file for 2019.2 that I can just modify?

TabMon.log.zip
here is the log and config file

sorry about this.  turns out the tabmon just didn't have correct permissions in the DB.

Hi @tslinkardKH,
Sorry about the delay on this.
If you have any other questions let me know.
Thanks,
Dan


Unable to open Properties window

Description
File Explorer is not affected by this issue. Note that this appears to have been introduced by the latest Windows update, and it happens regardless of object – both files and folders are affected alike.
Exception:
2022-02-20 09:27:10.5179|WARN|StartSTATask|The system cannot find the file specified. (0x80070002)
	System.IO.FileNotFoundException: The system cannot find the file specified. (0x80070002)
   at Vanara.PInvoke.HRESULT.ThrowIfFailed(String message)
   at Vanara.PInvoke.Shell32.ShellUtil.GetShellItemForPath(String path)
   at FilesFullTrust.Win32API.GetFileIconAndOverlay(String path, Int32 thumbnailSize, Boolean getOverlay, Boolean onlyGetOverlay)
   at FilesFullTrust.MessageHandlers.Win32MessageHandler.<>c__DisplayClass3_0.<ParseArgumentsAsync>b__0()
   at FilesFullTrust.Win32API.<>c__DisplayClass0_0`1.<StartSTATask>b__0()

Steps To Reproduce
No response
Expected behavior
Properties window can be opened normally.
Files Version
<IP_ADDRESS>
Windows Version
Windows 11 Dev 22557.1
Relevant Assets/Logs
debug.log
debug_fulltrust.log

Thank you for your feedback, this is an issue with the latest insider build and not connected to Files. Can you report the issue on the feedback hub?


extra space after contributor names on public activity and registrations pages

Steps

Visit the public activity page: https://osf.io/explore/activity/
Look for a project with multiple contributors—look a their names.
Visit the registrations page of a project with multiple contributors: https://osf.io/atgp5/registrations/
Look at the contributor names below the registration name

Outcome

Each contributor has an extra space after their name. In some cases this comes before a comma, in others, before an &

Expected Behavior

Appropriate spacing should be used—contributors should not automatically have an extra space after their names.
Spacing is correct in search results and project overview pages.

I noticed this too. It happens in all project/component containers, not just on the public activity page. Relevant Trello card.
This is an example from an OSF user's profile page Public Projects widget:

@haoyuchen1992 Are you working on fixing this?

Not on all containers though—not on project overviews.

@jlcohoon Ah weird!

@jinluyuan I think this is the unnecessary space before an & has been deleted in an unmerged PR. And as for the comma, I am not sure this would be treated as an bug. Some discussion need here.

@haoyuchen1992 I think this is a bug since the spacing should look like this:
Project overview page

Agreed, there should never be a space before a comma and should only ever have one before an ampersand

I think this will be solved in this PR- https://github.com/CenterForOpenScience/osf.io/pull/3713

Just noticed on staging that in the component widget there is no space before the ampersand. Maybe it got fixed too much?


Cloudformation Paginator PageSize does not work

We're trying to use page size with the CloudFormation paginator.  We're having the following issue.
Example code:
client = boto3.client('cloudformation')
paginator = client.get_paginator('list_stacks')
for page in paginator.paginate(StackStatusFilter=['CREATE_COMPLETE'], PaginationConfig={'PageSize': 1}):
......
Will throw error:
.....
botocore/paginate.py", line 175, in _make_request
return self._method(**current_kwargs)
TypeError: _api_call() keywords must be strings
We've traced it down to _limit_key being None, which is used in a kargs dictionary.  So you get {None: 1} (assuming 1 was the PageSize arg sent earlier).
Guessing what _limit_key string should be, we always get unexpected parameter (below trying _limit_key="PageSize'):
Unknown parameter in input: "PageSize", must be one of: NextToken, StackStatusFilter
Using
boto3==1.3.0
botocore==1.4.11
Are you able to reproduce?
Thanks,
Mike

Yeah I am able to reproduce. The reason is because the list_stacks operation does not have a max items related parameter to map the PageSize argument to: https://botocore.readthedocs.org/en/latest/reference/services/cloudformation.html#CloudFormation.Client.list_stacks. So in short there is not really a way to use page size because the api does not support it.
Here is what I was getting:
Traceback (most recent call last):
  File "foo.py", line 6, in <module>
    for page in paginator.paginate(StackStatusFilter=['CREATE_COMPLETE'], PaginationConfig={'PageSize': 1}):
  File "/Users/kyleknap/GitHub/botocore/botocore/paginate.py", line 104, in __iter__
    response = self._make_request(current_kwargs)
  File "/Users/kyleknap/GitHub/botocore/botocore/paginate.py", line 176, in _make_request
    return self._method(**current_kwargs)
TypeError: _api_call() keywords must be strings

I think that this is a pretty confusing error. We will to improve the error message and the docs.

Sorry on the late response.  Thanks much for looking.
Oh ok, we were looking at the paginator documentation.
https://botocore.readthedocs.org/en/latest/reference/services/cloudformation.html#CloudFormation.Paginator.ListStacks


MultiSelect field ignores disabled property

When I add "ui:options": { "disabled": true } to the uiSchema of a "ui:widget": "material-multiselect" field, the MultiSelect field is not disabled.
How to reproduce:

Go to https://react-jsonschema-form-material.herokuapp.com/
Add "ui:options": { "disabled": true } to the selectTest field in the uiSchema
The field is not disabled

Hi @tijpenga  -  Thanks for reporting the bug.
This issue has now been fixed as part of version 1.0.103 .
Thanks - V


Update to Apache Arrow 0.6.0 API

Looks good to me, let's see what Travis and AppVeyor have to say :-)

Codecov Report

Merging #123 into master will increase coverage by 0.13%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #123      +/-   ##
==========================================
+ Coverage      98%   98.13%   +0.13%
==========================================
  Files         143      143
  Lines        2651     2518     -133
==========================================
- Hits         2598     2471     -127
+ Misses         53       47       -6

Impacted Files
Coverage Δ

...urbodbc_arrow/Test/tests/arrow_result_set_test.cpp
100% <100%> (ø)
:arrow_up:

.../turbodbc_python/Library/src/python_result_set.cpp
93.75% <0%> (-0.54%)
:arrow_down:

...ary/src/result_sets/double_buffered_result_set.cpp
100% <0%> (ø)
:arrow_up:

.../Library/src/field_translators/date_translator.cpp
100% <0%> (ø)
:arrow_up:

cpp/turbodbc/Library/src/connection.cpp
100% <0%> (ø)
:arrow_up:

...pp/turbodbc/Library/src/result_sets/result_set.cpp
100% <0%> (ø)
:arrow_up:

..._python/Library/src/python_bindings/connection.cpp
100% <0%> (ø)
:arrow_up:

cpp/turbodbc_numpy/Library/src/binary_column.cpp
100% <0%> (ø)
:arrow_up:

cpp/cpp_odbc/Library/src/connection.cpp
100% <0%> (ø)
:arrow_up:

...c/Library/src/descriptions/unicode_description.cpp
100% <0%> (ø)
:arrow_up:

... and 48 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6afd8ee...fb2c792. Read the comment docs.

Thanks a lot for your contribution @xhochy!


Version control

As I said this is a different branch I created for the compilation of SIRF with specific version as discussed.
I did not create this PR.


Refactor of the loss class

Thanks for fixing the docstrings!
Regarding the VarianceLoss: My changes already simplify a lot! In the current version, you have to parse a list of losses to realize the variance regularization. That was not an intuitive solution in my opinion.
The variance regularization as a hyperparameter was implemented before, and I think, it also should stay, since this is a feature we want to highlight.
The fact that the loss is techically modified doesn't have to bother the user.

You're right, the variance argument was there already and it's a bit more intuitive now. Still don't you think using the loss like
QNNRegressor(...,SquaredLoss() + VarianceLoss(some_variance_factor),...)

or even
loss = SquaredLoss() + VarianceLoss(some_variance_factor)
QNNRegressor(...,loss,...)

could be more intuitive than
QNNRegressor(...,SquaredLoss(),...,variance=some_variance_factor,...)

Especially when we add other Losses we might combine it with. Then the whole loss construction could take place at once.

I think your first two approaches are more complicated, since the user has to know exactly what he/she is doing.
In the last interface, it is a hyper parameter of the method which might be more intuitive.
Nevertheless, the first implementation should already work, if a user wants to do it that way.

The "feature" might also be overlooked if it is not a hyper-parameter.


[BUG] Why are all secrets in the cluster frequently pulled?

Information about the Cluster

longhorb version: 1.3.3
Kubernetes version:1.18
Cluster Type (Local/Downstream): Local

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom)

Admin

Describe the bug
In my cluster, because a large number of helm charts are used to deploy services (1000+) and 10 historical versions are retained, this will generate a lot of secrets (5000+). In the test cluster, I did not prepare SSD for etcd. Disk, when longhorn pulls the secret, it will cause the kube apiserver cpu load to be too high and the machine hangs to death. I don't understand why longhorn needs to pull the secret?
Result
kube apiserver is crash
Additional context
$ cat audit* | jq '. | select(.userAgent == "longhorn-manager/v0.0.0 (linux/amd64) kubernetes/$Format")' | jq .requestURI | sort -rn | uniq -c | sort -rn | head -n 10
   3577 "/api/v1/namespaces/longhorn-system/secrets/longhorn-webhook-tls"
   2216 "/api/v1/pods?resourceVersion=1391102207"
   2084 "/api/v1/services?resourceVersion=1391102075"
   1832 "/api/v1/services?resourceVersion=1391098166"
   1818 "/api/v1/pods?resourceVersion=1391101604"
   1256 "/api/v1/secrets?resourceVersion=1391101112"
   1178 "/api/v1/secrets?resourceVersion=1391102484"
   1120 "/api/v1/pods?resourceVersion=1391105142"
   1100 "/api/v1/secrets?resourceVersion=1391102075"
    863 "/api/v1/secrets?resourceVersion=1391101925"

It doesn't pull the specified namespace or label

What's the longhorn version?
longhorb version: 1.13

What's the longhorn version?
longhorb version: 1.13

oh,sorry,is 1.3.3

helm values.yaml
annotations:
  prometheus.io/scrape: "true"
  prometheus.io/path: "/metrics"
  prometheus.io/port: "9500"

longhornManager:
  tolerations:
    - key: "node-role.kubernetes.io/storage"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"

longhornDriver:
  tolerations:
    - key: "node-role.kubernetes.io/storage"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"

defaultSettings:
  defaultDataPath: "/data/longhorn"
  createDefaultDiskLabeledNodes: "true"
  taintToleration: "node-role.kubernetes.io/storage=true:NoSchedule"

persistence:
  defaultClass: "false"

longhornUI:
  replicas: 1

What's the pull frequency?

~What's the pull frequency? In addition, how did you get the value?~ @PhanLe1010 explained the values in our internal meeting.

I opened the APIServer's audit logs and analyzed the logs within 3 hours to get this data。

@GounGG
Are the secrete mostly in a namespace which is different than longhorn-system namespaces? If yes, I think we have hit a known issue which is fixed at https://github.com/longhorn/longhorn/issues/6954 . Can you try to upgrade Longhorn to v1.5.5 or v1.6.2 to see if it is fixed?


Added yesterday_ds_nodash and tommorow_ds_nodash

The goal is to avoid too much logic in templates, the current workaround is to
write {{ macros.ds_format(yesterday_ds, '%Y-%m-%d', '%Y%m%d') }} which is too
verbose.

This is useful. Thanks for the PR.


docs: fix or remove @param

Description
php-cs-fixer has been modified to remove these tags.
  23) tests/_support/Config/Services.php (no_superfluous_phpdoc_tags, phpdoc_trim_consecutive_blank_line_separation)
      ---------- begin diff ----------
--- /home/runner/work/CodeIgniter4/CodeIgniter4/tests/_support/Config/Services.php
+++ /home/runner/work/CodeIgniter4/CodeIgniter4/tests/_support/Config/Services.php
@@ -28,8 +28,6 @@
     /**
      * The URI class provides a way to model and manipulate URIs.
      *
-     * @param string $uri
-     *
      * @return URI
      */
     public static function uri(?string $uri = null, bool $getShared = true)

      ----------- end diff -----------

Found 23 of 875 files that can be fixed in 92.546 seconds, 46.000 MB memory used
Error: Process completed with exit code 8.

See https://github.com/codeigniter4/CodeIgniter4/actions/runs/6018752536/job/16327489927#step:8:487
Checklist:

[x] Securely signed commits
[x] Component(s) with PHPDoc blocks, only if necessary or adds value
[] Unit testing, with >80% coverage
[] User guide updated
[x] Conforms to style guide

Thank you! @paulbalandan


make websites links in members avatars target="_blank"

On the collective page and/or on the github banner?

collective page
P.
piamancini.com
t: @piamancini
s: piamancini
OpenCollective https://opencollective.com
On Thu, Apr 28, 2016 at 4:56 PM, Xavier Damman<EMAIL_ADDRESS>wrote:

On the collective page and/or on the github banner?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/OpenCollective/OpenCollective/issues/56#issuecomment-215596739

fix: https://github.com/OpenCollective/opencollective-website/pull/245


Add fixer and new option to commentFormatRule

PR checklist

[x] Addresses an existing issue: fixes #649
[x] New feature, bugfix, or enhancement

[x] Includes tests

[x] Documentation update

Overview of change:

change existing rule to only emit one error
add fixer
add option allow-trailing-lowercase

Is there anything you'd like reviewers to focus on?

CHANGELOG.md entry:
[new-rule-option] "allow-trailing-lowercase" option for comment-format
[new-fixer] comment-format

Has this been completed?

I went ahead and fixed up the merge conflicts. Sorry this took so long @smoehrle - the changes look great and I'm excited to get to use them. Thanks so much for sending this PR back in the day! 😊
FYI @MichaelTontchev


Issue with the AuthorizationCodeContext

Not entirely sure if you guys are watching this repository or not. Issues are mentioned nearly a year ago.
I'm currently following the steps on the MS Website itself (https://learn.microsoft.com/en-us/openapi/kiota/tutorials/php-azure?tabs=portal)
The AuthorizationCodeContext is working fine with the authCode for once. When I refresh the page its gone and I need to repeat the whole process again. I need this 'working' without further consent of the user after it gave consent once. But there is nothing that mentions anything about the other options or how to make this work permenantly.
I was looking into the https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-client-creds-grant-flow - But not really sure how I could make that work and there isn't really some documentation about it..
I hope you'll see this and could reply to me!
Thanks in advance - Sanne

Hi @nedvice-sv, thanks for reaching out.
You can retrieve the access token fetched during the first request and cache it/store it more permanently for future re-use.
We have some docs gaps and some enhancements that can be made. Hopefully this unblocks you.
Here' some guidance docs (specific to Microsoft Graph's SDK generated by Kiota), but the concepts are similar e.g.
getting the access token fetched by the SDK & persisting it for future requests to reuse

$inMemoryAccessTokenCache = new InMemoryAccessTokenCache();
$requestAdapter = new GuzzleRequestAdapter(
    PhpLeagueAuthenticationProvider::createWithAccessTokenProvider(
        new PhpLeagueAccessTokenProvider(
            $authCodeContext,
            $scopes,
            [],
            null,
            $inMemoryAccessTokenCache
        )
    )
);
$client = new PostsApiClient($requestAdapter);

// Get access token for future storage
$accessToken = $inMemoryAccessTokenCache->getAccessToken();

Hi @Ndiritu,
Thanks for you reply. I'm sorry I couldn't get back to you earlier then possible. I'm working only 1 day a week on this project (hence why I reply now).
I've tested your code and sadly it doesnt work entirely. The access token I should receive isn't there. The AccessToken object is empty so I can't save it for later usages. Also the access token needs to be valid for quite some while. (Its being used as a provider to receive mails from an inbox and then being parsed)
I'm also using some code from this repo: "https://github.com/microsoft/kiota-samples/tree/main/get-started/azure-auth/php".
My current code looks like this:
           $code = $request->get('code');
            $authCodeContext = new AuthorizationCodeContext(
                tenantId: 'tenant-id',
                clientId: 'client-id',
                clientSecret: 'client-secret',
                authCode: $code,
                redirectUri: 'http://localhost',
            );

            $inMemoryAccessTokenCache = new InMemoryAccessTokenCache();
            $client = new GraphApiClient(
                new GuzzleRequestAdapter(
                    PhpLeagueAuthenticationProvider::createWithAccessTokenProvider(
                        new PhpLeagueAccessTokenProvider(
                            $authCodeContext,
                            ['User.Read', 'Mail.ReadWrite'],
                            [],
                            null,
                            $inMemoryAccessTokenCache
                        )
                    )
                )
            );
            $accessToken = $inMemoryAccessTokenCache->getTokenWithContext($authCodeContext);

So I do wonder where it goes wrong.. I can't install the https://github.com/microsoftgraph/msgraph-sdk-php because it has some composer issues...
Thanks in advance, again.

My apologies @nedvice-sv, I should have been clear that the access token is fetched before a request to the API.
So it will only be present in the cache after you make the first call using the client e.g.

$me = $client->me()->get()->wait();
$accessToken = $inMemoryAccessTokenCache->getTokenWithContext($authCodeContext);

Please let me know if this works
As for the lifetime of the token & future requests using the same token, after retrieving the access token, you can persist it securely & retrieve it, load it into the cache as shown here and in case it's invalid, the SDK will refresh it for you & update the cache with the newly acquired token.


create router

Remember to handle conflicts like /post/create and /post/:id : https://www.alexedwards.net/blog/which-go-router-should-i-use

https://github.com/matryer/way

also see: https://github.com/komuw/ong/issues/46


A valid semver tag (e.g. v1.0.0) was not found even though it exists

Describe the bug
When using the tool in a Github pipeline, it will systematically return the following message:
Tool 'versioning.net' (version '0.5.0') was successfully installed.
info: Application.GitVersioning.Handlers.GetCommitVersionInfosHandler[0]
      A valid semver tag (e.g. v1.0.0) was not found. Please add a valid semver tag to your repository and try again.
info: Application.GitVersioning.Handlers.IncrementVersionWithGitIntegrationHandler[0]
      Increment 'Unknown' was determined from the commits.

To Reproduce
Steps to reproduce the behavior:

Create a Github pipeline such as the following:

name: CI

on:

  push:
    branches: [ main ]

  pull_request:
    branches: [ main ]

  workflow_dispatch:

jobs:
  build:
    runs-on: windows-latest

    steps:
      - uses: actions/checkout@v2

      - name: Configure remote repository
        run: |
          git remote set-url origin<EMAIL_ADDRESS>          dotnet tool install --global Versioning.NET
          dotnet-version increment-version-with-git -g "." --branch-name main --author-email<EMAIL_ADDRESS>

Push a commit with a message such as:

fix(Versioning): Fixed issue where git tag could not be found

When running, the pipeline will show the message described above, and the new version is not computed

Expected behavior
A new version is successfully computed
Additional context
I did tag the repository with a SemVer (v0.1.0) before running the tool, but it did not change a thing 😢
Furthermore, shouldn't semantic versions NOT be prefixed with the 'v' character, as explained in this issue?

@cbcrouse Did you have time to check it out?
I'm trying to get it to work for CNCF Synapse.
If you have time, it would be absolutely amazing if you could contribute with a PR to add a pipeline for your awesome tool.

Hey @cdavernas I apologize, I don't recall seeing a notification of this issue, so thanks for following up on it. I'll have some time tomorrow to review this in more detail.

Furthermore, shouldn't semantic versions NOT be prefixed with the 'v' character, as explained in this issue
The versions in the project files themselves do not contain the 'v' character however, it helps to find the version tag when traversing through potentially many numeric-only tags such as dates. The SemVer C# library supports the 'v' when accepting the string formatted value and works quite well.

Hi @cdavernas - I was able to resolve this issue by setting the fetch-depth value of the checkout task to 0. I've added this as a troubleshooting step to a new section in the wiki.
Example
    steps:
      - name: Checkout
        uses<EMAIL_ADDRESS>        with:
          fetch-depth: 0

While I was troubleshooting, I also stumbled upon an unhandled exception for missing csproj files so I've published a fix for that. However, this issue did not require any code changes. Can you let me know if following the same steps also resolves your issue?

@cbcrouse Did you have time to check it out? I'm trying to get it to work for CNCF Synapse. If you have time, it would be absolutely amazing if you could contribute with a PR to add a pipeline for your awesome tool.

Let me know if you would still like me to contribute to your workflows. I recommend doing the versioning separately after a pull request is merged into the target branch. If main is the target branch, then create a workflow that triggers on main to version the code, and then create a separate workflow that triggers on completion of the versioning workflow to deploy your code if necessary.

Can you let me know if following the same steps also resolves your issue?

I'll check that today!

Let me know if you would still like me to contribute to your workflows

Well, sure! If its not too much hastle for you it would be wonderful! The reason is the more contributors, the better. If you do for that project, Ill make sure to implement your recommendations for another, just to get my hands on it!
Anyways, thank you very much and kudos for your great work!


Update gh-action-pip-audit to latest version

This also includes an update to the organization name. "trailofbits" redirects to pypa now, and while this is functional for now, having the current name decreases the likelihood of problems down the line.
Signed-off-by: Eric Ball<EMAIL_ADDRESS>

Eric, thanks for this, and sorry to have missed it when you filed it.  We have since fixed this problem in the active template-v2-scopes branch, both by updating the repo home (as you did here), updating the version to 1.0.3, and changing the version of setuptools to avoid a vulnerability warning (version 65.5.1).  I you can update this PR to do those two things, I can approve merge to develop.

xref #151

@MichaelTiemannOSC The requested changes have been made.

/approve


refactor(blockifier_reexecution): add chain ID to SerializableOfflineReexecutionData

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#1972  👈
#1971
#1970
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @dorimedini-starkware and the rest of your teammates on  Graphite

This change is

crates/blockifier_reexecution/src/main.rs line 137 at r1 (raw file):
Previously, dorimedini-starkware wrote…
still relevant?
the idea is:

RPC tests guess the chain ID from the node URL; or, get the chain ID explicitly if you want to override
offline tests need to fetch the chain ID from the JSON (hence the addition of the chain ID to the JSON struct)

Are we planning to run reexecution in other networks often or in tests?

crates/blockifier_reexecution/src/main.rs line 137 at r1 (raw file):
Previously, dorimedini-starkware wrote…
if we see interesting edge cases pop up in other networks, yes, I would like to be prepared.
maybe it will never but used in the CI but it was already useful for me, to manually test specific blocks from testnet

I understand that testing in other networks is useful, but why do we need to store the network in the file?

crates/blockifier_reexecution/src/main.rs line 137 at r1 (raw file):
Previously, dorimedini-starkware wrote…
because we need the block info for offline execution, right? the chain info requires knowledge of the network (in principle - in practice I think since the fee token addresses are the same it isn't really required, but good to have this for forward compatibility). no?

I don't think it's necessary to add it right to the file—this seems like something we might never really use, and if we need it, the user can add the network via CLI. Adding the network to the file means we plan to make tests that rely on it, but at the moment, there's nothing to suggest such tests will ever be needed. For all the current networks, the behaviour is the same.

crates/blockifier_reexecution/src/main.rs line 137 at r1 (raw file):
Previously, aner-starkware wrote…
I don't think it's necessary to add it right to the file—this seems like something we might never really use, and if we need it, the user can add the network via CLI. Adding the network to the file means we plan to make tests that rely on it, but at the moment, there's nothing to suggest such tests will ever be needed. For all the current networks, the behaviour is the same.

*right now

Merge activity

Nov 19, 10:11 AM EST: A user started a stack merge that includes this pull request via Graphite.


README.md does not make sense

#1
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ pip install yymake
Requirement already satisfied: yymake in /tmp/t/py3/lib/python3.10/site-packages (0.3.8)
Requirement already satisfied: pytest in /tmp/t/py3/lib/python3.10/site-packages (from yymake) (7.4.2)
Requirement already satisfied: colorama in /tmp/t/py3/lib/python3.10/site-packages (from yymake) (0.4.6)
Requirement already satisfied: networkx in /tmp/t/py3/lib/python3.10/site-packages (from yymake) (3.1)
Requirement already satisfied: colorlog in /tmp/t/py3/lib/python3.10/site-packages (from yymake) (6.7.0)
Requirement already satisfied: iniconfig in /tmp/t/py3/lib/python3.10/site-packages (from pytest->yymake) (2.0.0)
Requirement already satisfied: packaging in /tmp/t/py3/lib/python3.10/site-packages (from pytest->yymake) (23.2)
Requirement already satisfied: pluggy<2.0,>=0.12 in /tmp/t/py3/lib/python3.10/site-packages (from pytest->yymake) (1.3.0)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /tmp/t/py3/lib/python3.10/site-packages (from pytest->yymake) (1.1.3)
Requirement already satisfied: tomli>=1.0.0 in /tmp/t/py3/lib/python3.10/site-packages (from pytest->yymake) (2.0.1)
(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ tree
.
├── main.c
└── ya.py

0 directories, 2 files
(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ cat ya.py
project("yiyiya",
    version='12.0',
    desc='yiyiya is an os',
)

target('hello')
add_kind("binary")
add_files("./main.c")

(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ cat main.c
int main(void)
{
  return 0;
}

(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ ya
welcome to use ymake 0.3.8 ,make world happy ^_^!!
gcc main.c -c -o build/release/objs/main.o
[5%]: compile target main.o
ld build/release/objs/main.o -o build/release/hello -lc
ld: warning: cannot find entry symbol _start; defaulting to<PHONE_NUMBER>401000
[10%]: compile target build/release/hello
[100%]: compile target hello
build success! const 0:00:00.023146
(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ tree build/release/
build/release/
├── hello
└── objs
    └── main.o

1 directory, 2 files
(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ ./build/release/hello
-bash: ./build/release/hello: No such file or directory
(py3) xxh@xxh-virtual-machine:/tmp/t/t$
(py3) xxh@xxh-virtual-machine:/tmp/t/t$ nm ./build/release/hello
0000000000404000 D __bss_start
0000000000403f20 d _DYNAMIC
0000000000404000 D _edata
0000000000404000 D _end
0000000000401000 T main
                 U _start

Suspicions:

add_kind('binary'):   might be something like add_kind('executable')?
ld build/release/objs/main.o -o build/release/hello -lc: maybe wrong command line is generated by ya.py?

Suggestion:
Better test it more carefully, especially for those code-snippet in README

重新安装一下，没有问题就关闭了


Composer Versioning

Hi guys, we're developing a client services portal on behalf of a client that will tie into your API and serve some custom reports to their clientele.  My question is simply about composer and your versioning... I don't see any official "Releases" or tags, but I see that the current version is 2.6.
For obvious reasons it's important to be able to test a new version before pushing it to production, but without release tags, I can't do that.  Am I missing something, or, are you able to use tagging to tag your releases so we can properly test future releases before pushing to our production branch.
Thanks.
Kevin.

Hi It seems like the Composer Version is not updated correctly. Let me make a new release and see if it solves the problem

Thanks @hlu2 , please let me know when the release is up and I'll verify we can see it.

Hi @oucil Please try to see if it worked now. I update README file and some other configurations.

Looks good, see the releases now, thanks very much!