Vulkan Mobile renderer - Decals sometimes ignore cull mask and draw over everything

Godot version
v4.1.1.stable.official [bd6af8e0e]
System information
Godot v4.1.1.stable - Manjaro Linux - Vulkan (Mobile) - integrated AMD Radeon Vega 10 Graphics (RADV RAVEN) - AMD Ryzen 7 3700U
Issue description
When adding a decal using the Vulkan Mobile renderer, it will initially render over anything (Sprite3Ds, MeshInstance3Ds etc.) that are inside its AABB regardless of the cull mask set on it and the meshes.  Moving the decal slightly will temporarily fix this, but if the affected meshes are moved or hidden then shown again, the bug reappears.
This does not happen with the Forward+ renderer.
The decal setup - cull mask is set to only layer 1:

The sprite setup - layer is clearly set to 2 but the decal is still visible over it:

After moving the decal slightly, it no longer affects the sprite (as expected):

But after hiding and showing the sprite, the decal appears over it again:

Steps to reproduce

Create a 3D scene with a plane mesh in visibility layer 1 and a sprite or other mesh in layer 2
Add a decal with a cull mask set to only layer 1
Once the decal is added, and before it has been moved, it will render over the mesh in layer 2 even though the cull mask is set to layer 1

Minimal reproduction project
Mobile Decals Test.zip

Thanks for the report! Consolidating in https://github.com/godotengine/godot/issues/70774.


use additional sensorring argument

isolated the changes according to https://github.com/ipa320/cob_common/pull/202

merged https://github.com/ipa320/cob_common/pull/202 and retriggered travis

@ipa-nhg please verify that the correct robots have activated sensorring (cob4-1, cob4-2) and de-activated sensorring (cob4-5, cob4-7, cob4-10)

@ipa-fxm: list of active sensorrings is ok


purge unused setup.py clean_source custom command

🚀 Pull Request
Description
This PR performs more setup.py hygiene by removing the archaic and redundant clean_source custom command.
IMHO it's simply not necessary, and to my knowledge has never been used as part of our workflow, ever. Even as a developer convenience I've simply never used it.
Sounds harsh, but AFAIK this is classic deadwood infrastructure code that can be purged.

Consult Iris pull request check list

@lbdreyer I think that it may have come about when we were building pyke as part of iris and needed an automated way to "tidy-up".
But yeah, now that we've automated the process and building in clean, isolated environments, I really don't think this is now necessary.
Awesome, thanks :partying_face:


WIP: Kubernetes setup on docker-for-desktop

While still a work in progress, the current yml files will allow running the docker images on the single-node cluster on docker-for-desktop on Windows. It hasn't been tested on MacOS
You will need to edit your hosts file adding identityadmin.k8s.local to point to the IP of the machine running the cluster. The shortcut we took in docker-compose will not work on k8s as DNS resolving needs to happen at a different level.
I also added metallb to allow k8s to pass out external IP addresses all pointing at localhost. This should allow load balancing in future scenarios (by adding extra replicas).
This configuration is for dev usage only. Do NOT use it on production!!
Missing features are:

TLS support
Volume claims (to persist the data of sql server between reboots)
Configure the different services with config maps instead env variables
Store the database credentials in a k8s Secret.

We need to package the docker images and store them in a packages registry. Github offers this already, but we're not using that yet, so you will need to build the docker images manually. You can use docker-compose build to do so.
Prerequisites:

install Docker-for-desktop on windows
Enable Kubernetes in Docker-for-desktop in the settings
Have kubectl installed, ready from the command line.
Make sure you kubectl context is pointing to your local kubernetes environment. If you have no other k8s clusters, that should already be the case.

How to run:

Edit your host file as mentioned above. Add the following line in your C:\Windows\System32\drivers\etc\hosts file: <Your IP Address> identityadmin.k8s.local
Run the following commands

cd k8s
kubectl apply -f ./namespace.yml
kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.3/manifests/metallb.yaml
kubectl apply -f ./metallb.yml
kubectl apply -f ./sqlserver.yml
kubectl apply -f ./identity.admin.yml
kubectl apply -f ./identity.sts.yml
kubectl apply -f ./identity.api.yml

I might have missed a couple of things, so if someone could test this out on their machine, that'd be great.

Thanks @bravecobra - we will test this PR with @TomasHubelbauer

@bravecobra
I think a helm chart would be better for Kubernetes. I have done so in a private repo that unfortunately can't share. I will soon create helm charts in my own fork

@ekjuanrejon,
I agree, this first PR is just a proof that we can have a working environment on k8s. Putting such into a helm chart would indeed be the next step. The current images however could use some extra tcl before we can put them into production. The seed command for instance doesn't survive a subsequent run. That seed should run as an init-container.

@bravecobra - what do you suggest regarding to seed? Is it necessary to rework seed in project or event extract seed into separate project like console app for instance?

There are 2 options there: either

we let the seed do nothing if records already exist in the database (now it crashes on that) and assume thus assume a previous seed already did the job
use the /seed to do only the seeding and don't start listening so we can use this command-line option to only do the seeding part and nothing else. That way it can be used as an init-container.

I don't think a separate console project is required.

First use case is already covered, this should not crash it - before populating of data is check if something already exist in table, otherwise do nothing.
I faced another issue which is not solved yet, sometimes database is not ready and because of this seed failed.
What do you suggest with this issue?
@xmichaelx suggested library called Polly (https://github.com/App-vNext/Polly) - and add some retry mechanism with delay.
What do you think?

Actually in production, I would strongly recommend to use a SQL managed service from a cloud provider
Am currently using manged SQL service from Azure. I think that is the best scenario as the backing and restoring is managed by the cloud provider. No need to mound disk into kubernetes etc...
@skoruba when running integration tests or running locally via docker-compose it is good to use
restart: always
https://github.com/ufoscout/docker-compose-wait
or write a bash script...
Have done a combination of those

I agree, you would never run a sql database server in kubernetes in production. Hence the message in the beginning of the readme. Most cloud providers have their own/better solutions for that, that scale better you would ever be able to achieve.
However, again, for development purposes,  this would work for 99% of the cases.

Ideally we would have a helm chart that would conditionally include the correct stable helm chart of the configured database
Luckily those are available already:

https://github.com/helm/charts/tree/master/stable/mssql-linux
https://github.com/helm/charts/blob/master/stable/mysql
https://github.com/helm/charts/blob/master/stable/postgresql

Conditionally including a sub-chart is possible (https://stackoverflow.com/questions/54032974/helm-conditionally-install-subchart), however we need to keep https://github.com/helm/helm/issues/3742 in mind.
We also need to keep metallb out of the chart, as that won't be required for cloud providers. I'm only included it for completeness if you want to try this out on a bare metal cluster.
Although the API part is currently optional, I'd like to include that always. Personally I find the Admin is to call an API instead having the domain logic built-in as we currently have, but that a topic for another issue. Another thing I still need to test out is how the setup responds when Istio injection is enabled on the namespace as well as log-shipping to seq for instance. I first want to test these things out before pouring it into a more definitive solution like a helm chart. If we were to make a helm chart, I'd propose a separate PR that could handle production situations when deploying, not that dev test thingie.
Luckily I prefixed this PR with WIP (Work in progress)  😄

@bravecobra
How about the following:

Use a config map or secret to avoid loads or environment variables, or repetitive environment variables

For the k8s services it it possible to use an ingress instead of a load balancer,

@ekjuanrejon sure, a config map would be more k8s solution. See the missing features in the description where I already mention that. I just used the environment variables to keep the analogy with the docker-compose setup we already had.
An ingress would be easy way out, however then we wouldn't be able to test the statelessness of the current images. How would the service behave if we start adding replicas. In other words, I have no idea at this point whether scaling would work for the current images, just something I wanted to have tested out.

@bravecobra if that is your feature then you can scale your replicas to 1 and still use the ingress controller. it is better to use the ingress controller than to create three load balancers in the cloud

It's not a matter of fear, it's about scalability and being able to test it.


Insert column/row into DataFrame at specified location.

Hello.
Daru::DataFrame doesn't seem to have insert function.
set_at or set_row_at will overwrite.
[]= add_vector does not specify the insertion location.

[]= add_vector does not specify the insertion location.

Can you please elaborate a bit this line?

This is good point.
we can add another column easily but then we have to order it. You can also do df[:player] = df.index but yeah we should have a method to insert a column in specific place. may be we can have #insert method having information about index of column, vector name and vector value.

That's exactly what I wanted to say.

Thanks for letting us know this usecase.

Hello @Shekharrajak , I would like to work on this one.

@cyrillefr , please go ahead. You can discuss your approach here once so that we can verify that you are going in right way then you can open a PR.

Hello @Shekharrajak , sorry for the little delay.
My idea is : add some new data at the end, then reorder.
The method would look like this:
def insert_vector n, name, source
 vector = Daru::Vector.new(source, index: @index, name: @name)
 @data << vector
 @vectors = @vectors.add name
 ordr = @vectors.dup.to_a
 elmnt = ordr.pop
 ordr = ordr.insert n, elmnt
 self.order=ordr
end

Thanks @cyrillefr , please go ahead open a PR with few corner test cases. Love to see the changes in new release :)


Improving autocodeblocks

It's me again bringing more auto codeblocks

Why are you using msg.content rather than a separate string, if I may ask?

Why should i use a seperate string instead of msg.content?

Tested it, everything works as it should (at least when not using the get_tag func cause i can not really test it with that atm)


in CSV Report first column isn't formatted like the others

We want to use the csv Report and import the generated Files automatically into a database.
The first column with the hostname doesn't have apostrophes around the name.
CLIENT-246,"/Applications/Arduino.app/Contents/Java/log4j-core-2.12.0.jar","","Log4j 2","2.12.0","CVE-2021-44228","VULNERABLE","","2021-12-21 12:01:57"
Can you please format all columns in the same way.

@mhochhaus Sure. I will fix it.

Thx a lot mate.

@mhochhaus Do you need double quotes in CSV header line? Current version is:
Hostname,Path,Entry,Product,Version,CVE,Status,Fixed,Detected at

@mhochhaus Would you test v2.5.1 release?

We skip the header during import in Database.
Just checked Version 2.5.1 and it looks good.
"Hostname","Path","Entry","Product","Version","CVE","Status","Fixed","Detected at"
"Client-246","/Applications/Arduino.app/Contents/Java/log4j-core-2.12.0.jar","","Log4j 2","2.12.0","CVE-2021-44228","VULNERABLE","","2021-12-21 14:23:26"

@mhochhaus Thank you for test report!


Support for dynamic tab-content

Hey I have been playing around with your wizard, and I would love to update the number of tab-contents on the fly (managed by an array) depending on user preference the wizard would lengthen or shorten.
Here is a fiddle with what I would like to do. The remove first item button, though doesnt seem to remove the corresponding tab-content as I'd expect.
https://jsfiddle.net/bt5dhqtf/482/
Please let me know if this is the expected behaviour.

Hi @vl4d I think I know why the issue appears. On mobile right know but I will look into it today.

@vl4d Can you take a look at this fiddle https://jsfiddle.net/CristiJ/bt5dhqtf/497/
It references a develop branch so it's not ready for usage yet but wanted to know your opinion here on how steps should be handled.

What happens if the current checked step is removed?
What happens if a step before or after the current checked step is removed?
What happens if a step is added before or after the current checked step ?
Do you want similar cases to be handled or you have a more general use-case ?

@cristijora thanks for the quick response, and based on your questions i'd say:

If the current checked step is remove, I'd go to the step before it.
If the step before or after the current step is removed then I'd expect the wizard to remain at the current step. The wizard would just be 1 step shorter.
If a step is added before the current checked, it would mean that we send the user back to this newly added step (that way the user addresses the step immediately) or a visual representation that it was added and needs to be addressed. If a step is added after the current checked step, nothing would change but an extra additional step.
The use-cases you stated would handle most of what I'd think would come up.

The good thing is that the developer could validate the information and force the user to go back to step is deemed invalid. That is, can't finish the wizard unless all steps are valid.

Sorry for late response on this. I will most likely rewrite the wizard in jsx. Currently I'm a bit limited of rendering dynamic content from slots and don't have as much control over that as I have with render functions and jsx. I have tried implementing this but it very quickly got dirty and uncontrollable. This might take 2-3 weeks until I find time to port the wizard to jsx and consider this functionality again. Hope that's fine

If a step is added before the current checked, it would mean that we send the user back to this newly added step (that way the user addresses the step immediately) or a visual representation that it was added and needs to be addressed. If a step is added after the current checked step, nothing would change but an extra additional step.

This is the only point which I have not addressed yet since it's rather hard to detect where a step was added. If you have any ideas, please let me know. Kind of stuck on it.

@cristijora Regarding your comment 312689794. One idea might be the give every tab a private index in initializeTabs , then in reinitializeTabs you keep track of this.
Why not keep track of the tabs by hashing the tab-content by it's title and icon? Assuming these are unique per wizard.

@exnihilo-creatio Thanks for the idea. I sort of improved this with a new internal implementation but I still feel that it doesn't work as expected mainly due to the fact that different array operations (shift, pop etc) can cause multiple additions/removals which in the end provides inconsistent results. Don't know if hashing would help too much here but I could give a try.

The idea for the hash is to find out which tabs are currently being tracked. Once the array has been changed you loop over the new tabs and check which hashes are not present in the interal hash cache. Those not present are new tabs, and from the loop you can know the position of newly added tabs.

@exnihilo-creatio Do you have the desire to do a PR 😄  ?
The current logic for this is here
https://github.com/cristijora/vue-form-wizard/blob/master/src/components/FormWizard.vue#L226
and here
https://github.com/cristijora/vue-form-wizard/blob/master/src/components/FormWizard.vue#L237
You could add a watch property for tabs and add this hash as a computed property maybe inside tab-content and do the checks accordingly. When you know exactly what tabs to change you can use
this.changeTab(this.activeTabIndex, newIndex)

And potentially adjust this.maxStep = // max value the wizard reached

@cristijora Yes, but I can't guarantee a timeline, I'm swamped with work the coming weeks.

@exnihilo-creatio No worries about that. I would be grateful for the help regardless of timeline

Sorry for the delay. (finals came up)
@cristijora I had tried to do something hacky to introduce a hash as @exnihilo-creatio mentioned. If my proof of concept works I'll share and hopefully you can make it more elegant.

@vl4d Would be happy to see a working concept :)

Any more feedback on this issue? I'd be keen to have the ability to dynamically insert tabs based on the current tabs status. I'd only need to add/remove tabs in front of the current one, not behind.

@un-matthewdavis This is partially supported. It might not work as expected in some situations. It should be more or less reliable when you use an array to generate all steps rather then combine static html + a v-for
See this example as a reference https://jsfiddle.net/bt5dhqtf/1185/

Hi @cristijora ,
I have a problem.I have a question array,The number of questions is unknown, The number of answers is unknown(input type have text and checkbox and radio).
I want to dynamically generate step component,but I got a problem when I verified.I do not know how to verify(checkbox and radiobox).
I've seen you write ReadMe.md,I have more style in my vue file,
I want if can return a currentIndex and current Tab-content is good.
This is my demo  https://jsfiddle.net/YoungHarry/Lsmjrpeq/10/

Hi @zhang-yafei
The most simple way
https://jsfiddle.net/Lsmjrpeq/15/
Is to add :start-index.sync="stepIndex" on the wizard. The stepIndex will be synced and will be changed whenever the wizard goes to another step.
Another option would be on-change event
https://cristijora.github.io/vue-form-wizard/#/?id=events
<form-wizard @on-change="handleChange">

handleChange(prevIndex, nextIndex){
// This will be called when wizard goes from one step to another
}

Based on that you can access your arrayData

Thank you.I will try to solve it.Thanks again for your reply

Closing this since it's out of scope for now.
Dynamic tab contents are supported if used all together in a v-for. There is an example in the readme already https://jsfiddle.net/bt5dhqtf/1251/


SSL Connection establishment for an application that runs on WASM

Hi,
Our team is trying to build a web application that utilizes Axum for the backend, Yew for the frontend, and SQLx for middleware. We have been struggling with establishing an SSL connection between our frontend and backend through Reqwest, especially since our application runs on WASM. (Please note that we successfully built an application that communicates via HTTP. Thank you for your work!)
After dozens of days of research, we feel that Reqwest targeting "wasm32-unknown-unknown" does not currently support establishing an SSL connection because the documentation states, "Some of the features are disabled in WASM." (Reference: https://docs.rs/reqwest/latest/reqwest/#wasm)
Does Reqwest actually support SSL for applications compiled to WASM, or is this still under development?
By the way, the following code is something we have tried so far:
let response = reqwest::Client::new()         .post(url)         .json(&signup_data)         .send()         .await?;     if response.status().is_success() {         Ok(())     } else {         let status_code = response.status().as_str().to_string();         let message = response.text().await?;         Err(ApiError::Any(status_code, message).into())     }
We have also come across "reqwest-wasm," but I'm not entirely sure about the differences between this and the original Reqwest crate.
I would greatly appreciate any information you could provide regarding Reqwest(or Reqwest-wasm crate) for WASM.
Thank you!

Are you running the WASM in a browser? reqwest makes use of the fetch method, which uses the browser's ability to handle TLS connections.

Hi @seanmonstar, thank you for the reply.
Yes, we expect it to be compiled to WASM and run in a browser. To rephrase what you just mentioned, we should be able to connect these servers over SSL since Reqwest utilizes the Fetch API to handle TLS connections, provided we properly configure both the frontend and backend😏?
Would you also recommend Reqwest crate when targeting wasm32-unknown-unknown? Or you suggest Reqwest-wasm instead?
Best

The reqwest crate is meant to work for wasm targets.
I've never heard of reqwest-wasm before. I wouldn't be able to recommend it myself.

The reqwest crate is meant to work for wasm targets.
I seem to need a little bit more research and experimentation to figure out how I implement SSL connection with this crate...!

I've never heard of reqwest-wasm before
Reqwest-wasm can be found in the following URL:
https://docs.rs/reqwest-wasm/0.11.16/wasm32-unknown-unknown/reqwest_wasm/index.html
Please have a look if you are interested.

I am curious about the state in the document saying "Some of the features are disabled in WASM:tls, cookie, blocking".
Does this not mean "Reqwest is meant to work for WASM application, but not supported to work with TLS"?

As I mentioned, on wasm targets reqwest will use fetch from the browser. You can load your page in the browser and open a console and try using fetch, and if it's not working, you likely have an issue with your server.
That line in the docs is meant that you can't use the types in reqwest::tls to customize how the connection is made.


[haxe] structure types in function types are parsed incorrectly

class C {
	function foo(a:Int->{ s: String }->Void, b:Int):Void
	var bar:Int;
}

Int->?{ s: String }->Void is valid syntax for a Haxe type.
This produces some.. funny results in the outline panel. :D

@Gama11 Please, check it

Can't reproduce it anymore.


update chapel version to 2.0

Chapel 2.0 was released 3/21/2024. This PR updates the ATO Dockerfile for Chapel to use version 2.0.0

Thanks for the quick merge, @pxeger!


Extract BLE stuff into new Swift framework

@pzmudzinski Go ahead and merge.  If you decide not to change back to iOS 13 let's talk about that on Monday Standup.


Consider alternatives to dynamo

It's quite expensive. Could we use Cloud Bigtable? It seems cheaper. And we don't need all the indexy features of dynamo or Google Cloud Datastore.

The consistency requirements are "not entirely understood".
It seems intuitively that we ought to be able to get away with eventual
consistency for the chunkstore, but it's not mind-numbingly obvious how to
do that.
On Thu, Aug 11, 2016 at 2:28 PM, Eric Halpern<EMAIL_ADDRESS>wrote:

If you're considering self-hosted options as well...
Cassandra http://cassandra.apache.org/ is one obvious choice. It's the
OSS equivalent of Dynamo and is widely used. The running cost is likely to
be significantly lower than Dynamo or Cloud BigTable, but it's not trivial
to manage at scale.
Riak http://basho.com/products/riak-kv/#features is another interesting
option. It's significantly simpler and easier to manage than Cassandra but
is not entirely OSS. It is also built around eventual consistency
http://docs.basho.com/riak/kv/2.1.4/learn/concepts/eventual-consistency/
and only provides experimental support for strong consistency
http://docs.basho.com/riak/kv/2.1.4/learn/concepts/strong-consistency/.
Are the consistency requirements well understood? Is strong consistency
needed for anything other than updating the root reference? If not, then it
may be useful to separate the root update problem from everything else. For
example, one could use an eventually consistent store for all chunks and
use Zookeeper, etcd or equivalent for tracking roots.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239298504,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BFkY_H2HzPvwSLUNiQ_tn5joK5B-ks5qe5QAgaJpZM4I27-v
.

Got it. Thinking about it naively:
Since chunks and their hashes are immutable, there's no problem with
duplicate entries or conflict resolution.
So, it would come down to tolerating occasional chunk read failures by
retrying and failing gracefully when need be.
The root needs to be consistent since it's mutable and obtaining an old one
could lead to operating on old data.
Garbage collection may add some wrinkles here since it will not be possible
to accurately determine all referenced chunks (although that may already be
the case for other reasons).
Am I missing anything obvious here?
On Thu, Aug 11, 2016 at 2:33 PM, Aaron Boodman<EMAIL_ADDRESS>wrote:

The consistency requirements are "not entirely understood".
It seems intuitively that we ought to be able to get away with eventual
consistency for the chunkstore, but it's not mind-numbingly obvious how to
do that.
On Thu, Aug 11, 2016 at 2:28 PM, Eric Halpern<EMAIL_ADDRESS>wrote:

If you're considering self-hosted options as well...
Cassandra http://cassandra.apache.org/ is one obvious choice. It's the
OSS equivalent of Dynamo and is widely used. The running cost is likely
to
be significantly lower than Dynamo or Cloud BigTable, but it's not
trivial
to manage at scale.
Riak http://basho.com/products/riak-kv/#features is another
interesting
option. It's significantly simpler and easier to manage than Cassandra
but
is not entirely OSS. It is also built around eventual consistency
<http://docs.basho.com/riak/kv/2.1.4/learn/concepts/
eventual-consistency/>
and only provides experimental support for strong consistency
<http://docs.basho.com/riak/kv/2.1.4/learn/concepts/strong-consistency/
.
Are the consistency requirements well understood? Is strong consistency
needed for anything other than updating the root reference? If not, then
it
may be useful to separate the root update problem from everything else.
For
example, one could use an eventually consistent store for all chunks and
use Zookeeper, etcd or equivalent for tracking roots.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239298504,
or mute the thread
<https://github.com/notifications/unsubscribe-
auth/AAE6BFkY_H2HzPvwSLUNiQ_tn5joK5B-ks5qe5QAgaJpZM4I27-v>
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239299822,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0CBCKo9M-Jw6IzxSw4T5iox-TpOVLOks5qe5U8gaJpZM4I27-v
.

That's about right. Currently noms validates data on commit to ensure it is
complete (all claimed chunks present) and valid.
So assuming you can get commit to work then reads should just block until
requested chunk shows up. Since by definition commit should have ensured
that it will.
I think that getting commit to do this against a distributed backend
mightn't be tricky especially if you want to enable an individual
transaction to talk to multiple back ends, or be bigger than one machine
could store locally. But maybe we just don't do that.
On Thursday, August 11, 2016, Eric Halpern<EMAIL_ADDRESS>wrote:

Got it. Thinking about it naively:
Since chunks and their hashes are immutable, there's no problem with
duplicate entries or conflict resolution.
So, it would come down to tolerating occasional chunk read failures by
retrying and failing gracefully when need be.
The root needs to be consistent since it's mutable and obtaining an old one
could lead to operating on old data.
Garbage collection may add some wrinkles here since it will not be possible
to accurately determine all referenced chunks (although that may already be
the case for other reasons).
Am I missing anything obvious here?
On Thu, Aug 11, 2016 at 2:33 PM, Aaron Boodman<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>>
wrote:

The consistency requirements are "not entirely understood".
It seems intuitively that we ought to be able to get away with eventual
consistency for the chunkstore, but it's not mind-numbingly obvious how
to
do that.
On Thu, Aug 11, 2016 at 2:28 PM, Eric Halpern<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>>
wrote:

If you're considering self-hosted options as well...
Cassandra http://cassandra.apache.org/ is one obvious choice. It's
the
OSS equivalent of Dynamo and is widely used. The running cost is likely
to
be significantly lower than Dynamo or Cloud BigTable, but it's not
trivial
to manage at scale.
Riak http://basho.com/products/riak-kv/#features is another
interesting
option. It's significantly simpler and easier to manage than Cassandra
but
is not entirely OSS. It is also built around eventual consistency
<http://docs.basho.com/riak/kv/2.1.4/learn/concepts/
eventual-consistency/>
and only provides experimental support for strong consistency
<http://docs.basho.com/riak/kv/2.1.4/learn/concepts/
strong-consistency/
.
Are the consistency requirements well understood? Is strong consistency
needed for anything other than updating the root reference? If not,
then
it
may be useful to separate the root update problem from everything else.
For
example, one could use an eventually consistent store for all chunks
and
use Zookeeper, etcd or equivalent for tracking roots.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<https://github.com/attic-labs/noms/issues/1835#issuecomment-239298504
,
or mute the thread
<https://github.com/notifications/unsubscribe-
auth/AAE6BFkY_H2HzPvwSLUNiQ_tn5joK5B-ks5qe5QAgaJpZM4I27-v>
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239299822,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0CBCKo9M-
Jw6IzxSw4T5iox-TpOVLOks5qe5U8gaJpZM4I27-v>
.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239313220,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BLN05Va4oKeMFrxAaL6h8Utc7NFQks5qe6KLgaJpZM4I27-v
.

--
a (phone)

On Thu, Aug 11, 2016 at 4:09 PM, Aaron Boodman<EMAIL_ADDRESS>wrote:

So assuming you can get commit to work then reads should just block until
requested chunk shows up. Since by definition commit should have ensured
that it will.

Ah. That breaks the Has() method of ChunkStore though. Perhaps that should
go away. I'm not sure how much we use it.

We use Has() fairly productively for sync. It'd be a bummer to lose it.
One thing that we've discussed at various points is that you can imagine a caching layer which is only storing the set of Hashes which are reachable within a ChunkStore. This would have (at least) two benefits: 1) Has() would be dramatically cheaper, and 2) Get() can generally use eventually consistent reads, because it can know to retry or back off to strongly consistent on read failure (read failure should only actually occur in a vanishingly small number of reads).
IIRC, the two big questions from discussing this in the past are

Is 1% of storage size as RAM worth the trade off
If we lose the cache and need to re-create, it means reading through the entire datastore to re-create.

Sync is an interesting case because Has returning a false negative is nbd.
Just means you do a little extra work.
You could change Has to DefinitelyHas() :-).
Or just change the semantics of Has to mean it's not strongly consistent.
On Thursday, August 11, 2016, Rafael Weinstein<EMAIL_ADDRESS>wrote:

We use Has() fairly productively for sync. It'd be a bummer to lose it.
One thing that we've discussed at various points is that you can imagine a
caching layer which is only storing the set of Hashes which are reachable
within a ChunkStore. This would have (at least) two benefits: 1) Has()
would be dramatically cheaper, and 2) Get() can generally use eventually
consistent reads, because it can know to retry or back off to strongly
consistent on read failure (read failure should only actually occur in a
vanishingly small number of reads).
IIRC, the two big questions from discussing this in the past are

Is 1% of storage size as RAM worth the trade off
If we lose the cache and need to re-create, it means reading through
the entire datastore to re-create.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239357647,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BNmM9_qW4f8fTYCJxm7T2XiqkFacks5qe_QEgaJpZM4I27-v
.

--
a (phone)

That's a good point. It actually makes me wonder if the we should also push the softer read semantics up for Get(). In some sense, now that commit validation is in place, the only reason to use strongly consistent reads is to correctly fail a client which has requested a chunk which it has no reason to believe is present. Correctly behaved clients should only ever get chunks which validation has ensured are durable.

Although doing so is a bit of a layering violation, since ChunkStore doesn't know anything about values, commit or the DAG. Hmm.. Still, it's tempting.

On Thursday, August 11, 2016, Rafael Weinstein<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

Correctly behaved clients should only ever get chunks which validation has
ensured are durable.

 Right?
I think it is possible that a client commits, it passes validation; then
the client turns around and Get one of those chunks and the DynamoDB
replica that the app server talks to doesn't have it yet.
—

You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239359942,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BNcLCtiz3url4sjmLkAHySINB42tks5qe_orgaJpZM4I27-v
.

--
a (phone)

On Friday, August 12, 2016, Aaron Boodman<EMAIL_ADDRESS>wrote:

On Thursday, August 11, 2016, Rafael Weinstein<EMAIL_ADDRESS>wrote:

Correctly behaved clients should only ever get chunks which validation
has ensured are durable.

 Right?
I think it is possible that a client commits, it passes validation; then
the client turns around and Get one of those chunks and the DynamoDB
replica that the app server talks to doesn't have it yet.

(Using "DynamoDB" as standin here for any distributed db with eventual
consistency)

—

You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239359942,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BNcLCtiz3url4sjmLkAHySINB42tks5qe_orgaJpZM4I27-v
.

--
a (phone)

--
a (phone)

My point is that the client will only ever get one of two answers:

I don't have it
Here it is

This is different from other clients of eventually consistent k/v stores which also must wonder whether the data from (2) is current.
IOW, in noms, once data is committed, using a strongly consistent read has twice the cost and zero advantage to using a simple read and just re-reading if you get (1) back, since you are sure some node has it, and you just need to wait longer for the node you happen to be talking to to get it.

Ok I think we might be saying the same thing. How about we talk in person?
On Friday, August 12, 2016, Rafael Weinstein<EMAIL_ADDRESS>wrote:

My point is that the client will only ever get one of two answers:

I don't have it
Here it is

This is different from other clients of eventually consistent k/v stores
which also must wonder whether the data from (2) is current.
IOW, in noms, once data is committed, using a strongly consistent read has
twice the cost and zero advantage to using a simple read and just
re-reading if you get (1) back, since you are sure some node has it,
and you just need to wait longer for the node you happen to be talking to
to get it.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-239560900,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BG4wSI8v1cS4p1es9_WqZQq8KCZXks5qfOGggaJpZM4I27-v
.

--
a (phone)

What's the status here?

No change - it's open. Currently demo.noms.io uses leveldb which is not
durable. We also have code to support dynamo but it's expensive and slow.
It would be good to have alternatives either hosted or OSS.
On Thursday, September 15, 2016, Josh Taylor<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

What's the status here?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/1835#issuecomment-247501871,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BPpeS0x9zVwYYl1YrpHSbVVvjk13ks5qqfjzgaJpZM4I27-v
.

--
a (phone)

I'm going resolve this now that we have an AWS backend for nbs.


What's the advantage of afterware vs. middleware?

Middleware could easily function before/after a request:
const middleware = (req, options, next) => {
  options.headers = options.headers || {};
  options.headers['authorization'] = 'created token';
  return next(req, options);
};

const afterware = (req, opts, next) => {
  const res = next(req, opts);
  if (401 === res.status) {
    logout();
  }
  return res;
};

Is there a reason to have explicit middle/after stacks that I'm not seeing?

I should have a better answer for you in a couple of hours. The current reasons are simplicity of the middleware and aftwerware in their implementations and clarity of the order of their application. Your suggestion is certainly worth investigating. Are there any use cases you can think of that might need this sort of access to request and response?
While we are thinking about the middleware/afterware structure, another plan to look at is the specification for apollo-link, especially "Composing Links into a chain". Links are designed to modify GraphQL control flow with an HttpLink(calls ApolloFetch) or other transport link at the end of the chain. apollo-fetch deals with everything fetch and http specific. I'm curious to hear your thoughts on apollo-link and the interplay between link and fetch.

I guess I just think adding the concept of "afterware" just means that people have to deal with two concepts and the library has to handle two stacks/loops/etc. When everything is implemented as middleware it keeps it simple, and it allows for middleware that modifies the request and response. So in your auth example:
function makeAuthMiddleware(auth) {
  return (req, opts, next) => {
    opts.headers = opts.headers || {};
    opts.headers.authorization = auth.createAuthorizationToken();

    const res = next(req, opts);
    if (401 === res.status) {
      auth.logout();
    }
    return res;
  };
}

Using a pipeline is pretty straightforward:
function runMiddleware(middleware, req = {}, opts = {}) {
  const final = (req, opts) => {
    // FIXME:
    return `build body from ${JSON.stringify(req)} and do fetch() with ${JSON.stringify(opts)}`;
  };

  if (0 === middleware.length) {
    return final(req, opts);
  }

  if (1 === middleware.length) {
    return middleware[0](req, opts, final);
  }

  const pipeline = middleware.reduce((next, current) => {
    return (req, opts) => current(req, opts, next);
  }, final);

  return pipeline(req, opts);
}

Demo:
const req = { foo: 'bar' };
const opts = { headers: { 'x-bar': 'baz'}};
const middleware = [
  (req, opts, next) => {
    opts.headers = opts.headers || {};
    opts.headers.authorization = 'auth header';

    const res = next(req, opts);
    return `Check auth, then ${res}`;
  },
  (req, opts, next) => next(req, opts).replace(/"/g, ''),
];

console.log(runMiddleware(middleware, req, opts));

// Check auth, then build body from {foo:bar} and do fetch() with {headers:{x-bar:baz,authorization:auth header}}

I think the apollo-link spec looks interesting in a lot of ways, but it seems to me that apollo-fetch should be as simple as possible.

There are lots of cases where middleware would want to modify both the request and response. Another common one would be logging:
function log(req, opts, next) => {
  console.log('Requesting:', req, opts);
  const res = next(req, opts);
  console.log('Result:', res);
  return res;
};

@inxilpro Thank you for the suggestions! Right now, we are going to stick with the current interface for middleware and afterware, so that we get backwards compatibility with Apollo Client's middleware and afterware. Are you interested in contributing the functionality for middleware's next to return the fetch result from the afterware?
The proposed order of execution would be:
apolloFetch.use(m1).use(m2).useAfter(a1).useAfter(a2)

m1 > m2 > fetch > a1 > a2 > m2 > m1

Let me know what you think! This could be a good migration path toward deprecating afterware in apollo-fetch
It also sound like you had some ideas on how to apply the middleware/afterware more efficiently. If you are interested in that more, this code handles the ware application.

Another use case is for JWT-refresh.  The order of operation is basically:

request 1 gets a 401
separate request is made to another resource to refresh the JWT
request 1 is retried

The implementations I've seen (outside of Apollo) require keeping some state state:

"has the JWT refresh request been made?", so that the retry does not get stuck in a loop in case some other issue is causing a 401 that is unrelated to the JWT.
at the time of the response of request 1, a reference to the original request would be needed, so it could be retried.

Just my 2 cents. :)
(I eventually got here via this discussion.


Sample documentation for styling

Hello,
could you add a styling example in the documentation ? I'm not fluent using ChartJS and I didn't manage to style error bars, where should I put the configuration options ?

anything specific that you wanna achieve? Then I could create a code pen for it

Nothing fancy, the interface IErrorBarStyling is self explanatory but I was looking for a way to apply the options, something like what they did for the datalabels plugins.
Specifically I've tried this with variation of the 'errorbars' key (camel cased, with the '-', 'chart-error-bar', ...)
const chart = new Chart(ctx, {
  options: {
      plugins: {
          errorbars: {
              errorBarColor: '#36A2EB'
          }
      }
  },
  // ...
}

since they are regular graphical elements there are two locations where they can be configured.

similar to configure the backgroundColor of the rect: once per dataset (next to the data: ... )
in the default options of the elements: options: { elements: { rectangleWithErrorBar: { ... }}}.

see also https://codepen.io/sgratzl/pen/ExVYQXB.

Thanks, I didn't understood it that way. For the record, styling error bars on a scatter:
    elements: {
      pointWithErrorBar: {
        errorBarWhiskerSize: 10
      },
      point: {
        radius: 5
      }
    },


Improve test suite speed

This PR improves the test speed slightly by removing Jest snapshot testing and also adding an option to the Jest commands: --maxWorkers=1. The speed improvement is roughly around 10% for running all the test cases, whereas in watch mode it was around 300% when tested with Account.test.tsx.

@azangru @msmanoj I have now switched to using babel-jest instead of ts-jest. Tested on my machine after fixing NPM issues via npm audit fix. The time taken to run the tests is now ~10s.


fix: Set subject on CA cert used for signing self-signed certs

Signed-off-by: jannfis<EMAIL_ADDRESS>What type of PR is this?
/kind bug
What does this PR do / why we need it:
See https://github.com/redhat-developer/gitops-operator/issues/277
Recent versions of Firefox refuse to process TLS certificates that are signed by an unnamed CA. The Operator uses a CA certificate that's created without a X509 Subject, and thus, Firefox chokes.
For an existing installation, the argocd-tls and argocd-ca secrets must be deleted and reconciled back by the Operator.
Have you updated the necessary documentation?

[ ] Documentation update is required by this PR.
[ ] Documentation has been updated.

Which issue(s) this PR fixes:
Fixes https://github.com/redhat-developer/gitops-operator/issues/277
How to test changes / Special notes to the reviewer:

Install a recent version of Firefox (I used 97.0.1 (64-bit), installed via Flatpak)
Create simple ArgoCD for the Operator to reconcile and to create a new Argo CD instance. If you use route, make sure that .server.route.tls.termination is set to passthrough (or left blank, because passthrough is the default)
Point Firefox to the Argo CD UI
Connection should be successful (the unknown certificate message is expected, but should be skippable)

LGTM
thanks @jannfis


PLUGIN-1800: Adding read timeout to BQ Execute Config with default 120s

https://cdap.atlassian.net/browse/PLUGIN-1800
Retries were already configured on the BigQuery Client by default, 5xx errors are being retried 6 times before failing:
this.retrySettings = (RetrySettings)MoreObjects.firstNonNull(builder.retrySettings, getDefaultRetrySettings());

private static RetrySettings.Builder getDefaultRetrySettingsBuilder() {
    return RetrySettings.newBuilder().setMaxAttempts(6).setInitialRetryDelay(Duration.ofMillis(1000L)).setMaxRetryDelay(Duration.ofMillis(32000L)).setRetryDelayMultiplier(2.0).setTotalTimeout(Duration.ofMillis(50000L)).setInitialRpcTimeout(Duration.ofMillis(50000L)).setRpcTimeoutMultiplier(1.0).setMaxRpcTimeout(Duration.ofMillis(50000L));
  }

Hence, increasing the read timeout for BigQuery Execute action plugin (where we are validating SQL) to a default of 120s. Earlier timeout was the BigQuery default of 20s.
Confirmed via local sandbox testing that the default timeout for BigQuery Execute is 120s.

Please update the markdown as well.

As discussed offline, we will be keeping this new parameter hidden, hence not adding it to the markdown files.


Card type when user finished to add the Card Number

Hello Ben
I'm trying to add in my AddPaymentMethod-Form a flag with the brand of the card the user is adding, do you know if Stripe supports this feature? Or the card-type is just returned after the card is added to the API?
This is a picture with an example of what I need.

Check out angular-credit-cards


Add MESOP_STATE_SESSION_BACKEND_SQL_CONNECTION_URI to config docs.

Just a small fix, I noticed earlier today when chatting with docs bot. Thought docs bot was wrong, but turns out it was my fault...lol.

Lol


[FEATURE INTAKE] Improving Search relevancy through Reranker interfaces

This document captures the activities that needs to be performed in order to prepare the Re-ranking Feature https://github.com/opensearch-project/neural-search/issues/485 for release.
Release Activities
Below are the release activities that needs to be done to ensure that Re-ranking feature can be merged in 2.12 release of OpenSearch.
Code Freeze Date: Feb 6, 2024
Release calendar: https://opensearch.org/releases.html

[X] PR Merge
[x] Application Security Approval
[x] Benchmarking
[x] Documentation
[ ] Feature Demo

PR Merge
Once the PR is approved, it can be merged to the feature branch. The PR will move to main and 2.x branch once the security review and Benchmarking is done. We can wait for documentation to be completed.
Status: Completed
Application Security Approval
Status: In progress
Benchmarking
To ensure that this feature is fully tested and we are aware of latency and search relevancy impact team needs to run the benchmarks.
Status: Not started
Benchmarking Details
Benchmarking tool: https://github.com/martin-gaievski/info-retrieval-test/tree/score-normalization-combination-testing/beir/retrieval
Cluster Configuration

Config Key
Value

Data nodes
3

Data nodes Type
r5.8xlarge

Master Node
1

Master Node Type
c4.2xlarge

ML nodes
Use Datanodes as ML Nodes

ML Node Type
NA

ML Model Link
https://huggingface.co/sentence-transformers/msmarco-distilbert-base-tas-b

Heap Size
32gb

Number of Shards
12

Number of replicas
1

Number of Segments
No force merge is required.

Refresh Interval
default

Bulk Size
200

Bulk Client
1

Search Clients
1

K
100

KNN Algorithrm
hnsw

size
100

KNN engine
nmslib

space type
inner product

dimensions
768

Data sets

Data set Name
Link To download data set
Model Zip file Name
Model File Link

NFCorpus
https://github.com/martin-gaievski/info-retrieval-test/blob/score-normalization-combination-testing/README.md#beers-available-datasets
nfcorpus_traced.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

Trec-Covid
https://github.com/martin-gaievski/info-retrieval-test/blob/score-normalization-combination-testing/README.md#beers-available-datasets
trec_covid_tuned.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

Scidocs
https://github.com/martin-gaievski/info-retrieval-test/blob/score-normalization-combination-testing/README.md#beers-available-datasets
scidocs_tuned.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

Quora
https://github.com/martin-gaievski/info-retrieval-test/blob/score-normalization-combination-testing/README.md#beers-available-datasets
quora_tuned.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

Amazon ESCI
https://github.com/amazon-science/esci-data?tab=readme-ov-file#usage
amazon_traced.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

DBPedia
https://github.com/martin-gaievski/info-retrieval-test/blob/score-normalization-combination-testing/README.md#beers-available-datasets
dbpedia_tuned.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

FiQA
https://github.com/martin-gaievski/info-retrieval-test/blob/score-normalization-combination-testing/README.md#beers-available-datasets
fiqa_tuned.zip
https://huggingface.co/navneet1v/finetunedmodels/tree/main

Re-ranking Model
Use the models which are Opensource so that the results can be reproduced by other users.

Model Names
Link To download model

Once you have the results please paste the result table on the RFC. The benchmarking results will be reviewed by the maintainers of the Neural Search plugin to understand the tradeoffs. As a general guidance you should be able justify the latency trade-offs(if any) with improved relevancy metrics.
Documentation
As this is a new feature the feature owner needs to start writing the documentation for this feature. This task has not been started yet. We can add the new section under this table of Search Relevancy. https://opensearch.org/docs/latest/search-plugins/search-relevance/index/.
Status: Not started
Expectation from Documentation:

A working example should be provided to outline how to use the feature.
Examples needs to be provided on how upload a re-ranking local model.
Example ML commons blue print needs to be added and linked in this documentation on how to use remote re-ranking model like Cohere.
Example and details needs to be provided on how the query and processor can be configured. All different permutation and combination needs to be added.
Add a section of limitation if present.
If not planning to publish a blog on this, add the benchmarking details with the documentation.

Feature Demo
As this is new feature we need to a feature demo for this. Aryn team can provide a demo video.
Status: Not started

@martin-gaievski is there any chance you have a cloud formation script or otherwise that sets up the benchmarking cluster or am I on my own?

if you are ok with hosting cluster in AWS then you can use this tool - https://github.com/opensearch-project/opensearch-cluster-cdk. as your code isn't part of the official build you'll need to replace neural-search artifacts in data nodes before running benchmarks. You can try to build a complete deployable tarball using this tool https://github.com/opensearch-project/opensearch-build/, but setup maybe be a bit complex, so I suggest for one time test like this to use cluster-cdk. You can take this latest 2.12 distribution build as a basis:
https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.12.0/8999/linux/x64/tar/dist/opensearch/opensearch-2.12.0-linux-x64.tar.gz
https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.12.0/8999/linux/arm64/tar/dist/opensearch/opensearch-2.12.0-linux-arm64.tar.gz

Performance benchmarks:
First a couple notes:

Cluster configuration: 3 r5.8xlarge, 1 c4.2xlarge
I did not load dbpedia as I ran out of time (or overloaded the cluster with too many requests. both happened)
I did not test trec-covid as I ingested it with a different tool that didn't play well with the benchmarking script (mappings were misaligned)
I didn't even try Amazon ESCI
The benchmarking script appears to use the "took" field of search responses to measure time. This makes sense. Unfortunately, it appears that this field is not updated by search pipelines, so it's basically useless for the purposes of this experiment. Instead, I measured total time from request to response directly in the benchmarking script. So all measurements include the network call timing (ssh from my laptop to AWS (IAD) cluster). I wouldn't be surprised if that contributes in large part to some of the baseline measurements.
I only ran 300 queries from each dataset. This was essential for something like quora, which has 10,000 queries, and since reranking is rather resource intensive, that adds up quickly (I needed to finish this month)
I did not run these on a GPU instance. I was going for fidelity to the cluster specification listed above, but given my other issues I'm not sure that was worth it. But that would probably speed up the model inferences significantly.
All embeddings for neural search were created with "sentence-transformers/all-MiniLM-L6-v2", and all reranking experiments were done over neural searches
I used size=50 for all tests. Reranking latency scales linearly with size (except maybe if you used a remote API like cohere. But these are locally hosted model tests)
Ok, table

dataset
model
p50 (ms)
p90 (ms)
p99 (ms)

fiqa
neural, no reranking
506.0
974.4
1,335.25

fiqa
bge-rerank-base (quantized)
691.5
1,044.6
1,316.48

nfcorpus
neural, no reranking
446.0
948.2
1,326.14

nfcorpus
bge-rerank-base (quantized)
628.0
1,011.1
1,313.24

quora
neural, no reranking
506.0
952.7
1,307.02

quora
bge-rerank-base
5,497.5
6,361.3
7,446.49

quora
bge-rerank-base (quantized)
644.5
1,024.3
1,334.68

scidocs
neural, no reranking
509.0
961.4
1,302.03

scidocs
bge-rerank-base (quantized)
734.0
1,048.1
1,370.14

bge-rerank model
minilm rerank model
quantized bge-rerank model

more tests are running tonight but I'm kinda tired so I'll update this in the morning

@HenryL27 thank you for sharing these results. For the review we also need results for search relevancy, mainly nDCG@10, format can be a simplified version of one that shared in the blog for Hybrid query search

@HenryL27 I am closing this issue. As the feature is released.

hey @HenryL27 I am currently trying to replicate this results for reranking, I wanted to ask what you meant by this statement:

All embeddings for neural search were created with "sentence-transformers/all-MiniLM-L6-v2", and all reranking experiments were done over neural searches

For neural search benchmarking I am following what is in the example given by Martin to test search latency with neural query. However for reranking based on the docs I see for the feature, the query isn't a neural query. What do you mean by "all reranking experiments were done over neural searches"?

reranking is an extension of a query - you run a query and then you rerank the results from that query.

All embeddings for neural search were created with "sentence-transformers/all-MiniLM-L6-v2", and all reranking experiments were done over neural searches

means that the base query I used (and subsequently extended with reranking) was a neural query with minilm.

reranking is an extension of a query - you run a query and then you rerank the results from that query.

All embeddings for neural search were created with "sentence-transformers/all-MiniLM-L6-v2", and all reranking experiments were done over neural searches

means that the base query I used (and subsequently extended with reranking) was a neural query with minilm.

I see so the index you queried against was an knn index (or at least had embedding stored in one of the fields)? In the doc website I saw:
POST /_search?search_pipeline=rerank_pipeline
{
  "query": {
    "match": {
      "text_representation": {
        "query": "Where is Albuquerque?"
      }
    }
  },
  "ext": {
    "rerank": {
      "query_context": {
        "query_text_path": "query.match.text_representation.query"
      }
    }
  }
}

is it more relevant to benchmarking of a query that looks like this:
{
    "query": {
        "neural": {
          "passage_embedding": {
            "query_text": "Hi world",
            "k": 100
          }
        }
      },
  "ext": {
    "rerank": {
      "query_context": {
        "query_text_path": "query.neural.passage_embedding.query_text"
      }
    }
  }
}

My goal is to get a good base line for p50, p90, p99 query latency for reranker neural clause for comparison.

yeah, the second is essentially what I did in benchmarking. In general the reranking latency is gonna be substantially higher than the neural or bm25, just by nature of the computations that are going on.


Why GPU is not used when training bert model

Hi,
When training the BERT model, I didn't change the default settings file, but only 250 steps have been completed for nearly 2 days. Why is it so slow?
And when I run 'nvidia-smi' command, No running program detected, I attached below.

python run.py train experiments/spider-bert-run.jsonnet
[2020-08-03T12:37:20] Logging to logdir/bert_run/bs=2,lr=7.4e-04,bert_lr=3.0e-06,end_lr=0e0,att=1
[2020-08-03T12:44:43] Step 0 stats, train: loss =<PHONE_NUMBER>4277344
[2020-08-03T12:55:02] Step 0 stats, val: loss =<PHONE_NUMBER>2148438
^[[A^[[B[2020-08-03T13:06:20] Step 0: loss=147.2463
[2020-08-03T14:54:06] Step 10: loss=164.6095
[2020-08-03T16:26:34] Step 20: loss=207.7960
[2020-08-03T17:57:49] Step 30: loss=148.4615
[2020-08-03T19:31:26] Step 40: loss=204.2443
[2020-08-03T21:05:35] Step 50: loss=113.9567
[2020-08-03T22:38:39] Step 60: loss=147.5295
[2020-08-04T00:10:32] Step 70: loss=112.1358
[2020-08-04T01:43:07] Step 80: loss=208.5774
[2020-08-04T03:14:14] Step 90: loss=161.5002
[2020-08-04T04:54:04] Step 100 stats, train: loss =<PHONE_NUMBER>1816406
[2020-08-04T05:01:46] Step 100 stats, val: loss =<PHONE_NUMBER>779297
[2020-08-04T05:10:42] Step 100: loss=139.9932
[2020-08-04T06:45:24] Step 110: loss=97.4097
[2020-08-04T08:20:06] Step 120: loss=125.1138
[2020-08-04T09:54:11] Step 130: loss=184.3925
[2020-08-04T11:23:59] Step 140: loss=181.3083
[2020-08-04T12:58:00] Step 150: loss=106.3904
[2020-08-04T14:30:41] Step 160: loss=124.2398
[2020-08-04T16:01:37] Step 170: loss=90.2176
[2020-08-04T17:55:44] Step 180: loss=128.8443
[2020-08-04T19:14:49] Step 190: loss=124.6035
[2020-08-04T20:38:50] Step 200 stats, train: loss =<PHONE_NUMBER>8896484
[2020-08-04T20:47:12] Step 200 stats, val: loss =<PHONE_NUMBER>2470703
[2020-08-04T20:54:27] Step 200: loss=161.0717
[2020-08-04T22:16:30] Step 210: loss=182.9842
[2020-08-04T23:40:16] Step 220: loss=154.8580
[2020-08-05T01:00:57] Step 230: loss=184.8499
[2020-08-05T02:22:38] Step 240: loss=137.9579
[2020-08-05T04:04:16] Step 250: loss=91.6928

+--------------------------------------------------------------------------------+
| NVIDIA-SMI 440.33.01         Driver Version: 440.33.01        CUDA Version: 10.2     |
|---------------------------------+----------------------+-----------------------+
| GPU   Name          Persistence-M| Bus-Id               Disp.A | Volatile    Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|            Memory-Usage | GPU-Util  Compute M. |
|=======================+=================+=================|
|   0    Quadro   GV100           On   | 00000000:18:00.0  Off |                                    0 |
| 29%   35C     P2     24W / 250W |      0MiB   /    32508MiB |      0%              Default |
+--------------------------------+-----------------------+----------------------+
|   1   Quadro    GV100           On   | 00000000:3B:00.0  Off |                                  0 |
| 29%   37C     P2     25W / 250W |      0MiB    /   32508MiB |      0%             Default |
+-------------------------------+----------------------+----------------------+
|   2   Quadro    GV100          On   | 00000000:86:00.0  Off |                                  0 |
| 29%    36C     P2     25W / 250W |      1MiB   /   32508MiB |      0%              Default |
+-------------------------------+----------------------+----------------------+
|   3   Quadro    GV100          On   | 00000000:AF:00.0   Off |                                0 |
| 29%    36C       P2   25W / 250W |      0MiB   /   32508MiB |      0%            Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                                                GPU Memory |
|  GPU       PID   Type   Process name                                                        Usage      |
|=========================================================|
|  No running processes found                                                                                |
+-----------------------------------------------------------------------------+

It appears to me that CUDA is not installed in the Dockerfile. Can someone confirm?

I have same issue, my cuda version is 10.2, but default torch version is '1.3.1+cu92', then I manually reinstall torch 1.6.0, still cannot use cuda(out of the container, it works well).
root@865f29ef215f:/app# python
Python 3.7.8 (default, Jun 30 2020, 18:36:05)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import torch
import time
print(torch.version)
1.6.0
print(torch.cuda.is_available())
False

For the record, I managed to run this on GPU by changing the base image to pytorch/pytorch:1.5-cuda10.1-cudnn7-devel.


[e2p] Picture is not generated

Happens when an emote and skin tone modifier is combined.

Happens only when an emoji with a skin tone is executed on mobile devices.


Blur datepicker after date selection

Is there any possible way to force datepicker input to blur (lose focus) after a date has been selected? It is annoying because once the datepicker has been clicked it keeps focus even though it is closed so when you switch windows or tabs then return the datepicker will open by itself.

You can move focus to another input field or whatever  in onChange callback.

@alex-shamshurin can you elaborate?

@al3x-huang : Did you got solution?

I've tried using: document.activeElement.blur() and document.body.focus() in onChange event handler of DatePicker, but still the focus remains in DatePicker. I'm using Custom Date Picker here.
Snippet of my code:

handleStartDateChange = (newValue) => {
console.log("document ", document.activeElement)
document.activeElement.blur()
//document.body.focus();
console.log("document ", document.activeElement)
}
One interesting thing is on consoling log of document.activeElement. After blur() is being called the activeElement is logged as body, but the focus remains on DatePicker.
The issue surfaced when I've upgraded react-datepicker library to version: "react-datepicker": "^6.6.0"
It was working fine in version 2.9.6. By fine I mean that the DatePicker was losing focus when the date is selected.
Can somebody help me here.


Haptic feedback should be independent on system settings

I think that haptic feedback should not require having global system haptic feedback. I am not a fan of global haptic feedback as there is no reason for it IMO. This keyboard is a special case which is much better with haptic feedback. I would suggest (and volunteer) to implement independent haptic feedback with separate preference for vibration strength for these actions:

crossing diagonal separator (would be super helpful with smaller circles)
writing a letter

That would be great.
To be honest I didn't even knew we can do something like that in the first
place...
On Tue, 25 May 2021, 19:26 MichalLebeda, @.***> wrote:

I think that haptic feedback should not require having global system
haptic feedback. I am not a fan of global haptic feedback as there is no
reason for it IMO. This keyboard is a special case which is much better
with haptic feedback. I would suggest (and volunteer) to implement
independent haptic feedback with separate preference for vibration strength
for these actions:

crossing diagonal separator (would be super helpful with smaller
circles)
writing a letter

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/flide/8VIM/issues/184, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABX5WR6SZE3J7CWSJGSAKSTTPOUCRANCNFSM45PNXNFQ
.

Another FOSS keyboard I'm using, FlorisBoard,  has an independent haptic feedback option that may or may not be helpful to look at. (I'm not a coder, sadly.)

Agreed, but I would like to mention that some users might want an option to have the settings linked.


does not handle Negated lines

Is QuickUMLS able to handle negations?
(e.g no sign of fever)

Unless a concept appears in negated form in UMLS, it does not.

OK, thanks for the quick reply!

Any change to this?


FUSEDOC-1898 updated redis URL

Hi - I'm contributing a fix to the ASCII-doc documentation with a simple fix to a URL. Can you review it when you get a chance?

Thanks for the PR

Thank you!
On Fri, Nov 10, 2017 at 1:52 PM, Andrea Cosentino<EMAIL_ADDRESS>wrote:

Thanks for the PR
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/camel/pull/2089#issuecomment-343555791, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AecjFt1GmZ5vpqnbVii3qJxh3HBTA9UBks5s1JtrgaJpZM4QZ9cY
.


Updated ruby version required to 2.3

The theme doesn't require ruby 2.4.2. Set a sane requirement to ruby 2.3 which is a stable supported version.

@shovanuk @kylekirkby following up my mail to Kyle, here's the PR.

Thanks @fboudra !


Expose SagaRepository and SagaManager as Spring Beans when using @Saga

When using Spring autoconfiguration to initialize components for an @Saga, the SagaRepository and SagaManager aren't accessible as Spring beans in the context.

Moved to 3.1, as this requires a redesign on how Axon uses autoconfiguration (internally).
Instead of using Axon Configuration API, it should use "regular" Spring Boot Autoconfiguration to configure beans part of Saga infrastructure.

Access of the autoconfigured components for Sagas are available through the SagaConfiguration instance, which is published as Configuration in the application context.
That bean can be autowired (using @Qualifier).


feat: new content map for Android docs

depends on https://github.com/100mslive/100ms-docs/pull/1154
content map

in-flight JS pages: add to content map

[ ] session metadata

content gaps

[x] ordering of parent and child pages
[x] missing pages - call out in main spreadsheet

impact of link changes

[x] redirects to old links
[ ] checks for 404s - especially in FAQs

@arjunattam the goswami should likely start with a capital

has the drop-down on the docs landing always been broken?


openfinex account creation

Openfinex keeps track of the accounts and balances as well. Therefore we need to forward every deposit and withdraw call to openfinex as well. Openfinex will (not yet implemented on their side) register new accounts upon a deposit call on an account that is not yet registered.
Because this is a tbd feature, for now we need to manually register the accounts with exec:
Exec to database:
docker-compose -f compose/upstream_compose.yml exec database mysql -uroot --database finex_development

The members table of Openfinex looks as following:

New findings: We can not register our user by uid, because our AccountIds are too long.
Workaround: Register users with username.
Example commands to register //Alice:
INSERT INTO members(uid,email,level,role,state,created_at,updated_at,username) VALUES('ALICE','protected',3,'member','active','2021-05-20 14:43:58','2021-05-20 14:41:41', '5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY');
and //Bob:
INSERT INTO members(uid,email,level,role,state,created_at,updated_at,username) VALUES('BOB','protected_bob',3,'member','active','2021-05-20 14:43:58','2021-05-20 14:41:41', '5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty');


Step 5 incorrectly refers to enabling compute network

Step 5 of the ASE setup section contains the following contradictory text.
_"When carrying out the Enable compute network step of this procedure, ensure you use the port you've connected to your management network."
"Do not configure virtual switches, virtual networks or compute IPs."_
However, the 'enable compute' step is achieved by configuring a virtual switch, so these instructions contradict each other.
Compute is actually not enabled until later in the process (on the "Commission the AKS cluster" page) so the "When carrying out the Enable compute network step of this procedure..." text just needs to be deleted from step 5.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 73a16fd4-712b-11f5-88ad-b093cb9cec13
Version Independent ID: 814c890e-6d14-5059-1f0e-1988466b788f
Content: Prepare to deploy a private mobile network - Azure Private 5G Core
Content Source: articles/private-5g-core/complete-private-mobile-network-prerequisites.md
Service: private-5g-core
GitHub Login: @robswain
Microsoft Alias: robswain

@robertstarling
Thanks for your feedback! We will investigate and update as appropriate.

The fix for this is now live.
#please-close


gitlab: don't use deprecated token database column

Summary
This uses the "encrypted access token" column which we should have fully
migrated to by now. We don't need the fallback code at all in this case.
Change Type
Mark the type of change your PR introduces:

[ ] Bug fix (resolves an issue without affecting existing features)
[ ] Feature (adds new functionality without breaking changes)
[ ] Breaking change (may impact existing functionalities or require documentation updates)
[ ] Documentation (updates or additions to documentation)
[x] Refactoring or test improvements (no bug fixes or new functionality)

Testing
Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.
Review Checklist:

[x] Reviewed my own code for quality and clarity.
[ ] Added comments to complex or tricky code sections.
[ ] Updated any affected documentation.
[ ] Included tests that validate the fix or feature.
[ ] Checked that related changes are merged.

coverage: 53.098% (+0.01%) from 53.087%
when pulling c770b52ea429afde4f1395d50f37f6add3899e76 on gitlab-token-parsing
into ca957cb1e485501faa6e05aefc6ba593d6596f36 on main.


/endpoint gets proxied to /endpoint/

I have an endpoint /endpoint which I want to proxy to, however all my requests get redirected to /endpoint/ which doesn't exist.
It is set up like so:
app.use('/endpoint', proxy('http://somehost.com/endpoint'));
The error seems to be somewhere in slashJoin in index.js.

Running into this issue as well.  @0xR you have a PR lined up?  If not I'll see if I can make the change.  @andrewrk what are your thoughts on this?

My thoughts:
I don't write code for this module any more (I'm pretty sure the first version I released didn't have this problem), I just merge pull requests if the tests pass. It's an experiment in a community maintained module.

my PR broke the tests; working on fixing that now.  I'll let you know when they're up to snuff, thanks!

this issue can be closed.

jeez, feel stupid here, but when I fixed the tests I somehow reverted my fix ... :-1:
let's keep this open, sending another PR - its been a long week!

Thanks for making a fix. Instead of spending time making a PR I just switched to express-http-proxy. Good luck on recreating the fix haha.

We are experiencing the same issue. This module is used in ionic and our proxies pointing to /endpoint are redirecting to /endpoint/ causing an issue. Are there any other workarounds?

@koga73 I know it's late, but there is a workaround using the route option:
var url = require('url');
var options = url.parse('http://www.website.com');
options.pathname = '/endpoint';
options.route = '/api/endpoint';
app.use(proxy(options));

This is possible thanks to the url rewriting made to this route option.

@Pyo25  it's not the same question
app.use('/endpoint', proxy('http://somehost.com/endpoint'));
app.use('/api/endpoint', proxy('http://somehost.com/endpoint'));


New indoor temperature returns 0

The new sensor only returns 0 *C despite it actually being 24 *C

Enable all logs in plugin settings and debug mode in homebridge settings and restart homebridge. Then it will shows you log. Paste here response, it will be something that contains parameters {}

4.2.5 should fix this. You need to enable again sensor for inside temp.

You're awesome!


Upgrade apollo and related packages

Related Issue or Background Info

Resolves #2910

Changes Proposed

Upgrade @apollo/client to 3.4.17, the latest 3.4.x. The latest is 3.5.8, but upgrading to any 3.5.x broke our frontend unit tests, and I couldn't for the life of me figure out why. Changelog
Upgrade apollo-upload-client to 17.0.0. Changelog
Upgrade various @graphql-codegen packages, all minor version upgrades. Also regenerated the output. Changelog
Upgrade graphql to 16.3.0. Lots of breaking changes, but none seem to affect us. Changelog

Additional Information

decisions that were made
discussion of tradeoffs / future work
complaints about how bad the code is

Screenshots / Demos
Checklist for Author and Reviewer
Infrastructure

[ ] Consult the results of the terraform-plan job inside the "Terraform Checks" workflow run for this PR. Confirm that there are no unexpected changes!

Design

[ ] Any UI/UX changes have a designer as a reviewer, and changes have been approved
[ ] Any large-scale changes have been deployed to test, dev, or pentest and smoke-tested by both the engineering and design teams

Content

[ ] Any content changes (including new error messages) have been approved by content team

Support

[ ] Any changes that might generate new support requests have been flagged to the support team
[ ] Any changes to support infrastructure have been demo'd to support team

Testing

[ ] Includes a summary of what a code reviewer should verify

Changes are Backwards Compatible

[ ] Database changes are submitted as a separate PR

[ ] Any new tables that do not contain PII are accompanied by a GRANT SELECT to the no-PHI user
[ ] Any changes to tables that have custom no-PHI views are accompanied by changes to those views
(including re-granting permission to the no-PHI user if need be)
[ ] Liquibase rollback has been tested locally using ./gradlew liquibaseRollbackSQL or liquibaseRollback
[ ] Each new changeset has a corresponding tag

[ ] GraphQL schema changes are backward compatible with older version of the front-end

Security

[ ] Changes with security implications have been approved by a security engineer (changes to  authentication, encryption, handling of PII, etc.)
[ ] Any dependencies introduced have been vetted and discussed

Cloud

[ ] DevOps team has been notified if PR requires ops support
[ ] If there are changes that cannot be tested locally, this has been deployed to our Azure test, dev, or pentest environment for verification

Spent some time this morning digging into why 3.5.x is breaking our tests. Didn't get into a firm resolution but leaving finding here for posterity/in case it sparks anything for anyone.
It looks like the failing tests are generally failing because the mocks we're using on the relevant components aren't finding enough defined mocks as passed into the MockedProvider components we're using to simulate GraphQL calls. For example, from the MangedUsers.test.tsx file

Adding back in the specified mock in the error console (which in this case is just duplicating a mock that's already been defined) fixes the errors related to that issue.

There are some notes in the 3.5.0 changelog that describes changes to the way Apollo is handling the cache (which tbh I don't really understand) that might be related? Potentially defining those mocks once in 3.4.x and reading subsequent ones from the apollo cache (if this is how this works) might have changed in 3.5.0 given the changes. I might be way off base here though since I don't really understand much about the interactions at play here between RTL/Apollo, and React/GraphQL

Great digging Bob! I had seen the missing mock messages as well, but I didn't see the notes about changes to cache handling... I will take another peek there and see if anything stands out. Thanks for looking into it!!


Update installation

Coverage decreased (-45.6%) to 12.766% when pulling a3a9c8939dae319470f914a42da20b92834dcfa2 on installation into 9dc8a3cdc4167e5b843e9f893bfd886cac712d2c on master.

Coverage decreased (-0.05%) to 58.313% when pulling a5b792620763a078a9c10baccdd56f4103b5b144 on installation into 9dc8a3cdc4167e5b843e9f893bfd886cac712d2c on master.


[Feature request] Allow the macro to randomize unknown or renamed creatures

When a creature is renamed, the macro wont recognize it, and will not allow randomization of the token.

My request is that on such situations the macro will bring a dialogue with creature types, and if I click humanoid, it will make the token image humanoid. If I click dragon, it will make it into a dragon; Regardless of the token name.

Added new macro to todays version. Thank you for the suggestion. :D


Compile python requirements

Automated changes by create-pull-request GitHub action

Almost end of the week and time for the action to re-run, just going to close this one


Speed up shortest path discovery

Shortest path discovery is too slow, even on a laptop, when the number of nodes passes 50K. This is probably because links are stores in a compressed format that takes time to decode. Having the links in a simple array of integers and storing their coordinated & colors in a separate packed structure would be a better trade-off between speed & space.

Ah-hoc testing shows that the main time eater is finding the current shortest path during the traversal.
Sanity checking shows that Dijkstra's shortest path is overkill when all edges has the same weight. Switching to breadth-first search should speed thing up by at least a factor 10, but also rules out later tricks, such as prioritizing out-links over in-links when discovering a path from node A to B.

Next sanity check: The current implementation does behave as breath-first search and the current shortest path is simply the path at the head of the unvisited nodes. Quick-fix is to return that at once and postpone the challenge of making a decent priority queue in JavaScript until different weights for the edges becomes a factor.
Idea: Assign a very high weight to the "boring" domains google.com, facebook.com, etc. to ensure they are only visited as last resort.


Video hangs/stalls often

Hi,
I am using react native video to play MP4 videos. These videos are sent from a server (I.e. They are non local) and are no large than 5MB in size on average.
I'm pretty sure the loading time is fine as I'm getting the onLoad event, and the videos start playing. However, the videos tend to freeze often. I have to pause/play them to get them going again. I don't think it's a streaming / internet issue (because I think the videos are loaded entirely), so I think it might be an issue on the mobile side.
Is anyone else seeing something like this? Any help would be welcome.

Can you provide any sample code to allow us to reproduce the issue?

Dupe of #303


Appstudio update test-component-pac-mskc

Pipelines as Code configuration proposal

Pipelines as Code CI/test-component-pac-mskc-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
7 seconds
init

✅ Succeeded
12 seconds
clone-repository

✅ Succeeded
29 seconds
build-container

✅ Succeeded
22 seconds
inspect-image

✅ Succeeded
25 seconds
deprecated-base-image-check

✅ Succeeded
3 minutes
clair-scan

✅ Succeeded
52 seconds
clamav-scan

✅ Succeeded
20 seconds
sbom-json-check

✅ Succeeded
8 seconds
optional-label-check

✅ Succeeded
7 seconds
label-check

✅ Succeeded
6 seconds
show-summary

Pipelines as Code CI/test-component-pac-mskc-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
9 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
41 seconds
build-container

✅ Succeeded
12 seconds
inspect-image

✅ Succeeded
11 seconds
deprecated-base-image-check

✅ Succeeded
9 seconds
clair-scan

✅ Succeeded
33 seconds
clamav-scan

✅ Succeeded
9 seconds
sbom-json-check

✅ Succeeded
18 seconds
label-check

✅ Succeeded
19 seconds
optional-label-check

✅ Succeeded
5 seconds
show-summary


minor bug: tdi_T1.mif

I stumbled upon a minor but consequential bug in the script when verbosity >=2
the participant module of the script creates tdi_T1.mif in the scratch directory but after completing all the steps it wants to copy tdi_t1.mif (small t) to the output directory which does not exist.

Hi @chrisvriend,
This has already been fixed in 2433a906 (#100 / #104), but I am yet to merge the hotfix branch into master and release version 0.5.1. Unfortunately I'm so overloaded with other roles that this little side project is persistently neglected. But I'll try to get that done soon while there are less people demanding things of me.
Cheers
Rob


Update self-signed-certificates.md

Fixes #85273

@simonkurtz-MSFT : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Can you add support for Markdown highlighting?

I really love your theme. And as I switch the syntax to markdown, I am not seeing any hightlights.

Sure!

Done! Should be available in 0.2.0.


Add draft label from mergify

Signed-off-by: bindrad<EMAIL_ADDRESS>

@Mergifyio refresh

@Mergifyio refresh


Wrong Angola flag

I think the Angola flag is wrong ... the flag getting displayed is the Lichenstein one!

Apparently the Angola flag is missing in the PSD, we've to create it.
Thanks for the info!


[Feat]: Git auth

remove commenting diff from ci also pls, just ensure all the functions run correctly


All entities are "unavailable"

I got this issue today after integrating GOeCharger Version 0.15.1. Mostly all entities are in status "unavailable". Hope the following log entry will help to solve this issue.
WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that called async_setup_platforms
instead of awaiting async_forward_entry_setups; this will fail in version 2023.3. Please report issue to the custom
integration author for goecharger_mqtt using this method at custom_components/goecharger_mqtt/init.py, line 44:

Set base topic to "go-eCharger" without leading "/" in the configuration and it works now. Thx for your great support!


Creating a custom color map

Hi,
I'm accessing some results from an .rst file but would like to use a specific contour color map that I got directly from Ansys (see below and attached) when I plot these results using the result.plot_nodal_stress ( ) command. So far I think it's only possible to specify number of contours (n_colors=7) within this plotting function and use cmap to specify colors (cmap= ['green', 'red', 'blue'])
result.plot_nodal_stress(0, 'x' , cpos='iso',text_color='k' ,cmap= ['green', 'red', 'blue'])
Describe the solution you'd like
I would like to be able to specify the contour colors using the result.plot_xxx_xxx ( ) commands available in pyansys.
Something like:
result.plot_nodal_stress(0, 'x' , cpos='iso', cmap= ['1     0     0   100', '2     0   100   100', '3     0   100     0'])
Describe alternatives you've considered
I've been through the pymapdl, legacy pymadl reader and pyvista documentation but so far I have only seen things like building a custom map but don't seem to find a way to integrate this when using the result.plot_xxx_xx functions available in pyansys. Any help is appreciated.
Additional context
Contour color map from ansys:
*** CONTOUR COLOR MAP CREATED FOR THE WIN32C DRIVER ***
1     0     0   100
2     0   100   100
3     0   100     0
4   100   100     0
5   100     0     0
colormap_ansys.txt

Hi @marj44
I believe we have the capabilities you are requesting. You can get the default ANSYS colormap by importing:
from ansys.mapdl.core.examples import ansys_colormap

Then you can use it as:
mapdl.post_processing.plot_nodal_displacement(
        "NORM",
        cmap=ansys_colormap(),
        line_width=5,
        cpos="xy",
        scalar_bar_args={"title": "Displacement", "vertical": False},
    )

I do recognize though that this CMAP function is big hidden and not very advertise, but it has been very recently added.
I am closing the issue, but feel free to reopen it if something is not clear.

Is there any documentation on this CMAP function?

Hi @marj44
Plotting capabilities in PyMAPDL relies on Matplotlib (for 2D) and Pyvista (for 3D).
Hence you can find more information about colormaps (cmaps) in their correspondent help pages. It is not the intention of this package to "duplicate" documentation because that could lead to mistakes, outdated documentation, etc. Even more, since PyMAPDL does not contribute to any of those libraries, it would not make much sense to include their documentation from a copyright perspective.
However, I think it is a good point for you to start checking the colormap API (cm: https://matplotlib.org/stable/api/cm_api.html ).
This command ansys_colormap will always give you the same colors, because it is intended to replicate original ANSYS color palette.
If you check the ansys_colormap function source code in:
https://github.com/pyansys/pymapdl/blob/94ae01b51bd781dc1c29f8f48c951952daafa812/ansys/mapdl/core/examples/examples.py#L17-L31
You will see that is just a function which generates a colormap from a list of colors using Matplotlib ListedColormap (from matplotlib.colors import ListedColormap).
You could possibility write something like:
import numpy as np
from matplotlib.colors import ListedColormap

my_colors = np.random.randint(low=0, high=255, size=(10,3))/255  #Random colors
#Or replace this by your own colors.
my_colors = np.array([[140, 105, 255],
[95, 178, 255],
[0, 255, 255]])/255
my_cmap = ListedColormap(my_colors)

However, if you only supply 3 colors, you colormap is going to be restricted to them. You could probably have a look at interpolation tools to get intermediate values between your colors.


help

Hello, author:
This open source code of yours does not seem to be particularly studious, as it is more like an integrated third-party library, what would you recommend for beginners?
Looking forward to your reply.

Hello, thank you for raising this issue.
We have created a nice introductory Jupyter notebook where you can get an idea of how the simulator works and how to apply Reinforcement Learning with EV2Gym (https://github.com/StavrosOrf/EV2Gym/blob/main/tutorials/CreateYourCustomRLAlgorithm.ipynb).
Furthermore, I suggest you check the EV2Gym paper to get a better understanding of all the mathematical models used in the simulator. The paper is available for free at ArXiv (https://arxiv.org/abs/2404.01849), now also published at IEEE Transactions on Intelligent Transportation (https://ieeexplore.ieee.org/document/10803908).
Last but not least, I suggest you take a look at the README file which has a short guide on getting started. Also, try to run the example.py file and get some outputs (https://github.com/StavrosOrf/EV2Gym/blob/main/example.py)

Thank you very much for your reply
I wonder if it is convenient for you to add a requirements.txt file so that we can easily install the libraries that the project depends on.

This is indeed a nice suggestion! I added a requirements.txt file for the EV2Gym dependencies. Note that, for libraries not included in requirements.txt but are needed you can update the latest version. For example, stable-baselines3.


Whitespace changes from running clangformat on generated code

Summary

This PR is a change of whitespace formatting only
It does the following (modify list as needed):
Runs clangformat on sidre's shroud-generated code to make it consistent with expected style.

IIRC, we argued over the 'T* var' vs. 'T *var' thing in the past and we agreed to do the former. Is there a reason that this has to change to make the shroud generated code look pretty?

@rhornung67 -- I agree that our agreed-upon style is the former, but this change was automatically made by our clang-format setup. I think we'd need to make the change there.

Right. I though the changes in this PR were due to something related to shroud. Sorry for the confusion.


Make server-shelf compatible with Cloud Run

Cloud Run has a couple of requirements of apps that it attempts to run. The first is that they bind to the port specified in the PORT environment variable, and the second is that it listens to <IP_ADDRESS> instead of localhost. This change should make it as simple as adding a simple Dockerfile to the generated sample app to get it running on Cloud Run.

re-run pub run build_runner build – would be good.

PTAL @kevmoo

In theory we could publish this now, although I don't know if there is any urgency.
CC @kwalrath

This change came out of a Flutter Hackathon event where I and an attendee tried to use Dart on Cloud Run to act as the server side of a Flutter app. We both burned the majority of the event trying to figure out what we were doing wrong. Please publish this ASAP to prevent developer frustration.
Thanks!

Deployed as 3.3.1!


feat: Gracefully drain listeners before envoy shutdown on pod termination

What this PR does:
This PR is a step in the direction of achieving a hitless upgrade. Some challenges to achieving this have been outlined by @guydc in https://github.com/envoyproxy/gateway/issues/1712#issuecomment-1943985961 and https://github.com/envoyproxy/gateway/issues/2610
This proposed implementation of a PreStop lifecycle hook will allow Envoy proxies to gracefully shutdown when pods are terminated by Kubernetes. This is done by:
a) Initiating a graceful drain sequence via the drain_listeners?graceful admin endpoint.
b) Sleeping for a minimum drain duration of 15 seconds allowing ample time for endpoint deprogramming to occur.
c) Monitors total.server_connections statistic exiting once total connection count reaches 0
d) Times out after 600 seconds after which Envoy will be sent SIGTERM by Kubernetes regardless of how many open connections remain. This timeout lines up with the default value for --drain-time-s
e) Configures terminationGracePeriodSeconds to 900 seconds. This provides ample time for Envoy to cleanly exit after connections have been drained.
Some prior art w.r.t graceful shutdown for Envoy:

Contour: https://github.com/projectcontour/contour/blob/main/design/envoy-shutdown.md
Gloo Edge: https://docs.solo.io/gloo-edge/latest/operations/advanced/zero-downtime-gateway-rollout/
Contour: https://www.reddit.com/r/RedditEng/comments/1aqsxqf/proper_envoy_shutdown_in_a_kubernetes_world
Istio: https://github.com/istio/istio/blob/master/pkg/envoy/agent.go#L151-L152

Summary of why we need this:

Kubernetes uses SIGTERM to signal processes to gracefully shutdown, however Envoy does not support gracefully draining connections during shutdowns triggered by SIGTERM.
Endpoint deprogramming is eventually consistent and the time this takes varies somewhat based on the k8s implementation and size of the cluster.

Caveats:
When TCP listeners are used (rather than HTTP listeners), connections will not drain per the below note from draining architecture documentation. This means in the event of long-living TCP connections, Envoy proxies will simply wait in the PreStop hook for the 600 second timeout to occur after which these connections will be closed when Envoy receives the SIGTERM from Kubernetes.

Note that although draining is a per-listener concept, it must be supported at the network filter level. Currently the only filters that support graceful draining are Redis, Mongo, Thrift (if the envoy.reloadable_features.thrift_connection_draining runtime feature is enabled), and HTTP connection manager.

Reference: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/operations/draining

Testing / Load generator with unaltered EG quickstart.yaml:
run from another pod within the cluster
hey -c 100 -q 10 -z 300s -host www.example.com http://<IP_ADDRESS>/

PreStop logs from rolling restart of single replica deployment
$ kubectl logs -n envoy-gateway-system deploy/envoy-default-eg-e41e7b31 -f | grep -v start_time
Found 2 pods, using pod/envoy-default-eg-e41e7b31-85c848c76b-w8p5t
[2024-02-16 21:23:58.508][1][warning][main] [source/server/server.cc:923] There is no configured limit to the number of allowed active downstream connections.

This lines up with the default value for --parent-shutdown-time-s

Is there any significance to using the same value here, if we're not using the envoy hot reload feature?

@guydc No particular significance to the value being the same in the context of k8s, but it is why I chose 5 minutes greater than the drain timeout since the docs state the drain time should be less than the parent shutdown time set via the [--parent-shutdown-time-s](https://www.envoyproxy.io/docs/envoy/latest/operations/cli#cmdoption-parent-shutdown-time-s) option.
The important bit is that terminationGracePeriodSeconds must be higher than the graceful drain timeout so that k8s will send a SIGTERM instead of a SIGKILL if the drain runs up the clock against the full 600 second drain timeout.

/retest

/retest

/retest

/retest


Add skip drain flag.

Sometimes it's useful to deploy without draining some or all instance groups--for example, after you break a drain script 😇
cc @wjwoodson @timothy-spencer

Thanks for submitting this! Released as part of 2.11.0


Update CHANNELS.md

Coverage remained the same at 57.232% when pulling a762d8069d7d0a3a408bd4737306252a251ae0fe on v-kydela/inspector into c15f3ebad76832167dd4d7fab1b18c3dbac3750c on master.