soroban-rpc: fix datarace in bufferedResponseWriter.WriteOut

What
Fix data race #959 .
Why
When implementing bufferedResponseWriter.WriteOut, I attempted to implement the writing to the output stream asynchronously, so that in case of context expiration the writing to the channel would be suspended at once.
Unfortuntally, it led to conflict with go http server library : When the bufferedResponseWriter.WriteOut could not complete writing to the stream in time, the method would return. However, the writing operation was still going on it's own pace.
then, the http server was attempting to write to the stream, creating a data race.
The solution is to write to the stream in smaller chunks, and continuously check the ctx.Err in order to examine when it expires.
Known limitations
n/a

I would advocate to simply writing all at once. I think responses are small, so I think we can afford to surpass the deadline by the writing time.

I'd like to keep it for protection against malicious client leaving the connection open and draining it at a low rate.

chunks are not going to help you with that either, they can block when reading a single byte, so your solution doesn't really solve the problem


Update welcome_message.php

I have changed the styling of the default welcomes page.  It now uses the same styling as the website of codeigniter.com.
I have included external links to the logo and font files. It links to rawgit, it's a CDN service for github files. Although I don’t like a dependency to an external service I feel in this case it's a valid choice since I don't want to bloat up the installation size by including a bunch more files just to fancy up the installation file.
If the CDN is down it still looks good enough. Only the logo is missing and the heading text is Helvetica again.
One thing I was not quite sure about was using the heading font this way. Perhaps this is screwing to much with the corporate identity as the term "welcome to" is inserted between the fire image and the text part of the logo.
Perhaps an alternative would something like just have the flame and text logo in the red bar and put the "Welcome to CodeIgniter" text below the red bar and use a regular heading style on it (dark bold text).
Anyway, here is a screenshot of the result:

External assets are a blocker.
Any styling changes must be achievable via inline CSS alone (yes, this means no images and no custom fonts), and they must also be applied to default error templates.

I understand. Is an inline base64 image also a dealbreaker?

No, as long as it's inline - that's ok. Although it might be tricky to mask it on browsers that don't support Base64-encoded images, if any.
But unlike with other stylings, the logo (and any reference to CI itself) must stay only on the Welcome page. Obviously, the error templates will still have recognizable styling, but other than that, they shouldn't leak data about the platform.

Ok, I will give it another go if I have another spare half hour in the next couple of days. A quick search tells me inline images are supported widely (from IE8).
http://caniuse.com/#search=data

Ok I've had some to time to make the ajustments. I've created an inline image for the logo in the welcome page. Changed the error messages as you suggested to match the same color schema.
Welcome page:

404 error:

DB error:

Show_error example:

Looks better (the images in your comment at least).
But please, revert the coding style changes ... every second line is an empty one and you're replacing tabs indentation with 4 spaces instead - we don't do that. And more importantly, it's really hard to review and it shouldn't be - a few CSS property changes shouldn't result in a 100-line diff.

I've made the coding styles changes like you suggested. Still contains alot of diff rules because I added a wrapper around the main container, so I needed to indent the original code.

@DiederikLascaris Why do you need that wrapper?
@jim-parry What do you think about this one? Should it go into 3.0.x or 3.1?

I see no problem including it in 3.0.x

From: Andrey Andreev<EMAIL_ADDRESS>Sent: Thursday, August 20, 2015 6:25 AM
To: bcit-ci/CodeIgniter
Cc: James Parry
Subject: Re: [CodeIgniter] Update welcome_message.php (#4030)
@DiederikLascarishttps://github.com/DiederikLascaris Why do you need that wrapper?
@jim-parryhttps://github.com/jim-parry What do you think about this one? Should it go into 3.0.x or 3.1?

Reply to this email directly or view it on GitHubhttps://github.com/bcit-ci/CodeIgniter/pull/4030#issuecomment-133006114.

I see a couple of issues with this PR:

Inconsistency with hex colours. Use either upper-case or lower-case chars for A-F, not both.
You honestly could do away with wrapper and use the body if you're wanting to centre (which is what I presume you're trying to do)

This got lost under the radar ... sorry.
@DiederikLascaris If you're still interested, do what @softwarespot suggested and I'll be happy to merge it.

Interesting...

@DiederikLascaris if you are not in position to make this update now, I can do it -- let me know If I can go ahead.

@narfbg -- I just did, feel free to close it if you @DiederikLascaris makes the same changes on this one.

@narfbg @jim-parry - anything else that should be addressed on the PR #4973 ?

I am fine with this :)

@narfbg thanks for the reminder on this subject.
I have removed the wrapper and changed the hex colors to lowercase, also changed the color name 'white' to #fff to be more consistent.

Thanks!


Emoticons containing { can't be included in an intent utterance

If an emoticon such as :{ is included in the utterance text for an intent it's treated as a beginning of one of the special cases that begin with { - slot, option list, etc.
Need to create an exception - when a string is encountered that contains { and matches an emoticon and has white spaces around it (or is at start/end of the utterance) then it's treated as that emoticon.

resolved in v1.3.26


[1888] trainee record page change link for lead school

Context

This trello ticket
I also fixed a bug where the Schools::FormValidator was not being used for the confirm page
and that the Schools::FormValidator rehydrated the two forms with new_attributes from the FormStore

Changes proposed in this pull request

Confirmation Controller uses the Schools::FormValidator for the schools form
Schools::FormValidator uses form-store values as new attributes and merges them with the trainee
Change link from the record page goes to the schools search page

Guidance to review

Create a trainee on the School Direct Tuition Fee AND/OR the School Direct Salaried route and complete them
Submit the trainee for TRN
On their record page, change either their lead and/or employing school.
You should be redirected to their search page, and once a new school is chosen, you are redirected to the confirm page

 [M] Trainee record page 'Change' link for lead school

All good


Add FirebaseUI for interface for authentication

I don't know why but whenever I have added the dependency for FirebaseUI and synched the app, it shows error in the mainActivity file that 'R is unresolved'.

added firebase ui


Add Artists and works library tabs

-Use react router for url history management
-Various Refactors

Add library tabs for artist and each work type

Pull request moved to #12


Peer-to-peer code review

You can leave your feedback here 💡
Happy Coding!💻

Seeing as how I worked on this I don't exactly have any big suggestions. That said, it did bother me at the time that we ended up making the supporting-text element in the headline section an h2 element to make up for the one we didn't have.
I can't help but feel that it'd be much better from a structural point of view to have all then project cards in a section and have a title for that section which is the missing h2 element.
Of course Template 1 possesses no such element, but to be honest I always felt that was a lacking in the template from a structural point of view. This is really more a matter for design than code though.

You have done a great job with an effort, keep it up 👍 💯

I think you have done a good job, i have nothing to say about the code 💯
I just want to suggest you to add a function when someone click in the menu part in the header on portfolio, about or contact to go to the section 👍


Unable to deploy UAA locally using gradlew run

What version of UAA are you running?
75.6.0
What output do you see from curl <YOUR_UAA>/info -H'Accept: application/json'?
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to localhost (<IP_ADDRESS>) port 8080 (#0)
> GET /uaa/info HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: application/json
>

How are you deploying the UAA?
I am deploying the UAA

locally only using gradlew

What did you do?
cd uaa
git checkout 75.6.0
./gradlew run

What did you expect to see? What goal are you trying to achieve with the UAA?
I shoud be able to run curl localhost:8080/uaa/info -H "Accept: application/json" and get the expected output to confirm its working.
What did you see instead?
root@ip-XXX-XX-XX-XX:/workspace/uaa# ./gradlew run

> Task :cargoRunLocal
org.codehaus.cargo.container.ContainerException: Deployable [http://localhost:8080/cargocpc/index.html] failed to finish deploying within the timeout period [540000]. The Deployable state is thus unknown.

> Task :cargoRunLocal FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':cargoRunLocal'.
> org.codehaus.cargo.container.ContainerException: Deployable [http://localhost:8080/cargocpc/index.html] failed to finish deploying within the timeout period [540000]. The Deployable state is thus unknown.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 9m 10s
29 actionable tasks: 7 executed, 22 up-to-date

Please include UAA logs if available.

Hi , these errors were reported a view times already, see https://github.com/cloudfoundry/uaa/search?q=gradlew+run+timeout&type=issues
Please check, if your machine has enough power. Another solution would be to increase start timeout, see PR which provides such an option
https://github.com/cloudfoundry/uaa/pull/1594


Document AnimationNodeTimeSeek with clarified usage and code example

Document AnimationNodeTimeSeek with clarified usage and code example.
Issue was that users often ask "how can I play my animation from a certain position inside the AnimationTree?" but have no idea from the current documentation what the SeekNode can do and how it is used. The majority didn't even know what a seek command in this context is to begin with.

What did I do to offend the clang-god? Is it the empty line between the different codeblocks? How can I properly add a clang-happy empty line then?

Thanks!

Cherry-picked for 3.3.


VixDiskLib: Use a well-known IO to pass the URI

As we can be more confident it's the right thing now, we can lose the 'URI:' prefix.
In passing, also fix retries: previously, we would've attempted to re-read the URI upon retry, when it had already been consumed. That clearly wasn't going to work.

@jerryk55 this is entirely untested, mostly because I wouldn't know where to start. Feel free to use/ignore as you see fit...
If it were just some existing isolated code I may not have bothered, but as it sounds like we're about to replicate the pattern, it seemed worth a mention. (Also, I really don't see how that retry could possibly work.)

@matthewd I have implemented most of this logic by way of PR #4411 so I believe this one can be closed.


Created Searchplugin.java with default implementation

Signed-off-by: Varun Hegde<EMAIL_ADDRESS>Description
Add a new interface SearchExtension with default implementations for extension points matching SearchPlugin.java.
Issues Resolved
Copy pasted the default implementation of SearchPlugin.java ro SearchExtension.java
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Tell me If I have to add anything
Thanks and Regards
Varun

Yes I applied the spotlessApply and RequireJavaDoc and Documentation linkage was giving build errors. I fixed it as of now temporarily. But  still it throws some warnings as no @return for methods.
Tell If anything else has to be done to close this, I will do one more of this #323 ( I already copied and just there to commit ) and next I will proceed to #319 If its okay.
Thanks and Regards
Varun

Yes I applied the spotlessApply and RequireJavaDoc and Documentation linkage was giving build errors. I fixed it as of now temporarily. But still it throws some warnings as no @return for methods.

Yeah, you need @param and @return for constructors with parameters.   I think it's not required on the getters with same name as the field they are getting.
The one @link you removed goes to a not-yet-written Extension interface extension point, so for now to preserve the text could you just change what was @link to @code and we can put it back to link later?

If there is anything left out, I will do it tomorrow.
Thanks and Regards
Varun

Thanks @joshpalis. I Hope to learn and contribute more in the future


vagrant up fail on CentOS 7.3 & libvirt provider.

Host OS:

[root@localhost vagrant]# cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)

Vagrant version:

[root@localhost vagrant]# vagrant --version
Vagrant 1.9.1

Run the vagrant, got the following error msg and can't move on to finish the process.

==> master1: Setting hostname...
==> node1: Setting hostname...
==> admin1: Setting hostname...
==> master1: Automatic installation for Landrush IP not enabled
==> master1: Configuring and enabling network interfaces...
==> node2: Setting hostname...
==> node1: Automatic installation for Landrush IP not enabled
==> node1: Configuring and enabling network interfaces...
==> admin1: Automatic installation for Landrush IP not enabled
==> admin1: Configuring and enabling network interfaces...
==> node2: Automatic installation for Landrush IP not enabled
==> node2: Configuring and enabling network interfaces...
[landrush] Using eth0 (<IP_ADDRESS>)
==> master1: [landrush] adding machine entry: master1.example.com => <IP_ADDRESS>
==> master1: [landrush] starting dns server
admin1: [landrush] 'ruby /root/.vagrant.d/gems/2.2.5/gems/landrush-1.2.0/lib/landrush/start_server.rb 10053 /root/.vagrant.d/data/landrush /root/.vagrant.d/gems/gems'
[landrush] Using eth0 (<IP_ADDRESS>)
[landrush] Using eth0 (<IP_ADDRESS>)
==> node1: [landrush] adding machine entry: node1.example.com => <IP_ADDRESS>
==> node1: [landrush] starting dns server
admin1: [landrush] 'ruby /root/.vagrant.d/gems/2.2.5/gems/landrush-1.2.0/lib/landrush/start_server.rb 10053 /root/.vagrant.d/data/landrush /root/.vagrant.d/gems/gems'
[landrush] Using eth0 (<IP_ADDRESS>)
==> admin1: [landrush] adding machine entry: admin1.example.com => <IP_ADDRESS>
==> admin1: [landrush] starting dns server
admin1: [landrush] 'ruby /root/.vagrant.d/gems/2.2.5/gems/landrush-1.2.0/lib/landrush/start_server.rb 10053 /root/.vagrant.d/data/landrush /root/.vagrant.d/gems/gems'
[landrush] Using eth0 (<IP_ADDRESS>)
==> node2: [landrush] adding machine entry: node2.example.com => <IP_ADDRESS>
[landrush] Using eth0 (<IP_ADDRESS>)
[landrush] Need to configure dnsmasq on host.
[landrush] Momentarily using sudo to put the host config in place...
==> master1: Updating /etc/hosts file on active guest machines...
[landrush] Using eth0 (<IP_ADDRESS>)
[landrush] Using eth0 (<IP_ADDRESS>)
[landrush] Host dnsmasq config looks good.
==> node2: Updating /etc/hosts file on active guest machines...
[landrush] Host dnsmasq config looks good.
==> master1: Updating /etc/hosts file on active guest machines...
==> admin1: Installing SSHFS client...
==> node1: Updating /etc/hosts file on host machine (password may be required)...
==> admin1: An error occurred. The error will be shown after all tasks complete.
==> node2: Updating /etc/hosts file on host machine (password may be required)...
==> node2: Check for insecure vagrant key: OK (not present)
==> node2: Running provisioner: shell...
node2: Running: /tmp/vagrant-shell20170228-16586-1j7yblx.sh
==> node2: Check eth1 ip config for netbase: 192.168.50
==> node2: eth1 missing ip; restaring interface

Any advice to solve this problem? Thans!

/close


Kube-vip hang on releasing IP

Describe the bug
The kube-vip is used in Harvester project, we encountered such issue: Kube-vip hang on releasing IP
https://github.com/harvester/harvester/issues/4972
To Reproduce

Create & Release the VM type LB several times; internally, it creat/remove k8s LB type service
Encounter below phenomenon, the kube-vip stucks in last line of log

time="2024-01-08T15:33:17Z" level=error msg="request failed, error: unable to receive an offer: got an error while the discovery request: no matching response packet received (waiting 1m0s)"
time="2024-01-08T15:33:39Z" level=info msg="Using existing macvlan interface for DHCP [vip-5f95e2e7]"
time="2024-01-08T15:33:39Z" level=info msg="DHCP VIP [<IP_ADDRESS>] for [itpe-platform/test-2175] "
time="2024-01-08T15:33:39Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [itpe-platform/test-2175]"
time="2024-01-08T15:33:39Z" level=info msg="[service] synchronised in 147ms"
time="2024-01-08T15:42:55Z" level=info msg="Creating new macvlan interface for DHCP [vip-92c51f6a]"
time="2024-01-08T15:42:55Z" level=info msg="Generated mac: 00:00:6C:21:1f:22"
time="2024-01-08T15:42:55Z" level=info msg="New interface [vip-92c51f6a] mac is 00:00:6c:21:1f:22"
time="2024-01-08T15:42:56Z" level=info msg="DHCP VIP [<IP_ADDRESS>] for [itpe-platform/test-vlan2169] "
time="2024-01-08T15:42:56Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [itpe-platform/test-vlan2169]"
time="2024-01-08T15:42:56Z" level=info msg="[service] synchronised in 1146ms"
time="2024-01-08T16:01:46Z" level=info msg="[LOADBALANCER] Stopping load balancers"
time="2024-01-08T16:01:46Z" level=info msg="[VIP] Releasing the Virtual IP [<IP_ADDRESS>]"

The kube-vip also has no further log when adding/removing services.
note:
the normal releasing has such log:
time="2024-01-11T15:13:05Z" level=info msg="[LOADBALANCER] Stopping load balancers"
time="2024-01-11T15:13:05Z" level=info msg="[VIP] Releasing the Virtual IP [<IP_ADDRESS>]"
time="2024-01-11T15:13:05Z" level=info msg="release, lease: &{Offer:DHCPv4(xid=0xa8e6c374 hwaddr=00:00:6c:3a:36:ba msg_type=OFFER, your_ip=<IP_ADDRESS>, server_ip=<IP_ADDRESS>) ACK:DHCPv4(xid=0xa8e6c374 hwaddr=00:00:6c:3a:36:ba msg_type=ACK, your_ip=<IP_ADDRESS>, server_ip=<IP_ADDRESS>) CreationTime:2024-01-11 15:10:34.668103182 +0000 UTC m=+15916.446203689}"
time="2024-01-11T15:13:07Z" level=info msg="Removed [58b44bf9-5546-4224-beee-e5f3fc35b2db] from manager, [1] advertised services remain"
time="2024-01-11T15:13:07Z" level=info msg="service [default/lb-vm1] has been deleted"

After killing the kube-vip pod, the replaced new pod continue works. (skip the timestamp in following log which shows the normal working process of kube-vip)
ime="2023-10-26T20:59:21Z" level=info msg="Starting kube-vip.io [v0.6.0]"
time="2023-10-26T20:59:21Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[false], Services:[true]"
time="2023-10-26T20:59:21Z" level=info msg="No interface is specified for VIP in config, auto-detecting default Interface"
time="2023-10-26T20:59:21Z" level=info msg="prometheus HTTP server started"
time="2023-10-26T20:59:21Z" level=info msg="kube-vip will bind to interface [mgmt-br.2175]"
time="2023-10-26T20:59:21Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2023-10-26T20:59:21Z" level=info msg="beginning services leadership, namespace [harvester-system], lock name [plndr-svcs-lock], id [tekton22d]"
I1026 20:59:21.832553       1 leaderelection.go:248] attempting to acquire leader lease harvester-system/plndr-svcs-lock...
time="2023-10-26T20:59:21Z" level=info msg="new leader elected: tekton20d"
I0111 14:33:00.291975       1 leaderelection.go:258] successfully acquired lease harvester-system/plndr-svcs-lock
time="2024-01-11T14:33:00Z" level=info msg="starting services watcher for all namespaces"
time="2024-01-11T14:33:00Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [kube-system/ingress-expose]"
time="2024-01-11T14:33:00Z" level=info msg="[service] synchronised in 21ms"
time="2024-01-11T14:33:00Z" level=info msg="Creating new macvlan interface for DHCP [vip-1d84084b]"
time="2024-01-11T14:33:00Z" level=info msg="Generated mac: 00:00:6C:25:4c:ac"
time="2024-01-11T14:33:00Z" level=info msg="New interface [vip-1d84084b] mac is 00:00:6c:25:4c:ac"
time="2024-01-11T14:33:01Z" level=info msg="DHCP VIP [<IP_ADDRESS>] for [itpe-platform/test-valn2169] "
time="2024-01-11T14:33:01Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [itpe-platform/test-valn2169]"
time="2024-01-11T14:33:01Z" level=info msg="[service] synchronised in 1157ms"
time="2024-01-11T14:33:01Z" level=info msg="Creating new macvlan interface for DHCP [vip-5f95e2e7]"
time="2024-01-11T14:33:01Z" level=info msg="New interface [vip-5f95e2e7] mac is 00:00:6c:87:0a:69"
time="2024-01-11T14:33:01Z" level=info msg="DHCP VIP [<IP_ADDRESS>] for [itpe-platform/test-2175] "
time="2024-01-11T14:33:01Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [itpe-platform/test-2175]"
time="2024-01-11T14:33:01Z" level=warning msg="could not add iptables rules to limit traffic ports: could not add common iptables rules: could not add iptables rule to drop the traffic to VIP <IP_ADDRESS>: running [/sbin/iptables-legacy -t filter -I INPUT 2 -d <IP_ADDRESS> -m comment --comment itpe-platform/test-2175 kube-vip load balancer IP -j DROP --wait]: exit status 4: iptables: Resource temporarily unavailable.\n"
time="2024-01-11T14:33:01Z" level=info msg="[service] synchronised in 185ms"
time="2024-01-11T14:33:01Z" level=info msg="Creating new macvlan interface for DHCP [vip-00a62cb5]"
time="2024-01-11T14:33:01Z" level=info msg="Generated mac: 00:00:6C:2d:b7:d7"
time="2024-01-11T14:33:01Z" level=info msg="New interface [vip-00a62cb5] mac is 00:00:6c:2d:b7:d7"
time="2024-01-11T14:33:02Z" level=info msg="DHCP VIP [<IP_ADDRESS>] for [itpe-platform/test-vlan2179] "
time="2024-01-11T14:33:02Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [itpe-platform/test-vlan2179]"
time="2024-01-11T14:33:02Z" level=info msg="[service] synchronised in 1150ms"
time="2024-01-11T14:33:02Z" level=info msg="Creating new macvlan interface for DHCP [vip-53a2d28f]"
time="2024-01-11T14:33:02Z" level=info msg="Generated mac: 00:00:6C:eb:ed:07"
time="2024-01-11T14:33:02Z" level=info msg="New interface [vip-53a2d28f] mac is 00:00:6c:eb:ed:07"
time="2024-01-11T14:33:03Z" level=info msg="DHCP VIP [<IP_ADDRESS>] for [itpe-platform/test-vlan2175-23] "
time="2024-01-11T14:33:03Z" level=info msg="[service] adding VIP [<IP_ADDRESS>] for [itpe-platform/test-vlan2175-23]"
time="2024-01-11T14:33:04Z" level=info msg="[service] synchronised in 1205ms"

Expected behavior
A clear and concise description of what you expected to happen.
The kube-vip pod should not hang.
Did anyone happen to encounter such issue?
At the moment, we assumed  one possible place to hang: netlink.AddrDel.
https://github.com/harvester/harvester/issues/4972#issuecomment-2220078987
The netlink has an API SetSocketTimeout but seems not called in kube-vip.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):

OS/Distro: [e.g. Ubuntu 1804] SLE Linux 5.14.21-150400.24.116
Kubernetes Version: [e.g. v.1.18] v1.27.10
Kube-vip Version: [e.g. 0.2.3] ghcr.io/kube-vip/kube-vip-iptables:v0.6.0

Kube-vip.yaml:
If Possible add in your kube-vip manifest (please remove anything that is confidential)
Additional context
Add any other context about the problem here.

i don't think netlink.AddrDel is the key, anyway the netlink does have a default timeout.
https://github.com/vishvananda/netlink/blob/main/nl/nl_linux.go#L41
var SocketTimeoutTv = unix.Timeval{Sec: 60, Usec: 0}


NSIMD generates too much assembly

I recently found the need for a cross platform simd library and nsimd seems to be the perfect fit. However after testing a bit on Godbolt I discovered that even with optimizations enabled, the amount of assembly code generated is very large, exceeding even scalar operations in many cases. For example, implementing a simple add function results in over 15 lines of assembly:

Compared to this, using SSE intrinsics directly results in the expected much more compact assembly:

I have made sure to enable optimizations and the AVX2 GCC flag. I thought that maybe the integer operations were emulated as mentioned in the documentation so I also tried the same minimal example with floats, to no avail:

I wouldn't mind having to deal with some overhead but this too much to be normal. I also tested nsimd::if_else and it also suffers from this:

Is this intentional? Is this a bug? Is there any way to improve the quality of the generated code?

I believe you are misunderstanding the "length" argument of a pack. You should probably set it to 1 (not 4) to create a pack that has the natural size for your architecture. Choosing a larger value allows setting up SIMD vectors with a length that is a multiple of the natural hardware vector length. In other words, your nsimd examples are working on 16 elements, not just 4, and the code is thus much longer than expected. You are also seeing a lot of code related to loading and storing function arguments from and to the stack; these instructions will go away if the function is inlined, or is used inside a larger function.
I usually just use pack<double>, without specifying the length. The SIMD length is chosen automatically by nsimd.
As a test, you can evaluate sizeof(pack<float, 4>).

I believe you are misunderstanding the "length" argument of a pack. You should probably set it to 1 (not 4) to create a pack that has the natural size for your architecture. Choosing a larger value allows setting up SIMD vectors with a length that is a multiple of the natural hardware vector length. In other words, your nsimd examples are working on 16 elements, not just 4, and the code is thus much longer than expected. You are also seeing a lot of code related to loading and storing function arguments from and to the stack; these instructions will go away if the function is inlined, or is used inside a larger function.
I usually just use pack<double>, without specifying the length. The SIMD length is chosen automatically by nsimd.
As a test, you can evaluate sizeof(pack<float, 4>).

Thanks for pointing it out. The code looks much cleaner now. But if size of the vector depends on the architecture provided, is there any cross platform way of defining the size? In addition, though, I have been facing numerous build errors on compilers that aren't GCC when using the nsimd::cvt function. For example:

GCC

Clang

MSVC

I am not aware of a way to define the SIMD vector size. The suggested approach is to let nsimd choose the optimal vector length, and then to inquire the actual vector length at compile time. This allows efficient loop vectorization when traversing arrays in loops.
I am not encountering any such errors. I uses nsimd regularly, on various systems. I usually build with GCC 11.2 (as you do), and I specify -std=c++17 as well.
Most SIMD hardware works on a fixed number of bits. Thus converting from 4xi32 to 4xi16 is not a natural operation. The natural operation would generate an 8xi16 vector, presumably from two 4xi32 inputs. Maybe you need to use downcvt instead of cvt?

What I want to do is take each element of the int32 vector and cast it down to int16 without reducing the vector size (in elements). With intrinsics this can be done with _mm_cvtps_pi16 so I assumed that nsimd::cvt would work based on the similar name.

Closing is now as resolved


Fix build issue when nuget directory contains spaces

If $(MSBuildThisFileDirectory) contains spaces the command
dotnet exec $(MSBuildThisFileDirectory)../tools/T4.Build.dll

may fail during builds of a project having referenced T4.Build. This PR fixes the issue by enclosing the command in quotation marks.

Thanks a lot for your PR @hokb! 👍

v0.2.4 has been released with this fix


Fixes to Qnet

Previously, the Qnet layers were skipped over and not trained with the gradients, these changes resolve the issues. This allows for faster convergence and better pairing of a single number to each categorical value.  See results demonstration below:

Thanks a lot for your contribution! I will merge and update the README soon!

Yeah, no problem. It's a nice InfoGAN demo, so I wanted to make sure it works properly for anyone else who comes across it.


[zh-cn] sync manage-resources-containers.md secret.md

content/zh-cn/docs/concepts/configuration/manage-resources-containers.md
content/zh-cn/docs/concepts/configuration/secret.md

/lgtm


add explanation of file download link structure to docs

We should add Ernest's explanation https://github.com/pypa/warehouse/issues/3738#issuecomment-382083526 to https://warehouse.readthedocs.io/api-reference/integration-guide/#integration-guide . The file to update is docs/api-reference/integration-guide.rst.

Good First Issue: This issue is good for first time contributors. If you've already contributed to Warehouse, please work on another issue without this label instead. If there is not a corresponding pull request for this issue, it is up for grabs. For directions for getting set up, see our Getting Started Guide. If you are working on this issue and have questions, please feel free to ask them here, #pypa-dev on Freenode, or the pypa-dev mailing list.

I'll work on this


Package version conflict

Thank you very much for your outstanding contributions in this field. I have some questions about installing related packages.
tensorstore 0.1.54 requires ml-dtypes>=0.3.1.
But tensorflow-intel 2.15.0 requires ml-dtypes~=0.2.0.
And flashbax 0.1.0 requires tensorstore>=0.1.51
Could you please provide more details about the version of the related package? Thank you very much

Thanks so much for opening an issue. It looks like you solved the problem, but I thought I would leave a comment anyway.
We are aware of this dependency issue. But as long as you only install Flashbax after installing OG-MARL and its other dependencies, then everything will work.
If you have any other issues using OG-MARL, please do not hesitate to open another issue. Or if you just have a question, we would be happy to answer.
Good luck with your research!

非常感谢您提出问题。看来你解决了问题，但我想我还是要发表评论。
我们意识到这个依赖性问题。但只要您在安装 OG-MARL 及其其他依赖项之后才安装 Flashbax，那么一切都会正常。
如果您在使用 OG-MARL 时遇到任何其他问题，请随时打开另一个问题。或者，如果您只是有一个问题，我们很乐意回答。
祝您研究顺利！

Thank you very much for your reply. I reproduced qmix+cql in 3m_good scenario and found always starts to rapidly decline in performance after fewer than 1000 training steps。At present, the parameters in the code are used. Does this part still need additional parameters? It would be very helpful if you could provide me with more details. Thanks again for your patience and reply!
num_ood_actions: int = 10,
cql_weight: float = 5.0,
linear_layer_dim: int = 64,
recurrent_layer_dim: int = 64,
mixer_embed_dim: int = 32,
mixer_hyper_dim: int = 64,
discount: float = 0.99,
target_update_period: int = 200,
learning_rate: float = 3e-4,

Or could it be the result of the following warnings?
WARNING:tensorflow:From D:\ANACONDA\anaconda\envs\ogmarl\lib\site-packages\keras\src\losses.py:2976:  The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use  tf.compat.v1.losses.sparse_softmax_cross_entropy instead.
D:\ANACONDA\anaconda\envs\ogmarl\lib\site-packages\flashbax\buffers\trajectory_buffer.py:473: UserWarning:  Setting max_size dynamically sets the max_length_time_axis to be max_size//add_batch_size = 50000.This allows one  to control exactly how many timesteps are stored in the buffer.Note that this overrides the max_length_time_axis  argument.
warnings.warn(
D:\ANACONDA\anaconda\envs\ogmarl\lib\site-packages\flashbax\buffers\trajectory_buffer.py:498: UserWarning:  sample_sequence_length greater than min_length_time_axis,  therefore overriding min_length_time_axisto be set to sample_sequence_length,  as we need at least sample_sequence_length timesteps added to the buffer before we can sample.
warnings.warn(

Hi there, here are my latest baseline results: https://api.wandb.ai/links/off-the-grid-marl-team/f2ozf5vy
I do find that the performance does decay after extended training. But 1000 training steps does seem too soon. In my experiments performance decayed after 30k training steps. Could you double check that you are not running Qmix without CQL by mistake?

Oh wait, I see your x-axis there is "Step", which is logging steps not training steps. You should change the x-axis on the evaluator/episode_return to Trainer Steps (eval). Then you will see the training steps on the x-axis.
But yes, its expected that after long training the performance begins to degrade. This is because over time the algorithm trains on more and more out-of-distribution actions. This results in compounding extrapolation and eventually performance collapse. Other algorithms are more robust to this and do no suffer from performance degradation so soon (e.g. MAICQ and IDRQN+CQL)

Thank you very much for your reply!


Final pass updates samples 1-5 dotnetcore

updated samples 1, 2, 3, 4, 5
updated all bot files

going to open a new one, it's not picking up the new f5 pages for some reason

Jason, this is awesome progress! can you break these apart into separate PR's?
For sure we should look to lock down the echo bot sample since that has down stream dependencies with ABS, Vsix etc.
@daveta as FYI.

I can. I'll do it in the morning

I lied I'll do it now...

Jason, tomorrow is fine. Please get some rest.


allOf property is ignored when model marked as "type: object" in spec

Two valid specifications (verified by swagger.io online validator):
openapi: 3.0.0
info:
  version: 1.0.0
  title: Swagger Petstore
tags:
  - name: pet
    description: Everything about your Pets
paths:
  /pet:
    post:
      tags:
        - pet
      summary: Add a new pet to the store
      description: ""
      operationId: addPet
      responses:
        "405":
          description: Invalid input
components:
  schemas:
    Dog:
      allOf:
        - $ref: "#/components/schemas/Animal"
        - type: object
          properties:
            breed:
              type: string
    Animal:
      type: object
      discriminator:
        propertyName: className
      required:
        - className
      properties:
        className:
          type: string
        color:
          type: string
          default: red

generates public class Dog: Animal, but same spec with type: object added to Dog
openapi: 3.0.0
info:
  version: 1.0.0
  title: Swagger Petstore
tags:
  - name: pet
    description: Everything about your Pets
paths:
  /pet:
    post:
      tags:
        - pet
      summary: Add a new pet to the store
      description: ""
      operationId: addPet
      responses:
        "405":
          description: Invalid input
components:
  schemas:
    Dog:
      type: object
      allOf:
        - $ref: "#/components/schemas/Animal"
        - type: object
          properties:
            breed:
              type: string
    Animal:
      type: object
      discriminator:
        propertyName: className
      required:
        - className
      properties:
        className:
          type: string
        color:
          type: string
          default: red

generates public class Dog: APIModel without properties at all
version 4.3.1

The specific request this issue is about was probably addressed as of v4.4.0 and v4.6.0 (after some fixes). However there's still something broken, as reported here #295, not sure if that would affect you.

I've tested above samples with v.4.6.0.
Dog from first spec:
public class Dog: Animal {

    public var breed: String?

    public init(className: String, color: String? = nil, breed: String? = nil) {
        self.breed = breed
        super.init(className: className, color: color)
    }

    public required init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: StringCodingKey.self)

        breed = try container.decodeIfPresent("breed")
        try super.init(from: decoder)
    }

    public override func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: StringCodingKey.self)

        try container.encodeIfPresent(breed, forKey: "breed")
        try super.encode(to: encoder)
    }

    override public func isEqual(to object: Any?) -> Bool {
      guard let object = object as? Dog else { return false }
      guard self.breed == object.breed else { return false }
      return super.isEqual(to: object)
    }
}

and Dog from second spec:
public class Dog: APIModel {

    public init() {
    }

    public required init(from decoder: Decoder) throws {
    }

    public func encode(to encoder: Encoder) throws {
    }

    public func isEqual(to object: Any?) -> Bool {
      guard object is Dog else { return false }
      return true
    }

    public static func == (lhs: Dog, rhs: Dog) -> Bool {
        return lhs.isEqual(to: rhs)
    }
}


Remove direct react-navigation-tabs dependency.

This is almost NFC, considering how thin react-navigation's wrapper around react-navigation-tabs's createBottomTabNavigator is.
```
get createBottomTabNavigator() {
  return require('react-navigation-tabs').createBottomTabNavigator;
},
```

Oh, oops. There's one more usage of react-navigation-tabs — should be able to take care of that the same way; I'll update the PR and re-post here.

I like the reasoning for removing the direct dependency on react-navigation-tabs.
We should also remove the corresponding libdef in flow-typed/npm/.
For the fixmes, compare the fixmes that were removed in 0d2066f41 when we switched to using react-navigation-tabs. They were narrower, which meant we got type-checking on the rest of what we were passing to react-navigation there. That seems worth preserving -- there's a bunch of other options involved that it's good to check on.
In the analysis of what's causing the need for the fixmes:
One summary of the error on, e.g., `HomeTab`, is, "Hey, you should
write down the fact that HomeTab gets special treatment because it's
passed into `createBottomTabNavigator`. It gets certain props, for
example, like the `navigation` prop."

I don't think this is quite it, because if you look at the error it doesn't mention a name navigation. It seems to be entirely about the navigationOptions property. Which in particular is a property on the component type, i.e. a property on the value HomeTab itself, i.e. if it's a class then a static property on the class.
It's actually kind of puzzling to me what Flow is doing there, because the libdef says the property is optional, so it should be fine to simply not have it. (As indeed we don't.) I suspect the story involves something like React.ComponentType<…> being handled specially and not as "an object type" at all.
I think I can fix the issue that our typed connect doesn't pass through the information about static properties on the component type. The main annoying part will be testing that the fix doesn't cause Flow to lose type-checking precision somewhere else.
But also, on testing one possible fix for that, I find that the react-navigation libdef doesn't actually allow a useful navigationOptions anyway! I tried putting
 class HomeTab extends PureComponent<Props> {
+  static navigationOptions = { tabBarLabel: 'Home' };
+

and that produces an error saying tabBarLabel isn't allowed. And in fact it's the same error if I cut out connect entirely. So... we wouldn't get useful type-checking if we actually used this feature of the react-navigation API, which rather lessens the motivation to fix the connect type so that we could try.

Oh no! I'm about 80% sure I posted a lengthy reply to this, referring to my earlier thoughts on this extra property on the component, here. Hmm.

Oh no! I'm about 80% sure I posted a lengthy reply to this, referring to my earlier thoughts on this extra property on the component, here. Hmm, maybe I didn't follow through, or it got lost somehow.

Ah! I think @gnprice and I discussed it on the phone; that must have been it.

Thanks for the review! Just pushed my latest changes.

We should also remove the corresponding libdef in flow-typed/npm/.

Ah, just did this, and rebased.

I'm closing this, as we'll need to depend on it directly in react-navigation v4: https://reactnavigation.org/docs/4.x/upgrading-from-3.x/#install-the-new-packages (see the first subheader, "Install the new packages").


[BUG] elasticdump v. 6.15.4 only supports Node.js 10+

ElasticDump version: 6.15.4
Elasticsearch version: irrelevant
Node.js version: 8.15.0
Full command you are having issue with: elasticdump --output http://localhost:9200/someIndex --input=fileName
A data-set you can share publicly which reproduces the problem: irrelevant

Describe the bug
When running the simple command elasticdump --output=... --input=... on Node.js v8, I get the following error:
TypeError [ERR_INVALID_ARG_TYPE]: The "original" argument must be of type function
    at Object.promisify (internal/util.js:214:11)
    at Object.<anonymous> (/usr/local/lib/node_modules/elasticdump/lib/transports/file.js:9:28)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)

Additional context
The problem is with this change:
https://github.com/taskrabbit/elasticsearch-dump/commit/ec1e96b65492f57bda4a6de927afc93014fd8918#diff-f60b04ac83f174b93b06625be32a86f5R9
It adds use of Node.js's built in stream.pipeline function that was only added to Node v10, as can be seen here:
https://nodejs.org/api/stream.html#stream_stream_pipeline_streams_callback
I'm not sure if this is a bug since I couldn't find the minimal version of Node.js that elasticdump targets.
If elasticdump targets Node.js 10+ then I would add that to its readme.md and would bump the major version number to 7 since this is a breaking change.
If this is an unintentional bug then I would recommend running elasticdump's CI tests on the minimal targeted Node.js version to catch bugs like this.

That's an error my part.

Thanks for the swift response and fix!

It looks like the support of nodejs 8 was removed now with c9c66a64e316c4f7cf5c5e5b5b08dc5c2a23ecdc
so starting with elasticdump 6.32.0 nodejs 10 is required.

Just crashed into this. Provisioned a VM with Ubuntu 18.04, nodejs and npm are installed via apt (version 8.10.0) and elasticdump is installed via npm. Results in this error.
So you have to: Either install a fixed version of elasticdump or a fixed version of nodejs.
In my case:
npm i -g<EMAIL_ADDRESS>

I dropped support for node version lower than 10. Use 6.31.5 or lowered

Is it possible to unpublish/block versions greater than 6.31.5 for node v8?
currently, when using node v8, simply running npm i elasticdump results in a broken installation.

That's a function of the npm cli not elasticdump. They don't enforce engine restrictions, so there isn't much I can do.

i.e. I did add a note to the README.md

@ferronrsmith my small 2 cents on this issue are that changing minimal node version should change the package's major version to 7

I could have done this, but I didn't want to at the time.

Please, can you at least add a warning in elasticdump if node version is too old so we don't search like crazy for this error?

yea sure! I'll do that now

Ok done, from v 6.67.0 onward. If the node version doesn't match the engine restrictions specified in the package.json elasticsearch-dump will quit with an error and inform the user the node version it too low.


Feature Request: Add an optional command-line argument for a prefix URL

Would like to add a command-line argument that allows TensorBoard to run at a different URL location than the root domain.  So for example:

command-line: --base_url runhere
URL location: http://<address>:6006/runhere/

Motivation for request: There are locations where only minimal ports are open and it would be great to use nginx (or similar) to route TensorBoard through port 80.

@dandelionmane, could you take a look at this feature request. Thanks!

I've migrated this issue to our new repo over at tensorflow/tensorboard.


Possible alternatives to callbacks? Promise / Observables

I saw this reference in the v1.0 refactor by @moshen. Any thoughts on this?

Current implementation for me with callbacks
return new Bluebird(function (resolve, reject) {
        googleMapsAPI.placeSearch(options, function (error, response) {
            winston.debug("placeSearch result:", response);

            if (error) {
                return reject("Unable to query google maps api placeSearch", error);
            }

            return resolve(response);
        });
    });

With straight promises
return googleMapsAPI.placeSearch(options)
    .then(function (response) {
        winston.debug("placeSearch result:", response);
    })
    .catch(function (e) {
        winston.error("Unable to query google maps api placeSearch");
    });

Since all the calls now follow a similar pattern (CPS, callback last), you can use Bluebird to promisifyAll on the object.  So your example becomes:
var Promise = require('bluebird'),
  GoogleMaps = require('googlemaps'),
  gm = Promise.promisifyAll(new GoogleMaps());

return gm.placeSearchAsync(options)
.then(function(response) {
    winston.debug("placeSearch result:", response);
})
.catch(function(e) {
    winston.error("Unable to query google maps api placeSearch:", e);
});

Since it's so easy to wrap yourself, I'm not sure if it's worth putting into the library.  @fabriziomoscon ?

The current v1 implementation can be used by node 0.8 and 0.10 users as it doesn't require Promises. Using a third party Promises library inside the this code will add an extra dependency and maintenance costs. In my opinion the simpler the dependencies graph is the better for any library. Unless the use of Promises lowers the library maintenance costs or make it possible to achieve more functionalities I would prefer to keep the use of callback.
In the future if the version of node with native Promises support will become widespread adopted we could consider a v2 that drops the support for older node versions and rewrite the code to be Promises oriented.

@fabriziomoscon fair enough. Thanks for clarifying.


feat: Add progress_callback to snapshot method

Updated the snapshot method to include a progress_callback for better tracking of operation. This allows for real-time progress tracking during snapshot operations, improving logging and debuggability for delta backup.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 72.72%. Comparing base (db19245) to head (e5c7164).
Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #173      +/-   ##
==========================================
+ Coverage   72.64%   72.72%   +0.07%
==========================================
  Files          36       36
  Lines        4058     4073      +15
==========================================
+ Hits         2948     2962      +14
- Misses       1110     1111       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

This looks reasonable to me.
It would be nicer to have a debounced call to the progress callback inside each loop, but not essential for this iteration; it can always be added as a subsequent PR if need be.


Node security group rule "ingress_nodes_ephemeral" blocks inter-node communications

Description
I have a couple deployments on an EKS cluster exposed via ingress-nginx and some of them intermittently timeout when scheduled on different nodes.
Versions

Module version: 19.10.0

Terraform version: Terraform v1.3.4

Provider version(s): hashicorp/aws 4.53.0

Reproduction Code [Required]
module "eks" {
  source  = "registry.terraform.io/terraform-aws-modules/eks/aws"
  version = "19.10.0"

  cluster_name                           = "${local.global_prefix}-cluster"
  cluster_version                        = var.EKS_CLUSTER_VERSION
  cluster_endpoint_public_access         = true
  cluster_ip_family                      = "ipv4"
  create_cni_ipv6_iam_policy             = false
  cloudwatch_log_group_retention_in_days = 7
  cluster_enabled_log_types              = ["audit"]
  cluster_encryption_config              = {}
  create_kms_key                         = false
  aws_auth_users                         = var.EKS_MAPPED_IAM_USERS
  aws_auth_roles = concat(var.EKS_MAPPED_IAM_ROLES, [
    {
      rolearn  = aws_iam_role.eks_access_role.arn
      username = "aws-administrator"
      groups   = ["system:masters"]
    }
  ])

  cluster_addons = {
    coredns = {
      most_recent = true
    }
    kube-proxy = {
      most_recent = true
    }
    aws-ebs-csi-driver = {
      most_recent              = true
      service_account_role_arn = module.ebs_csi_irsa.iam_role_arn
    }
  }

  vpc_id                   = module.vpc.vpc_id
  subnet_ids               = module.vpc.private_subnets
  control_plane_subnet_ids = []
  node_security_group_additional_rules = {
    node_to_node = {
      from_port = 0
      to_port   = 0
      protocol  = -1
      self      = true
      type      = "ingress"
    }
  }

  manage_aws_auth_configmap = true

  eks_managed_node_group_defaults = {
    ami_type                   = "AL2_x86_64"
    instance_types             = ["m6i.large", "m5.large", "m5n.large", "m5zn.large"]
    iam_role_attach_cni_policy = true
  }

  eks_managed_node_groups = {
    spot = {
      name                       = "spot-eks-mng"
      use_name_prefix            = true
      subnet_ids                 = module.vpc.private_subnets
      min_size                   = 1
      max_size                   = 7
      desired_size               = 1
      ami_id                     = data.aws_ami.eks_default.image_id
      enable_bootstrap_user_data = true
      bootstrap_extra_args       = "--container-runtime containerd --kubelet-extra-args '--max-pods=50'"

      pre_bootstrap_user_data = <<-EOT
        export CONTAINER_RUNTIME="containerd"
        export USE_MAX_PODS=true
      EOT

      capacity_type        = "SPOT"
      force_update_version = true
      instance_types       = ["m5.large", "m6i.large"]
      labels = {
        type              = "general"
        instance_schedule = "spot"
      }

      update_config = {
        max_unavailable_percentage = 33
      }

      description = "EKS managed node group launch template"

      ebs_optimized           = true
      disable_api_termination = false
      enable_monitoring       = true

      block_device_mappings = {
        xvda = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size           = 50
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 150
            encrypted             = false
            delete_on_termination = true
          }
        }
      }

      metadata_options = {
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 2
        instance_metadata_tags      = "disabled"
      }

      create_iam_role          = true
      iam_role_name            = "eks-managed-node-group-spot"
      iam_role_use_name_prefix = false
      iam_role_description     = "EKS managed node group spot-instance role"
      iam_role_tags = {
        Purpose = "Protector of the kubelet"
      }
      iam_role_additional_policies = {
        AmazonEC2ContainerRegistryReadOnly = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
        additional                         = aws_iam_policy.node_additional.arn
      }

      tags = {
        "k8s.io/cluster-autoscaler/enabled"                        = "true"
        "k8s.io/cluster-autoscaler/${local.global_prefix}-cluster" = "owned"
        "misc.instance-schedule"                                   = "spot"
      }
    }

    spot_ml = {
      name                       = "spot-ml-eks-mng"
      use_name_prefix            = true
      subnet_ids                 = module.vpc.private_subnets
      min_size                   = 1
      max_size                   = 4
      desired_size               = 1
      ami_id                     = data.aws_ami.eks_default.image_id
      enable_bootstrap_user_data = true
      bootstrap_extra_args       = "--container-runtime containerd --kubelet-extra-args '--max-pods=50'"

      pre_bootstrap_user_data = <<-EOT
        export CONTAINER_RUNTIME="containerd"
        export USE_MAX_PODS=true
      EOT

      capacity_type        = "SPOT"
      force_update_version = true
      instance_types       = ["r5.large"]
      labels = {
        type              = "ai-ml"
        instance_schedule = "spot"
      }

      update_config = {
        max_unavailable_percentage = 33
      }

      description = "EKS managed node group launch template"

      ebs_optimized           = true
      disable_api_termination = false
      enable_monitoring       = true

      block_device_mappings = {
        xvda = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size           = 50
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 150
            encrypted             = false
            delete_on_termination = true
          }
        }
      }

      metadata_options = {
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 2
        instance_metadata_tags      = "disabled"
      }

      create_iam_role          = true
      iam_role_name            = "eks-managed-ml-node-group-spot"
      iam_role_use_name_prefix = false
      iam_role_description     = "EKS managed node group spot-instance role"
      iam_role_tags = {
        Purpose = "Protector of the kubelet"
      }
      iam_role_additional_policies = {
        AmazonEC2ContainerRegistryReadOnly = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
        additional                         = aws_iam_policy.node_additional.arn
      }

      tags = {
        "k8s.io/cluster-autoscaler/enabled"                        = "true"
        "k8s.io/cluster-autoscaler/${local.global_prefix}-cluster" = "owned"
        "misc.instance-schedule"                                   = "spot"
      }
    }
  }

  tags = local.global_tags
}

Steps to reproduce the behavior:
Try creating an EKS cluster using two managed node groups and have pods on both nodes with one ingress-nginx pod. Expose a few (demo nginx) deployments and try accessing them.
Expected behavior
You should be able to access all workloads without timeouts
Actual behavior
Some deployments are accessible, while some are not. Depends on the scheduled node.
Terminal Output Screenshot(s)
Additional context
I thought it was a cluster networking issue and removed the aws-vpc-cni addon and installed calico, but it wasn't the case.
Overriding the ingress_nodes_ephemeral with the following rule fixed the issue.
  node_security_group_additional_rules = {
    node_to_node = {
      from_port = 0
      to_port   = 0
      protocol  = -1
      self      = true
      type      = "ingress"
    }
  }

This falls under the users responsibility based on their workloads and applications - https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/docs/network_connectivity.md#security-groups

Thanks for the clarification. I was moving from v16 of the module and the rule for node-to-node coms there led me to the above "fix".
Closing issue.


Skip ads instead of muting

Apparently you can skip ads now, maybe add an option to skip instead of muting?

Where are you from? Because in The Netherlands you cannot skip ads....that would be amazing =)

Personal Opinion, skipping is kind like a "hack" which breaks their (Spotify) business model but muting is more like a workaround.

@li-xinyang I'm pretty sure their main model is converting people to premium. The main ads I get here in Denmark are advertisements for premium.


Change camera attributes while streaming

How can camera attributes be changed while streaming? The code snippets below are incomplete because blocking occurs while streaming. Attempts to stop streaming are also blocked because the camera instance is used in both threads. Without the camera reference in CameraStreamer, blocking would not occur.
CameraControl.py:
from CameraStreamer import CameraStreamer
import picamera
import threading

streamThread = threading.Thread()
cam = CameraStreamer()

def camera_worker(camera, ipAddress):
    cam.runStream(camera, ipAddress)

class CameraControl():

    def __init__(self):
		super(CameraControl, self).__init__()
		self.camera = picamera.PiCamera()
		self.camera.resolution=(640, 480)
		self.camera.framerate = 24

	def setBrightness(self, value):
		self.camera.brightness = value

	def	startStream(self, ipAddress):
		streamThread = threading.Thread(target=camera_worker, args=(self.camera, ipAddress))
		streamThread.start()

	def	stopStream(self):
		cam.sendStream = 0

CameraStreamer.py:
import socket
import time

class CameraStreamer():

    def __init__(self):
        super(CameraStreamer, self).__init__()
        self.sendStream = 0

    def runStream(self, camera, ipAddress):
		# Adapted from https://github.com/waveform80/picamera/issues/226
		client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
		client_socket.connect((ipAddress, 8000))
		# Make a file-like object out of the connection
		filestream = client_socket.makefile('wb')
		camera.start_recording(filestream, format='h264')
		self.sendStream = 1
		while self.sendStream == 1:
			# only the first of the following lines should be needed in practice:
			camera.wait_recording(1) # check for exceptions
			time.sleep(1) # pause
			time.sleep(0) # yield
		camera.stop_recording()
		filestream.close()
		client_socket.close()

This is related to a previous issue I submitted about Event driven socket integration.

My problem is solved. I unnecessarily created the problem by reusing other people's code and concepts, most of which I should have discarded. My project uses PyQt4 so I used its timer instead. Here is a simplified recipe of what I use now:
from PyQt4 import QtCore
import picamera
import socket

class CameraControl(ModuleBase):

    def __init__(self):
        super(CameraControl, self).__init__()
        self.filestream = None
        self.client_socket = None
        self.timer = QtCore.QTimer()
        QtCore.QObject.connect(self.timer, QtCore.SIGNAL("timeout()"), self.checkCamera)
        self.camera = picamera.PiCamera()
        self.camera.resolution=(640, 480)
        self.camera.framerate = 24

    def setBrightness(self, value):
        self.camera.brightness = value

    def checkCamera(self):
        try:
            self.camera.wait_recording(0)
        except BaseException as err:
            print('checkCamera: '+ str(err), flush=True)
            self.timer.stop()

    def stopStream(self):
        self.timer.stop()
        self.camera.stop_recording()
        self.filestream.close()
        self.client_socket.close()

    def startStream(self, ipAddress):
        # Adapted from https://github.com/waveform80/picamera/issues/226
        self.client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.client_socket.connect((ipAddress, 8000))
        # Make a file-like object out of the connection
        self.filestream = self.client_socket.makefile('wb')
        self.camera.start_recording(self.filestream, format='h264')
        self.timer.start(1000) # start a timer that fires every second


[WFLY-10951] allowing a SSLException with SocketException cause

https://issues.jboss.org/browse/WFLY-10951
(subtask of https://issues.jboss.org/browse/WFLY-10812 )
elytron testsuite failures on JDK-11:

UndertowTwoWaySslNeedClientAuthTestCase produce a bit different SSLException on JDK-11

Retest this please


ENH: Add binary mask operation to Mask Volume effect

Fixes #2.
@lassoan I think this addresses the request. Binary mask operation gives the user the option to choose the outside and inside fill values (0 and 1 by default), and the output is saved as a labelMapVolumeNode. The smallest possible scalar type is also used, based on the inputted fill values.

@lassoan I went with "Fill inside and outside" as the updated operation name in the widget as well as as replacement for FILL_BOTH, as I think this is the easiest to understand. The reference to the output being a labelmap is in the effect description and I also added a tooltip to the radio button to give even more info to the user.

Perfect, thank you!


⚠ Bofrost.at has degraded performance

In f85f24d, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1060 ms

Resolved: Bofrost.at performance has improved in 67731c0 after 305 days, 8 hours, 6 minutes.


why build fail?

I have sucessfully imported classes and create a bridge header when i add BEMCheckBox in identity inspector it says build Fail.

Hi @chnouman,
Thank you for opening this issue. Could you give more details and context? Are you talking about the IB designables failing?


Why auto resize ?

This is my situation.
The following code works as expected if the SVG tag is contained in the body. When the container is a modal (in my case I'm using a bootstrap modal), the nodes are resized (but not the text). This makes it impossible to adopt the zoom to fix the problem.
var renderer = new dagreD3.Renderer();
renderer.zoom(false);
renderer.edgeInterpolate('linear');

var svg = d3.select(parent),
    svgGroup = svg.select('g');

var layout = renderer.run(g, svgGroup);

var xCenterOffset = (svg.attr('width') - layout.graph().width) / 2;
svgGroup.attr('transform', 'translate(' + xCenterOffset + ', 20)');
svg.attr('height', layout.graph().height + 40);

Expected (in body)

Unexpected (in modal)

I have already made several attempts to force the size of the height, which acts only on the size of the layout. When it starts in modal, the height of layout is set to "0", so i forced it with a minimum heigth:
            var height = layout.graph().height;
            svg.attr('height', height>300?height:300 + 40);

Still in this case, the rendering is resized. Any indication?

In his case bootstrap was actually setting the svg to hidden during the rendering so the DOM gave values of 0 for dimensions.

I have the same problem. Is there is a workaround?

We changed it to set opacity to 0 instead of setting display to hidden

what do you mean by this?


Components not Rendering

There seems to be something wrong with the children prop and I can't get my components to render properly (anymore).
Not sure if this is a Next.js, TypeScript, or React issue.
We can certainly work on this together, as I'd love to learn how all these pieces fit together.
Note: I've run into this while working on #12

Fixed by reverting _app.tsx

Wonderful work


TLC Frontend

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (fd763c6) to head (1c7b0bf).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##              main       #23   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            9         9
  Lines         1030      1030
  Branches       107       107
=========================================
  Hits          1030      1030

Flag
Coverage Δ

unittests
100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Subsequent chapter headings not displayed if last line of previous chapter is a comment

I encountered this bug when finishing one chapter with a comment and noticing the main chapter heading as well as the TOC listing not showing up.
The content of the following chapter is merged with the previous chapter.
For example, from the repo I modified Chapter 2:
\chapter{Background}
\label{ch:background}

\todo[The big question is whether Memento framework~\cite{memento:rfc,memento:tr} can be utilized to count historically preserved frogs?
Is Hypertext Transfer Protocol (HTTP)~\cite{http:rfc} adequate to fetch hibernated frogs?]

% I am a comment.

Nothing else in the template has changed. The result is Chapter 3's content (starting with Mat counted JCDL) appended onto the end of Chapter 2 (ends with hibernated frogs?).

We are using \input command to include chapters that includes file content as it is. If the last line of a chapter is a comment and it does not have a trailing newline then the chapter heading of the next chapter is appended to that comment.
I will not call it a bug. This is possibly happening because your editor is not ensuring that a newline character is added at the end of the file. many text editors such as Atom or Nano add a newline character at the end of the file, which is generally a good thing. GitHub highlights missing newline character in files in the web interface as an error.

Hmm, I disagree. If we chose \input as the method of including files, we ought to account for its behavior in appending files. It is most definitely a bug, and a bug in oduwsdl/wsdlthesis and not the input command or anywhere else.

editor is not ensuring that a newline character is added at the end of the file

By design. I have my editor setup to not automatically manipulate my files. I am using TeXShop, an editor designed with LaTeX in-mind and not a general-purpose text editor. I believe that LaTeX does not mandate that files end in a newline.


problem about your lstm block

Hi, I read your code and found a problem about your lstm block. Usually we don't shuffle the input data of lstm, but in your provider model, you shuffle the file names in function provider.read_from().
Won't shuffle break the data sequence or I miss something?
Thank u~

Hi, @public1024. You are correct! When you run the lstm model, the data sequence has to be maintained. You need to comment this line like the provider (Provider2) for testing.
https://github.com/driving-behavior/DBNet/blob/53dd1b2d3d609257920bb3553ca3d012abe29f0e/provider.py#L103
Note that this demo code is only tested on the nvidia-pn model. You are welcome to submit pull-requests for other settings :) Recently, I am busy in doing other things. I will provide full supports as well as the results if I get a chance.

thanks! Wonderful job!


feat: setup project documentation

As the project grows, there is a strong need for a docs page for new (and existing) users to find information which might be too verbose to add to the repo README.
For the first version, we can add:

getting started
api reference
tutorial 1: deploy basic llm chain
tutorial 2: deploy chains with vector store / retriever
tutorial 3: deploy custom chains

Acceptance Criteria

[ ] study https://readthedocs.org/
[ ] publish docs on https://readthedocs.org/

@rogalvil any recommendations towards this?

@ajndkr Honestly, I don't have any recommendations. I don't have much experience with documentation.


visualization of equations

Hi team,
As per this issue: https://github.com/MicrosoftDocs/quantum-docs-pr.fr-FR/issues/1
The contributor reported the equations are not correctly displayed in the fr-FR page (Machine-Translated)
This is also correct for the other locales so I suspect there must be a format issue in the source that makes the format break during the translation process.
Kind regards
Microsoft DOCS International Team

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6516e6bf-4cd0-2d9b-4d18-565d7dbc38b6
Version Independent ID: a5b54d5e-8b9a-f7d5-2dac-ecd515478ba5
Content: Multiple qubits - Microsoft Quantum
Content Source: articles/concepts/multiple-qubits.md
Product: quantum
GitHub Login: @QuantumWriter
Microsoft Alias<EMAIL_ADDRESS>

Apparently, looking at the Markdown source code for the auto-translated version it seems that the translation software is modifying the LaTeX code and therefore leading to build errors. Some of the problems:

The software sometimes adds a space in between $$ symbols, breaking the equation command. Then the compiler just thinks there are two empty in-line math environments before and after all the rest of the text gets compiled as plain text.
It tries to translate sometimes some of the LaTeX commands (e.g. in the German page \ddots occasionally turned to \dpunkte).

@CeciAc Do you know who is in charge of the automated translation software? It seems like we cannot do much about this.

Fixed

Fixed


Request to Open the Discussions Section

I would like to request that the Discussions section be enabled for this repository. It probably is overdue, but we can at least streamline conversations going forward by enabling it now.
Reasons to use a Discussion section:

Community Gallery: I think a pinned discussion that would be used for people to show off their boards would be beneficial. This gallery could lead to exposure to new ideas and interest in different functions that can lead to innovation and additions to the boards provided to the community. I have gotten my chimera RAT-RATGDO/ultrasonic-car-presence-sensor working with few changes in the device's yaml file, but before offering it up in a Pull Request, I would like to gauge community interest and gather opinions for a possible change in the implementation. I think this would be better (and more cleanly) done by using Discussions.
Cleaning up the Issues: Many of the Issues have been closed (sometimes before the conversations have been resolved) because they're not really Issues or they would be better served as Pull Requests. There have also been a few Issues opened where people simply want to express gratitude, which, while very nice to see, isn't an issue to be dealt with. By opening Discussions, we can move these conversations over there and reserve the Issues section for problems that need to be addressed by contributors to the repository, whether that be original contributors or those who have added their own boards.
Dealing with Troubleshooting: Occasionally, we have people who come to the repository who need help implementing the wealth of knowledge that has been accrued. It's a lot harder to come into the conversation later down the road and very easy to miss bits of information buried in a conversation from way back at the beginning that for others has become common knowledge. Instead of making an Issue for each case where simple troubleshooting is necessary to work through, it might be better to have a Discussion thread open where common issues are addressed in the first post (which would then be edited from time to time), and new problems not fixed by any of the standard troubleshooting techniques can be brought up in one localized conversation that will hopefully be easier to browse through for similar problems.
General Conversations: Some conversations have appeared that could be categorized together, such as the other alternatives to the original RATGDO that have appeared on the market. The Discussions section would be perfect for such general, tangentially related conversations that interest the community. Then, if they develop in such a way that requires some kind of action in the repository, Issues or Pull Requests could be created relating and pointing to the conversations that preceded them.

TL;DR: The Discussions section would be good and useful.

Willdo, sorry for the delay.  My full time job decided to get in my way!


Add support for extended advertising via Rust-only API

Extended functionality is gated on an "unstable" feature
Designed for very simple use and minimal interferance with existing legacy implementation
Intended to be temporary, until bumble can integrate extended advertising into its core functionality
Added error case for controller's on_hci_le_set_scan_parameters_command (matching the BLE spec) to test error flow from Rust
Dropped HciCommandWrapper in favor of using bumble's HCI_Command.from_bytes for converting from PDL into bumble implementation
Refactored Address and Device constructors to better match what the python constructors expect

@barbibulle are there any other changes I should make to this PR to make it ready for merge?

@barbibulle are there any other changes I should make to this PR to make it ready for merge?

Sorry for the delay. Looks good to go. I'll merge it now.

Sorry for the delay. Looks good to go. I'll merge it now.

Not a problem, thank you!


Firebase auth does not delete Github credentials from cache upon sign out

Description
Description:
Once a user signs on to Firebase Auth with Github credentials and then signs out, there is no way to sign back into another Github account. Firebase Auth appears to be caching the sign-in information of the user and when the user enters the same "Sign in with Github" flow in the future, the user gets automatically signed back into the previous account without being given a chance to provide different credentials.
Actual:
User attempts to sign into Firebase Auth using Sign-in-with-Github flow
User has to enter their Github username and password and authorize the app that is requesting the credentials
User is authenticated
User signs out
User attempts to sign back into Firebase Auth using Sign-in-with-Github flow
User is automatically signed back into previously signed-in account
Expectation:
User attempts to sign into Firebase Auth using Sign-in-with-Github flow
User has to enter their Github username and password and authorize the app that is requesting the credentials
User is authenticated
User signs out
User attempts to sign back into Firebase Auth using Sign-in-with-Github flow
User has to re-enter Github username and password and authorize the app that is requesting the credentials to allow the user to log on to different Github account
Reproducing the issue

Integrate Firebase Auth for a macCatalyst project
Sign in with Github as explained in Docs
Note that in catalyst Firebase Auth uses an internal class named FIRAuthWebViewController to display Github sign in page to the user. However, for iOS clients Firebase Auth uses SFSafariViewController. This difference will be important in step 7.
Once successfully signed in, sign the user out of the as explained in the same integration guide.
Now assume that this user has another github account and wants to sign in with that instead. For example one account for personal or another one for work. Allow the user sign back in to Github using the same flow as before. Signing in to another Github account is impossible at this point because the user gets automatically signed back in to the previous account. There is no opportunity to provide different credentials or go back or cancel.
Expected: When the user signs out of Firebase all the data (cookies, temp data etc...) is purged from both  FIRAuthWebViewController and SFSafariViewController so that user can provide different Github credentials upon re-entering the flow.
However, if you test this sign-in-with-Github flow on an iOS device (not macCatalyst) AND erase all Safari data from Settings THEN user is able to provide Github credentials again and not automatically signed back into the previous account. This work around does not work for macCatalyst projects because FIRAuthWebViewController is a web-view.

Firebase SDK Version
10.13
Xcode Version
14.3
Installation Method
Swift Package Manager
Firebase Product(s)
Authentication
Targeted Platforms
iOS, macCatalyst
Relevant Log Output
No response
If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

Replace this line with the contents of your Package.resolved.

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

Replace this line with the contents of your Podfile.lock!

For step 7, does signing out of GitHub in the Safari app reset auth state as you would expect it to? I suspect this is working as intended or unfixable on iOS, since Safari state is untouchable by Firebase. On macOS Catalyst this is probably a bug.

signing on to Github on Safari app on iOS does not affect what happens in the app. In other words - even if you are sign on to an account on Github on Safari, Firebase auth (SFSafariViewController) still asks for credentials again.

We have the same problem for the login with oidc, using the signInWIthProvider() method.
Is there any known workaround? As I cannot clear any data for the SFSafariViewController. The only way for the user to actually "change" is by deleting the Safari Cache outside of the app.

Uninstalling and reinstalling is another not-great workaround

I've confirmed that adding SFSafariViewController.DataStore.default.clearWebsiteData() the signOut implementation enables a new GitHub id to sign in on iOS.
I'm hesitant to add it to the SDK since it may mess with other Safari data in the app or other SDKs.

I also feel like, that this shouldnt be a standard approach of the SDK on logout.
This is still not really what Apple wants you to do, as you have no way to clear specific website data, but all.
I think the "right" way to handle this, is to open the Logout Pages of those providers when clicking on logout, so that the session data can be cleared gracefully.
From this SDKs perspective, this is however not generally fixable as you probably just dont know which provider was used, it could be anything. The provider could have any logout page and any redirect page on logout. So Im not sure if this "bug" can be fixed from the SDK perspective at all.
Something I can think of, would be a parameter on the signOut which enforces this behaviour, but is optional.
Its not a super clean approach, but the only thing that comes to my mind, if you want to handle this within the SDK.

No clear follow up actions here, so closing.


🌟Feature request: use floating-ui as peer dependency

Is your feature request related to a problem? Please describe.
I am using floating-ui in my project already and would like to avoid shipping it twice because it comes with melt already.
Describe the solution you'd like
Use floating-ui as a peer dependency.
Describe alternatives you've considered
Additional context

Can you confirm that it's actually shipping twice?

Actually you're right, it's not shipping twice if it's the same version. But wouldn't it still be cleaner to have it as a peer dependency?

I haven't messed with peerDeps, but from what I've seen, deps are for something your library needs to function, while peerDeps are something that is used alongside it. Since we directly need floating UI for our functionality, it seems a dep is better.
Source: https://stackoverflow.com/questions/18875674/whats-the-difference-between-dependencies-devdependencies-and-peerdependencie
Will be closing this now, but we can still discuss 🙂


Vendor folder as a symlink

Hi,
With Symfony version 4.* this worked perfectly, I had the vendor folder in the shared folder set up, so there was no need for "composer install" every time on deployment...
Running "assets:install --symlink" in post_deploy,  worked perfectly.
After upgrading to Symfony 5, this is not working anymore... Symfony command complains that it can't find the .env file in the "shared" folder but it should look into the "current/vendor" folder.
Is this something that can be fixed?
Thanks
Alex

Hello Alex
Thank you for reaching out. As this ticket is old, we're closing it now. However, if you need further assistance, feel free to reach out. We've recently resumed development with the release of version 4.0.0-beta1.


Azure Durable functions - inconsistent version used in docs

the docs use python V1 for the Fan-out/Fan-in tutorial. this should be changed to v2 examples if v2 is recommended for use.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 62bbedb6-d4c5-a809-31ad-b8c843ac0f52
Version Independent ID: 07c4822d-0aad-5a8a-7f16-56835f325a42
Content: Fan-out/fan-in scenarios in Durable Functions - Azure
Content Source: articles/azure-functions/durable/durable-functions-cloud-backup.md
Service: azure-functions
Sub-service: durable
GitHub Login: @ggailey777
Microsoft Alias: azfuncdf

@yovelcohen
Thanks for your feedback! We will investigate and update as appropriate.


Added toast onHideCallback parameter

Per #134, an onHideCallback argument was added that is attached to each
toast instance.  On remove, the callback is invoked if it exists.
Cleaned up missing closure parameters and added angular as a global
reference.

@jirikavi If/when you merge this pull request, it might make sense to bump it to a new version since there's new features and cleanup.

Thanks for your contribution Stabzs. You really help me a lot with the repo. Would you like me to add you to the repo as a collaborator?
Jiri.

@jirikavi I'd be honored to be a collaborator on the project.  It's a fantastic piece of work and I greatly enjoy helping to improve it further.


Make request code smaller

If using Cordova as a component and the app is based in fragment, there is a 16bit size limit for the request code and we are currently using a 32bit one, reduce it to be 16 bit so it's compatible with fragment based apps.

@jcesarmobile merged


rename component class to just WebVR rather than WebVRCamera

Since it does more than just manage camera.

@caseyyee: we can close this, per your refactor work in PR #215?

Yes, thanks!


Using reindex API with a repository having a remote source causes an HTTP 400, no possibility to provide the CA certificate

Hi,
we are currently evaluating ways to move our data from elasticsearch to opensearch. To move our data asynchronously we want to use the reindex API (https://opensearch.org/docs/latest/opensearch/reindex-data/#reindex-from-a-remote-cluster), where the source repository points to the elasticsearch cluster:
{
   "conflicts":"proceed",
   "source":{
      "remote":{
         "host":"https://elasticsearch-es-http:9200",
         "username":"elastic",
         "password":"<redacted>"
      },
      "index":"..._log-ns-9_9"
   },
   "dest":{
      "index":"_log-ns-9_9",
      "op_type":"index"
   }
}

The POST to _reindex/ causes the following (expected) error:
elasticsearch API error: POST https://localhost:9201/_reindex 500: {"error":{"root_cause":[{"type":"s_s_l_handshake_exception","reason":"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}],"type":"s_s_l_handshake_exception","reason":"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target","caused_by":{"type":"validator_exception","reason":"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target","caused_by":{"type":"sun_cert_path_builder_exception","reason":"unable to find valid certification path to requested target"}}},"status":500}

The problem is not the error, its how to provide the CA certificate so opensearch will trust the self-signed certificate provided by elasticsearch. Any recommendations?
Operator version: 2.1.1
Opensearch version: 1.3.6
I have read https://github.com/Opster/opensearch-k8s-operator/blob/main/docs/userguide/main.md#add-secrets-to-keystore but I believe this is not the place to look for.
Best regards

Short update, I may have found a way to provide the CA.
I have configured the cluster with:
  general:
    additionalConfig:
      reindex.remote.whitelist: 'elasticsearch-es-http:9200'
      reindex.ssl.certificate_authorities: /var/run/secrets/elasticsearch/elasticsearch-http-ca.crt
    additionalVolumes:
      - name: elasticsearch-ca
        path: /var/run/secrets/elasticsearch
        secret:
          defaultMode: 511
          secretName: opensearch-elasticsearch-ca

This configuration mounts the secret correctly, however opensearch has problems with it:
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/var/run/secrets/elasticsearch/elasticsearch-http-ca.crt" "read")

I am not sure why this is the case, I can successfully cat the content from said secret:
sh-4.2$ ls -la /var/run/secrets/elasticsearch/elasticsearch-http-ca.crt
lrwxrwxrwx. 1 root root 32 Dec 16 13:26 /var/run/secrets/elasticsearch/elasticsearch-http-ca.crt -> ..data/elasticsearch-http-ca.crt

sh-4.2$ cat /var/run/secrets/elasticsearch/elasticsearch-http-ca.crt
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

Okay found the issue, opensearch does not allow using files outside of the opensearch config directory. Mounting the secret to /usr/share/opensearch/config/elasticsearch-tls enables opensearch to start up correctly.

Yes, the issue can be closed. I now have another issue with the actual application (graylog does not show data beyond the point where I actually moved to opensearch) but that is another story. The documents are shown in opensearch index just fine.

Closing as resolved.


Updated generation of Table of Contents to have a hierarchy and minor…

… clean up.
Biggest change is adding parents and children to the schema references to help build the hierarchy for the Table of Contents.
@bghgary
@HowardWolosky

Thanks @BeardedGnome!  Before we can take code from a new contributor, we need you to fill out the CLA (Contributor License Agreement) and email to<EMAIL_ADDRESS> This just makes it explicit that the code was intended to be shared.  Thanks!
@HowardWolosky do you have bandwidth for a code review?

Yup.  I started the review already and will finish it in the next couple hours.

Thanks all!
We now have a CLA from @BeardedGnome.

Thanks @BeardedGnome and @pjcozzi.
Back to you @HowardWolosky.  Now that the red tape is cut through, feel free to hit merge on this anytime you're happy with it.  Thanks!

I'm on it....will finish the review today.  Apologies for the delay.

Thanks for your contribution, @BeardedGnome!  This definitely makes a more readable TOC for the documentation.


Classifier requirement of APT makes using Spring's Dependency Management Gradle plugin awkward

Background
Currently, as stated in the Querydsl documentation, a classifier is necessary to use querydsl-apt:

Notice that you need to use a proper classifier when defining dependency to com.querydsl:querydsl-apt. Those additional artifacts define the annotation processor to be used in META-INF/services/javax.annotation.processing.Processor.
Available classifiers include:

general
hibernate
jdo
jpa

This means that the classifier needs to be specified in the build.  In a vanilla Gradle project, this would typically be done as follows:
dependencies {
    annotationProcessor 'com.querydsl:querydsl-apt:4.4.0:general'
    compileOnly 'com.querydsl:querydsl-core:4.4.0'
    implementation 'com.querydsl:querydsl-mongodb:4.4.0'
}

Spring's Dependency Management Plugin Gradle plugin allows specifying project dependency versions in a dependencyManagement block.  This allows versions to be consistent with the underlying frameworks, consistent cross-project versions, and adds convenience when specifying dependencies as the version numbers are excluded.
The spring-boot-dependencies BOM is commonly used with this by Spring Boot projects, and includes dependencies on Querydsl:
com.querydsl:querydsl-apt:4.4.0
com.querydsl:querydsl-collections:4.4.0
com.querydsl:querydsl-core:4.4.0
com.querydsl:querydsl-jpa:4.4.0
com.querydsl:querydsl-mongodb:4.4.0

The problem
However, the dependency management plugin does not support the use of classifiers (https://github.com/spring-gradle-plugins/dependency-management-plugin/issues/67).  This is stated to be due to a limitation with Gradle itself (https://github.com/gradle/gradle/issues/1947) not providing the classifier through its API.
Workaround
This means that a somewhat awkward workaround is necessary to use the dependency without hardcoding the version.  One solution is to use the importedProperties option to access the querydsl.version property in the spring-boot-dependencies Maven BOM, as suggested in another issue (https://github.com/querydsl/querydsl/issues/2459#issuecomment-659892795):
dependencies {
    annotationProcessor 'com.querydsl:querydsl-apt:4.4.0:general'
}

A similar solution that's worked for me is to use the managedVersions map to use the managed version of the querydsl-apt dependency.  This also has the benefit of working in a non-Boot Gradle project that uses the Spring Dependency Management plugin.
dependencies {
  annotationProcessor "com.querydsl:querydsl-apt:${dependencyManagement.managedVersions['com.querydsl:querydsl-apt']}:general"
}

Feature request
It would be nice if Querydsl & the Spring Dependency Management plugin would play together better so that the version would not need to be redundantly declared if it's brought in through dependencyManagement.  I don't have any clever solutions in mind; the best I've got is to have different artifacts rather than different classifiers (e.g. querydsl-apt-general instead of name: querydsl-apt, classifier: general.  But that seems a pretty heavy-handed solution to the problem, unless that was a change you wanted to make anyway.
At a minimum, it would be nice if the workaround was documented.  There does not appear to be much on the web around this specific issue, and the workaround within the Querydsl project is in large comment thread in an unrelated issue, linking to a Korean language blog entry.

You don’t actually need to use the classifier as long as you provide the APT processor classname as compiler argument.
Actually most of the Querydsl documentation/tutorials out there do precisely that.
That said, we currently do not actively support Gradle. As we provide just a plain APT plugin, it should work, but we do not provide working examples.

You don’t actually need to use the classifier as long as you provide the APT processor classname as compiler argument.
Actually most of the Querydsl documentation/tutorials out there do precisely that.
That said, we currently do not actively support Gradle. As we provide just a plain APT plugin, it should work, but we do not provide working examples.

I thought the spring dependency management plugin was deprecated and that we should use platform().

I thought the spring dependency management plugin was deprecated and that we should use platform().

You don’t actually need to use the classifier as long as you provide the APT processor classname as compiler argument.
Actually most of the Querydsl documentation/tutorials out there do precisely that.

That's great to know.  Is the purpose of the classifiers/usage of the APT processor classname documented somewhere?  This would be very helpful to have a canonical reference to indicate how this is intended to be used, and what the classifiers were used for.
In my own situation, I guess I was reading the Querydsl documentation documentation in the wrong context.  I misunderstood it to say that the usage of a classifier was required, when I guess it's only saying that it's necessary to use the maven-compiler-plugin:

Notice that you need to use a proper classifier when defining dependency to com.querydsl:querydsl-apt. Those additional artifacts define the annotation processor to be used in META-INF/services/javax.annotation.processing.Processor.

You don’t actually need to use the classifier as long as you provide the APT processor classname as compiler argument.
Actually most of the Querydsl documentation/tutorials out there do precisely that.

That's great to know.  Is the purpose of the classifiers/usage of the APT processor classname documented somewhere?  This would be very helpful to have a canonical reference to indicate how this is intended to be used, and what the classifiers were used for.
In my own situation, I guess I was reading the Querydsl documentation documentation in the wrong context.  I misunderstood it to say that the usage of a classifier was required, when I guess it's only saying that it's necessary to use the maven-compiler-plugin:

Notice that you need to use a proper classifier when defining dependency to com.querydsl:querydsl-apt. Those additional artifacts define the annotation processor to be used in META-INF/services/javax.annotation.processing.Processor.

@mjustin querydsl-apt contains various processors, for JPA, SQL, MongoDB, etc.
While it is possible to register the processor on the classpath using META-INF/services/javax.annotation.processing.Processor, having all these implementations in there would cause all modules to generate code individually.
The classifiers basically specify which implementation is provided in META-INF/services/javax.annotation.processing.Processor.
If you omit the classifier (use the default classifier), META-INF/services/javax.annotation.processing.Processor is simply missing and you have to register the APT processor as command line argument.
I'll make sure to change the documentation accordingly.
Did you manage to get APT processing to work in Gradle though?

@mjustin querydsl-apt contains various processors, for JPA, SQL, MongoDB, etc.
While it is possible to register the processor on the classpath using META-INF/services/javax.annotation.processing.Processor, having all these implementations in there would cause all modules to generate code individually.
The classifiers basically specify which implementation is provided in META-INF/services/javax.annotation.processing.Processor.
If you omit the classifier (use the default classifier), META-INF/services/javax.annotation.processing.Processor is simply missing and you have to register the APT processor as command line argument.
I'll make sure to change the documentation accordingly.
Did you manage to get APT processing to work in Gradle though?

Did you manage to get APT processing to work in Gradle though?

I currently have it working by using the classifier (com.querydsl:querydsl-apt:4.4.0:general).
I played around with using the default classifier.  Specifying the processor as a -processor compilation argument is not a good solution for me, since I have several other annotation processors declared (using annotationProcessor dependencies), and it looks like if I declare one in -processor I need to declare them all.
I confirmed that adding a META-INF/services/javax.annotation.processing.Processor to the classpath with a value of com.querydsl.apt.QuerydslAnnotationProcessor does cause that processor to be used.  However, I'm not sure there's a clean way in Gradle to use that file in the annotation classpath but not in the compile classpath without doing a fair amount of boilerplate build code, at which point using the classifier workaround mentioned in this issue becomes a win.

Did you manage to get APT processing to work in Gradle though?

I currently have it working by using the classifier (com.querydsl:querydsl-apt:4.4.0:general).
I played around with using the default classifier.  Specifying the processor as a -processor compilation argument is not a good solution for me, since I have several other annotation processors declared (using annotationProcessor dependencies), and it looks like if I declare one in -processor I need to declare them all.
I confirmed that adding a META-INF/services/javax.annotation.processing.Processor to the classpath with a value of com.querydsl.apt.QuerydslAnnotationProcessor does cause that processor to be used.  However, I'm not sure there's a clean way in Gradle to use that file in the annotation classpath but not in the compile classpath without doing a fair amount of boilerplate build code, at which point using the classifier workaround mentioned in this issue becomes a win.

QueryDSL supports Gradle for jpa code generation, as described in https://github.com/querydsl/querydsl/issues/2231#issuecomment-357956070 . Doing so neatly requires the use of the jpa classifier. Because Gradle does not support classifiers in the dependency management plugin (gradle/gradle#1947) this doesn't work properly when a version for querydsl-apt is provided through a bill-of-materials.
Probably best to fix this in Gradle or Springs BOM instead, although I would accept a PR for this change I am going to close this for now as the documentation aspect as far as Gradle support goes will already be covered in #2231.


Buefy Theme

Hi,
Thanks a lot for your vuepress-theme-bulma, could you tell me who I can use Buefy in your theme ?
Thanks in advance.

I updated at branch name buefy-demo URL: https://github.com/nakorndev/vuepress-theme-bulma/tree/buefy-demo
This commit should help you understand to use Buefy https://github.com/nakorndev/vuepress-theme-bulma/commit/c96f33fdaa2d3f47291d3cd818da33a674791581