weave-net failing to sleeve on EKS

What you expected to happen?
Weave works in fastdp mode
What happened?
Weave failing to sleeve mode
How to reproduce it?
Seems to immediately switch to sleeve mode
Anything else we need to know?

EKS
us-east-1
2 subnets in 2 AZs
4 nodes
r5.xlarge

Versions:

# /home/weave/weave --local version
weave 2.6.1
$ docker version
# uname -a
Linux ip-10-207-56-160.ec2.internal 4.14.171-136.231.amzn2.x86_64 #1 SMP Thu Feb 27 20:22:48 UTC 2020 x86_64 Linux
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-23T14:21:36Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.9-eks-502bfb", GitCommit:"502bfb383169b124d87848f89e17a04b9fc1f6f0", GitTreeState:"clean", BuildDate:"2020-02-07T01:31:02Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

Logs:
$ kubectl logs -n kube-system weave-net-pnv9s -c weave
		DEBU: 2020/03/10 22:51:36.008130 [kube-peers] Checking peer "66:df:31:d5:0e:9b" against list &{[{56:49:51:13:23:25 ip-10-207-56-224.ec2.internal} {8a:a4:a4:85:cb:6d ip-10-207-56-243.ec2.internal} {66:df:31:d5:0e:9b ip-10-207-56-160.ec2.internal} {62:77:0a:f2:8e:69 ip-10-207-56-144.ec2.internal}]}
INFO: 2020/03/10 22:51:36.060099 Command line options: map[conn-limit:200 datapath:datapath db-prefix:/weavedb/weave-net docker-api: expect-npc:true host-root:/host http-addr:<IP_ADDRESS>:6784 ipalloc-init:consensus=3 ipalloc-range:<IP_ADDRESS>/10 metrics-addr:<IP_ADDRESS>:6782 mtu:1376 name:66:df:31:d5:0e:9b nickname:ip-10-207-56-160.ec2.internal no-dns:true port:6783]
INFO: 2020/03/10 22:51:36.060129 weave  2.6.1
INFO: 2020/03/10 22:51:36.253396 Re-exposing <IP_ADDRESS>/10 on bridge "weave"
INFO: 2020/03/10 22:51:36.277079 Bridge type is bridged_fastdp
INFO: 2020/03/10 22:51:36.277095 Communication between peers is unencrypted.
INFO: 2020/03/10 22:51:36.330882 Our name is 66:df:31:d5:0e:9b(ip-10-207-56-160.ec2.internal)
INFO: 2020/03/10 22:51:36.330925 Launch detected - using supplied peer list: [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
INFO: 2020/03/10 22:51:36.346564 Checking for pre-existing addresses on weave bridge
INFO: 2020/03/10 22:51:36.346677 weave bridge has address <IP_ADDRESS>/10
INFO: 2020/03/10 22:51:36.351519 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.351668 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.351798 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.351907 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.351998 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352087 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352174 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352259 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352339 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352425 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352508 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352620 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352705 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352792 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352877 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.352963 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353051 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353137 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353221 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353331 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353447 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353534 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353718 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353813 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.353899 Found address <IP_ADDRESS>/10 for ID _
INFO: 2020/03/10 22:51:36.354138 [allocator 66:df:31:d5:0e:9b] Initialising with persisted data
INFO: 2020/03/10 22:51:36.354241 Sniffing traffic on datapath (via ODP)
INFO: 2020/03/10 22:51:36.360466 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2020/03/10 22:51:36.360571 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2020/03/10 22:51:36.360628 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2020/03/10 22:51:36.361538 ->[<IP_ADDRESS>:6783|62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)]: connection ready; using protocol version 2
INFO: 2020/03/10 22:51:36.361597 overlay_switch ->[62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)] using fastdp
INFO: 2020/03/10 22:51:36.361630 ->[<IP_ADDRESS>:6783|62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)]: connection added (new peer)
INFO: 2020/03/10 22:51:36.362526 fastdp ->[<IP_ADDRESS>:6784|62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)]: IPSec init SA remote
WARN: 2020/03/10 22:51:36.362554 fastdp ->[<IP_ADDRESS>:6784|62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)]: IPSec init SA remote failed: deserialize InitSARemote: empty msg
INFO: 2020/03/10 22:51:36.362722 overlay_switch ->[62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)] fastdp deserialize InitSARemote: empty msg
INFO: 2020/03/10 22:51:36.362735 overlay_switch ->[62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)] using sleeve
INFO: 2020/03/10 22:51:36.363398 ->[<IP_ADDRESS>:6783|8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)]: connection ready; using protocol version 2
INFO: 2020/03/10 22:51:36.363398 Listening for HTTP control messages on <IP_ADDRESS>:6784
INFO: 2020/03/10 22:51:36.363456 overlay_switch ->[8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)] using fastdp
INFO: 2020/03/10 22:51:36.363476 ->[<IP_ADDRESS>:6783|8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)]: connection added (new peer)
INFO: 2020/03/10 22:51:36.363635 fastdp ->[<IP_ADDRESS>:6784|8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)]: IPSec init SA remote
WARN: 2020/03/10 22:51:36.363658 fastdp ->[<IP_ADDRESS>:6784|8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)]: IPSec init SA remote failed: deserialize InitSARemote: empty msg
INFO: 2020/03/10 22:51:36.363660 Listening for metrics requests on <IP_ADDRESS>:6782
INFO: 2020/03/10 22:51:36.363991 overlay_switch ->[8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)] fastdp deserialize InitSARemote: empty msg
INFO: 2020/03/10 22:51:36.364003 overlay_switch ->[8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)] using sleeve
INFO: 2020/03/10 22:51:36.364140 ->[<IP_ADDRESS>:6783|56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)]: connection ready; using protocol version 2
INFO: 2020/03/10 22:51:36.364177 overlay_switch ->[56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)] using fastdp
INFO: 2020/03/10 22:51:36.364194 ->[<IP_ADDRESS>:6783|56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)]: connection added (new peer)
INFO: 2020/03/10 22:51:36.365314 fastdp ->[<IP_ADDRESS>:6784|56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)]: IPSec init SA remote
WARN: 2020/03/10 22:51:36.365347 fastdp ->[<IP_ADDRESS>:6784|56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)]: IPSec init SA remote failed: deserialize InitSARemote: empty msg
INFO: 2020/03/10 22:51:36.365417 overlay_switch ->[56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)] fastdp deserialize InitSARemote: empty msg
INFO: 2020/03/10 22:51:36.365545 overlay_switch ->[56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)] using sleeve
INFO: 2020/03/10 22:51:36.462200 ->[<IP_ADDRESS>:6783|56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)]: connection fully established
INFO: 2020/03/10 22:51:36.462418 ->[<IP_ADDRESS>:6783|62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)]: connection fully established
INFO: 2020/03/10 22:51:36.462519 ->[<IP_ADDRESS>:6783|8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)]: connection fully established
INFO: 2020/03/10 22:51:36.462665 sleeve ->[<IP_ADDRESS>:6783|62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)]: Effective MTU verified at 8939
INFO: 2020/03/10 22:51:36.463359 sleeve ->[<IP_ADDRESS>:6783|8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)]: Effective MTU verified at 8939
INFO: 2020/03/10 22:51:36.463529 sleeve ->[<IP_ADDRESS>:6783|56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)]: Effective MTU verified at 8939
INFO: 2020/03/10 22:51:37.068350 [kube-peers] Added myself to peer list &{[{56:49:51:13:23:25 ip-10-207-56-224.ec2.internal} {8a:a4:a4:85:cb:6d ip-10-207-56-243.ec2.internal} {66:df:31:d5:0e:9b ip-10-207-56-160.ec2.internal} {62:77:0a:f2:8e:69 ip-10-207-56-144.ec2.internal}]}
DEBU: 2020/03/10 22:51:37.079517 [kube-peers] Nodes that have disappeared: map[]
<IP_ADDRESS>
DEBU: 2020/03/10 22:51:37.148724 registering for updates for node delete events
INFO: 2020/03/10 22:51:38.906194 Discovered remote MAC 9e:0a:32:5b:2b:87 at 62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)
INFO: 2020/03/10 22:51:39.162080 Discovered remote MAC 8e:4f:03:51:da:69 at 8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)
INFO: 2020/03/10 22:51:39.217053 Discovered remote MAC d6:8b:82:d9:52:5f at 56:49:51:13:23:25(ip-10-207-56-224.ec2.internal)
INFO: 2020/03/10 22:51:39.925408 Discovered remote MAC e6:bd:bd:e7:72:23 at 8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal)
INFO: 2020/03/10 22:51:40.392799 Discovered remote MAC 7a:a0:dc:d9:08:11 at 62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)
INFO: 2020/03/10 22:52:06.210852 Discovered remote MAC 72:2f:ec:b2:15:80 at 62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)
INFO: 2020/03/10 22:52:08.926139 Discovered remote MAC 2a:59:10:8e:77:ab at 62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal)

Debug
Status:
# /home/weave/weave --local status

        Version: 2.6.1 (up to date; next check at 2020/03/11 06:10:37)

        Service: router
       Protocol: weave 1..2
           Name: 66:df:31:d5:0e:9b(ip-10-207-56-160.ec2.internal)
     Encryption: disabled
  PeerDiscovery: enabled
        Targets: 3
    Connections: 3 (3 established)
          Peers: 4 (with 12 established connections)
 TrustedSubnets: none

        Service: ipam
         Status: ready
          Range: <IP_ADDRESS>/10
  DefaultSubnet: <IP_ADDRESS>/10

Connections status:
 # /home/weave/weave --local status connections
-> <IP_ADDRESS>:6783    established sleeve 62:77:0a:f2:8e:69(ip-10-207-56-144.ec2.internal) mtu=8939
-> <IP_ADDRESS>:6783    established sleeve 8a:a4:a4:85:cb:6d(ip-10-207-56-243.ec2.internal) mtu=8939
-> <IP_ADDRESS>:6783    established sleeve 56:49:51:13:23:25(ip-10-207-56-224.ec2.internal) mtu=8939

thanks @hibbert for reporting the issue.
I can confirm the reported issue on 2.6.1. It seems like a regression with 2.6.1. Will invetigate further.


'flynn export' on a Docker image results in a corrupt tarball

Running flynn -a backups export > /tmp/backups-export works without error, returning:
2.81 KB 14.00 KB/s
flynn: pulling Docker image with "docker pull docker.flynn-sbx-qa-us-east-1.nexus.bazaarvoice.com/backups@sha256:4158c3f5ea96efd3b3194f8dc20a5f08dbd21e38f40982bcf233b88e56825a0a"
550.54 MB 63.86 MB/s 8s

However, the tarball appears corrupted:
$ tar xvvf /tmp/backups-export
-rw-r--r-- 1000/1000       390 2016-11-02 15:22 backups/app.json
-rw-r--r-- 1000/1000       844 2016-11-02 15:22 backups/routes.json
-rw-r--r-- 1000/1000      1182 2016-11-02 15:22 backups/release.json
-rw-r--r-- 1000/1000       458 2016-11-02 15:22 backups/artifact.json
-rw-r--r-- 1000/1000       234 2016-11-02 15:22 backups/formation.json
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

The CLI was writing the output of docker pull to stdout, which was corrupting the export (an explicit --file /tmp/backups-export is a quick workaround).
Fix proposed in #3615.


[SPARK-49511][SQL] Apply formatting rules to sql/api

What changes were proposed in this pull request?
This PR proposes to apply formatting rules to sql/api
Why are the changes needed?
The package is new, and we can start autoformatting. It won't have downside of backporting difficulty and etc.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Yes, by dev/lint-scala
Was this patch authored or co-authored using generative AI tooling?
No.

cc @HyukjinKwon we discussed this.

Thank you for the clean-ups.

Manually tested.
Merged to master.


Not downloading/receiving images

When someone sends me an image through iMessage (group or 1-to-1 chat), I see the loading icon for an image saying "Downloading Image" with a progress bar. Once the progress bar finishes, it displays "Failed to Download! Tap to try again". The same problem repeats when I click try again. I haven't checked if this also occurs with videos yet.
A video with the issue: https://github.com/user-attachments/assets/5e0a4256-82e4-48d0-94a8-771fa166aef7

The same thing happens to me. I thought I was the only one.
I keep getting this error when I try to send Images

Seems to be fixed in latest update.


Show current URL

For samples involving routing, it would be handy to be able to show the current URL somewhere in DartPad.

This is an interesting idea, I can definitely see the use cases.
I'm assuming you just want to show the path, query, parameters, etc? (So everything after tld and port)
There's not a lot of space, especially in embedded scenarios, so some sort of button would be nice. Likely defaulting to off, but controllable with a query parameter, as I'm assuming routes aren't a summer common use case.

This would be great! If you could include back/forward button as well this would be awesome 👌

I'm not sure we'll be able to invest the time to implement this, especially with our goal to keep the DartPad UI similar. It's perhaps something interesting to explore for a DevTools extension or IDE plugin though.
If someone would still like to see this though, please feel free to reopen and/or continue the discussion. Thanks!


Only log exceptions if they are true server errors

Issue #1151

Very tidy, thanks. Can you confirm that this cleans up the nosetests output?

@jeromekelleher yep it does

Great, +1.
@kozbo, are you happy with this change? This should mean that only genuine errors end up in the Apache error log as a stack trace. Normal user input errors should come up in error.log as a single line (indicating that they've been handled normally).

Yes, this seems like a very clean solution.

+1 Yeah, this is annoying. I've been looking at a lot of logs lately and the error logs will be cleaner. It would be great to be able to log those messages elsewhere as they are still useful but not "errors". Could you also check if the test_request_context removes the request path? The messages say they were from requests pointed at /GET, when I don't believe they are:
[Wed Apr 27 04:33:05.014055 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704] Exception on / [GET]
[Wed Apr 27 04:33:05.014155 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704] Traceback (most recent call last):
[Wed Apr 27 04:33:05.014166 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1475, in full_dispatch_request
[Wed Apr 27 04:33:05.014174 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     rv = self.dispatch_request()
[Wed Apr 27 04:33:05.014180 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1461, in dispatch_request
[Wed Apr 27 04:33:05.014187 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     return self.view_functions[rule.endpoint](**req.view_args)
[Wed Apr 27 04:33:05.014194 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/ga4gh-0.2.3.dev12+ng19af245.d20160422-py2.7.egg/ga4gh/frontend.py", line 550, in searchDatasets
[Wed Apr 27 04:33:05.014201 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     flask.request, app.backend.runSearchDatasets)
[Wed Apr 27 04:33:05.014207 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/ga4gh-0.2.3.dev12+ng19af245.d20160422-py2.7.egg/ga4gh/frontend.py", line 432, in handleFlaskPostRequest
[Wed Apr 27 04:33:05.014214 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     return handleHttpPost(flaskRequest, endpoint)
[Wed Apr 27 04:33:05.014223 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/ga4gh-0.2.3.dev12+ng19af245.d20160422-py2.7.egg/ga4gh/frontend.py", line 309, in handleHttpPost
[Wed Apr 27 04:33:05.014230 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     responseStr = endpoint(request.get_data())
[Wed Apr 27 04:33:05.014236 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/ga4gh-0.2.3.dev12+ng19af245.d20160422-py2.7.egg/ga4gh/backend.py", line 932, in runSearchDatasets
[Wed Apr 27 04:33:05.014243 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     self.datasetsGenerator)
[Wed Apr 27 04:33:05.014249 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   File "/srv/ga4gh/server/newserverenv/lib/python2.7/site-packages/ga4gh-0.2.3.dev12+ng19af245.d20160422-py2.7.egg/ga4gh/backend.py", line 685, in runSearchRequest
[Wed Apr 27 04:33:05.014255 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]     raise exceptions.InvalidJsonException(requestStr)
[Wed Apr 27 04:33:05.014261 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704] InvalidJsonException: Cannot parse JSON: '{
[Wed Apr 27 04:33:05.014281 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704]   "pageSize": 10,
[Wed Apr 27 04:33:05.014288 2016] [:error] [pid 13825:tid<PHONE_NUMBER>80704] }'

Great, we're all happy with this then. I think we'll need to look at what Apache prints out now once this is in @david4096, and see what we need to do logging-wise. Let's try and get a list of requirements that we have under the Logging milestone.


Update alfy to 0.8.0 in package.json

This will fix this issue: https://github.com/SamVerschueren/alfred-link/pull/20
It occurs when a user tries installing your workflow from npm as their very first workflow after installing Alfred 3. It is because a /workflows does not exist until you create a workflow in Alfred.

Oh, thanks for that, I'll do it. 👍

Fixed in https://github.com/jukben/alfred-emoji/commit/c5c1d89b674a2a9c3693d5db97ee660de04c6462 🚀 Deployed as 0.2.1


HWKAPM-533 | Add http method from Span to derived information objects

https://issues.jboss.org/browse/HWKAPM-533

@objectiser could you please look when you have some time?

Review finished - just a couple of minor changes.

Updated version is here.

@pavolloffay LGTM :+1:


version: post-release bump

Signed-off-by: Jiri Kuncar<EMAIL_ADDRESS>

Coverage remained the same at 100.0% when pulling 4b76590db6d3c79d61b52fc48f0ee65dbdc655ca on jirikuncar:version-bump into 5b74d55927ec2f728a8027869e8c839e0a6e72e7 on inveniosoftware:master.

Coverage remained the same at 100.0% when pulling fbdf35c8ca30a994b5ca0c852eb520419bf18ef6 on jirikuncar:version-bump into 43644435d9b79f374c0fb3619afdc72d2f6ead77 on inveniosoftware:master.


Audit logging

When using mikro-orm with nestjs (in my case) I want to use the EventSubscriber to act on Insert, Update and Delete events. However, I cannot create new AuditLog entities inside the EventSubscriber. I cannot inject my AuditLogService into the Subscriber
I'd like to be able to inject a service into the EventSubscriber which can then be used to insert new AuditLog entities into the db.
I used typeorm before, which did support something similar. With Mikro-Orm I have tried to use the @Injectable decorator and added the subscriber to the provider array of nestjs App Module.

It should work fine, all you need to do is to register the subscriber in the EventManager when the subscriber gets instanced. Something like this could work:
@Injectable()
export class MySubscriber {

  constructor(em: EntityManager) {
    em.getEventManager().registerSubscriber(this);
  }

}

But this won't allow you to flush new entities from inside the event handlers (not in the same UoW, you can fire native queries for sure). For that you need to use flush events:
https://github.com/mikro-orm/mikro-orm/blob/dev/docs/docs/lifecycle-hooks.md#flush-events

Does this mean that you can inject your own service into a subscriber along with the EntityManager and the subscribe the thing as an EventSubscriber? And what if in that case the service would try to flush a new (audit log) entity?

I created a repository displaying the problem I am having. If you have time would you mind looking at it? https://github.com/RiskChallenger/mikro-orm-issue (Second example in the repo)

As I already said, you can't use lifecycle events for this, you need to use flush events. Lifecycle events are fired when the transaction is already running, calling flush inside them is forbidden. What you can do is to create a new fork (via em.fork()) and use that - which will create separate transaction. Or you can use em.nativeInsert() (which is equivalent to using a QB).
Btw all event handlers will get EM instance in the EventArgs. You should use that one, as there can be multiple instances of EM (e.g. when doing nested transactions), and the one in EventArgs is always the one that triggered the event.
Will take a look at the repo, I believe it should be working, but I haven't tested this myself yet, so there might be some problem with this lazy registration (maybe you are registering it to the request scoped EM, but you need to register it to the global one).

I tried logging the different id's of the EM's and found that the EM that the EventSubscriber is registered to is different from the EM that is in the RequestContext. So I guess that the EventSubscriber is registered to the global one.
In an additional test I added a TestSubscriber that also subscribed to afterCreate Project events. This EventSubscriber was registered using the @Subscriber() annotation. After both are registered I logged console.log(em.getEventManager()['listeners']); what showed both subscribers. Even though both a registered, only the TestSubscriber is triggered after saving a new project.
I have updated the repo with the changes I made to test this.

Btw looking at your project - you don't need to use @InjectRepository with custom repositories - as long as there is actual class, it works fine (@InjectRepository is needed because by default all repositories are implementations of EntityRepository).

I have been looking into the code and I have a hunch, but you are probably better at confirming or debunking this.
My idea is that currently the request scoped EM triggers the events. This request scoped EM is created and then instantiates the EventManager like this new EventManager(this.config.get('subscribers'));. The configuration does not contain the EventSubscribers that were registered using the lazy registration.

Yeah that sounds correct.

I think I can make a PR to fix this, but what way do you think it should be fixed? When forking the em the EventManager is not copied, but recreated and then it forgets all the lazy registered EventSubscribers. So I guess we have at least 2 approaches:

In the fork method we create a copy of the EventManager and set that as the EventManager of the new EM. I guess we need a new EventManager and not reuse the EventManager of the global EM, right?
Push the lazy loaded EventSubscribers to the subscribers config `config.get('subscribers') when registering.

Maybe there are other better solutions?

Thinking about it a bit, and I'd say it should be fine to even reuse the EventManager instance. All it does is to hold the list of subscribers, the actual (global) parameter values are held in the EntityManager (and even those are copied when forking the EM).
So it should be just about making the property writable and doing em.eventManager = this.eventManager; in the fork method. Feel free to send a PR.


Implement tracking of filter values

Implemented filter tracking courtesy of Maik's draft implementation in https://github.com/boxwise/boxtribute/pull/1364/files#diff-ef01c5e03077013a4f49a11bf62dba2f8d1a35268b3f6438aa8bb1b1b5346461.
As I couldn't test it locally due to errors in running the frontend, could @fhenrich33 take a peek at running this branch locally and see if the console logs log the correct filter values?
Note: There is a slight deviation between Maik's changes in shared-components/statviz/components/filter/TimeRangeSelect.tsx and mine, see original version here- https://github.com/boxwise/boxtribute/pull/1364/files#diff-ef01c5e03077013a4f49a11bf62dba2f8d1a35268b3f6438aa8bb1b1b5346461L7.

Hey @HaGuesto  - I pushed a change again - again it's from Maik's implementation. I hope this works this time!

functional review passed


Cannot make resource from --resource

Laravel Version: 4.0.4
PHP Version: 7.4.10
Database Driver & Version:

Description:
I'm new in laravel, in order to make controller and resource I use command PHP artisan make:controller UserController --resource
but, it just generate controller not with the resource.

The make:controller {name} --resource makes Resource Controller, not json resource(app/Http/Resources) and controller
https://laravel.com/docs/8.x/controllers#resource-controllers

The make:controller {name} --resource makes Resource Controller, not json resource(app/Http/Resources) and controller
https://laravel.com/docs/8.x/controllers#resource-controllers

is the folder app/Http/Resources is still empty. but if I run the command separately like make:resource it work to make resource


Color consistency

Colors and legends' organization is not consistent between National and Regional graphs
NATIONAL

REGIONAL

Also, the legend title is not consistent having "Misura" and "Misurazione" titles

Both issues fixed. Update online from this evening.


Add license to rust-central-station

rust-central-station doesn't currently have any Open Source licensing explicitly listed. We'd like to relicense rust-central-station to Rust's standard MIT OR Apache-2.0 license, so that we can freely copy code between it and other Rust projects. In order to safely do that, we would like explicit permission from the following people:

[ ] @Mark-Simulacrum
[x] @RalfJung
[x] @SimonSapin
[ ] @aidanhs
[ ] @alexcrichton
[ ] @aturon
[x] @davidtwco
[ ] @frewsxcv
[x] @kennytm
[x] @oli-obk
[x] @phansch
[x] @pietroalbini
[x] @thibaultdelor

If you're OK with us using your contributions under Rust's standard Open Source license, please post a comment copy-pasting the following:

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

We'll track approvals using the checkboxes above, and once we have all the boxes checked, we'll relicense the repository.
Thank you!

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

I agree to license my contributions to rust-central-station under Rust's standard MIT OR Apache-2.0 dual-license.

The repository is now licensed under "MIT OR Apache-2.0"! Thanks everyone :heart:


Mutually recursive function declarations

Resources:

https://okmij.org/ftp/Computation/fixed-point-combinators.html
https://okmij.org/ftp/ML/fixpoints.ml
https://ahnfelt.medium.com/type-inference-by-example-part-7-31e1d1d05f56

let fix_poly : (('a -> 'b) list -> 'a -> 'b) list -> ('a -> 'b) list
 = fun l -> fix (fun self l -> List.map (fun li x -> li (self l) x) l) l

I had implemented in #515, but I'm temporarily dropping the code so that I can port the rest of the escalier_infer's behaviours over to escalier_hm.


PR Project Automation is failing

Description

It looks like the "PR Project Automation" workflow is failing: https://github.com/WordPress/openverse/actions/workflows/pr_project_automation.yml
This workflow is supposed to archive columns automatically after the weekly development chat.
Reproduction
Most recent run as of the creation of this issue: https://github.com/WordPress/openverse/actions/runs/4115406067
Additional context

We also make note in the Weekly Dev Chat Facilitation Guide that folks should archive these columns. We could remove that line once this is functional again since it will now be done automatically. @dhruvkb do you recall a reason why we shouldn't also archive the "Done" column of project 3 automatically too?

@AetherUnbound I think the only reason we did not implement it in our issues board is because of the satisfaction associated with archiving the closed issues. It should be very straightforward to automate that as well if that's preferable to the team.


Grid - display issue (resizableColumns: true, column.sortable=true)

I would say there is a problem with the position of the sort up/ down icons.
They will placed ever under the titel of the colums even if there is enough space at the right side of the title.
The user would expect the sort icon on the right side of the title after the resize of the colmn and the width of the colum is big enough.
Drag & Drop should not lead to a sort of the column.
Sample:
<script>
     $('#grid').grid({
         uiLibrary: 'bootstrap4', // same problem with Bootrap 3
         iconsLibrary: 'fontawesome',
         resizableColumns: true,
         dataSource: '/Players/Get',
         columns: [
             { field: 'ID', width: 42 },
             { field: 'Name', sortable: true },
             { field: 'PlaceOfBirth', sortable: true}
         ]
     });
 </script>

I'm tested with FF.

We fixed that in the new development version. Thanks for reporting this issue.

thanks for the fix


GEOMESA-2430 Upgrade to arrow 0.16

Arrow allocators are not exposed, but tied to the lifecycle of each class

Signed-off-by: Emilio Lahr-Vivaz<EMAIL_ADDRESS>

rebased and ready for merge


Fixed #24385 -- Added tests for distinct sum issue fixed in c7fd9b242d2d63406f1de6cc3204e35aaa025233

Ticket reference: https://code.djangoproject.com/ticket/24385
Copied the test from the reporter's project. Validates this usage of Sum and distinct in the ORM.
Bisect shows that this was fixed in commit: c7fd9b242d2d63406f1de6cc3204e35aaa025233

Thanks. It would be better if we could reuse existing models for this patch since adding new models makes the test suite slower and increases its complexity. Could you give that a try? It seems like the day field isn't needed to reproduce the issue.

Okay. I will look for an existing test with suitable models and add the test there.

One clarification please - shall I add this test to an existing test suite, or reuse a model from another test? I'm not sure if referencing a model from a different test directory is a valid approach.

I would look in the existing aggregation tests (aggregation or aggregation_regress and try to find an appropriate model there) - the test would go in whatever app has the models you end up using.

I did checkout 1.7 tag and applied this test as a stash to insure it fails on old code.

Could you please squash your commits? Thanks.

Commits appear to have been squashed.

buildbot, add to whitelist.

merged in 910638fc4e4f83bda31dd91db5017acb06dfc8d2, thanks!


Replace readable_org_guids_for_domains with query

A short explanation of the proposed change:
Replace the readable_org_guids_for_domains function with one that returns a SQL subquery as a filter rather than an actual Ruby array.
An explanation of the use cases your change solves
We have seen extremely large query strings when an admin or a power-user in many orgs requests domains as the entire list of (sometimes >100k) guids is turned into a giant WHERE IN ('abc', 'def' etc.) SQL query.
This is expensive for Ruby to turn into the query, expensive to send over the network and expensive for the DB to process.
Databases are very very good at working out the most efficient way to run a query so even though the subqueries produced by this look pretty complex (due to each role being a separate table and space roles needing a join to the orgs table), the actual runtime is much better this way.
Links to any other associated PRs
https://github.com/cloudfoundry/cloud_controller_ng/pull/2255 - similar problem of huge query strings that may actually be better addressed by this approach
Comparison with performance tests
Using the tests at https://github.com/cloudfoundry-incubator/cf-performance-tests I ran (locally) a comparison of the /v3/domains endpoint with:

50,000 orgs
100,000 spaces
20,000 domains

The results seem pretty clear cut:

[x] I have reviewed the contributing guide

[x] I have viewed, signed, and submitted the Contributor License Agreement

[x] I have made this pull request to the main branch

[x] I have run all the unit tests using bundle exec rake

[ ] I have run CF Acceptance Tests

Hi @andy-paine, thanks for the PR!  Looking forward to merging this.
Unfortunately I think the merge of https://github.com/cloudfoundry/cloud_controller_ng/pull/2405 caused most of the merge conflicts with this PR.  I think most of those files were just deleted, though, I hope the resolution of those conflicts are relatively straightforward...

Thanks for the heads up @sethboyles. I've rebase this onto main now so should be good to go


Slow Query

Interesting PostGIS quizz.
This query is very slow (8s) yet it only hits the index. Turns out the st_pointonsurface operation is a very, very expensive one.
Analyze result.
 Index Scan using osm_water_polygons_geom on osm_water_polygons  (cost=0.41..4.28 rows=2 width=663) (actual time=7448.206..7448.749 rows=2 loops=1)
   Index Cond: (geometry && '0103000020110F00000100000005000000747186C57E8468C166ABC68241D16041747186C57E8468C1062F0D220CD36041D1ED3F26B48268C1062F0D220CD36041D1ED3F26B48268C166ABC68241D16041747186C57E8468C166ABC68241D16041'::geometry)
 Planning time: 0.869 ms
 Execution time: 7448.797 ms

EXPLAIN ANALYZE SELECT topoint(geometry) AS geometry, osm_id,
	  name,
	  area,
	  coalesce(NULLIF(name_en, ''), name) AS name_en,
	  coalesce(NULLIF(name_es, ''), name) AS name_es,
	  coalesce(NULLIF(name_fr, ''), name) AS name_fr,
	  coalesce(NULLIF(name_de, ''), name) AS name_de,
	  coalesce(NULLIF(name_ru, ''), name) AS name_ru,
	  coalesce(NULLIF(name_zh, ''), name) AS name_zh
	  FROM osm_water_polygons
	  WHERE geometry && ST_SetSRID('BOX3D(-12854262.17266152 8817164.086751651,-12850593.19530383 8820833.064109337)'::box3d, 3857) AND
	  (
	    (
	      area >= 100000000
	      AND z(34123.7) >= 10
	    )
	    OR (
	      area >= 40000000
	      AND z(34123.7) >= 11
	    )
	    OR (
	      area >= 20000000
	      AND z(34123.7) >= 12
	    )
	    OR (
	      area >= 10000000
	      AND z(34123.7) >= 13
	    )
	    OR z(34123.7) >= 14
	  );

We are currently try to solve it with a expression index
CREATE INDEX osm_water_polygons_geom_point ON osm_water_polygons (topoint(geometry));

PostgreSQL often can't optimize when there's a function in the query,
Enhancement ideas: Is this function necessary? And should'nt you use GISt index?

Is this function necessary?

We need to turn the polygon into a point for labelling it.
I thought creating the expression index should help speed up the select part because the point is already precalculated - but it turns out this doesn't help.
My guess is that calculating the point on surface for a very large polygon is expensive.

And should'nt you use GISt index?

The GISt is actually used to find out, whether the polygon is even in the boundary.
geometry && ST_SetSRID('BOX3D(-12854262.17266152 8817164.086751651,-12850593.19530383 8820833.064109337)'::box3d, 3857)

You may try to precalculate the centroids you need to label later on during the PLANET import and just after creating of the database - before you even start the cluster... see the "materialized view" in PostgreSQL - this way we can at least calculate the centroid for each polyton always only once - not for each tile which is overlapping it again and again...

Identified same problem on layer poi_label:
SELECT geometry, osm_id, ref, website,
     poi_network(type) as network,
     poi_address(housenumber, street, place, city, country, postcode) as address,
     name,
     coalesce(NULLIF(name_en, ''), name) AS name_en,
     coalesce(NULLIF(name_es, ''), name) AS name_es,
     coalesce(NULLIF(name_fr, ''), name) AS name_fr,
     coalesce(NULLIF(name_de, ''), name) AS name_de,
     coalesce(NULLIF(name_ru, ''), name) AS name_ru,
     coalesce(NULLIF(name_zh, ''), name) AS name_zh,
     format_type(type) AS type,
     scalerank_poi(type, area) AS scalerank,
     coalesce(classify_maki_label(type), 'marker') AS maki,
     rank() OVER (PARTITION BY LabelGrid(geometry, 128 * 9.55463)
                  ORDER BY localrank_poi(type) ASC) AS localrank
   FROM (
	SELECT * FROM (
		SELECT geometry, osm_id, ref, website,
			housenumber, street, place, city, country, postcode,
			name, name_en, name_es, name_fr, name_de, name_ru, name_zh,
			type, 0 AS area,
            timestamp
		FROM osm_poi_point
        UNION ALL
		SELECT topoint(geometry) as geometry, osm_id, ref, website,
			housenumber, street, place, city, country, postcode,
			name, name_en, name_es, name_fr, name_de, name_ru, name_zh,
			type, area,
            timestamp
        FROM osm_poi_polygon
    ) AS poi_geoms
   ) as t
   WHERE z(34123.7) = 14
     AND geometry && ST_SetSRID('BOX3D(942927.1809259342 6018345.859061636,947819.1507361856 6023237.828871887)'::box3d, 3857);

The topoint function gets executed for each osm_poi_polygon even these which are not in the bbox. We solve it by taking the topoint function out of the view into the query. Result: 1913 ms to 31 ms
SELECT topoint(geometry), osm_id, ref, website,
     poi_network(type) as network,
     poi_address(housenumber, street, place, city, country, postcode) as address,
     name,
     coalesce(NULLIF(name_en, ''), name) AS name_en,
     coalesce(NULLIF(name_es, ''), name) AS name_es,
     coalesce(NULLIF(name_fr, ''), name) AS name_fr,
     coalesce(NULLIF(name_de, ''), name) AS name_de,
     coalesce(NULLIF(name_ru, ''), name) AS name_ru,
     coalesce(NULLIF(name_zh, ''), name) AS name_zh,
     format_type(type) AS type,
     scalerank_poi(type, area) AS scalerank,
     coalesce(classify_maki_label(type), 'marker') AS maki,
     rank() OVER (PARTITION BY LabelGrid(geometry, 128 * 9.55463)
                  ORDER BY localrank_poi(type) ASC) AS localrank
   FROM (
	SELECT * FROM (
		SELECT geometry, osm_id, ref, website,
			housenumber, street, place, city, country, postcode,
			name, name_en, name_es, name_fr, name_de, name_ru, name_zh,
			type, 0 AS area,
            timestamp
		FROM osm_poi_point
        UNION ALL
		SELECT geometry as geometry, osm_id, ref, website,
			housenumber, street, place, city, country, postcode,
			name, name_en, name_es, name_fr, name_de, name_ru, name_zh,
			type, area,
            timestamp
        FROM osm_poi_polygon
    ) AS poi_geoms
   ) as t
   WHERE z(34123.7) = 14
     AND geometry && ST_SetSRID('BOX3D(942927.1809259342 6018345.859061636,947819.1507361856 6023237.828871887)'::box3d, 3857);

Learning: Views are good to make the queries more readable. Expensive functions like topoint should be executed on the smallest set of geometries possible.


🐛 [Bug] source code compile error

Bug Description
bazel compile error.

To Reproduce
Steps to reproduce the behavior:

change WORKSPACE

2.change tensort (fix address problem)

change cudnn (fix address problem)

compile
use gcc-7.3.0
bazel build //:libtorchtrt --compilation_mode opt

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

Torch-TensorRT Version:1.1.0
LibTorch Version:1.9.0 (compiled with source-code(v1.9.0) by myself)
OS:linux
CUDA version:cuda10.2
GPU models and configuration:1/2 T4
TensorRT: tensorRT-<IP_ADDRESS>
Cudnn: v<IP_ADDRESS> (already copy to cuda directory)

Additional context

The compilation error is due to torch API update in 1.10.0
you are using older version of libtorch with torch-tensorrt v1.1.0. Please refer to the release page for correct dependency version

If you are attempting to use old version of libtorch as mentioned in #1047  you need to backport 1.1.0 code base.

If you are attempting to use old version of libtorch as mentioned in #1047 you need to backport 1.1.0 code base.

OK, thanks


Semantic token for destructor name does not cover the ~ character

This problem has existed for many years...
But I have never seen similar issues before. Why this problem only appearing on vsc?

System information
Clangd version (from the log, or clangd --version): 18.1.5 (but I think most versions all have this problem)
clangd extension version: 0.1.28
Operating system: win11

(I've moved the issue to the clangd repo since the change discussed in the issue title needs to be made in the clangd server.)

Note that clangd does attach a distinct modifier called constructorOrDestructor to the semantic tokens for the constructor and destructor.
So, it is possible to get vscode to color them differently today by adding something like this to vscode settings:
"editor.semanticTokenColorCustomizations: {
  "rules": {
    "*.constructorOrDestructor": {
      "foreground": "#D8D8A7"
    }
  }
}

(replacing "#D8D8A7" with the color you want).

Thanks for your help : )
In the past I had no choise (because I don't know this way), so I changed color of  ~ to color of class ...
And another problem is the color of operators delete and new are also incorrect.
I think the color of delete and new should be the same as [].
I'll try to find the similar way to solve it : (

So the reason for these issues is that the tokens provided by clangd server and VSC tokens are incompatible?
If the developers want to fix them, they must change the server instead of this plug?

And another problem is the color of operators delete and new are also incorrect. I think the color of delete and new should be the same as []. I'll try to find the similar way to solve it : (

These tokens get a semantic token type of operator, so their color can be customized with:
"editor.semanticTokenColorCustomizations: {
  "rules": {
    "operator": {
      "foreground": "..."
    }
  }
}

Note that you can use the VSCode command "Developer: Inspect Editor Tokens and Scopes" to see the semantic token type (and modifiers, if any) for any token in the code.

So the reason for these issues is that the tokens provided by clangd server and VSC tokens are incompatible? If the developers want to fix them, they must change the server instead of this plug?

I think of client-side tokens and server-side tokens as being complementary.

Client-side tokens color basic syntax like strings, keywords, numeric literals, etc.
Server-side tokens provide semantic information that allow identifier tokens to be colored differently based on whether they name a local variable, parameter, class member, etc.

If a token is colored by both the client and the server, the server color takes precedence.
I personally find that sometimes, vscode's default client-side coloring gets in the way (because it tries to also color identifiers which will be colored by the server anyways). There is an alternative client-side coloring engine in this extension which I find works better.


Dose composer support best checkpoint saver which can monitoring the checkpoint for best metrics or losses?

🚀 Feature Request

From the Documentation in Checkpointing part, I didn't find there is a best checkpoint saver function supported.
best checkpoint saver refers to the function that monitors the user-specified metrics or losses. It will save the best checkpoint for such particular best metrics or losses during the training process.
So I'm wondering if such a function already supports it or not.
In Keras, you can use the EarlyStopping function to achieve such needs.
https://stackoverflow.com/questions/48285129/saving-best-model-in-keras
Motivation

It would be convenient to have the best checkpoint saver function, in such case that I want to have the best checkpoint for a particular metric or loss. For example, for the Denoising task, I want to have the best checkpoint for PSNR metric and another best checkpoint for SSIM metric.
[Optional] Implementation
CheckpointSaver class in composer.

Additional context

This is a great suggestion!
We currently do have an EarlyStopping callback (see docs). Unfortunately, at this time we don't have a way to save best checkpoint, but we will add it to our roadmap.

I had just mentioned this issue on the mosaic slack here last week :).

is there any updates on this? this is a must-have feature.

Hi @mvpatel2000, we have a custom best checkpoint saver but the issue is how to load the best checkpoint before evaluating, this means after the fit but before evaluation.
Using the tooling provided by Composer, it does not seem to work in DDP. We are currently using checkpoint.load_checkpoint and we have tested to load it in a single rank or broadcast the checkpoint to several ranks without luck. Wrapping the model with the DistributedDataParallel, changes the keymapping.
We have not check with deepspeed nor FSDP.

Hi @mvpatel2000, we have a custom best checkpoint saver but the issue is how to load the best checkpoint before evaluating, this means after the fit but before evaluation.
Using the tooling provided by Composer, it does not seem to work in DDP. We are currently using checkpoint.load_checkpoint and we have tested to load it in a single rank or broadcast the checkpoint to several ranks without luck. Wrapping the model with the DistributedDataParallel, changes the keymapping.
We have not check with deepspeed nor FSDP.

@priba would you mind sharing a code example please? We're happy to take a look (and ideally we can add a unit test to ensure this works)

Hi @mvpatel2000, thanks for your answer, here are some snippets of the code we are using.
class BestCheckpointSaver(Callback):

    ...

    def fit_end(self, state: State, logger: Logger) -> None:
        if not self.save_last:
            return

        self.save_checkpoint("last" + DEFAULT_CHECKPOINT_EXTENSION, state, logger, None)

    def save_checkpoint(self, filename: str, state: State, logger: Logger, metric: Any) -> Path | None:
        destination = format_name_with_dist_and_time(
            os.fspath(Path(self.folder) / filename), state.run_name, state.timestamp
        )
        saved_path = checkpoint.save_checkpoint(state, destination, weights_only=self._weights_only)

        if saved_path is None:  # not all ranks save
            return None

        self.saved_checkpoints.append((Path(saved_path), state.timestamp.copy(), metric))
        log.debug(f"Uploading checkpoint to {destination} ...")
        logger.upload_file(remote_file_name=destination, file_path=saved_path)
        return Path(saved_path)

Then just before the eval call, we manually use a new entry point into the trainer:
class Trainer:
    ...

    def load_best_checkpoint(self) -> None:
        """Loads the best checkpoint as tracked by BestCheckpointSaver.

        Example:
            ```
            trainer = Trainer(
                model=model,
                max_duration="100ep",
                callbacks=[BestCheckpointSaver("cross-entropy", mode="min")],
                ...
            )
            trainer.fit(...)
            trainer.load_best_checkpoint()
            trainer.eval(...)
            ```
        """

        if is_model_ddp(self.state.model) or is_model_deepspeed(self.state.model) or is_model_fsdp(self.state.model):
            log.warn(
                "`load_best_checkpoint` is not implemented for DDP, DeepSpeed and FSDP. "
                "Last weights will be used. Run test independently to load the best model."
            )
            return

        checkpoint_savers = [callback for callback in self.state.callbacks if isinstance(callback, BestCheckpointSaver)]

        if len(checkpoint_savers) == 0:
            raise ValueError("A `BestCheckpointSaver` callback must be provided to call this method.")

        if len(checkpoint_savers) > 1:
            log.warning(f"Several `BestCheckpointSaver` were provided. Evaluating with {checkpoint_savers[0]!r}.")

        ckpt = checkpoint_savers[0].best_checkpoint
        if ckpt is None:
            raise ValueError("No best checkpoint was found. Have you ran `.fit()` with an `eval_dataloader`?")

        checkpoint.load_checkpoint(os.fspath(ckpt[0]), self.state, self.logger)

We have tried different combinations such as running in rank 0, brodcasting the checkpoint path and even brodcasting the model weights but we didn't manage to make it work in DDP.

@priba looking into this... it would also be helpful if you could elaborate on what "it does not seem to work in DDP" means. Could you provide an error trace or some kind of description? This design seems more or less correct to me...

Hi @mvpatel2000 , the issues we have are:

As it is, ranks besides 0 will not have a checkpoint path and the Value error in ckpt is None will be raised.
In case we just load the checkpoint in rank 0, it just runs into a deadlock
In our case, we brodcasted the string ckpt to all ranks so we can load it everywhere, however, the keys of the state dict are not matching as DDP adds a module.*. I've seen that this should be removed inside the load_checkpoint but I am not sure why it is not happening.

@priba

As it is, ranks besides 0 will not have a checkpoint path and the Value error in ckpt is None will be raised.
In case we just load the checkpoint in rank 0, it just runs into a deadlock
In our case, we brodcasted the string ckpt to all ranks so we can load it everywhere,
Only rank 0 saves the checkpoint to avoid duplicate work. But, all ranks need to load the checkpoint (with DDP, if you use FSDP we support a path for broadcasting). If you share a filesystem, broadcasting name should be sufficient as you described
however, the keys of the state dict are not matching as DDP adds a module.*. I've seen that this should be removed inside the load_checkpoint but I am not sure why it is not happening.
This is an ugly pytorch detail -- they add an extra level of module. when using DDP for reasons I am not familiar with. Composer should appropriately autostrip these on state_dict calls to state, but maybe this codepath fails. Could you share the exact error?

Torch 2.3 exposes a new API from torch get_model_state_dict which also handles this internally for PyTorch. So, when we release next week with 2.3 support and the new APIs, it might just go away for you

Thanks for the info @mvpatel2000
If we broadcast the string it does not throw any error message. It just skips loading the model as the keys are not there and throws a warning with the set of ignored keys. I don't think this is very informative for your case.
I will check once support for  2.3 is released and report back to you. Thank you so much

If we broadcast the string it does not throw any error message. It just skips loading the model as the keys are not there and throws a warning with the set of ignored keys. I don't think this is very informative for your case.

Ah, I think this is a bug. If you load from Trainer, it will default raise an error if the keys are not there (load_strict_model_weights defaults to True). However, if you call load_checkpoint directly, it defaults to False whereas it should be consistent. I've opened a PR to fix this! https://github.com/mosaicml/composer/pull/3219

I will check once support for 2.3 is released and report back to you. Thank you so much

Great! Please let me know if you encounter any errors after the release :)

@priba Hm... your snippet seems to work for me:
train          Epoch   0:  100%|_________________________| 25/25 [00:04<00:00,  6.76ba/s, loss/train/total=2.1885]                                                                                /mnt/workdisk/mihirp/composer/composer/core/data_spec.py:37: UserWarning: Cannot split tensor of length 904 into batches of size 1024. As it is smaller, no splitting will be done. This may happen on the last batch of a dataset if it is a smaller size than the microbatch size.
  warnings.warn(

eval           Epoch   0:  100%|_________________________| 5/5 [00:00<00:00,  6.62ba/s, metrics/eval/CrossEntropy=2.0713, metrics/eval/MulticlassAccuracy=0.2756]
train          Epoch   1:  100%|_________________________| 25/25 [00:03<00:00,  6.36ba/s, loss/train/total=1.9583]
eval           Epoch   1:  100%|_________________________| 5/5 [00:00<00:00,  6.92ba/s, metrics/eval/CrossEntropy=1.7521, metrics/eval/MulticlassAccuracy=0.3656]
eval           Epoch   0:  100%|_________________________| 5/5 [00:00<00:00,  7.07ba/s, metrics/eval/CrossEntropy=2.0718, metrics/eval/MulticlassAccuracy=0.2750]

Can you double check you are on Composer v0.22 and torch 2.3?

Great to hear! Please let me know if you encounter any further issues.

@mvpatel2000 out of curiosity, may I ask you which GPUs did you use to run this test? Its much faster than what I got. Are these H100?

I was debugging this on 2xA100

Hi @mvpatel2000 ! Thanks for providing feedback and insights into the the latest feature in PyTorch.
We are also using 2xA100 with the same code snippet than shared above but our throughput is quite a bit lower, around ~4.1 ba/sec when you reach >6.6ba/sec. We are using A100 with 40GB of vRAM with CUDA 12.1 and the latest version of PyTorch. Are you using A100 with that amount of vRAM or GPUS with 80GB? Would you mind sharing the output of composer_collect_env so I can compare?


/query endpoint not returning conversational text

The APP is deployed with Digital Ocean with Pinecone as a vector database. When the following is performed:
{
   "queries":[
      {
         "query": "What is banana",
         "top_k": "1"

      }
   ]
}

The results returned are not conversational from the OPENAI perspective when a POST request is sent to the endpoint; the /query only shows results that are similar to the search query.
What could be the issue?

same to me

@Katlego-Chagane That's exactly the intended behavior of this plug-in/API. The endpoint is intended to be called by ChatGPT when it needs information about a given topic as defined by your plug-in. ChatGPT will then use the response from the query api to create a conversational response to the user within the chat window.

Sorry, but can you clarify this? What's the use case of the query API if it only returns the top 3 embeddings and doesn't call ChatGPT? Is this package supposed to be part of some larger project using for example langchain?

This isn't a chatbot. This is an example plugin for ChatGPT. When a plugin like this is enabled, ChatGPT will determine if/when to send the end-users query to the /query endpoint. This plugin will return relevant document data to ChatGPT, and ChatGPT will then formulate a response.
If you want to make your own custom chatbot using similar methodology to this (embeddings and vector DBs), look at LangChain to get started.

I thought this is an API for the ChatGPT retrieval plugin, but looks like I was mistaken. So what this package does is that it enables very easy document upload. But how do I combine this with ChatGPT? Let's say I want to create a ChatGPT powered app that allows retrieval. My understanding is:

Have this package running on a server.
Create another app that is calling ChatGPT
Decide when to call the query API from the other app???
Thanks for any help.

This package includes APIs for you to add documents to your vector database, as well as the ChatGPT API endpoint for the plugin. No other custom apps are needed for this.
Are you looking to use ChatGPT (chat.openai.com) or just GPT API on your own site/app? They are separate, this is for ChatGPT Plugins. What's your end goal?

This package includes APIs for you to add documents to your vector database, as well as the ChatGPT API endpoint for the plugin. No other custom apps are needed for this.
Are you looking to use ChatGPT (chat.openai.com) or just GPT API on your own site/app? They are separate, this is for ChatGPT Plugins. What's your end goal?

Oh yeah, then I got it all wrong. I am planning to build an app using the ChatGTP API. I don't care about the ChatGPT UI at chat.openai.com. I thought this package is a supplement to langchain, but looks like I was mistaken. Thanks for clarification.


Broken link at Volume/iSCSI example, issue related to githubbranch param

This is a Bug Report

Problem:
On concept/storage document, the iSCSI example link is not working properly on 1.21 version. Because the link is broken. When clicking the iSCSI example, the document of the latest version redirects the url(https://github.com/kubernetes/examples/tree/main/volumes/iscsi
). And the document of release-1.21 version redirects the url(https://github.com/kubernetes/examples/tree/release-1.21/volumes/iscsi).

Proposed Solution:
the parameter "githubbranch" of the original document(docs/concepts/storage/volumes.md) occur this problem because the kubernetes/examples repository doesn't have main and release-1.21 branch.

docs/concepts/storage/volumes.md

See the [iSCSI example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/volumes/iscsi) for more details.

This issue has been solved partially by #29311.(This PR change {{<param "githubbranch"}} to master)
But, I think that the "githubbranch" parameter is changed according to the documentation branch in order to support minor version documentation.
Page to Update:
https://kubernetes.io/docs/concepts/storage/volumes/#iscsi

As a bug I think this was fully fixed in PR #29311
The examples repo doesn't have tags per release that we can hyperlink to.

/remove-kind bug
/priority awaiting-more-evidence
/language en

@ClaudiaJKang you might be requesting a backport of the fix from #29311 to the v1.21 docs
If that's what you want to happen, I recommend opening an issue framed around that backport request.


OOM error on exporting to MP4

I have around 125 images that i wish to convert to mp4 at 25fps in the browser. But the library fails saying OOM

how large are the images?
you might try exporting as webm (uses less memory) and then converting to mp4 separately on your computer.
see: https://github.com/amandaghassaei/canvas-capture#additional-notes

I use this method provided by this library
CanvasCapture.beginVideoRecord({ format: CanvasCapture.MP4 });

Which i see first converts to webm and then uses ffmpeg.wasm to convert to mp4

yeah I think if you export as webm instead and then use another tool to convert to mp4 you might have better luck.

see: https://github.com/amandaghassaei/canvas-capture#converting-webm-to-other-formats


for dataset

can you give me dataset for Adobe Deep Image Matting Dataset<EMAIL_ADDRESS>! thanks

can you give me dataset for Adobe Deep Image Matting Dataset<EMAIL_ADDRESS>thanks

I am not authorized to redistribute it, please contact the authors.


Some hosts missing Elasticsearch23 AWS Plugin

Hi, I've created a number elasticsearch23 service instances in GovCloud after PR #31 was merged in order to take advantage of the S3 snapshot utility. I noticed that some of the hosts don't appear to have been updated to reflect the added AWS Cloud plugin (or perhaps are having trouble installing the plugin).
Specifically, service instances on the 10.100.98.x block don't have the plugin listed when i call:
<svc_proxy>/_nodes/plugins
However, when a service instance is created on 10.100.36.x, this block does appear to have the AWS Cloud plugin available.

Closing because ES23 is no longer offered


Change session_info name, clashes with devtools::session_info

https://github.com/USGS-R/sbtools/blob/master/R/session_info.R i guess i made this - woopsy - what about session_details() ?

Works for me!

cooliio


Add async function support to beforeEach and afterEach

You now can write:
beforeEach(async () => {
  await somethingAsync();
});

Fix #1256

Waiting for unit tests to run on Travis since everything fails locally.

This is great! What about beforeAll/afterAll?
What is wrong locally? npm install and npm test should work.

What about beforeAll/afterAll?

I don't use them :P

What is wrong locally?

70 tests fail like
 FAIL  packages/jest-runtime/src/__tests__/Runtime-requireModuleOrMock-transitive-deps-test.js
  ● Test suite failed to run

    TypeError: Path must be a string. Received undefined

      at assertPath (path.js:7:11)
      at Object.dirname (path.js:1326:5)
      at Array.map (native)

Master fails too so I figured I'd just use Travis.

Master is passing for me and on travis. Seems like an issue with your node version?

I'm on Node 6.3.1 with npm 3. kpm didn't work for me so I had to use npm:
$ kpm install
kpm install v0.10.0
success Read lockfile kpm.lock
[1/3] 🚚  Resolving and fetching packages...
error The pattern<EMAIL_ADDRESS>not found in lock file

Current coverage is 90.27% (diff: 100%)

Merging #1546 into master will not change coverage

@@             master      #1546   diff @@
==========================================
  Files            28         28
  Lines          1121       1121
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           1012       1012
  Misses          109        109
  Partials          0          0

Powered by Codecov. Last update 6fa710c...d3609c7

thank you!


Add no_sync support, fix gradient accumulation, and logging and argument improvements

This PR adds FSDP no_sync support, which doesn't synchronize gradients until the gradient accumulation step. It also fixes gradient accumulation by truncating the dataset length and correcting the modulus comparison. It adds improved logging compatibility with tqdm and updates the readme and arguments.

LGTM


SettingsButton support

Telegram introduced the SettingsButton starting from version 7.0, as stated here: https://core.telegram.org/bots/webapps#settingsbutton
It would be great if this library supported it.

It's done :)

@ArthurStam Thank you


docs: 📚 template for syn-file

Description

Rationale

Acceptance criteria

duplicate of #569
closing this because in the other ticket is more content


"local" parameter only in compatible cat requests

Followup of #3059, removing local from common cat parameters and adding it only to those requests which allow it.
I can't test it with java because it has its own handcrafted test types, but it seems straightforward enough.

💚 All backports created successfully

Status
Branch
Result

✅
8.x

Questions ?
Please refer to the Backport tool documentation

💚 All backports created successfully

Status
Branch
Result

✅
8.17

Questions ?
Please refer to the Backport tool documentation


Shphschool

@alfredzimmer First stage of verification passed, official domain is shphschool.com as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@alfredzimmer The website of the school seems to be blocked from access from abroad. I cannot verify the school email domain eligibility for this repository.
Please apply for free educational license with an official document which proves that you are a student. You can do it using "OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students.
I have to close the request without merging.
Thank you for your understanding!


Added support for object alpha maps

Objects can now have an alpha map.
In case both the object and its material have alpha maps, they are combined.
The changes are backward compatible.

Added an alpha accessor to ShadingPoint that can be reused later to support curves and meshes per vertex alpha.

Refactored alpha handling code a bit.

Does not include the changes needed to set object alpha maps from appleseed.studio.

Wow, that was quite a bit of work :) Great job as usual.


Update README.md - configuration typo

thanks!

@prydonius you're welcome !
And there are two things to fix in the readme (I guess):
We can found:
TL;DR;

$ helm repo add monocular https://helm.github.io/monocular
$ helm install monocular/monocular

and
Installing the Chart

First, ensure you have added the Monocular chart repository:

$ helm repo add monocular https://kubernetes-helm.github.io/monocular

To install the chart with the release name my-release:

$ helm install --name my-release monocular

Two differents repos and two commands (and the second one is broken)

Codecov Report

Merging #405 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #405   +/-   ##
=======================================
  Coverage   86.84%   86.84%
=======================================
  Files          25       25
  Lines        1285     1285
=======================================
  Hits         1116     1116
  Misses        126      126
  Partials       43       43

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37fb830...c32da49. Read the comment docs.

@arthur-c oh you're right, the correct one is kubernetes-helm/monocular. helm install --name my-release monocular should also be helm install ... monocular/monocular


redisson cannot close

my application is jar project not web application project , init redisson run my task, but redisson cannot shutdown.
redisson have Daemon thread?
how ido to shutdown .when my task run over.?

You should shutdown it by Redisson.shutdown() method.

You should shutdown it by Redisson.shutdown() method.

need manual do  redisson.shutdown()?
@Bean(destroyMethod = "shutdown")
public RedissonClient redisson(){
}

@zhuyaoming
yes


Revise DefaultExchangeHolder

DefaultExchangeHolder should be revisited. It would be nice to use  json/avro/protobuf instead of serialization.
See https://github.com/apache/camel-quarkus/pull/2904#issuecomment-880483809

Sorry this is meant for Camel, closing.


Fix parameter name of IMessage::setHtmlPart according to docComment

Currently the comment and signature of IMessage::setHtmlPart is
    /**
     * [...]
     * @param string $contentTypeCharset the charset to use as the text/html
     *        part's content-type header charset value.
     */
    public function setHtmlPart($resource, $charset = 'UTF-8');

Symfony's DeprecationErrorHandler thinks that in the next main version of the package there will be a new parameter for this function:

Remaining indirect deprecation notices (1)
1x: The "ZBateson\MailMimeParser\Message::setHtmlPart()" method will require a new "string $contentTypeCharset" argument in the next major version of its interface "ZBateson\MailMimeParser\IMessage", not defining it is deprecated.

So I propose to change the parameter's name in the interface according to the name in the php doc.

@zbateson ping

Thanks @naitsirch


Can change chunksize per plugin

Since https://github.com/AxFoundation/strax/pull/277 it's possible to optimize the chunksize per plugin. This raises the default to 200 MB (uncompressed) but we could also use another value depending on the type of plugin.
For example, we can make a diagnostic plugin like pulse_counts very small.

There seems to have been little reason to implement this, on a large scale. We have some plugins that now (occasionally) use this


Task/opui 656 track session

ugh, sorry, did it again.  don't know why github makes the upstream the default to merge into :/


Fix for fractional container width rounding

element.clientWidth rounds the width value.
If the width is rounded up, then rowGrid.js overestimates how much
space it has and the last image can wrap to its own row.
This fix uses getBoundingClientRect() to get the true width of the
container. The width property of the DOMRect object wasn’t included
until IE9, so for compatibility with IE8 this fix computers the width
using the right and left properties.

That's great. Thank you for the pull request.


Updating links

@dend can you please merge this PR?

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-services/cosmosdb.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

#sign-off

@dend ping! can you please take a look at this PR.

approved changes to conceptual reference docs. Thanks @SnehaGunda !


please update the library

There are some new icons added to tabler-icons.io. we can't access those new icons

updated to tabler icons 2.23.0


Missing sourcemap flag

The build task for debugging should add --source-map to the build command, otherwise debugging goes in dist

Ah, good catch. When this was published, sourceMap was true by default. Since then, brighterscript has changed that to be false, so any new installs would cause this to generate no sourcemaps.


fix: tests throwing on hidden MacOS DS_Store directory

MacOS will regularly have a directory that interferes with the superblock directory causing the tests to fail.
I decided to return null instead of throwing an error would suffice when getting the superblocks. The tests seem to be able to check if a superblock is missing.
Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

@ojeytonwilliams  I have updated the proposed changes. Let me know what you think.

It's a bit tricky for me to check myself, so maybe @Nirajn2311 is the best person to review this. That said, my understanding was that .DS_Store is a file, not a directory, so it doesn't make sense that we would check if a directory is .DS_Store. Surely that's impossible.

Niraj's statement is not correct according to Wikipedia.  But that's beside the point, we are not checking IF it is a directory, we are supplying a path that is checked.

It's a bit tricky for me to check myself, so maybe @Nirajn2311 is the best person to review this. That said, my understanding was that .DS_Store is a file, not a directory, so it doesn't make sense that we would check if a directory is .DS_Store. Surely that's impossible.

 Not a directory. That being said, I do not own a Mac.

I can't test this properly. With this implementation the tests will pass.

Here's the python script to generate the .DS_Store file
import os

def create_file_in_each_child(filetree_path, filename):
    """
    Creates a new file with the specified filename in every child directory within the given filetree path.

    Args:
        filetree_path (str): The root directory of the filetree.
        filename (str): The name of the file to create in each child directory.
    """
    for root, _, _ in os.walk(filetree_path):
        filepath = os.path.join(root, filename)
        with open(filepath, 'w') as f:
            f.write("This file was created in directory: " + root)

And here's the bash command to delete the generated files after testing
find . -name ".DS_Store" -type f -delete


csrf_token issue during or after submission

while the user submit the form, the following error appears:
Forbidden (403)
CSRF verification failed. Request aborted.
Help
Reason given for failure:
CSRF token missing or incorrect.

In general, this can occur when there is a genuine Cross Site Request Forgery, or when Django's CSRF mechanism has not been used correctly. For POST forms, you need to ensure:
Your browser is accepting cookies.
The view function passes a request to the template's render method.
In the template, there is a {% csrf_token %} template tag inside each POST form that targets an internal URL.
If you are not using CsrfViewMiddleware, then you must use csrf_protect on any views that use the csrf_token template tag, as well as those that accept the POST data.
The form has a valid CSRF token. After logging in in another browser tab or hitting the back button after a login, you may need to reload the page with the form, because the token is rotated after a login.
You're seeing the help section of this page because you have DEBUG = True in your Django settings file. Change that to False, and only the initial error message will be displayed.
You can customize this page using the CSRF_FAILURE_VIEW setting.
------------------------------------------------------------+
I tried many way to solve it, but it seams deeper than what I expected, "BTW I toke steps well, following your installation documentations"

while the user submit the form, the following error appears:
Forbidden (403)
CSRF verification failed. Request aborted.
Help
Reason given for failure:
CSRF token missing or incorrect.

In general, this can occur when there is a genuine Cross Site Request Forgery, or when Django's CSRF mechanism has not been used correctly. For POST forms, you need to ensure:
Your browser is accepting cookies.
The view function passes a request to the template's render method.
In the template, there is a {% csrf_token %} template tag inside each POST form that targets an internal URL.
If you are not using CsrfViewMiddleware, then you must use csrf_protect on any views that use the csrf_token template tag, as well as those that accept the POST data.
The form has a valid CSRF token. After logging in in another browser tab or hitting the back button after a login, you may need to reload the page with the form, because the token is rotated after a login.
You're seeing the help section of this page because you have DEBUG = True in your Django settings file. Change that to False, and only the initial error message will be displayed.
You can customize this page using the CSRF_FAILURE_VIEW setting.
------------------------------------------------------------+
I tried many way to solve it, but it seams deeper than what I expected, "BTW I toke steps well, following your installation documentations"

Still waiting the answer please..

I think the problem is that the middleware is not passing csrf input to the block's template, while the block uses the customized template in the block.py of an app, rather than the iteration in the targeted template of a certain page model, I used the iteration in the new created project, started working properly, but, while I do same steps in a complicated project that is already live (production mode), it raises same error mentioned in this issue, will try to do investigation very deeply, will share the updates

it turned out that I was using multi-columns template file without considering to use include_block tag, therefore, if any of you faced same or more complicated issue about the CSRF, you can check wherever you have an itterable blocks object, and use the include_block  tag inside the loop, especially if you have nested blocks inside each others


Push notification: How to handle onClick ?

Short description of the problem:
I have setup the Ionic Push Notifications. Everything works fine. The notification is properly received when the app is off or background.
When i click on it I am redirected to a specific page that i setup in the onNotification, and works.
The problem is that, when the app is in the foreground, the notification is not received (and that's right) BUT the onNotification is still applied, thus I am redirected to that page EVERYTIME the cURL request is made.
What behavior are you expecting?
I expect a kind of onClick callback function that is called ONLY when i actually click on the notification.
Steps to reproduce:

Setup the callback function inside the onNotification (see code below)

var push = new Ionic.Push({

		"debug": true,
		"onNotification": function(notification)
		{
			$rootScope.get_my_account_data();
			$location.path('app/attendee_list');
		},
		"onRegister": function(data)
		{
			console.log(data.token);
		},

	});

Other information:
There is another place where I should setup the callback for onClick?
Which Ionic Version? 1.x or 2.x
Ionic Framework Version: 1.1.1
Run ionic info from terminal/cmd prompt: (paste output below)
Cordova CLI: 6.3.1
Gulp version:  CLI version 3.9.1
Gulp local:   Local version 3.9.1
Ionic Framework Version: 1.1.1
Ionic CLI Version: 1.7.16
Ionic App Lib Version: 0.7.3
ios-deploy version: Not installed
ios-sim version: Not installed
OS: Mac OS X El Capitan
Node Version: v0.12.7
Xcode version: Xcode 7.3.1 Build version 7D1014

This issue was moved to driftyco/ionic-cloud-issues#176

Hello! Because this issue was an issue related to the Ionic cloud and not the framework I have moved this issue to the appropriate repo. Thanks for using Ionic!


Implement Record Summaries.

This implements Record Summaries in both the API and Watcher.
Record Summaries allow for clients to associate simplified information
to their Results. The purpose of this is to allow Results to capture
basic high level information such as status and start/end times without
needing to perform an additional API call.
The API does not enforce that the data must match the underlying
Records, because how the data maps to the Records is type specific.
Clients are responsible for ensuring freshness. Cluster Watcher handles
this for Tekton Pipeline/TaskRun types today.
Fixes #147

/lgtm

/kind feature


Could you improve "Class Based"?

?

You have a "class based" section in your settings list. Here I propose an innovation. Make bleeding after stepping out of a pocket dimension of SCP-106.
scp106_bleeding_duration: 9
scp106_bleeding_damage: 3
scp106_bleeding_health: 2


chore(deps): update module google.golang.org/grpc to v1.56.3 [security]

This PR contains the following updates:

Package
Type
Update
Change

google.golang.org/grpc
require
minor
v1.48.0 -> v1.56.3

GitHub Vulnerability Alerts
GHSA-m425-mq94-257g
Impact
In affected releases of gRPC-Go, it is possible for an attacker to send HTTP/2 requests, cancel them, and send subsequent requests, which is valid by the HTTP/2 protocol, but would cause the gRPC-Go server to launch more concurrent method handlers than the configured maximum stream limit.
Patches
This vulnerability was addressed by #6703 and has been included in patch releases: 1.56.3, 1.57.1, 1.58.3.  It is also included in the latest release, 1.59.0.
Along with applying the patch, users should also ensure they are using the grpc.MaxConcurrentStreams server option to apply a limit to the server's resources used for any single connection.
Workarounds
None.
References
#6703
CVE-2023-44487
HTTP/2 Rapid reset attack
The HTTP/2 protocol allows clients to indicate to the server that a previous stream should be canceled by sending a RST_STREAM frame. The protocol does not require the client and server to coordinate the cancellation in any way, the client may do it unilaterally. The client may also assume that the cancellation will take effect immediately when the server receives the RST_STREAM frame, before any other data from that TCP connection is processed.
Abuse of this feature is called a Rapid Reset attack because it relies on the ability for an endpoint to send a RST_STREAM frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly resets the request. The request is canceled, but leaves the HTTP/2 connection open.
The HTTP/2 Rapid Reset attack built on this capability is simple: The client opens a large number of streams at once as in the standard HTTP/2 attack, but rather than waiting for a response to each request stream from the server or proxy, the client cancels each request immediately.
The ability to reset streams immediately allows each connection to have an indefinite number of requests in flight. By explicitly canceling the requests, the attacker never exceeds the limit on the number of concurrent open streams. The number of in-flight requests is no longer dependent on the round-trip time (RTT), but only on the available network bandwidth.
In a typical HTTP/2 server implementation, the server will still have to do significant amounts of work for canceled requests, such as allocating new stream data structures, parsing the query and doing header decompression, and mapping the URL to a resource. For reverse proxy implementations, the request may be proxied to the backend server before the RST_STREAM frame is processed. The client on the other hand paid almost no costs for sending the requests. This creates an exploitable cost asymmetry between the server and the client.
Multiple software artifacts implementing HTTP/2 are affected. This advisory was originally ingested from the swift-nio-http2 repo advisory and their original conent follows.
swift-nio-http2 specific advisory
swift-nio-http2 is vulnerable to a denial-of-service vulnerability in which a malicious client can create and then reset a large number of HTTP/2 streams in a short period of time. This causes swift-nio-http2 to commit to a large amount of expensive work which it then throws away, including creating entirely new Channels to serve the traffic. This can easily overwhelm an EventLoop and prevent it from making forward progress.
swift-nio-http2 1.28 contains a remediation for this issue that applies reset counter using a sliding window. This constrains the number of stream resets that may occur in a given window of time. Clients violating this limit will have their connections torn down. This allows clients to continue to cancel streams for legitimate reasons, while constraining malicious actors.

Release Notes

grpc/grpc-go (google.golang.org/grpc)
v1.56.3: Release 1.56.3
Compare Source
Security

server: prohibit more than MaxConcurrentStreams handlers from running at once (CVE-2023-44487)
In addition to this change, applications should ensure they do not leave running tasks behind related to the RPC before returning from method handlers, or should enforce appropriate limits on any such work.

v1.56.2: Release 1.56.2
Compare Source

status: To fix a panic, status.FromError now returns an error with codes.Unknown when the error implements the GRPCStatus() method, and calling GRPCStatus() returns nil. (#6374)

v1.56.1: Release 1.56.1
Compare Source

client: handle empty address lists correctly in addrConn.updateAddrs

v1.56.0: Release 1.56.0
Compare Source
New Features

client: support channel idleness using WithIdleTimeout dial option (#6263)

This feature is currently disabled by default, but will be enabled with a 30 minute default in the future.

client: when using pickfirst, keep channel state in TRANSIENT_FAILURE until it becomes READY (gRFC A62) (#6306)
xds: Add support for Custom LB Policies (gRFC A52) (#6224)
xds: support pick_first Custom LB policy (gRFC A62) (#6314) (#6317)
client: add support for pickfirst address shuffling (gRFC A62) (#6311)
xds: Add support for String Matcher Header Matcher in RDS (#6313)
xds/outlierdetection: Add Channelz Logger to Outlier Detection LB (#6145)

Special Thanks: @s-matyukevich

xds: enable RLS in xDS by default (#6343)
orca: add support for application_utilization field and missing range checks on several metrics setters
balancer/weightedroundrobin: add new LB policy for balancing between backends based on their load reports (gRFC A58) (#6241)
authz: add conversion of json to RBAC Audit Logging config (#6192)
authz: add support for stdout logger (#6230 and #6298)
authz: support customizable audit functionality for authorization policy (#6192 #6230 #6298 #6158 #6304 and #6225)

Bug Fixes

orca: fix a race at startup of out-of-band metric subscriptions that would cause the report interval to request 0 (#6245)
xds/xdsresource: Fix Outlier Detection Config Handling and correctly set xDS Defaults (#6361)
xds/outlierdetection: Fix Outlier Detection Config Handling by setting defaults in ParseConfig() (#6361)

API Changes

orca: allow a ServerMetricsProvider to be passed to the ORCA service and ServerOption (#6223)

v1.55.1: Release 1.55.1
Compare Source

status: To fix a panic, status.FromError now returns an error with codes.Unknown when the error implements the GRPCStatus() method, and calling GRPCStatus() returns nil. (#6374)

v1.55.0: Release 1.55.0
Compare Source
Behavior Changes

xds: enable federation support by default (#6151)
status: status.Code and status.FromError handle wrapped errors (#6031 and #6150)

Special Thanks: @psyhatter

New Features

xds/xdsclient: support ignore_resource_deletion server feature as per gRFC A53 (#6035)
security/advancedtls: add min/max TLS version selection options (#6007)

Special Thanks: @joeljeske

Bug Fixes

xds: stop routing RPCs to deleted clusters (#6125)
client: fix race between stream creation and GOAWAY receipt, which could lead to spurious UNAVAILABLE stream errors (#6142)

Performance Improvements

server: improve stream handler goroutine worker allocation when NumStreamWorkers is used (#6004)

Special Thanks: @SaveTheRbtz

v1.54.1: Release 1.54.1
Compare Source
Bug Fixes

credentials/alts: revert a change that causes a crash in the handshaker

v1.54.0: Release 1.54.0
Compare Source
Behavior Changes

xds: remove support for xDS v2 transport API (#6013)

New Features

server: expose SetSendCompressor API to set send compressor name (#5744)

Special Thanks: @jronak

xdsclient: include Node proto only in the first discovery request message, to improve performance (#6078)

Bug Fixes

metadata: fix validation logic and properly validate metadata appended via AppendToOutgoingContext (#6001)

Special Thanks: @ktalg

transport: do not close connections when we encounter I/O errors until after all data is consumed (#6110)
ringhash: ensure addresses are consistently hashed across updates (#6066)
xds/clusterimpl: fix a bug causing unnecessary closing and re-opening of LRS streams (#6112)
xds: NACK route configuration if sum of weights of weighted clusters exceeds uint32_max (#6085)

Documentation

resolver: update Resolver.Scheme() docstring to mention requirement of lowercase scheme names (#6014)
resolver: document expected error handling of UpdateState errors (#6002)

Special Thanks: @fho

examples: add example for ORCA load reporting (#6114)
examples: add an example to illustrate authorization (authz) support (#5920)

Special Thanks: @KenxinKun

v1.53.0: Release 1.53.0
Compare Source
API Changes

balancer: support injection of per-call metadata from LB policies (#5853)
resolver: remove deprecated field resolver.Target.Endpoint and replace with resolver.Target.Endpoint() (#5852)

Special Thanks: @kylejb

New Features

xds/ringhash: introduce GRPC_RING_HASH_CAP environment variable to override the maximum ring size. (#5884)
rls: propagate headers received in RLS response to backends (#5883)

Bug Fixes

transport: drain client transport when streamID approaches MaxStreamID (#5889)
server: after GracefulStop, ensure connections are closed when final RPC completes (#5968)
server: fix a few issues where grpc server uses RST_STREAM for non-HTTP/2 errors (#5893)
xdsclient: fix race which can happen when multiple load reporting calls are made at the same time. (#5927)
rls: fix a data race involving the LRU cache (#5925)
xds: fix panic involving double close of channel in xDS transport (#5959)
gcp/observability: update method name validation (#5951)

Documentation

credentials/oauth: mark NewOauthAccess as deprecated (#5882)

Special Thanks: @buzzsurfr

v1.52.3: Release 1.52.3
Compare Source
Bug Fixes

Fix user-agent version

v1.52.1: Release 1.52.1
Compare Source
Bug Fixes

grpclb: rename grpclbstate package back to state (#5963)

v1.52.0: Release 1.52.0
Compare Source
New Features

xdsclient: log node ID with verbosity INFO (#5860)
ringhash: impose cap on max_ring_size to reduce possibility of OOMs (#5801)

Behavior Changes

client: return an error from Dial if an empty target is passed and no custom dialer is present; the ClientConn would otherwise be unable to connect and perform RPCs (#5732)

Special Thanks: @huangchong94

Bug Fixes

transport (net/http server handler): respond to bad HTTP requests with status 400 (Bad Request) instead of 500 (Internal Server Error). (#5804)

Special Thanks: @sjbarag

transport: Fixed closing a closed channel panic in handlePing (#5854)
server: fix ChainUnaryInterceptor and ChainStreamInterceptor to allow retrying handlers (#5666)

Special Thanks: @yiminc

transport: ensure value of :authority header matches server name used in TLS handshake when the latter is overridden by the name resolver (#5748)

Special Thanks: @holdno

Documentation

examples: add an example to illustrate the usage of stats handler (#5657)

Special Thanks: @Yash-Handa

examples: add new example to show updating metadata in interceptors (#5788)

Special Thanks: @richzw

v1.51.0: Release 1.51.0
Compare Source
Behavior Changes

xds: NACK EDS resources with duplicate addresses in accordance with a recent spec change (#5715)

Special Thanks: @erni27

grpc: restrict status codes that can be generated by the control plane (gRFC A54) (#5653)

New Features

client: set grpc-accept-encoding header with all registered compressors (#5541)

Special Thanks: @jronak

xds/weightedtarget: return a more meaningful error when all child policies are in TRANSIENT_FAILURE (#5711)
gcp/observability: add "started rpcs" metric (#5768)
xds: de-experimentalize the google-c2p-resolver (#5707)
balancer: add experimental Producer types and methods (#5669)
orca: provide a way for LB policies to receive OOB load reports (#5669)

Bug Fixes

go.mod: upgrade x/text dependency to address CVE 2022-32149 (#5769)
client: fix race that could lead to an incorrect connection state if it was closed immediately after the server's HTTP/2 preface was received (#5714)

Special Thanks: @fuweid

xds: ensure sum of the weights of all EDS localities at the same priority level does not exceed uint32 max (#5703)

Special Thanks: @erni27

client: fix binary logging bug which logs a server header on a trailers-only response (#5763)
balancer/priority: fix a bug where unreleased references to removed child policies (and associated state) was causing a memory leak (#5682)
xds/google-c2p: validate URI schema for no authorities (#5756)

v1.50.1: Release 1.50.1
Compare Source
New Features

gcp/observability: support new configuration defined in public preview user guide

v1.50.0: Release 1.50.0
Compare Source
Behavior Changes

client: use proper "@" semantics for connecting to abstract unix sockets. (#5678)

This is technically a bug fix; the result is that the address was including a trailing NULL byte, which it should not have.  This may break users creating the socket in Go by prefixing a NULL instead of an "@", though, so calling it out as a behavior change.

Special Thanks: @jachor

New Features

metadata: add experimental ValueFromIncomingContext to more efficiently retrieve a single value (#5596)

Special Thanks: @horpto

stats: provide peer information in HandleConn context (#5589)

Special Thanks: @feihu-stripe

xds: add support for Outlier Detection, enabled by default (#5435, #5673)

Bug Fixes

client: fix deadlock in transport caused by GOAWAY racing with stream creation (#5652)

This should only occur with an HTTP/2 server that does not follow best practices of an advisory GOAWAY (not a grpc-go server).

xds/xdsclient: fix a bug which was causing routes with cluster_specifier_plugin set to be NACKed when GRPC_EXPERIMENTAL_XDS_RLS_LB was off (#5670)
xds/xdsclient: NACK cluster resource if config_source_specifier in lrs_server is not self (#5613)
xds/ringhash: fix a bug which sometimes prevents the LB policy from retrying connection attempts (#5601)
xds/ringhash: do nothing when asked to exit IDLE instead of falling back on the default channel behavior of connecting to all addresses (#5614)
xds/rls: fix a bug which was causing the channel to be stuck in IDLE (#5656)
alts: fix a bug which was setting WaitForReady on handshaker service RPCs, thereby delaying fallback when required (#5620)
gcp/observability: fix End() to cleanup global state correctly (#5623)

v1.49.0: Release 1.49.0
Compare Source
New Features

gcp/observability: add support for Environment Variable GRPC_CONFIG_OBSERVABILITY_JSON (#5525)
gcp/observability: add support for custom tags (#5565)

Behavior Changes

server: reduce log level from Warning to Info for early connection establishment errors (#5524)

Special Thanks: @jpkrohling

Bug Fixes

client: fix race in flow control that could lead to unexpected EOF errors (#5494)
client: fix a race that could cause RPCs to time out instead of failing more quickly with UNAVAILABLE (#5503)
client & server: fix a panic caused by passing a nil stats handler to grpc.WithStatsHandler or grpc.StatsHandler (#5543)
transport/server: fix a race that could cause a stray header to be sent (#5513)
balancer: give precedence to IDLE over TRANSIENT_FAILURE when aggregating connectivity state (#5473)
xds/xdsclient: request correct resource name when user specifies a new style resource name with empty authority (#5488)
xds/xdsclient: NACK endpoint resources with zero weight (#5560)
xds/xdsclient: fix bug that would reset resource version information after ADS stream restart (#5422)
xds/xdsclient: fix goroutine leaks when load reporting is enabled (#5505)
xds/ringhash: fix config update processing to recreate ring and picker when min/max ring size changes (#5557)
xds/ringhash: avoid recreating subChannels when update doesn't change address weight information (#5431)
xds/priority: fix bug which could cause priority LB to block all traffic after a config update (#5549)
xds: fix bug when environment variable GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION is set to true (#5537)

Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

ℹ Artifact update notice
File name: go/go.mod
In order to perform the update(s) described in the table above, Renovate ran the go get command, which resulted in the following additional change(s):

3 additional dependencies were updated

Details:

Package
Change

google.golang.org/genproto
v0.0.0-20200526211855-cb27e3aa2013 -> v0.0.0-20230410155749-daa745c078e1

google.golang.org/protobuf
v1.28.1 -> v1.30.0

github.com/golang/protobuf
v1.5.2 -> v1.5.3


Native Interactive - Cell history inconsistent behavior

Environment data

Version: 1.58.0-insider
Commit: 2d23c42a936db1c7b3b06f918cde29561cc47cd6
Date: 2021-07-07T12:08:00.172Z
Jupyter Extension version: v2021.8.1007436305
Python Extension version: v2021.7.992362372-dev
OS: OSX 11.4
Python and/or Anaconda version: 3.9.2
Type of virtual environment used: poetry
Jupyter server running: Local

When using arrow keys to move through previously executed cells, the most recent cell seems to require a second up arrow press even when the cursor is at the start of the cell. This makes it a bit hard to know exactly what the behavior will be depending on where you are in the cell history.
Notice how at 00:13 I need to press up arrow twice to move from test3 to test2, but only once to test:
https://user-images.githubusercontent.com/29611875/124806120-2dc11a00-df11-11eb-8c45-7ffa151ab613.mp4
This is a simple example, so I'm not sure if its necessarily always the last cell only. In general things feel a bit weird, can't tell if I'm also needing to double press to move through other cells in the history.
(related functionality request, but not this specific bug):
In my opinion, it would be best if a single up/down arrow press moved to the previous cell if cursor is on the top/bottom-most line, as opposed to jumping to the start/end of the line, (how it worked in the old webview interactive).
Expected behaviour

single up arrow press moves to previous executed cell

Actual behaviour

last executed cell requires two up arrow presses

We're discussing this exact problem over at https://github.com/microsoft/vscode/issues/127709 😊 I've linked this feedback in a comment there so we don't lose track of it.


Make <div class="line-numbers"> fill the container height

When looking at the container for code, there seems to be a hairline gap at the bottom of the code lines. Adding height: 100% to the .line-numbers class seems to fix this.
I wanted to post this here in the event that it was helpful. Happy to make a PR if desired.
Without height: 100%:

With height: 100%:

Hey @Repjarms, looks like that gap shows up due to rounding error. Would love a PR on this!


[Web] Add TVMArgBool to ArgTypeCode

In a recent PR https://github.com/apache/tvm/pull/16183, TVMArgTypeCode introduced TVMArgBool. This PR is needed for the web runtime when a TVM packed function returns a bool.

temp block this in light of https://github.com/apache/tvm/pull/16183#issuecomment-2273212033

#17240 has now landed, so this PR should be able to be revisited.


PRE-COMMIT HOOK (FORMATTING AND LINTING)

Feature: Pre-Commit Linting
Description
This PR introduces a pre-commit hook for enforcing formatting and linting locally before commits can be made, using husky and lint-staged npm packages. Helps all devs to integrate quality and uniformly-styled code.
Changes
Frontend:

Added husky and lint-staged
Modified package.json with new script and lint-staged configurations

Testing

Manual testing was conducted in the command line

Notes

Tested in the command line, after several attempts and alterations, all is working correctly

Checklist

[x] Code compiles correctly
[x] No merge conflicts

Tags
enhancement

Why was npm run lint not functioning? We may look into this in the future, but this fix is quick and simple. Will delete the feature branch after the merge. Approved.