[Server] Multi-Cluster: GraphQL Session Hijacked

Current Behavior
Meshery has erratic behavior with three local Kubernetes clusters connected and is unusable. Clusters:

Docker Desktop
Minikube
KinD

Meshery Operator is deployed to each cluster. Meshery Server built and running locally. Local system has enough free resources (CPU and Mem).
Note: WriteHeader on hijacked connection from github.com/99designs/gqlgen/graphql/handler/transport.SendError
Screenshots/Logs
ERRO[2022-07-14T17:53:32-05:00] rpc error: code = Canceled desc = context canceled  app=meshery code=2041 probable-cause="unable to retrieve the requested data" severity=2 short-description="Error getting operations for the mesh.Error getting service mesh name" suggested-remediation="Make sure adapter is reachable and running"
ERRO[2022-07-14T17:53:32-05:00] rpc error: code = Canceled desc = context canceled  app=meshery code=2041 probable-cause="unable to retrieve the requested data" severity=2 short-description="Error getting operations for the mesh.Error getting service mesh name" suggested-remediation="Make sure adapter is reachable and running"
ERRO[2022-07-14T17:53:32-05:00] rpc error: code = Canceled desc = context canceled  app=meshery code=2041 probable-cause="unable to retrieve the requested data" severity=2 short-description="Error getting operations for the mesh.Error getting service mesh name" suggested-remediation="Make sure adapter is reachable and running"
ERRO[2022-07-14T17:53:32-05:00] rpc error: code = Canceled desc = context canceled  app=meshery code=2041 probable-cause="unable to retrieve the requested data" severity=2 short-description="Error getting operations for the mesh.Error getting service mesh name" suggested-remediation="Make sure adapter is reachable and running"
ERRO[2022-07-14T17:53:32-05:00] rpc error: code = Canceled desc = context canceled  app=meshery code=2041 probable-cause="unable to retrieve the requested data" severity=2 short-description="Error getting operations for the mesh.Error getting service mesh name" suggested-remediation="Make sure adapter is reachable and running"
ERRO[2022-07-14T17:53:32-05:00] rpc error: code = Canceled desc = context canceled  app=meshery code=2041 probable-cause="unable to retrieve the requested data" severity=2 short-description="Error getting operations for the mesh.Error getting service mesh name" suggested-remediation="Make sure adapter is reachable and running"
ERRO[2022-07-14T17:53:32-05:00] write tcp [::1]:9081->[::1]:62537: write: broken pipe  app=meshery code=2013 probable-cause="Object is not a valid json object" severity=2 short-description="Unable to marshal the : .user config data" suggested-remediation="Make sure if the object passed has json tags"
server.go:3160: http: superfluous response.WriteHeader call from github.com/layer5io/meshery/handlers.(*Handler).SessionSyncHandler (session_sync_handler.go:80)
INFO[2022-07-14T17:53:37-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:53:37-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:53:37-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:53:42-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:53:42-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:53:42-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:53:47-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:53:47-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:53:47-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:53:52-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:53:52-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:53:52-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:53:57-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:53:57-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:53:57-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:54:02-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:54:02-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:02-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:54:07-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:54:07-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:07-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:54:12-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:54:12-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:12-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery
websocket.go:79: unable to upgrade *http.response to websocket write tcp [::1]:9081->[::1]:62586: write: broken pipe:
server.go:3160: http: response.WriteHeader on hijacked connection from github.com/99designs/gqlgen/graphql/handler/transport.SendError (error.go:15)
server.go:3160: http: response.Write on hijacked connection from github.com/99designs/gqlgen/graphql/handler/transport.SendError (error.go:20)
INFO[2022-07-14T17:54:17-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:54:17-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:17-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
INFO[2022-07-14T17:54:22-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:54:22-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:22-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery

INFO[2022-07-14T17:54:27-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:54:27-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:27-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
DEBU[0596] token : eyJhY2Nlc3NfdG9rZW4iOiJleUpoYkdjaU9pSlNVekkxTmlJc0ltdHBaQ0k2SW5CMVlteHBZenBsT0dWbU5ERmpNeTFpWldWbUxUUmlZakV0T0dVNE1DMHpOakExTVRZeU4yTTJNakVpTENKMGVYQWlPaUpLVjFRaWZRLmV5SmhkV1FpT2x0ZExDSmpiR2xsYm5SZmFXUWlPaUp0WlhOb1pYSjVMV05zYjNWa0lpd2laWGh3SWpveE5qVTNPRFF5T0RZM0xDSmxlSFFpT250OUxDSnBZWFFpT2pFMk5UYzRNemt5Tmpjc0ltbHpjeUk2SW1oMGRIQnpPaTh2YldWemFHVnllUzVzWVhsbGNqVXVhVzh2YUhsa2NtRXZJaXdpYW5ScElqb2lNek14TkdVM056SXRORFprT1MwME1HTmhMVGt6WTJRdE0yTmxaRFE0T0dVeE9UZzBJaXdpYm1KbUlqb3hOalUzT0RNNU1qWTNMQ0p6WTNBaU9sc2liM0JsYm1sa0lpd2liMlptYkdsdVpTSmRMQ0p6ZFdJaU9pSmlSMDVvWWtkT2RtUkhWVDA2WkVoa2NHUklVbXhqWnowOUluMC4yc0NwOGo0SlNLaE11M3dmaGJtdlZlaTZrek9CdkpKeFZkcDdqZXpnU1FnTkdodWZGcUJiX2sxbmF5cjlWVzI3UDdsbUxZXy1GTk9kOEhQbVdmV2taN28xSnZkVEFnOHFEVDk4OVJWZmU3MG1ueHFvOWI0TFlwYWNZeDVfd0wtZjREVnZTUVExMkE4TU42M1QxX2NDVUtYT1J1Njljd1NsUF9OTlpHdU5OSEd2eVNvR1l2bTBWR1JPS2dETXBLUGs3cE5MRXlpbkVEOExWb2hscmR6c3R0MlJWMUg5MU92QW9FelNpNHpfeTdUaHc3SVktT0tSTUI3clBtMWZwUGpRYklBX09CU0poMGpLcmdKcERRdWZVQVA1VFBraU1kck9iakFwWk15RU9wdFRDZDVoQ2k2VmZhdlNFUzFIR19aenpzWTBwaGoxYkV6Zk1ZVWFJNWVKZFVIM0VSaHRkcElPNHdJWDVPTlF1akhWTnBvYnJ6RHg3ejdSYXBsRV95cXFmQ3ROZVpOU2RNUTdyNEh5MmE1bTRFVC1WdVByVG1XTDJHYkFQdm94YTZHQzZ6S19uLW5aMzdqaGxjdWxKLW85NXpYQUl5X0d6eGcwbkd4RUxYUk9VQUNveXhiRkt6V2RIdnFiZHNJQUI3UGdlMnlFT0FSSHlxcjRXQzNoYmFwaFNtbEVNQV9lV3BfODF5MnJ2Q05lUk9jNWlOQXJ3NUpTZ0Z4NkEwNWktWXprNkliLVVCRUVKZl9UZ0VUYWtlaDM0QUtiRzMxdUFHSEhoRnRhVzJEZ2l0Q0kyUUZqbmVpeFFqVHFFNXpFaWtDUmF3aVVYNndqQlpQaDBmRWRQb3RjaFZDSVZET1V5UTFJZ1dTU2RDdml4eWs4TXRLUlF0dTBLUnRnSU5kZ2R1dyIsInRva2VuX3R5cGUiOiJiZWFyZXIiLCJyZWZyZXNoX3Rva2VuIjoiYWFBcFpoTUhxa1J1QUxnS0o3WWJaMEVyWVZnallFdzlKcDc0Q3p0a0FzZy51d1UzSkN3cGVha3ZVNUZxbDY0VU1fLXRCUUU5Q0otRUM5anlmYmNBd3RBIiwiZXhwaXJ5IjoiMjAyMi0wNy0xNFQyMzo1NDoyNi4zNjg5MjA4NDhaIn0
DEBU[0596] [Initialize]: Package found at /Users/lee/.meshery/provider/Meshery/v0.6.0-rc.6b-3 skipping download
INFO[0596] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0596] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0596] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0596] kubernetes contexts successfully retrieved from remote provider
INFO[0596] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0596] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0596] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533

INFO[0596] kubernetes contexts successfully retrieved from remote provider
INFO[0596] kubernetes contexts successfully retrieved from remote provider

INFO[0597] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0597] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0597] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0597] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0597] kubernetes contexts successfully retrieved from remote provider
INFO[0599] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0599] kubernetes contexts successfully retrieved from remote provider
INFO[0600] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0600] kubernetes contexts successfully retrieved from remote provider
INFO[2022-07-14T17:54:32-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:54:32-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:32-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery
INFO[0603] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0603] kubernetes contexts successfully retrieved from remote provider
INFO[2022-07-14T17:54:37-05:00] found meshery-broker endpoint: localhost:31318 for contextId: fc89362d677d2e2ce4942792a1a506c4  app=meshery
WARN[2022-07-14T17:54:37-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:37-05:00] skipping meshsync data handler setup for contextId: fc89362d677d2e2ce4942792a1a506c4 due to: nats: no servers available for connection  app=meshery
INFO[0607] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0608] kubernetes contexts successfully retrieved from remote provider
INFO[2022-07-14T17:54:42-05:00] found meshery-broker endpoint: localhost:31164 for contextId: e21ad3cc3711dd894fafda9050e3d4eb  app=meshery
WARN[2022-07-14T17:54:42-05:00] nats: no servers available for connection     app=meshery code=11000 probable-cause="Endpoint might not be reachable" severity=2 short-description="Connection to broker failed" suggested-remediation="Make sure the NATS endpoint is reachable"
INFO[2022-07-14T17:54:42-05:00] skipping meshsync data handler setup for contextId: e21ad3cc3711dd894fafda9050e3d4eb due to: nats: no servers available for connection  app=meshery
websocket.go:79: unable to upgrade *http.response to websocket write tcp [::1]:9081->[::1]:62594: write: broken pipe:
server.go:3160: http: response.WriteHeader on hijacked connection from github.com/99designs/gqlgen/graphql/handler/transport.SendError (error.go:15)
server.go:3160: http: response.Write on hijacked connection from github.com/99designs/gqlgen/graphql/handler/transport.SendError (error.go:20)
INFO[0612] attempting to fetch kubernetes contexts from cloud for Meshery instance: 158c2256-ea2e-4d3a-8bb2-c2074329f533
INFO[0613] kubernetes contexts successfully retrieved from remote provider

Environment

Host OS: Mac
Meshery Server Version: stable-v0.6.0-rc.6b
Meshery Client Version: stable-v0.6.0-rc.6b

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

We may close this one as this issue if fixed by https://github.com/meshery/meshery/pull/5844

Closed with https://github.com/meshery/meshery/pull/5844


Feature request: selectable orientation when you arrive at a photosphere

I'm trying out the new photo tour theme. (which is very nice, I'd started doing my own mixed info / nav hotspot theme and this covers it completely)
consider when you have 3 spheres linked to each other.  When you nav from A to C, you expect to be facing away from A. When you nav from B to C, you expect to be facing away from B. Right now, your orientation is just the default based on the sphere image, so it's jarring to navigate around in VR.
So it'd be nice to be able to specify a direction that your view is facing by default in each location.
(I'm trying to do this myself but I've been running into issues with setting the yaw in webVR)

The next release of Ideaspace will have a rotation field type which allows you to set the default rotation of a photo sphere. I implemented this feature in the upcoming Photo Tour Pro theme, but I ran into issues as well if the theme is used in WebVR, because the rotation info from the HMD is overriding the manual settings, even if I set the initial rotation in a wrapper entity around the camera.


Open theme properties to public

Check this dump
["theme"]=>
    object(Timber\Theme)#907 (10) {
...
      ["theme":"Timber\Theme":private]=>
      object(WP_Theme)#908 (12) {
        ["update"]=>
        bool(false)
        ["theme_root":"WP_Theme":private]=>
        string(52) "censored"
        ["headers":"WP_Theme":private]=>
        array(11) {
          ["Name"]=>
          string(6) "censored"
          ["ThemeURI"]=>
          string(34) "censored"
          ["Description"]=>
          string(43) "Bla bla"
          ["Author"]=>
          string(10) "This is what I need"
          ["AuthorURI"]=>
          string(20) "This is what I need"
...

I would like to change ["theme":"Timber\Theme":private] to ["theme":"Timber\Theme":public] so I can access to the Author and AuthorURI fields. Adding watermark is important so please open for public that field.
In the meantime, can you teach me how to extends the Theme class to print the Author field please?

A temporary solution is make one for ours, this is not optimized (in fact, it's ugly) but it can be used.
// Put this in functions.php
function fixMetaTheme() {
	$theme = wp_get_theme();
	$meta = [
		'name' => $theme->get('Name'),
		'meta' => [
			'author' => $theme->get('Author'),
			'author_uri' => $theme->get('AuthorURI')
		]
	];
	return $meta;
}

After that, I override the theme array like this:
$context['theme'] = fixMetaTheme();

From there, I could use in twig template:
This bloody awesome {{theme.name}} made by {{theme.meta.author}}


Terminal client for keybase chat

If you are on desktop, your options for using Keybase for interactive chat are either to run the Electron desktop app, or to run a sequence of keybase chat send and keybase chat read commands. The former isn't great from a security perspective (Electron apps are a notorious security problem due to the bundled browser and libraries), and personally I don't like having more applications open than I need to. So, I have opted to (for now: #22140) use the command-line interface itself, paired with a little demon that notifies me when I have new messages.
This works, but is very cumbersome. If I get a message, I run keybase chat read <name of conversion> (which I think is also not tab-completeable?), get a list of messages, type keybase chat send <name of conversaion>, type my response in something is almost, but not quite, a readline environment, and then send. If they reply, I then repeat the process again and again.
It would be very handy if there was a (simple) TUI for having interactive chat sessions. Something straightforward like an ncurses interface for a single conversation would help enormously. All it would need is to show new messages as they arrive, and allow me to type messages to respond. I think this is all buildable through keybase api, but before I sit down and try to hammer something together myself, I figured I'd open an issue here in case this is something you've already thought about.

@jonhoo this should be quite straight forward to implement yourself (or others can for you) now that bot api and libs are available: https://github.com/keybase/keybase-bot


created '/raw' endpoint to test local endpoint and fastapi function

Description
Created a /raw endpoint to return raw data from the site-assessment dataset. This mostly serves as a "dummy" endpoint for DS to test fast-api function and AWS deployment of somewhat representative data.
Clearly documents the reason for the pull request and what has changed.

[ ] This is a bugfix.
[x] This is a new feature.
[x] This is part of a feature.

Organization

[ ] Have you linked this pull request to a Trello card?

Review

[ ] Have you added two reviewers to this pull request?
[ ] Is this a work in process?
[ ] Is this ready to be reviewed?

 Setting up AWS and git-secrets to host DS endpoint

Working on the README.md currently, but to split it into separate pull request to keep pull requests clean.

Consolidating /raw endpoint files to Feature/ds endpoints sites villages pull request, as this branch is getting stale and the new pull request allows for "conflict free" merging.
Closing this pull request.


RFC: Monitoring across the stack

Additionally some preliminary sketched out code as a starting point
Respond by: 2022-03-25
Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@zackkrida I've removed Ronny from the requested reviewers and replaced his infrastructure sign-off role with you, hope that's okay.
I still need two people from the team to approve this if it looks good @WordPress/openverse. Thanks!

Is there a way (or use) to add monitoring for the size of the responses that we send?

Yeah totally, that would be a histogram metric that we could implement with relative ease.


[SPARK-29504][WEBUI] Toggle full job description on click

On clicking job description in jobs page, the description was not shown fully.
Add the function for the click event on description.
when there is a long description of a job, it cannot be seen fully in the UI.
The feature was added in https://github.com/apache/spark/pull/24145
But it is missed after https://github.com/apache/spark/pull/25374
Before change:

After change: on Double click over decription

No
Manually test

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@gengliangwang i have raised for 2.4 branch. Could you verify?

Test build #113041 has started for PR 26336 at commit 14de92f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17962/
Test PASSed.

Test build #113041 has finished for PR 26336 at commit 14de92f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113041/
Test PASSed.

Test build #113119 has started for PR 26336 at commit edbfd60.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18033/
Test PASSed.

Test build #113119 has finished for PR 26336 at commit edbfd60.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113119/
Test PASSed.

Hi @PavithraRamachandran ,
After building and running test manually with the changes, I found it doesn't work well: on double click the class description-input is removed.
In the original PR https://github.com/apache/spark/pull/25484, the file additional-metrics.js is not removed.
I am closing this PR now.
Sorry I didn't look close into https://github.com/apache/spark/pull/25484 .


非管理员P-agent安装使用全业务安装查询Job无权限问题(fixed #112)

Codecov Report

Merging #113 (3f9ea5c) into V2.1.346rc (b50b239) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           V2.1.346rc     #113   +/-   ##
===========================================
  Coverage       58.90%   58.91%
===========================================
  Files             395      395
  Lines           25943    25944    +1
===========================================
+ Hits            15283    15284    +1
  Misses          10660    10660

Impacted Files
Coverage Δ

apps/backend/components/collections/agent.py
75.27% <100.00%> (ø)

config/default.py
75.75% <100.00%> (+0.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b50b239...3f9ea5c. Read the comment docs.


Best way to output scaled version in canvas?

writeCanvas use context.putImageData which is not considered as a draw function that can be influenced by something like context.scale(x, y).
At first I tried resizing the image with image.resize but that's obviously very slow using it for interactive purpose.
Is there a way to use drawImage instead of putImageData to allow scaling directly on draw?

Why should we scale to a canvas ?
We can wrap the canvas in CSS to resize it.

This is done in the 'screw' project using the following library:
https://github.com/Hackuarium/screws-sorting/blob/main/package.json#L50


pervasive/seq_lib: adding support for Seq to Set conversion

This adds a new method to_set() to the Seq implementation in the seq_lib module that converts the sequence into a set. This also includes a proof that the resulting set is finite via a recursive definition of to_seq.

Pushed the changes that include the suggestions and the broadcast_forall.

Feel free to merge.


feat(qrm): use move_pages syscall to migrate numa memory

What type of PR is this?
Enhancements
What this PR does / why we need it:
Use move_page syscall intead of migrate_pages to migrate numa memory

move_page syscall has more more fine-grained locks  than migrate_page
control the number of pages migrated each time to ensure the move latency is within a acceptable range (20ms).

Which issue(s) this PR fixes:

reduce the blocked time for the process migrated，block time is dropped from minutes to milliseconds.

Special notes for your reviewer:

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.zhangyadong.0808 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


adjust eslint base and node

Recreate "tslint" scripts for backward compatibility
Add/change base rules for eslint

Too much conflicts with package-lock.json


Can't open anything besides prompt

Description of your problem
Anaconda 5.0.0 - Only the prompt shows in the start menu
Previous versions - Can't run anything besides the prompt
What steps will reproduce the problem?
Only thing i did was install and uninstall different versions of conda, none of them worked. I had installed one a year ago, which worked at the time. I also installed python independently from anaconda. I unninstalled one version which stubbornly remains in the start menu and i can't remove.
Please provide any additional information below
My problem is similar to the one here:
https://github.com/ContinuumIO/anaconda-issues/issues/264
I used spyder --reset, which gave:

Then I used conda install spyder-app, and the result:

On that previous issue, one of the users fixed the problem by removing an apostrophe in his name, but mine as none.
Any tips?
Versions and main components

Spyder Version:
Python Version:
Qt Version:
PyQt Version:

Dont know how to check this, since i cant open anything, but right now i used the anaconda 4.2.0 installer

Operating system: win 10, 64 bits

Dependencies
Don't know how to check these..
Thanks in advance

Please uninstall Anaconda completely and reinstall it again.

Did it with different versions of Anaconda, with no result. I think there might be something left up in the registry or somewhere else that is messing thigns up. Will try to format the entire thing and reinstall, just to make sure.

Worked fine after formatting the pc!


When Private-Reviewer changes quota in the Request, they need to enter Justification fields

Description:
When Private-Reviewer changes quota in the Request, they need to enter Justification fields
When Private-Reviewer changes quota to higher in the Admin Decision page of the Request that they review, Justification fields become empty and they have to fill out Justification fields in order to Approve the Request
Expected result:
If Private-Reviewer changes quota to higher in the Admin Decision page of the Request that they review:

they do not have to fill out Justification fields
the Justification fields keep the values that were input by the user

Now if private-reviewer changes quota during the review, the Justification fields do not appear for filling out.
But they also are not visible. I believe they need to be visible for the reviewer for making an informed decision:

So we need to make them visible but not editable by the reviewer.

@funtigr there is a dedicated section to check the information, do you remember?

Oh, right.
The reviewer can see it in the Summary tab in Justification section.


"disallowCommaBeforeLineBreak": { "allExcept": [ "function" ] } does not work

jscs -V
2.2.1
disallowCommaBeforeLineBreak: Error running rule disallowCommaBeforeLineBreak: This is an issue with JSCS and not your codebase.
Please file an issue (with the stack trace below) at: https://github.com/jscs-dev/node-jscs/issues/new
TypeError: Cannot read property 'some' of undefined
    at canSkip (/usr/local/lib/node_modules/jscs/lib/rules/disallow-comma-before-line-break.js:90:53)
    at /usr/local/lib/node_modules/jscs/lib/rules/disallow-comma-before-line-break.js:96:17
    at /usr/local/lib/node_modules/jscs/lib/js-file.js:603:17
    at Array.forEach (native)
    at Object.JsFile.iterateTokensByTypeAndValue (/usr/local/lib/node_modules/jscs/lib/js-file.js:601:26)
    at Object.module.exports.check (/usr/local/lib/node_modules/jscs/lib/rules/disallow-comma-before-line-break.js:95:14)
    at null.<anonymous> (/usr/local/lib/node_modules/jscs/lib/string-checker.js:165:22)
    at Array.forEach (native)
    at StringChecker._checkJsFile (/usr/local/lib/node_modules/jscs/lib/string-checker.js:161:31)
    at StringChecker.checkString (/usr/local/lib/node_modules/jscs/lib/string-checker.js:106:14) at rules/browserify._js :

I belive https://github.com/jscs-dev/node-jscs/commit/cd8ce2e36b4c7fece12ad33cc7e7709ff71fd561 is responsible.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/27186318-disallowcommabeforelinebreak-allexcept-function-does-not-work?utm_campaign=plugin&utm_content=tracker%2F281640&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F281640&utm_medium=issues&utm_source=github).

If it is then it might be the same as https://github.com/jscs-dev/node-jscs/issues/1746? @dreef3
Can you post a code snippet that would result in the error? Thanks for the report!

/cc @dreef3

Ok we're going to release 2.3.0 soon


maintenance: bundle miniflare

This bundles miniflare into the wrangler package. We already bundled most other dependencies, this adds miniflare as well. Of note, it's actually bundling miniflare twice, once into the main bundle, and once separately as a cli to be called with wrangler dev as local mode. We could optimise this in the future as a separate package. Regardless, I expect this to anyway install fewer dependencies (naturally, because it's not a nested dep anymore) and roughly be the same size for download, so it should be a faster install.
I don't think this should break anything else. This would've been a good usecase for the arbitrary npm publishes, haha. Anyway, we can test the alpha after we land this.
Closes https://github.com/cloudflare/wrangler2/issues/66

Just for information...
Before this PR:
=== Tarball Details ===
name:          wrangler
version:       0.0.15
filename:      wrangler-0.0.15.tgz
package size:  2.4 MB
unpacked size: 12.7 MB
total files:   88

After this PR:
=== Tarball Details ===
name:          wrangler
version:       0.0.15
filename:      wrangler-0.0.15.tgz
package size:  4.1 MB
unpacked size: 20.0 MB
total files:   86

More interesting info...
I tried using the npm packed version of wrangler from main and this PR.
The result of npm install:
main
37,456,739 bytes (39.1 MB on disk) for 808 items

This PR
38,472,480 bytes (38.9 MB on disk) for 194 items

So the size on disk is almost identical. But you can see that for this PR we download about a quarter as many items.
Also the package-lock.json indicates many many fewer dependencies (as suggested by @threepointone in the PR description).
Finally, "ad hoc" install time measurement went from 14 secs to 7 secs 🎉

So all-in-all, this looks like a good improvement.
I think we should definitely consider bundling the miniflare library separately from the miniflare CLI, so we can reduce the duplication there (see the size increase in the bundle, above). Or alternatively work out a way to kick off miniflare for dev mode without having to use the miniflare CLI...?

Unfortunately, calling the CLI is necessary so we can attach a debugger :( That's also why we're manually invoking the cli with node --inspect instead of invoking miniflare's bin. Maybe the thing to do here is for pages, and our kv commands etc to use the bundled version instead, which would remove all instances of import {Miniflare} from 'miniflare' from the source. I'll leave that for a subsequent PR.

Thanks for the headsup! Aight, I'll make sure I handle html-rewriter-wasm correctly.

I think a simple fix should be to just mark html-rewriter-wasm as an external, and add it as a dependency. Do you remember how to trigger the undici issue?

Ok thank you @mrbbot, I'm able to reproduce the issue. I'll park this PR until I fix it. Thanks for the assist, I appreciate you!

This PR has got a bit stale and is not crucial to getting to 2.0.0.
Let's close it for the time-being and revisit after we release 2.0.0.


ATAC QC error

I got this error from running ac.pl.fragment_histogram(atac, region='chr1:1-2000000')
KeyError                                  Traceback (most recent call last)
 in
----> 1 ac.pl.fragment_histogram(atac, region='chr1:1-2000000')
~\Anaconda3\lib\site-packages\muon_atac\plot.py in fragment_histogram(data, region, groupby)
330         raise TypeError("Expected AnnData or MuData object with 'atac' modality")
331
--> 332     fragment_path = adata.uns["files"]["fragments"]
333     fragments = tools.fetch_regions_to_df(fragment_path=fragment_path, features=region)
334
~\Anaconda3\lib\site-packages\anndata\compat_overloaded_dict.py in getitem(self, key)
98             return self.overloaded[key].get()
99         else:
--> 100             return self.data[key]
101
102     def setitem(self, key, value):
KeyError: 'files'
I got another error by running ac.tl.nucleosome_signal(atac, n=1e6)
KeyError                                  Traceback (most recent call last)
 in
----> 1 ac.tl.nucleosome_signal(atac, n=1e6)
~\Anaconda3\lib\site-packages\muon_atac\tools.py in nucleosome_signal(data, n, nucleosome_free_upper_bound, mononuleosomal_upper_bound)
1031
1032     if "files" not in adata.uns or "fragments" not in adata.uns["files"]:
-> 1033         raise KeyError(
1034             "There is no fragments file located yet. Run muon.atac.tl.locate_fragments first."
1035         )
KeyError: 'There is no fragments file located yet. Run muon.atac.tl.locate_fragments first.'
Not sure what to do?

Hey, in order to work with fragments, a fragments file (with a tabix index) is required. It's a BED-like file, and you can find more information about it for instance here.
This is a large file that is not loaded into the AnnData object but rather a path to the file is recorded.
For 10X Genomics CellRanger ATAC output, the reader in muon.atac will pick the file automatically.
It seems like there has been no fragments file located and, as indicated by the error message, it has to be located with ac.tl.locate_fragments().

I've had this issue while going through the tutorial.
The ac.tl.locate_fragments() solution as suggested by @gtca works, but it's not clear why it is required from the tutorial.
The reason, I believe, is that tutorial assumes the presence of pysam in which case mu.read_10x_h5 will automatically add the fragment file. But pysam not in the pyproject.toml. If pysam is not present it will show a little warning that is very easy to miss (at least for me).
So the solution would probably be to just explicitly add pysam as a dependency even though it may not be necessary for all use cases.

Thanks, @maxim-h, this is a fair point. I believe we can improve error & warning messages.
The one that this thread has started with seems to be good enough already though? It does provide an action point:
KeyError: 'There is no fragments file located yet. Run muon.atac.tl.locate_fragments first.'

Currently, I would argue for continuing not having pysam as a hard dependency for muon — even when working with ATAC-seq data it might not be required, depending on the task, not speaking of other omics.
Maybe we can take advantage of features like extras in poetry — or requires-extra in flit, which we're currently using.

As this has been recently addressed e.g. in 60c979b6442903228dbaa5f4d9807fc4f66a8967 and 17eb76c183e0739b96d634963c6e82a02c71c4b8, I'll close it for now. Feel free to re-open or to open a new one if there are other suggestions how to improve user experience with ATAC while it's a part of muon with optional dependencies.


Don't skip ZipReader data twice.

This could occur when a zip file has a normal file header AND a post descriptor.
Fixes https://github.com/adamhathcock/sharpcompress/issues/269

Could you also add a test for BadStateArchive.zip from #269 issue?

Done


VerneMQ cluster not working in Ipv6 only environment on Kubernetes

Hi,
We have two environments where we are trying to deploy VerneMQ using the helm chart. One is dual stack and the other ipv6.
The dual stack environment works fine with the latest chart version. The issue however is the ipv6.
Firstly, vmq admin does not work and I see the same issue with vernemq ping.
~ $ vmq-admin
Node<EMAIL_ADDRESS>not responding to pings.
~ $ vernemq ping
Node<EMAIL_ADDRESS>not responding to pings.
~ $ netstat -tunlup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 <IP_ADDRESS>:4369            <IP_ADDRESS>:*               LISTEN      191/epmd
tcp        0      0 <IP_ADDRESS>:1883          <IP_ADDRESS>:*               LISTEN      131/beam.smp
tcp        0      0 :::9100                 :::*                    LISTEN      131/beam.smp
tcp        0      0 2600:1f14:22b:5502:dd4a::1:8080 :::*                    LISTEN      131/beam.smp
tcp        0      0 :::4369                 :::*                    LISTEN      191/epmd
tcp        0      0 2600:1f14:22b:5502:dd4a::1:44053 :::*                    LISTEN      131/beam.smp
tcp        0      0 ::1:8888                :::*                    LISTEN      131/beam.smp
tcp        0      0 2600:1f14:22b:5502:dd4a::1:8888 :::*                    LISTEN      131/beam.smp
tcp        0      0 ::1:1883                :::*                    LISTEN      131/beam.smp
tcp        0      0 2600:1f14:22b:5502:dd4a::1:1883 :::*                    LISTEN      131/beam.smp

Here is the same on the dual stack cluster:
~ $ vmq-admin
Usage: vmq-admin <sub-command>

  Administrate the cluster.

  Sub-commands:
    node        Manage this node
    cluster     Manage this node's cluster membership
    session     Retrieve session information
    retain      Show and filter MQTT retained messages
    plugin      Manage plugin system
    listener    Manage listener interfaces
    metrics     Retrieve System Metrics
    api-key     Manage API keys for the HTTP management interface
    trace       Trace various aspects of VerneMQ
  Use --help after a sub-command for more details.

~ $ vernemq ping
pong
~ $ netstat -tunlup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 <IP_ADDRESS>:8888          <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 <IP_ADDRESS>:8888         <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 <IP_ADDRESS>:1883          <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 <IP_ADDRESS>:1883         <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 <IP_ADDRESS>:9100            <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 <IP_ADDRESS>:8080         <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 <IP_ADDRESS>:4369            <IP_ADDRESS>:*               LISTEN      194/epmd
tcp        0      0 <IP_ADDRESS>:44053        <IP_ADDRESS>:*               LISTEN      136/beam.smp
tcp        0      0 :::4369                 :::*                    LISTEN      194/epmd

I feel Im missing some listener for the ipv6 setup. But I have exhausted all my options. I have tried most of the things I could find. Please help me figure out what I might be missing.
I can provide more logs/configs if required.

@avinakollu
There's an open issue: https://github.com/vernemq/vernemq/issues/1664
The vmq-admin scripts do not connect when the cluster communication is configured to use ipv6.
(which means having  -proto_dist inet6_tcp in vmq.args enabled, so that Erlang cluster comm uses ipv6).
I'm not sure here whether your nodes still cluster. Can you access the status page on port 8888 for one of the nodes, and check whether it shows a full cluster?
We need to find a way to fix the script issue.

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@ioolkos Thanks for your response.
So, I have two scenarios. When I enable -proto_dist inet6_tcp in vm.args, only vernemq-0 comes up. The second one fails with the following trace:
Permissions ok: Our pod vernemq-1 belongs to StatefulSet vernemq with 2 replicas
Will join an existing Kubernetes cluster with discovery node at vernemq-0.vernemq-headless.messaging.svc.cluster.local
Did I previously leave the cluster? If so, purging old state.
Cluster doesn't know about me, this means I've left previously. Purging old state...
Password:
Reenter password:
config is OK
-config /vernemq/data/generated.configs/app.2<IP_ADDRESS>.56.21.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args
Exec:  /vernemq/bin/../erts-<IP_ADDRESS>/bin/erlexec -boot /vernemq/bin/../releases/<IP_ADDRESS>/vernemq               -config /vernemq/data/generated.configs/app.2<IP_ADDRESS>.56.21.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args              -pa /vernemq/bin/../lib/erlio-patches -- console -noshell -noinput
Root: /vernemq/bin/..
Protocol 'inet6_tcp-eval': not supported
Protocol 'vmq_server_cmd:node_join('VerneMQ@vernemq-0.vernemq-headless.messaging.svc.cluster.local')': not supported```

Without the flag though, all 3 replicas do come up but I do not see the nodes in the cluster status page. Here is the status page:
<img width="1167" alt="Screenshot 2023-02-27 at 3 46 03 PM" src="https://user-images.githubusercontent.com/87093061/221717438-b5d74934-adcb-4add-a96d-3c929a25dead.png">

The reason is that the node tries to cluster automatically. And vmq_server_cmd:node_join/1 is a wrapper for a vmq-admin call which bring us back to the mentioned incompatibility.
https://github.com/vernemq/vernemq/blob/5c14718469cc861241caa2b920ef5bca25283d71/apps/vmq_server/src/vmq_server_cmd.erl#L28
I don't see what's wrong with scenario 2. Any logs from the nodes?

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

Note to self: find a way to inject -proto_dist inet6_tcp into the noderunner escript. Maybe we need to have a second ip6-enabled version of the script and then make vmq-admin choose via a flag.

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

Yeah I do have logs.
Node 1:
Permissions ok: Our pod vernemq-0 belongs to StatefulSet vernemq with 1 replicas
Password:
Reenter password:
config is OK
-config /vernemq/data/generated.configs/app.2<IP_ADDRESS>.02.10.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args
Exec:  /vernemq/bin/../erts-<IP_ADDRESS>/bin/erlexec -boot /vernemq/bin/../releases/<IP_ADDRESS>/vernemq               -config /vernemq/data/generated.configs/app.2<IP_ADDRESS>.02.10.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args              -pa /vernemq/bin/../lib/erlio-patches -- console -noshell -noinput
Root: /vernemq/bin/..
00:02:12.624 [info] alarm_handler: {set,{system_memory_high_watermark,[]}}
00:02:12.728 [info] writing (updated) old actor <<217,63,70,135,63,206,115,43,49,140,165,14,237,32,235,220,239,75,136,229>> to disk
00:02:12.736 [info] writing state {[{[{actor,<<217,63,70,135,63,206,115,43,49,140,165,14,237,32,235,220,239,75,136,229>>}],1}],{dict,1,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[['VerneMQ@vernemq-0.vernemq-headless.messaging.svc.cluster.local',{[{actor,<<217,63,70,135,63,206,115,43,49,140,165,14,237,32,235,220,239,75,136,229>>}],1}]],[],[],[]}}},{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}} to disk <<75,2,131,80,0,0,1,36,120,1,203,96,206,97,96,96,96,204,96,130,82,41,12,172,137,201,37,249,69,185,64,81,145,155,246,110,237,246,231,138,181,13,123,150,242,189,85,120,125,231,189,119,199,211,172,68,198,172,12,206,20,6,150,148,204,228,146,68,198,68,1,32,228,72,12,72,52,200,16,200,66,3,25,140,168,98,96,43,64,4,83,10,131,93,88,106,81,94,170,111,160,67,25,136,206,45,212,53,208,131,177,50,82,19,83,114,82,139,139,245,114,129,68,98,122,102,94,186,94,113,89,178,94,114,78,105,113,73,106,145,94,78,126,114,98,14,105,238,5,185,11,225,102,6,82,220,12,210,10,0,163,254,97,243>>
00:02:12.757 [info] Opening LevelDB SWC database at "./data/swc_meta/meta1"
00:02:12.781 [info] Opening LevelDB SWC database at "./data/swc_meta/meta2"
00:02:12.791 [info] Opening LevelDB SWC database at "./data/swc_meta/meta3"
00:02:12.800 [info] Opening LevelDB SWC database at "./data/swc_meta/meta4"
00:02:12.810 [info] Opening LevelDB SWC database at "./data/swc_meta/meta5"
00:02:12.819 [info] Opening LevelDB SWC database at "./data/swc_meta/meta6"
00:02:12.828 [info] Opening LevelDB SWC database at "./data/swc_meta/meta7"
00:02:12.837 [info] Opening LevelDB SWC database at "./data/swc_meta/meta8"
00:02:12.847 [info] Opening LevelDB SWC database at "./data/swc_meta/meta9"
00:02:12.858 [info] Opening LevelDB SWC database at "./data/swc_meta/meta10"
00:02:12.910 [info] Try to start vmq_swc: ok
00:02:12.956 [info] Opening LevelDB database at "./data/msgstore/1"
00:02:12.971 [info] Opening LevelDB database at "./data/msgstore/2"
00:02:12.985 [info] Opening LevelDB database at "./data/msgstore/3"
00:02:12.994 [info] Opening LevelDB database at "./data/msgstore/4"
00:02:13.001 [info] Opening LevelDB database at "./data/msgstore/5"
00:02:13.010 [info] Opening LevelDB database at "./data/msgstore/6"
00:02:13.019 [info] Opening LevelDB database at "./data/msgstore/7"
00:02:13.028 [info] Opening LevelDB database at "./data/msgstore/8"
00:02:13.036 [info] Opening LevelDB database at "./data/msgstore/9"
00:02:13.044 [info] Opening LevelDB database at "./data/msgstore/10"
00:02:13.053 [info] Opening LevelDB database at "./data/msgstore/11"
00:02:13.062 [info] Opening LevelDB database at "./data/msgstore/12"
00:02:13.122 [info] Try to start vmq_generic_msg_store: ok
00:02:13.230 [info] loaded 0 subscriptions into vmq_reg_trie
00:02:13.249 [info] cluster event handler 'vmq_cluster' registered ```

Node 2:
``` $ kubectl logs vernemq-1 -n messaging
Permissions ok: Our pod vernemq-1 belongs to StatefulSet vernemq with 2 replicas
Will join an existing Kubernetes cluster with discovery node at vernemq-0.vernemq-headless.messaging.svc.cluster.local
Did I previously leave the cluster? If so, purging old state.
Cluster doesn't know about me, this means I've left previously. Purging old state...
Password:
Reenter password:
config is OK
-config /vernemq/data/generated.configs/app.2<IP_ADDRESS>.04.01.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args
Exec:  /vernemq/bin/../erts-<IP_ADDRESS>/bin/erlexec -boot /vernemq/bin/../releases/<IP_ADDRESS>/vernemq               -config /vernemq/data/generated.configs/app.2<IP_ADDRESS>.04.01.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args              -pa /vernemq/bin/../lib/erlio-patches -- console -noshell -noinput
Root: /vernemq/bin/..
00:04:03.212 [info] alarm_handler: {set,{system_memory_high_watermark,[]}}
00:04:03.314 [info] writing (updated) old actor <<165,158,8,12,24,41,0,246,32,145,173,99,202,109,217,6,192,216,199,63>> to disk
00:04:03.322 [info] writing state {[{[{actor,<<165,158,8,12,24,41,0,246,32,145,173,99,202,109,217,6,192,216,199,63>>}],1}],{dict,1,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[['VerneMQ@vernemq-1.vernemq-headless.messaging.svc.cluster.local',{[{actor,<<165,158,8,12,24,41,0,246,32,145,173,99,202,109,217,6,192,216,199,63>>}],1}]],[],[],[]}}},{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}} to disk <<75,2,131,80,0,0,1,36,120,1,203,96,206,97,96,96,96,204,96,130,82,41,12,172,137,201,37,249,69,185,64,81,145,165,243,56,120,36,52,25,190,41,76,92,155,124,42,247,38,219,129,27,199,237,179,18,25,179,50,56,83,24,88,82,50,147,75,18,25,19,5,128,144,35,49,32,209,32,67,32,11,13,100,48,162,138,129,173,0,17,76,41,12,118,97,169,69,121,169,190,129,14,101,32,58,183,80,215,80,15,198,202,72,77,76,201,73,45,46,214,203,5,18,137,233,153,121,233,122,197,101,201,122,201,57,165,197,37,169,69,122,57,249,201,137,57,164,185,23,228,46,132,155,25,72,113,51,72,43,0,185,179,95,4>>
00:04:03.346 [info] Opening LevelDB SWC database at "./data/swc_meta/meta1"
00:04:03.368 [info] Opening LevelDB SWC database at "./data/swc_meta/meta2"
00:04:03.377 [info] Opening LevelDB SWC database at "./data/swc_meta/meta3"
00:04:03.386 [info] Opening LevelDB SWC database at "./data/swc_meta/meta4"
00:04:03.395 [info] Opening LevelDB SWC database at "./data/swc_meta/meta5"
00:04:03.404 [info] Opening LevelDB SWC database at "./data/swc_meta/meta6"
00:04:03.417 [info] Opening LevelDB SWC database at "./data/swc_meta/meta7"
00:04:03.425 [info] Opening LevelDB SWC database at "./data/swc_meta/meta8"
00:04:03.434 [info] Opening LevelDB SWC database at "./data/swc_meta/meta9"
00:04:03.444 [info] Opening LevelDB SWC database at "./data/swc_meta/meta10"
00:04:03.493 [info] Try to start vmq_swc: ok
00:04:03.530 [info] Opening LevelDB database at "./data/msgstore/1"
00:04:03.539 [info] Opening LevelDB database at "./data/msgstore/2"
00:04:03.547 [info] Opening LevelDB database at "./data/msgstore/3"
00:04:03.556 [info] Opening LevelDB database at "./data/msgstore/4"
00:04:03.564 [info] Opening LevelDB database at "./data/msgstore/5"
00:04:03.572 [info] Opening LevelDB database at "./data/msgstore/6"
00:04:03.581 [info] Opening LevelDB database at "./data/msgstore/7"
00:04:03.589 [info] Opening LevelDB database at "./data/msgstore/8"
00:04:03.598 [info] Opening LevelDB database at "./data/msgstore/9"
00:04:03.608 [info] Opening LevelDB database at "./data/msgstore/10"
00:04:03.616 [info] Opening LevelDB database at "./data/msgstore/11"
00:04:03.624 [info] Opening LevelDB database at "./data/msgstore/12"
00:04:03.658 [info] Try to start vmq_generic_msg_store: ok
00:04:03.763 [info] loaded 0 subscriptions into vmq_reg_trie
00:04:03.771 [info] cluster event handler 'vmq_cluster' registered
00:04:04.610 [info] Sent join request to<EMAIL_ADDRESS>00:04:04.615 [info] Unable to connect to<EMAIL_ADDRESS>```

00:04:04.615 [info] Unable to connect to<EMAIL_ADDRESS>
There is no connectivity on the Erlang distribution level, judging from that log line. net_kernel:connect_node/1 fails. Whether this comes from one of the configs, or some Kubernetes configs (maybe done previously), I don't know.

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@ioolkos Can you please tell me which endpoint this tries to connect to? I will verify if the connection is successful. As I said, this is an ipv6 only cluster where I had to change the listeners to get it to this point.

I see, then all listeners are IPv6 but the Erlang distribution is not enabled for IPv6.
For an idea on the ports involved (IPv4) see: https://docs.vernemq.com/vernemq-clustering/communication
But in any case really need to enable IPv6 in the noderunner script (see my remark above); otherwise the join command will not work.

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@avinakollu yes, that's the context. I count on having this fixed in the next release, but I have no ETA.
Do you plan on supporting the VerneMQ project?

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@avinakollu here's the PR that builds the nodetool/vmq-admin script dynamically to adapt for ipv4 or ipv6: https://github.com/vernemq/vernemq/pull/2134
Once a new release is out, we'll need to ensure this works in Docker too. (it should, but you never know). For a normal build it works perfectly.

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.


Improve delete event performance

Fix #1286
Relocate the responsibility of cascading Reading deletion when deleting
an Event.
Add a new function to the DBClient interface as well as the Mongo and
Redis implementations which abstract the process of deleting all
readings associated with a Device.
Update the business logic to leverage the newly added function for more
effectively deleting readings associated with the specified device.
The Mongo implementation has gone from O(n) to O(1) where n is the
number of readings associated with an Event. However, the Redis
implementation still remains at O(n) for event deletion process as a
whole where n is the number of readings associated with the event. The
deletion functionality is split into two processing units. One which
renames the reading Redis key, this is functionaly the same as a delete
from the perspective of other clients as they cannot retrieve the
original data. The other process cleans up the lingering data. The API
has been updated to rename the Redis reading keys which are to be
deleted, asynchronously delete the lingering data, and respond to the
user. Therefore, the API executes at O(1), but the deletion process as a
whole executes at O(n). The reason is that the process as a whole
executes at O(n) is because we must retrieve the full object that is
persisted so that we can remove all the data(Sorted sets) that reside in
the data-store. For example, there exists sorted sets for time
created(reading:created), and name(reading:name:{r.Name}) associated
with the reading. Given only the device information we are not be able
to remove the data from Redis(ZREM) without constructing the Redis key
which is compromised of the data which is stored.
Signed-off-by: Anthony M. Bonafide<EMAIL_ADDRESS>

The changes should be merged after the updates to Mongo have been merged to eliminate any performance issues related to the lack of indexes on the fields needed by the newly added queries. The related PRs can be found here:

https://github.com/edgexfoundry/docker-edgex-mongo/issues/19
https://github.com/edgexfoundry/developer-scripts/issues/116

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@624da0c). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master    #1400   +/-   ##
=========================================
  Coverage          ?   16.95%
=========================================
  Files             ?       78
  Lines             ?     8792
  Branches          ?        0
=========================================
  Hits              ?     1491
  Misses            ?     7148
  Partials          ?      153

Impacted Files
Coverage Δ

internal/core/data/event.go
59.63% <0%> (ø)

internal/core/data/init.go
0% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 624da0c...7a74f43. Read the comment docs.

I finally caught up on this thread and one thing to consider is all the work being done to avoid doing this in Lua. For the same batch operation, Lua will always be the fastest implementation. The tradeoff is when the Lua script is running, all other commands will be queued so there would be a need to chunk the key deletions.

As I see it, an implementation completely in Lua will not be as effective as you cannot pipeline the Redis calls from a script, which will then result in the client being blocked. On the other hand, we can do a hybrid approach which we have a Lua script which handles the deletion and the client makes "batched" calls to the server by invoking the script with smaller chunks of IDs which to delete. I am assuming the client portion which is doing the invoking would do this in the background to reduce the wait time for the user, which is similar to what is implemented here only minus the Lua script which we can add for the go-routine to leverage for the background deletion. Should we introduce a Lua script for the background deletion process?

You don't need to pipeline from the Lua script because you can't get run any faster than a Lua script; the script and the server are running in the same memory space.
What I'm recommending is investigating a strategy to avoid creating the complexity that I'm seeing arise from using go routines. I'll pull in one of my colleagues and we'll noodle on this side.

I'm personally wary of using Lua here because it's an additional skillset that will be necessary to support the Redis provider. In either case -- Mongo or Redis -- some knowledge of querying capabilities and syntax are required, but now for Redis we're talking about Lua as well. I'm not aware of anyone outside of our Redis team members who has familiarity with Lua. I'm not sure asking anyone who wants to extend a feature that touches the database layer to potentially learn Lua is a viable option.

I spoke briefly with @andresrinivasan yesterday and it seems like we can merge this and possibly work on improvements in a follow up PR. Based on @tsconn23 previous comment, it seems like we would ideally like to find an improvement that does not introduce a Lua script for the sake of logical complexity.


No overlapping events in day type calendar

is it possible to block so that there is on overlapping events in day type of calendar by default?

Not in this version, maybe when v-calendar v2.0 is used in next version


exit fee change based on amount

when i change the amounts i would like to remove, the exit fee changes.
this should not be the case as the percentage is not related to the amount i liquidate.
bonus bug: the "output" amount when selecting 25% is showing as 5e-7 instead of the actual number (0.00000050) in a readable format.
https://streamable.com/fyly6d

Solved the 'bonus bug'
Are we sure about the exit fee? I can only make it change on very small amounts, exit fee is calculated from on-chain data. @ashachaf

might be related to very small numbers where you lose accuracy for lack of decimals which results in different exit fee.
can only reproduce when i try to remove liquidity for very small numbers.


[DNNL][BYOC] Enable Altering Dense Weight Layout

The patch including four parts:

Enable nn.contrib_dense_pack to be partitioned and offloaded by dnnl byoc.
Enable alter dense layout function during build relay model by introducing nn.contrib_dense_pack.
Make some minor fixes in tensor_requisite.h.
Add UT for pack dense.

Thanks for contributing to TVM!   Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

@apeskov @masahi @yangulei @crazydemo @linlifan @Qianshui-Jiang Please take a look :-)

Hi @apeskov, what you mentioned is a common issue of blocked layout.

Let's take a close look on next example. Dense with next shapes: data_shape [128, 10], weight_shape [17, 10], weight_layout NC, output_shape will be [128, 17]. Assume that we applies alter op layout and change layout to NC8c. Weight shape will be changed to [3, 10, 8], with some additional padding. Unexpectedly, output shape will also be changed to [128, 24]. Weight layout conversion changes output shape, that's very strange behavior.

If additional padding applied when transformed from a plain layout to blocked layout, a cropping must be applied too when transformed back to plain layout. A bijective transformation should ensure origin = backward(forward(origin)), but it's not guarantied so far.
Padding is natural, while cropping needs extra information. We use the extra information from the definition of Conv to solve this problem with the blocked weights, but it's a workaround instead of a general solution. I think we need both logical shape and concrete shape for tensor, just like the dims and padded dims in DNNL memory descriptor.
Maybe we need a Pass to Infer the original logical shapes and save them as attributes for later usage, do you have any idea about this?

Hi @masahi , Could you shed some light on the ci failure? I could not find a way to reproduce it on local environment. Thanks!

Sorry I couldn't tell what the error was either, cc @driazati @areusch

sorry it's unclear from the logs, we really should aggregate common error phrases automatically. If you search through the logs for Fatal Python error: Aborted you can see the failed test (e.g. in https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm/branches/PR-11966/runs/10/nodes/384/steps/1167/log/?start=0 it's
[2022-08-01T04:22:42.467Z] tests/python/frontend/pytorch/test_forward.py::test_convert_torch_script_with_input_types free(): invalid pointer
[2022-08-01T04:22:42.467Z] Fatal Python error: Aborted
[2022-08-01T04:22:42.467Z]
[2022-08-01T04:22:42.467Z] Thread 0x00007fbdec1c2700 (most recent call first):
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 300 in wait
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 552 in wait
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/tqdm/_monitor.py", line 60 in run
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap
[2022-08-01T04:22:42.467Z]
[2022-08-01T04:22:42.467Z] Thread 0x00007fbe37727700 (most recent call first):
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/socket.py", line 212 in accept
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pytest_rerunfailures.py", line 429 in run_server
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 870 in run
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap
[2022-08-01T04:22:42.467Z]
[2022-08-01T04:22:42.467Z] Current thread 0x00007fbe6c265740 (most recent call first):
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/torch/jit/_serialization.py", line 162 in load
[2022-08-01T04:22:42.467Z]   File "/workspace/tests/python/frontend/pytorch/test_forward.py", line 4077 in test_convert_torch_script_with_input_types
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/python.py", line 192 in pytest_pyfunc_call
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_callers.py", line 39 in _multicall
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_manager.py", line 80 in _hookexec
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_hooks.py", line 265 in __call__
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/python.py", line 1761 in runtest
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/runner.py", line 166 in pytest_runtest_call
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_callers.py", line 39 in _multicall
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_manager.py", line 80 in _hookexec
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_hooks.py", line 265 in __call__
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/runner.py", line 259 in <lambda>
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/runner.py", line 338 in from_call
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/runner.py", line 259 in call_runtest_hook
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/runner.py", line 219 in call_and_report
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/runner.py", line 130 in runtestprotocol
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pytest_rerunfailures.py", line 497 in pytest_runtest_protocol
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_callers.py", line 39 in _multicall
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_manager.py", line 80 in _hookexec
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_hooks.py", line 265 in __call__
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/main.py", line 347 in pytest_runtestloop
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_callers.py", line 39 in _multicall
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_manager.py", line 80 in _hookexec
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_hooks.py", line 265 in __call__
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/main.py", line 322 in _main
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/main.py", line 268 in wrap_session
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/main.py", line 315 in pytest_cmdline_main
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_callers.py", line 39 in _multicall
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_manager.py", line 80 in _hookexec
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pluggy/_hooks.py", line 265 in __call__
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/config/__init__.py", line 165 in main
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/_pytest/config/__init__.py", line 187 in console_main
[2022-08-01T04:22:42.467Z]   File "/usr/local/lib/python3.7/dist-packages/pytest/__main__.py", line 5 in <module>
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/runpy.py", line 85 in _run_code
[2022-08-01T04:22:42.467Z]   File "/usr/lib/python3.7/runpy.py", line 193 in _run_module_as_main
[2022-08-01T04:22:42.722Z] tests/scripts/setup-pytest-env.sh: line 49: 28733 Aborted                 TVM_FFI=${ffi_type} python3 -m pytest -o "junit_suite_name=${suite_name}" "--junit-xml=${TVM_PYTEST_RESULT_DIR}/${suite_name}.xml" "--junit-prefix=${ffi_type}" "${extra_args[@]}"
[2022-08-01T04:22:42.722Z] + exit_code=134

)

That surely looks unrelated to this PR (it fails in PyTorch). The same issue is reported in https://github.com/apache/tvm/issues/12276. The error looks similar to the one in https://github.com/apache/tvm/issues/9362, but I don't know why we start getting this now...

@tvm-bot rerun

Hi @masahi , Thanks for quick response. I found another testcase failed in https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm/branches/PR-11966/runs/11/nodes/382/steps/1159/log/?start=0.
[2022-08-03T01:24:59.821Z] tests/python/frontend/pytorch/qnn_test.py::test_serialized_modules free(): invalid pointer
[2022-08-03T01:24:59.821Z] Fatal Python error: Aborted
Is it a random issue?

Yeah that looks also unrelated and flaky. It's strange, I did a PR yesterday and met none of these issues. https://github.com/apache/tvm/pull/12263

@tvm-bot rerun


Read performance issues

OS
Your OS: Ubuntu 20.04
Versions
The db-sync version (eg cardano-db-sync --version): <IP_ADDRESS>
PostgreSQL version: 14
Build/Install Method
The method you use to build or install cardano-db-sync: N/A
Run method
The method you used to run cardano-db-sync (eg Nix/Docker/systemd/none): containerd
Problem Report
It took nearly 2 minutes to fetch a single block (and all of its transactions). After some investigation, the following indices were created and the fetch time is now ~5 seconds:
CREATE INDEX idx_redeemer_tx_id ON redeemer(tx_id);
CREATE INDEX idx_collateral_tx_in_tx_in_id ON collateral_tx_in(tx_in_id);
CREATE INDEX idx_collateral_tx_out_tx_id ON collateral_tx_out(tx_id);
CREATE INDEX idx_reference_tx_in_tx_in_id ON reference_tx_in(tx_in_id);

Possibly duplicate of https://github.com/input-output-hk/cardano-db-sync/issues/1349. Since adding indexes helped, maybe we can close?

@kderme The aforementioned indices are not mentioned in #1349.

How is it determined which indices should be managed by db-sync and which should be customer-managed?

This is described in https://github.com/input-output-hk/cardano-db-sync/blob/master/doc/migrations.md#upgrading-to-13100
Basically with <IP_ADDRESS> we splitted existing indexes to two groups. The stage 3 migration, which are indexes created at the beginning and are crucial for db-sync itself. And the stage 4, the remaining indexes that historically db-sync had, which are created when db-sync is 30mins behind the tip of the chain. Users can add new inderxes in stage 3 and add new or remove any index in stage 4.

Okay, but the indices referenced in my original report do not exist in any of the schema files. Moreover, the document you've linked to does not provide any justification for including or excluding a given index in the schema. In other words, it does not answer the question, "Under what conditions will an index be included in the migration set, and under what conditions will it be excluded?"

db-sync will create any index added in the migration file. By default these migrations files have indexes based on what db-sync needs (stage 3) and what was historically there (stage 4). I'm not sure I can provide more justifications.

Okay, but the indices referenced in my original report do not exist in any of the schema files

Then you have to create them yourself.


Fix shm.exists() check in ingress drop monitor

If we are recording ingress drops to a counter, use the right check for
the counter's existence.  Addresses
https://github.com/snabbco/snabb/pull/1047#pullrequestreview-5090751.

+1


[Suggestion][.NET] Update .csproj  on Script addition/deletion

I'm sharing a porject on linux and windows, and i can't use it on windows, cause Visual Studio dosn't add the scripts i added using Consulo
 <ItemGroup>
    <Compile Include="Program.cs" />
    <Compile Include="Test.cs" />
    <Compile Include="Test.Something\Test.cs" />
    <Compile Include="Config.cs" />
    <Compile Include="Properties\AssemblyInfo.cs" />
  </ItemGroup>

It would be cool if Consulo could update the .csproj project file :)

Moved to Visual Studio plugin repository


Function fixed: comoving_volume_element

Previous implementation was using the comoving_angular_distance. Fix to use the angular_diameter_diatance.

What about this line?
https://github.com/LSSTDESC/CCL/blob/d762d995d9edb23522ff42818aa2343af3ec2d34/pyccl/background.py#L311

@pennalima I think the function was correct as it was. Note that the function is meant to be dV / (da dOmega), not dV / (dz dOmega), which is maybe what you were thinking.

Hi  @hsinfan1996,
Thanks a lot!
Yes, I was reading it as dV/(dzdOmega).
So, we can ignore it.

Ok, thanks! I'll close this then.


Python API

I find the idea behind this project really cool. Are you planning to create Python API for Vectorflow? I think, that would be really great and would raise the number of users greatly.

Thank you. We don't have any plan/bandwidth at the moment to offer a Python API (see https://github.com/Netflix/vectorflow/issues/4 for more details) as the library heavily relies on some of D compile-time features to offer an efficient way to iterate over the data and feed it to the DAG.
However if you're familiar with SWIG, it's fairly easy to offer a minimalist API. All you have to do is extern(C) or extern(C++) the basic vectorflow functions you need and let SWIG do the rest.


Implement a ticketing system.

[ ] Have id for each case.

[ ] Able close each case id.

[ ]

Hi @naveenfrancis, apologies, I couldn't understand what you meant here.
Github already has a ticketing system implemented as Issues and gives a unique ID to them as well.

Each request should have a ticket id like PTM0001. We need able to call the requestor and close the case.
Request are piling up. Now it looks like 5500 requests. Many are calling the same requestor called bt volunteers again and again.
Not ticketing for github.
Sample ticket PTM0001, TCR0020

Hijacking/piggybacking this discussion. Can we move to an already well developed system specifically for this purpose? I think it might be best to use Eden from the Sahana Foundation projects because it is well tested in disaster scenarios and provides lots of features. Additionally, they do provide hosted Eden instances too so we can simply contact them and get it done.
Most features we think of will be already implemented there. We can customize and self-host Eden instance if required than relying on a hosting provided by them.

Any ticketing mechanism would fine now. If it is tested well good. Let's use that.

@dnivra @naveenfrancis if  it is good then we can test and move  to it

I have not used Eden and from a brief glance, it seems to have good features. We might have to investigate what features it offers and then perhaps implement additional features we need. Sahana Foundation might also offer some volunteers for this purpose: experienced developers who could help.

Demo site if anyone wants to check it out: demo.sahanafoundation.org

@dnivra  it looks goods, Location tracking is also available

Please let me know if anyone need help from DevOps kind of activities: ping me here<EMAIL_ADDRESS>

Can you assign this to me?
I'm planning to implement something like this:

If you're aim for something like:
Volunteer calls existing requests, confirms request is fullfilled, marks as fullfilled
Then we already have it, it is only given to District Project Managers  of KSITMand Volunteer POCs
We shall seek how to optimize the work flow.

@biswaz @naveenpf  Can we get access to the volunteers list on the field asap ? that will help us to route the request manually for the time being. please share your whatsapp number.

better to just use a paginated listing and assigning to volunteer leaders based on location can help.

right @bobinson , i am trying it with a cloud based app. if it works we can integrate that with this website.

@naveenpf
userid: naveenpf
pwd:12345
https://dev58610.service-now.com/login.do
Try this
also availble as mobile application.so easy for eveyone. we can ask international volunteers to input the whatsapp or facebook help msg here
@bobinson @biswaz

Created a Telegram group here : https://t.me/RescueKerala

@biswaz - please join and will give the admin access.

Already District officials are taking the requests and updating.
If the status update is given public, officials are worried about wrong updates.
Still discussions are going.
Please wait for update

agree.
And also if there is someway for the authorities to prioritize areas that will be good I believe. The context is some one stuck in area is asked to wait for 1.5 hours as the rescue is happening in an emergency basis. Which means prioritization of locations can be helpful for authorities. (wishlist)

Is there an admin interface for Changing status for each case ("help requests")?

Is anyone working on this?

Hi please assign this to me. Will be working on it

@naveen could we get some insight on this issue please.

@alokbabu I have just started looking at this and yes will be working on this. Shall upload the pr asap.

@naveenpf @vjMagar @alokbabu There should be an admin view to update status as well as sync up reference from any other tracking sheet. #157

We are planning to implement.
-Each rescuecase once logged, has to create a ticket. (external app)
-Volunteers can only update tickets.
-Only Districts officials will close the ticket.
Currently District officials handling the requests.

So here is the base idea.
We create new app, naming it ticket management.
Model shall have FK to request

Sound good

We make new app naming it ticket management
It's shall have one model,
Ticket -> FK to request model.
Assuming that only district office have access they are the admin stuff, using this logic to restrict the status option on can see and update.
Now on views we shall have different two panel one showing details of tickets, and then second panel to update the status and all.
And display on second panel shall be based on the access user have. Eg volunteer can update ticket, and so.
We shall have another view for district officers for quick response. This shall show the status of all requests with district wise filters and all. (Similar to admin dashboard)

@naveenpf which external system you are going to use over here?

@naveenpf  Is there any way we can get the volunteer list ? so that we can manually assist the request for the time being ?

@naveenpf  can you pls ping me in whatsapp ? I m on +919745653615.  or pls share yours I shall reach out to you

what's the status of this?

hi i have just reached home and have started with it. @jishnunand would you like to join in so that we can complete this faster?

Guys, Please look at this
dev58610
userid: test_user
password:12345
also , mobile application available
Playstore -> ServiceNow (Download the application) and use the same credentials .
Its quite easier and already available in app. Volunteers and other people who is abroad also can work as it is on cloud. Request you all to try and deploy it faster rather than coding it from scratch

GOK officials internal requirements on this. It will be developed by internal team.
If help is needed I will update.

It might make sense to integrate #246 in this system.

Ticketing System - Usecases

UA1: How to prioritise requets ?
UB1: How to get geo-location?
UB3: How to track a request is handled ?
UA2: We need to avoid duplicate requests -- this is an important issue as people requesting help are requesting help via multiple methods using social media, whatsapp etc as well via TV/Radio channels etc. This creates dupicate requests.
UA3: Scheduling - the requests are handled on a priority basis. ie if some location is under threat, that request is first taken care by the volunteers. This means, the post-poned requests will get missed out. We need a mechanism to schedule.
UA4: Verify resucue - We need to update the status of rescue and close duplicates.
UA5: Location Priority : locations undere water should be prioritized. This should be ideally done with the feedback from the volunteers on field.
UJ1: SMS / USSD linking for OTPs and send confirmations etc. (low priority)
UT1: Last Known Location from GSM towers (in many cases the last known location from the Telecom HLR can help to traingulate  a person with upto 4 meters of accuracy. Will need help from Telcos. )

Talked this through with @naveenfrancis in slack, this is the current requirements/plan:
Requirements: volunteers need to be able to update the status of a request, to indicate if it's being handled or already handled.
Plan:

Add a descriptive request ID to the request (Eg: EKM1234)
Add a page that accepts a request ID and allows looking up and updating the status of that request. Collect a name, phone number, and comments/actions taken when updating.

There might be more things we want here, but this seems to be the immediate priority. Jump in #ticketingsystem in slack if you want to help!

Can requestee phone number be the basis of closing /updating a case?

Guys , What is the status of this one . If this feature is not implemeted , you can use this .
https://keralafights.com/manage/
We have pulled the data from keralarescue.in and created a ticketing system here.
As of now we have 30K tickets there .  Right now , we can make use of community to do a dedupe check , call and confirmation and corresponding status updates and resolved , the Govt officials can later verify the closure.
We have more than 12000 requests coming in as duplicates. The work flow what we implemented is what was mentioned on the SOS pages.
Along with every status update , we keep track of a work log as well.
All of the data is exposed as API's and we are keeping a remote_id to refer the ids of the tickets on keralarescue.in as well.  So any time you can sync the data back to the rescue kerala.

@giju  Great Work bro...
One suggestion. Can you make an open portal(page) where somebody can just come in and put their mobile number and mark safe. No need to display the request or anything. Just capture mobile number and allow them to mark safe. This we can share to relief centers(+) and ask them to update.
And then we can use these mobile numbers to put a separate status for the related request.
(And another note: In dashboard, selecting external and clicking next button on the bottom keeps loading..)

Thanks @nikhildev4nk - Will do and update everyone.

I have opened this https://github.com/IEEEKeralaSection/rescuekerala/issues/449 and got linked to this...
https://keralafights.com/manage/ - This looks great.
Option to filter the records with location is important.
My parents were rescued from Chengannur 2 days ago after 2 days in water... No Electricity, Network Coverage in any of these critical areas.. Can there be a status like "UNKOWN" or something?
'Status' is required field and none of those are applicable in a critical situation.
Also, very critical to provide the option to enter google map link. In my opinion, there is no other trusted sources to provide the coordinates to helicopter rescue. I have requested this issue also earlier - related to this. https://github.com/IEEEKeralaSection/rescuekerala/issues/252

please consider the criticality, There is already questions from national/international media to Major Kishore how the location is passed to them. soon things will come back to this request and in my openion, this should be rolled to production immediately so that relatives can start mark/change the status and pin the locations of their lowed ones

If the status can change only by authorities, we may didn't get correct status. Let relatives or friends can update status, and only authorised person can verify that post.
In person i have requested for my friend who's in aarattupuzha, and now he's safe. But the request still there.

clearly of the topic here but can you also include an option or as a seperate thread to add cleaning materials to a specified list of panchayats where the flood has receeded but left a great aftermath.i think it would be good for cleaning up and stuff


Add -v cli flag to build step to control amount of build output

Currently, the CMAKE_VERBOSE_MAKEFILE=ON setting in the zephyr build step creates a huge amount of build output. For most use-cases this provides very little additional information while obfuscating and hiding warnings and errors from own code.
This commit only introduces this flag and implements it for the zephyr rtos, but it can obviously later get used by other RTOS build scripts.

Super useful, should we set the variable with a default value?

The default value i implemented here is UROS_VERBOSE_BUILD=off, as i believe there are not so many cases where you actually want the verbose cmake output. This default is set both in the build_firmware.sh script and in the build.sh zephyr-specific script.

Nice, I will merge when the CI pass. Thanks a lot!

thanks, really nice to have this extremely short response time :+1:

Thanks to you for being one of the first contributors to the micro-ROS tooling and build system.
Looking forward to hearing about your use cases and projects!


Version 184 to 195 : Items to add

Entities ID 374 to 384,
Except 379 which is the Canteen Full

Canteen Wiki page is not created, so it's postpone at the moment,
Behemoth Stone Dinosaur Gateway : image missing, item information added.


Automated Package Upgrade

:tada: This PR is included in version 25.6.8 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Need to add exception handling all over the filesystem for when things are used outside of app

lots of crashing all of the time

http://stackoverflow.com/questions/12690287/does-system-io-filesystemwatchers-created-event-fire-synchronously-or-asynchron


Home Accessories updating slowly

Problem/Motivation
When launching the Home App, all Home Assistant accessories (added via Homedridge addon) are taking several seconds to update their status (up to 10 most times), whilst the ones added natively like Eve Elgato or Philips Hue lights are readily available for interaction. This is becoming very frustrating to deal with, especially considering that there are instances where things are running fast and smooth but more times than not, I am stuck looking at my iPhone screen waiting for the device to be ready so I can turn it on/off.
I am not sure if this is a known issue or it's simply expected behaviour. I did some research online and couldn't fine any conclusive answer/solution to my problem hence this Issue post.
My Hardware setup:

Raspberry Pi 3 Model B
SanDisk 32GB Extreme PRO Micro SD 667x R100/W90
Pi powered by a 5V 2.5A power supply
Internet 200 Mbps Fibre Optic connection
Google Wifi Mesh network (Pi connected to the Primary Point via cat7 ethernet cable)
iPhone X

My Software setup:

Hassio 0.60
Supervisor 0.79
Host OS 1.1
Homebridge addon 2.2.0
iOS 11.2.1

Homebridge addon Options:
{
  "log_level": "info",
  "avahi_interfaces": "",
  "avahi_hostname": "",
  "avahi_domainname": "local",
  "enable_ipv6": false,
  "packages": [],
  "init_commands": [],
  "plugins": []
}

Homebridge config.json:
{
  "bridge": {
    "name": "Home Assistant",
    "username": "B8:27:EB:A8:84:FA",
    "port": 51826,
    "pin": "087-70-688"
  },
  "description": "Homebridge for Home Assistant",
  "accessories": [],
  "platforms": [
    {
      "platform": "HomeAssistant",
      "name": "HomeAssistant",
      "host": "https://MY_HA.duckdns.org:8123",
      "password": "MY_PASSWORD",
      "default_visibility": "visible",
      "supported_types": [
        "automation",
        "binary_sensor",
        "climate",
        "cover",
        "device_tracker",
        "fan",
        "group",
        "light",
        "lock",
        "media_player",
        "remote",
        "scene",
        "script",
        "sensor",
        "switch"
      ],
      "logging": true,
      "verify_ssl": false
    }
  ]
}

It's worth noting that I initially had the "host" set to http://hassio/homeassistant, password was empty and "verify_ssl" set to true, however that was giving me the worse results. I also tested switching around "enable_ipv6", I tried both true and false, but didn't see any noticeable difference.
Expected behaviour
Been able to interact with accessories immediately (or at least within a few seconds) after launching the Home App.
Actual behavior
Every time I launch the Home App, I need to wait up to 10 seconds for the Home Assistant accessories to be available since they 'stuck' on Updating, as per below screenshot (please note that the kitchen oven is showing correctly since it's NOT a HA device - it's in fact connected to a Bluetooth Eve Elgato smart switch):

Steps to reproduce

Have Hassio and Homebridge 2.2.0 addon installed
After correctly setting up Homebridge in iOS, launch Home App
Look out for any delays in the accessories status

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

I had the same issue. Changing the URL from http://hassio/homeassistant to my DuckDNS address sped things up.