Bad tf tree

@spuetz
I wanted to  run mesh navigation in simulation.
So, According to the tutorial, I used following command.
roslaunch pluto_gazebo pluto_botanical_garden.launch
roslaunch pluto_navigation botanical_garden_osnabrueck.launch
But, I can't run mesh navigation.
I checked my tf tree. Then I noticed that tf between map and odom doesn't exist and Strange tf tree exists.
In general, tf tree of ros navigation is " map ==> odom ==> base_footprint "
Please tell me the solution to this problem
frames.pdf

The transformation map -> base_footprint is published from Gazebo. If you have odomerty in between just use your own config. Btw. mesh navigation is still under construction / development. Cheers.

@spuetz
Thanks for your reply.
if you don't mind, Please show me your tf _frames and rqt_graph when you run mesh navigation using pluto in simulation.
I want to use them as a reference.

I may have solved this problem. I set value of  enable_odom_tf to false and add     to pluto_gazebo.launch.
Before (pluto_gazeo.launch)
<node name="gazebo" pkg="gazebo_ros" type="$(arg script_type)" respawn="$(arg respawn_gazebo)" output="screen"         args="$(arg command_arg1) $(arg command_arg2) $(arg command_arg3) -e $(arg physics) $(arg extra_gazebo_args) $(arg world_name)">     <remap from="/pluto_velocity_controller/cmd_vel" to="/cmd_vel"/>     <param name="/pluto_velocity_controller/enable_odom_tf" value="true"/>
After (pluto_gazeo.launch)
<node name="gazebo" pkg="gazebo_ros" type="$(arg script_type)" respawn="$(arg respawn_gazebo)" output="screen"         args="$(arg command_arg1) $(arg command_arg2) $(arg command_arg3) -e $(arg physics) $(arg extra_gazebo_args) $(arg world_name)">     <remap from="/pluto_velocity_controller/cmd_vel" to="/cmd_vel"/>     <param name="/pluto_velocity_controller/enable_odom_tf" value="false"/>     <remap from="tf" to="gazebo_tf"/>
After_frames.pdf


Add initial implementation based on prior work in EXtra-data

Let's go into a third round.
Based on earlier work and discussions here and here, here now a polished version in its own cleaned up repository.
Short primer, as seen in the README, the idea is to provide easy parallelization in a functional programming style transparently on anything iterable while taking care of details like handling shared memory across processes:
import numpy as np
import pasha as psh

# Get some random input data
inp = np.random.rand(100)

# Allocate output array via EXtra-pasha.
outp = psh.array(100)

# Define a kernel function multiplying each value with 3.
def triple_it(worker_id, index, value):
    outp[index] = 3 * value

# Map the kernel function.
psh.map(triple_it, inp)

# Check the result
np.testing.assert_allclose(outp, inp*3)

A major selling point is of course the transparent support for EXtra-data's types, e.g. (including reduction)
import numpy as np
import extra_data as xd
import pasha as psh

ion_trace_src = ('SQS_DIGITIZER_UTC1/ADC/1:network',
                 'digitizers.channel_1_A.raw.samples')
run = xd.open_run(2310, 490) \
      .select_trains(xd.by_index[:500]) \
      .select(*ion_trace_src)

def sum_kernel(wid, tid, data):
    averages[wid] += data[ion_trace_src[0]][ion_trace_src[1]] \
                     .astype(np.float64)

averages = psh.array_per_worker(run[ion_trace_src].entry_shape, dtype=np.float64)
psh.map(sum_kernel, run)
averages = averages.sum(axis=0) / len(run.train_ids)

New compared to philsmt/pashmap#1 is the support for extra_data.KeyData and a new way of adding pasha support to custom types by implementing a _pasha_functor_() method (Thanks, Thomas). In the long run, this should also be the preferred way of adding pasha support for EXtra-data.
Given it's in no way bound to EuXFEL, I found it more fitting to skip the EXtra- prefix here.
Any thoughts?

Given it's in no way bound to EuXFEL, I found it more fitting to skip the EXtra- prefix here.

This seems somewhat contrary to "A major selling point is of course the transparent support for EXtra-data's types" :wink: I understand that your aim is to move all of the specifics of that into EXtra-data, but if a key selling point is how conveniently you can use it with EuXFEL data, and the EuXFEL DA group is going to use & support it, it's kind of EuXFEL-related.
Anyway, names are the eternal bikeshedding issue, so I won't push this. I think you should consider putting it in the European XFEL Github organisation, whatever you prefer to call it.
I'll have a look at the implementation now.

It would be nice if map would return an iterator. That is maybe hard to return output a value or slice of the output array, but at least a counter and a worker id (and a trainId)?

If map returns anything, I'd expect it to contain/represent the return values of the function somehow - that's common across map as a builtin function, in multiprocessing, concurrent.futures, Dask, ipyparallel, etc. I'd accept that, as pasha is designed to use shared memory, map doesn't return/yield anything, but it would feel a bit weird for it to yield some other information.

I also think the project should move to the European-XFEL organisation... (I also prefer the EXtra- prefix in the name

I agree. I think I said so before, so I won't labour the point, but I think it's appropriate for pasha to join our group projects. The two pieces don't have to go together - it can live under the European-XFEL organisation and still be called pasha if you prefer.

Thanks for the review, other Thomas!

It would be nice if map would return an iterator. That is maybe hard to return output a value or slice of the output array, but at least a counter and a worker id (and a trainId)?

As Thomas (K.) mentions, this sounds somewhat unintuitive for the sake of returning something related to the map. As map itself is part of Context, it has no concept of length (defined by Functor by the way it splits itself) or indexing (defined by Functor by the way it iterates).
I have the faint idea of somehow passing output arrays to map(), returning the same number (and order) of values in your kernel and letting it to the assignment in shared memory. But even then you'd need to allocate the arrays manually beforehand and there'd be little point in returning them again.
One could think of some kind of auto allocation scheme, e.g. define the output array shape and its dtype and map() allocates and returns them automatically. But it's all just band-aids for specific cases, and I'd prefer a separate API for that.

I back @takluyver on splitting tasks in n parts (observed non-optimal distribution while testing with long running tasks)

As it turns out, my function was actually buggy under some circumstances 🙈

I also think the project should move to the European-XFEL organisation, it would gain in visibility with staff and users, and shows that it is a tool designed to work with our data and we support.
(I also prefer the EXtra- prefix in the name as it shows it is part of the extra- galaxy ; as it is compatible with DataCollections. But also it's a bit more generic than that, so I don't care that much 🤷 )

Usually, I'm hesitant to call something EXtra- when it contains nothing specific to our systems or infrastructure (similar to metropc). As it turned out in the review process, even the EXtra-data integrations can now be moved into EXtra-data itself. What's left here is generic and may be used in a completely different context than analyzing our data. It feels odd to then carry the name tag around.
How about we'd expose it as an optional part of the extra_data package? With a stub sub-package simply doing from pasha import *, there wouldn't be an automatic import with EXtra-data, but it's still directly connected to us for most of "our" users as another means of parallelizing analysis with it.
from extra_data import open_run
from extra_data.pasha import map

run = open_run(proposal=123, run=456)
map(cool_analysis, run['AGIPD', 'module1.image'])

What's left here is generic and may be used in a completely different context than analyzing our data. It feels odd to then carry the name tag around.

I don't think 'EXtra' has to mean it's EuXFEL specific functionality - it can simply mean it's something created & supported as part of our work. EXtra-geom isn't actually EuXFEL specific - another facility with a JUNGFRAU detector could use it - though it's obviously not as generic as this.
I appreciate your preference, though. I'd still encourage you to put it in the EuXFEL Github org regardless of the name.

How about we'd expose it as an optional part of the extra_data package? With a stub sub-package simply doing from pasha import *...

I'm -0.5 on this - it would be confusing to make it look like part of EXtra-data when it's really a separate project.

How about we'd expose it as an optional part of the extra_data package? With a stub sub-package simply doing from pasha import *...

I'm -0.5 on this - it would be confusing to make it look like part of EXtra-data when it's really a separate project.

While thinking about arguments to push you by that +0.5, I conviced myself of the opposite 🙃
I think I'd like to stick with the name without EXtra- while still moving it to the EuropeanXFEL repositories. I'm happy to list it as part of in-house solutions wherever we do, though, such as a section in the data analysis documentation.
It's your fault really, it's the idea of magic _pasha_functor_ methods that pushed me beyond the edge 😀

I think I'd like to stick with the name without EXtra- while still moving it to the EuropeanXFEL repositories.

I'm happy with that. :slightly_smiling_face:

It's your fault really, it's the idea of magic pasha_functor methods that pushed me beyond the edge :grinning:

:smiling_imp:


Added withoutConnecting flag to RoomProvider

Added option withoutConnecting to skip the initial connection on mount of <RoomProvider /> component.
<RoomProvider id={roomId} withoutConnecting>
  <App />
</RoomProvider>

We're using the internal option to skip the connection in SSR mode:
https://github.com/liveblocks/liveblocks/blob/528ff8d3d0d589d0078dddf340c9b22fc655f4f2/packages/liveblocks-react/src/factory.tsx#L396-L402
https://github.com/liveblocks/liveblocks/blob/c70fed4818f243dc476fbb6a9d7286568f6e6486/packages/liveblocks-client/src/client.ts#L23-L28
Related to https://github.com/liveblocks/liveblocks/discussions/477

First of all, sorry for not getting back to you sooner, we completely missed that. And thank you for submitting this PR!
Before we can move further with this, I asked a few questions on this discussion to make sure to understand your needs properly.

This is now fixed by #517 and published as part of 0.18.4. Thanks for your initial bug report and contributions here, @kachar 🙏 !


Add Sdk version by default to all telemetry. Delete SdkVersionTelemet…

…ryInitializer so it cannot be removed by customer which would result in not adding the prefix.

Current coverage is 81.65%

Merging #218 into develop will increase coverage by +0.05% as of a12da8d

@@            develop    #218   diff @@
=======================================
  Files           129     128     -1
  Stmts          4196    4193     -3
  Branches        548     548
  Methods           0       0
=======================================
  Hit            3424    3424
+ Partial         179     177     -2
+ Missed          593     592     -1

Review entire Coverage Diff as of a12da8d

Powered by Codecov. Updated on successful CI builds.

:shipit:


Make nacos dynamic configcenter work

Make nacos dynamic configcenter work

Codecov Report

Merging #525 into develop will decrease coverage by 0.11%.
The diff coverage is 0%.

@@             Coverage Diff              @@
##             develop    #525      +/-   ##
============================================
- Coverage      21.52%   21.4%   -0.12%
  Complexity       303     303
============================================
  Files             84      84
  Lines           3428    3447      +19
  Branches         544     547       +3
============================================
  Hits             738     738
- Misses          2614    2633      +19
  Partials          76      76

Impacted Files
Coverage Δ
Complexity Δ

...admin/registry/config/impl/NacosConfiguration.java
1.17% <0%> (-0.34%)
1 <0> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b976ba2...c869636. Read the comment docs.

do not merge first

Please resolve the conflicts first

Sorry, because you didn't reply for a long time, we will close this PR first.
Please feel free to reopen it if you have any question.


Icons for sidebar collapse/expand

This change uses bi-arrow-bar-left and bi-arrow-bar-right.  This is a minimal change, but changes from .
I started off with just changing the open one to use bi-layout-sidebar, which is the mirror of the collapsed icon.  I also considered changing the icon that shows when collapsed to bi-layout-sidebar-text-reverse.  Both of those seemed less clear than these.
Note that the "shown"/"collapsed" classes on these elements appear to be reversed.  I didn't touch that.

LGTM

Note that the "shown"/"collapsed" classes on these elements appear to be reversed. I didn't touch that.

If you did want to fix that, I wouldn't mind...


Add link to resetting account during provisioning

Adds another row to the device list during provisioning that leads to an account reset link. Also tweaks the wording slightly on desktop.

And after clicking the row:

@keybase/react-hackers cc @malgorithms @patrickxb

The commits 7bb3376d1575...8b49ec4bffa5 introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)

Looks good.  RESET MY ACCOUNT goes to website, I presume?
We could do stronger wording to tell the user what happens with a reset.  But the website also will tell them, so maybe not necessary:
"RESET MY ACCOUNT AND START FROM SCRATCH"
or
"DISABLE ALL MY EXISTING DEVICES AND RESET MY ACCOUNT"

The commits 7bb3376d1575...48d8a73b863b introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)

The commits 7bb3376d1575...97313e71125b introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)

@patrickxb 'RESET MY ACCOUNT' goes to https://keybase.io/#account-reset.
How's this?

I'm hesitant to get more specific because the website has a good spiel about the specifics and we can't fit more than 'RESET MY ACCOUNT' on mobile.

Text looks good to me, but I defer to @malgorithms for any text...
One question:  what happens after they reset their account (or not) and come back to the app?  Are they still on this screen or back at the first login screen?

Text looks good to me, but I defer to @malgorithms for any text...

looks good to me.

One question: what happens after they reset their account (or not) and come back to the app? Are they still on this screen or back at the first login screen?

good question. it seems they should be back at the login screen.

The commits 7bb3376d1575...69c1518fa025 introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)

The commits 7bb3376d1575...6591f0e37bc3 introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)

OK, modified so clicking the link will be equivalent to hitting the back button, going to the Login cancelled screen

talked to @malgorithms about this for a little bit. Lets actually just change the blue text to say 'Reset my account' and let the webpage do the more detailed information

The commits 4e8b1094f6f8...040ab237357b introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)

Done swapping out the div's etc., PTAL @chrisnojima

The commits 4e8b1094f6f8...9f619aaaa25d introduce visual changes on linux. :mag_right: 2 changed

Changed: Select_Other_Device-Normal (before) (after)

Changed: Select_Other_Device-No_User_Pass_Login (before) (after)


Turn off Anti-aliasing

Hi, is it possible to turn off anti-aliasing?
I've tried doing it with css(-webkit-font-smoothing: none), but it doesn't work.
Thanks

Did anyone find a solution for that?
font-smooth: never; doesn't work either.

Yes, I switched to firefox version. There you can simply set a user preference: user_pref("gfx.text.disable-aa", true);

Thanks. But unfortunately that is not a solution for me. I run it on AWS Lambda and haven't succeeded in getting Firefox running there. Isn't there a user preference for Chromium that one can set?

Anyone who has figured it out yet? I print receipts on a thermal printer (via AWS lambda -> base64 -> react native) and they come out somewhat blurry.


Modernize tech stack and add support for node.js 18 and 20

Hi @Prior99, will this PR be merged? If you have time, could you please review it? Thank you.


DO NOT MERGE: Trigger needs rebase bot

This PR is to trigger a merge conflict against master branch in order to test the needs-rebase bot.

Oh this doesn't work, actions don't run if conflicts exist initially.


feat(rootScope): stopImmediatePropagation method

Adding stopImmediatePropagation method to allow stopping of all immediate listeners on the scope. This is similar in behavior to jQuery's stopImmediatePropagation. Decision to put it together came out of this thread - https://github.com/angular/angular.js/issues/3564#issuecomment-23607481. Issue #3564

@IgorMinar well if YOU can't imagine how angular apps would benefit from it then it must be true.


Add some indication of when data is loading from the server

Users need to be made aware when the app is loading data.

This could be done using a singleton loading UI class. The first call creates the UI object. Later calls just display it for a given period.


API requests are throttled

I'm testing the kit on Harvester, after switching to the new OS, I notice the requests to collect cluster (k8s) resources are throttled. It takes more time to generate a support bundle.
logs:
+ exec tini -- /usr/bin/support-bundle-kit manager
time="2021-09-01T04:58:45Z" level=info msg="running phase start"
time="2021-09-01T04:58:45Z" level=info msg="succeed to run phase start. Progress (20)."
time="2021-09-01T04:58:45Z" level=info msg="running phase cluster"
time="2021-09-01T04:58:45Z" level=debug msg="generating cluster bundle..."
W0901 04:58:45.097998       7 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
I0901 04:58:51.664301       7 request.go:655] Throttling request took 1.198813749s, request: GET:https://<IP_ADDRESS>:443/apis/apps/v1?timeout=32s
I0901 04:59:01.863794       7 request.go:655] Throttling request took 11.397921448s, request: GET:https://<IP_ADDRESS>:443/apis/upgrade.cattle.io/v1?timeout=32s
W0901 04:59:08.479082       7 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
I0901 04:59:25.264319       7 request.go:655] Throttling request took 1.198192283s, request: GET:https://<IP_ADDRESS>:443/apis/certificates.k8s.io/v1beta1?timeout=32s
I0901 04:59:35.464259       7 request.go:655] Throttling request took 11.397711805s, request: GET:https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1beta1?timeout=32s
W0901 04:59:42.067340       7 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
I0901 04:59:58.864473       7 request.go:655] Throttling request took 1.198562157s, request: GET:https://<IP_ADDRESS>:443/apis/apiregistration.k8s.io/v1?timeout=32s
I0901 05:00:09.063571       7 request.go:655] Throttling request took 11.397542858s, request: GET:https://<IP_ADDRESS>:443/apis/admissionregistration.k8s.io/v1beta1?timeout=32s
W0901 05:00:15.667231       7 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
I0901 05:00:32.463723       7 request.go:655] Throttling request took 1.197947123s, request: GET:https://<IP_ADDRESS>:443/apis/apiregistration.k8s.io/v1beta1?timeout=32s
I0901 05:00:42.664039       7 request.go:655] Throttling request took 11.397188614s, request: GET:https://<IP_ADDRESS>:443/apis/helm.cattle.io/v1?timeout=32s
W0901 05:00:49.264993       7 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
time="2021-09-01T05:01:16Z" level=info msg="succeed to run phase cluster. Progress (40)."
time="2021-09-01T05:01:16Z" level=info msg="running phase node"
time="2021-09-01T05:01:16Z" level=debug msg="expected bundles from nodes: map[node1: node2: node3: node4:[]"
time="2021-09-01T05:01:16Z" level=debug msg="creating daemonset supportbundle-agent-foo with image rancher/support-bundle-kit:dev"

...

The behavior is expected because the rest-client repeatedly sends API requests to fetch resource information.
Not sure if there is room to improve it.

Do we know what's the threshold? I am wondering if we throttled the request by ourselves, would it be faster to finish all the retrivals.

Do we know what's the threshold? I am wondering if we throttled the request by ourselves, would it be faster to finish all the retrivals.

It looks like the values can be adjusted by the client. Create a PR to fix that.


Fix code scanning alert - Use of a broken or weak cryptographic hashing algorithm on sensitive data

Tracking issue for:

[x] https://github.com/cools9/ElementalDB/security/code-scanning/2

Well if its for auth then im thinking of reniving the auth feature it will added again but later as it slows the db down


net/http: client.Do(request) panics in time.stopTimer

What version of Go are you using (go version)?

$ go version
go version go1.14.1 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build562962017=/tmp/go-build -gno-record-gcc-switches"

What did you do?
Unfortunately it's not possible to post the code here but it should be pretty straightforward to reproduce this.
My application makes concurrent GET requests to another API at around 280 req/sec using goroutines. This application makes about 3 million requests to the API every day which takes about 4-5 hours to finish, however when using the latest version of Go it panics after 10-120 minutes of running (the length of run before panic is not deterministic). Rolling back to Go 1.13.5 fixes this issue.
I have seen other threads here that refer to the same issue in older versions of Go, however this seems to have re-appeared in 1.14.1 again. I'm not sure if it's for a different reason this time.

What did you expect to see?
200 OK
What did you see instead?
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x451cad]

goroutine 646692 [running]:
time.stopTimer(0xc00035fa48, 0x0)
	/usr/local/go/src/runtime/time.go:224 +0x2b
time.(*Timer).Stop(...)
	/usr/local/go/src/time/sleep.go:78
net/http.(*http2ClientConn).roundTrip(0xc001e68780, 0xc001e8c900, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/h2_bundle.go:7480 +0x9f
net/http.(*http2Transport).RoundTripOpt(0xc0001965a0, 0xc001e8c900, 0x9b3800, 0xc0001af080, 0xc000809770, 0x5)
	/usr/local/go/src/net/http/h2_bundle.go:6948 +0x16f
net/http.(*http2Transport).RoundTrip(...)
	/usr/local/go/src/net/http/h2_bundle.go:6909
net/http.http2noDialH2RoundTripper.RoundTrip(0xc0001965a0, 0xc001e8c900, 0xb53460, 0xc0001965a0, 0x0)
	/usr/local/go/src/net/http/h2_bundle.go:9103 +0x3e
net/http.(*Transport).roundTrip(0xf4e580, 0xc001e8c900, 0xc002c91e60, 0xc001080840, 0x40c7f8)
	/usr/local/go/src/net/http/transport.go:515 +0xd94
net/http.(*Transport).RoundTrip(0xf4e580, 0xc001e8c900, 0xf4e580, 0x0, 0x0)
	/usr/local/go/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc001e8c900, 0xb52420, 0xf4e580, 0x0, 0x0, 0x0, 0xc00073c2f8, 0xc00002a015, 0x1, 0x0)
	/usr/local/go/src/net/http/client.go:252 +0x43e
net/http.(*Client).send(0xf58ba0, 0xc001e8c900, 0x0, 0x0, 0x0, 0xc00073c2f8, 0x0, 0x1, 0xc001e8c900)
	/usr/local/go/src/net/http/client.go:176 +0xfa
net/http.(*Client).do(0xf58ba0, 0xc001e8c900, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:699 +0x44a
net/http.(*Client).Do(...)
	/usr/local/go/src/net/http/client.go:567

I have seen other threads here that refer to the same issue in older versions of Go, however this seems to have re-appeared in 1.14.1 again. I'm not sure if it's for a different reason this time.

Can you point to these threads/issues?

@bradfitz @bcmills @rsc

This sounds like #37400 (specifically, the symptom reported in #37490), but the fix for that (by @ianlancetaylor) was released in Go 1.14.1 (#37494).

Milestoning to 1.15 based on similarlity to #37400.

IMO this should be 1.14.2, this is a massive regression from 1.13.x, I ran into the same issue with the exact backtrace.

I can tell from the line numbers that the stack backtrace in the original report is from 1.14, not 1.14.1, so I'm fairly certain that this was fixed by the fix for #37494.
@OneOfOne Do you still see this when using the 1.14.4 release?  Can you show us the backtrace that you get?  Thanks.


Magnetometer Values are wrong on NodeMcu esp8266 when used with ROS

I Have used I2C communication on the nodemcu with MPU9250 . The output is correct when i run the example sketches. The board works well with Ros serial and its Wireless implementation as well (that is i have tried it out and it sends in any value fine). The problem is reading of one of the axis of magnetometer is like -1e+34. I checked the reading using serial terminal while ros topic was being published through wifi and it was this big value being read. This is the code i am using.
#include "MPU9250.h"
#include <ros.h>
#include <sensor_msgs/Imu.h>
#include <sensor_msgs/MagneticField.h>
#include <std_msgs/String.h>
#include <Wire.h>
#ifdef ESP8266
  #include <ESP8266WiFi.h>
  const char* ssid = "No";
  const char* pass = "abcdq1987654";
  IPAddress server(192,168,43,18);
  const uint16_t serverPort = 11411;
#endif
MPU9250* IMU;
int stat;
ros::NodeHandle nh;

sensor_msgs::Imu msg_dat;
sensor_msgs::MagneticField mag_dat;
ros::Publisher raw("imu/data_raw",&msg_dat);
ros::Publisher mag("imu/mag",&mag_dat);
void setup() {
#ifdef ESP8266
  Wire.begin(); //D1,D2);
  Serial.begin(115200);
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid,pass);
  while(WiFi.status()!=WL_CONNECTED){
      delay(500);
      Serial.print(".");
    }
  nh.getHardware() -> setConnection(server,serverPort);
  nh.initNode();
  Serial.print("IP = ");
  Serial.println(nh.getHardware()->getLocalIP());

#else
  Wire.begin();
  nh.initNode();
#endif
  IMU = new MPU9250(&Wire,(uint8_t)0x68);
  while(!nh.connected()){
      nh.spinOnce();
    }
  stat = IMU->begin();
  char buff[7];
  if(stat<0){
      nh.logfatal("IMU initialization unsuccessful");
      nh.logerror("Check IMU wiring or try cycling power");
      itoa(stat,buff,10);
      nh.loginfo(strcat("Status: ",buff));
      while(1){nh.spinOnce();delay(1000);}  // Nodemcu doesnt like empty while loops
    }
  nh.advertise(raw);
  nh.advertise(mag);
  msg_dat.header.frame_id="/case_link";
  mag_dat.header.frame_id="/chase_link";
  msg_dat.orientation.x=0;msg_dat.orientation.y=0;msg_dat.orientation.z=0;msg_dat.orientation.w=0;
  for(int i=0;i<9;i++)msg_dat.orientation_covariance[i] =-1;
  for(int i=0;i<9;i++)msg_dat.angular_velocity_covariance[i]=0;
  for(int i=0;i<9;i++)msg_dat.linear_acceleration_covariance[i] = 0;
  for(int i=0;i<9;i++)mag_dat.magnetic_field_covariance[i] = 0;
}

void loop() {
  if(nh.connected()){
    IMU->readSensor();
    msg_dat.angular_velocity.x = IMU->getGyroX_rads();
    msg_dat.angular_velocity.y = IMU->getGyroY_rads();
    msg_dat.angular_velocity.z = IMU->getGyroZ_rads();
    msg_dat.linear_acceleration.x = IMU->getAccelX_mss();
    msg_dat.linear_acceleration.y = IMU->getAccelY_mss();
    msg_dat.linear_acceleration.z = IMU->getAccelZ_mss();
    mag_dat.magnetic_field.x = IMU->getMagX_uT();
    mag_dat.magnetic_field.y = IMU->getMagY_uT();
    mag_dat.magnetic_field.z = IMU->getMagZ_uT();
    mag_dat.header.stamp = nh.now();
    msg_dat.header.stamp = nh.now();
    mag.publish(&mag_dat);
    raw.publish(&msg_dat);
  }else{
    Serial.println(".");
    delay(100);
  }
  nh.spinOnce();
  delay(50);
}

And a Important thing to tell is that i modded the library files to takein the pointer of the  Twowire object and not run _i2c->begin , so that i would not be running it twice  begin as i tried different ports of Wire.begin(D5,D6) and soon.
I believe the problem is to do with the value of float being used or interpreted and calculated in a way suitable for a 8 bit Processor and not generalizing for 32 Bit Processor and due to some hickups in performance the value is being calculated wrong or being bit shifted.
This is a demo output of ROS node that i am receiveing from it. The problem only exits for magnetometer and not for Accel or gyro. Thanks in advanced.
---
header:
  seq: 15151
  stamp:
    secs:<PHONE_NUMBER>
    nsecs: 921860963
  frame_id: "/chase_link"
magnetic_field:
  x: 24.4541053772
  y: 84.6991043091
  z: -1.03845937171e+34
magnetic_field_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
---
header:
  seq: 15152
  stamp:
    secs:<PHONE_NUMBER>
    nsecs: 973860963
  frame_id: "/chase_link"
magnetic_field:
  x: 24.4541053772
  y: 84.6991043091
  z: -1.03845937171e+34
magnetic_field_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
---

All of our libraries were developed and tested on the Teensy 3.x and LC line of 32 bit microcontrollers. I don't have any experience with ESP8266 or ROS. I would try to see if the unmodified library example outputs reasonable magnetometer values to the serial monitor as a starting point for debugging.

Yes the modification doesnt make any difference both work perfectly as expected even the ros topics work but the code that has both together just fails to work by giving a random value like this. I initially thought ROS took up the heap memory overflow there where wrong values outputted.
Executable segment sizes:
IROM   : 256700          - code in flash         (default or ICACHE_FLASH_ATTR)
IRAM   : 28220   / 32768 - code in IRAM          (ICACHE_RAM_ATTR, ISRs...)
DATA   : 1268  )         - initialized variables (global, static) in RAM/HEAP
RODATA : 1452  ) / 81920 - constants             (global, static) in RAM/HEAP
BSS    : 28736 )         - zeroed variables      (global, static) in RAM/HEAP

But that is not the case as well i thought there were some pins that was intefering with I2C but i disabled anykind of serial communication from ROS libs as well only have one serial port (the standard Uart over usb one) at 11500 the one that you can see in the code. I thought many cases were it would be caused in the ros side of the things but i couldnt find one. So i what i was thinking is that there is some error in calculation from the processor as it may be liitle too resource intensive or something but that didnt make any sense to me . So for now i have been debugging this for a pretty good amount of time and i can say ROS is working MPU9250 library is working but them together is causing some kind of interference. I even tried it with Arduino Mega where in i was getting 2 values with this mutated form (with the ros obviously the them seperately work well and ros delievers the right message as i tried trivial zeros and ones which it passes through well but the sensor reading  is messaged up). So could you say me if there is some specific memory locations that your library tries to access that is not allocated and reserved for itself so that i can debug further
Thanking you

All memory allocation occurs in the constructor and the library doesn't access memory outside of its class.

Yes I believe the problem is not the MPU9250 library but the esp8266/arduino compiler as WiFi and I2C interfere #1331 was a issue in there repo. And i have not been able to trackdown the version of it which worked for them . I did try SPI as well but i guess i have no more luck down this lane.


Limiting number of edges displayed

In the Edge Explorer tab, the browser sometimes freezes when querying nodes with lots of edges. Would it make sense to set a limit on the number of edges displayed somehow?

Edge explorer is shelved for now so closing.


SNPmatch error

Dear Rahul,
Hello,
I tried to use SNPmatch but it generated error.
Can you please take a look?
``
/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/csmatch.py:52: RuntimeWarning: invalid value encountered in less
NumAmb = np.where(likeliHoodRatio < snpmatch.lr_thres)[0]
2018-05-10 14:56:04,744 - root - ERROR - The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Traceback (most recent call last):
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/init.py", line 163, in main
args'func'
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/init.py", line 105, in snpmatch_cross
csmatch.potatoCrossIdentifier(args)
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/csmatch.py", line 232, in potatoCrossIdentifier
crossIdentifier(args['binLen'],snpCHR, snpPOS, snpWEI, DPmean, GenotypeData, GenotypeData_acc, args['outFile'])
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/csmatch.py", line 187, in crossIdentifier
(ScoreList, NumInfoSites, NumMatSNPs) = crossWindower(binLen, snpCHR, snpPOS, snpWEI, DPmean, GenotypeData, outFile)
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/csmatch.py", line 101, in crossWindower
writeBinData(out_file, i, GenotypeData, ScoreList, NumInfoSites)
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/csmatch.py", line 50, in writeBinData
(likeliScore, likeliHoodRatio) = snpmatch.calculate_likelihoods(ScoreList, NumInfoSites)
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/snpmatch.py", line 41, in calculate_likelihoods
LikeLiHoods = [likeliTest(NumInfoSites[i], int(ScoreList[i])) for i in range(num_lines)]
File "/mnt/d/scratch/bin/python_default/local/lib/python2.7/site-packages/snpmatch/core/snpmatch.py", line 27, in likeliTest
if n > 0 and n != y and y > 0:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

I use pip version and it doesn't work.
But with git cone version 2.0, it worked.
AraGeno is web version that i use.
By the way, can you tell me about SNPmatch output?
Thanks.
Won

Won Cheol Yim
Research Scientist
+1<PHONE_NUMBER> |<EMAIL_ADDRESS>| http://www.plantbioinformatics.org/
MS330/Department of Biochemistry & Molecular Biology
1664 N. Virginia Street, Reno NV
University of Nevada - Reno
89557

From: Rahul Pisupati<EMAIL_ADDRESS>Sent: Friday, May 11, 2018 4:24:00 AM
To: Gregor-Mendel-Institute/SNPmatch
Cc: Won C Yim; Mention
Subject: Re: [Gregor-Mendel-Institute/SNPmatch] SNPmatch error (#8)
Hi @ascendohttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fascendo&data=01|01|wyim%40unr.edu|b52f7a76e8d04a59cb2d08d5b731b206|523b4bfc0ebd4c03b2b96f6a17fd31d8|1&sdata=MWdUV504%2FbGl98IellKGZ9WlnQUNbtxo5hLRQJLbIaA%3D&reserved=0 ,
Can you provide a sample file on which you are running SNPmatch? Is this the error on the AraGeno or standalone version?
Rahul
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FGregor-Mendel-Institute%2FSNPmatch%2Fissues%2F8%23issuecomment-388336780&data=01|01|wyim%40unr.edu|b52f7a76e8d04a59cb2d08d5b731b206|523b4bfc0ebd4c03b2b96f6a17fd31d8|1&sdata=K2E9KzAh5ZJksE%2Ft76rYjEx8IQ96Ewi8MRLnDWu9R68%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAA3XIebY8Jx7HBrraX8AjYiVLJedpK9cks5txXTQgaJpZM4T6nXp&data=01|01|wyim%40unr.edu|b52f7a76e8d04a59cb2d08d5b731b206|523b4bfc0ebd4c03b2b96f6a17fd31d8|1&sdata=SSwf2tkl6N5RXVR0UguIPYsaRtp6jmQCqDxj449MvCI%3D&reserved=0.

Hi Won,
We will update version on AraGeno soon and with a detailed output in the README file.
Briefly, SNPmatch output contains accession number, number of SNPs that were matched compared to total SNPs and likelihood scores. Json file generated contains only matched accessions with 95% confidence based on likelihood ratio test.
Rahul

@timeu can you update AraGeno to version 2.0 of SNPmatch..? Thanks


Xenserver 6.5 support?

Was wondering if this would work with xenserver 6.5?

I've not tested it directly; but I also don't see any updates for 6.5 in the XML feed yet, so we won't truely know until they release a patch (assuming they haven't)  :)

They released a patch now. What do I have to do to test it/make it work?
Thx in advance

I tested the code its working on xenserver 6.5.
How to do it.
download the patcher.py on your xenserver under tmp
wget --no-check-certificate -O patcher.py https://raw.github.com/dalgibbard/citrix_xenserver_patcher/master/patcher.py
chmod +x patcher.py
./patcher.py
Below are my screenshot from xenserver 6.5
./patcher.py
Getting host list using: /opt/xensource/bin/xe host-list name-label=grep "^HOSTNAME=" /etc/sysconfig/network | awk -F= '{print$2}' params=uuid --minimal
Detected HOST UUID: 180f8b5a-d232-458a-ab44-05a282de8e27
Get patch list using: /opt/xensource/bin/xe patch-list hosts="180f8b5a-d232-458a-ab44-05a282de8e27" --minimal
The following Patches are pending installation:
uuid: 5200911d-5f79-4149-abca-0556af77b14d
url: http://support.citrix.com/article/CTX142061
timestamp: 2015-02-18T00:00:00Z
name_label: XS65E003
patch_url: http://downloadns.citrix.com.edgesuite.net/10196/XS65E003.zip
after_apply_guidance: restartHost
name_description: Public Availability: security fixes to glibc
uuid: 492ca007-bf7b-454f-8e5c-63a991a52449
url: http://support.citrix.com/article/CTX142141
timestamp: 2015-03-03T00:00:00Z
name_label: XS65E005
patch_url: http://downloadns.citrix.com.edgesuite.net/10201/XS65E005.zip
after_apply_guidance:
name_description: Public Availability: Storage fixes to prevent disk corruption
uuid: 30d3992b-ac0a-45e8-9e93-d4b2e9bb235f
url: http://support.citrix.com/article/CTX142147
timestamp: 2015-03-11T00:00:00Z
name_label: XS65E006
patch_url: http://downloadns.citrix.com.edgesuite.net/10207/XS65E006.zip
after_apply_guidance: restartHost
name_description: Public Availability: security fixes to Xen
uuid: d3c08fcb-daa0-4410-bdb2-c298109e88ad
url: http://support.citrix.com/article/CTX142266
timestamp: 2015-04-08T00:00:00Z
name_label: XS65E008
patch_url: http://downloadns.citrix.com.edgesuite.net/10320/XS65E008.zip
after_apply_guidance: restartHost
name_description: Public Availability: Fix for Dom0 kernel
NOTE: Installation of these items will require a reboot!
You will be prompted to reboot at the end.
Would you like to install these items? [y/n]: y
Starting patching...
Downloading: XS65E003.zip
Download Size: 31742971 Bytes
31742971  [100.00%]
Applying: XS65E003
Uncompressing...
Internal Upload...
Completed: 5200911d-5f79-4149-abca-0556af77b14d
Applying Patch 5200911d-5f79-4149-abca-0556af77b14d
/opt/xensource/bin/xe patch-apply uuid=5200911d-5f79-4149-abca-0556af77b14d host-uuid=180f8b5a-d232-458a-ab44-05a282de8e27
Downloading: XS65E005.zip
Download Size: 3316830 Bytes
3316830  [100.00%]
Applying: XS65E005
Uncompressing...
Internal Upload...
Completed: 492ca007-bf7b-454f-8e5c-63a991a52449
Applying Patch 492ca007-bf7b-454f-8e5c-63a991a52449
/opt/xensource/bin/xe patch-apply uuid=492ca007-bf7b-454f-8e5c-63a991a52449 host-uuid=180f8b5a-d232-458a-ab44-05a282de8e27
Downloading: XS65E006.zip
Download Size: 10675554 Bytes
10675554  [100.00%]
Applying: XS65E006
Uncompressing...
Internal Upload...
Completed: 30d3992b-ac0a-45e8-9e93-d4b2e9bb235f
Applying Patch 30d3992b-ac0a-45e8-9e93-d4b2e9bb235f
/opt/xensource/bin/xe patch-apply uuid=30d3992b-ac0a-45e8-9e93-d4b2e9bb235f host-uuid=180f8b5a-d232-458a-ab44-05a282de8e27
Downloading: XS65E008.zip
Download Size: 108511340 Bytes
108511340  [100.00%]
Applying: XS65E008
Uncompressing...
Internal Upload...
Completed: d3c08fcb-daa0-4410-bdb2-c298109e88ad
Applying Patch d3c08fcb-daa0-4410-bdb2-c298109e88ad
/opt/xensource/bin/xe patch-apply uuid=d3c08fcb-daa0-4410-bdb2-c298109e88ad host-uuid=180f8b5a-d232-458a-ab44-05a282de8e27
A reboot is now required. Would you like to reboot now? [y/n]: y
Rebooting...


Remove Launchpad client

Closes #947

[x] Remove Launchpad support from faucet
[x] Rewrite examples/delegate.ts to use the stargate backend
[x] Rewrite examples/local_faucet.ts to use the stargate backend and adjust packages/cli/README.md
[x] Remove Launchpad support from cli
[x] Remove launchpad tests from CI
[x] Remove launchpad package
[x] Remove launchpad scripts
[x] Update references in docs

Code changes look good.
But wh are there so many new zip files if you only removed dependenciey? Also why did you change checksumBehavior?

Had to delete yarn cache and yarn install again because tests didn't work anymore (checksum errors). If there's a better solution, please let me know.
checksumBehaviorwas changed for local debugging reasons. Changed it back now.

Why is .yarn/cache committed to the repo anyway?

Why is .yarn/cache committed to the repo anyway?

That's the best practice way to handle dependencies in yarn 2+. This way, you always have all dependencies available in your repo and don't need to download them. They are zipped and stored in git LFS. "cache" is a questionable name though.

That's the best practice way to handle dependencies in yarn 2+. This way, you always have all dependencies available in your repo and don't need to download them.

Let's download the internet in case we need it 😄

I ran into all kind of mess trying to check out this PR. I could fix it as follows:
git checkout -b rm-launchpad-v2
wget https://patch-diff.githubusercontent.com/raw/cosmos/cosmjs/pull/1015.diff
git apply --exclude=".yarn/cache/*" 1015.diff

Then you can yarn && yarn build with no change to the zip files at all. Will follow-up with a fresh PR.


KubeWatchApi

Closed by #4653 which was merged successfully but due some glitch status didn't update.


Running 'cargo -h' does not display the clippy subcommand.

Problem
I tried this code:
cargo -h

I expected to see this happen:
...
Some common cargo commands are (see all commands with --list):
    build, b    Compile the current package
    check, c    Analyze the current package and report errors, but don't build object files
    clean       Remove the target directory
    clippy      <An introduction to the clippy command>
...

Instead, this happened:
...
Some common cargo commands are (see all commands with --list):
    build, b    Compile the current package
    check, c    Analyze the current package and report errors, but don't build object files
    clean       Remove the target directory
...

Steps
in windows, run 'cargo -h'.
Possible Solution(s)
I believe you should be able to add the description directly at this link:https://github.com/rust-lang/cargo/blob/a6c0d43787269d48510c0a4e8acf2207790a6daa/src/bin/cargo/cli.rs#LL465C9-L465C9.
Notes
No response
Version
rustc 1.69.0 (84c898d65 2023-04-16)
binary: rustc
commit-hash: 84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc
commit-date: 2023-04-16
host: x86_64-pc-windows-msvc
release: 1.69.0
LLVM version: 15.0.7

Thanks for the report!
Although cargo-clippy is an official external command, it doesn't always come with Cargo if you installed Rust via rustup. If you choose minimal profile when downloading a toolchain, it won't contain cargo-clippy nor cargo-fmt component. I guess that was one of the reason that it's intentionally not included in the cargo -h help manual.
When you run cargo +1.69.0 --list | grep clippy to see a full list of commands, cargo clippy should still be there. This keeps short help manual concise — if people want more info they always have a way to do so.

thank you very much, I see!


Piet wgsl wasm

Run cargo run-wasm piet-wgsl from the root of the repository and navigate to http://localhost:8000 on Chrome Canary to view the demo.
There are a few rough edges that I would like to clean up in a follow-up eventually, in particular:

Reorganize the wasm vs non-wasm example code
Make the shader assembly code that has to use include_str! more compact. Shared and non-shared versions of the shaders currently need to be enumerated in two different places. The code could be made tighter using a procedural macro
Add a texture output alternative to the current out_buf with a buffer usage flag. The blit from the output buffer to the swapchain texture currently involves some hand-coded resizing for the ImageCopyBuffer layout and extent to align with the output buffer dimensions and row stride.

I'll close this with thanks, as it's subsumed by #220 (which adapted a lot of code in here) and pretty much working in the browser.


Add or Convert to .NET Core

We should convert the aaLogReader and aaLogWebAPI projects to .NET Core at a minimum. I've started looking into it.

@ryanvs so sorry I didn't see this come across - for some reason I didn't get notified.  I think it's a great idea to bring the codebase up to something a little more current.  I don't have a lot of spare cycles right now but open to a PR if you get something you are happy with.

I don't have a lot of spare cycles either, but I'll try to find some time here and there and see if I can get it done.


How do I find shortest path via Java API and can I limit the search range?

Currently I'm using this approach:
"select shortestPath("+ sourceId + ", "+ destinationId +")"
But I'm only interested in certain depth and this seems to exhaust every possibility (which is a overkill for me)

For search range what do you mean? Can you do a concrete example of what you'd like to get?

I would like to limit the number of hops/degrees the shortest path can go (I'm only interested to know if a relationship is within X degrees)

Meaning: I need to know if the two entities are linked over, for example, 4 degrees or less. I do not need a complete/exhaustive search to happen that returns a path of 20 hops/degrees and I'm trying to avoid the time that it takes. We have ~40 million entities in our db and density is an issue so I would also like to limit the Edge types that can be traversed in search of the shortest path.

@luigidellaquila WDYT about supporting a limit on this function? Or is there any other way, for example with the MATCH to have the same goal?

I think shortestPath() function can be enhanced with many parameters, one of them is for sure the max depth.
MATCH statement is not optimized (yet) to calculate shortest paths, you can write a query to get the same result but the elaboration time will be much higher than shortest path
Thanks
Luigi

Ok, so this is not an option.  The time spent will be determined by the density of the graph so I would really appreciate a query that would achieve this. The only other option is for us to create our own traversing using the Java API and I would like to avoid that at all cost.

Ok, let me see what I can do on shortestPath(), maybe I can quickly enhance it
Thanks
Luigi

Hi @acmeguy
I think I have a solution for this, I'm running all the tests and then I'll push it to 2.1.x branch
Thanks
Luigi

Great stuff, thank you :)

Ok, pushed on both 2.1.x and develop branches.
Now shortestPath() accepts a new (fifth) parameter that is a map and can contain multiple options.
The only supported option is "maxDepth" right now:
SELECT shortestPath(#8:32, #8:10, null, null, {"maxDepth": 5})

Here are full docs
https://github.com/orientechnologies/orientdb-docs/blob/master/SQL-Functions.md#shortestpath
The enhancement will be released with 2.1.12
Thanks
Luigi

thank you!


Nonzerorepr expr

tempted to merge, even though it isn't the prettiest, it seems to function and I believe I have cought every case that could lead to zero.


Only show announcement every 30 days

I asked on Slack and no one seems to be against it. I'll wait a bit longer to see if someone says anything.


Add constants to the class and replace static text with constants

If you can fix these and get the tests passing I'd love to get this merged!
PS: you can run the unit tests locally with phpunit

@Crisfole your comments make sense! I've updated this PR

Looks fantastic! Waiting on the tests, if/when they pass I'd say this is a go.
@brianr eyes?

LGTM

Merging later today along with a few others and a minor version bump.


Increase tolerances in substract_vec_mat test

Summary
We are still experiencing some issues on develop after https://github.com/stan-dev/math/pull/1469 was merged. This increases the tolerances for another test. If new test failures come up then I would say we should increase the tolerance globally. I will also investigate what causes the "randomness" of this tests that does not use any randomness whatsoever. Might be limited to Windows.
Tests
This increases the tolerances in the tests.
Side Effects
/
Checklist

[x] Math issue #1453

[x] Copyright holder: Rok Češnovar

[x] the basic tests are passing

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

I'm guessing it's a different compiler or settings or different hardware.

On Dec 4, 2019, at 3:55 AM, Rok Češnovar<EMAIL_ADDRESS>wrote:
Summary
We are still experiencing some issues on develop after #1469 was merged. This increases the tolerances for another test. If new test failures come up then I would say we should increase the tolerance globally. I will also investigate what causes the "randomness" of this tests that does not use any randomness whatsoever. Might be limited to Windows.
Tests
This increases the tolerances in the tests.
Side Effects
/
Checklist
•  Math issue #1453
•  Copyright holder: Rok Češnovar
•  the basic tests are passing
•  the code is written in idiomatic C++ and changes are documented in the doxygen
•  the new changes are tested
You can view, comment on, or merge this pull request online at:
https://github.com/stan-dev/math/pull/1483
Commit Summary
• increase tolerances in substract_vec_mat test
File Changes
• M test/unit/math/mix/mat/fun/subtract_test.cpp (7)
Patch Links:
• https://github.com/stan-dev/math/pull/1483.patch
• https://github.com/stan-dev/math/pull/1483.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

This is ready for review. I will examine the strange windows behavior in tests in a separate issue/PR. This should be merged to prevent Jenkins issues for other developers.


Design/implement user security model

Write the medrina policy, define all of the roles, etc.

This is done.


DownmixMono "channels_first" kwarg is unexpected keyword

transform = torchaudio.transforms.DownmixMono(channels_first=True)
 __init__() got an unexpected keyword argument 'channels_first'

Hash : d92de5b

Hmm that's strange. It works for me
>>> torchaudio.transforms.DownmixMono(channels_first=True)
DownmixMono()

and it should work since it is in the code:
https://github.com/pytorch/audio/blob/d92de5b/torchaudio/transforms.py#L108
and it was added a while back.
Could you run test_transforms.py and see if an error appears?
python -m unittest test/test_transforms.py

Running:
python -m unittest test/test_transforms.py
gives me
RuntimeError: Failed to parse the argument list of a type annotation: name 'Optional' is not defined

What would be the proper method to install torchaudio on colab?
@jamarshon please help.

@deepwilson I ran this on a Python3 colab and it worked. Let me know if it does 😄, although you may have to adjust the pytorch installation depending on desired Python and Cuda versions.
# Install dependencies
!apt-get install sox libsox-dev libsox-fmt-all

# Install pytorch 1.1.0 (https://pytorch.org/). Depends on Python and Cuda versions so look for correct commands
!pip3 install https://download.pytorch.org/whl/cpu/torch-1.1.0-cp36-cp36m-linux_x86_64.whl
!pip3 install https://download.pytorch.org/whl/cpu/torchvision-0.3.0-cp36-cp36m-linux_x86_64.whl

# install torchaudio
!pip install<EMAIL_ADDRESS>
# Run torchaudio
import torchaudio
transform = torchaudio.transforms.DownmixMono(channels_first=True)
print(transform)  # prints DownmixMono()
print(torchaudio.__version__)  # prints 0.2.0a0+7d7342f

Thank you so much @jamarshon !
I ran the following code which was wrong:
!apt-get install sox libsox-dev libsox-fmt-all
!git clone https://github.com/pytorch/audio.git
import os
os.chdir("audio")
!git checkout 5c9d33d  #301e2e9 d92de5b
!python setup.py install
import torchaudio

Correct code/procedure to follow as per your instructions:
import sys
sys.version
'3.6.8 (default, Jan 14 2019, 11:02:34) \n[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]'

!cat /usr/local/cuda/version.txt
CUDA Version 10.0.130

# Install dependencies
!apt-get install sox libsox-dev libsox-fmt-all

Go to the following link and check for links corresponding to your python and CUDA versions

!pip3 install https://download.pytorch.org/whl/cu100/torch-1.1.0-cp36-cp36m-linux_x86_64.whl
!pip3 install https://download.pytorch.org/whl/cu100/torchvision-0.3.0-cp36-cp36m-linux_x86_64.whl

# install torchaudio
!pip install<EMAIL_ADDRESS>
# Run torchaudio
import torchaudio
transform = torchaudio.transforms.DownmixMono(channels_first=True)
print(transform)  # prints DownmixMono()
print(torchaudio.__version__)  # prints 0.2.0a0+7d7342f

Thanks once again @jamarshon !

@deepwilson Glad to hear it worked 😄


Add document validation for Python code

The Python console app needs to validate the input document. Please follow validation rules from documentation.

Hi, I'd like to try!

Hi, I'd like to try!

Please go ahead, and submit your pull request!

Closed on PR #26


[V1] Optimize the CPU overheads in FlashAttention custom op

With piece-wise CUDA graphs, we have to make sure that the attention custom op causes minimal CPU overheads. This PR made a few changes to optimize the CPU overheads in the FlashAttention custom op:

We directly use torch.ops.vllm_flash_attn_c.varlen_fwd rather than flash_attn_varlen_func, since FlashAttnFunc which inherits torch.autograd.Function causes unnecessary overheads.
We move the reshapes and shape check logics to outside of the custom op, so that they can be done at the CUDA graph capture time.

Results of python benchmarks/benchmark_latency.py (opt-125m) on a single H100 GPU:

V1 main: 227 ms
V1 this PR: 192 ms
V0 + 8-step: 130 ms

Next step: further reduce the unnecessary CPU ops inside the FlashAttention op.

well, I think I forgot to update the v1 flash attention file, after https://github.com/vllm-project/vllm/pull/10558/ , you don't need the torch.ops.vllm.unified_v1_flash_attention call.

@youkaichao @mgoin As we merged https://github.com/vllm-project/flash-attention/pull/30, we don't have to directly use torch.ops.vllm_flash_attn_c.varlen_fwd. We can just use flash_attn_varlen_func as we currently do. Both V0 and V1 already gets the benefits after https://github.com/vllm-project/flash-attention/pull/30.

One weird phenomenon I found is that V1 has a spike in latency:
Avg latency: 0.20093455887205589 seconds
10% percentile latency: 0.1931818482640665 seconds
25% percentile latency: 0.19354040725738741 seconds
50% percentile latency: 0.19391279752017 seconds
75% percentile latency: 0.19426249974640086 seconds
90% percentile latency: 0.1961068181961309 seconds
99% percentile latency: 0.3368887884780999 seconds

This is highly reproducible on my dev machine. Can this be because of Python gc or something like that?

One weird phenomenon I found is that V1 has a spike in latency:
Avg latency: 0.20093455887205589 seconds
10% percentile latency: 0.1931818482640665 seconds
25% percentile latency: 0.19354040725738741 seconds
50% percentile latency: 0.19391279752017 seconds
75% percentile latency: 0.19426249974640086 seconds
90% percentile latency: 0.1961068181961309 seconds
99% percentile latency: 0.3368887884780999 seconds

This is highly reproducible on my dev machine. Can this be because of Python gc or something like that?

It’s probably the prefix caching …

Hmm but benchmark_latency.py does sample each prompts separately: https://github.com/vllm-project/vllm/blob/main/benchmarks/benchmark_latency.py#L36

Hmm but benchmark_latency.py does sample each prompts separately: https://github.com/vllm-project/vllm/blob/main/benchmarks/benchmark_latency.py#L36

Just found that it has a warmup phase. It's still possible due to prefix caching if all prompts are cached. Suggest to explicitly disable prefix caching to double check.

@comaniac @robertgshaw2-neuralmagic You're right. The latency becomes stable when prefix caching is turned off.
Avg latency: 0.1945609479948568 seconds
10% percentile latency: 0.19310778125654907 seconds
25% percentile latency: 0.19390572598786093 seconds
50% percentile latency: 0.19475348049309105 seconds
75% percentile latency: 0.195164829317946 seconds
90% percentile latency: 0.19570096801035106 seconds
99% percentile latency: 0.1962820820847992 seconds


Include License

Hello! Which license do you support for this project?

Just added MIT license.

Thank you for the prompt response! That was surprisingly fast.


Expose query building blocks

Great! I assume these are necessary because Pyjnius has trouble with statics? If so, a quick note to document would be great!
Also, please merge in latest master?


Make ImageReference strongly-typed and checked up-front

Turn ImageReference into its own type so it is validated early on in request submission time, and we don't end up with malformed IDs, etc.
Add in support for shared image galleries since that was easy enough to add while I'm doing this.
Explicitly document which image sources are permitted.

This addresses/closes #1464 for the C# port.

Need to do some validation that shared images work as expected before merging.

Waiting for #2375 to merge in order to rebase this on top.

Codecov Report

Merging #2369 (d060da8) into main (4cc4de9) will increase coverage by 4.27%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #2369      +/-   ##
==========================================
+ Coverage   31.03%   35.31%   +4.27%
==========================================
  Files         254      135     -119
  Lines       28753    16959   -11794
==========================================
- Hits         8924     5989    -2935
+ Misses      19829    10970    -8859

Impacted Files
Coverage Δ

src/ApiService/ApiService/OneFuzzTypes/Model.cs

src/ApiService/ApiService/OneFuzzTypes/Requests.cs

...rc/ApiService/ApiService/OneFuzzTypes/Responses.cs

src/ApiService/ApiService/Program.cs

...ApiService/ApiService/onefuzzlib/OnefuzzContext.cs

...piService/ApiService/onefuzzlib/ProxyOperations.cs

...piService/ApiService/onefuzzlib/ReproOperations.cs

...ervice/ApiService/onefuzzlib/ScalesetOperations.cs

src/ApiService/ApiService/onefuzzlib/Scheduler.cs

...ApiService/ApiService/onefuzzlib/TaskOperations.cs

... and 108 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Doing manual verification, I can create scalesets with:

[] /subscriptions/{id}/resourceGroups/{rg}/providers/Microsoft.Compute/galleries/{gallery}/images/{image}
[] /subscriptions/{id}/resourceGroups/{rg}/providers/Microsoft.Compute/galleries/{gallery}/images/{image}/versions/{version}


Added action to Github-Actions List

Hey @mariamrf
Thanks for the contribution over on github-actions site.
I have added a badge if you are interested.
Keep up the good work 👍 🎉

Cool, thank you!


runtime: mallocs cause "base outside usable address space" panic when running on iOS 14

What version of Go are you using (go version)?

$ go version
go version go1.16.5 darwin/amd64 (also reproducible on 1.15 and 1.14)

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
ios/arm64
What did you do?
Create a simple Go package, doing a few medium sized allocs (creating a few byte slices of several tens of MB will do, as long as it forces the Go runtime to ask the OS for more memory), compile it with gomobile to an iOS .framework file, use the resulting framework in an iOS project and compile and run for iOS 14.
If the iOS project doesn't do a lot of memory allocs itself, everything works fine.
If the iOS project does use reserve a lot of memory (either for CPU or GPU) before initialising the Go framework, the Go runtime crashes with the following panic.
runtime: memory allocated by OS [0x2ac000000, 0x2b0000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

Note that the exact amount of memory needed seems to be device dependent, although in all cases the panic happens way before the typical OOM point for the device in question. Issues was reproduced using iPhone 12 (allocating 1.1GB of RAM in the iOS project is sufficient to make it crash), iPhone 12 Pro and iPhone SE 2020. Note that everything works fine when compiling with pre-iOS 14 versions of XCode.
Although I'm way out of my league here, the addresses indicated in the panic seem to indicate that iOS no longer has a 33-bit memory address limit, as assumed by the Go runtime. Perhaps related to iPhone 12 Pro having 6GB of RAM, which would mean the 4GB limit assumed by the Go runtime would no longer be sufficient?

/cc @mknyszek @hyangah

Although I'm way out of my league here, the addresses indicated in the panic seem to indicate that iOS no longer has a 33-bit memory address limit, as assumed by the Go runtime. Perhaps related to iPhone 12 Pro having 6GB of RAM, which would mean the 4GB limit assumed by the Go runtime would no longer be sufficient?

This might be the case. To be honest, I couldn't find any documentation about this. This was the best I could infer from experiments and the experiments of others I found. Basically, the assumption I was operating under is that each process has 4 GiB of address space, starting from the 4 GiB offset (the bottom 4 GiB are reserved).
The amount of RAM on the phone theoretically shouldn't matter, because I think (but cannot say for sure) iOS keeps each process's reserved address space somewhat limited, IIUC. The aforementioned experiments revealed you could barely make a 2 GiB mapping (PROT_NONE!) without iOS saying we were out of memory. But, I think you're right that something changed in iOS 14.
One solution here is to just run new experiments and increase the size of the address space that we assume. I'll try to think of something better, but until there's actually any documentation on this, I think the address space structure we have is maybe not a great fit for a platform that doesn't document the size of its available address space. Sigh.

Thanks @mknyszek.
i did some searching on changes in iOS 14, and it seems Apple introduced something called Extended Virtual Address Space. Details are scarce, but it sounds very related to our problem above.

Maybe this will be of use: https://github.com/apple/darwin-xnu/blob/a1babec6b135d1f35b2590a1990af3c5c5393479/bsd/kern/kern_exec.c#L2928

Update: I can reproduce with an empty iOS project, and having the Go library allocate e.g. 2GB of data on it's own. Instead of the alloc succeeding, or throwing an OOM error, it throws:
runtime: memory allocated by OS [0x2ac000000, 0x2b0000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

@mknyszek Given that Go 1.13 doesn't suffer from the problem, I was able to trace the issue back to this single-line commit https://github.com/golang/go/commit/198f0452b0292eb245bb5122510b1dfd0050dacc
I've confirmed that when I change the 33 into 35, everything works fine.
That said, I don't understand this code well enough to determine what side-effects that might have. What are the potential risks of me doing so? Is the base outside usable address space error just an extra safeguard against something that should never happen in practice because the OS would never assign a memory address outside of addressable space? Or is me changing it to 35 a serious risk that could result in runtime panics?

Increasing that number is exactly what I was talking about earlier. The risk with increasing that number naively, though, is that it

causes a larger virtual mapping to be created for a data structure in the runtime, and
slows down access to that data structure slightly (not enough to matter, I think).

This may or may not cause errors in older iOS versions, but I think the mapping for 39 bits should still be relatively small. I should calculate that. The reason why you don't see an issue with Go 1.13 is that I added that in Go 1.14.
My question, however, is: is 39 the right number? What are the actual virtual address space limits when you have the Extended Virtual Addressing entitlement? I can't actually find any documentation on this.

I double checked the extended virtual addressing entitlement, but we actually have it turned off. So my theory is that something changed in the way iOS does memory addressing to allow for that feature to be possible, but those changes are also active when the entitlement is not used.
I'll do some more searching to find the 'right' number. The highest memory address I've seen so far is 0x2b8000000.
But I'm afraid changing it to 39 has serious issues. I've had the following crash on Swift side at least once:
2021-06-25 16:14:36.961698+0200 flat[43222:11907111] Uncaught exception: NSInvalidArgumentException: *** NSAllocateMemoryPages(18446744071865635812) failed
(
    0   CoreFoundation                      0x000000019bbfa5c8 58500388-BF36-397C-84CF-17315A3445B6 + 1217992
    1   libobjc.A.dylib                     0x00000001b06797a8 objc_exception_throw + 60
    2   Foundation                          0x000000019cf81224 NSZoneMalloc + 0
    3   Foundation                          0x000000019ce53f14 63D26DEE-A1FB-34B0-9ADA-CC52E7F1D60C + 57108
    4   Foundation                          0x000000019ce569b8 63D26DEE-A1FB-34B0-9ADA-CC52E7F1D60C + 68024

That failure suggests to me (but I'm not 100% sure) that Swift's malloc is having trouble getting virtual address space. If that's the case, I think 39 is too high.
Thanks for looking into this! Does 35 work for you? I noticed your original reply (which got sent to my email) had 35 instead of 39.

Yes, indeed. I set it to 35 before, but then I noticed it was not high enough; the issue still appeared, albeit at higher memory levels. I'll try a few more values until I find the 'right' one. The main issue is that it is difficult to say for sure when a value is correct. The Swift error above for example only seems to appear when the circumstances are just right and is not easy to reproduce.

Seems like iOS 15 will complicate things even further: https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_kernel_increased-memory-limit?changes=latest_beta

Documenting results of various experiments here:
33-35: Go panic: runtime: memory allocated by OS [0x2ac000000, 0x2b0000000) not in usable address space: base outside usable address space. fatal error: memory reservation exceeds address space limit
37: Go panic: fatal error: out of memory allocating heap arena metadata
39: Swift exception
Uncaught exception: NSInvalidArgumentException: *** NSAllocateMemoryPages(18446744071865635812) failed
(
    0   CoreFoundation                      0x000000019bbfa5c8 58500388-BF36-397C-84CF-17315A3445B6 + 1217992
    1   libobjc.A.dylib                     0x00000001b06797a8 objc_exception_throw + 60
    2   Foundation                          0x000000019cf81224 NSZoneMalloc + 0
    3   Foundation                          0x000000019ce53f14 63D26DEE-A1FB-34B0-9ADA-CC52E7F1D60C + 57108
    4   Foundation                          0x000000019ce569b8 63D26DEE-A1FB-34B0-9ADA-CC52E7F1D60C + 68024

@mknyszek see above. I'm afraid there is no 'good' value. Setting it to 38 instead of 33 makes things better, in the sense that the crash comes later (i.e. higher memory address) than with 33, but we're still not at the real OS memory limit (still a couple of MB of from reaching the OS OOM limit). At 39 though, we get the Swift exception. Not sure how to continue...

@rayvbr It occurs to me that you might want to switch to treating iOS as "64-bit" instead of "32-bit." Just changing the constant results in the aforementioned runtime data structure getting mapped in entirely as read/write. In the "64-bit" case, we make a reservation, and only map what we need. This may be needed to make 39+ work. At that point, I think we should just eliminate all the iOS workarounds in treating it as "32-bit" and assume a 48-bit address space. 2^39 is already so large that I expect address space to be plentiful.
We could do iOS version detection and do things the old way for iOS <14, and remove the workarounds for >=14 (and then later remove the workarounds altogether when iOS <14 is no longer supported), but what's tricky here is the fact that the data structure is configured by compile-time constants. I'd rather not add a compile-time flag for this.
Another alternative is to switch iOS over to the "64-bit" structure, then detect the version. If the version is <14, the runtime can limit the amount of memory actually mapped, but the length of the slices over that mapped memory can be shortened, so that if we ever try to access beyond the corresponding address space what iOS <14 actually supports (say, due to a bug or something), we won't access random memory and will instead panic (which is preferable).
It's a bit of a hack, but this might be a way forward. It also gives us a clear way out for iOS 15 and beyond.
I really wish we were just allowed to know what the actual address space limits are. That would make this sort of planning a whole lot simpler.

Thanks for the explanation @mkevac, we'll continue our investigation. Two short follow up questions:

You mention "you might want to switch to treating iOS as "64-bit" instead of "32-bit." How do I enable 64-bit mode for iOS if I want to test that?
What surprises me in the results above is that when we increased the address size from 33-bit to 38-bit, the memory address signalled in the panic has increased from 0x2ac000000 to 0x2bc000000, which seemingly indicates that we only increased the allowed maximum memory address from ~11.4GB to ~11.77GB (which corresponds to what we see in XCode Profiler). Shouldn't an increase of 5 additional bits normally result in way more addressable memory than that? Doesn't this indicate that something else is going on? Perhaps Apple changing the internal structure of the memory addresses? Or them using a different base offset?

@rayvbr RE: your second point... that's a very good point. I should've looked more closely at your earlier message. I think that might mean there's something else that needs to change. Let me double check and see if I can prepare a patch for you if there's something else that has to change.
RE: your first point, I'll just explain how this works in more detail.
The Go runtime manages a data structure that allocates one bit per 8 KiB of address space (this happens lazily, so is generally not a problem), and 8 bytes per 4 MiB of address space (this bit is mapped up-front). On systems with small address spaces (like 32-bit systems), this data structure is relatively small, just a few KiB. So, we map the whole thing into memory (see mpagealloc_32bit.go). On systems with large address spaces (like amd64, which has a 48-bit address space in practice), this data structure is much larger, using ~6 GiB of address space if memory serves. Of course, we don't want every Go process to consume 6 GiB of memory by default, so we only make a reservation, and then map in new pieces of the data structure as read/write whenever the heap grows (see mpagealloc_64bit.go).
Currently iOS uses mpagealloc_32bit.go's implementation, because the 6 GiB mapping on iOS was causing all Go processes to error out immediately (not enough address space!). Prior to that exception, it was treated like every other arm64 system.
My suggestion, then, was to make iOS use mpagealloc_64bit.go. The way to do this is to just change the go:build lines in those files and remove the exception for iOS in malloc.go (though I think for other reasons we may still want to keep 4 MiB arenas, so this is going to be a little subtle, but still very little code).
However, using mpagealloc_64bit.go on iOS means that older versions of iOS are going to break. So, I was proposing that we use mpagealloc_64bit.go but add some logic to make the mappings smaller only on iOS. It's not ideal, but it's something, and it should be easy to remove that logic once iOS <14 is no longer supported.

@mknyszek, it seems the mpagealloc_64bit.go trick works.
I removed the ios-specific build tags from mpagealloc_64bit.go and mpagealloc_32bit.go, removed the 33-bit exception in malloc.go, and the result is that I can finally use the full available memory on an iPhone 12. When I do cross the app memory limit, I now get a proper fatal error: out of memory error, instead of the memory reservation exceeds address space limit panic. On the Swift side everything looks fine as well, and I don't have any runtime exceptions like when I changed 33 to 39.
Note that I did not touch the following two lines:
case GOARCH == "arm64" && GOOS == "ios":
     p = uintptr(i)<<40 | uintptrMask&(0x0013<<28)

Unfortunately, I don't have any iOS 13 (or earlier) device I can test on. And downgrading seems to be far from trivial. Do you happen to have one?

Hey @mknyszek, any suggestions on best way forward? While we've had the above running in production for some time now, I'd love to at some point go back to using an official Go release, instead of relying on homemade patches, especially where it concerns low-level code like this...

@mknyszek We are facing the same issue on iOS 14. Do you have plans to patch this in the 1.16.x/1.17.x?
The idea of having a DIY-patched Go binary in the CI feels uncomfortable.
Anything we can help with?

I think what this has to look like is switching to the 64-bit implementation on iOS (for all versions) but adding a case to limit the address space used for iOS < 14. I can't think of anything better, and this workaround is pretty unfortunate. The workaround would then disappear once we no longer supported iOS < 14.
I'll ask around this week and see if anyone else has any ideas. I'll also see if it's possible to make a small, safe change so this is fixable in a minor release. For a major release, I'm going to start considering eliminating the separate 32-bit implementation and just have every platform do the dynamic mapping. That should make this fix cleaner going forward.

Actually, I just thought of something better: a combination of what @rayvbr tried before. Something like: make iOS use the 64-bit implementation, but limit it to a 40-bit address space. This should prevent that Swift exception seen earlier, but hopefully is large enough to accommodate the addresses that we saw iOS trying to return earlier. This is a small change that I believe could be backported. @rayvbr @requilence Would y'all be willing to try this out?
Eventually, once old iOS versions are pronounced obsolete, we could promote iOS to a full 48-bit address space like every other arm64-based platform.

Err... I guess one snag is we don't actually know where that Swift exception at 39-bit addresses came from. The amount of additional address space used should only be O(MiB), so I don't fully understand why Swift would have trouble allocating pages. Given that it's the Go runtime is running out of memory sometimes, perhaps that's independent and just chance that Swift does instead?
Anyway, let's try this patch and see how it goes.

Thanks @mknyszek, that sounds great!

@rayvbr @requilence Would y'all be willing to try this out?

Of course!

Hey @mknyszek, we did a thorough set of tests on various devices, and I can report the following.

We can no longer reproduce the not in usable address space: base outside usable address space fatal error: memory reservation exceeds address space limit error when using https://golang.org/cl/344401. We can allocate up until the device limit without issues.
When having Go allocate memory that crosses the physical limit (e.g. 2.1GB on a non-Pro iPhone 12), by repeatedly allocating chunks of 10MB until it crashes, the panic thrown is out of memory allocating heap arena metadata.

I'm not sure if the panic thrown is the expected one or whether you would expect a regular 'out of memory' panic? Either way, it definitely fixes the issues we've been having, and that is great news!
Unfortunately, we were not able to obtain an iOS 13 or earlier device to test on. Perhaps @requilence has one?

Huh. That's not quite the out of memory error I was expecting, but I guess since the arena size is 4 MiB that's the luck of the draw. Or, maybe it's actually more likely than I think, because I'm pretty sure that structure gets zeroed whereas each 10 MiB chunk isn't paged in yet (and we don't need to zero it because we assume the OS does for us).
It might be hard to hit the "real" out-of-memory error on any Darwin-based (or any UNIX-y) system, since the actual out of memory condition depends on what's paged in. In any case, I think this is a good sign.
RE: iOS 13, I'm pretty sure our builders are iOS 13 or below. There's currently a failure, but it's purely test related (a test I forgot to update). As far as I can tell, it works.

Sounds great. I'm not sure what the criteria are for adding something to a minor release vs. waiting for the next major release. Would you say this qualifies for a subsequent 1.17 minor release?

@gopherbot Please open backport issues for 1.16 and 1.17.
I think so. This failure doesn't seem to have a reasonable workaround (use less memory?) and otherwise means iOS >=14 Go apps can just break.

@mknyszek, this patch works for us. BTW, I have only iOS 14 devices right now

I am seeing this same issue on M1 Mac devices, although I'm not familiar enough with the memory allocation logic to know if this would have the same cause as the issue on iOS.
runtime: memory allocated by OS [0x288000000, 0x290000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

runtime stack:
runtime.throw(0x12d4b371d, 0x2e)
	/usr/lib/go-1.15/src/runtime/panic.go:1116 +0x54 fp=0x172a629f0 sp=0x172a629c0 pc=0x12d0369a4
runtime.(*mheap).sysAlloc(0x12d82d540, 0x400000, 0x172a62b08, 0x12d02b298)
	/usr/lib/go-1.15/src/runtime/malloc.go:718 +0x654 fp=0x172a62aa0 sp=0x172a629f0 pc=0x12d012114
runtime.(*mheap).grow(0x12d82d540, 0x1, 0x0)
	/usr/lib/go-1.15/src/runtime/mheap.go:1344 +0x74 fp=0x172a62b10 sp=0x172a62aa0 pc=0x12d02b4c4
runtime.(*mheap).allocSpan(0x12d82d540, 0x1, 0x12d002a00, 0x12d848308, 0x10000012d000000)
	/usr/lib/go-1.15/src/runtime/mheap.go:1160 +0x5d8 fp=0x172a62b90 sp=0x172a62b10 pc=0x12d02b2c8
runtime.(*mheap).alloc.func1()
	/usr/lib/go-1.15/src/runtime/mheap.go:907 +0x58 fp=0x172a62bf0 sp=0x172a62b90 pc=0x12d05ef68
runtime.(*mheap).alloc(0x12d82d540, 0x1, 0x172a6012a, 0x12d01c7a4)
	/usr/lib/go-1.15/src/runtime/mheap.go:901 +0x64 fp=0x172a62c40 sp=0x172a62bf0 pc=0x12d02a924
runtime.(*mcentral).grow(0x12d8303e8, 0x0)
	/usr/lib/go-1.15/src/runtime/mcentral.go:506 +0x74 fp=0x172a62c90 sp=0x172a62c40 pc=0x12d01cb14
runtime.(*mcentral).cacheSpan(0x12d8303e8, 0xbfe62e42fefa39ef)
	/usr/lib/go-1.15/src/runtime/mcentral.go:177 +0x40c fp=0x172a62d20 sp=0x172a62c90 pc=0x12d01c86c
runtime.(*mcache).refill(0x128e58108, 0x2a)
	/usr/lib/go-1.15/src/runtime/mcache.go:142 +0xbc fp=0x172a62d50 sp=0x172a62d20 pc=0x12d01c1dc
runtime.(*mcache).nextFree(0x128e58108, 0xfffffaebbf68752a, 0xfffffc088, 0x172a62df0, 0xfffffaebbf6875cc)
	/usr/lib/go-1.15/src/runtime/malloc.go:880 +0x8c fp=0x172a62da0 sp=0x172a62d50 pc=0x12d01230c
runtime.mallocgc(0x180, 0x12d498ee0, 0x1, 0x12d826040)
	/usr/lib/go-1.15/src/runtime/malloc.go:1061 +0x6d4 fp=0x172a62e60 sp=0x172a62da0 pc=0x12d012b74
runtime.newobject(0x12d498ee0, 0x172a62eb8)
	/usr/lib/go-1.15/src/runtime/malloc.go:1195 +0x38 fp=0x172a62e90 sp=0x172a62e60 pc=0x12d012fa8
runtime.malg(0x100008000, 0x65cc92d2a9a87978)
	/usr/lib/go-1.15/src/runtime/proc.go:3520 +0x2c fp=0x172a62ee0 sp=0x172a62e90 pc=0x12d040fcc
runtime.mpreinit(...)
	/usr/lib/go-1.15/src/runtime/os_darwin.go:285
runtime.mcommoninit(0x12d826040, 0xffffffffffffffff)
	/usr/lib/go-1.15/src/runtime/proc.go:663 +0xe4 fp=0x172a62f30 sp=0x172a62ee0 pc=0x12d03a344
runtime.schedinit()
	/usr/lib/go-1.15/src/runtime/proc.go:565 +0xa4 fp=0x172a62fa0 sp=0x172a62f30 pc=0x12d039ed4
runtime.rt0_go(0x0, 0x0, 0x991700018a49dd3c, 0x0, 0x0, 0x230e060975165a0d, 0x0, 0x0, 0x1729d7010, 0x0, ...)
	/usr/lib/go-1.15/src/runtime/asm_arm64.s:69 +0xb8 fp=0x172a62fd0 sp=0x172a62fa0 pc=0x12d064948

Sorry for the delay here. There's some problem with the patch on our builders, and I'm having trouble accessing the builders to identify the issue. Filed #48772.

Thanks @mknyszek. What are the chances of this making it into the next minor release? Or do you expect it will have to wait for 1.18?

I noticed there is no Go 1.18 label attached to this. Should one be added to make sure it doesn’t go unnoticed?

We think this is potentially release-blocking, let us know what you think.

Actually, I just thought of something better: a combination of what @rayvbr tried before. Something like: make iOS use the 64-bit implementation, but limit it to a 40-bit address space. This should prevent that Swift exception seen earlier, but hopefully is large enough to accommodate the addresses that we saw iOS trying to return earlier. This will likely use 2 MiB or so more address space in iOS <14, but luckily that's not that much and shouldn't impact most applications. This is a small change that I believe could be backported.

A 2 MB increase in address space use may affect the Tailscale app, because iOS limits memory use of their VPN extension. Paging @bradfitz.

I believe iOS 15 increased extension memory limit very significantly (from 15 to 50MB)

@mknyszek, https://go-review.googlesource.com/c/go/+/344401/9 also works for the keybase client build. Would love to see this mainlined.
cc @maxtaco @mmaxim

@eliasnaur @rayvbr The 2 MiB of address space is never committed, so it shouldn't count toward that 15 MiB limit, even on older versions iOS (I think, maybe Tailscale folks can chime in, but I'd be surprised if it's limited it by address space). It just matters in this case because old iOS versions have a very small address space, so we need to keep that small to maintain compatibility across this change in address space size iOS is doing. In the future this won't matter -- it'll just be like any other arm64 platform.
@joshblum Unfortunately I'm still blocked on having a working builder. I'll see if there's anything else we can do about it.

Builders are back. Going to take another stab at this.

Great news: it was just a test issue, and relatively easy to fix at that. Passed the trybots just fine. Hopefully I'll land this soon.
The timeline looks like this: if it lands before the beta goes out, then it'll be in that, otherwise it'll be part of the first release candidate in about a month (probably mid-January because of the holidays). I'll get the backported fix in ASAP and that'll also be out in about a month, since the last minor releases just went out yesterday.

Thanks @mknyszek , awesome to see this fixed!

I'm on 1.17.4 and this issue only happens if I have Address Sanitizer enabled on my scheme. I can't risk using beta at this point on my production app so I'll wait till an actual version comes up w/ this fix.

@brunomunizaf Go 1.17.6 released 2 days ago has the fix. (See https://go.dev/doc/devel/release#go1.17.minor and https://github.com/golang/go/issues?q=milestone%3AGo1.17.6+label%3ACherryPickApproved)


Update protobuf to get a fix from upstream

We already have a fix in protbuf https://github.com/google/protobuf/pull/2246
This makes TF build with Bazel at HEAD again.
Fix https://github.com/bazelbuild/bazel/issues/1929
I understand TF is currently using a release version of protobuf, is this change acceptable?
BTW @mrry 's fix will also get fetched by this change
https://github.com/google/protobuf/pull/2203

Can one of the admins verify this patch?

Mr. Jenkins: test this please

We'll find out in a moment if it's acceptable.

Approved. I'll let the sync oncall merge this.
Tangential: I noticed our protobuf rules don't build with Bazel sandboxing enabled. This seems to be related to the upstream fix. But the problem does go deeper with us. I filed #4974.

I just spotted another issue reporting this @rohan100jain. It might be better if I just go ahead and merge this to fix the build.


Does not work at all.

Describe the bug
If you attempt to run WITH grabber on, it brings up a blatant error message telling you that the discord_token_grabber is not working, something like that. If you try WITHOUT grabber, it mentions that on line 3 of source_compiled.py, that pyinput dosent exist. There is probably more errors if i resolve this.
To Reproduce
Steps to reproduce the behavior:

Load config
Make source
Compile
Run it
Expected behavior
Virus to run, and output onto discord.

Screenshots

Desktop (please complete the following information):

OS: Windows 10
Python Version: 3.6.5 (?)

Additional context
Add any other context about the problem here.

I keep seeing errors about not having access?

Updated python to 3.12.7, noticed ui changes but still the same issues. sometimes it compiles but dosent change from .notanexe also there is a ton of permission denied errors. no more password grabber denied perms, but token grabber mainly. could be main cause?

add 266sebee on discord for help


Adding the color to the background of the wallets also with slight tr…

…ansparency

look nice, thanks !


Request to add a "Copy location to subsequent images" button in the upload wizard

What is the user problem or growth opportunity you want to see solved?
As all images of any specific upload will essentially depict the same item, it's imperative that their location would be same. Thus, a "Copy location to subsequent images" button, similar to the "Copy to subsequent media" button for description/caption would be a good feature to have.
How do you know that this problem exists today? Why is this important?
I have experienced this issue myself while uploading images of the same item from different angles but without EXIF data. I have to go through an extra step to manually add the location to each of the images.
Who will benefit from it?
The general users of the app, who use the multi-upload feature of the app to upload images without EXIF data, will greatly benefit from it, as it will reduce the time taken to add a location to each image individually.
Anything else you would like to add?
I would like to work on this enhancement if it's required by the community :)

We already have a mechanism to suggest the coordinates of a picture taken shortly before or after, and we remember the last selected geolocation, so I am not sure a dedicated UI element would justify its screen real estate...

Yeah, we do have a similar image coordinate dialog, but that pops up only when the one image in the previous set of images had EXIF data. If my current set of images do not have any EXIF data, and I add location for the first image, there is no mechanism to copy that to the other images :(

It only takes two quick taps for each picture. 🙂

It only takes two quick taps for each picture. 🙂

Fair point. I just thought it would be good to add such an element to enhance the upload wizard further and fasten the process :)

Huge support to this enhancement. I have taken thousands photos in the Olympics and I'm uploading about 30 photos by match with my phone (I don't have my computer here). The two quick taps by photo are turned into 60 taps by match and are making me crazy.


[eslint] Move from e to event

This one is for me :dizzy:. The event name has far more meaning than a single letter e.
For instance, have a look at the React documentation https://facebook.github.io/react/docs/forms.html.
I will take care of react/jsx-handler-names once this PR is merged.

This must be the happiest day of your life :laughing: :laughing: Looks really good :+1: Thanks a lot :grin:

All cleand and green notwithstanding npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.

This must be the happiest day of your life

We are close to it :roquet:.
Hum, that's weird. Let's merge this before I get any conflict :grin:

npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed

num update && npm install hadn't helped, but deleting node_modules and reinstalling made that go away.


Support turbo bootstrap

Currently it is not ideal to run verification on entire project for packages or apps. Running bootstrap on root should run bootstrap on all projects, however running bootstrap on apps or projects should only bootstrap their dependents. Furthermore, by relying on turbo's architecture we can refactor the bootstrap script to gain filter support. Possibly, even getting rid of it all together.

See commit c68c52b7bc7a26bbdb8c349c6ffd69bd44c7200d


[Admin][Appointments][Dialog][Edit] - Success Scenario #9

resolves https://github.com/users/otaKenneth/projects/7?pane=issue&itemId=41868570

resolves #9


Hotfix to Fix an azure-core-test Bug During Recording

API changes have been detected in com.azure:azure-core. You can review API changes here
API changes
-     public interface ReferenceManager {
-         ReferenceManager INSTANCE = new ReferenceManagerImpl();
-         void register(Object object, Runnable cleanupAction)
-     }


app crash if file url outside xCode project

I printed url in console
document return nil

iOS Version: 13
Xcode Version: 11.5
Swift Version: 5

I just edit to you create link in "PDFDocument"
was ->  URL(fileURLWithPath: filePath, isDirectory: false)
after my edit -> self.fileUrl = URL(string: filePath)


please provide a C/C++ library code for the Raspberry Pi!

please provide also a  C/C++ library code for the Raspberry Pi!
(in case I missed it: please link to the github repository in the overview  if it already might exist)

Github issues here are only for bug reports and pull requests. Please use the Adafruit forums for general discussion, thanks.


error stacktrace not remapped inside launchChromium

Does not work because pupetter does not support them.
see

https://stackoverflow.com/questions/45872698/sourcemaps-in-puppeteer
https://github.com/shannonmoeller/run-headless/issues/1
https://github.com/GoogleChrome/puppeteer/issues/985

The fix would certainly be to use Error.prepareStackTrace client side together with fetch and sourcemap module from firefox.

Moved to https://github.com/jsenv/jsenv-chromium-launcher/issues/1


openthread seems like Contiki

Hi, everyone
maybe you have earned about @contiki operating system  which has the same network architecture present in @openthread ; Maybe I'm wrong but what is the principal difference between  @openthread  and the competitors like @contiki  or @RTOS operating systems.
What is Contiki?

Contiki is an open source operating system for the Internet of Things.
Contiki connects tiny low-cost, low-power microcontrollers to the Internet.
Contiki is a powerful toolbox for building complex wireless systems.

From my understanding, Contiki is an OS, where as OpenThread is a networking stack.  Although OT can be run bare metal (as is common in our example code), it can also be run with an OS like Contiki or FreeRTOS.

Adding to @aeliot's comments, OpenThread is designed to be OS and platform agnostic.  You can read more in a recent blog post.

And as an example of how OpenThread runs in an OS and is not a competitor to something like Contiki, check out the integration of OpenThread with RIOT, another RTOS.

Hi everyone
Thanks you so much for clarify my doubts.
I'm understood  that this openthread is quite interesting and soon I will try to get involved.
again thanks you so much.


[WIP] Expanding the BEM surface when they intersect.

Partial fix to https://github.com/mne-tools/mne-python/issues/3291
I was playing around with the plotter in https://github.com/mne-tools/mne-python/pull/3778 and wanted to try this. This is how it looks for sub002 of the biomag demo.

Before, calling make_bem_model would fail because the outer_skin is inside the outer_skull. This moves the outer_skin slightly outwards.
I'll let you decide if we want something like this.

It's probably hard to figure out from the picture since the face is punched in for anonymity. I think the expansion happened on the left side (lower left part that's sticking out a bit).

Current coverage is 86.36% (diff: 40.00%)

Merging #3785 into master will decrease coverage by 0.01%

@@             master      #3785   diff @@
==========================================
  Files           343        343
  Lines         61111      61120     +9
  Methods           0          0
  Messages          0          0
  Branches       9337       9342     +5
==========================================
- Hits          52786      52784     -2
- Misses         5618       5628    +10
- Partials       2707       2708     +1

Powered by Codecov. Last update a85cd39...ee05b37

I am not convinced it's the right fix. I guess many scenarios can happen depending which surface intersects with what.


Interactive Brokers API 9.72

Last month Interactive Brokers released beta version API 9.72 (release notes at bottom right of the page). This version offers some new functionalities. There is an option to send conditional orders  that get executed only when Time/Price/Volume conditions are met and seems like a very useful feature. I was wondering whether there are any plans of implementing version 9.72 in IbPy. I will be happy to contribute, but my understanding of EClient and ESocket methods is limited.

I believe IbPy2 only support API 9.70 right now. It will be great if we can add support for the current official release 9.71 first. Because IB maintains backward compatibility, this effort won't be wasted when 9.72 is officially released.

Sure thing, I just need to figure out how to best do it.

Let me know if you need any help @fzumstein. Would love to switch over to 9.72.... reqSecDefOptParams in particular would be nice to have.

I would also try to help with this effort.  Agreed about reqSecDefOptParams.
Btw, the makefile that references the IB API version still points to 9.69, however as I understand it the latest release supports 9.70.  This was from this commit Should this be updated in the Makefile?
It also seems that the code generation is no longer done by the end user?  This was the case when I first started using IbPy.

@dam5h what do you mean by "code generation" exactly?

I mean the java to python translation stuff, since much of the python code gets auto-generated from translation from the java API.  I forget what the tool was called (maybe jeeves?) that did this step for you, but installing ibpy used to be more involved than just running setup.py.  I assumed that java2python was doing this now during a build step at installation time, but as I look around it seems like this is not the case.  The end user doesn't need to have the java source at all, is this correct?
I ask because getting up to speed on this would help me to assist with an effort to support 9.72.

I don't know if any of this will help, but I genereated these instructions
(README) when I used to build the IbPy package years ago. You will probably
need to make some modifications to add the new features.
https://github.com/guidov/ib-py
On Mon, Nov 7, 2016 at 8:55 AM, dam5h<EMAIL_ADDRESS>wrote:

I mean the java to python translation stuff, since much of the python code
gets auto-generated from translation from the java API. I forget what the
tool was called (maybe jeeves?) that did this step for you, but installing
ibpy used to be more involved than just running setup.py. I assumed that
java2python was doing this now during a build step at installation time,
but as I look around it seems like this is not the case. The end user
doesn't need to have the java source at all, is this correct?
I ask because getting up to speed on this would help me to assist with an
effort to support 9.72.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/blampe/IbPy/issues/32#issuecomment-258841170, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AC2IML-lAM7gMCq4IThyb7bucAEmrS2Eks5q7y3LgaJpZM4KWg-5
.

I don't think that this was ever necessary to be done by the enduser.
The code conversion is done via https://github.com/natural/java2python
However, the current output is pretty different from the current code base that has been massively manually tweaked. So in the past, I think in most occasions the Java source was diffed and the python code adjusted manually.
I haven't come around to do this and so can't really decide what the best way going forward should be.

Okay got it.  I suppose its possible that I didn't need to do it, but I did it anyway.  This was back in 2008 or so before it was tracked on git.  There were instructions at the time for the code generation steps but that may have just been for thoe wanting to build themselves.
Good to know about the manual edits, a diff against the java and hand edits makes sense as a reasonable approach.  Not sure how big the 9.72 change is from 9.7 but will look.