Feature request - Proximity queries

I was wondering if we currently have something akin to the distance and collision queries supported by Bullet [1] (e.g. getContactPoints, getClosestPoints). Is there any documentation on something equivalent to that in MuJoCo or could that be a potential feature to implemented in the future? Many thanks for the presentation at CoRL!
[1] - https://docs.google.com/document/u/0/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/mobilebasic#h.hrpbzrwzgy1p

Hi, thanks for input!
Regarding getContactPoints, simply iterate over the data->contact array to read out the contact points that have been found in the last call to the dynamics.
Regarding getClosestPoints, finding the distance and/or contact point between two arbitrary geoms -- independently of the dynamics -- there is currently no easy way to do this with the current API, but we can fairly easily add that. I've marked this as a feature request and will update here when we have something.

simply iterate over the data->contact array to read out the contact points

Hello. In what function or part of the cc file would you do this?
I've tried to print out a lot of the data with d-> or m-> and I get Segmentation fault (core dumped)

May I ask has there have been any updates on finding the distance between two arbitrary geoms? Thank you!

Thanks for the reminder!


SMB error : STATUS_USER_SESSION_DELETED

Hello,
I'm having some trouble when using the tool. It worked some days ago, I was able to list shares. But after that success I did not managed to get I working. I get the following error :
python crackmapexec.py IP.AD.DR.ES -u user -d DOMAIN -p password --shares
99-99-1999 99:99:99 [*] IP.AD.DR.ES:445 is running **************** (name:*********) (domain:DOMAIN)
99-99-1999 99:99:99 [-] IP.AD.DR.ES:445 DOMAIN\user:password SMB SessionError: STATUS_USER_SESSION_DELETED(The remote user session has been deleted.)

But it is working with smbclient :
smbclient -W DOMAIN -U user -L IP.AD.DR.ES
Enter user's password:
Domain=[DOMAIN] OS=[*********************] Server=[**********************]

	Sharename       Type      Comment
	---------       ----      -------
	ADMIN$          Disk      ********************
	C$              Disk      *************************
	IPC$            IPC       ********************
Connection to IP.AD.DR.ES failed (Error NT_STATUS_RESOURCE_NAME_NOT_FOUND)
NetBIOS over TCP disabled -- no workgroup available

Could such issue be generated by a special group policy or an antivirus ?
Thanks anyway for this amazing tool dude :D

Sorry for the late response, this could be related to #36, although the error is slightly different, I've tried reproducing this with no success, could you give me some info about the OS your targeting? It shouldn't have anything to do with the AV.
Thanks!

No problem :D if I'm correct it was a Windows 7 professional on x86 architecture. I'm often dealing with strange SMB tools behaviour and I'm pretty sure I cannot help you on this use case :/
Good luck man, and thanks

Closing since this commit https://github.com/CoreSecurity/impacket/commit/dc4606822e489e62d276bc3c928af31cd860f4a0 seems to resolve this issue


Shallow map or deep map?

https://github.com/open-feature/java-sdk/pull/51#discussion_r955642085
Should the Structure object map be shallow or deep copies?

In the use case I hypothesize for this I think that shallow copies make sense. That use case being the transcription of an EvaluationContext into a different provider specific object.
If we did deep copies then structures contained inside other structures would get deep copied multiple times depending on their depth. Any intermediate copies would only be used to deep copy themselves again when asMap was called on them.
So, for that use case, I prefer the shallow copy.

IMO shallow copy is fine, but If the intent is to return a view of the map then Collections.unModifiableMap can be used. But I am not sure if I am being paranoid here!

I think I like the idea of using Collections.unModifiableMap.

AFAIR, Collections.unModifiableMap will only help you by protecting the top-level map.
But Structure can be nested, eg, a Structure could be (pseudocode)
Structure mystruct = {
  topKey: new Structure({
    innerKey: new Value("innerValue")
  })
}

If you were to use unMofifiableMap, then it would protect you from doing things like mystruct.add(...) or any mutation operations, but you could still do mystruct.get("topKey").add("whatever", "stuff") or mystruct.get("topKey").remove("innerKey")


How to make open tag and close tag have the same color?

In your first render method,  <provider> and</Provider>have different color.

nvm. figured out.

But why close the issue without explaining how?

You can customize colors via variables.

But why close the issue without explaining how?

You can customize colors via variables.


Start angle and end angle not supported

This is not customisable want to achieve
something like below this functionality is not supported.

found solution


feat(nndb)!: added support for null-safety

Status
READY
Breaking Changes
YES
Description

Bumbed version of Dart, and migrated code to support null-safety.
Type of Change

[x] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[x] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🗑 Chore
[ ] 🧪 Testing

Published v0.0.1-nullsafety.0 🎉

Published v0.0.1-nullsafety.0 🎉


Deprecated fields

This is to enhance the MgmtNet and the NodeDefinition structs to also accept the depricated underscore fields

this was mixed up stuff... closing it


⚠ Persistent BinderHub - Health has degraded performance

In 574ce85, Persistent BinderHub - Health (https://notebooks.gesis.org/services/binder/health) experienced degraded performance:

HTTP code: 200
Response time: 1567 ms

Test comment

Resolved: Persistent BinderHub - Health performance has improved in 3084fc2.


Nick: Watch sekenkoum.com

Nick requests the watch of the watch_keyword sekenkoum\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

sekenkoum\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Feat: Add machine for initializing Transaction

Test: Two passing tests and 1 failing test for initialize transaction machine

I had to find a new way to not allow my latest commit from mixing with the already created pull request to remain atomic


Kubevirt e2e use simple tcp server

- What this PR does and why is it needed

- Special notes for reviewers

- How to verify it

- Description for the changelog

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

201 of 426 (47.18%)  changed or added relevant lines in 27 files are covered.
57 unchanged lines in 13 files lost coverage.
Overall coverage increased (+0.4%) to 52.407%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

go-controller/pkg/ovn/base_network_controller.go
0
1
0.0%

go-controller/pkg/ovn/namespace.go
20
21
95.24%

go-controller/pkg/node/gateway_init.go
5
7
71.43%

go-controller/pkg/ovn/controller/services/node_tracker.go
0
2
0.0%

go-controller/pkg/ovn/gress_policy.go
9
11
81.82%

go-controller/pkg/ovn/zone_interconnect/zone_ic_handler.go
0
3
0.0%

go-controller/pkg/cni/cni.go
0
4
0.0%

go-controller/pkg/ovn/egressfirewall.go
11
16
68.75%

go-controller/pkg/util/node_annotations.go
0
6
0.0%

go-controller/pkg/ovn/controller/admin_network_policy/admin_network_policy.go
0
7
0.0%

Files with Coverage Reduction
New Missed Lines
%

go-controller/pkg/node/default_node_network_controller.go
1
18.78%

go-controller/pkg/ovn/base_secondary_layer2_network_controller.go
1
46.69%

go-controller/pkg/ovn/egressfirewall.go
1
82.43%

go-controller/pkg/ovn/gress_policy.go
1
88.09%

go-controller/pkg/util/kube.go
1
49.13%

go-controller/pkg/ovn/default_network_controller.go
2
56.71%

go-controller/pkg/ovn/hybrid.go
2
79.52%

go-controller/pkg/ovn/master.go
2
60.57%

go-controller/pkg/node/gateway_shared_intf.go
3
52.46%

go-controller/pkg/clustermanager/egressip_controller.go
8
60.33%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
26989

Relevant Lines:
51499

💛  - Coveralls


Update the detection method for docker overlay2 issue

This PR updates the detection method for docker overlay2 issue. We no longer count how many times the log message is seen.

/assign @Random-Liu
/cc @yguo0905
/cc @jiayingz

Remove the one from docker-monitor-counter.json?

Remove the one from docker-monitor-counter.json?

I cannot  remove it now. The presubmit and CI jobs are using k8s master head to test NPD e2e tests, which needs docker-monitor-counter.json. We can remove this config file after updating the k8s master.

/retest

/lgtm
/approve


Input lag bug

For some reason when I press an input related to the mod it is delayed a decent bit, I've also noticed that the adjustment keys are super buggy.  (Latest release.)

Please provide the resulting crash log from pressing F3 + C for 10 seconds while ingame.

crash-2021-05-22_18.50.47-client.txt

After a bit of experimenting, I've found that the delayed input updates when normal input is used, and also after a certain period of time.
The glitchy adjustment is still strange though.

Do you know how to build from source?

I have spotted some issues regarding keybinds glitching out. This should be fixed in the latest commit. If you know how to build from source you can try it out and see if it fixes your issue. Though i was not able to reproduce your "delayed input".
To build from source run the following commands in a terminal:
git clone https://github.com/Exopandora/ShoulderSurfing.git
cd ./ShoulderSurfing
gradlew build
The jar should be located in ./build/libs/

Fantastic, it works wonderfully!  The weird delay seems to only be an issue with this mod and my mouse.
Oh yeah, there's is a bit of a flickering sometimes, my framerate is rather high so it might be that.
Thank you for the quick response and prompt bugfix!

Regarding the flickering: Did you press the buttons in any special pattern?

I dont think that its a frame rate related issue though, as i get up to 600fps and did not encounter any flickering. It might be caused by different file observing behaviours of linux (your setup) and windows (my setup). When the system detects a file change it will automatically reload the config file. Maybe that sometimes resets the values to an earlier state, causing the flickering.

That's more likely the issue.  I didn't do anything particular, it just happens when I use the adjustment keys.  It's really noticeable when you hold it down, but it's nothing so bad that it breaks the function of the mod.

I made some more changes and i'm fairly confident it will be fully fixed this time. Again though you need to build from source in order to test it. If you still have the repo from the previous build you can simply run the following commands in ./ShoulderSurfing/ to get the updated version:
git pull
./gradlew build
The jar will again be located in ./build/libs/

A release version is now available on curseforge

Yep, works fine.  Flickering is better for the most part, but it still happens from time to time.  Everything else is fixed though!
Glad I could help out, this is a good mod that I very much enjoy.

What exact linux distro are you using? And what type of drive (hdd/ssd) are you using?

Ubuntu, and an NVMe SSD.


Support additional fields in ServiceRole.

Support notPaths, notMethods, ports, notPorts, hosts, notHosts in ServiceRole (+ unit tests and refactoring).
PR for #11516.
Thank you Yangmin for the help.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: pitlv2109
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: ayj
If they are not already assigned, you can assign the PR to them by writing /assign @ayj in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @pitlv2109. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: pitlv2109
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: geeknoid
If they are not already assigned, you can assign the PR to them by writing /assign @geeknoid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pitlv2109: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-integ-k8s-tests

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-integ-k8s-tests

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-integ-k8s-tests

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-integ-k8s-tests

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-unit-tests

prow/istio-integ-local-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-integ-local-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-unit-tests

prow/istio-integ-local-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-integ-local-tests

prow/istio-integ-k8s-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-unit-tests

prow/istio-integ-local-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-integ-local-tests

prow/istio-integ-k8s-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-integ-k8s-tests

prow/istio-presubmit.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
925915345454bf0fb4abe67de17301812875f461
link
/test istio-integ-k8s-tests

prow/istio-unit-tests.sh
1ffb05c54b4423a0b5ddbc81e7e7ee0db9c2d321
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

The PR currently includes all changes not related to RBAC, we need to do a rebase to split out the RBAC changes. You can also create a new PR if you find that is more easier to do.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
8e96fa894a5735adffa1d3e48e13ccfc730bbfb1
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
1ffb05c54b4423a0b5ddbc81e7e7ee0db9c2d321
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
1ffb05c54b4423a0b5ddbc81e7e7ee0db9c2d321
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@pitlv2109: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
1ffb05c54b4423a0b5ddbc81e7e7ee0db9c2d321
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
1ffb05c54b4423a0b5ddbc81e7e7ee0db9c2d321
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
1ffb05c54b4423a0b5ddbc81e7e7ee0db9c2d321
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/uncc costinm

/close
This PR is now here: https://github.com/istio/istio/pull/11712

@pitlv2109: Closed this PR.

In response to this:

/close
This PR is now here: https://github.com/istio/istio/pull/11712

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


basic-classifier-no-backtracking gives high weight to nonexistent room

-5585241004967735254 at 51200 47

This seems to be an issue with the model and not with the input.


Test Report

Today ,I have tested this fork.
I am using a 2160p 120fps H264 AVC 8Bit video to test.
My CPU is Intel Xeon E3 1230 V2, the GPU is Nvidia Geforce GTX 760 and the memory is 2 x 8GB DDR3 1333.
I only adjust the Windows SDK version to 10.0.10240.0 and use the merged FFmpeg dynamic linked library compiled by myself.(https://github.com/M2Team/FFmpegUniversal)
And I found something need to report.

An error occurs when you try to replay the video after it has finished playing.

2. Although we don't talk about reaching the target of 120fps, but the frame curve is not smooth enough.

3. GPU uses a lot of time on copying.

4. CPU usage.
I hope my report can help improve this fork.
Mouri

Hi Mouri, thank you for taking time to test the fork. I have just fixed the first issue (error on file restart).
About the performance: I think you can't avoid the GPU copy operations. When you do software decode, the result is stored in RAM and must be copied into GPU memory. At 4K 120hz that is a lot of copying., which can only be avoided when using hardware decoding. And if decoding does not happen at real-time, it is no surprise that FPS is unstable. Some scenes are more complex than others, so on some you reach higher fps than on others. Fps will only be stable if you reach desired playback speed.
To improve performance you can try to set Config.VideoOutputAllowIyuv = true (see new toggle switch in updated sample). This should avoid sws_scale calls, freeing up CPU resources. Iyuv has some disadvantages, that is why it is not enabled by default. But if you need every fps, this is the right setting for you.

When using Iyuv, I get only ~0.3% CPU usage in FFmpegInterop lib, the rest is external code...

Hi lukasf, I have tested it just now.
The replay bug has been solved. But the performance is not seemed better than before.

This is my settings.

This is the performance graph.
Mouri

Then there is not much we can do. At least, I don't have any idea right now. And as you can see, there is almost zero CPU time spent in our code anymore with iyuv. Maybe a newer ffmpeg version could improve performance, but that could require changes in the code. Plus, the last stable version 3.4 is pretty old, I'd rather wait for 3.5 before doing the work.

I'd say this is pretty impressive for software decoding...

https://github.com/FFmpeg/FFmpeg/tree/release/3.4
This is the source code of FFmpeg I used, updated 12 days ago.
I find there are some API changes in FFmpeg 3.5 when I read the master branch's FFmpeg doxygen.
Yes, I am agree with you in most cases.
I also wanted to know what it will happen if implement the Media Foundation interface directly.
If someone can answer to me, I will be glad because one of my friends suggests me to implement the Media Foundation interface. (He thinks it may solve my problem.)

@mcosmin222
I know that the video I'm testing can be played using PotPlayer smoothly.
But I hope the FFmpegInterop can reach this goal because I used many Media Player Applications used Windows Runtime and find its software decoding performance so bad.
Eat more CPU and GPU but not show result smoothly.
Recently, I tried to modify the FFmpegInterop/master branch, removed all hardware decoders only use the software decoder because hardware decoder is useless when playing a video which used the future mainstream resolution or bit rate.
If I do some works about the software decoders in the future, I want to contribute my work to this fork or the master.
Good luck to you and me. Thank you, @lukasf and @mcosmin222 .
Mouri

I hope that there is not much additional overhead by using MediaStreamSource. It should be possible to more or less directly map MediaStreamSample to IMFSample. But you will only know if you try it. It would sure be interesting to see how performance is with a direct MF implementation, compared to MediaStreamSource.
You would need to implement the IMFByteStreamHandler and register your class with the MediaExtensionManager. I can tell you that it is quite some work, the MF interfaces are a bit cumbersome. I have done it a few times for audio decoders (FLAC, OGG). If you try it, please let us know how it goes.

Thank you.
But I am curious how to make all file formats use my handler. (Because the FFmpeg can get supported formats, but the information is not complete.)

You can register the same MF class it for multiple file extensions and mime types. You'd have to find out which file types are supported by ffmpeg and how their extensions and mime types are. But for a performance test, it would be sufficient to just register it for video/mp4. The rest can be done later, if you see that it really has performance benefits. I am not 100% sure, but it might be that MF will try your ByteStreamHandler if it does not find another codec to handle it, even if you did not register that file type. But if you want to override existing codecs, you definitely must register for all those common file types.
There is one thing I just remembered: Some videos (especially HEVC but sometimes also H264) have special color range. Those are rendered incorrectly in most players such as MPC-HC (which uses ffmpeg/LAV). We pass the required info to MF to allow color range conversion and correct display of those files. I have noticed about 10% higher CPU usage with this conversion (which is surprisingly high). It might be that your file also has this color space. You can search for MF_MT_VIDEO_NOMINAL_RANGE and disable that line, and see if it improves performance.  It only has an effect if you use Iyuv, so enable the Iyuv switch and disable that code line, then check if performance is better. It probably won't be enough to get you to 120fps, still, it could be an improvement (if your file uses this color range).

Thank you very much. I will try it recently.
Today, I try to optimize your work at my repository.  I tried and have found some ideas, and I hope I can help.

About the NativeBufferFactory::CreateNativeBuffer method. The less your invoked, the better performance you get. Because even create an Windows Runtime object written by WRL, there is still a visible overhead.
There is also a visible overhead in the UncompressedVideoSampleProvider::SetSampleProperties method. Because the Windows Runtime implicitly converts C/C++ variable to a reference or a Windows Runtime object.  (I think you can do it better,)

Guid g_MFMTVideoChromaSiting(MF_MT_VIDEO_CHROMA_SITING);

IBox<uint32>^ g_MFVideoChromaSubsampling_MPEG2 = ref new Box<uint32>(MFVideoChromaSubsampling_MPEG2);
IBox<uint32>^ g_MFVideoChromaSubsampling_MPEG1 = ref new Box<uint32>(MFVideoChromaSubsampling_MPEG1);
IBox<uint32>^ g_MFVideoChromaSubsampling_DV_PAL = ref new Box<uint32>(MFVideoChromaSubsampling_DV_PAL);
IBox<uint32>^ g_MFVideoChromaSubsampling_Cosited = ref new Box<uint32>(MFVideoChromaSubsampling_Cosited);

IBox<int>^ g_TrueValue = ref new Box<int>(TRUE);
IBox<int>^ g_FalseValue = ref new Box<int>(FALSE);

Guid g_MFSampleExtension_Interlaced(MFSampleExtension_Interlaced);
Guid g_MFSampleExtension_BottomFieldFirst(MFSampleExtension_BottomFieldFirst);
Guid g_MFSampleExtension_RepeatFirstField(MFSampleExtension_RepeatFirstField);

HRESULT UncompressedVideoSampleProvider::SetSampleProperties(MediaStreamSample^ sample)
{
	MediaStreamSamplePropertySet^ ExtendedProperties = sample->ExtendedProperties;

	if (m_interlaced_frame)
	{
		ExtendedProperties->Insert(g_MFSampleExtension_Interlaced, g_TrueValue);
		ExtendedProperties->Insert(g_MFSampleExtension_BottomFieldFirst, m_top_field_first ? g_FalseValue : g_TrueValue);
		ExtendedProperties->Insert(g_MFSampleExtension_RepeatFirstField, g_FalseValue);
	}
	else
	{
		ExtendedProperties->Insert(g_MFSampleExtension_Interlaced, g_FalseValue);
	}

	switch (m_chroma_location)
	{
	case AVCHROMA_LOC_LEFT:
		ExtendedProperties->Insert(g_MFMTVideoChromaSiting, g_MFVideoChromaSubsampling_MPEG2);
		break;
	case AVCHROMA_LOC_CENTER:
		ExtendedProperties->Insert(g_MFMTVideoChromaSiting, g_MFVideoChromaSubsampling_MPEG1);
		break;
	case AVCHROMA_LOC_TOPLEFT:
		if (m_interlaced_frame)
		{
			ExtendedProperties->Insert(g_MFMTVideoChromaSiting, g_MFVideoChromaSubsampling_DV_PAL);
		}
		else
		{
			ExtendedProperties->Insert(g_MFMTVideoChromaSiting, g_MFVideoChromaSubsampling_Cosited);
		}
		break;
	default:
		break;
	}

	return S_OK;
}

Mouri

I also changed some methods in MediaSampleProvider class because there is a visible overhead. (All the overheads I mentioned were watched in the Visual Studio Performance Profiler by myself.)
Here is the method definitions which I changed.
void QueuePacket(AVPacket& packet);
void PopPacket(AVPacket& packet);
HRESULT GetNextPacket(AVPacket& avPacket, LONGLONG & packetPts, LONGLONG & packetDuration);

Frankly, this all sounds like unneccessary micro-optimizations to me. In your last profiler result, 99.6% of CPU time was spent outside our library. Now you are really trying to optimize those 0.4%? Sorry, I think you are on the wrong track. You can spend weeks doing with micro-optimizations on those 0.4% and you will never get any noticeable improvement.
When you want to optimize performance, you need to find the real performance bottlenecks and tackle them. And the bottlenecks are obviously not in our library data flow. If other players using ffmpeg/LAV are really faster, then it could be the ffmpeg build, the way we configure/use ffmpeg, MediaFoundation, the UWP rendering part, or the MediaStreamSource overhead. But it is clear from your results, that it is not our internal data handling that is holding back performance. In older versions, we had a lot of unneccessary buffer copy and other slowdowns. It is very satisfying to see that this has all been solved now.
Still some info on the three points you mentioned:

This is true, but MediaStreamSource requires use of IBuffer, so we must instantiate a WinRT class. We could add a pool of IBuffers which can be re-used (or even a pool of MediaStreamSamples). But I guess it is quite a lot of work. So personally, I would not put work in that area right now, given the good performance numbers. If you want to do it and you can give proof that is has noticeable impact (profile before and after), I will happily merge it.

This makes the code a bit ugly. The Guid redefinitions are definitely unneccessary, you are just creating a second copy of them. Using fixed IBox<> values probably has a very small performance benefit. I will think about adding it.

As far as I know, reference and pointer is internally exactly the same to the compiler. It is only a syntactic difference and should even lead to the exact same binary code. A small google search confirmed my assumption so far. Please add more info if you are really sure that this helps.

Please don't get me wrong, I really appreciate that you want to help improving performance. It's just that I do not think that these are the areas which really need improvements right now.
If you want to work on the library, I can add you as a contributor of this fork. Then you can create your own branches, and create PRs if you think you have improvements. Just let me know.

Thank you, lukasf.
If the external code, I remember that the FFmpeg load accounted for only about 60%~70% of the CPU time.
If we still use the FFmpeg default compilation options, It seems we can't get a visible improve.
About the optimization I said before, it seems that the FPS value curve of my device is smoother than before, so this is why I recommended. (Maybe it's just a psychological consolation.)
Mouri

I want to improve this library.
But I don't have more ideas now because I basically said all things I thought.
I think I should read documents of FFmpeg carefully, may find a good idea.

I think part of the overhead comes from the winRT sandboxing model, particularly from IO through the runtime broker.
It is interesting to see what PotPlayer really is (maybe it is a packaged desktop app) and how it does this. Microsoft states in the docs that if you want as little overhead as possible, you need to go with MF directly. I would advise against it, unless you really need that 4k at 120fps

Almost forgot to say,  IYUV mode eat more memory, but performance is same as the software scalar.

@mcosmin222
PotPlayer is the only desktop app can play my test video smoothly on my PC.
MPC-HC's performance looks like your fork when I tested few years ago. (The test video is from 2013.)

I have been wondering why Potplayer can decode that video with a 70%~90% CPU usage.
So I desired to find the answer, and this is the reason why I wrote so many replies in this issue.

Anyway, thank you everyone.
I think this discussion should be ended now.
I will open a new issue in the future if I have new ideas.

Just for information: I have tested current PotPlayer, MPC-HC and FFmpegInterop with a few 4K files I had here. I got mixed results. For one file PotPlayer was noticeably faster than MPC-HC or FFmpegInterop. For other files, it was slower. It might depend on the pixel format used. The file where PotPlayer was faster had 10bit pixel format, the others had 8bit. Not sure why PotPlayer is faster, in settings 10bit was disabled so I guess it does downscale output to 8bit (but it's not possible to check the actual output format in the player). MPC-HC did output 10bit, FFmpegInterop downscaled to 8bit, but both were slower than PotPlayer. But then again, for two other files PotPlayer was slower, for one it was about same speed. They seem to do some things different, sometimes better, sometimes worse.
FFmpegInterop was usually on par or sometimes even faster than MPC-HC. FFmpegInterop seems to have higher CPU usage for a few seconds, then it goes to similar level as MPC-HC. It might be due to buffering and multi threaded decoding. I just did some experiments with adding direct 10bit output to avoid scaler, but I was not successful. Generally, I am satisfied with the performance I see. Having 10bit output would be nice, but it's not a top priority for me right now.
Feel free to open a new issue if you find ways to improve performance or find other things.

 Still adding this for those who want the extra quality (and have 10/12bit display).


Add Styling

In this Milestone I made the following additions ➕ ➕ 🟢
- Added Style for the whole Application.

Added CSS files - Book.css, Books.css, Navigation.css, BookForm.css and Categories.css. 📁
Imported CSS files to App Components. ⬇
Updated CSS files to complete Application Style.

Thanks for checking the project and the review too.


Plant release notes for next release moving troubleshooting to wiki

In scope for https://www.pivotaltracker.com/story/show/155400593
Apart from the content of this PR, please review also:

https://github.com/aeternity/epoch/wiki/Troubleshooting
How https://github.com/aeternity/epoch/wiki#usage links https://github.com/aeternity/epoch/wiki/Troubleshooting

The idea is being able to update release notes contextually to PR.


proposal: function and method arguments and results should be passed in registers

Performance would be generally somewhat improved if arguments-to and results-from function and method calls used registers instead of the stack.  Projected improvements based on a limited prototype are in the range of 5-10%.
The running CL for the prototype: https://go-review.googlesource.com/c/28832/
The prototype, because it is a prototype, uses a pragma that should be unnecessary in the final design, though helpful during development.
(This is a placeholder bug for a design doc.)

5% for amd64 doesn't feel like it justifies cost of breaking all the assembly written to date.

Additionally, passing argument in registers will make displaying function
arguments in traceback much harder, if not impossible. One reason I like Go
so much is how Go's traceback makes debugging so much easier by providing
enough contextual information about the frames.
Frankly, I'd have expected much larger performance improvement for
switching the calling convention to use registers. If the benefit is indeed
only 5-10%, IMHO it's not worth the trouble. We should pursue better
inlining and automatic vectorization instead.
The Intel CPUs have invested so much circuit in optimizing the stack
(operations), so I'm not too surprised though. The benefit should be larger
on RISC architectures.

I agree with Minux, optimising function calls for 10% speedup vs inlining and all the code generation benefits that unlocks doesn't feel like a good investment.

I'm still thinking about the proposal, but I will note that for asm functions with go prototypes and normal stack usage, we can do automated rewrites. Which is also a reminder that the scope for this proposal should include updating vet's asmdecl check.

I wouldn't worry too much about assembly functions,
as in the worse case, we can have an opt-in mechanisms
for assembly functions and then develop a mode in
cmd/asm to help people rewrite (or better yet, automate
the translation in cmd/asm so that people won't even
notice the difference. we can have a new textflag for new
style assembly functions).
However, I do care deeply about arguments in tracebacks.
I tried patch set 30 of CL 28832 on this example:
https://play.golang.org/p/UV-E4wyL2T
And the result is:
panic: 0
goroutine 1 [running]:
panic(0x456a80, 0xc42000e118)
$GOROOT/src/runtime/panic.go:531 +0x1cf
main.F(0x50)
/tmp/x.go:8 +0x6b
main.F(0x0)
/tmp/x.go:6 +0xa
main.F(0x0)
/tmp/x.go:6 +0xa
main.F(0xc420052000)
/tmp/x.go:6 +0xa
main.F(0x0)
/tmp/x.go:6 +0xa
main.F(0x7)
/tmp/x.go:6 +0xa
main.F(0x7f6c34290000)
/tmp/x.go:6 +0xa
main.F(0x60)
/tmp/x.go:6 +0xa
main.F(0xc420070058)
/tmp/x.go:6 +0xa
main.F(0x0)
/tmp/x.go:6 +0xa
main.F(0xc4200001a0)
/tmp/x.go:6 +0xa
main.main()
/tmp/x.go:13 +0x9
I'd like to hear if we have any plans for restoring the existing
behavior of showing the first few arguments for each frame.

@minux Yes, we will have all the information necessary to print args as we do now.  I suspect it just isn't implemented yet.

I'm curious to know how do you plan to implement it without
incurring the overhead of storing the initial value into memory?

The values enter in registers but they will be spilled to the arg slots if they are live at any call.  So all live arguments will still be correct.  Even modified live arguments should work (except for a few weird corner cases where multiple values derived from the same input are live).
Dead arguments are not correct even today.  What you see in tracebacks is stale.  They may get more wrong with this implementation, so it may be worth marking/not displaying dead args.
Things will get trickier if we allow callee-saved registers.  We're thinking about it, but it isn't in the design doc yet.

Just passing arguments in register while still reserving stack slots
for them looks like MS's ABI. Isn't that negating a lot of the benefit
of passing arguments in registers in the first place? Passing
argument in register is precisely about not generating memory
traffic for register arguments.
Anyway, I'd like to read the full design docs of this and see the
discussion of various design decisions and trade offs made.

When we pass an argument in a register, the corresponding stack slot is reserved but not initialized, so there's no memory traffic for it.  Only if the callee needs to spill it will that slot actually get initialized.

I'm not sure how "full" the design docs are.
I've got a CL up, and here's a more readable version of it that I'll try to keep up to date: https://gist.github.com/dr2chase/5a1107998024c76de22e122ed836562d
(Repeating some of that gist) I reviewed a bunch of ABIs, and the combination of

need someplace to spill if the stack must grow, and we have no frame yet
callee-saves makes tracing stacks for GC a good deal more painful; the impact on assembly language is also a good deal larger.
there's no memory traffic unless there's a spill, and then we must spill somewhere
most of the existing ABIs (all but Arm64) are unfriendly to multiple return values in registers
assurances from debugger/dump-inspection people (Delve and Backtrace.io) that they use DWARF, if we take the time to get it right

caused me to to decide to try something other than standard ABI(s).  The new goal was to minimize change/cost while still getting plausible benefits, so I decided to keep as much of the existing design as possible.
The main loss in reserving stack space is increased stack size; we only spill if we must, but if we must, we spill to the "old" location.
As far as backtraces go, we need to improve our DWARF output for debuggers, and I think we will, and then use that to produce readable backtraces.  That should get this as right as it can be (up to variables not being live) and would be more generally accessible than binary data.
So actually turning this on may be gated by DWARF improvements.

Because we still spill arguments to stack when it's live across calls, this
means using registered parameters won't help performance for most non-leaf
functions (and any functions with moderate size.)
Therefore, it seems registered parameters will mostly help small leaf
functions. And if that's true, I imagine inlining those functions will
actually provide even more speedup because then the compiler can optimize
across function call boundaries.
I'm not sure improving DWARF would help traceback. True, DWRAF provides an
elaborated way to specify location of arguments (and variables), but for
the case of traceback, we can only use SP relative addressing for the
arguments: even though DWARF allows us to describe that at certain point in
the program, certain argument is in a certain register, at the time of
traceback, such information is useless because registers are almost
guaranteed to be clobbered.
What I'd like to see is concrete result that changing ABI is worthy the
effort as compared to, say, better inlining support. Specifically, I like
to see evidence that registering arguments help large functions that cannot
be inlined.
If we have to change the ABI, I think allowing some callee-saved registers
will actually provide more benefit. And we need to cache g in a register on
amd64 (like the RISC architectures). I remember Russ proposed that we
maintain some essentially callee-saved register to save g. We should
investigate general callee-saved ABIs. Prior experience shows that
introducing callee-saved registers in a register-rich architecture helps
performance considerably, and it's not restricted to small functions.
Additionally, if we can align our callee-saved registers with the platform
ABI, cgo callbacks will be faster because we always save all platform
callee-saved registers at entry.

if we can align our callee-saved registers with the platform ABI, cgo callbacks will be faster

Not to mention more reliable.  A lot of the signal trampolines in the runtime could be a whole lot simpler if they didn't have to deal with the ABI mismatch between signal handlers (which must use the platform ABI) and the runtime's Go functions.  (I'm still not entirely convinced that there aren't more calling-convention bugs lurking.)

One problem of callee-saved registers is that we must figure out how to
make GC cope with it. Otherwise if we can only save non-pointer in
callee-saved registers across function calls, the benefit will be much
smaller.
This is complicated by the problem that when the callee decided to spill a
callee-saved register, how do they know if it should be stored in pointer
slots or not?
Pointer tagging is out of the question as we don't box everything. My
initial idea is having the caller reserves some stack space above the
outgoing arguments area for all callee-saved registers and have
(conditional) stack map for those. The callee will always save to
designated slot for the register. To avoid clearing the save area, we need
more pcdata to tell GC which of the callee-saved registers have been
spilled to their respective slots. This solution does negate some of the
benefit of having callee-saved registers, but coupled with better inlining,
I think it could still outperform register arguments alone.
Of course, one simple solution is to allow only pointers in callee-saved
registers.

At each call, have the PCDATA for that call record, for each callee-saved register, whether it holds 1) a pointer; 2) a non-pointer; 3) whatever it held on function entry.  Also at each call, record the list of callee-saved registers saved from the caller, and where they are.
Now, on stack unwind, you can see the list of callee-saved registers.  Then you look to the caller to see whether the value is a pointer or not.  Of course you may have to look at the caller's caller, etc., but since the number of callee-saved registers is small and fixed it's easy to keep track of what you need to do as you walk up the stack.

5% for amd64 doesn't feel like it justifies the cost of breaking all the assembly written to date.

Much like SSA, I expect this change will have a much more meaningful effect on minority architectures.
Those fancy Intel chips can resolve push/pop pairs halfway through the instruction pipeline (without even emitting a uop!), while a load from the stack on a Cortex A57 has a 4-cycle result latency..

On Wed, Jan 11, 2017 at 9:39 PM, Philip Hofer<EMAIL_ADDRESS>wrote:

5% for amd64 doesn't feel like it justifies the cost of breaking all the
assembly written to date.
Much like SSA, I expect this change will have a much more meaningful
effect on minority architectures.
Those fancy Intel chips can resolve push/pop pairs halfway through the
instruction pipeline (without even emitting a uop!), while a load from the
stack on a Cortex A57 has a 4-cycle result latency.
http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Cortex_A57_Software_Optimization_Guide_external.pdf
.

Note: the gc compiler don't ever generate push or pop instructions. It
always uses
mov to access the stack.
Result latency doesn't matter much on a out-of-order core because L1D on
Intel
chips has similar latency.
I suspect the benefit could be higher, but for non-leaf functions we still
need to
spill the argument registers to stack, which negates most of the benefit
for large
and non-leaf functions.

Result latency doesn't matter much on a out-of-order core because L1D on
Intel
chips has similar latency.

It especially doesn't matter if you have shadow registers, but AFAIK none of the arm/arm64 designs have 'em.

I suspect the benefit could be higher, but for non-leaf functions we still
need to
spill the argument registers to stack, which negates most of the benefit
for large
and non-leaf functions.

You can't just move them to callee-saves if they're still live? Is that for GC, or a requirement for traceback?

On Wed, Jan 11, 2017 at 11:59 PM, Philip Hofer<EMAIL_ADDRESS>wrote:

I suspect the benefit could be higher, but for non-leaf functions we still
need to
spill the argument registers to stack, which negates most of the benefit
for large
and non-leaf functions.
You can't just move them to callee-saves if they're still live? Is that
for GC, or a requirement for traceback?

In the current ABI, every register is caller-save. My counterproposal is
introducing callee-save
registers but keep arguments passing on stack to preserve the current
traceback behavior.

In the current ABI, every register is caller-save. My counterproposal is
introducing callee-save
registers but keep arguments passing on stack to preserve the current
traceback behavior.

Ah. I guess I assumed, incorrectly, that this proposal included making some registers callee-save.

Therefore, it seems registered parameters will mostly help small leaf
functions. And if that's true, I imagine inlining those functions will
actually provide even more speedup because then the compiler can optimize
across function call boundaries.

Inlining doesn't help on function pointer calls.

@dr2chase

there's no memory traffic unless there's a spill, and then we must spill somewhere

That's not strictly true.  Empty stack slots still decrease cache locality by forcing the actual in-use stack across a larger number of cache lines.  There's no extra memory traffic between the CPU and cache unless there's a spill, but reserving stack slots may well increase memory traffic on the bus.

most of the existing ABIs (all but Arm64) are unfriendly to multiple return values in registers

Not true.  It would be trivial to extend the SysV AMD64 ABI for multiple return values, for example: it already has two return registers (rax and rdx), and it's easy enough to define extensions of the ABI that pass additional Go return values in other caller-save registers.
For example, we could do something like:
rax: varargs count; 1st return
rbx: callee-saved
rcx: 4th argument; 3rd return
rdx: 3rd argument; 2nd return
rsp: stack pointer
rbp: frame pointer
rsi: 2nd argument
rdi: 1st argument
r8: 5th argument; 5th return
r9: 6th argument; 6th return
r11: temporary register
r12-r14: callee-saved
r15: GOT base pointer

Am I understanding correctly that this proposal is calling for structs to always be unpacked into registers before a call? Has any thought been given to passing structs as read-only references? I think this is how most of the ELF ABIs handle structs, particularly larger ones.
This way the callee gets to decide whether it needs to create a local copy of anything, avoiding copying in many cases. It is also presumably fairly common for only a subset of struct fields to be accessed so unpacking the all or part of the struct may be unecessarily expensive (particularly if the struct has boolean fields). Obviously the reference would have to be to something on the stack (copied there, if necessary, by the caller) or to read-only memory.
For Go in particular it seems like this would be a nice fit because the only way to get const-like behaviour is to pass structs by value and internally passing them by reference instead would potentially make that idiom a lot cheaper.
Arrays and maybe slices could also be handled the same way.

On Jan 12, 2017 11:59 AM, "cherrymui"<EMAIL_ADDRESS>wrote:
Therefore, it seems registered parameters will mostly help small leaf
functions. And if that's true, I imagine inlining those functions will
actually provide even more speedup because then the compiler can optimize
across function call boundaries.
Inlining doesn't help on function pointer calls.
Yes, but I doubt passing arguments help much either (at least in the
current proposal where it might still spill to memory). A better fix could
be de-virtualization or speculation. On the other hanf, having callee-saved
saved registers could help more (we can have more callee-saved registers
that most function have arguments.)

It might be interesting to gather empirical data about the distribution of pointer vs non-pointer arguments. Or perhaps more precisely, GC-relevant vs non-GC-relevant arguments, since there are some pointers that GC doesn't care about.
Ian's suggestion for tracking pointer vs non-pointer vs inbound in PCDATA is nice and clean and flexible, but even simpler and cheaper would be to simply partition into fixed sets of GC-able registers and non-GC-able registers.

I don't know if I made this point adequately well, but the estimated performance gains are based on benchmarks where X% of calls are hand-enabled (tied to function) to pass arguments in registers, spills and all.  Rick helped me with VTune to figure out the smallest set of functions to opt-in to get a meaningful percentage of calls covered.  The 5 and 10% improvement estimates are a conservative scale-up of the observed improvements to 100% coverage of calls.
I'm in the process of attempting to get both -gcflags=-l=4 and the current experiment working simultaneously so I can estimate their combined benefit in the same way; there are bugs in the interaction.
The current version of the proposal does call for structs to be unpacked field-by-field.  This is influenced by a desire to get strings, slices, and interfaces to land entirely within registers, combined with lack of support for selecting a "field" from a register in the current backend and some worries about how this might work on amd64p32.  I don't think that "field selection" from a register is something that we've implemented (yet).

One thing we could definitely do without impairing traceback is returning
values in registers. This also doesn't have GC complications as there is no
GC safe point immediately following a return instruction.
We do have to make sure to support at least three/four pointer return
values for the common case of return &T (or an interface) and an error.

I don't think there is any GC complications for register arguments as the
callee know which is pointer. Ian's suggested approach is for general
callee-saved registers. Partition the callee-save registers into pointer
and non-pointer classes is also a viable solution, but we need to gather
the types of hot values that live across a function call, not types of
function input arguments.

we need to gather the types of hot values that live across a function call, not types of function input arguments.

Right, thanks.

@josharian

even simpler and cheaper would be to simply partition into fixed sets of GC-able registers and non-GC-able registers.

That isn't obvious to me.  Partitioning registers adds complexity to the register allocator, while PCDATA tracking adds complexity to the GC.  Either approach seems straightforward, but both require additional code.

@mundaym

It is also presumably fairly common for only a subset of struct fields to be accessed so unpacking all or part of the struct may be unecessarily expensive (particularly if the struct has boolean fields).

That assumes that you've constructed the struct somewhere in memory in the first place.  For large and/or long-live structs, that's a valid assumption — but for many small structs (given reasonably effective inlining) it is not.

On Jan 13, 2017 3:01 PM, "Josh Bleecher Snyder"<EMAIL_ADDRESS>wrote:
we need to gather the types of hot values that live across a function call,
not types of function input arguments.
Right
Related idea: sometimes the compiler can choose whether to use an integer
or a pointer, esp. for loop induction variables, we might also need to take
this into account.
Additionally, if the compiler is sure that the GC can find another pointer
to a slice (e.g. the slice itself is on caller's stack), then the induction
pointer to its backing array (when saved in callee-saved registers) doesn't
have to be labeled as pointer.

the induction pointer to its backing array (when saved in callee-saved registers) doesn't have to be labeled as pointer.

I think it would, since stack copying still needs to update it.

Returning values in registers is slightly more difficult when runtime shims are needed (e.g., reflectcall).  On the call side, there's already a "natural" potential second entrypoint for all functions that do a stack size check (provided that the original spill-site memory layout is retained) that makes it relatively easy to implement go statements.
Notes:

"if the compiler is sure that the GC can find another pointer to a slice" -- and, also, if the slice's backing store is known not to reside on a stack, since stacks can relocate and grow.
The pointer-to-(large-)struct trick also requires that the struct not be mutable from multiple threads, else we can end up with nasty concurrency issues.

Do we need separate proposals for all the other great ideas put forth as alternatives?

Do we need separate proposals for all the other great ideas put forth as alternatives?

I say we continue the conversation here, and once it settles, someone--you, I imagine :)--digests all the input and comes back with an updated proposal taking it all into account.

Josh, it took a fair amount of work to obtain the credible-to-me 5 and 10% estimates for the existing proposal, and I took the time to do it because the potential impacts of doing this work are relatively large.  Anything that doesn't have a large cost-to-other-people I think just gets put on a list of stuff to do, and we prioritize by roughly-expected bang-for-buck.
My view on this is that a 5-10% gain in general (geomean) compiled code performance is worthwhile (back in the price-workstations-by-benchmark-performance days, we'd have been giddily happy for such a gain, and that was back when pipelines were short and simple) and the possibility that this might have an annoying impact makes me want to do it sooner, rather than later, since the more users we have writing "interesting" code, the more concrete is poured around our feet.  Low-impact changes we can do later.
I've been trying to get a feel for what the "actual problems" are with this proposal, where "hey, what about this other optimization" is not an actual problem.  The ones that I see are:

hex numbers in stack traces will get less informative, possibly even misleading;
stacks will not shrink as much as we might wish because we're allocating spill slots that might not be necessary (that we hope are usually not necessary);
treatment of structs containing lots of small fields might get worse, not better
there's an opportunity here to change how (large) structs are passed that might be very beneficial, and that's not been thought about in any great detail.
the assembly language migration story is not entirely satisfying; outbound calls through pointers remain a problem.

I would add one more specific concern to your list:

if the new convention isn't a superset of the platform ABI, we may lose the opportunity to simplify assembly functions.  (The migration is likely a lot of work, and I doubt we'll do it twice.)

I'm not saying that you necessarily need to fundamentally alter the proposal. It might well be that the outcome is that the proposal gets an "Alternatives" section added explaining why the various alternative suggestions here were declined in favor of the current one.

I've been trying to get a feel for what the "actual problems" are with this proposal, where "hey, what about this other optimization" is not an actual problem.

It depends to what extent the "what about this other optimization" suggestions interact with the original proposal. At least at first blush, that interaction is non-trivial. And having potentially better optimizations available is an "actual problem", since the work required to implement, debug, and migrate to a new ABI will dwarf the (not inconsiderable) work it took to validate your 5-10% number. That says to me it is worth at least taking some of the alternatives seriously, and updating the proposal in light of them.

@bcmills

if the new convention isn't a superset of the platform ABI, we may lose the opportunity to simplify assembly functions. (The migration is likely a lot of work, and I doubt we'll do it twice.)

As an aside: the s390x ELF ABI requires a 160-byte (!) caller-allocated save area, so I would rather not adhere strictly to it in Go due to the effect it would have on goroutine stack growth. Probably fine to keep the register allocations the same though.

For callee-saved pointer registers, we need callers to do some work.  In particular, they need to make sure to zero any such registers which are dead before a call.  Otherwise, callees will dutifully pass the values in those registers on to the GC which will then preserve the (maybe otherwise collectible) objects they point to.  That's another cost that must be borne by callee-saved pointer registers.
Maybe that cost goes away with Ian's strategy, I'm not sure.

I agree that it's important to get this right so we only have to do it once.  I don't want to force people to rewrite their assembly twice.  To the extent that we can do meaningful experiments to gauge the effect of ABI-modifying optimizations (e.g. callee-saved registers), we should do that.

On Fri, Jan 13, 2017 at 10:48 PM, Keith Randall<EMAIL_ADDRESS>wrote:

For callee-saved pointer registers, we need callers to do some work. In
particular, they need to make sure to zero any such registers which are
dead before a call. Otherwise, callees will dutifully pass the values in
those registers on to the GC which will then preserve the (maybe otherwise
collectible) objects they point to. That's another cost that must be borne
by callee-saved pointer registers.
Maybe that cost goes away with Ian's strategy, I'm not sure.

If the caller function know the pointer is dead at the call site, it just
need to record that it contains non-pointer at that time.
Basically, we need a separate PCDATA for each call instruction in the
program that records whether each callee-saved register is used (owned) by
the enclosing function and if used, whether it contains pointer. For each
function using callee-saved register, there will be FUNCDATA telling GC
where does it save the incoming callee-saved register.
When unwinding the stack, the traceback routine maintain a vector of
callee-saved registers contents and reload their contents as necessary as
it walks down the stack, and passing newly loaded pointer content to GC as
root.

The goal of the proposal is that assembly-language meddling be minimized
(readable proposal here https://gist.github.com/dr2chase/5a1107998024c76de22e122ed836562d ).
Assumption is that there will be a new tag set for adapted assembly language, the assembler will look for that tag, and if it does not see it then the code will automatically be made to work, provided that it lacks indirect CALLs (and perhaps it will error out for unadapted code containing indirect CALL).
For call-free assembly language, the prologue/epilogue is modified to call adapters emitted by the compiler to handle loads/stores from the argument/result portion of the stack.
For calls to named functions, one option is to surround the call with calls to compiler-generated wrappers.  Doing this for every function leads to a lot of mostly-unused extra boilerplate to be discarded by the linker; a slightly more complex example requires some back-and-forth with the compiler to ensure that these function-type-specific adapters are generated on demand.
And indirect calls in Go asm are rare, certainly within Google where we can (and did) search for them.
So I'd like to suggest that separating these args-in-registers and callee-saves is not necessarily "doing it twice", though it would be for last-cycle performance-critical assembly.
Adding callee-saves seems to guarantee a lot more meddling in existing assembly language, unless we come up with some clever plan.  If the clever plan were a tool, the tool must notice writes to callee-save registers, modify stack frame to add spill area for those registers, include FUNCDATA to indicate presence of spill.  How does the tool know where in the assembly frame to put the spills?
Alternately, we could choose the callee-saves registers in this turn of the crank and strongly suggest that anyone rewriting their assembly language now avoid using them, or preemptively add the save/restore code, so that when we do start using them for callee-saves in the future there is not a roadblocking need to repair code in a hurry.
Another earlier proposal by Keith appears to get some of the benefits of callee-save for direct calls with less impact on assembly language; there, the compiler records which registers a function does not clobber, records this in the export data, and for each direct call instruction its unclobbered registers can be left unspilled.  There's still interaction with GC, and this does nothing for indirect calls, but it has reduced source code impact.
For helping Cgo, might we be better off conforming exactly to the platform ABI for such calls, and only for those calls?

I wouldn't worry too much about assembly function migration.
In the worst case, we can implement automatic wrapper for them. (New style
assembly sets a new textflag, and linker will generate wrapper for old
style assembly function on the fly as needed. This scheme will work for
both register arguments and callee-saved registers.)
My main concern for the current proposal is still argument display in
tracebacks.
My position is that if we just use registers for return value and implement
callee-saved registers, we can get the benefit of register arguments
(potentially much more, because callee-saved registers doesn't necessarily
need spilling cross function calls), but will not lose the benefit of
argument shown in tracebacks. Even if it's not a performance win, I still
prefer the debugability provided by argument values in tracebacks (e.g.
invalid arguments like nil stand out, and it's usually the first few
arguments that will be passed in registers).

My main concern for the current proposal is still argument display in tracebacks.

@minux, can you expand on the problem you see with tracebacks? As I understand it, in the current proposal, any "controlled" traceback (where the goroutine is at a safe-point) works exactly like it does right now because the arguments will all be spilled to exactly the same places on the stack (unless, perhaps, their last use is before the first call?). Tracebacks caused by signals (nil pointer, divide-by-zero) may be a problem for printing the arguments of the inner-most frame, though all of the other frames will be fine. Is there some other problem you see?
Maybe we need a more elaborate way to communicate argument locations to the runtime for traceback. I think this is going to be more of a problem for inlining than for register arguments; we currently have no hope of recovering arguments for inlined frames (so, at least for now, we're not; perfect is the enemy of the good and all that.)

On Tue, Jan 17, 2017 at 11:26 AM, Austin Clements<EMAIL_ADDRESS>wrote:

My main concern for the current proposal is still argument display in
tracebacks.
@minux https://github.com/minux, can you expand on the problem you see
with tracebacks? As I understand it, in the current proposal, any
"controlled" traceback (where the goroutine is at a safe-point) works
exactly like it does right now because the arguments will all be spilled to
exactly the same places on the stack (unless, perhaps, their last use is
before the first call?). Tracebacks caused by signals (nil pointer,
divide-by-zero) may be a problem for printing the arguments of the
inner-most frame, though all of the other frames will be fine. Is there
some other problem you see?

This unnecessary spilling will have performance costs (and not to mention
the stack usage is the same as before). That's why I proposed callee-saved
register and only using registers for return values instead. They have
nicer interactions with traceback without losing performance.

This unnecessary spilling will have performance costs (and not to mention the stack usage is the same as before).

Backing up a step, are you concerned about tracebacks with the current proposal (which does potentially unnecessary spilling, partly to support finding arguments for things like tracebacks), or with an alternate proposal that does less spilling?

Adding callee-saves seems to guarantee a lot more meddling in existing assembly language, unless we come up with some clever plan. If the clever plan were a tool, the tool must notice writes to callee-save registers, modify stack frame to add spill area for those registers, include FUNCDATA to indicate presence of spill. How does the tool know where in the assembly frame to put the spills?

We don't need to notice writes to callee-save registers: we can spill them unconditionally.  (If that's a performance problem, the trivial workaround is to rewrite the assembly function to use the new calling convention.)
We would need to do the spills at the point where we change calling conventions: when we go to call across conventions, we spill all the callee-saved registers, then copy all of the arguments to the stack, call the assembly function, copy return-parameters to registers, restore callee-saved registers, and we're done.
IIUC, that would only require FUNCDATA to indicate the spills at the call sites, for which we would generate the same FUNCDATA as for any other spill (indicating that those registers do not contain GC roots after the spill occurs).

Alternately, we could choose the callee-saves registers in this turn of the crank and strongly suggest that anyone rewriting their assembly language now avoid using them, or preemptively add the save/restore code, so that when we do start using them for callee-saves in the future there is not a roadblocking need to repair code in a hurry.

I like that idea.  We're already trending a bit in that direction (see https://go-review.googlesource.com/#/c/35068/).

For helping Cgo, might we be better off conforming exactly to the platform ABI for such calls, and only for those calls?

Which calls do you mean, exactly?  There are (unfortunately) several routes by which Go functions can end up being called from C ABI functions: at least explicit cgo calls and signal handlers.  At any rate, I do think that (at least for platform ABIs that don't have crazy stack bloat) we should try to conform to the platform ABI.

On Tue, Jan 17, 2017 at 3:39 PM, Austin Clements<EMAIL_ADDRESS>wrote:

This unnecessary spilling will have performance costs (and not to mention
the stack usage is the same as before).
Backing up a step, are you concerned about tracebacks with the current
proposal (which does potentially unnecessary spilling, partly to support
finding arguments for things like tracebacks), or with an alternate
proposal that does less spilling?

To clarify my thinking:
Introducing register arguments in the most optimum case will make traceback
arguments incorrect,
but spilling the register arguments on function calls to solve the
traceback problem will reduce performance benefit of passing arguments in
registers.
Therefore, I suggest that we still pass arguments in stack (so that
traceback behavior is unchanged) and use call-saved registers and register
return values to get the speed up from changing ABI.

Thinking about tracebacks.  At any given point in the program, we know whether an argument is still valid or not; we shouldn't show invalid tracebacks, but we may have some arguments unavailable.
The most interesting arguments are in two directions: in one, the direction of "things that could have been GC'd but weren't yet", which favors passing in registers without spilling to preserve tracebacks.  In the other, the direction of "values which were passed further down the stack".  The latter could be traced by FUNCDATA annotations using a more general version of @ianlancetaylor's suggested approach: if we know where the contents of the registers came from, we can reconstruct the arguments that are still around because they were passed as arguments (or stored as locals) further down the call chain.
Does DWARF already support that kind of value-propagation annotation?

Can you clarify "things that could have been GC'd but weren't yet" ?
Do you mean pointers that are functionally dead, and if a GC has occurred might now point to reused memory, or something else?  That's a bad idea unless there is some indication (to debugger and/or traceback) that the pointer is stale and that examining its referent might lead to confusion.

Do you mean pointers that are functionally dead, and if a GC has occurred might now point to reused memory, or something else?

I mean pointers for which:

the last remaining reference is a function argument and
the execution of the corresponding function call has passed the last use of that argument.

If function arguments are passed on the stack (or spilled for the lifetime of the function), then collecting those pointers requires extra work.  On the other hand, if function arguments are passed in registers (and only spilled if they have live references after the spill), then collecting those pointers is automatic — and preserving them for display in tracebacks requires extra work.

Additionally, I'd want the arguments show in the traceback to be the
initial arguments to the function. Not updated values.
Because otherwise the meaning of the argument also depends on the exact
line number, which significantly reduces the value of showing them.
The main value of having the arguments is spotting plain wrong arguments
like 0xdeaddeaddead or nil without looking at the source code. If we only
flush the arguments to stack at call sites, then there is no guarantee the
arguments shown is the initial one and it could be changed. Having context
dependent argument not only is confusing, it also makes the value less
useful (considering when you don't actually know the version of Go used to
build the program, context dependent argument values are useless because
you can't refer to code; And this happens a lot when you're triaging new
issues).

We're already in a state where what is displayed in the traceback is context-dependent.
func f(x *int) {
   x = new(int)
   g()
   .. use x ..
}

If there is a panic during the call to g, the value displayed in the stack traceback for x's argument slot will show the newly allocated pointer, not the original value of x at the start of the function.
The reason this happens is that the newly allocated pointer is spilled to the argument slot for x.  We used to do this to ensure that the old value x pointed to could be garbage collected during the call to g.  Now that (as of go 1.8) we use our precise liveness information for arguments (and have runtime.KeepAlive available to override if necessary), we could revisit this decision.  It would cost some stack space, but probably not much.
In any case, all that is pretty tangential to the proposal at hand.  Not making tracebacks worse is certainly worth discussing here.  Making them better should be a separate proposal.

On Thu, Jan 19, 2017 at 5:56 PM, Keith Randall<EMAIL_ADDRESS>wrote:

In any case, all that is pretty tangential to the proposal at hand. Not
making tracebacks worse is certainly worth discussing here. Making them
better should be a separate proposal.

But my point is that making traceback better might conflict with the
current proposal, which means we can't really discuss these two problems in
isolation.

That the proposals conflict is ok.  And we can discuss those conflicts here.  But without a concrete proposal about how you'd like to make tracebacks better it's hard to see exactly what those conflicts would be.

I'm interested in knowing how this would interact with non-optimized compilation. Right now non-optimized compilations registerize sparingly, which is great for debuggers since the go compiler isn't good at saving informations about registerization. Putting a lot more things into registers without getting better at writing the appropriate debug symbols would be bad.

I'm interested in knowing how this would interact with non-optimized compilation. Right now non-optimized compilations registerize sparingly, which is great for debuggers since the go compiler isn't good at saving informations about registerization.

Presumably part of the changes for the calling convention would be making the compiler emit more accurate DWARF info for register parameters.

On hold until @dr2chase is ready to proceed.

Current plan is 1.10, time got reallocated to loop unrolling and whatever help was required to get the Dwarf support better in general, so that we have the ability to say what we're doing with parameters.
There's been a lot of churn in the compiler in the last couple of months -- new source position, pushing the AST through, making things more ready for running in parallel, and moving liveness into the SSA form -- so I am okay with waiting a release.

Would the proposed calling convention omit frame pointers for leaf functions? I can see this varying based on whether the function stores the state of a callee-save register, takes the address of a local variable, etc. In that case, is there still a feasible way to obtain the call stack?

I can be wrong, but register-based calling convention (CC) could give more performance boost if additional SSA rules are added that actually take advantage of it.
Currently, many small and average functions have many mem->reg->mem patterns that make certain optimizations on amd64 not worth it.
It is not fair to say that 10% is not significant.
It's 10% with optimizer that works on GOARCH=amd64 like it is GOARCH=386.
Lately, I was comparing 6g with gccgo and even without machine-dependent optimizations, aggressive inlining and constant folding there was about 25-40% performance difference in some allocation-free benchmarks. I do believe that this difference includes CC impact (because most other parts of output machine code look nearly the same).

As noted above, the performance benefit for a register-based calling convention is higher on RISC architectures like ppc64le & ppc64. If the above CL is not stale I would be willing to try it out. To me this is one of the biggest performance issues with golang on ppc64le.
Inlining does help but doesn't handle all cases if there are conditions that inhibit inlining.  Also functions written in asm can't be inlined AFAIK.

The CL is very stale. First we have to get to a good place with debugger information (seems very likely for 1.10), and then we have to redo some of the experiment (it will go more quickly this time) with a better set of benchmarks.  Some optimizations that looked promising on the go1 benchmarks turn out to look considerably less profitable when run against a wider set of benchmarks.

@dr2chase
Is the method used to determine the 5% performance improvement somewhere in the discussion? I'm curiously interested in what the function sample space actually look like.

I think the measurement we want is how much does this help once inlining is fully enabled by default.

The estimate was based on counting call frequencies, looking at performance improvement on a small number of popular calls "registerized", and extrapolating that to the full number of calls.
I agree that we want to try better inlining first (it's in the queue for 1.11, now that we have that debugging information fixed), since that will cut the call count (and thus reduce the benefit) of this somewhat more complicated optimization.   Mid-stack inlining, however, was one of the optimizations that looked a lot less good when applied to the larger suite of benchmarks.  One problem with the larger benchmark suite is selection effect -- anyone who wrote a benchmark for parts of their application cares enough about a performance to write that benchmark, and has probably used it already to hand-optimize around rough spots in the Go implementation, so we'll see reduced gains from optimizing those rough spots.

and has probably used it already to hand-optimize around rough spots in the Go implementation

From my point of view, the goal of having a better inliner is to have more readable/maintainable (less "hand-optimized"), not really faster code.

There is an updated proposal at #40724, which addresses many of the issues raised here.  Closing this proposal in favor of that one.


Refactor code

Need to refactor to eliminate all the duplicate code blocks for the different device types

Code done, needs testing.

In 0.5.0


[MERGE ON 17 JUNE] EID-1519: Prepare IDP recommendation AB test

Variant version will present recommended 2 top IDPs more prominently and not
recommended ones a bit less to test user interaction
if the recommendation model would be deployed.

I think this looks good now. I'm happy to approve if @jakubmiarka has no objections.

We've had three people work and look at this so this should be fine.
And yes - we'll merge everything in the right order on day one of the test.


Error: Installing<EMAIL_ADDRESS>failed.

error showing during Atom installation session, plz help! thx!

solved


The triple titles problem

Problem description
Titles are defined in three places:

/_data/toc.yaml
the title: directive in the pages' front matter YAML
The # title (<h1>) at the top of most documents

I can see there being a use case for having a different title in the left-navigation than the top of the page. There is less space to work with in the left-nav and there is more context around the title that may make longer titles unnecessary.
However, we should probably get rid of all the # titles at the top of each doc and modify _layouts/docs.html to render an <h1> of {{ page.title }}, making the title: directive in the front matter YAML the canonical page title.

Fixed in 558314bf56f4ee7e15a805e0b54cc9fee157c145


Font used to be bigger in the Pandas data grid output

Once #1482 is merged, the font size in the Pandas data grid at /examples/pydata.html is going to be smaller.
I'm creating this issue to track a potential follow-up task to #1482 to increase the font size in the data grid.

Before #1482 (font big)
After #1482 (font small)

Do we really need to change anything here ?
nbsphinx and myst-nb embark there own set of custom css and classes and I would argue that sizing the pandas dataframe is out of the scope of the theme.
We try to reduce as much as possible the impact we have on the third-party libs by only implementing the dark theme and overflow issues.
I'm personally +1 for closing without changing anything.

I thought I had commented here, but I guess I missed it.

Do we really need to change anything here ?

I do not think so; if anything, the current data grid font is more in line with the rest of the font size within the rest of the theme, so I would keep on closing this issue, too.


PaginatorPlugin data not sync with store

store.add(value)

{
"perPage": 10,
"lastPage": "10",
"currentPage": "3",
"data": [...]
}

here data not updated after add value into store
unable to get added value in paginationRef

@bh86 @amateurhuman @hoisel @dintel

The data property should come from your server. What do you mean?

@NetanelBasal
after performing
store.add(value)
value must be added in

{
"perPage": 10,
"lastPage": "10",
"currentPage": "3",
"data": [...]
}

otherwise there is no advantage of akita

i have two pages

1 . add computer
2 list computer  ( using pagination)
everytime i add a computer it does not reflect in list computer.... i have to refresh whole page for that

Can you share the code, please?

private getAll() {
    const sortByValue = this.paginatorRef.metadata.get('sortBy') || 'createdAt';
    this.sortByControl = new FormControl(sortByValue);
    const perPage$ = this.perPageControl.valueChanges.pipe(startWith(10));
    const sort$ = this.sortByControl.valueChanges.pipe(startWith(sortByValue));

    this.items$ = combineLatest(
      this.paginatorRef.pageChanges,
      combineLatest(sort$, perPage$).pipe(tap(_ => this.paginatorRef.clearCache()))
    ).pipe(
      switchMap(([page, [sortBy, perPage]]) => {
        const requestFn = () => this._service.getAll({ page, sortBy, perPage }, this.where);
        this.paginatorRef.metadata.set('sortBy', sortBy);
        this.paginatorRef.metadata.set('perPage', perPage);
        this.paginatorRef.metadata.set('currentPage', (page as any).offset || 0);
        return this.paginatorRef.getPage(requestFn).pipe(
          tap((page: any) => {
            this.paginatorRef.metadata.set('total', page.total);
          })
        );
      }),
      // untilDestroyed(this)
    );
  }

The plugin performs server-side pagination. When you add a new entity, Akita can't know the new pagination data. You need to re-fetch it to get the updated data from the server.

we are using akita for faster processing , i think, this plugin will slow down the process.

This is not related to Akita. This is how server-side pagination works. The client can't know the updated data such as lastPage, etc.

But what if I want to change a property from an item in the list where we don't need to invalidate the cache.
Not every change in the list will affect the paginated result from the server.

But what if I want to change a property from an item in the list where we don't need to invalidate the cache.
Not every change in the list will affect the paginated result from the server.

The plugin performs server-side pagination. When you add a new entity, Akita can't know the new pagination data. You need to re-fetch it to get the updated data from the server.

How do you do this?


A filter expression that results in a stack overflow

The filter expression
name pr and not (first eq "test") and not (another eq "test")
parses into an expression with an really deep (inifinite?) LogicalExpression.Left chain.
and results in a stack overflow if printed
=== RUN   Test_Parsing/crashes?
runtime: goroutine stack exceeds<PHONE_NUMBER>-byte limit
runtime: sp=0xc0201e0578 stack=[0xc0201e0000, 0xc0401e0000]
fatal error: stack overflow

This one give the same result name pr and not (first eq "test") and another ne "test"

Here's a test I added to filter_test.go
func ExampleParseFilter_crashing_filter() {
	fmt.Println(ParseFilter([]byte(`name pr and not (first eq "test") and not (another eq "test")`)))
	// Output:
	// name pr and not (first eq "test") and not (another eq "test") <nil>
}

Hey @DagW.
The PR above should fix this.

Tried it today and it works a charm,
Thanks!


Fixed #6

Hey @Soumya-Kushwaha , Please follow the guideline to create a .md file with description in it.
Thank you!


Direct connect

This change is

r? @canndrew
(maidsafe_highfive has picked a reviewer for you, use r? to override)


How to modify one of the global values

hello, I want to modify 8888 to 7777；
Now I can only query the full values( "8888^9999"), then modify to ("7777^9999") ,And then save?
Can this be the case?

Hi @verydemo, there are two way you can modify the global values.

In the place of your screen shot, click on "8888^9999" directly and change its value to "7777^9999". Then click Save.

In the provided IDE, modify the Python code to set the global value to "7777^9999" and run the Python file again.

So the answer to your question is yes. You have to take full value and modify it.
Please let me know if you have any other question.


Forward openmp variant to strumpack

Some downstream dependencies turn off openmp, so we should forward that variant to strumpack.

Codecov Report

Merging #1016 (017ccae) into develop (a2d7924) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #1016   +/-   ##
========================================
  Coverage    89.53%   89.53%
========================================
  Files          139      139
  Lines        10965    10965
========================================
  Hits          9817     9817
  Misses        1148     1148

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@white238 , can you take a look at this?


Can't open single file via Enter/o in Build 4143

a works, but o doesn't. Works fine in build 4126, but not in 4142 or 4143. Any idea on what's causing this?

Also did open an issue here so maybe that helps: https://github.com/sublimehq/sublime_text/issues/5682

https://github.com/aziz/BetterFindBuffer/pull/37 fixes this :)

This still isn't fixed. Is there a better fork that took the merge? If so, could that be on Package Control instead?


Package/Projects/Class drop down menus get "break at step point 1" menu item (3.0.82)

If you disable breakpoints, and in a browser you have a method selected, the Projects, Packages, Classes, Categories methods all get a "Break at step point 1" menu item.

Appears fixed in v3.0.82

I was incorrect; this does reproduce in 3.0.82; it's just completely unrelated to breakpoints and enablement thereof.
To reproduce: in PB, go to a method (find a long one).  Put focus on the method text, say, near the end.
Now, go to the menu bar drop-down menus.
The Project, Package, and Classes method all have the menu item "Break at step point 1".  This makes no sense.  The menu item "Break at step point 1" could, reasonably, be on the Methods menu.  But it's NOT there.

Shouldn't see any breakpoint messages except on the context menu as of 3.0.92pre-release2


Apply-PnPTenantTemplate : The 'value' argument cannot be null.

Category

[ ] Question
[X] Bug
[ ] Enhancement

Expected or Desired Behavior
Observed Behavior

Brand new O365 Tenant Developer
Have followed the instructions for provisioning the PnP Starter Kit:

prepared my tenant for Targeted Relase for Everyone and waited over 24 hours
added my user account to the role of global administrator and SPO admin (just to make sure)
provisioned a new Apps Catalog site collection to host the package
installed NodeJS LTM 10
installed the NPM packages (YO, GULP, and Yemon for SP)
added my user account to the Term Store admin group
added the custom property in the User Profile
all instructions from: https://github.com/SharePoint/sp-starter-kit

The error output is the following:

PS C:\dev\sp-starter-kit\provisioning> Apply-PnPTenantTemplate -Path starterkit.pnp -Parameters @{"PORTALURL"="/sites/demoportal"; "MARKETINGALIAS"="demomarketing"; "HRALIAS"="demohr" } -Verbose
Apply-PnPTenantTemplate : The 'value' argument cannot be null.
Parameter name: value
At line:1 char:1

Apply-PnPTenantTemplate -Path starterkit.pnp -Parameters @{"PORTALURL ...

  + CategoryInfo          : WriteError: (:) [Apply-PnPTenantTemplate], ArgumentNullException
  + FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.Provisioning.Tenant.ApplyTenantTemplate

Steps to Reproduce

Started PowerShell session ad Admin

Connected to my Tenant:  Connect-PnPOnline -url https://NAME.sharepoint.com/

Apply-PnPTenantTemplate -Path starterkit.pnp -Parameters @{"PORTALURL"="/sites/demoportal"; "MARKETINGALIAS"="demomarketing"; "HRALIAS"="demohr" } -Verbose

I checked my connection right after the error:

PS C:\dev\sp-starter-kit\provisioning> Get-PnPConnection

ConnectionType     : O365
InitializationType : Unknown
MinimalHealthScore : -1
RetryCount         : 10
RetryWait          : 1
PSCredential       : System.Management.Automation.PSCredential
TelemetryClient    : Microsoft.ApplicationInsights.TelemetryClient
Url                : https://NAME.sharepoint.com/
TenantAdminUrl     :
Context            : OfficeDevPnP.Core.PnPClientContext

Yes all good, I was connected and ran the Apply-PnPTenantTemplate from the provisioning folder.
All instructions to prepare the tenant were followed to the letter, I have since then removed everything and "cleared" my tenant from all previous steps, removed all packages, properties, delete site collections from the failed starter kit, removed groups, etc. and then tried again to re-run a second time observing one more time all the instructions - same result, nothing different.
Submission Guidelines
Is there a simple script to do all the provisioning instructions? Could this be provided in the future? The instructions given are simple to follow and doesn't require much from anybody but it seems impossible to reproduce consistent results for all - could we have an online tool like the: https://provisioning.sharepointpnp.com ?
Thanks for your contribution! Sharing is caring.

So it seems you can't skip the "optional step" for the Alpha Vantage Key :(
From the starter kit documentation my understanding about this step seemed like an optional step in case you wanted to use live stocks web part... but it seems I was incorrect, without the Alpha Vantage key you will fail... :_(


Question : Systrace Stages in record window

I don't understand what's the purpose of this option? Could you care to explain little bit?

On one of the previous projects, the developers actively monitored the app startup time. To do this, metrics are sent corresponding to different stages of the application start: for example, Application.onCreate(), Activity.onCreate, FirstFrameDrawn, FirstDataShown, and so on. Usually, the drawdown of a particular metric occurred due to the fact that some method was called earlier than it needed to be. For the convenience of tracking such cases, I introduced the concept of Stage: the time interval from one metric to another. It turns out that all calls during the start can be attributed to one or another Stage. Yamp has functionality that allows you to see an early method call that should have been called at a later stage. Such a tool helped to find regression during the new release. so Stages could be defined in 2 ways: choose a chain of methods that will correspond to the beginning of the Stage. This approach has a disadvantage: a method that is too fast may not get into trace due to sampling. The second way to determine the Stage is to label the Systrace labels with the desired prefix.


GPG error in Ubuntu install - public key is not available?

I'm attempting to sudo apt-get update in Ubuntu 17.04 after adding http://qgis.org/debian/ to my /etc/apt/sources.list,  and getting the following error message:
"W: GPG error: http://qgis.org/debian zesty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY CAEB3DC3BDF7FB45
E: The repository 'http://qgis.org/debian zesty InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details."
I'm checking out the manpage entry, but obviously I'd prefer to do a secure update ;) .
So can 'http://qgis.org/debian zesty InRelease' be signed/updated?
Thanks!

Add the following lines in terminal:
wget -O - http://qgis.org/downloads/qgis-2017.gpg.key | gpg --import
gpg --fingerprint CAEB3DC3BDF7FB45
Should output:
pub   2048R/BDF7FB45 2017-08-16 [expires: 2019-08-16]
Key fingerprint = 61E0 A086 749E 463E DE50  2255 CAEB 3DC3 BDF7 FB45
uid                  QGIS Archive Automatic Signing Key (2017<EMAIL_ADDRESS>sub   2048R/E959BBCF 2017-08-16 [expires: 2019-08-16]
then add:
sudo apt-get update
gpg --export --armor CAEB3DC3BDF7FB45 | sudo apt-key add -
sudo apt-get update to see if there are still any errors
That should resolve it.
User justinmnoor posted this on 31 March for the NO_PUBKEY 089EBE08314DF160 error:
Hi. If you have time, please review the QGIS Installers Documentation for Ubuntu Xenial 16.04.
I installed Las Palmas 2.18.5 on Ubuntu Xenial 16.04 LTS, following the instructions at:http://www.qgis.org/en/site/forusers/alldownloads.html#debian-ubuntu
The following error was generated:
W: GPG error: http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu xenial InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 089EBE08314DF160
I resolved it with: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key 089EBE08314DF160
This key is not referred to in the documentation. Sorry in advance if I've missed anything.
These fixed it for me.

@dennilopez6 thanks so much! The steps above fixed the issue. I noticed that the steps above are now at http://www.qgis.org/en/site/forusers/alldownloads.html#debian-ubuntu .

Still an acceptable workaround, though it's a mess that it's not fixed in late 2018!!!

The key mentioned in this issue is now expired as of 8/16/2019. Workaround no longer works.

I'm having the same issue with Debian stable. Attempts to retrieve a new key only result in retrieval of the old key and gpg reports that the key is unchanged as a result. I noticed this around the same time as user from the previous comment. It had been working fine for the last couple months.

@jef-n I just had somebody in xenial trying to install QGIS with deb http://qgis.org/ubuntu-ltr  xenial main ubuntu-ltr but apt update complains that the key on the server is outdated. do you have any hints?

plain xenial has also long been dropped - 3.4 requires GDAL2 at least.  You can go for ubuntugis-ltr + ubuntugis.

ah, I missed the foot notes thanks

Today on 2019  i had to addapt this solution to:

wget -O - http://qgis.org/downloads/qgis-2019.gpg.key | gpg --import

gpg --fingerprint 51F523511C7028C3

sudo apt-get update

gpg --export --armor 51F523511C7028C3 | sudo apt-key add -

sudo apt-get update

Today on 2019 i had to addapt this solution to:

See also https://qgis.org/en/site/forusers/alldownloads.html#debian-ubuntu