public RequestID generator

In my project we can start running code not only through http request, but with cli command or just background processes. I want to be able to trace everything that happened in different part of system by unique id with the same format. For http I use RequestID middleware, but there is no public methods which I can use to generate ID. Copy-past of that functions in project will cause different formats in future.
Do you mind to move request id generator to separate package, or just provide public function to generate new key?

@smacker I see, so you want these ID's to be consistent between http requests and other actions? yes, I can move https://github.com/pressly/chi/blob/master/middleware/request_id.go#L65 to a public function to generate the next request id, thats okay with me

@smacker I've added middleware.NextRequestID() for you :)

thank you sir!

thank you sir!


Signing CSR

Hello,
congrats on the project, I think something like this is widely needed.
It would be great though, if it were possible to sign CSRs.
Regards
Alexander

Hi Alexander,
Thank you.
I will look into it. Just to make sure, this would be to behave like a real CA where clients would send their CSR and the easypki would have no idea about the client private key ?
This will need a bit of refactor, as the current Sign method also generate the private/public key. So I will need to separate CSR creation and Signing (which should have been done since the beginning).

I will look into it. Just to make sure, this would be to behave like a real CA where clients would send their CSR and the easypki would have no idea about the client private key ?

Yeah in my case some hardware generates the private key and the only component I (can) get is the CSR.

I did not get time to get this done, and I unfortunately won't get any in the coming weeks. I am unassigning myself for now.

Secondary example: I set up Easy PKI as my network CA (it lives on a protected server).  I have a Unifi controller that I am trying to get a CA cert for ([OT] UNMS seems to be able to accept a separately generated key FWIW).  The official certificate installation process is to have the Unifi controller generate a CSR (which it gives you in DER form- I can convert to PEM and back), and you provide it back with a signed cert and any needed CA certs which it then installs into its keystore file


Include parent title in topics dropdown and sort by both

To make it easier to find topics to select.
Trello: https://trello.com/c/tUXtPH7a/91

:+1:


Social Media

[ ] Set up a big3D YouTube channel with graphic on top
[ ] Setup Facebook account
[ ] Setup InstaGram account


Member (de)serialization ignores actual type and uses member's declared type

When (de)serializing a field or property, Serializer uses the type that the field or property was declared as, not the type that's encoded in XML or the value's actual type.
Using GetType() (when serializing) or GetTypeToDeserialize() (when deserializing) would solve this issue, albeit at the cost of some performance.

It appears that the same thing happens when serializing as well - Serializer does not copy over the Type attribute from serialized members' XML nodes.

Fixed with https://github.com/Carnagion/GDSerializer/commit/6a3655a3354aa409fa0133d6c356849dcec6ada6.


Can we create a new tag release?

@elsh I need a new tag release with this change https://github.com/uber/mockolo/pull/76

Let's wait till https://github.com/uber/mockolo/pull/79 is merged. Feel free to add yourself to the reviewer list (I tried to add you but it couldn't seem to find).

https://github.com/uber/mockolo/releases/tag/1.1.3


Wrong showing of type description

TypeScript Version:  2.0.3
Code
I have next declaration of phoneCheck function

Expected behavior:
Must to show Callback<boolean>
Actual behavior:
TS shows wrong type name for Callback param on phoneCheck function.

TS shows correct type name for Callback param on data param of callback.

Please provide a self-contained repro, including text.
It's also not clear from your description exactly which text is wrong or what you expected the description to be, nor is it obvious where the caret is.

type Callback<T> = (error: Error, data: T) => void;

class Test {
    post(params: number, cb: Callback<boolean>) {
    }
}

class Test2 extends Test {
    run () {
        this.post(1, (err, data2) => {})
    }
}

You can try this example. If I put mouse curcsor over this.post in Test2 class I see method post(params: number, cb: Callback<this>). If I put mouse co=ursor over data2, it shows parameter data2: boolen. It's not crytical but cb: Callback<this> is not right
I use VS Code IDE.

What do you expect data2 should be?

@mhegazy It's boolean. But it has wrong description when I check post definition. It must be Callback<boolean>, but shows Callback<this>

@mhegazy It's boolean. But it has wrong description when I check post definition. It must be Callback<boolean>, but shows Callback<this>

I updated TS to v2.0.6. I can't repeat this error

thanks!


update INSTALL.md: fix typo in receive

Fix typo in receive word
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

No longer valid


Add support for distinguishing CREATE from UPDATE (and DELETE)

The current model contains a single action attribute that combines the operation and target for the event, i.e. GET_ALL_ARBEIDSFORHOLD, GET_ARBEIDSFORHOLD, UPDATE_ARBEIDSFORHOLD.
One improvement (or extension) is to separate these, having a separate attribute for the action (CREATE, READ, UPDATE, DELETE, ...) and the target (ARBEIDSFORHOLD, ELEV, PERSONALRESSURS).

I think this is a good improvement, one issue we need to look into if we go for this solution is that the generated Actions class in the model is used for verification that the adapter uses:
https://github.com/FINTprosjektet/fint-sse-adapter-skeleton/blob/master/src/main/java/no/fint/provider/adapter/event/EventStatusService.java
There are probably more places as well that would need to take into account two values instead of the old way where it was just one string.

Perhaps we should leave the generated Action classes as-is.  In this way the adapter can convey whether it supports only read, or whether it supports any modifications at all.
Then the Operation enum is used to convey the purpose of the modification requested through the consumer.

I think this is a good idea. If we do this we can also deprecate ACTION and add a new field called target. This will normalize the modell. ACTION as it is today has to "informations" - the noun and the subject.
Should we wait a bit before we do this?

What if we do it like this:

Add operation and target as members to the modell and leave them blank for now. Of course we also need to make sure to be backwards compatible.
Add values to operation and target
Start implement internal stuff with the new model.
Mark action as deprecated.

Timeline:

Before summer we start using the model internally
At end of 2019 all adapter vendors is using the new model.

Fixed by b091c40e5da4bc80ffc1a48a09414383101f8ff4

In the update I can only see Operation added, should it have a target as well? As @fsjovatsen mentioned.
Also, can we start populating these fields already? we should have all the necessary information from action, right?


Show unread comments, even if boring or written by a troll

https://hup.hu/cikkek/20161128/hupper_2?comments_per_page=9999#comment-2044942

Released in v2.3.0


Is responsive-images.php still relevant?

I'm trying to wrap my head around if the three filters added in responsive-images.php
is needed. Or where they added at a time when WordPress core didn't support responsive images and are just "technical debts" now?
For me they seems to cause more issues than they solve (like most stuff in cleanup.php that I stopped using long ago), but then again I'm no master at responsive images.

@Aetles I'm not sure how necessary it is, but I found the following which might explain why we have it in FP:

While WordPress does an exceptional job at inserting all available sizes into the srcset attribute, the sizes attribute is slightly more difficult to predict. This is because the sizes attribute is responsible for telling the browser how wide the image will be at every available viewport. Since that information will be different depending on the styling of the user’s theme, the best we can do is provide the following reasonable default:

sizes="(max-width: {{image-width}}) 100vw, {{image-width}}"

This default sizes attribute accomplishes two things. First, it ensures that a valid sizes attribute exists on the image, which recently became a mandatory requirement according to the specification. Second, it ensures that the browser doesn’t provide an image source that is larger than the original width requested. If there is any CSS manipulating the size of the image at different viewport widths, however, this default sizes attribute becomes less helpfull.

Since the default sizes attribute will only help with images that are not altered by CSS, filter hooks have been provided to allow theme developers to adjust the sizes attribute of every image, ensuring that a perfect sizes attribute can be delivered at every breakpoint.

Source: https://www.smashingmagazine.com/2015/12/responsive-images-in-wordpress-core/

Ah, great digging.
I had some weird issues with the wrong (low res) image being loaded instead of the appropriate one and removing these filters solved it. I never looked into what the issue really was (and I don't have deep knowledge of responsive images in WP to begin with).
(Generally speaking I'm trying to have as few filters as possible that modifies WP behaviour or "clean things up" since those tend to become irrelevant or even causing issues as WP develops over time. Have good documentation in code about why they were added and/or what issue they solve helps a lot.)


Downgraded Scalatra For Use With javax.servlet.servlet-api-2.5

Fixes #40. servlet-api-2.5.jar was in the hadoop classpath and causing conflicts with mango app, which uses servlet-api-3.0.
Though downgrading the servlet-api removes asynchronous processing support, this allows use of the latest hadoop distribution, which still uses servlet-api-2.5.jar

Merged build triggered.

Merged build started.

Merged build finished. Test PASSed.

Merged! Thanks @erictu!


1.23: consolidate SDK patch

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm

/approve


avt_vimba_camera: 0.0.10-0 in 'kinetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository avt_vimba_camera to 0.0.10-0:

upstream repository: https://github.com/srv/avt_vimba_camera.git
release repository: https://github.com/srv/avt_vimba_camera-release.git
distro file: kinetic/distribution.yaml
bloom version: 0.5.26
previous version for package: null

avt_vimba_camera
* Merge pull request #26 <https://github.com/srv/avt_vimba_camera/issues/26> from 130s/k/add_ci
  [CI] Add config for ROS Kinetic (and Lunar as an option).
* [CI] Add config for ROS Kinetic (and Lunar as an option).
* Merge pull request #25 <https://github.com/srv/avt_vimba_camera/issues/25> from mintar/fix_open_camera
  Fix opening the camera
* Simplify openCamera() logic
* Open camera on init
  Without this patch, the camera is never opened. This bug was introduced in 63f868791.
* Give more information to the user
* Merge pull request #22 <https://github.com/srv/avt_vimba_camera/issues/22> from plusone-robotics/dyn_reconfig_fix
  Dynamic reconfigure fix
* Fix #21 <https://github.com/srv/avt_vimba_camera/issues/21>: Retry camera opening and handle SIGINT
* Changed file mode in order to build
* Fixed crashing of dynamic reconfigure
* Fixed dynamic reconfigure configuration that did not allow camera parameters to be updated
* Fix call QueueFrame() method
* Fix CPU overhead issue
* Fix #18 <https://github.com/srv/avt_vimba_camera/issues/18>
* Merge pull request #17 <https://github.com/srv/avt_vimba_camera/issues/17> from josepqp/kinetic
  Added ARM 32 to CMakeList.txt
* Stop camera before destroy it
* Added ARM 32
* Merge branch 'kinetic' of github.com:srv/avt_vimba_camera into kinetic
* Change variable scope
* Fix #15 <https://github.com/srv/avt_vimba_camera/issues/15>: do not depend on turbot_configurations
* Merge pull request #14 <https://github.com/srv/avt_vimba_camera/issues/14> from josepqp/kinetic
* Added ARM 32 bits libraries

  Modified CMakeList.txt to compile with ARM 32 bits

  Added Iris Parameter
* kinetization
* Fix sync problems after camera tests
* Add a sync timer
* Try stereo image sync
* Add a check timer
* Fix #12 <https://github.com/srv/avt_vimba_camera/issues/12>: allow bigger resolutions
* Fix camera info
* Fix camera config
* Fix camera info when decimation
* Make sync node acts as stereo sync checker
* Include a check timer on stereo_camera
* Perform the stereo_sync in a separate node
* Publish camera temperatures
* Change the way of reset
* Increase the initial wait time before checking sync
* Add a sync watcher node
* Fix branch mix
* Remove unused variables
* Left and right callback in a separate thread
* Change default sync time
* change logging messages
* fix binning
* add stereo launchfiles
* removed prints
* set stereo launchfiles
* removed unused params
* calibration epi. 4
* improvements to stereo node
* merge with v2.0 SDK
* upgrade to VIMBA SDK 2.0
* upgrade to 1.4
* changed ros prints from info to debug
* removed comment
* changed stereo camera launchfile
* Merge pull request #11 <https://github.com/srv/avt_vimba_camera/issues/11> from lucasb-eyer/indigo
  Set the frame_id of the image header, too.
* Set the frame_id of the image header, too.
* Contributors: Isaac I.Y. Saito, Martin Günther, Miquel Massot, Shaun Edwards, SparusII, agoins, josep, lucasb-eyer, plnegre, shaun-edwards

Is it intended to use different branches for doc and src?

No, it should be kinetic. If it's wrong it must have been me wrongly
pressing Enter too fast...
On 16 Aug 2017 2:44 p.m., "Dirk Thomas"<EMAIL_ADDRESS>wrote:

Is it intended to use different branches for doc and src?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ros/rosdistro/pull/15785#issuecomment-322758914, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABiVjHASUwVVpDKs6oQBLTNLgvgh-u_Rks5sYuRCgaJpZM4O4u0y
.

Thanks for the support!


Pane Navigation not visible

Hello,
thank you for this really nice theme, which I enjoy a lot! One small issue: I use the Pane Relief Plugin, which enables per pane history, so that you can go back and forth. With your theme enabled, the two arrows are not visible anymore. Can you fix this, please? The image shows how it should look like (in the image I use the minimal theme)

With release 1.4, I added support for the Style Settings Plugin. Using that plugin, you can simply re-enable the titlebar to use the Pane-Relief arrows 🙂


React native app crashes on launch

When I run the app, an error appears: ".mask" blabla.

Okay, can you provide the screenshot of the error message? and the commands?

whatev, I'll try to create a pull request


custom-server-egg

This is a simple example with egg.js.

Can you run yarn lint --fix?
https://travis-ci.org/zeit/next.js/jobs/450321027
I'll close this PR for now, feel free to open a new PR with the updated linting 👍


realm name and proxy_uri configurable via docker-env

First of all: thanks a lot for this project. It helps a lot and works quite well!
I use the resulting docker image from your Dockerfile and needed to change the realm name in keycloak and to add my own nginx reverse proxy via docker environment.
Please have a look in to the application.yaml, attached.
application.yaml.zip
The keycloak.realm.id property defaults to "camunda" (as it was hard coded) and the plugin.identity.keycloak.proxyUri is taken from the env, too.
Maybe you can adapt this change into the code?

I recently added a pull request: #49

Hi @mf75,
the SSO Example's Docker image and the underlying application configuration is not intended to cover all use cases. It's just an example. Even if the example shows how to solve a wide range of challenges when it comes to combining SSO with the Keycloak Identity Provider it is strongly recommended to build your own configuration and Docker image according to your own needs.
Anyway, I'm going to have a look at that.
Thanks,
Gunnar

I never intended to call it a bug - just struggled to see, where to change the type of ticket I post here.
Please don't mind, if I open some more "issues" of this not-a-bug category.
Thanks a lot
Markus

Your're welcome 👍


Feature Request: Support different formats of summaries

Summaries are currently printed out as a single JSON line. It would be handy to be able to change this format to a more readable format (such as YAML or multiline JSON).
These summaries are more likely to be checked into git. As such, having them in a more human-readable format would be beneficial.

While the issue itself seems easy to implement, it unravels new problem that we had to address at some point: dvclive configuration.

Can we start with the easy implementation 😆 ?

@dberenbaum yes we can, we can probably pass it silently by replacing summary flag with string arg summary_path that will also tell dvc what format we want the summary in. Though in that case there would be some reworking needed also on DVC side.

I was thinking even smaller, like can we just dump the json with an indent value so that it gets pretty printed?

Ahh, ok, I guess my imagination went too far. Give me a sec...

@Benjamin-Etheredge would properly formatted json be good enough for start? Created PR for that.

Fixed in #78. We can revisit it if we need yaml support, though purely aesthetics reasoning is not enough to justify amount of changes needed to support different summary formats. (It would require configuration, which is currently not necessary for dvclive)


Intellisense doesn't add namespace imports

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.65.2
OS Version: Windows 11

Steps to Reproduce:

Create files test.ts and bam.ts
Create and export an helloworld() function in bam.ts
Type bam in test.ts and try to get VS Code to add in a namespace import in the form: import * as bam from "./bam";

I was led to believe that this was possible by the following comment: https://github.com/microsoft/TypeScript/issues/23830#issuecomment-590572306_

This is the intended behavior - that comment only refers to unqualified module names, but "./bam" is a qualified path

This is the intended behavior - that comment only refers to unqualified module names, but "./bam" is a qualified path

What's stopping Intellisense from resolving my own modules using the filename, as a code action?

No one has suggested that it do so nor implemented it

Sure :)


settings -> livestreamer -> hls live edge (and other number boxes) broken on os x

Thanks for reporting an issue!
Please read the contribution guidelines first!
Feel free to use the following template. Be as detailed as possible.
Don't forget to remove this text before submitting.

Description
When changing the value in settings for numeric fields in os x, the up/down arrows in the input box don't do anything.   Also, clicking in the box and using the track pad to scroll up/down is very broken.   It seems to be able to scroll one direction after clicking, but the number shoots up/down so fast and then the page starts scrolling.   Presumably this is the same trackpad interaction issues as the problem with smooth scrolling was.   This functionality should probably be disabled on OS X and then the up/down arrows should be fixed.
Entering a number via keyboard works fine.
...
Expected / Actual behavior
...
Reproduction steps

...
...
...

Environment details (operating system, etc.)
os x, chrome -- latest stable everything    14.1 livestreamer
...
Comments, logs, screenshots, etc.
Thanks so much for making this!   Still loving it!
...

These input fields are default HTML5 number fields and there are no additional event listeners of any kind added to them, so this has nothing to do with smooth scrolling. It's Chromium which is adding these features to the number fields.
This issue seems to be related to the used Chromium version of NW.js, which is a little bit old (NW.js 0.12.3 -> Chromium 41.0.2272.76) compared to one of the new Chrome releases (52/53/54). An upgrade to a new NW.js version could fix this (#275), but I'm not sure.
Also, scrolling on top of the number fields doesn't do anything for me in my OSX VM (I don't have a trackpad, though). Using the arrow keys on the keyboard does work, clicking the arrow buttons inside the field is indeed broken.

With regards to the smooth scrolling, I was referring to the rate at which
the numbers change when scrolling in the number box.   it's very jerky like
the smooth scrolling is/was.
https://www.youtube.com/watch?v=DCiM0aP5QNw
Everything in that movie is with the trackpad (pretty sure -- at least
almost all of it).  The seemingly random times when the number changes is
the trackpad.
--Zac
On Sun, Aug 7, 2016 at 9:02 AM, Sebastian Meyer<EMAIL_ADDRESS>wrote:

These input fields are default HTML5 number fields
https://github.com/bastimeyer/livestreamer-twitch-gui/blob/v0.14.1/src/templates/settings/SettingsLivestreamer.hbs#L47
and there are no additional event listeners of any kind added to them, so
this has nothing to do with smooth scrolling. It's Chromium which is adding
these features to the number fields.
This issue seems to be related to the used Chromium version of NW.js,
which is a little bit old (NW.js 0.12.3 -> Chromium 41.0.2272.76)
compared to one of the new Chrome releases (52/53/54). An upgrade to a new
NW.js version could fix this (#275
https://github.com/bastimeyer/livestreamer-twitch-gui/pull/275), but
I'm not sure.
Also, scrolling on top of the number fields doesn't do anything for me in
my OSX VM (I don't have a trackpad, though). Using the arrow keys on the
keyboard does work, clicking the arrow buttons inside the field is indeed
broken.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bastimeyer/livestreamer-twitch-gui/issues/282#issuecomment-238090606,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIycpG7MX1wZt9RMMMZvW3kKfLbg_Acks5qdgGcgaJpZM4JeeIr
.


Problem about this operator

Hi I have a problem with traspiling class and mixing it with existing es5 code to better esplain the issue I've created a repo. the problem is to use in this simple function a method of class generated via typescript
function setObjVal(fn,val){
    fn(val);
}

repo:
https://github.com/mydiscogr/jscontextproblem
There's is a workaround for this problem thanks

You are passing a prototype member function which will only work when called by the prototype chain. This will always work, BTW.
setObjVal((val) => obj.setVar1(val), val);

It sounds like you're proposing that we emit all classes as constructor functions which re-allocate a closure per method. This isn't a good idea, both in terms of ES6 compat and in terms of runtime performance (one closure per method per instance is much worse than one closure per method). But if this is actually a question or something (I can't tell), we'll need more information to answer it.

prequel.
"I have wrote time ago one app in node so i would integrate with typescript"
Sorry, I would better explain the problem.
I have a simple ES5 function:
function setObjVal(fn,param){
fn(param);
}
well I see only, that if ,fn is a method, and the object method use this operator,
in my many way to declare in es5 an object don't work.
I've found that only in one way works, with closure, as below.
module.exports = (function(){
var self = {

    var1: "unmod",
    setVar1: function(val){

        self.var1 = val;
    }
}

return self;

})();
So I dunno if closure can cause a problem (why?? is bad??),
or memory wasting of memory, or cause memory lacks,
but my problem is more simple, I would like to create an object in safe way, just to
avoid critical bugs (you know the difficult to find bug sometimes).
So I hope that if you declare a class in typescript your way to generate a class
should (must?) be safe(so I think). Because this is a transpiler, and can be a really pain to find
a "bug" of code not wrote by you...(think about c++ compiler, nobody is interested as create asm src..., we assume that code generated is correct)
So wrapping up. I'm not asking to generate and object as I wrote in my repo,
I'm not a great js programmer
but I ask to taking care of this problem.
I hope to make more clear my question.
thanks for attention.

No other comment? .. a solution.

Class emit in TypeScript is meant to be inline with the ES6 spec. this allows you to interoperate your classes with other ES6 classes and vice versa. TS will place methods on your prototype, and will support inheritance as well.
You can still port your implementation to TS by giving your function a type annotation as:
// type of instance of MyClass
interface MyClass {
    var1: string;
    setVar1(val: string): void;
}

// constructor function of MyClass
var MyClass: { new (): MyClass; } = <any>function () {
    var self = {
        var1: "unmod",
        setVar1: function (val) {
            self.var1 = val;
        }
    };
    return self;
};

// Make it the export of my module
export = MyClass;
})();


Autocomplete: missing Watch decorator for property 'value'

Prerequisites

[X] I have searched for duplicate or closed issues
[X] I have read the contributing guidelines

Describe the issue
Changing the value of the autocomplete property "value" using javascript is not being honored. The options are filtered by the previous value. The property "value" has no @Watch decorator.
Reduced test cases
No response
What operating system(s) are you seeing the problem on?
No response
What browser(s) are you seeing the problem on?
No response
What is the issue regarding ?
@trimble-oss/modus-web-components
What version of npm package are you using ?
No response
Priority
Medium
What product/project are you using Modus Components for ?
e-builder
What is your team/division name ?
e-builder
Are you willing to contribute ?
Yes
Are you using Modus Web Components in production ?
No response

Closing this one as the PR is merged.


WIP Added Depoly to IBM Cloud Support

Includes a Deploy to IBM Cloud button and a manifest.yml file that uses a static ML end point on IBM Cloud
Closes #7

@rhagarty Two question for you on this pr:

I used a static ip for the ml-endpoint when deploying to IBM Cloud. That end point is a permanent one that CODAIT set up on IBM Cloud with the plan to make it publicly known as a permanent access point in the future. Is having this static ok in a Code Pattern or do we need to find a way for the user to deploy the model to IBM cloud and chain them together?
I can't seem to find any documentation on how to add the badge for tracking the number of times the web app has been deployed. It almost seemly like the feature was deprecated from what I could find.

Note this is WIP since I still need to add actually instructions after the deploy button, though it will just be a screen shot telling them to click the web button once it's finished deploying.

1 - not sure about that. If the plans are to make it public, it seems fine. Would be better if this was a host name.
2 - the tracking badges has issues, so we removed it from the code patterns.

Added deploy steps if you could take a look @djalova @rhagarty

LGTM


SCons: Refactor Linux linker options with linker=<bfd|gold|lld|mold>

The new option is linker and lets the user specify the argument to
the-fuse_ld= linker flag directly. The supported options are:

default: No change, typically uses GNU ld (bfd) unless the user or
distro picked a different default /usr/bin/ld.
bfd: GNU ld from binutils
gold: GNU gold from binutils
lld: lld from LLVM
mold: mold, an extremely fast modern linker, not (yet) intended for
use in production but great for development speed. Provided by distro
mold package or needs to be compiled from source and installed to
/usr otherwise.

Removes the use_lld=yes option, and make lld actually usable with GCC
too.
Not all the above are compatible or recommend for LTO, we recommend
using GNU ld with GCC LTO, or lld with LLVM ThinLTO.

Implements and closes https://github.com/godotengine/godot-proposals/issues/4531.
Supersedes #61433.

As a side note, @Calinou suggested this in https://github.com/godotengine/godot-proposals/issues/4531#issuecomment-1125505587:

Replace use_lto=yes and use_thinlto=yes with lto=none|thin|full.

I think that makes sense, could be done in a follow-up PR.

Is this really needed? I always use LINKFLAGS="-fuse-ld=mold" and it works fine. I see this as a compiler dependent feature tbh

Is this really needed? I always use LINKFLAGS="-fuse-ld=mold" and it works fine. I see this as a compiler dependent feature tbh

Well the same can be said of most other build options. You can set CC and CXX instead of use_llvm=yes, etc., and LINKFLAGS="-fuse-ld=lld" instead of use_lld=yes which we currently have.
We just abstract them in a way that makes them either to use. Personally I always have to double check the exact syntax of the link flag so IMO having an option we can validate makes a lot of sense.


fix: Replace imported stylesheet(#93)

Applies the solution from arief135 to recurring duplicate issues  (material-components#139, material-components#198, material-components#155). Build should complete without error without applying the earlier workaround. However, the MDC-103 Web codelab instructions must be changed accordingly to prevent further issues.

@googlebot I signed it!


achcli: fix typo in usage instructions

correction to usage instructions for reformat

Codecov Report

Merging #737 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #737   +/-   ##
=======================================
  Coverage   93.14%   93.14%
=======================================
  Files          66       66
  Lines        5804     5804
=======================================
  Hits         5406     5406
  Misses        239      239
  Partials      159      159


Ipopov/evening fixes

Fix for the latest found issue by @JaroslawRomanovich .

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: IgorPopov-MS sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@orezhko please merge this PR after #733 because it include #733  changes.


test failure with CPython 3.12b3: err::tests::fetching_panic_exception_resumes_unwind

I'm trying to update the pyo3 packages on Fedora Linux to the latest version (v0.19.1), but I'm getting a weird test failure with CPython 3.12b3:
---- err::tests::fetching_panic_exception_resumes_unwind stdout ----
--- PyO3 is resuming a panic after fetching a PanicException from Python. ---
Python stack trace below:
note: panic did not contain expected string
      panic message: `"Unwrapped panic from Python code"`,
 expected substring: `"new panic"`

failures:
    err::tests::fetching_panic_exception_resumes_unwind

c.f. https://github.com/PyO3/pyo3/blob/v0.19.1/src/err/mod.rs#L855-L868
I'm not sure how or why this is happening, but it looks like PanicException message is no longer extracted successfully here:
https://github.com/PyO3/pyo3/blob/v0.19.1/src/err/mod.rs#L317-L321

Yes, we haven't got 3.12 passing in CI yet, I think realistically this will come in PyO3 0.20 which might be a month off. Is it helpful for you if we try to expedite and backport onto a 0.19.2 release?

That would be very helpful, yes :+1: Let me know if I can help with something.
Note that I also need to support PyO3 v0.15, v0.16, v0.17, and v0.18 on top of Python 3.12 (mostly due to Python packages like cryptography using ancient PyO3 version to be compatible with debian stable or something) ... I'll likely need to do some patching to make that work, would you be interested in getting fixes out for those older branches as well?

Oof. I'll take a quick spin at fixing the 3.12 test suite now. (Might take a few days depending on how much is broken.)
TBH I think it will be a lot of work to backport proper releases for all those versions, as they will have multiple ways that their CI has bitrotted by now. I don't think I'll have capacity to do that, but if there's any information I can offer to help you with patching I will do my best to share it.
cryptography have updated their PyO3 dependency to 0.19 not too long ago, so you may be able to update that first to avoid some patching pain. I'd be interested in hearing what other packages require old PyO3 versions, potentially it would be better for the ecosystem if we collaborate to update those packages' PyO3 dependencies?

Oof. I'll take a quick spin at fixing the 3.12 test suite now. (Might take a few days depending on how much is broken.)

Thanks, no hurry. :)

TBH I think it will be a lot of work to backport proper releases for all those versions, as they will have multiple ways that their CI has bitrotted by now. I don't think I'll have capacity to do that, but if there's any information I can offer to help you with patching I will do my best to share it.

I'll take a look how much work it would be to fix things on older branches - if it's doable, I'd of course not keep those patches for myself but submit PRs for them.

cryptography have updated their PyO3 dependency to 0.19 not too long ago, so you may be able to update that first to avoid some patching pain. I'd be interested in hearing what other packages require old PyO3 versions, potentially it would be better for the ecosystem if we collaborate to update those packages' PyO3 dependencies?

That's good to know! Sadly cryptography is known to subtly break things with every release, so it's not as easy as just updating it, since that will break a dozen other things. :( But yes, in general, it would be great if I didn't have to support 5 different versions of pyo3.
Right now, Packages in Fedora Linux that depend on different versions of pyo3 are:

pyo3 v0.15: fapolicy-analyzer, bcrypt (looks like they bumped to pyo3 v0.19 in the dev branch recently), cryptography (also bumped to v0.19 recently)
pyo3 v0.16: y-py
pyo3 v0.17: matrix-synapse
pyo3 v0.18: orjson (recently bumped to pyo3 v0.19)

@decathorpe FWIW if you're interested in discussing things we can be doing to reduce the maintenance burden with pyca/cryptography, @reaperhulk and I are happy to discuss and share the different constraints we operate under (and yes, the fact that many of our users are on distros with hilarious out of date software, and they wish to use this week's cryptography with last millenium's OS is, hilariously one of them).

Looks like #3305 is part of the story for 3.12b3. There is still an occasional refcounting failure in some tests which is probably due to the immortal objects changes (probably our FFI definitions need updating).

Can confirm that this was fixed with pyo3 v0.19.2. Thanks again!


Add basic generation config

This one setups up the generator based on the requirement of the codicons repo being cloned into dist. Seems there are no real changes of note as far as new icon releases. There were some icons that had no viewBox so I opted to set those to a decent default. This could just as easily be changed to simply skip adding one too.

Looks absolutely perfect.
I would like the viewBox default to be of 0 0 16 16 with sync to other sibling icons of the set.
@mallardduck - thanks for this. I will use this as a reference and update the other packages too. There are few icon sets which are little complicated, I will tag you wherever needed so that we can discuss & sort it out.

Thanks, glad to help! Dries has already accepted the PR I sent for icon filename suffix.
So the current version of the generator is 1.1.0 meaning once we see a new release the custom repo in composer.json can be removed.

Thanks @mallardduck !


⚠ Processors has degraded performance

In 77029dc, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 83 ms

Resolved: Processors performance has improved in c8a8978.


[BUG/QUESTION] Character encoding bug

Please prefix your issue with one of the following: [BUG] [PROPOSAL] [QUESTION].
Package version, Laravel version
"php": ">=5.5.9",
"laravel/framework": "5.2.*",
"maatwebsite/excel": "^2.1"
Expected behaviour
Chunk import work as same as normal import if the character encoding is ISO8859-2
Actual behaviour
If i load the csv with chunks:
Excel::filter('chunk')->load('teszt.csv','ISO-8859-2')->chunk(250,function($rows){
dd($rows);
});
CSV content:

All columns that contains accented characters will be null.
Here is the dump:

But if i load the same csv without chunk, everythink is ok. Like this:
$records = Excel::load('teszt.csv','ISO-8859-2')->all();
dd($records);

Exception stack trace
No Exceptions thrown.
Screenshot of Excel file
Steps to reproduce the behaviour
Excel::filter('chunk')->load('teszt.csv','ISO-8859-2')->chunk(250,function($records){
dd($records);
});

@tomizej I'm having the same issue, did you manage to solve it?


feat(docker): implement the structure

Implement for the container the structures requests, properties, store and service.

I have renamed docker to containers.


[Query] Process killed by linux kernel when CPU usage goes high

The processes executed with this library got killed by kernel when there is CPU contention.
Is there any way to prevent this?

@kunaldawn Is it related to the library? Can you provide code snippet, environment description and steps to reproduce the issue.

@sevlyar let me create an isolated example and come back to you


Set default_prepared_statements for trilogy

Trilogy doesn't support prepared statements yet so until https://github.com/github/trilogy/pull/3 is merged and support is implemented in the adapter this should be set to false.
Fixes #5

👋🏻 Thank you!


Resolve correct link path for tsconfig.extends

This PR fixes #131643
@mjbvz Sorry for the latency, but its here. I tried, but failed to reuse existing resolve algorithm. However I hope it would be possible in future with standalone lib like tsconfig-paths.
I think this is almost 1:1 implementation to what we have in TypeScript repo today. Now, if the file doesn't exist, it won't create a link to a non-existing file.
I didin't handle a case where tsconfig might be referenced from the root since its a rare case and I won't believe someone actually does that (however we can check whether path starts with / at least).
I made everything up to the provider method async only because of async exists (fs.stat).
@fregante I remember you compared it to WebStorm. This implementation is much better!
However, I noticed a bug that if you write path to the existing file and then swap it with a non-existing, the provider would be called and it'd return undefined, but the old link wouldn't be dropped.

I just remembered that I forgot to describe purpose of resolveNodeModulesPath. Here is example with library from linked issue:
- 📁 node_modules
    - 📁 @sindresorhus
        - 📁 tsconfig
            - tsconfig <-- would resolve to here!
- packages/some-package
    - 📁 node_modules
        - 📁 @sindresorhus
            - 📁 another-lib
    - 📁 lib
        - tsconfig.json (extends: @sindresorhus/tsconfig)

Can you also please open a new issue to track using node resolution for the tsconfig paths. #131643 was about the referencing the package directory without a tsconfig.json at the end, but since this PR adds a lot more functionality we should track that separately

I thought #131643 was about incorrect extends resolving algorithm in general, but sure I'll do it in a moment.

I changed the implementation and added new internal command.

I didn't add the nls key, I looked into ./rename.ts and can see it wasn't defined too.

I tried to test the command, it opens the tsconfig, but doesn't display error message. Breakpoints works from time to time, so I can't figure out things (everything worked perfectly with previous commit).

When i try to open nonexisting file it seems that window.showTextDocument and workspace.openTextDocument complain about in console (cannot open ... Unable to resolve nonexistent file)
When it returns undefined nothing happens

Can you also please open a new issue to track using node resolution for the tsconfig paths

I opened #141112, that has a link reference, feel free to request details if I missed something.

Please review all opened comments when you have time for it. I'll be waiting for reply!

Thanks! Will be in the next insiders build for testing


proposal: runtime: add per-goroutine CPU stats

Per-process CPU stats can currently be obtained via third-party packages like https://github.com/elastic/gosigar. However, I believe that there exists a need for a certain type of applications to be able to understand CPU usage at a finer granularity.
Example
At a high level in CockroachDB, whenever an application sends a query to the database, we spawn one or more goroutines to handle the request. If more queries are sent to the database, they each get an independent set of goroutines. Currently, we have no way of showing the database operator how much CPU is used per query. This is useful for operators in order to understand which queries are using more CPU and measure that against their expectations in order to do things like cancel a query that's using too many resources (e.g. an accidental overly intensive analytical query). If we had per-goroutine CPU stats, we could implement this by aggregating CPU usage across all goroutines that were spawned for that query.
Fundamentally, I think this is similar to bringing up a task manager when you feel like your computer is slow, figuring out which process on your computer is using more resources than expected, and killing that process.
Proposal
Add a function to the runtime package that does something like:
type CPUStats struct {
    user time.Duration
    system time.Duration
    ...
}

// ReadGoroutineCPUStats writes the active goroutine's CPU stats into
// CPUStats.
func ReadGoroutineCPUStats(stats *CPUStats)

Alternatives
From a correctness level, an alternative to offering these stats is to LockOSThread the active goroutine for exclusive thread access and then get coarser-grained thread-level cpu usage by calling Getrusage for the current thread. The performance impact is unclear.
Additional notes
Obtaining execution statistics during runtime at a fine-grained goroutine level is essential for an application like a database. I'd like to focus this conversation on CPU usage specifically, but the same idea applies to goroutine memory usage. We'd like to be able to tell how much live memory a single goroutine has allocated to be able to decide whether this goroutine should spill a memory-intensive computation to disk, for example. This is reminiscent of https://github.com/golang/go/issues/29696 but at a finer-grained level without a feedback mechanism.
I think that offering per-goroutine stats like this is useful even if it's just from an obervability standpoint. Any application that divides work into independent sets of goroutines and would like to track resource usage of a single group should benefit.

A possible solution to showing high level usage of different query paths can be achieved by setting profiling labels on the goroutine:
https://golang.org/src/runtime/pprof/runtime.go?s=1362:1432#L26
And doing background profiling on the job:
https://go-review.googlesource.com/c/go/+/102755

Yes, this is exactly what labels are for.  A nice thing about labels is that they let you measure CPU or heap performance across a range of goroutines all cooperating on some shared task.
https://golang.org/pkg/runtime/pprof/#Labels
https://golang.org/pkg/runtime/pprof/#Do
Please let us know if you need something that is not addressed by labels.

Thanks for the suggestion. My main concern with profiling is that there is a non-negligible performance overhead. For example, running a quick workload (95% reads and 5% writes against a CockroachDB SQL table) shows that throughput drops by at least 8% when profiling with a one second interval.
I'm hoping that this information can be gathered by the scheduler in a much cheaper way since the question to answer is not "where has this goroutine spent most of its time" but "how much CPU time has this goroutine used". Would this even be feasible?

Ah, I see.  I would think that always collecting CPU statistics would be unreasonably expensive.  But it does seem possible to collect them upon request in some way, at least when running on GNU/Linux.  Every time a thread switched to a different goroutine it would call getrusage with RUSAGE_THREAD.  The delta would be stored somewhere with the old goroutine.  Then it could be retrieved as you suggest.  Memory profiling information could be collected separately.
I don't know how useful this would be for most programs.  In Go it is very easy to start a new goroutine, and it is very easy to ask an existing goroutine to do work on your behalf.  That means that it's very easy for goroutine based stats to accidentally become very misleading, for example when the program forgets to collect the stats of some newly created goroutine.  That is why runtime/pprof uses the labels mechanism.
Perhaps it would also be possible for this mechanism to use the labels mechanism.  But then it is hard to see where the data should be stored or how it should be retrieved.

And doing background profiling on the job:
https://go-review.googlesource.com/c/go/+/102755
Overall Go program usage can be queried from the enclosing container or process stats from the Operating system directly.

I'm curious what the internal experience with this mechanism is and how it is used exactly. I suppose some goroutine will be tasked with continuously reading the background profiling stream, but what does it do with it? Is there any code I can look at for how to get to "here's a stream of timestamped label maps" to "this label used X CPU resources"? Do we "just" create a profile every X seconds from the stream? And if that is the idea, how is that different from doing "regular" profiling at a lower sample rate (you might miss a beat every time you restart the sample, but let's assume this is ok)? I feel like I'm missing the bigger picture here.

Ah, I see. I would think that always collecting CPU statistics would be unreasonably expensive. But it does seem possible to collect them upon request in some way, at least when running on GNU/Linux. Every time a thread switched to a different goroutine it would call getrusage with RUSAGE_THREAD. The delta would be stored somewhere with the old goroutine. Then it could be retrieved as you suggest. Memory profiling information could be collected separately.

With the RUSAGE_THREAD idea, wouldn't we be adding a syscall per scheduler tick? That seems very expensive and would likely be a non-starter for use cases such as ours where we want to track usage essentially at all times for most of the goroutines.
The most lightweight variant I have seen is based on nanotime() per scheduler ticks, as my colleague @knz prototyped here (the code just counts ticks, but imagine adding up the nanotime() instead of the ticks:
https://github.com/golang/go/compare/master...cockroachdb:taskgroup?expand=1
I understand that this basic approach has caveats (if threads get preempted, the duration of the preemption will be counted towards the goroutine's CPU time, and there's probably something about cgo calls too) but it seems workable and cheap enough to at least be able to opt into globally.

I don't know how useful this would be for most programs. In Go it is very easy to start a new goroutine, and it is very easy to ask an existing goroutine to do work on your behalf. That means that it's very easy for goroutine based stats to accidentally become very misleading, for example when the program forgets to collect the stats of some newly created goroutine. That is why runtime/pprof uses the labels mechanism.

The PR above avoids (if I understand you correctly) this by using propagation rules very similar to labels. A small initial set of goroutines (ideally just one, at the top of the "logical task") is explicitly labelled via the task group (identified by, say, an int64) to which the app holds a handle (for querying). The ID is inherited by child goroutines. Statistics are accrued at the level of the task group, not at the individual goroutine level (though goroutine level is possible, just create unique task group IDs). I recall that the Go team is averse to giving users goroutine-local storage by accident, which this approach would not (there would not need to be a way to retrieve the current task group from a goroutine) - given the task group ID, one can ask for its stats. But one can not ask a goroutine about its task group ID.

Perhaps it would also be possible for this mechanism to use the labels mechanism. But then it is hard to see where the data should be stored or how it should be retrieved.

I agree that getting labels and either of the new mechanisms proposed to play together would be nice, but it doesn't seem straightforward. You could assign to each label pair (key -> value) assigned a counter (i.e. two goroutines with both have label[k]=v share a task group <k,v>. If a goroutine with labels map[string]string{"l1": "foo", "l2": "bar"} ran for 10ms, we would accrue these to both of these label pairs, i.e. conceptually somewhere in the runtime m[tup{"l1", "bar"}] += 10 // ms and similarly for l2. Perhaps the way to access these metrics would be via the runtime/metrics package.
One difficulty is that we lose the simple identification of a task group with a memory address which we had before, because two maps with identical contents identify the same task group.
On the plus side, any labelled approach would avoid the clobbering of counters that could naively result if libraries did their own grouping (similar to how one should not call runtime.SetGoroutineLabels), as counters could be namespaced and/or nested.
You mention above that

A nice thing about labels is that they let you measure CPU or heap performance

but profiler labels are not used for heap profiles yet, and I think it's because of a somewhat similar problem - a goroutine's allocations in general outlive the goroutine, so the current mechanism (where the labels hang off the g) doesn't work and the lifecycle needs to be managed explicitly. (Everything I know about this I learned from https://github.com/golang/go/issues/23458#issuecomment-377973247). Maybe the way forward there opens a way forward for sane per-label-pair counters?

The most lightweight variant I have seen is based on nanotime() per scheduler ticks, as my colleague @knz prototyped here (the code just counts ticks, but imagine adding up the nanotime() instead of the ticks:

The implementation using nanotime() is in fact ready already here: https://github.com/cockroachdb/go/commit/b08903328e01603dc1a620779ca9a926f056023e

Perhaps the way to access these metrics would be via the runtime/metrics package.

Done here: https://github.com/cockroachdb/go/commit/9020a4bfcf68d37c5b4d88627357ee0cff29137c

You could assign to each label pair (key -> value) assigned a counter (i.e. two goroutines with both have label[k]=v share a task group <k,v>

One elephant in the room is that looking up labels and traversing a Go map in the allocation hot path is a performance killer.

A nice thing about labels is that they let you measure CPU or heap performance

but profiler labels are not used for heap profiles yet,

This is the other elephant in the room: partitioning the heap allocator by profiler labels would run amok of the small heap optimization. (plus, it would be CPU-expensive to group the profiling data by labels)

In case it wasn't clear from the latest comments from @tbg and myself: we believe that doing things at the granularity of goroutines is too fine grained, and raises painful questions about where to accumulate the metrics when goroutines terminate.
While @tbg is trying to salvage pprof labels as the entity that defines a grouping of goroutines, I am promoting a separate "task group" abstraction which yields a simpler implementation and a lower run-time overhead. I don't know which one is going to win yet—we need to run further experiments—but in any case we don't want a solution that does stats per individual goroutines.

@knz I really like the feature in https://github.com/cockroachdb/go/commits/crdb-fixes. Could you create a pull request to go official?

thank you for your interest!
We're still evaluating whether this approach is useful in practice. Once we have a good story to tell, we'll share it with the go team.

@irfansharif sent https://go.dev/cl/387874 today which adds a /sched/goroutine/running:nanoseconds metric to runtime/metrics, which total time in _Grunning for the current goroutine. This is effectively equivalent to the ReadGoroutineCPUStats API from the first comment but hidden behind the runtime/metrics.
FWIW, I don't think this API is a good fit for runtime/metrics, which typically returns the same results regardless of the calling goroutine [1]. Given the other metrics available, like /sched/latencies:seconds, I expected this to be a histogram of time spent in running, which I think could be useful, but not for the resource isolation requirements in this proposal.
[1] We also expect some monitoring systems to discover and export all metrics. This metric would be meaningless to export directly, as it would only report on a single reporter goroutine.
cc @golang/runtime @mknyszek

On the CRDB side I've filed https://github.com/cockroachdb/cockroach/issues/77041 which I hope adds more color for as to why we're eager to get more observability into runtime internals.
Unrelated but perhaps of interest to the golang/runtime folks, we also use the # of runnable goroutines as an indicator for CPU saturation. We use this to power our admission control machinery for stability + better tail latencies. For that metric, we were able to get by peeking into already tracked runtime internals. For this running nanos however, I don't think we had similar allowances, hence the patch.
https://github.com/cockroachdb/cockroach/blob/0679c4cb6578/pkg/util/goschedstats/runnable.go#L22-L34
https://github.com/cockroachdb/cockroach/blob/0679c4cb6578/pkg/util/admission/granter.go#L29-L37

For the number of runnable goroutines, that would be quite reasonable to add to runtime/metrics. I actually thought we had that, but it looks like we only have to total number of goroutines. It would perhaps make sense to add something like the /memory/classes hierarchy where we subdivide the total number of goroutines by state.

For example, running a quick workload (95% reads and 5% writes against a CockroachDB SQL table) shows that throughput drops by at least 8% when profiling with a one second interval.

I find this incredibly surprising. I would expect the impact of profiling to be well under 1%. In fact, it's not obvious to me that statistical profiling would be more expensive than tracking time on every goroutine switch in an application that's heavy on goroutine scheduling.
If profiling didn't have this overhead, would it make sense to use profiling labels plus a stream of profile events to accumulate the information you need?

If profiling didn't have this overhead, would it make sense to use profiling labels plus a stream of profile events to accumulate the information you need?

We attempted something similar in https://github.com/cockroachdb/cockroach/pull/60508. I can try re-running a stock workload with profiling enabled to measure what the overhead is, perhaps things have changed since that 2020 comment. I'll do it next week.

In fact, it's not obvious to me that statistical profiling would be more expensive than tracking time on every goroutine switch in an application that's heavy on goroutine scheduling.

For my own understanding:

wouldn't tracking time on goroutines be faster than profiling since we'd avoid logging the entire stack trace? This presents also as memory overhead.
wouldn't it be slower to compute the total on-CPU time for a trace by analyzing the stream of profile events instead of reading off of a counter?
does the overhead for https://go-review.googlesource.com/c/go/+/387874/ come primarily from the use of nanotime() on every goroutine switch? Or are we also worried about the two added fields to type g struct? I didn't know how to evaluate the latter. Given we invoke nanotime() today every 8th switch today to track scheduler latencies, is this still a concern?

To add to what Irfan has said, the inability to opt-in to per goroutine CPU tracking is a critical gap in the Go runtime for us. We're building a Serverless database where we charge users per request, and doing that well requires finer granularity of CPU tracking. Using the profiling functionality isn't a good match for this scenario, because we want a more precise measurement, not a statistical sample. Simple prepared queries can take just a few hundred microseconds to execute, which (as I understand it) is below the sampling granularity. In addition, charging usage based on samples is a blunt instrument.
While the details of Irfan's patch may not fit the Go runtime patterns well enough, can we get guidance on what would work better? As Irfan said, the current code calls nanotime every 8th goroutine switch. Even if the Go team is not comfortable with calling nanotime on every switch as in this patch, perhaps there could be an opt-in mechanism that enables apps to set gTrackingPeriod equal to 1 (and makes it a variable instead of a constant), so that only those apps pay the overhead (and are you sure calling nanotime here is actually a performance bottleneck to begin with?).

I also wanted to get the temperature of the Go team around putting some Linux perf events around the scheduler. In particular, a few days ago, this user events patch was merged into the kernel allowing user space programs to register perf events that can be dynamically enabled and disabled from outside the program through the tracefs file system. The program can check whether a particular event is enabled and, if it is, it outputs data to trace by writing to a tracefs file. The events can then be consumed through standard interfaces (perf, BPF, etc).
In theory, runtime scheduler events outputted to perf can be joined with kernel scheduler events, and true on-cpu time can be computed for goroutines.
I don't know the cost of producing the respective events when enabled (the cost if basically zero when disabled).
Irrespective of other decisions for adding Go native APIs for introspection discussed in this proposal, would a patch exploring the perf path be welcomed?

@andreimatei I think supporting something like that would be of great interest to us.
The devil is in the details however. Like, it would be great to just dump the execution trace (runtime/trace) events into a kernel trace and be able to correlate everything together. But how exactly would that be enabled? Maybe a if a GODEBUG setting is present, StartTrace actually starts a kernel trace instead? I'm honestly not sure how well that fits with how user tracepoints are modeled. I'm interested in hearing and/or seeing more details about what you had in mind.

Maybe a if a GODEBUG setting is present, StartTrace actually starts a kernel trace instead?

@mknyszek what I had in mind was to add some type of userspace tracepoints into the scheduler, making available the scheduler events as well as different arguments to tracers such as perf. For the use cases discussed in this issue, I imagine that the profiler labels of the goroutines involved would be good arguments to include in the tracepoints, as they can be used by the go program to attach arbitrary information to its goroutines. Then, one could use standard Linux techniques perf record, perf script, etc to collect the data at runtime, analyze it, and profit. Depending on exactly how we do it, BPF programs could also be attached to these tracepoints (courtesy of the Linux tracing infrastructure), which opens the door to collecting more arbitrary information from the running program.
I would try to not have anything control by a GODEBUG, but instead have the probes always available for a tracer to attach to them.
As you perhaps suggest, demonstrating that an execution trace equivalent to runtime/trace can be generated purely based on these tracepoints, and additionally that it can contain information invisible to the runtime around the kernel thread scheduling, would be very cool.
There are different details to work out. For example, what kind of tracepoints would we use. The new user events type that I mentioned in my previous message seem to have quickly fallen out of favor in the kernel; I think they might end up being removed. But, there are other types. In particular, there's DTrace USDTs, which are supported by the various Linux tools. These also have a feature (USDT semaphores) that makes them practically zero cost when tracing is not enabled. Except USDTs are not natively available to Go programs, I believe, or event to the Go runtime (as you need the compiler and linker to cooperate). There are tricks available out there, but I don't know how suitable they are for the runtime and whether they preserve the zero-cost property. (Btw, I think that supporting USDTs would generally be a major improvement to Go). In theory, for a fixed version of the runtime, we could also use dynamic uprobes to extract the information we want. These don't need any changes to the runtime, but reading the arguments from memory is tedious. However, a simple noinline dummy function that promises a somewhat stable interface and groups the arguments together for the only purpose of them being easily readable by a probe attached to that function could be workable.

I can try re-running a stock workload with profiling enabled to measure what the overhead is, perhaps things have changed since that 2020 comment and/or we didn't measure correctly. I'll do it next week.

Apologies for being slow. The profiling overhead is not as stark as @asubiotto (hi Alfonso!) originally described, so perhaps we did measure things incorrectly.

would it make sense to use profiling labels plus a stream of profile events to accumulate the information you need?

In addition to @andy-kimball's point around wanting to measure at a granularity finer than 100Hz/10ms for microsecond accounting, another reason to not rely on the profile approach is its inaccuracy (profile event proportions for stack traces don't match actual on-CPU proportions) and imprecision (repeated measurements have a lot of variability) that we'd inherit if looking to build control loops on top of it; I'm borrowing the verbiage and experimental results from https://github.com/golang/go/issues/36821. I was also curious to see how measured CPU time would fare with tests in that proposal, so I tried the following two things (available at https://github.com/irfansharif/runner):
// TestEquivalentGoroutines is a variant of the "parallel test" in
// https://github.com/golang/go/issues/36821. It tests whether goroutines that
// (should) spend the same amount of time on-CPU have similar measured on-CPU
// time.
func TestEquivalentGoroutines(t *testing.T) {
	mu := struct {
		sync.Mutex
		nanos map[int]int64
	}{}
	mu.nanos = make(map[int]int64)

	f := func(wg *sync.WaitGroup, id int) {
		defer wg.Done()

		var sum int
		for i := 0; i < 500000000; i++ {
			sum -= i / 2
			sum *= i
			sum /= i/3 + 1
			sum -= i / 4
		}

		nanos := grunningnanos()
		mu.Lock()
		mu.nanos[id] = nanos
		mu.Unlock()
	}

	var wg sync.WaitGroup
	for i := 0; i < 10; i++ {
		i := i // copy loop variable
		wg.Add(1)
		go f(&wg, i)
	}
	wg.Wait()

	mu.Lock()
	defer mu.Unlock()

	total := int64(0)
	for _, nanos := range mu.nanos {
		total += nanos
	}

	minexp, maxexp := float64(0.085), float64(0.115)
	for i, nanos := range mu.nanos {
		got := float64(nanos) / float64(total)

		assert.Greaterf(t, got, minexp,
			"expected proportion > %f, got %f", minexp, got)
		assert.Lessf(t, got, maxexp,
			"expected proportion < %f, got %f", maxexp, got)

		t.Logf("%d's got %0.2f%% of total time", i, got*100)
	}
}

// TestProportionalGoroutines is a variant of the "serial test" in
// https://github.com/golang/go/issues/36821. It tests whether goroutines that
// (should) spend a proportion of time on-CPU have proportionate measured on-CPU
// time.
func TestProportionalGoroutines(t *testing.T) {
	f := func(wg *sync.WaitGroup, v uint64, trip uint64, result *int64) {
		defer wg.Done()

		ret := v
		for i := trip; i > 0; i-- {
			ret += i
			ret = ret ^ (i + 0xcafebabe)
		}

		nanos := grunningnanos()
		atomic.AddInt64(result, nanos)
	}

	results := make([]int64, 10, 10)
	var wg sync.WaitGroup

	for iters := 0; iters < 10000; iters++ {
		for i := uint64(0); i < 10; i++ {
			i := i // copy loop variable
			wg.Add(1)
			go f(&wg, i+1, (i+1)*100000, &results[i])
		}
	}

	wg.Wait()

	total := int64(0)
	for _, result := range results {
		total += result
	}

	initial := float64(results[0]) / float64(total)
	maxdelta := float64(0.5)
	for i, result := range results {
		got := float64(result) / float64(total)
		mult := got / initial
		assert.InDelta(t, float64(i+1), mult, maxdelta)

		t.Logf("%d's got %0.2f%% of total time (%fx)", i, got*100, mult)
	}
}

The results suggest that the measured on-CPU time is both accurate and precise.
        === RUN   TestEquivalentGoroutines
            0's got  9.98% of total time
            1's got  9.53% of total time
            2's got  9.22% of total time
            3's got 10.42% of total time
            4's got  9.84% of total time
            5's got 10.43% of total time
            6's got 10.50% of total time
            7's got 10.21% of total time
            8's got 10.03% of total time
            9's got  9.86% of total time

        === RUN   TestProportionalGoroutines
            0's got  1.87% of total time (1.000000x)
            1's got  3.60% of total time (1.931999x)
            2's got  5.41% of total time (2.899312x)
            3's got  7.21% of total time (3.864451x)
            4's got  9.11% of total time (4.880925x)
            5's got 10.94% of total time (5.864723x)
            6's got 12.77% of total time (6.842004x)
            7's got 14.34% of total time (7.685840x)
            8's got 16.58% of total time (8.885060x)
            9's got 18.18% of total time (9.741030x)

the inability to opt-in to per goroutine CPU tracking is a critical gap in the Go runtime for us.

How do the package maintainers feel about making https://github.com/golang/go/pull/51347 opt-in with zero overhead if opted out? We could for e.g. only track the nanoseconds if a GODEBUG var is set, relying on branch prediction to get it right ~always +  never invoke the additional nanotime() if unset.

The report in #36821 reflects bugs in / shortcomings of Go's CPU profiling on Linux as of early 2020. @irfansharif , have you found that it is still inaccurate on Linux as of Go 1.18, or is it possible that the runtime/pprof sampling profiler could give acceptable results for on-CPU time?
Some of the earlier comments here pointed to map-based goroutine labels adding unacceptable computation overhead. If those were more efficient, would goroutine labels plus the sampling profiler work well enough?
There's also the question from @andy-kimball of granularity, in particular for billing customers for their CPU time. I don't fully understand why sampling isn't an option, even for (repeated) operations that take only a few hundred microseconds each. It seems like the work of repeated operations would show up in over 1000 samples (at 100 Hz) well before it's consumed $0.01 of CPU time (at current cloud prices).
Overall it seems to me that the current tools we have are close, so I'm interested in how we can improve them enough to be useful here.

We would like to get to the point where a user can run an EXPLAIN ANALYZE and get back a cost for that single statement. This should work on a dev/staging cluster (i.e. with little or no other load), and give results that are similar to a production cluster that is fully loaded. Sampling would not work well for that.
Similarly, we'd like to be able to show a list of all recent SQL statements that have been run and show the CPU consumed by each. While some statements may have thousands of iterations and show an accurate number, there's often a "long tail" of other SQL statements that have only a few iterations. Those would often show 0, since we didn't happen to get a "hit". While it's perhaps better than nothing, we're trying to enable a better customer experience than that.

Collecting a timestamp every time the scheduler starts or stops a goroutine, tracking goroutines' running time to sub-microsecond levels, sounds close to key features of the execution tracer (runtime/trace). That data stream also describes when goroutines interact with each other, which allows tracking which parts of the work each goroutine does are for particular inbound requests.
Rather than build up additional ways to see data that's already available via runtime/trace, are there ways to pare down or modify the execution tracer's view to something with lower complexity and overhead? I have guesses (below), but I'm curious for your view on how the execution tracer falls short today for your needs.
Is it:

CPU overhead (from collecting timestamps, from collecting stack traces, writing out the byte stream, or some other part) is too high?
Expensive startup (does work proportional to the number of living goroutines)?
Runtime does not flush partial results in a timely manner?
Inconvenient to parse?

@rhysh I am currently having similar thoughts, and I think a low-enough overhead, user-parseable trace (with some way of inserting markers at the user level) sounds like it could potentially resolve this situation, assuming it's OK to stream this data out and analyze it programmatically out-of-band.
For this to work, though, the trace format itself has to be more scalable and robust, too. On scalability, the traces come out to be pretty large, and they unfortunately basically need to be read fully into memory. On robustness, you really always need a perfect trace to get any useful information out (or at least, the tools back out if it's not perfect). For instance, partial traces don't exist as a concept because there's no enforcement that, say, a block of really old events actually appears early in the trace, forcing you to always deal with the whole trace (from StartTrace to StopTrace).
I'm currently thinking a bit about tracing going forward, and this has been on my mind. No promises of course, but Go's tracing story in general could use some improvements.

What would it take to get this proposal moved into the Active column of the Proposals board?

Anyone interested in tracing, I can't recommend highly enough Dick Sites's new book Understanding Software Dynamics.
For an intro see his Stanford talk.
It seems to me that really good tracing along these lines would help a lot more than scattered per-goroutine CPU stats.

I agree that really good tracing would help, but for the environments we're hoping to use per-goroutine CPU stats, and the contexts we're hoping to use it under (CPU controllers), we're not running with the kinds of kernel patches (https://github.com/dicksites/KUtrace) Dick's tracing techniques seem to be predicated on. I've not evaluated how the more mainstream kernel tracing techniques (ftrace, strace) or use of bpf probes fare performance wise.
I also recently learned about how Go's own GC is able to bound it's CPU use to 25% (https://github.com/golang/proposal/blob/master/design/44167-gc-pacer-redesign.md#a-note-about-cpu-utilization, authored by participants in this thread), which effectively uses similar idea: by capturing the grunning time for time spent in GC assists. This kind of CPU control is similar to what we could do if per-goroutine CPU stats were exposed by the runtime, it would let us hard-cap a certain tenant on a given machine to some fixed % of CPU (with the usual caveats around how accurate this measure is).

The setting where tracing would not help is if you want the program to observe itself and respond, like the pacer does.

Ack. This is precisely the class of problems I'm hoping to push on with per-goroutine CPU stats (or at least just the grunning time).

@irfansharif, have you found that it is still inaccurate on Linux as of Go 1.18, or is it possible that the runtime/pprof sampling profiler could give acceptable results for on-CPU time?

For posterity, running the reproductions from https://github.com/chabbimilind/GoPprofDemo#serial-program:
$ go version
go version devel go1.19-7e33e9e7a3 Fri May 13 14:03:15 2022 -0400 darwin/arm64

$ time go run serial.go && go tool pprof -top serial_prof |  grep 'expect'
      60ms 20.00% 20.00%       70ms 23.33%  main.H_expect_14_546 (inline)
      50ms 16.67% 36.67%       50ms 16.67%  main.G_expect_12_73 (inline)
      50ms 16.67% 53.33%       60ms 20.00%  main.J_expect_18_18 (inline)
      30ms 10.00% 63.33%       30ms 10.00%  main.F_expect_10_91 (inline)
      20ms  6.67% 70.00%       20ms  6.67%  main.C_expect_5_46 (inline)
      20ms  6.67% 76.67%       20ms  6.67%  main.E_expect_9_09 (inline)
      20ms  6.67% 83.33%       20ms  6.67%  main.I_expect_16_36 (inline)
      10ms  3.33%   100%       10ms  3.33%  main.D_expect_7_27 (inline)

________________________________________________________
Executed in  793.74 millis    fish           external
   usr time  473.95 millis    0.13 millis  473.83 millis
   sys time  178.80 millis    3.86 millis  174.94 millis

Still somewhat inaccurate + imprecise (we observe variances across multiple runs) for small total running time (controllable by -m), increasing in accuracy + precision when sampling over larger durations.

CockroachDB's latest release (v22.2) builds with a patched Go runtime, using effectively the diff from https://github.com/golang/go/pull/51347. In the comments above we talked about the "kind of CPU control [..] we could do if per-goroutine CPU stats were exposed by the runtime" and how "it would let us hard-cap a certain tenant on a given machine to some fixed % of CPU". We did this exact thing for bulky, scan-heavy, CPU work in the system (think backups or range feed catch up scans) where we've found them to be disruptive to foreground traffic due to elevated scheduling latencies. https://www.cockroachlabs.com/blog/rubbing-control-theory/ is a wordy write up of the simple idea: dynamically adjust what % of CPU is used for elastic work by measuring at high frequency what the Go scheduling latency p99 is, and then use per-goroutine CPU time to enforce the prescribed %. It's been effective.
@mknyszek: Could we move this further along the proposals board? I believe this issue was also referenced in https://github.com/golang/go/issues/57175#issuecomment-1343129614 ("There's an issue about measure CPU time on-line").

@irfansharif nice article. I like the idea of per-goroutine scheduler stats in general (even if it's a niche use case). A few thoughts.

So scheduling latencies increase non-linearly with CPU utilization [from the article]

Sounds like Little's Law?

and then use per-goroutine CPU time to enforce the prescribed %

Why do you need to enforce a prescribed % instead of just deciding if you want more or less elastic work to happen? Specifically, wouldn't it be enough to have a P(ID) controller that outputs a throttle (sleep) value for elastic work based on the error of (actual scheduler latency - desired scheduler latency)? Sorry if I'm missing something mentioned in the article or that's otherwise obvious.

semi-frequent calls to runtime.Gosched() during elastic work didn’t help [from the article]

Off topic for this issue, but related to your blog post, @irfansharif : You might be interested in https://github.com/golang/go/issues/56060#issuecomment-1270262896 . Gosched appears to yield only to goroutines in the P's local run queue.

RE: https://www.cockroachlabs.com/blog/rubbing-control-theory/, nice! I think this demonstrates an interesting use-case for wanting to ingest metrics produced by the Go runtime on-line.
By any chance have you measured whether there's any additional overhead incurred to tracking per-goroutine CPU times for every goroutine?

Could we move this further along the proposals board?

I don't think we can should move forward with exactly what #51347 does, since runtime/metrics really wasn't built for this per-goroutine use-case. Specifically, it breaks the use-case of just slurping up all the available metrics and having them be meaningful (IMO it would be easy to misinterpret the new metric since it would only apply to the metrics-collection goroutine in most cases, which is typically spending most of its time sleeping).
I think something like this probably needs a new API, though I'm not sure what that looks like.

Why do you need to enforce a prescribed % instead of just deciding if you want more or less elastic work to happen? Specifically, wouldn't it be enough to have a P(ID) controller that outputs a throttle (sleep) value for elastic work based on the error of (actual scheduler latency - desired scheduler latency)? Sorry if I'm missing something mentioned in the article or that's otherwise obvious.

Slightly off-topic, but this is exactly how the Go runtime's background scavenger goroutine works. There's a PI controller that controls sleep time to achieve a desired CPU utilization. It seems to work reasonably well.

I think something like this probably needs a new API, though I'm not sure what that looks like.

Since we have at least two UCs for this at Datadog (EXPLAIN ANALYZE and Resource Isolation) that are not well served by CPU profiling with pprof labels, I decided to give this a shot. It's similar to @asubiotto's original proposal, but with a few differences:

It lives in runtime/debug rather than runtime. I think it's a good place because the package docs say it "contains facilities for programs to debug themselves while they are running". Additionally the package contains SetPanicOnFault which establishes precedent for controlling per-G beahvior.
SetGStats prevents any potential tracking overhead from impacting Go programs that don't make use of it. And even programs using this feature will only have to pay for it on the goroutines they are interested in, rather than all goroutines used by the program. An efficient implementation could try to overload the meaning of an existing g field or find some unused padding to avoid increased memory usage per goroutine.
The "Running" clarifies that this is not actually tracking CPU time, but a close proxy for it. An alternative name would be "Executing" which is the term used by the runtime/trace goroutine analysis.

package debug // runtime/debug

// GStats collect information about a single goroutine.
type GStats struct {
  // Running is the time this goroutine was in running state. This should usually
  // be similar to the CPU time used by the goroutine.
  Running time.Duration
  // could be extended in the future, e.g. with cumulative scheduler latency
  // or time spend in different wait states.
}

// SetGStats enables the tracking of goroutine statistics for the current
// goroutine. It returns the previous setting.
func SetGStats(enabled bool) bool {}

// ReadGStats reads statistics about the calling goroutine into stats.
func ReadGStats(stats *GStats)

I haven't checked with @irfansharif, but I suspect this API would be compatible CRDBs use cases.

I'm happy to wholly adopt any alternative, sticking this into runtime/metrics was done without much forethought on my part. In CRDB we didn't use runtime/metrics (it's a tad costlier), we instead exposed a private func grunningnanos() int64 that we go:linknamed against: See https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20220602_fine_grained_cpu_attribution.md#design and https://github.com/cockroachdb/cockroach/blob/060cdcb6661d1a16d262d37bef5787aa5061f235/pkg/util/grunning/enabled.go#L22-L26.
As for keeping this opt-{in,out}, there too I'll happily defer to folks with better intuition about the performance overhead. Some microbenchmarks against go1.19 on darwin/arm64: https://gist.github.com/irfansharif/256a6864de0d4d2a797d5c815fcd679a. And also since Mac's vDSO-like mechanism ("commpage") is not the same thing as Linux, and Linux is actually what we use on servers, the results looked like below:
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.20GHz
BenchmarkGRunningNanos
BenchmarkGRunningNanos-24       195321096               30.67 ns/op
BenchmarkGRunningNanos-24       195100147               30.77 ns/op
BenchmarkGRunningNanos-24       195415414               30.71 ns/op
BenchmarkGRunningNanos-24       195564742               30.70 ns/op
BenchmarkGRunningNanos-24       195472393               30.70 ns/op
PASS

name                             old time/op    new time/op    delta
PingPongHog-24                      482ns ± 9%     472ns ± 9%    ~     (p=0.065 n=25+25)
CreateGoroutines-24                 306ns ± 4%     308ns ± 4%    ~     (p=0.225 n=25+25)
CreateGoroutinesParallel-24        34.2ns ± 2%    34.2ns ± 3%    ~     (p=0.963 n=25+23)
CreateGoroutinesCapture-24         2.85µs ± 5%    2.82µs ± 4%    ~     (p=0.231 n=24+25)
CreateGoroutinesSingle-24           407ns ± 2%     407ns ± 2%    ~     (p=0.541 n=25+25)
Matmult-24                         0.97ns ± 3%    0.97ns ± 4%    ~     (p=0.613 n=25+25)
ChanNonblocking-24                 0.41ns ± 0%    0.41ns ± 1%    ~     (p=0.055 n=23+24)
ChanUncontended-24                  348ns ± 0%     348ns ± 0%    ~     (p=0.193 n=24+24)
ChanContended-24                   48.0µs ± 2%    46.9µs ± 4%  -2.34%  (p=0.000 n=23+24)
ChanSync-24                         237ns ± 2%     239ns ± 9%    ~     (p=0.645 n=20+24)
ChanSyncWork-24                    33.6µs ± 3%    32.9µs ± 3%  -2.04%  (p=0.000 n=25+25)
ChanProdCons0-24                   1.07µs ± 1%    1.06µs ± 2%  -0.78%  (p=0.006 n=21+25)
ChanProdCons10-24                   847ns ± 2%     842ns ± 2%    ~     (p=0.094 n=25+24)
ChanProdCons100-24                  577ns ± 3%     579ns ± 3%    ~     (p=0.362 n=24+22)
ChanProdConsWork0-24               1.08µs ± 1%    1.08µs ± 2%  +0.56%  (p=0.017 n=24+22)
ChanProdConsWork10-24              1.01µs ± 2%    1.01µs ± 2%    ~     (p=0.732 n=24+23)
ChanProdConsWork100-24              891ns ± 2%     888ns ± 2%    ~     (p=0.418 n=24+25)
ReceiveDataFromClosedChan-24       31.1ns ± 0%    31.1ns ± 0%    ~     (p=0.303 n=25+25)
ChanCreation-24                    61.7ns ± 3%    60.6ns ± 2%  -1.87%  (p=0.000 n=24+24)
ChanSem-24                          439ns ±10%     444ns ± 7%    ~     (p=0.603 n=25+25)
ChanPopular-24                     1.24ms ± 3%    1.24ms ± 3%    ~     (p=0.539 n=25+23)
ChanClosed-24                      0.47ns ± 0%    0.47ns ± 0%  +0.13%  (p=0.006 n=24+24)
SelectUncontended-24               8.63ns ± 0%    8.60ns ± 0%  -0.26%  (p=0.000 n=24+24)
SelectSyncContended-24             3.91µs ± 2%    3.89µs ± 2%  -0.69%  (p=0.019 n=25+25)
SelectAsyncContended-24             705ns ± 3%     702ns ± 4%    ~     (p=0.277 n=25+22)
SelectNonblock-24                  1.44ns ± 0%    1.44ns ± 0%    ~     (p=0.783 n=25+24)
SelectProdCons-24                  1.25µs ± 1%    1.24µs ± 1%  -0.92%  (p=0.000 n=24+22)
GoroutineSelect-24                 1.90ms ± 1%    1.90ms ± 2%    ~     (p=0.775 n=23+25)
WakeupParallelSpinning/0s-24       14.5µs ± 2%    14.5µs ± 3%    ~     (p=0.547 n=25+25)
WakeupParallelSpinning/1µs-24      19.5µs ±10%    19.8µs ± 8%    ~     (p=0.185 n=24+22)
WakeupParallelSpinning/2µs-24      25.5µs ± 4%    25.5µs ± 6%    ~     (p=0.785 n=24+25)
WakeupParallelSpinning/5µs-24      37.2µs ± 7%    37.4µs ± 7%    ~     (p=0.672 n=25+25)
WakeupParallelSpinning/10µs-24     54.3µs ± 1%    54.4µs ± 1%    ~     (p=0.613 n=25+25)
WakeupParallelSpinning/20µs-24     96.0µs ± 1%    96.0µs ± 1%    ~     (p=0.712 n=24+23)
WakeupParallelSpinning/50µs-24      222µs ± 0%     222µs ± 0%    ~     (p=0.510 n=24+23)
WakeupParallelSpinning/100µs-24     399µs ± 3%     396µs ± 3%    ~     (p=0.104 n=25+25)
WakeupParallelSyscall/0s-24         172µs ± 4%     173µs ± 3%    ~     (p=0.410 n=25+25)
WakeupParallelSyscall/1µs-24        172µs ± 2%     173µs ± 3%    ~     (p=0.284 n=23+22)
WakeupParallelSyscall/2µs-24        177µs ± 3%     175µs ± 3%  -1.02%  (p=0.003 n=25+24)
WakeupParallelSyscall/5µs-24        183µs ± 2%     183µs ± 3%    ~     (p=0.845 n=23+23)
WakeupParallelSyscall/10µs-24       196µs ± 9%     194µs ± 4%    ~     (p=0.295 n=24+23)
WakeupParallelSyscall/20µs-24       219µs ± 4%     217µs ± 3%    ~     (p=0.133 n=25+23)
WakeupParallelSyscall/50µs-24       279µs ± 2%     282µs ± 5%    ~     (p=0.109 n=24+23)
WakeupParallelSyscall/100µs-24      394µs ± 4%     391µs ± 3%    ~     (p=0.099 n=25+25)

name                             old alloc/op   new alloc/op   delta
CreateGoroutinesCapture-24           144B ± 0%      144B ± 0%    ~     (all equal)

name                             old allocs/op  new allocs/op  delta
CreateGoroutinesCapture-24           5.00 ± 0%      5.00 ± 0%    ~     (all equal)

I did want to learn from the Go team what would be a better way to evaluate the overhead. We didn't observe anything at the CRDB-level {micro,benchmarks}.

Thanks @irfansharif. I'll try to reproduce the bench results with a bare metal amd64/linux box tomorrow. I'm okay if we decide the SetGStats part of my proposal is not needed. I also don't know what's acceptable for this code path, and how much to weigh linux/darwin/windows/etc against each other in case the overhead is platform dependent.

If we're reasonably certain that it doesn't make a difference on microbenchmarks, I'm not all that concerned about bigger systems.
On the API side, I think it'd be nice to have something less committal than GStats, since we'd have to live with that forever. What about something like:
package metrics

func ReadPerGoroutine([]Sample)
func AllPerGoroutine() []Description

As long as the metrics are scalars, I expect this to be only slightly less efficient than the struct-based API. One thing that might be interesting also is including a dump of this information in the goroutine profile.

@irfansharif I just tried on a big AWS machine with 128 vCPUs and I still see a significant performance regression on PingPongHog. I've also tried with -cpu 24 to use the same number of procs as you did above, but the results are the same. I don't understand why your results are so different. Do you have other hardware to try with?
$ curl http://<IP_ADDRESS>/latest/meta-data/instance-type
c6i.metalubuntu
$ head -n4 before.txt
goos: linux
goarch: amd64
pkg: runtime
cpu: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
$ benchstat before.txt after.txt
name             old time/op  new time/op  delta
PingPongHog-24   610ns ±16%   808ns ±32%  +32.38%  (p=0.000 n=50+50)
PingPongHog-128   603ns ±17%   803ns ±19%  +33.20%  (p=0.000 n=49+50)

@mknyszek how do you feel about making this opt-in if the results above are true? I believe these micro benchmarks will not translate to noticable regression for most Go programs, but I'm still a little worried about the numbers.
The API you proposed sounds nice as well, I'd be okay with that 👍.

One thing that might be interesting also is including a dump of this information in the goroutine profile.

As a new sample type? How would one interpret the resulting profile? For most programs the stack traces in a goroutine profile will be biased towards Off-CPU time, but then the flame graph would be scaled by On-CPU (executing) time? I'd imagine this to lead to confusing results.

Was there any conclusion in the end? What happened to this?


[Suggestion] Merge with pydantic & polyfactory

Hello,
I am impressed with your library @mitchelllisle. It is great complementing other known libraries. Have you ever thought to merge your solution to Pydantic & Polyfactory? Now I'm using these stuff to generate test data in JSON with Polyfactory from Pydantic model and validate data with Pydantic. Your library helps me to validate data in the Spark based on the same pydantic models. Polyfactory has nice feature to generate an object filled with validated data based on Pydantic constraints in a model with one-line class factory. In the other hand your library could generate an DataFrame with passed length. They are complementary features that could work together, but I don't have any idea how to merge it. Could you give me an advice to do it?

Hi there!
Thanks for raising. I think this functionality would be most suitable to live in Pydantic directly, however I don't know if implementing something like this fits in with the goals of that project. We'd have to open up a discussion on whether it would be suitable or not with the Pydantic maintainers. Also the way this library is designed would probably means we couldn't do a direct lift and shift. I'll try to spend some like looking how this might be done.


hugo convert, documentation: add section 'Available commands'

When using the help function of the convert command, a section Available commands is printed out:
$ hugo convert --help
Convert your content (e.g. front matter) to different formats.

See convert's subcommands toJSON, toTOML and toYAML for more information.

Usage:
  hugo convert [command] [flags]
  hugo convert [command]

Available Commands:
  toJSON      Convert front matter to JSON
  toTOML      Convert front matter to TOML
  toYAML      Convert front matter to YAML

Flags:
  -h, --help            help for convert
...

This section is not given on the documentation page for the  convert command. Were nice if this section could be added. Alternatively, we could add links to the three subcommands toJSON, toTOML and toYAML which link to their corresponding documentation site. Not sure if this an option since the documentation page is auto-generated AFAICS.

You can find the subcommands and links to their documentation at the bottom of the page under the "SEE ALSO" section.
Going through the source code of Cobra (see md_docs.go), I was not able to find a way to add the very same "Available Commands" section to the generated MD docs.

You can find the subcommands and links to their documentation at the bottom of the page under the "SEE ALSO" section.

I know.

Going through the source code of Cobra (see md_docs.go), I was not able to find a way to add the very same "Available Commands" section to the generated MD docs.

I submitted a PR in the cobra repo that addresses the issue raised by me. Feel free to give it a like!


Unable to load Medieval Dynasty save

I'm trying to edit a Medieval Dynasty save. There seem to be three missing types. I tried every type, that made sense to me, but could not get it to load. The missing types are .LandscapeData.Value, .Buildings.ToolsBuffer_118_137A92324AB6747A399BA4B732BE0166.Value and .NPCs.PresetsIDs_166_980BA472412B10BA3984DCB92C30E02A.Value.
I'm always getting this error: Error: at offset 6133756: io error: failed to fill whole buffer
Is there something I'm missing?
Here is my save file:
Autosave.zip

ByteProperty has two variants, one with just a single byte value and one with a string label in the case the ByteProperty is represending an enum with named keys. In the case of ByteProperty being used as keys in a MapProperty, it's currently assuming the string variant, but this save has both, and I don't see a way to disambiguously parse such structures. I may need to add a way to manually specify, like how types work currently. This is a patch that allows this particular save to parse:
diff --git a/src/lib.rs b/src/lib.rs
index c89ac32..12b13ed 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1605,7 +1605,17 @@ impl PropertyValue {
                 PropertyValue::SoftObject(read_string(reader)?, read_string(reader)?)
             }
             PropertyType::ObjectProperty => PropertyValue::Object(read_string(reader)?),
-            PropertyType::ByteProperty => PropertyValue::Byte(Byte::Label(read_string(reader)?)),
+            PropertyType::ByteProperty => PropertyValue::Byte({
+                let byte_keys = [
+                    ".NPCs.Dialogue_140_6358380044D8558348534F8C345DE45E.FirstConvers_42_848629774FB793214FB315B79A315190",
+                    ".UsedPresetsPerVillage",
+                ];
+                if byte_keys.contains(&reader.path().as_str()) {
+                    Byte::Byte(reader.read_u8()?)
+                } else {
+                    Byte::Label(read_string(reader)?)
+                }
+            }),
             PropertyType::EnumProperty => PropertyValue::Enum(read_string(reader)?),
             PropertyType::StructProperty => {
                 PropertyValue::Struct(StructValue::read(reader, st.as_ref().unwrap())?)

Awesome, this seems to work. Thank you.


Explain the difference between lbry and lbry fdroid

The Issue
There is no documentation on the changes made to make it fdroid special
Steps to reproduce

Expected behaviour
Tell us what should happen
Actual behaviour
Tell us what happens instead
System Configuration

Device:
Android version:

Anything Else

Screenshots

Internal Use
Acceptance Criteria

Definition of Done

[ ]  Tested against acceptance criteria
[ ] Tested against the assumptions of the user story
[ ] The project builds without errors
[ ] Unit tests are written and passing
[ ] Tests on devices/browsers listed in the issue have passed
[ ] QA performed & issues resolved
[ ] Refactoring completed
[ ] Any configuration or build changes documented
[ ] Documentation updated
[ ] Peer Code Review performed

This repo is just to for the fdroid build process. We'll get this added to the readme.

You haven't added this in the fdroid repository readme

Just run a hash on the executable.  It should be the same as the one lbry uses.

So it is the same thing. Could have said that


Add containerport and prometheus env for monitpring

related issue: https://github.com/pmint93/helm-charts/issues/50

Love that !


Review suggestions

@MikeWasson,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/compliance/pci-dss/index.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/compliance/pci-dss/index.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/compliance/pci-dss/index.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


date filter added on various pages

Fixes #2077
Sample:- https://shubham3211.github.io/google-filter/index.html

This PR broke these tests: https://github.com/fossasia/open-event-wsgen/blob/c89beec0ad9d92528c508b69ccdcc3a1b37707d7/test/generatorAndSchedule.js#L273-L291


Issue in Package include in service page

Describe the bug

No Package Includes Available... should be No Package Available...
save & next button not available
default sort order should be 1
show order in list because if there is more record then we can not identify order
show red star on sort order and show correct message
Description shows 2000 characters limit but accept only 1988 characters

Screenshots
If applicable, add screenshots to help explain your problem.

save & next button not available
show order in list because if there is more record then we can not identify order
validation shows 2000 characters limit but accept only 1988 characters

same comment in #859 also


[WIP] connect travis to saucelabs

Automate testing with saucelabs
https://docs.travis-ci.com/user/sauce-connect/

@natekonimbo is it still WIP because you want to add more tests before integrating?


Table keyboard interaction (draft / partial implementation)

Pull Request
🤨 Rationale
#1137 (HLD)
Storybook build for this branch
👩💻 Implementation
The KeyboardNavigationManager class does the bulk of the work:

Tracks what is currently focused / should be focused (focusState property which contains focusType - there are multiple TableFocusType values like columnHeader/row/cell/cellActionMenu/etc)

Note: If the user clicks in the table away from the current keyboard focus, focusState would end up out-of-date. There's code in handleFocus() to handle for this (see additional comments there)

Listens for table keypresses to handle arrow key navigation, Tab/Shift-Tab presses
If the user unfocuses and refocuses the table, re-focuses the appropriate elements in the table (handleFocus() again)
Tab/Shift-Tab behavior should match the HLD / expectations for ARIA treegrid. (Basically, Tab/Shift-Tab go through the tabbable elements of the header row / current row, not including cells/column headers, until the end is reached, then the table blurs and elements before/after it will be focused with additional Tab presses.)
Not implemented in this version:

Home/End/PageUp/PageDown keys
Ctrl-Enter / F2 to focus elements in a cell (action menu can still be reached via Tab/Shift-Tab)

The overall approach of only setting tabindex=0 on the single element in the table we want focused is called roving tabindex.

@rajsite @jattasNI @atmgrifter00 I think I've now commented on the main areas of this code that I'd like feedback on, so feel free to start looking it over now. If there's any demos/walkthroughs of it that you'd like to see to understand something better (or any discussions that you think would be better as Teams calls) we can do that next week.