Enable "Protected branches"?

Since we have so many collaborators that have push access to this repo, can we enable protected branches for some branches to disable force pushing, prevent branches from being deleted, and require status checks before merging? Also as we need to create pull requests instead of pushing to the main branches directly, etc..., this partially helps to enforce everyone following the governance.

Have we ever broken something by not having this?

I don't think so. I think its a general good practice to enforce this anyway?

I'm not sure this is necessary...we don't have this on https://github.com/nodejs/node do we?

This is relatively new so not many repos are doing it now.

Well, it would go along with our contribution guidelines:

All modifications to the Node.js code and documentation should be performed via GitHub pull requests. Only the Website Admin group can merge their own work and should do so with great care.

Might be worth a shot, I haven't tested this feature in detail yet.

I believe this is necessary for all nodejs projects. @nodejs/tsc @nodejs/collaborators

-1 (At least for for nodejs/node)
It's git, there's nothing to loose. Many of us have full up-to-date copies.
nodejs/node master is purposefully force-pushed to correct meta-data and other problems here and there, though there is a "within 10-minutes" rule. This has been done on many occasions.
Moreover, it should definitely not be asserted as necessary. Force pushing exists for a reason.

Keep in mind that force-pushing is required to update Pull Requests, so people should have some understanding of it.

I believe this is necessary for all nodejs projects.

The only repo we auto-deploy is the website which is some justification for turning on additional restrictions although I don't think it's a good idea to add any potential barriers to contribution to solve problems we haven't actually experienced yet.

@Fishrock123 sorry I didn't see "within 10-minutes" rule before (it's not mentioned in this repo). I would think node would forbid force push at all. If force pushing is required in the workflow then just ignore me :)

Yeah, definitely -1 on this.

It's not just a -1, it's a technical infeasibility. Release branches are regularly rebased. If this were to happen then either 1) these branches would have to live on an individual's repo or 2) a new branch would have to be created with the rebased commits.

Also -1 from me. Disabling force-push is a hinderance in the rare case it is needed.
Here's the info on the 10 min rule.

-1 also. Not a good idea, not necessary with git. background on protected branches


Can't change default Organization Folder

maybe i'm dumb, but i'm unable to change those folders
instruction says

To do this simply open or create ./.vscode/settings.json and fill in the following fields:
{
"unity-tools.defaultOrganizationFolders" : ["Your","New", "Folder","Names"]
}

tried doing that in every way i could think off for last two hours
all i get is "Unknown configuration setting" error.

Huh, you're not dumb. Doesn't work for me now anymore either. I thought maybe something changed? But quick searching seems to say I'm making a setting configuration correctly...
Don't have the time at the moment to dive into it and figure out why. This definitely was working in a previous version, but now I can't seem to pull up app settings and I'm not sure why.

Would love to see a fix for this in the near future,
really handy when starting new projects.

Should be fixed in 0.1.12
@Mixtrate  Can you test that it works now for you after updating?

@TobiahZ yes I can. Did you update it on VScode yet? It's still 0.1.9 on there.

Feeling proud of myself, I went to lunch after submitting. Turns out, I had an unrelated issue with my CI that prevented the new version from automatically being submitted to the marketplace.
Just fixed it. Version 0.1.14 should be live.

So it works but still give "Unknown configuration setting" in the settings. But it works. So I'm assuming that VScode is not recognizing the strings but your scripts are.

Works with no issues in 0.1.15 for me.
Thanks for the hard work today


#[bincode(bound)] for generic types

I'm looking into switching to bincode v2 and I'd love to get rid of serde - the only thing that seems to be lacking in the derive macros is that I can't specify custom bounds for generics. e.g.:
#[derive(Encode, Decode)]
struct A<B: MyTrait> {
    foo: usize,
    b: B,
    thing: B::Assoc,
}

With serde, I had a #[serde(bound(deserialize = "B::Assoc: serde::Deserialize<'de>", serialize = "B::Assoc: serde::Serialize"))]; it'd be awesome to have a similar feature for bincode v2's derive.

Oh, also, this is the generated code for a type with a default type param:
impl<C: Constant = ConstantData> ::bincode::Encode for CodeObject<C>

which is invalid syntax.

This is being implemented as a part of #526. We'll have to start splitting the Decode and BorrowDecode trait, which might take a while.


Adapting Anchors & Angles for Extremely Skewed Dataset

Hey,
My dataset has extremely skewed aspect ratios, so for example the width of an object could be 800px, whilst the height may only be 20px. So the objects are extremely wide and thin, and generally run across from left to right of the image.
I was wondering whether/if I need to adapt the ANCHOR_RATIOS and ANCHOR_ANGLES (in libs/configs/cfgs.py) to my own dataset? Or should I use the default ratios?
Any other recommendations/tips for dealing with such skewed objects?

anchor free based method may be more suitable


Fix CoreRT TODOs for Buffer.CopyBlock and Array.Copy

Remove CoreRT TODOs in Buffer.CopyBlock and Array.Copy
Emit proper EEType flags in CppCodeGen - required for "Hello world" with TODOs removed

@gkhanna79 Could you please take a quick look?

LGTM


JpaConfiguration_Spring4Vaadin

I have 2 maven projects one that contain the backend and all spring jpa and presentation project that contain vaadin4spring with ui and views etc..and all that work fine with embedded tomcat 7 but when i change it to deploy in my tomcat 7  I got the ui but when i use a view that use that has some autowired services from backend i got  "classnotfoundexception" ,after searching for the reason for this error i can say that  this class from Backend is not in the frontend ,

When I try @Import({DomainAndPersistenceConfig}) tomcat wont start

And this my hierachy

Have you verified that your WAR file that you deploy to your Tomcat also contain the backend JAR?

Yes the JAR is there

How do you deploy your app, from within Eclipse or manually by dropping the WAR into the Tomcat webapps directory?

from eclipse

Please try to build the WAR from the command line and deploy it manually. It it works then, the problem is within Eclipse. If it does not, the problem is in the Maven configuration or in the code.

Sorry, but another question if there is a vaadin addons to work with simple spring with xml files (applicationContext.xml etc ...) not for spring boot ?

The official Spring add-on, which is based on Vaadin4Spring, supports simple Spring XML files. You can find it here: https://github.com/vaadin/spring
First draft of tutorial: https://vaadin.com/wiki?p_p_id=36&p_p_lifecycle=0&p_p_state=normal&p_p_mode=view&p_p_col_id=row-1&p_p_col_pos=2&p_p_col_count=4&_36_struts_action=%2Fwiki%2Fview&p_r_p_185834411_nodeName=vaadin.com+wiki&p_r_p_185834411_title=I+-+Getting+started+with+Vaadin+Spring

Closing due to lack of activity


typos: 1.13.10 -> 1.13.12

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/crate-ci/typos/releases.
meta.description for typos is: Source code spell checker
meta.homepage for typos is: https://github.com/crate-ci/typos/
Updates performed

Rust version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.13.12 with grep in /nix/store/x4prrzkx0nay8lq4qfg35qlsnsrhvfck-typos-1.13.12
found 1.13.12 in filename of file in /nix/store/x4prrzkx0nay8lq4qfg35qlsnsrhvfck-typos-1.13.12

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
typos

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/x4prrzkx0nay8lq4qfg35qlsnsrhvfck-typos-1.13.12 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A typos https://github.com/r-ryantm/nixpkgs/archive/0309bd6a4c5d10e7ba1be385806a09689f6e9f14.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/x4prrzkx0nay8lq4qfg35qlsnsrhvfck-typos-1.13.12
ls -la /nix/store/x4prrzkx0nay8lq4qfg35qlsnsrhvfck-typos-1.13.12/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

typos

Maintainer pings
cc @mgttlinger for testing.

Result of nixpkgs-review pr 218398 run on aarch64-darwin 1

1 package built:

typos


Clear/wipe requests with sensitive information

When making a request with sensitive data (e.g. a password or token), I found the request stays in the heap for some time afterward the request is made, until GC'd. If I dump the heap, I can then get the sensitive data out.
While all data is going to inevitably be in cleartext in memory, it would be good to have a mechanism to minimise the time sensitive data is exposed. Rather than waiting for GC, these request objects could be cleared after the request is made.

This is extremely difficult to do well. HTTP/2 header compression remembers headers! Also Okio buffers don’t have any mechanism to wipe to their previous contents; that's usually a useful optimization.
I don't expect we'll ever be able to implement this enhancement. If you need it, I recommend finding another library!

@jack-webb Did you solve your issue? We had a Fortify on Demand audit and they have found the login json in the heap "a reasonable amount of time after such information may have been entered" Is there another network library that can effectively clear the heap right after the request is done?

This is probably a broader question worthy of a discussion on stack overflow. Security APIs in Java have done this with char[] over String, but this isn't something that a library like OkHttp can solve for you.  So it's worth having a discussion with a wider set of participants.

@kaciula I'm afriad not; We're still using OkHttp.
My request also stemmed from an audit. We repeated their findings and decided to let it ride, instead of replacing our network lib, and the work associated with that. Sorry, and good luck finding a solution! It would be interesting to see further discussion on this re: Yuri's comment, for sure.


Create Walls

Extends #17, add walls with hotboxes to create individual rooms within a level. Basic building block for gameplay

 US4: Level Assets


SQL operations based on ('r', 'alpha', 'a', 'b')

I already have the RFM generation streamlined in my database. However I'm interested in taking it to the next level..
Does anyone know how feasible it is to do SQL table updates based on the parameters derived from the BetaGeoFitter? I'm aiming to do daily updates of predictions on customer data in standard SQL operations to avoid hassle with python/container jobs in my ETL.
Specifically I'm referring to the function conditional_expected_number_of_purchases_up_to_time() for the BetaGeoFitter.

That's an interesting question. A lot of the code in conditional_expected_number_of_purchases_up_to_time is there for stability reasons in the general case. If have reasonable parameters and your distribution of data is also reasonable (I am leaving it up to you to define that, but no/few outliers, moderately sized values, etc.), then the code can be simplified quite a but. However, you still need to implement hyp2f1 in SQL, or approximate it somehow. I haven't seen that done.


Frontend Refactor: EmissionsOverTimeCard and ApexLineChart

Context
Refer to Issue #329 for the full context. This story will focus on refactoring the CloudUsage.tsx file and supporting components to follow the presentational and container component pattern for better maintainability and separation of concerns. This may follow and possibly overlap with the extraction that will happen from the EmissionsMetricsPage.
Tasks

[x] Extract component's style hook into a separate .ts named after the component in camelcase (cloudUsageStyles.ts)
[x] Extract the component's rendered HTML markup into a separate Presentational component .tsx file with the component's name (i.e. CloudUsage.tsx) so that it can be rendered as one component by the container
[x] Convert any required data that the presentational component needs to props that are passed from the original component
[x] Rename the original component that now has all of the original logic and state as the component's name followed by the word container (i.e. CloudUsageContainer or shorter)
[x] Extract any necessary subcomponents
[x] Extract or create any new tests as necessary, especially for the presentational component

Tech Tasks

[x] Run Changesets, bump any packages that have been updated/changed
[x] Did you make changes to the client, api or cli package? If so, considering updating the create-app templates.

Tech/Implementation Details
The following article is super helpful for understanding the Container/Presentational component pattern (also known as Smart/Dumb components!)
https://medium.com/@dan_abramov/smart-and-dumb-components-7ca2f9a7c7d0)

Some components may be small and not even worth breaking into two components. In these cases, best judgement can be used to decide if the effort is worth it based on the possibility of that component being expanded in the future.
There may be some repeated styles or reused components that may arise. These may need to be extracted or moved to the src/common directory.

I pushed some small improvements to reduce the number of type errors, with the introduction of the new accountId key in the EstimationResult type, but other than that this look good! I think we can close it off now.


Message.requeue() delay param

https://github.com/bitly/pynsq/blob/master/nsq/message.py#L89
There is information about param "delay" in the method help text . But to set requeue delay to work I should use time_ms param, eg. Message.requeue(time_ms=sec*1000).

fixed in #116


Ukrainian translation added

"UK" is a abbreviation for Ukrainian language according to ISO-639-1. All strings translated

@grsmv Thank you :)


maintainers.expipiplus1: add key

Motivation for this change
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Thanks!
On Fri, 12 Nov 2021, 4:18 pm Fabian Affolter, @.***>
wrote:

Merged #145593 https://github.com/NixOS/nixpkgs/pull/145593 into master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/145593#event-5608094539, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGRJXBGID3EVLSW6AVCYL3ULTETZANCNFSM5H4G3SGA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Run linter on build step

Is your feature request related to a problem? Please describe.
As a developer, I want my work to be linted for style consistency and for quality during the build step, so that it is harder to accidentally merge defect-containing code into master.
Moreover, I want to apply airbnb coding style which helps reduce a larger range of code defects.
Describe the solution you'd like

npm install --save-dev cross-env
Add cross-env EXTEND_ESLINT=true flag to the start, build and test commands so that create-react-app's build steps use our provided .eslintrc file.
Run the eslint-prettier-airbnb-react configuration script to configure prettier and airbnb to work together.
Add npm run lint command to run eslint .
Add npm run lint-fix command to run eslint --fix .
Run npm run lint-fix
Fix remaining linting issues case by case.
Remove extraneous eslintConfig field in package.json since that field is ignored by eslint if the .eslintrc file is present.

Describe alternatives you've considered
Just sticking to prettier rules.
Additional context
See #20 and #21.

I approve

Thanks! I reckon one approval is enough since this is part of the initial things we want to set up.
Fixing the linting issues will affect a lot of files though. It may be slightly painful when others try to merge back.

Would probably be nice to setup the linter for the backend too as part of the test step


[1.4.z] Support FIPS-compatible client certificate generation

Summary
backports #1114
Please check the relevant options

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Dependency update
[ ] Refactoring
[x] Backport
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] This change requires a documentation update
[x] This change requires execution against OCP (use run tests phrase in comment)

Checklist:

[x] Example scenarios has been updated / added
[x] Methods and classes used in PR scenarios are meaningful
[x] Commits are well encapsulated and follow the best practices

run tests

I suppose there is some backport missing in 1.4.z branch that breaks JVM. I'll have to find it and backport it first.


Feat/input

Close #28
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Only
[ ] Refactor

Description, Motivation and Context
New component input added to the NextUI components

Screenshots - Animations

Codecov Report

Merging #42 (5892f02) into main (5f84f79) will decrease coverage by 0.80%.
The diff coverage is 92.30%.

@@            Coverage Diff             @@
##             main      #42      +/-   ##
==========================================
- Coverage   95.78%   94.98%   -0.81%
==========================================
  Files          55       66      +11
  Lines         830     1056     +226
  Branches      266      374     +108
==========================================
+ Hits          795     1003     +208
- Misses         32       46      +14
- Partials        3        7       +4

Impacted Files
Coverage Δ

.../nextui/src/components/input/input-block-label.tsx
73.33% <73.33%> (ø)

...s/nextui/src/components/input/input-icon-clear.tsx
77.27% <77.27%> (ø)

...ages/nextui/src/components/input/input-content.tsx
81.25% <81.25%> (ø)

...ckages/nextui/src/components/input/input-label.tsx
81.25% <81.25%> (ø)

packages/nextui/src/components/input/input.tsx
96.47% <96.47%> (ø)

...ages/nextui/src/components/button/button-group.tsx
100.00% <100.00%> (ø)

packages/nextui/src/components/button/button.tsx
93.05% <100.00%> (+0.19%)
:arrow_up:

...ages/nextui/src/components/image/image-browser.tsx
100.00% <100.00%> (ø)

...ges/nextui/src/components/image/image.skeleton.tsx
100.00% <100.00%> (ø)

...ackages/nextui/src/components/input/clear-icon.tsx
100.00% <100.00%> (ø)

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f84f79...5892f02. Read the comment docs.


Change "codelet" to code snippets

Instead of trying to promote the codelet term just use the damn code snippet, and get rid of aka

maybe change later in code to from codelet to snippet


API name cleanup and adding log support

@jasonsandlin,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


re-export dtype_real from jax.dtypes

This was being exported previously.

Hi - thanks for flagging this.
The change you reference deliberately cleaned up JAX's public API. We chatted a bit about this offline, and I think we'd prefer for this utility to not be part of JAX's public API, as it really has more to do with numpy dtypes than anything special to JAX. If you want a quick way to get this result, you can do something like this:
def dtype_real(dtype):
  return np.finfo(dtype).dtype if np.issubdtype(dtype, np.complexfloating) else dtype

I see.
Indeed it might be something that should be in numpy.
But it isn't.
Most people probably don't use complex numbers, but since jax's support of complex numbers in AD is somewhat... contrived, this is a fundamental function to have.
Looking at jax.dtpes I see the following symbols:
>>> jax.dtypes.
jax.dtypes.bfloat16(            jax.dtypes.finfo(               jax.dtypes.iinfo(               jax.dtypes.result_type(
jax.dtypes.canonicalize_dtype(  jax.dtypes.float0               jax.dtypes.issubdtype(          jax.dtypes.scalar_type_of(

result_type In particular! if I need to compute the result type of an operation that is R->R or C->R, I have the need for dtype_real, otherwise how can I compute the result?
This, for me, is telling that it is essential when computing dtypes.

I'm not certain I understand the relationship to result_type, which is basically JAX's version of np.result_type specialized to JAX's promotion logic.
The dtype_real utility is only used in one place in JAX; unless there are good arguments for keeping it, I'd probably be in favor of removing it altogether.

My argument is that np.result_type allows you to compute the type promotion logic of some operation involving two types.
So if I write a custom jax primitive doing something like a sum +(a,b) = a+b, I can implement the abstract evaluation logic using np.result_type(a, b).
 You can write your own, dtype_real with a series of if,else, but that is ugly.
use cases I have for it are (what we are using it in NetKet):

computing the resulting dtype of statistical objects (variance, autocorrelation time...) which are all real-valued even if the input data is complex-valued.
In some layers with precise mathematical properties, we want some parameters to be real no matter what, while the rest can be complex too. We ask the user to specify the dtype of the layer, which can be any floating-point type, but then need to extract the real_dtype.

In essence, yes, dtype_real is fairly niche in that not a lot of people care about complex numbers.
For people who care about complex numbers, this is an important tool and avoids littering the code with
dtype_real= lambda typ:jnp.ones((), typ)).real or smarter implementations.
Again, I do think that this should be a numpy thing really, but as it's not, I think it's a fairly useful thing to be included in jax.


golang proxy request incorrect

when use proxy to request nginx, nginx can not be use correctly, After I rewrite some code in readRequest method then it can be succeed. The code I have already upload. So you can take a look. Waiting for your reply. Thank you
Now request.go code (from readRequest method):
req.Header = Header(mimeHeader)

	// RFC 7230, section 5.3: Must treat
	//	GET /index.html HTTP/1.1
	//	Host: www.google.com
	// and
	//	GET http://www.google.com/index.html HTTP/1.1
	//	Host: doesntmatter
	// the same. In the second case, any Host line is ignored.
	req.Host = req.URL.Host
	//if req.Host == "" {
	//	req.Host = req.Header.get("Host")
	//}
	if deleteHostHeader {
		delete(req.Header, "Host")
	}

So if I run the following code, ngxin won't report that it can't find "project error"
	remote, err := url.Parse(p.path)
			if err != nil {
				panic(err)
			}
			proxy := httputil.NewSingleHostReverseProxy(remote)
			req.URL.Path = Substring(urlPath.Path, StringLen(p.pattern), -1)

			if req.Method == "OPTIONS" {
				w.Header().Set("Access-Control-Allow-Origin", "*") //
				w.Header().Add("Access-Control-Allow-Headers", "content-type, accept, x-auth-token, X-Subject-Token,x-openstack-nova-api-version")
				w.Header().Add("Access-Control-Allow-Methods", "POST")
				w.Write([]byte("{\"test\":\"OPTIONS\"}"))
				return true
			}
			w.Header().Add("Access-Control-Allow-Origin", "*") //x-openstack-nova-api-version
			w.Header().Add("Access-Control-Allow-Headers", "content-type, accept, x-auth-token, X-Subject-Token,x-openstack-nova-api-version")
			w.Header().Add("Access-Control-Expose-Headers", "X-Subject-Token, x-openstack-nova-api-version,X-Auth-Token")
			w.Header().Add("Access-Control-Allow-Methods", "PUT,POST,GET,DELETE,OPTIONS")
			w.Header().Add("Access-Control-Allow-Credentials", "true")
			proxy.ServeHTTP(w, req)
		}

Looks like dup of #28168. Let me know if that's not correct.


Add message middleware

Required for CRUD operations in admin

Sorry, this PR is broken, wait for anouther one


chore: change oauth2-proxy from k8s-at-home to the official chart

/hold

It might be a smoother transition to first bump the existing k8s-at-home/oauth2-proxy to v5.0.7 as this version used the latest oauth2-proxy version: v7.1.3
Then switch over to the official chart?

I switched locally to the official chart and it works. Plus, I'm of the vain we should use official since it is likely to have the most support and eyes on it. Also, I tried v5.0.7. It didn't work because for some reason it was still trying to use network/v1beta1 which the nginx in the cluster didn't like.

@pokerface3 Your totally right! My bad.
The latest v5.0.7 version of k8s-at-home/oauth2-proxy uses the outdated syntax in its Template: k8s-at-home/oauth2-proxy template

@ankitm123 I can confirm that the setting in this PR will unblock 1.22 users and close Issue https://github.com/jenkins-x/jx/issues/8212
Confirmation steps:

I already had the setup from the JX docs site: OAuth: How to protect Pipeline Visualizer with OAuth
Thus, I removed k8s-at-home repositories and releases helmfile settings but kept pv-values.yaml file

Added the following snippets to helmfiles/jx/helmfile.yaml

repositories:
- name: oauth2-proxy
  url: https://oauth2-proxy.github.io/manifests

AND
releases:
- chart: oauth2-proxy/oauth2-proxy
  version: 6.2.1
  name: oauth2-proxy
  values:
  - ./charts/oauth2-proxy/oauth2-proxy/values.yaml.gotmpl
  - jx-values.yaml
  - extraArgs:
      provider: google
    ingress:
      annotations:
        kubernetes.io/ingress.class: nginx

Creates a helmfiles/jx/charts/oauth2-proxy/oauth2-proxy/ folder with the defaults.yaml, secret-schema.yaml, and values.yaml.gotmpl from this PR.

Ran helmfile --file helmfile.yaml template --validate --include-crds --output-dir-template /tmp/generate/{{.Release.Namespace}}/{{.Release.Name}} to confirm generated Ingress config.

After committing change I confirmed OAuth2 works for Dashboard and Lighthouse Merge Status pages. 🥳

Great work @pokerface3!

I will review it this week. We are planning to migrate the jx internal GCP infra to use oauth instead of password based authentication for things like the dashboard. Once I complete the migration, I can upgrade and test this PR out :)

@pokerface3 can you rebase with the latest master? The release pipeline has been fixed partially.

/hold cancel

Closing, if this pr isn't dead, feel free to comment and I'll reopen!


Api.ai Webhook integration

This package provides integration in apex for the voice recognition and TTS service api.ai. Useful when used with AWS Gateway.

nice! I wonder if we're maybe best keeping it to aws-only stuff in here at least, and linking to others. Not a huge deal really but if someone sends another PR for it I'll have no clue if it's valid or not

Sure! I've sent the PR because of the Slack implementation but I guess I could host this on my Github, no problem.

Ah true haha I happen to know the Slack one, but we should have done the same there

actually whatever, probably won't be a huge deal in practice, we'll see haha

ahah, I was about to click on close but you were faster ;-) anyway, let me know if it's an issue I can do a PR on the README with link to my repo. In any case if any PR on this occurs I will definitely help since this comes from a work-related development


Provide a way to quickly squash auto-commits

Auto-commits allow you to easily view and revert your work, however they can generate a lot of commits. If a user could easily “squash” the auto-commits into a single commit, with a specified message, then they could have all of the local/personal benefits of auto-committing, that can be easily consolidated before pushing to the repo’s remote.

This has now been shipped, and is available via the Collapse Version(s) Above context menu on the Timeline tree: https://github.com/vsls-contrib/gitdoc#collapsing-versions.


mkimage-yum.sh to support dnf

mkimage-yum.sh to support dnf
included changes @tianon proposed in #17362
Signed-off-by: Elvir Kuric<EMAIL_ADDRESS>

ping @tianon @jfrazelle PTAL

LGTM

Looks like this addresses @tianon's comments, so LGTM
thanks @ekuric!


【Don't Merge】upgrade BookKeeper to 4.14.0

Motivation
Pulsar 2.8.0 should dependent on BookKeeper 4.14.0,  otherwise will cause direct memory copy to heap.  Discuss in https://github.com/apache/pulsar/pull/10330
This bug is fixed by https://github.com/apache/bookkeeper/pull/2701
Modification

upgrade BookKeeper to 4.14.0
fix prometheus metric incompatible with bookkeeper 4.14.0, causing by interface changed by https://github.com/apache/bookkeeper/pull/2650

BookKeeper 4.14.0 has prometheus metric bug, which will cause BookKeeper prometheus metric can't be accepted by Prometheus. Please don't merge this PR before it addressed.

BookKeeper 4.14.0 has prometheus metric bug, which will cause BookKeeper prometheus metric can't be accepted by Prometheus. Please don't merge this PR before it addressed.

Which bug?
Can you please link here the bug in BK issue tracker?

BookKeeper 4.14.0 has prometheus metric bug, which will cause BookKeeper prometheus metric can't be accepted by Prometheus. Please don't merge this PR before it addressed.

Which bug?
Can you please link here the bug in BK issue tracker?

The BK bug is https://github.com/apache/bookkeeper/pull/2718
Please help take a look before Pulsar 2.8.0 release. @sijie @jiazhai @merlimat @codelipenghui @eolivelli

@hangc0276 The release 4.14.1 is already available on Maven central. You can update this PR.

@merlimat @eolivelli @codelipenghui  I have upgrade bookkeeper dependency to 4.14.1, PTAL, thanks.


[Proposal] Ecto.Multi enhancements

Ecto 2.0 is still in beta, yet we already got some feedback on the use of the newly introduced features.
This focuses on the feedback we got on Ecto.Multi.
The most requested feature is ability to introspect previeus changes with insert/update/delete to gain access to fields generated in the database, like ids. It's currently possible to do this with the generic run function, but it's not very convenient.
A possible solution is to allow insert/update/delete to accept a function taking changes_so_far as an argument (similar to run) and returning a changeset that is to be processed. This solves the above issue, but limits the visibility into the Multi and hinders the ability to introspect it without actually running the transaction. That is mainly to the fact that funs are opaque and can't be externally analyzed. It's not yet entirely clear for us how important the introspection is in terms of Multi especially with migration to db_connection that allowed concurrent tests hitting the database.
Proposed usage
Multi.new
|> Multi.insert(User.changeset(%User{}, user_params))
|> Multi.insert(&Auth.changeset(assoc(&1.user, :auths), auth_params))
|> Multi.insert(&Session.changeset(assoc(&1.auth, :sessions), session_params))

To decide

Should those new operations be encoded as a special type of insert/update/delete, get a new operation type or be translated to run calls?
It is not yet clear if this will land in 2.0 or will be deferred until 2.1.

:+1 for these proposals.

Should those new operations be encoded as a special type of insert/update/delete, get a new operation type or be translated to run calls?

Both seem fine. Write the code, document them, see which one works better. I am have a hunch the run version will be simpler.

It is not yet clear if this will land in 2.0 or will be deferred until 2.1.

It depends on how fast you are. :)

This would be very neat!
Since people will use Ecto.Multi.run/2 anyway to circumvent that limitation I don't think loosing the introspectability is too bad.


Use smart-open for opening historical files

smart_open is a drop-in replacement for Python's built-in open():
it can
do anything open can (100% compatible, falls back to native open
wherever possible), plus lots of nifty extra stuff on top.

The main bit of "nifty extra stuff" it is being used for here is for transparent decompression of compressed files.
Previously in docs/performance.md smart-open was mention and a monkey patch was provided:
with patch("builtins.open", smart_open.open):
    framework.add_strategy(strategy)
    framework.run()

Anyone who was using this monkey patch will not have their workflow distrupted by this change (as I have essentially hard-coded in what was being patched in).
At the time of writing the latest release version of smart-open is 6.3.0.
https://github.com/RaRe-Technologies/smart_open

Fixes: https://github.com/betcode-org/flumine/issues/688


Fix for distributions with no data and emit only strict JSON

If a Distribution aggregation matched no data, it currently threw an exception (range generation was bad). This was handled and the user saw the no results scenario (records empty) but the metadata contained this error. While the behavior is still acceptable, this fixes that issue and always returns the result from the Sketch (which was one of Double.{NEGATIVE_INFINITY, POSITIVE_INFINITY or NAN}).
We were also forcing Gson to emit these invalid Doubles as Javascript equivalent but the JSON spec does not officially support them. This fixes it to convert these bad doubles to Strings instead.

Coverage decreased (-0.06%) to 99.159% when pulling 1f64eeb33c2716b8519b1ba22b237800b6b2f88d on quantile-points-and-json-fix into c52cdbaa04b05fd90c82e124e7dd34d008e55261 on master.

Coverage increased (+0.001%) to 99.223% when pulling 027f1705cead02fe758fa9d7fa770706ca307b79 on quantile-points-and-json-fix into c52cdbaa04b05fd90c82e124e7dd34d008e55261 on master.

Coverage increased (+0.001%) to 99.223% when pulling c63d49f5bf573297d8438b6a7e7a6b2e97985494 on quantile-points-and-json-fix into c52cdbaa04b05fd90c82e124e7dd34d008e55261 on master.


Docs WG Meeting #2 - 2016-01-20 @ 10:00AM PST

Time for another meeting.
This meeting is scheduled for 2016-01-20 @ 10:00AM PST, and will be a public-viewable Google Hangout on the Air.
If you participated in the Doodle poll, then your Github username should be in this list, and you should have received an invite. If not, and you want to participate, please email me (bryan@bryanenglish.com).
Minutes will be recorded in this Google Doc and then committed back to this repo after the meeting. Please let me know if you want write access to the doc.
Participants

@chrisdickinson
@TheAlphaNerd
@benjamingr
@evanlucas
@qard
@distracteddev
@bengl

Agenda

#50 WG Ratification

Who is invited?

@benjamingr We're inviting all current Docs WG members, as well as any collaborators (or website WG members) that have committed to the docs/ directory on nodejs/node within the last six months.

@nodejs/documentation if you haven't yet put your availability for the meeting into the doodle, please do so. http://doodle.com/poll/8weupkr8x9kaetdd

@nodejs/documentation reminder, this meeting is today at 10AM PST. Please let me know if you don't have the hangout link.

@bengl I don't have the hangouts, my email is benjamingr at gmail dot com

@benjamingr sent

I don't have the hang outs link either :S


License

Hi,
Great work on Bunny, super impressive! Would you mind adding a license for the code & weights?
Thx!

Hi, we have added the Apache-2.0 license for both code and model weights. Thx!

Thanks!


500 {"errorMessage": "Unable to import module 'source'"}

Hi, guys, I took a look at doc,  still not figure it out, this is my Function, BTW, how to install dependency, boto3 and botocore are needed.
apiVersion: extensions.triggermesh.io/v1alpha1
kind: Function
metadata:
  name: ecr-push-deploy-for-gaia
spec:
  runtime: python
  public: false
  entrypoint: lambda_handler
  code: |
    # -*- coding: utf-8 -*-
    import json
    from botocore.vendored import requests

    def lambda_handler(event, context):
        print(event)
        print(context)

        payload = {
            'repo_name':event["detail"]["repository-name"],
            'image_tag':event["detail"]["image-tag"],
            'event':event["detail"]["action-type"]
        }

and I got error 500, logs:
2022/03/14 05:07:36 <-<PHONE_NUMBER>601989221 {"Attributes":{"SentTimestamp":"1647234456512"},"Body":"{"version":"0","id":"57583141-3bad-8e6f-b000-8d06e76b8863","detail-type":"ECR Image Action","source":"aws.ecr","account":"XXX","time":"2022-03-14T05:07:34Z","region":"us-west-2","resources":[],"detail":{"result":"SUCCESS","repository-name":"hpbp/gaia","image-digest":"sha256:00dfb39ecb807e4365e489bd56718eceb3dba837d50e659c897082c55b4417ec","action-type":"PUSH","image-tag":"PR-1058_a189a7d"}}","MD5OfBody":"c7817ede2026b9d8e13e3675c96d63e9","MD5OfMessageAttributes":null,"MessageAttributes":null,"MessageId":"625e7509-4b59-49fd-a727-8d7071647033","ReceiptHandle":"AQEBxubPSxHGe4tDMF6aLbfnDsA7CTdc7BkKrR6pPqOVXDH+/xzNV6A3dCWrrfAmI+uf/IsrIaGffHd/gJVe4i0gj+zRl6NguLwdD3c367KiYJapihS3ANWSkRijNtPiWXSXNaVPInR+F78bSrP+uzNvg7B7wBtFGvZpPnORIP79UmIEa1mpnPIvC0kPEhNOtHXAr9HOi+g7pbFc06H4na/w7lTnKcv5BvguZXwLDmtnk1bthUAb1l+3yM5k6bjLxXIn3Op5we7o8tHxrf4+5o994N3SusuX8fTNsRig+o6gilX3zAT7B1zRO/Qv7juASXufKKUJWW2sulx1gaWvDoy/b22jOTzXOMNDkMXMFqL50ETiwBBMW8S3jufHX674ayu2VAjNYuV3Qtz5R/iyvPizpw=="}
2022/03/14 05:07:36 -><PHONE_NUMBER>601989221 500 {"errorMessage": "Unable to import module 'source'"}
but source.py is just there, I am confused now

@life-learner1 Function resource's main purpose is the event transformation and it does not support the import of external libraries. If the function is responding with the error even without the external dependencies, please post the full code of the function so that we could debug it.
If you want to have more flexibility with your code and use external libraries, check out the KLR project - the runtime of Function CR:
https://github.com/triggermesh/knative-lambda-runtime

Already did, but seems it doesn`t support to use local folder ?  it stuck when I execute this command
tm -n knative-samples deploy service ecr-msg -f ./ \
                              --build-template knative-python37-runtime \
                              --build-argument HANDLER=source.lambda_handler \
                              --wait

@life-learner1 it should work fine with the local folder. The command that you're executing has --wait argument which makes CLI wait for the deployment result and it may look like it is stuck. Try adding -d flag to get the debug output and see what it says. Please note that the function build process requires tekton pipelines installed in the cluster and docker registry auth configured to push the resulting images.


Start vendoring ptx-builder as rust-cuda-builder

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 63.09524% with 93 lines in your changes missing coverage. Please review.

Project coverage is 33.00%. Comparing base (a915a77) to head (f1a08e6).

Files with missing lines
Patch %
Lines

rust-cuda-kernel/src/kernel/link/mod.rs
64.85%
84 Missing :warning:

rust-cuda-kernel/src/kernel/link/error.rs
0.00%
3 Missing :warning:

rust-cuda-kernel/src/kernel/specialise/function.rs
0.00%
3 Missing :warning:

...st-cuda-kernel/src/kernel/specialise/param_type.rs
0.00%
3 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #24      +/-   ##
==========================================
+ Coverage   31.88%   33.00%   +1.11%
==========================================
  Files          54       54
  Lines        6576     6763     +187
==========================================
+ Hits         2097     2232     +135
- Misses       4479     4531      +52

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix compatibility issues with Pydantic 2

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily receiving feedbacks. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Running command
python3 -m lmdeploy.serve.openai.api_server ./workspace <IP_ADDRESS> 8080  --instance_num 32 --tp 1

and start to call the /chat/completions api will show error messages like this:
/output/.pylibs/lib/python3.8/site-packages/lmdeploy/serve/openai/api_server.py:170: PydanticDeprecatedSince20: The `json` method is deprecated; use `model_dump_json` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.3/migration/
  data = chunk.json(exclude_unset=True, ensure_ascii=False)
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 436, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/fastapi/applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/output/.pylibs/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/output/.pylibs/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/routing.py", line 69, in app
    await response(scope, receive, send)
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/responses.py", line 277, in __call__
    await wrap(partial(self.listen_for_disconnect, receive))
  File "/output/.pylibs/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 597, in __aexit__
    raise exceptions[0]
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/responses.py", line 273, in wrap
    await func()
  File "/output/.pylibs/lib/python3.8/site-packages/starlette/responses.py", line 262, in stream_response
    async for chunk in self.body_iterator:
  File "/output/.pylibs/lib/python3.8/site-packages/lmdeploy/serve/openai/api_server.py", line 170, in completion_stream_generator
    data = chunk.json(exclude_unset=True, ensure_ascii=False)
  File "/output/.pylibs/lib/python3.8/site-packages/typing_extensions.py", line 2360, in wrapper
    return arg(*args, **kwargs)
  File "/output/.pylibs/lib/python3.8/site-packages/pydantic/main.py", line 960, in json
    raise TypeError('`dumps_kwargs` keyword arguments are no longer supported.')
TypeError: `dumps_kwargs` keyword arguments are no longer supported.

Modification
Follow the instructions of the warning messages and migrate codes for Pydantic 2.
BC-breaking (Optional)
Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.
Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
The documentation has been modified accordingly, like docstring or example tutorials.

@aisensiy Hi, could you provide the fastapi and pedantic version? I used pydantic==2.3.0 and fastapi==0.103.1 and lmdeploy works fine.

The same, pydantic==2.3.0 and fastapi==0.103.1. Maybe it is because of the python version? I have this issue using python 3.8.

I tried python 3.8 inside the docker. Could not reproduce the error.

Hi, I reproduced the error finally with the streaming case. Thanks for the wonderful pull request.


[NotificationQueue] Add missing none to NotificationCoalescing

https://developer.apple.com/reference/foundation/notificationqueue.notificationcoalescing/1415826-none

I think this API probably goes against our usual conventions of not providing a 'none' option (just pass empty), but we should have this in the name of source compatibility.

@swift-ci test and merge


Update UI code in list.rs

This commit updates the code in ui function to be compatible with the latest version of ratatui.

Hi there - the code you're updating to is actually the older style - it's been updated, but a release has not yet been created that matches the example code. See https://github.com/ratatui-org/ratatui/issues/779 for more info on why this is.
There are a few ways to fix this.

view the version of the examples aligns with the version of Ratatui that you're using (e.g. https://github.com/ratatui-org/ratatui/tree/v0.25.0/examples)
compile the examples with a local version of the source code rather than the released version
you may be able to update to the latest alpha (released weekly on Saturdays) https://crates.io/crates/ratatui/0.26.0-alpha.1 if you need the functionality.


i use the keras to train my model and save the model to pkl format .My question is to show the detailed information about the model_pkl .I don't know how to do .Can you give me some advice please?

Please make sure that the boxes below are checked before you submit your issue. Thank you!

[ ] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

[ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

[ ] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

From FAQ: How can I save a model
Load your model and call model.summary() to see detail information

Thanks. i check the ( how can I save a model) and don't find the SUMMARY function. Then i load my model and call the model.summary(). Like this.

But it show me wrong .


Feature/factual

Loads and sets social media buttons for a specific location via the Factual Crosswalk API. Social media buttons don't show if there is no associated account of that type for a location.

What was the situation here with api keys? All I see on the Factual docs is for OAuth.

As suggested, the only authentication method is Oauth2 client + secret which is unsafe from client-only applications.
Closing this PR, if we decide to populate our backend database with social urls for each of the different social media sites, we can pull the social media directive from this PR and use it with those fields.


Implement DIDAuth

Depends on: https://github.com/spruceid/ssi/pull/105
A DIDAuth command/function is added to:

CLI
C
JNI
Flutter/Dart
Node
WASM
tests
CLI readme

I didn't add it to the HTTP interface because this is not specified in the VC HTTP API or DID Resolution HTTP(S) Binding.
The DIDAuth function takes a holder URI (e.g. a DID), linked data proof options (same as for issuing a credential or presentation), and a JWK, and generates a Verifiable Presentation (which does not contain a verifiableCredential).
Reference: https://w3c-ccg.github.io/vp-request-spec/#did-authentication-request

Great work, thanks for adding cross-platform support.


iTunes metadata is not recognized at all

I guess this issue is due to this commit: https://github.com/l-smash/l-smash/commit/294d62ca7c96209ad251e67fc4363107e26e77cf
At preasent, when meta box is created in isom_read_meta(), default (non null) ilst field is set (copied from isom_meta_box_default).
On the other hand, when ilst field is already set, isom_read_ilst() calls isom_read_unknown_box().
This probably is meant to refuse the second ilst in the meta box, but it means that isom_read_ilst() always branches into isom_read_unknown_box() at present.

Thank you for reporting. I'll check the behavior.

Please check this. https://github.com/l-smash/l-smash/commit/6480a78e833bcf588adc6a982b5073f721a6bd04

Thanks, it works now.


TreeError [SettingsTree] Tree element not found: [object Object]

Error: TreeError [SettingsTree] Tree element not found: [object Object]
at Fe.o (out/vs/workbench/workbench.desktop.main.js:131:31498)
at Fe.expandTo (out/vs/workbench/workbench.desktop.main.js:131:30891)
at Ue.reveal (out/vs/workbench/workbench.desktop.main.js:551:1841)
at <anonymous> (out/vs/workbench/workbench.desktop.main.js:2285:61016)
at <anonymous> (out/vs/workbench/workbench.desktop.main.js:86:4361)
at g.invoke (out/vs/workbench/workbench.desktop.main.js:86:145)
at n.deliver (out/vs/workbench/workbench.desktop.main.js:86:2066)
at h.fire (out/vs/workbench/workbench.desktop.main.js:86:1704)
at $.k (out/vs/workbench/workbench.desktop.main.js:550:47903)
at $.set (out/vs/workbench/workbench.desktop.main.js:550:47811)
at G.setFocus (out/vs/workbench/workbench.desktop.main.js:550:50330)
at G.focusNext (out/vs/workbench/workbench.desktop.main.js:168:9584)
at p.focusNext (out/vs/workbench/workbench.desktop.main.js:551:1476)
at U.q (out/vs/workbench/workbench.desktop.main.js:1945:3182)
at R.Z [as C] (out/vs/workbench/workbench.desktop.main.js:1945:2893)
at R.run (out/vs/workbench/workbench.desktop.main.js:86:14599)
at D.runAction (out/vs/workbench/workbench.desktop.main.js:86:14962)
at D.run (out/vs/workbench/workbench.desktop.main.js:86:14873)
at i.onClick (out/vs/workbench/workbench.desktop.main.js:182:11377)
at HTMLLIElement.<anonymous> (out/vs/workbench/workbench.desktop.main.js:182:10955)

Go to Errors Site

Seems to be fixed.


Update Replicate Extension

Description
Updates @KevinBatdorf's Replicate extension. Now users can:

Run models on Replicate
View prediction data
Explore models / open replicate dashboard

Screencast
https://www.loom.com/share/7afb734f7a814829af8655ecb5470108
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Replicate employee here. 👋🏼
Took this for a spin and it works like a charm. Looks good to me!
cc @pernielsentikaer @mathieudutour

Excited to see that land! 🙏🏼


0602 Tag with reserved word OR/AND should be more lenient

Some tags should not be reported, for example:
[Tags]    Creditor

Maybe we just need to report this case sensitive:
[Tags]    CrediOR  # this should be reported since it will try to match Credi OR (next tag/empty)

False negative - it is actually reporting only on AND/OR, I had different tags in the same line.

What if there's nothing after 'OR' or 'AND'. How it will be parsed by RF? For example CreditOR will be parsed as Credit and OR?


[FUZZ] Crash reproducer

Fuzz testing failed and there are new reproducers available.

This looks promising ...


Docwarden has hard coded dependency on file locations in Java repo.

A recent change to Docwarden:
https://github.com/Azure/azure-sdk-tools/commit/90793333756e3cc91a44aba4414232ba44655279
Introduced a hard-coded dependency on the location of the client SDK parent POM. As part of some restructuring work we want to move the parent into sdk/parents/azure-client-sdk-parent/pom.xml. We could potentially just fix up this path, but I wonder whether it is a good thing to have hard coded paths specified in code in the azure-sdk-tools repo which makes assumptions about the structure of another repo.
Additionally, post parent POM restructuring the parent POM won't contain a module listing, which means the logic being used here wouldn't work anyway.
/cc @chidozieononiwu this was a fairly recent change ... is it easy to back out, and if so what are the consequences?

Am not aware of any other way of determining what packages are client libraries in Java. I can update the path to the parent pom.xml. If there is a better way to determine which libraries are client that will be great.

If you scan for all pom.xml files, any POM file that has a parent of azure-client-sdk-parent is a track 2 library.

Fixed in DocWarden 0.4.2


Documentation versioning

Fixes #3846.
Docusaurus' built in support for versioning seems to do exactly what is needed actually! I have just run yarn run docusaurus docs:version 3.4 and everything seems to work as required. I now need to go through the list of changes and make sure they are documented accordingly. I strike things when I consider they do not need documenting so it is fine to leave the 3.4 and the master docs identical.
New features

[ ] Wikidata support has been generalized to arbitrary Wikibase instances. (#1640)
[x] The cross function now accepts implicit project and column names (#2504)
[x] The left panel can be collapsed (#1038)
[x] Support for more Wikidata constraints was added (multi-value, difference within range, conflicts with, and citation needed constraints) (#2354)
[x] Splitting multi-valued cells is now possible by transition between uppercase/lowercase (#2238)
[x] When importing multiple archive files, importers can store the filename of the archive file each row was extracted from (#1963)
[x] It is now possible to go to a page of the project table directly (#2638)
[x] The pagination sizes offered by the UI can now be configured by setting the ui.browsing.pageSize preference to values such as [100,500,1000,2000] (#2624)
[x] Format detection at the import stage was improved (#2805, #2800)
[x] The split/join multivalued cells dialogs now remember the last separator used (#2197)
[x] The forEach GREL function works on JSON objects (#3149)
[x] A new GREL function wholeText can be used to extract all the text inside an XML element (including in its children) (#3180)
[x] A dialog to confirm the removal of starred expressions was added (#501)
[x] HTTP host validation was added (#3288)
[x] The Wikibase extension can now be used to add BCE dates (#3816)

Incompatible changes

[x] The behavior of the unescape function has changed in CSV mode. Previously it would not remove the first and last quote characters from a string, due to a bug in the supporting library. Now it correctly removes these quotes. unescape("mystring", "csv") would return "mystring" previously, but now will return mystring.

The last main chunk is the documentation of the support for arbitrary Wikibase instances, which has been requested before.


IN-727 Copy test data to project and check jobs complete successfully

A step is added (Copy test data to test project) which, based on a job ID for the assay (CEN or TWE) given in the repository's GitHub Actions variables, copies over the Dias single folder from the 002 project to our 004 test project (because eggd_artemis fails if files are not in the same project where it's run :upside_down_face:)

If this has been copied into the project (i.e. from a previous workflow run or manually) then it just moves the files into the folder for the current GitHub Actions workflow run

Adds step (Check jobs complete successfully) to hold the workflow to check that all jobs which have been launched by the eggd_dias_batch job complete successfully
Modified the YML workflow file so the workflow can be run from another repository using on: workflow_call (i.e. keeping the workflow in this repo and calling it from a repo for a batch config file)

Now takes input of secret variable (i.e. DX_TOKEN) as this can't just be set in the caller repository, and steps to check out code of both repos (so it can access the changed JSON config file in the caller repo, and Python scripts in this repo)

Workflow now adds extra comments to the PR for:

The config file updated in the PR being uploaded to DNAnexus (with a URL link to it)
The diff report being created (and a URL link to download it)
The eggd_dias_batch job being set off in DNAnexus (with a URL link to it)

Adds beginning of a README to list variables and secrets required
Weirdly it thinks I've deleted and made a new run_testing_jobs.py, but only small changes made so that a CNV calling job ID can be given if RUN_CNV_CALLING repository variable is set to False

This change is

Hello @rklocke! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file .github/workflows/copy_test_data.py:

Line 59:1: E302 expected 2 blank lines, found 1
Line 88:80: E501 line too long (117 > 79 characters)
Line 98:80: E501 line too long (82 > 79 characters)
Line 380:32: W605 invalid escape sequence '.'
Line 467:1: E305 expected 2 blank lines after class or function definition, found 1

In the file .github/workflows/run_testing_jobs.py:

Line 95:5: E303 too many blank lines (2)

Manually cancelled workflow run as only README updated


Dependency version is old

[Enter feedback here]
In the documents, old dependency version was wrote as 1.0.8.
However now the version is 2.3.1. So please modify the version?
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>applicationinsights-web</artifactId>
            <version>2.3.1</version>
        </dependency>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e4fe1c72-eef2-fc3a-975a-bcc1c3c50d19
Version Independent ID: e4d3b889-4643-2ffa-6632-73ffbb599677
Content: Azure Application Insights libraries for Java
Content Source: docs-ref-services/appinsights.md
Service: appinsights
Product: azure
Technology: azure
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

@yoshioterada can you please submit a PR associated to this issue?

Dear
@brunoborges san
Thank you so much !!
After confirmed the version, it already modified.
So I will close this issue.


Missing documentation on acceptable field

Description
When calling the API about a device, there's a field called acceptable
To me, it sound like a bool that represents if a device can be accepted.
It turns out it isn't (see #3526) and the documentation needs to be updated
Edition
Community
Version
0.14.1

To me, it sound like a bool that represents if a device can be accepted.

In fact, acceptable is exactly this. When a device cannot be accepted for any reason, its status changes to false. Otherwise, unless it is already accepted, its state is true for acceptable.
Isn't the current behavior?

When I see this variable, I expect it to be True when a device is pending and False when a device is already accepted.
@henrybarreto you explained in #3526 that acceptable is only used in your cloud service for device limits, hence the confusion :)
I suppose a big explanation in the docs would be welcome for this field as, at first glance, it really isn't clear what it represents

When I see this variable, I expect it to be True when a device is pending and False when a device is already accepted (Nothing to do with device limits etc).

You are correct. This should be the right behavior. What I have explained was the “why” of it. It was a misunderstanding from my side.
Are the field working as expected?

You are correct. This should be the right behavior. What I have explained was the “why” of it. It was a misunderstanding from my side.

No worries!

Are the field working as expected?

Sadly no. The status field is correct and it is what I am using for my accept() method on my sdk (see https://github.com/Seluj78/shellhub-python/blob/e9080a3010808ac87a04eef3a5ef5a62b70f0af0/shellhub/models/device.py#L135)
But when I tried to accept a pending device only based on the acceptable field, it was set as False but the device was pending (see https://github.com/Seluj78/shellhub-python/commit/a6631514c52329157eb98eb1d00bffb6b3bb06f7#diff-0df71759cb5f7aa308a761c2561b6edbe3fea6b3211cb747a1e0bc94a165aa0bL109-L110).

Amazing, thank you very much!
Do let me know if there is anything I can do to help

I see you closed this as completed. Can I have an overview of what's changed ? Has the documentation updated ? :)


Interpreter hangs when running parallel function in subprocess with workqueue backend

Reporting a bug

[x] I have tried using the latest released version of Numba (most recent is
visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
[x] I have included a self contained code sample to reproduce the problem.
i.e. it's possible to run as 'python bug.py'.

Description
The Python interpreter hangs when using Numba in the following way:

Select the "forksafe" workqueue threading backend
Run a Numba parallel function
Start a subprocess (e.g. using Python multiprocessing)
Run a Numba parallel function in the subprocess

If I understand correctly, this pattern should be allowed when selecting a forksafe threading layer. Is this correct? Or is one only allowed to run parallel functions in a subprocess, but not mix parent and subprocess work?
Versions
I ran into the issue on Numba 0.55.1 (installed from pip), and then I installed from master (0.56.0dev0+298.g53ea89fee) and found the same issue.  I thought it might be a specific issue to my local machine, but it also fails on GitHub CI in the same way.
I've been testing using Python 3.9 on Linux.  GitHub CI fails on Python 3.7-3.10.
https://github.com/numba/numba/issues/5890 seemed related, so I tried installing the PR from https://github.com/numba/numba/pull/7625, but it has no effect on this problem.
I haven't tried a conda install or a TBB threading layer yet; will try that and report back.
Debugging Attempts
Valgrind gives a pleasantly coherent view of what's happening. It appears the subprocess segfaults inside a workqueue mutex, thus leaving the parent process hanging, waiting for the child to return its result.  This segfault story also meshes with the GitHub CI result from above, where pytest seems to notice that something segfaulted.
Note the PID changes after a few lines, as Valgrind starts tracking the child.
(venv) lgarrison@ccalin008:~/abacusutils$ valgrind --suppressions=valgrind-python.supp python3 repro_gh47.py
==1400719== Memcheck, a memory error detector
==1400719== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==1400719== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info
==1400719== Command: python3 repro_gh47.py
==1400719==
Single-process result: [ 0  2  4  6  8 10 12 14 16 18]
numba.threading_layer()='workqueue'
==1401030== Invalid read of size 4
==1401030==    at 0x58B7D00: pthread_mutex_lock (in /usr/lib64/libpthread-2.17.so)
==1401030==    by 0xE845DA63: queue_condition_lock (workqueue.c:89)
==1401030==    by 0xE845DA63: queue_state_wait (workqueue.c:260)
==1401030==    by 0xE845DA63: ready (workqueue.c:554)
==1401030==    by 0xE845DA63: parallel_for (workqueue.c:401)
==1401030==    by 0x404E19A: ???
==1401030==    by 0x3: ???
==1401030==    by 0x17: ???
==1401030==    by 0xFEFF706F: ???
==1401030==    by 0x8: ???
==1401030==  Address 0x40 is not stack'd, malloc'd or (recently) free'd
==1401030==
==1401030==
==1401030== Process terminating with default action of signal 11 (SIGSEGV)
==1401030==  Access not within mapped region at address 0x40
==1401030==    at 0x58B7D00: pthread_mutex_lock (in /usr/lib64/libpthread-2.17.so)
==1401030==    by 0xE845DA63: queue_condition_lock (workqueue.c:89)
==1401030==    by 0xE845DA63: queue_state_wait (workqueue.c:260)
==1401030==    by 0xE845DA63: ready (workqueue.c:554)
==1401030==    by 0xE845DA63: parallel_for (workqueue.c:401)
==1401030==    by 0x404E19A: ???
==1401030==    by 0x3: ???
==1401030==    by 0x17: ???
==1401030==    by 0xFEFF706F: ???
==1401030==    by 0x8: ???
==1401030==  If you believe this happened as a result of a stack
==1401030==  overflow in your program's main thread (unlikely but
==1401030==  possible), you can try to increase the size of the
==1401030==  main thread stack using the --main-stacksize= flag.
==1401030==  The main thread stack size used in this run was 8388608.
==1401030==
==1401030== HEAP SUMMARY:
==1401030==     in use at exit: 24,269,054 bytes in 24,666 blocks
==1401030==   total heap usage: 639,280 allocs, 614,614 frees, 1,540,075,539 bytes allocated
==1401030==
==1401030== LEAK SUMMARY:
==1401030==    definitely lost: 576 bytes in 12 blocks
==1401030==    indirectly lost: 0 bytes in 0 blocks
==1401030==      possibly lost: 985,934 bytes in 4,064 blocks
==1401030==    still reachable: 23,282,544 bytes in 20,590 blocks
==1401030==                       of which reachable via heuristic:
==1401030==                         multipleinheritance: 6,896 bytes in 8 blocks
==1401030==         suppressed: 0 bytes in 0 blocks
==1401030== Rerun with --leak-check=full to see details of leaked memory
==1401030==
==1401030== For lists of detected and suppressed errors, rerun with: -s
==1401030== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 84136 from 1044)

Reproducer
#!/usr/bin/env python3

import numba
numba.config.THREADING_LAYER = 'workqueue'

import numpy as np

# === Run a Numba parallel function in a single process ===

@numba.njit(parallel=True)
def f(a):
    return 2*a

res = f(np.arange(10))
print(f'Single-process result: {res}', flush=True)
print(f'{numba.threading_layer()=}')

# === Now run a Numba parallel function in a forked process ===

@numba.njit(parallel=True)
def g(a):
    return 3*a

import multiprocessing
with multiprocessing.Pool(1) as p:
    mres = p.map(g, [np.arange(10)])  # hangs

print(f'Forked process result: {mres}', flush=True)

x-ref: https://github.com/abacusorg/abacusutils/issues/47

@lgarrison thank you for reporting this. @stuartarchibald and I took a closer look at this for some time today. We discovered that the initial run of f causes corruption somewhere in the threading layer and so the run of g within multiprocessinghangs. Specifically this is caused by calling fork from a non-main thread. The best way to reproduce this is to swap the threading layer to tbb. In this case you get:
Single-process result: [ 0  2  4  6  8 10 12 14 16 18]
numba.threading_layer()='tbb'
Numba: Attempted to fork from a non-main thread, the TBB library may be in an invalid state in the child process.
Forked process result: [array([ 0,  3,  6,  9, 12, 15, 18, 21, 24, 27])]

Which means that tbb is indeed "smart" enough to detect this, it seems.
So, we came to the conclusion that the first thing to do, is to upgrade the workqueue backend to also detect this situation and then attempt to isolate exactly what the cause is of this fork attempt and fix that.

I think I figured it out!  The thread pool in the subprocess never gets started because at the Python level, the global numba.np.ufunc.parallel._is_initialized is True even after the fork, when in reality threads always need to be restarted after a fork.  I know this is true for pthreads, and I imagine it holds for tbb as well.
A slightly modified reproducer that demonstrates a workaround:
#!/usr/bin/env python3

import multiprocessing

import numba
numba.config.THREADING_LAYER = 'workqueue'

import numpy as np

# === Run a Numba parallel function in a single process ===

@numba.njit(parallel=True)
def f(a):
    return 2*a

res = f(np.arange(10))
print(f'Single-process result: {res}', flush=True)
print(f'{numba.threading_layer()=}')

# === Now run a Numba parallel function in a forked process ===

def h():
    numba.np.ufunc.parallel._is_initialized = False
    a = np.arange(10)
    mres = g(a)
    print(f'Forked process result: {mres}', flush=True)

@numba.njit(parallel=True)
def g(a):
    return 3*a

p = multiprocessing.Process(target=h)
p.start()
p.join()

This is now valgrind clean, and it prints the expected result.
I imagine the proper fix involves registering an after-fork handler.  It looks like os.register_at_fork() is one way to do this.  I'd be happy to PR this if you agree with the diagnosis and fix.

@lgarrison thank you, this looks great! I can confirm that your reproducer works for both the workqueue and tbb backends. IIRC @stuartarchibald also suggested to modify or update the fork-handler during our debug session and will be able to give further advice. Personally, I concur with your diagnosis.

I augmented the MWR as follows (to ensure everything is under __main__ and to add a couple of guaranteed places where a known symbol will be called (using numba.gdb_breakpoint())):
if __name__ == "__main__":

    import numba
    numba.config.THREADING_LAYER = 'workqueue'
    from numba import gdb_breakpoint

    import numpy as np
    print("Calling known gdb_breakpoint location 1")
    gdb_breakpoint()

    # === Run a Numba parallel function in a single process ===
    @numba.njit(parallel=True)
    def f(a):
        return 2*a

    print("Running single-process")
    res = f(np.arange(10))
    print(f'Single-process result: {res}', flush=True)

    # === Now run a Numba parallel function in a forked process ===

    @numba.njit(parallel=True)
    def g(a):
        return 3*a

    import multiprocessing
    print("Running forked-process")
    print("Calling known gdb_breakpoint location 2")
    gdb_breakpoint()
    with multiprocessing.Pool(1) as p:
        mres = p.map(g, [np.arange(10)])  # hangs

    print(f'Forked process result: {mres}', flush=True)

running this in gdb (I've added comments and removed superfluous details):

Details:
$ gdb --args python issue7872.py
<snip>
Reading symbols from python...

(gdb) b main # break on main so we can make absolutely sure that we know the main thread id
Breakpoint 1 at 0x11a430: file <python_binary>/Programs/python.c, line 15.

(gdb) b fork # break on fork
Function "fork" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 2 (fork) pending.

(gdb) commands # When breaking on fork print the thread-id of the caller
Type commands for breakpoint(s) 2, one per line.
End with a line saying just "end".
>p/x (long long)pthread_self()
>end

(gdb) b numba_gdb_breakpoint # break on the special numba internal gdb breakpoint symbol
Function "numba_gdb_breakpoint" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 3 (numba_gdb_breakpoint) pending.

(gdb) r # run
Starting program: <env>/bin/python issue7872.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, main (argc=2, argv=0x7fffffffca28)
    at <python_binary>/Programs/python.c:15
15      <python_binary>/Programs/python.c: No such file or directory.

(gdb) info threads # hit main, main thread is id 0x7ffff7fd9740
  Id   Target Id                                  Frame
* 1    Thread 0x7ffff7fd9740 (LWP 11360) "python" main (argc=2, argv=0x7fffffffca28)
    at <python_binary>/Programs/python.c:15

(gdb) c # continue
Continuing.

Breakpoint 2, 0x00007ffff74c6b90 in fork () from /lib64/libpthread.so.0
$1 = 0x7ffff7fd9740

(gdb) c # hit fork, thread id is 0x7ffff7fd9740 i.e. main thread
Continuing.

Breakpoint 2, 0x00007ffff71ada34 in fork () from /lib64/libc.so.6
$2 = 0x7ffff7fd9740

(gdb) c # hit fork, thread id is 0x7ffff7fd9740 i.e. main thread
Continuing.
[Detaching after fork from child process 11364]

Breakpoint 2, 0x00007ffff74c6b90 in fork () from /lib64/libpthread.so.0
$3 = 0x7ffff7fd9740

(gdb) c # hit fork, thread id is 0x7ffff7fd9740 i.e. main thread
Continuing.

Breakpoint 2, 0x00007ffff71ada34 in fork () from /lib64/libc.so.6
$4 = 0x7ffff7fd9740

(gdb) c # hit fork, thread id is 0x7ffff7fd9740 i.e. main thread
Continuing.
[Detaching after fork from child process 11369]
Calling known gdb_breakpoint location 1

Breakpoint 3, 0x00007fffdd15b830 in numba_gdb_breakpoint ()
   from <numba_path>/numba/_helperlib.cpython-39-x86_64-linux-gnu.so

(gdb) info threads # hit the numba gdb breakpoint, still just one thread, the main thread
  Id   Target Id                                  Frame
* 1    Thread 0x7ffff7fd9740 (LWP 11360) "python" 0x00007fffdd15b830 in numba_gdb_breakpoint ()
   from <numba_path>/numba/_helperlib.cpython-39-x86_64-linux-gnu.so

(gdb) c # continue, the next part shows the workqueue threadpool starting with 4 threads
Continuing.
Running single-process
[New Thread 0x7fffd96e2700 (LWP 11375)]
[New Thread 0x7fffd8ee1700 (LWP 11376)]
[New Thread 0x7fffd86e0700 (LWP 11377)]
[New Thread 0x7fffd7edf700 (LWP 11378)]
Single-process result: [ 0  2  4  6  8 10 12 14 16 18]
Running forked-process
Calling known gdb_breakpoint location 2

Thread 1 "python" hit Breakpoint 3, 0x00007fffdd15b830 in numba_gdb_breakpoint ()
   from <numba_path>/numba/_helperlib.cpython-39-x86_64-linux-gnu.so

(gdb) info threads # hit the numba gdb breakpoint on the main thread, the workqueue threadpool threads are all waiting
  Id   Target Id                                  Frame
* 1    Thread 0x7ffff7fd9740 (LWP 11360) "python" 0x00007fffdd15b830 in numba_gdb_breakpoint ()
   from <numba_path>/numba/_helperlib.cpython-39-x86_64-linux-gnu.so
  2    Thread 0x7fffd96e2700 (LWP 11375) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  3    Thread 0x7fffd8ee1700 (LWP 11376) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  4    Thread 0x7fffd86e0700 (LWP 11377) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  5    Thread 0x7fffd7edf700 (LWP 11378) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0

(gdb) t a all f 1 # look up the stack 1 frame on all threads to confirm that threads 2-5 are all from workqueue

Thread 5 (Thread 0x7fffd7edf700 (LWP 11378) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 4 (Thread 0x7fffd86e0700 (LWP 11377) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 3 (Thread 0x7fffd8ee1700 (LWP 11376) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 2 (Thread 0x7fffd96e2700 (LWP 11375) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 1 (Thread 0x7ffff7fd9740 (LWP 11360) "python"):
#1  0x00007fffd7617010 in _3cdynamic_3e::_gdb_func_injection<abi> ()

(gdb) c # continue
Continuing.

Thread 1 "python" hit Breakpoint 2, 0x00007ffff74c6b90 in fork () from /lib64/libpthread.so.0
$5 = 0x7ffff7fd9740

(gdb) c # hit a fork on main thread, continue
Continuing.

Thread 1 "python" hit Breakpoint 2, 0x00007ffff71ada34 in fork () from /lib64/libc.so.6
$6 = 0x7ffff7fd9740

(gdb) c # hit a fork on main thread, continue, next we see three new threads starting, threads 6-8
Continuing.
[Detaching after fork from child process 11384]
[New Thread 0x7fffd760a700 (LWP 11385)]
[New Thread 0x7fffd6e09700 (LWP 11386)]
[New Thread 0x7fffd6608700 (LWP 11387)]
[Switching to Thread 0x7fffd6e09700 (LWP 11386)]

Thread 7 "python" hit Breakpoint 2, 0x00007ffff74c6b90 in fork () from /lib64/libpthread.so.0
$7 = 0x7fffd6e09700

(gdb) info threads # fork is hit on thread 0x7fffd6e09700, this is thread 7, i.e. a fork on a non-main thread
  Id   Target Id                                  Frame
  1    Thread 0x7ffff7fd9740 (LWP 11360) "python" 0x00007ffff74c3b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
  2    Thread 0x7fffd96e2700 (LWP 11375) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  3    Thread 0x7fffd8ee1700 (LWP 11376) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  4    Thread 0x7fffd86e0700 (LWP 11377) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  5    Thread 0x7fffd7edf700 (LWP 11378) "python" 0x00007ffff74c1a35 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib64/libpthread.so.0
  6    Thread 0x7fffd760a700 (LWP 11385) "python" 0x00007ffff71dbddd in poll () from /lib64/libc.so.6
* 7    Thread 0x7fffd6e09700 (LWP 11386) "python" 0x00007ffff74c6b90 in fork () from /lib64/libpthread.so.0
  8    Thread 0x7fffd6608700 (LWP 11387) "python" 0x00007ffff74c475d in read () from /lib64/libpthread.so.0

(gdb) t a all f 1 # look at frame 1 for all threads, thread 1 (main) is waiting/locked, 2-5 are from workqueue and are waiting, 6 is polling, 7 is forking, 8 is read()ing

Thread 8 (Thread 0x7fffd6608700 (LWP 11387) "python"):
#1  0x000055555578c896 in read () at <sysroot_path>/sysroot/usr/include/bits/unistd.h:44
44      <sysroot_path>/sysroot/usr/include/bits/unistd.h: No such file or directory.

Thread 7 (Thread 0x7fffd6e09700 (LWP 11386) "python"):
#1  0x00007ffff7e0f623 in subprocess_fork_exec (self=<optimized out>, args=<optimized out>) at <py_src_path>/Modules/_posixsubprocess.c:844
844     <py_src_path>/Modules/_posixsubprocess.c: No such file or directory.

Thread 6 (Thread 0x7fffd760a700 (LWP 11385) "python"):
#1  0x00007ffff7e05a9b in select_poll_poll_impl (timeout_obj=<optimized out>, self=0x7fffd76aced0) at <py_src_path>/Modules/selectmodule.c:652
652     <py_src_path>/Modules/selectmodule.c: No such file or directory.

Thread 5 (Thread 0x7fffd7edf700 (LWP 11378) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 4 (Thread 0x7fffd86e0700 (LWP 11377) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 3 (Thread 0x7fffd8ee1700 (LWP 11376) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 2 (Thread 0x7fffd96e2700 (LWP 11375) "python"):
#1  0x00007fffd96e99d4 in queue_state_wait () from <numba_path>/numba/np/ufunc/workqueue.cpython-39-x86_64-linux-gnu.so

Thread 1 (Thread 0x7ffff7fd9740 (LWP 11360) "python"):
#1  0x00007ffff74c3bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0

it's evident that the fork() is coming from the non-main thread. These 3 new threads associated with multiprocessing.Pool are likely from here:
https://github.com/python/cpython/blob/6a14330318c9c7aedf3e9841c3dfea337064d8e6/Lib/multiprocessing/pool.py#L223-L250
the Pool._handle_workers eventually ending up in a call to https://github.com/python/cpython/blob/6a14330318c9c7aedf3e9841c3dfea337064d8e6/Lib/multiprocessing/pool.py#L319 which is what creates the fork(2) call by default on linux systems.
The issue with this is that whilst workqueue is "fork safe" it is evidently so only from the thread that loaded/launched the library as pthread_atfork handlers are installed only in the calling thread.

Ah, I see, so there may be two issues: one is that Pool calls a fork from a non-main thread, which tbb (at least) doesn't like, and the other is that the workqueue threadpool is never reinitialized after a fork because numba.np.ufunc.parallel._is_initialized is not reset.  Is that your understanding?
Is calling fork from a non-main thread always an illegal thing, or just something tbb doesn't like?  My understanding is that this is generally allowed in pthreads, but that there is still plenty that could go wrong.

Ah, I see, so there may be two issues: one is that Pool calls a fork from a non-main thread, which tbb (at least) doesn't like, and the other is that the workqueue threadpool is never reinitialized after a fork because numba.np.ufunc.parallel._is_initialized is not reset. Is that your understanding?

Mostly, yes. The non-main thread issue is going to be a problem for TBB, it shouldn't be a problem for workqueue.
The workqueue threadpool reinitialisation is seemingly now having the problems noted above under fork calls (it's thankfully easy to replicate with a simple os.fork() call). I don't recall all the details as it's from a few years ago, but I think it used to work correctly by virtue of the thread pool launch being non-lazy. The pthread_atfork() handlers registered in the launch_threads() call would reset the state in the child processes on fork, and then the next parallel function launch would reinitialise the pools in the child processes as they'd see the reset state. This is threadpool launch is now both delayed until needed and only done once such that the state is reset in a child process but the thread pool is not launched, hence the problem with the global mutex being invalid.
Setting the _is_initialized to False with os.register_at_fork(after_in_child=some_reset_function) would have an effect similar to the original non-lazy implementation and I think this is why your examples using that technique seem to work.
I think there is a self contained workqueue library level fix available, but it would mean that child processes automatically have a new thread pool launched. This may or may not be desirable. I've a patch to this effect that I'm testing and I'll raise this at the public development meeting next week and see if anyone has concerns about this approach.

Is calling fork from a non-main thread always an illegal thing, or just something tbb doesn't like? My understanding is that this is generally allowed in pthreads, but that there is still plenty that could go wrong.

As I understand it it's "legal" to call fork from any thread, it just can be challenging to handle.

Thanks, that makes sense—I can see the appeal of keeping the fork handling logic internal to the workqueue library.  On the other hand, I can imagine lots of Numba-unrelated libraries may cause a process to fork, and will thus incur some small overhead if the threads are always started automatically.  Platforms with 128 cores are not uncommon in HPC these days, and starting 128 threads on every fork starts to sound not so lightweight.  I don't think it will be a huge issue either way, though; I'll try to drop by the development meeting, I'm interested to hear people's thoughts!

@lgarrison following on from the discussion at the development meeting... first, apologies for the delay on this! Second, since that meeting we've done some more investigations and think there's potentially also a fix available for the "calling fork(2) from non-main-thread" problem. Hope to get some potential fixes into PRs next week.

Great! Happy to help/test however I can.

I think https://github.com/numba/numba/pull/7625 ought to go in first else the conflicts both in terms of moving the same areas of the code base and in handling the complexity of the change will end up prohibitively hard. Will hopefully finish reviewing that next week and then propose a patch on top of that.

I think I've got a patch for this, it's written on top of https://github.com/numba/numba/pull/7625 with view of getting that merged shortly.

@lgarrison we finally got #7625 merged, many thanks for your patience whilst it was reviewed (it was a long and complicated review!)! I've now opened https://github.com/numba/numba/pull/8195 to hopefully fix this issue. If you would like to test/have thoughts on #8195, feedback is very welcome, as discussed at the public meeting a while back. Thanks again!


The destination of Basic Agent

Hi,
I'm wondering what is the actual destination of a basic agent and how to set a destination.
In the automatic_control.py, the destination is set to Location(x=227.31, y=-5.08551, z=1.8431),  but the vehicle will always stop at Location(x=-77.8, y=24.2).
Is this a bug? Or is there any conversion between two locations?
(Using Carla 0.9.3, latest code from GitHub built from source, running on Ubuntu.)

As far as i know when using the basic agent, you set the destination to any location. The agent will then find the nearest possible waypoint and set it as the true destination. Furthermore there is a factor of 10 between the worldcoordinates in the UE4Editor and in the PythonAPI locations (but that won't explain your coordinates).
Maybe the vehicle will stop by the end of the RoutePlanner spline? And thats why the coordinates are so far apart.

@slyer0009 Thanks for your response.
But, the destination in automatic_control.py is from the get_spawn_points(), which supposed to be a valid waypoint, right? So, I guess the vehicle should be navigated to that specific position.

@slyer0009 Thanks for your response.
But, the destination in automatic_control.py is from the get_spawn_points(), which supposed to be a valid waypoint, right? So, I guess the vehicle should be navigated to that specific position.

Have you solved yet? Ive found the same question. Im thinking if there is a bug in the program.

I think in their latest release 0.9.5, this problem is solved.


Feature/212 add direct ndvi calculation

Develop branch fix for #212, also added in pipeline PR #238.
May need some minor tweaks on the analysis side but I can fix those once we have some data downloaded, and I have checked the NDVI scaling (#244).

Add saving of direct NDVI mean, std to the non-pipeline code (it is in the pipeline code also)
Modify scale_tif to use a global NDVI transform (that is not dependent on the NDVI values found in the current image) - see #244
Fix a couple of imports for the non-pipeline code
Add function to plot NDVI time series to analysis code

just waiting for #261 to merge


Should disable resource setup for external transports when the external transports are disabled

Hit a user. Easy to understand, quick fix.

I think the solution to this would be to add a check to all implementations of IStatefulResource.Setup?
Or only the transport ones?


As-built Report hangs with no errors.

Bug description
Process stuck for me at the AsBuiltReport.Microsoft.AD 0.8.0 is currently installed line

Command-line input
$Creds = Get-Credential
New-ASBuiltReport -Report Microsoft.AD -Target server01.contoso.com -Credential $Creds -Format Html,Text -OutputFolderPath 'C:\Temp' -EnableHealthCheck -AsBuiltConfigFilePath "C:\Users\test\AsBuiltReport\AsBuiltReport.json" -ReportConfigFilePath "C:\Users\test\AsBuiltReport\AsBuiltReport.Microsoft.AD.json"
Steps to reproduce
I've tried running this same command line from both AD server and Windows 10 machine.
Expected behaviour
Report to be completed.
Screenshots
No response
Operating System
Tried from Windows 10 and Server 2016
PowerShell Version
5.1.14393.6343
PowerShell Modules
AsBuiltReport.Core              1.4.0
AsbuiltReport.Microsoft.AD 0.8.0
PScribo                                 0.10.0.
PScriboCharts                      0.9.0
PSPKI                                   4.2.0
Additional Context
This used to work for me in the past but now it just seems to hang or never finish.  Please let me know if I can do anything to remediate from my end.
Before submitting

[X] I have read and followed the bug reporting guidelines.
[X] I have read the documentation, and referred to the known issues before submitting this bug report.
[X] I have checked for previously opened & closed issues before submitting this bug report.

Can you run the report with the Verbose option and see if any errors message  is displayed?

I started it with just -Verbose after the initial command line.  Let me try again changing the global verbose parameters as well.  Do I also include the -Verbose switch?

I started it with just -Verbose after the initial command line. Let me try again changing the global verbose parameters as well. Do I also include the -Verbose switch?

Yes.

That section looks for missing subnet in the netlogon.log file of each DC maybe a slow bandwidth is slowing the connection.
Anyway 14 minutes is a long time I am not sure why the cmdlet is not getting a timeout event.
if (Test-Connection -ComputerName $DC -Quiet -Count 2) {
    try {
        $DCPssSession = New-PSSession $DC -Credential $Credential -Authentication $Options.PSDefaultAuthentication -Name 'MissingSubnetinAD'
        $Path = "\\$DC\admin`$\debug\netlogon.log"
        if ((Invoke-Command -Session $DCPssSession { Test-Path -Path $using:path }) -and (Invoke-Command -Session $DCPssSession { (Get-Content -Path $using:path | Measure-Object -Line).lines -gt 0 })) {
            Write-PScriboMessage "Collecting Missing Subnet in AD information from $($Domain)."
            $NetLogonContents = Invoke-Command -Session $DCPssSession { (Get-Content -Path $using:Path)[-200..-1] }
            foreach ($Line in $NetLogonContents) {
                if ($Line -match "NO_CLIENT_SITE") {
                    $inObj = [ordered] @{
                        'DC' = $DC
                        'IP' = $Line.Split(":")[4].trim(" ").Split(" ")[1]
                    }

                    $OutObj += [pscustomobject]$inobj
                }

                if ($HealthCheck.Site.BestPractice) {
                    $OutObj | Set-Style -Style Warning -Property 'IP'
                }
            }
        } else {
            Write-PScriboMessage "Unable to read $Path on $DC"
        }
        if ($DCPssSession) {
            Remove-PSSession -Session $DCPssSession
        }
    } catch {
        Write-PScriboMessage -IsWarning "Missing Subnet in AD Item Section: $($_.Exception.Message)"
    }
}

Can you add the that DC to the Exclude.DCs	option so we can rule out if it is a DC connection problem?

Sorry forgot to add the image:
The Exclude.DCs is located at the AsBuiltReport.Microsoft.AD.json
You can generate the file using this docs link:
https://github.com/AsBuiltReport/AsBuiltReport.Microsoft.AD?tab=readme-ov-file#pencil2-configuration

Can you add the that DC to the Exclude.DCs option so we can rule out if it is a DC connection problem?

I executed repadmin /showrepl and found a problem replicating to a DR.

Sorry forgot to add the image:
The Exclude.DCs is located at the AsBuiltReport.Microsoft.AD.json
You can generate the file using this docs link: https://github.com/AsBuiltReport/AsBuiltReport.Microsoft.AD?tab=readme-ov-file#pencil2-configuration

I had gotten, Invalid JSON primitive: dc01.contoso.com (I think I was missing the quotes on the json file around the server name)

Yes, the quotes are a requirement in json

Yes, the quotes are a requirement in json

Thank you, it's further along.  Update to follow:

Now it seems to be stuck in this process: (about 15 minutes now)

If still stuck in that section you can try to disable the $HealthCheck.Domain.DFS in the AsBuiltReport.Microsoft.AD.json file.
Set the value to false to disable it and rerun the report.

However, I think there must be a connection problem in your environment.

Thank you for your help,  I will attempt this workaround step.  Is there a way to set timeouts to continue throug the reporting process?

Currently there is no way to set a timeout in the report.
Powershell have the capability to set a timeout for a job but coding it for a multi-Domain/DC is very difficult. Also there is no right way to define a timeout for a cmdlet that is called remotely in a distributed Active Directory environment.
There are also many thing to take into consideration like bandwidth, server resources and implemented security mechanism in the environment that can prevent the report to run successfully.
Maybe in a future release a per job timeout can be implemented.

Currently there is no way to set a timeout in the report.
Powershell have the capability to set a timeout for a job but coding it for a multi-Domain/DC is very difficult. Also there is no right way to define a timeout for a cmdlet that is called remotely in a distributed Active Directory environment.
There are also many thing to take into consideration like bandwidth, server resources and implemented security mechanism in the environment that can prevent the report to run successfully.
Maybe in a future release a per job timeout can be implemented.

It looks like after resolving the memory allocation issues with one of the DC and replication flowed across the environment the reports is now flowing right through.  Thanks again for all your time and help!

If you manage to generate the report. Could you evaluate the content and give me recommendations?
I am open to new ideas or sections that may be needed.
Best regards,


First draft specification to store geometries in Arrow data as nested lists

See the discussion in https://github.com/geopandas/geo-arrow-spec/issues/4
This starts with describing a core subset of what has been discussed in https://github.com/geopandas/geo-arrow-spec/issues/4, i.e. the memory layout based on Arrow's ListArrays (with a variable number of nesting levels, depending on the geometry type), with some examples to illustrate this layout.
There are still several open questions (eg I didn't include a description of how unions can be used for GeometryCollections or mixed geometry types), but wanted to first get the core subset done.

Really excited to see this moving forward! One additional question: should the spec enforce winding order of polygons?

@paleolimbot thanks a lot for all the prototyping!
You raise several good points and questions. Regarding extension type naming and metadata, I will open a separate issue to discuss that, to focus here on the actual type / memory layout. Reacting to some other aspects here:

It defines the entire point/linestring/polygon/multipoint/multilinestring/multipolygon stack. Seemed like a good idea at the time and wasn't too much harder to write.

It's indeed probably fine to also define linestring and polygon. As you already mentioned, my initial reason to limit it to point/multipoint/multilinestring/multipolygon was that this makes each of them use a unique type (i.e. different nesting depth). But being able to still distinguish them based on either the naming of the levels or the general name (in the extension name, or in schema-level metadata) is probably good enough.

it's about twice as fast to read points as a struct from a Parquet file [compared to fixed-size list] and the file is slightly smaller (with default snappy compression)

Yeah, for Parquet it is true that using fixed-size list instead of struct is going to be less optimal. I think this is also related to the fact that Parquet doesn't really have a concept of fixed-size lists. Whether you have a variable or fixed size list in Arrow, (I assume) they are stored the same way in Parquet using the definition/repetition levels. Now given that for a fixed size list, the repetition levels are going to contain a lot of repeated values, and those data is also encoded/compressed (eg with run-length encoding), I think for Parquet itself it might not be too bad.

One additional question: should the spec enforce winding order of polygons?

It would be interesting to check how other formats deal with this, but I think agreed with @paleolimbot that if you rely on this, you will probably have to check this anyway (even if we specify it the spec, it's probably difficult to enforce that this is actually done)

I am going to merge this to at least have a first version of the format description in the main repo, we can further iterate on that in follow-up issues / PRs.
So also @thomcom further feedback from the cuspatial side is certainly welcome!