Run om prior

Description

[x] Get the calculation of emission for all layers running again
[x] Add a test that checks the full process is running
[x] Speed up the process to calculate emissions for Agriculture, LULUCF and waste layer.

Checklist
Please confirm that this pull request has done the following:

[x] Tests added
[ ] Documentation added (where applicable)

Notes
@prayner When we check the data with omPriorVerify.py, we get the following output. Does that still work?
PASSED - Livestock CH4 within bounds of total agriculture CH4: 0.16963856
FAILED - Discrepency of -51.030272990142386% in agriculture emissions
FAILED - Discrepency of -65.28372899999995% in electricity emissions
FAILED - Discrepency of -89.48823911165206% in fugitive emissions
FAILED - Discrepency of -73.59460500000147% in industrial emissions
FAILED - Discrepency of -77.15502137254902% in lulucf emissions
FAILED - Discrepency of -73.59460972972967% in stationary emissions
FAILED - Discrepency of -73.59461375000018% in transport emissions
FAILED - Discrepency of -79.06976740181267% in waste emissions

I tried to send links to 10km versions of these files yesterday,
clearly that didn't work. You can find these at
https://www.dropbox.com/scl/fi/h8quykmt9u16ggmmv8zjr/GRIDDOT2D_2?rlkey=sr5ka3rjp8y3vkew8r2lln6zb&dl=0
https://www.dropbox.com/scl/fi/hyd7tdk3ows3asytzxl4y/GRIDCRO2D_2?rlkey=jn2kj9lxtkycsfeat1atx1m4w&dl=0
The geo_em file is produced by the wps process before running WRF, the
GRID* files are produced in the MCIP process run as part of the
setupCMAQinputs script and is generated from the wrfout files. They're
pretty similar except that the GRID files have some edge gridcells
removed since the meteorology here is less reliable. I'd be very
surprised if the code will run with different resolutions.

Thanks for sending the files @prayner. They are now in the repository.
The tests in omPriorVerify.py now pass.

Great! We should refactor the verify script into something that can be tested in a future MR

I'm wary of including the GRID* files in the repo since they're not
really input files nor are they part of the prior process itself but
are used to create om-domain-info.nc. Can we perhaps include
om-domain-info.nc itself?

In my normal workflow om-domain-info is produced from the GRID* files
then the GRID* files are not used again. If omCreatDomainInfo runs
successfully we should (a) have consistent grids and (b) not need the
GRID* files again. If the prior is only being developed for Open
Methane then none of this matters much.
Peter Rayner (he/him), Honorary Professorial Fellow, University of Melbourne
mobile +61 402 752 379
zoom id<PHONE_NUMBER>, join at  https://unimelb.zoom.us/j/4431343191?pwd=a1E5Z3JEOTRVQUJsaVdRbVUvR1QyZz09
mail-to: @.***
google scholar: https://scholar.google.com.au/citations?user=H3up71wAAAAJ&hl=en
I acknowledge the Traditional Custodians of the land on which I work, the Wurundjeri people of the Kulin nation, and pay my respect to their Elders, past and present
I am sending this email when convenient for me, please only respond when convenient for you


Site Built Successfully but Quickly Crashed afterwards

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/HugoBlox/hugo-blox-builder/issues?q=is%3Aissue
[X] This issue is not a question, feature request, or anything other than a bug report directly related to Hugo Blox Builder. Please post those things on Discord: https://discord.gg/z8wNYzb

Description
I follow the guidelines to deploy my site to the GitHub page (it works well locally)
However, once the page was published, the top toolbar was not responsible, and the whole page crashed.
For instance, I want to check one of the publications under the home page: publication/, the browser raised a 404 error, and the full site crashed. Only the readme was shown on github.io afterward (as shown in screenshots).

After navigating through the site

The website ends up with

Reproduction Link
https://github.com/petal-cs-pitt/petal-cs-pitt.github.io
Steps to Reproduce

locally hugo server (works)
On GitHub, I run the Action

...
Expected Result
The website should be run on http://petal-cs-pitt.github.io
Actual Result
The website loaded initially, but the entire site was down afterward. The website loaded in the beginning, but the full site was down afterward.
What Hugo Module versions does your site use?
module github.com/wowchemy/starter-hugo-research-group
go 1.15
require (
github.com/Agos95/wowchemy-shortcode-collection/shortcodes/gallery-with-caption v0.0.0-20220820161609-3537320c2f23 // indirect
github.com/rodrigoalcarazdelaosa/wowchemy-widget-hero-lottie v0.0.0-20210723222755-5607ebac6b4c // indirect
github.com/wowchemy/wowchemy-hugo-themes/modules/wowchemy-plugin-netlify v1.0.0 // indirect
github.com/wowchemy/wowchemy-hugo-themes/modules/wowchemy-plugin-netlify-cms v1.0.0 // indirect
github.com/wowchemy/wowchemy-hugo-themes/modules/wowchemy/v5 v5.7.1-0.20230418194707-ca2775cd57a2 // indirect
)
What operating system(s) are you seeing the problem on?
Linux
What browser(s) are you seeing the problem on?
Chrome
Which Hugo Blox template are you using?
acedemic-group (but using the older version)
What version of Hugo are you using?
0.97.3

This issue is unrelated to the Hugo Blox Research Group theme.
For GitHub Pages deployment issues, reach out to GitHub Support: https://github.com/community
For Hugo crash issues, reach out to the Hugo team: https://discourse.gohugo.io/


Fixed bug with Android < 23

On Android < 6.0 , trying to record a video from the camera using react-native-camera will result in an error message java.io.FileNotFoundException·/storage/emulated/0/...: open failed: EACCES (Permission denied)
This article details the steps that need to be done in need to ensure the permissions are granted correctly.
###Rationale
Asking for WRITE_EXTERNAL_STORAGE will automatically grant READ_EXTERNAL_STORAGE but the opposite isn't true. Most developers need write access to storage when requesting it using react-native-permission so this makes more sense.

LGTM, thanks :)


Investigate different method of template replacement instead of $-prefixed

Description of Problem & Solution
Right now using templates in bash requires escaping the dollar signs.  Investigate using a style similar to Python instead to help with this, e.g.:
scenedetect [...] split-video -f {VIDEO_NAME}-Scene-{SCENE_NUMBER}.mp4

Proposed Implementation:
Allow both dollar signed template $VARIABLES as well as braced {VARIABLES} to mean the same thing.  However since {} may need to be escaped (see below), another solution might need to be investigated.
If any best practices exist, those should be followed.
Alternative Solutions:
TODO: Research how most other command line interfaces approach this before finalizing a solution.

Supposedly braces need to be matched in some bash terminals as well, so might need to just use another delimiter symbol.

As found in #291, there also might be issues with only using a prefix, so using a suffix should be considered.

I was doing some reading on the python Template class and found out why the behavior identified in #291 occurs. See here for an explanation (tl;dr is that _ is a reserved python character and can break substitution unless braces are also used). For example '$VIDEO_NAME_test.csv' does not work because the variable is followed by _. However, '${VIDEO_NAME}_test.csv' does work because the brackets separate the template variable from the _.
If changing the delimiter is desired, then creating a custom template class that inherits from Template and then rewriting the delimiter might be a good way to go. Template doesn't seem to support a prefix/suffix substitution scheme that I can tell, so this kind of issue with a template variable being followed by a reserved python character would require a different solution (regex substitution?).
Alternatively, this edge case could be documented in the docs with the solution of including braces.


multi-gpu failed in train_localization.py

unlike multi-gpu training using "train.py", in localization training using "train_localization.py", I am facing the following issue when trying to use multi-gpu. With single gpu, it is working, but I am running out of memory using only one Gefore 2080ti RTX gpu.
CUDA_VISIBLE_DEVICES=3,4 python train_localization.py --folds-csv folds.csv --config configs/se50_loc.json --logdir logs --predictions predictions --data-dir /datasets/xView2/train_tier3_combined --gpu 3,4 --output-dir weights
bottleneck  1280 256
bottleneck  704 192
bottleneck  384 128
bottleneck  128 64
Selected optimization level O0:  Pure FP32 training.

Defaults for this optimization level are:
enabled                : True
opt_level              : O0
cast_model_type        : torch.float32
patch_torch_functions  : False
keep_batchnorm_fp32    : None
master_weights         : False
loss_scale             : 1.0
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled                : True
opt_level              : O0
cast_model_type        : torch.float32
patch_torch_functions  : False
keep_batchnorm_fp32    : None
master_weights         : False
loss_scale             : dynamic
Freezing encoder!!!
  0%|                                                                                | 0/1266 [00:09<?, ?it/s]
Traceback (most recent call last):
  File "train_localization.py", line 302, in <module>
    main()
  File "train_localization.py", line 191, in main
    args.local_rank)
  File "train_localization.py", line 268, in train_epoch
    out_mask = model(imgs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
    raise output
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
    output = module(*input, **kwargs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/apex/amp/_initialize.py", line 197, in new_fwd
    **applier(kwargs, input_caster))
  File "/space/export/data/azim_se/xView2_second_place/models/unet.py", line 390, in forward
    x = stage(x)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/amajid/anaconda3/envs/xV2-con36-cu90-tch110-tv030/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 338, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: Expected tensor for argument #1 'input' to have the same device as tensor for argument #2 'weight'; but device 1 does not equal 0 (while checking arguments for cudnn_convolution)

DDP version worked, it should be run with the following params in your case  CUDA_VISIBLE_DEVICES=3,4 python  -u -m torch.distributed.launch --nproc_per_node=2 --master_port 9901 train_localization.py --distributed  --folds-csv folds.csv --config configs/se50_loc.json --logdir logs --predictions predictions --data-dir /datasets/xView2/train_tier3_combined  --output-dir weights


iOS: collapsing scrolling behavior

In the current iOS branch when I try to collapse a folder it is causing some weird scrolling behavior. Please see attached
http://sabi.net/temp/newsblur-scrolling.mov.

I can't reproduce it either any more. Will reopen if I can.

Not sure if this is related but now I’m getting the search box appearing when I swipe back to the feed list even if it wasn’t visible before. This is happening in the 8.0 release and didn’t happen before.

#1139 is a better encapsulation of what's happening; closing this one.


strict check on repackCommandString

Without proposed strict check, a value of 0 is evaluated as empty string. So, for example, passing ['-skip' => 0] in commands would result in a wrong parameter, since the zero is dropped.

Thanks @garak, good catch. I added failing test assertion that your patch resolves.


./node_modules/react-popper/lib/esm/Popper.js Module not found: Can't resolve '@babel/runtime/helpers/objectWithoutPropertiesLoose' in 'mypath\node_modules\react-popper\lib\esm'

after installed this module in my project,the console  printed this error

I get a different error that seems to be related, after updating to Babel 7.4 and core-js 3.
ERROR in ./node_modules/babel-runtime/core-js/object/assign.js
    Module not found: Error: Can't resolve 'core-js/library/fn/object/assign' in '/home/.../node_modules/babel-runtime/core-js/object'
     @ ./node_modules/babel-runtime/core-js/object/assign.js 2:13-56
     @ ./node_modules/babel-runtime/helpers/extends.js
     @ ./node_modules/react-popper/lib/esm/Popper.js
     @ ./node_modules/react-popper/lib/esm/index.js
     @ ./node_modules/react-datepicker/es/index.js
...

It seems to be related to react-popper using babel-runtime which uses core-js 2, while we're using @babel/runtime and core-js 3.
Is there any way to make it work on this situation?

@leslie555 I got the same error as you and updating react-scripts worked for me.
yarn remove react-scripts
yarn add react-scripts
If you have a proxy setup in local development, you will probably need to convert to using a setupProxy.js file.


k8s apiversion deprecation warnings

Hi we see warnings about deprecated kubernetes apiversions.
W0226 14:19:49.344630       1 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0226 14:19:49.448353       1 warnings.go:67] certificates.k8s.io/v1beta1 CertificateSigningRequest is deprecated in v1.19+, unavailable in v1.22+; use certificates.k8s.io/v1 CertificateSigningRequest

Tested with machine-controller v2.16.0

/remove-lifecycle stale

any news here?


Change the condition in mask_to_segm() to eliminate invalid bboxes

Resolves #859

pfnCI, test this please

LGTM


Access new values in $afterUpdate

Issue #828 was closed but I believe it is still valid.

When running an insert() with .returning('*') on postgres, the $afterInsert hook contains all fields of the model record in this.
When running a patch() with .returning('*') on postgres, the $afterUpdate hook contains only the data that was passed for the update.

This is pretty unexpected as it doesn't allow to run the same code in the $afterInsert and $afterUpdate hooks.
I think the reason is that there is special treatment of the returning values in https://github.com/Vincit/objection.js/blob/5b6c8afe07ac5a297daa0d789dd72586e6c96d8d/lib/queryBuilder/operations/InsertOperation.js#L54-L60
which is lacking https://github.com/Vincit/objection.js/blob/master/lib/queryBuilder/operations/UpdateOperation.js

Yeah, I think we could fix that. But instead I'd take a look at the static afterUpdate hook. The result passed to that method should already contain all the properties.

Yeah, I think we could fix that. But instead I'd take a look at the static afterUpdate hook. The result passed to that method should already contain all the properties.

Thanks for the hint @koskimas. Using the static hooks instead worked well for me. My hook setup is now much more consistent. I'll leave it up to you to close this or keep it open, I for my part no longer need this feature.

Thanks for the hint @koskimas. Using the static hooks instead worked well for me. My hook setup is now much more consistent. I'll leave it up to you to close this or keep it open, I for my part no longer need this feature.


Modify explicit page title and description

Description & motivation
An astute Community member has pointed out  that we have inconsistent capitalizations for dbt on the docs site. We think this is because of Google Search taking liberties with auto-generating page structure. This PR attempts to address this by setting an explicit title that is not a URL.
To-do before merge

[x] Test locally to make sure nothing breaks

Pre-release docs
Is this change related to an unreleased version of dbt?

[ ] Yes: please

update the base branch to next
add Changelog components: <Changelog>[New/Changed] in v0.x.0</Changelog>
add links to the "New and changed documentation" section of the latest Migration Guide

[x] No: please ensure the base branch is current
[ ] Unsure: we'll let you know!

Checklist
no page additions/removal

FYI - doesn't look from the source of docs.getdbt.com like tagline compiles to a meta:description or og:description.  On individual docs pages in frontmatter it looks like it's the description field that populates those.

💯  that pages use the description field to populate the tag (although we aren't super good at setting these right now 🙃  )
According to the v1 Docusaurus docs (! I couldn't find the equivalent v2 docs page) tagline should provide the default that then gets superseded by the page description if it's available. But we can also set something at the React level if we really want to be sure :)

heads up that the reason the page title was set to docs.getdbt.com was to differentiate between tabs that are the website generated by "dbt docs generate" (left), and this page (right). It wasn't an intentional choice! (As was the old teal dbt icon for docs.getdbt.com
Can someone (@jasnonaz ?) think through how to differentiate these two tabs?

m)

Thanks @clrcrl! @grantwinship-consulting had a great idea that could help us distinguish here - interested to hear your thoughts.


Jmeter  code stopped building after removal of com.sun.activation » javax.activation >> 1.2.0

Expected behavior
I'm using Apache Jmeter 5.5 libraries to generate .jmx file & integrated with IDE.
Recently my company found vulnerability in com.sun.activation » javax.activation >> 1.2.0 Jar file. Enterprise support team deleted this jar file from repository. My Gradle build is failing, please suggest any alternatives ways to come over this.
https://mvnrepository.com/artifact/org.apache.jmeter/ApacheJMeter_components/5.5 com.sun.activation » javax.activation  is runtime dependency.
Tried excluding this jar file from build , Also this jar don't have old or newer version available in maven repository make iit work.
Please suggest how to come over this issue. Is there any alternative for this issue ?
Actual behavior
No response
Steps to reproduce the problem
Remove  com.sun.activation » javax.activation >> 1.2.0 Jar file from classpath
JMeter Version
5.5
Java Version
1.8
OS Version
Microsoft Windows 10

Well, don't delete jars, when you don't know how to fix it :)
You can try to replace that jar with the file jakarta.activation-api-1.2.2.jar found at https://mvnrepository.com/artifact/jakarta.activation/jakarta.activation-api/1.2.2
I will do a PR to replace that file in trunk and would be happy to hear, if replacing helped you.

After looking at the build failures with Java 17 of #5721 it seems, we (and that would include you) need to include the jar jakarta.activation-1.2.2.jar, which can be found via https://mvnrepository.com/artifact/com.sun.activation/jakarta.activation/1.2.2

Thank you for your email.  Issue is resolved now.  our Artifactory is
restored both  javax.activation & Jakarta.activation under
com.sun.activation group.
Code is working as expected.
Thanks,
Shrikant
On Fri, Oct 28, 2022 at 10:04 AM Felix Schumacher @.***>
wrote:

After looking at the build failures with Java 17 of #5721
https://github.com/apache/jmeter/pull/5721 it seems, we (and that would
include you) need to include the jar jakarta.activation-1.2.2.jar, which
can be found via
https://mvnrepository.com/artifact/com.sun.activation/jakarta.activation/1.2.2
—
Reply to this email directly, view it on GitHub
https://github.com/apache/jmeter/issues/5720#issuecomment-1295041293,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQ5RB5UDEUFTLS4LPZLJ6M3WFPMNRANCNFSM6AAAAAARQTOO4M
.
You are receiving this because you authored the thread.Message ID:
@.***>


Change define: - test=1 to - test: 1

I recommend to change the syntax for define symbols to the standard key: value in YML.
Reason: there might be tools that read/modify the YML data files, i.e. csolution.yml to adjust define values.  Having the non-standard syntax text=1 does not work with a standard yml reader.
So instead of
      define:
         - test = 1

This syntax that is consistent with YML.
      define:
         - test: 1

It is suggested to implement this as a breaking chang.e

Would it make sense to use a map instead of a list of maps? E.g.:
define:
  test: 1
  THE_KEY: the_value

define:
  - test: 1
  - MY_KEY2: the_value
  - MY_KEY3                  # no value

This creates an array (define) of maps with only a single entry, each.
I.e.;
define[0][test] = 1
define[1][MY_KEY2] = 'the_value'
define[2] = 'MY_KEY3'

Versus
define:
  test: 1
  MY_KEY2: the_value
  MY_KEY3: true

would create a map with three key-value pairs, i.e.,
define[test] = 1
define[MY_KEY2] = 'the_value'
define[MY_KEY3] = true

see: cmsis-toolbox 2.0.0-dev0


fix: cannot wrap Provider to nested page

背景：对于 src/pages/Home/Me/index 这样的页面，目前没有包上 store.Provider

Codecov Report

Merging #829 (d7cd708) into master (705a9b8) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #829   +/-   ##
=======================================
  Coverage   51.94%   51.94%
=======================================
  Files          47       47
  Lines        1232     1232
  Branches      260      260
=======================================
  Hits          640      640
  Misses        592      592


closes #281, closes #237

adds --job-info scriptname option and JOBSUB_EXTRA_JOB_INFO, JOBSUB_EXTRA_LINES, JOBSUB_EXTRA_ENVIRONMENT environment variables to augment the command line argument lists --job-info, --environment , and --lines,
So now you can export  JOBSUB_EXTRA_ENVIRONMENT=IFDH_DEBUG=1 and all your submissions will get ifdh debugging turned on, without changing your scripts.
But it's really to get rid of the ugliness that is poms_jobsub_wrapper...

All unit tests pass now - thanks!


Add more emojis to AWS upload logs

Now that we know emojis work properly with osbuild-composer and systemd,
let's add some more.
Signed-off-by: Major Hayden<EMAIL_ADDRESS>


Benefits?

Hi @gabomdq,
I stumbled upon your repo whilst adding game controller support to my SDL game. Out the box SDL appears to support the few controllers I have to hand so I'm wondering if I'm missing something - is this database a more exhaustive list? What benefits would I get from adding it to my game?
Kind regards,
Deon

Diff this database against what SDL provides. Gabriel started this repo to get wider (unofficial) coverage than the SDL project itself provides. There are a lot of legacy and third-party manufactured gamepad devices out there. All of the major first-party devices are covered in SDL's db eventually. If you want more thorough coverage, use this db. The maintainers of this DB do our best to keep the data of high quality and in parity as a superset of SDL's, but unlike SDL's maintainer we do not work at Valve. If you prefer SDL's rigor and oversight, use SDL's db. Many high quality shipped titles use this DB, presumably many more do not.

@offalynne Thanks for taking the time to answer the question, it's much appreciated! I understand now, I'll give this database a whirl :)


chore: release 1.2.129

:robot: I have created a release *beep* *boop*
1.2.129 (2022-07-15)
Features

custom-tag: add custom tags params for converge and bundle-publish (4cc2802)
kubedog: show Ready resources only once (d8281b8)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/werf/werf/releases/tag/v1.2.129 :sunflower:


Whitespace in additional address line field cause error in url generation

If a customer enters a whitespace in the additional address line field the url can't be generated. It's possible to fix it by an array map trim over billing and shipping address after building the orderData Array in PaymentHandler .

@ThLind In which version of the plugin are you experiencing this issue?

@milan-kiener It's Mollie Version 1.0.14 on Shopware Version 6.2.3

@ThLind This issue should be allready resolved in the latest version, 1.3.6. Can you confirm that?


Doc/rewrite (and code changes inspired by trying to document it)

Re-wrote the docs to include mpl_gui.registry.

Possible "short" versions (and we can change the module name to be something consistent):

flt (FigureLisT and reminds you of plt), fl
mfigs, my_figs, myfigs, mf, managed_figs
mwindows, mwins, mw
fig_reg, freg, fr

The other red-flag is that I got my self confused that FigureRegistry does not live in registry.

Should the module currently known as registry have something like “global” in its name, to distinguish it from the FigureResistry objects?

OK, I guess the use here is to mimic
fig, ax = plt.subplots()
... # do plotting
plt.show()

I almost wish this were mpl_gui.subplots() and mpl_gui.show().  I don't remember why you don't want to do it that way?  It's hard to think about what the name would be other than "gui".  Registry isn't bad I guess, just very abstract

I almost wish this were mpl_gui.subplots() and mpl_gui.show() I don't remember why you don't want to do it that way?

Because if the global state lives at the top-level there is no way to opt out of it!  By putting it in a submodule users who do not want the global state to be around can either not import it or put None is sys.modules depending on how cranky they are.

Sure, but... is that really a problem?  If they don't want to use it, they don't have to - it's just an empty registry object at that point.
I guess if this goes in mpl proper it could be a side module instead of a sub module.

We have a number of issues due to users using the global state without realizing it.
I started down this project with the goal of getting rid of global state and only added it back reluctantly (mostly to prove we could re-implement pyplot-like on top of this).  If the net result is to replace on unavoidable bit of global with another bit of global state it is not clear to me that there is actually any upside of the code churn.
If (when?) we pull this in to the main repo I think we would need FigureRegistry, a global singleton instance of said registry, and pyplot in three different modules (pyplot would use the singleton and the singleton would obviously import the FigureRegistry) so users could pick their level of global-state opt in.

I took @rcomer 's suggestion and put "global" in the name.

OK, I guess if it makes it into main, it could be matplotlib.gui and matplotlib.figreg, or something, and the global state would be in figreg, and folks would do mfr.subplots() or something like that.

... thought now that I think about it, I forget why we need a registry at all.  The original suggestion was to just make subplots etc available as well as figure.

We need it for the pyplot-replacement and it seems nice to expose publicly.
I think it would be a nice pattern to have user plotting functions that take in a FigureRegistry to either find by name or fabricate what ever figures they need, something like
fr = FigureRegistry()
plot1(..., fr)
plot2(..., fr)
fr.show()

I made the helper functions at the top level private, but could easily be convinced to put them back.  Writing the docs, it felt like I had 3 copies of exactly the same code block and "fixed" it by making one different.

We need it for the pyplot-replacement and it seems nice to expose publicly.

I'm still confused by this - happy to sidebar about it if easier...
Of course we need to keep pyplot around, but why can't it have its own registry and mpl_gui remain registry free?
Previously you were offering fig = mpl_gui.figure() and, if I recall correctly, no registry.  I suggested it would be nice to have  a helperfig, axs = mpl_gui.subplots() (and subplots_mosaic and whatever other shortcuts there. are), which is just an extra call to fig.subplots().   I'm forgetting why that morphed into needing a registry?

 fr = FigureRegistry()
 plot1(..., fr)

What is the advantage of that over just allowing such libraries to continue to use pyplot?  Better is surely plot1(..., fig=None) or plot1(..., ax=None) unless you want the methods to know about the Registry for some reason?

Going to go ahead and merge this (to get built docs) and continue in a new PR.


EMFILE: too many open files when runing dev mode

Hi. I'm building a monorepo project using TurboRepo. I have 13 packages and 2 apps, so not so many files, in comparison to other gigantic monorepos. This error started happening when I created the 2nd app, crashing at the moment it wants to start.
The error is shown at the 2nd app folder ("backoffice-psae"), when trying to import a package ("ui") that imports another package ("hooks") that imports a library ("react-hook-form"). The whole error:
Error: EMFILE: too many open files, open 'C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\apps\backoffice-psae\node_modules\ui\node_modules\hooks\node_modules\react-hook-form\dist\utils\isWeb.d.ts'

@weyert commented on this 3 weeks ago in #986:

I have tried turbowatch but it appears to watch the whole monorepo which I don't think right? I would expect it to watch any input that turbo itself considers? E.g. why would it need to watch node_modules directories?
I keep getting the error Error: EMFILE: too many open files, watch when running it based on the example given by @mattieb
I have to admit my monorepo is pretty big; based on the tree 60.000 files or via git ls-files 15.000

Needless to say, my project is by a big difference smaller, listing 156 files with git ls-files. In reality, counting node_modules, there are 47.549 files, still smaller than @weyert 's, however a lot in comparison to his 15.000 tracked files...
The only way I've found to workaround this issue is deleting the 2nd app folder. Not even runing dev mode with --filter and only the 1st app. Of course, not a solution. It throws away the whole point of creating a monorepo.
Call stack:
✓ Compiled /page in 4.3s
web:dev:  ⨯ ModuleBuildError: Error evaluating Node.js code
web:dev: Error: EMFILE: too many open files, open 'C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\apps\backoffice-psae\node_modules\ui\node_modules\hooks\node_modules\react-hook-form\dist\utils\isWeb.d.ts'
web:dev:     at processIssues (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\lib\router-utils\setup-dev-bundler.js:256:23)
web:dev:     at Object.ensurePage (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\lib\router-utils\setup-dev-bundler.js:1095:33)
web:dev:     at async DevBundlerService.ensurePage (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\lib\dev-bundler-service.js:18:20)
web:dev:     at async DevServer.ensurePage (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\dev\next-dev-server.js:518:9)
web:dev:     at async Object.ensure (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\dev\next-dev-server.js:164:17)
web:dev:     at async DevRouteMatcherManager.matchAll (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\future\route-matcher-managers\dev-route-matcher-manager.js:94:13)
web:dev:     at async DevRouteMatcherManager.match (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\future\route-matcher-managers\default-route-matcher-manager.js:155:26)
web:dev:     at async NextNodeServer.handleCatchallRenderRequest (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\next-server.js:222:31)
web:dev:     at async DevServer.handleRequestImpl (C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\node_modules\.pnpm\next@14.0.1_react-dom@18.2.0_react@18.2.0\node_modules\next\dist\server\base-server.js:751:17)
web:dev:  ✓ Compiled /_error in 369ms
web:dev:  ✓ Compiled /favicon.ico/route in 86ms

Some relevant info:
Dev command: env-cmd -f ./.env.development turbo run dev --concurrency 18
Dev config at turbo.json:
"dev": {
      "cache": true,
      "persistent": true,
      "dotEnv": [".env.development"]
    },

Env:
Windows 11
Node 21.0.0
pnpm 8.9.2
Apps runing with Next.js 14.0.1
I'll be glad to help if any more information is needed. Thanks!

I updated Node to 21.2.0, but still got the error.
I removed the persistent config at turbo.json, and the --concurrency 18 from the dev script, but still got the error.
I tried running only 1 app, directly moving my cmd to that folder and runing pnpm dev there, and removing from it any connection it may have to the other app (for example, a multizones config that goes in the next.config.js file). Still got the error (which is weird, because it isn't running the other app!)
My PC has 16GB of RAM and a 12th Gen Intel i3 processor, so performance shouldn't be a problem.

Can you try running with --no-daemon and see if you still get the same error?

Can you try running with --no-daemon and see if you still get the same error?

Hi! I´m on the same team, with the same issue. I tried this and I´m still getting the error

Hi @chris-olszewski, thanks for your quick reply. Running dev directly at backoffice-psae gave the same error. The dev command here, as well as in web but with other port, is next dev --port 3001 --turbo. The turbo flag was added some days ago at trying to solve this issue, so without it the problem remains.
The first time I had this problem was when creating this new app folder and trying to run it. As I reached nowhere, I checkouted to other branch to keep working on other stuff from packages folder. The error was still there, and it stayed until I realized that .next and .turbo folders, inside apps/backoffice-psae remained in the branch I was now working. This was because git didn't track them - normal behaviour. I removed those and everything went on as normal, but with only 1 app. So I've now tried many times to remove and recreate those folders, both from web and from backoffice-psae, and still nothing was usefull. Even in this last step you suggested.

If the issue is there without using turbo, then this is an underlying issue with how many files are open when running the dev server. If this issue is also there without the --turbo flag it also isn't a Turbopack related issue.

until I realized that .next and .turbo folders

.next is probably the culprit here. .turbo should just have task logs for the workspace.
I'd suggest looking for an existing issue in vercel/next.js or opening one there would probably be a better place to look for help.


Matter Image destroy() throws error

A bug in the API (always say which version you're using!)

createBody (x, y, key, index) {
    this.circ = this.matter.add
      .image(x, y, GENERAL_ASSETS_KEY, key)
      .setInteractive()
    this.circ.setBody(
      {
        type: 'circle',
        radius: 60 * 0.8,
      },
      {
        label: `Circ${index}`,
        mass: 11,
      },
    )
  }

so when i'm calling
this.circ.destroy()
i see this error

Transform.js:36 Uncaught TypeError: Cannot read property 'position' of undefined
at MatterImage.get [as x] (Transform.js:36)
at Camera.cull (Camera.js:561)
at InputManager.hitTest (InputManager.js:380)
at InputPlugin.hitTestPointer (InputPlugin.js:450)
at InputPlugin.update (InputPlugin.js:1490)
at EventEmitter.emit (index.js:203)
at Systems.step (Systems.js:278)
at SceneManager.update (SceneManager.js:480)
at FingerBallsGame.step (Game.js:367)
at TimeStep.step (TimeStep.js:527)

i saw that MatterImage extends from Image which is extends from GameObject, the last have destroy() method that destroying body, than itself, and i thought that destroy is the function to remove MatterImage from my scene

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.


Another attempt to reduce size_of

Changes made

data field is removed, instead the allocation layout is changed such that first element of control bytes start exactly at one past last element of bucket table. So we just use negative index to access bucket table without touching size field. Allocation layout looks like this: https://github.com/rust-lang/hashbrown/blob/7027781128a4896aea0aebd4f3f6590b8776dea9/src/raw/mod.rs#L359-L361
Changed calculate_layout implementation:

Layout is now [Paddings] | [Buckets] | [Ctrls], previously it was [Ctrls] | [Paddings] | [Buckets]
Returned offset is now start of control bytes in the allocation and also one past last element of buckets. (Previously it was data offset)

Renamed Bucket::add to Bucket::next_n

Drawbacks

Development cost? accessing with negative index seems awkward
Code is slower on cases where start of data pointer is needed

Issues

calling Bucket::as_ptr() on one past last element yields UB (crosses allocation boundary).
data_start() doesn't equal data_end() when table is empty because empty table returns buckets() == 1

cargo benchcmp (from local)
 name                         before ns/iter  after ns/iter  diff ns/iter   diff %  speedup
 clone_from_large             1,454           1,858                   404   27.79%   x 0.78
 clone_from_small             17              22                        5   29.41%   x 0.77
 clone_large                  1,985           1,731                  -254  -12.80%   x 1.15
 clone_small                  47              39                       -8  -17.02%   x 1.21
 insert_ahash_highbits        9,314           8,935                  -379   -4.07%   x 1.04
 insert_ahash_random          9,556           9,509                   -47   -0.49%   x 1.00
 insert_ahash_serial          9,739           9,475                  -264   -2.71%   x 1.03
 insert_erase_ahash_highbits  20,957          20,920                  -37   -0.18%   x 1.00
 insert_erase_ahash_random    21,220          22,805                1,585    7.47%   x 0.93
 insert_erase_ahash_serial    25,188          24,349                 -839   -3.33%   x 1.03
 insert_erase_std_highbits    50,005          50,587                  582    1.16%   x 0.99
 insert_erase_std_random      49,894          50,796                  902    1.81%   x 0.98
 insert_erase_std_serial      42,161          44,495                2,334    5.54%   x 0.95
 insert_std_highbits          22,460          22,433                  -27   -0.12%   x 1.00
 insert_std_random            22,554          22,943                  389    1.72%   x 0.98
 insert_std_serial            19,504          19,748                  244    1.25%   x 0.99
 iter_ahash_highbits          1,659           1,747                    88    5.30%   x 0.95
 iter_ahash_random            1,626           1,743                   117    7.20%   x 0.93
 iter_ahash_serial            1,492           1,520                    28    1.88%   x 0.98
 iter_std_highbits            1,639           1,792                   153    9.33%   x 0.91
 iter_std_random              1,600           1,834                   234   14.62%   x 0.87
 iter_std_serial              1,466           1,500                    34    2.32%   x 0.98
 lookup_ahash_highbits        4,445           4,912                   467   10.51%   x 0.90
 lookup_ahash_random          4,467           4,964                   497   11.13%   x 0.90
 lookup_ahash_serial          6,240           4,738                -1,502  -24.07%   x 1.32
 lookup_fail_ahash_highbits   4,241           4,444                   203    4.79%   x 0.95
 lookup_fail_ahash_random     4,514           4,603                    89    1.97%   x 0.98
 lookup_fail_ahash_serial     4,471           4,622                   151    3.38%   x 0.97
 lookup_fail_std_highbits     18,394          18,476                   82    0.45%   x 1.00
 lookup_fail_std_random       18,129          18,378                  249    1.37%   x 0.99
 lookup_fail_std_serial       15,348          15,716                  368    2.40%   x 0.98
 lookup_std_highbits          18,042          18,351                  309    1.71%   x 0.98
 lookup_std_random            18,247          18,567                  320    1.75%   x 0.98
 lookup_std_serial            15,054          15,767                  713    4.74%   x 0.95

Some of results vary a lot in my tests, they seem roughly same except clone_from_large is slower.

@bors r+

:pushpin: Commit b500a0a8b72fbf5ae08c28285e4c20c650c1697b has been approved by Amanieu

:hourglass: Testing commit b500a0a8b72fbf5ae08c28285e4c20c650c1697b with merge 9498006e0c535d660b9bc229e430ea45f556e899...

:sunny: Test successful - checks-travis
Approved by: Amanieu
Pushing 9498006e0c535d660b9bc229e430ea45f556e899 to master...


Operations manager

Operations manager
Originally posted by @Evilantireversed in https://github.com/jlord/sheetsee.js/issues/376

Operations manager
Originally posted by @Evilantireversed in jlord/sheetsee.js#376


Update anti-entropy-api.md

added more detail about where this API exists and it's integration with the influxd-ctl tool
Fixes #2793

need to make sure this gets pulled through to 1.8 as well...

thank you, @timhallinflux!

pulled to 1.8


Updates to polish gliders for consistency

I've recognized a bunch of inconsistencies with most of the polish gliders, which I tried to correct. The original list looks like its based on the DMSt index list. Some of the changes should also be corrected there, but that's a different story.
Happy to discuss every changed line and ready to split-up the changes if necessary.

Thank you for fixing these inconsistencies. Indeed I would need some more explanation as I am not too familiar with polish gliders. I think we should split the updates on the aircraft naming and the updates on the manufacturer naming into two separate commits. Also I have mixed feelings about updating names for 'Jantar' and 'Diana' as these are the common names they are known by in most countries.

Thank you for fixing these inconsistencies. Indeed I would need some more explanation as I am not too familiar with polish gliders. I think we should split the updates on the aircraft naming and the updates on the manufacturer naming into two separate commits. Also I have mixed feelings about updating names for 'Jantar' and 'Diana' as these are the common names they are known by in most countries.

Also I have mixed feelings about updating names for 'Jantar' and 'Diana' as these are the common names they are known by in most countries.

Yes, I understand that. Unfortunately there are 3 Jantar Standard versions and one of them has a different fuselage, which according to your reasoning needs a separate entry. Their "real" designation (SZD-41 / SZD-48) reflect that. The club class IGC index list also has a different index for those two.
Question: Does it really matter when your search algorithm can pull up both (and the open class ones) entries when typing in "Jantar"?
The SZD-59 Acro is even weirder in the current index list. It does only exist without winglets in its Acro (13.2m) configuration, but that should for sure not have an index of 100.

Also I have mixed feelings about updating names for 'Jantar' and 'Diana' as these are the common names they are known by in most countries.

Yes, I understand that. Unfortunately there are 3 Jantar Standard versions and one of them has a different fuselage, which according to your reasoning needs a separate entry. Their "real" designation (SZD-41 / SZD-48) reflect that. The club class IGC index list also has a different index for those two.
Question: Does it really matter when your search algorithm can pull up both (and the open class ones) entries when typing in "Jantar"?
The SZD-59 Acro is even weirder in the current index list. It does only exist without winglets in its Acro (13.2m) configuration, but that should for sure not have an index of 100.

Yes I think you are right and we should go with the full name. I had a look at the type certificate and I think we should go with
SZD-41 Jantar Std.
SZD-48 Jantar Std. 2
SZD-48-3 Jantar Std. 3
and also add WL variants for all of them.

Yes I think you are right and we should go with the full name. I had a look at the type certificate and I think we should go with
SZD-41 Jantar Std.
SZD-48 Jantar Std. 2
SZD-48-3 Jantar Std. 3
and also add WL variants for all of them.


Query prioritisation support

Problem
We lack a prioritisation mechanism for queries for instance

At shard level fetch phase could have a higher priority than query phase requests, so that requests complete faster with higher probabilities
Async search queries running across multiple clusters should have a lower priority than usual search requests
Resource intensive queries could similarly have a lower priority

I was having a discussion about this literally just today. Thanks for this @Bukhtawar

Throwing some additional thoughts and copying details from #1140 and closing it, since this was created first.
As part of #1042 we are planning to do resource mapping of queries, and selective rejection when in duress. We want to extend the solution to also allow query prioritisation, which provides mechanism to selectively execute queries, when there are multiple queries with different priorities contending for the same resources.
Not all queries in the workload are of equal importance to customer. Often performance of one request or set of queries might be more important than others. With query prioritisation, customers can define the relative importance of queries in a workload by setting a priority value. The priority is specified for a dynamic queue such as one of (CRITICAL, HIGHEST, HIGH, NORMAL, LOW, LOWEST).
Opensearch will use the priority when accepting queries for execution, and to determine the amount of resources to be allocated to a query. By default, queries run with their priority set to NORMAL. These priority of query is also used under duress, to selectively cancel resource guzzling queries, and recover the system.

I wanted to capture a practical use case for this feature in the company I work at. We have three main sources of load to OpenSearch:

Live user queries (essentially powering the search results of a search box) which have strict reliability, availability and latency SLOs. These queries need to be run immediately as there is an end user looking for search results. Queries of these type are observed with daily peaks under business hours and lows outside business hours. An outage of these use cases would have material customer impact
Background batch jobs which execute queries which can be slower to execute, can be retried and can be run at any point in the day.

Ideally if OpenSearch was under stress (particularly on specific data nodes), it would be able to first load shed the background jobs before it starts to load shed live user queries. This could be done outside of OpenSearch, but either the backpressure would be too crude, creating failing requests which would have executed just fine as the target data node/shard was not under stress but others were, or requires complex logic to track which shards and data node a request will map to and use external monitoring to determine if that shard/data node is under stress. The OpenSearch cluster itself is much better suited to be able to determine what shards are overloaded, provided it knows what requests are more or less important.


Was onClucterClick removed as an event?

Hello there. I'm trying to do an activity where I click on a cluster and something happens. However, I am unable to trigger it on the cluster. Below an example on codesandbox.  Try clicking on the cluster. You will see that nothing happens. Was it removed?
https://codesandbox.io/s/gifted-liskov-9rw91?file=/src/App.js

The thing with this component implementation is that createPathComponent needs 2 arguments. The first is the function that called when the component is created, and the second is a function that called when the component is updated.
And this implementation defines only the first one.
https://github.com/yuzhva/react-leaflet-markercluster/blob/398a2e16479c2073f286af628e3576268fa5753b/src/react-leaflet-markercluster.js#L7
So if you dynamically add/remove handlers to your component, changes are not applied.

was also looking for an answer to this but seems i have found a solution, just call onClick not onClusterClick.
this line is causing the clustering event to be called clusterclusterclick, not sure if there was a need to ad this extra text for other use cases or can be removed.


Better Error Notifications for Development Mode

As originally discussed in #513. It would be great to have DoneJS "yell" a bit more loudly when your app breaks. @matthewp liked the idea of injecting them into the DOM. I also suggested finding a way to throw terminal alerts.
Use Case
I am furiously developing a new component and have my browser window in the background. There is very little to catch my attention when I create a bug (such as a stache parsing error or a js syntax error). With hot module swapping you can introduce an error and the app will still appear to be loading correctly. Would be great if the terminal yelled at me or we injected a big red error banner in the app. :)

I like the idea but have 3 reservations:

usefulness
best practices
maintainability

but I have ideas on how to work around most of them.
usefulness

have my browser window in the background.  ....
we injected a big red error banner in the app. :)

If your browser is in the background, you wouldn't see the banner.  I think errors do show up in the dev console and in the command line (but aren't highlighted).
A solution to both problems might be using a browser's notifications API.  A person would see the error no matter where there browser is and we wouldn't be modifying the DOM.
best practices
I'm in general of adding any warnings to the DOM (at least not by default), because devs should be trained to look in their console.  Thats where every other project logs its errors.
maintainability
I wouldn't want something like can-set, which has no dependencies, to have code looking for some supperLogger.error().
Perhaps we could build a project that would intercept console.warns and perform any additional level of altering.  Otherwise, to make every subproject use something else might be overkill.

Otherwise, to make every subproject use something else might be overkill.

I think most of these errors are caught by live-reload, so that's where the logic would go, not in subprojects.  Currently we reload the page if there's a reload error: https://github.com/stealjs/live-reload/blob/master/live.js#L213

@matthewp what about a template parsing warning I might want to warn about in can/view/parse?

A solution to both problems might be using a browser's notifications API. A person would see the error no matter where there browser is and we wouldn't be modifying the DOM.

I think this might make a good OS dev tool...it would be cool to package it as a chrome extension, and it will always throw alerts for js warnings/errors for specified domains like localhost, if you can do that. That way you don't have to bake it into any of your projects...but then this escapes the realm of our OS tools.


项目怎么跑起来

我把代码拉下来之后不知道该怎么运行起来

Please checkout the readme for installation section. You can use the online ide, text editor extensions or install using npm. Thanks

我用npm安装了，找不到应该怎么把这个运行到浏览器

@antfu project is about  ancient Chinese, reply is English, seems weird. I think you can：
有一书简，名曰 ReadMe，汝可观之。或假以online ide试之。

吾乃略读ReadMe,但仍未解鄙人之惑。不知兄台用其方法可否实现？

If you want to run the compiler by yourself, you can read this document https://github.com/LingDong-/wenyan-lang/blob/master/documentation/API.md

If you want to run the compiler by yourself, you can read this document https://github.com/LingDong-/wenyan-lang/blob/master/documentation/API.md

thanks,but page cannot be opened

@antfu project is about ancient Chinese, reply is English, seems weird. I think you can：
有一书简，名曰 ReadMe，汝可观之。或假以online ide试之。

吾乃略读ReadMe,但仍未解鄙人之惑。不知兄台用其方法可否实现？

thanks,but page cannot be opened

Sorry the document name is recently updated, check https://github.com/LingDong-/wenyan-lang/blob/master/documentation/Compiler-API.md

thanks,but page cannot be opened

Sorry the document name is recently updated, check https://github.com/LingDong-/wenyan-lang/blob/master/documentation/Compiler-API.md

Ok，thank you very much

@puchenhui As Antfu suggested:
npm i
npm run build
npm t

npm run build


User Interface Improvement

This pull requests has user interface improvement.

hi @habib-itopa , it seems the wrong branch was selected


Added tooltip to finalise submission

Description

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Code quality improvements

How to test

Checklist

[ ] I have tested this code
[ ] I have updated the documentation

One of the snapshot tests is failing, as spotted by CI. Could you take a look @AustinHuang1203 ? After verifying that it's ok, you can update the snapshot via:
$ yarn test --updateSnapshot

Not enough follow-up by original author


Add Xamarin.Forms Notes for professionals

What does this PR do?
Add a book: Xamarin.Forms notes for professionals
For resources
https://books.goalkicker.com/XamarinFormsBook/
Description
Why is this valuable (or not)?
It is a complete guide to Xamarin.Forms and is totally free of cost. Can be referred as a handbook as well
How do we know it's really free?
All the books and notes provided by Goalkicker are free and require no active email address or signup
For book lists, is it a book? For course lists, is it a course? etc.
It is a book
Checklist:

[x] Read our contributing guidelines.
[x] Search for duplicates.
[x] Include author(s) and platform where appropriate.
[x] Put lists in alphabetical order, correct spacing.
[x] Add needed indications (PDF, access notes, under construction).
[x] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

Thanks, and welcome to free-programming-books!


Improve perfomance using OffscreenCanvas

This is really cool and is the first time I'm hearing of the OffscreenCanvas.  Thanks for the hard work. I'm really motivated to get this one in, but I have to find the time.


[BUG] Bad MANIFEST.in exclude can globally break sdist when setuptools_scm is installed

setuptools version
62.4.0
setuptools-scm 6.4.2
Python version
3.10.5
OS
Arch Linux, reproducible on fedora
Additional environment information
No response
Description
I discovered an interesting setuptools foot-gun that took me a few hours to figure out.

Using wildcard global excludes in MANIFEST.in, it is possible to create installable but broken packages by excluding certain metadata files. Minimal example:

$ tar -taf normal-sdist.tar.gz
fnord-23.0.42/
fnord-23.0.42/MANIFEST.in
fnord-23.0.42/PKG-INFO
fnord-23.0.42/fnord.egg-info/
fnord-23.0.42/fnord.egg-info/PKG-INFO
fnord-23.0.42/fnord.egg-info/SOURCES.txt
fnord-23.0.42/fnord.egg-info/dependency_links.txt
fnord-23.0.42/fnord.egg-info/top_level.txt
fnord-23.0.42/setup.cfg
fnord-23.0.42/setup.py
$ tar -taf broken-sdist.tar.gz
fnord-23.0.42/
fnord-23.0.42/PKG-INFO
fnord-23.0.42/fnord.egg-info/
fnord-23.0.42/fnord.egg-info/SOURCES.txt
fnord-23.0.42/setup.cfg

Notice how the following files are missing:
fnord-23.0.42/fnord.egg-info/PKG-INFO
fnord-23.0.42/fnord.egg-info/dependency_links.txt
fnord-23.0.42/fnord.egg-info/top_level.txt

setuptools'es sdist command trips up with a TypeError: expected string or bytes-like object when it finds such a package installed in the user's ~/.local/lib/python*/site-packages (stack trace below).

Expected behavior

setuptools should refuse to build a package that is missing essential metadata files.
I'm not sure what the intended semantics of MANIFEST.in are, but matching its includes/excludes against package-internal metadata files that are autogenerated during the package build seems counter-intuitive to me. If this is required behavior, it would be good if it were hidden behind a switch.
sdist should not break when encountering an invalid package on disk
When sdist due to a bug errors out while parsing package meta from site-packages, it should print the name and path of the offending package along with the error to give the user some hint on how to fix the issue.

How to Reproduce
In a fedora container, run:
python3 -m ensurepip
python3 -m pip install -UI 'setuptools'
python3 -m pip install -UI 'setuptools_scm'
python3 -m pip install -UI 'build'

mkdir /tmp/testpkg
cd /tmp/testpkg
cat <<EOS > setup.py
from setuptools import setup, find_packages

setup(
    name='fnord',
    version='23.0.42',
    python_modules=['fnord']
)
EOS
cat <<EOS > MANIFEST.in
global-exclude *
EOS
python3 setup.py install --user
python3 -m build --sdist -n

Output
Future calls to setup.py sdist  or python3 -m build --sdist -n will fail with a TypeError: expected string or bytes-like object, even in setup.py of completely unrelated projects under the same account:
[root@f5db61a4677a /]# python3 -m ensurepip
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Looking in links: /tmp/tmpt3_2fo1d
Processing /tmp/tmpt3_2fo1d/setuptools-49.1.3-py3-none-any.whl
Processing /tmp/tmpt3_2fo1d/pip-20.2.2-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-20.2.2 setuptools-49.1.3
[root@f5db61a4677a /]# python3 -m pip install -UI 'setuptools'
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting setuptools
  Downloading setuptools-62.4.0-py3-none-any.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 41.0 MB/s
Installing collected packages: setuptools
Successfully installed setuptools-62.4.0
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
[root@f5db61a4677a /]# python3 -m pip install -UI 'setuptools_scm'
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting setuptools_scm
  Downloading setuptools_scm-6.4.2-py3-none-any.whl (37 kB)
Collecting packaging>=20.0
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 14.4 MB/s
Collecting setuptools
  Using cached setuptools-62.4.0-py3-none-any.whl (1.2 MB)
Collecting tomli>=1.0.0
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
     |████████████████████████████████| 98 kB 22.0 MB/s
Installing collected packages: pyparsing, packaging, setuptools, tomli, setuptools-scm
Successfully installed packaging-21.3 pyparsing-3.0.9 setuptools-62.4.0 setuptools-scm-6.4.2 tomli-2.0.1
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
[root@f5db61a4677a /]# python3 -m pip install -UI 'build'
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting build
  Downloading build-0.8.0-py3-none-any.whl (17 kB)
Collecting tomli>=1.0.0; python_version < "3.11"
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pep517>=0.9.1
  Downloading pep517-0.12.0-py2.py3-none-any.whl (19 kB)
Collecting packaging>=19.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Installing collected packages: tomli, pep517, pyparsing, packaging, build
Successfully installed build-0.8.0 packaging-21.3 pep517-0.12.0 pyparsing-3.0.9 tomli-2.0.1
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
[root@f5db61a4677a /]#
[root@f5db61a4677a /]# mkdir /tmp/testpkg
[root@f5db61a4677a /]# cd /tmp/testpkg
[root@f5db61a4677a testpkg]# cat <<EOS > setup.py
> from setuptools import setup, find_packages
>
> setup(
>     name='fnord',
>     version='23.0.42',
>     python_modules=['fnord']
> )
> EOS
[root@f5db61a4677a testpkg]# cat <<EOS > MANIFEST.in
> global-exclude *
> EOS
[root@f5db61a4677a testpkg]# python3 setup.py install --user
/usr/local/lib/python3.9/site-packages/setuptools/_distutils/dist.py:257: UserWarning: Unknown distribution option: 'python_modules'
  warnings.warn(msg)
running install
/usr/local/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
creating fnord.egg-info
writing fnord.egg-info/PKG-INFO
writing dependency_links to fnord.egg-info/dependency_links.txt
writing top-level names to fnord.egg-info/top_level.txt
writing manifest file 'fnord.egg-info/SOURCES.txt'
reading manifest file 'fnord.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'fnord.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

creating build
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/fnord-23.0.42-py3.9.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing fnord-23.0.42-py3.9.egg
Copying fnord-23.0.42-py3.9.egg to /root/.local/lib/python3.9/site-packages
Adding fnord 23.0.42 to easy-install.pth file

Installed /root/.local/lib/python3.9/site-packages/fnord-23.0.42-py3.9.egg
Processing dependencies for fnord==23.0.42
Finished processing dependencies for fnord==23.0.42
[root@f5db61a4677a testpkg]# python3 -m build --sdist -n
* Getting dependencies for sdist...
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
    main()
  File "/usr/local/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 345, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/local/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 297, in get_requires_for_build_sdist
    return hook(config_settings)
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 181, in get_requires_for_build_sdist
    return self._get_build_requires(config_settings, requirements=[])
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
    self.run_setup()
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 281, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 174, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 3, in <module>
    setup(
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 86, in setup
    _install_setup_requires(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
    dist = MinimalDistribution(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 57, in __init__
    super().__init__(filtered)
  File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 475, in __init__
    for ep in metadata.entry_points(group='distutils.setup_keywords'):
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 999, in entry_points
    return SelectableGroups.load(eps).select(**params)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 449, in load
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 996, in <genexpr>
    eps = itertools.chain.from_iterable(
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/_itertools.py", line 16, in unique_everseen
    k = key(element)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 931, in _normalized_name
    return self._name_from_stem(stem) or super()._normalized_name
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 600, in _normalized_name
    return Prepared.normalize(self.name)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 855, in normalize
    return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/usr/lib64/python3.9/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_sdist

Hi @jaseg, thanks for reporting this. Unfortunately, I cannot reproduce the behaviour you are seeing in an isolated fashion:
> docker run --rm -it fedora:36

python3 -m ensurepip
python3 -m pip install -UI 'setuptools==59.5.0'

mkdir /tmp/testpkg
cd /tmp/testpkg
cat <<EOS > setup.py
from setuptools import setup, find_packages

setup(
    name='fnord',
    version='23.0.42',
    python_modules=['fnord']
)
EOS
cat <<EOS > MANIFEST.in
global-exclude *
EOS
python3 setup.py install --user
python3 setup.py sdist
python3 setup.py sdist  # once again ...

This example runs successfully.
Could you provide a modified reproducer that don't depend on the specific state of your machine? (You can work in a modified version of the example above that uses containers for isolation).
Please also note the following:

Setuptools don't provide patches for old releases, so please verify if the behaviour still exists in the latest version.
Please don't use the python setup.py install command, as it is considered deprecated.
Instead you can try python -m pip install .
Please don't use the python setup.py sdist command. It is considered deprecated.
Instead you can try python -m build --sdist (build needs to be installed first). (python -m build --wheel is also "equivalent" to python setup.py bdist_wheel)

I am still trying to think about the expected behaviour you proposed. Somethings seem fair. However the *.egg-info directory is not something that is standardised, and currently considered an internal mechanism for setuptools.
What is a standardised metadata file is the PKG-INFO file on the root directory of the sdist, and this file seems to be there...

Hmm, after some debugging I found that the issue only happens if the setuptools_scm package is installed.
I'm not sure if maybe this is a setuptools_scm issue then? This reproduces the issue on fedora:
python3 -m ensurepip
python3 -m pip install -UI 'setuptools'
python3 -m pip install -UI 'setuptools_scm'

mkdir /tmp/testpkg
cd /tmp/testpkg
cat <<EOS > setup.py
from setuptools import setup, find_packages

setup(
    name='fnord',
    version='23.0.42',
    python_modules=['fnord']
)
EOS
cat <<EOS > MANIFEST.in
global-exclude *
EOS
python3 setup.py install --user
python3 setup.py sdist
python3 setup.py sdist  # once again ...

Output:
[root@1d65ff136570 /]# python3 -m ensurepip
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Looking in links: /tmp/tmphags28qn
Processing /tmp/tmphags28qn/setuptools-49.1.3-py3-none-any.whl
Processing /tmp/tmphags28qn/pip-20.2.2-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-20.2.2 setuptools-49.1.3
[root@1d65ff136570 /]# python3 -m pip install -UI 'setuptools'
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting setuptools
  Downloading setuptools-62.4.0-py3-none-any.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 36.6 MB/s
Installing collected packages: setuptools
Successfully installed setuptools-62.4.0
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
[root@1d65ff136570 /]# python3 -m pip install -UI 'setuptools_scm'
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting setuptools_scm
  Downloading setuptools_scm-6.4.2-py3-none-any.whl (37 kB)
Collecting setuptools
  Using cached setuptools-62.4.0-py3-none-any.whl (1.2 MB)
Collecting packaging>=20.0
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 14.6 MB/s
Collecting tomli>=1.0.0
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
     |████████████████████████████████| 98 kB 26.3 MB/s
Installing collected packages: setuptools, pyparsing, packaging, tomli, setuptools-scm
Successfully installed packaging-21.3 pyparsing-3.0.9 setuptools-62.4.0 setuptools-scm-6.4.2 tomli-2.0.1
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
[root@1d65ff136570 /]#
[root@1d65ff136570 /]# mkdir /tmp/testpkg
[root@1d65ff136570 /]# cd /tmp/testpkg
[root@1d65ff136570 testpkg]# cat <<EOS > setup.py
> from setuptools import setup, find_packages
>
> setup(
>     name='fnord',
>     version='23.0.42',
>     python_modules=['fnord']
> )
> EOS
[root@1d65ff136570 testpkg]# cat <<EOS > MANIFEST.in
> global-exclude *
> EOS
[root@1d65ff136570 testpkg]# python3 setup.py install --user
/usr/local/lib/python3.9/site-packages/setuptools/_distutils/dist.py:257: UserWarning: Unknown distribution option: 'python_modules'
  warnings.warn(msg)
running install
/usr/local/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
creating fnord.egg-info
writing fnord.egg-info/PKG-INFO
writing dependency_links to fnord.egg-info/dependency_links.txt
writing top-level names to fnord.egg-info/top_level.txt
writing manifest file 'fnord.egg-info/SOURCES.txt'
reading manifest file 'fnord.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'fnord.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

creating build
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/fnord-23.0.42-py3.9.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing fnord-23.0.42-py3.9.egg
Copying fnord-23.0.42-py3.9.egg to /root/.local/lib/python3.9/site-packages
Adding fnord 23.0.42 to easy-install.pth file

Installed /root/.local/lib/python3.9/site-packages/fnord-23.0.42-py3.9.egg
Processing dependencies for fnord==23.0.42
Finished processing dependencies for fnord==23.0.42
[root@1d65ff136570 testpkg]# python3 setup.py sdist
Traceback (most recent call last):
  File "/tmp/testpkg/setup.py", line 3, in <module>
    setup(
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 86, in setup
    _install_setup_requires(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
    dist = MinimalDistribution(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 57, in __init__
    super().__init__(filtered)
  File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 475, in __init__
    for ep in metadata.entry_points(group='distutils.setup_keywords'):
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 999, in entry_points
    return SelectableGroups.load(eps).select(**params)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 449, in load
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 996, in <genexpr>
    eps = itertools.chain.from_iterable(
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/_itertools.py", line 16, in unique_everseen
    k = key(element)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 931, in _normalized_name
    return self._name_from_stem(stem) or super()._normalized_name
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 600, in _normalized_name
    return Prepared.normalize(self.name)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 855, in normalize
    return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/usr/lib64/python3.9/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

It affects the latest versions of both packages:
[root@1d65ff136570 testpkg]# python3 -m pip list setuptools
Package        Version
-------------- -------
gpg            1.14.0
libcomps       0.1.15
packaging      21.3
pip            20.2.2
pyparsing      3.0.9
rpm            4.16.0
setuptools     62.4.0
setuptools-scm 6.4.2
tomli          2.0.1
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

The issue only appears when the package is installed using python3 setup.py install --user and when using python3 setup.py sdist.
When using python3 setup.py install --user, the following egg-zip gets copied to the user's site-packages:
[root@cdefa744e88c testpkg]# unzip -l ~/.local/lib/python3.9/site-packages/fnord-23.0.42-py3.9.egg
Archive:  /root/.local/lib/python3.9/site-packages/fnord-23.0.42-py3.9.egg
  Length      Date    Time    Name
---------  ---------- -----   ----
        1  06-14-2022 09:40   EGG-INFO/zip-safe
---------                     -------
        1                     1 file

When the package is installed using python3 -m pip install --user .,  the following directory gets created instead. This does not cause a problem with any sdist invocation:
[root@cdefa744e88c testpkg]# ls -lh ~/.local/lib/python3.9/site-packages/fnord-23.0.42-py3.9.egg-info/
total 16K
-rw-r--r-- 1 root root 51 Jun 14 09:44 PKG-INFO
-rw-r--r-- 1 root root  0 Jun 14 09:44 SOURCES.txt
-rw-r--r-- 1 root root  1 Jun 14 09:44 dependency_links.txt
-rw-r--r-- 1 root root 56 Jun 14 09:44 installed-files.txt
-rw-r--r-- 1 root root  1 Jun 14 09:44 top_level.txt

The issue disappears when using python3 -m build --sdist due to the automatic venv isolation. It re-appears when using the following invocation:
[root@cdefa744e88c testpkg]# python3 -m build --sdist -n
* Getting dependencies for sdist...
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
    main()
  File "/usr/local/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 345, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/local/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 297, in get_requires_for_build_sdist
    return hook(config_settings)
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 181, in get_requires_for_build_sdist
    return self._get_build_requires(config_settings, requirements=[])
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
    self.run_setup()
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 281, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 174, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 3, in <module>
    setup(
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 86, in setup
    _install_setup_requires(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
    dist = MinimalDistribution(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 57, in __init__
    super().__init__(filtered)
  File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 475, in __init__
    for ep in metadata.entry_points(group='distutils.setup_keywords'):
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 999, in entry_points
    return SelectableGroups.load(eps).select(**params)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 449, in load
    ordered = sorted(eps, key=by_group)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 996, in <genexpr>
    eps = itertools.chain.from_iterable(
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/_itertools.py", line 16, in unique_everseen
    k = key(element)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 931, in _normalized_name
    return self._name_from_stem(stem) or super()._normalized_name
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 600, in _normalized_name
    return Prepared.normalize(self.name)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 855, in normalize
    return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/usr/lib64/python3.9/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_sdist

I suspect that the venv-isolation hides the setuptools_scm package that triggers the issue.

I have updated the issue description with repro instructions. I find it weird that python3 setup.py install --user and python3 -m pip install --user . produce different results given the same input.

Hi @jaseg, thank you very much for checking that and for the detailed investigation.
What seems to be happening here is the following:

The .egg-info directory is not exactly part of any standard created by PyPA.
The PKG-INFO file at the root of the sdist is the only thing that seems to be required.
The sdist created by setuptools does seem to conform with that spec...
importlib.metadata seems to be making an extra effort to support existing .egg-info folders (which are in theory would only be installed via deprecated mechanisms such as an .egg distribution or python setup.py install). But seems to assume (understandably) that .egg-info directories would contain a PKG-INFO.

There are several things that could be done in this situation:

Modify setuptools to ensure .egg-info contains PKG-INFO
Modify importlib.metadata to ignore .egg-info directories that don't contain PKG-INFO files
Modify setuptools-scm to catch the error and ignore it
Modify setuptools to catch the error and ignore it.

However, it is important to recognise that the situation just seems to be problematic when a deprecated installation/build method is used. Moreover, we can also consider that the cause of the problem is a edge case very unlikely to happen...
With all these things in mind, I don't think it is worth right now to diverge the setuptools development efforts to fix a feature that is inevitably going to be removed and that we actively tell the users to not use...
That said, I would be more than happy to review any PR addressing this issue. If you would like to work on this, I think we can leave this issue open for a while, otherwise it seems to make little sense to me to keep it open (unless another person would like to tackle it).
If you (or other member of the community) would like to provide a PR, the following is an important tip:

Avoid changing setuptools/_distutils directory. This directory is periodically pulled from the https://github.com/pypa/distutils repository and any changes made there are likely to be overwritten.
If anyone would like to change code in setuptools/_distutils, the recommendation is that they contact the pypa/distutils team to see if they are supportive of the change, and then provide the PR directly to that repository.

What are your thoughts?

I have updated the issue description with repro instructions. I find it weird that python3 setup.py install --user and python3 -m pip install --user . produce different results given the same input.

python3 setup.py install --user uses easy_install behind the scenes, while pip install --user relies on, well, pip. easy_install is long deprecated.

Hi @abravalheri,
I think there are two things that would be good to change.

When any error is thrown inside setuptools during parsing of a package, setuptools should augment the error with human-readable information on the path of the package it tried to parse. Something like raise SystemError(f'Cannot parse package at {path}') from e . This would allow easier troubleshooting by users and would make sense regardless if you think this particular bug is likely or not. Some python users might not even be aware that ~/.local/lib/pythonX.XX/site-packages is a thing, and if they encountered an issue like this that would make it really difficult for them to debug. This is compounded by the fact that a crash like this might happen long after the questionable package was installed.

Given that broken packages can happen in practice, and it may not even be the user's fault (e.g. they downloaded a broken package from somewhere else, or they had a system crash and some package metadata files got lost in that), I think setuptools should never throw an error while parsing a package. The brute-force approach would be to catch any exception inside the package parsing code and transform it into a warning, the nicer way would be to make sure everything that parses packages properly checks the package for correctness first.

I think setuptools_scm is totally not at fault here, because AFAICT it only uses the setuptools API as intended. Though I think the fact that merely having it installed changes the behavior of setuptools is odd and may cause more trouble in the future, I think that is a separate issue.

Hi @jaseg, I would be very happy to review a PR addressing the suggestions you provided, they seem pretty reasonable to me. Would you be interested in submitting one?

It seems I've ran into this bug in the wild. I'm running OpenSuSE Tumbleweed, ans so far it seems that I'm experiencing this bug with both python3.9 and python3.10 system wide installs. Is this issue solved already by the approach suggested by @jaseg ? If not I'd like to post whichever info would be relevant, if you could provide me with some instructions as to which information is important exactly and how to get it.
I'm experiencing this bug with and without setuptools_scm installed.
Could any of you advise me on the best way to find and remove the invalid package(s)?

@sandersantema I used pdb to debug the issue. By running setup.py with pdb (something like python3 -m pdb setup.py ...), pdb will break and give you a debug prompt when setuptools encounters the error parsing the invalid package. You can then walk up the stack using where and up, and print variables using p [expr]. If you haven't used it before, the pdb manual is in the python docs.

Thank you very much for that recommendation. Can't believe I didn't learn about pdb earlier, such a useful tool. I unfortunately wasn't able to learn much about the what caused the bug itself, but I did find the package causing the error. I was thinking it might be useful for me to upload a zip of the package directory but it seems that the whole package consists of nothing but directories.
Here's the output of tree -a (with repeating non-unique directories omitted) :
├── EGG-INFO
│   └── scripts
├── pynicotine
│   ├── plugins
│   │   ├── anti_shout
│   │   │   └── __pycache__
│   ├── __pycache__
│   └── upnp
│       └── __pycache__
└── share
    ├── applications
    ├── doc
    │   └── nicotine
    ├── icons
    │   └── hicolor
    │       ├── scalable
    │       │   ├── apps
    │       └── symbolic
    │           └── apps
    ├── locale
    ├── man
    │   └── man1
    └── metainfo

Given that I install this package from their git repo manually with setup.py I assume something went wrong once, so that might explain why the package is there.


Pass through target attribute on continue link.

So links that should open in a new window can do.
@incuna/js Please merge, ta!

Perhaps we could clone the original element so everything persists?

Thanks!

Released as a minor upgrade because it's new functionality. See https://github.com/incuna/angular-external-link-interceptor/releases/tag/1.2.0


support encodeImage method

I see there is an encodeImage proc in the nim library, would it be possible to support that as well as the already existing write_file() in the python layer?
https://nimdocs.com/treeform/pixie/pixie.html#encodeImage%2CImage%2CFileFormat

Could you describe how you'd use this API and what you'd expect to get for a datatype back? This would be the first time we return a "binary blob" back, which is not an actual string and thus has Python encode() / decode() implications.


Document process used to extract cardano-coin-selection repository.

Context
As part of the work to create a separate coin selection and fee balancing library, we created a repository for this library from a clone of the cardano-wallet library.
When creating this clone, we filtered the commit history so that only commits relevant to coin selection and fee balancing were retained. However, this process was non-trivial, as the internal structure of cardano-wallet has been changed many times during its history. A naive repository extraction would have resulted in history being curtailed for files that have content from multiple ancestor files at different file locations.
Given that we'll need to extract further libraries in the future, it would be beneficial to document the process we used for extracting cardano-coin-selection.
Decision
Document the process we used for extracting cardano-coin-selection, so that we have a simple set of steps to following when extracting further libraries.
Acceptance Criteria

[x] There must be a guide that explains the process of extracting a new repository from an existing one.
[x] The guide must explain how to retain the full history of all files that are included in the extracted repository.

QA

[ ] Take a clone of the cardano-wallet repository, delete an arbitrary subset of files, and then follow the steps in the guide to make a new repository based on the remaining subset of files. Confirm that the extraction has worked, with full history preserved for the subset of files included.

@jonathanknowles, I wonder if better place for this type of document would be here -> https://github.com/input-output-hk/adrestia/wiki#engineering-practices

Linked the article on wiki -> https://github.com/input-output-hk/adrestia/wiki.
LGTM!

@jonathanknowles, I wonder if better place for this type of document would be here -> https://github.com/input-output-hk/adrestia/wiki#engineering-practices

After a brief internal discussion, we came to the conclusion that we would like to:

Keep the existing article within cardano-coin-selection to serve as a historical record of how we extracted that repository from cardano-wallet.

Create a link from the Adrestia engineering wiki to the above article. If in future we need to create a general process that deviates from the process we used to extract cardano-coin-selection, we can update the wiki with the updated process.


add make update-chart

An attempt at making git subtree more usable and consistent across our repositories.
This can be run as follow, with kube-prometheus-stack-23.2.0 being an upstream reference (tag, commit sha, branch name, etc ...)
$ make update-chart version=kube-prometheus-stack-23.2.0

It requires the Chart.yaml to contain information about the upstream repository we are syncing against.
e.g.
$ cat helm/prometheus-operator-app/Chart.yaml|yq -y -r '.annotations'
repository: https://github.com/prometheus-community/helm-charts
path: charts/kube-prometheus-stack/

Sadly it depends on https://github.com/kislyuk/yq to process Chart.yaml informations.
TODO: test run with & without conflicts

Closing outdated and abandoned PR.


Link to php implementation on website

Please add https://github.com/lyoshenka/php-triplesec to the list of implementations.
I'm happy to file a PR if you point me to the website source.

The HTML apparently comes from here: https://github.com/keybase/triplesec/blob/master/index.html

I don't think it does. That page doesn't match what's on https://keybase.io/triplesec/
Either way, they added a link so this can be closed.


1.18 Forge port

Requires new branch to be created

I actually sent a message via curseforge about a Forge port, where I stated that I would handle any bug reports or version porting. I realize now I did not make that clear here. If you do not want the code to be hosted here though, that's also fine


Fixing issue #284 : Reviewed unit tests related tests in BaseOnHeapStoreTest

I have reviewed the tests in StoreEventListenerTest that are related to tests in BaseOnHeapStoreTest and to me, it makes sense to retain these tests as they can validate the listener if behaviour of Store is changed. I've also added a few negative tests for OnEviction and OnExpiration events. However I did not add tests for each and every store operation as eviction and expiration events would be same for each of them.
Please provide comments/suggestions on the same

@aurbroszniowski Can you have a look at this PR as it relates to an issue you created?

@rishabhmonga Sorry if the issue wasn't described clearly enough... the goal was not to review SPI tests, but the unit tests in BaseOnHeapStoreTest (as indicated in the title)
for instance, the negative test case 'no expiration' that you added is missing in the unit tests, e.g. adding in the BaseOnHeapStoreTest some cases like:
  public void testGetNotExpired() throws Exception {
    TestTimeSource timeSource = new TestTimeSource();
    OnHeapStore<String, String> store = newStore(timeSource,
        Expirations.timeToLiveExpiration(new Duration(2, TimeUnit.MILLISECONDS)));
    StoreEventListener<String, String> listener = addExpiryListener(store);
    store.put("key", "value");
    timeSource.advanceTime(1);
    assertThat(store.get("key"), not(nullValue()));
    verify(listener, never()).onExpiration(Matchers.<Cache.Entry<String, String>>any());
  }

And to be more explicit, I would be quite strict NOT to include the tests you are proposing in the SPI tests as they do not verify any explicit contract, but rather adapt the cases to the unit tests in the BaseOnHeapStoreTest unit tests.

I have reviewed the unit tests in BaseOnHeapStore and included a few more to add coverage to StoreEventListener. Additionally I have reverted the changes in StoreEvntListenerTest as these were not required as part of the issue. The PR is updated and needs to reviewed

@aurbroszniowski: would you please review the changes to the PR


?dtgePreview=1 post 404

I have a standard 0.4.0 nuget install in Umbraco v7.6.1 (.Net 4.6.2) and I'm getting the following error message when I load a page where I have added a doc type.

The config is:
{
	"name": "Content picker",
	"alias": "docType",
	"view": "/App_Plugins/DocTypeGridEditor/Views/doctypegrideditor.html",
	"render": "/App_Plugins/DocTypeGridEditor/Render/DocTypeGridEditor.cshtml",
	"icon": "icon-item-arrangement",
	"config": {
		"allowedDocTypes": ["GridWidget$"],
		"nameTemplate": "",
		"enablePreview": true,
		"viewPath": "/Views/Partials/Grid/Editors/DocTypeGridEditor/",
		"previewViewPath": "/Views/Partials/Grid/Editors/DocTypeGridEditor/Previews/",
		"previewCssFilePath": "",
		"previewJsFilePath": ""
	}
}

I have created a doc type called "Basic Content Grid Widget" (alias of "basicContentGridWidget") and I have a file at both /Views/Partials/Grid/Editors/DocTypeGridEditor/BasicContentGridWidget.cshtml and /Views/Partials/Grid/Editors/DocTypeGridEditor/Preview/BasicContentGridWidget.cshtml
When the page in the CMS loads, there is no preview and I can see from my proxying tool that the following POST request is returning a 404:
http://localhost:56529/1196?dtgePreview=1

@digbyswift - thanks for raising this. Does your website have any published nodes?
Currently DTGE's preview mechanism requires at least the homepage to be published.  It's not ideal, but at the time when we developed DTGE, it was the only option.

All of the nodes are published. However the root node (which is an immediate parent of the current node) is a document type without a template which performs an internal redirect to a homepage node. Could it be this root node?

@digbyswift Yes, that'll be it.
I've experienced that problem too, which led me to add in a check for RedirectResult.  Then in my controller code for the homepage redirect, I ended up adding in a check if it was a DTGE preview request.
if (Request.QueryString["dtgePreview"] == "1" && Request.HttpMethod == "POST" && Request.QueryString["nodeId"] != null)
{
    return View(model);
}

I know it's not ideal, but again it's due to how the preview mechanism currently works.  (I'd love to rewrite it! #59)

So just to be sure, I've recreated the content structure but with a new node at the root which has a template. I have also removed the redirect property from the root. I've then republished all the content. However, unfortunately the child nodes are still all displaying the same error as before and the preview is still not displaying. I've rolled the client dependency and restarted the app too.

Ok, I have fixed the error but not the previews not displaying. The error was caused because I had set web.routing > disableFindContentByIdPath=true in the umbracoSettings.config as part of my default setup.
Is there any reason why the previews would not be displaying?

@digbyswift Cool that you've resolved it.
Hopefully one day soon I can rewrite the preview mechanism! 😃

It's the first time I've used it and nearly gave up but now that I've got it working, it looks real good. Thanks for your help and time!


change unsupported character

That character is the Greek xi, ξ.
AFAIK this is used in memory.py as well, can you replace them all for consistency?

Oops, missed that it was used elsewhere. Yep, I'll update them everywhere.