Factor the mipmap generation code out of bloom so it can be used elsewhere.

Objective
Many postprocessing passes, such as bloom, depth of field, and screen-space reflections (SSR), want to generate mipmaps of the framebuffer. The logic for this is currently hard-coded into the bloom code. It would be helpful to generalize this in preparation for adding depth of field and screen-space reflections, especially since workarounds are necessary on WebGL 2 and it'd be ugly to duplicate them everywhere. Furthermore, it'd be nice to write this using compute shader eventually, as SSAO does for the depth buffer, and having the mipmap generation code nicely factored should allow the compute version to be written once and used everywhere.
The following passes can or could benefit from mipmap generation:

Bloom—Downsampling generates mipmaps using a custom shader. This pull request ports it over to use this new infrastructure.
SSAO—Downsampling the depth buffer creates mipmaps. This is currently done using compute shader, so this PR doesn't port it over to avoid performance regressions. Eventually, it probably could be ported, once this mipmapping logic gains compute shader functionality.
Automatic texture mipmap generation on import—This PR makes an implementation of this frequently-requested feature straightforward.
Depth of field—This effect is frequently implemented using mipmaps to create the blur.
Screen space reflections—Mipmaps of the framebuffer are frequently used here to create blurry reflections.
Two-phase occlusion culling—This modern occlusion culling technique generates mipmaps of the depth buffer in order to optimize occlusion checks and could benefit from this infrastructure.
Reflection planes—To properly handle rough materials, real-time reflection planes need to generate mipmaps.

Essentially, this PR, or something like it, is probably a prerequisite for these features.
Solution
This pull request introduces a Mipmap trait that can be implemented on a PipelineKey to, effectively, automatically generate the following types:

PipelineIds
Pipeline
PipelineKey
Texture
BindGroups
SpecializedRenderPipelines

Along with generating these types, a generate_mipmaps function is available that is intended for use in a render node and encapsulates all the logic needed to repeatedly invoke a shader on successively smaller mip levels.
Note that the mipmapping logic is generic over the shader, so generating mipmaps doesn't have to involve a straight blit. It can even involve custom inputs, as the bloom downsampling code does. More functionality can be added over time as the need arises.
In WebGL 2, each mipmap level is stored in a separate texture. This is because individual mipmap levels can't be bound to a single texture view on that platform. This fact was an important motivation for this patch, because duplicating that WebGL 2 workaround for every pass that requires mipmap generation is messy.

Changelog
Added

A new Mipmap trait is available that simplifies writing render passes that repeatedly call the same shader in order to generate mipmap levels.

There's another use case I've been interested in this sort of thing for, which is rough transmissive materials. For example: hazed windows, rain effects and the like. Does this work for that too or would a different mechanism be needed there?

@NotAFile bevy already supports transmission as of 0.12 https://bevyengine.org/news/bevy-0-12/#standardmaterial-light-transmission
It's very possible the code from this PR could be used to improve it though. I personally, do not know.

Comments addressed

Added a screenshot.

Closing this as it's no longer needed.


x/tools/present: initial comment line now interpretted as a title

The present tool now supports Markdown syntax but also supports the previous present markup syntax. The switch to enable Markdown syntax use is that the first line (the talk title) be preceded with a # . Unfortunately, this coincides with the use of #  as a comment mark in the old syntax. In some cases, slides may contain copyright notices (I use present for lecture presentations and have done for some years now, so we require copyright notices). This puts the slide syntax into Markdown mode, which subsequently breaks because of the mutually unintelligible syntaxes.
It would be nice if there were a way to prevent present from assuming a #  prefix is a note to present to use Markdown. Or at the very least a more prominant warning that old slide have a significant risk of being broken (I'm actually happy we now have Markdown, but this will result in a fair bit of work for me to fix in my lecture presentations).
/cc @rsc

Hash-Bang rather than a pure Hash? I missed whatever discussion may
have occurred, so please ignore if that's already been rejected.
Lucio.
On 4/22/20, Dan Kortschak<EMAIL_ADDRESS>wrote:

The present tool now supports Markdown syntax but also supports the previous
present markup syntax. The switch to enable Markdown syntax use is that the
first line (the talk title) be preceded with a # . Unfortunately, this
coincides with the use of #  as a comment mark in the old syntax. In some
cases, slides may contain copyright notices (I use present for lecture
presentations and have done for some years now, so we require copyright
notices). This puts the slide syntax into Markdown mode, which subsequently
breaks because of the mutually unintelligible syntaxes.
It would be nice if there were a way to prevent present from assuming a #
prefix is a note to present to use Markdown. Or at the very least a more
prominant warning that old slide have a significant risk of being broken
(I'm actually happy we now have Markdown, but this will result in a fair bit
of work for me to fix in my lecture presentations).
/cc @rsc
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/golang/go/issues/38578

--
Lucio De Re
2 Piet Retief St
Kestell (Eastern Free State)
9860 South Africa
Ph.: +27 71 471 3694
Cell: +27 83 251 5824

Yeah, pretty much anything that's not /# / works. In the file I'm writing at the moment, I just did s/^# /#_/.

@rsc


Problem with NPM Install in root directory

When I run npm install in the file directory I get a few package downloads followed by a couple of errors see attached log.
2019-05-22T14_12_04_887Z-debug.log

I'm same problem.. go to other template

you can try installing node-pre-gyp with sudo and --unsafe-perm
sudo npm install  node-gyp
or
sudo npm install  node-gyp  --unsafe-perm

Any Update on this, Same issue!


create new profile rinshin jalal

Added a new profile!!

It looks like you have used npm install instead of npm ci from the docs, this causes the generated files to be updated but these should not be committed without specific changes to them.
https://linkfree.eddiehub.io/docs/environments/local-development
Please remove these changes with:

git checkout main -- package.json
git checkout main -- package-lock.json

It is always best to check file changed with git status -s before doing a commit and never use . in your git commands as this will include files you may not want.
Any questions come and ask in the EddieHub Discord http://discord.eddiehub.org

Do you need any help with the requested changes @Rinshin-Jalal ?

I will fix in a temporary branch


Release a new version of bytes

Could you make a new bytes release and bump mio's bytes dependency? The buffers that mio re-exports still exhibit the bugs fixed in c4546493eaad57e69e37aa4205a08000bcf82153

Done. Thanks.
Bytes: https://crates.io/crates/bytes/0.2.7
Mio: https://crates.io/crates/mio/0.3.3

Great, thank you! :smiley:


Add timeout to container.exec

Issue

For some reason, alertmanager and prometheus pods are getting constantly restarted and we couldnt figured out why.
Versions:
Juju 3.1.6
Alertmanager latest/edge 98
Prometheus latest/edge 170
Solution

I was not able to put my finger on it but this PR just adds timeout to the container.exec calls to make sure that this is not leaking and making pebble unresponsive.
Context

Pebble
Testing Instructions

No instructions. It happens randomly.
Release Notes

Thanks @amandahla and sorry for the long delay.

Do you still find the timeout useful?
Could 3 sec be too short?
Iiuc, a timeout would put the charm in error state. Not sure which approach is best.

This has been stale for more than six months, so I will go ahead and close this. If you still think this is relevant, please open a new PR and answer the question stated in the last comment above.


[improve] update grafana auth method and add expose url

What's changed?

update api auth method

for some maybe use xxx@xxx as password, it will work failed when pre auth method.

add expose url param

for some env the inner access grafana server url and extern access url may not same.

Checklist

[ ]  I have read the Contributing Guide
[ ]  I have written the necessary doc or comment.
[ ]  I have added the necessary unit tests and all cases have passed.

Add or update API

[ ] I have added the necessary e2e tests and all cases have passed.

Codecov Report
Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 32.55%. Comparing base (9a00f5a) to head (8e05694).

Files with missing lines
Patch %
Lines

.../history/vm/VictoriaMetricsClusterDataStorage.java
0.00%
3 Missing :warning:

...e/store/history/vm/VictoriaMetricsDataStorage.java
0.00%
3 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2818      +/-   ##
============================================
- Coverage     32.56%   32.55%   -0.02%
  Complexity     1903     1903
============================================
  Files           496      496
  Lines         18310    18314       +4
  Branches       2249     2251       +2
============================================
- Hits           5963     5962       -1
- Misses        11651    11654       +3
- Partials        696      698       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


bot not working

I invited it to my server and typed pm!help but nothing happens. Discord says it's online.

sad because this seems like a very good poll bot

Same here. 😞

@dullbananas did you verify the bot has permissions to read and send in the channel you are sending the commands?

If the bot does not respond at all, it usually lacks the permissions to read your message or answer.
As long as you can see the bot in the user list type @Pollmaster debug to see if your permissions are correct.
Closing this for now.


Added local segment validation to email validator

Manually handling exceptions caused by invalid local segments of email addresses is always a hassle, o I figured it made sense to extend this library to cover the full gamut of requirements outlined in RFC-5322, specifically sections 3.2.3 and 3.2.4. I specifically eschewed using any regular expressions per this library's documentation.
For a more accessible guide to the rules I'm matching check out this discussion.
Thanks for your consideration! Let me know if there's anything you think should be tweaked!

Hey @hallelujah, is this in line with what you'd like to see for this project? Or is there anything I can do to make it a better fit?

Sorry I am a little out of business now. But will come back soon.
Just merged the PR. OK for me

Awesome, thanks! I really appreciate you taking a look!
Have a nice day!

It would be nice to document his change in a UPDATE.md since this could change the validity of models already stored in the database.


NSMBe5 has stopped working

So, I got NSMBe5 a little while back and now for some odd reason, it doesn't open. Whenever i try opening it, a popup appears saying "NSMBe5 has stopped working, we're sending crash details to Microsoft". I kept deleting and re-downloading NSMBe5 but the same thing happened. I then proceeded to restart my pc and the same outcome came. I have no clue what to do. Please help.

wrong place

Idk bro have you tried resetting your internet router? If so, maybe a fresh windows 10 install will help? As a last resort, try deleting System 32, it might have something that crashes your program


LuAnimator-patch

Edited LuAnimator.json for better usage

What does this do other than reformatting the file and setting twoHanded to false (which I did because without it the alignment was wrong when flipping your character)?
Scripts is misspelled, by the way.

Actually, it doesn't change "twoHanded" parameter
The thing it does is disabling the scripts and setting allowFlip to true


DPI Fixes.

Added proper DPI manifests that work on Windows 10 and Windows 7.
Fixed a bug where the WPF control would render at 25% of the size at 200% DPI scaling. Fixes #137.
Initialize the CoreWebView2Environment explicitly to eliminate some reflection.

Works here on Surface Book 2 @ 200% DPI and also on external monitor @ 200% DPI.


Error Code: GENERIC_VERIFY_FAIL

In Ubuntu-18.04.3 Tor Browser will install.  However, the verification key will not download.  So, Tor can not run...or browse.  Any method of acquiring the key will be an asset.  Please give generously.

Confirmed. Fixed my problem as well


Remove: Unity.Unity.6000.LinuxServer version 6000.0.13f1

Microsoft Reviewers: Open in CodeFlow

URL: https://download.unity3d.com/download_unity/53a692e3fca9/TargetSupportInstaller/UnitySetup-Linux-Server-Support-for-Editor-6000.0.13f1.exe
OK
(Automated message - build 923)

https://github.com/microsoft/winget-pkgs/pull/186121#issuecomment-2445759087

[Policy] Needs-Manual-Merge


Update extending validation error messages docs

Closes #1740

Coverage remained the same at 82.274% when pulling 805b8139e13741867a24826f846291dc79c7489b on nickbabcock:validation-docs into d687cdce03198ce54bed98c19524ee361852d290 on dropwizard:master.


Option to create the bucket full

Hello.
The reason I need is solely for testing sake, so I'll welcome any other suggestion for how to elegantly create a TokenBucket "ready to spill" without waiting for it or ugly monkeypatching. Still, I think it's a valuable functionality, and it maintains previous behavior.
Feel free to modify this PR of course.

We need it also for real life - we are creating a token bucket on demand (per user), and want the user to be able to work with it immediately.
This is similar to https://github.com/jhurliman/node-rate-limiter/issues/18, which propose a different approach which I think is more generic.

We need it also for real life - we are creating a token bucket on demand (per user), and want the user to be able to work with it immediately.
This is similar to https://github.com/jhurliman/node-rate-limiter/issues/18, which propose a different approach which I think is more generic.

BTW, I now see that event the code of RateLimiter is manually filling the TokenBucket after creating it (in https://github.com/jhurliman/node-rate-limiter/blob/master/lib/rateLimiter.js#L20-L21).
I guess this means that this functionality should be exposed.
@jhurliman, @SirR4T what do you say?
Maybe make the new option part of an "options" argument of the constructor to allow additional options in the future?
Or, just expose the "content" member in the Typescript?

BTW, I now see that event the code of RateLimiter is manually filling the TokenBucket after creating it (in https://github.com/jhurliman/node-rate-limiter/blob/master/lib/rateLimiter.js#L20-L21).
I guess this means that this functionality should be exposed.
@jhurliman, @SirR4T what do you say?
Maybe make the new option part of an "options" argument of the constructor to allow additional options in the future?
Or, just expose the "content" member in the Typescript?

@yoshigev I am leaning toward the second option of exposing content in the Typescript defs.

@yoshigev I am leaning toward the second option of exposing content in the Typescript defs.

content is now exposed in the 2.0.0 rewrite


Fix and add tests for IPv4 and IPv6 parsing from proxy X-Forwarded-For headers

Note: This patch does not fix IPv6 parsing with ports
IP parsing is broken because of a broken past IIS patch for port resolving, and does not function correctly on e.g. AWS Elastic Beanstalk.
Standard formatted IPv4 addresses are reduced to empty string in current version:
$ python
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ip = '<IP_ADDRESS>'
>>> ip = ''.join(ip.split(':')[:-1])
>>> ip
''

The correctly functioning port parsing is of format:
>>> ip = '<IP_ADDRESS>'
>>> ip = ''.join(ip.split(':', 1)[0])
>>> ip
'<IP_ADDRESS>'

This behaviour seems to be because of duplicate and conflicting patch entires for the get_ip(request) function.
This patch fixes IP parsing for normally formatted IPv4 addresses with ports, and IPv6 addresses without ports. (It does not mutilate IPv6 addresses, but keeps the current functionality). It cleans up the internal implementation of get_ip and adds a test set for the get_ip(request) function.
To accommodate the changes, test runner setup had to be refactored, and different settings added for axes setups running behind proxies. This only affects internal APIs and does not modify existing functionality.
This patchset should fix #186 and #206 and can be merged with #171.

Coverage increased (+2.3%) to 88.581% when pulling 41877cdecd400231c7642e511ace194ecea8a76f on Vincit:master into ef3d527bee81e10455f73e57aaea13a222c9b792 on jazzband:master.

Thank you

Was it really necessary to remove fix for IPv6 parsing with ports? IIS still uses such address in the X-Forwarded-For header. Fixing one trouble while removing the useful functionality.

Well, not absolutely necessary, but it did fix a honest bug for more people than it broke things for.
If you wish to have the IIS header parser in upstream and new releases, we can merge PRs and make new releases rather quickly.
Thank you for your understanding and appreciation for the voluntary maintenance and R&D that has gone into this project.


Update lib/em-dir-watcher.rb

Deprecated 'Config' replaced with 'RbConfig' to avoid rake error during 'bundler' installation

This is identical to #11.


feat(nifs): compute_F

fn compute_F(
    beta: F,
    delta: F,
    w: PlonkWitness,
    S: PlonkStructure // S.gates
) -> UnivariatePoly<F> {
   // evaluate over t+1 points
   // do ifft to get coefficient form
   // 1. Take gates, eval for witness to get f_i(w)
   // 2. Calc pow_i(\beta + X * \delta) over X={1,w',w'^2,...}
   // 3. pow_i()*f_i(w) over X={1,w',w'^2,....}
   // 4. ifft
   todo!()
}

$F(X)=\sum_{i=0}^{n-1}pow_{i}(\boldsymbol{\beta}+X\cdot\boldsymbol{\delta})f_i(w)$

Define an internal function to calculate this with complexity $O(n)$:
$$\sum_{i=0}^{n-1}pow_{i}(\beta_1,\cdots,\beta_t)v_i$$
Here the $(\beta_1,\cdots,\beta_t)$ can be of the form $(\beta_1+\alpha_i\delta_1,\cdots,\beta_t+\alpha_i\delta_t)$ for different points $\alpha_i$ in case of compute_F. Or it can be of the form $(\beta_1+\alpha\delta_1,\cdots,\beta_t+\alpha\delta_t)$ for some fixed $\alpha$.
 // suppose  n = values.len()
// t = beta.len() with t = log(n)
 // this algorithm gives O(n) complexity
//  values = {v_0,...,v_{n-1}}
fn random_linear_combine(beta: Vec<F>, values: Vec<F>) -> F {

}

Define an internal function to evaluate all $v_i$ from $f_i$.
Suppose there are $m$ gates, and $n$ rows. We convert it to a size $m\cdot n$ vector $(f_i)_{i=0}^{m\cdot n-1}$. Then we evaluate them f_i(witness)=v_i to become the valuation vector  {v_i}.
And we need to do this for all the instance to be folded.

Define a new data evaluation domain instead of PlonkEvalDomain, since we don't have cross term anymore:
pub struct ProtogalaxyEvalDomain {
            num_advice,
            num_lookup,
            selectors,
            fixed,
            challenges,
            W,
}

Define a new data evaluation domain instead of PlonkEvalDomain, since we don't have cross term anymore:

The new domain will repeat the old one almost completely. It would be better to generalize the domains to a single type later, simplifying index handling. However, it's not a priority right now, so I'll just fill the W2s with a blank, which won't affect the result. Let me know if I've overlooked something here

Within the framework of the task (especially the part that evaluate_witness) we need a real circuit for testing and evaluating the performance of the solution. The point is that the approach to counting and parallelization of calculations is very different depending on performance.
That's why at this stage I plan to create a separate repository where I will copy real circuits from other repositories (they are not imported normally) and use them in tests


Update UT entry point to correspond with OSAL/UT Assert change

Describe the contribution
Fix #435
Use UtTest_Setup instead of OS_Application_Startup as the entry point function for test applications.
This needs to be merged in coordination with a related change in OSAL/UT Assert.  This also updates the UT application final link to use "ut_assert" rather than "ut_bsp", as the separate bsp for unit test is also getting phased out.
Testing performed
Build CFE for all supported platforms (native Linux, RTEMS, VxWorks) in conjunction with related OSAL pull request and confirm unit test build and run successfully
Expected behavior changes
No change to behavior
System(s) tested on:
Ubuntu 18.04 LTS 64 bit
Additional context
Needs to be merged in coordination with nasa/osal#294
Contributor Info
Joseph Hickey, Vantage Systems, Inc.
Community contributors
You must attach a signed CLA (required for acceptance) or reference one already submitted

CCB 20190108 - Reviewed and approved for integration candidate


does leveling up work? or does it also require a checksumm?

Hi!
does crew leveling work? or does it also require a checksumm?

And does crew training work?

@nIKsL both crew training and crew leveling works.
I plan to refactor the code to use a unique identifier and automatically move the crew to their designated rooms if their training is complete or to a training room because no checksums are required for those endpoints. I have support for all the training rooms in the manageTraining method.
Crew leveling works but my implementation is too bruteforce. It attempts to upgrade each crew that's not level 40 even if the crew doesn't have enough exp for an upgrade. I have not figured out how to determine if the crew has enough exp for an upgrade. Therefore, I didn't use that method because I didn't want to produce so much necessary next traffic.
I run my script using a GitHub action at an hour interval.
By the year, this codebase will have significant improvements.

which class method is responsible for training?
or did you remove it from the code?
I also run the script locally via cron every hour :)
In another script I implemented auto-battles via pyautogui - I’m working with the game client.

The client.manageTraining method on line 660 is the method responsible for
crew training.
In sdk/client.py:
Line 687 - 761 has a static definition for crew training.
On Sat, Oct 21, 2023 at 6:47 AM nIKsL @.***> wrote:

which class method is responsible for training?
or did you remove it from the code?
I also run the script locally via cron every hour :)
In another script I implemented auto-battles via pyautogui - I’m working
with the game client.
—
Reply to this email directly, view it on GitHub
https://github.com/raelldottin/tachikoma/issues/58#issuecomment-1773750124,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACNMV6IAV2LUH7URSDFHG3YAOR25AVCNFSM6AAAAAA6J6XNO6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTG42TAMJSGQ
.
You are receiving this because you commented.Message ID:
@.***>

^ I mean class method for level up, not about training

By the way, in the laboratory there is a study of rockets, which is not shown in the game client. But it is launched via API.
Its ID is 5 or something like that (can be seen from alldesign xml request)
I advise you to add it to the exception in the study.

client.upgradeCharacter
On Sat, Oct 21, 2023 at 7:09 AM nIKsL @.***> wrote:

^ I mean class method for level up, not about training
—
Reply to this email directly, view it on GitHub
https://github.com/raelldottin/tachikoma/issues/58#issuecomment-1773758106,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACNMVYRFCIAHHLP3B57FULYAOUQJAVCNFSM6AAAAAA6J6XNO6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTG42TQMJQGY
.
You are receiving this because you commented.Message ID:
@.***>

Ok, thank you. You can always submit a PR.
On Sat, Oct 21, 2023 at 7:19 AM nIKsL @.***> wrote:

By the way, in the laboratory there is a study of rockets, which is not
shown in the game client. But it is launched via API.
Its ID is 5 or something like that (can be seen from alldesign xml request)
I advise you to add it to the exception in the study.
—
Reply to this email directly, view it on GitHub
https://github.com/raelldottin/tachikoma/issues/58#issuecomment-1773760317,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACNMVYEKWTYDFYRGS4YCY3YAOVS7AVCNFSM6AAAAAA6J6XNO6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTG43DAMZRG4
.
You are receiving this because you commented.Message ID:
@.***>


Support IRI as DTDL type

The following DTDL document produces two errors in the VSCode extension, although it's accepted by the parser
{
  "@context": "dtmi:dtdl:context;2",
  "@id": "dtmi:com:example:myThing;2",
  "@type": "Interface",
  "displayName": "myThing",
  "contents": [
    {
      "@id": "dtmi:com:example:myThing:temperature;1",
      "@type": "Telemetry",
      "name": "temperature",
      "dtmi:dtdl:property:schema;2": "double"
    }
  ]
}

Problems reported
[{
	"resource": "/C:/code/temp/scratch-repo/dtmi/com/example/mything-2.json",
	"owner": "_generated_diagnostic_collection_name_#0",
	"severity": 8,
	"message": "Miss required properties: schema",
	"startLineNumber": 7,
	"startColumn": 5,
	"endLineNumber": 7,
	"endColumn": 5
},{
	"resource": "/C:/code/temp/scratch-repo/dtmi/com/example/mything-2.json",
	"owner": "_generated_diagnostic_collection_name_#0",
	"severity": 8,
	"message": "dtmi:dtdl:property:schema;2 is not a valid property.",
	"startLineNumber": 11,
	"startColumn": 7,
	"endLineNumber": 11,
	"endColumn": 36
}]

Impacting https://github.com/microsoft/vscode-dtdl/issues/94


Fix the broken footer social link buttons

I would like to fix the broken buttons in the footer s shown below:

@Anupkjha2601  sir kindly assign me this issue


About "the following command to start the test cluster"

What happened:
When I executive this shell:
bash hack/local-up-karmada.sh

My server of karmada-apiserver will be forced to change to 127:0:0:1 in karmada.config. But my local server of  karmada-host-control-plane is <IP_ADDRESS>. It's queried by the shell like this:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'  "CONTAINER ID of karmada-host-control-plane"

So that, I unable to execute the shell:
kubectl apply -f artifacts/dashboard/karmada-dashboard-sa.yaml

The error will be located at:
[2024-07-20 20:47:50][INFO  ][main:299] Create namespace on karmada-apiserver
error: error validating "/var/folders/rw/254jjhsx2758fk20v9btxgcm0000gn/T/tmp.19wJcCm5ry/deploy/namespace.yaml": error validating data: failed to download openapi: Get "https://<IP_ADDRESS>:5443/openapi/v2?timeout=32s": dial tcp <IP_ADDRESS>:5443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
[2024-07-20 20:47:50][INFO  ][main:300]

What you expected to happen:
I expected the command can be optimized to fit the local environment.
How to reproduce it (as minimally and precisely as possible):
If we ignore the error and continue, this situation will occur after "apply"
base) yuhowmean@192 dashboard % kubectl apply -f artifacts/dashboard/karmada-dashboard-sa.yaml
clusterrolebinding.rbac.authorization.k8s.io/karmada-dashboard created
Error from server (NotFound): error when creating "artifacts/dashboard/karmada-dashboard-sa.yaml": namespaces "karmada-system" not found
Error from server (NotFound): error when creating "artifacts/dashboard/karmada-dashboard-sa.yaml": namespaces "karmada-system" not found

Anything else we need to know?:
If we create namespace of karmada-system manually, like this:
kubectl create namespace karmada-system

The problem that the token is empty will occur, like this:
(base) yuhowmean@OminMacBook-Pro ~ % kubectl -n karmada-system get secret/karmada-dashboard-secret -ojsonpath='{.data.token}' | base64 -d && echo

(base) yuhowmean@OminMacBook-Pro ~ %

And the number of DATA is 0.
Environment:

Karmada version:The latest version
Karmada Dashboard version:The latest
Others:

It seems that the process of starting  karmada test environment broken, the karmada-system namespace did not be created correctly. After that you create karmada-system namespace manually. I'll try to recurrent the strange problem. @chouchongYHMing and you can try start the test environment by bash local-up-karmada.sh ${you host ip address}.


Overhaul website content for proc resource

Reordered lots of content for better legibility. Reduced overall content substantially. This was for a few reasons, but primarily because:

Much of the copy dated to Q4 2021 and was out of date or no longer relevant
The copy that was here was mostly copied from the white paper without much attention paid to organization or easy navigability on the web
We are hoping to direct more traffic to the site with the imminent launch of our procurement resource so it seemed like a good time for a clean up

These changes lgtm. I'm a little hesitant on the removal of the process page but ultimately I think taking it out is right, until we have the staff to commit to substantive ongoing updates and stick to what's committed to. The history information about launch is no longer needed.


Migrations: Replace usage of BasicModelBuilder with direct calls to core metadata

Currently model snapshots get scaffolded as calls to BasicModelBuilder, but the latter is going away as described in #908.
Although most scenarios we originally envisioned for BasicModelBuilder can be handled by a "regular" ModelBuilder without the conventions, after discussing with @bricelam and @ajcvickers we agreed that the ModelBuilder API is not the best choice for model snapshots.
Instead we would like to use direct calls to the core metadata API.
We were considering a JSON representation of the model, described in #1355, but we believe that now should be moved to the backlog.

@smitpatel please figure out with @bricelam how to best integrate this change with his work on migrations.

@smitpatel I wrote a blog post, C# Object Notation, on some language features we can use to try and make the format as terse as possible.


[CI] simplify workflow, remove matrix

It seems like GH-actions is not reliable enough for the previous setup.
Hopefully this simplification improves it.

why does macOS take foreverrrrrrrrrrrr


Add a new Qos operator for acks, and gated acknowledgement continuations

Interceptor for manually handling of circuit breaker acknowledgments in a custom tremor-script function/callback.

Actually part of #517


version L5.3 dependency update

Bumping compatibility to Laravel 5.3. I've been using this in a local project and it seems like it works just fine without any other changes.

Added support for laravel 5.* on version 3.*


fetchPackageMetaData error for @material-ui/core@latest zlib: unexpected end of file

npm install @material-ui/core fails with zlib: unexpected end of file
I've tried in both an existing react project and a new one and same result.

[ x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior

Install succeeds.
Current Behavior

Install fails on:
silly fetchPackageMetaData error for @material-ui/core@latest zlib: unexpected end of file
10 timing stage:rollbackFailedOptional Completed in 1ms
11 timing stage:runTopLevelLifecycles Completed in 13519ms
12 verbose stack ZlibError: zlib: unexpected end of file
12 verbose stack     at Zlib.(anonymous function).onerror (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:131:21)
12 verbose stack     at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:255:31)
12 verbose stack     at Unzip.flush (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:222:10)
12 verbose stack     at Unzip.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:229:10)
12 verbose stack     at Object.end (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:415:21)
12 verbose stack     at PassThrough.onend (_stream_readable.js:595:10)
12 verbose stack     at Object.onceWrapper (events.js:313:30)
12 verbose stack     at emitNone (events.js:111:20)
12 verbose stack     at PassThrough.emit (events.js:208:7)
12 verbose stack     at endReadableNT (_stream_readable.js:1064:12)
12 verbose stack     at _combinedTickCallback (internal/process/next_tick.js:138:11)
12 verbose stack     at process._tickCallback (internal/process/next_tick.js:180:9)
Steps to Reproduce (for bugs)

Context

Your Environment

Tech
Version

Material-UI
1.0.0-rc.0

React
16.3.2

browser

etc

I was able to reproduce by:
npx create-react-app mui-test
cd mui-test
npm install @material-ui/core
Regards,
Scott Humphrey

On 16 May 2018, at 15:32, Olivier Tassinari<EMAIL_ADDRESS>wrote:
What's the reproduction processus?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

--
Whizz Education Limited is registered in England No. 4494788 and its
registered office is Macmillan House, Paddington Station, London W2 1FT.
The information in this Email and any attachments is confidential and is
intended solely for the addressee. If an addressing or transmission error
has misdirected this Email, please notify the author.

@whizz-shumphrey Thanks, it's working in my environment. I'm assuming it's an issue with zlib and not Material-UI:
``
$ node --version
v9.10.0
$ npm --version
5.6.0


2b. verify  if results from sphere to mesh is the same that results computed directly on sphere

The idea is to apply spangy directly onto a sphere where the curvature information has been mapped. There are tons of methods to map a mesh onto a sphere but ywe could use the one in Lefevre & Auzias, 2015 that can be implemented rapidly in slam
https://hal.archives-ouvertes.fr/hal-01222933/file/74-Lefevre.pdf

Two conclusions:

It is not easy to compare the numerical eigenvectors with the analytic ones because the eigenvalues are not unique (the eigenspace is degenerated).
But we could use the inverse mapping to back-project the spherical harmonics onto the brain mesh and check whether
a. the functions are (almost) orthogonal and
b. they allow to obtain comparable results when looking at the spectral bands.


Suggestion:  Chat scanner

🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅🔅-->
🚀 Chat Scanner
Describe the solution you'd like
A chat scanner like mercury trade. You put filters and when somebody write it on the chat an alert apear. For example, if you put hello and somedoby write it, an alert apear, but if write Hello, !not bye, if somebody write "Hello bye" the alert dont apear.
When the alert apear you can click in some button like trade display and can whisper that player that wrote it.
Its the only tool that i use now on that app, for my is so necesary to find groups in global 820
Additional context

Would be a nice thing to have

yes, chat scanner would be really nice

would be a great feature

Scanner is the thing that I really miss.

Scanner is the thing that I really miss.


Problem loading file in MSBuild workspace

[X] Include the log file upgrade-assistant.clef that is produced in the working directory

Describe the bug
When performing an analysis, all the projects in my solution generate the following error:
[15:29:48 DBG] [Failure] Problem loading file in MSBuild workspace Msbuild failed when processing the file 'C:\temp\UpgradeTest\UpgradeTest.csproj' with message: C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\Microsoft.NET.Build.Extensions.ConflictResolution.targets: (30, 5): The "ResolvePackageFileConflicts" task could not be loaded from the assembly C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\\tools\net6.0\Microsoft.NET.Build.Extensions.Tasks.dll. Could not load file or assembly 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\tools\net6.0\Microsoft.NET.Build.Extensions.Tasks.dll'. The system cannot find the path specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask.

I believe this is due to the fact that because of the version of msbuild being used it is looking for some assemblies under Microsoft.NET.Build.Extensions that do not exist.
To Reproduce
UpgradeTest.zip
Further technical details
Processor	Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz   2.11 GHz
Installed RAM	16.0 GB (15.7 GB usable)
System type	64-bit operating system, x64-based processor
Edition	Windows 10 Pro
Version	20H2
Installed on	10/26/2020
OS build	19042.1526

Include the output of dotnet --list-sdks
5.0.404 [C:\Program Files\dotnet\sdk]
6.0.201 [C:\Program Files\dotnet\sdk]

Include the output of upgrade-assistant --version
0.3.310801+057ffba0656270dc5d83c015748ac56daed55886

Visual Studio
Microsoft Visual Studio Enterprise 2022
Version 17.1.2
VisualStudio.17.Release/17.1.2+32319.34
Microsoft .NET Framework
Version 4.8.04084

Installed Version: Enterprise

Hi @skushubar,
Did you ever find a solution to this problem? I'm experiencing the same issue.

Any update on this issue?
I am getting the exact same error with all projects I run analyze against.
If i specify --msbuild-path, and indicate the folder containing msbuild, I get the error below and the analyze process stops:
[15:17:41 ERR] No MSBuild instance was found at C:\Program Files\dotnet\sdk\6.0.202
[15:17:41 ERR] Exception of type 'Microsoft.DotNet.UpgradeAssistant.UpgradeException' was thrown.
If I do not specify an --msbuild-path, the upgrade tool finds all SDKs installed and throws the same error as "skushubar" gets, but the analyze process continues.

For some reason, --msbuild-path must end in \\ and --vs-path must not end in \. Try adding \\.

I have VS 2022 and .NET SDK 6.0.301 installed. I get the following error when I run upgrade-assistant analyze MyApp.sln:
[14:37:16 INF] Using MSBuild from C:\Program Files\dotnet\sdk\6.0.301\
[14:37:16 INF] Using Visual Studio install from C:\Program Files\Microsoft Visual Studio\2022\Professional [v17]
...
[14:37:17 DBG] Unable to resolve assembly Microsoft.Build.resources, Version=<IP_ADDRESS>, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a
...
[14:37:18 DBG] Unable to resolve assembly Microsoft.Build.Tasks.Core.resources, Version=<IP_ADDRESS>, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a
...
[14:37:18 DBG] [Failure] Problem loading file in MSBuild workspace Msbuild failed when processing the file 'C:\Users\gordon.tyler\dev\MyApp\MyApp.csproj' with message: C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\Microsoft.NET.Build.Extensions.ConflictResolution.targets: (30, 5): The "ResolvePackageFileConflicts" task could not be loaded from the assembly C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\\tools\net6.0\Microsoft.NET.Build.Extensions.Tasks.dll. Could not load file or assembly 'C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\tools\net6.0\Microsoft.NET.Build.Extensions.Tasks.dll'. The system cannot find the path specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask.

The DLL that it's looking for doesn't exist in the VS2022 install but it does exist in the dotnet 6 sdk folder.

Getting similar issue, is there a work around

Fixed in new versions of UA CLI.


More interactions needed

There are a couple of non-functioning buttons at the top of the screen. The cat also suggests playing a game as a way to help the user.
It would be good to have the 'stroke' interaction and maybe one other implemented for this prototype.

On second thoughts, the easiest thing might be to have the cat talk to the user after stroking, feeding etc to make it more interactive. And developing proper interactions would be for the roadmap.

Closing this as #25 captures sufficient interactions for now.


Fixed calculations for stroking one pixel thick paths.

This fixes issue #1 by correctly doing the calculations for the paths. As a "side-effect", the intrinsicContentSize hack is not needed any more. The rendering in retina is now exactly one on-screen pixel higher and wider (which is also a fix in this case: the right-most line was blurry).

Non-retina: before

Non-retina: after

Retina: before (notice blurry right-most line; you may need to click on the image for full size)

Retina: after

I was one pixel off on the right side of the color swatch. Fixed in 32879004aaa17ed024c0a612ee43db0e060be6b7.

Nice work, thanks very much! I will take a closer look and then merge.


Validate remote source endpoint and make permissions consistent

Fixes https://github.com/BlueBrain/nexus/issues/2212
Fixes https://github.com/BlueBrain/nexus/issues/2214


does it support hdpi? font size looks so much different when i move between screens

my laptop is 14-inch with a 27-inch monitor
it looks so fine when both in all-screen mode，but not when moving between screens
is there anyway to solve it (maybe by settings that i haven't noticed?)

Try the latest update, just released now!
If you installed it, you can manually switch to different display resolutions in top menu Options > Display.
-TT


Benchmark integration main

Description

Changes to integrate with agbenchmark
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Docs update

Checklist

[x] My pull request is atomic and focuses on a single change.
[ ] I have read the contributing guide and my code conforms to the guidelines.
[ ] I have documented my changes clearly and comprehensively.
[ ] I have added the required tests.

Unit Tests are failing too, please check for it


Improper cursor placement on multi-hierarchical bullet lists

When you add a second level of bullets, the cursor starts at the beginning of the line instead of after the bullet.  Only after you begin typing again will the cursor snap back into place.
https://user-images.githubusercontent.com/37011672/204637114-5601d515-2247-44ef-a1b0-3fc6bde67c73.mp4

I tried to reproduce this in the example app on Mac and web but didn't notice any issue with caret placement.
@transcendr can you confirm whether this is still a problem on main?

This seems to be a duplicate of https://github.com/superlistapp/super_editor/issues/904 - closing in favor of that one.


Azure OpenAI: set header in order to avoid adding multiple Authorization headers on retries

Fix https://github.com/Azure/azure-sdk-for-net/issues/46109
Thanks @mikeharder  for https://github.com/Azure/azure-sdk-for-net/issues/46109#issuecomment-2389642980!

API change check
API changes are not detected in this pull request.

Can we add a test for this?  Mock the service returning a 429, and ensure duplicate headers aren't sent on retry?


Blog post for new functionality

The report command is more awesome, the network visualization shows lot's more things.  Announce all these in a blog post.

Closing due to staleness.


Logout is not working after user sign in

Description
Logout is not working when the user clicks on Logout button.
OBS: If the user reloads the page and clicks the Logout button a few times, the logout works
Steps to Reproduce

Access Matrix login page
Sign in Matrix using a google account
Once you have redirected to the office page, click the Logout button in the navbar

Expected Behavior

Your account must be disconnected and you must be redirected to the login page.

Current Behavior

An error occurs and logout is not working

I'm gonna try to solve it!


Text input on top of bootstrap-slider

Are you requesting a feature, reporting a bug or asking a question?
Asking a question
What is the current behavior?
A slider shows which is from "bootstrap-slider" lib
What is the expected behavior?
Implement input text on top of it that copies the value of slider into it

surveyjs platform (angular or react or jquery or knockout or vue): React
surveyjs version: Latest

Details
I would like to implement an input on top of the slider that changes according to the slider and according to user input, how can I achieve this? Please check out the pictures below to know exactly what I want:
Without Input:

With Input:

PS: This is only for demonstration/design purpose, I will not use that input for any kind of UX functionality, I'm already getting the result from the slider.
Thanks!

You can create a custom widget implementing desired design and behavior.
You can take this file - https://github.com/surveyjs/widgets/blob/master/src/bootstrap-slider.js as a strating point

Ok so I think I can change this part:
slider.on("change", function(valueObj) {
    question.value = slider.getValue();
});

to something like:
slider.on("change", function(valueObj) {
question.value = slider.getValue();
customInput.value = slider.getValue();
});
in the node_modules folder, right?

@CedricHadjian hello! I think that changing some module in the node_modules folder is bad practice (because of npm i or npm update will rewrite it). So you can create a separate file with your own custom widget based on file that @tsv2013 suggested above. Also please check the "tutorial-example": https://plnkr.co/edit/HdnYE5?p=preview.

@CedricHadjian hello! I think that changing some module in the node_modules folder is bad practice (because of npm i or npm update will rewrite it). So you can create a separate file with your own custom widget based on file that @tsv2013 suggested above. Also please check the "tutorial-example": https://plnkr.co/edit/HdnYE5?p=preview.

Ok, I will. Thanks Dmitry!

@CedricHadjian please feel free to ask additional questions or post the examples.

I want something exactly like this but in react and surveyjs: http://jsfiddle.net/FPCRb/2771/
Having trouble achieving it

Been working on creating the custom file, how will I exactly require it inside widgets function?

the following code do the trick:
function init(Survey, $) {
    const widget = {...};
    Survey.CustomWidgetCollection.Instance.addCustomWidget(widget, "customtype");
}
export default init;

then you can use it something like:
import * as Survey from "survey-react";
import initCustomSlider from "./somepath/customslider.js";
...
initCustomSlider(Survey);
...

also you can check our "nouislider": https://github.com/surveyjs/widgets/blob/master/src/nouislider.js
https://surveyjs.io/Examples/Library/?id=custom-widget-nouislider&platform=Knockoutjs&theme=default

Been working on creating the custom file, how will I exactly require it inside widgets function?

the following code do the trick:
function init(Survey, $) {
    const widget = {...};
    Survey.CustomWidgetCollection.Instance.addCustomWidget(widget, "customtype");
}
export default init;

then you can use it something like:
import * as Survey from "survey-react";
import initCustomSlider from "./somepath/customslider.js";
...
initCustomSlider(Survey);
...

also you can check our "nouislider": https://github.com/surveyjs/widgets/blob/master/src/nouislider.js
https://surveyjs.io/Examples/Library/?id=custom-widget-nouislider&platform=Knockoutjs&theme=default

Thank you!! Will try it now.
I checked nouislider before, not something I exactly want.

Perfect! Working great :)
Thank you for the support @dmitrykurmanov !


Incorrect Exponent Handling for Integers in SignificantNumber Conversion

Issue:
The bug affected the handling of integers when converting them to significant numbers. Specifically, the logic for trimming trailing zeros did not correctly adjust the exponent value for integers, leading to incorrect representation of the significant digits and exponent values.
Symptom:
When converting integers or decimal integers with trailing zeros, the resulting Significand and Exponent values were inaccurate. For example, converting 3600 resulted in incorrect significant digits and exponent values.

Fix:

Introduced a check to determine if the input number is an integer.
Adjusted the trimming logic for trailing zeros to correctly update the exponent value for integers, ensuring accurate representation and maintaining significant digits.

Verification:
Added new unit tests (TestDecimalInteger and TestInteger) to confirm the correct handling and conversion of integers and decimal integers, ensuring that the Significand, Exponent, and SignificantDigits are accurately represented.


Font not used when exporting to png, svg

Bug report
CODEPEN
(https://codepen.io/anaschamkhi/pen/mdjXGoa)
Explanation
the problem here im using a different font-family for the chart it works in page view, but when i download PNG,SVG the font family change .
What is happening instead?
Font gets replaced with a generic one
What error message are you getting?
None

I guest I have a clue. I tried to export with Montserrat font and it didn't work but with Arial it's ok.  So the problem come from the font that the export can use I guess.

Any updates on this issue?
We've just encountered this bug in a fairly new, major project at work and are considering switching to a different charting library, if we cannot get accurate exports from ApexCharts.

hello everyone.
I figured that it's not a bug. everything works correct !
I checked exported SVG font-family in browser and it turned out, every font you use, will be applied without any problem.
you just have to install the font that you are using for your charts in your system to see changes when you export. (either PNG/SVG).
note that if you change the font name that is different from your installed system fonts, it won't be applied.
example: I used 'Shabnam-Farsi' font for my charts but in my system its installed as 'Shabnam FD'.
so all I had to do was to rename my font in my back-end and done!
wish you luck.

Thanks for the tip, @Arc-T! You're correct that the SVG exports with the correct font family.
For my use case, there are only two problems left, one being that ApexCharts offers no officially documented way to simply get the SVG string, it only offers a function to trigger a download of the SVG, which isn't ideal, if you want to do anything with the SVG.
In my case, I need to send the SVG to the backend, so it can be added to a PDF file.
Fortunately, this issue can be worked around fairly easily by getting ahold of the chart object and calling chart.exports.cleanup() followed by chart.exports.getSvgString(), which is what happens internally when you use the download SVG function.
The other issue is a major one: the legend is exported as a <foreignObject> containing HTML within the SVG, meaning common SVG rendering libraries can't render it.
So while the legend is part of the SVG, it's simply not rendered in the PDF I'm creating, making the charts largely useless, as nobody can tell what's what.
I haven't found a workaround for this yet. I've tried grabbing the HTML manually and throwing it into some online HTML to SVG and HTML to PDF converters, but they fail to render it correctly.
At this point, I think the only way to deal with this is to write a custom converter to convert the embedded HTML to a new SVG and layer that SVG on top of the original, which will be a major pain.


Support SPM

supporting Swift Package Manager in 2020 is must have

also does this project support iOS < 13?

@uuttff8
Thanks. I think so too.
However, I'm not sure how we could do this with Objective-C based library (Texture).
If you have any idea, could you submit the PR to get this?

hmm seems like Texture does not support SPM, so it's not possible for this project.

I'd still like to see this project vended through SPM, assuming an app already has texture in it some other way. I'm going to try to throw up a PR


feat: add python sdk

Description

[ ] Implement python ( 3.x) sdk

python sdk task can assign to me

Hei @fatelei
This is great. Thank you.
Let me know if you need any infos.

Hei @fatelei
Just wanted to ask if you are still working on this issue?
Or if you neeed anything from us.

Hei @fatelei Just wanted to ask if you are still working on this issue? Or if you neeed anything from us.

yes，working on it

@fatelei when can we expect to see any code?


More Gen 21: splitting the rest of the stores

This splits up the types out of constants file.
The primary motivation was to remove the massive cyclical dep of
constants/blah.js defines the State and selectors, which pull in TypedState
then
constants/reducer.js defines the TypedState and pulls in constants/blah
This causes flow crappiness and causes the mobile packager to get confused and leave tons of things undefined

@keybase/react-hackers this is ready


Early exit from @testitem

Often in my code I will have stuff like:
@testitem "my test" begin
    if VERSION >= v"1.8"
        using MyPkg
        do_stuff()
    end
end

It could be nice to have a way to exit early from a test instead, like
@testitem "my test" begin
    if VERSION < v"1.8"
        @testitembreak
    end
    using MyPkg
    do_stuff()
end

Nice! Yeah that would be perfect.


Nuitka generates unsafe code

I've found a few crashes on code generated by Nuitka. Repro steps: https://github.com/n-bes/nuitka-crash-poc
Example 1
# cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"

# nuitka3 --version
1.6.5
Commercial: None
Python: 3.6.9 (default, Mar 10 2023, 16:46:00)
Flavor: Debian Python
Executable: /usr/bin/python3
OS: Linux
Arch: aarch64
Distribution: Ubuntu (based on Debian) 18.04.6
Version C compiler: /usr/bin/gcc (gcc).

Asan
==22==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x51d00001e920 at pc 0xaaaab53edf48 bp 0xffffff971280 sp 0xffffff970a70

READ of size 2216 at 0x51d00001e920 thread T0
    #0 0xaaaab53edf44 in __asan_memcpy (/code/cmake-build/crash-nuitka+0x39df44) (BuildId: f53aebe4f7e8c1cf)
    #1 0xaaaab549ad64 in DEEP_COPY_DICT /code/nuitka-build/static_src/HelpersDictionariesGenerated.c:427:17
    #2 0xaaaab57116e0 in main /code/run.c:115:35
    #3 0xffffa453579c in __libc_start_main /build/glibc-L0whdJ/glibc-2.27/csu/../csu/libc-start.c:310
    #4 0xaaaab5350360 in _start (/code/cmake-build/crash-nuitka+0x300360) (BuildId: f53aebe4f7e8c1cf)

0x51d00001e920 is located 0 bytes after 2208-byte region [0x51d00001e080,0x51d00001e920)
allocated by thread T0 here:
    #0 0xaaaab53eff94 in malloc (/code/cmake-build/crash-nuitka+0x39ff94) (BuildId: f53aebe4f7e8c1cf)
    #1 0xffffa49de3cc in _PyObject_Alloc build-shared/../Objects/obmalloc.c:1446:22
    #2 0xffffa49de3cc in _PyObject_Malloc build-shared/../Objects/obmalloc.c:1456:12
    #3 0xffffa49f0a8c in new_keys_object build-shared/../Objects/dictobject.c:537:14
    #4 0xffffa49f0b2c in _PyDict_NewPresized build-shared/../Objects/dictobject.c:1386:16
    #5 0xaaaab5700eb4 in _unpackBlobConstants /code/nuitka-build/static_src/HelpersConstantsBlob.c:566:27
    #6 0xaaaab566bd00 in unpackBlobConstants /code/nuitka-build/static_src/HelpersConstantsBlob.c:1219:5
    #7 0xaaaab566b74c in loadConstantsBlob /code/nuitka-build/static_src/HelpersConstantsBlob.c:1349:5
    #8 0xaaaab5710f58 in main /code/run.c:51:5
    #9 0xffffa453579c in __libc_start_main /build/glibc-L0whdJ/glibc-2.27/csu/../csu/libc-start.c:310
    #10 0xaaaab5350360 in _start (/code/cmake-build/crash-nuitka+0x300360) (BuildId: f53aebe4f7e8c1cf)

HWAsan
==11==ERROR: HWAddressSanitizer: tag-mismatch on address 0xedcdfffedc00 at pc 0xaaaad5f3340c
READ of size 2216 at 0xedcdfffedc00 tags: 17/87 (ptr/mem) in thread T0
Invalid access starting at offset 2208
    #0 0xaaaad5f3340c in __hwasan_memcpy (/code/cmake-build/crash-nuitka+0x27340c) (BuildId: 8ace36754df69c33)
    #1 0xaaaad5fc121c in DEEP_COPY_DICT /code/nuitka-build/static_src/HelpersDictionariesGenerated.c:427:17
    #2 0xaaaad620932c in main /code/run.c:115:35
    #3 0xffffbd31479c in __libc_start_main /build/glibc-L0whdJ/glibc-2.27/csu/../csu/libc-start.c:310
    #4 0xaaaad5f26618 in _start (/code/cmake-build/crash-nuitka+0x266618) (BuildId: 8ace36754df69c33)

[0xedcdfffedc00,0xedcdfffee600) is a small allocated heap chunk; size: 2560 offset: 0

Cause: heap-buffer-overflow
0xedcdfffedc00 is located 0 bytes inside a 2208-byte region [0xedcdfffedc00,0xedcdfffee4a0)
allocated by thread T0 here:
    #0 0xaaaad5f2eb6c in malloc (/code/cmake-build/crash-nuitka+0x26eb6c) (BuildId: 8ace36754df69c33)
    #1 0xffffbd7bd3cc in _PyObject_Alloc build-shared/../Objects/obmalloc.c:1446:22
    #2 0xffffbd7bd3cc in _PyObject_Malloc build-shared/../Objects/obmalloc.c:1456:12
    #3 0xffffbd7cfa8c in new_keys_object build-shared/../Objects/dictobject.c:537:14
    #4 0xffffbd7cfb2c in _PyDict_NewPresized build-shared/../Objects/dictobject.c:1386:16
    #5 0xaaaad61fa2bc in _unpackBlobConstants /code/nuitka-build/static_src/HelpersConstantsBlob.c:566:27
    #6 0xaaaad6176b04 in unpackBlobConstants /code/nuitka-build/static_src/HelpersConstantsBlob.c:1219:5
    #7 0xaaaad61766a0 in loadConstantsBlob /code/nuitka-build/static_src/HelpersConstantsBlob.c:1349:5
    #8 0xaaaad6208e84 in main /code/run.c:51:5
    #9 0xffffbd31479c in __libc_start_main /build/glibc-L0whdJ/glibc-2.27/csu/../csu/libc-start.c:310
    #10 0xaaaad5f26618 in _start (/code/cmake-build/crash-nuitka+0x266618) (BuildId: 8ace36754df69c33)

Example 2
OS: Ubuntu 24.04 / CentOS Stream 9
Nuitka: 2.3

==1==ERROR: HWAddressSanitizer: tag-mismatch on address 0xffff7ddf3bb0 at pc 0xffff7dc06ddc
READ of size 8 at 0xffff7ddf3bb0 tags: f5/00 (ptr/mem) in thread T0
    #0 0xffff7dc06ddc in PyInit_module /code/module.build/module.module.c:1123:26
    #1 0x66def0 in _PyImport_LoadDynamicModuleWithSpec /usr/src/python3.12-3.12.3-1/build-static/../Python/importdl.c:169:9
    #2 0x66d338 in _imp_create_dynamic_impl /usr/src/python3.12-3.12.3-1/build-static/../Python/import.c:3775:11
    #3 0x66d338 in _imp_create_dynamic /usr/src/python3.12-3.12.3-1/build-static/../Python/clinic/import.c.h:506:20
    #4 0x502d4c in cfunction_vectorcall_FASTCALL /usr/src/python3.12-3.12.3-1/build-static/../Objects/methodobject.c:422:24
    #5 0x5652ac in _PyEval_EvalFrameDefault /usr/src/python3.12-3.12.3-1/build-static/Python/bytecodes.c:3254:26
    #6 0x4c2f00 in _PyObject_VectorcallTstate /usr/src/python3.12-3.12.3-1/build-static/../Include/internal/pycore_call.h:92:11
    #7 0x4c2f00 in object_vacall /usr/src/python3.12-3.12.3-1/build-static/../Objects/call.c:850:14
    #8 0x4c4ae4 in PyObject_CallMethodObjArgs /usr/src/python3.12-3.12.3-1/build-static/../Objects/call.c:911:24
    #9 0x58bee8 in import_find_and_load /usr/src/python3.12-3.12.3-1/build-static/../Python/import.c:2779:11
    #10 0x58bee8 in PyImport_ImportModuleLevelObject /usr/src/python3.12-3.12.3-1/build-static/../Python/import.c:2862:15
    #11 0x565f6c in import_name /usr/src/python3.12-3.12.3-1/build-static/../Python/ceval.c:2482:15
    #12 0x565f6c in _PyEval_EvalFrameDefault /usr/src/python3.12-3.12.3-1/build-static/Python/bytecodes.c:2135:19
    #13 0x560070 in _PyEval_EvalFrame /usr/src/python3.12-3.12.3-1/build-static/../Include/internal/pycore_ceval.h:89:16
    #14 0x560070 in _PyEval_Vector /usr/src/python3.12-3.12.3-1/build-static/../Python/ceval.c:1683:12
    #15 0x560070 in PyEval_EvalCode /usr/src/python3.12-3.12.3-1/build-static/../Python/ceval.c:578:21
    #16 0x598f70 in run_eval_code_obj /usr/src/python3.12-3.12.3-1/build-static/../Python/pythonrun.c:1722:9
    #17 0x598f70 in run_mod /usr/src/python3.12-3.12.3-1/build-static/../Python/pythonrun.c:1743:19
    #18 0x67e6f0 in pyrun_file /usr/src/python3.12-3.12.3-1/build-static/../Python/pythonrun.c:1643:15
    #19 0x67e2c4 in _PyRun_SimpleFileObject /usr/src/python3.12-3.12.3-1/build-static/../Python/pythonrun.c:433:13
    #20 0x67e090 in _PyRun_AnyFileObject /usr/src/python3.12-3.12.3-1/build-static/../Python/pythonrun.c:78:15
    #21 0x6890e8 in pymain_run_file_obj /usr/src/python3.12-3.12.3-1/build-static/../Modules/main.c:360:15
    #22 0x6890e8 in pymain_run_file /usr/src/python3.12-3.12.3-1/build-static/../Modules/main.c:379:15
    #23 0x6890e8 in pymain_run_python /usr/src/python3.12-3.12.3-1/build-static/../Modules/main.c:629:21
    #24 0x6890e8 in Py_RunMain /usr/src/python3.12-3.12.3-1/build-static/../Modules/main.c:709:5
    #25 0x688ca4 in Py_BytesMain /usr/src/python3.12-3.12.3-1/build-static/../Modules/main.c:763:12
    #26 0xffff7fb384c0 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #27 0xffff7fb38594 in __libc_start_main csu/../csu/libc-start.c:360:3
    #28 0x5f24ec in _start (/usr/bin/python3.12+0x5f24ec) (BuildId: 18160fe6beb052a7e6830ecc99e313a3498c377d)

Thread: T0 0xeffe00002000 stack: [0xfffff1f19000,0xfffff2719000) sz: 8388608 tls: [0xffff808f1460,0xffff808f2320)

Memory tags around the buggy address (one tag corresponds to 16 bytes):
  0xffff7ddf3300: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3400: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3500: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3600: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3700: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3800: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3900: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3a00: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
=>0xffff7ddf3b00: 00  00  00  00  00  00  00  00  00  00  00 [00] 00  00  00  00
  0xffff7ddf3c00: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3d00: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3e00: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf3f00: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf4000: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf4100: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf4200: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
  0xffff7ddf4300: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
Tags for short granules around the buggy address (one tag corresponds to 16 bytes):
  0xffff7ddf3a00: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..
=>0xffff7ddf3b00: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  .. [..] ..  ..  ..  ..
  0xffff7ddf3c00: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags

Registers where the failure occurred (pc 0xffff7dc06ddc):
    x0<PHONE_NUMBER>b1a670  x1  0000ffff808f1bb0  x2<PHONE_NUMBER>b266d8  x3  0000ffff7de09de0
    x4<PHONE_NUMBER>009d90  x5<PHONE_NUMBER>b2d8d0  x6  ea00ef3dffffbef0  x7  0000ffff7db92060
    x8<PHONE_NUMBER>000000  x9<PHONE_NUMBER>000000  x10<PHONE_NUMBER>000000  x11<PHONE_NUMBER>000000
    x12 9100ed9dfffeba00  x13<PHONE_NUMBER>000027  x14<PHONE_NUMBER>36fa65  x15 fffffffffffff000
    x16 0000ffff7fe3b350  x17<PHONE_NUMBER>000007  x18<PHONE_NUMBER>000004  x19<PHONE_NUMBER>b266b0
    x20 0200efff00000000  x21 f500ffff7ddf3bb0  x22<PHONE_NUMBER>0000f5  x23 00000ffff7ddf3bb
    x24 0000ffff7f0fa990  x25<PHONE_NUMBER>70f5b8  x26 0000ffff7dc06ac4  x27 0000ffff7f0fb920
    x28<PHONE_NUMBER>b8a278  x29 0000fffff2717e00  x30 0000ffff7dc06de0   sp 0000fffff2717e00
SUMMARY: HWAddressSanitizer: tag-mismatch /code/module.build/module.module.c:1123:26 in PyInit_module

Nuitka 1.6 is not too relevant anymore.

@kayhayen example 2 based on fresh version: 2.3

@n-bes You are right, but 1.6 stopped me from reading on.

At some point a commercial customer provided information about Nuitka and these kinds of reports. I do not recall them being very actual. I think it was copying unused memory and such, resulting in the micro-optimization of the dictionary copy, but it wasn't a real bug in that sense.
I still have what I used to check it, from those days:
ASAN_OPTIONS=detect_leaks=0 LDFLAGS="-fsanitize=address" CCFLAGS="-Og -fno-omit-frame-pointer -fsanitize=address -w -DNDEBUG" NUITKA_EXTRA_OPTIONS="--clang --lto=no --unstripped --static-libpython=no" python3.10 tests/basics/run_all.py search

I am currently re-running that, with 3.10 and then 3.12 maybe. It seems that HWAsan is something different, can you provide me with the flags to use to get that, I am not all that familiar with any of this.

I do not recall them being very actual

False-positive reports are very rare, please, do not ignore them.

I think it was copying unused memory and such, resulting in the micro-optimization of the dictionary copy, but it wasn't a real bug in that sense.

Coping unused memory is not marked as a problem by ASAN/HWasan, only incorrect memory access (out-of-bounds, use of uninitialized memory, etc.)

It seems that HWAsan is something different

Clang docs describes it.

Can you provide me with the flags to use to get that, I am not all that familiar with any of this.

All information available in the repo, useful links:

https://github.com/n-bes/nuitka-crash-poc/blob/init/Dockerfile#L65
https://github.com/n-bes/nuitka-crash-poc/blob/init/Dockerfile#L90
https://github.com/n-bes/nuitka-crash-poc/blob/init/docker-compose.yaml#L136

@kayhayen reopen?

I actually resolved them all, what I was saying is that they were not observable issues, like allocating or copying more memory than necessary, but never in a way that causes overflow.

I am not sure, if I have AArch64 hardware, does my M1 mac quality for that? Otherwise I have 64 bit ARM, but not too much. The code you compiled looks very small-ish. Can you maybe provide the lines around the one where your stack trace points to, if I am never going to have it?
Also, I only ever checked programs like that, never modules, I might have to do that too.

From what I am reading, ASAN should be good enough and not cost more than HWASAN, just at a lower cost, so I don't have to worry about using it. Should issues be apparent in either? Can you confirm to me, that my theory holds for the 2.3 issue you are seeing?

It seems that I was getting linker errors for module builds. I updated from llvm-15 to llvm-19, and now it no longer works at all for me locally on my Ubuntu 22.04. So I downgraded to llvm-14 as in Ubuntu, and now it works again. The LLVM snapshots appears to be only so good.
However, it appears I need to build a Python that is built with ASAN in mind. I cannot quite see that in your Dockerfile, though, and maybe I need to try hwsanitize, maybe it doesn't need it.

So, I think I convinced it to compile semi-properly for packages
LD_LIBRARY_PATH=/usr/lib/llvm-14/lib/clang/14.0.0/lib/linux/ ASAN_OPTIONS=detect_leaks=0 LDFLAGS="-fsanitize=hwaddress -g -shared-libasan -fuse-ld=lld -Wl,--no-relax" CCFLAGS="-Og -fno-omit-frame-pointer -fsanitize=hwaddress -w -DNDEBUG" NUITKA_EXTRA_OPTIONS="--clang --lto=yes --unstripped" python3.10 tests/packages/run_all.py resume

However, getting FATAL: HWAddressSanitizer requires a kernel with tagged address ABI. in my WSL. Guess I need to spin up some Ubuntu it seems.

I actually resolved them all, what I was saying is that they were not observable issues, like allocating or copying more memory than necessary, but never in a way that causes overflow.

I found a lot of null dereferences on generated code by static analysis tool and it is the main reason why I just run it with ASAN/HWasan and prove it.

I actually resolved them all, what I was saying is that they were not observable issues, like allocating or copying more memory than necessary, but never in a way that causes overflow.

I'll try to reproduce example 1 on latest nuitka version.

I am not sure, if I have AArch64 hardware, does my M1 mac quality for that? Otherwise I have 64 bit ARM, but not too much.

I don't know about m1, my local setup is M3 Max with Docker Desktop (kernel version is Linux d3e7e642290b 6.6.26-linuxkit)

It seems that I was getting linker errors for module builds. I updated from llvm-15 to llvm-19, and now it no longer works at all for me locally on my Ubuntu 22.04. So I downgraded to llvm-14 as in Ubuntu, and now it works again. The LLVM snapshots appears to be only so good.

I prepared Ubuntu 22.04 with clang-19, and it works (and crashes): https://github.com/n-bes/nuitka-crash-poc/blob/init/Dockerfile#L65

Can you maybe provide the lines around the one where your stack trace points to, if I am never going to have it?
Also, I only ever checked programs like that, never modules, I might have to do that too.

See:
source code: https://github.com/n-bes/nuitka-crash-poc/tree/init/ubuntu-2204-hwasan-env-with-nuitka/module.build
stack trace: https://github.com/n-bes/nuitka-crash-poc?tab=readme-ov-file#ubuntu-2204-hwasan-env-with-nuitka

However, it appears I need to build a Python that is built with ASAN in mind. I cannot quite see that in your Dockerfile, though, and maybe I need to try hwsanitize, maybe it doesn't need it.

I think that compiled python interpreter from sources with ASAN should be enough to detect problems. It is my first HWAsan launch, so ... ASan also replace allocator and can't detect the problem, for now we may only guess about results.

The code in question is this:
    if (_Py_PackageContext != NULL) {
        module_full_name = _Py_PackageContext;
    }

For 3.12 that is using this:
#if PYTHON_VERSION >= 0x3c0
#define _Py_PackageContext (_PyRuntime.imports.pkgcontext)
#endif

and
static char const *module_full_name = %(module_name_cstr)s;

I am not getting what that should mean in terms of a illegal write. It would appear that in libpython it's defined similar,
const char * pkgcontext;
So, can you explain what this means. Is it complaining that I am copying a pointer into a DLL (the extension module) that is exported from the main process?

I didn't see the use of llvm.sh. Unfortunately my Virtualbox hates even after an update the Ubuntu 24.04 installation so hard, it hurts, kind of surprising. I avoid working with Docker when I can, I have had issues on macOS with architecture mismatches, but you probably are using arm there.

I found a lot of null dereferences on generated code by static analysis tool and it is the main reason why I just run it with ASAN/HWasan and prove it.

These would probably be of interest, but lets just say, the stacktraces you provided, are clearly not one of those. It does a NULL check after all, but even if it didn't that would copy a NULL pointer and that would be an issue only later, most definitely a crash with segfault.

I gave macOS a try, and for packages, it refuses to work, because the Python is not instrumented, but for programs, it seems to not find any stuff with -fsanitize=address and that's good and all. The hwaddress is not support for arm macOS proper it seems.
So, I think you reproduced the results I once had with 1.6 and I don't feel like their is new evidence. The report given I would have to read up what it could even mean, but I am like 100% sure that for pre-3.12 it is definitely not a bug, and for 3.12 it is working pretty well for getting the right name to use. Also, in your example, it probably should be NULL, I can check that.
Something that might be wrong, now that I think of it, that it might have to do a strdup if it was to use it that much later, but I think it doesn't do that, i.e. when the module returns, the usage is stopped or should be. But I don't think that is what the message means, what does it mean?

I have pushed to the factory branch a change, where the assignment only happens when the names differ and now it is done via strcpy so your stacktrace should change. I am still not sure how valid it is, but checking PyModule_Create doesn't see to remember the string other than as Python objects, but the md_def remains in the module, but it seems the name is never used, but it's better to be safe than sorry, dangling pointers are not a good thing.
Please retest with current factory state, and lets see what pops up. I would expect nothing, since I didn't understand hwaddress to be better. Yet, I didn't have a chance to see module mode myself. But I am not entirely inclined right now to reproduce what I feel is a false alarm. Also, does it only give a single report at a time?

python3.10

In Ubuntu 22.04 python version is 3.10

I'll try to reproduce example 1 on latest nuitka version.

Do not reproduced with gcc (ASAN) and factory version of nutika. I'll try with clang and HWASAN.

So, can you explain what this means. Is it complaining that I am copying a pointer into a DLL (the extension module) that is exported from the main process?

I think that something happens while trying to load module in memory, not related with copying of pointer.

Please retest with current factory state, and lets see what pops up.

Ready:

https://github.com/n-bes/nuitka-crash-poc/tree/init?tab=readme-ov-file#ubuntu-2404-hwasan-factory-env-with-nuitka
https://github.com/n-bes/nuitka-crash-poc/blob/init/docker-compose.yaml#L187

Also, does it only give a single report at a time?

Yes, it is stopped on first error.

I am not convinced that your report is not an issue with hwsanitizer at all. And I would ask you to use not module mode, but executable mode and find something. It seems I wasn't able to do that, and it seems while it links an extension module, that doesn't mean, it likes being run inside the DLL only while talking to the outside. So until there is new evidence that doesn't just look like a false alarm, I am closing this, sorry.

I would of course be interested in your static analysis that claims there might be errors in the code, even if providing the evidence for it is not as easy as you hope to be, it could be something and I can of course decide if it's true.

@kayhayen I added results for to the repository in two formats:

plain-text
sarif

So, the ones about unreachable code in generated code make no sense:
* UNREACHABLE_CODE: The condition in ternary operator is always true at HelpersComparisonGe.c:364.
    ternary operator at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersComparisonGe.c:364

This is generated from a Jinja2 template, that is using a bool to nuitka_bool conversion with the explicit intent to let the C compiler resolve this at its compile time rather than creating unnecessarily complex template code. If you can, ignore these things in those files.
Similarily this one makes no sense to me either.
    default:
        NUITKA_CANNOT_GET_HERE("invalid PYGEN_ result");
    }

I have those to catch implementation errors at runtime. Having a default is a good practice, is it not.
Then those
* TAINTED_PTR.MIGHT: Variable 'home_path' is obtained from an untrusted source at HelpersFilesystemPaths.c:951 by calling function 'getenv'. The variable is used in a vulnerable operation at HelpersSafeStrings.c:57 by passing as 2nd parameter to function 'appendStringSafe' at HelpersFilesystemPaths.c:963. Please double check if input string is properly validated in error-prone manner
    [vulnerable operation] Variable 'home_path' is passed to function 'appendStringSafe' as 2nd parameter at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersFilesystemPaths.c:963
    [tainted pointer] Call of 'getenv' at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersFilesystemPaths.c:951

These functions e.g. appendStringSafe will abort on overflows, and thus the usage is safe.
And this
            coercion c2 =
                (type2->tp_as_number != NULL && NEW_STYLE_NUMBER_TYPE(type2)) ? type2->tp_as_number->nb_coerce : NULL;

Again generated code, NEW_STYLE_NUMBER makes it so that this doesn't happen on Python3.12, but it's not the only version this code is used with. Similar story, not making generated code do things, the C compiler will do for us.
This one is actually good:
* REDUNDANT_COMPARISON.ALWAYS_FALSE: Redundant comparison '0' > 'res' (0 > {[0, 1]}) is always false at HelpersBuiltin.c:624.
    comparison at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersBuiltin.c:624

I probably changed this function to be bool result at some point, and code still treats it as int result in a few places, I found at least one more spot.
Also this
* NO_VA_END: Function 'va_end' was not called before HelpersHeapStorage.c:31 inside function 'Nuitka_PreserveHeap'. (Function 'va_start' was called at HelpersHeapStorage.c:17.)
    [no va_end] no va_end inside function 'Nuitka_PreserveHeap' at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersHeapStorage.c:31

Seems not too relevant for the compilers and code I have, but the standard has a "must call" there, so doing it will probably just be something the C compiler knows if it needs it or not.
* NEGATIVE_CODE_ERROR.EX: Variable 'wb.len', which might receive a negative value at HelpersOperationInplaceAddUtils.c:71, is used without checking at HelpersOperationInplaceAddUtils.c:91 by calling function 'memcpy'.
    [value use] Variable 'wb.len' is passed to function 'memcpy' as 3rd parameter at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersOperationInplaceAddUtils.c:91
    [error code] Assign the negative value to variable 'wb.len' at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersOperationInplaceAddUtils.c:71

In this specific case, the function is known to always work, and in fact wb.len may not make sense to be set. Adding an assert(wb.len >= 0); can of course affirm that.
This one is in line with what Python does:
* HANDLE_LEAK.EX: The handle 'handle' is created at MetaPathBasedLoader.c:766 by calling function 'dlopen' and lost at MetaPathBasedLoader.c:823.
    [acquire] Call of 'dlopen' at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/MetaPathBasedLoader.c:766
    [leaked] leak at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/MetaPathBasedLoader.c:823

Since we use the code from the DLL with dlsym, releasing it seems not possible.
This one
* BUFFER_SIZE_MISMATCH: Pointer 'args', referencing a memory location of size 8 bytes, is passed as a parameter to function 'memcpy' at HelpersCallingGenerated.c:1577 by calling function 'memcpy' with a size parameter that is always equal to 16. This can lead to a buffer overflow.
    Variable 'args' is passed to function 'memcpy' as 2nd parameter at /usr/local/lib/python3.12/dist-packages/nuitka/build/static_src/HelpersCallingGenerated.c:1577

Seems to be about a false alarm, the memcpy is done on a    PyObject *const *args = &PyTuple_GET_ITEM(pos_args, 0); and two array elements are intended to be copied here memcpy(python_pars, args, 2 * sizeof(PyObject *));  which would be fine. I am not sure if helping the tool via &args[0] can help it, this is just a wrong detection. The size is statically known to be 2 and even asserted in this generated code assert(PyTuple_GET_SIZE(pos_args) == 2); so that is false alarm.
There is one on Nuitka_Generator_send which checks for NULL, but only once. That is indeed unnecessary, because it's used as a single argument function, so the one check there is can actually be removed and replaced by an assertion I guess.

I have a bunch of changes on factory, I invite you to re-run this once it hits develop or now. Factory is unusually a bit unstable to run, but I guess your analysis need not care. If possible ignore list the generated C helper issues, that I addressed. For the LOOKUP_ATTRIBUTE, I mean to add an asserting variation, as for some values, it's known they will not raise an error. For the UNICODE_APPEND one I need to find a strategy, but it seemed the lines didn't quite match, and it's not important yet.

The example you gave is a memcpy with size 0 on a NULL pointer, and I don't want to make it slower by adding a check. However, since it's common not to have a closure, I consider specializing this case, as I have done before, or just outright attaching the closure to the function after it has been created. The memcpy(NULL, NULL, 0) is not defined behavior according to C standards, but I won't care much about that. I am adding an assertion that actually confirms that like assert(closure_given == 0 || closure != NULL); and maybe it's going to help.

I don't think I looked at HWSan results with findings yet. The module mode is not suited. Have you produced results that I have not discovered yet. I only looked at the static analysis results recently.
This static analysis tool is not reliable and doesn't know enough about the code. I am done with looking at it. What remains are false alarms to me. The LOOKUP_ATTRIBUTE stuff, I have taken note of to be dealt with when I write Jinja2 template code to generate the existing code helpers, such that a expected-not-possible-to-fail one will be easy to add, but that's not now. It is however not a possible issue to not check for NULL there.
Part of the changes are in the hotfix, part are in develop, and a few minor things were added for factory just now.

This is part of the stable release 2.4 that I just made.


removed  so full text displays on pub.dev

I don't know why those u tags were there 😅
Anyway, thank you very much!


shipping ppapi components

ppapi_proxy and ppapi_ipc are hard-coded to be built as shared_library, so they need to be treated specially like #104.

@zcbenz
As far as I know, the ppapi_ipc and ppapi_proxy aren't hard-coded in gyp. see https://code.google.com/p/chromium/codesearch#chromium/src/ppapi/ppapi_internal.gyp&sq=package:chromium&l=81&q="ppapi_ipc"&type=cs

@hokein Ah I misunderstood the code, thanks!

thanks @hokein


Converting back to JSON

Is it possible to convert back to JSON from form data in node?
My use case is that I would like to assert the form data I have sent has been mapped correctly
I.E the flow is
JSON data  > Mapped to correct structure > Form data created > Payload sent
This would then be used to perform tests like the code below demonstrate ( Nock will pass my the API calls payload automatically making these kind of tests very convenient )
expect(formPayload.products.length).toEqual(originalJSON.items.length)

FormData is a Stream. It's drained when data is sent out. You may try to use one of the stream tools, like through2 – to inspect content going through "pipes".

@alexindigo thanks for the information - the issue is I don't have scope to the stream within my tests I just have a string which looks like the following
12312321
----------------------------584171990226462722659061
Content-Disposition: form-data; name="city"

Caceres
----------------------------584171990226462722659061
Content-Disposition: form-data; name="zipcode"

10001
----------------------------584171990226462722659061
Content-Disposition: form-data; name="address"

Calle Test
----------------------------584171990226462722659061
Content-Disposition: form-data; name="streetNumber"

Do you know if I can convert this without writing some hacky parser myself?

If this library implemented the official FormData API specifications we would be able to iterate over instances and return key value pairs.

Any advance with this issue? Im struggling with this trying to get key value pairs from Form Data.

Any advance with this issue? Im struggling with this trying to get key value pairs from Form Data.

Any update with this issue? Im also struggling to get key-value pairs from FormData or convert them back to JSON.

HACKY fix for testing:
const parseFormData = (form) => form._streams.reduce((result, line) => {
  if (typeof line === 'string') {
    const matches = line?.match(/name="(.+)"/);
    const key = matches?.[1];

    if (key) {
      result._currentKey = key;
    } else if (line !== '\\r\\n') {
      result[result._currentKey] = line;
      delete result._currentKey;
    }
  }

  return result;
}, {});


Create README.md

Description 📜
Javascript call stack
Fixes #3634

Type of change 📝

[x] Audio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)

Domain of Contribution 📊

[x] Frontend Dev #HTML/CSS/JS

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

Good job @Arya-Gupta !
@2024-SANDHYA Have a look, it's good to merge from my side.


[BUG]: Two typos on arneis-conveyor-20220312.pdf

Description of the bug
There are two typos in the first three lines of the first page of arneis-conveyor-20220312.pdf

Steps To Reproduce

Open arneis-conveyor-20220312.pdf
Notice the destination of the first link
Notice the destination of the second link

Additional Information
No response

@alv67 I must assign the issue to you, unless you first create the HOWTO has hinted in https://github.com/B-AROL-O/ARNEIS/issues/232#issuecomment-1066038803 🤣


Rename IEEE methods to fit convention

Fixes #1

Thanks Velovix, you are now the first official contributor :)

Glad to do it :)


Setting a timeline record's state to skipped in a logging command results in no icon

Agent version and platform
2.111.1
Windows
VSTS type and version
VSTS (https://vs01.visualstudio.com)
What's not working?
Using the commands documentation, the following is a valid timeline record command:
##vso[task.logdetail id=guid here;state=skipped;progress=100;]Task D

However, using this command in a PowerShell Script step results in the record appearing in the timeline with a 0:00 duration and a white square for an icon, as in the below screenshot:

Thanks for reporting!  This is a service side web UI issue (and not an agent issue).  We'll get it taken care of.  Unfortunately the service side bugs and tasks aren't open.


Internal compiler error when comparing if map is empty using emptyMap function

Are you using the latest released (or pre-released, a.k.a. "next") version?

[X] I'm using the latest Tact version

Tact source code
fun test(): Bool {
   let s: map<Int,Int> = emptyMap();
   return s == emptyMap();
}

contract TestContract {
}

Relevant Tact/build system log output
Tact compilation failed
Error: (Internal Compiler Error) Invalid types for binary operation: /home/jesus/tact/tact/bin/test.tact:3:15
    at writeExpression (/home/jesus/tact/tact/dist/generator/writers/writeExpression.js:278:19)
    at writeCastedExpression (/home/jesus/tact/tact/dist/generator/writers/writeFunction.js:25:77)
    at writeStatement (/home/jesus/tact/tact/dist/generator/writers/writeFunction.js:56:32)
    at /home/jesus/tact/tact/dist/generator/writers/writeFunction.js:475:25
    at WriterContext.body (/home/jesus/tact/tact/dist/generator/Writer.js:188:9)
    at /home/jesus/tact/tact/dist/generator/writers/writeFunction.js:463:21
    at WriterContext.fun (/home/jesus/tact/tact/dist/generator/Writer.js:141:9)
    at writeFunction (/home/jesus/tact/tact/dist/generator/writers/writeFunction.js:454:17)
    at /home/jesus/tact/tact/dist/generator/writeProgram.js:277:43
    at Array.forEach (<anonymous>)

What happened?
Checking if the map is empty using the emptyMap function, produces the error. But note that if we remove the contract declaration, like this:
fun test(): Bool {
   let s: map<Int,Int> = emptyMap();
   return s == emptyMap();
}

the error disappears.
Also, if we compare against null, the error disappears:
fun test(): Bool {
   let s: map<Int,Int> = emptyMap();
   return s == null;
}

contract TestContract {
}

What did you expect?
To compile without errors. Is this probably caused by the compiler not able to determine the type of emptyMap()?
Steps to reproduce
No response
How do you run Tact?
Tact CLI
Anything else?
No response

return s == null;

works because the compiler is too smart now -- it sees the comparison is always true at compile time and the code generator does not actually deal with map comparisons


misc fixes

PS: that coveralls decrease is because most of the tests now just run the commands via shelljs, so it's done in a separate node process that istanbul doesn't track :(

Coverage decreased (-30.56%) when pulling 282596aab0585292f007dcf49abffc8fb278c799 on slang800:master into d666b14513d5dc48964602b43840f2e7cd41201d on carrot:master.

Hey just want to give you a heads up that, as mentioned in the first comment here, I still am going to want separate pull requests per change rather than one gigantic full rewrite "misc" pull request. I don't want this to turn into the same sticky situation as we are in with the accord PR, so let's try to keep this one on track from the start.

Coverage decreased (-30.56%) when pulling 60df04487e503e5c59d998e9cd71e2dac3241152 on slang800:master into d666b14513d5dc48964602b43840f2e7cd41201d on carrot:master.

one PR per commit?

Nope, one PR per idea/feature. We're pretty good at breaking up commits into logical chunks and submitting PRs based on that around here. For example, a PR called "miscellaneous" with 30 commits is not really useful. If you are having trouble with the logical chunking concept, grab one of us for help as soon as you start to notice things getting out of hand, and we can discuss it and figure it out.

what logical chunks do you see here?

 Are you unable to do so? If you are not, I can add this to my todo list.

Nope, I grouped all of my changes into commits, where each commit is one logical grouping of changes. As a whole they're not really working towards any particular feature, they're just a set of miscellaneous improvements.
Though I'd like you to review the commits anyway & make sure there aren't any bugs or things I missed.

If that was your approach, then yes I'd like one PR per commit. When you submit all the commits in a single PR, I can either take or leave all of them at the same time. When they are in separate PRs, we can merge in ones that we both agree on and get them shipped right away, and for ones that are more conflicted, there is a place for discussing just those conflicts.

Oh, you can use git cherrypick to accept individual commits.

I know that. But I'd rather use github pull requests for their strengths here, and they are very good at providing an area to discuss a proposed change to code.

I'm fine with using the line comments for discussing individual commits... I don't think I see the problem you're getting at.

I'm not fine with that. I would rather have a pull request per feature.

what's your issue with it? maybe we could find a way around it so I don't need to make 17 separate pull requests?

https://github.com/carrot/accord-cli/pull/3#issuecomment-69814119 <-- I like this functionality and prefer to use it. I'm sorry about the 17 pull requests, but the good news is that next time we can start from the beginning submitting a pull request per feature and you don't have to do this separation again.

You can definitely do that on any comment. For example: https://github.com/slang800/accord-cli/commit/24499e1977fad0a0dec70729d255eaf933c94a1c#commitcomment-7541742
You just right-click on the date & select "copy link address"

I would rather use pull requests. I'm sorry. I know it's a pain. But we have to do it.