[SPARK-47045][SQL] Replace IllegalArgumentException by SparkIllegalArgumentException in sql/api

What changes were proposed in this pull request?
In the PR, I propose to replace all IllegalArgumentException by SparkIllegalArgumentException in sql/api code base, and introduce new legacy error classes with the _LEGACY_ERROR_TEMP_ prefix.
Why are the changes needed?
To unify Spark SQL exception, and port Java exceptions on Spark exceptions with error classes.
Does this PR introduce any user-facing change?
Yes, it can if user's code assumes some particular format of IllegalArgumentException messages.
How was this patch tested?
By running existing test suites like:
$ build/sbt "core/testOnly *SparkThrowableSuite"

Was this patch authored or co-authored using generative AI tooling?
No.

@HyukjinKwon @LuciferYang @panbingkun @srielau @hvanhovell @beliefer Could you review this PR, please.

Merging to master. Thank you, @srielau for review.


Fix State Router Throw Exception when Address is Empty

What is the purpose of the change
Brief changelog
Verifying this change

Checklist

[x] Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Add some description to dubbo-website project if you are requesting to add a feature.
[ ] GitHub Actions works fine on your own branch.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #8951 (6219266) into 3.0 (401e6eb) will increase coverage by 0.01%.
The diff coverage is 72.72%.

@@             Coverage Diff              @@
##                3.0    #8951      +/-   ##
============================================
+ Coverage     63.27%   63.28%   +0.01%
- Complexity      313      314       +1
============================================
  Files          1182     1182
  Lines         50259    50256       -3
  Branches       7522     7500      -22
============================================
+ Hits          31802    31806       +4
+ Misses        14973    14967       -6
+ Partials       3484     3483       -1

Impacted Files
Coverage Δ

...java/org/apache/dubbo/rpc/cluster/RouterChain.java
79.71% <72.72%> (+0.98%)
:arrow_up:

...ng/transport/dispatcher/all/AllChannelHandler.java
62.06% <0.00%> (-20.69%)
:arrow_down:

...dubbo/common/status/support/LoadStatusChecker.java
46.15% <0.00%> (-15.39%)
:arrow_down:

...ache/dubbo/remoting/transport/AbstractChannel.java
75.00% <0.00%> (-12.50%)
:arrow_down:

...in/java/org/apache/dubbo/common/utils/JVMUtil.java
81.13% <0.00%> (-11.33%)
:arrow_down:

...ng/transport/dispatcher/WrappedChannelHandler.java
47.82% <0.00%> (-10.87%)
:arrow_down:

.../org/apache/dubbo/remoting/ExecutionException.java
15.78% <0.00%> (-5.27%)
:arrow_down:

...ng/exchange/support/header/HeartbeatTimerTask.java
68.42% <0.00%> (-5.27%)
:arrow_down:

.../apache/dubbo/remoting/transport/AbstractPeer.java
56.52% <0.00%> (-4.35%)
:arrow_down:

...he/dubbo/remoting/transport/netty/NettyServer.java
70.17% <0.00%> (-3.51%)
:arrow_down:

... and 38 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 401e6eb...6219266. Read the comment docs.


How do I migrate content to PRIVATE channels?

I can't find private channels in the migration tool (they are simply not shown).
And since I cannot convert a non-private-channel to a private channel, I can't migrate the content and make the channel private later.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 36d01d11-eea9-7f49-693d-25c3fecc6253
Version Independent ID: 6749a619-2b75-1561-81e9-b45888c58c4b
Content: How to use the SharePoint Migration Tool - Migrate to SharePoint and OneDrive
Content Source: migration/how-to-use-the-sharepoint-migration-tool.md
Service: sharepoint-online
GitHub Login: @JoanneHendrickson
Microsoft Alias: jhendr

I solved it myself:
A private channel is in fact a completly different SharePoint-site so for each private channel you have to find the corresponding Sharepoint-site 😯
Please include in the documentation.
Thanks.

@RipRapRob Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.


Unable to run Jasmine tests with fixtures

Hi,
I'm attempting to use jasmine-rails to process coffeescript (both code & tests), but I keep seeing "ReferenceError: Can't find variable: Regions in [...]/tmp/jasmine/assets/jasmine-specs.js (line 10723)". I check the compiled jasmine-specs.js file and I see Regions defined before line 10723. Can you offer any suggestions on how to get these tests running? Also, let me know if you need to see anything else. Thanks.
Relevant gems in use:

jasmine (1.3.2)
jasmine-core (1.3.1)
jasmine-rails (0.7.1)

Includes:

jasmine-jquery.js

Failing test (settings_spec.js.coffee):
describe 'Settings', ->
  beforeEach ->
    loadFixtures('settings.html')
    Regions.setup()
    Modal.setup()
    return

  it 'should disable all locations dropdowns except APAC on page load', ->
    expect($("#locations_apac")).toBeHidden()
    expect($("#locations_emea")).toBeVisible()

Thank you @nathany! Helped me a lot! :heart: :grin:

You’re welcome. ;-)

@nathany thank you very much!!!! :+1:

Please have a look to my PR https://github.com/travisjeffery/jasmine-jquery-rails/pull/12. It will add jasmine-jquery fixture support for Rails without special configs necessary and it even supports HAML!

It seems like I can only choose to run the specs on the command line or via http://localhost:3000/specs using the solution above.
jasmine.getFixtures().fixturesPath = "../../spec/javascripts/fixtures";

It would be nice to conditionally load the fixtures based on whether it was via the browser or the command line. I'll look into that.

Hi, @ccallebs, were you able to figure out how to make it run for both command line and via browser?
I am running into the same problem.

@enkanba I never was -- we just settled on the command line. Sorry :(

(Maybe) fixed by #216


AVD Baseline deployment fails due policy violation

What happened? Provide a clear and concise description of the bug, including deployment details.
Using ALZ-Bicep v0.19 prior to AVD Baseline deployment, and fails on RequestDisallowedByPolicy error on Enforce recommended guardrails for Azure Key Vault. Azure Key Vault should use RBAC permission model
Key vault module needs to be updated to use RBAC instead of access policies for Key vault.
Please provide the correlation id associated with your error or bug.
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
What was the expected outcome?
No response
Relevant log output
New-AzDeployment: /home/vsts/work/_temp/38ae158e-e968-459b-89f9-2b7e64ae7b42.ps1:6
Line |
6 |  New-AzSubscriptionDeployment -DeploymentName $DeploymentName `
|  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|  InvalidTemplateDeployment - Long running operation failed with status
| 'Failed'. Additional Info:'The template deployment failed because of
| policy violation. Please see details for more information.'
| RequestDisallowedByPolicy - Resource 'kv-sec-smt-test-usc-xt' was
| disallowed by policy. Policy identifiers:
| '[{"policyAssignment":{"name":"Enforce recommended guardrails for Azure
| Key
| Vault","id":"/providers/Microsoft.Management/managementGroups/xxxxxx-landingzones/providers/Microsoft.Authorization/policyAssignments/Enforce-GR-KeyVault"},"policyDefinition":{"name":"Azure Key Vault should use RBAC permission model","id":"/providers/Microsoft.Authorization/policyDefinitions/12d4fa5e-1f9f-4c21-97a9-b99b3c6611b5"},"policySetDefinition":{"name":"Enforce recommended guardrails for Azure Key Vault","id":"/providers/Microsoft.Management/managementGroups/xxxxxxxx/providers/Microsoft.Authorization/policySetDefinitions/Enforce-Guardrails-KeyVault"}}]'.
##[error]PowerShell exited with code '1'.

@ThojoUno thank you for your feedback, we are working on it.


Integration Candidate 2020-03-25

Describe the contribution
Integration Candidate 2020-03-25,
incorporates changes to sample_app, cFE, and OSAL
Fix #28, #60, and #47,
Testing performed
PR #61
Bundle CI
Expected behavior changes
See related Integration Candidate PRs
nasa/osal#52
nasa/osal#401
nasa/cFE#579
Updates to Bundle CI:

Deploy documentation,
config warnings
removes permissive mode hack
Adds enforce of no doxygen warnings for osal and users guide

System(s) tested on
Bundle CI - Ubuntu:Bionic
Code contibutions
Jacob Hageman - NASA/GSFC
Gerardo E. Cruz-Ortiz - NASA/GSFC

@skliper https://travis-ci.com/github/nasa/cFS/jobs/311547104

This is a merge issue - the global variable was renamed from Sample_AppData to SAMPLE_AppData to adhere to the capitalization patterns used in other units and the naming convention.


Option to have GT handle state.

Allow the user to tell GT to keep track of the state of the system for us. Optional when creating the GT object.

Completed.


This still works

I just upgraded to a new development system, still using VS 2010 with very wide screens and all of my forms were screwing up and throwing split container errors.  Applying the above registry fix cleaned up the problem, thanks.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3e9b977f-c58f-1cd4-a146-5246723932ef
Version Independent ID: b615485b-d573-7ba1-62db-be2ad27d0a48
Content: Automatic Scaling in Windows Forms
Content Source: docs/framework/winforms/automatic-scaling-in-windows-forms.md
Product: dotnet-framework
GitHub Login: @dotnet-bot
Microsoft Alias: dotnetcontent

The article does not discuss registry changes.

@todduk we have since removed the registry change. @OliaG can you comment on this issue? Should we close this or do we want to add some additional information to the topic?

Registry change is not a suggested way of fixing that problem since the feature is not officially published to external users and supported. In the next release 15.8 we will have a solution for that issue in Visual Studio. Meanwhile it's up to you if you prefer to update the registry key, but we are not going to promote this solution in the docs.

Thanks @OliaG. I'll close this one as by-design then. We can document the new feature then once it's out.


Ticket/04 social media

Hi @younglust!
I have created a SocialMedia component which has a li corresponding to a circle of social networks and this component I use in SocialMediaList to generate the 4 options.

Hello @irenegf83 I just seen socialMedia code and I think it's cool and you can merge it. :)


LowNodeUtilization policy needs all thresholds to be violated

I am testing out LowNodeUtilization policy with the following value:
           nodeResourceUtilizationThresholds:
             thresholds:
               cpu: 60
               memory: 60
               pods: 5
             targetThresholds:
               cpu: 100
               memory: 100
               pods: 1000

However all nodes are appropriately utilized.
Eg:
I0514 20:35:09.877111       1 lownodeutilization.go:147] Node "gke-asia-northeast1-std--default-pool-36ae422e-wnp4" is appropriately utilized with usage: api.ResourceThresholds{"memory":48.0697066631997, "pods":12.727272727272727, "cpu":34.64443045940843}

For the above node
"memory":48.0697066631997, < 60
"cpu":34.64443045940843 < 60
But  "pods":12.727272727272727 > 5
I checked the code and  it looks like IsNodeWithLowUtilization will return false if any threshold is not violated - https://github.com/kubernetes-incubator/descheduler/blob/master/pkg/descheduler/strategies/lownodeutilization.go#L298
This means that ALL thresholds need be violated instead of ANY. Is that by design?

This means that ALL thresholds need be violated instead of ANY. Is that by design?

Yes. For a node to be considered under utilized all thresholds should be lower or violated as you mentioned.

On the other hand, for a node to be considered over utilized any one of them should be higher than configured targetthresholds.

@ptagr if you do not have any further concern, could you close this issue?


show XCA federation vs centralization

with pro and con of each
This might be an independent whitepaper?

this should be careful to be written for executive level audience, do not duplicate technical details found elsewhere and in the profile text. Should likely lean on the basic concepts of the definition of centralization and federation.
If this text gets too long (more than a half sheet of paper), then it is a clear indication that we need a whole whitepaper on the topic.

Centralized has an Initiating-Gateway/Responding-Gateway at the center. Where all traffic must traverse through this central actor. It has the benefit of having full knowledge of the traffic patterns, needs, usage, etc. Audit Logging can be maintained centrally. It also has the ability to 'fixup' requests/responses/content to satisfy dissonance in capability/conformance of various partners. For example: when one partner needs highly constrained content, where a responding partner is an older generation that does not meet those highly constrained content. Similar fixup can be done with security tokens. The drawback of this model is that provenance of all data and requests is reliant on the central system, the central system has full visibility to all protected health information, the central system can modify any request/response, the central system is a bottle-neck from a performance, and robustness.

Federation enables each community to communicate within a trust domain. That trust domain is all that is managed centrally, which usually is a certificate authority(s) and organizational endpoint directory. All communications are between the partners, and are thus not exposed to any central system, thus no bottle-neck or provenance concern.
There is a perception that a federation is harder to maintain, but the central trust domain and configuration management simplifies that to the same degree as centralized.

Should we cover a hybird where the basic federation is augmented with a Record Location Service (Patient Discovery) that is implemented centrally? That is to say that XCPD is always used with a central service, but XCA is federated?

does more need to be said?
Seems diagrams would help. I am not sure how to show in a diagram the difference. Need creative proposed diagrams

@jlamy did you look at the comments on this issue and come to a conclusion of what should be said in the HIE whitepaper vs more detail that exists elsewhere?
The discussion lately on identity, centralized vs federated, did get me thinking that centralized vs federation does need to be highlighted. Where centralized everyone is trusting completely that central authority, federation is transaction and purpose specific trust. Meaning I can decide to be selective in my trust of a federated partner, but I must fully trust all the centralized partners. I think this is true of identity (XCPD) but also XCA like metadata

@jlamy do you have a recommendation for text changes? Or do we consider the whitepaper is sufficient detail today?

I like what is said now, but let's consider the following.
Section 3.3 merges the concepts of community and gateway, implying a federation consists of a bunch of communities that each have gateways. There are two counterexample use cases; should we mention them here?

An XCA RG fronts multiple communities and exposes these in XCA query responses and potentially as parent-child relationships in a directory. The IG can either look for the community in the directory and retrieve directly from it, or retrieve from the original gateway. See ITI TF 2b: <IP_ADDRESS>.3.2.
An XCDR RG fronts multiple communities and exposes these as parent-child relationships in a directory. The IG, if it wants to direct a submission to a specific community, includes that HCID as routing information when submitting to the RG.

I notice that we never mention the concept of the Home Community ID, which I assume must be intentional. Nor do we ever connect the concepts of directories and communities, e.g. to say you could look up a community in a directory. Without this background, my cases above will be hard to follow for the reader.
If we wanted to allude to these cases without getting too technical, we could do the following:

Modify section 3.3: "Behind each gateway may be a single organization, like the USA Veterans Administration, a complex community of many organizations, or a more simple organization like a single small hospital or facility. The gateway may hide the internal structure of each participant in the cross-community exchange, or may expose them as identifiable communities reachable through the gateway."
Add a short section 3.4 Federated Push, patterned after 3.3:
3.4.1 Cross-Community Document Reliable Interchange (XCDR)
The XCDR profile provides a point-to-point method of sending documents to a specific community. It leverages common principles as described in Section 2. It sends documents and metadata using the same Web-Services reliable transport that is used to publish documents in the Centralized Discovery and Retrieve Document model.

An XCDR Responding Gateway may support submissions to multiple communities that are behind it. These communities would typically be described in a common directory, with a parent-child relationship linking them to the community with the XCDR gateway.


Publish backstop results to gh pages

Maybe we should merge this one? @mayank99 @veekeys

I think this can be improved simply by adding branch name to url. We have 8 PRs right now and we don't want them conflicting each other.
At root, we can still host test html pages.


Simple usage example doesn't work

Hi @ohler55
Thanks for this tool.
But i think it's doesn't work right!
Command: perfer -t2 -c2 -d2 http://localhost:4000
Results:
➜  ~ perfer -t2 -c2 -d2 http://localhost:4000
*-*-* Only one URL is allowed.

usage: perfer [options] <server-address>

Saturates a web server with HTTP requests while tracking the number of requests,
latency, and throughput. After the duration specified no more requests are sent
and the application terminates when all responses have been received or 10
seconds, which ever is sooner.

  -h                      This help page.
  --help

  -v                      Verbose mode.

  --version               Version of this application.

  -d <duration>           Duration in seconds for the run. Positive decimal
  --duration <duration>   values are accepted.

  -m <rate>               Set a metering rate in request per second.
  --meter <rate>          (default: 0, indicating no metering)

  -t <number>             Number of threads to use for sending requests and
  --threads <number>      receiving responses. (default: 1)

  -c <number>             Total number of connection to use for sending
  --connections <number>  requests (default: 1)

  -b <number>             Maximum backlog for pipeline on a connection.
  --backlog <number>      (default: 1, range 1 - 15)

  -l <percent,...>        Percentages of latency spread to report.
  --latency <percent,...> (example: 10,20,30,40,50,60,70,80,90,99)

  -g <wide>x<high>        Print a latency graph with the dimensions specified.
  --graph <wide>x<high>

  -r <file>               File with the full content of the HTTP request.
  --request <file>        The -keep-alive option is ignored.

  -k                      Keep connections alive instead of closing.
  --keep-alive

  -a <name: value>        Add an HTTP header field with name and value.
  --add <name: value>

  -p <content>            HTTP POST with the content provided.
  --post <content>        example: -p 'mutation { repeat(word: "Hello") }'

  -j                      JSON output.
  --json

  <url>                   URL for requests.
                          example: http://localhost:6464/index.html

Or i'm doing something wrong?

Options need a space separating the key from the value so this would be the correct syntax.
prefer -t 2 -c 2 -d 2 http://localhost:4000

Thanks. I’d think it’s normalizes args, seems not. Nice tool

Duration parameter doesn't work as i'm giving -d 2, but it runs for 0.7 second.
Also, wrk works much accurate and shows real requests per second, but perfer shows very low values and it's impossible.

Perfer attempts to run for the designated time. Usually it is closer than what you are seeing. There are a number of parameters you should try. By default wrk uses keep-alive. You must set the -k option of perfer for keep alive. You can also set the pipelining option -b to some value like 3 if you want.

Did any of those suggestions help?

Yep, it helped. Thanks

With the changes did you find prefer gave faster results or about the same?

Perfer doesn't gives faster results, i understand new things with perfer.
Perfer gives accurate results!
Nice tool, i recommend to others to understand, numbers on wrk isn't accurately

I'm curious why you say wrk is not accurate.

Perfer shows really accurate results by creating new connection by default.
This means, it shows real life example, not higher numbers. Because i really tried improve performance, knew about network delay, but didn’t know good about KeepAlive. On different users not single Connection, yes? I always wanted benchmark how much requests will my library handler at same time from different location, not from single location.
I seem, my library does not differ much from express. Does not worth improving performance, because there TCP/Network layer...

Both new connections and old are important. Once a connection is made there are many pages that must be retrieved. CSS, Javascript, HTML, and data. Keep-Alive lets this pages be downloaded faster.
Anyway, you have options. Thank you for the feedback.

Yep, i agree about keep alive, but my focus more connection handling at same time


Module '*-compliler.1.0.4' depends on one or more Android Libraries but is a jar

Module 'com.tojc.ormlite.android:ormlite-content-provider-compiler:1.0.4' depends on one or more Android Libraries but is a jar
:app:prepareDebugDependencies FAILED

Did you add both the library com.tojc.ormlite.android:ormlite-content-provider-library:1.0.4 and the compiler?

Yes, I already added "library" and "compiler" then I got it. But If I just added library is everything seems to be OK. Then, some guideline from you can be completed without compiler for generating.

I'm sorry, I'm a bit unclear on what you are saying.
You probably need a library for annotation processing if you're using the compiler.
Have you got an annotation processor, such as https://bitbucket.org/hvisser/android-apt installed?
You would need something like the following in your gradle file:
// Needed for annotations processing
apply plugin: 'com.neenbedankt.android-apt'
buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.neenbedankt.gradle.plugins:android-apt:1.8'
    }
}
apt {
    arguments {
        resourcePackageName android.defaultConfig.applicationId
        androidManifestFile variant.outputs[0]?.processResources?.manifestFile
    }
}

dependencies {
    apt 'com.tojc.ormlite.android:ormlite-content-provider-compiler:1.0.4'
    compile 'com.tojc.ormlite.android:ormlite-content-provider-library:1.0.4'
}

I had the same issue when I had
compile "com.tojc.ormlite.android:ormlite-content-provider-library:1.0.4@aar"
compile "com.tojc.ormlite.android:ormlite-content-provider-compiler:1.0.4"
in my Gradle script. Replace it with
compile "com.tojc.ormlite.android:ormlite-content-provider-library:1.0.4@jar"
compile "com.tojc.ormlite.android:ormlite-content-provider-compiler:1.0.4"
and you are all set.

The project is no longer actively. Just close it.


Cannot connect to NICE-DCV (from Windows)

Hello,
I'm having issues trying to connect to NIVE-DCV. The aws-parallecluster version is 2.10.1. I downsized the cluster size and boiled down the configuration script to its skeleton:
[global]
update_check = true
sanity_check = true
cluster_template = g1cluster

[aws]
aws_region_name = us-east-1
aws_access_key_id = ********...
aws_secret_access_key = ********...

[cluster g1cluster]
vpc_settings = odyvpc
placement_group = DYNAMIC
placement = compute
key_name = ********
master_instance_type = t3.medium
compute_instance_type = c5.large
cluster_type = spot
disable_hyperthreading = true
initial_queue_size = 2
max_queue_size = 2
maintain_initial_size = true
scheduler = slurm
base_os = centos8
dcv_settings = wdcv

[vpc odyvpc]
master_subnet_id = subnet-********
vpc_id = vpc-********

[dcv wdcv]
enable = master
access_from = <IP_ADDRESS>/0

[aliases]
ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}

Everything is working fine and the dcvserver is up and running:
 sudo systemctl status dcvserver
? dcvserver.service - NICE DCV server daemon
   Loaded: loaded (/usr/lib/systemd/system/dcvserver.service; enabled; vendor p>
   Active: active (running) since Mon 2021-02-22 18:35:10 UTC; 5min ago
 Main PID: 6162 (dcvserver)
    Tasks: 4 (limit: 23063)
   Memory: 26.5M
   CGroup: /system.slice/dcvserver.service
           ??6162 /bin/bash /usr/bin/dcvserver -d --service
           ??6173 /usr/libexec/dcv/dcvserver --service

Feb 22 18:35:06 ip-172-31-47-117 systemd[1]: Starting NICE DCV server daemon...
Feb 22 18:35:10 ip-172-31-47-117 systemd[1]: Started NICE DCV server daemon.

with no running dcv sessions as expected. The issue happens when I try to connect (from Windows) with:
C:\Users\Arturo\.parallelcluster>pcluster dcv connect -k ~/.ssh/id_rsa g1cluster
The authenticity of host '<IP_ADDRESS> (<IP_ADDRESS>)' can't be established.
ECDSA key fingerprint is SHA256:u2tUM52iIzq56r18rUdv0R3Oi7fQMJ23qwGE0RbgY9A.
Are you sure you want to continue connecting (yes/no)?
ERROR: Something went wrong during DCV connection.
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Please check the logs in the /var/log/parallelcluster/ folder of the head node and submit an issue https://github.com/aws/aws-parallelcluster/issues

Thanks,
Arturo
pcluster_dcv_authenticator.log

Hi Arturo,
I wonder the key path ~/.ssh/id_rsa g1cluster might not work in Windows. You could try using backslashes, like c:\path\to\key. The rest of your configuration seems OK to me.

Hi @ddeidda,
Not sure if I'm following you. Why would the id_rsa key be in my Windows system? Isn't the pcluster command supposed to grab that key from the master instance and open the dcv connection?
Thanks.

@afernandezody
With the -k argument you specify the local path to the private ssh key that should be hosted on your system.
More information here: https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster.dcv.html

Hi @ddeidda,
You're right, I had not paid attention to the NICE DCV HTTPS certificate section. It's not exactly how I assumed it would work. Let me close the ticket because I need to think about about going about this via a different route, and your proposed solution should work.
Thanks.


Update Mojang Scrolls game client.

Update Mojang Scrolls game client (close source).
The linux client is still in beta. So the binary on the fetch URL changes from time to time.

Please properly name commit following our guide in CONTRIBUTING.md.

Hi,
I changed the commit message accordingly to the guide.
Thanks


Decouple zebra-scanner database format version from the one used by zebrad

Motivation
The zebra-scanner binary currently uses the same database version as zebrad through its Storage struct, so any bump to the database format version in zebra-state for zebrad also bumps scanner's database format version.

This caused a problem here where the version file is deleted for restore-able major format version bumps because zebra-scanner's Storage skips format upgrades via the debug_skip_format_upgrades parameter, so when the zebra-state major database format version is bumped, DiskDb::try_reusing_previous_db_after_major_upgrade() will delete the disk version file after its just been created by database_format_version_on_disk(), and the database format upgrades don't restore it.
The immediate problem causing a panic in zebra-scanner during a shutdown can be solved trivially by updating the disk version to 0.0 in try_reusing_previous_db_after_major_upgrade(), but

We want to have an independent database format version in code for the zebra-scanner Storage to avoid unnecessarily updating its config.db_path(), this may be best to do while separating ZebraDb into its own crate.

I misdiagnosed this.


User registration API

User info API that will return user email and is user fully registered
2.User registration API that will receive email, name and work place

Part of #1469

That's the data that we need to get from the user


FLINK-15649 and FLINK-20359. Adding Volume and Configuration Mount decorators and Owner reference

…ce support

What is the purpose of the change

Support mounting volumes, including PVCs, emptydir and hostpaths and configurations, including Secrets and config maps
Support for owner reference

Brief change log
(for example:)

Adding new additional decorators - volume mount and configuration mounts and their specialization for Task and Jobcmanagers
Adding required keys to the KubernetesConfigOptions
Adding unit tests for new decorators
Adding decorators to Task and job manager factories
Adding owner reference to job manager factory
Updating unit job manager unit test
removing Secret mount and its unit test (COnfiguration decorator provides more powerfull support)

Verifying this change
This change added tests and can be verified as follows:
(example:)

Added unit test for new decorators and updated JM factory test to support owner reference
Extended integration test for recovery after master (JobManager) failure

Does this pull request potentially affect one of the following parts:
no
Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? (JavaDocs )

@blublinsky is this PR still valid? If not, then let's close it.

Yes, lets close, thank you


fix: fix saving version & backend port saving & new sdk

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#218  👈
#217
#216
#215
#214
#213
#209
#208
#207
#206
#205
#204
#203
#202
#201
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @NathanFlurry and the rest of your teammates on  Graphite


System.InvalidOperationException: WaitForInputIdle failed.

What steps will reproduce the problem?
1. While running SuperPuTTY try to start an SSH session
2. Window pops up saying "Invalid Operation Error  WaitForInputIdle failed.
This could be because the process does not have a graphical interface.
3. Clicking ok on this window causes window that looks like attached file to
appear

What version of the product are you using? On what operating system?
Using latest version of SuperPuTTY on Windows 7.

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 1 Jul 2015 at 11:40
Attachments:

superputty_error.png

I'm getting this on Windows 10 with version <IP_ADDRESS> of superputty.
When clicking on the OK button after the initial error message, I get the following stacktrace:
System.InvalidOperationException: WaitForInputIdle failed.  This could be because the process does not have a graphical interface.
  at System.Diagnostics.Process.WaitForInputIdle(Int32 milliseconds)
  at SuperPutty.ApplicationPanel.OnVisibleChanged(EventArgs e)
  at System.Windows.Forms.Control.AssignParent(Control value)
  at System.Windows.Forms.Control.ControlCollectionAdd(Control value)
  at System.Windows.Forms.Form.ControlCollection.Add(Control value)
  at SuperPutty.ctlPuttyPanel.CreatePanel()
  at SuperPutty.ctlPuttyPanel..ctor(SessionData session, PuttyClosedCallback callback)
  at SuperPutty.ctlPuttyPanel.NewPanel(SessionData sessionData)
  at SuperPutty.SuperPuTTY.OpenPuttySession(SessionData session)
  at System.Windows.Forms.TreeView.WndProc(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
  at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr Iparam)

Same problem on 1.4.0 8 win10.

Same here (SuperPuTTY <IP_ADDRESS>, putty 0.70, Windows 10).
FTR: I tried whether installing the x86 version of putty would fix it, but that didn't help.

Check if you installed putty via chocolatey (and putty.exe resides in the chocolatey path C:\ProgramData\chocolatey\bin). That version wasn't getting uninstalled and had a glitch (though being 0.70?). Once I changed the path to a freshly installed putty in superputty, everything worked again.


Support for compiling in Chinese paths and Chinese output directory, Chinese executes and Chinese module names not perfect yet

Nuitka version, full Python version

> python -m nuitka --version
2.2.1
Commercial: None
Python: 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)]
Flavor: CPython Official
Executable: C:\Users\ASUS\AppData\Local\Programs\Python\Python311\python.exe
OS: Windows
Arch: x86_64
WindowsRelease: 11
Nuitka-Scons: Non downloaded winlibs-gcc 'D:\llvm-mingw-20240502-ucrt-x86_64\bin\gcc.exe' is being ignored, Nuitka is
Nuitka-Scons: very dependent on the precise one.
Version C compiler: ~\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\gcc.exe (gcc 13.2.0).

How did you install Nuitka and Python
I installed Nuitka using pip.

Log
> nuitka --include-package-data=selenium --enable-plugin=tk-inter --standalone --onefile --windows-icon-from-ico=main.ico --output-dir="D:\Silly\CL - P\dist" "D:\Silly\CL - P\Cl.py"

Nuitka-Options: Used command line options: --include-package-data=selenium --enable-plugin=tk-inter --standalone
Nuitka-Options: --onefile --windows-icon-from-ico=main.ico --output-dir="D:\Silly\CL - P\dist" "D:\Silly\CL -
Nuitka-Options: P\Cl.py"
Nuitka: Starting Python compilation with Nuitka '2.2.1' on Python '3.11' commercial grade 'not installed'.
Nuitka-Plugins:WARNING: options-nanny: Using module 'attr' (version 0.3.2) with incomplete support due to condition
Nuitka-Plugins:WARNING: 'version("attrs") < (23,1)': 'attrs' is not fully supported before version 23.1.0
Nuitka-Plugins:options-nanny: Using module 'trio._core._ki' (version 0.22.0) with incomplete support: Disabled careful
Nuitka-Plugins:options-nanny: handling of KeyboardInterrupt in 'trio'
Nuitka-Plugins:options-nanny: Note, when using 'tkinter', consider using '--disable-console' option. Otherwise a
Nuitka-Plugins:options-nanny: terminal window will open. However for debugging, terminal output is the easiest way to
Nuitka-Plugins:options-nanny: see informative traceback and error information, so delay this until your program is
Nuitka-Plugins:options-nanny: working and remove once you find it non-working, and use '--enable-console' to make it
Nuitka-Plugins:options-nanny: explicit and not see this message.
Nuitka: Completed Python level compilation and optimization.
Nuitka: Generating source code for C backend compiler.
Nuitka: Running data composer tool for optimal constant value handling.
Nuitka: Running C compilation via Scons.
Nuitka-Scons: Non downloaded winlibs-gcc 'D:\llvm-mingw-20240502-ucrt-x86_64\bin\gcc.exe' is being ignored, Nuitka is
Nuitka-Scons: very dependent on the precise one.
Nuitka will use gcc from MinGW64 of winlibs to compile on Windows.

Is it OK to download and put it in 'C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1'.

Fully automatic, cached. Proceed and download? [Yes]/No :
Nuitka: Downloading
Nuitka: 'https://github.com/brechtsanders/winlibs_mingw/releases/download/13.2.0-16.0.6-11.0.1-msvcrt-r1/winlibs-x86_64-posix-seh-gcc-13.2.0-llvm-16.0.6-mingw-w64msvcrt-11.0.1-r1.zip'.
Nuitka: Extracting to
Nuitka: 'C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\gcc.exe'
Nuitka-Scons: Backend C compiler: gcc (gcc 13.2.0).
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from __constants.c:2:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [__constants.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.OpenSSL.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.OpenSSL.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.OpenSSL._util.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from __helpers.c:4:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.OpenSSL._util.o] Error 1
scons: *** [__helpers.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.OpenSSL.SSL.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.OpenSSL.SSL.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.OpenSSL.crypto.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.OpenSSL.version.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.OpenSSL.crypto.o] Error 1
scons: *** [module.OpenSSL.version.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.async_generator.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.async_generator.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.attr._compat.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.attr._compat.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.async_generator._version.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.async_generator._version.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.__parents_main__.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.async_generator._impl.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.async_generator._util.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.__parents_main__.o] Error 1
scons: *** [module.async_generator._impl.o] Error 1
scons: *** [module.async_generator._util.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.attr._cmp.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.attr._cmp.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.__main__.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.__main__.o] Error 1
In file included from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/exceptions.h:10,
                 from C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/prelude.h:431,
                 from module.attr._config.c:19:
C:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\Lib\SITE-P~1\nuitka\build\include/nuitka/constants.h:216:10: fatal error: __constants.h: No such file or directory
  216 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

scons: *** [module.attr._config.o] Error 1
FATAL: Failed unexpectedly in Scons C backend compilation.
Nuitka:WARNING:     Complex topic! More information can be found at
Nuitka:WARNING: https://nuitka.net/info/scons-backend-failure.html
Nuitka-Reports: Compilation crash report written to file 'nuitka-crash-report.xml'.

I wonder if the rejected gcc is causing this, or if the output directory with spaces is causing it, it seems the source files are found, and the constants file would be generated in the same spot.

Can you provide the output when compiling with --show-scons option please?

I think I understand the reason for this issue now. It seems to be because the path contains Chinese characters.

我想知道是被拒绝的gcc导致了这种情况，还是带有空格的输出目录导致了这种情况，似乎找到了源文件，并且常量文件将在同一位置生成。

I wonder if the rejected gcc is causing this, or if the output directory with spaces is causing it, it seems the source files are found, and the constants file would be generated in the same spot.

I think I understand the reason for this issue now. It seems to be because the path contains Chinese characters.

Which you carefully removed from your report, right? I would be interested where they show up, because they are supposed not to, and --show-scons will help me see it.

log.txt

Which you carefully removed from your report, right? I would be interested where they show up, because they are supposed not to, and --show-scons will help me see it.

Thanks, very helpful. It's actually that the --output-dir is chinese, so result_name and result_exe get passed with Chinese characters to scons. That happens in both the path and the base filename. I am not sure it would resolve that properly, but likely it will, but that's only at link time.
The file the C compiler does not find however, strangely lives in . and I don't see how that could go wrong. I will have to try and use --output-dir with that kind of path, maybe I can then reproduce it.

Thanks, very helpful. It's actually that the --output-dir is chinese, so result_name and result_exe get passed with Chinese characters to scons. That happens in both the path and the base filename. I am not sure it would resolve that properly, but likely it will, but that's only at link time.
The file the C compiler does not find however, strangely lives in . and I don't see how that could go wrong. I will have to try and use --output-dir with that kind of path, maybe I can then reproduce it.

If there's anything I can assist you with, please let me know.

I managed to reproduce an issue for ccache with this, compiling in a chinese directory, is translated well, but the output dir is not, it seems: ccache: error: Failed to write to C:\Users\kayha\repos\CCA1E~1\???C\dist\AssertsTest.build\ccache-34228.txt: Invalid argument, so that's an angle to resolve things.

So, the issue is that . is expanded in scons, to the full chinese path. For me, result_exe and result_name are not chinese, but correct, not sure why they are for you, maybe the final program filename is it. That would indeed possibly pose a challenge hard to overcome.

Using a chinese filename, I am getting this from MinGW

__loader.c:40:1: warning: hex escape sequence out of range

It seems a value of 344828 is used there, according to MSVC.
That seems to only happen in accelerated mode, where the original paths are stored for use inside the binary, and it reveals a mistake in how unicode strings are encoding for use in C there.

So, next up the CMD script couldn't handle unicode, but that can be worked around. With that MSVC is working fine. But MinGW64 will not handle -o C:\Users\kayha\repos\CCA1E~1\CCA1E~1\dist\开启.exe and will need some help to do that it seems, like we supply a short path, and pre-create the file, so the filenames alias, but I am not sure that can work.

So, I am adding workarounds, basically enforcing the use of short names for MinGW64 but we need to create an empty file to know the shortname, and then it seems we can make MinGW64 overwrite it. With that, executables work with Chinese result names, modules I also changed, but for some reason, we end up having C++ detected as the language or our C files, and this gives an error. No idea why yet, but it could well fail to detect things inside of unicode paths there.

So, cleaning up things, Nuitka in Scons now checks if the target filename is encodable, and if not, changes it to "_nuitka_temp.pyd" or "_nuitka_temp.exe" and after successfully compiling, uses a rename operation to go to the real needed name.
Then, it seems to be a similar problem, where dependency walker was using the basename of the created binary, for temp files it uses, so we do something similar there, these names are only there to easily associate things for debugging, so onefile works now in this setup too.
One issue remaining, is that MinGW64 doesn't recognize files as C from their name, e.g. C:\Users\kayha\repos\CCA1E~1\CCA1E~1\dist\4ADC~1.BUI\MODULE~1.C when I use a Chine output directory, I am researching that now.

So, compiling now works for modules even with output directories, but now I get this:
Nuitka: Launching '键开启C\dist\开启.cp310-win_amd64.pyd'.
Traceback (most recent call last):
  File "<string>", line 2, in <module>
ImportError: dynamic module does not define module export function (PyInitU_drrp5w)

The name of the init function and the module name have to match. It seems that U_drrp5w is some sort of encoding that Python uses, most likely DLL entry points, cannot be unicode, and that's their way of working around that, but Nuitka doesn't yet know that kind of code.

So, it seems it uses something called punycode for the entry points, and now it was complaining about it not returning a 2 phase init object, and not having per sub-interpreter support for new style loads. So I added that as well, and now it works for extension modules. The created extension modules now load fine into the interpreter, but I expect that Nuitka standalone won't be able to make use of them until we mirror the entry point name change there too.

So, I added that entry point change, and the 2 phase module load successfully as well. And then there was also an issue with the filename of DLL being wrong, which has been corrected as well. At some point, I also saw the original error you had, since it's using `.' as source_dir in some configurations, and otherwise the source_dir not shortened, this can happen as well.
I am putting this through my internal CI now, and then we should have something which is a lot closer to working for many more Chinese setups. I am still a bit confused why result_exe had non-short paths in there. I am suspecting a bug in the caching of short paths, that I want to look at.

Thanks for your report, this is fixed on the factory branch, which is a development version under rapid development. You can try it out by going here: https://nuitka.net/doc/factory.html
Feedback on whether this is working is very welcome. Please do not share plans to do it; only confirm or deny that it is working.

It would seem, that you should be able to compile in your environment with the exact options successfully now with the factory version. Let me know if that's true. When something fails, I would love to have a report. I found huge amounts of fixes, surprising, mostly related to unicode output filenames, extension modules, etc. and I would love to see if things improved for you. I care much about out of the box experience for Nuitka on Asian systems.

Not working under Macos too, you can reproduce the problem like this:
with open("test.txt","a+") as fp: fp.write("测试")

That doesn't look like Python code, but a data file, when I renamed it to .py which is what you intended probably, I indeed got a crash from the NameError. So that's more like core Nuitka, checking it out.

So, I have updated factory with fixes for this. The code was assuming it can always take a UTF8 string for the exception, but that's not true, and for Python3 there is actually a better method to use, so I applied this everywhere I saw similar code, function repr, class names, method repr, local variable errors, etc.
This actually could be part of a hotfix, but I feel that 2.3 is approaching fast, and I had to make an emergency one for PySide6 today, so I don't think there will be another one necessarily, and then it's not work the risk. Right now my CI has better assertions on, and you can try --debug if you still have crashes, should catch those cases better.

Strangely, it work well in debug mode and I can not get any crash log.

I found that I can execute test.app/Contents/main, but cant run test.app directly.  Maybe the console-less mode caused breaking down?

Actually `--disable-console´ has had no effect on macOS for a while now. What do you mean with "directly"? How sure are we that macOS itself handles this fine, or that signing with Chinese filenames works. But it seems you are hijacking this issue, which was about Windows. Please make a new issue @susundingkai esp one with a reproducer.

Actually `--disable-console´ has had no effect on macOS for a while now. What do you mean with "directly"? How sure are we that macOS itself handles this fine, or that signing with Chinese filenames works. But it seems you are hijacking this issue, which was about Windows. Please make a new issue @susundingkai esp one with a reproducer.

I mean, open the test.app folder, find the executable file, and double click it to execute the file. It will open a console and run without crash.

The variable names related stuff is now on develop and will be in 2.3 release as well, which is bound to happen any day now.

Thanks for your report, this is fixed on the factory branch, which is a development version under rapid development. You can try it out by going here: https://nuitka.net/doc/factory.html
Feedback on whether this is working is very welcome. Please do not share plans to do it; only confirm or deny that it is working.

I tried installing the factory branch using pip install -e git+https://github.com/Nuitka/Nuitka.git@factory#egg=nuitka, but after installation, Nuitka still couldn't find __constants.h.

That's too bad, can you provide the output with --show-scons again?

That's too bad, can you provide the output with --show-scons again?

log.txt

Ok, I think it's recognizable, that the translation to short paths somehow fails there. Both result_exe and nuitka_src are being wrong. The call of the scons file and the Python for the scons itself, might also be wrong. Maybe Nuitka does caching that is broken or maybe the shortname translation fails for your specific paths. I am seeing that there is this code:
            # Permission denied.
            if ctypes.GetLastError() == 5:
                return filename

Can you comment that out in nuitka.utils.FileOperation and see if you instead get a Python exception? Not sure, why permission could be denied there, but maybe short paths are not really always available to use and would require priviledge.

Ok, I think it's recognizable, that the translation to short paths somehow fails there. Both result_exe and nuitka_src are being wrong. The call of the scons file and the Python for the scons itself, might also be wrong. Maybe Nuitka does caching that is broken or maybe the shortname translation fails for your specific paths. I am seeing that there is this code:
            # Permission denied.
            if ctypes.GetLastError() == 5:
                return filename

Can you comment that out in nuitka.utils.FileOperation and see if you instead get a Python exception? Not sure, why permission could be denied there, but maybe short paths are not really always available to use and would require priviledge.

I tried commenting out the code, but the same error occurred.
log.txt

I don't really know what could be causing this. My current fixes are part of 2.3 release, but it seems there is something to get to the bottom of still.
There is a line options["nuitka_src"] = getExternalUsePath(options["nuitka_src"]) and that seems to either not run for you, but I am seeing the impact of options["source_dir"] = "." so I think it is, and somehow said function, which you commented out code on its path for me now, is not running, or we are being dumb. But the version information seems to imply you have the same code as I there. Use 2.3 now and let me know, if you manage to somehow figure out how it gets to return an absolute path with Chinese characters there.

I don't really know what could be causing this. My current fixes are part of 2.3 release, but it seems there is something to get to the bottom of still.
There is a line options["nuitka_src"] = getExternalUsePath(options["nuitka_src"]) and that seems to either not run for you, but I am seeing the impact of options["source_dir"] = "." so I think it is, and somehow said function, which you commented out code on its path for me now, is not running, or we are being dumb. But the version information seems to imply you have the same code as I there. Use 2.3 now and let me know, if you manage to somehow figure out how it gets to return an absolute path with Chinese characters there.

It outputted:
Before: D:\Silly\真一键脚本 - 抢先版\venv\src\nuitka\nuitka\build
After: D:\Silly\真一键~3\venv\src\nuitka\nuitka\build

I tried using D:\Silly\真一键~3, which is the correct short path for Chinese characters.

Ah, it is? But I have serious doubts that encoding-wise, MinGW64 does well there. Short Chinese paths are a huge surprise to me. I thought the whole idea was not for them to be short but for them to be ASCII. I need to study the MS docs there.

So, it seems nothing in the docs guarantees this to be anything but a shorter path, lengthwise. However, in my testing, e.g., Chinese extension module names became usable through this. Here, the current directory is where the included file lives, and we switch to the short path. Then GCC reads it and seemingly cannot handle it, with it being Chinese still. So, where we use "." there, not even relative paths are working out.
What's striking, though, is that this is the only relative path used, and that should cause it to not be found. That's very strange, is it not?
In this command line:
"C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\ccache.exe" "C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\gcc.exe" -o module.attr.setters.o -c -std=c11 -fvisibility=hidden -fwrapv -pipe -fpartial-inlining -ftrack-macro-expansion=0 -Wno-deprecated-declarations -fno-var-tracking -Wno-misleading-indentation -fcompare-debug-second -O3 -D_WIN32_WINNT=0x0501 -D__NUITKA_NO_ASSERT__ -DMS_WIN64 -D_NUITKA_STANDALONE -D_NUITKA_ONEFILE_MODE -D_NUITKA_ONEFILE_TEMP_BOOL -D_NUITKA_CONSTANTS_FROM_RESOURCE -D_NUITKA_FROZEN=153 -D_NUITKA_EXE -D_NUITKA_PLUGIN_MULTIPROCESSING_ENABLED=1 -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\inline_copy\zlib -IC:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\include -I. -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include -

The part not working there would be the -I. part. The absolute paths to parts of Nuitka did come into play already, it's loading __constants.h from that.
See this
D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka/constants.h:220:10: fatal error: __constants.h: No such file or directory
  220 | #include "__constants.h"

So either this file really doesn't exist in the build folder, please confirm, or the fact that it is . which should be a short path too, somehow gets lost, and then that's an issue for MinGW64. I swear I saw MinGW64 fail when Nuitka did live in a Chinese path, and making it short helped.
To verify, you could change this:
            options["source_dir"] = "."
            options["nuitka_src"] = getExternalUsePath(options["nuitka_src"])

to
            options["source_dir"] = getExternalUsePath(options["source_dir"])
            options["nuitka_src"] = getExternalUsePath(options["nuitka_src"])

That should change the -I. and make it an absolute path, too. These are actively avoided to achieve caching across different build folders, but it might be what triggers the issue for you. The Chinese short paths seem to work. Maybe MinGW64 creates a long form path out of . or the Windows process forked this way:
            if source_dir is not None:
                _createSconsDebugScript(
                    source_dir=source_dir, scons_command=scons_command
                )

                source_dir = getExternalUsePath(source_dir)
            try:
                result = subprocess.call(scons_command, shell=False, cwd=source_dir)
            except KeyboardInterrupt:
                Tracing.scons_logger.sysexit("User interrupted scons build.")

So, please if you can try these things, I think I am very close to understanding this one.

So, it seems nothing in the docs guarantees this to be anything but a shorter path, lengthwise. However, in my testing, e.g., Chinese extension module names became usable through this. Here, the current directory is where the included file lives, and we switch to the short path. Then GCC reads it and seemingly cannot handle it, with it being Chinese still. So, where we use "." there, not even relative paths are working out.
What's striking, though, is that this is the only relative path used, and that should cause it to not be found. That's very strange, is it not?
In this command line:
"C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\ccache.exe" "C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\gcc.exe" -o module.attr.setters.o -c -std=c11 -fvisibility=hidden -fwrapv -pipe -fpartial-inlining -ftrack-macro-expansion=0 -Wno-deprecated-declarations -fno-var-tracking -Wno-misleading-indentation -fcompare-debug-second -O3 -D_WIN32_WINNT=0x0501 -D__NUITKA_NO_ASSERT__ -DMS_WIN64 -D_NUITKA_STANDALONE -D_NUITKA_ONEFILE_MODE -D_NUITKA_ONEFILE_TEMP_BOOL -D_NUITKA_CONSTANTS_FROM_RESOURCE -D_NUITKA_FROZEN=153 -D_NUITKA_EXE -D_NUITKA_PLUGIN_MULTIPROCESSING_ENABLED=1 -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\inline_copy\zlib -IC:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\include -I. -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include -

The part not working there would be the -I. part. The absolute paths to parts of Nuitka did come into play already, it's loading __constants.h from that.
See this
D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka/constants.h:220:10: fatal error: __constants.h: No such file or directory
  220 | #include "__constants.h"

So either this file really doesn't exist in the build folder, please confirm, or the fact that it is . which should be a short path too, somehow gets lost, and then that's an issue for MinGW64. I swear I saw MinGW64 fail when Nuitka did live in a Chinese path, and making it short helped.
To verify, you could change this:
            options["source_dir"] = "."
            options["nuitka_src"] = getExternalUsePath(options["nuitka_src"])

to
            options["source_dir"] = getExternalUsePath(options["source_dir"])
            options["nuitka_src"] = getExternalUsePath(options["nuitka_src"])

That should change the -I. and make it an absolute path, too. These are actively avoided to achieve caching across different build folders, but it might be what triggers the issue for you. The Chinese short paths seem to work. Maybe MinGW64 creates a long form path out of . or the Windows process forked this way:
            if source_dir is not None:
                _createSconsDebugScript(
                    source_dir=source_dir, scons_command=scons_command
                )

                source_dir = getExternalUsePath(source_dir)
            try:
                result = subprocess.call(scons_command, shell=False, cwd=source_dir)
            except KeyboardInterrupt:
                Tracing.scons_logger.sysexit("User interrupted scons build.")

So, please if you can try these things, I think I am very close to understanding this one.

I think I may have discovered something. I checked the D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka path, but I did not see __constants.h. (It's also possible I don't understand C code? Is it normal for this file to be missing?)

So next, I will try modifying getExternalUsePath(options["source_dir"]).

No, that would be normal, it's supposed to exist in the build folder, where it is generated into for the specific program.
When I reproduced the other issues, it was not true, but because MinGW64 can not use non-Unicode paths for that build folder. It is a short path, but still Chinese, I think, is what is causing the symptoms. However, I also observed, that it is the only path that is not used absolute, and that short Chinese paths appear to be working.
I don't understand why I didn't have short Chinese paths. Maybe because I have a non-Chinese locale, my NTFS doesn't do them at all. Maybe then work with a Chinese locale, but not as relative paths, or something like that. That is why I suggested the change to make, and also kind of expect it to help your case.

So, it seems nothing in the docs guarantees this to be anything but a shorter path, lengthwise. However, in my testing, e.g., Chinese extension module names became usable through this. Here, the current directory is where the included file lives, and we switch to the short path. Then GCC reads it and seemingly cannot handle it, with it being Chinese still. So, where we use "." there, not even relative paths are working out.
What's striking, though, is that this is the only relative path used, and that should cause it to not be found. That's very strange, is it not?
In this command line:
"C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\ccache.exe" "C:\Users\ASUS\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\gcc.exe" -o module.attr.setters.o -c -std=c11 -fvisibility=hidden -fwrapv -pipe -fpartial-inlining -ftrack-macro-expansion=0 -Wno-deprecated-declarations -fno-var-tracking -Wno-misleading-indentation -fcompare-debug-second -O3 -D_WIN32_WINNT=0x0501 -D__NUITKA_NO_ASSERT__ -DMS_WIN64 -D_NUITKA_STANDALONE -D_NUITKA_ONEFILE_MODE -D_NUITKA_ONEFILE_TEMP_BOOL -D_NUITKA_CONSTANTS_FROM_RESOURCE -D_NUITKA_FROZEN=153 -D_NUITKA_EXE -D_NUITKA_PLUGIN_MULTIPROCESSING_ENABLED=1 -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\inline_copy\zlib -IC:\Users\ASUS\AppData\Local\Programs\Python\PYTHON~1\include -I. -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include -

The part not working there would be the -I. part. The absolute paths to parts of Nuitka did come into play already, it's loading __constants.h from that.
See this
D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka/constants.h:220:10: fatal error: __constants.h: No such file or directory
  220 | #include "__constants.h"

So either this file really doesn't exist in the build folder, please confirm, or the fact that it is . which should be a short path too, somehow gets lost, and then that's an issue for MinGW64. I swear I saw MinGW64 fail when Nuitka did live in a Chinese path, and making it short helped.
To verify, you could change this:
            options["source_dir"] = "."
            options["nuitka_src"] = getExternalUsePath(options["nuitka_src"])

to
            options["source_dir"] = getExternalUsePath(options["source_dir"])
            options["nuitka_src"] = getExternalUsePath(options["nuitka_src"])

That should change the -I. and make it an absolute path, too. These are actively avoided to achieve caching across different build folders, but it might be what triggers the issue for you. The Chinese short paths seem to work. Maybe MinGW64 creates a long form path out of . or the Windows process forked this way:
            if source_dir is not None:
                _createSconsDebugScript(
                    source_dir=source_dir, scons_command=scons_command
                )

                source_dir = getExternalUsePath(source_dir)
            try:
                result = subprocess.call(scons_command, shell=False, cwd=source_dir)
            except KeyboardInterrupt:
                Tracing.scons_logger.sysexit("User interrupted scons build.")

So, please if you can try these things, I think I am very close to understanding this one.

I have changed the code here, but the error still occurs. I noticed that the log indicates Nuitka is still using the short path, which might be the problem. Additionally, I found that after modifying the code, the .build and .dist folders were correctly generated in the dist directory, which wasn't happening before.

It seems that the __constants.h and __constants.c files are there, yet the MinGW64 does not find them in an include path scan, although it does find the request to include nuitka_src from a Chinese short path.
We lack an explanation of how it could be. My guess that . is not good for them, maybe expanding that to a long Chinese path it cannot handle, should be out the Window, if you used the external path for it, i.e. passed as absolute. I am out of ideas honestly.

log.txt

It seems that the __constants.h and __constants.c files are there, yet the MinGW64 does not find them in an include path scan, although it does find the request to include nuitka_src from a Chinese short path.
We lack an explanation of how it could be. My guess that . is not good for them, maybe expanding that to a long Chinese path it cannot handle, should be out the Window, if you used the external path for it, i.e. passed as absolute. I am out of ideas honestly.

I have uploaded the log, which might be helpful. I am now wondering why the compiler is still using the short path for lookup even after using getExternalUsePath(options["source_dir"]).

That is what it does, turning a path absolute and making it short. Long chinese paths are known to not work with MinGW64, and it seems short paths always work.
The gist of the issue is here:
-ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\static_src -ID:\Silly\真一键~3\venv\src\nuitka\nuitka\build\inline_copy\libbacktrace module.attr.setters.c

Notice, how it tells it to look in short paths, and how MinGW64 does so successfully:
In file included from D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka/exceptions.h:10,
                 from D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka/prelude.h:464,
                 from __helpers.c:4:
D:\Silly\真一键~3\venv\src\nuitka\nuitka\build\include/nuitka/constants.h:220:10: fatal error: __constants.h: No such file or directory

Then I fails to find this file:
  220 | #include "__constants.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.

And that is from -I. which it ought to not use I guess, because this is asking it to go from the current directory (which is a short chinese path, and it seems to fail at that. This is source_dir though and if you did the changes I suggest, it would not be "." but an external path. I think you maybe didn't do what I asked you to do this time? Make sure to not see -I. in the scons output.
Yours,
Kay

It's Python code, I cannot remote debug it. It could be that shallCompileWithoutBuildDirectory is not true for you, if you use --debugger, that would be the case.

It's Python code, I cannot remote debug it. It could be that shallCompileWithoutBuildDirectory is not true for you, if you use --debugger, that would be the case.

To be honest, I think Nuitka has done its job; the issue seems to lie with MinGW64. Can you provide a compressed package of MinGW64 for me to check if the issue is with my MinGW64?

The URL for download is in the source code of Nuitka. If you are not using that exact MinGW64, then of course you may have bugs that I do not have. It being buggy is unfortunately extremely common, which is why Nuitka rejects non-downloaded gccs on Windows, if you cheated there, then of course it's all off and a waste of time. In case it's not, too bad, but I will have to wait for a report with a genuine install.

The URL for download is in the source code of Nuitka. If you are not using that exact MinGW64, then of course you may have bugs that I do not have. It being buggy is unfortunately extremely common, which is why Nuitka rejects non-downloaded gccs on Windows, if you cheated there, then of course it's all off and a waste of time. In case it's not, too bad, but I will have to wait for a report with a genuine install.

If you mean the MingW64 from https://github.com/brechtsanders/winlibs_mingw/releases/download/13.2.0-16.0.6-11.0.1-msvcrt-r1/winlibs-x86_64-posix-seh-gcc-13.2.0-llvm-16.0.6-mingw-w64msvcrt-11.0.1-r1.zip, I've already tried it. 😥
log.txt

You have not done what's needed to not have -I in the scons calls of the gcc. My theory is that it will be better, but it seems you do not manage. That's OK, lets wait until somebody with the same issue can.

You have not done what's needed to not have -I in the scons calls of the gcc. My theory is that it will be better, but it seems you do not manage. That's OK, lets wait until somebody with the same issue can.

I upgraded Nuitka to version 2.3.11 and changed MinGW to use 13.2.0-16.0.6-11.0.1-msvcrt-r1. If I'm still doing something wrong, it seems I might not be suited to using Nuitka.

You don't have to change anything about MinGW, but let it download. There will be another chance once I switch Nuitka to an Anaconda-based setup in the 2.5 time frame, and maybe their gcc, but yeah, let's give up for now. Even though your report is great, your debugging ability is not there, and my remote debugging skills are equally insufficient.


feat: add guest binding for wasi-http

We are migrating the code from here that is implementing a basic HTTP client on top of the generated binding: https://github.com/bytecodealliance/preview2-prototyping/blob/47b30fdad11553841bbee4c5632101bbd645854b/wasi/src/http.rs
And based on the example from the wasmtime repository: https://github.com/bytecodealliance/wasmtime/blob/362696a2d363d583887a726d979944d221eb5b4b/crates/test-programs/wasi-http-tests/src/bin/outbound_request.rs

Hey, sorry, this one just popped up on my feed despite it being here for a little while.
I haven't yet figured out a plan for how we are upstreaming the p2-p stuff into this crate. I suppose, for wasi-http, that since we are publishing a new crate and api, we can just go ahead and do it.
Upstreaming the functionality that currently lives in the wasi crate is a little bit trickier, since we have existing users there.
@alexcrichton do you have any objections to creating this new wasi-http crate here, with these more-idiomatic bindings on top of the wasi-http wit bindgen?

I finally got another PR working by using the guest binding from here, so the moment we have a new version published I can move forward.

This all seems useful to have somewhere for sure, but I feel that this is pretty significantly diverging from what the wasi crate is doing at least. The wasi crate is effectively a "glorified *-sys crate" where it's very low-level, mostly auto-generated, and gives you effectively the raw functionality of the preview1 APIs. This new wasi-http crate, however, isn't doing that and is instead intentionally hiding the wit-bindgen-generated functionality and only exposing interop with the http crate.
Given the precedence of wasi I'd expect that the wit-bindgen-generated code would be the main public API of the wasi-http crate, with perhaps a different crate performing http interop or an optional off-by-default feature on the wasi-http crate to build in support.
There's also sort of the question of "are things ready yet" which I don't know how best to answer. This crate is relatively stable/official for preview1 but preview2 and its related support/etc aren't quite at that same level yet. I don't know how best to bridge that gap. One route is something like this where it's largley "just land it and let everyone deal with breakage later", but there's also the possibility of doing something more nuanced/measured where this is fit into a larger plan of rolling out preview2, evalutating it, etc. Or perhaps such a plan already exists and I'm not aware, unsure!

@alexcrichton , I pushed some changes as per your suggestion by exposing the HTTP related interfaces (those would probably make more sense the wasi crate when it lands here). Also, I hid under a non-default feature flag the default HTTP client.
I believe the wasi-http will match the http crate in some sense and, later when preview2 stabilizes, we can build functionality for hyper, reqwest, surf, ureq, etc, (in their own respective repositories) when targeting wasm-wasi-preview2 on top of this crate.

In DM and in https://github.com/bytecodealliance/preview2-prototyping/pull/157 Eduardo and I discussed that we don't really have a great way to land preview 2 stuff in the wasi crate yet. We are going to defer figuring that out for a while by publishing the preview 2 version of the wasi crate as wasi-preview2-prototype. So, I think we are ok to close this PR for now, and revisit what to do with wasi and wasi-http in a little while.


Further improve entity memory footprint

Storing components in a dictionary is slow, that's why Entitas is using arrays and stores components at a certain index. This is super fast for the cost of memory, because each entity is created with an array with a capacity of the total number of components. Currently we can use multiple pools and assign components to one or more pools with Attributes like [Core] or [UI] to minimize the initial capacity of the array.
It would be nice to find or create a data structure which doesn't need a initial capacity and can grow while at the same time reaches array-like performance.

@mzaks @cloudjubei @arne-schroppe

I still think that a hash map with lazily created heaps as values is a promising idea. The hash function could simply be bit-masking on the component id. So every component id that ends in e.g. 0b010 would end up in the same heap.
But we need to keep in mind that this is not a completely uniform hash function, since the number of components will probably not be something that ends in all 1's, so everything below num_component_ids & 0b111 will potentially have one more entry. And the hash table itself also needs extra memory, so we might even end up using more memory with this method.
One way to mitigate this could be, that we expect the heaps not to be filled up completely. So to go with the current example, if we hash with the last three bits and thereby potentially have 8 entries per heap, we would at first only allocate space for 4 entries, hoping that the remaining slots will never be used. Whether that is a good idea and what numbers to use exactly is something we'd have to measure in a couple of real applications.
All in all I'm not sure if this is worth it, though. We might end up using more memory and we're going from 1 O(1) access + 1 dereference to  O(log n) access + 2 dereferences (first from hash table to heap and then from heap entry to component data). But it sure sounds like a fun experiment :)

Actually now that I think about it: @antonlicht's idea of simply using a heap sounds way better. We would start with a capacity for maybe 20% of the total number of components (the best percentage would still need to be measured in real applications). This means that we might need to reallocate, but that should ideally not happen too often. Access would still be O(log n) instead of O(1) but even with 200 different components that would be at most 8 comparisons, so not really much.

Will close becaus of #112


what does the code line: "r3d200_KM_200ep.pth: --model resnet --model_depth 200 --n_pretrain_classes 1039" mean ?

I am really confused about the code:
r3d200_KM_200ep.pth: --model resnet --model_depth 200 --n_pretrain_classes 1039
what does this really mean? what should I do if I want to load a pre-trained  model ?

I am a new learner about this, something maybe very common for you guys but confused for me. Someone can help me with this? thanks very much.


ValueError occurs when try to evaluate task "bigbench_multiple_choice"

Hello,
I'm trying to evaluate some hf🤗 models on lm-eval. When I use the "bigbench_multiple_choice" task, I encounter a ValueError in certain subtasks. I'd appreciate help with resolving this.
below is my script
lm_eval --model hf \     --model_args pretrained=allenai/OLMo-7B,trust_remote_code=true \     --tasks bigbench_multiple_choice \     --device cuda:0 \     --batch_size auto \     --log_samples \     --output_path logit_result
I also tried with some other models(llama 3, mistral ), but it still makes error
like
"Task: bigbench_conlang_translation_multiple_choice] has_training_docs and has_validation_docs are False, using test_docs as fewshot_docs but this is not recommended.
....
File "
However, when I select only one specific task, for example, "bigbench_implicit_relations_multiple_choice," it runs without any issues. I suspect the error might be occurring during the task configuration stage. Do you have any ideas?
my environment is like below with RTX A6000 GPU, UBUNTU 22.04
absl-py                  2.1.0
accelerate               0.29.3
aiohttp                  3.9.5
aiosignal                1.3.1
attrs                    23.2.0
certifi                  2024.2.2
chardet                  5.2.0
charset-normalizer       3.3.2
click                    8.1.7
colorama                 0.4.6
DataProperty             1.0.1
datasets                 2.18.0
dill                     0.3.8
evaluate                 0.4.1
filelock                 3.13.4
frozenlist               1.4.1
fsspec                   2024.2.0
huggingface-hub          0.22.2
idna                     3.7
Jinja2                   3.1.3
joblib                   1.4.0
jsonlines                4.0.0
lm_eval                  0.4.2       /home/ohs/eval/lm-evaluation-harness
lxml                     5.2.1
MarkupSafe               2.1.5
mbstrdecoder             1.1.3
more-itertools           10.2.0
mpmath                   1.3.0
multidict                6.0.5
multiprocess             0.70.16
networkx                 3.3
nltk                     3.8.1
numexpr                  2.10.0
numpy                    1.26.4
nvidia-cublas-cu12       <IP_ADDRESS>
nvidia-cuda-cupti-cu12   12.1.105
nvidia-cuda-nvrtc-cu12   12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12        <IP_ADDRESS>
nvidia-cufft-cu12        <IP_ADDRESS>
nvidia-curand-cu12       <IP_ADDRESS>
nvidia-cusolver-cu12     <IP_ADDRESS>
nvidia-cusparse-cu12     <IP_ADDRESS>
nvidia-nccl-cu12         2.19.3
nvidia-nvjitlink-cu12    12.4.127
nvidia-nvtx-cu12         12.1.105
packaging                24.0
pandas                   2.2.2
pathvalidate             3.2.0
peft                     0.10.0
pip                      24.0
portalocker              2.8.2
psutil                   5.9.8
pyarrow                  15.0.2
pyarrow-hotfix           0.6
pybind11                 2.12.0
pytablewriter            1.2.0
python-dateutil          2.9.0.post0
pytz                     2024.1
PyYAML                   6.0.1
regex                    2024.4.16
requests                 2.31.0
responses                0.18.0
rouge-score              0.1.2
sacrebleu                2.4.2
safetensors              0.4.3
scikit-learn             1.4.2
scipy                    1.13.0
setuptools               65.5.0
six                      1.16.0
sqlitedict               2.1.0
sympy                    1.12
tabledata                1.3.3
tabulate                 0.9.0
tcolorpy                 0.1.4
threadpoolctl            3.4.0
tokenizers               0.19.1
torch                    2.2.2
tqdm                     4.66.2
tqdm-multiprocess        0.0.11
transformers             4.41.0.dev0 /home/ohs/eval/transformers
triton                   2.2.0
typepy                   1.3.2
typing_extensions        4.11.0
tzdata                   2024.1
urllib3                  2.2.1
word2number              1.1
xxhash                   3.4.1
yarl                     1.9.4
zstandard                0.22.0`

I think the error is caused at here (or anywhere else) line 9: doc_to_target: "{{multiple_choice_targets.index(targets[0])}}"
I think the index method raises the ValueError

Oh, I found there are some subtasks that are not fit in the format.
I did not conduct a complete survey, but some of the data from above and the data I encountered problems with are as follows.
strategyqa_zero_shot
ascii_word_recognition_zero_shot

https://github.com/EleutherAI/lm-evaluation-harness/pull/1686


In FoldingPresets only Top Left, Top Wide, Left Wide are available.

Hello.
Can you help me why only Top Left, Top Wide, Left Wide are available in FoldingPresets?
I am using v4.3.stable.mono.official [77dcf97d8]
I tried everything and copied what you show in the tutorial video but it doesn't change.

If you click the TIPS button shown in the screenshot (bottom right button in the FoldingPresets) it will print this out in the console:
For more FoldingDirections: parent to a Container (a MarginContainer is recommended). Ensure that the Container has the same minimum size as this node for the intended effect. (Copy and paste the custom_minimum_size from this inspector into the MarginContainer's custom_minimum_size!)
In other words, just parent the collapsible container to any Container node, and make sure the parent Container node has the same minimum size as the collapsible container (when it is fully open).
The tutorial video is older than the FoldingDirections feature, so it has different steps to reproduce this effect.

Ahh yes I missed that. Thank you!


Remove edge case code for clusterID fetching

What?
We recently updated the PD service orchestrations and alertmanager configurations to all be consistent. The cluster_id is now always in alert.custom_details.cluster_id. Therefore, we no longer need edge case handling per service to retrieve the cluster_id.
I renamed all occurences of external id to cluster id, as we have don't know which id is contained in the cluster_id field, per example for CPD it's the internal one, for CHGM it's the external one. This is already handled by CAD and always converted to internal_id in the first parts of the investigation.go code.
I also had to extract custom details parsing from the GetAlerts function, as it made no sense to have it in there. We now have a function that gets alerts for an incident, and a function that extracts the needed details from the alerts. The localAlert objects were renamed, as they are not alerts, they are details of alerts (only clusterID and alertID).
Why?
https://issues.redhat.com/browse/OSD-17557
Tests
Tested for SilenceAlerts and CHGM.

/hold
We should merge this after SilenceAlerts canary testing is complete.

/unhold

/label tide/merge-method-squash


Swipe left and swipe right does not update active index on expo web

You can see the issue on the official expo web example.
https://snack.expo.dev/@krissnawat/intro-slider-app

I'm also experiencing the same problem. Do you have any work-around to make this work?

@lelimit If i recall correctly, I just manually handled the gestures and set the slide index using a gesture handler.

How to detect gesture for horizontal scroll on web with middle button on mouse for example?

@lelimit On react native web? Using the gesture handler, I think it was https://docs.swmansion.com/react-native-gesture-handler/docs/api/gestures/gesture-detector/


QA Column: Previously signed-off pulls are at the bottom

Judging from Gilbert's commit message here, he wanted to move pulls with stale QA signoffs to the top of the queue, not the bottom.
https://github.com/iFixit/pulldasher/blob/master/views/standard/spec/columns.js#L207
By adding 1000 to the score, they're pretty much guaranteed to go to the bottom. This should be score -= 1000.

https://github.com/iFixit/pulldasher/blob/master/views/standard/spec/columns.js#L207