KHR_XMP_JSON_LD extension support

Is your feature request related to a problem? Please describe.
We want to support KHR_XMP_JSON_LD extension in the form of exposing the json string at the potential components (asset, scene, node, mesh, material, image, animation). But in reality we may limit the access to a few components based on the filament design.
Describe the solution you'd like

We first make sure we parse the information in cgltf.h and have code commit there. So cgltf.h contains the code to read this extension and also possible to write it out in cgltf_write.h

then in filament/libs/gltfio/include/gltfio/FilamentAsset.h introduce function std::vector<std::string> getKHRXmpJsonLd(Entity entity = {}) to expose the vector of the json_ld. Since it is possible to refer multiple index in the extension section of the component, the output is a vector.

This change is similar to how extra is handled in filament right now.

However, I notice that FilamentAsset is not designed to expose information on all the components. I cannot find way to expose, for example, extra for image, animation.  So how to expose the json_ld strings in all components for filament user to use need consider?

Describe alternatives you've considered

An alternatives is to let user to deal with cgltf data themselves. So we do not expose this extension data at FilamentAsset level.

OS and backend
This feature is not specific to a special backend/OS.

Filament is a renderer, and not a generic gltf library. The alternative solution above seem more appropriate.


nvim-lsp: Add settings for rust-analyzer

Because the settings are quite complex they are auto-generated. This commit adds the script creating it & the generated options.

By manually what do you mean ? All the options have défaut values that are filled in from the package.JSON
If ra chnages an option that would ne problematic, si I could wrap evrything in nullOr whit null default

Yeah I mean wrapping it in nullOr so that we don't have a massive init file generated for options that aren't set.
Just tried it out and just enabling rust-analyzer leads to an almost 6KB init file so definitely something to fix

I made the change to add 'nullOr' everywhere. Sometimes there may be some non-optimal behiviour with nullOr (nullOr ...). I can change that, but it would make the autogenerating script more complex

@pta2002 I made a significant change, I added a package in the top level flake in order to build the options with the package.json version that corresponds to the rust-analyzer in nixpkgs. I saw that my approach was not very resilient to settings rename


Release new, supported, listed version...

[LAST UPDATED - 14th May 2019]
Mighty 3.0.4 Release is now up on NuGet.
The documentation is complete!
Task list:

[x] Add async support to Mighty

[x] Upgrade entire project and tests, initially to async only (later merge back in sync as below)
[x] Add and test CancellationToken support
[x] Add async support to dynamic-only methods and to Mighty's Npgsql automatic cursor dereferencing code
[x] Merge the sync and async branches back together, to support both at once

[x] Support .NET Core 1.0 and 1.1 via .NET Standard 1.6 instead of directly as previously

The .NET Framework 4.0 and 4.5 versions are still provided and have config file and transaction scope support which isn't in the .NET Standard version (.NET 4.0 has no Async)

[x] Add support for .NET Core 2.0 via .NET Standard 2.0
[x] Add support for .NET Core 3.0
[x] Remove dynamic method support (First, Last, Find, Get, etc.) and support all the same functionality in non-dynamic methods:

Allows IntelliSense for all methods
Removes the overhead for all calls to Mighty (even non-dynamic onces) which having dynamic method support added

[x] Add XML comments to everything which didn't have them
[x] Decide on (NuGet) versioning issues
[x] Confirm everything is still running against the old Massive test suite

With any required minor changes documenting what the minor differences are!

[x] Put up a documentation website

[x] Jekyll documentation - working on finishing the remaining pages

[ ] Make a permanent dll of the old test suite to confirm that new builds remain binary compatible

Last item still to do above, but anyway: supported, listed version is now released.


Can you specify a graphics card in the ollama deployment model?

For example, qwen2 is in CUDA_VISIBLE_DEVICE=2 and glm4 is in CUDA_VISIBLE_DEVICE=4

You can do this if you run a separate ollama instance for each graphics card.


How to handle missing/incomplete fragments more elegantly

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I'm running yt-dlp version 2023.01.06 (update instructions) or later (specify commit)
[X] I've searched the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
First some observations regarding fragments:

--fragment-retries infinite bangs its head against the same wall forever
--abort-on-unavailable-fragments bails on the first sign of trouble
--keep-fragments floods the directory with hundreds of fragments
--no-keep-fragments iteratively merges fragments while downloading (but if a fragment is missing it will be left as a separate file, the merged video will lack this fragment, and video/audio/subtitles will be out of sync)

So, is there a way to tell yt-dlp:

If some fragments can't be downloaded, skip them
Focus on getting all the fragments that are available
Iteratively merge all sequential fragments that are complete
When finished, if some fragments are missing/incomplete, don't merge or delete anything
Let the user re-run the download at a later time to try to quickly get the (probably few) remaining fragments
Merge and delete once all fragments are available

Pseudo:
foreach video in batch
  failures = 0
  foreach fragment in video
    if fragment missing or incomplete
      get fragment, try=10
    if fragment complete
      merge fragment with preceding complete fragments
    else
      print "fragment # missing. download will continue, but merging will be skipped."
      failures++
  if failures == 0
    merge video
    delete fragments

Example of a condensed way to keep/merge fragments:
.part-Frag001-020
.part-Frag021.part
.part-Frag022-049
.part-Frag050.part
.part-Frag051-099

PS! When running a standard yt-dlp <video> that was merged, but has at least one .part-Frag###.part (in which case video/audio/subtitles will be out of sync), is there currently any way to re-run the download to get the missing fragment and merge with the already merged video? (I assume no, and though possibly technically possible to implement, would be better handled pre-emptively as described above.)
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
$ yt-dlp -vU
[debug] Command-line config: ['-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.06 [6becd2508] (source)
[debug] Git HEAD: 88d8928
[debug] Python 3.10.9 (CPython x86_64 64bit) - macOS-12.6.2-x86_64-i386-64bit (OpenSSL 1.1.1s  1 Nov 2022)
[debug] exe versions: ffmpeg git-2022-06-01-c6364b71 (setts), ffprobe git-2022-06-01-c6364b71, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.06.15, sqlite3-2.6.0, xattr-0.9.9
[debug] Proxy map: {}
[debug] Loaded 1735 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.06, Current version: 2023.01.06
yt-dlp is up to date (2023.01.06)

Not quite, but you can get close

If some fragments can't be downloaded, skip them
Focus on getting all the fragments that are available
Iteratively merge all sequential fragments that are complete

These are the default

When finished, if some fragments are missing/incomplete, don't merge or delete anything
Let the user re-run the download at a later time to try to quickly get the (probably few) remaining fragments
Merge and delete once all fragments are available

"don't merge" is contradicting since you ask the merge to be iterative in (3). You can however prevent the fragments from being deleted with --keep-fragments. Then you can verify the final file manually before deleting the fragments

However, I don't get the motivation behind your question. In practice, how is what you are doing any different from the default behavior? If the fragments were not fetchable with yt-dlp's own retries, it is unlikely to succeed with a manual retry either

Thanks for reminding me about the importance of explaining core motivation when asking questions :)
The core motivation is to not be stuck with a merged/post-processed file that has video/audio/subtitles out of sync, because then you have to re-download the entire file, which may take hours under certain circumstances.
So, one can use keep-fragments to prevent this, but this even keeps fragments even if download/merge/PP is 100% successful, and if you download by batch, there will be thousands of fragments in the directory that you don't really need.
(Also, I forgot to mention that I had -i on by default, which causes generation of the final file and removal of fragments even with missing fragments. I doubt anyone ever wants to do this as it causes video/audio/subtitles to be out of sync.
So my thought was:

Detect if missing fragments causes video/audio/subs to be out of sync, and if so, either refuse to delete fragments, or merge the file by "least common denominator", eg. if video is missing 30:00-30:10, remove that for audio and subtitles too.
Keep directory flooding of fragment files down by merging consequtive fragments (as illustrated in OP).
Only keep fragments as long as they are actually needed. (If all fragments are downloaded, and the final file is successfully merged, they are not needed anymore)

Options for this could be something in the lines of:
--merge-final-file=[always | only-if-have-all-fragments | yes-but-prevent-out-of-sync-issues]
--merge-fragments-while-downloading=[always | consequtive-only | never]
--keep-fragments-after-merging-final-file=[always | only-if-missing-fragments | never]
I hope this is more understandable and contributes to bringing the discussion forward.

@chrizilla: Thanks for the support. Yes, it's about being smarter about exceptions. Usually, you just want to download some media into a single file, and usually that goes all well. But sometimes, some of the below happens for a shorter or longer period of time,  and you want to handle that smart and gracefully, but also non-intrusively, and as efficiently as possible in long run. (Eg. if a fragment is missing, is it best to try again immediately, in 1 minutes, 1 hour or 1 day?)

Missing fragments
Missing connections
Missing disk space
Interrupted program

My initial problem was files being merged when there were missing fragments that cause out-of-sync, and I don't want to keep fragments as a default, since that's meaningless for the majority of files that finish 100%.
General questions during code design:

What information is available on which to base decisions?
What confident choices can you make on behalf of the user?
When choices are not evident, when do you introduce program options and when do you prompt user input?

Maybe borrow some ideas from Torrent protocols. They are built specifically with exception resilience in mind.

However, I don't get the motivation behind your question. In practice, how is what you are doing any different from the default behavior? If the fragments were not fetchable with yt-dlp's own retries, it is unlikely to succeed with a manual retry either

With CBS, maybe five to ten percent of videos (my experience) have problems (server overload?) when attempting to download them. The vast majority show absolutely none of the errors displayed in this comment:
https://github.com/yt-dlp/yt-dlp/issues/3445#issuecomment-1462413029
Does some parameter exist to retry immediately if I get a "fragment not found" message?

This probably won't help much, but for the small percentage of files I have problems with (almost all CBS), i generally know that everything exists, and want yt-dlp to keep pounding away until the server coughs up the fragment.

@2011: Define "exists". And "pounding away" when? (Retry the same fragment forever before moving on?)

I don't know if you looked at the comment (and the entire related issue), but the problems I encounter (which, of course, others might find completely irrelevant) come from recently released material accessed (usually) through the CBS web site. The site would (except in the case of some weird and rare error) have all of the fragments available (and I imagine people would let them know otherwise) for the (generally) few weeks that the content remains accessible by the public. I can usually repeat a download that failed (meaning at least one missing fragment), and have success on the second attempt (again, see the linked issue).
I would rather (in this case) retry the fragment infinitely (and simply wait as long as it takes to retrieve it). I don't know what happened to the failed fragment in the linked comment. The output doesn't specifically say that yt-dlp tried ten times to retrieve it (those errors don't happen often enough for me to know exactly what happened, and the generic extractor provides more error information than the CBS extractor).
I would guess that (non-live) videos very seldom actually have "missing" fragments, and instead that most of the not available erros come from temporary overloads or outages, but again, I don't have enough experience to make that claim with a high level of confidence.
Thanks for taking a look at (and thinking about) this.

When starting, possibly do a rough estimate of if there will be enough disk space for the upcoming downloads. If not, then warn immediately about how much more is needed with ANSI colored terminal text (but go on downloading).

Out of scope

If disk full, bail everything and play a notification with sound to make user immediately aware of this so he can free up disk space and manually re-run the download.

Seems unrelated to this issue.

Continuously merge all consequtive fragments (see OP) to keep the dir as clean as possible.

Keeping directory "clean" is not an objective of yt-dlp. Set a -P temp: instead to keep the main directory clean. The reason yt-dlp currently merges files on the go is because it is faster and so it can be played as it's downloaded etc. If fragments are missing, we don't need to add the additional complexity of creating sequential blocks. (Similar to -N)

If a fragment fails, immediately skip to the next fragment
If the next n fragments also fail, skip to the next file in the playlist (or exit). Let the user decide when to re-run.
When tried/fetched all fragments, retry those that failed the first time, still following the logic in the previous point.

This feels to me like it is unnecessarily complicating the current behavior[^1] with no benefit. Pls explain how this is better

Download consequtive fragments as long as no problems.

If there are missing fragments when the script has finished (or given up on) a file, the user must pass explicit options to both merge and delete fragments regardless of this.

I will not change the defaults as explained here. However, a new option (say, --continue-on-unavailable-fragments) can be added to get this behavior https://github.com/yt-dlp/yt-dlp/issues/6757#issuecomment-1505005929. As I understood OP, this is really the only thing you want anyway. Idk why you are complicating it with so many other "ideas".

When merging media with separate video/audio/subs, if eg. an audio fragment is missing, merging should cut out the corresponding pieces in the other media (ie. video/subs) to always prevent out-of-sync (no-one ever wants that).

Not possible to implement. Audio and video fragments may not have same durations/count. Or one of them may not even be fragmented. We can't even detect, much less cut out the correct portions.
[^1]: If fragment fails, retry. If all retries fail, go to next fragment.

@2011 Look into --fragment-retries inf. This issue is unrelated to what you are asking.

Continuously merge all consequtive fragments (see OP) to keep the dir as clean as possible.

Keeping directory "clean" is not an objective of yt-dlp. Set a -P temp: instead to keep the main directory clean.

This is actually a pretty decent workaround. Thanks for pointing this out.

I will not change the defaults as explained here. However, a new option (say, --continue-on-unavailable-fragments) can be added to get this behavior #6757 (comment). As I understood OP, this is really the only thing you want anyway. Idk why you are complicating it with so many other "ideas".

When merging media with separate video/audio/subs, if eg. an audio fragment is missing, merging should cut out the corresponding pieces in the other media (ie. video/subs) to always prevent out-of-sync (no-one ever wants that).

Not possible to implement. Audio and video fragments may not have same durations/count. Or one of them may not even be fragmented. We can't even detect, much less cut out the correct portions.

Important clarification. Out-of-sync media was my initial concern. If suggestions got unnecessarily complicated it was trying to be creative, not knowing every detail of the application, and related issues from other users.
Aren't HLS segments usually 10 seconds for both video and audio, and disregarding weird edge cases where they aren't, isn't it quite easy to do a logical "and" between available video and audio pieces? (Subtitles require a bit of code, but not that much, or one could simply choose to forsake subtitles for edge cases like this.)
If this is completely undoable (or you can't be bothered), at least --continue-on-unavailable-fragments will prevent out-of-sync situations, but at the risk and cost of the following:

You start a download that will take hours before you leave home or go to sleep.
An early fragment is missing. The application bails.
You return, expecting the download to be complete, and get upset because you have to restart it, which the further chance of the same thing happening any time.

With this in mind, putting the temp directory to /tmp, and using --keep-fragments for sites known to take hours and have missing fragments, absolutely seems like the best solution. The download dir is kept clean, downloading is not abruted by mishaps, and the (usually few) fragments missing at the end can quite quickly be fetched with a second manual run. This can even be semi-automated with a until shell loop, assuming the application gives the proper exit codes depending on whether the download was complete and successful.

Aren't HLS segments usually 10 seconds for both video and audio,

No. Where did you get that? They can be any length afaik. Besides, there's dash and ism too.

If this is completely undoable (or you can't be bothered), at least --continue-on-unavailable-fragments will prevent out-of-sync situations, but at the risk and cost of the following:

You start a download that will take hours before you leave home or go to sleep.
An early fragment is missing. The application bails.
You return, expecting the download to be complete, and get upset because you have to restart it, which the further chance of the same thing happening any time.

No, what you are describing is the existing behavior. The whole point of the new option is to avoid this. Here's what will happen instead.

You start a download that will take hours with the new option before you leave home or go to sleep.
An early fragment is missing. The application still downloads all available fragments, but doesnt merge them.
You return, expecting the download to be complete, and when you look at log of why it didn't you see a fragment was missing.

If you want to try and get that fragment, restart the download with same options and yt-dlp will try to just download the missing pieces and then merge everything
If you think the fragment is unobtainable, remove the new option and run yt-dlp again causing the file to be merged while skipping the fragment

Am I missing something?

So --continue-on-unavailable-fragments means --merge-final-file-despite-missing-fragments? If so, I'm totally on board. I would set this to "NO" then, and TBH would expect it to be the default.
What about the part about deleting fragments when the file is 100%? (ie. only keeping fragments as a safety measure as long as they might be needed to ultimately get a full and complete merged file)

@chrizilla: Happy to hear we seem to have found a concise common ground that that makes good sense (avoid corruption, without unnecessary inconvenience), and would seem easy to implement. Hopefully this is a step closer to an implementation and improvement in the near future.

You have to be impressed with the work involved in maintaining a project like this and keeping tabs on so many suggestions and opinions. Maybe others will weigh in. Let's be patient and hope for the best, whatever that may be.


Bug: destroy not working as expected when changing audio url too quickly

Bug description
If it's 'autoplay: true' and the audio url is changed too quickly, the wavesurfer instance is not destroyed, which results in multiple audios instead of destroying the old instance and creating a new one with the updated url.
In order to demonstrate the bug, I have provided a video below and a code using React.js.
Environment

Browser: Firefox and Chrome

Minimal code snippet
https://codesandbox.io/s/wavesurfer-js-change-song-bug-ylf48s?file=/src/App.js
Expected result
The audio url should be changed and the previous wavesurfer instance destroyed, instead of multiple audios being used
Obtained result
Multiple audios
Video
https://vimeo.com/839866689?share=copy

It's most likely a bug in the hook that creates/destroys a wavesurfer instance.
Your code is based on the React example which is just that, an example. If you manage to figure it out, please feel free to make a PR.
Ideally, it shouldn't even create and destroy a new instance on every render. It should instead reuse the same instance and call load() to set a new URL, and setOptions to set new visual props.

That said, I did notice an inconsistency in the destroy method of the player, will create a PR to address it shortly.

That said, I did notice an inconsistency in the destroy method of the player, will create a PR to address it shortly.

Yeah, I had a feeling that the problem was in the destroy method.
Thank you for the quick answer, waiting for merge!  🙏🏼


Fix typo for CPM

Missing a hyphen between workerbulk causing errors. It was late.

 (3) [Tech debt] Use only publishing api to preload items


Where to store a key size?

We discussed two options to express a key size in algorithmProperties:

Use an existing security strength field. (Was this referring to the classicalSecurityLevel field?)
Introduce a new field, keysize or just size (to alternatively cover block size etc. where required) or something along those lines.

IIUC the security strength concept might be intuitive enough for the community—probably helped by some documentation how to set and interpret it.

Closing with the upstream to CycloneDX 1.6 and the merge of https://github.com/CycloneDX/specification/pull/339.


overhaul design for STATS report

[x] make it look nicer with art colleague

[ ] add background pattern
[ ] tweak logo sizes (should be bigger)
[ ] tweak gaps and positions (now its awkwardly inconsistent)
[ ] tweak fontWeights
[ ] verify responsive / line break with different data
[ ] somewhat add a nice border around the avatar of the streamer so that it looks like shit
[ ] experiment with shadows (timeboxed)

Current design


az connectedk8s connect: FAILURE.

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az connectedk8s connect Extension Name: connectedk8s. Version: 1.3.6.
Errors:
The command failed with an unexpected error. Here is the traceback:
[Errno 8] Exec format error: '/home/vitorafgomes/.azure/helm/v3.6.3/linux-amd64/helm'
Traceback (most recent call last):
  File "/home/vitorafgomes/.local/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/vitorafgomes/.local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/home/vitorafgomes/.local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/vitorafgomes/.local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/home/vitorafgomes/.local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/home/vitorafgomes/.local/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/home/vitorafgomes/.azure/cliextensions/connectedk8s/azext_connectedk8s/custom.py", line 199, in create_connectedk8s
    release_namespace = get_release_namespace(kube_config, kube_context, helm_client_location)
  File "/home/vitorafgomes/.azure/cliextensions/connectedk8s/azext_connectedk8s/custom.py", line 853, in get_release_namespace
    response_helm_release = Popen(cmd_helm_release, stdout=PIPE, stderr=PIPE)
  File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1845, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/home/vitorafgomes/.azure/helm/v3.6.3/linux-amd64/helm'

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az connectedk8s connect --name {} --resource-group {} --location {} --correlation-id {}

Expected Behavior
Environment Summary
Linux-5.15.0-1021-raspi-aarch64-with-glibc2.35, Ubuntu 22.04.1 LTS
Python 3.10.6
Installer: PIP

azure-cli 2.43.0

Extensions:
connectedk8s 1.3.6

Dependencies:
msal 1.20.0
azure-mgmt-resource 21.1.0b1

Additional Context

route to CXP team

@vitorafgomes Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

@vitorafgomes Could you please follow the below plan and let us know if that helps ?

Please use the most recent version of Helm and check if that helps.
Could you please confirm if you are logged into AzCLI using Service Principal ?
Could you please try setting the HELM_CLIENT_PATH (PATH environment variable) for the HELM (download & Install path for HELM ?
Could you try running the below command and check if that helps ?

az connectedk8s connect -g resourceGroupName -n connectedClusterName --kube-config /path/to/kubeconfig --kube-context kubeContextName

Reference articles:
https://github.com/Azure/azure-cli-extensions/blob/816c8d2175d2310e044464853760006b8c55428b/src/connectedk8s/azext_connectedk8s/custom.py#L853
https://learn.microsoft.com/en-us/cli/azure/connectedk8s?view=azure-cli-latest#az-connectedk8s-connect
https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/troubleshooting
https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/quickstart-connect-cluster?tabs=azure-cli

@vitorafgomes I wanted to do quick follow-up to check if you had a chance to look at my above comment. Please let us know if you had any updates on this. Awaiting your reply.

@vitorafgomes The action plan is currently pending on you to provide an update on the above questions. Once you are ready to resume troubleshooting on this, please feel free to reopen this thread. We would be happy to help.

navba-MSFT
Sorry for the delay, could you tell me why I have this error below?


Issue about the ETa in the "getLGP_G_5_and_10" function

https://github.com/gicait/PyAEZ/blob/5ca84abe7311e9e79495da2ebb6fb53dc3c9c60c/pyaez/ClimateRegime.py#L352
Based on page 28 of the model document (Global Agro-Ecological Zones v4 – Model documentation.pdf), ETa = ETm, when P > ETm or when P < ETm but P+Wb-Wr > ETm.
But the this line 375,  ”self.totalPrec_daily[i_r,i_c,ii] + W >= SaD(1-pc)“ is only P+Wb >= Wr,  not meets "P+Wb-Wr > ETm". https://github.com/gicait/PyAEZ/blob/5ca84abe7311e9e79495da2ebb6fb53dc3c9c60c/pyaez/ClimateRegime.py#L375

Dear @Saibo-Li, the current team of developers are not sure if that was a numerical trick or an actual glitch in the code from the previous version. We are glad to share the news that we are releasing PyAEZ version 2 at the end of November 2022, and this specific issue you raised had already been fixed, the source codes now should follow the GAEZv4 Documentation (within the PyAEZ extent) and are more readable/understandable. An updated PyAEZv2 Documentation will also be released.

Dear @kboonma, many thanks to you and your team. We're looking forward to this latest version. However, we really want to use this getLGP_G_5_and_10 function recently, I wonder if you can only share the ClimateRegime.py (or this function ). My email is<EMAIL_ADDRESS> We are sorry for this rude request!

@Saibo-Li I sent you some fixes as requested. Please consider registering for the upcoming official PyAEZ launching event and technical training.

Thank you very much! I have registered. The official launch event and technical training are very helpful to me!


Messages are queued when window is inactive

If I leave a tab open w explorer in zone and then return, messages are queued and then executed in a hurry
While this queue is executed, I can't interact with other things (or it gets queued maybe)
Go to this scene, switch tabs and leave it open five minutes as you do other things, then return
https://explorer.decentraland.zone/?ENABLE_WEB3&position=68%2C-68

I don't think this keeps happening
or if it does, it's a lot less obvious
I'd say we can close it


add to Textual gallery

https://www.textualize.io/gallery-instructions
Figure more people will be exposed to the project there as well.

I have already done PR https://github.com/Textualize/textualize.io/pull/123 .
But looking on other issues, it requires some patience.
Thank you for thinking highly of this project!

Aha, ok, didn't see you in the gallery so figured hadn't already submitted yet. Good to hear!


Better metrics for performance evaluation

It would be nice to be able to better compare different loss functions and training runs with some other metric. One example could be implementing and reporting on KNN classification accuracy on validation data during training.

Fixed by e2a4fea.

Fixed by e2a4fea.


[RFC] Programmatic payments

Problem
There is no way in Saleor to manage recurring payments (aka subscriptions) and stored payment methods. Existing solutions (e.g. availablePaymentGateways) work only with Payment plugins and don’t have any management API that could be used by customers.
🏁 Goals

Add API for fetching and managing stored Payment Methods in Payment Provider Apps

Add mutation inspired by existing storedPaymentSources for fetching Payment Methods
Add sync webhooks to allow communication for fetching and managing stored payment methods in Payment Apps
Add mutations and sync webhooks for adding, updating, deleting payment methods
Add mutation for Payment Provider Apps and notifications when payment method expires / is no longer available
Add mutation for setting user’s default Payment Method per channel

Create API for making programmatic charges by staff members from customer stored payment methods (aka offline flow)

There should be a special permission for this mutation
We should create a new webhook similar to transaction-initialize-session and transaction-process-session webhook - that would return information about charge / auth success, failure, manual action required (3d secure) for manual charges
Add async webhook for receiving notifications about failed manual charges (so that user can re-authenticate)

Create API for requesting payment from users (interactive)

Add mutation for staff members to request a payment for the order / draft order from customer
Add async webhook for receiving notifications about payment requests from store staff

Incurred subscription charges should be stored in Saleor as transactions

Transactions should link to the Payment Method which was used

❌ Non goals

Create an API for managing subscription contracts / mandates

This will be tackled in a separate RFC in future

Status
## PaymentMethod API
- [ ] Add PaymentMethod types
- [ ] Add `User.storedPaymentMethod` query
- [ ] Add `Checkout.storedPaymentMethod` query
- [ ] Add `LIST_PAYMENT_METHODS`
- [ ] Update Transactions object to have linked payment methods

## Payment method management API
- [ ] Add `paymentSourceDelete` mutation and types
- [ ] Add `PAYMENT_SOURCE_DELETE` webhook
- [ ] Add `PAYMENT_SOURCE_UNAVAILABLE` webhook
- [ ] Add `User.defaultPaymentMethod` query
- [ ] Add `paymentSourceSetDefault` mutation

## Tokenize payment method API
- [ ] Add `paymentMethodTokenizeInitialize` mutation and types
- [ ] Add `PAYMENT_METHOD_TOKENIZE_INITIALIZE` webhook
- [ ] Add `paymentMethodTokenizeProcess` mutation and types
- [ ] Add `PAYMENT_METHOD_PROCESS_TOKENIZE` webhook
- [ ] Add `paymentSourceAboutToExpire` mutation
- [ ] Add `PAYMENT_SOURCE_EXPIRING` webhook

## Programatic charges API
- [ ] Add `paymentSourceCharge` mutation and types
- [ ] Add `PAYMENT_METHOD_CHARGE` webhook
- [ ] Add `CHARGE_REQUEST_ACTION_REQUIRED` webhook

API changes
storedPaymentMethod query
Updated schema:
type PaymentMethodType {
  CREDIT_CARD
  DEBIT_CARD
  DIRECT_DEBIT
  WALLET
  BUY_NOW_PAY_LATER
  OTHER
}

enum SupportedChargeFlowsEnum {
  INTERACTIVE
  SUBSCRIPTION
  NON_SCHEDULED
}

"""
Represents a payment source stored for user in payment gateway, such as credit card.

@deprecated in Saleor 3.xx - use PaymentMethod instead
"""
type PaymentSource {
  # ...
}

type PaymentGateway {
  name: String!
  id: String!
  config: [GatewayConfigLine!]!
  currencies: [String!]!
}

"""
Represents a payment method stored for user (tokenized) in payment gateway, such as credit card or SEPA direct debit

Can be accessed only by user the PaymentMethod is attached to
or by staff members and app with permission `HANDLE_CHARGE`
"""
type PaymentMethod @doc(category: "Payments") {
  """Payment gateway that stores this payment method"""
  gateway: PaymentGateway!

  """
  ID of stored payment method used to make charges
  Note: method ID is unique only within the payment gateway
  """
  paymentMethodId: String!

  """
  Unique ID of payment method (gateway.id + paymentMethodId)
  """
  id: ID!

  """
  Channels in which the payment method can be used in.
  If the list is empty the payment method can be used in all channels
  """
  channels: [Channel!]

  """Stored credit card details if available."""
  creditCardInfo: CreditCard

  """
  List of supported charge flows that can be performed on this payment method

  INTERACTIVE - Payment method can be used for 1 click checkout - it's prefilled in checkout form (might require additional authentication from user)
  SUBSCRIPTION - Payment method can be used for recurring subcriptuions (fixed amount charged on regular interval)
  NON_SCHEDULED - Payment method can be used for irregular subscriptions or charged by staff members
  """
  supportedChargeFlows: [SupportedChargeFlowsEnum!]!

  """
  Type of the payment method
  """
  type: PaymentMethodType!

  """
  Name of the payment method

  Example: SEPA Direct Debit / Apple Pay / Klarna Pay later
  """
  name: String!

  """
  Uniquely identifiable name of the payment method

  Example: last 4 digits of credit card, obfuscated email
  "**** 1234" /<EMAIL_ADDRESS>  """
  uniqueName: String

  """
  Date when the payment method was saved in payment gateway
  """
  created: DateTime!

  """
  Date when the payment method was last updated by gateway or by user
  """
  updatedAt: DateTime!

  """
  List of public metadata items.
  """
  metadata: [MetadataItem!]!
}

type User {
  # ...
  """Returns a list of user's stored paymetn methods that can be used in provided channel"""
  storedPaymentMethods(channelId: ID!): [PaymentMethod!]
}

type Checkout {
  # ...
  """
	Returns a list of user's stored payment methods that can be used in this checkout session
	It works the same way as User.storedPaymentMethods, it uses the channel that the checkout was created in.
	"""
  storedPaymentMethods: [PaymentMethod!]
}

Transaction object
If a transaction was created programmatically from a stored payment source, it will be linked in Transaction object
type TransactionSavedPaymentMethod {
  """
  Unique ID of payment method - gatewayId + paymentMethodId
  """
  id: ID!
  """
  Identifier of payment method inside payment gateway
  """
  paymentMethodId: String!
  """
  ID of payment gateway that has this payment source stored
  """
  gatewayId: String!
  """
  Human readable name of payment gateway
  """
  gatewayName: String
}

type Transaction {
  # ...
  paymentMethod: TransactionSavedPaymentMethod
}

Fetching payment sources from app
LIST_PAYMENT_METHODS [SYNC WEBHOOK]
↗ Request
type ListPaymentMethods implements Event {
   """Time of the event."""
  issuedAt: DateTime

  """Saleor version that triggered the event."""
  version: String

  """The user or application that triggered the event."""
  issuingPrincipal: IssuingPrincipal

  """The application receiving the webhook."""
  recipient: App

  """The user the request to fetch payment methods was issued for"""
  sourceObject: User

  """Channel in context which was used to fetch the list of payment methods"""
  channel: Channel
}

↘ Response
type ChannelId = string;

type PaymentMethod = {
  id: string;
  creditCardInfo?: {
    brand: string;
    firstDigits?: string;
    lastDigits: string;
    expMonth: number;
    expYear: number;
  };
  supportedChargeFlows: Array<"INTERACTIVE" | "SUBSCRIPTION" | "NON_SCHEDULED">;
  type: CREDIT_CARD | DEBIT_CARD | DIRECT_DEBIT | WALLET | BUY_NOW_PAY_LATER | OTHER;
  name: string;
  uniqueName?: string;
  channelIds?: Array<ChannelId>;
  created: Date;
  updatedAt: Date;
  metadata: Record<string, string>;
};

type WebhookResponse = Array<PaymentMethod>;

The response from Payment App is mapped to PaymentMethod object:
{
  gateway: {
    id: "<App id>",
    name: "<App name>",
  },
  paymentMethodId: "<Response.id>",
  id: "base64(<App.id>.<Response.id>)",
  // rest is left as is
}

Adyen Example
{
  gateway: "app.saleor.adyen",
  gatewayId: "Adyen",
  paymentMethodId: "ZFD443FSD54CGFD",
  creditCardInfo: {
    brand: "mc",
    firstDigits: "1234",
    lastDigits: "0000",
    expMonth: 01
    expYear: 2024
  },
  supportedChargeFlows: [
    'ONE_CLICK'
		'SUBSCRIPTION',
    'CARD_ON_FILE'
  ],
  type: 'credit_card'
}

Storing new payment method
Method 1: Zero Auth
In some payment providers to create a payment method without placing an order, storefront must create an authorization with amount 0. If your payment provider requires that, use the regular “[Processing a payment with Payment App](https://docs.saleor.io/docs/3.x/developer/payments#initialize-payment-gateway)” flow with amount set to 0

      Example in Adyen SDK

await checkout.create({
  amount: {
    value: 0,
    currency: "EUR",
  },
  storePaymentMethod: true,
  shopperInteraction: CreateCheckoutSessionRequest.ShopperInteractionEnum.Ecommerce,
  recurringProcessingModel: CreateCheckoutSessionRequest.RecurringProcessingModelEnum.Subscription,
  reference: "test-1234",
  returnUrl: "http://localhost:1234",
  merchantAccount: process.env.ADYEN_MERCHANT_ACCOUNT,
  countryCode: "PL",
  shopperReference: SHOPPER_REFERENCE,
  shopperEmail<EMAIL_ADDRESS>});

→ See a flowchart of this flow
Method 2: Setup session
In others, there is a separate API for saving payment methods (e.g. Stripe [Setup Intents API](https://stripe.com/docs/payments/setup-intents)). Use the following mutations for creating payment sources if your payment provider requires that.

Note
implementation of this webhook is not required from Payment Provider App developers if the payment provider uses Zero Auth method described in previous section

→ See a flowchart of this flow
1. Channel.availablePaymentGateways query
Fetch available payment gateways, without creating new checkout session
type Channel {
  # ...
  availablePaymentGateways: [PaymentGateway!]!
}

2. paymentMethodTokenizeInitialize mutation
Initialize new session to store a payment method in payment gateway
enum PaymetnMethodTokenizeInitializeErrorCode {
  GRAPHQL_ERROR
  INVALID
  NOT_FOUND
  GATEWAY_ERROR
}

type PaymentMethodTokenizeInitializeError {
  field: String
  message: String
  code: PaymetnMethodTokenizeInitializeErrorCode!
}

type PaymentMethodTokenizeInitializeSession {
  data: JSON
  errors: [PaymentMethodTokenizeInitializeError!]
}

type Mutation {
  """
  Creates new session in payment gateway to tokenize a payment source and save it for later use

  Requires to be called by registered user
  """
  paymentMethodTokenizeInitialize(
    """
    The identifier of the payment gateway app to initialize.
    """
    gatewayId: String!
    """
    The data that will be passed to the payment gateway.
    """
    data: JSON
  ): PaymentMethodTokenizeInitializeSession
}

3. PAYMENT_METHOD_TOKENIZE_INITIALIZE [SYNC Webhook]
This webhook is sent to the Payment Provider app after paymentMethodTokenizeInitialize is called
↗ Request
type PaymentMethodTokenizeInitialize implements Event {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user that triggered the event.
  """
  issuingPrincipal: User

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Additional data sent from storefront to the app to create a session
  """
  data: JSON
}

↘ Response
type SuccessResponse = {
  ok: true;
  data: JSON;
};

type FailureResponse = {
  ok: false;
  error: string;
};

type WebhookResponse = SuccessResponse | FailureResponse;

4. Saving payment method
Send payment method details directly to payment provider by using their SDK/API Stripe example.

Note
Saving payment methods is done out-of-band, due to PCI requirements: don’t want users to pass raw credit card data through Saleor.

5. Optional step: paymentMethodTokenizeProcess + PAYMENT_METHOD_PROCESS_TOKENIZE [SYNC Webhook]
Request additional action from Payment Provider App
enum PaymetnMethodTokenizeProcessErrorCode {
  GRAPHQL_ERROR
  INVALID
  NOT_FOUND
  GATEWAY_ERROR
}

type PaymentMethodTokenizeProcessError {
  field: String
  message: String
  code: PaymetnMethodTokenizeProcessErrorCode!
}

type PaymentMethodTokenizeProcessSession {
  data: JSON
  errors: [PaymentMethodTokenizeProcessError!]
}

type Mutation {
  """
  Sends additional data to payment provider app

  Requires to be called by registered user
  """
  paymentMethodTokenizeProcess(gatewayId: String!, data: JSON): PaymentMethodTokenizeProcessSession
}

Webhook payment-source-process-tokenize is sent to Payment Provider App
↗ Request
type PaymentSourceProcessTokenize {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user that triggered the event.
  """
  issuingPrincipal: User

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Additional data sent from storefront to the app to process a session
  """
  data: JSON
}

↘ Response
type SuccessResponse = {
  ok: true;
  data: JSON;
};

type FailureResponse = {
  ok: false;
  error: string;
};

type WebhookResponse = SuccessResponse | FailureResponse;

Managing stored payment sources
paymentSourceDelete mutation + PAYMENT_SOURCE_DELETE [SYNC webhook]
To delete a payment method, use this mutation:
enum PaymentSourceDeleteErrorType {
  GRAPHQL_ERROR
  INVALID
  GATEWAY_ERROR
}

type PaymentSourceDeleteError {
  """
  Name of a field that caused the error. A value of `null` indicates that the error isn't associated with a particular field.
  """
  field: String
  message: String
  code: PaymentSourceChargeErrorCode!
}

type PaymentSourceDeleteResult {
  errors: [PaymentSourceDeleteError]
}

type PaymentSourceToDelete {
  id: String!
  gatewayId: String!
}

type Mutation {
  paymentSourceDelete(paymentSource: PaymentSourceToDelete!): PaymentSourceDeleteResult
}

It sends a payment-source-delete webhook to Payment Provider App:
↗ Request
type PaymentSourceDelete implements Event {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user or app that triggered the event.
  """
  issuingPrincipal: IssuingPrincipal

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Payment source that was requested for deletion
  """
  sourceObject: PaymentSourceToDelete!
}

↘ Response
type SuccessResponse = {
  ok: true;
};

type FailureResponse = {
  ok: false;
  error: string;
};

type WebhookResponse = SuccessResponse | FailureResponse;

Get user’s default payment source: User.defaultPaymentMethod query
type User {
  """
  Returns users default payment method for provided channel
  """
  defaultPaymentMethod(channelId: ID!): PaymentSource
}

Update user’s default payment source: paymentSourceSetDefault mutation
enum PaymentMethodSetDefaultErrorCode {
  GRAPHQL_ERROR
  PAYMENT_SOURCE_NOT_FOUND
  USER_NOT_FOUND
  CHANNEL_NOT_FOUND
}

type PaymentMethodSetDefaultError {
  field: String
  message: String
  code: PaymentMethodSetDefaultErrorCode!
}

type PaymentMethodSetDefault {
  user: User
  paymentSource: PaymentSource
  errors: [PaymentSourceSetDefaultError!]
}

type Mutation {
  """
  Sets payment source as user's default in `User.defaultPaymentMethod`
  """
  paymentMethodSetDefault(
    paymentMethodId: String!
    userId: ID!
    channelId: ID!
  ): PaymentSourceSetDefault
}

Notifications about changes in payment sources
About to expire: PAYMENT_SOURCE_EXPIRING [ASYNC webhook]
Notification is sent by Payment Provider app before a payment method expires
Mutation
enum PaymentSourceAboutToExpireErrorType {
  GRAPHQL_ERROR
  INVALID
}

type PaymentSourceAboutToExpireError {
  field: String
  message: String
  code: PaymentSourceAboutToExpireErrorType!
}

type PaymentSourceAboutToExpire {
  paymentSource: PaymentSource
  errors: [PaymentSourceAboutToExpireError!]
}

type Mutation {
  paymentSourceAboutToExpire(paymentSourceId: String!, expiresAt: Date!): PaymentSourceAboutToExpire
}

Async Webhook request
type PaymentSourceExpiring implements Event {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user that triggered the event.
  """
  issuingPrincipal: User

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Payment method the notification was issued for
  """
  sourceObject: PaymentMethod

  """
  Date when the payment source will expire
  """
  expiresAt: Date
}

Deleted: PAYMENT_SOURCE_UNAVAILABLE [ASYNC webhook]
Notification is sent automatically once paymentMethodDelete mutation completes without errors or when a payment method deletion is reported by Payment App via the same mutation (e.g. payment method expired and is no longer available, payment method was deleted externally - via customer portal, payment method was deleted by bank)
type PaymentSourceDeleted {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user or app that triggered the event.
  """
  issuingPrincipal: IssuingPrincipal

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Payment source that was deleted deletion
  """
  sourceObject: PaymentSource!
}

Charging customer programmatically from saved payment method

To charge a customer, you must first create an order or draft order.
To charge customer, you must have a new permission: HANDLE_CHARGES
→ See a flowchart for this flow
paymentSourceCharge mutation
GraphQL
scalar JSON
scalar ID

type PaymentMethodChargeErrorCode {
  GRAPHQL_ERROR
  INVALID
  ORDER_NOT_FOUND
  PAYMENT_SOURCE_NOT_FOUND
  MISSING_DEFAULT_PAYMENT_SOURCE
  PAYMENT_SOURCE_EXPIRED
  PAYMENT_SOURCE_REJECTED
  PAYMENT_APP_UNAVAILABLE
}

type PaymentMethodChargeError {
  """
  Name of a field that caused the error. A value of `null` indicates that the error isn't associated with a particular field.
  """
  field: String
  message: String
  code: PaymentMethodChargeErrorCode!
}

type PaymentMethodCharge {
  transaction: TransactionItem
  event: TransactionEvent
  errors: [PaymentMethodChargeError!]
}

type Mutation {
  """
  Charges customer with their default saved payment method (can be overwritten)
  by sending request to payment app

  Charge request must be made to an existing order or draft order with customer attached.

  By default the new transaction will use the amount from `order.totalBalance` - can be overwritten.
  Currency is also taken from the order, but cannot be overwritten.

	Requires permission: `HANDLE_CHARGES`

  Added in Saleor 3.xx
  """
  paymentMethodCharge(
    """
    Order or draft order for which a charge will occur. It must have a customer attached
    The new transaction amount will be taken from `order.totalBalance` if not specified
    """
    order: ID!

	  """
	  Use payment method other than specified in `User.defaultPaymentSource` for the order's channel
	  """
	  paymentMethodId: String

	  """
	  Charge other amount than `order.totalBalance`.
	  The currency will be taken from order's currency
	  """
	  amount: PositiveDecimal!

	  """
	  Additional reference passed to payment provider
	  """
	  merchantReference: String

	  """
	  Determines the transaction flow strategy - AUTHORIZATION or CHARGE
	  """
	  actionType: TransactionFlowStrategyEnum!

	  """
	  Custom data passed to payment app - can be used by other apps for more advanced flows
	  """
	  data: JSON
  ): PaymentMethodCharge
}

PAYMENT_METHOD_CHARGE webhook [SYNC Webhook]
↗ Request
scalar JSON

type PaymentMethodChargeAction {
  amount: PositiveDecimal!
  currency: String!
  actionType: TransactionFlowStrategyEnum!
  customer: User!
}

type PaymentMethodChargeRequest implements Event {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user or application that triggered the event.
  """
  issuingPrincipal: IssuingPrincipal

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Order for which the charge request was created from
  """
  sourceObject: Order

  """
  Details for requested action
  """
  chargeAction: PaymentMethodChargeAction

  """
  Payment method that's meant to be charged
  """
  paymentMethod: PaymentMethod

  """
  Merchant reference passed from Saleor
  """
  merchantReference: String

  """
  Additional data passed to app
  """
  data: JSON
}

↘ Response
type PositiveDecimal = number; // non negative number

type WebhookResponseCommon = {
  pspReference?: string;
  data?: JSON;
  amount: PositiveDecimal;
  time?: Date;
  externalUrl?: string;
  message?: string;
};

type WebhookResponseUnion =
  | {
      result:
        | "CHARGE_SUCCESS"
        | "CHARGE_FAILURE"
        | "CHARGE_REQUEST"
        | "AUTHORIZATION_SUCCESS"
        | "AUTHORIZATION_FAILURE"
        | "AUTHORIZATION_REQUEST"
        | "AUTHORIZATION_ACTION_REQUIRED"
        | "CHARGE_ACTION_REQUIRED";
      actionUrl: never;
    }
  | {
      result: "AUTHORIZATION_ACTION_REQUIRED" | "CHARGE_ACTION_REQUIRED";
      /** */
      actionUrl: string;
    };

type WebhookResponse = WebhookResponseUnion & WebhookResponseCommon;

Notification about additional action required: CHARGE_REQUEST_ACTION_REQUIRED [Async webhook]
This notification is sent once Payment App returns AUTHORIZATION_ACTION_REQUIRED or CHARGE_ACTION_REQUIRED action from PAYMENT_METHOD_CHARGE webhook.
The returned actionUrl can be passed directly to the user in order to authenticate the charge request.
type ChargeRequestActionRequired implements Event {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The payment app that triggered the event.
  """
  issuingPrincipal: App

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  The transaction that has the ACTION_REQUIRED event
  """
  sourceObject: Transaction

  """
  Order for which the request was created for
  """
  order: Order

  """
  Customer for which the charge request is asosiated with
  """
  user: User

  """
  Payment method that needs additional action
  """
  paymentMethod: PaymentMethod
}

Paying for the order
Use the Payments App flow described in Saleor Docs:
In paymentGatewayInitialize provide the ID of the Order that needs payment.
Workflow
Fetching list of saved payment methods
sequenceDiagram
    actor User
    User->>Storefront: Visits a storefront page for managing saved payment methods
    alt Fetching payment methods through Saleor
        Storefront->>Saleor: User.storedPaymentMethods query
        par Fetching from Payment Apps
        Saleor->>Adyen App: LIST_PAYMENT_METHODS webhook
        Adyen App->>Saleor: List of stored payment methods for the user
        end
        Saleor->>Storefront: Views list of stored payment methods
    else Fetching payment methods directly via SDK
        Storefront->>Adyen API: Fetch list of payment methods (uses merchantReference = Saleor User.id)
        Adyen API->>Storefront: List of stored payment methods
    end
    Storefront->>User: Sees a list of saved payment methods

Using saved payment method to pay for the order
sequenceDiagram
    actor User
    User->>Storefront: Visits a checkout page to complete the checkout
    rect rgba(194, 219, 255, 0.2)
        Storefront->>Saleor: Channel.availablePaymentGateways query
        Saleor->>Storefront: Returns list of payment gateways
        Storefront->>Saleor: paymentGatewayInitialize mutation
        par Initlizing payment gateways
        Saleor->>Adyen App: PAYMENT_GATEWAY_INITIALIZE_SESSION webhook
        Adyen App->>Adyen API: Create session
        Adyen API->>Adyen App:
        Adyen App->>Saleor: Return session data for dropin
        end
        Saleor->>Storefront: Returns session data from payment apps
    end
    opt Optional way of fetching payment methods - in case methods are not returned in session data from payment app
        Storefront->>Saleor: Checkout.storedPaymentMethods
        par Fetching payment methods from Payment Provider apps
            Saleor->>Adyen App: LIST_PAYMENT_METHODS webhook
            Adyen App->>Adyen API: Fetch stored payment methods for provided user in channe's merchant account
            Adyen API->>Adyen App: Return list of stored payment methods
            Adyen App->>Saleor: Respond to webhook call
        end
        Saleor->>Storefront: Returns list of PaymentMethod objects
    end
    Storefront->>User: Sees a list of available payment methods
    User->>Storefront: Selects a saved payment method
    User->>Storefront: Enters CVC for saved credit card
    Storefront->>Saleor: transactionInitialize mutation
    Saleor->>Adyen App: TRANSACTION_INITIALIZE_SESSION webhook
    Adyen App->>Adyen API: Charge customer from tokenized payment method
    Adyen API->>Adyen App: Return pspReference
    Adyen App->>Saleor: Respond to TRANSACTION_INITIALIZE_SESSION webhook
    Saleor->>Storefront: Return created transaction
    note over Adyen API: processes charge asynchronously
    Adyen API->>Adyen App: Charge result + saved credit card token
    Adyen App->>Saleor: transactionEventReport mutation

Authorizing credit card for the first time with zero auth - Adyen example
sequenceDiagram
    actor User
    User->>Storefront: Visits a storefront page for managing saved payment methods
    rect rgba(194, 219, 255, 0.2)
        Storefront->>Saleor: Channel.availablePaymentGateways query
        Saleor->>Storefront: Returns list of payment gateways
        Storefront->>Saleor: paymentGatewayInitialize mutation, with amount = 0
        par Initlizing payment gateways
        Saleor->>Adyen App: PAYMENT_GATEWAY_INITIALIZE_SESSION webhook
        Adyen App->>Adyen API: Create session
        Adyen API->>Adyen App:
        Adyen App->>Saleor: Return session data for dropin
        end
        Saleor->>Storefront: Returns session data from payment apps
    end
    Storefront->>User: Sees a list of available payment methods
    User->>Storefront: Enters credit card data (through iframe)
    Storefront->>Saleor: transactionInitialize mutation
    Saleor->>Adyen App: TRANSACTION_INITIALIZE_SESSION webhook
    Adyen App->>Adyen API: Charge customer + save credit card
    Adyen API->>Adyen App: Return pspReference
    Adyen App->>Saleor: Respond to TRANSACTION_INITIALIZE_SESSION webhook
    Saleor->>Storefront: Return created transaction
    note over Adyen API: processes charge asynchronously
    Adyen API->>Adyen App: Charge result + saved credit card token
    Adyen App->>Saleor: transactionEventReport mutation
    User->>Storefront: Chooses default payment method
    Storefront->>Saleor: paymentMethodSetDefault mutation

Customer adds payment method with special SDK - Stripe example
sequenceDiagram
    actor User
    User->>Storefront: Visits a storefront page for managing saved payment methods
        Storefront->>Saleor: Channel.availablePaymentGateways query
        Saleor->>Storefront: Returns list of payment gateways
        Storefront->>Saleor: paymentMethodTokenizeInitialize mutation
        Saleor->>Stripe App: PAYMENT_METHOD_TOKENIZE_INITIALIZE webhook
        Stripe App->>Stripe API: Create SetupIntent
        Stripe API->>Stripe App:
        Stripe App->>Saleor: Return session data for dropin
        Saleor->>Storefront: Returns session data from payment apps
    Storefront->>User: Sees a list of available payment methods
    User->>Storefront: Enters credit card data (through iframe)
    Storefront->>Stripe API: Stripe.confirmSetup called in SDK
    Stripe API->>Storefront: Returns result

Charging customer from saved Payment Method in Saleor Dashboard
sequenceDiagram
    actor CS as Customer Support

    CS->>Saleor: paymentMethodCharge mutation
    Saleor->>Adyen App: PAYMENT_METHOD_CHARGE webhook
    Adyen App->>Adyen API: Charge customer with `UnscheduledCardOnFile + provided payment method ID
    Adyen API->>Adyen App: Returns pspReference
    Adyen App->>Saleor: Respond to PAYMENT_METHOD_CHARGE webhook
    note over Adyen App, Saleor: event = CHARGE_REQUEST
    Saleor->>CS: Mutation success
    note over Adyen API: Responds after some time
    alt Charge succeeded
        Adyen API->>Adyen App: Returns success result
        Adyen App->>Saleor: transactionEventReport mutation
        note over Adyen App, Saleor: event = CHARGE_SUCCESS
    else SCA failed
        Adyen API->>Adyen App: Returns failed result
        Adyen App->>Saleor: transactionEventReport mutation
        note over Adyen App, Saleor: event = CHARGE_ACTION_REQUIRED
        Saleor->>Email app: CHARGE_REQUEST_ACTION_REQUIRED webhook
        note over Email app: Webhook call includes actionUrl to redirect user to
        Email app->>Email app: Sends email to user
    end

To Test

[ ] Transactions API
[ ] Transaction Webhooks

This looks great and well-considered. I've got a couple of questions:

The description for the SUBSCRIPTION flow says "Payment method can be used for recurring subcriptuions (fixed amount charged on regular interval)". Would this interval be managed in Saleor? If so, is there any difference between this and UNSCHEDULED?
Is it down to the client to keep track of subscription orders and run the payment?
If a customer doesn't have a card saved should they complete checkout with only subscription items first and non-subscription products separately to generate the correct authorisation?

Hi @fetimo

The description for the SUBSCRIPTION flow says "Payment method can be used for recurring subcriptuions (fixed amount charged on regular interval)". Would this interval be managed in Saleor? If not, is there any difference between this and UNSCHEDULED?

No, this interval wouldn't be managed by Saleor itself, we're planning to leave that part to app developers, so that the billing schedule fits your business needs.
This distinction is made due to requirements in some payment providers (like Adyen) to provide an information whether the charge is initiated:

by user from previously saved payment method
by merchant on a scheduled basis (e.g. monthly subscription)
by merchant on an unscheduled basis (e.g. account top-up).

Is it down to the client to keep track of subscription orders and run the payment?

Yes, as I mentioned, the scheduling of subscriptions and running the payments is out of scope for this RFC.

If a customer doesn't have a card saved should they complete checkout with only subscription items first and non-subscription products separately to generate the correct authorisation?

Yes, the authorization for subscriptions should be made separately so that the card can be charged later without manual input from the user (e.g. 3D Secure).
E.g. In Adyen, when you save a card with recurringProcessingModel set to CardOnFile (used for one-off, non-subscription payments), the user must be present when making another authorization (usually a CVC of the card and 3D Secure is required).
With recurrringProcessingModel set to Subscription 3D Secure is only required when making initial authorization.

Updated RFC:

Fixed naming issues in webhooks (paymentSource -> paymentMethod)
Fixed response examples in LIST_PAYMENT_METHODS

Removed from the RFC:
Transaction object
If a transaction was created programmatically from a stored payment source, it will be linked in a Transaction object
type TransactionSavedPaymentMethod {
  """
  Unique ID of payment method - gatewayId + paymentMethodId
  """
  id: ID!
  """
  Identifier of payment method inside payment gateway
  """
  paymentMethodId: String!
  """
  ID of payment gateway that has this payment source stored
  """
  gatewayId: String!
  """
  Human readable name of payment gateway
  """
  gatewayName: String
}

type Transaction {
  # ...
  paymentMethod: TransactionSavedPaymentMethod
}

Note
This feature will be discussed further in the future

LIST_PAYMENT_METHOD response
{
  // ...
  supportedChargeFlows: Array<"INTERACTIVE" | "SUBSCRIPTION" | "NON_SCHEDULED">;
}

Note
These additional fields will be added once Subscriptions are added in Saleor

Get user’s default payment source: User.defaultPaymentMethod query
type User {
  """
  Returns users default payment method for provided channel
  """
  defaultPaymentMethod(channelId: ID!): PaymentSource
}

Note
This query and mutation will be added later, the API might change a bit

Update user’s default payment source: paymentSourceSetDefault mutation
enum PaymentMethodSetDefaultErrorCode {
  GRAPHQL_ERROR
  PAYMENT_SOURCE_NOT_FOUND
  USER_NOT_FOUND
  CHANNEL_NOT_FOUND
}

type PaymentMethodSetDefaultError {
  field: String
  message: String
  code: PaymentMethodSetDefaultErrorCode!
}

type PaymentMethodSetDefault {
  user: User
  paymentSource: PaymentSource
  errors: [PaymentSourceSetDefaultError!]
}

type Mutation {
  """
  Sets payment source as user's default in `User.defaultPaymentMethod`
  """
  paymentMethodSetDefault(
    paymentMethodId: String!
    userId: ID!
    channelId: ID!
  ): PaymentSourceSetDefault
}

Notifications about changes in payment sources

Note
This notification will be removed because currently we've found that none of the payment providers we support sends a notification before the card expires.
Furthermore the requirement to send such notification to Saleor for app creators based on card expiration date would be too much

About to expire: PAYMENT_SOURCE_EXPIRING [ASYNC webhook]
Notification is sent by Payment Provider app before a payment method expires
Mutation
enum PaymentSourceAboutToExpireErrorType {
  GRAPHQL_ERROR
  INVALID
}

type PaymentSourceAboutToExpireError {
  field: String
  message: String
  code: PaymentSourceAboutToExpireErrorType!
}

type PaymentSourceAboutToExpire {
  paymentSource: PaymentSource
  errors: [PaymentSourceAboutToExpireError!]
}

type Mutation {
  paymentSourceAboutToExpire(
    paymentSourceId: String!
    expiresAt: Date!
  ): PaymentSourceAboutToExpire
}

Async Webhook request
type PaymentSourceExpiring implements Event {
  """
  Time of the event.
  """
  issuedAt: DateTime

  """
  Saleor version that triggered the event.
  """
  version: String

  """
  The user that triggered the event.
  """
  issuingPrincipal: User

  """
  The application receiving the webhook.
  """
  recipient: App

  """
  Payment method the notification was issued for
  """
  sourceObject: PaymentMethod

  """
  Date when the payment source will expire
  """
  expiresAt: Date
}


Wordle Game using Python

issue Fixed #267

WORDLE GAME USING PYTHON

It's a simple WORDLE GAME

Tech Stack Used:
PYTHON
Demo:

@TusharAMD kindly check

pls solve conflicts

@TusharAMD I have closed it by mistake please reopen this🙂🙂sorry for this


Update maven plugins used by build tools

We are using old versions of all the maven plugins. Next release could be a good time to upgrade to the last ones when possible. Initial contents of this PR upgrades everything except 2 plugins that were not backward compatible...
Still work in progress.... Also some cleanup in various component pom files could be needed to have the version for every plugin only defined in the main pom file of the build tools.

Initial tests have been done against CCM and ncm-metaconfig.

@jouvin can we already merge the PR as it is now? at least those parts will receive some testing during the 15.12 release

@jouvin can this be merged yet?

@jouvin?

I'd say that what is in the PR is ready to be merged... I'll confirm quickly.

Thanks

I'd a look. I don't see why this would cause any problem but should it happen, I will do my test to fix them... It's difficult (even though not impossible) to test without merging and trying to run the build_all_repos script. In case of too many problems, we can always revert!


Void Macerator turns Lead Ore into Void Copper Dust

As shown in screenshots, Lead Ore (with oredictionary -oreLead) is turned into void copper dust by the void macerator.

Was this the only oreDict miss-match? All other metals, mainly modded, work correctly?

Similar issue with aluminum ore upon further testing:

Right, so it's my entire macerator system thats broken, good to know

Iron and emeralds work correctly.


UI - catalog app page can't load page correctly if it doesn't contain the questions.yml file

What kind of request is this (question/bug/enhancement/feature request):
UI bug
Steps to reproduce (least amount of steps as possible):

add custom catalog e.g https://github.com/guangbochen/charts-test to the global catalog
go to the project level catalog apps page and click the Nginx app
the catalog app page can't display properly

Result:
catalog app page can't load page correctly if it does not contain the questions.yml file
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): rancher master 06/05
Installation option (single install/HA): HA&single

I can confirm this is happening on v2.2.3-rc6. Any already installed apps are not loading if we try to upgrade or if we try install the app at the cluster level. Multi cluster seems to be loading.
Uncaught TypeError: Cannot read property 'forEach' of undefined
    at n.parseQuestionsAndAnswers (component.js:288)
    at n.initTemplateModel (component.js:266)
    at component.js:70
    at e.invoke (0:34688)
    at e.flush (0:34604)
    at e.flush (0:34766)
    at e._end (0:35237)
    at MutationObserver._boundAutorunEnd (0:34934)

Close in favor of https://github.com/rancher/rancher/issues/19890


Problems with Node.js --experimental-detect-module

Node.js’s --experimental-detect-module flag poses a problem for TypeScript. If a future release of Node.js enables it by default, there is little we can do to model its semantics in our own module system under --module nodenext, which could lead to a confusing developer experience for TypeScript and JavaScript authors relying on tsc or editor language features in projects configured for Node.js.
This is fundamentally a TypeScript problem, and I don’t know that the severity merits lobbying Node.js to change course. However, I still thought it was important to document and share as feedback.
The problem

When TypeScript reads a type declaration file, it needs to know the true module format of the JavaScript file it represents.
The contents of a declaration file do not always provide a reliable indication of the true module format of the corresponding JavaScript file.

When a user compiles a TypeScript file like
export default 0;

they can vary the module format of the output JavaScript file by changing the --module flag, but the output declaration file always looks like
declare const _default: 0;
export default _default;

So, that declaration file text on its own is ambiguous. It could represent either of these JavaScript files:
export default 0;

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.default = 0;

or scripts in AMD or SystemJS format.
For TypeScript users who want to run their code in Node.js, it’s important that TypeScript can distinguish between these two cases, because it affects whether those users can access that module with certain types of imports, and what type should be assigned to the names imported from that module:
import z1 = require("export-default-zero"); // Error if resolved file is ESM

import z2 from "export-default-zero";
z2.default; // 0 if resolved file is CJS
            // undefined if resolved file is ESM

Currently, Node.js’s own module format detection algorithm resolves this ambiguity for us. The file extension of the declaration file tells us the file extension of its JavaScript counterpart, which tells us how Node.js will interpret that file’s module format:

Declaration extension
JavaScript extension
Module format

.d.mts
.mjs
ESM

.d.cts
.cjs
CJS

.d.ts
.js
Determined by package.json

These constraints allow us to make a safe assumption about the format of every JavaScript file represented by a declaration file. For example, when we see a .d.ts file, we know it represents a .js file whose format is determined by its package.json scope—if we find that package.json and see that it does not include a "type" field, we assume the JavaScript file contains CommonJS syntax. If that assumption is wrong, the user’s problem is that their JavaScript file is misconfigured for usage in Node.js—they don’t have a problem with TypeScript.
--experimental-detect-module prevents us from being able to make safe assumptions about the module format of .js files whose package.json scope does not define a "type". The flag makes Node.js detect the module format of those files by reading their contents, but TypeScript can’t do the same in the parallel world of declaration files due to the syntax ambiguity discussed earlier.
Possible mitigations
Resolve and read JavaScript files when declaration files are ambiguous
TypeScript avoids reading, or even checking for the existence of, JavaScript files when declaration files are found first. The declaration files currently tell the compiler everything it needs to know about the presence and contents of JavaScript files, so there’s no point in spending extra memory and file I/O reading JavaScript files. If that changes, we could potentially resolve ambiguity by looking at JavaScript content. This would come with a performance penalty. Additonally, multiple team members expressed discomfort at the prospect of giving up the invariant of declaration files being ultimate sources of truth for the compiler. As things stand now, this is the mitigation we’re least likely to pursue.
Eliminate declaration file ambiguity going forward
We could begin changing our declaration file emit format to ensure that module formats are not ambiguous in the future. This is something I strongly advocate for, but it isn’t a solution on its own, because so many existing packages are already ambiguous, and many will never update.
Assume ambiguous files are CommonJS (i.e., do nothing)
If Node.js made --experimental-detect-module the default in the future, we would advocate strongly that all packages define a "type" in their root package.json to avoid ambiguity. (We would likely consider mandating this when compiling local projects under --module nodenext.) We could rely on this becoming best practice for new and actively maintained packages, and assume that packages lacking a "type" field were published before --experimental-detect-module, when such a lack of a "type" implied CommonJS.
Type imports of ambiguous files as permissively as possible
It might be possible to do some type system trickery to make something like this work:
import z1 from "export-default-zero";
z1;         // Type: AmbiguousModule<0> ~= 0 & { default: 0 }
z1.default; // Type: 0

This would be an attempt to get out of the way and never issue a false positive error, at the expense of missing some real errors.
Allow users to assert the format of ambiguous imports
In combination with some other behavior as the default, we could allow users to resolve the ambiguity themselves, perhaps with an import attribute or some central configuration.

we would advocate strongly that all packages define a "type" in their root package.json to avoid ambiguity. (We would likely consider mandating this when compiling local projects under --module nodenext.

I'd in favor of that, even if we end up using one of the other options to workaround that issue, or if Node.js never makes --experimental-detect-module the default; pushing for less ambiguity is a win for the whole ecosystem IMHO.

we would advocate strongly that all packages define a "type" in their root package.json to avoid ambiguity.

Node also encourages everyone to always include a "type" field, especially for published packages: https://nodejs.org/api/packages.html#determining-module-system:~:text=For this reason,should be interpreted. Perhaps tsc could sometimes error when "type" is missing, such as when run in a version of Node where detect-module is enabled by default or when certain settings are specified.

That’s something we might consider for a user’s project package.json, but the real issue for us is the thousands of existing npm packages that have no "type" and will never be updated to get one. We can’t reasonably error on those, but we still need to know what format their .js files are.

2a) npm serves all package.json files which are published without "type" as package.json files with "type": "commonjs"

Node asked npm about this when --experimental-default-type was being designed, and they said that it’s a hard requirement that files aren’t modified by npm once they’re published, for security reasons. That’s why --experimental-default-type doesn’t apply to folders below node_modules, because all typeless packages would be broken under --experimental-default-type=module and therefore the flag would be unusable unless users did some kind of post-install transform of all their dependencies to add the missing "type": "commonjs" entries.
--experimental-detect-module doesn’t have this issue, as code that can run as CommonJS will run as CommonJS, and so it doesn’t have a node_modules exception. It’s also preferable to have code behave the same way whether or not it’s under node_modules, so that the library author and the library consumer have code run the same way (at least when they’re both using the same version of Node).

TypeScript avoids reading, or even checking for the existence of, JavaScript files when declaration files are found first.

For what it’s worth, within Node I implemented a containsModuleSyntax function in C++ to answer this one question very quickly. We could expose this as a public API if TypeScript might find it useful. It could be exposed before --experimental-detect-module is enabled by default, so therefore TypeScript could include logic like “if the Node version is < the version where module detection is enabled by default, assume CommonJS (prior behavior); else read the file and use Node’s containModuleSyntax to see how Node would interpret it.” The new API would exist in any version of Node where TypeScript would need it, because of the ability to assume the current behavior for prior versions.
Yes this would be a performance penalty, just like detection itself is a performance penalty, which is why we encourage everyone (especially package authors) to include "type" fields. (Or potentially even add missing "type" fields to dependencies in a post-install pass.) But it would be the most direct solution, at least until packages are updated with "type" fields and/or unambiguous type definition files.

I experimented with ways to mitigate the performance penalty of reading and analyzing additional JavaScript files. We have two limitations that take all the good options off the table: (1) our API is synchronous, so parallelizing the work within the same process is not practical, and (2) file I/O is delegated through a host object provided via the JavaScript API (this is what lets us read from unsaved editor files in TS Server, for example), so calling into another process (whether native code or parallelized JS) is not possible.

It’s complicated. I keep waffling between two plausible perspectives.
On one hand, if we think about think about the scenarios that are most likely to be problems for TypeScript in an unflagged future, those are scenarios that are currently problems for Node.js and will be fixed by unflagging this behavior—loading “unmarked” ESM from packages that included it for bundlers, for example. Meanwhile, TypeScript’s declaration files being ambiguous is a problem of TypeScript’s own creation. Node.js is doing right by their users, and it’s on TypeScript to solve their side of things.
On the other hand, given my present feedback that TypeScript won’t be able to solve our side of things to the degree that we would hope, Node.js probably needs to adjust their expectations for how much unflagging --experimental-detect-module is going to improve the end user experience. If Node.js Just Works™ in more cases than it used to, but most of those cases retain red squigglies in VS Code, that’s not a very satisfying improvement for a huge portion of Node.js users. The Node.js team might appreciate knowing that the problem lies squarely with TypeScript, but users are less interested in watching Node.js and TypeScript perform spiderman-pointing-at-spiderman.png than having an experience that works end to end, and I wonder if incrementally changing interop rules, even if in a nominally desirable direction, creates more confusion than it resolves.
I think a good analysis here would require a better understanding of the most common consequences for users. I’ve laid out a theory of how things could go wrong, but I’m not yet sure how often we would see problems and how bad they would be. I’m curious what other feedback about this Node.js has received, and whether they are still leaning towards unflagging, and on what kind of timeline?
I should also point out that because our issues center around declaration files, node_modules packages are a much bigger problem for us than local project files, and especially the loose .js file scenario. I don’t know if it would be on the table to reconsider disabling this behavior inside node_modules, but that would largely eliminate my concerns.

I’m curious what other feedback about this Node.js has received

We haven’t received much. There’s really only one open issue, https://github.com/nodejs/node/issues/50917, which involves the edge case of non-ESM-related parse errors in a file. The “try again as ESM” behavior is triggered by an ESM-related parse error, but if V8 errors on something else before getting to the first import or export statement, the retry never happens; and some of these parse errors are valid when parsed as ESM. The solution is to either just document this, that this is another edge case that won’t trigger running as ESM (like how we currently document top-level await as not triggering detection) or we could remove the limitation of “only ESM parse errors” and try again for any parse error. Either way, though, the solution would be simple.
Once that’s resolved, unless anything else comes up then it’s really just a question of how much more time we want to wait before considering this having been out in the wild long enough to unflag it. Optimistically the earliest that could happen would be in the next few months, sometime near the end of the 21.x line so that we get maximum feedback before 22.0.0 in April. Or we could unflag it as part of 22.0.0, though it doesn’t necessarily need to wait until then as it’s not a semver-major change. And of course it could always happen later, in the 22.x line or beyond, especially if issues are reported.
I think the fundamental thing here is that Node has accepted (or will be accepting, if we unflag this) a conditional performance hit in the name of usability. TypeScript could do the same: that’s the first option in the above list of potential solutions, where tsc reads the accompanying JavaScript file. If Node and TypeScript were managed together as one project, that would presumably be a decision made together; that if Node behaves one way, TypeScript would behave the same. I am very sympathetic to the desire to not slow down any users, but there are many ways to both match Node’s behavior and mitigate or eliminate the performance impact on TypeScript’s side; for example, some new tsconfig option like "dependencyTypeDefinitionsModuleFormat": "commonjs" / "module" / "detect", where the default is "detect" (performance hit, but matches Node) and users could set "commonjs" to preserve current behavior and current performance. I’m not sure it’s worth creating the potential footgun on the Node side of a node_modules exception for this feature, thereby limiting everyone, when TypeScript has the ability to create such an exception just for itself.

watching Node.js and TypeScript perform spiderman-pointing-at-spiderman.png

It’s honestly amazing how often this meme is relevant to the dev scene in general, especially JS.  Tech stacks become so intertwined that everyone starts looking at what everyone else is doing for direction, and then before you know it, bam, Spider-Man meme happens.

I don’t know if it would be on the table to reconsider disabling this behavior inside node_modules, but that would largely eliminate my concerns.

I don't think this would be an option unfortunately, since having packages work locally then break on publish and install would be a really bad footgun.
It's plausible we could unflag in 22 as @GeoffreyBooth describes. So if you do have any strong arguments against that now would be the time to mention I think.

"not unflagging" still seems like the best option here; the usability isn't worth the risks.

I momentarily forgot about the footgun of a package working during local development and breaking when installed into node_modules. I agree that’s not worth it.
My instinct is that there are non-performance reasons why relying on JavaScript files to resolve declaration file ambiguity is unappealing, but so far they’re mostly rooted in principle and history. @DanielRosenwasser and I discussed this recently and he felt the same way. Maybe he can weigh in on how much that approach feels on or off the table.

This isn’t really an issue yet, is it? Like until someone publishes a package where they rely on detection to work—so they put ESM syntax in .js files and leave out the type field—the ambiguous type definition files always refer to “running as CommonJS” JavaScript files, right?
And it’ll be a long time before anyone publishes a package that relies on detection without getting complaints from users, since any users installing that package in Node <= 21 would try to run it as CommonJS and it would error for them.
So it’s probably safe the vast majority of the time for TypeScript to assume that ambiguous type definition files refer to CommonJS sources. If TypeScript changes the type definition files to be unambiguous soon, then by the time the day comes that someone publishes a “relies on detection” package that runs in most of the versions of Node active by then, that person will likely also be using a future version of TypeScript that outputs an unambiguous type definition file.
There are several “most of the time” and “vast majority” qualifiers in the above, though. It’s definitely possible for someone to slip through the cracks of the edge cases and publish a “relies on detection” package that can’t run in Node < 21 but for whatever reason they generated type definition files with TypeScript 4. That said, it’s always possible for someone to publish incorrect types for a package. I guess the question is what your tolerance level is for this kind of thing. People publish broken packages all the time, that fail in certain versions of Node or certain runtimes or bundlers, so it’s perhaps an acceptable edge case to assume that ambiguous type definition files mean CommonJS and going forward they need to be unambiguous.

I don't think this is an issue about this new option, but rather about TS in general.
More specifically, supporting for CJS, ESM, TS(CJS), TS(ESM) from a single .js file is very hard.
Specifically we have to do stuff like this in fastify:
module.exports = myfn
module.exports.default = myfn
module.exports.myfn

This needs to be matched by a .d.ts file like this:
namespace myfn {
  export const myfn: MyFn
  export { myfn as default }
}

export = myfn

With all of this,  import myFn from 'myfn' and import { myfn } from 'myfn' would work  on all the combinations of CJS+ESM+TS above.
Note that this problem is so significant that multiple modules have to:
const X = require('anothermodule')

const x = new X.default() // because default export is not recognized

An alternative solution for module authoring is to https://github.com/isaacs/tshy and have both a CJS and ESM being built.
I do think the new option makes it worse, mostly because it exposes most developers to the same complex DX we have to do on the module authoring side.

I momentarily forgot about the footgun of a package working during local development and breaking when installed into node_modules. I agree that’s not worth it.

This is something that we might want to revisit. Can you open a separate issue about this feedback?

We should not plan to unflag in 22 before the TypeScript story is sorted.
However, I think simplifying the developer experience for CJS+ESM+TS is something we should strive for, as it's one of the major DX complaints we are hearing from Node.js users. Let's collaborate.
@andrewbranch @DanielRosenwasser what behavior would work for you? How do you think we can solve this problem?

@mcollina I would love to talk about default exports but I think it’s a separate issue to work out, pretty much unrelated to --experimental-detect-module. Unless Node.js is prepared to allow require(esm) and recognize __esModule the same way Bun and every bundler do (and my understanding is neither of those is on the table), we’re going to need to know the true module format of every file in a program because they affect the types we need to assign to imports. Even if we were to come up with a perfect solution around default export interop, we would still like to protect people from trying to require(esm) at compile time. If both of those things were solved, --experimental-detect-module wouldn’t matter to us because interop would be transparent enough that we would never need to know whether a declaration file represented a CJS or ES module. (This is how TypeScript works when configured for Bun, tsx, bundlers, etc.)
Unless I’m mistaken and those are up for discussion, we’ll need to continue to find ways of discerning the module format of files Node.js is going to load, and we can iterate on default export transpilation to try to make it more portable. But (a) I think it’s a separate issue, and (b) it’s a conversation that’s not specific to TypeScript; it would be better to discuss with Babel, esbuild, Rollup, etc.

Yes, DefinitelyTyped can be updated in bulk, which helps for a large number of old/stale but still-popular packages, but not for ones that ship their own types. I’m not sure we would want to use DT to add module format info for packages that do ship their own types (much less duplicate those types to DT), but Daniel and I did briefly mention the idea of shipping a Bloom filter encoding a set of popular npm packages that are definitely CommonJS. I forgot to mention it in my OP because it was one of the more out-there ideas. I’m not sure it would be a big improvement over “always assume CommonJS.”

Here's an alternative solution, that I think would be easy for developers to understand, though it would involve many more organizations than just Node.js and TypeScript, though mainly npm.

When under node_modules, packages with a package.json without "type" are always interpreted as CommonJS. These packages will be assumed to be "old".
Various tools start refusing to publish new packages that don't have "type" set

Package registries, such as npm, Github, Gitlab
Package publishing tools, such as npm, Yarn, pnpm
Possibly TypeScript stops compiling such packages

I don't know if this is at all workable, but it's the easiest solution I can think of.

I don't think it's workable for anyone to refuse to publish new packages that don't have "type".

Various tools start refusing to publish new packages that don’t have "type" set

We asked npm about this when we were working on --experimental-default-type and the answer was basically no, because the npm registry is for more than just Node stuff, and targets like browsers don’t need a type field. They also won’t ever change an already-published package, say to add the type field, because of security; current consumers of old packages might be checking integrity checksums, and those are never supposed to change.

I’m not sure it would be a big improvement over “always assume CommonJS.”

Stepping back for a second, what if we did assume all ambiguous type definition files were just always CommonJS? Isn’t the problem limited to a potential future case where someone publishes a package with ambiguous type definition files where those packages only work when consumed by users with detection enabled (so ESM syntax and no type field)? But no one’s published such a package yet, and likely won’t for months if not years. If you changed the type definition output to be unambiguous in the meantime, you’ll more or less prevent the problem unless someone is trying to slip through the cracks (i.e. using an old version of TypeScript while targeting only very new, currently future, versions of Node). That’s not really very different than someone intentionally publishing incorrect type definition files which they can already do today. So if you’re okay with the types being possibly incorrect in rare cases (aren’t we only talking about default exports here?) then the problem is solved.
So it’s really a question of what your tolerance is. If you truly wanted to ensure correct types 100% of the time, then I’d think you would always read the accompanying JavaScript file and not simply trust that the type definitions alongside it are correct. But that’s not what TypeScript currently does, so we’re already compromising on correctness in favor of performance at least somewhat.

Actually to amend my previous comment slightly, what if:

tsc is updated ASAP to start emitting unambiguous type definition files; and
Ambiguous type definition files are assumed to be CommonJS, however if a type error is thrown that could be caused by an ambiguous type definition file, the accompanying JavaScript file is read to resolve the ambiguity.

This would close the edge case at the cost of performance, but only under a condition that would be rare to occur (someone relying on detection enabled by default, and who generated the type definition files using an old version of TypeScript). The vast majority of the time would be either correctly-guessed ambiguous type definition files that are CommonJS, or unambiguous type definition files.

I don't see a resolution of https://github.com/microsoft/TypeScript/issues/56678. So maybe it's better to ship as a major.


fix: support IPv6 during ClientRequest options parsing

Following of this comment, I revisited the options parsing rules according to Node.js source and docs, and a few insights:

host and hostname are the same, hostname is preferred if both are specified.
host (and hostname) is Domain Name and doesn't include the port.
Node doesn't parse the port from the host. if a user includes the port in the host, Node refers to it as part of the host, and the result is an invalid URL like: http://google.com:1234:80

I also added tests for ipv6.
Last, the current implementation only differs from Node.js in that we return an Invalid URL instead of firing a bad request. This is due to the fact that we convert options to URL first. However, I believe this behavior is acceptable.
should fix #451

Released: v0.29.0 🎉
This has been released in v0.29.0!

📄 Release notes
📦 npm package

Make sure to always update to the latest version (npm i @mswjs/interceptors@latest) to get the newest features and bug fixes.

Predictable release automation by @ossjs/release.


[Bug] The spelling of Alumni is wrong in the Simple footer

Description
There is a typo in the simple footer. The alumni is spelled as "alumini" in the simple footer. I would like to correct this bug.
Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSOC'24 contributor.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

Go ahead @nidhipai78


Ferhatsirin77 patch 1

This looks great, but it's turkish in cz.json? if you can rename that cz.json to tu.json would be great! :)

Merged in https://github.com/kodadot/nft-gallery/pull/81


chore: bdd-spring-1585506535 to 0.0.1

chore: Promote bdd-spring-1585506535 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


stage.attestation.id in the AW internal browser

I am trying to create attestation with http://stage.attestation.id/ using AlphaWallet internal browser.
I discovered that background is dark somehow, but in Metamask and Chrome it looks correct. Maybe its because of dark theme?
AlphaWallet:

Chrome Devtools Mobile View:

AlphaWallet:

And another issue: it breaks when I try to connect ti the wallet with walletconnect.
@JamesSmartCell , can you help me to use AW correctly?

This issue stems from com/alphawallet/app/web3/Web3View.java:236:
/WebSettingsCompat.setForceDark(getSettings(), FORCE_DARK_ON);

This issue stems from com/alphawallet/app/web3/Web3View.java:236:
WebSettingsCompat.setForceDark(getSettings(), FORCE_DARK_ON);

great finding. is any option to update this seting from the browser? @JamesSmartCell

No unfortunately it's hardcoded into the webview settings. I think it should probably be removed, as I suspect it would cause issues with a lot of websites. But let's wait until James is back to confirm its ok.

Bump @justindg in case it's a bug.

Will remove webview dark mode for now. It will cause high contrast for most websites, maybe let's revisit in the future.

@JamesSmartCell , can you tell me when new wallet version will be deployed, so we can make sure it works good?


Clarify max body size units and format readme

It's my first contribution, as @SebCorbin said, It's an docs issue, so I've updated it BODY -> BYTE
Fixes #300

The issue with BODY/BYTES was fixed in d40bb9dc.  The formatting changes are fine but are separate from what this PR was trying to fix.