Fix various typos in markdown files.

Hi there,
As per title, just fixing typos in various .md files.
Didier
Signed-off-by: Didier Durand<EMAIL_ADDRESS>/kind documentation
What this PR does / why we need it:
Just some language cleanup
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
NO
NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Welcome @didier-durand! It looks like this is your first PR to falcosecurity/falco 🎉

/assign @leodido

LGTM label has been added.  Git tree hash: 3c9f404688175c2709767b2a93857e5779e287b0

LGTM label has been added.  Git tree hash: 3c9f404688175c2709767b2a93857e5779e287b0

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@leogr: it is my pleasure to contribute. Didier

@leogr: it is my pleasure to contribute. Didier

/milestone 0.28.0

/milestone 0.28.0

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leodido, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leodido,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leodido, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leodido,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/hold

/hold

@leodido : my pleasure to contribute! Didier

@leodido : my pleasure to contribute! Didier

@didier-durand: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@didier-durand: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

New changes are detected. LGTM label has been removed.

New changes are detected. LGTM label has been removed.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: b34392c4b919ba4729fe5785f099c75d9ef1b08c

LGTM label has been added.  Git tree hash: b34392c4b919ba4729fe5785f099c75d9ef1b08c

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/hold cancel

/hold cancel

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leodido, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leodido,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leodido, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leodido,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fntlnz, leodido, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fntlnz,leodido,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fntlnz, leodido, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fntlnz,leodido,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Failed on android with com.android.dex.DexException: Multiple dex files define Lcom/google/android/gms/internal/zzpq;

On Android gradles build process gives the following exception:
Unknown source file : UNEXPECTED TOP-LEVEL EXCEPTION:
Unknown source file : com.android.dex.DexException: Multiple dex files define Lcom/google/android/gms/internal/zzpq;
Unknown source file :   at com.android.dx.merge.DexMerger.readSortableTypes(DexMerger.java:596)
Unknown source file :   at com.android.dx.merge.DexMerger.getSortedTypes(DexMerger.java:554)
Unknown source file :   at com.android.dx.merge.DexMerger.mergeClassDefs(DexMerger.java:535)
Unknown source file :   at com.android.dx.merge.DexMerger.mergeDexes(DexMerger.java:171)
Unknown source file :   at com.android.dx.merge.DexMerger.merge(DexMerger.java:189)
Unknown source file :   at com.android.dx.command.dexer.Main.mergeLibraryDexBuffers(Main.java:502)
Unknown source file :   at com.android.dx.command.dexer.Main.runMonoDex(Main.java:334)
Unknown source file :   at com.android.dx.command.dexer.Main.run(Main.java:277)
Unknown source file :   at com.android.dx.command.dexer.Main.main(Main.java:245)
Unknown source file :   at com.android.dx.command.Main.main(Main.java:106)

When removing react-native-device-info-stuff from build.gradle, settings.gradle and MainActivity.java, everything works fine.

I'm having this same problem, did you find any way to fix it?

Not yet

Did you try ./gradlew clean?

Dex issues are caused by conflicting dependencies between libraries, from my experience very hard to track, one option as @kompot says is try
./gradlew clean
However if this does not solve your problem you need to go through each of your dependencies and check for conflicts, if I remember correctly in my case it was a sub-dependency on google play services, and try to force a specific version in the gradle build settings.

Resolve with that in android/app/build.gradle :
compile (project(':react-native-device-info')){
       exclude group: "com.google.android.gms"
}

I write whole article about this problem:
<EMAIL_ADDRESS>Hope I helped you :)

This is now mentioned in the troubleshooting section of the README file.
Closing this issue.


Added support for Telegram multiple messages

Hi,
please also update existing telegram tests and add the tests for the features you added before we can merge this https://github.com/claudiajs/claudia-bot-builder/tree/master/spec/telegram

Done!

Thanks, I'll review it during the weekend

@jveres I have a few minor comments but this looks good. Thanks!

Merged, thanks!

👍


RFC: Remove Dependency Criteria for Audits

Introduction
Audits in cargo-vet are associated with audit criteria, which allows the system to track and enforce audits for multiple sets of properties. We further generalized this with the concept of dependency criteria, which allows audits for a crate to specify the criteria that should be enforced for that crate’s dependencies. Put differently: once we allow users to require arbitrary combinations of criteria for top-level third-party dependencies, we need to specify what gets enforced for transitive dependencies. We concluded that the requirements should be inherited down the crate graph by default, but that the default behavior could be overridden by specifying dependency-criteria on either an audit or a policy entry.
I’ve come to believe that dependency-criteria on audits was a mistake, and that we should remove the feature. It has never once been used in practice in any cargo-vet instance I’m aware of, which is a strong indicator that it is not successfully serving user needs. To understand why, let’s look at the original motivating use-cases.
Requirement Strengthening
The first and primary use-case was what we’ve sometimes referred to as “requirement strengthening”. The idea was that an auditor might conclude that the validity of the audit depends on certain properties of a dependent crate. For example, whether or not a crate is safe-to-deploy might depend on whether a dependent crate returns the right-sized parameters for an unsafely-allocated memory buffer, a properly-validated path for a filesystem operation, or the correct definition of a constant for an FFI call. Because semantic versioning generally allows the contents of the dependency to vary across deployments at the publisher’s discretion, it’s technically not enough even to verify these properties directly in the current version of the dependent crate, because they might change out from under you.
Unfortunately, I think it’s unrealistic to expect people to do this consistently and at-scale. A corner-cutting auditor might eyeball a block of unsafe code and conclude it’s fine without fully tracing through the information flow of its inputs. A judicious auditor might trace those inputs back through rest of the crate. A tenacious auditor might even trace them across crate boundaries into the current version of a dependent crate. But approximately nobody is going to do all that and then derive the full set of required properties, articulate those properties as coherent custom criteria, and then figure out how to encode all of that in cargo-vet.
Realistically most auditors will end up assuming that external crates behave as advertised, and so I don’t think it’s productive to try to set the bar higher than that. This does create some potential failure modes and attack vectors — i.e. the introduction of an exploitable defect via separate seemingly-innocuous changes in two different crates — but the concrete risk level is debatable and I don’t want to let perfect be the enemy of good.
Requirement Weakening
The second use-case is “requirement weakening”. The idea is that an auditor might discover while auditing crate X that its transitive dependency Y is only used in a very limited way (e.g. in development mode) that requires a weaker audit. Unfortunately, people aren’t thinking about this consideration when auditing X. They might think about it when auditing Y (“why do we have this dependency anyway? Oh I see, it’s used by X's tests”) — but at that point it’s generally too late, since the flight rules of the system generally don’t permit you to just go and amend the existing audit for X.
So in practice nobody is doing any requirement weakening, which means in practice it’s not saving us any work. Moreover, much of the value here is automatically captured by cargo-vet's default behavior of requiring only safe-to-run for dev-dependencies. We have found it occasionally useful to record dependency-criteria overrides on policy entries, and I propose we keep the concept there. These aren’t shareable across projects, but they’ve proven themselves to have actual utility whereas the ones on audit entries have not.
Simplifying the Implementation
Beyond simplifying the feature set and aligning it with real user needs, dropping dependency-criteria from audit entries would dramatically simplify the implementation. They are a large (perhaps the largest) driver of complexity in the algorithm, and I suspect removing them would make the code much easier to reason about.

Would like to roll this into the next release unless somebody objects.

The main usecase (in my books) for this was definitely the Strengthening for "crypto-safe" for whatever cryptography libraries. Google has found a way to invert the encoding so it's only Weakening, but it still requires an asymmetric boundary between "library that implements crypto" and "library that uses it".

I might have a use case for this (unless there is a better way to solve that which I haven't discovered yet):
Build-dependencies are difficult to map to criteria, as it all depend on what they are used for. The default mapping to safe-to-deploy is the correct conservative mapping, but safe-to-run will often be sufficient.  This must be determined on a case-by-cases basis by looking at how the crate is used, but I have found this difficult to encode in the configuration.
It seems very useful to be able to encode this as dependency-criteria.  Let's say you have a crate that pulls in some crates to implement some logic in a build.rs file.  If this does not affect the generated binary then it would make sense to be able to encode in dependency-criteria that this dependency only needs safe-to-run.
In fact, it would also be very useful to allow the user to specify dependency-criteria on first party crates as well for the same reason.  I tried this but could not make it work.  I suspect that audits for first-party crates are ignored.
One could argue that this not important because build-dependencies will have to be audited anyway and one can consider usage then.  This is not always the case.  Consider a project where one is permitted to use externally sourced audits for safe-to-run, but must do internal audits for safe-to-deploy.  This seems a reasonable policy for many projects where the security of the produced binary is important.  In this case, specifying dependency-criteria can make it possible to map build-dependencies to safe-to-run where appropriate and make many internal audits unnecessary.  However, a complete solution requires the ability to specify dependency-criteria for first-party crates, which does not seem to work.
There might be a better way to achieve the same result, but I have not found one yet.

@hpenne Thanks for the detailed use-case analysis. To be clear, under this proposal you'll still be able to specify dependency-criteria on policy entries in config.toml to achieve the effect you're looking for (which I suspect will also solve your first-party problem — firefox does exactly this). The change here is removing the ability to specify this kind of constraint in an audit.

As a followup to the above — @mystor pointed out to me that policy entries currently only apply to first-party crates because you were supposed to use the dependency-criteria features on audit entries for third-party crates. However, her patch to implement this ticket (and remove that feature) allows policy entries to apply to any crate, so the above technique should work going forward.

@bholley Thanks for clarifying. I missed the detail about policy entries only working for first-party crates, which explains some of the problems I had with configuration yesterday.  I would very much welcome a change where this is also allowed for third-party crates. I hope it gets merged and released soon.
I can see why having dependency-criteria for audits adds complexity. However, this mapping should be set up based on how a crate actually uses its dependencies, and as such it is logically tied to a specific crate version. It makes sense that it is a part of the audit of a specific version. Specifying this in the policy section makes the mapping independent of the crate version.  The crate could theoretically be released in a new version that changes how a dependency is used in such a way that the dependency-criteria needs to be changed. Having dependency-criteria only in policies means that this can "go under the radar". I'm not sure that removing this is a good idea.

@bholley Thanks for clarifying. I missed the detail about policy entries only working for first-party crates, which explains some of the problems I had with configuration yesterday. I would very much welcome a change where this is also allowed for third-party crates. I hope it gets merged and released soon.

Work is underway so I'd expect it to be merged in the next few days. We're aiming to do an 0.4 release in a few weeks, but you can use it with cargo install --git before then.

I can see why having dependency-criteria for audits adds complexity. However, this mapping should be set up based on how a crate actually uses its dependencies, and as such it is logically tied to a specific crate version. It makes sense that it is a part of the audit of a specific version. Specifying this in the policy section makes the mapping independent of the crate version. The crate could theoretically be released in a new version that changes how a dependency is used in such a way that the dependency-criteria needs to be changed. Having dependency-criteria only in policies means that this can "go under the radar". I'm not sure that removing this is a good idea.

That's a really good point. I still think dependency-criteria don't belong in the audit table, but I'm persuaded that entries for third-party crates in the policy table should be associated with a specific version. I'll file an issue.

That's a really good point. I still think dependency-criteria don't belong in the audit table, but I'm persuaded that entries for third-party crates in the policy table should be associated with a specific version. I'll file an issue.

I suppose the difference will be that if I publish an audit and include dependency-criteria in that, then everyopne using my audit will benefit from my analysis of how it uses its dependencies. If you move dependency-criteria to the policy table then everyone who uses my audit will have to inspect the version I audited themselves to determine dependency usage.
The question is if this is really a realistic scenario, though.

I suppose the difference will be that if I publish an audit and include dependency-criteria in that, then everyopne using my audit will benefit from my analysis of how it uses its dependencies. If you move dependency-criteria to the policy table then everyone who uses my audit will have to inspect the version I audited themselves to determine dependency usage.

Yep — this is the reason we originally added dependency-criteria to audits.

The question is if this is really a realistic scenario, though.

And this is the reason we're removing them. :-)

@hpenne I've filed #381 with a potential solution which could be used for that scenario in the future without requiring the full infrastructure of dependency-criteria. We're not planning to implement it or something like it right now, but I think we'll be able to get those benefits without dependency-criteria's drawbacks in the future if we need it.


FR: Calling an Apple Revoke Token endpoint

Feature proposal

Firebase Component: Auth

Apple's new guidelines say,
If your app offers sign-in with Apple, you must use sign-in with the Apple REST API to revoke your user token when you delete your account.
https://developer.apple.com/news/?id=12m75xbj
Does Firebase Auth offer any functionality to assist with account deletion?

A similar question has been asked on stack overflow.
https://stackoverflow.com/questions/72415814/where-i-get-from-ios-firebase-api-the-apple-revoke-tokens-endpoint-parameters-c

Hi @linkillhz -
you can delete user accounts in Firebase Auth using user.delete():
if let user = Auth.auth().currentUser {
  do {
    try await user.delete()
  }
  catch {
    print(error)
  }
}

See the docs for more details.
To delete any of the user's associated data, you can use the Delete User Data Extension. This essentially is a Cloud Function that will run whenever a Firebase Auth user account is deleted (either using the call above or via the Firebase console), and can delete the user's data in Cloud Firestore, Cloud Storage, and the Realtime Database.
If you find that your data model is different from what the Extension currently supports:

Please consider leaving feedback on the Firebase Extensions repo (file a feature request outlining how your data model looks like)
Consider implementing your own Cloud Function based on the code of the Delete User Data Extension

As for revoking the token, @rosalyntan and @renkelvin can provide more details.

Hi,  @rosalyntan and @renkelvin
Firebase SDK did not call Apple's revoke token endpoint when I deleted my Firebase account.
I have a few questions.

What should be the flow for calling Apple's revoke token endpoint when deleting a Firebase account?
Do I need to implement my own Cloud Function?
Also, can I get the necessary parameters for Apple's revoke token endpoint?

Are there any plans to support calling Apple's revoke token endpoint in the Firebase SDK?

Regards

+1 Apple is complaining about this being a requirement June 30th. Looks like apps wont be accepted after that date if they do not delete tokens. This is the message:

Starting June 30, 2022, apps submitted to the App Store that support account creation must also include an option to initiate account deletion.
We noticed this app may support account creation. If it does not, you may disregard this message. If it already offers account deletion or you’re working to implement it, we appreciate your efforts to follow the App Store Review Guidelines. Apps submitted after June 30 that do not comply with the account deletion requirements in guideline 5.1.1(v) will not pass review.
Learn more about the account deletion requirements. If your app offers Sign in with Apple, use the [Sign in with Apple REST API](https://developer.apple.com/go/?id=ar-ad-si

wa) to revoke user tokens.

+1 this is a real problem to be addressed before June 30.
Either implement it automatically in the SDK (would be great) or give us access to the requested parameters so we can do it ourselves.

I don't think it makes sense to offer this with Firebase SDK. This is something the developer has to handle independently with apple auth. You can get the client secrets and token strings during login or sign up. It is decoupled from the Firebase flows.
However, apple has provided very poor documentation for how to implement this. Even if you test their API with something like postman, you will always get status code 200 even with incorrect parameters. w

I don't think it makes sense to offer this with Firebase SDK. This is something the developer has to handle independently with apple auth. You can get the client secrets and token strings during login or sign up. It is decoupled from the Firebase flows.
However, apple has provided very poor documentation for how to implement this. Even if you test their API with something like postman, you will always get status code 200 even with incorrect parameters.

Perhaps you could help explain how this could be implemented, for clarity, I'm using Firebase & Flutter, I use Firebase Auth with flutter and sign_in_with_apple to authenticate with Firebase via the method in the documentation: https://firebase.google.com/docs/auth/flutter/federated-auth
To delete my user, the documentation for firebase/flutter states to call user.delete().
This however would cause an app rejection via Apple's new policy. Is your position that users should know to create their own server outside Firebase, track secrets there after authenticating on Firebase, and then call that server to revoke tokens alongside the calls to firebase to delete the authenticated user?
If so where should that be documented and would a cloud function and Firestore be fine to use to keep track of tokens and revoke on delete?

Hi all -
Thanks for raising this issue - we are aware of this and are actively working on a solution. We will update this issue once we can share more details.
Documentation (and accompanying sample code) would definitely be part of the solution. This will likely be a blog post that shows how to use our solution, fast-followed by an update of our docs (https://firebase.google.com/docs/auth/ios/apple).
We would want to avoid developers having to stand up their own servers outside of Firebase - this would defeat the whole purpose of an mBaas platform.

To @CalHoll,
I would be delighted to know that the proposed solution being worked on by @peterfriese (and the Firebase team) could cover the upcoming requirement.
However, you can see my attempt at this issue here: [https://stackoverflow.com/questions/72399534/how-to-make-apple-sign-in-revoke-token-post-request/72488104?noredirect=1#comment128373667_72488104]
The general principle of my approach is that I am attempting to perform a POST request with limited understanding of the parameters at play. I have not been able to verify that my solution works when running in a debug environment. To my understanding, I did not think it would have been possible to meet the requirement within the Firebase ecosystem, but will be very pleasantly surprised to be educated otherwise.

Apple's revoke API is called based on JWT signed by the developer's key, so I think it will be difficult for the client to implement the function. I think it's most intuitive to upload a key file from the Firebase console (like cloud messaging) and automatically process it when user.delete(). However, with Apple's instruction request date just around the corner, the way to generate JWT and invoke API via Firebase functions seems not bad.

Firebase functions require the Blaze plan, though. My app is small and so far I haven't used Firebase functions, yet. It would be great if we could just upload the JWT file to Firebase and Firebase takes care of the rest after calling delete() on the user account.

For now I'm trying to implement the revocation calls myself, but I can't find a way to get access_token and refresh_token that I need to revoke. Does Firebase allow to get them? Maybe in some delegate calls during authentication?

I'm also trying to fill in the "token" parameter,
The identityToken I get when I try to log in with Firebase
But this parameter filling does not work,
Trouble Firebase tell me if there is a way to get the token parameters required for Post
And what is the parameter type refresh_token or access_token

Hi all -
Thanks for raising this issue - we are aware of this and are actively working on a solution. We will update this issue once we can share more details.
Documentation (and accompanying sample code) would definitely be part of the solution. This will likely be a blog post that shows how to use our solution, fast-followed by an update of our docs (https://firebase.google.com/docs/auth/ios/apple).
We would want to avoid developers having to stand up their own servers outside of Firebase - this would defeat the whole purpose of an mBaaS platform.

Sadly this blog is not very helpful

After getting the appleIDCredential.authorizationCode from the app, you need to get the refresh token through the auth/token API.
The whole process is as follows.

Get authorizationCode from App where user log in.
Get a refresh token with no expiry time using authorizationCode with expiry time.
After saving the refresh token, revoke it when the user leaves the service.

I implemented it in Firebase functions and put the code to be reference on the link below.
https://github.com/jooyoungho/apple-token-revoke-in-firebase
If you did it right, it would have disappeared from your device's Settings - Apps using Apple Sign-in.

Thanks,it works!

TEAM ID, CLIENT ID, and KEY ID can all be found on the Apple site.
TEAM ID: Your team ID, which can be found at https://developer.apple.com/account/#!/membership.
CLIENT ID: For apps, this is your App ID (IDENTIFIER) located at https://developer.apple.com/account/resources/identifiers/list.
KEY ID: This is the Key ID that appears when you go to https://developer.apple.com/account/resources/authkeys/list.

Is there a general timeframe for when this will be implemented in the Firebase Auth flow? Curious if it is worth putting in the time for a custom solution or just wait for Firebase to implement this.

@peterfriese Is there any expected solution within the coming days?
June 30th is getting closer and we wouldn't wanna be rejected due the Apple's requirements, and we'd rather not use some cumbersome custom solutions for that.

From collective analysis on https://github.com/invertase/react-native-apple-authentication/issues/282 it appears the only way to do the revoke is to generate a fresh valid authorization code + a fresh JWT built with a client secret that Firebase Auth has already
It is possible to pre-generate and store a JWT locally (in app) with a maximum expiry of 6 months (per documentation) at cost of possible spoofing with exposed JWT plus need to update app periodically or  you have a server API somewhere (could be here!) that can take the known secret firebase auth already has to generate the JWT for post to the Apple REST API for token revocation
It does not appear there is any avoiding some user interaction though as I believe you need an authorization code from the Sign In With Apple APIs which is short-lived and based on my understanding of the Firebase Auth APIs, Firebase does not itself maintain a token to revoke?
So if I understand correctly and all the above is correct - the optimization possible here is the elimination of the need for a server by use of some possible new firebase auth API and existing firebase servers, that can generate take a non-expired apple authorization code from a caller (perhaps react-native-apple-authentication, for react-native people) and do the rest (JWT generation, REST API calls). That's definitely useful so I'm interested in a timeline too if possible.
As a library maintainer, it will take me a while to wrap anything that comes up for react-native-firebase and to communicate how to use it for react-native-apple-authentication users

After getting the appleIDCredential.authorizationCode from the app, you need to get the refresh token through the auth/token API.
The whole process is as follows.

Get authorizationCode from App where user log in.
Get a refresh token with no expiry time using authorizationCode with expiry time.
After saving the refresh token, revoke it when the user leaves the service.

I implemented it in Firebase functions and put the code to be reference on the link below. https://github.com/jooyoungho/apple-token-revoke-in-firebase
If you did it right, it would have disappeared from your device's Settings - Apps using Apple Sign-in.

Hey @jooyoungho.
I am able to retrieve the  appleIDCredential.authorizationCode at signIn/signUp, but how do I retrieve it while the user is already signed in using firebase? The user never goes through ASAuthorizationControllerDelegate if they have already signed in since last time they used the app.

Please, Firebase (@peterfriese) , when you implement the solution, make it possible to revoke the Apple token of the user also from a web server, and not only from the app.
I delete the user account from a web server (using this firebase php api https://github.com/kreait/firebase-php), because It does not force the user to re-authentify before account deletion. (I already display 2 confirmations popup in the app, to ensure that the user really wants to delete his account).

Thanks for creating the code generator, @jooyoungho - this looks like a useful resource!
Regarding UserDefaults - I think it would be useful to call this out more prominently in your repo. Storing this in the iOS keychain might be a better option - this is what Apple recommends in Storing Keys in the Keychain, and what Firebase Auth does to store the currently signed in user (we store the Firebase ID token using FIRAuthStoredUserManager)

but how do I retrieve it while the user is already signed in using firebase? The user never goes through ASAuthorizationControllerDelegate if they have already signed in since last time they used the app.

@Knapiii You can implement a user-facing 'reauthentication' flow. That way you can require reauthentication prior to displaying the account deletion UI in your application. Doing that effectively requires that the user tap on another Sign In With Apple or Continue With Apple button.
See https://firebase.google.com/docs/auth/ios/apple#reauthentication_and_account_linking for more details. Doing this safely guarantees you will have the sensitive information required to revoke the token without any of it expiring.
Keep in mind, when reading the Firebase account deletion documentation, it can be considered a 'sensitive' action which would require reauthentication. Firebase recommends performing reauthentication for 'sensitive' actions. I think account deletion and revoking the token from Sign in With Apple are considered sensitive with regards to backend services.

I had a question regarding the refreshToken cloud functions.  If I use a reauthenticate workflow, I'll can get the authorizationCode needed for the cloud functions mentioned.  However, can we also just get the refresh tokens by performing this in authorizationController(controller: ASAuthorizationController, didCompleteWithAuthorization authorization: ASAuthorization)?
Auth.auth().currentUser?.reauthenticate(with: credential) { (authResult, error) in     print ("received refresh token : \(authResult?.user.refreshToken) :: \(error)")  }
Or can we not revoke this refresh token?
Thanks,

but how do I retrieve it while the user is already signed in using firebase? The user never goes through ASAuthorizationControllerDelegate if they have already signed in since last time they used the app.

@Knapiii You can implement a user-facing 'reauthentication' flow. That way you can require reauthentication prior to displaying the account deletion UI in your application. Doing that effectively requires that the user tap on another Sign In With Apple or Continue With Apple button.
See https://firebase.google.com/docs/auth/ios/apple#reauthentication_and_account_linking for more details. Doing this safely guarantees you will have the sensitive information required to revoke the token without any of it expiring.
Keep in mind, when reading the Firebase account deletion documentation, it can be considered a 'sensitive' action which would require reauthentication. Firebase recommends performing reauthentication for 'sensitive' actions. I think account deletion and revoking the token from Sign in With Apple are considered sensitive with regards to backend services.

Hi @andrejandre, were you able to get this to work with the 'reauthentication' flow?  I'm getting the authorization code via:
            guard let authorizationCode = appleIDCredential.authorizationCode,                   let codeString = String(data: authorizationCode, encoding: .utf8) else {                 print ("unable to get codeString")                 return             }
Unfortunately, the revokeToken cloud function isn't revoking my tokens using the new authorization code.  I'm using the cloud functions provided by @jooyoungho.
Thanks.

Hi there,
I successfully revoke token by followed this guide. and
I checked
iPhone Settings >  Click iCloud Account > Password & Security > There's no app in Apps Using Apple ID
(thanks for the guide jooyoungho)
Also delete the user using this code
but, when I try to sign up using the previous apple Id I delete.
I am unable to get user's email and name

Usually, if u revoke Sign in with Apple manually from Settings  or from here https://appleid.apple.com/
later the app will ask if I want to share name, email or not.
But I am unable to get same result by revoking token via Apple's API

Does anyone here having same issue? how to solve it.

Hey - I'm trying to follow the steps laid out by @jooyoungho (thanks)
The Apple login wasn't initially set up by me, and oddly I have found that whilst it is enabled in our app identifier, there is no key attached to it.
I assume that means that I need to create one and attach it to the identifier in order to even begin this process?
I'm using react native with @mikehardy's https://github.com/invertase/react-native-apple-authentication
I'm kind of confused as to how the initial dev wasn't required to set this up. Does anyone know what effect this change might have on our current Apple Sign In Users once released?
Also - after 7 days its worth asking how the Firebase in-house solution is coming along.
Thanks for all your work guys - I'd be totally lost without these threads.

I too did not have a key setup for 'Sign In with Apple' in my developer account and it has always worked for me w/o the key. I followed this example...
https://firebase.google.com/docs/auth/ios/apple
I guess I never setup a key because of this instruction from the documentation....
"In the Firebase console, open the Auth section. On the Sign in method tab, enable the Apple provider. If you are only using Sign In with Apple in an app, you can leave the Service ID, Apple Team ID, private key and key ID fields empty."
But since the solution proposed by @jooyoungho (thanks, btw) uses firebase functions (ie, web environment), I think a key is required.

@dillydallygames I was puzzled about that difference in reported success / difficulties from others, and your explanation was really clear, thanks for posting that

Is there any kind of updated timeline on this from the Firebase team? As a newer developer, and still working on understanding authorization flows, the manual implementation is proving to be very challenging for me.

Despite trying multiple options, the suggested Cloud Function option is not working.  I'm really hoping the Firebase team starts to take this as serious as they should have prior to Apple making their requirement changes.   Any updates are greatly appreciated...

Hi @blinkin the Cloud Function option worked for me. I'm using React Native and using the Cloud Function's endpoint with a POST request method passing the data by a query parameter. Why it is not working for you? How did you try?

Hi @arangurenlucas, I've been trying multiple variations of @jooyoungho's solution.  Apple's REST API continues to respond with 400's and unfortunately they don't provide much in terms of why so it's a lot of trial and error (unfortunately w/o any success).   I'm sure it's a PEBKAC error but a very slow process to diagnose -- I'm hoping Apple will temporarily accept option #2 suggested by @peterfriese while I continue to work through CFs handling properly.

Any update available on this? Still looking good for 4th quarter?

We're planning to finish implementation in Q4/2022. Due to the EOY code freeze, the GA launch might slip into early Q1/2023.

@peterfriese.  Is the solution at a point where we should just wait for the Firebase implementation in early Q1?  Or implement the workaround?  Thx!!

I stuck with my iOS App Version because of this issue. Anyone got a interim solution with the JS SDK running?

@peterfriese. Is the solution at a point where we should just wait for the Firebase implementation in early Q1? Or implement the workaround? Just wanted to see if it was still on schedule for early Q1. Thx!! PS - Need any guinea pigs? :)

Yes, we're still on track for early Q1. We'll reach out to this thread once we've got something that you can take for a spin.

I stuck with my iOS App Version because of this issue. Anyone got a interim solution with the JS SDK running?

https://github.com/jooyoungho/apple-token-revoke-in-firebase is the best interim solution according to me. I successfully published my iOS app on the App Store with this solution after getting a rejection the first time.

Doesn't realy help with the JS SDK as I don't understand how to get the mentioned authorizationCode. I also created a Q&A Post here to get help: https://github.com/firebase/firebase-ios-sdk/discussions/10565
Do I understand it right that the early Q1 solution will not depend on any SDK but directly revoke Apple ID Logins when deleteing a firebase User connected to Apple ID?

Do I understand it right that the early Q1 solution will not depend on any SDK but directly revoke Apple ID Logins when deleteing a firebase User connected to Apple ID?

To revoke a token, you will have to call the revokeToken method, similar to this:
let appleIDCredential = authorization.credential
let auth = Auth.auth()
do {
  try await auth.revokeToken(appleIDCredential.authorizationCode)
} catch {
    print("Token revocation failed with error \(error)")
}

We considered making this a back-end only implementation, but this would require the Sign in with Apple access/refresh tokens/authorization code to be stored in our backend. We decided against this for a number of reasons:

Storing the tokens in our backend increases the risk of them being stolen in the (unlikely) event of our backend being compromised. In the interest of security, we decided to err on the side of being more secure.
Since deleting the user account is a security-sensitive operation, you need to re-authenticate the user anyway (see the Firebase Authentication documentation). So even if we did store the token in the backend, you would have to re-auth the user to get a fresh token.

You can follow #10580 for the implementation of the token revocation API.

@peterfriese
The revokeToken() could fail, so I assume it's meant to be called first and if it works successfully, then we delete the user's account?

To revoke a token, you will have to call the revokeToken method, similar to this:
let appleIDCredential = authorization.credential
let auth = Auth.auth()
do {
  try await auth.revokeToken(appleIDCredential.authorizationCode)
} catch {
    print("Token revocation failed with error \(error)")
}

So it sounds like the Q1 solution will be something like:

Have the user re-login to get the Apple credential/auth code.
Call the revokeToken method on the Firebase auth object
Magic happens.

If it's that easy.. and all done in my UI.. I'm definitely waiting until Q1!
Thanks!!!

Hey guys, have there been any more updates on this?

ditto

If anyone is using Flutter and hung up on this, here is a temporary workaround.
Delete user (App side)
Since deletion is "security-sensitive operation", asking the user to log in again is recommended. I tried the Firebase Auth for providing the token and it did not work. I had to use the_apple_sign_in for that.

Flutter Code
deleteUser() async {
  final AuthorizationResult res =
      await TheAppleSignIn.performRequests([
    const AppleIdRequest(
        requestedScopes: [Scope.email, Scope.fullName])
  ]);

  switch (res.status) {
    case AuthorizationStatus.authorized:
      try {
        // Get Token
        final AppleIdCredential appleIdCredential = res.credential!;

        String accessToken = String.fromCharCodes(
            appleIdCredential.authorizationCode!);

        try {
          HttpsCallableResult resp = await FirebaseFunctions.instance
              .httpsCallable('revokeToken')
              .call(accessToken);
          debugPrint(resp.data.toString());

          if (resp.data["status"] != null) {
            if (resp.data["status"].toString() == "success") {
              await FirebaseAuth.instance.currentUser!.delete();
              await signOut();
            }
          }
        } catch (firebaseFunctionError) {
          debugPrint(firebaseFunctionError.toString());
          return firebaseFunctionError.toString();
        }
      } on PlatformException catch (error) {
        debugPrint(error.message);
        return error.code;
      } on FirebaseAuthException catch (error) {
        debugPrint(error.message);
        return error.code;
      }
    case AuthorizationStatus.error:
      return 'auth-error';
    case AuthorizationStatus.cancelled:
      return 'auth-cancelled';
    default:
      return 'auth-error-other';
  }
}

Revoke function (Server side)
To revoke, use the authorization code obtained in the app to generate a request_token. Use this request_token to perform the revocation. The makeJWT() and the original code I modified for this was provided by @jooyoungho https://github.com/firebase/firebase-ios-sdk/issues/9906#issuecomment-1162716985
Replace com.example.app in both instances of "client_id": "com.example.app", with your bundleID.

Firebase Function
exports.revokeToken = functions.https.onCall(async (request, response) => {
  // import the module to use
  const axios = require("axios");
  const qs = require("qs");

  const authorizationCode = request.toString();
  const clientSecret = makeJWT();

  let data = {
    "code": authorizationCode,
    "client_id": "com.example.app",
    "client_secret": clientSecret,
    "grant_type": "authorization_code",
  };

  // first get the refresh token
  try {
    const res = await axios.post("https://appleid.apple.com/auth/token", qs.stringify(data), {
      headers: {
        "Content-Type": "application/x-www-form-urlencoded",
      },
    });

    const refreshToken = res.data.refresh_token;

    data = {
      "token": refreshToken,
      "client_id": "com.example.app",
      "client_secret": clientSecret,
      "token_type_hint": "refresh_token",
    };

    // use the refresh token to revoke
    try {
      const res = await axios.post("https://appleid.apple.com/auth/revoke", qs.stringify(data), {
        headers: {
          "Content-Type": "application/x-www-form-urlencoded",
        }});

      functions.logger.log(res.status);
      return {"status": "success"};
    } catch (revokeError) {
      functions.logger.log(revokeError);
      return {"status": "failed", "message": revokeError.toString()};
    }
  } catch (refreshError) {
    functions.logger.log(refreshError);
    return {"status": "failed", "message": refreshError.toString()};
  }
});

Note!
This is just a proof-of-concept code. Do the appropriate error checks and take security measures as needed.

Is it still in progress?

Is it still in progress?

Yes, it is.

spent the last 2 days trying to get the cloud functions working just keep getting a 400 code from apple. Really hope this is fixed soon.

For those who are running into 400 error:
AuthorizationCode leaves only 5 minutes, and can be used only once. Otherwise, I'm getting 400 error. I haven't found this information anywhere except apple official documentation
https://developer.apple.com/documentation/sign_in_with_apple/generate_and_validate_tokens

We are still looking for a practical solution to this problem.

This will be launching in the next release of the Firebase SDK.

Actually, my understanding is that this was fixed in #11001 and already released in 10.8.0.
cc: @renkelvin

@paulb777 Just to confirm, is the expected behaviour that calling Auth.auth().currentUser.delete { .. } should revoke the token automatically also? I tested this now so using 10.8.1, called delete but the token still appears in the apple id page. Could it be because the app isn't live on the app store?

Hi @CristianMoisei - token revocation needs to triggered separately from deleting the user account.
We're currently working on updating the docs (and a video is also on the way), but broadly speaking, you will have to do the following:

Make sure you filled out the Services ID and OAuth code flow configuration
section of the Sign in with Apple provider configuration in the Firebase console

Since Firebase does not store user tokens when users are created with Sign in with Apple, you must ask the user to sign in again before revoking their token and deleting the account. To avoid confusing users, you should show a dialog first, telling the user that they need to authorize deleting their account. For an example of how to do this, have a look at the Craft app

Obtain the authorization code from the ASAuthorizationAppleIDCredential, and use it to call Auth.auth().revokeToken(withAuthorizationCode:) to revoke the  user's tokens.

Finally, delete the user account (and all associated data)

I try to understand Peter's recommendations, but do not understand how to get ASAuthorizationAppleIDCredential. We use flutter with firebase_auth.

I try to understand Peter's recommendations, but do not understand how to get ASAuthorizationAppleIDCredential. We use flutter with firebase_auth.

I've filed a feature request on the FlutterFire repo for you, @TheoBo

Thank you @peterfriese, works great!
I was wondering what would be best practice for handling token revocation while using the local Firebase emulator?
When attempting to revoke the token with the Auth.auth().revokeToken(withAuthorizationCode: authCode) function, FIRAuthErrorDomain 17004 error is thrown which refers to "ERROR_INVALID_CREDENTIAL".
Would simply skipping the above function be the way to go if the local emulator is being used?
Thanks!

@peterfriese  sorry, but Auth.auth().revokeToken(withAuthorizationCode:) method is absent in the latest master branch.

@peterfriese sorry, but Auth.auth().revokeToken(withAuthorizationCode:) method is absent in the latest master branch.
I imported FirebaseCore, FirebaseAuth

It should be there: https://github.com/firebase/firebase-ios-sdk/blob/8efafcb340eb097f4faa2a3c24e46185ff28b28e/FirebaseAuth/Sources/Public/FirebaseAuth/FIRAuth.h#LL858C31-L858C31 - can you double check you're either on master or on 10.8.0 or higher?

CocoaPods and SPM use the same versioning. We release them at the same time.


fixup(gateway)!: Remove broken compression feature

This feature was broken, hence removing it. Also needed for cargo hack --feature-powerset check to work.
Sort of related to #1314

Why is it broken, and should we fix it instead?

Why is it broken

compiling gateway with default-features=false, features = ["compression"] gives the following lovely output:
error[E0433]: failed to resolve: use of undeclared crate or module `miniz_oxide`
 --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:6:5
  |
6 | use miniz_oxide::deflate::core::CompressorOxide;
  |     ^^^^^^^^^^^ use of undeclared crate or module `miniz_oxide`

error[E0433]: failed to resolve: use of undeclared crate or module `miniz_oxide`
 --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:7:5
  |
7 | use miniz_oxide::inflate::stream::InflateState;
  |     ^^^^^^^^^^^ use of undeclared crate or module `miniz_oxide`

error[E0432]: unresolved imports `miniz_oxide`, `crate::ffi::Backend`, `crate::ffi::Deflate`, `crate::ffi::DeflateBackend`, `crate::ffi::ErrorMessage`, `crate::ffi::Inflate`, `crate::ffi::InflateBackend`, `miniz_oxide::deflate::core::CompressorOxide`, `miniz_oxide::inflate::stream::InflateState`
 --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/mem.rs:6:24
  |
6 | use crate::ffi::{self, Backend, Deflate, DeflateBackend, ErrorMessage, Inflate, InflateBackend};
  |                        ^^^^^^^  ^^^^^^^  ^^^^^^^^^^^^^^  ^^^^^^^^^^^^  ^^^^^^^  ^^^^^^^^^^^^^^
  |
 ::: /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:6:5
  |
6 | use miniz_oxide::deflate::core::CompressorOxide;
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
7 | use miniz_oxide::inflate::stream::InflateState;
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
8 | pub use miniz_oxide::*;
  |         ^^^^^^^^^^^ use of undeclared crate or module `miniz_oxide`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:10:32
   |
10 | pub const MZ_NO_FLUSH: isize = MZFlush::None as isize;
   |                                ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:11:37
   |
11 | pub const MZ_PARTIAL_FLUSH: isize = MZFlush::Partial as isize;
   |                                     ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:12:34
   |
12 | pub const MZ_SYNC_FLUSH: isize = MZFlush::Sync as isize;
   |                                  ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:13:34
   |
13 | pub const MZ_FULL_FLUSH: isize = MZFlush::Full as isize;
   |                                  ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:14:30
   |
14 | pub const MZ_FINISH: isize = MZFlush::Finish as isize;
   |                              ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared type `DataFormat`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:31:9
   |
31 |         DataFormat::Zlib
   |         ^^^^^^^^^^ use of undeclared type `DataFormat`

error[E0433]: failed to resolve: use of undeclared type `DataFormat`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:33:9
   |
33 |         DataFormat::Raw
   |         ^^^^^^^^^^ use of undeclared type `DataFormat`

error[E0433]: failed to resolve: use of undeclared type `InflateState`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:58:20
   |
58 |             inner: InflateState::new_boxed(format),
   |                    ^^^^^^^^^^^^ use of undeclared type `InflateState`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:70:21
   |
70 |         let flush = MZFlush::new(flush as i32).unwrap();
   |                     ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared crate or module `inflate`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:72:19
   |
72 |         let res = inflate::stream::inflate(&mut self.inner, input, output, flush);
   |                   ^^^^^^^ use of undeclared crate or module `inflate`

error[E0433]: failed to resolve: use of undeclared type `MZStatus`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:78:17
   |
78 |                 MZStatus::Ok => Ok(Status::Ok),
   |                 ^^^^^^^^ use of undeclared type `MZStatus`

error[E0433]: failed to resolve: use of undeclared type `MZStatus`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:79:17
   |
79 |                 MZStatus::StreamEnd => Ok(Status::StreamEnd),
   |                 ^^^^^^^^ use of undeclared type `MZStatus`

error[E0433]: failed to resolve: use of undeclared type `MZStatus`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:80:17
   |
80 |                 MZStatus::NeedDict => {
   |                 ^^^^^^^^ use of undeclared type `MZStatus`

error[E0433]: failed to resolve: use of undeclared type `MZError`
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:85:17
   |
85 |                 MZError::Buf => Ok(Status::BufError),
   |                 ^^^^^^^ use of undeclared type `MZError`

error[E0433]: failed to resolve: use of undeclared type `MZFlush`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:148:21
    |
148 |         let flush = MZFlush::new(flush as i32).unwrap();
    |                     ^^^^^^^ use of undeclared type `MZFlush`

error[E0433]: failed to resolve: use of undeclared crate or module `deflate`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:149:19
    |
149 |         let res = deflate::stream::deflate(&mut self.inner, input, output, flush);
    |                   ^^^^^^^ use of undeclared crate or module `deflate`

error[E0433]: failed to resolve: use of undeclared type `MZStatus`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:155:17
    |
155 |                 MZStatus::Ok => Ok(Status::Ok),
    |                 ^^^^^^^^ use of undeclared type `MZStatus`

error[E0433]: failed to resolve: use of undeclared type `MZStatus`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:156:17
    |
156 |                 MZStatus::StreamEnd => Ok(Status::StreamEnd),
    |                 ^^^^^^^^ use of undeclared type `MZStatus`

error[E0433]: failed to resolve: use of undeclared type `MZStatus`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:157:17
    |
157 |                 MZStatus::NeedDict => mem::compress_failed(ErrorMessage),
    |                 ^^^^^^^^ use of undeclared type `MZStatus`

error[E0433]: failed to resolve: use of undeclared type `MZError`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:160:17
    |
160 |                 MZError::Buf => Ok(Status::BufError),
    |                 ^^^^^^^ use of undeclared type `MZError`

error[E0412]: cannot find type `DataFormat` in this scope
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:29:43
   |
29 | fn format_from_bool(zlib_header: bool) -> DataFormat {
   |                                           ^^^^^^^^^^ not found in this scope

error[E0412]: cannot find type `InflateState` in this scope
  --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:38:16
   |
37 | pub struct Inflate {
   |                   - help: you might be missing a type parameter: `<InflateState>`
38 |     inner: Box<InflateState>,
   |                ^^^^^^^^^^^^ not found in this scope

error[E0412]: cannot find type `CompressorOxide` in this scope
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:111:16
    |
110 | pub struct Deflate {
    |                   - help: you might be missing a type parameter: `<CompressorOxide>`
111 |     inner: Box<CompressorOxide>,
    |                ^^^^^^^^^^^^^^^ not found in this scope

error[E0412]: cannot find type `CompressorOxide` in this scope
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/ffi/rust.rs:131:28
    |
126 | impl DeflateBackend for Deflate {
    |     - help: you might be missing a type parameter: `<CompressorOxide>`
...
131 |         let mut inner: Box<CompressorOxide> = Box::default();
    |                            ^^^^^^^^^^^^^^^ not found in this scope

error[E0425]: cannot find value `MZ_DEFAULT_WINDOW_BITS` in module `ffi`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/mem.rs:202:59
    |
202 |             inner: Deflate::make(level, zlib_header, ffi::MZ_DEFAULT_WINDOW_BITS as u8),
    |                                                           ^^^^^^^^^^^^^^^^^^^^^^ not found in `ffi`

error[E0425]: cannot find value `MZ_DEFAULT_WINDOW_BITS` in module `ffi`
   --> /home/<user>/.local/share/cargo/registry/src/github.com-1ecc6299db9ec823/flate2-1.0.22/src/mem.rs:382:52
    |
382 |             inner: Inflate::make(zlib_header, ffi::MZ_DEFAULT_WINDOW_BITS as u8),
    |                                                    ^^^^^^^^^^^^^^^^^^^^^^ not found in `ffi`

    Checking rand_chacha v0.3.1
Some errors have detailed explanations: E0412, E0425, E0432, E0433.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `flate2` due to 29 previous errors
warning: build failed, waiting for other jobs to finish...
error: build failed

should we fix it instead?

Could make compression enable zlib-stock, but I prefer to be explicit. I figured this would be expected behavior since http's rustls feature flag was removed.


hex conversion

Hi, everything is looking good but I found a inconsistent behavior with .hex.  It drops the leading nibble(s) depending upon the value of a fractional number.
Technically this is not wrong if you read its value as a human would.  I am working with large fixed point numbers and push them through a 32-bit interface.  I was using .hex() to do that but learned the hard way that the length varies.  While not impossible to work around, I found it just easier to use .bin() which is consistent in the converted string length.
I am doing something like below to push the raw bits through a hardware interface between 2 processing systems, a cpu and fpga:
f = Fxp(float128(1)/float128(3), True, 256, 253)
s = f.bin()  # or .hex()
n = int(s[0:32], base=2)  # [0:4], base=16 if hex
# Push value n through a 32-bit hardware fifo, get next chunk
n = int(s[32:64], base=2)
# Push next value n through a 32-bit hardware fifo, get next chunk
#  ... do the same for the remaining 32-bit words

# on the other side pull them out of fifo and reassemble into the original fixed point value, then do processing

Hi Fred,
All of this problem with hex it is because is replicating hex python function.
In the last commit I update that method to include padding by default. The repo is updated, but I`m not going to do a release yet. So, you can pull the repo and install from sources.
An alternative option is to use the next workaround:
h = '0x{0:0{1}X}'.format(int(f.val), int(np.ceil(f.n_word / 4)))

That was I implemented to solve this problem.

Hi Franco,  Thanks for the quick response!  I'm using .bin() for my situation.  I will retest, I'm sure it is fixed, closing this.


Failed to apply variable substitution

I have a workflow that failed with "Failed to apply variable substitution".
Is there a way to understand why? Is it because I use "." in the variables?

Your link is dead, which makes it hard for anybody to help you. What type of file you were using, json, yaml, xml? What is it's structure, what are your environment variables named, etc.
Anyway, I found out while stumbling upon an issue myself, that apparently it is possible to have more detailed debug messages from this action like so: https://github.com/actions/toolkit/blob/main/docs/action-debugging.md
Also, if you have no choice, like I didn't, you can have a look at the action source code here:
https://github.com/microsoft/variable-substitution

It's been a while so I'll just close the issue - Sorry for the inconvenience.


Api Update

Environment information

ape and plugin versions:

$ ape --version
0.7.23

$ ape plugins list
Installed Plugins
  alchemy      0.7.3
  etherscan    0.7.4
  foundry      0.7.5
  optimism     0.7.3
  safe         0.7.0b2
  solidity     0.7.3

Python Version: 3.12
b- OS: osx/linux/win linux

What went wrong?
The safe api changed for fetching transactions, i believe v1/safes/{address}/transactions/ no longer works and is now
https://safe-transaction-optimism.safe.global/
/v1/safes/{address}/all-transactions/
https://github.com/ApeWorX/ape-safe/blob/eddc804f771b791d3891d3cfe6245edc98f90cf8/ape_safe/client/__init__.py#L84
Please include information like:

what command you ran

txn            = multisend.MultiSend()
txn(sender=self.safe,gas=0,submit_transaction=False,nonce=nonce)

Trying to push

the code that caused the failure (see this link for help with formatting code)
full output of the error you received

Traceback (most recent call last):
  File "/home/thesis/repos/gnosis_perp_ape/main.py", line 58, in <module>
    self.setup_gnosis_perp_parameter_change(args.p)
  File "/home/thesis/repos/gnosis_perp_ape/utils/controller.py", line 75, in setup_gnosis_perp_parameter_change
    self.post_gnosis_transaction(contractName='perp_settings',functionName=functionName,args=stagedParams)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/repos/gnosis_perp_ape/utils/gnosis_safe.py", line 24, in wrapper
    return func(self,*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/repos/gnosis_perp_ape/utils/gnosis_safe.py", line 64, in post_gnosis_transaction
    nonce          = self.get_next_safe_nonce()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/repos/gnosis_perp_ape/utils/gnosis_safe.py", line 24, in wrapper
    return func(self,*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/repos/gnosis_perp_ape/utils/gnosis_safe.py", line 60, in get_next_safe_nonce
    return self.safe.new_nonce
           ^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/miniconda3/envs/gnosis_perp_ape/lib/python3.12/site-packages/ape_safe/accounts.py", line 322, in new_nonce
    if latest_tx := next(self.client.get_transactions(confirmed=False), None):
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/miniconda3/envs/gnosis_perp_ape/lib/python3.12/site-packages/ape_safe/client/base.py", line 81, in get_transactions
    for txn in self._all_transactions():
  File "/home/thesis/miniconda3/envs/gnosis_perp_ape/lib/python3.12/site-packages/ape_safe/client/__init__.py", line 86, in _all_transactions
    response = self._get(url)
               ^^^^^^^^^^^^^^
  File "/home/thesis/miniconda3/envs/gnosis_perp_ape/lib/python3.12/site-packages/ape_safe/client/base.py", line 128, in _get
    return self._request("GET", url)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thesis/miniconda3/envs/gnosis_perp_ape/lib/python3.12/site-packages/ape_safe/client/base.py", line 153, in _request
    raise ClientResponseError(api_url, response)
ape_safe.exceptions.ClientResponseError: Exception when calling 'https://safe-transaction-optimism.safe.global/api/v1/safes/xxxxxxxxxxxxxxxxxxxxx/transactions/':

<!doctype html>

  <title>Not Found</title>

Not Found

The requested resource was not found on this server.

How can it be fixed?
Need to update the api

@solarthesis please check out https://github.com/ApeWorX/ape-safe/pull/47 and let me know if that solves your problem!


Raise the —target typescript compilation for ESM build

Objective is to lower the package size and increase performance at the same time.
If the package consumer supports ESM then the consumer should support a higher compilation target than the default of ES3.

CommonJS consumers should not be affected. CJS should build the same.

v3.3.0


Parameters failure in nested directory structure nuxt 2.14

This is an odd issue I got on one of my customers websites after an update from Nuxt 2.12 to Nuxt 2.14.
Parameters are no longer working correctly in deeply nested directory structures.
Versions

nuxt: v2.14.5
node: v12.18.3

Reproduction
https://codesandbox.io/s/cocky-rosalind-rfzy0?file=/pages/test/nest/_param1/test/_param2/_param3/_param4/index.vue
https://rfzy0.sse.codesandbox.io/test/nest/param1/test/param2/param3 <-- Failure
https://rfzy0.sse.codesandbox.io/test/nest/param1/test/param2/param3/param4 <-- works correctly when last param is not omitted
Steps to reproduce
Nested directory structure with parameters
What is Expected?
Parameters in the right place
What is actually happening?
When going to a deeply nested URL and omitting the last param, the next to last param gets moved to last param

Could it be related to https://github.com/nuxt/nuxt.js/pull/7843 ?

@manniL I think it's rather related to #7845

But I could be wrong

Seems more likely!
cc @farnabaz

I see, I believe despite that is works before, you were doing it in the wrong way. Let's assume you have pages structure like the image

According to official docs param4 will be required and other params (params1..3) will be optional, the generated route will be like /test/nest/:param1?/test/:param2?/:param3?/param4
Ok, now you are going to visit /test/nest/1/test/2/3 path in the browser. Nuxt router expects that you provide a value for the param4 (because it is required) and it picks 3 as the value of param4.
In case you want to catch urls with missing param4 consider creating an index inside param3 directory. Take a look at https://codesandbox.io/s/params-failure-forked-1d8se?file=/pages/test/nest/_param1/test/_param2/_param3/index.vue

It worked perfectly fine in 2.12, so it was unexpected when the website stopped working properly in 2.14.
In the real-world application the last parameter is an optional date. I'd rather not make a duplicate index file for this purpose, so how would I solve this otherwise?

It worked perfectly fine in 2.12, so it was unexpected when the website stopped working properly in 2.14.

Yes, I agree with you. Your project was working perfect with 2.12 because of a bug inside route generation system. A bug sometimes may become usefull, in your case you were lucky back then that uses this bug to achieve your goal 🍀
But the docs are our source of truth in this matter.

In the real-world application the last parameter is an optional date. I'd rather not make a duplicate index file for this purpose, so how would I solve this otherwise?

You could extract params4/index into a component and use that component in both param4/index and param3/index.
Or you can possible ways to extends nuxt router.


docs: Failed to initialize NVML: Driver/library version mismatch

URL(s) with the issue:
https://www.tensorflow.org/install/gpu#install_cuda_with_apt
Description of issue (what needs changing):
I'm following the Ubuntu 18.04 (CUDA 10.1) instructions. I installed the NVIDIA package repositories & NVIDIA driver 450 and rebooted, then ran nvidia-smi:
$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           On   | 00000000:00:1E.0 Off |                    0 |
| N/A   62C    P0    63W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I then installed cuda-10-1, libcudnn7=<IP_ADDRESS>-1+cuda10.1, libcudnn7-dev=<IP_ADDRESS>-1+cuda10.1 as per the instructions. Now I get:
$ nvidia-smi
Failed to initialize NVML: Driver/library version mismatch

I'm running Ubuntu 18.04.5 LTS (Bionic Beaver)

I've rebooted after installing cuda & libcudnn, and now it works:
$ nvidia-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.32.00    Driver Version: 455.32.00    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           On   | 00000000:00:1E.0 Off |                    0 |
| N/A   24C    P8    29W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

It looks like the nvidia driver version was upgraded to 455.
Would it be worth updating the docs to move the reboot to the end of the instructions?

Actually, a better solution might be to update the driver instructions to install 455 instead of 450, eg:
sudo apt-get install --no-install-recommends nvidia-driver-455

Because cuda-10-1 installs cuda-drivers-455, which installs nvidia-driver-455

Because cuda-10-1 installs cuda-drivers-455, which installs nvidia-driver-455, perhaps a better solution is to remove this line from the instructions:
sudo apt-get install --no-install-recommends nvidia-driver-450

@tekumara
Please, see this SO link and see if it helps you. Is this issue related to Tensorflow?
Thanks!

It's working for me now but I believe the docs could be improved as mentioned above.

@tekumara
The requested change has been implemented in this PR#1749.Please, verify once and close the issue. Thanks!


NER - “input sequence after bert tokenization shouldn’t exceed 512 tokens” (ner_bert_base)

DeepPavlov version:
The latest docker container deeppavlov/deeppavlov, published last month
Python version:
3.10
Operating system:
The latest docker container deeppavlov/deeppavlov, published last month.
Docker is running on CentOS/AlmaLinux.
Issue:
I’m looking to understand how to prevent this crash from happening.

input sequence after bert tokenization shouldn’t exceed 512 tokens.

I’m using the REST API, so I’m calling ner_bert_base like this:
{
  "x": [
    "A huge text. Blah blah blah... No line breaks. I'm a 28 year-old person called John Smith, etc..."
  ]
}

While researching about this error, I found: https://github.com/deeppavlov/DeepPavlov/issues/839#issuecomment-492209665
which says:

Sorry, but the BERT model has positional embeddings only for first 512 subtokens. So, the model can’t work with longer sequences. It is a deliberate architecture restriction. Subtokens are produced by WordPiece tokenizer (BPE). 512 subtokens correspond approximately to 300-350 regular tokens for multilingual model. Make sure that you performed sentence tokenization before dumping the data. Every sentence in the dumped data should be separated by an empty line.

But I don’t fully understand what I need to do to resolve the problem.
What does “Make sure that you performed sentence tokenization before dumping the data” mean? Is it some function I need to call first, that returns the list of tokens? Is it something that I can call with the REST API from my application/code?
I was also looking to see if I could have my application (the caller) to somehow tokenize the words and punctuation, and then only send the first 512, but the thing is that it’s hard to preserve the spacing, and even if I send 512, it somehow still passes that limit in the model, crashing anyway.
I feel like I’m trying to reinvent the wheel.
Can’t we have the API and/or the model just silently (or by setting a flag/parameter in the input) truncate the text input past 512 tokens?
(Note that my application is not made in Python)
Thank you very much!

This actually resolves my issues: https://github.com/deeppavlov/DeepPavlov/pull/1657/files#diff-c2feefe4ebd288d44761cad4fbe6c29d43da997a00c597f6281e89ceca3a57d2


Flutter Web: Authentication Post becomes Options

Hey out there,
first of all, thank you for sharing this amazing work. I just fell in love with flutter and in order to do some full stack dev Angel is just awesome.
I recently tried to run my Flutter Client App on a web server. The app is a quiet simple authentication example. I found out that, while it works perfectly fine on Android, the Angel Client sends OPTION posts instead of "normal" POSTs when run on Flutter Web.
Is this intended behaviour? I tested this for both implementations ('browser.dart' & 'flutter.dart')
I know Flutter Web is still not release perfect yet but it would be awesome to provide support for it right away.
If you need any information i'd be glad to help out.
Thanks to all contributors for what you achieved so far :+1:

Alright, i am kind of a newbie when it comes to this.
The problem seems to be caused by CORS policy. Can someone provide a simple example on how to handle this problem?

I just found out that i had a really big lack of knowledge about CORS. In case anyone else occurs this error you can simply add teh angel_cors dependency add these lines to your server and you should be good to go.
import 'package:angel_cors/angel_cors.dart';
...
app.options('/*', cors());
app.all('/*', cors(CorsOptions(origin: "*")));

If someone else has a more secure way of achieving the same i'd be glad to see it.


runtime/metrics: /memory/classes/heap/unused:bytes spikes

Go version
go1.22.1
Output of go env in your module/workspace:
N/A - many different prod environments

What did you do?
We're trying to build nice dashboards to expose runtime/metrics to all Datadog users. For this reason we started rolling out a package that collects these metrics across our fleet.
What did you see happen?
While graphing the data, we noticed occasional spikes in the memory metrics. Upon closer inspection, we discovered that all of these spikes were caused by the /memory/classes/heap/unused:bytes metric.

These spikes are pretty rare (e.g. 18 spikes per day in the last 24h for a very large fleet), but frequent enough to cause problems with building nice dashboards. The issue occurs across architectures (arm64, amd64), instance types, and hyperscalers without any clear pattern.
We suspect the large values are the result of an underflow in the runtime/metrics code:
out.scalar = uint64(in.heapStats.inHeap) - in.heapStats.inObjects

To investigate further we started logging the values (we internally use float64 for storage). Below are a few values we logged and their distance from math.MaxUint64 (assuming it's indeed an underflow we're seeing here).

value
math.MaxUint64 - value

18446744073337960688
371652608

18446744073325609896
383950848

18446744073455472696
254150656

18446744073690885408
18751488

18446744073702264952
7352320

We also logged the values of all KindUint64 runtime metrics that were collected as part of the same metrics.Read() call. I've dumped the results into this sheet (apologies for the formatting)
What did you expect to see?
No spikes.
cc @mknyszek

Hm... those differences are so large (300+ MiB??) that it doesn't seem like a situation where we're transiently racy in some corner case (as is the usual case for accounting bugs). It's a bit difficult to explain this.
This is a shot in the dark, but is it possible these services are using GOEXPERIMENT=arenas? That's maybe the one area where I could see both (1) skew this large and (2) accounting bugs still lurk, because it's used less widely.

I've checked and think I can rule out arena usage. There are no search hits for arena imports in our mono-repo, and we're seeing this issue across most of our services.
Any other ideas for things we could do to help debug this? Do you have a chance to see if the issue happens with Google's fleet? It's possible we're doing something weird, but I'm not sure what 🤔 . This is the only memory metric that we see behave oddly like this.

I think I found the bug. The span can be freed (and the "bytes in heap" stats can be updated) before we update the object stats for large objects on sweep. The subtraction assumes that object stats are always handled before the "bytes in heap" stat.

@gopherbot Please open backport issues for Go 1.21 and Go 1.22.
This issue can cause spurious overflow in runtime metrics with no workaround.

Reproducer:
package main

import (
	"runtime/metrics"
)

var sink any

func main() {
	go func() {
		s := []metrics.Sample{
			{Name: "/memory/classes/heap/unused:bytes"},
		}
		for {
			metrics.Read(s)
			val := s[0].Value.Uint64()
			if val > 1<<40 {
				panic("overflow")
			}
		}
	}()
	for {
		sink = make([]byte, 10<<20)
		sink = nil
	}
}


[Bug]: Zone side margins are harder to access when it’s empty and user is trying to create a new zone

Is there an existing issue for this?

[x] I have searched the existing issues

Description
Zone side margins are harder to access when it’s empty and user is trying to create a new zone
Steps To Reproduce

Interact with borders on an empty zone on the anvil DP (https://ce-28238.dp.appsmith.com/ currently)

Public Sample App
No response
Environment
Deploy Preview
Severity
Low (Inconsistent UI/UX)
Issue video log
https://www.loom.com/share/08a43a14937f47adabb6295940eec82c
Version
cloud

The issue also exists on zones that have widgets
https://theappsmith.slack.com/archives/C025SE88KNE/p1711975980229339


Fix BPM <-> MIDI tempo conversions

This PR addresses #102 and contains the fixes needed to correctly consider the time signature when converting between BPM (beats per minute) and MIDI tempo (microseconds per quarter note).
Before being able to be merged, these points should be discussed/addressed:

[ ] ticks_per_beat is a bit misleading since it rather should be ticks_per_quarter_note. Renaming the variable, however, may break existing code, so this is a rather bad option. Maybe adding a new variable (and deprecate the old one, but keep it around for backwards compatibility) is a better approach.
[ ] Tests for Python 2 fail at the moment since Python 2 and 3 round differently. This was revealed by adding tests for corner cases.
[ ] Extend the documentation to reflect all introduced changes, especially update the image in midi_files.rst.
[ ]  Add more descriptive docstrings, but which docstring format should be used?

Status?

I have added the fix I was suggesting in #454 and merged your changes.
Thank you for your contribution!

Thanks a lot!


new contributors are now pointed to correct issue label

I've changed the contributing.md to point new contributors to issues labeled 'easy' since the previous labels are not used anymore (and did not exist in the first place)
@bkase !ci-build-me can be run

This change landed in a different PR. Closing.


Swap Example

I am not able to get a swap working. The transaction will be sent successfully, but there is no exchange. Is there an error in my code?
const { publicKey, secretKey } = await mnemonicToWalletKey(
    mnemonic.split(' ')
  );

  const provider = new TonWeb.HttpProvider(
    'https://toncenter.com/api/v2/jsonRPC',
    {
      apiKey:
        '...',
    }
  );
  const tonweb = new TonWeb(provider);

  const Wallet = tonweb.wallet.all.v4R2;
  const wallet = new Wallet(provider, {
    publicKey,
  });
  const address = await wallet.getAddress();
  const nonBouncableAddress = address.toString(true, true, false);
  const seqno = (await wallet.methods.seqno().call()) || 0;

  const stonfiRouter = new Router(provider, {
    revision: ROUTER_REVISION.V1,
    address: ROUTER_REVISION_ADDRESS.V1,
  });

  const tonToJettonTxParams = await stonfiRouter.buildSwapProxyTonTxParams({
    userWalletAddress: nonBouncableAddress,
    proxyTonAddress: 'EQCM3B12QK1e4yZSf8GtBRT0aLMNyEsBc_DhVfRRtOEffLez',
    offerAmount: new TonWeb.utils.BN('100000000'),
    askJettonAddress: 'EQA2kCVNwVsil2EM2mB0SkXytxCqQjS4mttjDpnXmwG9T6bO', //STON
    minAskAmount: new TonWeb.utils.BN(1),
    queryId: 12345,
    referralAddress: undefined,
  });

  const result = await wallet.methods
    .transfer({
      secretKey,
      toAddress: ROUTER_REVISION_ADDRESS.V1,
      amount: tonToJettonTxParams.gasAmount,
      seqno: seqno,
      payload: tonToJettonTxParams.payload,
    })
    .send();

console.log(result);

/*
{
  "@type": "ok",
  "@extra": "1714167550.242306:8:0.9612518955091519",
}
*/

So the result looks okay to me, but I get TON sent back by the router:

Any idea?

Fixed: Forgot to use sendMode


Bookmarker_application

Describe the project
Tech stack : Html, CSS and JavaScript and little bit of bootstrap for styling.
Issue Type:

[ ] Bug Report
[ ] Feature Request
[ ] Documentation
[ ✔] New Project

Additional context
Beginner to medium level web developer friendly project.

Pls assign me this issue @TusharKesarwani

Go for it @Amit0412 🚀


Added CORS support to the v2 Content API

no-issue
When trying to use /api/v2/content from a different domain, the requests
were failing with CORS errors. This doesn't use the shared cors middleware,
because it should be open to all hosts, and not locked down via our
whitelist or trusted domains.

@allouis See approval message 👍

As soon as an integration wants to speak to the admin v2 api, it would throw a cors error too i think?

This is true - however we do not want the admin integration keys in the client at all - so we deffo don't wanna support CORS for them.
Any requests made via cURL or request or whatever serverside aren't subject to CORS checks, so they're be fine :relaxed:

Yeah okay, let's leave the referenced line as is for now. Thanks 👍


Add LICENSE to releases

Is your feature request related to a problem? Please describe.
N/A
Describe the solution you'd like
See title
Describe alternatives you've considered
No response
Additional context
No response

Completed this in PR68


Beefy relay compatibility

This PR adds support for running a relay chain with the beefy pallet support. All e2e tests still pass.
Related:

Using polkadot-launch from: https://github.com/paritytech/polkadot-launch/pull/62
Using polkadot from: https://github.com/paritytech/polkadot/pull/2101

We may not want to merge this to main yet, as eventually the above 2 PRs will be part of polkadot-launch/polkadot master, but I do think the team and staging and all future PRs should base themselves off this and work to maintain compatibility with this.
@denalimarsh @philipstanislaus This PR also includes a node script (subscribeBeefyJustifications), which, if you run it after everything is setup, will connect to the appropriate RPC endpoints on the relay chain and parachain and subscribe to all the data we'll need to be relayed into the Solidity light client. This script should help make it easy to integrate all the endpoints and relaying of all data types into our Golang relayer. It includes:
1 - Getting initial list of authorities (note: currently a small bug, so hardcoded)
2 - Getting each new beefy commitment produced and the collection of signatures on that commitment (which we need for the Relay Chain Light Client)
3 - Getting the latest new MMR block that the beefy commitment commits to, which we need to process the commitment for validator set updates/epoch changes/new parachain headers and the proof for that MMR
4 (WIP/TODO) - Getting the parachain heads commitment in that MMR, a merkle proof of our parachain's new head within parachain heads, and the new head data itself
Note: I had a minor issue building the suggested branch of Polkadot with an error related to polkadot-node-primitives. If you encounter this, you can fix it by deleting the redundant node-primitives line from node/network/statement-distribution/Cargo.toml

included in https://github.com/Snowfork/polkadot-ethereum/pull/343


Centralize *gen_problem functions. Parameterize graph type.

karate and singlegraph have parts in common.

Idea: instead of centralize all, step one, end up with two, one gen_problem for multigraph and another one for singlegraph.


Low memory option

Find a way to do format conversion with lower memory.
Ideas:

Try to find if someone else has a more efficient conversion somewhere else in the world
Use BerkeleyDB to take advantage of disk instead of memory

CC @mbhall88

I mean it should be possible to convert a square matrix to phylip without keeping both matrices in memory right? Read the first line, which is sample (taxon) names, then each line after that is essentially a row in the phylip right?

I think you're right. I can add a special case when converting matrix to tsv so that it only has to read one line at a time. However, that would require maybe two passes: 1) find the number of taxa because phylip requires that count at the top and also to make sure that the row and column order matches and then 2) copy over the values. I'll see what I can cook up.

I haven't tested it on large files yet but I did the two pass method. It makes a compressed temporary file in TMPDIR that contains all the phylip, except the numsamples field. Then it streams correct phylip to stdout including the numsamples.

Can you not get the number of taxa from the first line in the matrix?

Can you not get the number of taxa from the first line in the matrix?

Dangit why didn't I think of that


bug: Vimscript function must not be called in a fast event context

Did you check docs and existing issues?

[X] I have read all the noice.nvim docs
[X] I have updated the plugin to the latest version before submitting this issue
[X] I have searched the existing issues of noice.nvim
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
NVIM v0.11.0-dev-1161+g6ef80eb42c
Operating system/version
Arch Linux 6.11.7
Describe the bug
Run :echo "test" or any other command which causes Noice to print a message.
An error happened while handling a ui event
....local/share/nvim/lazy/nui.nvim/lua/nui/text/init.lua:31: E5560: Vimscript function must not be called in a fast event context

stack traceback:
	...local/share/nvim/lazy/noice.nvim/lua/noice/util/call.lua:144: in function <...local/share/nvim/lazy/noice.nvim/lua/noice/util/call.lua:143>
	[C]: in function 'strlen'
	....local/share/nvim/lazy/nui.nvim/lua/nui/text/init.lua:31: in function 'set'
	....local/share/nvim/lazy/nui.nvim/lua/nui/text/init.lua:18: in function 'init'
	.../.local/share/nvim/lazy/nui.nvim/lua/nui/object/init.lua:132: in function 'NuiText'
	....local/share/nvim/lazy/nui.nvim/lua/nui/line/init.lua:20: in function '_append'
	...ocal/share/nvim/lazy/noice.nvim/lua/noice/text/block.lua:186: in function 'append'
	...ocal/share/nvim/lazy/noice.nvim/lua/noice/text/block.lua:129: in function 'set'
	...y/.local/share/nvim/lazy/noice.nvim/lua/noice/ui/msg.lua:108: in function 'handler'
	.../.local/share/nvim/lazy/noice.nvim/lua/noice/ui/init.lua:190: in function <.../.local/share/nvim/lazy/noice.nvim/lua/noice/ui/init.lua:189>
	[C]: in function 'xpcall'
	...local/share/nvim/lazy/noice.nvim/lua/noice/util/call.lua:149: in function 'safe_handle'
	.../.local/share/nvim/lazy/noice.nvim/lua/noice/ui/init.lua:95: in function <.../.local/share/nvim/lazy/noice.nvim/lua/noice/ui/init.lua:77>
	[C]: in function 'nvim_out_write'
	vim/_editor.lua: in function 'print'
	[string ":lua"]:1: in main chunk

This issue started happening as of https://github.com/neovim/neovim/commit/de48fbbd5f8800bd7f1909a6fb41e53e871cf74c (https://github.com/neovim/neovim/pull/27874), which changes vim.ui_attach message callbacks to execute in fast context.

Should be fixed now.
msg_show events are no longer processed realtime, so this might have other side-effects.
Let me know if you notice anything else broken.
Thank you for reporting!

Thanks for the quick fix! One issue now is if you try searching with / for a pattern which doesn't exist in the buffer, the "not found" message isn't printed and the search bar gets stuck until you do something to trigger a redraw.

Also a similar thing happens with a mapping like :confirm qall with unsaved changes.

Yeah indeed. Some things are broken now...
Not sure how I can fix that confirm message.
The searching thing works for me with Snacks.notifier.
nvim-notify has some issues with redraw.

Opened a tracking issue. Feel free to add anything else you find is broken...


Removing a table doesn't remove it from the section headers

Removed participation table from ModuleDefinition, but its section selector box still exists in the left panel.

Seems to have resolved itself. I can't replicate the issue. Will return if it occurs again.