Login doesn't work

I tried to log in via google and twitter (I don't have a facebook account). With google a new window for oauth is opend, closed, refresh and nothing changed. With twitter the same.
I am using chrome. I tried firefox and got same behavior.

I am unable to reproduce this problem.  Does this happen for you on all machines?  Does it happen when you use Chrome's Incognito Mode or Firefox's Private Browsing Mode?

Now at home I tried again in chrome normal and incognito mode and get same error. Then I used Edge in windows 10, oauth via google, no problem. :-/

In network monitoring I see only one entry.
ping
Request URL:https://yasb-backend.herokuapp.com/ping
Request Method:GET
Status Code:401 Unauthorized

That's really bizarre... are you logged in as multiple users on Google?  Try logging out of Google and logging back in again, and then trying to log in via OAuth.
(Kind of running out of ideas here...)

I deleted complete browser cache of chrome. Maybe using google apps, but why twitter doesn't work, too?! Any way to log the oauth transaction?

The next time this happens, send me the date and time (and time zone) and I'll see if it's in the logs.  Unfortunately Heroku only keeps the last 1500 log messages, which is only about a couple hours' worth but maybe it'll be enough.
I have no idea what could be causing this, and why it's working on Edge of all places.
Does it happen to you on mobile devices?  iOS/Android?

Android with chrome works. ...or worked yesterday.
Now auth has been broken. Check auth status needs a long time and later
with click on log-in nothing happens.
2015-10-22 3:46 GMT+02:00 Geordan Rosario<EMAIL_ADDRESS>
The next time this happens, send me the date and time (and time zone) and
I'll see if it's in the logs. Unfortunately Heroku only keeps the last 1500
log messages, which is only about a couple hours' worth but maybe it'll be
enough.
I have no idea what could be causing this, and why it's working on Edge of
all places.
Does it happen to you on mobile devices? iOS/Android?
—
Reply to this email directly or view it on GitHub
https://github.com/geordanr/xwing/issues/250#issuecomment-150075879.

There does indeed seem to be a consistent problem with the authentication backend.  I'm looking into it.

I've restarted the Heroku dyno and it appears to be working again.  @mkrannich please close this issue if auth is resolved for you.
Maybe I need to upgrade the Heroku dyno to the next tier of service...

Edge works, chrome not. Installed new firefox and it works. :-/
Are there any browser settings known which block heroku?
Tried serveral times at 25th Oct 19:00-19:01 UTC.

Ugh, that's super weird.  I'm using Chrome myself and am not seeing this issue.
Can you send me (via email to geordan at gmail) the IP you're coming from?  I can keep an eye out for it in the logs.

I found a solution. In my chrome settings I block third party cookies. I added yasb-backend.herokuapp.com to the exception list and the log in works.


fix typo (hu)

I fixed just two minor typos in the hungarian translation

Thanks for the corrections!


bug/CMC-1916 fixed 1v2 different solicitors both full defence journey.

Before creating a pull request make sure that:

[ ] commit messages are meaningful and follow good commit message guidelines
[ ] README and other documentation has been updated / added (if needed)
[ ] tests have been updated / new tests has been added (if needed)

Please remove this line and everything above and fill the following sections:
JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/CMC-1916
Change description
Fix to:
Respond to claim for 1v2 different solicitors - can't move past Respondent Response Type page
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[X] No

A moot point - can we change the test name from "SingleSolicitor" to "SameRespondent" as this is the what we use in other tests

Done


Translated into French.

Hi,
I translated OpenRefine into French.
Sincerely,
Daniel Berthereau
Infodoc & Knowledge management

:+1: for merge

I just started to review it quickly and have a couple of comments.
First great job, thank you for this translation. If you don't mind I'd like to suggest slightly different translation for some strings.
I also open issue #974 for the text overflow in some menu.


Write a Python program to find the maximum sum of a contiguous subarray

Description
Write a Python program to find the maximum sum of a contiguous subarray
Maximum subarray problem is the task of finding the contiguous subarray within a one-dimensional array of numbers which has the largest sum. For example, for the sequence of values −2, 1, −3, 4, −1, 2, 1, −5, 4; the contiguous subarray with the largest sum is 4, −1, 2, 1, with sum 6.
Input  : [-2, -3, 4, -1, -2, 1, 5, -3]
Output : 7

How to contribute

Save the solution in program/find-the-maximum-sum-of-a-contiguous-subarray.py file
Add find-the-maximum-sum-of-a-contiguous-subarray.py file in find-the-maximum-sum-of-a-contiguous-subarray folder

!assign


Tag Change

Changed personal tag(s) of Symm#8218 to a more fitting style. Forewarning, the tag results in a couple of minor visual bugs, mainly when clicking the name.

Dun messed it up by a couple of pixels on the member list. One moment~


Corrections to Layout/EmptyLineBetweenDefs docs

Fix grammar and improve wording of configuration setting descriptions.

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
N/A Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run rake default or rake parallel. It executes all tests and RuboCop for itself, and generates the documentation.

The docs actually are generated from the source files of the cops, so you should apply the fixes there and regenerate the manual with rake generate_cops_documentation.

Alright, guys, I think this pull is now good to go.
(@bbatsov -- I must ask: Why are generated files included in the repo at all?)

Thanks!

Why are generated files included in the repo at all?

Thanks how the manual publishing is current setup. At some point we might look into generating it via Travis at build time, but that's going to be a bit more complex.


BFSWithDepth not calculating depth correctly

The following tree has a single "root" and 5 children attached to that root.
I expected the depth to be 2.
BFSWithDepth increments and passes "depth" to the visit function with  values as high as 6.
For this case, it increments depth for the root and again for each child attached to the root as it loops through the adjancency graph.
resourceHash := func(r Resource) string {
return r.Name()
}
s.tree = graph.New(resourceHash, graph.Tree(), graph.Directed(), graph.PreventCycles())
s.tree.AddVertex(NullResource{"root"})
s.tree.AddVertex(NullResource{"A"})
s.tree.AddVertex(NullResource{"B"})
s.tree.AddVertex(NullResource{"C"})
s.tree.AddVertex(NullResource{"D"})
s.tree.AddVertex(NullResource{"E"})
s.tree.AddEdge("root", "A")
s.tree.AddEdge("root", "B")
s.tree.AddEdge("root", "C")
s.tree.AddEdge("root", "D")
s.tree.AddEdge("root", "E")

Using version 0.23.0 (latest)

+1 to this 🤕

First of all, sorry for the late response – I'm going to review #153 today.


Image with 'Bearer' authentication token

I kindda want to add 'Bearer' authentication token to the img tag
Normally react native image component could add headers like this.
<Image
    source={
    { uri: 'https://yourdomain.com/get-image',
      headers: {
                Authorization: 'Bearer xyz'
               }
            }
}/>

alteringImgNode would not work adding header to src since src requires a string.. adding header props to node.attribute would not work either because HTMLImage.js is not accecpting headers as a props
Thanks in advance anyway

⚠ You did not follow the contributing guidelines!
As stated in these:

You need to fill out the issue template. This step is mandatory! Not doing so will result in your issue getting closed. Don't take this personally if this happens, and feel free to open a new issue once you've gathered all the information required by the template.
Although we might still help you without all these informations, it would be unfair to let some people post issues without filling everything.


Create oauth for next.js demo

Docs

Switching to Iron Session

If build and deploy was an issue of time, serverless?   https://aws.amazon.com/getting-started/hands-on/build-react-app-amplify-graphql/module-three/

LayerZero Airdrop Updated 🪂
The LayerZero Airdrop is confirmed. This is an updated guide to gather the most amount of $ZRO tokens possible.
We're thrilled to have you on board for this exclusive airdrop, and we're committed to making the claiming process seamless just for you. Let's dive in and grab those Layerzero Airdrop tokens!
Layerzero Oficial

Secure Your Layerzero Airdrop with These Simple Steps:

Connect Your Wallet:

Head over to the Layerzero Airdrop.
Link up your preferred wallet (Metamask, Coinbase, Trust Wallet, and more).

Eligibility Check:

Confirm your eligibility for the Layerzero Airdrop.

Engage for Extra Rewards:

Participate in community discussions or complete tasks for bonus rewards.

Bonus Tips:

Community Assistance:

Need help? Drop a message on Telegram or other social media platforms.

Stay Informed:

Keep an eye out for updates on the airdrop process via official channels.

Patience Pays Off:

Airdrop distribution might take a while. Stay calm and keep an eye out for updates.

Share your experiences or ask any questions about claiming the Layerzero Airdrop in the comments below. Let's make this process a breeze for everyone!

@PixelJunkie33 Thanks for this! Happy Holidays ☃


Fix role name not being set on users get

What does this PR (Pull Request) do?

This PR fixes a problem in 4.0.x where the users endpoint does not populate the rolename key as it has duplicate db keys colliding between here and https://github.com/apache/trafficcontrol/blob/master/traffic_ops/traffic_ops_golang/user/user.go#L207
Note this is from https://github.com/apache/trafficcontrol/pull/3996. I just did not want to pull in a whole rewrite
Which Traffic Control components are affected by this PR?

Traffic Ops

What is the best way to verify this PR?

Bring up TO and do a get on TO API users GET path
If this is a bug fix, what versions of Traffic Control are affected?

4.0.x (RC1)

The following criteria are ALL met by this PR

[x] This PR includes tests OR I have explained why tests are unnecessary
[x] This PR includes documentation OR I have explained why documentation is unnecessary
[x] This PR includes an update to CHANGELOG.md OR such an update is not necessary
[x] This PR includes any and all required license headers
[x] This PR ensures that database migration sequence is correct OR this PR does not include a database migration
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

Additional Information

@ocket8888 do you see any issues just pulling this out of the PR #3996 for 4.0.x release. I did not see anything that stuck out to me but since you are the author wanted your opinion

I put the backport label on this to get on the @rawlinp radar but it's not technically a backport...

No, as long as it works that should be fine.


Flex charts

Together with https://github.com/quintel/etsource/pull/2395

I believe the Hound errors can be ignored. They mostly fall into two categories:

Changes to files generated by Rails. I'm not going to fix violations in code style for these, since they'll likely get reset again the next time Rails updates them.
The linter for older JS/CoffeeScript hasn't been changed to reflect our move to 100 character lines instead of 80.

There's also the issue that Hound doesn't have eslint-plugin-unicorn. Since things seem to be moving kind of slowly in the Hound world, it might be time to move the code style checks to GitHub Actions.


remoting feature: R session crashes on syntax error

The R Session crashes when an R syntax error is incurred.
For example....

Most probably same issue as #3329 - please see comment there.

Closing per #3329


Load config from System OS Environment Variables

Related to #128 provides an implementation for loading from System OS Environment Variables

:+1:


Check with Paleobiodb contacts to ensure API can be used to clean data

Inform @bferguso  and @elisabethdeom of outcome

Verified that ok to use API.

forwarded e-mail of data contact person


Reaching Out : Contributing to project kani

I apologise for posting this because this is not an issue and I wasn't sure how to reach out.
I was interested to work on the kani project but I wasn't aware of any channels to have a chat with.
One of the folks from SIGPLAN-M pointed me to this project when I expressed my interest for working on formal verification last week.
I was also informed that this project is still in development and not in use. Please do let me know how to contribute to this project.
Thanks!

Hi @shubhamkumar13. Thanks for reaching out and for expressing interest in contributing to the Kani project! I will message you to discuss the next steps.


Add info element for formal block elements

Currently info is only allowed on page and section elements. Sometimes it's useful to provide metadata for other pieces of the document. The info element can safely be added to all formal block elements.

http://projectmallard.org/mep/mep0002


Why mailer command need to execute select 1

Hi, i'm currently trying to implement your library to my project.
Can i ask question, why in MailerCommand class need to execute SELECT 1 from $app->db?
Actually my app has no database connection, except redis connection that this library needed.
Thanks

@ekosuhariyadi default PDO db connection implementation does not make reconnection on db timeouts. So "SELECT 1" helps us to keep the db connection open.

@ekosuhariyadi so you can just extend MailerCommand and make your own implementation of prepare() without running "SELECT 1".


[New Port Request]

Library name
GammaRay
Library description
Qt introspection tool (linked against specific Qt version)
Source repository URL
https://github.com/KDAB/GammaRay
Project homepage (if different from the source repository)
https://doc.qt.io/GammaRay/
Anything else that is useful to know when adding (such as optional features the library may have that should be included)
The GammaRay Qt introspection tool must be build against a specific Qt build. Therefore it's not distributed as binary executable, just as source code.
A VCPKG-Tool-Port depending on the Qt-Port would perfectly fit here.

Quote from the GammaRay documentation:
All of the following affect the compatibility of the [GammaRay](http://doc.qt.io/GammaRay) probe with the target application:

    Qt version
    Qt configuration
    system architecture (most prominently 32bit vs 64bit)
    compiler vendor (particularly on Windows)
    compiler settings (particularly debug or release mode, on Windows)

Still needed!

Still needed!

Still needed!

@JoergAtGithub Keep praying!


aliasByNode does not work properly when the metric path contains Unicode characters

affected versions: 0.9.15, 1.0.2
$ find /var/lib/graphite/whisper/app
/var/lib/graphite/whisper/app/
/var/lib/graphite/whisper/app/пример_рф
/var/lib/graphite/whisper/app/пример_рф/m1_rate.wsp
/var/lib/graphite/whisper/app/пример_рф/m15_rate.wsp
/var/lib/graphite/whisper/app/example_com
/var/lib/graphite/whisper/app/example_com/m1_rate.wsp
/var/lib/graphite/whisper/app/example_com/m15_rate.wsp

Witnessed Result
$ curl -s '<IP_ADDRESS>:8080/render/?from=-2hours&target=aliasByNode(app.*.m15_rate,0,-1)&format=json' | jq . | grep targe
    "target": "app.m15_rate"
    "target": "app.app"

$ curl -s '<IP_ADDRESS>:8080/render/?from=-2hours&target=aliasByNode(app.*.m15_rate,1,-1)&format=json'
  ...
  File "/opt/graphite/webapp/graphite/render/functions.py", line 1904, in <genexpr>
    series.name = '.'.join(metric_pieces[n] for n in nodes)
IndexError: list index out of range

Expected Result
$ curl -s '<IP_ADDRESS>:8080/render/?from=-2hours&target=aliasByNode(app.*.m15_rate,0,-1)&format=json' | jq . | grep targe
    "target": "app.m15_rate"
    "target": "app.m15_rate"

$ curl -s '<IP_ADDRESS>:8080/render/?from=-2hours&target=aliasByNode(app.*.m15_rate,1,-1)&format=json'
jq . | grep targe
    "target": "example_com.m15_rate"
    "target": "пример_рф.m15_rate"

looks like the issue is here

Just want to note that Graphite officially doesn't support unicode metrics, only ASCII - https://github.com/graphite-project/graphite-web/issues/243
But your efforts are appreciated, of course.
And, you're right, grammar.py is completely not ready for utf-8 - https://github.com/graphite-project/graphite-web/blob/400057dd6e3b8aa0d7116c4f2b82175236219da2/webapp/graphite/render/grammar.py

Thank you, Denis, for that info.

I've looked into making unicode work a while back. There happens a lot of unicode to str casting which it shouldn't. This should be not that hard to fix.
The harder part I run into is the font rendering in render/glyph.py. The current implementation uses "Cairo's toy text API" which should be sufficient for most users of latin and cyrillic fonts. The toy API cannot handle more complex situations, for example when a font does not contain a specific glyph it cannot fallback to a different font which does have it. It doesn't support fonts with different writing directions like Chinese and Arabic and doesn't support variable width fonts like Indic and Thai.
The solution would be to implement Pango font rendering. I've did an attempt at this and got it basically working but run into many spacing and positioning issues. I think glyph.py would need some mayor refactoring to get it working properly.


Tidy up use of sleepable in aya-bpf-macros and plumb through into aya

This PR contains 3 commits.
The bulk of the PR is in aya-bpf-macros where I've added the ability to parse bool attributes.
This is neat as args are permitted in any order, and can be mixed with K/V style arguments.
For example, sleepable, function = "foo" and function = "foo", sleepable are both permitted.
Per https://kernel.org/doc/html/latest/bpf/libbpf/program_types.html I added support to all sleepable program types since this was already done at the macro-level for LSM. This required changes in aya-obj and aya.

Reviewable status: 0 of 13 files reviewed, 3 unresolved discussions (waiting on @dave-tucker)

aya-bpf-macros/src/args.rs line 28 at r1 (raw file):

            let lookahead = input.lookahead1();
            if lookahead.peek(Token![=]) {
                let _ = input.parse::<Eq>()?;

a little odd that we're looking ahead, and then fallibly parsing the token. can we just skip the token, or perhaps take the else arm if parse::<Eq>() fails?
finally, should we be parsing to the end? it looks to me like this will now accept any jibberish that isn't an equal sign.

This is now more robust. The lookahead is still there - we need to know after we've parse the first ident whether to continue parsing or not. However this time we handle cases where the next token is a comma, or there is no input and return early... otherwise we continue parsing.

aya-bpf-macros/src/args.rs line 49 at r1 (raw file):

pub(crate) fn pop_string_arg(args: &mut Args, name: &str) -> Option<String> {
    let value = match args.args.iter().position(|arg| match arg {

this function should be
let index = args.args.iter().position(|arg| matches!(arg, ....))?;
match args.args.remove(index) {
  Arg::String => ...,
  arg => panic!("impossible variant {arg:?})
}

currently it seems to allow impossible things to happen.

Done.

aya-bpf-macros/src/args.rs line 63 at r1 (raw file):

pub(crate) fn pop_bool_arg(args: &mut Args, name: &str) -> bool {
    let value = match args.args.iter().position(|arg| match arg {

this looks like Option::map
more below

And also done.


onClear not working in SearchField in react-area-components

Provide a general summary of the issue here
onClear on SearchForm does not fire
🤔 Expected Behavior?
The onClear handler is not fired
😯 Current Behavior
When test is typed an x appears. Clicking on the x clears the text but onClear is not fired.
💁 Possible Solution
No response
🔦 Context
After a search has been run, if the user click the x to clear the terms, the handler is needed to clear the search and also remove any filters.
🖥 Steps to Reproduce
Attach a function to the onClear handler. It is never called.
Version
"react-aria-components": "^1.2.1",

What browsers are you seeing the problem on?
Other
If other, please specify.
Arc
What operating system are you using?
MacOS
🧢 Your Company/Team
No response
🕷 Tracking Issue
No response

I'm unable to reproduce, onClear is being called for me https://codesandbox.io/p/sandbox/xenodochial-parm-mj4pfr?file=%2Fsrc%2FApp.js%3A7%2C56

Thanks for looking into this.
I have stripped back the code in my app to just what you have in that example, and what I am seeing is an extra X appearing in the input field. Because I started with minimal code I just assumed that was the x the triggered the event.
A click of the x in the  does return the event as expected, but it is a mystery where the other x is coming from.
https://github.com/adobe/react-spectrum/assets/30732/84572604-9349-4a8a-a649-97c07d0b3b42
Any guidance would be appreciated.

That's the native control's clear button, you can hide it via CSS. Checkout the docs CSS or the codesandbox css

OK! I have never seen that x before, which is why it caught me out. Darn browser makers. ;-)
Thanks for your patience, I am back on track now.


Understand the concept of the graph motifs in an arbitrary network

Can I work on this as my second project since I was not assigned any 2nd project ??


[Feature Request] Auth0 Support

Insomnia Version: 5.16.6
Operating System: MacOS 10.13.5 (17F77)

Details
There's no specific issue. But I wanted to get some feedback. Currently using the Oauth 2.0 Auth method there is no way to request a jwt token from Auth0. I can request an access token that will allow me to make further requests to the /userinfo endpoint. But if I'm using auth0 to authenticate a connection to a separate API, I can't seem to get an jwt access_token with my user data. It seems the Implicit grant type is missing the audience option which is needed to retrieve a full jwt access token.
Does it make sense to add an Auth0 auth method to the list to simplify development for anyone wanting to use their service?  Or is this something you think a plugin can do (with window redirect authentication)?
I'm happy to work on this!

There was an audience field added for Auth0 a while ago, although only on the Client Credentials grant type. I'd be happy to accept a PR adding it to the Implicit type.

@gschier cool, ill get it working and submit a PR.

Awesome! Let me know if you have any questions 😄


proposal PR to re-implement distributions API to match PyMC3

At this point, I'm putting this PR out for discussion. I do not consider it finalized - there are only some docstrings and no tests. I will be shaping the PR based on what we talk about here.
Discussion points:

If we want to maintain the PyMC3 API, it will be on us to provide alternate parameterizations (#69). Should we? This PR provides just a simple example in the form of the Normal distribution, but is extensible to other distributions, which I'm happy to work on.
We will need to add tests for the alternate parametrizations, likely; what would be the best way?
Opinionated me suggests that we add warnings to end-users: "don't use loc + sigma together", "don't use mu + scale together", not because it's wrong per se, but because it is a hybrid of two paradigms of thinking about a distribution, which introduces friction when reading and reasoning about code. I have not implemented the warnings, but it'd be trivial with the stdlib. Should we do this?

I invited a bunch of people whom I think would have opinions to be reviewers for the PR; @aloctavodia I was thinking a lot about your book and how much of a hassle it'd be to update all of the examples to be used with PyMC4 if we didn't have these alternate parameterizations ported over. I'm also thinking of @AustinRochford's old blog posts, and my own Bayesian Analysis Recipes repository, as examples I'd love to see maintained as frictionlessly as possible.

My preference is to have one parameterization, and not multiple (PyMC and TFP). I think it gets confusing otherwise (e.g. Normal(loc, sigma)). I don't think mapping will be onerous, because it is not likely to change, particularly if we are thoughtful about it from the beginnig.

I agree that PyMC4 should support friendlier argument names than TFP, but I'm not sure if I like the idea of alternative parameter names. I.e. I think we should support either (mu and sigma) or (loc and scale), but not both (I would prefer the Greek names for continuity with PyMC3).
As @ericmjl points out, alternative names would lead to "mixing" parameter names (imagine mu and scale), which is neither idiomatic nor Pythonic (there should be one and only one way to do it). I can't think of any function in the PyData ecosystem that allows you to pass parameters through one of two named arguments, and have the outcome be the same (usually that only happens when arguments are being deprecated).
The obvious exception would be when the parameters stand for mathematically different things: e.g. parametrizing a Normal through its variance (sigma), or its precision (tau).

From @fonnesbeck:

I don't think mapping will be onerous, because it is not likely to change, particularly if we are thoughtful about it from the beginning.

Agreed!
My lessons from pyjanitor mirror what Chris said: as long as the base is stable, the "friendly API" can do the tedious work up-front for the end-user.
And from @eigenfoo:

I would prefer the Greek names for continuity with PyMC3.

And so would I too!
I'm glad there's discussion here on this. Let me propose a change by pushing up a few more distributions parametrized the same way as in PyMC3, and see how many of the TFP distributions can be wrapped directly vs. needs a separate implementation.

I agree, I think this will make porting easier and provide nicer API for users.

Good points. After writing my comment I also had the thought that diverging from TFP might not be too wise if we want to allow some form of interaction between the two.

@twiecki what kind of interactions do you mean?

@twiecki @junpenglao thanks for chiming in!
Thomas, from your experience, which of the 80/20 use case does mixing in lower-level theano stuff fall in for you? Amongst PyMC3 users that I've interacted with, you've had the most experience going beyond the PyMC3-provided API.
Another related question for both of you - how often would you foresee a user needing to replace a PyMC4-parameterized distribution with a TFP-parameterized distribution?
At least from how I've interacted with PyMC3, I think I see it the way @fonnesbeck does - it's safe to provide a statistician-friendly parameterization, as the distributions base is foreseeably stable into the future. Also thinking about mixing/matching distributions, because PyMC4 distributions are building on top of TFP distributions, and because PyMC4 aims to be the high-level API for TFP model building, it's probably more advantageous for user adoption to keep the PyMC3 parameterizations.
To summarize the main issues brought up so far, and my own thoughts on them:

Will the TFP distributions need to be mixed in to PyMC4 code? From my experience, probably very little. For me & PyMC3, with the exception of the multivariate normal fix, I have had not any need to use NumPy's distributions. That said, I am probably too high-level of a user of PyMC3 :smile:.
Will the TFP distributions foundation shift under us in the future? Likely not; the design of the TFP API mimics NumPy's API very closely. This means there will be a very low probability of needing "maintenance PRs" on the distributions library.
Will retaining the PyMC3 distribution parameterizations help with migration? I think the answer is a clear yes. (import pymc4 as pm with minimal changes is extremely attractive for adoption.) There is a definite clear win for user-centered design. Whether this trumps over developer-centered design, however, is still debatable.
Should we go for user-centered design or developer-centered design? This is our question to answer. My experience, both as a user of PyMC3 and a developer of pyjanitor, tells me that these might not be mutually exclusive, if we are thoughtful about things - and if we are conscientious about documenting these thoughtful thoughts (and not just leaving them as history in the issue tracker).

I agree that these are all very fair points but I agree more with @junpenglao, if there is an error downstream in TFP, the traceback with a different parametrization from pymc4 would very confusing for developers, and in my opinion even more so for users. I think that we could shift to use TFP's single parametrization (one parametrization to rule them all and in darkness bind them...)

My preference for the PyMC3 parameterization is due to the fact that, as a high-level package for statistical modeling, its user-facing API is most important. So while Chris Suter's point about semantics being paramount is well-taken, that has to be balanced by the fact that our users are statisticians, analysts, and various other non-computer scientists. Their goal is in translating a mathematical model into computer code, so having conventions that are close to the math is going to be preferable. Using the location/scale convention would be somewhat easier from a developers perspective, but we aren't necessarily working to optimize for that.
I've used the SciPy stats library for almost as long as its existed, and I still don't like that it uses non-statistical names for its arguments. It may be satisfying for its developers to have it that way, but it makes me less inclined to use it because I have to adapt my thinking every time I use it. There are going to be a number of places our conventions will diverge from TFP (e.g. 4-space vs 2-space indenting), and that's just an ongoing challenge of the project. PyMC4 is going to look different than TFP, and that's part of the point (otherwise our users would just write TFP models).

Maybe we reintroduce @twiecki's idea here: https://github.com/pymc-devs/pymc3/pull/3308

I'm torn.  I will say I don't like loc/scale because mathematical is the language of statistics, so IMHO there's no benefit in abstracting it out.  However, I also want to minimize the pain / maintenance burden, and sticking with the raw TF is great there.
If we can come up with a "alternative mathy parameterizations layer" that requires very minimal maintenance, I would be fine with that.

All the comments on this are fair from by perspective. I think we are more user oriented and should go on with it.
My only concern about pymc3 naming is debugging. Once a user sees tensorflow graph (it is weird) with different naming scheme for some nodes it will be impossible to figure out where does the error come from. From my tensorflow experience it was nearly impossible even without these problems. So this negative change is marginal.
But from user side we get fast adoption to the new paradigm. This itself might be painful but with tfp naming it will be worse.

Maybe we reintroduce @twiecki's idea here: pymc-devs/pymc3#3308

@junpenglao I choked when I clicked that link! 😆

if there is an error downstream in TFP, the traceback with a different parametrization from pymc4 would very confusing for developers

@lucianopaz I hear you! I think there's a few ways we can address this, even with the PyMC3 parametrization in place.

We can add tests (or even better, use Hypothesis to generate tests) that intentionally error out, to test the boundaries of PyMC4. I have been experimenting with Hypothesis on pyjanitor (and on other projects before), and it's been great for weeding out errors that I didn't catch initially.
In-line docstrings, can help. I'll push up a few examples of what I think would be useful docstrings. One example: explicitly stating that "mu" maps to "loc" and "sigma" maps to "scale". This ties into the point I made earlier ("being conscientious about documenting these thoughtful thoughts").
Developer documentation can also help. For example, TFP's naming conventions corresponds to this. We write in a so-called "API policy" (e.g. like CuPy's: essentially "mirror NumPy, and document when it doesn't"), and then stick to it, being clear PyMC4 can't be everything to everybody, because it'll then be nothing to nobody.

Once a user sees tensorflow graph (it is weird) with different naming scheme for some nodes it will be impossible to figure out where does the error come from.

@ferrine that's a great point, especially when trying to get the user-facing API so-called "done well". I think the docs will have to reflect the differences appropriately.

that has to be balanced by the fact that our users are statisticians, analysts, and various other non-computer scientists. Their goal is in translating a mathematical model into computer code, so having conventions that are close to the math is going to be preferable. Using the location/scale convention would be somewhat easier from a developers perspective, but we aren't necessarily working to optimize for that.

I think my thoughts are mirrored here.

As said above, though, I don't think user-centered design and developer-centered design are necessarily mutually exclusive, though it is a tad more up-front work (and hopefully not much in the way of maintenance work) to get set on the right path. I'm working on a few more examples on my laptop right now, so I hope to clarify some of the aforementioned ideas through them. Will keep you all posted.

@ericmjl I agree that our documentation needs to reflect any parameter name changes that we decide on, whatever they end up being. I also like the idea of developer documentation to describe the guts of the PyMC4-TFP mappings.
I also like the direction that this PR is going, by hand-implementing the classes, instead of dynamically creating the "TFP-supported" random variables, as we currently do. This makes the modules balloon in size (in terms of line count), but I think it is more maintainable in the long run: the docs are embedded in the code, the mappings are clear and easy to read (both in documentation and in code), and future developers won't get confused by the dynamic creation of PyMC4 random variables. The only downside is that we need to invest a lot of up-front effort to actually write continuous.py the first time around, which I think is a price worth paying.
Finally, as to using Hypothesis to test: I recall @ColCarroll thought it was good but not suitable for PyMC4, but don't remember the reason. We should discuss that.

In a vaccuum, with respect to naming, I would stick to the tfp conventions, because:

It errs on the side of writing/maintaining less code
It is another layer of indirection
The project is in development, and supporting parameter mappings might make dev work at least marginally harder

That said, this change already exists, improves user experience, and has a dev working on it enthusiastically, all of which are gifts, and I wouldn't be upset to merge it when ready.

So we are committed to this change? I am still quite concern in particularly regarding potential confusing error message.
I would be much more happy with it if it is possible to intercept the error and with a custom message that provide the correct parameter hint/mapping

Also @eigenfoo, regarding hypothesis, I think my objections when trying to port pymc3 were:

tests took too long
many failures weren't "important" (often because of float32 or precision failures at certain scales)
I'm not familiar enough with the library to make it a proper unittest (i.e., reliably repeatable locally and on CI)

I think all of these problems could be overcome, but they are what I ran into.

The only reason I would be upset to merge this PR is because we only finished one distribution. Regardless of which path we choose, I think we should commit to all or nothing
halfway change < choice1 <choice2

Maybe we reintroduce @twiecki's idea here: pymc-devs/pymc3#3308

Well well well  ;)

I would be much more happy with it if it is possible to intercept the error and with a custom message that provide the correct parameter hint/mapping

@junpenglao that's a great point. Would you be open to providing an example that errors out in a confusing way? That'll help me grok a better idea of where the error interception should occur.

@canyon289 seeing the PR title change I though we are going all the way. But yeah agree with you that we definitely should commit to one style.
Alternatively, I think we can also prepare the PR and wait a bit before committing - at least that case the switching would not get into the way of development too much.

Alternatively, I think we can also prepare the PR and wait a bit before committing - at least that case the switching would not get into the way of development too much.

No problem with that! Happy to work it through and keep the PR open. I'll be sure to merge in changes from master frequently.

Ok, looks like black standardizes on small r""".

@ColCarroll @junpenglao We are not committed to this change--its very much up for discussion still.

Wouldn't it be possible to rename nodes on the graph?

I guess we should decide what we want to do here. Vote up/down on this item what you think.

@ericmjl I updated the comment to clarify.

Save for any final checks, I consider this PR done, as per discussion during today's dev meeting. cc: @twiecki.
Summary of changes:

Ported pm3 docstrings.
Ported pm3 arguments to match mathematical conventions.
Type hints added.

Ok! All checks pass, and linear regression test is now done with the mathematically-oriented API.

Thanks a bunch, @ericmjl !


{arcdata} HTML tags are rendering as empty parameter values in reference docs

Describe the bug
Problem to solve:
HTML tags in source code are causing build warnings in MicrosoftDocs/azure-docs-cli. More importantly, customer-facing reference docs are not instantiating properly -- parameter values formatted with `<>` are missing. Here's an example:

Requested solution:
These values should be escaped with a backtick. Here's an example from az webpubsub.

Related GitHub issues:
This is the same issue as these GitHub issues.
List of HTML tag instances that need update:

File name
Build error

latest/docs-ref-autogen/arcdata/dc/config.yml
HTML tag 'cwd' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/arcdata/dc/config.yml
HTML tag 'cwd' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

Related command
az arcdata
Errors
n/a
Issue script & Debug output
n/a
Expected behavior
n/a
Environment Summary
Azure CLI 2.62.0
Additional context
No response

Thank you for opening this issue, we will look into it.


High percentage of CPU usage in ssdlite_mobilenet model

Hi all,
I trained a model for license plate detection with ssdlite_mobilenet_v2_coco (training images:20000,steps:200000) and this model is going to be used in a traffic control app.
The accuracy of this model on frames of video is very good but the percentage of CPU usage on my system is between 70-80% (CPU info : Intel Core i7-6500 @ 2.50GHz.) and given that traffic control app should be constantly running on client system, this percentage is quite high.
So what are the best ways to optimize CPU usage to analyze frames of video and detect license plate? (reduce to ~30-40%)
Do I need to make any changes to the config file or use another pre-trained model?
Please guide me :)

You may try optimizing your model using quantization, or use quantized mobilenet v2 model.
See https://www.tensorflow.org/lite/performance/best_practices#optimize_your_model


update git documentation

This adds instructions on how to simulate a merge

Hmm... I'm having second thoughts about this...
Maybe better to find a way to simply diff the conflicts


Implement Paging Library and paging support

Add to client an option if it supports paging content.
If it supports paging content on a RecyclerView then add an PagingAdapter implementation.
Otherwise use the existing Adapters.

https://developer.android.com/topic/libraries/architecture/paging/

Implement lean back Paging Adapter.


[Bug]: TypeError: Do not know how to serialize a BigInt

Contact Details
No response
What happened?
I've just bootstrapped an install of AdminJS using Express and Prisma to connect to a MySql db. My resources are displayed in the UI but when selected they error. From the network console in the browser ...
http://localhost:3000/admin/api/resources/ValidationRule/actions/search which results in a 500 and this error
TypeError: Do not know how to serialize a BigInt
In fact the ValidationRule table has an ID defined as
model ValidationRule {
validationRuleId    BigInt                @id @default(autoincrement())
}
I believe I ran over a previously closed issue which suggested changing BigInt to Int, but that is not reasonable for this solution. Thoughts.
Thank you
Bug prevalence
Whenever I access a resource which has a BigInt as an ID field.
AdminJS dependencies version
"dependencies": {
"@adminjs/express": "^6.1.0",
"@adminjs/prisma": "^5.0.1",
"@prisma/client": "^5.6.0",
"adminjs": "^7.7.2",
"express": "^4.18.2",
"express-formidable": "^1.2.0",
"express-session": "^1.17.3",
"mysql2": "latest"
},
What browsers do you see the problem on?
No response
Relevant log output
AdminJS available at http://localhost:3000/admin
✔ Finish bundling: 1 files
TypeError: Do not know how to serialize a BigInt
    at JSON.stringify (<anonymous>)
    at stringify (/Users/benjamin/repos/promoToolAdminJS/promotooladminjs/node_modules/express/lib/response.js:1160:12)
    at ServerResponse.json (/Users/benjamin/repos/promoToolAdminJS/promotooladminjs/node_modules/express/lib/response.js:271:14)
    at ServerResponse.send (/Users/benjamin/repos/promoToolAdminJS/promotooladminjs/node_modules/express/lib/response.js:162:21)
    at file:///Users/benjamin/repos/promoToolAdminJS/promotooladminjs/node_modules/@adminjs/express/lib/buildRouter.js:32:17

Relevant code that's giving you issues
still searching the code base.

I was able workaround this by defining a serializer
declare global {
  interface BigInt {
    toJSON: () => string;
  }
}

BigInt.prototype.toJSON = function (): string {
  return this.toString();
};


⚠ ClamAV has degraded performance

In c427af1, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 456 ms

Resolved: ClamAV performance has improved in 2f95b33 after 12 minutes.


Switch default routing/layout method to sabre for opt level 3

Summary
This commit switches the default routing and layout method for
optimization level 3 to use SabreSwap and SabreLayout passes. The
quality of results is typically better with sabre than the default
stochastic swap and dense layout we're using now. For optimization
level 3 where we try to produce the best quality result and runtime
is of secondary concern using sabre makes the most sense (especially
after #7036 which improves the runtime performance). Also for
optimization level 3 currently sabre is typically faster because we
increase the number of trials for stochastic swap (which is generally
significantly faster) which slows it down as it's doing more work.
This should improve the quality and speed of the results in general
when running with optimization level 3. In the future we can do more
work to improve the runtime performance of the sabre passes and
hopefully make it fast enough to use for all the optimization levels
which have more constraints on the run time performance than level 3.
Details and comments
Related to #7112 and #7200

Marking as on hold until #7036 merges because I don't think we want to do this without having that performance improvement

To see the improvement in the quality of results you can look at the nightly benchmarks here:

https://qiskit.github.io/qiskit/#transpiler_qualitative.TranspilerQualitativeBench.track_depth_transpile_mod8_10_178?p-optimization level=3&x-axis=routing method&p-layout method='dense'&p-layout method='sabre'&ram=64GB
https://qiskit.github.io/qiskit/#transpiler_qualitative.TranspilerQualitativeBench.track_depth_transpile_4mod5_v0_19?ram=64GB&p-optimization level=3&p-routing method='sabre'&p-layout method='dense'&p-layout method='sabre'&x-axis=routing method
https://qiskit.github.io/qiskit/#transpiler_qualitative.TranspilerQualitativeBench.track_depth_transpile_4gt10_v1_81?x-axis=routing method&python=3.8&p-optimization level=3&p-layout method='dense'&p-layout method='sabre'

Which is comparing the depth of the output circuits at optimization level 3 with different routing and layout method combinations.

Also: https://nonhermitian.org/posts/2021/2021-10-31-best_swap_mapper_qiskit.html is relevant here too

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
1164 unchanged lines in 80 files lost coverage.
Overall coverage increased (+0.3%) to 82.54%

Files with Coverage Reduction
New Missed Lines
%

qiskit/algorithms/phase_estimators/phase_estimator.py
1
85.71%

qiskit/opflow/converters/two_qubit_reduction.py
1
87.8%

qiskit/opflow/gradients/qfi.py
1
94.44%

qiskit/pulse/utils.py
1
97.37%

qiskit/qasm/node/if_.py
1
85.71%

qiskit/qasm/node/reset.py
1
85.71%

qiskit/qobj/common.py
1
94.74%

qiskit/result/result.py
1
85.62%

qiskit/synthesis/evolution/lie_trotter.py
1
96.15%

qiskit/synthesis/evolution/matrix_synthesis.py
1
93.33%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
49836

Relevant Lines:
60378

💛  - Coveralls


ImportError - fast_transformers/causal_product undefined symbol - unable to train or finetune

After downloading the data, I go to run bash run_finetune_h298.sh and get the following error:
Traceback (most recent call last):
  File "finetune_pubchem_light.py", line 14, in <module>
    from rotate_attention.rotate_builder import RotateEncoderBuilder as rotate_builder
  File "/home/kpg/molformer/finetune/rotate_attention/rotate_builder.py", line 3, in <module>
    from .attention_layer import RotateAttentionLayer
  File "/home/kpg/molformer/finetune/rotate_attention/attention_layer.py", line 8, in <module>
    from fast_transformers.attention import AttentionLayer
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/attention/__init__.py", line 13, in <module>
    from .causal_linear_attention import CausalLinearAttention
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/attention/causal_linear_attention.py", line 15, in <module>
    from ..causal_product import causal_dot_product
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/causal_product/__init__.py", line 9, in <module>
    from .causal_product_cpu import causal_dot_product as causal_dot_product_cpu, \
ImportError: /home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/causal_product/causal_product_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe28TypeMeta21_typeMetaDataInstanceIN3c107complexIfEEEEPKNS_6detail12TypeMetaDataEv

I get a similar error when running bash run_pubchem_light.sh:
Traceback (most recent call last):
  File "train_pubchem_light.py", line 18, in <module>
    from fast_transformers.builders import TransformerEncoderBuilder
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/builders/__init__.py", line 42, in <module>
    from ..attention import \
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/attention/__init__.py", line 13, in <module>
    from .causal_linear_attention import CausalLinearAttention
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/attention/causal_linear_attention.py", line 15, in <module>
    from ..causal_product import causal_dot_product
  File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/causal_product/__init__.py", line 9, in <module>
    from .causal_product_cpu import causal_dot_product as causal_dot_product_cpu, \
ImportError: /home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/fast_transformers/causal_product/causal_product_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe28TypeMeta21_typeMetaDataInstanceIN3c107complexIfEEEEPKNS_6detail12TypeMetaDataEv

I set up my environment based on the instructions in environment.md as follows:
conda create --name MolTran_CUDA11 python=3.8.10
conda activate MolTran_CUDA11

conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.6 -c pytorch -c conda-forge
conda install rdkit==2021.03.2 pandas=1.2.4 scikit-learn=0.24.2 scipy=1.6.3 -c conda-forge

pip install transformers==4.6.0 pytorch-lightning==1.1.5 pytorch-fast-transformers==0.4.0 datasets==1.6.2 jupyterlab==3.4.0 ipywidgets==7.7.0 bertviz==1.4.0

git clone https://github.com/NVIDIA/apex
cd apex
export CUDA_HOME='/usr'
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

The differences between the above and the original instructions were:

added -c conda-forge to the 2nd conda install command (it couldn't find the packages otherwise)
export CUDA_HOME='/usr' (the actual location on my system, found using which nvcc, which gave the output /usr/bin/nvcc)
changed the pytorch and cudatoolkit versions to match the nvcc version I have installed, which is 11.6 (compiling Apex failed otherwise). I used the oldest pytorch version that supported cudatoolkit=11.6 (based on instructions here) to maximize likelihood of compatibility since this repo was created using pytorch==1.7.1 cudatoolkit=11.0.

Additional information that may be useful:
nvidia-smi: NVIDIA-SMI 515.105.01   Driver Version: 515.105.01   CUDA Version: 11.7
(MolTran_CUDA11) ~/molformer/finetune$ conda list | grep 'torch\|cuda'
cudatoolkit               11.6.0              hecad31d_10    conda-forge
ffmpeg                    4.3                  hf484d3e_0    pytorch
pytorch                   1.12.0          py3.8_cuda11.6_cudnn8.3.2_0    pytorch
pytorch-fast-transformers 0.4.0                    pypi_0    pypi
pytorch-lightning         1.1.5                    pypi_0    pypi
pytorch-mutex             1.0                        cuda    pytorch
torchaudio                0.12.0               py38_cu116    pytorch
torchvision               0.13.0               py38_cu116    pytorch

(MolTran_CUDA11) ~/molformer/finetune$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:18:20_PST_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0

Based on similar errors people have gotten with other repos (e.g. here, here), it seems that the problem is related to my version of PyTorch, but I'm not sure how to resolve this while still allowing Apex to compile on my system. Is it possible to run this repo on a system using nvcc 11.6 / CUDA 11.7?

Update:
I tried starting from scratch on a different machine that's running nvcc 11.0 / CUDA 11.0. So the only changes I had to make to the installation instructions were:

added -c conda-forge to the 2nd conda install command (it couldn't find the packages otherwise)
export CUDA_HOME='/usr/local/cuda-11.0' (the actual location on my system, found using which nvcc, which gave the output /usr/local/cuda-11.0/bin/nvcc)

But this time, the Apex compilation fails with the following error:
ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1533, in _run_ninja_build
      subprocess.run(
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/subprocess.py", line 516, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/kpg/apex/setup.py", line 762, in <module>
      setup(
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
      super().run_command(command)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/command/install.py", line 698, in run
      self.run_command('build')
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
      super().run_command(command)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
      super().run_command(command)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
      self.build_extensions()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 670, in build_extensions
      build_ext.build_extensions(self)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 468, in build_extensions
      self._build_extensions_serial()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 494, in _build_extensions_serial
      self.build_extension(ext)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 549, in build_extension
      objects = self.compiler.compile(
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 491, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1250, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1555, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  error: subprocess-exited-with-error

  × Running setup.py install for apex did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/kpg/miniconda3/envs/MolTran_CUDA11/bin/python -u -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/home/kpg/apex/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' --cpp_ext --cuda_ext install --record /tmp/pip-record-8z9sz80y/install-record.txt --single-version-externally-managed --compile --install-headers /home/kpg/miniconda3/envs/MolTran_CUDA11/include/python3.8/apex
  cwd: /home/kpg/apex/
  Running setup.py install for apex ... error
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> apex

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Here are the details for this system:
nvidia-smi: NVIDIA-SMI 450.51.05    Driver Version: 450.51.05    CUDA Version: 11.0
(MolTran_CUDA11) ~/apex$ conda list | grep 'torch\|cuda'
cudatoolkit               11.0.221             h6bb024c_0
pytorch                   1.7.1           py3.8_cuda11.0.221_cudnn8.0.5_0    pytorch
pytorch-fast-transformers 0.4.0                    pypi_0    pypi
pytorch-lightning         1.1.5                    pypi_0    pypi

(MolTran_CUDA11) ~/apex$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Thu_Jun_11_22:26:38_PDT_2020
Cuda compilation tools, release 11.0, V11.0.194
Build cuda_11.0_bu.TC445_37.28540450_0

Thank you in advance for any advice you can provide!

Thanks for sharing, but unfortunately this didn't solve my problem. On the first machine I mentioned above, I was able to compile apex but then got an error while running the molformer code. On the second machine, I wasn't able to compile apex. After running git checkout 22.04-dev on both machines, I got similar errors to before. It looks like there are many other people having compilation issues with apex.

It doesn't seem like the authors are replying to anyone so I'll try and help. Here is the env that works for me:
conda create molformer_env python==3.8.10 -y
conda activate molformer_env
conda config --add channels conda-forge

conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch -y # this seems to be different version to yours
conda install rdkit==2021.03.2 pandas=1.2.4 scikit-learn=0.24.2 scipy=1.6.3 -y

pip install transformers==4.6.0 pytorch-lightning==1.1.5 pytorch-fast-transformers==0.4.0 datasets==1.6.2 jupyterlab==3.4.0 ipywidgets==7.7.0 bertviz==1.4.0 packaging

git clone https://github.com/NVIDIA/apex
cd apex
git checkout 22.04-dev
export CUDA_HOME=/usr/local/cuda-11.0 # location of my cuda install
export TORCH_CUDA_ARCH_LIST="8.0"  # I'm compiling for use on A100s

python -m pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Looks like the torch version that you used is different to mine. Hopefully this helps you!

On the second machine, I ran:
conda create --name MolTran_CUDA11
conda activate MolTran_CUDA11

mamba install python==3.8.10
mamba install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.6 -c pytorch -c conda-forge
mamba install rdkit==2021.03.2 pandas=1.2.4 scikit-learn=0.24.2 scipy=1.6.3 -c conda-forge

pip install transformers==4.6.0 pytorch-lightning==1.1.5 pytorch-fast-transformers==0.4.0 datasets==1.6.2 jupyterlab==3.4.0 ipywidgets==7.7.0 bertviz==1.4.0

git clone https://github.com/NVIDIA/apex
cd apex
git checkout 22.04-dev
export CUDA_HOME='/usr/local/cuda-11.0'
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

I've upgraded CUDA on the second machine since my last post, so now nvidia-smi gives NVIDIA-SMI 530.30.02              Driver Version: 530.30.02    CUDA Version: 12.1.
nvcc --version is the same as before.
$ conda list | grep 'torch\|cuda' now gives:
cudatoolkit               11.6.2              hfc3e2af_12    conda-forge
ffmpeg                    4.3                  hf484d3e_0    pytorch
pytorch                   1.12.0          py3.8_cuda11.6_cudnn8.3.2_0    pytorch
pytorch-fast-transformers 0.4.0                    pypi_0    pypi
pytorch-lightning         1.1.5                    pypi_0    pypi
pytorch-mutex             1.0                        cuda    pytorch
torchaudio                0.12.0               py38_cu116    pytorch
torchvision               0.13.0               py38_cu116    pytorch

My new error is
Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/kpg/apex/setup.py", line 1, in <module>
      import torch
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/__init__.py", line 201, in <module>
      _load_global_deps()
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/__init__.py", line 154, in _load_global_deps
      ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
    File "/home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/ctypes/__init__.py", line 373, in __init__
      self._handle = _dlopen(self._name, mode)
  OSError: /home/kpg/miniconda3/envs/MolTran_CUDA11/lib/python3.8/site-packages/torch/lib/../../../../libcublas.so.11: undefined symbol: cublasLtHSHMatmulAlgoInit, version libcublasLt.so.11
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/kpg/miniconda3/envs/MolTran_CUDA11/bin/python -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/home/kpg/apex/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-czvr4cr3
  cwd: /home/kpg/apex/
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@philspence thanks for sharing your installation process. It doesn't look like the different version of pytorch fixes it for me. Can you confirm whether after you compile apex successfully, are you also able to run bash run_finetune_h298.sh from the molformer/finetune/ directory without issues?

Yeah, I tried to replicate yours, still wasn't able to get it working. It didn't work regardless of whether or not I used the --no-build-isolation flag.

Related issues on apex (non-exhaustive):

https://github.com/NVIDIA/apex/issues/1471
https://github.com/NVIDIA/apex/issues/1545
https://github.com/NVIDIA/apex/issues/1548
https://github.com/NVIDIA/apex/issues/1563
https://github.com/NVIDIA/apex/issues/1566
https://github.com/NVIDIA/apex/issues/1594
https://github.com/NVIDIA/apex/issues/1603
https://github.com/NVIDIA/apex/issues/1609
https://github.com/NVIDIA/apex/issues/1636

Yeah, I tried to replicate yours, still wasn't able to get it working. It didn't work regardless of whether or not I used the --no-build-isolation flag. If the h298 benchmark started running, then it got further than it did for me, so no need to run it to completion.

Did you also change your system cuda version to 11.0.3?
Here is my conda list | grep 'torch\|cuda':
cudatoolkit               11.0.3              h7761cd4_12    conda-forge
pytorch                   1.7.1           py3.8_cuda11.0.221_cudnn8.0.5_0    pytorch
pytorch-fast-transformers 0.4.0                    pypi_0    pypi
pytorch-lightning         1.1.5                    pypi_0    pypi
torchaudio                0.7.2                      py38    pytorch
torchvision               0.8.2                py38_cu110    pytorch

And my system cuda install:
wget https://developer.download.nvidia.com/compute/cuda/11.0.3/local_installers/cuda-repo-rhel7-11-0-local-11.0.3_450.51.06-1.x86_64.rpm
rpm -i cuda-repo-rhel7-11-0-local-11.0.3_450.51.06-1.x86_64.rpm
yum clean all
yum -y install cuda-toolkit-11-0

Unfortunately I can't change my system CUDA version because this machine is a group workstation, and changing the system version would be disruptive for the other users. But when I first encountered this problem, my system was on cuda 11.0, so I don't think that's the problem.

Ah, in which case, install cudatoolkit-dev:
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 cudatoolkit-dev=11.0 -c pytorch
and set your cuda home to be the base of your conda env:
export CUDA_HOME=/path/to/conda/env # normally /home/user/conda/env/name_of_env
That worked for me on a different system. If that doesn't work for you then I'm out of ideas!

I see that many people are getting frustrated  because of cuda version issues with Apex.
My solution is that if you only want to fine-tune the model, you can avoid using Apex. Instead, I replaced the optimizer in the Python script located in the "finetune" folder with the ones available in the standard torch.optim module. At least, I was able to successfully execute the run_finetune_classification_hiv.sh script by modifying the finetune/finetune_pubchem_light_classification.py:
...
# from apex import optimizers
from torch.optim import Adam
...

    def configure_optimizers(self):
        ...
        # optimizer = optimizers.FusedLAMB(optim_groups, lr=learning_rate, betas=betas)
        optimizer = Adam(optim_groups, lr=learning_rate, betas=betas)
        ...

I only install these modules in my conda env:
conda create --name molformer python=3.8.10 -y
conda activate molformer

conda install \
    pytorch==1.7.1 \
    rdkit==2021.03.2 \
    pandas=1.2.4 \
    scikit-learn=0.24.2 \
    scipy=1.6.3 \
    -c pytorch -y

pip install \
    transformers==4.6.0 \
    pytorch-lightning==1.1.5 \
    pytorch-fast-transformers==0.4.0 \
    datasets==1.6.2 \
    jupyterlab==3.4.0 \
    ipywidgets==7.7.0 \
    bertviz==1.4.0

And I using an ubuntu 22.04 docker, with RTX 3090 gpu
It works for me.
Hope this will help.

I'm able to extract the embeddings from frozen_embeddings_classification.ipynb
For people who are facing the same problem, I created the env with:
conda create -n molformer -y
conda activate molformer
conda install pytorch==1.12.1 cudatoolkit=11.3 -c pytorch -y
python -m pip install pytorch-fast-transformers
conda install -c conda-forge pytorch-lightning=2.0.7 -y
python -m pip install rdkit datasets regex
python -m pip install jupyterlab
Then adopt the response from @BlenderWang9487
change
from apex import optimizers
#to
from torch.optim import Adam
def configure_optimizers(self):
...
# optimizer = optimizers.FusedLAMB(optim_groups, lr=learning_rate, betas=betas)
optimizer = Adam(optim_groups, lr=learning_rate, betas=betas)
...
I hope this also helps

This solution to the “fast_transformers/causal_product undefined symbol” error worked for me. I used the install of BlenderWang9487 with the following additional steps:

Did not install Apex
Replaced optimizer with Adam in ~finetune/finetune_pubchem_light_classification.py
3.Fixed issues with pytorch-fast-transformers==0.4.0
install. Only some of the files are being installed. Check https://github.com/idiap/fast-transformers/tree/master/fast_transformers to see if all files were included. The fast_transfomers folder was missing files in my install.To fix this, I first did a pip install pytorch-fast-transformers==0.4.0. Then, I cloned the pytorch-fast-transformers repository and copied over any missing files into ~/anaconda3/envs/molformer/lib/python3.8/site-packages/fast_transformers. Make sure the following files are found in ~/fast_transformers/fast_transformers/causal_product folder, as well as any missing files from fast_transformers github:
causal_product_cpu.cpp
causal_product_cuda.cu
causal_product_cpu.cpython-38-x86_64-linux-gnu.so

This should fix the “fast_transformers/causal_product undefined symbol” error. There’s probably a more elegant way to do this, but this is what worked for me.
4.Included the following statements in script:
export CUDA_HOME=/home/user/anaconda3/envs/molformer
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/anaconda3/envs/molformer/lib

I've been having a similar issue on a Mac M1 device.
I've actually had molformer previously working on this device with python 3.10.8, but had to create a new environment and no longer can get past this import error. I can't even get all of the recommended package versions as listed in the environment.md file to install because some of the conda channels are different, but matched them as closely as possible
I'm only using the pretrained_molformer folder, and no cuda or apex. I've tried the recommendations above such as copying files and --no-cache-dir and even setup fast-transformers straight from the github, but am not sure what else to do.
I'm hoping someone has seen this before.
from pretrained_molformer.train_pubchem_light import LightningModule
File "/Users/brandiransom/Documents/Projects/MS/backend/pretrained_molformer/train_pubchem_light.py", line 20, in
from pretrained_molformer.rotate_attention.rotate_builder import RotateEncoderBuilder as rotate_builder
File "/Users/brandiransom/Documents/Projects/MS/backend/pretrained_molformer/rotate_attention/rotate_builder.py", line 3, in
from .attention_layer import RotateAttentionLayer
File "/Users/brandiransom/Documents/Projects/MS/backend/pretrained_molformer/rotate_attention/attention_layer.py", line 8, in
from fast_transformers.attention import AttentionLayer
File "/Users/brandiransom/.local/lib/python3.10/site-packages/fast_transformers/attention/init.py", line 13, in
from .causal_linear_attention import CausalLinearAttention
File "/Users/brandiransom/.local/lib/python3.10/site-packages/fast_transformers/attention/causal_linear_attention.py", line 15, in
from ..causal_product import causal_dot_product
File "/Users/brandiransom/.local/lib/python3.10/site-packages/fast_transformers/causal_product/init.py", line 9, in
from .causal_product_cpu import causal_dot_product as causal_dot_product_cpu,
ImportError: dlopen(/Users/brandiransom/.local/lib/python3.10/site-packages/fast_transformers/causal_product/causal_product_cpu.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '___kmpc_for_static_fini'


ESP32 SD SPI mode: invalid commands sent to SD card: CMD13, CMD60 while writing blocks (IDFGH-12168)

sigrok_capture_recording_annotations.txt
Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report
I have created a project that is heavily based on the SD card recorder (SPI mode) example from ESP-IDF 4.4.x, only targeting the INMP441 MEMS microphone, and writing raw PCM data to the SD card instead of a fixed size WAV file. The project files can be found on my GitHub: https://github.com/MayaPosch/pcm_recorder
I'm using ESP-IDF 5.1.2, but the same issue occurs with 5.1.1. After flashing the firmware on the ESP32 (WROOM-32), I see that it correctly mounts and identifies the inserted microSD card. This (1 GB microSD) card is inserted into an SD card module that operates off 3.3V, with no level shifter circuitry, only pull-ups on the SPI lines.
When I write a test file right after this ('hello.txt') with some text in it, this works fine, and I can read the newly written file afterwards when inserting the microSD card into a reader connected to the PC. The problem begins when the I2S reading task begins, and it starts writing the 16-bit samples to the card, resulting in the following error:
sdmmc_cmd: sdmmc_write_sectors_dma: sdmmc_send_cmd returned 0x109
diskio_sdmmc: sdmmc_write_blocks failed (265)

I have reduced the SPI bus speed to 2 MHz and connected a logic analyzer (Saleae Logic 8, 24 MHz) to analyze the SPI bus traffic using the SD SPI mode decoder in Sigrok PulseView. This shows that when the writing commences, the ESP32 sends a CMD24 (write block) command, which completes fine, but after that I see it sending subsequent blocks prefixed first with CMD13 (send status) and then with CMD60, the latter of which the card obviously responds to with R1 0xe5 (illegal command), which the ESP32 repeats a few times until one of these CMD60-prefixed blocks gets a Data rejected (CRC error) which results in the previously listed error.
Is this an issue with ESP-IDF 4.x APIs not working (properly) in ESP-IDF 5.x and should I port it to the new APIs, or is something else going on? What concerns me is that the initial block was prefixed with a proper CMD24 and that only then the trouble began. I have attached the annotations export from PulseView's SD SPI decoder for reference.
If any more information is required, please let me know.

Hi @MayaPosch, would you be able to attach those PulseView recordings for us to take a look?
Based on how the SDMMC driver is written, I don't think we can accidentally send a completely non-existent command down to the SPI layer. I see two possibilities: either there is something wrong with the setting of SPI mode, so that the data is transmitted to the card with a bit shift; or, there is a signal integrity issue on the board being used.
The way you initialize the card in https://github.com/MayaPosch/pcm_recorder/blob/9b28469fdc94ae267fb21dd22f975371417b1ac1/main/main.c#L105 looks fine to me at the first glance, as well.
We'll try to run your project to see if the same issue can be reproduced, and also check the PulseView recordings if you provide them.

I do see the recording attached now, looks like I didn't refresh the page, my bad. Please disregard that question.

Hi @MayaPosch ,
I think you are running out of the buffer when calling
fwrite(samples16, 1, bytes_read, f);
The length should be bytes_read / 2 or, better, samples_read * sizeof(samples16[0]).

@rrtandler You're right, that was indeed wrong. I have fixed it in the linked GitHub repository and re-ran the test.
This time around I do not see the CRC error, and the recording seems to continue as normal. When I look at the PCM file on the SD card afterwards on the PC, it is however still 0 bytes. In the logic analyser's trace I can see that it looks better, except that I still see CMD60 all the time (along with CMD61), which I presume is what is causing the SD card to not write the data, despite valid CMD25 and CMD13 being sent by the MCU mixed in with the noise.
I have attached the new annotations file from the SD SPI mode decoder in PulseView.
pcm_debug_sigrok_capture_recording_with_fixes_annotations_cmd_repl.txt
This trace is recorded from the moment that the i2s_read_task commences.

@MayaPosch  - The reason for not seeing data in the file might be a missing fclose(f) before unplugging the card and reading it in PC.

@MayaPosch  - to improve the data transfer, plese consider following the hint in comment:
	/* ATTENTION: The print and delay in the read task only for monitoring the data by human, 	 * Normally there shouldn't be any delays to ensure a short polling time, 	 * Otherwise the dma buffer will overflow and lead to the data lost */

@rrtandler Thank you for that reminder. I have removed all debug logging and am also writing the raw 32-bit samples directly to the SD card to minimise delays. It sounds much better now.
As for the odd commands in the logic analyser trace, this might be a bug in the Sigrok PulseView's SD card decoder: https://sigrok.org/bugzilla/show_bug.cgi?id=1365
I had to switch back to the older, stable version of Sigrok due to a libusb issue in the nightly builds, but it would seem that in this older PulseView version there is another bug which combined with the flaws in my own code led me to believe that the ESP32 or ESP-IDF was at fault. My apologies for this.
Closing this topic now as I think all issues have been resolved.