Single Logout URL is never called by Azure AD B2C

Is the Single Logout/Sign-out still only available for custom policies as per the current docs?

If it is, would you please direct me to the docs I need to follow to implement it? Possibly other than these: https://docs.microsoft.com/fr-fr/azure/active-directory-b2c/session-behavior-custom-policy#configure-the-token-issuer because it doesn't really show us what we need to add to our custom policy.
If it is not, and is now also available for built-in policies, why does Azure AD B2C fail to send a request to the configured Logout URL? It's just expecting a GET endpoint and 200 HTTP OK response, right? In any way, the call is never made when I logout from another app or the end_session_endpoint.

Thank you for any clarification.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5761bd51-2480-33fe-0e82-c8e7b58fdcd5
Version Independent ID: 59194957-7c80-9c86-d903-18b15022adde
Content: SSO sessions in Azure Active Directory B2C
Content Source: articles/active-directory-b2c/session-overview.md
Service: active-directory
Sub-service: b2c
GitHub Login: @msmimart
Microsoft Alias: mimart

@iliassk
Thanks for your question! We will investigate and update as appropriate.

@iliassk
I was able to look into this issue and found a Single Sign-Out SAML Protocol doc. However, besides that documentation it looks like there's still a feature request for Single Sign out in B2C.
Additionally, if you'd like to further troubleshoot your issue, can you please post it on MS Q&A so our community team can further look into the issue.

The docs, I linked above, say:
When you redirect the user to the Azure AD B2C sign-out endpoint (for both OAuth2 and SAML protocols), Azure AD B2C clears the user's session from the browser. However, the user might still be signed in to other applications that use Azure AD B2C for authentication. To enable those applications to sign the user out simultaneously, Azure AD B2C sends an HTTP GET request to the registered LogoutUrl of all the applications that the user is currently signed in to.
Applications must respond to this request by clearing any session that identifies the user and returning a 200 response. If you want to support single sign-out in your application, you must implement a LogoutUrl in your application's code.
https://docs.microsoft.com/fr-fr/azure/active-directory-b2c/session-overview#single-sign-out
So it should be working for both SAML AND OAuth2.

@illiassk
Thank you for pointing that out! I'll work with my team to help answer your original question:
Is the Single sign-out feature only available for custom policies:

If it's still only available for custom policies, I'll see if we have any other public docs on how you can implement this.

If it's available for built-in policies, I'll get clarification on why AzureAD B2C fails to send a request to the Logout URL. Since the call is never made when you logout from another app or the end_session_endpoint.

@iliassk
I spoke with my team and was given the below GitHub issues for your reference. As of right now, the documentation is up to date and the only implementation/how to docs we have regarding AzureAD B2C Custom Policies can be found here.
Issues:
https://github.com/MicrosoftDocs/azure-docs/issues/34577
https://github.com/MicrosoftDocs/azure-docs/issues/49090

If you'd like to submit a feature request you can do so using this link. However, if you'd like us to continue to further troubleshoot your issue, please submit your question to our Q&A forums.

@iliassk
I just wanted to check in and see if you had any other questions regarding my previous post? If not, I'll go ahead and close out this GitHub issue at the end of the day.
Thank you for your time and patience, if you'd like us to continue to further troubleshoot your issue, please submit your question to our Q&A forums.

Hi @JamesTran-MSFT, I apologize for the delayed response. I was a bit discouraged by your response. None of the issues you linked answer my question.
I have no problem logging out my users on App A, when I initiate the logout on App A. The issue is when the logout has been initiated on App B. Azure AD B2C is supposed to call the Logout URI configured on App A to notify App A that the user has logged out (as per the docs and OIDC specs).
If the Single Logout-Out/Single Sign-Out is correctly implemented on your end and works as described above, I want to know:
- Why does AzureAD B2C fail to send a request to the Logout URL? Using the built-in user flow.
- If it only works with custom policies: can you please show me how can I configure my custom policy to handle the SLO. I went through this entire doc section, with no luck.
Thank you !

@iliassk
Thank you for your time and the detailed response!
After speaking with my team, I've re-opened your issue and have reached out to our product team for further guidance/troubleshooting.
I'll update you as soon as I receive a response from their end.

@iliassk
I just wanted to let you know that I reached out to our product team this morning and will update as soon as I hear back from them.

For custom policies, the guidance for modifying the policy files for SLO (OIDC&SAML) is here:
https://docs.microsoft.com/en-us/azure/active-directory-b2c/session-behavior-custom-policy#single-sign-out

@iliassk
I just wanted to check in and see if the response from @JasSuri helped with your issue. It looks like the link provided points to the only location we have regarding AzureAD B2C custom policy documentation.

Please let us know if you have any other questions.
Thank you for your time and patience throughout this issue.

@iliassk
I just wanted to check in and see if you had a chance to review the above posting by JasSuri.
If you have any other question, please let us know. Otherwise, I'll go ahead and close out your issue at the end of the week.
Thank you for your time and patience.

Closing out the issue.
If you require additional assistance or have any questions, please feel free to tag me and I will be more then happy to continue working with you on this issue.
Thank you for your time and patience!

@JasSuri @JamesTran-MSFT I've already tried what was suggested in the docs but it's not working for me. AAB2C never sends a request to my Logout URL:
Since adding a new JwtIssuer throws this error :

I changed its id. This is my configuration:
<ClaimsProvider>
      <DisplayName>Local Account SignIn</DisplayName>
      <TechnicalProfiles>
        <TechnicalProfile Id="login-NonInteractive">
          <DisplayName>Local Account SignIn</DisplayName>
          <Protocol Name="OpenIdConnect" />
          <Metadata>
            <Item Key="UserMessageIfClaimsPrincipalDoesNotExist">We can't seem to find your account</Item>
            <Item Key="UserMessageIfInvalidPassword">Your password is incorrect</Item>
            <Item Key="UserMessageIfOldPasswordUsed">Looks like you used an old password</Item>

            <Item Key="ProviderName">https://sts.windows.net/</Item>
            <Item Key="METADATA">https://login.microsoftonline.com/{tenant}/.well-known/openid-configuration</Item>
            <Item Key="authorization_endpoint">https://login.microsoftonline.com/{tenant}/oauth2/token</Item>
            <Item Key="response_types">id_token</Item>
            <Item Key="response_mode">query</Item>
            <Item Key="scope">email openid</Item>

            <!-- Policy Engine Clients -->
            <Item Key="UsePolicyInRedirectUri">false</Item>
            <Item Key="HttpBinding">POST</Item>
          </Metadata>
          <InputClaims>
            <InputClaim ClaimTypeReferenceId="signInName" PartnerClaimType="username" Required="true" />
            <InputClaim ClaimTypeReferenceId="password" Required="true" />
            <InputClaim ClaimTypeReferenceId="grant_type" DefaultValue="password" />
            <InputClaim ClaimTypeReferenceId="scope" DefaultValue="openid" />
            <InputClaim ClaimTypeReferenceId="nca" PartnerClaimType="nca" DefaultValue="1" />
          </InputClaims>
          <OutputClaims>
            <OutputClaim ClaimTypeReferenceId="objectId" PartnerClaimType="oid" />
            <OutputClaim ClaimTypeReferenceId="tenantId" PartnerClaimType="tid" />
            <OutputClaim ClaimTypeReferenceId="givenName" PartnerClaimType="given_name" />
            <OutputClaim ClaimTypeReferenceId="surName" PartnerClaimType="family_name" />
            <OutputClaim ClaimTypeReferenceId="displayName" PartnerClaimType="name" />
            <OutputClaim ClaimTypeReferenceId="userPrincipalName" PartnerClaimType="upn" />
            <OutputClaim ClaimTypeReferenceId="authenticationSource" DefaultValue="localAccountAuthentication" />
          </OutputClaims>
        </TechnicalProfile>

        <TechnicalProfile Id="JwtIssuerOidc">
          <DisplayName>JWT token Issuer</DisplayName>
          <Protocol Name="OpenIdConnect" />
          <OutputTokenFormat>JWT</OutputTokenFormat>
          <UseTechnicalProfileForSessionManagement ReferenceId="SM-OAuth-issuer" />
        </TechnicalProfile>

        <!-- Session management technical profile for OIDC based tokens -->
        <TechnicalProfile Id="SM-OAuth-issuer">
          <DisplayName>Session Management Provider</DisplayName>
          <Protocol Name="Proprietary" Handler="Web.TPEngine.SSO.OAuthSSOSessionProvider, Web.TPEngine, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null" />
        </TechnicalProfile>
      </TechnicalProfiles>

    </ClaimsProvider>

I'm not sure I'm getting where I should add this XML tag:
          <UseTechnicalProfileForSessionManagement ReferenceId="SM-OAuth-issuer" />

What am I doing wrong? Thank you!

@iliassk
Thank you for the response and details.
I'll reach out to @JasSuri regarding this info and we will update as soon as possible.

It looks like you are putting everything in Base file, which already contains a technical profile called JWTIssuer. The error states that you have defined the technical profile with the ID JWTIssuer twice.
The snippets you posted are correctly configured, but instead take the config of JwtIssuerOidc and merge it with JWTIssuer. Then the last step in the userJourney should have a sendClaims step referencing JwtIssuer

Thanks for your response @JasSuri
I applied your suggestions and exported everything to my TrustFrameworkExtensions.xml :
<ClaimsProviders>
        ...
        <ClaimsProvider>
            <DisplayName>Token Issuer</DisplayName>
            <TechnicalProfiles>
                <TechnicalProfile Id="JwtIssuer">
                    <Protocol Name="OpenIdConnect" />
                    <Metadata>
                        <Item Key="token_lifetime_secs">28800</Item>
                        <Item Key="id_token_lifetime_secs">3600</Item>
                        <Item Key="refresh_token_lifetime_secs">86400</Item>
                        <Item Key="rolling_refresh_token_lifetime_secs">86400</Item>
                        <Item Key="IssuanceClaimPattern">AuthorityAndTenantGuid</Item>
                        <Item Key="AuthenticationContextReferenceClaimPattern">None</Item>
                    </Metadata>
                    <UseTechnicalProfileForSessionManagement ReferenceId="SM-OAuth-issuer" />
                </TechnicalProfile>
            </TechnicalProfiles>
        </ClaimsProvider>

        <ClaimsProvider>
            <DisplayName>Session Management</DisplayName>
            <TechnicalProfiles>
                <!-- Session management technical profile for OIDC based tokens on SLO (Single Log-Out)-->
                <TechnicalProfile Id="SM-OAuth-issuer">
                    <DisplayName>Session Management Provider</DisplayName>
                    <Protocol Name="Proprietary" Handler="Web.TPEngine.SSO.OAuthSSOSessionProvider, Web.TPEngine, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null" />
               </TechnicalProfile>
          </TechnicalProfiles>
     </ClaimsProvider>
</ClaimsProviders>

My GET /api/slo endpoint is now correctly called by the IdP but Azure AD B2C doesn't include any information about the session that should be invalidated/logged out. Usually, other IdPs call the targeted Logout URI by including the domain's browser cookies. Am I missing some configuration in my custom policy? Here are the request parameters:
:authority: localhost:443
:method: GET
:path: /api/slo
:scheme: https
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
referer: https://<tenant>.b2clogin.com/<tenant>.onmicrosoft.com/oauth2/v2.0/logout?post_logout_redirect_uri=https%3A%2F%2Flocalhost%3A5555&p=B2C_1A_custom_signup_signin
sec-fetch-dest: iframe
sec-fetch-mode: navigate
sec-fetch-site: cross-site
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36

Adding the cookies would be the browser&domain combo, B2C has no control on what cookies are presented at your endpoint. There shouldn't be anything further to configure in policy now.

Then I don't get why the cookies are added by the browser when the request is directly sent via the browser and not added when the request is sent via Azure AD B2C after a logout?

I'm not sure either, it should work with just a redirect as B2C is doing.
Wonder if its a same-site cookie issue, anything in the Browser console?

That was it.
On both Firefox and Chrome, same-site on the session cookie has to be set to None instead of Lax:
This cookie was blocked because it had the "Samesite=Lax" and the request was made from a different site and was not initiated by a top-level navigation
I guess it's blocked because the Azure AD B2C request was not initiated by a top-level navigation?
Also, for those using incognito mode, be sure to disable this:

@iliassk
I just wanted to check in and see if you still required JasSuri's assistance on this issue? Or if setting same-site to none resolved your issue.
Please let me know if you have any other questions.
Thank you again for your time and patience throughout this!

Yes, it does resolve my issue but it's probably not the best solution security-wise. I was wondering what was @JasSuri opinion about this. Is it a limitation of the way AADB2C is calling the Logout URI endpoint?
But yes, you can close the issue. Thank you for your assistance James!

@iliassk
Thank you for the confirmation! I'll go ahead and close out this issue.
@JasSuri, when you get the chance, would you be able to respond to iliassk's question above? Regarding your opinion on this issue.

I tried the same custom policy configs for single sign out.
To support single sign-out, the token issuer technical profiles for both JWT and SAML must specify:

The protocol name, such as <Protocol Name="OpenIdConnect" />
The reference to the session technical profile, such as UseTechnicalProfileForSessionManagement ReferenceId="SM-OAuth-issuer" />.

I updated the configs inextension b2c policy file and after calling the b2c end session end point url the log out url configured is not called.

@iliassk Can you tell me what end session end point url is called?Do we need to pass the parameter post_logout_redirect_uri?

For me it is working.

@jasvinder-bhomra-by how did you fix it?

I'm 4 months late on here, but I've also had this issue when configuring an Azure Static Web App to use Active Directory B2C. The 'Front-channel logout URL' doesn't seem to be getting called, and it makes not difference what I change the value to be. However, I was able to generate what appears to be a full logout by sending a GET request to https://{tenant}.b2clogin.com/{tenant}.onmicrosoft.com/{name-of-b2c-login-userflow}/oauth2/v2.0/logout?post_logout_redirect={static-webapp-url}/.auth/logout.

Hi, I've the same issue. Any news since then ?


Omnibus Launcher GUI and CLI

Description
Implemented a simple PyQT GUI and CLI for the launcher.
Developer Testing

Ran python launcher.py, GUI popped up in new window and all launcher functionalities worked as expected
Ran python launcher.py -- text, CLI showed up in terminal and all launcher functionalities worked as expected

Reviewer Testing

Run python launcher.py to see GUI
Run python launcher.py --text to see CLI

This change is

Closing because the commit history is messed up from pre-wdr merge. Need to move changes to new branch and re-open.


Do not emit Resolver type when not used

Definitely an interesting idea I will use in other projects, however, there is a slight issue that this is also generated when skipSchema is set ultimately causing 'Resolver' is declared but never used. and unable to compile the code.
cc @kamilkisiela

@kamilkisiela 👍


Added german informal "du" as new language "de-DU"

Facebook and others address their users with the german "Du" instead of the mor polite variant "Sie". This PR adds a new language "de-DU".
To use just modify the language in the main configuration file web.php
$config = [
    ...
    'language' => 'de-DU',
    ....
];

Thank you! But can you please send a pull-request which only includes translation?


Support for specifying the customer on the /api/v1/messages endpoint

Problem
Wanting to create messages that come from customers programmatically inside an existing Conversation with the REST API
Solution
Add support to the /api/v1/messages endpoint to provide a customerId
Describe alternatives you've considered
Implementing the existing websockets system from the widget
Additional context
We make a Slack bot called https://chatfox.app/ — We’re using Papercups in our web dashboard side of things but also want to integrate it inside of Slack itself. When someone is having issues they can message us in their Slack and we use the Papercups API + Webhooks to create/update conversations. Which we can then reply to via our Slack in Papercups. Its like a complex Slack to Slack system
Testing
Can you change what customer a message is from? Success

makes sense! thanks for flagging this @matthewhall-ca, we'll update it this week :+1:

hey @matthewhall-ca — we're deploying the update for this right now :+1: let us know if you have any issues with it!

@reichert621 Working PERFECT! 🎉


Improve performance of Factors with scope known at compile time.

This is an optimization feature and therefore not yet a high priority.
It may be advantageous to use factors with xt::xtensor<double, N> or perhaps xt::xtensor_fixed<double, xshape<I, J, K>> in the case where the shape of the factor is known at compile time. This is not simply a drop-in replacement for xt::xarray and will require essentially a duplication of the API for what one may call statically-shaped vs. dynamically-shaped factors.

Note also that any practical dense representation of a discrete factor will not have a high dimensionality to its scope. Whether that means we can make an arbitrary limit as low as, say, 5 variables, I'm unsure. But it seems like a plausible and useful design constraint.


todo -h fails with error

When I try to run todo -h, it fails with the following error
Traceback (most recent call last):
  File "/usr/bin/todo", line 11, in <module>
    load_entry_point('todoman==3.7.0', 'console_scripts', 'todo')()
  File "/usr/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1236, in invoke
    return Command.invoke(self, ctx)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3.8/site-packages/todoman/cli.py", line 34, in wrapper
    return f(*a, **kw)
  File "/usr/lib/python3.8/site-packages/todoman/cli.py", line 312, in cli
    invoke_command(
  File "/usr/lib/python3.8/site-packages/todoman/cli.py", line 324, in invoke_command
    click_ctx.invoke(cli.commands[command], args)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/todoman/cli.py", line 34, in wrapper
    return f(*a, **kw)
  File "/usr/lib/python3.8/site-packages/todoman/cli.py", line 645, in list
    click.echo(ctx.formatter.compact_multiple(todos))
  File "/usr/lib/python3.8/site-packages/todoman/formatters.py", line 68, in compact_multiple
    due = self.format_datetime(todo.due)
  File "/usr/lib/python3.8/site-packages/todoman/formatters.py", line 219, in format_datetime
    rv = humanize.naturaltime(self.now - dt)
TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'datetime.date'

This has been fixed in master already.
I'm releasing v3.8.0 today which includes this fix.

Thanks @WhyNotHugo.


Unable to change recovery redirect

Not sure if this is the best place to ask, apologies if not.
I need a bit of guidance in setting up the forgotten password functionality correctly. The forgotten password form sends an email which can be seen in Inbucket (I'm using a local installation of Supabase for development).
When I click on the link in the email the URL is http://localhost/auth/v1/verify?token=pkce_7ca51de7cf50d3aa016b59ca0a019609a0d46eb6c1c329138e2889e3&type=recovery&redirect_to=http://localhost:5173/
The first issue is that http://localhost/auth... should be http://localhost:54321/auth... - I don't think this is much of an issue though as I'm sure the correct url will be used once the hosted version of Supabase is used.
However what is important is being able to change the redirect URL - I need it to be the URL of where my update password form is which is http://localhost:5173/auth/update-password.  Where can this be done?
Update
I should add that the redirectTo prop value is ignored.
Update
I have edited the config.toml file with the following values and restarted supabase - still the same issue though.
site_url = "http://localhost:5173"
additional_redirect_urls = ["http://localhost:5173/auth/callback","http://localhost:5173/auth/update-password"]
Update
The redirect issue has been solved - the value I was feeding to the prop was using a variable that was not actually outputting what I thought it was and was not an actual url so the root url was used.
The email link now contains the correct redirect url, however there is now a loop - I have an onAuthStateChange listening for SIGNED_IN event - when that happens the user is redirected to the home page. So when the user is automatically logged in after clicking the email link this event is triggered and the user is redirected to the home page instead of the redirect url.
From reading the docs it seems a PASSWORD_RECOVERY event should be fired - it isn't. Maybe this is a bug in my local Supabase. I think the only way forward with this is to make use of the hosted version and go from there. I have to say that the reason I opted to use Supabase was to make things easier, but setting up this auth is a pretty painful experience.

I had the same problem(s) and just figured out that one can use the USER_UPDATED event:
supabase.auth.onAuthStateChange((event, newSession) => {
   if (event == "USER_UPDATED") router.push("/");
 });


[SPARK-13763] [SQL] Remove Project when its projectList is Empty

What changes were proposed in this pull request?
As shown in another PR: https://github.com/apache/spark/pull/11596, we are using SELECT 1 as a dummy table, when the table is used for SQL statements in which a table reference is required, but the contents of the table are not important. For example,
SELECT value FROM (select 1) dummyTable Lateral View explode(array(1,2,3)) adTable as value

Before the PR, the optimized plan contains a useless Project after Optimizer executing the ColumnPruning rule, as shown below:
== Analyzed Logical Plan ==
value: int
Project [value#22]
+- Generate explode(array(1, 2, 3)), true, false, Some(adtable), [value#22]
   +- SubqueryAlias dummyTable
      +- Project [1 AS 1#21]
         +- OneRowRelation$

== Optimized Logical Plan ==
Generate explode([1,2,3]), false, false, Some(adtable), [value#22]
+- Project
   +- OneRowRelation$

After the fix, the optimized plan removed the useless Project, as shown below:
== Optimized Logical Plan ==
Generate explode([1,2,3]), false, false, Some(adtable), [value#22]
+- OneRowRelation$

How was this patch tested?
Added a new unit test case into the suite ColumnPruningSuite.scala

cc @marmbrus @cloud-fan @dilipbiswal

Test build #52721 has started for PR 11599 at commit 0fa21ac.

Test build #52724 has started for PR 11599 at commit a31b1b5.

Test build #52721 has finished for PR 11599 at commit 0fa21ac.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52721/
Test PASSed.

Merged build finished. Test PASSed.

Test build #52724 has finished for PR 11599 at commit a31b1b5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52724/
Test PASSed.

@cloud-fan Added another two cases. Feel free to let me know if you want me to add more cases. Thanks!

Test build #52740 has started for PR 11599 at commit 68decd1.

Test build #52740 has finished for PR 11599 at commit 68decd1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52740/
Test PASSed.

nit: we need to update the title and description. Technically we can't remove Project with empty projectList, only when the child also output Nil.

Done. The title and PR description are corrected. Thanks!

Thanks, merging to master.


Chore: Utilize content collections

Chore: Utilize content collections
Upgrade: Update all dependencies

Vercel had an outage which broke the automated checks in this PR. A new PR will be opened, which should have the checks working again.


SharedTree: lint fixes

Knocked off a little low-hanging fruit.

@DLehenbauer Is my eslint upgrade PR (#4900) going to cause you any problems with these changes? Let me know if you want to coordinate.

@DLehenbauer Is my eslint upgrade PR (#4900) going to cause you any problems with these changes? Let me know if you want to coordinate.

I got a little excited and merged #4900 in. Hope it doesn't cause too much hassle.

I got a little excited and merged #4900 in. Hope it doesn't cause too much hassle.

I got a little excited and merged #4900 in. Hope it doesn't cause too much hassle.

No worries, @tylerbutler!  I'm excited to see #4900 go in too.

I got a little excited and merged #4900 in. Hope it doesn't cause too much hassle.

No worries, @tylerbutler!  I'm excited to see #4900 go in too.


XFA - An image can be a stream in the pdf (bug 1718521)

hrefs can be found in catalog > Names > XFAImages

/botio makeref

@calixteman This is causing failures all over the place, looking at the makeref logs.
Please back this out for now, and re-land once that's fixed and the tests have actually been run (since makeref without running tests first is always risky).

@calixteman This is causing failures all over the place, looking at the makeref logs.
Please back this out for now, and re-land once that's fixed and the tests have actually been run (since makeref without running tests first is always risky).

Sorry about that I thought I ran the tests...


Solid Pod

We still have to upload and fetch data from the Solid Pod.
Thus far, I've managed to fetch the user name and profile picture, and a list of contacts. We still have to figure out how to save chat messages.

Solved in pull request #45


thanks and a couple of questions

Hi @hjanuschka , thanks for writing this plugin. I have been invoking this script to upload my dsyms to NR https://gist.github.com/chrislavender/e2daf0a21f23f4b8e1d0 but would like to switch over to your plugin. I have two questions.
First related to your plugin. For dsym_path, I 'm guessing I can just pass in lane_context[SharedValues::DSYM_OUTPUT_PATH]? Can you clarify what the new_relic_upload_libs is used for? The zip file usually contains dSYMS for all dependent pods so what is the typical use of new_relic_upload_libs setting?
Second is a general fastlane question but somewhat related to New Relic uploading (have this issue even with the script I currently use). I have multiple apps in New Relic and so I specify the app ids in scheme/target specific user defined setting (called NEW_RELIC_ID) in my XCode project. In the gym log file in ~/Library/Logs/gym I can see that it initializes and  prints this variable correctly for the target being built like this:
export NEW_RELIC_ID=xxxxx
However, if I try to grab is via ENV['NEW_RELIC_ID'] it comes up blank. I 'd like to pass it to new_relic_license_key setting of your plugin.

@lakamsani

DSYM_OUTPUT_PATH should be set automatically.
new_relic_upload_libs - you can set a comma seperated list of libs wich should be included, default should be empty - wich means everything is uploaded (i guess this is the most common case)
have you tried setting the env inside the fastfile ? like  ENV['NEW_RELIC_ID'] ='xxxx'

can you run fastlane env  to get a clue about your setup ?

So looks like I can completely skip the entries dsym_path and new_relic_upload_libs and it will do the right things from the gym context (i.e. use its DSYM_OUTPUT_PATH and grab all the libs). Only new_relic_license_key and new_relic_app_name are required settings to your plugin. I will give that a shot. That definitely simplifies and brings it close to the upload_symbols_to_crashlytics action in how it works.
On the NEW_RELIC_ID, the problem is that unlike Fabric/Crashlytics New Relic requires the api key to be supplied in my app code using NewRelicAgent.startWithApplicationToken. So I need this in the XCode build settings and access it via NSBundle.mainBundle().infoDictionary.  If the same can be accessed in the lane it would be perfect. It is definitely printing it in gym logs. So I wonder why the lane can't get it. I would rather not to repeat this key via en explicit variable setting in my Fastfile. The other problem with setting to one value  in the fastfile is that I have 10 or so new relic ids. So it can't be static. For Fabric API key, I 'm indeed setting it in one place in an env file used by Fastfile.

you can try getting the value via
version_number = get_info_plist_value(path: plist_file_path, key: 'CFBundleShortVersionString')

adopting the key for sure, and checking the path to plist file.
let me know if it works for you

Thanks for the suggestion.
I ran into the problem described here. https://gist.github.com/phatblat/6f4410cbd8873738479e
and here: https://www.snip2code.com/Snippet/856253/Fastlane-Info-Plist
I need to get the plist file dynamically. Even if I get, there is just a single plist file per target. I have multiple XCode schemes per target (one for Development, one for Adhoc and one for AppStore) and each has its own new relic id. So for now I will explore grepping (via sh)  the gym build log file which is able to print the correct scheme specific value for the new relic app id.
I was however able to confirm your plugin works by specifying just the two settings new_relic_license_key and new_relic_app_name like this.
   # app is a hash with some app specific meta-data. The command to grab the nrKey always grabs the last one i.e. for the Release build config. parsing gym build logs to get the right one is the only solution I can think of for now.
    nrAppId = app['nrAppId']
    scheme =  app['scheme']
    nrKey = sh("xcodebuild -workspace ../My.xcworkspace -scheme #{scheme} -showBuildSettings | grep NEW_RELIC_ID | sed 's/[ ]*NEW_RELIC_ID = //'").chomp

    upload_symbols_to_new_relic(
        new_relic_license_key: nrKey,
        new_relic_app_name: nrAppId
    )

Here is my lane that is doing everything I need.
  desc "Upload dsyms to New Relic via upload_symbols_to_new_relic plugin"
  lane :newrelic do |options|
      nrAppId = options['nrAppId']
      scheme =  options['scheme']
#after gym command set options['logPath'] = Gym::BuildCommandGenerator.xcodebuild_log_path
      logPath =  options['logPath']
      pname = sh("xcodebuild -workspace ../My.xcworkspace -scheme #{scheme} -showBuildSettings  | grep -v FULL | grep PRODUCT_NAME | sed 's/[ ]*PRODUCT_NAME = //'").chomp
      nrKey = sh("grep NEW_RELIC_ID '#{logPath}' | tail -1 | sed 's/export NEW_RELIC_ID=//'").chomp
      puts "  #{nrKey} #{nrAppId}"
      unless nrAppId.nil?
        upload_symbols_to_new_relic(
            new_relic_license_key: nrKey,
            new_relic_app_name: nrAppId
        )
      end

  end

glad to hear  -  you got it working


ThrowsCount config excludeGuardClauses misspelled or does not exist

Using detekt 1.17.1
ThrowsCount:
    active: true
    max: 3
    excludeGuardClauses: true

Run failed with 1 invalid config property.
        - Property 'style>ThrowsCount>excludeGuardClauses' is misspelled or does not exist.

Run failed with 1 invalid config property.
- Property 'style>ThrowsCount>excludeGuardClauses' is misspelled or does not exist.

I'm sorry but I'm unable to reproduce (I've tried on detekt's main to add the same config you posted).
Could you paste a link to your project or a reproducer?

If you comment out excludeGuardClauses in detekt.yml in https://github.com/eygraber/detekt-rules you should be able to repro it.

Thanks for providing a reproducer @eygraber 🙏
Seems like you stumbled upon an undocumented feature: https://github.com/detekt/detekt/pull/3774#discussion_r631998983
This was fixed on main. Will be released on 1.18.0.
For the time being you can suppress the config file validation by adding this to your config file:
config:
  ...
  excludes:
    'style>ThrowsCount>excludeGuardClauses'


Getting Involved

Hey Connor,
I notice you labeled this project as open source and I was interested in getting involved. What's the best way to do that?
For some context :
I'm a structural engineer by trade who's pursuing their masters in computer science ( I hold a bachelor and masters in Structural Engineering as well). I initially started programming because I was interested in developing my own analysis software.
I'm mainly proficient in Java and Python but have some C# knowledge and can continue learning if you do need help with the project.
Thanks!

Hey @jcooper67,
That is awesome to hear. Sounds like we have very similar backgrounds.
Yeah, this repo is, at the moment, pretty much all c#. That decision was made because this is the language that many apis in the AEC industry are written in such as Revit, ETABS, and Tekla Structures. I even chose to use Blazor as the frontend framework because being able to push the c# execution to the client for heavy structural calcs will definitely save some money for me on the server hosting.
However, many structural engineers who have enough programming abilities to use something like this are probably more comfortable in Python. It would be interesting to see if a python SDK would be valuable to people.
This repo needs a ton of work in order for it to actually be useful. Unfortunately, I'm not really organized enough to let you know exactly what needs worked on at the moment, but I'm definitely open for suggestions if you see something that you want to work on. I'm happy to help you understand and navigate the existing code if you have any specific questions. I know it can be intimidating to make a PR to someone's repo, but you probably know more than I do about comp science so I wouldn't worry about that. I'm happy to help you understand and navigate the existing code if you have any specific questions.
In terms of non-coding involvement, I'm always on the lookout for good examples of solved text-book problems that I can use to create tests. I've mostly been working on implementing the Direct Stiffness Method for solving indeterminate structures, and I haven't found a ton of great examples that deal with 3D frames.

Thanks for the reply! Totally agree on your implementation C# is definitely the way to go for something like this especially with how calculation heavy it will be. I'm still pretty fluent in C# for the most part so I should be able to piece everything together. I'll dive into it when I get some good free time.
As for examples I actually have some matrix structural analysis books I can dig through. They're somewhat old but I'll see if I can find a PDF for you.

@connorivy I've got some examples from this book. I can send them to you or I can scan them and push them to the repo, whichever you prefer. Also, I'm having an issue building, is there a place I can ping you quickly to clarify some things? I'm sure I'm just misunderstanding the way Blazor apps run.


huggingface device_map="auto" doesn't work for multi-gpu

Hey there! I'm trying to get a simple LLM example going with multiple gpus, but for some reason when using device_map="auto" I run into cuda errors. The model works fine when I only use 1 gpu. Any ideas?
Here's the example I'm using:
import time

from modal import Image, method, gpu, Stub
from pathlib import Path

huggingface_model_registry = {
    "vicuna": "lmsys/vicuna-7b-v1.5",
}

stub = Stub(name="test")

def download_model():

    from transformers import AutoModelForCausalLM

    # download all models
    for _, model in huggingface_model_registry.items():
        model = AutoModelForCausalLM.from_pretrained(
            model,
            token="...",
            trust_remote_code=True
        )

stub.container_image = (
    Image.debian_slim(python_version="3.10")
    # .apt_install("git", "gcc", "clang", "build-essential")
    .apt_install("git")
    # .env({
    #     "CUDA_LAUNCH_BLOCKING": "1",
    #     "TORCH_USE_CUDA_DSA": "1"
    # })
    .pip_install(
        "scipy",
        "protobuf==3.20.*",
        "accelerate~=0.25",
        "transformers",
        "torch",
        "torchvision",
        "torchaudio",
        "openai",
        "tqdm",
        "pyyaml",
        "vllm==0.2.6",
        "anthropic",
        "pandas",
        "sentence-transformers",
        "numpy",
        "pytz",
        "wandb",
        "fschat[model_worker,webui]",
        "einops",
        "google-generativeai")
    .run_function(download_model)
)

@stub.cls(image=stub.container_image, gpu=gpu.A100(memory=80), container_idle_timeout=300)
class Vicuna:
    def __enter__(self):

        import torch
        from transformers import AutoTokenizer, AutoModelForCausalLM

        self.model_name = "lmsys/vicuna-7b-v1.5"
        print("loading tokenizer")
        self.tokenizer = AutoTokenizer.from_pretrained(self.model_name, token="...")
        print("loading model")
        self.llm = AutoModelForCausalLM.from_pretrained(
            self.model_name,
            trust_remote_code=True,
            torch_dtype=torch.float16,
            device_map="auto",
            token="..."
        )

        # self.llm.eval()

        self.max_tokens = 500
        self.temperature = 1
        self.top_p = 0.7

    @method()
    async def generate(self, input, history=[]):
        inputs = ["test"] * 10
        input_ids = self.tokenizer(
            inputs,
            return_tensors="pt",
            padding=True).to("cuda")

        print("generating....")
        output_ids = self.llm.generate(
                **input_ids,
                max_new_tokens=self.max_tokens,
                temperature=self.temperature,
                top_p=self.top_p,
                do_sample=self.temperature != 0)
        print("finished generating...")

        if not self.llm.config.is_encoder_decoder:
            output_ids = output_ids[:, input_ids["input_ids"].shape[1]:]

        completion_texts = self.tokenizer.batch_decode(output_ids, skip_special_tokens=True)

        return completion_texts

@stub.local_entrypoint()
def main(input: str):
    llm = Vicuna()
    sentence = llm.generate.remote(input)
    print(sentence)
    return sentence

Changing the gpu count results in the following error:
RuntimeError: CUDA error: mapping of buffer object failed
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

, and compiling with those flags on doesn't help either. Can confirm the model works on the cpu alone.

Hey @lithafnium, this could be a runtime issue on our part, affecting multi-GPU applications. We're actively working on it, and thank you for the reproduction script.
If you reach out in our Slack, https://modal.com/slack, I can put your workspace on our legacy runtime to unblock you.

Hey @lithafnium, using your script I can confirm that it works on our legacy runtime. Please reach out with your workspace name and I'll enable the legacy runtime for your workspace.
In the meantime, we're working on the issue.

@lithafnium we've merged a fix for this, and checked that your reproduction script now works with this multi-GPU configuration: @stub.cls(image=stub.container_image, gpu=gpu.A100(count=2, memory=80), container_idle_timeout=300)
Closing this issue as fixed :)

Hi @lithafnium ,
I am new to Modal. Where are the model weights stored when you run "download_model" at image creation time? Is this in some volume?
Thanks


Add validation functions and fixup sanitizer build

This PR adds 2 new functions to validate encoded streams. These are useful when loading data from untrusted sources (disk, network, etc...) since it's typically trivial to change the uncompressed size that's stored alongside the
encoded data. This has caused crashes for us where the decode functions have trusted the corrupted size and ended up reading into an unmapped page.
I was split between doing it this way or adding a streamvbyte_decode_safe() which also took in the size of the input, but this appeared to be the simpler of the two for error handling (at the expense of having to traverse the data twice if you need to check and then decode).
Also as part of this PR I've fixed up the ASAN build so that it links (which caught a leak in the tests), fixed the Makefile build (presumably not used?), and fixed a typo. See individual commits for more detail.

Fantastic work! Running tests.

Looks like the macOS jobs are failing because the macos-11 runners were removed: https://github.blog/changelog/2024-05-20-actions-upcoming-changes-to-github-hosted-macos-runners/. Should I bump that to macos-latest or a specific version?

The speed of this ended up being a big performance hit on macOS because AppleClang16 doesn't appear to be able to unroll a loop. Giving it a hand significantly improves performance and I've added a basic NEON version that gives the same performance (at least under AppleClang16). Note that I haven't added an x64 version since we see no performance impact on Linux (using GCC, clang untested but should be able to unroll the loop on x64 too).
I've also changed how basictests creates its data since for each test it always produced numbers that had the same encoding width, so it would pass even if keyPtr was never incremented.

@blawrence-ont Don't worry about some tests failing in CI for technical reasons. We can fix that in some other ways.

@blawrence-ont Could you add a tiny bit of documentation in the README file ? Only a couple of sentences would be enough. If you don't do it, we can do it later.

Merging!


Attribute priority: first wins

Accoridng to the HTML spec when it comes to duplicate attributes the first defined attribute is the one that wins. Subsequent attributes of the same name are ignored:

I just want to confirm that we are adhering to this as well in the SwiftUI client.

Core is used to fetch attributes by name with the getAttribute function on Node, so I think this issue should be moved there.


Package Manager: Add button for installing dependencies

(Especially useful after the upgrade all button.)

I think this is not a good idea for two reasons:

the package manager isn't meant to be something messy. It shouldn't have too many buttons (it already has two)
every time you add/remove a package, there is no need to call yarn install again, it deals with it automatically


Uncaught ReferenceError: VueClipboard

Hello, this my code attempting to set up use of Vue clipboard, note this is cdn because its a completely standalone page with no back end what so ever. I know nothing about webpack or any of the other massively complex packages i've seen that actually support these "20 lines or less" examples.
  <script type="text/javascript" src="https://unpkg.com/vue"></script>
  <script type="text/javascript"<EMAIL_ADDRESS>  </script>
<script>

 Vue.use(VueClipboard)

When I load the page i get this in the console
 Uncaught ReferenceError: VueClipboard is not defined
    at html:80

How do I actually reference the code you have packaged here? Why does Vue work just fine from a cdn but I cannot reference anything in the second one?

You used wrong package vue-clipboards which is not vue-clipboard2

Ok that did in fact resolve the VueClipboard reference error. Its digging into code from the CDN. Do you know where I can resolve an issue with vue-clipboard.js using the "require" function? Or am I just wasting my time trying to get this to work in a stand alone environment? I believe that technically vue-clipboard isn't you so I don't want to ask you to fix it but "require" is node.js according to my google search.

Here is a example using standalone js file: http://jsbin.com/behecujoze/edit?html,output

Yay it works! Thanks!


Add status badges to README.md

Coverage remained the same at 100.0% when pulling 559f960bb6ccdb71dc0ee58471f00afa94307629 on add_badges_to_readme into 874da2880858669318392d8ba79705e3153e749b on master.


Bad version name in jw.util package

Bad source in Weblate
Link to the source string: https://pypi.org/project/jw.util/-class.-jw.util.version.Version-/
Description of the problem: There is a strange version that seems wrong in the versions list of jw.util package in PyPI repository.
Link to the release history of the package - you can see that this versions is in the bottom of the list.
Please let me know if this version name is normal.
Thanks!

Thanks for the report. This is no longer a valid version, but was possible at the time the release was created (2014) and thus is just a historical artifact.


[Suggestion] Laser Redirection Cube

With the upcoming laser improvements from #11 , the possibility for laser redirection cubes opens up. They'd be a new entity (not compatible with regular cubes or enemies), and would redirect lasers to the angle the player points at when placing. This would require some radical changes to how lasers behave (such as being angled in non-grid-aligned angles).

What do you mean by "not compatible with regular cubes"? What makes this distinction?

I mean that, since this item will interact with lasers and with the player in unique ways, these behaviors can't be applied as enemy behavior, unless with some serious modifications.


Using CopyFiles with CleanTargetFolder fails if the target folder does not exist

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name:  CopyFilesV2
Environment

Server - Azure Pipelines

If using Azure Pipelines, provide the account name, team project name, build definition name/build number: DevDiv, DevDiv, https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=6535057&view=logs&j=3dc8fd7e-4368-5a92-293e-d53cefc8c4b3&t=c961bbc5-e1d7-56a7-1309-8966a4feaaf3&s=6884a131-87da-5381-61f3-d7acc3b91d76

Agent - Private

If using private agent, provide the OS of the machine running the agent and the agent version:  Windows Server 2019, Current agent version: '2.206.1'

Issue Description
This PR seems to have regressed using the CopyFilesV2 task with CleanTargetFolder: true when the folder doesn't exist. I believe this change is the cause since the function call switched from ignoring ENOENT to throwing.
Turning off CleanTargetFolder resolves the issue.
Task logs
Starting: Copy localization files
==============================================================================
Task         : Copy files
Description  : Copy files from a source folder to a target folder using patterns matching file paths (not folder paths)
Version      : 2.208.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/copy-files
==============================================================================
found 878 files
Cleaning target folder: D:\a\_work\1\a\loc
##[warning]Entry "D:\a\_work\1\a\loc" does not exist
##[error]Error: Entry "D:\a\_work\1\a\loc" does not exist
Finishing: Copy localization files

Note: 2.200.0 was working for us yesterday, and today it seems that 2.208.0 is being used, so I do believe the linked PR is the root cause since the version bumped from 2.200.0 -> 2.208.0 in that PR.
Successful run:
Starting: Copy localization files
==============================================================================
Task         : Copy files
Description  : Copy files from a source folder to a target folder using patterns matching file paths (not folder paths)
Version      : 2.200.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/copy-files
==============================================================================
found 878 files
Cleaning target folder: D:\a\_work\1\a\loc
...

Thanks for working on this so quickly. Wondering about an ETA on this fix, since many of our builds are currently failing due to it.

@delmyers we are going to roll out the rollback today.
@tydunkel thank you for your pull request with the fix. We will review and merge it before the next release of the tasks.

Confirmed this morning that 2.208.1 is pulled which contains the revert.

@tydunkel, let me close this issue since your PR with the fix was merged


Gempak Colormaps

Add parser for gempak colormap files. Not sure how best to integrate this with matplotlib's colormap + norm machinery.

Right now the registry just returns the pair. Don't know if we really need more than that. Bigger question now is, how does the GEMPAK format differ from our current format.


Working with React Form and Handling Event

In this branch I:

[x] Added checkboxes to the Todo items;
[x] Implemented a delete button for the Todo items;
[x] Added a text input field and submit button;
[x] Implemented a random ID generator for the Todos list items;

Hello Vanessa  😇,
You actually have done great work even without finalizing the whole project yet like:-

write a descriptive README file.
the commit message is written well.
Follow the best practice of git-flow.
No linter errors.
The application working well with the same functionalities as you described in the pull request description.

Well Done Vanessa  🎉🎉 I think this pull request is ready to merge now ✔


Support for selecting the ACME preferred chain

This is an optional parameter to indicate the preferred chain to retrieve from ACME when finalizing the order.
This is applicable to Let's Encrypts recent migration https://letsencrypt.org/certificates/_ to their
own root, where they provide two distinct certificate chains (fullchain_pem vs. alternative_fullchains_pem);
he main chain will be the long chain that is rooted in the expiring DTS root, whereas the alternative chain is rooted in X1 root CA.
Select "X1" to get the shorter chain (currently alternative), leave blank or "DST Root CA X3" for the longer chain.

Coverage increased (+0.06%) to 60.765% when pulling e48f4ffe7748bc838e9e0ceb654dec61fdbf49d1 on hosseinsh:acme-preferred-chain into df17337075fee5ab7dbb414fa0019751760613aa on Netflix:master.


Yt-dlp cannot fetch video list information in a channel of bilibili

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
china
Provide a description that is worded well enough to be understood
Yt-dlp cannot fetch video list information in a channel of bilibili.
It can still downloads single bilibili video.
yt-dlp https://space.bilibili.com/1984487658/video
[BilibiliSpaceVideo] Extracting URL: https://space.bilibili.com/1984487658/video
[BilibiliSpaceVideo]<PHONE_NUMBER>: Downloading JSON metadata
[BilibiliSpaceVideo]<PHONE_NUMBER>: Downloading page 0
ERROR:<PHONE_NUMBER>: An extractor error has occurred. (caused by KeyError('page')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
I'm using the latest yt-dlp.
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [615a84447] (pip)
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
D:\myweb\test\turtle_diary_test>yt-dlp -vU https://space.bilibili.com/1984487658/video
[debug] Command-line config: ['-vU', 'https://space.bilibili.com/1984487658/video']
[debug] Encodings: locale cp936, fs utf-8, pref cp936, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [615a84447] (pip)
[debug] Python 3.8.10 (CPython AMD64 32bit) - Windows-10-10.0.26085-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg git-2020-02-18-ebee808, ffprobe git-2020-02-18-ebee808, rtmpdump 2.3
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {'http': 'http://<IP_ADDRESS>:10809', 'https': 'http://<IP_ADDRESS>:10809', 'ftp': 'http://<IP_ADDRESS>:10809'}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1803 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[BilibiliSpaceVideo] Extracting URL: https://space.bilibili.com/1984487658/video
[BilibiliSpaceVideo]<PHONE_NUMBER>: Downloading JSON metadata
[BilibiliSpaceVideo]<PHONE_NUMBER>: Downloading page 0
ERROR:<PHONE_NUMBER>: An extractor error has occurred. (caused by KeyError('page')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "C:\Python\lib\site-packages\yt_dlp\extractor\common.py", line 732, in extract
    ie_result = self._real_extract(url)
  File "C:\Python\lib\site-packages\yt_dlp\extractor\bilibili.py", line 1069, in _real_extract
    metadata, paged_list = self._extract_playlist(fetch_page, get_metadata, get_entries)
  File "C:\Python\lib\site-packages\yt_dlp\extractor\bilibili.py", line 973, in _extract_playlist
    metadata = get_metadata(first_page)
  File "C:\Python\lib\site-packages\yt_dlp\extractor\bilibili.py", line 1058, in get_metadata
    page_size = page_data['page']['ps']
KeyError: 'page'

Cookies to need to be passed to yt-dlp. The error message has been improved


please support to post json body just use string content

In the next release https://github.com/rybalkinsd/kohttp/tree/release/0.7.0#post-with-json-body

Feature implemented in 0.7.0 release
In your case:
body(type = "application/json") {
    string("your json content")
}

Also checkout docs for more details
@SeanCai


NODDI recon producing weird values for ICVF and ISOVF

Hello!
I have the following DTI scheme:
-> Loading data:
	* DWI signal
		- dim    = 108 x 129 x 109 x 96
		- pixdim = 1.800 x 1.800 x 1.800
	* Acquisition scheme
		- 96 samples, 2 shells
		- 6 @ b=0 , 60 @ b=2000.0 , 30 @ b=1000.0
	* Binary mask
		- dim    = 108 x 129 x 109
		- pixdim = 1.800 x 1.800 x 1.800
		- voxels = 206359

Images were acquired on a GE scanner
Two separate sequences were acquired:

b=1000 with 30 directions, in one phase direction,
then b=2000 with 60 directions in opposite phase direction

QSIprep was used to preprocess the images and they are concatenated into one .nii.gz file (which is default behaviour for QSIprep)
running AMICO recon runs no problem
However, the ISOVF and ICVF look like they are being miscalculated in some regions over others, in a very obvious way. Best way to visualize is with an image:

What am I doing wrong?

Hi @weberam2 how does the distortion correction look? There are b=0 images from both phase encoding directions, right? I also wonder if there's something odd with the colormap in your image plotting tool. What does the histogram of ICVF and ISOVF values look like inside the brain mask?

Hi @mattcieslak
Thanks for getting back to me!
"how does the distortion correction look?" It looks great!
"here are b=0 images from both phase encoding directions, right?" Correct: The two scans are as follows:

b=0 x 3, then b=1000 x 30, all in AP direction
then a separate scan: b=0 x 3, then b=2000 x 60, all in PA direction

" I also wonder if there's something odd with the colormap in your image plotting tool. What does the histogram of ICVF and ISOVF values look like inside the brain mask?" No, this does not seem to be the case. Here is the histogram from ICVF:

I do have an update:
I managed to do the preprocessing with MRtrix3:
mrcat sub-Rett01_dir-AP_dwi.mif sub-Rett01_dir-PA_dwi.mif all_dwis.mif -axis 3
dwidenoise all_dwis.mif all_dwis_denoise.mif -noise noise.mif
dwifslpreproc all_dwis.mif dwi_preprocessed.mif -rpe_header -eddy_options " --slm=linear "

And when I run AMICO NODDI, my images look like this (and the histogram)
ICVF:

So I do believe it is something happening with QSIprep preprocessing...
Any suggestions of what we could try? I am happy to share the dwi .nii.gz / .json and the anat .nii.gz / .json files

with default settings the qsiprep workflow is extremely similar to the mrtrix preprocessing pipeline. I wonder if it is the b=0 image harmonization, which isn't present in the mrtrix version of the workflow. Could you try rerunning preprocessing with --no-b0-harmonization?

I'm running that now, and will let you know the results
But in the mean-time, I also realize the preprocessing doesn't finish properly

230217-21:39:41,928 nipype.workflow ERROR:
could not run node: qsiprep_wf.single_subject_Rett01_wf.dwi_preproc_wf.pre_hmc_wf.raw_rpe_concat

But it still outputs the preprocessed files:
sub-Rett01_space-T1w_desc-eddy_cnr.nii.gz
sub-Rett01_space-T1w_desc-preproc_dwi.bvec
sub-Rett01_desc-SliceQC_dwi.json
sub-Rett01_space-T1w_desc-preproc_dwi.b
sub-Rett01_space-T1w_desc-preproc_dwi.nii.gz
sub-Rett01_space-T1w_desc-brain_mask.nii.gz
sub-Rett01_space-T1w_desc-preproc_dwi.bval
sub-Rett01_space-T1w_dwiref.nii.gz

more info of the error:
Traceback:
        Traceback (most recent call last):
          File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run
            runtime = self._run_interface(runtime)
          File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/nilearn.py", line 134, in _run_interface
            new_nii = concat_imgs(self.inputs.in_files, dtype=self.inputs.dtype)
          File "/usr/local/miniconda/lib/python3.8/site-packages/nilearn/_utils/niimg_conversions.py", line 487, in concat_niimgs
            for index, (size, niimg) in enumerate(zip(lengths, _iter_check_niimg(
          File "/usr/local/miniconda/lib/python3.8/site-packages/nilearn/_utils/niimg_conversions.py", line 160, in _iter_check_niimg
            raise ValueError(
        ValueError: Field of view of image #1 is different from reference FOV.
        Reference affine:
        array([[-1.79477441e+00, -6.63911998e-02,  1.19901352e-01,
                 1.28985382e+02],
               [-8.74731839e-02,  1.76716912e+00, -3.30843806e-01,
                -9.69735107e+01],
               [ 1.05513543e-01,  3.35716337e-01,  1.76523221e+00,
                -1.38833572e+02],
               [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
                 1.00000000e+00]])
        Image affine:
        array([[-1.79477453e+00, -6.63908795e-02,  1.19907245e-01,
                 1.28985336e+02],
               [-8.74728560e-02,  1.76716900e+00, -3.30862254e-01,
                -9.69734955e+01],
               [ 1.05512999e-01,  3.35716873e-01,  1.76532757e+00,
                -1.38833649e+02],
               [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
                 1.00000000e+00]])
        Reference shape:
        (140, 140, 80)
        Image shape:
        (140, 140, 80, 33)

What version are you using? There was a similar error a few versions ago

What version are you using? There was a similar error a few versions ago

pennbbl/qsiprep:0.16.1

@mattcieslak is there a solution to this when the source data has different orientation?
@weberam2 I've seen this in some data as well. If you have the DICOM headers, you can compare ImagePositionPatient and ShimSetting - in affected data I've seen, there's been a re-shim and change in alignment of the image FOV.

Just a quick update, I ran the QSIprep with the option: --no-b0-harmonization
and still get the bad NODDI images

@mattcieslak is there a solution to this when the source data has different orientation?
@weberam2 I've seen this in some data as well. If you have the DICOM headers, you can compare ImagePositionPatient and ShimSetting - in affected data I've seen, there's been a re-shim and change in alignment of the image FOV.

OK, so the first series: REL Image Position Patient//-121.315-144.362-69.2205
The second: REL Image Position Patient//-127.311-127.82\19.0426
For ShimSetting, I can't find anything with that in the dicom files...

Sorry, I got mixed up - ShimSetting is in the BIDS JSON sidecar

I don't see the ShimSetting in there either.
Here are both .json files:
{
    "Modality": "MR",
    "MagneticFieldStrength": 3,
    "ImagingFrequency": 127.773,
    "Manufacturer": "GE",
    "InternalPulseSequenceName": "EPI2",
    "ManufacturersModelName": "DISCOVERY MR750",
    "InstitutionName": "BCCH MRI Research Facility",
    "DeviceSerialNumber": "00000604875MR750",
    "StationName": "MR01OC01",
    "BodyPartExamined": "BRAIN",
    "PatientPosition": "HFS",
    "SoftwareVersions": "27\\LX\\MR Software release:DV26.0_R06_2132.a",
    "MRAcquisitionType": "2D",
    "SeriesDescription": "DTI B1000_30dir_AP",
    "ProtocolName": "RESEARCH - AMRETT",
    "ScanningSequence": "EP\\SE",
    "SequenceVariant": "NONE",
    "ScanOptions": "CL_GEMS\\SAT_GEMS\\EDR_GEMS\\EPI_GEMS\\HYPERBAND_GEMS\\ACC_GEMS\\PFF\\FS",
    "ImageType": [
        "ORIGINAL",
        "PRIMARY",
        "OTHER"
    ],
    "SeriesNumber": 7,
    "AcquisitionTime": "10:07:41.000000",
    "AcquisitionNumber": 1,
    "SliceThickness": 1.8,
    "SpacingBetweenSlices": 1.8,
    "SAR": 0.219551,
    "EchoTime": 0.082,
    "RepetitionTime": 4.1,
    "FlipAngle": 90,
    "PhaseEncodingPolarityGE": "Unflipped",
    "CoilString": "Nova32ch",
    "MultibandAccelerationFactor": 3,
    "PercentPhaseFOV": 100,
    "PercentSampling": 100,
    "AcquisitionMatrixPE": 140,
    "ReconMatrixPE": 140,
    "ParallelReductionFactorInPlane": 2,
    "EffectiveEchoSpacing": 0.000382072,
    "TotalReadoutTime": 0.053108,
    "PixelBandwidth": 3571.43,
    "PhaseEncodingDirection": "j-",
    "ImageOrientationPatientDICOM": [
        0.997097,
        0.048596,
        0.0586183,
        -0.0368838,
        0.981761,
        -0.186509
    ],
    "InPlanePhaseEncodingDirectionDICOM": "COL",
    "ConversionSoftware": "dcm2niix",
    "ConversionSoftwareVersion": "v1.0.20211006",
    "Dcm2bidsVersion": "2.1.6"
}

{
    "Modality": "MR",
    "MagneticFieldStrength": 3,
    "ImagingFrequency": 127.773,
    "Manufacturer": "GE",
    "InternalPulseSequenceName": "EPI2",
    "ManufacturersModelName": "DISCOVERY MR750",
    "InstitutionName": "BCCH MRI Research Facility",
    "DeviceSerialNumber": "00000604875MR750",
    "StationName": "MR01OC01",
    "BodyPartExamined": "BRAIN",
    "PatientPosition": "HFS",
    "SoftwareVersions": "27\\LX\\MR Software release:DV26.0_R06_2132.a",
    "MRAcquisitionType": "2D",
    "SeriesDescription": "DTI B2000_60dir_PA",
    "ProtocolName": "RESEARCH - AMRETT",
    "ScanningSequence": "EP\\SE",
    "SequenceVariant": "NONE",
    "ScanOptions": "CL_GEMS\\SAT_GEMS\\EDR_GEMS\\EPI_GEMS\\HYPERBAND_GEMS\\ACC_GEMS\\PFF\\FS",
    "ImageType": [
        "ORIGINAL",
        "PRIMARY",
        "OTHER"
    ],
    "SeriesNumber": 8,
    "AcquisitionTime": "10:10:26.000000",
    "AcquisitionNumber": 1,
    "SliceThickness": 1.8,
    "SpacingBetweenSlices": 1.8,
    "SAR": 0.219551,
    "EchoTime": 0.082,
    "RepetitionTime": 4.1,
    "FlipAngle": 90,
    "PhaseEncodingPolarityGE": "Flipped",
    "CoilString": "Nova32ch",
    "MultibandAccelerationFactor": 3,
    "PercentPhaseFOV": 100,
    "PercentSampling": 100,
    "AcquisitionMatrixPE": 140,
    "ReconMatrixPE": 140,
    "ParallelReductionFactorInPlane": 2,
    "EffectiveEchoSpacing": 0.000382072,
    "TotalReadoutTime": 0.053108,
    "PixelBandwidth": 3571.43,
    "PhaseEncodingDirection": "j",
    "ImageOrientationPatientDICOM": [
        0.997097,
        0.0485962,
        0.0586186,
        -0.036884,
        0.981761,
        -0.186509
    ],
    "InPlanePhaseEncodingDirectionDICOM": "COL",
    "ConversionSoftware": "dcm2niix",
    "ConversionSoftwareVersion": "v1.0.20211006",
    "Dcm2bidsVersion": "2.1.6"
}

If I run diff sub-Rett01_dir-AP_dwi.json sub-Rett01_dir-PA_dwi.json on these I get:
15c15
<     "SeriesDescription": "DTI B1000_30dir_AP",
---
>     "SeriesDescription": "DTI B2000_60dir_PA",
25,26c25,26
<     "SeriesNumber": 7,
<     "AcquisitionTime": "10:07:41.000000",
---
>     "SeriesNumber": 8,
>     "AcquisitionTime": "10:10:26.000000",
34c34
<     "PhaseEncodingPolarityGE": "Unflipped",
---
>     "PhaseEncodingPolarityGE": "Flipped",
45c45
<     "PhaseEncodingDirection": "j-",
---
>     "PhaseEncodingDirection": "j",
48,50c48,50
<         0.048596,
<         0.0586183,
<         -0.0368838,
---
>         0.0485962,
>         0.0586186,
>         -0.036884,

Ah, I see you have a GE scanner, I'm looking at Siemens data here. I guess the ShimSetting might be a Siemens-specific output from dcm2niix.
Anyway, it looks like your patient orientation is identical between series - this is good because it means the bvecs should have the same coordinate system. Sorry to derail your thread!

Ah, I see you have a GE scanner, I'm looking at Siemens data here. I guess the ShimSetting might be a Siemens-specific output from dcm2niix.
Anyway, it looks like your patient orientation is identical between series - this is good because it means the bvecs should have the same coordinate system. Sorry to derail your thread!

Don't be sorry! I'll take any help I can get. I like the idea of QSIprep and would like to see it work...

@weberam2 do you know if the mrtrix pipeline is bias correcting the images individually before running eddy? I wonder if there is something happening in the bias correction that is making the b>0 images have very different intensities in them across the two scans.
QSIPrep will refuse to concatenate images with different phase encoding directions before denoising (which for us means unringing, mp-pca and b1 bias correction). Maybe --dwi-no-biascorr in addition to --no-b0-harmonization will make this less of a problem?

That worked!!
So, this is what I ran:
docker run --rm -v /mnt/WeberLab/license.txt:/opt/freesurfer/license.txt:ro -v $PWD:/data:ro -v $PWD/derivatives/FastSurfer:/sngl/freesurfer-input:ro -v $PWD/derivatives:/out -v $HOME/work:/scratch pennbbl/qsiprep:0.16.1 /data /out participant --freesurfer-input /sngl/freesurfer-input --output-resolution 1.8 --participant-label sub-Rett01 -w /scratch --no-b0-harmonization --dwi-no-biascorr

and then
docker run --rm -v /mnt/WeberLab/license.txt:/opt/freesurfer/license.txt:ro -v $PWD:/data:ro -v $PWD/derivatives/qsiprep:/qsiprep-output:ro -v $PWD/derivatives:/out -v $HOME/work:/scratch pennbbl/qsiprep:0.16.1 /data /out participant --recon-input /qsiprep-output --recon-spec amico_noddi --participant-label sub-Rett01 --recon-only -w /scratch

and now I get nice NODDI images (here is the ISOVF):

I still have some questions though:

What are the negative consequences of using --dwi-no-biascorr and --no-b0-harmonization for my preprocessing?
My qsiprep preprocessing still doesn't spit out a boilerplate, even though it does seem to finish the preprocessing (attaching the crash report). Why is that?
crash-20230219-013059-root-raw_rpe_concat-42fbb6b6-65ca-4d00-be47-71d3bd6bfdbe.txt

What are the negative consequences of using --dwi-no-biascorr and --no-b0-harmonization for my preprocessing?

B0 harmonization is designed to reduce signal drift over time. I believe this was inspired by the HCP pipelines, where there are typically four DWI series taking a total of 20-30+ minutes, that are separated in time (ie, not acquired back to back). It scales each series by a constant factor, and with many b=0 volumes in each series, should be fairly reliable.
The DWI bias correction estimates a spatial multiplicative correction caused by B1 inhomogeneity. The bias field affects SIFT results. In mrtrix, this is applied after running topup and eddy
https://mrtrix.readthedocs.io/en/latest/fixel_based_analysis/mt_fibre_density_cross-section.html

QSIPrep will refuse to concatenate images with different phase encoding directions before denoising (which for us means unringing, mp-pca and b1 bias correction). Maybe --dwi-no-biascorr in addition to --no-b0-harmonization will make this less of a problem?

This seems different to what mrtrix does, and a bit worrying. If the bias field is estimated independently between series, it can provide a different correction both temporally and spatially. Subsequent b=0 harmonization could account for a global scaling between the series, but it won't affect spatial differences in the bias field. There's also the possibility that brain masking will be different between series, which will also change the bias field.

Should we move the bias correction to after the resampling? Thinking more about it now, I'm leaning towards moving it. In addition to the other downsides you mention I wonder if it may also confuse uncorrected signal pileup and dropout for B1 inhomogeneity, which is obviously also bad.

I wonder if it may also confuse uncorrected signal pileup and dropout for B1 inhomogeneity, which is obviously also bad.

It definitely will, this is a great point. For optimal performance, outliers should be winsorized. However, if the bias field is less than perfect, the impact of any errors will be mitigated if it is applied once to the whole data set.


fixed rendering layers Bubbles + Points

Description
Points + Bubbles Layering inconsistency affecting chromatic checks.
This PR seperates Points and Bubbles into separate layers, ensuring Points layers on top of bubbles
Video:

Multiple storybook refreshes showing points layered above bubbles
2 browser refreshes showing points layered above bubbles

https://github.com/user-attachments/assets/39435e78-941e-408c-b064-09f38553e45a
Checklist

[x] For UI or styling changes, I have added a screenshot or gif showing before & after
[x] I have added a changeset
[ ] I have added to the docs where applicable
[ ] I have added to the VS Code extension where applicable

@zachstence looking into this now

We are creating new panes depending on bubbles or points, to ensure the points are layered on top of bubbles.

Users can use the new paneType and z prop to control multiple layers of points or bubbles

Note:
If users have 2 points in a basemap without panteType and z, the map layers the points pseudo-randomly (I believe whichever point renders first). We can try to automate this for this for the user.

@kwongz whats the syntax for pane type and z? And what would the user be specifying with those?

fixes #2560

@hughess
Syntax: (layers blue points on top of red)
<Points ... paneType="blue points" z='2'/>
<Points ... paneType="red points" z='1'/>
Docs: panetype accepts strings (spaces included), z accepts numbers

additional:
Solves layering click issue, where clicking blocks other points/bubbles/areas
Current Issues:
https://github.com/user-attachments/assets/f2207fb3-3544-42ad-bc7f-88aa26ea4648
This PR Solves:
https://github.com/user-attachments/assets/1059c71d-17a5-4ed7-a901-fac3240a5b3f

@hughess I would also suggest removing the double-click zoom on areas points and bubbles.
Select and deselect can invoke double-click zoom, which is a minor annoyance. I can imagine  users':

selecting + deselecting
map zooms in
The user needs to zoom out to access other areas of the map
OR user just keeps zooming deeper and deeper while interacting with the map

I think bubbles and points should not be too much of a problem, but since areas can cover a large portion of the map, this would mean user would have to double-click outside of any area layers (e.g. bodies of water, non-area layers on the map etc). Users can always use the scroll to zoom if needed.

Is there any way for the component to automatically choose which layer should be on top? I'm thinking the easiest thing from a user perspective would be ordering the nested components by the order they want them to appear. E.g.:
<!-- Points appear on top because they are first in the order below -->
<BaseMap>
   <Points/>
   <Bubbles/>
</BaseMap>

@hughess
Ah, i understand, sorry to clarify,

Currently without z or paneType, points will layer on top of bubbles. By default we check the chart type, and if it's points, it's assigned to the points pane (z-index 401), and if it's bubbles it on the bubbles pane (z-index 400).

the reasoning for this is regardless of how they are structured, leaflet would rendered which ever loaded the data first (I believe). resulting in storybook, and docs pages pseudo-randomly layer points or bubbles on top of each other.

the reasoning for paneType prop and z is for @zachstence situation where users may want to differentiate and layering of multiple points or multiple bubbles

The separation of panes also solves the onClick blocking issue, if we render on the same pane and depend on the ordering, we break the ordering and cause a layering issue

@hughess
Spoke to Zach about a new handling for layers.

Default Points layer on top of bubbles
automatically handle paneType for every point/bubble component used in basemaps
removed paneType prop
Use z to handle multiple points and bubbles

Default points on top of bubbles
<Bubbles data={la_locations} lat="lat" long="long" pointName="point_name" value="sales" size="sales" tooltipType="hover" opacity="0.6" />
<Points data={la_locations} lat="lat" long="long" color="red" />

Red Points will always layer on top
<Points data={la_locations} lat="lat" long="long" color="red" z="3" />
<Points data={la_locations} lat="lat" long="long" color="purple" z="2" />
<Points data={la_locations} lat="lat" long="long" color="blue" z="1" />

Is there any way to detect the ordering of the child components, or do we need the user to tell us?

fixes #2613

Is there any way to detect the ordering of the child components, or do we need the user to tell us?
I'll look into this!

@zachstence  @hughess
Bubble and Point layering now respect order user defines them in Basemap automatically.
-Example
The order of layering from bottom - top = [bubbles (yellow), points (green), points(red)]
`<BaseMap title="My Map" height="300">
	<Bubbles {...$$restProps} color="yellow"/>
	<Points {...$$restProps} color="green" />
	<Points {...$$restProps} color="red" />
</BaseMap>`

-How it Works

We first register the components in the order they are  written in an array
We create seperate panes for each component and add their points/bubbles to their pane
We give those Panes a z-index based on their order in the array

-Changes

Removed paneType and z prop
No longer handle points over bubbles (up to user design)
Panes solve onClick layering issue

This is perfect!

removed props from docs


Conda install Tensorflow 2.1 with mkl (avx/avx2 support) not working

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
TensorFlow installed from (source or binary):  tensorflow 3.1 with mkl binary from Anaconda installation
TensorFlow version: 3.1.0
Python version: 3.7.6
Installed using virtualenv? pip? conda?: conda
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: na
GPU model and memory: na

I was using tensorflow 2.0 with mkl (avx/avx2 optimization) and has zero problem. I just upgraded to tf 2.1 using Anaconda. After upgrade, tf complains that "Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2".
To make sure I upgraded correctly. I uninstalled all tensorflow components, and then do a fresh install using command conda install tensorflow-mkl. The installation succeed but the problem persists.
when I run the same training script, I saw the performance is worse than the 2.0. So, I guess the avx2 is not used in 2.1

@wangzhizhengduke, Can you provide the standalone complete code to analyze the issue. Thanks!

@wangzhizhengduke, Can you provide more information to understand the issue better. Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

I have tensorflow2.0 with mkl. Can you please help here?
Here's the code:
import tensorflow as tf
from tensorflow import keras

input_A = keras.layers.Input(shape=[5], name="wide_input")
input_B = keras.layers.Input(shape=[6], name="deep_input")
hidden1 = keras.layers.Dense(30, activation="relu")(input_B)
hidden2 = keras.layers.Dense(30, activation="relu")(hidden1)
concat = keras.layers.concatenate([input_A, hidden2])
output = keras.layers.Dense(1, name="output")(concat)
model = keras.models.Model(inputs=[input_A, input_B], outputs=[output])

Here's the output:
2020-12-12 17:06:46.037846: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2020-12-12 17:06:46.040928: I tensorflow/core/common_runtime/process_util.cc:147] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.

Please let me know how I can assist in debugging this issue.
Here's the mkl check from https://software.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation-guide.html
Code:
major_version = int(tf.__version__.split(".")[0])
if major_version >= 2:
   from tensorflow.python import _pywrap_util_port
   print("MKL enabled:", _pywrap_util_port.IsMklEnabled())
else:
   print("MKL enabled:", tf.pywrap_tensorflow.IsMklEnabled())

Output:
MKL enabled: True


JS (buildArea.html) - add table functionality. Append parts and prices.

Table will appear below the 6 images on build.html.

Scott drove, Jon navigated for a bit.

Elizabeth drove, Jon navigated for a bit.

Final testing in process. Closing issue as goal achieved.


MultidimensionalArray Push

thank you so much for providing this amazing cls!
I love this feature very much
dim arr1(1 to 2) as variant,arr2(1 to 1) as variant
arr1(1,1)=1 : arr1(1,2)="A"
arr1(2,1)=2 : arr1(2,2)="B"

arr2(1,1)=1 : arr2(1,2)="C"

dim arr as new BetterArray
arr.items=arr1
arr.concat arr2

Thank you :)

Thank you :)


Mac development B: Add Delete Button on Node GUI

Macs do not have a Del button

Now all Nodes have a Del button (as well as an info button and separate buttons for the netwrok set up and the logging)


[MD] How to get the list of contacts

How to get the list of contacts

How to get the list of contacts

https://github.com/adiwajshing/Baileys/issues/1015

thanks


Marketplace

one-click app. marketplace. think about these things from the beginning. let your mockups be the forcing function for community building before trying to sell the first set.

The marketplace needs to have several functions built in. Buy booster packs, gems, sleeves, skins, avatars, etc. Auction cards you don't need. Trade for cards you want. Show your library, or at least cards you want to sell in an interactive gallery where the potential buyer can  pick on the card to interact and make an offer. Sell from the gallery if the offer is agreed on. Say if the hidden reserve is met by the offer at the end of the auction.

The marketplace needs to have several functions built in. Buy booster packs, gems, sleeves, skins, avatars, etc. Auction cards you don't need. Trade for cards you want. Show your library, or at least cards you want to sell in an interactive gallery where the potential buyer can  pick on the card to interact and make an offer. Sell from the gallery if the offer is agreed on. Say if the hidden reserve is met by the offer at the end of the auction.

The gems could be tokens and they could be used internally for all transactions, but minting tokens on a side chain and buy/selling to ADA can get complicated in ways you may not want to deal with. They could, for example, be used to arbitrage the  changing price of ADA. There could end up being some regulatory impact to using them. Need to research further into this function.

Port over some comments now living in the core section and continue the discussion here. Xsolla has all the earmarks of a complete solution to the revenue generation mechanic that we need. They have even partnered with Bitpay and thus have the ability to handle Bitcoin transactions. Not ADA. Not yet. Inevitable? Yes.
This still doesn't address the issues around selling gems or trading items between players. However, the player inventory system appears to have the foundation for this feature when integrated with Cardano smart contracts. Also, it appears the inventory system tracking all the cards a player would have in their inventory appears also to be handled within tools Xsolla offers. How well could it be to integrate remains to be seen, however.

When you are ready to discuss whether ADA will be supported in their integration application, email<EMAIL_ADDRESS>to begin that conversation.

You can't give Xsolla 5% of the auctions and trades between players. At most, you would be taking 1% for processing those in a safe and secure manner.

There would also have to be a way to pass a token for an NFT if you are going to use it in the same forging system that produces copies of other cards.

There's a big problem with having thousands of cards in your library if you are going to be storing that information on the blockchain. If on a side chain, who takes care of the absolute security of that information? Customer inventory and virtual currency balance cannot ever be put at risk. It must retain the utmost security with very frequent backups. (Such as every time there is a change in the account?)

Game modes and revenue generation. In addition to selling packs, there has to be a way to generate revenue in an ongoing basis. Entry fees for tiers and game modes might be one way to do this.
Win or lose, you gain experience? Maybe just gain it when you win? Gain more when you win? Experience would just be a way to measure how much you've played. The leaderboard would be more about how many games you've won.
If you divide by seasons, you can charge an entry fee for each season that brings in fresh $ADA into the system. You then offer unlocks for something that is really hard to earn, but could be reached faster if you bought some of that unit (such as crystals, gems, prismatic gems, etc.).
You would also be able to unlock these items by winning (harder, but you eventually get there). Reference the MTG: Arena store.
The key is that you can buy or earn gems, but you can only ever spend them. You can't extract them from your inventory and use them elsewhere (this saves some very complicated issues).
The gold and gems you earn remain captured in the system (I'm not sure how, but I'm fairly certain this avoids certain regulatory issues with handling market tradeable tokens of value) and are used for the microtransactions within. (yes, including vanity items and skins).

There is a serious financial matter that has to be considered with regard to the functionality of the marketplace. What problems do we bear in creating an environment where people can buy gems in cash and sell them into ADA? Where do we get the ADA to fill those trades? What regulatory issues are we tripping over when the application goes parabolic? Taxes? If we use XSolla, what would we be setting ourselves up for? Would we ever consider selling gems back into cash?

When it comes to creating incentives for people to keep playing and keep spending into the system, certainly rewarding success in the different elements is a natural way to accomplish this. The addition of being able to make money reselling cards and NFTs has to also be a part of the story.

Consider a subscription option that would ensure consistent and continued support. This would open the door for giving out keys for that subscription as well. One month, three months. Some interval of time to get used to the benefits of the subscription vs Free play options. All of this can tie into the Xsolla platform offering as well.

One way to solve the problem of winning or buying or trading into NFTs might be to preload your Receive address, but they change whenever they get used.

Until there are safe APIs that can interface with wallets, I don't think there's a solution other than copy/paste receive address. Probably, you should use Buffy Bot, or at least the application, even if it gets reskinned for the game.


Change Log – Binance API Documentation

https://binance-docs.github.io/apidocs/delivery/en/#change-log

Hi,
Can I help in API documentation?
Thanks!


fix(config): remove system params from config file

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
As title. Closes #7860. The removed config will be acceptable from meta node CLI.
Documentation

My PR contains user-facing changes.

Types of user-facing changes
Please keep the types that apply to your changes, and remove the others.

Installation and deployment
RisingWave cluster configuration changes

Release note
System parameters are no longer accepted in the configuration file. Instead, they can be initialized from the CLI of the meta node. Here is a list of the removed configs:

Name
Section
Argument

sstable_size_mb
storage
--sstable-size-mb

block_size_kb
storage
--block-size-kb

bloom_false_positive
storage
--bloom-false-positive

data_directory
storage
--data-directory

checkpoint_frequency
streaming
--checkpoint-frequency

barrier_interval_ms
streaming
--barrier-interval-ms

storage_url
backup
--backup-storage-url

storage_directory
backup
--backup-storage-directory

It seems the end-to-end test becomes much slower after this PR. Would you please help to take a look? 🤔


Adding support for maskedPan and Details fields

Is your feature request related to a problem? Please describe.
When looking at some account types, I noticed that the account fields maskedPan and details are not passed through the library, this PR adds support for these types.
A sample sanitised API response from the /api/v2/accounts/:id/details/ endpoint:
{
    "account": {
        "resourceId": "resource_id",
        "currency": "GBP",
        "ownerName": "Account User",
        "cashAccountType": "CARD",
        "status": "enabled",
        "maskedPan": "XXXX-XXXXXX-12345",
        "details": "A Sample Card"
    }
}

Describe the solution you'd like
Adding support for the maskedPan and details items in the API
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context

Noticed that this field is present and populated in AMEX data feeds

Was resolved in #174


Add additional copilot studio hunter directories to package data

This fixes issues with fuzzing.  Still having some problems with puppeteer.

Thanks, will review and check the Puppeteer issues as well.


Rename rounded-* to radius-*, switch to numeric scale

This PR changes the name of our border-radius classes from rounded-* to radius-*.

Old Class
New Class

rounded-none
radius-0

rounded-sm
radius-1

rounded
radius-2

rounded-lg
radius-4

rounded-full
radius-full

The primary motivation for this is to make the naming mirror the CSS property more closely, and because I was constantly typing rounded-0 instead of rounded-none, and the whole idea of something being "rounded but not" is just stupid sounding.
This PR also removes the idea of a default border radius value (the rounded) class in favor of always specifying a modifier. That approach was okay with the name rounded but the class radius doesn't make as much sense without a modifier.
I've switched to a numeric scale because radius-0 is more intuitive to me than radius-none, but we could also just use 0 for the zero value, and a descriptive scale like sm, md, lg, etc. for the other values. Don't have a strong opinion on that either way, especially because the numeric scale I'm using in the PR at the moment doesn't match our spacing scale or anything anyways.
If we do keep a numeric scale, I'm not sure if it should be proportional like I have right now (notice that 3 is missing), or if it should just be radius-1, radius-2, radius-3.
Interested in any feedback on this :+1:

Maybe I'm missing something obvious, but what was the context in which you were:

constantly typing rounded-0 instead of rounded-none

I don't think I've ever used rounded-none.

I am slightly pro rounded variants since in web development we are primarily using it to round the corners on things (cards, buttons, etc). radius variants do map to the CSS property, but I mentally associate those more with circles, somethings that -- aside from radius: 999 to make a circle avatar -- doesn't come up that often. Part of the joy of Tailwind is that I don't need to care as much about the CSS property names and I like the flow of composing classes based on how you would talk: "a button with a blue background (bg-blue), white text (text-white) and rounded corners (rounded)".

I agree with @swanson that "rounded" is the expected usage in this context. However, also I agree with @adamwathan that we should consider renaming the rounded-none class to something else. I would propose not-rounded (similar to the not-italic class which was recently merged) or hard-edges.

I don't think I've ever used rounded-none.

It's mainly to remove a radius at another screen size, e.g. rounded sm:rounded-none.
It seems that the general direction for 1.0 is to as close to the metal (CSS) as possible while maintaining a great developer experience. I don't really mind these little abstractions, but I think I prefer consistency, and most class names reflect the underlying CSS property in some way. I was also not a fan of the rounded-none class, and I'd rather have a consistent prefix for utilities that affect the same property so I'm not a fan of not-rounded either. If you really like rounded and this gets merged, you can always add it as a key to the config, and have a radius-rounded class that would be like your default border-radius.
The only name that strikes me as odd now is radius-full. rounded-full felt like it meant "fully rounded", but if we're talking about a really high border-radius, I think I prefer radius-max.

The only name that strikes me as odd now is radius-full. rounded-full felt like it meant "fully rounded", but if we're talking about a really high border-radius, I think I prefer radius-max.

Yeah I could be persuaded to change that, I kept full because we use that in a lot of other places to mean "100%", which is what we are doing here as well (at least in spirit, damn you 9999px hack.)

Well, border-radius: 100% is valid and gives a different result (except for square elements, they become circles in both cases).

...and gives a different awful result

Fixed it for you.

Haha it creates ovals. That can work in some designs...

For unsetting the border radius I would propose no-radius or radius-unset (and unset seems also a valid value for the property)
For the scale naming is very subjective, but I would stick to sm, md, lg, as switching to radius-1 would not be closer to the value that will be assigned to the property

If we do keep a numeric scale, I'm not sure if it should be proportional like I have right now (notice that 3 is missing), or if it should just be radius-1, radius-2, radius-3.

@adamwathan The scale should be proportional since it will come given by the micro-spacing scale (regardless this is something explicitly defined in Tailwind core or not). widths and heights will come given by the macro-spacing scale.
Here's the first link I found to a design system I remembered defined it this way: https://www.carbondesignsystem.com/guidelines/spacing/

The only name that strikes me as odd now is radius-full. rounded-full felt like it meant "fully rounded", but if we're talking about a really high border-radius, I think I prefer radius-max.

+1 to @benface's note on full making sense for the class with value 100%. Ovals, ovoids, and ellipses are a thing so let's just

For unsetting the border radius I would propose no-radius or radius-unset (and unset seems also a valid value for the property)

This could also be an option but since we're not using initial, inherit, and unset on other modules, I don't think it makes sense to introduce it now.
Also, imho this values should be given by the micro-spacing scale of the system, 0 would be a value of that scale while unset would not. Design tokens > CSS things.

I like the rounded classes. My vote is to leave it as-is. I usually only need one or two variants anyway.

Going to keep everything as-is 👍


VS code Resize Slightly

Does this issue occur when all extensions are disabled?: Yes

VS Code Version:
Version: 1.73.0 (user setup)
Commit: 8fa188b2b301d36553cbc9ce1b0a146ccb93351f
Date: 2022-11-01T15:34:06.111Z
Electron: 19.0.17
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19044
Sandboxed: No
OS Version:
Edition	Windows 10 Pro
Version	21H2
Installed on	2021-03-13
OS build	19044.1766
Experience	Windows Feature Experience Pack 120.2212.4180.0

Steps to Reproduce:

Start VS Code
Click X button on the right-top
Repeat (1 and 2)
The more times repeating the procedure, the bigger VS Code window size is gonna be.

The size grows really slightly.
I realized that I need to resize the window every week.
So I repeated it for 3 minutes, and I recorded a video as well.

Unable to repro the issue with latest stable 1.74


Fixed issue where a 200 response is returned when server throws exception

Added handling of all exceptions in LambdaContainerHandler.proxyStream() to return an error rather than a 200.
Issue #, if available: #273
Description of changes:
If an exception is thrown that is not explicitly handled by the LambdaContainerHandler.proxyStream() method an empty 200 will be returned. I've only seen this behavior using the Jersey handler but I believe this will also occur in the other handlers. The Jersey handler returns empty 200s if HK2 is unable to resolve dependencies. If the service framework is unable to start I'd expect a 500 to be returned.

Thanks for the PR, @mikaelq. This does seem to be a bug. However, we should not just marshal all exceptions and return them. The correct way to implement this is to rely on the ExceptionHandler interface. Can you modify this pull request to follow a similar logic to the proxy() method. Also, we'll need unit tests to verify the behavior. I'll open an issue to track this.

@sapessi  thanks for the quick reply. Oops, I didn't see proxy() method, it handles all exceptions. My initial fix did not work but I found what the issue is.
The issue is the JerseyServletResponseWriter.failure() eats failures ultimately returning a 200. I modified it to throw an InternalServerError exception. I verified this fixes the issue via unit tests. I did not add integrations tests. I'll try to do this soon.

I'll merge this manually in the core branch where I'm working on other changes.

I've merged this manually in the core branch. Thanks for the fix @mikaelq!