[WIP] Automated deployment

We're super excited about your automation work @gavinbarron! @LuiseFreese asked me to validate the work and provide feedback, and overall I'm delighted that this drastically reduced the deployment time!
We did notice some small issues and like to ask for some additional functionality, so we can reuse this script for upgrades as well.
First run: cannot create the "templates" container in the storage account
Whatever I do, the script always fails to create the "templates" container for me. This of course fails the rest of the deployment. I worked around it by manually creating it, and then rerunning but that's not a great workaround :

)
Script parameters + documentation
The script takes already a SubscriptionId parameter, and we'd like the option to optionally provide a ResourceGroup parameter too.
Given both optional parameters, we'd think this could be made more clear:

Explain both optional parameters in the documentation, and make it clear how they can be used
Ask for confirmation in the script for the parameters like "does this look correct?", and only on approval we start the script

Rerunning of the script
We'd like the script to be idempotent, so you can run it even if a previous run has occurred. This is helpful when parts of the deployment failed, or when the ARM template gets updated in an upgrade scenario.
Currently, there are some situations where that isn't happening as clean as we'd like it to be.
With the app registration:

Also, it creates a NEW client secret on every run and keeps the old one. We feel like the old one should be removed, but can't really assess the impact. Any idea?
With the uploading/updating of the template files

With adding permissions to the managed identity

Downloading of the template files
Looking into this whole approach, we started wondering if we really need the deployment storage account. Given that we already copy all the files with a GIT checkout to the CLI storage account, can't we just use it from there?
We're unsure if this is a good idea or not, so we appreciate your input and ideas regarding this.
Let us know what you think about all of this? We are already more than grateful for the work you already did, so if you don't feel like taking up these improvements (all or part of them), just tell us. We can share the work if you'd like.

Thanks for the feedback @YannickRe :)
Can't create the container.

Can you please validate that the account you are using has the Contributor role assignment at the subscription level? I ran into this when I had an account with the classic owner permission but no modern permission assigned. I believe that permission level is called out as a pre-requisite at present.

Optional ResourceGroup parameter

❤ I had this in there, but took it out based on feedback.

Script idempotency

Cool, happy to make this better and all of that makes sense to me.
The upload of templates is probably failing as the container and SasToken step failed, might need to set this script up with $ErrorActionPreference = Stop, thoughts?

Downloading template files
Options:

We could likely get the Blob URL for the templates cloned into cloud shell and use that
I believe we can use the raw github url for the main branch as the input to the ARM deployment which simplifies the script a whole lot. Will investigate and confirm.

Assuming that 2 is feasible it would be my preferred approach, but I'll differ to you and @LuiseFreese as your preference here and go that way
And I'm more than happy to help get this wrapped up.

Can't create the container.
Can you please validate that the account you are using has the Contributor role assignment at the subscription level? I ran into this when I had an account with the classic owner permission but no modern permission assigned. I believe that permission level is called out as a pre-requisite at present.

Ok, that makes sense. I only have classic administrator rights, not a modern permission level assigned. I'll try again with that permission. It's ok if this is a prerequisite for now, but in that case I would propose two things:

we document this prerequisite and the associated error message clearly
we try and assign this permission on the Resource Group level to circumvent this

Script idempotency

The upload of templates is probably failing as the container and SasToken step failed, might need to set this script up with $ErrorActionPreference = Stop, thoughts?

It only fails the second time (first time works after manually creating the container), it doesn't fail to get the SAS token (but that doesn't matter as it isn't used, because the script uses the --login option). I will see if this is fixed with the previous point when adding a modern Contributor permission level.

Downloading template files
Options:

We could likely get the Blob URL for the templates cloned into cloud shell and use that
I believe we can use the raw github url for the main branch as the input to the ARM deployment which simplifies the script a whole lot. Will investigate and confirm.
Assuming that 2 is feasible it would be my preferred approach, but I'll differ to you and @LuiseFreese as your preference here and go that way
I let @LuiseFreese make the executive decision, but for me, it makes sense to go with Option 1. Option 2 is great but would always use the latest version from GitHub, and sometimes you'd like to checkout a moment in time and rerun it after days or weeks. I like it better when I checkout that point in time version and keep it on my CloudShell storage account.

And I'm more than happy to help get this wrapped up.
Thank you, this is highly appreciated!

There's another usecase that I think bears considering.
With using the -deploy Resource Group and storage account users who have PowerShell + Azure CLI installed locally can choose to run the script locally and it will "just work".
If we move to the assumption that this is only ever run from CloudShell as Option 1 locks in we stop these users from running the script locally.

First of  all thank you @gavinbarron for putting so much hard work into this ❤ Your expertise is HIGHLY appreciated - this will ease so much for everyone who deploys ProvisionGenie 🧞!
Also thank you @YannickRe for reviewing, testing and bringing your expertise to the table!
You both take this - yet again - to another level 🚀
my feedback:

container-issue: any prerequisites need to be documented and we should also give a hint in the script. assigning the correct permissions would be amazing.
Optional ResourceGroup parameter: yes please, we just need to change this then in the ARM template as well
Downloading template files: after a quick check with Yannick I think option 1 is the best for us at the moment. Do you @CarmenYsewijn have anything against that?

Thanks again everyone, I am very much grateful for all your help 🙏

I did some digging. There's a dedicated PowerShell cmdlet to get all the info about the storage account from cloud shell Get-CloudDrive This gives us everything we need to provision the SAS token and use the templates in the cloned repo.
Added PR #64 to fix the issues raised in here

Hi @gavinbarron , thank you for the awesome work and the quick turnaround! We're in the final stretch, just tested and noticed some small things (and one unfortunate thing)...
ManagedIdentity rerun issue
You adjusted the code to check for existing permission assignments, but it currently does not work. This is due to the following:
$currentRoles = (az rest `
>>     --method get `
>>     --uri https://graph.microsoft.com/v1.0/servicePrincipals/$principalId/appRoleAssignments `
>>     | ConvertFrom-Json).value `
>>     | ForEach-Object { $_.appRoleId }

returns
19dbc75e-c2e2-444c-a770-ec69d8559fc7
9492366f-7969-46a4-8d15-ed1a20078fff
23fc2474-f741-46ce-8465-674744c5c361
bf7b1a76-6e77-406b-b258-bf5c7720e98f
62a82d76-70ea-41e2-9197-370581804d09
0c0bf378-bf22-4481-8f81-9e89a9b4960a

but the following $teamCreate = az ad sp show --id $graphId --query "appRoles[?value=='Team.Create'].id | [0]" returns "23fc2474-f741-46ce-8465-674744c5c361" and as a result, the matching fails: $roleMatch = $currentRoles -match $appRoleId
Non-existing subscription id
If you provide a subscription id that does not exist, it tells you that it does not exist and then continues with the default subscription, instead of canceling. We prefer this "issue" to stop the execution.
ProvisionGenieApp as optional parameter
We'd like to provide the option to deploy ProvisionGenie multiple times in the same Azure, and as such we noticed that ProvisionGenieApp better not be hardcoded as an app name.
Could you maybe add this as an optional parameter?
CloudShell as a storage account
I had a chat with Luise, and we agree with your assessment that switching to CloudShell would disable the option to run the deployment locally. Additionally, the script fails when using a different subscription id because it cannot find CloudShell storage account IN that subscription etcetc.
So after a quick chat with @LuiseFreese , it looks like the best course of action is to revert the usage of CLoudShell and go back to the dedicated separate storage account.
We do like to change one thing tho, and feel like this storage account should be created in the main "ProvisionGenie" resource group instead of its own -deploy group.
I'm very sorry to not have assessed this better/sooner 😔

great work, guys! @YannickRe and @gavinbarron!
here is one puzzle tile 🧩 to solve the ManagedIdentity rerun issue: we can replace
$teamCreate = az ad sp show --id $graphId --query "appRoles[?value=='Team.Create'].id | [0]"
by
$teamCreate = az ad sp show --id $graphId --query "appRoles[?value=='Team.Create'].id | [0]" -o tsv
so that it outputs Tab-separated values, with no keys.
for reference see Output formats for Azure CLI commands
@YannickRe tested this successfully already 🥂

AAD App Name
Greate idea, easy enough.
Storage
Sure, I can revert that and target the Storage Account to be inside the resource group being deployed.
And don't worry about the churn on that, I got to learn stuff with the change.
I'll change the permission assignment to grant the user running the script Contributor on the Resource Group being deployed which will prevent the issue we saw with classic admins.
Managed Identity Permissions
Weird, I tested the Managed Identity permissions piece a few times, admittedly it was a little late. Shouldn't be too hard to make it work right.
Managed Identity Location
Not sure if you spotted that I'd changed the location of the Managed Identity to be passed down from the parent template rather than hard coded to westeurope. This could be considered a breaking change 😦 as anyone who has deployed with the Managed Identity in westeurope and their ResourceGroup elsewhere attempting to re-deploy will have the deployment fail as you can't change the location of a resource.
This can be reverted if you think it necessary, but it seemed like an error when I spotted it last night.

Storage

Good idea.

Managed Identity Permissions

No worries. Just adding the --output tsv to fetching the permission scope id's (all of them) will fix this.

Managed Identity Location

I saw, and it is accepted 😀. For future deployments, it is better that this is not hardcoded.

PR #65 should address the items raised thus far :)

@gavinbarron impressive work! I tested and retested it, tried many different parameter combinations and it now seems super robust and stable. Thank you for this effort, the team and I appreciate it enormously!
There are two more things before we can merge:

@LuiseFreese / @carmenysewijn will need to do an addition documentation check, to make sure the docs are still end-to-end consistent with the new approach and all the changes we made. It looks like it from my side, but I prefer a second opinion
The script changes the order of some installation instructions, and we suspect this has an impact but we haven't been able to fully pinpoint it. Before we merge, we need to make sure what it is and what our approach is to handle this. We'll circle back asap!

I'm so excited to get this done: the timesaving is gigantic🚀

Oh, @LuiseFreese do we need the AAD App registered before deploying the Dataverse pieces?
In that case there are two options:

Add a pause using a Read-Line input check into the script that so that we can create the app registration and then stop until the user gives input to continue execution
Split the script in two and output the ClientId. ClientSecret, TenantId from the first for input into the second

@gavinbarron This went a bit faster than expected 😊.
First the issue: We need the app registration first to make the Application user in Dataverse, otherwise the Logic Apps can't register their webhooks with the Dataverse tables.
@LuiseFreese already restructured the docs to accommodate for this, to make it more clear for you what needs to happen. It seems you noticed from the commits 😁😉.
Indeed, those are the two options that we discussed and the decision fell on the second one. Not sure we need the tenant id as input? We could assume it from the logged-in user, if necessary?
Our idea:

Run the App Registration script, output client id / secret (/ tenant id?)
Go to Dataverse, do the required steps
Run the second script with the rest of the deployment, taking client id and secret as input

We have been asking a lot from you already, so if this is too much: we can do this part too. Please don't feel obliged, we're already super grateful for your contributions.

+1
super happy, grateful and very much impressed on this. This totally rocks!

I'll make the changes to split the script later tonight :)

I realized that we can differ resetting the client app credentials until during the main deployment.
This should allow leaving the current parameter list to the main deployment script unchanged :)
Hopefully this approach works.

@gavinbarron What a great realisation! I glanced over the commit, and it sure looks like this should do the trick.
I will test further this evening and then we can et this thing merged for all to use 😀. Great work, again!

@gavinbarron Made some final tweaks:

We don't need subscription information to do the app registration, it doesn't depend on a subscription so I removed that from the first script
The app registration result has a password property instead of a clientCredential property, and a tenant property instead of a tenantId property. I adjusted the script for that.
Passing the app registration properties into the ARM deploy command required extra encapsulation with $(), so I adjusted the script for this.

It all works now, the whole thing is deployed in minutes and a new entry in Dataverse kicks off the provisioning!
Amazing work! I'm very excited for this, it's so much easier now!
Thank you for the effort, it was great working with you on this ♥.
THis leaves the executive decision with @LuiseFreese and @CarmenYsewijn to merge it 😁.

WRT subscriptionId and tenantId.
The challenge I was allowing for becomes apparent when you have CLI access to multiple Subscriptions with different AAD tenants.
If you're looking to target a subscription and tenant other than the one associated with the current default subscription then you cannot without either having the SubscriptionId parameter to switch the active subscription, or manually doing so beforehand.

Oh oh, that's a scenario I didn't think about. The different subscriptions doesn't really matter, but multiple tenants might become an issue 😀. Let me reset that quickly...

What an amazing work! Thank you so much! I love how you elegantly splitted the script and how you envisioned scenarios that at least didn't had in mind in the first place (but that made so much sense!). Thank you for bearing with all of our improvement-requests and for contributing to this open-source project. This only exists because of community and I am happy that you are part of it. I couldn't be happier to have your support and I am incredibly grateful that you put so much effort into this.
A big thank you also on behalf of the entire team 💖

☝☝ what she said, so it is now merged 😁


New version: JetBrains.dotUltimate version 2024.2.2

Created by 🥟 Dumplings in workflow run #7620.

Log
Updated: 2024.2.1 → 2024.2.2
Submitting WinGet manifests
Creating manifests
Uploading manifests and making commits
Creating a pull request

Microsoft Reviewers: Open in CodeFlow


UI: Allow admin/user to filter by state and other parameter for resources

As an operator, I want to see the resources in the UI by a particular state. Refer to the specific list API parameters and implement a drop-down filter to allow filtering by state or some other API param/field:

Account: filter by state https://github.com/apache/cloudstack/pull/7365
Hosts: filter by host state
Cluster: filter by allocation state
Pods: filter by allocation state
Systemvms: filter by state
Routers: filter by state
Storage pool: filter by state (this is UI + API fix to have a new param added)

ISSUE TYPE

Improvement Request

COMPONENT NAME

UI, API

@kishankavala can you implement this minor UI improvement for 4.18 branch, for hosts, clusters, pods
@vishesh92 can you implement this minor UI improvement for 4.18 branch, systemvms, routers and storage pool?

Both PRs are raised to address this issue, closing.


i am not find databinding.ActivitySharedelementBinding class

Hello dear
Please help me.
I am not find derectory com.lgvalle.material_animations.databinding
and class related to databinding like (ActivitySharedelementBinding)

put :
dataBinding {
enabled = true
}
on your gradle:app


Adding flask route for malaria model and required frontend pages for the same

Feature request

Add a new html page for malaria disease and integrate it with malaria model present in model section. This page will take image as the input and give result .[...]

I would like to work if assigned.

Assigned @ayush-raj8 🚀

Please review and merge

@kritikaparmar-programmer  @utkarsh0702 please update with proper level and update my score.


Support the workflow under Windows.

The current release does not work under Windows.
There are a few known issues:

bazel and all the required dependencies are cumbersome to setup in Windows.
some path handling in J2CL does not work well with windows paths.

Detailed documentation on how to setup under Windows would be very helpful and address the first point.
For now we recommend to use WSL under Windows 10.

Does anyone have any plans to make this work with Maven or any other tool that works everywhere or it really can work only with Basel ?

The GWT Community has started work on Maven and Gradle plugins. They are designed towards GWT 3 though, not "pure J2Cl".

Detailed documentation on how to setup under Windows would be very helpful and address the first point.

As a start, I can describe my WIN 10 Bazel setup based on Using Bazel on Windows.
Would external-link to "md file" be useful?

@vojkog Did you try everything explained in README.md of J2CL ? If yes, was everything successful on Win 10 for you or not ?

@markposterone see #7

Basel is possible to install on Windows as pointed by @vojkog. The GWT community is working hard on Maven but it seams that nobody is looking into "some path handling in J2CL does not work well with windows paths" as they expect that creators of J2CL will take a look at this problem.

We would like the community to take on making J2CL work in Windows natively. None of the core developers uses Windows machines. We do not consider this a blocker since it works under WSL in Windows 10.
W.r.t the path handling I referred to in the previous comment is mostly constrained to the the reading of the source files. It is very localized and should be pretty straightforward to debug and fix.

Could we update the closure-compiler to version to v20190215 to solve
"SourceFileTest.testSourceFileResolvesZipEntries failes for Windows #3206" issue?!

What's the status of this ?

I am working on updating Closure rules workspace to work with the newer Closure release; should be completed next week.

Closure compiler version is updated to latest release in https://github.com/bazelbuild/rules_closure/pull/347.

As of 2021Jan13, I attempted to setup the j2cl in Windows 10. Following the "Getting Started" guide and installing bazel for Windows 10, the setup was successful. But, I saw an error related to "zipper.exe Argument list too long".
d:\j2cl\samples\helloworld> bazel build src/main/java/com/google/j2cl/samples/helloworld:helloworld
INFO: Analyzed target //src/main/java/com/google/j2cl/samples/helloworld:helloworld (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: C:/users/myuser/_bazel_myuser/myku6orf/external/com_google_j2cl/jre/java/BUILD:45:16: error executing shell command: 'C:/msys64/usr/bin/bash.exe -c set -eu
shopt -s extglob
zip_args="c bazel-out/x64_windows-fastbuild/bin/external/com_google_j2cl/jre/java/jre_java_files.srcjar"
for src in "$@"
do
relative_name=${...' failed (Exit 126): bash.exe failed: error executing command C:/msys64/usr/bin/bash.exe -c ... (remaining 304 argument(s) skipped)
: line 11: external/bazel_tools/tools/zip/zipper/zipper.exe: Argument list too long
Target //src/main/java/com/google/j2cl/samples/helloworld:helloworld failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.781s, Critical Path: 0.27s
INFO: 3 processes: 3 internal.
FAILED: Build did NOT complete successfully
Error message seems to be related to bazel and may not related to j2cl. Someone please shed light on this.
Does anyone faced this issue ?

As of 2021Jan13, I attempted to setup the j2cl in Windows 10. Following the "Getting Started" guide and installing bazel for Windows 10, the setup was successful. But, I saw an error related to "zipper.exe Argument list too long".
d:\j2cl\samples\helloworld> bazel build src/main/java/com/google/j2cl/samples/helloworld:helloworld
INFO: Analyzed target //src/main/java/com/google/j2cl/samples/helloworld:helloworld (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: C:/users/myuser/_bazel_myuser/myku6orf/external/com_google_j2cl/jre/java/BUILD:45:16: error executing shell command: 'C:/msys64/usr/bin/bash.exe -c set -eu
shopt -s extglob
zip_args="c bazel-out/x64_windows-fastbuild/bin/external/com_google_j2cl/jre/java/jre_java_files.srcjar"
for src in "$@"
do
relative_name=${...' failed (Exit 126): bash.exe failed: error executing command C:/msys64/usr/bin/bash.exe -c ... (remaining 304 argument(s) skipped)
: line 11: external/bazel_tools/tools/zip/zipper/zipper.exe: Argument list too long
Target //src/main/java/com/google/j2cl/samples/helloworld:helloworld failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.781s, Critical Path: 0.27s
INFO: 3 processes: 3 internal.
FAILED: Build did NOT complete successfully
Error message seems to be related to bazel and may not related to j2cl. Someone please shed light on this.
Does anyone faced this issue ?

Yes, see #121

Yes, see #121


Idle Connection Resiliency doesn't work with Connection Pooling

Driver version or file name
5.2.0-rc
SQL Server version
SQL Server 2016
Client operating system
Windows 10
PHP version
7.2.1
Microsoft ODBC Driver version
ODBC17 preview
Table schema
(c1 INT, c2 VARCHAR(40))
Problem description
With connection pooling enabled, connection is not resumed after it is broken.
Expected behavior and actual behavior
Connection should resume after it is broken.
Repro code
function BreakConnection( $conn, $conn_break )
{
    $stmt1 = sqlsrv_query( $conn, "SELECT @@SPID" );
    if ( sqlsrv_fetch( $stmt1 ) )
    {
        $spid=sqlsrv_get_field( $stmt1, 0 );
    }

    $stmt2 = sqlsrv_prepare( $conn_break, "KILL ".$spid );
    sqlsrv_execute( $stmt2 );
    sleep(1);
}

// create a connection for create the table and breaking other connections
$conn_break = sqlsrv_connect( $server, array( "Database"=>$dbName, "UID"=>$uid, "PWD"=>$pwd) );

$tableName = "test_connres";
$sql = "CREATE TABLE $tableName ( c1 INT, c2 VARCHAR(40) )";
$stmt = sqlsrv_query( $conn_break, $sql );

$sql = "INSERT INTO $tableName VALUES ( ?, ? )";
    for( $t = 200; $t < 209; $t++ )
    {
        $ts = substr( sha1( $t ),0,5 );
        $params = array( $t,$ts );
        $stmt = sqlsrv_prepare( $conn_break, $sql, $params );
        sqlsrv_execute( $stmt );
    }

// first connection
$connectionInfo = array( "Database"=>$dbName, "UID"=>$uid, "PWD"=>$pwd, "ConnectionPooling"=>true,
                         "ConnectRetryCount"=>10, "ConnectRetryInterval"=>10 );

$conn = sqlsrv_connect( $server, $connectionInfo );

BreakConnection( $conn, $conn_break );

$stmt = sqlsrv_query( $conn, "SELECT * FROM $tableName", array(), array( "Scrollable"=>"buffered" ) );
if( $stmt === false )
{
    echo "Error in connection 1.\n";
    print_r( sqlsrv_errors() );
}
else
{
    echo "Connection 1 resumes successfully.\n";
    $rowcount = sqlsrv_num_rows( $stmt );
    echo $rowcount." rows in result set.\n";
}

sqlsrv_close( $conn );

// second connection
$conn = sqlsrv_connect( $server, $connectionInfo );

BreakConnection( $conn, $conn_break );

// connection fails to resume here if connection pooling is enabled
$stmt2 = sqlsrv_query( $conn, "SELECT * FROM $tableName" );
if( $stmt2 === false )
{
    echo "Error in connection 2.\n";
    print_r( sqlsrv_errors() );
}
else
{
    echo "Connection 2 resumes successfully.\n";
    $rowcount = sqlsrv_num_rows( $stmt2 );
    echo $rowcount." rows in result set.\n";
}

sqlsrv_close( $conn );

Fixed as of ODBC driver 17.1. Closing the issue.


Move non-secret config to govuk-puppet

Some of our non-secret asset-manager configuration options (e.g. the AWS_S3_BUCKET_NAME environment variable) are currently set in govuk-secrets. This is unnecessary (because they're not secret) and makes them hard for us to change (because we don't have access to the secrets repo). We think it's possible to move these config options to govuk-puppet which would allow us to change them when required.

I've opened https://github.com/alphagov/govuk-puppet/pull/6352 to move the S3 bucket name and region to govuk-puppet.

I've just merged the changes in https://github.com/alphagov/govuk-puppet/pull/6352 without first double checking that the values match those in govuk-secrets! I'll try to get someone to check the values now.

I've asked Paul B to check, and then remove, the equivalent settings in govuk-secrets.

Paul B has confirmed that the values in govuk-secrets matched those in https://github.com/alphagov/govuk-puppet/pull/6352. He's also opened https://github.com/alphagov/govuk-secrets/pull/39 to remove those values from govuk-secrets.
I don't think there's anything more I can do about this so I'm closing the issue.


Upgrade WebdriverIO dependency to V9

Simple version update to make this v9 compatible. The only real change was working around this temporary bug in Chromium
https://github.com/webdriverio/webdriverio/issues/13440

Thanks @klamping , could we update the pipeline action versions to see if the test pass?

Looks like I need to update the node version... Just a moment


Set encoding to UTF-8 in Android. Fixes #4873

Test Plan:

Load the HTML below into a webview on Android:

let html = `<html lang="fr">
<head>
</head>
<body>
commandité
</body>`;

return (<WebView
        originWhitelist={['*']}
        source={{ html: html }}
      />);

The French "e" should have the proper accent.

Release Notes:
[Android][BUGFIX][WebView] - Set encoding to UTF-8 in Android. Fixes #4873

Could you please use and check if the issue is in https://github.com/react-native-community/react-native-webview and create pull request for fix. WebView in RN will be removed in the future.

@dulinriley That has the same issue. I'll open a PR for that there. So you don't want this fix here as well?

@bendangelo please read http://facebook.github.io/react-native/blog/2018/11/01/oss-roadmap, WebView is now community maintained in react-native-webview, and will be removed from RN core in the future.


Adds unit test for pkg/agent/agent.go

Adds unit test for pkg/agent/agent.go
Signed-off-by: Harshitha U R<EMAIL_ADDRESS>

@urharshitha Windows UT check failed.

Closing because there is another PR for this change https://github.com/antrea-io/antrea/pull/4703


[SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch

What changes were proposed in this pull request?
Builds upon the work done by @hhbyyh in https://github.com/apache/spark/pull/7871 . This replaces all occurrences of TimeTracker with the more useful MultiStopWatch. More useful because it is possible to bench the total time across the worker nodes as well, for instance in the method binsToBestSplit using the DistributedStopwatch. It is also very useful to measure the optimizations in terms of time done in https://github.com/apache/spark/pull/13959 and should be merged before that gets reviewed. It also removes the TimeTracker since it is not being used elsewhere except the tree module.
How was this patch tested?
It was run using setLogLevel("INFO") and the following timings are printed out.
16/07/19 16:45:18 INFO RandomForest: {
binsToBestSplit: 26ms,
chooseSplits: 301ms,
findBestSplits: 307ms,
findSplitsBins: 553ms,
init: 1229ms,
total: 1572ms
}

Test build #62566 has started for PR 14273 at commit e5b077d.

ping @jkbradley @mengxr

Test build #62566 has finished for PR 14273 at commit e5b077d.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62566/
Test FAILed.

Test build #62620 has started for PR 14273 at commit abff51b.

Test build #62620 has finished for PR 14273 at commit abff51b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62620/
Test PASSed.

@jkbradley Would you be able to have a look?

bump?

Sorry about the delay here.  Do you still have time to work on this?

@jkbradley I do not think @MechCoder is working on Spark for the time being.

OK apologies @MechCoder for the delay.  I guess we can close this issue, and someone else can open up a PR based on yours.


@tmcw This repository seems not maintained. Can you recommend a replacement?

@tmcw This repository seems not maintained. Can you recommend a replacement?

Sorry, this one isn't relevant enough to me or my work to maintain a fork. Someone from the community should do it, if it's important to them.


Show the name of each account imported

For each account, show Imported <old account> as <new account>.
Does not currently support localization.
Replaces // TODO: show each account imported (and possibly old name) with // TODO: localization.

Have you tested the UX of this with large numbers of accounts?

I have not tested this with large numbers of accounts; I don't have many of my own to test with. I may get around to adding more sample accounts to androidTest, which would help to test other areas as well.


chore: remove cloud sources feature flag

Description

Remove the cloud sources feature flag and turn it on permanently.
User-facing documentation
(must be 2 items and both must be checked)

[ ] CHANGELOG is updated
[ ] CHANGELOG update is not needed
[ ] Documentation PR is created and linked above
[ ] Documentation is not needed

Testing

[ ] inspected CI results

Automated testing
(must be at least 1 item and all items must be checked)

[ ] added unit tests
[ ] added e2e tests
[ ] added regression tests
[ ] added compatibility tests
[ ] modified existing tests
[ ] contributed no automated tests

How I validated my change

change me!

/test all


Remove plone links & update CI job docs

See https://trello.com/c/esoA5qL5/590-update-team-comms-page-in-contributing-docs and https://trello.com/c/yVcZIVWh/582-which-repos-for-which-ci-jobs
This removes the remaining plone links from the team communication page, adds a note about which gh repos are being used by which jobs & a few other little updates to the CI docs, like current versions etc.

Pushed another to address @mtbc comment on https://trello.com/c/RoSbF1W0/450-document-gitter-irc-slack (although it isn't showing up here yet)

Looks fine to me; one very minor nitpick noted.

Thanks, merging and releasing in that case as it'll cut down the number of public places we are still linking to the old website.


notepad ++ crash

Hallo,
following string, causes the plugin to crash notepad ++
[
"BeginOfEnumerable",
[
{
"profileid": 10820
}
],
"EndOfEnumerable"
]
JsonViewer 1.32
Notepad++ v7.6.3   (64-bit)
Build time : Jan 27 2019 - 17:16:47
Path : C:\Program Files\Notepad++\notepad++.exe
Admin mode : OFF
Local Conf mode : OFF
OS : Windows 10 (64-bit)
Plugins : ComparePlugin.dll DSpellCheck.dll mimeTools.dll NppConverter.dll NPPJSONViewer.dll NppXmlTreeviewPlugin.dll PluginManager.dll SqlInFormNpp64.dll XMLTools.dll

also on Npp 7.6.6

The issue is fixed in v1.34. Get it from the releases section. I've already pushed an update to the plugin admin regarding the same.


[Refactor] Remove signer argument from fun init(ctx, signer)

Some module init functions need signer to call account_storage::global_move_to.
The signer is the module address account. This will ensured by the VM. The developer may be suspicious about the correctness of the account, so will make an extra check on the address.
We can replace the signer with moveos_std::signer::module_signer<T>, and remove the signer argument.
https://github.com/rooch-network/rooch/blob/23ebddb7155979da5cc05b4b74a96253609d01fa/crates/rooch-framework/sources/address_mapping.move#L28-L36

🙋


Update to Minecraft 1.20.2

A simple port of the internal scoreboard changes to 1.20.2

More scoreboard changes happened in 1.20.4, do you think you can port that too? Thanks in advance.


Set up travis

Do you believe you could setup travis in order to have it running and being able to test against all PHP versions.

Hi @jdorn, do you think to continue to keep this library and update it to support new versions of php or can we consider it abandoned?
Thank you!

FYI we started a fork at Doctrine: https://github.com/doctrine/sql-formatter


VS Code: require userInfo prop

Context

Fixes the issue caused by the fact that userInfo was not available in the critical place required for a telemetry event creation. This PR makes userInfo a required prop in the React component tree for all the intermediate components to avoid such an issue in the future. It sets the userInfo value to { isDotComUser: true, isCodyProUser: false } for the components reused in the web version. @ chenkc805, let me know if other default values make more sense.
Closes https://github.com/sourcegraph/cody/issues/2468
Slack thread

Test plan

Hit the rate limit events as a DotCom user and ensure the correct event name/tier is used.
Ensured that this won't happen again with the updated types.

So confirming, will the intended behavior be:

If user is Free & on Dotcom & they hit rate limits, we log an upsell
If user is Pro & on Dotcom & they hit rate limits, we log an abuse message
If user is Enterprise & not on Dotcom & they hit rate limits, we log an abuse message
Every other case (ie. Enterprise & Dotcom, Free & not Dotcom, Pro & not Dotcom) are not relevant and can be ignored. Did I get that right?

If user is Free & on Dotcom & they hit rate limits, we log an upsell

If a user hits the rate limit error and an upgrade is available (based on response headers, it can only happen on DotCom for users on the Free tier), we log an upsell ✅

If user is Pro & on Dotcom & they hit rate limits, we log an abuse message

If a user hits the rate limit error and an upgrade is NOT available (based on response headers, it can only happen on DotCom for users on the Pro tier), we log an abuse message ✅

If user is Enterprise & not on Dotcom & they hit rate limits, we log an abuse message

If a user hits the rate limit error and an upgrade is NOT available (based on response headers, it can only happen on DotCom for users on the Pro tier), we log an abuse message ✅
This PR does not change the logic related to choosing the suitable event. It fixes the data propagation so we have all the required information to make the right choice.

@abeatrix, @chenkc805, end-to-end tests worked for this use case by accident. I fixed them in my latest commit. Before that, we ran all the upgrade rate limit E2E tests, thinking we were on the enterprise instance.

@abeatrix, @chenkc805, end-to-end tests worked for this use case by accident. I fixed them in my latest commit. Before that, we ran all the upgrade rate limit E2E tests, thinking we were on the enterprise instance.

HHmm this is different than how I remembered and it looks like something were changed while I was on PTO, but when we first set up the test, we assumed the client would return different value for DotCom and non-DotCom user, and we use the custom endpoint in E2E test to differentiate DotCom and non-DotCom instances: https://github.com/sourcegraph/cody/blob/8bbb73e0087567850add91d39dd8064a228928af/vscode/test/fixtures/mock-server.ts#L122-L126

Yes, that's correct, @abeatrix. This changes response headers for completion requests. But we also look at the authStatus.endpoint to understand whether we are on DotCom. This was not taken into account in E2E tests AFAIU.

Yes, that's correct, @abeatrix. This logic changes response headers for completion requests. But we also look at the authStatus.endpoint to understand whether we are on DotCom. This was not taken into account in E2E tests AFAIU.

Ah. I meant to say we added the custom completion endpoint for the test to workaround that instead of relying on the authStatus.endpoint in the webview, but if that's changed then what I mentioned is now irrelevant 😅

to workaround that instead of relying on the authStatus.endpoint in the webview

But we still have code in the codebase that relies on authStatus.endpoint to determine whether we are on DotCom. Do you know if the intention was to remove all such instances, too?


Godot 4.0.alpha15 : iOS app keep crashing on startup

Godot version
4.0.alpha15
System information
macOS 12.3.1, Rx560, Vulkan(MoltenVK), iPhone7 (MN8X2KH/A), iOS 15.6.1
Issue description
Hello, I'm in testing on several godot4 versions. Including alpha13,14,15 and custom builds. Unfortunately, no succeed for ios till now.
Here is the log messages when I start the iOS app which exported through godo4.alpha.15, even same output other alpha versions.
Is the iOS platform not yet ready? Or is there something I missed?
I love godot engine. Please give me advice.
2022-09-02 15:34:20.913424+0900 gd4-app[12542:420301] Metal GPU Frame Capture Enabled
2022-09-02 15:34:20.913671+0900 gd4-app[12542:420301] Metal API Validation Enabled
*********** main.m
running app main
2022-09-02 15:34:21.059753+0900 gd4-app[12542:420301] Warning: Unable to create restoration in progress marker file
Path: /var/containers/Bundle/Application/70C5A51F-02EC-4BC8-ADDB-EC8316430E3A/gd4-app.app
godot_ios /var/containers/Bundle/Application/70C5A51F-02EC-4BC8-ADDB-EC8316430E3A/gd4-app.app/gd4-app
cwd /private/var/containers/Bundle/Application/70C5A51F-02EC-4BC8-ADDB-EC8316430E3A/gd4-app.app
os created
2022-09-02 15:34:21.065358+0900 gd4-app[12542:420301] USER ERROR: Condition "!configured" is true.
2022-09-02 15:34:21.065422+0900 gd4-app[12542:420301]    at: StringName (core/string/string_name.cpp:327) - Condition "!configured" is true.
USER ERROR: Condition "!configured" is true.
   at: StringName (core/string/string_name.cpp:327) - Condition "!configured" is true.
...
...
...
2022-09-02 15:34:21.071958+0900 gd4-app[12542:420301] USER ERROR: Class  already has a method .
2022-09-02 15:34:21.071981+0900 gd4-app[12542:420301]    at: bind_methodfi (core/object/class_db.cpp:1291) - Condition "has_method(instance_type, mdname)" is true. Returning: nullptr
USER ERROR: Class  already has a method .
   at: bind_methodfi (core/object/class_db.cpp:1291) - Condition "has_method(instance_type, mdname)" is true. Returning: nullptr
2022-09-02 15:34:21.072034+0900 gd4-app[12542:420301] USER ERROR: Condition "!configured" is true.
2022-09-02 15:34:21.072059+0900 gd4-app[12542:420301]    at: StringName (core/string/string_name.cpp:267) - Condition "!configured" is true.
USER ERROR: Condition "!configured" is true.
   at: StringName (core/string/string_name.cpp:267) - Condition "!configured" is true.
...
...
2022-09-02 15:34:21.074308+0900 gd4-app[12542:420301] USER ERROR: Class  already has a method .
2022-09-02 15:34:21.074350+0900 gd4-app[12542:420301]    at: bind_methodfi (core/object/class_db.cpp:1291) - Condition "has_method(instance_type, mdname)" is true. Returning: nullptr
USER ERROR: Class  already has a method .
   at: bind_methodfi (core/object/class_db.cpp:1291) - Condition "has_method(instance_type, mdname)" is true. Returning: nullptr
...
...
...
2022-09-02 15:34:21.154268+0900 gd4-app[12542:420301] USER ERROR: Class '' already has signal ''.
2022-09-02 15:34:21.154290+0900 gd4-app[12542:420301]    at: add_signal (core/object/class_db.cpp:790) - Condition "check->signal_map.has(sname)" is true.
USER ERROR: Class '' already has signal ''.
   at: add_signal (core/object/class_db.cpp:790) - Condition "check->signal_map.has(sname)" is true.
2022-09-02 15:34:21.154326+0900 gd4-app[12542:420301] USER ERROR: Condition "!configured" is true.
2022-09-02 15:34:21.154594+0900 gd4-app[12542:420301]    at: StringName (core/string/string_name.cpp:327) - Condition "!configured" is true.
USER ERROR: Condition "!configured" is true.
   at: StringName (core/string/string_name.cpp:327) - Condition "!configured" is true.
...
...
...
2022-09-02 15:34:21.169085+0900 gd4-app[12542:420301] USER ERROR: Condition "!configured" is true.
2022-09-02 15:34:21.169366+0900 gd4-app[12542:420301]    at: StringName (core/string/string_name.cpp:327) - Condition "!configured" is true.
USER ERROR: Condition "!configured" is true.
   at: StringName (core/string/string_name.cpp:327) - Condition "!configured" is true.
2022-09-02 15:34:21.169408+0900 gd4-app[12542:420301] USER ERROR: Class  already has a method .
2022-09-02 15:34:21.169432+0900 gd4-app[12542:420301]    at: bind_methodfi (core/object/class_db.cpp:1291) - Condition "has_method(instance_type, mdname)" is true. Returning: nullptr
USER ERROR: Class  already has a method .
   at: bind_methodfi (core/object/class_db.cpp:1291) - Condition "has_method(instance_type, mdname)" is true. Returning: nullptr
setting data dir to /var/mobile/Containers/Data/Application/C6D2B158-74EB-4251-93DA-AC9921F39FD7/Documents from /var/mobile/Containers/Data/Application/C6D2B158-74EB-4251-93DA-AC9921F39FD7/Documents
2022-09-02 15:34:21.187947+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.188030+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
   at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.188526+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.188593+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
...
...
...
setup 0
2022-09-02 15:34:21.231024+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.231100+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
   at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
******** setting up keyboard input view
******** adding observer for keyboard show/hide
start animation!
2022-09-02 15:34:21.309411+0900 gd4-app[12542:420301] USER ERROR: AudioOutputUnitStart failed, code: -50
2022-09-02 15:34:21.309554+0900 gd4-app[12542:420301]    at: start (drivers/coreaudio/audio_driver_coreaudio.cpp:251) - AudioOutputUnitStart failed, code: -50
USER ERROR: AudioOutputUnitStart failed, code: -50
   at: start (drivers/coreaudio/audio_driver_coreaudio.cpp:251) - AudioOutputUnitStart failed, code: -50
2022-09-02 15:34:21.327620+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.327742+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
   at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.329585+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.329648+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
   at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
...
...
...
2022-09-02 15:34:21.576546+0900 gd4-app[12542:420301] Vulkan API 1.1.216 - Using Vulkan Device #0: Apple - Apple A10 GPU
Vulkan API 1.1.216 - Using Vulkan Device #0: Apple - Apple A10 GPU
2022-09-02 15:34:21.583300+0900 gd4-app[12542:420301] fopen failed for data file: errno = 2 (No such file or directory)
2022-09-02 15:34:21.583384+0900 gd4-app[12542:420301] Errors found! Invalidating cache...
2022-09-02 15:34:21.917764+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.917847+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
   at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.918336+0900 gd4-app[12542:420301] USER ERROR: Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:21.918408+0900 gd4-app[12542:420301]    at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
USER ERROR: Condition "!classes.has(ti.inherits)" is true.
   at: _add_class2 (core/object/class_db.cpp:402) - Condition "!classes.has(ti.inherits)" is true.
2022-09-02 15:34:22.098801+0900 gd4-app[12542:420476] fopen failed for data file: errno = 2 (No such file or directory)
2022-09-02 15:34:22.098889+0900 gd4-app[12542:420476] Errors found! Invalidating cache...
2022-09-02 15:34:26.492816+0900 gd4-app[12542:420476] validateWithDevice, line 3718: error 'Render Pipeline Descriptor Validation
No valid pixelFormats set.
'
validateWithDevice:3718: failed assertion `Render Pipeline Descriptor Validation
No valid pixelFormats set.
'
dyld4 config: DYLD_LIBRARY_PATH=/usr/lib/system/introspection DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libBacktraceRecording.dylib:/Developer/usr/lib/libMainThreadChecker.dylib:/usr/lib/libMTLCapture.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib
validateWithDevice:3718: failed assertion `Render Pipeline Descriptor Validation
No valid pixelFormats set.
'
(lldb)

Steps to reproduce

Create simple godot-project
Export to iOS with godot4.0.alpha15-template
Build and run on iOS Device

Minimal reproduction project
gd4.alpha15.project.zip

Duplicate of https://github.com/godotengine/godot/issues/64471.

Hi @Calinou @luxmargos,
I have the same errors as you in your logs:
"2022-09-02 15:34:22.098801+0900 gd4-app[12542:420476] fopen failed for data file: errno = 2 (No such file or directory)
2022-09-02 15:34:22.098889+0900 gd4-app[12542:420476] Errors found! Invalidating cache..."
It appears ONLY on first run. Do you have an idea why it happens? Do I need to do something with it?


Conv2DTranspose shape gets None when exporting SavedModel

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): TF2.1
Python version: 3.6
Bazel version (if compiling from source): NA
GCC/Compiler version (if compiling from source): NA
CUDA/cuDNN version: 10.2/7.6
GPU model and memory: V100, 32Gb

Describe the current behavior
The shape information after Conv2DTranspose layer is incomplete and the spatial dimensions are missing when exporting the saved model.
Describe the expected behavior
The shape information should include the spatial dimensions when exporting the saved model.
Code to reproduce the issue
import tensorflow as tf

def _crop_and_concat(inputs, residual_input):
  factor = inputs.get_shape().dims[1].value / residual_input.get_shape().dims[1].value
  return tf.concat([inputs, tf.image.central_crop(residual_input, factor)], axis=-1)

class UNet(tf.keras.Model):
  def __init__(self, name):
    super(UNet, self).__init__(name)
    self.conv1 = tf.keras.layers.Conv2D(filters=8,
                                        kernel_size=(3, 3),
                                        activation=tf.nn.relu)
    self.conv2 = tf.keras.layers.Conv2D(filters=8,
                                        kernel_size=(3, 3),
                                        activation=tf.nn.relu)
    self.maxpool = tf.keras.layers.MaxPool2D(pool_size=(2, 2),
                                             strides=2)
    self.deconv = tf.keras.layers.Conv2DTranspose(filters=16,
                                                  kernel_size=(2, 2),
                                                  strides=(2, 2),
                                                  padding='same',
                                                  activation=tf.nn.relu)
    self.conv3 = tf.keras.layers.Conv2D(filters=8,
                                        kernel_size=(3, 3),
                                        activation=tf.nn.relu)

  @tf.function
  def call(self, x):
    print(">>> Input Shape", x.shape)
    out = self.conv1(x)
    print(">>> conv1 Shape", out.shape)
    skip = self.conv2(out)
    print(">>> conv2 Shape", skip.shape)
    out = self.maxpool(skip)
    print(">>> maxpool Shape", out.shape)
    out = self.deconv(out)
    # the deconv shape will be (None, None, None, 16) when exporting saved model
    print(">>> deconv Shape", out.shape)
    out = self.conv3(out)
    out = _crop_and_concat(out, skip)

    return out

model = UNet("dummy")

res = model.predict(tf.ones((1, 400, 400, 1)))

print("Finish prediction")

tf.keras.models.save_model(model, "/results/SavedModel",
    save_format="tf", overwrite=True, include_optimizer=False)

Other info / logs
>>> Input Shape (None, 400, 400, 1)
>>> conv1 Shape (None, 398, 398, 8)
>>> conv2 Shape (None, 396, 396, 8)
>>> maxpool Shape (None, 198, 198, 8)
>>> deconv Shape (None, 396, 396, 16)
>>> Input Shape (None, 400, 400, 1)
>>> conv1 Shape (None, 398, 398, 8)
>>> conv2 Shape (None, 396, 396, 8)
>>> maxpool Shape (None, 198, 198, 8)
>>> deconv Shape (None, 396, 396, 16)
Finish prediction
>>> Input Shape (None, 400, 400, 1)
>>> conv1 Shape (None, 398, 398, 8)
>>> conv2 Shape (None, 396, 396, 8)
>>> maxpool Shape (None, 198, 198, 8)
>>> deconv Shape (None, **None, None**, 16)
... <Then the TF crashes and complains about the None value since we need it to compute the fraction in _crop_and_concat()> ...

The above log shows that in the prediction, all shapes are correctly inferred. But when we are exporting the saved model: the shape becomes incomplete and the spatial info are lost only after the deconv (Conv2DTranspose) and all the other layers still looks fine with correct shape info. So, we have two questions:
(1) Why do we need to calculate the shapes again when exporting the saved model?
(2) Why is the spatial info lost only after deconv? And this one looks like a bug.
FYI @nluehr

@houtoms Saving a subclassed model is little different. Check for more details in this response. In short, can you try saving the weights using model.save_weights("weights.tf", save_format='tf') and then (1) load the model, (ii) run on some data, and (iii) load_weights. This approach is clearly mentioned in the TF website here. Thanks!
Please let us know if you have any questions. Please close the issue if this resolved your issue. Here is the gist for our reference. Thanks!

Thx. Closing.


AffineLogStackTransform: uninitialized local variable 'rdorigin' used

AffineLogStackTransform::ReadCenterOfRotationIndex(ReducedDimensionInputPointType&) has a suspicious compilation warning from VS2017:

elastix\components\transforms\affinelogstacktransform\elxaffinelogstacktransform.hxx(574): warning C4700: uninitialized local variable 'rdorigin' used

dummyImage->SetOrigin( rdorigin );

https://github.com/SuperElastix/elastix/blob/32a1b346956cf3c250ce357249cbc16bf30310f0/Components/Transforms/AffineLogStackTransform/elxAffineLogStackTransform.hxx#L574
How shall we fix it? Is this function still in use?

I think the dummyImage->SetOrigin( rdorigin ) call can just be removed. But I need to figure out how to test it!

fixed by the above commits


Card Elevation does not work when outlined is true

Bug type
Component
Component name
MudCard
What happened?
<MudCard Elevation="3" Outlined="true" Square="true" Style="width: 100%">
...
</MudCard>

The above results in the following DOM element:

Whereas if I remove the "Outlined" property, "mud-elevation-#" class is added as expected:

Expected behavior
Not sure if this is by design or a bug, but I couldn't find anything about these properties being mutually exclusive in the docs.
Reproduction link
.
Reproduction steps

Create a MudCard
Set Elevation Property
Set Outlined=true
...

Relevant log output
No response
Version (bug)
6.0.17
Version (working)
No response
What browsers are you seeing the problem on?
Chrome
On what operating system are you experiencing the issue?
Windows
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

My thoughts exactly, @fondraco! Maybe it is just a matter of updating documentation.

Same behavior for the MudPaper (what MudCard is based on) as of 7.6.0. Here's a playground https://try.mudblazor.com/snippet/QaweYCwtgAiBmVZq.
mud-elevation-x class is simply not added to the container and still no mention of Outlined preventing Elevation from working in the documentation, which is confusing issue-prone if it's by design.

From the docs: Note: Elevation is disabled when a component supports and uses the Outlined property.

From the docs: Note: Elevation is disabled when a component supports and uses the Outlined property.

Good to know it's written somewhere, although I still think there should be at least a link to that page in the docs of the components that supports elevation, such as MudPaper where it doesn't specify, neither warn about Outlined disabling Elevation


Please document use of wildcards in Reply URLs

Reply URLs accept wildcards. This does not appear to be documented and the resulting behavior is confusing. For example, if I have an app that sends an AuthnRequest with an ACS URL of https://foo.bar.someapp.com/portal/login, then:
These potentially configured Reply URLs work:
https://foo.bar.someapp.com/portal/login
https://foo.bar.someapp.com/*
https://*.bar.someapp.com/*
But these do not:
https://*.someapp.com/*
https://*.someapp.com/portal/login
https://foo.*.someapp.com/portal/login
Please document this functionality and its rules. Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f97b43d9-4a25-1552-c719-f0f0b418b0a8
Version Independent ID: 13445782-bdb8-b9fa-b130-630bb770187c
Content: Configure single sign-on - Azure Active Directory
Content Source: articles/active-directory/manage-apps/configure-single-sign-on-portal.md
Service: active-directory
Sub-service: app-mgmt
GitHub Login: @CelesteDG
Microsoft Alias: celested

@jaffadog
Thanks for your feedback! We will investigate and update as appropriate.

#reassign:msmimart

Thanks @jaffadog for pointing this out. We're working on this guidance with the product team and I'm tracking this issue in an internal work item. Just wanted to let you know this is in the works - for now I'll close the issue but we'll update the related articles as soon as possible.
#please-close


Watch out for null originating elements

https://linear.app/replay/issue/RUN-724

Landing without review so we can get these crash fixes out.


new(libsinsp,libscap): add is_exe_writable flag (take 2)

Signed-off-by: Luca Guerra<EMAIL_ADDRESS>What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature
Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area driver-kmod
/area driver-ebpf
/area libscap
/area libsinsp

/area tests

/area proposals

What this PR does / why we need it:
This PR adds:

flags as an argument to execve exit to be used for additional flag
^ the first of these, EXE_WRITABLE implemented
proc.is_exe_writable as a field tracked by libsinsp that we can use to base Falco rules on

This flag is true if the user that executed a process can also write to the executable file that is used to spawn it or is its owner or otherwise capable. This can be meaningful in Falco rules, since in some scenarios, especially in containerized application where we follow best practices, we don't expect a non-root user to bring its own binaries and run them.
This PR was originally proposed long ago, and now it is hopefully improved because it now works with the eBPF probe as well, also considering user namespaces. The latter can be tricky to handle and there may be a better way to work with them from eBPF, in which case I'd be very happy to know.
One confusing aspect of this flag may be that while computing is_exe_writable the kernel already resolved symlinks, so an user-writable symlink pointing to a non-writable file will not be an exe_writable, even if proc.exepath reports an user-writable symlink. Personally, I think this is more of an issue in how we handle proc.exepath and so possibly suitable for another PR.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

new(libsinsp,libscap): added proc.is_exe_writable flag

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: LucaGuerra
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Welcome @LucaGuerra! It looks like this is your first PR to falcosecurity/libs 🎉

Hi @LucaGuerra. Thanks for your PR.
I'm waiting for a falcosecurity member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: d2d533359aaf5d4ffceddddd73e9abe3526e5733

@LucaGuerra: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

build-libs-minimal
c109239fa1d650aaa1f683412080b9186b7ca17a
link
true
/test build-libs-minimal

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

New changes are detected. LGTM label has been removed.

LGTM label has been added.  Git tree hash: 6ea0b12c6cd7d3743172b9aafe47b476f5430ac3

/test build-libs-minimal

New changes are detected. LGTM label has been removed.

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

c84dac5 track is_exe_writable when collected from proc

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/retest

@LucaGuerra: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

build-libs-minimal
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-minimal

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@LucaGuerra: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

build-libs-minimal
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-minimal

build-libs-bundled-deps
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-bundled-deps

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@LucaGuerra: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

build-libs-minimal
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-minimal

build-libs-bundled-deps
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-bundled-deps

build-libs-with-chisels
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-with-chisels

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@LucaGuerra: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

build-libs-minimal
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-minimal

build-libs-bundled-deps
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-bundled-deps

build-libs-with-chisels
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs-with-chisels

build-libs
fb3dccde827ef8bb0a9e60e2ef0a6217efe095fa
link
true
/test build-libs

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Addressed all comments, including @ldegio 's suggestion to add a flag field for execve instead of an entire integer for a single bit so it can be used for more flags, and I think I know which one we could implement next

LGTM label has been added.  Git tree hash: 954820bfc3dffe7eaca7c9028bc8e7609f73b87b


migration to ESM

[x] npm run test
[x] npx webpack --bail (dist.es5+esm)
[ ] npx run test:browser
[ ] test the baked packages

TypeScript 5.0 supports import *.ts but it isn't intended to be used in code like this library. Even under that feature, libraries that are released to npm with *.js + *.d.ts should have import *.mjs.

"import *.ts" enables deno to use code from the src directory directly.


fix: ensure that clearing and re-entering swap amount generates a new quote

Description
This one has been bugging me for a while. There is some workaround to prevent duplicate quotes from being fetched, which also prevented genuine quotes from being refetched.
Before:
https://user-images.githubusercontent.com/20150449/202700637-66ea9459-7202-47fc-b6dc-465a3989870e.mp4
After:
https://user-images.githubusercontent.com/20150449/202700612-bbebae0e-a015-404b-82fd-13e42c03af22.mp4
Other changes
N/A
Tested
Manually
How others should test
N/A
Related issues
N/A
Backwards compatibility
N/A

Thanks so much, this has been bothering me too!! Glad you got to it.
But no test to ensure it's fixed and won't come back? 😅

fiiiiine i added a test 🙈 i also got sidetracked into fixing the last test that was flakey (i've noticed it breaking in a few unrelated PRs, with a jest error like Ran 100000 timers, and there are still more!) - i suspect it is because of runAllTimers so i changed everything that i could to runOnlyPendingTimers. so there's a few more changes than expected in the test file, maybe you want to take another look @jeanregisser


feat(cli): Extend CLI with v1 authorization commands

Closes #19765
This commit add a series of new subcommands to the influx CLI tool. The specific commands are:

influx v1 to host any InfluxDB v1 specific commands
influx v1 auth for v1 authorization token management.
influx v1 auth set-password to assign a password to a token for V1 API authorization

The v1 auth commands enable access to the private authorization APIs (see #19815 and #19837) used to manage authorization tokens for v1 compatibility APIs, including:

/write
/query

The final issue, #19768 will update the authorization handlers to use these new tokens and their assigned passwords.

[ ] CHANGELOG.md updated with a link to the PR (not the Issue)
[x] Well-formatted commit messages
[x] Rebased/mergeable
[ ] Tests pass

@gavincabbage the team-assigner gods picked you!

@gavincabbage I'll request a review when this is ready, as we need to make a few additional changes

@gavincabbage this PR is ready for review

would it be appropriate to add some tests for all the business logic in v1_authorization.go?

I'm not really sure we have established a way to test CLI commands and I don't see precedent for testing any of the other ones. We do test the HTTP APIs that the CLI commands call, however.

Well, it looks like bucket has some tests:
https://github.com/influxdata/influxdb/blob/6bc4158a46f61897a2796c841958c8b74ed424fd/cmd/influx/bucket_test.go#L33

I'll take a closer look at the bucket tests and verify what they are testing – if they add value, I'll do the same.

@gavincabbage I'm going to add a follow up task to investigate refactoring these commands so they are testable.
The v1_authorization command was copied from authorization.go and is not testable, as the dependencies are not injected.


Ledger signing not working on Firefox on Beta

@jeesunikim , when you work on this, could you please update @ledgerhq dependencies? 🙏

closing this. Firefox 114 broke U2F compatibility (ledger)

Firefox users: Firefox 114 broke U2F compatibility -  disrupting the ability to connect Ledger or any other hardware wallets to MetaMask. Please use other browsers which support U2F or WebHID connection method type (eg. Chrome, Brave).


Change Default Linker to Gold for AArch64

For all the same reasons outlined in PR https://github.com/apple/swift/pull/19875
AArch64 also needs to default to the "Gold" linker.

I'm getting pretty convinced about the arguments in #19875 that BFD is a dead end (cc @jckarter). Thanks, Neil!
@swift-ci Please smoke test and merge

Yeah, I don't know of any common platforms where gold wouldn't be available but BFD would.

lld sounds nice. :-) toolchains::Windows isn't a toolchains::GenericUnix, so this shouldn't matter there.

I agree, lld would be ideal, assuming it's mature enough and there aren't any showstopping bugs. We may however need to still maintain support for ld64 on Apple platforms.


release-19.2: sql: fix locality information in show ranges

PR: https://github.com/cockroachdb/cockroach/pull/44312
From release notes:

SHOW RANGES now shows locality information consistent with the range descriptor when node ID and store ID don't match. [#44312][#44312] {% comment %}doc{% endcomment %}

This is a bug fix that doesn't change expected (and documented) behavior.


Can't install on Windows

I can't install Sanic on Windows, because of the forced uvloop dependency not working on Windows. It raises RuntimeError: uvloop does not support Windows at the moment and aborts the installation. It should still be possible to use Sanic without the uvloop speedup.

If I'm not mistaken the httptools dependency also doesn't work on Windows

httptools installs and works fine for me on Windows. They have a binary wheel.

Try the Windows Subsystem . It's based of Ubuntu 14 kernel . Then Google how to update it to Ubuntu 16 should be straight forward. That's how I'm using Sanic with Uvloop on Windows 10 .https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

Yeah we don't have any control over whether or not uvloop supports windows.
This relates to #17. I guess we should make that clearer in the README

There is actually a way to install sanic on windows without uvloop. I usually set my requirements.txt file to
# install dependencies in setup.py
-e .

to get the dependencies in setup.py. I managed to get it work like this:
# install dependencies in setup.py
-e<EMAIL_ADDRESS>-e .

and doing on the command line:
set SANIC_NO_UVLOOP=1
pip install -r requirements.txt

this will clone sanic from github at tag 0.5.4 and will install without the uvloop dependency and then all the other dependencies from setup.py. Obviously you can specify your dependencies directly in requirements.txt, just write the sanic bit as above
I'm not sure you can install it from the wheel without the uvloop dependency though.

try to install Visual studio build tools  then install sanic it's work for me


Rename HZ module

From cas-server-support-hazelcast to cas-server-support-hazelcast-ticket-registry to be consistent with other ticket registry modules naming convention

Cool. Probably should do the same for all others as well.


Add sorts in generate to make the compiler output stable

This means that if we compile the same code repeatedly the output is identical. The motivation for this is to allow adding lightweight infrastructure to detect whether/how a PR changes the compile output for the example Python files. Having a stable output is a prerequisite for this to avoid false positives where eg. functions are just rearranged.
Where we've used HashMap data structures the sort order is undefined, and in practice if you compile multiple times you'll get different outputs.
See suggestion in Discord: https://discordapp.com/channels/1005658120548270224/1006027721127776388/1042345155522474034
To test this I compiled the examples 20 times each:
# examples/
for f in ./*.py
do
    for i in {1..20}
    do
        ../target/debug/seahorse compile $f > compiled/$f$i.rs
    done
done

And then verified with md5 compiled/* that all instances of the same file are identical
Intentionally didn't update changelog BTW! :D

Alternate minimal-effort idea: replace every HashSet/HashMap with a BTreeSet/BTreeMap. Assuming we're not using any hash-specific functionality, this might just work

Alternate minimal-effort idea: replace every HashSet/HashMap with a BTreeSet/BTreeMap. Assuming we're not using any hash-specific functionality, this might just work

TIL about those, thanks! I'll give that a go, I think it'd be better to keep the insertion order than sort by name if possible. And hopefully a bit more straightforward too :)

I think it'd be better to keep the insertion order than sort by name if possible

Could be done with yet another type! I know that Python does that, this post explains how (hopefully it's correct lol, just picked the first one I found). I think I like name order slightly better though, one advantage is that you have some flexibility with changing how things are added without accidentally changing the output. What draws you to insertion order?

I think I like name order slightly better though, one advantage is that you have some flexibility with changing how things are added without accidentally changing the output. What draws you to insertion order?

Ohh I was assuming BTreeMap etc were using insertion order. Reasoning was just that it'd be closer in structure to the input. I'm not strongly opinionated on the output order though.
I've updated the PR to use those data structured, think that's a lot better! Same test I used before worked! I just needed to update inferred_accounts and namespace. I also updated features kinda speculatively, it won't affect any current examples but I think it would affect output if we had multiple features.


Cannot easily use NiceDCV if only connection is via ssh on port 22

Environment:

AWS ParallelCluster / CfnCluster version [e.g. aws-parallelcluster-2.6.0]

dcv_settings = hpc-dcv
[dcv hpc-dcv]
enable = master
Bug description and how to reproduce:
We only have access via ssh so everything has to be tunnelled via port 22.
To get NiceDCV to work I have to do this:
I have a script on the cluster that runs pcluster_dcv_connect.sh and parses the output to form a connection string that is a URL for localhost so I can then use an ssh tunnel to connect through that.
If I run the standard connect
$ pcluster dcv connect -s pmjmon30-01
Please use the following one-time URL in your browser within 30 seconds:
https://<IP_ADDRESS>:8443?authToken=n2D_WcBJdEOCbS9gv737KfoZpfZMI7h1OAzrKNmtu_eouwkKDuK4UR0ApEg_xVfSN_RQebhr6ndW-hTmzQDe4s05pGnhJBhzOSv5oPhroFmT2vrC5sv_TaLo0H7ZnQs-TkHQnANgmd7StyNLI9AN08U3sLMQ-W8t9bvJPiOgadDsz_AOdFMdE6oY4Q4-hYrxMppvwLAhvJTYzoYTT51rC7z6PEujXjSl0okHKtOUs7rNm4kSm0yf53nleHwGOtCw#Hj6Avm86itfs8BaCAu0r

I get a URL that does not work. Instead, I have to run a script such as below
#!/bin/bash
dcvstr=`/opt/parallelcluster/scripts/pcluster_dcv_connect.sh /tmp`
dcvarr=($dcvstr)
pattern=\*\=
port=${dcvarr[0]#$pattern}
sess=${dcvarr[1]#$pattern}
auth=${dcvarr[2]#$pattern}
echo "https://localhost:"$port"?authToken="$auth"#"$sess

which outputs
https://localhost:8443?authToken=UuD-XC8li-8VcYRm1TFcRwgDlAIeFNPFo5y95A1BzPzjqyr1W2YIpxqhYic2clWhMv2V1lvzDa9ahUAgSzXNW7euRGf4RjdwyluveyMVzEP4-FVqBUgELe4oIE-0VhGAkZsmxpPt2zJhn3dPhEv21Uxi_6-KCUEzRAQ7a8RBw42uofQWE0u31AKjOZMW-5s5bhpUhG33XYMeEYEYNVWb0XEORFYwUnu4_Yul4UESE70Bg2nlhlfvQdabYaWePk1T#Hj6Avm86itfs8BaCAu0r

It would be good to add this sort of option to the pcluster script so e.g.
pcluser dcv connect -show-url --local-url ...

Hi Philip,
thanks for opening this feature request.
As an alternative, instead of replacing the actual master ip with localhost you could modify the /etc/hosts file so that the public IP of the master gets resolved to localhost.

Hi @PhilipMorrisJones, while this request remains valid, it is not a feature we are able to prioritize at this time within our roadmap. We are continuing to work closely with the DCV team within AWS so that we can evolve our integration to cover a wider variety of use cases and enhancements. Thanks


Bug in some multiple requests

First of all. Your project is fantastic and very useful.
I'm trying to use it in our Picostrap Theme
There is a lot of GDPR compliance concern regarding usage of Google Fonts on the web.
Your solution is really clever and so much easier to implement, compared to locally hosting the fonts.
Tinkering with your service I think I've found a bug.
Some requests dealing with multiple fonts, when not including weights, apparently fail.
One example:
https://api.fonts.coollabs.io/css2?family=Playfair+Display&family=Josefin+Sans&display=swap
Returns Playfair Display (correctly)  and Montserrat(?)
To compare, the corresponding Google URL,
https://fonts.googleapis.com/css2?family=Playfair+Display&family=Josefin+Sans&display=swap
works as expected.
Thanks so much again for your project, and for your time looking into this.

One more interesting case:
https://api.fonts.coollabs.io/css2?family=Barlow+Semi+Condensed:wght@700&family=Abhaya+Libre&display=swap
Here, removing the weight fixes the issue apparently.

Nice findings! I will check them!

Sorry for the late answer. I think this is already solved, I just forgot to write here (oops). Could you please check?

Hello Andras,
looks like it's working fine now. Amazing!
Thank you again for your effort in maintaining this useful project.


tx_version

Abstract

Testing
To test this PR, it's suggested to attempt these user flows, or variations of these:

Step 1
Step 2
Step 3

If any errors are found, the PR works unexpectedly, or you have viable suggestions to improve the UX or functionality of the PR, let me know!

What does this PR address?

What features or improvements were added?

How does this benefit users?

wrong repo xD


browser.awaitNavigation() for browser-initiated navigation

Even with https://github.com/angular/protractor/issues/3857, browser-initiated navigation is going to cause problems if/when a page unload interrupts a selenium command (example).  I suggest we add a new function to help users to deal with this:
browser.awaitNavigation: (matcher: string | (url: string) => boolean) => wdpromise<void>

This function will behave basically like browser.get does after the location change.

Blocked on https://github.com/angular/protractor/issues/3857, since without the defer label will not get set


Update vSphere images in docs to be version independent

This updates all vSphere OVA download instructions to be version
independent so we can use them for all releases, namely v0.11.0 and on.
Fixes: https://github.com/vmware-tanzu/community-edition/issues/3518

@stmcginnis @dvonthenen @jpmcb @kcoriordan  could one of you look at this and merge it?
ty!


TEED preprocessor node problem (probably model loading): [WinError 206] The filename or extension is too long:

Running the portable version using the latest comfyui and custom nodes version
On comfyui I got that error
Error occurred when executing TEEDPreprocessor:
[WinError 206] The filename or extension is too long: 'F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\bdsqlsz/qinglong_controlnet-lllite\cache\models--bdsqlsz--qinglong_controlnet-lllite\snapshots\e51c0770bd1ec5cf6419a7e0692e57d7ed5abe23\Annotators'
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 372, in new_func
res_value = old_func(*final_args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\teed.py", line 19, in execute
model = TEDDetector.from_pretrained().to(model_management.get_torch_device())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\controlnet_aux\teed_init_.py", line 25, in from_pretrained
model_path = custom_hf_download(pretrained_model_or_path, filename, subfolder="Annotators")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\controlnet_aux\util.py", line 264, in custom_hf_download
model_path = hf_hub_download(repo_id=pretrained_model_or_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\huggingface_hub\file_download.py", line 1390, in hf_hub_download
os.makedirs(os.path.dirname(pointer_path), exist_ok=True)
File "", line 225, in makedirs
I tried using the perfect resolution on the AIO nodes, and the normal node but I still got that error.

The max_path length in windows is 260.
7_model.pth needs 11 and  your path 'F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\bdsqlsz/qinglong_controlnet-lllite\cache\models--bdsqlsz--qinglong_controlnet-lllite\snapshots\e51c0770bd1ec5cf6419a7e0692e57d7ed5abe23\Annotators' takes 249, but we need a extra slashe '\' here.
11+249+1=261 > 260, That's why this error occur.
Maybe, you can rename the folders in a shorter way and try again.
Or you can download the model directly and put that into 'F:\Stable-Diffusion\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\bdsqlsz\qinglong_controlnet-lllite\' folder.

I tried renaming the folder and installing the model and still it ain't working

Let's try this way:
First, Hold "windows buton" and press "R" on your keyboard to open the Run Command box
then type "gpedit.msc" and press "ok".After that you will get into the Group Policy Editor

Second, find the Enable Win32 long paths which belong in Computer Configuration > Administrative Templates > System > Filesystem > Enable Win32 long paths . Double-click it.

Finally, enable this setting, apply it and press 'ok', then you path limit will gone.

Still ain't working I tried this technique, and after the renaming technique.

Is there a way to use the downloaded model instead the downloaded using the huggingface model with the long path?

Here is my gpedit setting (I am on Windows 11 so I don't know if that is the reason it don't works)

Is there a way to use the downloaded model instead the downloaded using the huggingface model with the long path?
yes, you can create a file named config.yaml to move the ckpts folder.
for example, if I put ckpts in Disk I,

and I will create config.yaml in the comfyui_controlnet_aux folder like this:

Then, all models will download into I:\\ckpts folder.

What's more, you can dowload models from huggingface:

For example, the model url is https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite/blob/main/Annotators/7_model.pth.
you can replace https://huggingface.co to your ckpts path (like me is I:\\ckpts), then delete /blob/main.
that will be the model path. namely: I:\\ckpts\\bdsqlsz\\qinglong_controlnet-lllite\\Annotators\\7_model.pth

Thank you so much, it works amazingly well!


[BUG-python/deployment]  SSL Certificate Verify Failed When Instancing httpx_extra_kwargs={"verify": False}

Describe the bug
When instancing httpx_extra_kwargs={"verify": False} on rg.init, still getting a SSL_CERTIFICATE_VERIFY_FAILED  when using rg.load_dataset_settings()
Stacktrace and Code to create the bug
settings = rg.load_dataset_settings("my_dataset", "my_workspace")

ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

Expected behavior
# Operate through labels within the same dataset

settings.label_schema

# Perform further operations

Add a new label: settings.label_schema.add("new_label")
Remove a label: settings.label_schema.remove("label_to_remove")
Get all labels: labels = settings.label_schema.get_all()
Check if a label exists: exists = settings.label_schema.contains("label_name")

Environment:

Argilla Version [1.21.0]
Databricks

Additional context
The httpx_extra_kwargs parameter is used to provide additional arguments to the httpx library when making HTTP requests using the {"verify": False} argument in httpx_extra_kwargs, specifying that SSL certificate verification should be disabled, not verifying the SSL certificate of the server.
As we're working with sensitive data within a corporate domain, when making HTTP request SSL certificate verification would be verified by default.
Our hypothesis would be that a previous issue was solved regarding this problem, but it hasn't migrated into these processes as attribute.

"mydataset" corresponds to DatasetForTokenClassification


consider specific version of providers instead of latest in docker build

Hello Folks,
it would be worth to consider explicit tag when cloning azuredevops provider (https://github.com/aztfmod/rover/blob/1e1c15cb3c78c79c83acdb2d7341f6258bad3476/Dockerfile#L59) in docker build.
the concern I have is the docker image of rover every time it is built even from the same commit it does not need to be idempotent due to "dynamic" clone of the latest code from different repository of azuredevops provider instead of cloning specific version.
maybe it is intentional but I don't see benefit.
I have come into the problem that I have built the rover again today, from the same code base and couldn't pass the bootstrap steps of caf foundation which were working the days before.
after investigation I realized that there is clone from different repo as I described above.
and what is more. the investigation had been difficult because in container when I attached inside, unfortunately - there is used hard-coded value in the name of azure devops provider - more here issue reported - https://github.com/microsoft/terraform-provider-azuredevops/issues/341
this could be also fix on this codebase -  (https://github.com/aztfmod/rover/blob/1e1c15cb3c78c79c83acdb2d7341f6258bad3476/Dockerfile#L46) when before executing build.sh you overwrite file PROVIDER_VERSION.txt :
`# to force the docker cache to invalidate when there is a new version
ADD "https://api.github.com/repos/microsoft/terraform-provider-azuredevops/git/ref/tags/v${versionAzureDevopsTerraform}" version.json

RUN cd /tmp && \
git clone --branch "v${versionAzureDevopsTerraform}" https://github.com/microsoft/terraform-provider-azuredevops.git && \
cd terraform-provider-azuredevops && \
echo ${versionAzureDevopsTerraform} > PROVIDER_VERSION.txt && \
./scripts/build.sh

hi @eissko, understand the pain point, we'll look to have a short term fix for it, happy to get a PR on it :). Also we are quite looking forward to use registry sourcing for the providers as a longer term solution: https://www.hashicorp.com/blog/adding-provider-source-to-hashicorp-terraform/

@eissko terraform-provider-azuredevops started implementing releases recently so we will transition to their releases soon.


Ignore Readme Files In Workflow

This will ignore the running of the workflow file when someone pushes (commits) edits to a readme file or when someone pull requests readme ( markdown ) file.
If you merge this I'll try to pull request some readme edits contain badges.

Please get rid of the merge commit.

Please get rid of the merge commit.

You mean what

Hi @sudo-nautilus . I guess @BtbN meant that you should provide a linear history. In other words, the two commits should be on the same branch from the beginning.

Ok thanks @pzhlkj6612

Wew

No need to close this, just force-push the rebased commits into your PR branch.


After is not called if test is aborted by SIGINT

I know Ginkgo does this, and I'm sure it comes with some complexity, but is this a known/intentional feature that spec doesn't run any After code? Can you think of how I might be able to setup a signal handler in my own tests to call the After?

You could catch SIGINT (and perhaps SIGTERM) and then panic. The After blocks are deferred and would get called before the tests terminate. I'd be open to adding a helper to spec that does this.

On second thought, the panic would need to be in the same goroutine as the deferred After blocks, which would make the suggestion above difficult.
Parallelism makes this tough to solve cleanly, but maybe we could introduce an option that stops testing on SIGINT/SIGTERM and executes any deferred After blocks?

It looks like Go recently introduced a *T.Cleanup function. There's an upstream issue discussing the issues with this feature and parallelism: https://golang.org/pkg/testing/#T.Cleanup
As a path forward, we could introduce an option that just stops new subtests from running on SIGINT/SIGTERM.

https://github.com/golang/go/issues/41891

Here's what I'm thinking now:

Replace spec.S.After's defer-based implementation with *testing.T.Cleanup.
On first SIGINT/SIGTERM, *T.Skip all subtests that haven't run yet.
Introduce an spec.S.Context() that is canceled on the first SIGINT/SIGTERM.
If three SIGINT/SIGTERM are observed within any five second period, os.Exit(1)

To allow users to enable this, we could add:

A spec.Interrupt() function, returning a spec.Option, that provides a context that's cancelled on SIGINT/SIGTERM
A  spec.Context(ctx) function, returning a spec.Option, that allows you to provide a custom context

This would also allow you to specify a test timeout via ctx.Deadline.

@benmoss could you try this branch? https://github.com/sclevine/spec/tree/spec-1.4-rc
Just pass spec.Interrupt() as an option to spec.Run() to enable the new behavior.
If you have a long running individual spec, make sure it exits when it.Context() is canceled.

Seems good to me, still doesn't fix my exact problem of the test cleanup, but I guess that's just the upstream issue

If you only hit Ctrl-C once, all of the After blocks should still execute. It may take a while for the tests to stop running, but you can use it.Context() to wind them down more quickly.

ctrl-c once doesn't seem to do anything: https://asciinema.org/a/EAbKe6nHvexc4F2rquiD3HFLg

Ctrl-C once:

Stops new tests from running
Cancels the context returned by it.Context()

If you have individual long-running specs, you could end them early when it.Context() is canceled.
Unfortunately, I think this is the best we can do while using Go's testing package for parallelism.

Got it, that seems to do the trick:
func TestHelloWorld(t *testing.T) {
  spec.Run(t, "object", func(t *testing.T, when spec.G, it spec.S) {

    it.After(func() {
      fmt.Println("after")
    })

    it("should have some default", func() {
      fmt.Println("1")
      <-it.Context().Done()
      fmt.Println("2")
    })

    it("runs other tests", func() {
      fmt.Println("3")
    })

  }, spec.Interrupt())
}

$ go test -v ./foo/
=== RUN   TestHelloWorld
=== RUN   TestHelloWorld/object
=== RUN   TestHelloWorld/object/should_have_some_default
1
^C2
after
=== RUN   TestHelloWorld/object/runs_other_tests
--- PASS: TestHelloWorld (2.06s)
--- PASS: TestHelloWorld/object (2.06s)
--- PASS: TestHelloWorld/object/should_have_some_default (2.06s)
--- SKIP: TestHelloWorld/object/runs_other_tests (0.00s)
PASS
ok      github.com/sclevine/spec/foo    2.062s


Add evaluation support for contact and id annotator

Is your proposal related to a problem?
Contact, Id, covid symptom annotators needs to be added to the evaluation code

Depends on:

https://github.com/nlpsandbox/nlpsandbox/issues/46
https://github.com/nlpsandbox/nlpsandbox/issues/47

@thomasyu888 Removing support for covid annotator from the scope of this ticket.

The generator will generate code for the covid annotator because it's in the schema.

@tschaffter This is for the evaluation script code?  I don't think the scoring script is auto generated at all.

Your right. Assigning to @yy6linda