userCertificate required on AD Computer account for AAD Connect to sync

In its default configuration from version 1.1.553 Azure AD Connect wont synchronise Computer objects unless the userCertificate attibute is populated. Is this attribute required for implementing hybrid domain join? Can I safely disable this Scoping Filter on the Out to AAD - Device Join SOAInAD rule in AAD Connect? This isn't documented above (or anywhere else that I can find).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 72cdc251-618a-e84e-f40d-960b0ac69f99
Version Independent ID: 8cbaba3c-e883-2c71-28fb-f88db3fcaf7d
Content: How to configure hybrid Azure Active Directory joined devices
Content Source: articles/active-directory/device-management-hybrid-azuread-joined-devices-setup.md
Service: active-directory
GitHub Login: @MarkusVi
Microsoft Alias: markvi

@jeremyhagan Thanks for the feedback! We are currently investigating and will update you shortly.

@jeremyhagan Hi, Please note as per below document regarding Azure AD Connect version 1.1.553 below -
https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnect-version-history#new-features-and-improvements-10
Specific to userCertificate attribute on Device objects, Azure AD Connect now looks for certificates values required for Connecting domain-joined devices to Azure AD for Windows 10 experience and filters out the rest before synchronizing to Azure AD. To enable this behavior, the out-of-box sync rule “Out to AAD - Device Join SOAInAD” has been updated.

This is self referential. What user certificates are required and why?
Where do they come from? If they aren't there what is the behaviour?
On 20 March 2018 at 11:06, Mohit Garg<EMAIL_ADDRESS>wrote:

@jeremyhagan https://github.com/jeremyhagan Hi, Please note as per
below document regarding Azure AD Connect version 1.1.553 below -
https://docs.microsoft.com/en-us/azure/active-directory/
connect/active-directory-aadconnect-version-history#
new-features-and-improvements-10
Specific to userCertificate attribute on Device objects, Azure AD Connect
now looks for certificates values required for Connecting domain-joined
devices to Azure AD for Windows 10 experience and filters out the rest
before synchronizing to Azure AD. To enable this behavior, the out-of-box
sync rule “Out to AAD - Device Join SOAInAD” has been updated.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/5872#issuecomment-374427472,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AWnfbTIjPoISXgqrIlrD0lvC1qrSwkV0ks5tgEgjgaJpZM4StSsS
.

@jeremyhagan Out to AAD - Device Join SOAInAD sync rule is used to implement Hybrid Azure ad join / Domain Join in a managed domain. In a federated domain this rule is not used as the STS / AD FS would authenticate the device. In a managed domain the certificate for the device would be used to authenticate the device in AAD.
You can disable the sync rule as long as you are using a federated environment.
Please note that if you have further query regarding it please open a new MSDN forum thread so that community can assist you with it. Please use below link for MSDN forum -
https://social.msdn.microsoft.com/Forums/azure/en-US/newthread?category=windowsazureplatform&prof=required

@jeremyhagan We will now proceed to close this thread. If there are further questions regarding this matter, please open a new MSDN forum post and community will help address your query. Please refer to below link to MSDN forum to ask new question -
https://social.msdn.microsoft.com/Forums/azure/en-US/newthread?category=windowsazureplatform

I know this case closed, but I am looking at the same situation where the device objects do not have a userCertificate attribute populated so are being filtered out and not joined to AAD. We are using PTA with SSO. Any recommendation on populating this attribute?

The computers require internet access via WinHTTP in order to probe the device registration service. Once they successfully probe the service they create a self-signed certificate which they then use to populate the userCertificate attribute. Check the User Device Registration event log. If you see errors saying winhttp call back failed it is because the machines can't find a proxy or access the internet.

I stumbled across this when facing the same issue. Hybrid Azure AD is already setup in the environment and can confirm it is working for most machines. I have a couple problem computer objects that upon domain join did not receive the userCertificate attribute. These machines of course are not being synced up.
I have tried Initial Syncs, confirmed connectivity, even removed the machine from the domain and rejoined it and still no userCert attribute. Completely at a loss right now.
Any ideas anyone?
Thanks,

Check the User Device Registration Event Log. The errors will be indicated in there. The machine wont generate the userCertificate until it is sure it can reach the device registration endpoint. Obviously the computer account has to be within the scope of replication (i.e. not in the wrong OU if you aren't syncing the whole domain) and the GPO telling it to join need to be not configured or enabled (win10) or enabled (win81).

@jeremyhagan The machine is within the scope and I do see a warning under the event viewer > apps and services > Microsoft > User Device Registration, but no errors. The warning states it can not be joined the process must run as NT authority.
I am not trying to get it to join right now though, I just need the object to sync,

The object wont sync until the user certificate is created. The workstation
wont create the user certificate and push it to AD until it decides it want
to do the hybrid join. It is just the way it works.
On 29 May 2018 at 23:04, toanyonebutyou<EMAIL_ADDRESS>wrote:

@jeremyhagan https://github.com/jeremyhagan The machine is within the
scope and I do see a warning under the event viewer > apps and services >
Microsoft > User Device Registration, but no errors. The warning states it
can not be joined the process must run as NT authority.
I am not trying to get it to join right now though, I just need the object
to sync,
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/5872#issuecomment-392767984,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AWnfbXauCZaAtLbHeJYjCN74rwwrg4zVks5t3UdLgaJpZM4StSsS
.

@jeremyhagan  First off, thank you for your time in replying to me. Secondly I must apologize as there is something that I am not understanding, it is not clicking for me.
I did a test in my lab, I disabled AAD Connect and joined a new machine and that new machine recieved the usercertificate if that sheds any light.
Can you describe the process a little more for me? At least the flow of events? I am not sure what you mean by
'The workstation wont create the user certificate and push it to AD until it decides it want
to do the hybrid join.'
Thank you for your time.

Sure @toanyonebutyou There are a couple of disjoint processes in play.
AAD Connect has a sync rule which prevents a computer account from syncing unless the userCertificate attribute is populated. It has no involvement in populating that attribute on the computer account. It is just a simple sync rule. The question is: How does this attribute get populated? I'm going to stick to describing the process for Windows 10 since that is where I did all my testing and implementation work.
The userCertificate attribute on the computer account in your on-premises AD gets populated by the User Device Registration Scheduled Task on the workstation. It generates a self-signed certificate and populates the computer account with the public key of this cert. The next question is, how/when does the workstation decide to do this and why?
Well it does it when it is configured to perform a Hybrid Domain Join. Which it will do if there is a Service Connection Point in AD configured with your Tenant ID and if there is a GPO setting set to Not Configured or Enabled on the workstation (see here for doco on the SCP and GPO https://docs.microsoft.com/en-us/azure/active-directory/device-management-hybrid-azuread-joined-devices-setup). So if you think that these two things are set up properly, then you need to monitor the Microsoft-Windows-User-Device-Registration/Admin Event Log to see the flow of events.
A summary of event (from my memory) is as follows:
1: The workstation finds the SCP and decided to try for a hybrid domain join
2: It probes the Azure Device Registration endpoint to see if it can join, if this is successful it then generates the certificate and populates the userCertificate attribute
3: It tried to hybrid join and fails because there is no synced account in Azure AD
4: Time passes until the next AAD Connect sync interval and now, since the userCertificate attribute is populated
5: Meanwhile, the workstation keep periodically trying to Hybrid Domain join, eventually the computer account exists in Azure AD and it matches up the certificate with the one it generated and the hybrid join is successful
The problem is that the workstation need Internet access for the probe to succeed. If this probe fails, it wont generate the certificate. It would be much better if it generated the certificate regardless of the success of the probe. What's worse, is that the Scheduled Task either needs WPAD to be working of to have the WinHTTP proxy settings set to work behind an proxy. It wont use the proxy settings of the logged in user. These two issues (the timing of the cert generation and the proxy requirements) make this a very confusing issue to troubleshoot, ESPECIALLY since the documentation for this process is virtually non-existent.
I hope this makes it clearer for you.
Cheers,
Jeremy.

I had a similar issue where some devices had a UserCertificate and others did not, a full sync resolved the issue.

What kind of full sync you mean here?

I had a similar issue where some devices had a UserCertificate and others did not, a full sync resolved the issue.

What kind of full sync you mean here?

Its been a while, but I imagine I meant I ran a full sync on the user object so it completely updated in AAD from the on-premise object.

I had a similar issue where some devices had a UserCertificate and others did not, a full sync resolved the issue.

What kind of full sync you mean here?

Full AAD Connect Sync:
Start-ADSyncSyncCycle -PolicyType Initial
From: https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-sync-feature-scheduler


Improve documentation for wsdlLocation option

By default cxf codegen puts the absolute build path in the generated class
e.g.
@WebServiceClient(name = "tcnService",
wsdlLocation = "file:/builds/arctravel/current-state/atpco-tcn/src/main/resources/wsdl/tcn.wsdl",
while can lead to runtime issues:
Caused by: javax.wsdl.WSDLException: WSDLException: faultCode=PARSER_ERROR: Problem parsing 'file:/builds/arctravel/current-state/atpco-tcn/src/main/resources/wsdl/tcn.wsdl'.: java.io.FileNotFoundException: /builds/arctravel/current-state/atpco-tcn/src/main/resources/wsdl/tcn.wsdl (No such file or directory)
Fix (which should be atleast noted in documentation to help others not struggle) is to set the wsdlLocation to the classpath: location, however is there any way this can be handled in the plugin I feel this would be a great addition,
tasks.register("tcnWSDL",Wsdl2Java){
toolOptions{
wsdl.set(file("src/main/resources/wsdl/tcn.wsdl"))
wsdlLocation.set("classpath:wsdl/tcn.wsdl")
xjcArgs.add("-Xts")
}
}

It would have to be a documentation note somewhere since this plugin does not participate in the actual code generation process that the wsdl2java tool does.

I was also affected by this issue, using the configuration from the guide at
https://ciscoo.github.io/cxf-codegen-gradle/docs/current/user-guide/#minimal-usage
will generate source code with a an absolute path to your project in the filesystem, which I believe no one actually wants and it creates code which builds and works on your machine but not if you try it somewhere else.
It would be nice to have a separate configuration option for the common usecase that you want to use a wsdl-file from a classpath resource. Or maybe even do like https://github.com/bjornvester/xjc-gradle-plugin so that by default it would use any wsdl-file found in src/main/resources, providing a zero-configuration option. This could even be backwards compatible if it is used as a fallback if none of the wsdl* options is set.

@anderso can you elaborate more as to why one would need the generated code to use classpath?
I'm not familiar with this particular use case and primarily only familiar with generating code from WSDL to consume an external SOAP service of some sort.

Thanks for the detailed info @anderso. I get your point specifically on reproducible builds. As a look at the generated code from a example service, I can see that my filesystem is embedded within the generated code.

Maybe the reason you have not experienced it is because you don't instantiate the service without supplying the wsdl?

I think this is it, see below.
I also think because I'm not using 'pure'/plain Java EE/Jakarta constructs, but rather some specific Apache CXF functionality.

Using this service as an example http://www.dneonline.com/calculator.asmx and the WSDL located at the project root.
Here is how I typically construct a service given a WSDL:

Gradle Contents
import io.mateo.cxf.codegen.wsdl2java.Wsdl2Java
import org.springframework.boot.gradle.plugin.SpringBootPlugin

plugins {
    java
    id("org.springframework.boot") version "3.2.0"
    id("io.mateo.cxf-codegen") version "2.2.0"
}

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(21)
    }
}

repositories {
    mavenCentral()
}

cxfCodegen {
    cxfVersion = libs.versions.cxf
}

dependencies {
    implementation(platform(SpringBootPlugin.BOM_COORDINATES))
    implementation(platform(libs.cxfBom))

    implementation("org.springframework.boot:spring-boot-starter-web")
    implementation("org.apache.cxf:cxf-rt-transports-http")
    implementation("org.apache.cxf:cxf-rt-frontend-jaxws")
}

tasks.register<Wsdl2Java>("calculator") {
    toolOptions {
        wsdl = layout.projectDirectory.file("calculator.wsdl").asFile.toPath().toAbsolutePath().toString()
    }
}

And a small Spring Boot example:
package io.mateo;

import org.apache.cxf.jaxws.JaxWsProxyFactoryBean;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.annotation.Bean;
import org.springframework.context.event.EventListener;
import org.tempuri.CalculatorSoap;

@SpringBootApplication
public class ExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(ExampleApplication.class, args);
    }

    @Bean
    public CalculatorSoap calculator() {
        JaxWsProxyFactoryBean jaxWsClientFactoryBean = new JaxWsProxyFactoryBean();
        jaxWsClientFactoryBean.setAddress("http://www.dneonline.com/calculator.asmx");
        jaxWsClientFactoryBean.setServiceClass(CalculatorSoap.class);
        return (CalculatorSoap) jaxWsClientFactoryBean.create();
    }

    @EventListener(ApplicationReadyEvent.class)
    public void testCalculator(ApplicationReadyEvent event) {
        var calculator = event.getApplicationContext().getBean(CalculatorSoap.class);
        System.out.println(calculator.add(1, 1));
        System.out.println(calculator.multiply(2, 4));
    }

}

Yields the following printed to stdout:
2023-12-18T18:19:33.840-06:00  INFO 57082 --- [           main] io.mateo.ExampleApplication              : Starting ExampleApplication using Java 21.0.1 with PID 57082 (/Users/cisco/code/wsdl2java-issue/build/classes/java/main started by cisco in /Users/cisco/code/wsdl2java-issue)
2023-12-18T18:19:33.841-06:00  INFO 57082 --- [           main] io.mateo.ExampleApplication              : No active profile set, falling back to 1 default profile: "default"
2023-12-18T18:19:34.136-06:00  INFO 57082 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8080 (http)
2023-12-18T18:19:34.140-06:00  INFO 57082 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2023-12-18T18:19:34.140-06:00  INFO 57082 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.16]
2023-12-18T18:19:34.156-06:00  INFO 57082 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2023-12-18T18:19:34.157-06:00  INFO 57082 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 300 ms
2023-12-18T18:19:34.219-06:00  INFO 57082 --- [           main] o.a.c.w.s.f.ReflectionServiceFactoryBean : Creating Service {http://tempuri.org/}CalculatorSoapService from class org.tempuri.CalculatorSoap
2023-12-18T18:19:34.461-06:00  INFO 57082 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8080 (http) with context path ''
2023-12-18T18:19:34.465-06:00  INFO 57082 --- [           main] io.mateo.ExampleApplication              : Started ExampleApplication in 0.746 seconds (process running for 0.988)
2
8

Last two lines are calls to the service which worked without any issues related to wsdlLocation
I think I'd need a concrete example that demonstrates the issue pertaining to wsdlLocaltion since as you said, given the above arrangement, is not something I experience issues with.
I'm not against adding something to this plugin to account for that scenario, I require a working example otherwise I'm shooting/developing in the dark.


Time alignment

This PR implements time alignment.
@CodyCBakerPhD Would be great to get your help on a few final things:

Can you point me to an example for setting up a Dummy Recording for a sorting interface so that we can simply set timestamps / starting time?
The current iteration isn't properly setting timestamps on the preview conversion. Is there a reason this is happening?

Besides the above caveats, this is properly updating the time alignment information and requesting it back on the Source Data popup.

Can you point me to an example for setting up a Dummy Recording for a sorting interface so that we can simply set timestamps / starting time?

Not a 'dummy' example, but a proper test case from example data: https://github.com/catalystneuro/neuroconv/blob/179711773fc67a2a733ebc6522afa0bea6f2f305/tests/test_on_data/test_sorting_interfaces.py#L29-L33
https://github.com/catalystneuro/neuroconv/blob/179711773fc67a2a733ebc6522afa0bea6f2f305/src/neuroconv/tools/testing/data_interface_mixins.py#L586-L589
Easy way to truly do a 'dummy' one is the use the official Mock one: https://github.com/catalystneuro/neuroconv/blob/179711773fc67a2a733ebc6522afa0bea6f2f305/tests/test_ecephys/test_ecephys_interfaces.py#L127-L131

The current iteration isn't properly setting timestamps on the preview conversion. Is there a reason this is happening?

I'll look into it

And just to confirm, we actually do want to allow users to specify timestamps or a start time for Sorting Interfaces, independent of any active Recording Interfaces?
Mentioned this to Ben at the developer hackathon and he was a bit confused about what this actually would mean.

If the exact recording interface used to generate the sorting is present in the pipeline, then they should link the sorting to the recording and adjust timestamps or shift starting time of the recording
Otherwise, they still need to be able to set timestamps and shift starting time as if they had chosen to include the original recording

Ah gotcha. Makes sense now

So this is practically ready for review.
Now that I've expanded my tests beyond the SpikeGLX-Phy test case, though, I've realized that the Phy Sorting Interface in the tutorial data seems to be locked to the SpikeGLX Recording Interface regardless of any user-defined parameters. Is this an expected behavior?

This was the previous behavior before https://github.com/catalystneuro/neuroconv/pull/823—though an error was thrown about the None and int addition. If providing a mock recording interface, the specified start time would take effect but it wasn't clear what it was shifting.

Also confirmed that this time offset on the Recording Interface does not show up in stub files but can be seen from Neurosift after a full conversion.

@garrettmflynn I don't see the link recording button on my end

This is by design. I've implemented a check that determines what interfaces are compatible with a given SortingInterface before display, ensuring that users only select valid options.

It's possible to disable the option—possibly with a related message on hover—instead of removing it completely. Would that be preferable?

This is by design. I've implemented a check that determines what interfaces are compatible with a given SortingInterface before display, ensuring that users only select valid options.

Ahh, right you are - silly me I was using a nonsensible pipeline
That part is working as intended then

A couple things I noticed testing this
i) the option of linking between compatible sorting and recording is now failing for the tutorial data (no button shows up)
ii) I set timestamps for Suite2p (single plane; 250 frames); they seem to have 'uploaded' to the interface fine because the alignment bar did say the proper start and stop interval according to the timestamps; but then they do not show up even in the final converted file
iii) Nothing seems to be applying to stub mode, I'm investigating that one myself

Updated so that (i) works! Can you try again? Would be interested to get the file you're using too so I can replicate.

When I try to do (I), I am getting an error during the conversion—but I can successfully associate the two.

Just FYI I was able to use the Suite2P test pipeline and align to an arbitrary start time, which I could observe across stub and full conversions.

I discovered the issue with recording interfaces specifically, it's on the NeuroConv side so I'll push a fix over there but it shouldn't hold up this end of things
One question about error handling and then this should be good to go

FYI the issue I had with timestamps was a non-issue; the timestamps vector I was trying to set was regular, and NeuroConv magic automatically detected that and set to a regular rate. There's ongoing discussions over there about whether that should even happen at  all though


Assets incorrectly loaded from unrelated page

Scenario is that I have a solid-js start application (SSR enabled).
I have pages in the routes folder (e.g. index and admin/page). However, when I load index, I am seeing that js files from the admin/page is being loaded. In another words, loading the index page is also loading assets from other routes/pages!
Tracing back the issue all the way back to createPageEvent in pageEvent.js of @solidjs/start. It appears that import.meta.env.MANIFEST["client"] and import.meta.env.MANIFEST["ssr"] could have wrongly loaded assets for the page. I checked that Vinxi seems to be the one populating the 2 global variables. Can you provide any additional assistance as to where I could try to trace the issue?
Any ideas why a page is loading assets from other unrelated page?

I am tracing to see why the assets are wrongly loaded, and I noticed that some js files are unnecessary added in the MANIFEST.
Specifically: virtual:$vinxi/handler/client has some external imports that I do not think should be there.
How are the assets/imports for virtual:$vinxi/handler/client determined?


Pin definitions for NVIDIA edge embedded devices

Hi,
I am using Syslogic's AGX based embedded device which has its own GPIO pins and they are further connected to AGX's GPIO pins. Would you know if this package can be used to with Syslogic's controller?
Device I'm using - (https://www.syslogic.com/eng/ai-rugged-computer-rml-jetson-agx-xavier-106848.shtml?parentPageId=106961)
Thank you
Kavish

We can try brut-forcing pin numbers and see if there are any signals on any pin


Disable CURLOPT_SSL_VERIFYPEER

Bug?                     no
New Feature?            no
Are you using composer?  yes
Hi!
I'm trying to use this client. Unfortunatly i keep getting a fatal error because of my self signed certificate.

Uncaught GuzzleHttp\Ring\Exception\RingException: cURL error 60: SSL certificate problem

I know you should be able to disable the verify peer option. What i have tried is.
$guzzleClient = new \GuzzleHttp\Client(array( 'curl' => array( CURLOPT_SSL_VERIFYPEER => false, ), )); $li->setHttpClient($guzzleClient);
and
$guzzle = new Http\Adapter\Guzzle5\Client(['curl' => array( CURLOPT_SSL_VERIFYPEER => false ) ]); $li->setHttpClient($guzzle);
which gives me Class 'Http\Adapter\Guzzle6\Client' not found or paramater must be instance of GuzzleHttp\ClientInterface.
Please advice.

Have a look at the HTTPlug documentation: http://docs.php-http.org/en/latest/clients/guzzle5-adapter.html#usage
You need to create the Guzzle client and configure it. Then you need the Guzzle5 adapter. Also, it seams that you are mixing Guzzle versions. Decide if you want to use Guzzle5 or Guzzle6.

Hi and thanks for you response! I forgot to answer you...I ended up with hardcoding the verifiypeer to false on my local environment. Still getting "paramater must be instance of GuzzleHttp\ClientInterface message". Even after following the docs.

For future reference here is what you should do:
$linkedIn = new Happyr\LinkedIn\LinkedIn('client_id', 'client_secret');
$linkedIn->setHttpClient(new \Http\Adapter\Guzzle6\Client(new \GuzzleHttp\Client(['curl' => [CURLOPT_SSL_VERIFYPEER => false ]])));

You basically need to create a Client instance, pass it to the Client Adapter instance and pass that into the LinkedIn instance.
Why? because composition. This whole Guzzle thing is a mess!

This is now a part of HTTPlugs documentation. http://docs.php-http.org/en/latest/clients/guzzle6-adapter.html


Fix build & update to the latest build convs

coverage: 99.026% (+0.06%) from 98.965%
when pulling 402bb5d14fe464cd34375a044f52cbb8c6a63f94 on fix-build
into 80f9063651aea688652d2d827692895487e96e61 on main.


add custom html head extension

This is mostly so tracking extension could be added before anything else loaded.
There is one place where I didn't rename htmlResources to htmlBodyResources as there are some internal clients that may be using this already.  @jyeakel it would help if we could work with them and rename it while there are no outside users of this feature yet.

I remember one team requesting help with tracking. I assume you searched for extensions.json file?
@jyeakel could you reach to folks who use tracking and ask them to update the code if we decide to rename a field in extensions.json file?


Added support for GitHub reviews

Currently, status: and such in a review body isn't parsed by Carson. This is a bit annoying, as it means we either have to manually assign the status label or write an issue comment with just the status:.
This PR adds a new status subscriber for GitHub reviews. Besides parsing review bodies, this subscriber does 2 more things:

Updating the status based on the review state (approved/changes requested)
Updating the status to needs review when a GitHub review is requested

@weaverryan I'm still not sure how to implement the functional testing bit, if you can do that before/after the merge, I would be very happy :)

@weaverryan any news on it ?

@weaverryan ping

I just made some changes! I would appreciate any review - it's still a bit of "guess work" when processing webhooks, and I'd like to not get it wrong :)

this PR should close both #50 and #59

This is merged and deployed! But, I don't know if the Pull request review webhook is checked on symfony/symfony or not. So, if this doesn't work, we need to ping fabpot about that. I just tested on weaverryan/symfony and the feature definitely works (assuming that webhook is being delivered)

Pull request reviews was already checked on symfony/symfony, and I've just checked it on symfony/symfony-docs.

Can confirm that it's working now! :+1: https://github.com/symfony/symfony-docs/pull/8899#pullrequestreview-117854022


Duplicate peers

Peers can wind up "doubly-connected" to each other.  Then, since you're (probably) not well-connected, you'll try to connect to that address.
This shouldn't cause any breaking behavior, but it's definitely something we want to prevent.

I'm not sure there's an easy fix for this. There isn't a straightforward way to associate the callback address with the connected address, because it gets advertised just like any other address.
We'll probably end up copying Bitcoin (surprise surprise) and just have makeOutboundConnections try to connect to geographically distinct addresses. So if the network is very small there's still a fair chance you'll wind up doubly-connected, but once we cross a certain threshold it should become very unlikely.

Fixed by 629db238eb172bc6c08585d9828efbd57d1f45dd, which bans duplicate IPs.


Suppress input, allow Ctrl+C

Close #126

Codecov Report
Patch coverage: 54.54% and project coverage change: -6.71 :warning:

Comparison is base (d2bcc54) 87.09% compared to head (9cc6488) 80.39%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #153      +/-   ##
==========================================
- Coverage   87.09%   80.39%   -6.71%
==========================================
  Files           1        1
  Lines          31       51      +20
==========================================
+ Hits           27       41      +14
- Misses          4       10       +6

Impacted Files
Coverage Δ

src/Spinners.jl
80.39% <54.54%> (-6.71%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

This is my ugliest code yet!


Error while assigning a Built-in Policy Initiative using SystemAssigned Identity

Our client would like the PCI v3.2.1:2018 Policy Initiative (id: /providers/Microsoft.Authorization/policySetDefinitions/496eeda9-8f2f-4d5e-8dfd-204f0a92ed41) assigned to their subscriptions, however when attempting this we get the following error:
Error: Duplicate object key

  on .terraform/modules/enterprise_scale/locals.policy_assignments.tf line 74, in locals:
  72:   azurerm_policy_set_definition_external_lookup = {
  73:     for policy_set_definition_id in local.policy_assignments_with_managed_identity_using_external_policy_set_definition :
  74:     policy_set_definition_id => {
  75:       name                  = basename(policy_set_definition_id)
  76:       management_group_name = try(regex(local.regex_extract_provider_scope, policy_set_definition_id), null)
  77:     }
  78:   }

Two different items produced the key
"/providers/Microsoft.Authorization/policySetDefinitions/496eeda9-8f2f-4d5e-8dfd-204f0a92ed41"
in this 'for' expression. If duplicates are expected, use the ellipsis (...)
after the value expression to enable grouping by key.

Policy Assignment document looks like this:
{
  "name": "PCI-Benchmark",
  "type": "Microsoft.Authorization/policyAssignments",
  "apiVersion": "2019-09-01",
  "properties": {
    "description": "Enables PCI benchmark initative.",
    "displayName": "PCI-Benchmark",
    "notScopes": [],
    "parameters": {},
    "policyDefinitionId": "/providers/Microsoft.Authorization/policySetDefinitions/496eeda9-8f2f-4d5e-8dfd-204f0a92ed41",
    "scope": "${current_scope_resource_id}",
    "enforcementMode": false
  },
  "location": "${default_location}",
  "identity": {
    "type": "SystemAssigned"
  }
}

Not quite sure what I'm doing wrong here, any help is appreciated.

Thank you for reporting this @john-nicholls-development... I'll try to reproduce this behaviour and come back to you on this ASAP.
Please can you confirm which version of the module you are using when you get this error?

We are currently building with 0.0.8

@john-nicholls-development, we implemented a few fixes to the logic for this in v0.1.0 along with a fairly comprehensive policy refresh. Are you able to re-produce this issue in the latest release? If so, please provide details of the error and we will investigate further.
Please also take note of the upgrade guidance.

Hi,
I get the same problem in both 0.8 and in. 1.1.
We are trying to create a parallel pseudo root in our management group structure in our tenant which is the idea that we want to use as Dev Environment.
It is after we have the init backend and then run the plane step that we get the error message.
These are policies that bothered us:

ES-Deploy-VM-Antimalw
ES-Deploy-VM-Monitoring
ES-Deploy-ASC-Monitoring
Deploy-AKS-policy
Deploy-ASC-ContExport
Deploy-LX-Arv-monitoring
Deploy-SQL-DB-Auditing
Deploy-VM-Antimalw
Deploy-WS-ARC-monitoring

There were more policies that do not go through after our upgrade to 1.1.
Has a feeling that it has to do with how ID is created. It may have to do with several assignments in the same environment.
BR
Magnus

Just a quick update. We have managed to duplicate this error and are working on a fix.

We're making good progress on this, however we still have an outstanding issue which is preventing us from releasing a fix.
The original error reported above by @john-nicholls-development occurs when the module detects a Policy Assignment with a Policy Set Definition not known by the module. We refer to this within the module as an external definition, which covers both built-in and custom Policy Definitions and Policy Set Definitions which are not being managed within the current module context.
Under "normal" operation this part of the module processes correctly and I am able to correctly assign the PCI v3.2.1:2018 Policy Initiative mentioned above, without any changes. With the additional context provided by @magnus78boy I was able to identify that the error occurs when the module needs to lookup multiple assignments using the same external Policy Set Definition. This can also be repeated when a Policy Assignment uses a Policy Definition not known by the module, producing the same error but on line 128.
Error: Duplicate object key

  on .terraform/modules/enterprise_scale/locals.policy_assignments.tf line 128, in locals:
  126:  external_policy_definitions_from_internal_policy_assignments = {
  127:    for policy_set_definition_id in local.policy_assignments_with_managed_identity_using_external_policy_definition :
  128:    policy_set_definition_id => {
  129:      name                  = basename(policy_set_definition_id)
  130:      management_group_name = try(regex(local.regex_extract_provider_scope, policy_set_definition_id), null)
  131:    }
  132:  }

Two different items produced the key
"/providers/Microsoft.Authorization/policySetDefinitions/<redacted>"
in this 'for' expression. If duplicates are expected, use the ellipsis (...)
after the value expression to enable grouping by key.

We have identified a fix to handle these duplications, ensuring the module can successfully perform the lookup on external definitions when duplicates exist, however this has just pushed the problem further downstream.
Despite this not being an issue when no duplicates exist (with or without the "fix"), the module now generates the following error, which is actually harder to identify as this being the cause:
Error: Invalid for_each argument

  on .terraform/modules/enterprise_scale_lz1/resources.role_assignments.tf line 2, in resource "azurerm_role_assignment" "enterprise_scale":
   2:   for_each = local.azurerm_role_assignment_enterprise_scale

The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.

I will continue to investigate this particular issue (as the data model is correctly generating th required values as part of plan). Assuming this is within our control in the module and not something which needs addressing in the provider or Terraform itself, we hope to provide a full fix soon.
During our investigations, we also identified a minor bug causing the module to miss Role Assignments for internal Policy Set Definitions so will cut a new release with these initial fixes in place, but please note this will not resolve this issue fully and requires further work.
In the meantime, we will work on documentation for an example deployment to show how you can decompose your deployment into multiple instances of the module to better handle nested deployments and allow you to create these assignments at multiple scopes within a single hierarchy.

For the issue
Error: Invalid for_each argument

  on .terraform/modules/enterprise_scale_lz1/resources.role_assignments.tf line 2, in resource "azurerm_role_assignment" "enterprise_scale":
   2:   for_each = local.azurerm_role_assignment_enterprise_scale

It is still in place for version 0.3.1.
It works assigning empty map in local.azurerm_role_assignment_enterprise_scale for the first apply.
# The following locals are used to extract the Role Assignment
# configuration from the archetype module outputs.
locals {
  es_role_assignments_by_management_group = flatten([
    for archetype in values(module.management_group_archetypes) :
    archetype.configuration.azurerm_role_assignment
  ])
  es_role_assignments_by_subscription = []
  es_role_assignments = concat(
    local.es_role_assignments_by_management_group,
    local.es_role_assignments_by_subscription,
    local.es_role_assignments_by_policy_assignment,
  )
}

# The following locals are used to build the map of Role
# Assignments to deploy.
locals {
  /*
  azurerm_role_assignment_enterprise_scale = {
    for assignment in local.es_role_assignments :
    assignment.resource_id => assignment
  }
  */

  azurerm_role_assignment_enterprise_scale = {}
}

After the first apply, the change can be reverted and applied again. For the second apply the module output is already in state and TF can determine the values correctly.
So guess it cannot determine the output from the module if that may help.

@petr-stupka... thank you for your additional feedback on this issue. Would you be able to share your configuration with us please so we can try to reproduce?

Regarding my previous response where I mention:

In the meantime, we will work on documentation for an example deployment to show how you can decompose your deployment into multiple instances of the module to better handle nested deployments and allow you to create these assignments at multiple scopes within a single hierarchy.

Please refer to our recent Wiki page, Deploy Using Module Nesting. Hopefully this covers how to do this in a bit more detail.

Hi @krowlandson
regarding the 'Error: Invalid for_each argument'
So I played with it a bit and realized I been wrong with the module like I mentioned before. So I hope I'm correct now :)
The issue is with built-in policy which have managed identity roles defined:
"roleDefinitionIds": [
  "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
],

This section in ES module is trying to create list of those roles for the specific policy from  'local.policy_roles':
es_role_assignments_by_policy_assignment = flatten([
  for policy_assignment_id, policy_id in local.policy_assignments_with_managed_identity : [
    for role_definition_id in try(local.policy_roles[policy_id], local.empty_list) : [
      {
        resource_id          = "${local.azurerm_policy_assignment_enterprise_scale[policy_assignment_id].scope_id}${local.provider_path.role_assignment}${uuidv5(uuidv5("url", role_definition_id), policy_assignment_id)}"
        scope_id             = local.azurerm_policy_assignment_enterprise_scale[policy_assignment_id].scope_id
        principal_id         = try(local.principal_id_by_policy_assignment[policy_assignment_id], null)
        role_definition_name = null
        role_definition_id   = role_definition_id
      }
    ]
  ]
])

However as you can see the 'Mod-Inherit-Tags-RG' as example will be known after apply so it can not read the roles from built-in policy during plan/apply
  policy_roles = {
    "/providers/Microsoft.Authorization/policyDefinitions/cd3aa116-8754-49c9-a813-ad46512ece54"                                                           = (known after apply)
    "/providers/Microsoft.Management/managementGroups/brz365-dev-management/providers/Microsoft.Authorization/policyDefinitions/Deny-Required-Tag-RG"     = []
    "/providers/Microsoft.Management/managementGroups/brz365-dev-management/providers/Microsoft.Authorization/policySetDefinitions/Deny-Required-Tags-RG" = []
    "/providers/Microsoft.Management/managementGroups/brz365-dev-management/providers/Microsoft.Authorization/policySetDefinitions/Mod-Inherit-Tags-RG"   = (known after apply)
  }

Any 'workaround' tip? I'm happy to give it try.

@petr-stupka... I'm not sure whether this is just because you've setup an output for policy_roles, but something doesn't look quite right here. Would you mind just confirming whether you are consuming the module from the Terraform Registry, or have forked/cloned the repository and are using it by making configuration changes within the module?
The reason I ask is that a default deployment of the module would have many more values in the policy_roles object, and this isn't something we would expect a customer to typically need to investigate.
One observation I can make from the above (regardless of approach), is that Mod-Inherit-Tags-RG is showing as (known after apply) because the module doesn't know about it. This means it will try to lookup the Policy Set Definition from Azure, and fail as it presumably doesn't exist. This is usually caused by the policy (set) definition template not being defined within the module. All customer defined policies should be stored under the library_path location with the expected naming convention. They must also then be declared in the corresponding archetype_definition template to ensure the module will deploy them at the correct scope. In your case, this would be against the brz365-dev-management Management Group.
Our recommendation would be to follow the module user guide and examples page for a successful deployment. Specifically, I would recommend looking at our guide for Deploy Custom Landing Zone Archetypes and Expand built in archetype definitions. We have limited documentation currently on creating custom policies and roles, but this is on our backlog and will be published soon.

Closing issue as we believe this was fixed in release v0.4.0


LoadP12Credentials test flakes on Windows

https://github.com/googleapis/google-cloud-cpp/actions/runs/9559232391
[ RUN      ] GoogleCredentialsTest.LoadP12Credentials
D:\a\google-cloud-cpp\google-cloud-cpp\google\cloud\storage\oauth2\google_credentials_test.cc(549): error: Value of: creds
Expected: code is equal to OK and message is anything
  Actual: 40-byte object <90-52 3D-F5 0D-02 00-00 00-CC CC-CC CC-CC CC-CC 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 CC-CC CC-CC CC-CC CC-CC>, whose status is INVALID_ARGUMENT: No private key found in PKCS#12 file (C:\Users\RUNNER~1\AppData\Local\Temp\credentials.p12): Key does not exist.
 (error code<PHONE_NUMBER>) error_info={reason=INVALID_ARGUMENT, domain=gcloud-cpp, metadata={gcloud-cpp.source.filename=D:\a\google-cloud-cpp\google-cloud-cpp\google\cloud\internal\win32\parse_service_account_p12_file.cc, gcloud-cpp.version=v2.26.0-rc, gcloud-cpp.source.line=171, gcloud-cpp.source.function=GetKeyFromProvider}}, with a code that isn't equal to OK, but a message that is anything

[  FAILED  ] GoogleCredentialsTest.LoadP12Credentials (206 ms)

The following tests FAILED:
    346 - storage_oauth2_google_credentials_test (Failed)

https://github.com/googleapis/google-cloud-cpp/actions/runs/9881692763/job/27293859323#step:12:3358
This time it was CreateFromP12ValidFile

https://github.com/googleapis/google-cloud-cpp/actions/runs/10101370384/job/27934825274?pr=14558#step:12:3381
a repeat of GetKeyFromProvider().

I've been aware of this issue and will try alternative methods to obtain the private key when I have time.

https://github.com/googleapis/google-cloud-cpp/actions/runs/10394011938/job/28782826586?pr=14644#step:12:3393

Has this reproduced since #14645? Maybe we can close the issue.

I haven't seen a flake. I am fine with closing the issue. If it happens again, we can always reopen the issue.


make the GuiMain thread not block forever

Why is that needed? I don't think you should need to run the GUI in any situation where blocking may matter

well, in my case i'm launching the converter GUI from inside minecraft... i guess that's not really an expected use case though, so feel free to not merge if this is out-of-scope.

Running the GUI in MC... that seems very wrong. But also, wrapping it in EventQueue.invokeLater should already be enough to make it not block

i considered that, but figured there was some reason why you added the Thread.sleep() to keep the main thread alive. would doing that actually be safe?

The reason I added the Thread.sleep() is that when I was initially writing the code, the JVM I ran it on probably had a bug where it would shut down the event loop thread even after calling EventQueue.invokeLater if the main thread quit.

Or you could just call EventQueue.invokeLater(() -> new GuiFrame().init()); yourself and make the change unnecessary

true, alright then. closing.


[BEAM-3595] Migrate to "v1" URNs for standard window fns.

Add missing documentation.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

R: @boyuanzz

Retest this please


Refactoring of SB to  HuggingFace interface

@TParcollet requested a more flexible interface to use HuggingFace than only with wav2vec2.
In another conversation, @Moumeneb1 pointed me to AutoModel of the HuggingFace transformers library.

Upfront a worthwhile side note—recently released:

transformers 4.22.2 on Sep 27, 2022
datasets 2.5.2 on Oct 5, 2022
huggingface-hub 0.10.0 on Sep 28, 2022

Should we opt for a minimum HF library support ?
(edit: the hub is now at 0.7.0 in requirements.txt - the other two are optional; should they remain so ? )

What has happened at the opening of this Draft PR:

availed handling of HuggingFace cache_dir throughout (so it can be put outside of $HOME/.cache)
=> minor impact to speechbrain/pretrained/fetching.py
removed unnecessary imports and dicts (handled via AutoModel)
placed static functions outside of a class

_check_model_source(path, save_path) // has changes to check if source is downloaded already

created helper static function - some are used implicitly by the pre-existing two classes

config_return_hidden_states(config)
model_set_spectral_augmentation(model, apply_spec_augment)
modify_state_dict_wav2vec2(path)
default_forward(model, data)
wav2vec2_forward(model, data, output_all_hiddens)
wav2vec2_pretraining_forward(model, data, mask_prob, mask_length)

These functions are intended to be used as partials - use them -or- plug-in your own :)

new class: HuggingFaceModel(nn.Module) to handle all interfaces with HuggingFace transformers - with init doing:

determine AutoConfig (adjust if wanted)
create/download model from AutoModel (adjust if wanted)
prepare forward function abstraction

set input layer norm flag
assign inner forward function from given/default partial Callable
(e.g., default_forward; wav2vec2_forward; wav2vec2_pretraining_forward)
set output layer norm flag
output of a variable -or- tuple

Wrapper: forward() -> _forward() -> self.forward_partial_fn(data=data)

handle Freezing

HuggingFaceWav2Vec2 inherits now from HuggingFaceModel and is reduced to a super().init call
same for HuggingFaceWav2Vec2Pretrain; different init parameterization (serves here as proof-of-concept)
docstring examples for the three classes were working on my end

Drafting status:

[x] initial PR (docstring examples & linters)
[x] create integration test folder with YAML examples
[x] whether/not pythonapp workflow integration tests should install transformers>=4.22.2 (or: skip their integration examples)
[x] resolve TODO comments
[x] check on single GPU if nothing breaks & on DDP for wav2vec2 training
[x] minimize online communication overheads (once downloaded, that's it)

Edit (2022-10-11).

[x] dissolve current file & create a nested folder structure with main interface & helper functions
[x] drop normalization functions (note: they have been migrated correctly BUT were ontologically superfluous in the starting code prior to this PR)
[x] expedite further auto-general use features provided by HF
[x] explore to provide further hub examples (beyond w2v2)
[x] expand briefly the existing tutorial for how to make use of this PR

Edit (2022-12-13).

[x] merge testing from #1600
[x] re-test HF pretrained models & apply fixes
[x] fix failing recipes (when transformers integration of this PR is the issue)

The above pre-review was resolved entirely in commit fb80a9d.
Since then, the following above TODOs have been completed:

create integration test folder with YAML examples
resolve TODO comments (note: those in py files)
check on single GPU if nothing breaks & on DDP for wav2vec2 training
--> instead: on cluster cpu node (since the rest is covered through other tests)
minimize online communication overheads (once downloaded, that's it)
--> cpu node ran offline

The four test yamls target the existing interfaces:

speechbrain.lobes.models.huggingface_wav2vec.HuggingFaceWav2Vec2
HuggingFaceWav2Vec2 immediately mapped from yaml through its new base HuggingFaceModel
speechbrain.lobes.models.huggingface_wav2vec.HuggingFaceWav2Vec2Pretrain
HuggingFaceWav2Vec2Pretrain immediately mapped from yaml through its new base HuggingFaceModel

Tested with
transformers       4.22.2
huggingface-hub    0.10.0
datasets           2.5.2

Upfront - the provided integration tests are limited in their scope;
Not: throughout tests if all possible wav2vec2 settings would work or not.
Goal: does anything break immediately, crash, or not run to the end of a "minimal" example.
A list for potential legacy testing of HF integration:
pip install huggingface_hub==0.7.0 datasets==2.0.0 transformers==4.18.0
2022: May 24 - Mar 15 - Apr 6

pip install huggingface_hub==0.4.0 datasets==1.18.2 transformers==4.16.0
2022: Jan 11 - Jan 28 - Jan 27

pip install huggingface_hub==0.1.0 datasets==1.15.0 transformers==4.12.3
2021: Nov 2 - Nov 2 - Nov 3

pip install huggingface_hub==0.0.1 datasets==1.1.3 transformers==4.0.0
2020: Dec 23 - Nov 19 - Nov 30

As for the warning messages about changes in transformers v5, I'd recommend sticking with:
pip install huggingface_hub>=0.10.0 datasets>=2.5.2 "transformers>=4.22.2<5.0"
@TParcollet what's your take ?

Note: when jumping between the HF versions, on legacy testing, lots of cache-related errors appeared—one of the logs showed:
The new cache file layout looks like this

Locking the latest versions would be another option:
pip install huggingface_hub==0.10.0 datasets==2.5.2 transformers==4.22.2

A question for the scope of this PR. Thus far, it was to make more transformers model available. In that sense, future PRs could explore more. Yet, what is more to be explored? :hugs:

data preparation: datasets is considered feature-complete
streaming (HF-tested on w2v2 fine-tuning), see: https://huggingface.co/docs/datasets/stream

HF/datasets related pointers:
https://huggingface.co/docs/datasets/audio_load
https://huggingface.co/datasets/mozilla-foundation/common_voice_11_0
https://huggingface.co/blog?tag=audio
I'd see these features different to the nature of this PR.
Then, there is the transformer Auto~ and Pipelines.

pipeline for inference (comparable to SB architectures); see: https://huggingface.co/docs/transformers/pipeline_tutorial
AutoClass classes beyond AutoModel (currently the only one provided in this PR): AutoTokenizer, AutoFeatureExtractor, AutoProcessor & AutoModel; see: https://huggingface.co/docs/transformers/autoclass_tutorial

While we already do the former in SB (but in the SB way), the latter could have a broader impact: from which of these AutoClasses to choose from?
// perhaps an init check for alternatives to AutoModel would be a go-to here; structurally it appears of same use style.

@Adel-Moumen Just noticed that we will need to update w2v2 related YAMLs on HF when this PR gets merged. What is the best way forward in your view?
For now, I'll put back in the speechbrain.lobes.models.huggingface_wav2vec interfaces.
Their use could be resolved when we do the v0.6 migration (which is becoming some sort of mantra...).

Status: added a before/after testing for changing HF yamls of depending wav2vec2 repos. All of them use input norm & output norm. When substituting output norm with a layer norm (manually), only some of the one-short examples were still working.

With a bit of re-naming, in-/output norms can be substituted using this yaml snippet
wav2vec2: !new:speechbrain.nnet.containers.LengthsCapableSequential
    w2v2_in: !new:speechbrain.processing.features.InputNormalization
        norm_type: sentence
        dim: -1
    w2v2: !ref <w2v2>
    w2v2_norm: !new:speechbrain.processing.features.InputNormalization
        norm_type: sentence
        dim: (-2, -1)

Until v0.6, both ways to de/activate wav2vec2 in/out norms should be available. Another PR should drop the legacy one (that PR would be to the unstable branch).

Added two files for testing whether/not HF interface refacotrings break anything for uploaded pretrained models
tests/integration/HuggingFace_transformers/refactoring_checks.py
tests/integration/HuggingFace_transformers/overrides.yaml

with which one can also test a before/after refactoring of HF hyperparams.yaml files (interface refactorings might require these to change as well). They are provided with comment detail level at the one of other script files in tests/integration (i.e.  none).
What can they do:

test single files (from HF repo) as specified in another branch (yaml files not even visible to the develop branch)
test with pre-prepared test partitions of specified/related recipe datasets to check with specified testers

Example of one reference "to-test/new" YAML files (as of now, the original - PR 1623 seeks into their updating):
https://github.com/speechbrain/speechbrain/blob/testing-refactoring/updates_pretrained_models/asr-wav2vec2-librispeech/hyperparams.yaml
Example of updating PR changes to hyperparams.yaml files:
https://github.com/speechbrain/speechbrain/pull/1623/files
Example of test specification:
https://github.com/speechbrain/speechbrain/blob/testing-refactoring/updates_pretrained_models/asr-wav2vec2-librispeech/test.yaml
For testing some single file from its repo:
sample: example.wav
cls: EncoderASR
fnx: transcribe_batch

For testing with a dataset test partition, add:
dataset: LibriSpeech
recipe_yaml: recipes/LibriSpeech/ASR/CTC/hparams/train_hf_wav2vec.yaml
overrides:
  output_folder: !ref tests/tmp/<dataset>
dataio: from recipes.LibriSpeech.ASR.CTC.train_with_wav2vec import dataio_prepare
test_datasets: dataio_prepare(recipe_hparams)[2]
test_loader: test_dataloader_opts
performance:
  CER:
    handler: cer_computer
    field: error_rate
  WER:
    handler: error_rate_computer
    field: error_rate
predicted: predictions[0]

They relate to the depending recipe hparams file.
Example of testing with custom interface of pretrained models hosted on HF:
https://github.com/speechbrain/speechbrain/tree/testing-refactoring/updates_pretrained_models/asr-wav2vec2-ctc-aishell
The tests/integration/HuggingFace_transformers/overrides.yaml file allows to specify a glob filter:
# Filter HF repos (will be used in a local glob dir crawling)
# glob_filter: "*wav2vec2*"
# glob_filter: "*libri*"
glob_filter: "*"

As for now, legacy YAMLs are supported as well as new interface. I'd like to merge this PR on develop and to later drop the legacy support, as further YAML interface changes will come into play (e.g. with PR 751 and alikes).

Tutorial update:
https://colab.research.google.com/drive/1Ggjw5sao7dFh2GSNwe7hTJFWzkGBzpJB#scrollTo=psdnsUwfsGgf

Alright, shouldn't we also update all the recipes, even if it's backward compatible? Because the example are using the new yaml notation for wav2vec2, but not the recipes. This could be confusing?

Alright, shouldn't we also update all the recipes, even if it's backward compatible? Because the example are using the new yaml notation for wav2vec2, but not the recipes. This could be confusing?

Yes — recipes need updating, too.
No — there are bigger issues to get to first.
Meanwhile this yes/no situation is there

These two files show-case legacy-preserved usability:
https://github.com/speechbrain/speechbrain/blob/d3264fcba7097b14246e01439de961a5e73a4ed9/tests/integration/HuggingFace_transformers/wav2vec2_for_pretraining.yaml
https://github.com/speechbrain/speechbrain/blob/d3264fcba7097b14246e01439de961a5e73a4ed9/tests/integration/HuggingFace_transformers/wav2vec2_from_pretrained.yaml

and these two demonstrate — for early adapters — the new interface that will be breaking the legacy at v0.6:
https://github.com/speechbrain/speechbrain/blob/d3264fcba7097b14246e01439de961a5e73a4ed9/tests/integration/HuggingFace_transformers/wav2vec2_for_pretraining_immediate.yaml
https://github.com/speechbrain/speechbrain/blob/d3264fcba7097b14246e01439de961a5e73a4ed9/tests/integration/HuggingFace_transformers/wav2vec2_from_pretrained_immediate.yaml

Why do I provide both?

=> The current SB testing is non-functional to this type of refactoring.

SB is a merger of YAML and python - on git & on HF - at the core modules; in the recipes; in the pretrained models - the SB ecosystem was not set up for this type of refactoring PR to be easy.

First, SB needs to be made ready to have tools that provide testing for continuity of purpose & consistency, especially for multi-domain refactoring operations, like this PR.
Second, SB needs to transition through the refactoring phase. For this, either it is one PR (business as usual) which does not work that well as seen with PR #751 already since 15+ months. OR - we transition in a two phases: I. legacy-preserving & II. legacy-breaking. Here, the attempt is to roll out the latter strategy for two reasons:

improve testing while at it (see below)
do the major changes, so the legacy-breaking will hurt less, in a follow-up PR

// that then will be painful in combination with all PRs that are legacy-breaking for v0.6

Why not be legacy-preserving in the first place, when you can? [Type I error]

=> HF changed their interfaces and avail more services, especially through the AutoClass - to open up SB services, one approach is to provide a general interface to HF with SB-compliance. The implication is simple: what is old in SB will be put into comparison with many more new functions of alike types, such that sticking on one particular interface to one particular transformer, the wav2vec2, becomes of a conservative artifact if we "conserve" it for too long.

Why not ado with it at once then, either? [Type II error]

=> As mentioned before, SB is a multi-domain, distributed & multi-language ecosystem. Automated testing is limited to:
A. Changes to function body &/or interface invocation via YAML
but not:
B. Changes to function signature (interface), legacy-preserving
C. Changes to function signature (interface), legacy-breaking

Details are explained here (the file is under development):
https://github.com/speechbrain/speechbrain/blob/127e509e93f26cb38cebda365f0986b40385e724/tests/consistency/COVERAGE.md
This PR is also concerned with providing tools for B.
Please take a look at the most important file of this PR:
https://github.com/speechbrain/speechbrain/blob/d3264fcba7097b14246e01439de961a5e73a4ed9/tests/integration/HuggingFace_transformers/refactoring_checks.py
It might need better placement, as it is the basis to check refactorings for YAML files that are hosted on HF to have the pretrained models there to be functional with the changes of this PR. It tests the changes put forth in PR #1623 which is concerned with the updating of the HF yaml files for pre-trained models only.

Updating recipe yamls can be done here, or later.

PR #1600 needs to be wrapped-up first, so that we then can check the recipes to be functional after their refactoring. At the moment I have this knowledge only because of the tests/integration/HuggingFace_transformers/refactoring_checks.py just mentioned above, and this one works only for the HF repos - not for the recipes. (That's why 1600 is necessary.)
I can update the recipe yamls here after:

other fixes we need to do to this interface (not redoing that too often)
the recipe testing ++ PR is finished, otherwise reviewers have belief but no tools

As testing is present at the moment:

recipe YAMLs will change; tests will say: you used all variables & all objects instantiate, bravo

What current testing will not say:

data flow is not broken and the overall pipeline still works
the refactored recipe/pipeline gives the same result

Point 2. is a given when the other two points before are consolidated.
This & the 1600 PRs add testing tools that are needed to ensure point 1.
This PR can serve only as an intermediate refactoring, otherwise one does not have control of consistency to the last develop version before/after. Legacy-breaking is one; continuity of consistency breaking is another.
As such, the updating of recipe YAMLs is a choice of When - in this PR on in its follow-up PR.

Alright, shouldn't we also update all the recipes, even if it's backward compatible? Because the example are using the new yaml notation for wav2vec2, but not the recipes. This could be confusing?

My main concern is the testing tool that this PR provides. In a way, that tool could be "copied" to the 1600 PR. Then, this PR could go directly to unstable and I'd start refactoring the recipes also. Didn't see that option 2-3 weeks ago.

As a reminder for next year.
One could retrain all transformer recipes, since there is a correct feature norm in-place now. One could also refer to the SpeechBrain git commit ID under which a model was trained and re-run performance checks on the testing sets with existing pretrained models (otherwise performance reporting wouldn't be there in the first place). That way, one spares the hassle while having the latest results. Rightfully, one might argue that it is not a 1:1 match between training & testing, which is true. Yet, the added feature norm concerns the final part of an utterance that is padding. As such, we are rather removing artefacts during scoring than we are treating major junks of speech in an unbalanced fashion compared to model training. This change on feature norm concerns the padding which was unaccounted for before, such that now, we expect improvements (generally speaking).

Should we run tests of the first recipe training stages if close/exact same results are yielded?
Due to the nature of DNN training, this option might be unavailable, though (randomness is a key element).

A future PR might look into training with one epoch & testing then (one-shot) for improvements to zero-shot (testing with pretrained model as-is).
Merge procedure:

PR #1600 -> develop
0. Read https://github.com/anautsch/speechbrain/blob/refactor-recipe-testing/tests/coverage/README.md

test all recipes are working
use tests/utils/recipe_tests.py to gather results before refactoring (single-audio interface results)

Read PR #1623 -> testing-refactoring

should the branch be pruned from any non-relevant files?
some changes are about the nature of testing, should they be pushed directly before PR merger?
develop an idea of the changes to come for the HF repos (performance results on model card & interface + github READMEs)

This PR #1596 -> unstable-v0.6

push latest develop to unstable-v0.6 (this mimics stopping pushes to develop)
run HF tests for (a) single-audio interface & (b) entire test partitions
check working tests -> are file changes making sense?
for failing tests -> are there obvious flaws in the changed files?

retrain 1x wav2vec2 or whisper model to see whether/not this is going as expected (one-shot)

Please keep in mind that here, commits to PR 1600 are included as well (68 commits w/ +3,965 -923 changes). The shown 115 commits with +6,403 -1,761 changes are not as "dramatic". I merged in a recent develop version to unstable-v0.6 and rebased this PR twice so the commits are managed properly by git.
I prepared a tutorial update - but there was another update on whisper, so I'll tackle it next year.
All changes to existing yamls are conceptually aligned to this way:
[ wav -> w2v2_input_norm -> w2v2 -> w2v2_output_norm -> Dense ] = encoder
         \--------------- wav2vec2 ----------------/

What was wav2vec2 before is now a w2v2 hparam. Benefit: a correct input feature norm is used.

Intermezzo notes:

115 commits with +6,403 -1,761 changes is now 49 commits with +2,498 -972 changes
PR #1623 is now #1801 (cleaned-up before 1600/testing refactoring was merged)
meanwhile, whisper came in & changed
meanwhile, w2v2 was hotfixed
meanwhile, the tutorial changed

Wrapping this one up over the next few weeks.

To follow-up on the merge procedure described in:
https://github.com/speechbrain/speechbrain/pull/1596#issuecomment-1354727118
For how to use the advanced test tools, when interface refactoring touches upon pretrained model interfaces (e.g. YAML files on HuggingFace), please read:
https://github.com/speechbrain/speechbrain/blob/develop/tests/utils/README.md
To-date, the following SpeechBrain branches & PRs are related:

Branch
Purpose
PR

develop
v0.5.14
to-come v0.6

unstable-v0.6
v0.6.0
#1596 flexible transformer integration

hf-interface-testing
keeping track of HF interfaces (YAMLs & custom.py)
#1868 adds two recent HF repos (original interfaces)

hf-interface-testing
PR 1596 changes interfaces, so YAMLs on HF repos need to change
#1801 supplements 1596 (after 1868)

unstable-v0.6
v0.6.0
#751 CTC decoding & scoring refactorings

PR 1868 should be a simple comparison with what's currently on HF
PR 1801 can be used for tutorials to show how YAMLs need to be updated; here, we keep track of our changes
PR 1596 is the transformers lib refactoring (this PR), to make use of their AutoConfig, AutoTokenizer, AutoModel, etc.
PR 751 the legendary v0.6 development - this PR will also need supplementing edits to YAMLs & pretrained interfaces on HF, in a similar tracking fashion as demonstrated with this PR for refactoring the transformers integration

The then "to-come v0.6" will be progressively enhanced on the unstable-v0.6 branch. When the moment of its merging comes, all YAMLs & interfaces that are on  hf-interface-testing are ready to be put on HuggingFace. That in-parallel update procedure implies that each HF repo will have a PR which updates YAML & custom.py interface files with what we kept track of in the hf-interface-testing branch.
Before merging this PR on unstable-v0.6, please:

merge the latest develop on the unstable-v0.6 branch
rebase this PR away from & back to the unstable-v0.6 branch => git tree needs that for updating...

Here's how I went for testing this PR.
Preparation steps:

Clone & checkout https://github.com/anautsch/speechbrain/tree/hf-integration
Please create a fresh Python 3.9 environment (some testinig tools require >= 3.9)
Install SpeechBrain from local repo + requirements + find recipes | grep extra | xargs cat | sort -u | grep -v \# | xargs -I {} pip install {}

Next => Test if the refactored recipes (SpeechBrain repo only) still work (here, whisper & wav2vec2 only).

Run recipe tests by Hparam_file

python -c 'from tests.utils.recipe_tests import run_recipe_tests; print("TEST FAILED!") if not(run_recipe_tests(filters_fields=["Hparam_file"], filters=[["recipes/LibriSpeech/ASR/transformer/hparams/train_hf_whisper.yaml", "recipes/LibriSpeech/ASR/CTC/hparams/train_hf_whisper_encoder.yaml", "recipes/TIMIT/ASR/transducer/hparams/train_wav2vec.yaml", "recipes/TIMIT/ASR/seq2seq/hparams/train_with_wav2vec2.yaml", "recipes/SLURP/direct/hparams/train_with_wav2vec2.yaml", "recipes/IEMOCAP/emotion_recognition/hparams/train_with_wav2vec2.yaml", "recipes/LibriSpeech/ASR/CTC/hparams/train_sb_wav2vec.yaml", "recipes/LibriSpeech/ASR/CTC/hparams/train_hf_wav2vec.yaml", "recipes/LibriSpeech/self-supervised-learning/wav2vec2/hparams/wav2vec2_base.yaml", "recipes/Switchboard/ASR/CTC/hparams/train_with_wav2vec.yaml", "recipes/DVoice/ASR/CTC/hparams/train_fon_with_wav2vec.yaml", "recipes/DVoice/ASR/CTC/hparams/train_amh_with_wav2vec.yaml", "recipes/DVoice/ASR/CTC/hparams/train_sw_with_wav2vec.yaml", "recipes/DVoice/ASR/CTC/hparams/train_dar_with_wav2vec.yaml", "recipes/DVoice/ASR/CTC/hparams/train_wol_with_wav2vec.yaml", "recipes/DVoice/ASR/CTC/hparams/train_multi_with_wav2vec.yaml", "recipes/CommonVoice/ASR/seq2seq/hparams/train_en_with_wav2vec.yaml", "recipes/CommonVoice/ASR/seq2seq/hparams/train_fr_with_wav2vec.yaml", "recipes/CommonVoice/ASR/seq2seq/hparams/train_rw_with_wav2vec.yaml", "recipes/CommonVoice/ASR/seq2seq/hparams/train_it_with_wav2vec.yaml", "recipes/CommonVoice/ASR/CTC/hparams/train_en_with_wav2vec.yaml", "recipes/CommonVoice/ASR/CTC/hparams/train_fr_with_wav2vec.yaml", "recipes/CommonVoice/ASR/CTC/hparams/train_de_with_wav2vec.yaml", "recipes/CommonVoice/ASR/CTC/hparams/train_rw_with_wav2vec.yaml", "recipes/CommonVoice/ASR/CTC/hparams/train_it_with_wav2vec.yaml", "recipes/CommonVoice/self-supervised-learning/wav2vec2/hparams/wav2vec2_base.yaml", "recipes/AISHELL-1/ASR/transformer/hparams/train_ASR_transformer_with_wav2vect.yaml", "recipes/AISHELL-1/ASR/CTC/hparams/train_with_wav2vec.yaml", "recipes/timers-and-such/direct/hparams/train_with_wav2vec2.yaml", "recipes/CommonVoice/ASR/transformer/hparams/train_ar_hf_whisper.yaml", "recipes/CommonVoice/ASR/transformer/hparams/train_mn_hf_whisper.yaml", "recipes/CommonVoice/ASR/transformer/hparams/train_hi_hf_whisper.yaml", "recipes/CommonVoice/ASR/transformer/hparams/train_sr_hf_whisper.yaml", "recipes/CommonVoice/ASR/transformer/hparams/train_fa_hf_whisper.yaml", "recipes/CommonVoice/ASR/transformer/hparams/train_fr_hf_whisper.yaml"]], do_checks=False, run_opts="--device=cuda")) else print("TEST PASSED")'

Note: cat tests/recipes/*.csv | cut -d ',' -f2 | sort -u will list you the currently recorded recipe datasets available for testing.

Prepare testing of YAMLs + custom.py (the to-come HF repo updates/PRs).

python -c 'from tests.utils.refactoring_checks import init;init(new_interfaces_git="https://github.com/anautsch/speechbrain", new_interfaces_branch="hf-integration")'

This clones the specified branch (it's PR 1801) to the nested tests/tmp/hf_interfaces folder. There, the tests/utils/refactoring_checks.py tool will be able to access updated YAMLs & custom.py files. The same tool gets to-date versions of these files directly from HuggingFace. To compare before and after the refactoring, we take advantage from the local repo installation of SpeechBrain: we can switch between develop & PR branches.

Run single-file tests for pretrained interfaces

# Let's revisit the old way to integrate transformers into SpeechBrain
git checkout develop
python -c "from tests.utils.refactoring_checks import gather_expected_results;gather_expected_results()"

# so we can compare it with the proposed way to integrate all of the latest features from the transformers library
git checkout hf-integration
python -c "from tests.utils.refactoring_checks import gather_refactoring_results;gather_refactoring_results()"

Note: A yaml summary file will be created at tests/tmp/refactoring_results.yaml.
For the following, please ensure that test partitions of depending recipe datasets are available. As for this example, we assume there's access only available to LibriSpeech. As such, path specifications for other recipe datasets are empty, thus depending pretrained models cannot be tested. The following step aims to reproduce performance metrics using test partitions, as they are reported on the SpeechBrain recipe folders. Therefore, however, pretrained models are used.

Prepare your datasets using the recipe, but point their output to the testing structure. Example: LibriSpeech (run on cpu-only; more CPUs, less waiting)

cd recipes/LibriSpeech/ASR/CTC && python train_with_wav2vec.py hparams/train_hf_wav2vec.yaml --data_folder=/path/to/dataset --output_folder=../../../../tests/tmp/LibriSpeech || cd -

Manually cancel after the data preparation finished (when recipe training starts). Repeat for other datasets (each one recipe); check on step (9) for using expected folder names.

To avoid that the recipes folder cannot be found as a module, we create a symbolic link. (try w/o, you'll see)

cd tests/utils && ln -s ../../recipes  && cd -

Run tests with pretrained models on the test partitions of recipe datasets

git checkout develop
python tests/utils/refactoring_checks.py tests/utils/overrides.yaml --LibriSpeech_data="/path/to/dataset" --CommonVoice_EN_data="" --CommonVoice_FR_data="" --IEMOCAP_data="" --after=False

git checkout hf-integration
python tests/utils/refactoring_checks.py tests/utils/overrides.yaml --LibriSpeech_data="/path/to/dataset" --CommonVoice_EN_data="" --CommonVoice_FR_data="" --IEMOCAP_data="" --after=True

Note: Other refactorings might have expected changes in their testing performance; then, this tool can be used to measure those changes as well.

Logs from (4)
(1/35) Running test for TIMIT_row_4...
	... 582.89s
(2/35) Running test for TIMIT_row_18...
	=> skipped; took too long – i.e. restart w/o the two TIMIT yamls

(1/33) Running test for LibriSpeech_row_2...
	... 167.02s
(2/33) Running test for LibriSpeech_row_3...
	... 44.51s
(3/33) Running test for LibriSpeech_row_23...
	... 15.72s
(4/33) Running test for LibriSpeech_row_24...
	... 28.28s
(5/33) Running test for LibriSpeech_row_25...
	... 23.78s
(6/33) Running test for DVoice_row_2...
	... 122.35s
(7/33) Running test for DVoice_row_3...
	... 110.42s
(8/33) Running test for DVoice_row_4...
	... 92.57s
(9/33) Running test for DVoice_row_5...
	... 91.19s
(10/33) Running test for DVoice_row_6...
	... 91.76s
(11/33) Running test for DVoice_row_7...
	... 88.60s
(12/33) Running test for AISHELL-1_row_2...
	... 177.38s
(13/33) Running test for AISHELL-1_row_5...
	... 99.93s
(14/33) Running test for timers-and-such_row_6...
	... 67.50s
(15/33) Running test for CommonVoice_row_2...
	... 93.06s
(16/33) Running test for CommonVoice_row_3...
	... 70.77s
(17/33) Running test for CommonVoice_row_4...
	... 70.30s
(18/33) Running test for CommonVoice_row_5...
	... 73.74s
(19/33) Running test for CommonVoice_row_6...
	... 74.12s
(20/33) Running test for CommonVoice_row_12...
	... 104.15s
(21/33) Running test for CommonVoice_row_13...
	... 123.71s
(22/33) Running test for CommonVoice_row_14...
	... 183.87s
(23/33) Running test for CommonVoice_row_15...
	... 144.55s
(24/33) Running test for CommonVoice_row_18...
	... 54.74s
(25/33) Running test for CommonVoice_row_19...
	... 34.37s
(26/33) Running test for CommonVoice_row_20...
	... 33.29s
(27/33) Running test for CommonVoice_row_21...
	... 34.68s
(28/33) Running test for CommonVoice_row_22...
	... 31.71s
(29/33) Running test for CommonVoice_row_23...
	... 40.47s
(30/33) Running test for CommonVoice_row_24...
	... 26.99s
(31/33) Running test for Switchboard_row_2...
	... 33.13s
(32/33) Running test for SLURP_row_4...
	... 25.21s
(33/33) Running test for IEMOCAP_row_2...
	... 20.28s

=> Ok, so the refactored recipes are not crashing.
Logs from (6)
$ grep same tests/tmp/refactoring_results.yaml
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true
  same: true

Note: emotion-recognition-wav2vec2-IEMOCAP entries need manual clean-up.
=> Ok, so for single audios, the refactoring does no harm on inference when using pretrained models.
Logs from (9)
Run tests on: asr-wav2vec2-librispeech
speechbrain.core - Beginning experiment!
speechbrain.core - Experiment folder: tests/tmp/LibriSpeech
	repo: asr-wav2vec2-librispeech
	speechbrain.pretrained.EncoderASR
	obj.from_hparams({'source': 'speechbrain/asr-wav2vec2-librispeech', 'savedir': 'pretrained_models/asr-wav2vec2-librispeech', 'run_opts': {'debug': False, 'debug_batches': 2, 'debug_epochs': 2, 'debug_persistently': False, 'device': 'cuda:0', 'data_parallel_backend': False, 'distributed_launch': False, 'distributed_backend': 'nccl', 'find_unused_parameters': False, 'tqdm_colored_bar': False}})
speechbrain.pretrained.fetching - Fetch hyperparams.yaml: Delegating to Huggingface hub, source speechbrain/asr-wav2vec2-librispeech.
speechbrain.pretrained.fetching - Fetch custom.py: Delegating to Huggingface hub, source speechbrain/asr-wav2vec2-librispeech.
Some weights of the model checkpoint at facebook/wav2vec2-large-960h-lv60-self were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-large-960h-lv60-self and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
speechbrain.lobes.models.huggingface_wav2vec - speechbrain.lobes.models.huggingface_wav2vec - wav2vec 2.0 is frozen.
speechbrain.pretrained.fetching - Fetch wav2vec2.ckpt: Using existing file/symlink in pretrained_models/asr-wav2vec2-librispeech/wav2vec2.ckpt.
speechbrain.pretrained.fetching - Fetch asr.ckpt: Using existing file/symlink in pretrained_models/asr-wav2vec2-librispeech/asr.ckpt.
speechbrain.pretrained.fetching - Fetch tokenizer.ckpt: Using existing file/symlink in pretrained_models/asr-wav2vec2-librispeech/tokenizer.ckpt.
speechbrain.utils.parameter_transfer - Loading pretrained files for: wav2vec2, asr, tokenizer
Some weights of the model checkpoint at facebook/wav2vec2-large-960h-lv60-self were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-large-960h-lv60-self and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
speechbrain.lobes.models.huggingface_wav2vec - speechbrain.lobes.models.huggingface_wav2vec - wav2vec 2.0 is frozen.
speechbrain.dataio.encoder - Load called, but CTCTextEncoder is not empty. Loaded data will overwrite everything. This is normal if there is e.g. an unk label defined at init.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 328/328 [01:26<00:00,  3.77it/s]
speechbrain.utils.train_logger - [LibriSpeech] - BEFORE: asr-wav2vec2-librispeech, set: test-clean - test CER: 5.00e-01, test WER: 1.90
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 368/368 [01:22<00:00,  4.46it/s]
speechbrain.utils.train_logger - [LibriSpeech] - BEFORE: asr-wav2vec2-librispeech, set: test-other - test CER: 8.83e-01, test WER: 2.95

We can compare this with:
https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriSpeech/ASR/CTC
where for train_hf_wav2vec.yaml a Test Clean WER of 1.90 is reported.
The log for the refactored interfaces:
Checking out files: 100% (122/122), done.
Switched to branch 'hf-integration'
Your branch is up to date with 'origin/hf-integration'.

Run tests on: asr-wav2vec2-librispeech
speechbrain.core - Beginning experiment!
speechbrain.core - Experiment folder: tests/tmp/LibriSpeech
	repo: asr-wav2vec2-librispeech
	speechbrain.pretrained.EncoderASR
	obj.from_hparams({'source': 'tests/tmp/hf_interfaces/updates_pretrained_models/asr-wav2vec2-librispeech', 'savedir': 'pretrained_models/asr-wav2vec2-librispeech', 'run_opts': {'debug': False, 'debug_batches': 2, 'debug_epochs': 2, 'debug_persistently': False, 'device': 'cuda:0', 'data_parallel_backend': False, 'distributed_launch': False, 'distributed_backend': 'nccl', 'find_unused_parameters': False, 'tqdm_colored_bar': False}})
speechbrain.pretrained.fetching - Fetch hyperparams.yaml: Linking to local file in tests/tmp/hf_interfaces/updates_pretrained_models/asr-wav2vec2-librispeech/hyperparams.yaml.
speechbrain.pretrained.fetching - Fetch custom.py: Linking to local file in tests/tmp/hf_interfaces/updates_pretrained_models/asr-wav2vec2-librispeech/custom.py.
Some weights of the model checkpoint at facebook/wav2vec2-large-960h-lv60-self were not used when initializing Wav2Vec2Model: ['lm_head.bias', 'lm_head.weight']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-large-960h-lv60-self and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
speechbrain.lobes.models.transformer.HuggingFace - speechbrain.lobes.models.HuggingFaceTransformer is frozen.
speechbrain.pretrained.fetching - Fetch wav2vec2.ckpt: Using existing file/symlink in pretrained_models/asr-wav2vec2-librispeech/wav2vec2.ckpt.
speechbrain.pretrained.fetching - Fetch asr.ckpt: Using existing file/symlink in pretrained_models/asr-wav2vec2-librispeech/asr.ckpt.
speechbrain.pretrained.fetching - Fetch tokenizer.ckpt: Using existing file/symlink in pretrained_models/asr-wav2vec2-librispeech/tokenizer.ckpt.
speechbrain.utils.parameter_transfer - Loading pretrained files for: wav2vec2, asr, tokenizer
Some weights of the model checkpoint at facebook/wav2vec2-large-960h-lv60-self were not used when initializing Wav2Vec2Model: ['lm_head.bias', 'lm_head.weight']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-large-960h-lv60-self and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
speechbrain.lobes.models.transformer.HuggingFace - speechbrain.lobes.models.HuggingFaceTransformer is frozen.
speechbrain.dataio.encoder - Load called, but CTCTextEncoder is not empty. Loaded data will overwrite everything. This is normal if there is e.g. an unk label defined at init.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 328/328 [01:30<00:00,  3.61it/s]
speechbrain.utils.train_logger - [LibriSpeech] - AFTER: asr-wav2vec2-librispeech, set: test-clean - test CER: 5.27e-01, test WER: 2.04
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 368/368 [01:25<00:00,  4.30it/s]
speechbrain.utils.train_logger - [LibriSpeech] - AFTER: asr-wav2vec2-librispeech, set: test-other - test CER: 9.21e-01, test WER: 3.15
	before: {'test-clean': {'CER': 0.50, 'WER': 1.90}, 'test-other': {'CER': 0.88, 'WER': 2.95}}
	 after: {'test-clean': {'CER': 0.53, 'WER': 2.04}, 'test-other': {'CER': 0.92, 'WER': 3.15}}
	  same: False

=> Well, there's more going on ;-)
It's in the range & with the 751 PR, more refacotrings are coming in—also, we had a hotfixing of wav2vec2 to the end of 2022 & in early 2023, some more edits to whisper (without much performance re-checking). As of internal discussions, at some point the retraining of SpeechBrain models will be necessary.

Still getting a handle on this PR, but one quick question from the start: many of the changes seem to be related to removing device from the arguments to the checkpointer's .load() function. Is this a necessary part of this PR or would it make sense to put in a different PR? I would just like to hear the motivation for the change itself and for putting it in this PR, and I can't find a discussion of this point in the comments above.

Hey @pplantinga, the device argument as already been removed in unstable-v6.0 (see: https://github.com/speechbrain/speechbrain/blob/unstable-v0.6/recipes/LibriSpeech/ASR/seq2seq/train.py#L370) thanks to @lucadellalib's PR (see: https://github.com/speechbrain/speechbrain/pull/1743).
I think you should not pay attention to this. The PR is not sync anymore to the latest commits in unstable-v6.0.

@mravanelli @Adel-Moumen @pplantinga I have an idea to simplify this PR. What about we take only the 'HuggingFaceTransformer(nn.Module):' class and make it an abstract class. THe idea is to force the user to override a few mandatory functions, like the forward, and the freeze params functions. THis would be MUCH simpler to grasp. There is no way we release a SB v1.0 with huggingface_wav2vec.py in the lobes while it's not even wav2vec2 anymore ...

I agree on the need for a simplification. Could you show us a code snippet
to figure out how the code can look like with this proposal?
On Sun, Aug 6, 2023 at 2:26 PM Parcollet Titouan @.***>
wrote:

@mravanelli https://github.com/mravanelli @Adel-Moumen
https://github.com/Adel-Moumen @pplantinga
https://github.com/pplantinga I have an idea to simplify this PR. What
about we take only the 'HuggingFaceTransformer(nn.Module):' class and make
it an abstract class. THe idea is to force the user to override a few
mandatory functions, like the forward, and the freeze params functions.
THis would be MUCH simpler to grasp. There is no way we release a SB v1.0
with huggingface_wav2vec.py in the lobes while it's not even wav2vec2
anymore ...
—
Reply to this email directly, view it on GitHub
https://github.com/speechbrain/speechbrain/pull/1596#issuecomment-1666938782,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEA2ZVQCBLKBELEHWEEZ2VDXT7OT5ANCNFSM6AAAAAAQ5SH6PY
.
You are receiving this because you were mentioned.Message ID:
@.***>


Updated for Oracle v19c

Adjusted content to work with the latest version of Oracle

@Dbakevlar
Can you review the proposed changes? Note that they're extensive enough that they'd benefit from staging and automated checks in the private repo. Can you move the commits into the private repo or let us know if you'd like assistance with that?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

@Dbakevlar @glbwell0 - This content can't be merged in its present state. The commits must be moved to the private repo for automated checks and staging. Also, the placeholder content (lines 177, 181, 185) must be replaced. HTML text formatting must be replaced with Markdown. @Dbakevlar can you move the commits? Let us know if you'd like help doing that.
#hold-off

@Dbakevlar - we'll move the commits in this PR to the private repo as soon as we have time... and notify you in the PR there. Thanks.

@glbwell0 @Dbakevlar - before we can move the commits, we need to resolve the merge conflicts. I will do attempt to do that now, but heads up to check my choices. thanks.

@glbwell0 @Dbakevlar -
IMPORTANT NOTE:  When I was addressing the merge conflicts, I noticed several issues that will block our ability to merge these changes even after we move the commits to the private repository. These are:

use of HTML for formatting. (eg. <b>) This is not allowed. Can you replace all HTML formatting with markdown? In the case of <b> you can use ** .
placeholders for <insert image>. I removed several of these, but you should make sure none remain. You can put the images into the PR or we can move the commits without reference to where the images will go (and they can be added in a later PR).

@glbwell0 - can you update your PR to solve the above issues and let us know by entering #sign-off when you're done.
@Dbakevlar - we can technically move the commits without these issues addressed, but they would still need to be fixed in the private repo (and you'd be on task for that).

@jjaygbay1 my review is finished, recommended some changes to content.

I apologize for not being available last week -I was enjoying some vacation.
Are there any conflicts remaining?   I can take it from here.
If not...  Who can complete the Merging?

@glbwell0 please see the previous comments left by the ms review team and resolve those issues, thank you.

@jjaygbay1, @Dbakevlar, @glbwell0  - Many of the changes in this PR overlap with the changes that were merged in https://github.com/MicrosoftDocs/azure-docs-pr/pull/234432, resulting in 27 merge conflicts.
Should this PR be closed? If not, can you resolve the conflicts?

I believe we can close this PR.  Looks like another PR with this work was released at https://learn.microsoft.com/en-us/azure/virtual-machines/workloads/oracle/configure-oracle-dataguard.

I believe we can close this PR.  Looks like another PR with this work was released at https://learn.microsoft.com/en-us/azure/virtual-machines/workloads/oracle/configure-oracle-dataguard.


[Feature]: Python support for api integration

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Awesome project.
postman is cool but can python programmers get basic REPL environment for integration.
Maybe something like https://replit.com
.Thanks
Why should this be worked on?
Postman is cool, but python programmers are use to the terminal. Postman has too much fancy stuff.  Adding REPL  will make api integration bearable for python programers like myself.

@Nllii thank you for the feature request! I have a question about this. As of today, our REST API integration is only used to talk to an already created API. Are you looking to create an API inside Appsmith with Python or call a Python script that is already created somewhere on your server

Yes. To call a python script.
I have tried bokeh, plotty, pyscript and Mongodb charts to create a dashboard, it takes up too much resources and too much fancy stuff.
This project has what I need.
Now adding python for basic  CRUD and HTTP request. With a drag and drop UI attached to ones code.
You know. Like scratch or REPL apps

The current  flow for attaching QUERIES/JS(postman)  is in Javascript and  API  calls are in postman; then binding to canvas widget.
JSObject is in javascript, this  is where python is needed for creating functions for python users.
To conclude  I'm using cUrl imports for now to get my API urls in, then creating my functions in javascript. but;  I am I want  to write python scripts inside Appsmith to bind to the widgets and make api calls using
requests.get(url = URL, params = PARAMS)  instead of postman and JSObject.

Closing in favor of #23506 13577


logger problems on fmu-v4 with latest master 5b52cd0fe6

This is the console output for 2 successive arm/disarm sequences (via RC):
nsh>
nsh> INFO  [logger] Start file log
INFO  [logger] Opened log file: /fs/microsd/log/sess001/log001.ulg
INFO  [logger] closed logfile, bytes written: 372848
INFO  [logger] Start file log
ERROR [logger] Can't open log file /fs/microsd/log/sess001/log002.ulg

Result is that only the first arm/disarm segment is logged. Must reboot in order to recover.
logger on/off works properly, creating sequentially numbered files within the same session folder.
sdlog2 never opens a logfile when armed via RC, or QGC or commander.
sdlog2 on/off works correctly also.

@bkueng Can you look into that?

@kd0aij I cannot reproduce this. When arming/disarming via RC several times, logger always successfully starts/stops. The same with sdlog2 then started with -a.
I know the symptoms don't look like it, but did you try another SD card or reformatting?

Yes, I tried a second SD card. And both cards have been fine in previous flights.
I first encountered the problem flight testing the pr-mixerEnhancements branch, and reproduced it on the bench in that branch. Then I fetched upstream, checked out/pulled master, "make clean" then "make px4fmu-v4 upload" to generate the results filed in this issue.
Did I forget something? There must be something wrong with my builds.

@bkueng I'm also having trouble with mixing on the FMU in https://github.com/PX4/Firmware/pull/5720
I'd appreciate it if you could try that PR on fmu-v2 to see if the new AUX_TRIM parameters have any effect on the aux pwm outputs. In my testing, although the new MAIN_TRIM parameters work as expected for the main pwm outputs, on the IO coprocessor, the aux outputs do not. Perhaps the two problems are somehow related.

@bkueng I just used QGC to flash master onto the same pixracer, and again reproduced the problem on the bench with a third SD card. That rules out my local build (gcc 4.9.3) as the cause.

Sure, I'll look into it when I find the time. Can you try to find out why logger fails to open the file (print the errno)? Maybe it runs out of file descriptors? Do you use a custom topics list?

not using a custom file list. errno=24 which is "too many open files" according to nuttx/include/errno.h
I would have though regression testing is done using the default (all?) topics.

It is totally acceptable to not be able to log all topics on a resource constrained system. So I would like to see this fixed by closing stdin right before opening the log file. The logger should also output a MAVLink message that it was not able to log all topics.

@LorenzMeier I don't understand why @bkueng couldn't reproduce the problem on fmu-v2 though.
Is there a particular reference platform (e.g. fmu-v2) that default configs are tested against?

Just to clarify: closing stdin will free the file descriptor needed for the log file

@bkueng I just verified by bumping CONFIG_NFILE_DESCRIPTORS up to 52, that the new logger is failing to release the descriptor used for writing the logfile. With that change, it succeeds in opening and writing two logs, then fails as before:
INFO  [logger] Start file log
INFO  [commander] Takeoff detected
INFO  [logger] Opened log file: /fs/microsd/log/sess001/log001.ulg
INFO  [logger] closed logfile, bytes written: 253952
INFO  [commander] Landing detected
INFO  [logger] Start file log
INFO  [logger] Opened log file: /fs/microsd/log/sess001/log002.ulg
INFO  [logger] closed logfile, bytes written: 287199
INFO  [logger] Start file log
ERROR [logger] Can't open log file /fs/microsd/log/sess001/log003.ulg, errno: 24

I can't see any obvious differences between the way sdlog2 closes the file and the way the new logger does it, but obviously something is different. Is there any relationship between these files and "virtual" files? Logging fewer topics is also a usable workaround...

I was able to reproduce, w/o any HW attached. We were leaking a file descriptor, which is why it worked the first N times before it failed.
@kd0aij can you test #5783?

verified in latest master on fmu-v4


Nublado v3

Per our discussion at co-work, I'm going to merge this up and then we will reverse polarity and Russ will do a PR for me to review.


Redact sensitive information

Sensitive information #62 is redacted.  By convention sensitive information is any property whose name starts with a ~.

A redacted resource
{
   standard: 'foo',
   '~dob': '██████',
  _metadata: {
     redactions: { '/~0dob': 'replaced' }
} }


Interaction in custom render causes Notebook cell to switch to Markdown

VS Code Version: 1.54.0-insider
OS Version: Darwin x64 19.6.0

Steps to Reproduce:

Clone github.com/tanhakabir/rest-book
Run in Extension Host
Create an empty notebook by creating a file with a .restbook ending
Create 2 simple REST requests (one GET and one illegal POST) in two separate cells
Try to search for something in either cell that exists within the the response from the call.
See that the bottom cell automatically get flipped into markdown.

Only when the word exists in the response: I'm adding span tags on search to highlight the search term. This was not an issue in the past.

This wasn't the behavior before. One cell didn't affect the rendering of another.

I'm not able to reproduce this with latest API. can you confirm?

Can confirm still occurs with latest API:

Ran one more time with 1.54.0-insiders to be sure: