Multi-select widget dropdown: animation and position

@Tooluloope Some comments about the multi-select widget animation: https://www.loom.com/share/fd7cb9d07db841cf938d49c75ba85d97
Originally posted by @areyabhishek in https://github.com/appsmithorg/appsmith/issues/5069#issuecomment-895313653

Agreed, Note @Tooluloope
We should have consistency between the select and multi-select widget

The dropdown position need a solution, How can we always open it at bottom
The animation applied to the widget should be fade in/out just like the select widget.

@somangshu the dropdown will need to open from the top too for long lists.

@areyabhishek @somangshu From the video, The reason why it opens from the top is because of the limited space from the widget to the end of the page.
The Multiselect on the Left is closer to the bottom and the one of the right is a bit higher. That's why there's a repositioning.
Select WIdget also does the same, due to availability of space.

@Tooluloope i feel that this behaviour is alright given that we have consistency between the select and multi-select widgets. Let change the animation to what we see with the select widget


[AVM Module Issue]: 500 Internal Server Error When Creating Container App Job

Check for previous/existing GitHub issues

[x] I have checked for previous/existing GitHub issues

Issue Type?
No response
(Optional) Module Version
0.1.3
(Optional) Correlation Id
3c35f989-77ec-0577-99b8-32e0b61d8239
Description
Environment
Cloud Adoption Framework (CAF) Landing Zones, using vWAN in a hub-and-spoke configuration. Spokes are in their own Landing Zone subscriptions, with a VNet deployed and connected to the vWAN Hub. This also uses a centralized Private DNS Zone configuration.
Issue
While deploying the module, using an existing VNet, subnets, and Private DNS Zone, the terraform apply throws the following error:
module.github_runners.module.container_app_job[0].azapi_resource.job: Creating...
module.github_runners.module.container_app_job[0].azapi_resource.job: Still creating... [10s elapsed]
module.github_runners.module.container_app_job[0].azapi_resource.job: Still creating... [20s elapsed]
module.github_runners.module.container_app_job[0].azapi_resource.job: Still creating... [30s elapsed]
module.github_runners.module.container_app_job[0].azapi_resource.job: Still creating... [40s elapsed]
module.github_runners.module.container_app_job[0].azapi_resource.job: Still creating... [50s elapsed]
╷
│ Error: Failed to create/update resource
│
│   with module.github_runners.module.container_app_job[0].azapi_resource.job,
│   on .terraform/modules/github_runners/modules/container-app-job/main.tf line 1, in resource "azapi_resource" "job":
│    1: resource "azapi_resource" "job" {
│
│ creating/updating Resource: (ResourceId "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.App/jobs/caj-ghr"
│ / Api Version "2023-05-01"): PUT
│ https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.App/jobs/caj-ghr
│ --------------------------------------------------------------------------------
│ RESPONSE 500: 500 Internal Server Error
│ ERROR CODE: InternalServerError
│ --------------------------------------------------------------------------------
│ {
│   "error": {
│     "code": "InternalServerError",
│     "message": "Internal server error occurred. correlation ID: 3c35f989-77ec-0577-99b8-32e0b61d8239",
│     "details": null,
│     "target": null,
│     "additionalInfo": null,
│     "traceId": null
│   }
│ }
│ --------------------------------------------------------------------------------

It seems that most/all other resources were successfully created though.

Terraform Code
module "github_runners" {
  source  = "Azure/avm-ptn-cicd-agents-and-runners/azurerm"
  version = "~> 0.1.3"

  location = "Canada Central"
  postfix  = "ghr"

  resource_group_creation_enabled = true

  version_control_system_type                  = "github"
  version_control_system_organization          = "OrgName"
  version_control_system_repository            = "RepoName"
  version_control_system_personal_access_token = var.github_personal_access_token

  virtual_network_creation_enabled = false
  virtual_network_id = local.virtual_network_id

  container_registry_private_dns_zone_creation_enabled = false
  container_registry_dns_zone_id = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/RGName/providers/Microsoft.Network/privateDnsZones/privatelink.azurecr.io"

  container_app_subnet_id   = local.container_app_subnet_id
  container_instance_subnet_id = local.container_app_subnet_id
  container_instance_subnet_name = var.subnet_name

  container_registry_private_endpoint_subnet_id = local.private_endpoint_subnet_id
  container_registry_private_endpoint_subnet_name = var.private_endpoint_subnet_name
}

terraform {
  required_version = ">= 1.9.2"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.113"
    }
    modtm = {
      source  = "azure/modtm"
      version = "~> 0.3"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.5"
    }
    time = {
      source  = "hashicorp/time"
      version = "~> 0.12"
    }
  }
}

provider "azurerm" {
  use_oidc = true
  features {}
}

data "azurerm_client_config" "current" {}
data "azurerm_subscription" "current" {}

locals {
  virtual_network_id = format("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Network/virtualNetworks/%s",
    data.azurerm_subscription.current.subscription_id, var.virtual_network_resource_group, var.virtual_network_name
  )

  container_app_subnet_id = format("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Network/virtualNetworks/%s/subnets/%s",
    data.azurerm_subscription.current.subscription_id, var.virtual_network_resource_group, var.virtual_network_name, var.subnet_name
  )

  private_endpoint_subnet_id = format("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Network/virtualNetworks/%s/subnets/%s",
    data.azurerm_subscription.current.subscription_id, var.virtual_network_resource_group, var.virtual_network_name, var.private_endpoint_subnet_name
  )
}

Thanks for reporting this. I will try to take a look in the next week.

I re-ran the same execution today (as a continuation of the original terraform apply), and it progressed farther but still failed.
In the portal, the Container App Job did appear to be created...

But the Terraform output threw the following error:
module.github_runners.module.container_app_job[0].azapi_resource.job: Still creating... [20m51s elapsed]
2024-11-05T12:52:05.943-0500 [ERROR] provider.terraform-provider-azapi_v1.15.0: Response contains error diagnostic: @module=sdk.proto diagnostic_severity=ERROR tf_proto_version=6.4 @caller=github.com/hashicorp/terraform-plugin-go@v0.22.1/tfprotov6/internal/diag/diagnostics.go:58
  diagnostic_detail=
  | creating/updating Resource: (ResourceId "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.App/jobs/caj-ghr" / Api Version "2023-05-01"): GET https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.App/locations/canadacentral/containerappsjobOperationStatuses/3628c3a5-089e-4130-a625-4ba6036b44c8
  | --------------------------------------------------------------------------------
  | RESPONSE 200: 200 OK
  | ERROR CODE: ContainerAppOperationError
  | --------------------------------------------------------------------------------
  | {
  |   "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.App/locations/canadacentral/containerappsjobOperationStatuses/3628c3a5-089e-4130-a625-4ba6036b44c8",
  |   "name": "3628c3a5-089e-4130-a625-4ba6036b44c8",
  |   "status": "Failed",
  |   "error": {
  |     "code": "ContainerAppOperationError",
  |     "message": "Failed to provision revision for container app 'caj-ghr'. Error details: ."
  |   },
  |   "startTime": "2024-11-05T17:31:11.2073342"
  | }
  | --------------------------------------------------------------------------------
   diagnostic_summary="Failed to create/update resource" tf_provider_addr=registry.terraform.io/Azure/azapi tf_req_id=0327e601-5d83-babd-ebc1-12abcf264d8c tf_resource_type=azapi_resource tf_rpc=ApplyResourceChange timestamp=2024-11-05T12:52:05.943-0500
2024-11-05T12:52:06.050-0500 [ERROR] vertex "module.github_runners.module.container_app_job[0].azapi_resource.job" error: Failed to create/update resource
╷
│ Error: Failed to create/update resource
│
│   with module.github_runners.module.container_app_job[0].azapi_resource.job,
│   on .terraform/modules/github_runners/modules/container-app-job/main.tf line 1, in resource "azapi_resource" "job":
│    1: resource "azapi_resource" "job" {
│
│ creating/updating Resource: (ResourceId "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.App/jobs/caj-ghr"
│ / Api Version "2023-05-01"): GET
│ https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.App/locations/canadacentral/containerappsjobOperationStatuses/3628c3a5-089e-4130-a625-4ba6036b44c8
│ --------------------------------------------------------------------------------
│ RESPONSE 200: 200 OK
│ ERROR CODE: ContainerAppOperationError
│ --------------------------------------------------------------------------------
│ {
│   "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.App/locations/canadacentral/containerappsjobOperationStatuses/3628c3a5-089e-4130-a625-4ba6036b44c8",
│   "name": "3628c3a5-089e-4130-a625-4ba6036b44c8",
│   "status": "Failed",
│   "error": {
│     "code": "ContainerAppOperationError",
│     "message": "Failed to provision revision for container app 'caj-ghr'. Error details: ."
│   },
│   "startTime": "2024-11-05T17:31:11.2073342"
│ }
│ --------------------------------------------------------------------------------

Container App Job
When looking at the Container App Job in the portal, the Settings > Secrets view throws this error: "Internal server error occurred. correlation ID: 1b4ef41b-b351-4bff-9795-cca9311c19f9"

Any update on this, as it's an "internal server error"?

I have been unable to replicate this issue so far. It could be region specific of course. I'll try Canada Central, but if you could try another region to see if you get the same issue that would be helpful.

I have added further tests to this branch to try to reproduce your issue and I hard coded Canada Central too, but no luck reproducing your issue so far: https://github.com/Azure/terraform-azurerm-avm-ptn-cicd-agents-and-runners/pull/81
Could this be DNS related? How is DNS configured on your VNET here?

Thanks @jaredfholgate,
I am not able to deploy outside of Canada, due to region restrictions for this client.
The client's environment setup, is using the Cloud Adoption Framework (CAF), and Azure Landing Zones. We are trying to deploy the GitHub Runners in a spoke VNet. All spokes direct DNS traffic through the central Azure Firewall, which itself is a proxy to the Private DNS Resolver (which is in its own spoke).

Thanks @jaredfholgate, I am not able to deploy outside of Canada, due to region restrictions for this client.
The client's environment setup, is using the Cloud Adoption Framework (CAF), and Azure Landing Zones. We are trying to deploy the GitHub Runners in a spoke VNet. All spokes direct DNS traffic through the central Azure Firewall, which itself is a proxy to the Private DNS Resolver (which is in its own spoke).
Also, interestingly, after attempting to deploy, the Container App Environment cannot be deleted. It throws an error Failed to delete Container Apps environment {0}: {1}: . We've opened a Support Request (ID:<PHONE_NUMBER>011826) to help us remove the empty CAE.

That sounds fine. I think the only real difference in my testing is the fact that the private DNS zone is in a different subscription. I'm unsure if this would cause the 500 error. There isn't really very much to go on there. Unless you can find something more detailed in the logs I don't think there is much more I can do from the Terraform perspective.

Looking at this again, I re-tested with the latest version (v0.2.0) of the AVM module.
It still threw the same 500 Internal Server Error error.
All the other resources seem to create without issue. The error consistently occurs when it attempts to create the module.github_runners.module.container_app_job[0].azapi_resource.job resource.
So, thinking about it further, I though I'd try to manually create the Container App Job in the already created Container App Environment, using the exact values from the terraform plan.
Terraform Plan:
Terraform will perform the following actions:

  # module.github_runners.module.container_app_job[0].azapi_resource.job will be created
  + resource "azapi_resource" "job" {
      + body                      = {
          + properties = {
              + configuration = {
                  + eventTriggerConfig = {
                      + parallelism            = 1
                      + replicaCompletionCount = 1
                      + scale                  = {
                          + maxExecutions   = 100
                          + minExecutions   = 0
                          + pollingInterval = 30
                          + rules           = [
                              + {
                                  + auth     = (sensitive value)
                                  + metadata = {
                                      + owner                     = "bcgov-c"
                                      + repos                     = "ecf-azure-startup-sample-app-serverless"
                                      + runnerScope               = "repo"
                                      + targetWorkflowQueueLength = "1"
                                    }
                                  + name     = "github-runner"
                                  + type     = "github-runner"
                                },
                            ]
                        }
                    }
                  + registries         = [
                      + {
                          + identity = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.ManagedIdentity/userAssignedIdentities/uami-ghr"
                          + server   = "acrghr.azurecr.io"
                        },
                    ]
                  + replicaRetryLimit  = 3
                  + replicaTimeout     = 1800
                  + secrets            = (sensitive value)
                  + triggerType        = "Event"
                }
              + environmentId = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.App/managedEnvironments/cae-ghr"
              + template      = {
                  + containers = [
                      + {
                          + env       = (sensitive value)
                          + image     = "acrghr.azurecr.io/github-runner:bc4087f"
                          + name      = "caj-ghr"
                          + resources = {
                              + cpu    = 1
                              + memory = "2Gi"
                            }
                        },
                    ]
                }
            }
        }
      + id                        = (known after apply)
      + ignore_casing             = false
      + ignore_missing_property   = true
      + location                  = "Canada Central"
      + name                      = "caj-ghr"
      + output                    = (known after apply)
      + parent_id                 = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr"
      + schema_validation_enabled = true
      + tags                      = {
          + "account_coding" = "000000000000000000000000"
          + "billing_group"  = "e833c2"
          + "ministry_name"  = "CITZ"
        }
      + type                      = "Microsoft.App/jobs@2023-05-01"

      + identity {
          + identity_ids = [
              + "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/rg-ghr/providers/Microsoft.ManagedIdentity/userAssignedIdentities/uami-ghr",
            ]
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
          + type         = "UserAssigned"
        }
    }

Interestingly, in the Azure portal, when entering the values manually, the value that Terraform is trying to use for the container job name (ie. caj-ghr), produces an error!

The value doesn't seem to conflict, since "Value must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character and cannot have '--'. The length must not be more than 32 characters."
Manually changing that to cajghr passes validation though!
Further, the Container details shows the error "Cannot access ACR 'acrghr.azurecr.io' because admin credentials on the ACR are disabled."

When I manually change the Admin Credentials option on the Azure Container Registry via the portal, and re-run terraform apply it still throws the 500 internal server error.
A few thoughts come to mind...

Q: Is the API version used for Microsoft.App/jobs correct? Looking at the latest Microsoft.App jobs API documentation, I see 7 API versions newer!
Q: Is the Container Registry required to have Admin User enabled?

Hi. I am confident the configuration, settings and API version are correct since we run end to end tests, which consistently pass. We have also run them in Canada Central.
The job uses an managed identity to access to the ACR, so not admin credentials are required.
You can supply your own container image if you don't want to use our dockerfiles. We provide them as a convenience to get started quickly.
The name validation of the container app job is something that would need to be raised via a support ticket as it is outside of our scope here.
You could try adding a longer postfix to see if the 3 letter post fix is the underlying cause? We can add some validation to avoid short names if that is the case.

Hey @jaredfholgate, quick (and semi-positive) update for you...
We've deployed the module (specifically for GitHub), using the basic/standard example provided.
As previously mentioned, this environment is created based on the Cloud Adoption Framework (CAF) and Enterprise Scale (ES). Because of that, we had to make a few adjustments to make the "basic" implementation deploy, namely:

Provide a custom container_registry_name

The default deployment throws an error that the name (which uses acr${var.postfix}) was not unique

Create an Azure Policy exemption for the following policies:

Subnets should have a Network Security Group
Deny the creation of private DNS
Configure Azure PaaS services to use private DNS zones
Public network access should be disabled for PaaS services

We successfully tested the GitHub Runner (based on the Azure Container App implementation) using one of our GitHub repos.
Future Forward
We can close this GitHub Issue, since the basic implementation works. However, we now plan to "walk back" the basic deployment to adhere to the controls in place in the CAF Landing Zone.
We can provide feedback to the AVM team (in the form of GitHub Issues) on how the module should be changed to support  the CAF-ES landing zone environment.

Hi
Glad you got somewhere with it.
It would be really helpful if you could provide those policy and naming related finds in a new issue.
I am confident they can all be resolved, apart from the publicly accessible one. Would be good to understand which resource(s) that applies to.
We intend to use this module as part of our application landing zones accelerator, which is an extension of the platform landing zones accelerator. Therefore would need to make these changes anyway to work with the standard policy set.

Thanks for the support @jaredfholgate. Very happy to hear that there's focus on making the module work with Landing Zones.
I'll submit GitHub Issues as we work through each component, and potential challenges with using them/getting them to work in a CAF Landing Zone.
Here's one that we already submitted: [AVM Module Feature Request]: Option To Disable NAT Gateway Deployment

FYI @jaredfholgate,
Now that we've successfully deployed the GitHub Runner version of the module, I attempted to deploy the module again in the same CAF environment (with the same Azure Policy exceptions in place as the successful deployment), but this time with the following customizations:

Creating a Resource Group externally (ie. for naming conventions)
Using an existing VNet/subnets (ie. not letting the module create a new one)
Using existing (and more importantly, centralized) Private DNS Zones (ie. instead of letting the module create one and link it with the VNet)

So this (in my mind), tells me there is more than just the Azure Policies that are causing the deployment issues. I'll continue to dig into it, and open any applicable GitHub Issues with suggestions for CAF-based environment implementation.


Add allow_untyped option to NoSuperfluousPhpdocTagsFixer

This PR continues #3930.
I described my point of view in that PR conversation.

In particular, such a requirement also has PHPStan

`@param mixed $foo` means you can pass any type of value to $foo, which is also what having no typehint in the definition of $foo means, hence the rule considers it superfluous. I see no value in adding such tag.

(https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/3930#issuecomment-406184572)
The value is adding itself, it states that the param is documented, i.e. the author did not forgot to document the param. When there is more than one param and one is typed and one mixed I like those both to be documented and only the typed ones.
This is why I typically don't use the NoSuperfluousPhpdocTagsFixer. I think the fixer does what it should do so I don't agree on adding the option by this PR. If you want to keep the params; remove the fixer from your config :)
That does leave the initial valid issue report on the conflict between the fixers that we should try to resolve in some other way.

I respect your point, but what should I do in my case?
The coding standards in our company require, among other things, the following rules:

Always explicitly specify the type of each argument and the return type (typehint or docblock). PHPStan helps with this (why this rule was added you can read here https://github.com/phpstan/phpstan-strict-rules/issues/14). The very existence of a rule in PHPStan means that someone other than me has encountered a similar issue.
Docblock should not contain superfluous tags @param. Superfluous one means that the method signature already contains the corresponding typehint. NoSuperflousPhotsTagsFixer can help with this except for the mixed type because it contradicts the first rule.

Adding an option to the fixer will make it more flexible, solve my issue and does not break BC.

Add a description to the @param mixed $vars and it will no longer consider them superfluous.
Also, mixed is not always really mixed, it can't always handle everything. While a lot of work, you could specify what it actually accepts. (And use the first option in other cases.)

In the sample the mixed is most likely not correct; you cannot serialize most resource-es so the doc could've been made better ;)
Adding the description might be the best way to go to explain why something is mixed.
I can see both sides now, so I'm +0 on this change now.

Actually in Symfony implementation the serializer can get any type (even resource) but throws exception if serialization of this type does not supported.
We can argue this is good or bad, but not in this MR. This works that way and this needs to be documented.

#4002 was just merged with similar feature so I'm closing here. Thank you for your time.


Can't use collection helper on FS.Collection.

Say Photos is a FS.Collection. And when you do Photos.files.helpers({...}); you get Error: [Can't apply helpers to 'cfs.photos.filerecord' a transform function already exists!] error message. CollectionFS doc says underlying Mongo.Collection instance available through myFSCollection.files.

Hey @zayaisshawn just as the error states, there's already a transform function defined for that collection. Perhaps collectionfs has already created one? I don't have any experience using that package, maybe it has its way for you to add methods to the transform function it has setup. Have you opened an issue over there?

Hi @dburles I have not opened an issue there. Yes, it turns out FS.Collection has its own transform function created. In my understanding, It is different from normal transform function you use on any mongo.collection. I have not found the answer to this problem yet. However, since I use reywood:publish-composite, it's easier to just publish related cursor and use template helper. Someone on stackoverflow suggested to define custom function on fs.collection prototype. I lack experience to do that but I will look further into that later. Thank you.

Hey thanks @zayaisshawn if you do find how to work around the incompatibility I'd appreciate it if you would let me know

collectionFS could add support for this package

True. If there is no pull request already, I'll add one.


Tag namespaces cause cssSelector() to fail

I have been seeing the following error parsing a huffingtonpost site:
Exception in thread "main" org.jsoup.select.Selector$SelectorParseException: Could not parse query 'fb:comments': unexpected token at ':comments'
	at org.jsoup.select.QueryParser.findElements(QueryParser.java:196)
	at org.jsoup.select.QueryParser.parse(QueryParser.java:65)
	at org.jsoup.select.QueryParser.parse(QueryParser.java:39)
	at org.jsoup.select.QueryParser.combinator(QueryParser.java:81)
	at org.jsoup.select.QueryParser.parse(QueryParser.java:51)
	at org.jsoup.select.QueryParser.parse(QueryParser.java:39)
	at org.jsoup.select.Selector.<init>(Selector.java:83)
	at org.jsoup.select.Selector.select(Selector.java:105)
	at org.jsoup.nodes.Element.select(Element.java:273)
	at org.jsoup.nodes.Element.cssSelector(Element.java:495)

There is a tag "fb:comments" in this site; which by XML is not totally nonsense, but of course does not work well with CSS.
Namespaces supposedly are supported by CSS, but with a pipe instead of a colon.
Note: I have not tested this change - feel free to solve this differently.

Thanks. Do you want to add a test? Also good to have a comment on the replace so we remember what it's doing.

With unit test + comment now.

Thanks! Merged.


Create reference when accepting a share

A,K.A mounting the share

Codecov Report

Merging #243 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #243   +/-   ##
=======================================
  Coverage   74.25%   74.25%
=======================================
  Files           7        7
  Lines         268      268
=======================================
  Hits          199      199
  Misses         64       64
  Partials        5        5

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97f589f...f0c4fa2. Read the comment docs.

Overcomed by #394


fix: Update pr-test-file-check.yml

Description

[!TIP]
Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content team).
Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.

Fixes Typo in Step name

[!WARNING]
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation
/ok-to-test tags=""
:mag: Cypress test results

[!CAUTION]
If you modify the content in this section, you are likely to disrupt the CI result for your PR.

Summary by CodeRabbit

Chores

Corrected a typo in the workflow file to ensure the correct action is invoked during execution.

@nidhi-nair  Can i please request your time for this PR review.

Hey @gurudattgd04, thank you so much for fixing this. ❤


How do I use cargo like make?

I want to rewrite my kernel via rust.
But I find it lack of a makefile to build it even though I could write it in program with package gcc and ld.

Thanks for the report! Currently Cargo is not striving to be a complete build system, however, and for use cases like this the recommended steps are to specify crate-type = ["staticlib"] in Cargo.toml and then use that library to then call gcc or ld manually to link everything together. This means you'll probably still need a Makefile shim, but you can leave most of the heavy lifting to Cargo.


Can't approve users in the waiting room.

I updated the plugin to the newest version and now I can't approve users from the waiting room. See the screenshot.

I noticed that when the waiting room is enabled, there is no visible logo in the desktop view. There is a visible logo in the mobile view though, but it is not centered. The same is true for the PlugnMeet native logo.
Is it only the case with my instance of the plugin or some bug? Thanx.

@vitaliybaran sorry. You can try from demo api information or https://demo.plugnmeet.com/login.html

@vitaliybaran sorry. You can try from demo api information or https://demo.plugnmeet.com/login.html

seems to be ok


Exclude JS file in sniffing

For example when there is modernizr.min.js, sniffer shows error ERROR	Stylesheets must be registered/enqueued via wp_enqueue_style
May be we need to exclude JS file from such check.

There is a checkbox to sniff only PHP files, and I'm checking for a minimised files here:
https://github.com/WPTRT/theme-sniffer/blob/ccb7bc0682c07bddccaab259d2162a846b6bc06e/src/callback/class-run-sniffer-callback.php#L376-L381
Although this is a super tricky thing to check so any sanity checking is welcomed 🙂

Fixed in the latest release


Update eslint to v9

Waiting on sub dependencies to be updated

fixed in https://github.com/montanatrouthunters/montanatrouthunters.com/commit/df249b247ba555f1822e10e0c83ed35d8184eca8


stats.truncnorm.rvs 100x slower in v1.4.x than v1.3.3

The 1.4.x releases of scipy seem to have resulted in greater than 100x slow-down of stats.truncnorm.rvs sampling relative to the 1.3.x versions:
Reproducing code example:
With scipy==1.3.3
>>> pip install scipy==1.3.3
>>> python -m timeit "from scipy.stats import truncnorm; truncnorm.rvs(-2, 2, size=(50, 50, 50))"
1 loop, best of 5: 6.45 msec per loop

With scipy==1.4.0
>>> pip install scipy==1.4.0
>>> python -m timeit "from scipy.stats import truncnorm; truncnorm.rvs(-2, 2, size=(50, 50, 50))"
1 loop, best of 5: 885 msec per loop

Similar slowdown has also been observed with scipy==1.4.1.
Scipy/Numpy/Python version information:
To reproduce the slow sampling behavior:
1.4.0 1.16.4 sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)

To reproduce the fast sampling behavior:
1.3.3 1.16.4 sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)

Other details of my environment:
- python=3.7.3 (compiled with Clang 8.0.0)
- platform=darwin_x86_64
- cpu=Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz

Probably due to gh-10104. @pvanmulbregt is this expected?

The new version uses np.vectorize and looses standard vectorization AFAICS
It might be better to split the distribution into the standard usecase version (that prefers speed over good tail behavior) and a slow tail usecase version if the slowdown is widespread across methods.
https://github.com/scipy/scipy/issues/2477#issuecomment-17748051

Apologies if this question should be obvious from the code-base, but are there currently any unit tests that check the speed of random sampling? I was a little incredulous at first when a colleague brought this issue to my attention because a 100x slowdown is pretty substantial.

I'm not aware of any unit tests that test for speed. Scipy does have an airspeedvelocity framework that is run on a semi regular basis, see https://pv.github.io/scipy-bench/, but it's not comprehensive by any means. A test for any speed regression could be put in there, but I'm not sure what the formality is for noticing that there are negative changes. This issue is certainly critical for those who want to do calculations with large amounts of random numbers.

IIRC the issue was that truncnorm was broken and was therefore rewritten completely. That's a case where you review for correctness rather than speed (at least, that's what I did). Would be good to look into whether the vectorized approach can still be used - I suspect np.vectorize was used for a good reason.

For my specific user case I'm interested in large numbers of random variates from truncnorm with tails chopped off, i.e. tail behaviour isn't important. Thus, a two state behaviour, as suggested by @josef-pkt, would be useful.

Some comments:
I believe that I've slowed down truncnorm twice in the past year, though each time it was a known side-effect, rather than the goal, of the change.
gh-9900 fixed thread-safety issues arising from distributions setting member variables inside _argcheck().
gh-10104 was a rewrite to fix some asymmetry and overflow issues.
Both impact the speed difference observed here.
truncnorm needs to determine the probability in the truncated tails. Prior to gh-9900, truncnorm calculated this once and stored it in the class. It used scipy.special.ndtr to do that.
After gh-9900, the result is not stored but computed each time. It still uses scipy.special.ndtr if the tail endpoints are in the range [-40, 40], otherwise it does a calculation using logs.
For the ppfs, the calculation may be able to use scipy.special.ndtri as before, or it may have to do a root solving operation.
One other slowdown is by using np.vectorize.  IIRC, this turned out to be needed because during a call to _ppf(a, b, x), the  arguments a, b and c may still be arrays.  I.e. It is not the case that dist.ppf(a, b, q) handles the arrays, and dist._ppf(a, b, q) computes the scalar inverse ppf.  Instead _ppf() may have to do both.  It is not arrays of q that are the issue, it is arrays of a and/or 'b'.
In the case above, for a,b = (-2, 2), the new code should be using the same underlying scipy.special calls, but it has to compute the probability in the tails for every value of q.
Freezing the distribution doesn't help either as that doesn't allow for the precomputation of the tail probabilities, it merely remembers the a,b parameters.

To be a bit more explicit on the need for the np.vectorize:
The call stats.truncnorm.rvs([-10, 10], [-5, 11], size=(5, 2)) generates a call truncnorm._ppf(U, a, b) where U has  size=(5, 2) and contains a random samples from Uniform(0, 1). E.g.
U= [[0.19151945 0.62210877]
    [0.43772774 0.78535858]
    [0.77997581 0.27259261]
    [0.27646426 0.80187218]
    [0.95813935 0.87593263]]
a = [-10 10]
b = [-5 11]

and is expected to return a (5, 2) array such as
[[-5.30971183 10.09591425]
 [-5.15699038 10.15125262]
 [-5.04769914 10.03146819]
 [-5.24240404 10.15906068]
 [-5.00823854 10.20459498]]

truncnorm._ppf calls _truncnorm_ppf to do the work. The np.vectorize wraps _truncnorm_ppf  and turns this one call with arrays into 5*2=10 calls to _truncnorm_ppf(u_, a_, b_), passing in scalars.
Is there a standard way to turn this into just 2 calls, one call for a_,b_=-10, -5 with the first column of U, and one for a_,b_=10, 11, with the 2nd column of U.  E.g.
_truncnorm_ppf([0.19151945, 0.43772774, 0.77997581, 0.27646426, 0.95813935], a=-10, b=-5)
_truncnorm_ppf([0.62210877, 0.78535858, 0.27259261, 0.80187218, 0.87593263], a=10, b=11)

That would enable computing the tail probabilities just once for each instance of distribution shape values.

Added links to gh-2069 and gh-5526 which have relevant discussions.

im still getting crazy slowdowns using the new truncnorm from v1.5.0rc1. A sampler that uses the truncnorm in a joblib.Parallel process takes over 50min to produce 2000 samples whereas in v1.2.1 it took just over 1min @pvanmulbregt


BelongsToMany return soft deleted records of pivot table

Laravel Version: 6.x
PHP Version: 7.2.28
Database Driver & Version: Mysql

Description:
When i use belongstomany pivot tables soft deleted records coming in instance
though raw query for the same givng correct results.
relation i am using
$this->belongsToMany('Navnorth\Core\Http\Models\Group', 'group_members', 'profile_id', 'group_id')
->distinct('group_id')
->where('group_type_id', '!=', (int) array_search('ll_roster', config('core.group.grouptypes')))
->whereNull('group_members.deleted_at');
my tables
groups
id , name, created_at,updated_at,deleted_at
group_members
id,group_id,profile_id,created_at,updated_at,deleted_at
Steps To Reproduce:

Laravel does not support soft deletes on pivot tables.


Release beta

Description
Release new features and updates:

custom claims can be queried for when GraphQL middleware is enabled
multiSig signers are supported in getPermissions
Batch transactions can be subsidized

Breaking Changes

JIRA Link

Checklist

[ ] Updated the Readme.md (if required) ?

/fsat-forward

:tada: This PR is included in version 26.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Update definition of power entering and leaving the battery, update use of efficiency

Implement a new definition of how to deal with efficiency:

If we specify a charging curve (e.g. constant 100 kW), then this 100 kW comes from the grid. In other words, less power arrives in the (vehicle) battery due to the efficiency.
When discharging a (vehicle) battery with 100 kW, 100 kW goes into the grid or wherever it goes. Meaning more power is taken from the (vehicle) battery due to its efficiency.

The (dis)charging curve therefore defines P_in  and P_out in this picture:

More info: https://smooth.readthedocs.io/en/latest/smooth.components.html#module-smooth.components.component_battery
Additionally: Ensure that the efficiency is considered in all strategies and for vehicles and stationary batteries!
Example: In peak_load_window the stat. battery does not consider the efficiency when discharging.

This is already the behavior when using target_power of the battery load/unload functions. See test_components for example, test_charging and test_unload in particular. Do you have an example file where the efficiency is not accounted for, in your opinion?


qt wallet data dir access error handling

Currently qt wallet using exception to signal IO issues with data dir access.
The rest of app doesn't handle it.
Since QT doesn't use exceptions, as well as the rest of the app. Let's switch error processing model for error flags with error messages.

Testing:
This issue is hard to reproduce normally, so far we get the only report.
But I think it should be possible to produce it with permissions.
You can target at directory ~/mwc-qt-wallet/.   For example logs.  You can delete 'logs' and make 'mwc-qt-wallet' read only. In this case error should be reported during logs initialization.

Have deleted the wallet log files and changed the directory ~/mwc-qt-wallet/ permission to READ-Only; afterwards got the below error message.
Wallet got a critical error:
mwc713 process exited due some unexpected error.
mwc713 exit code: 101
thread 'main' panicked at 'could not create address book backend!: LmdbErr(Error::Code(5, 'Input/output error'))', src/main.rs:491:9
You might use command line for troubleshooting:
'C:/Program Files/MWC/mwc713.exe' '--config' 'C:/Users/pavit/mwc-qt-wallet/wallet713v2.toml' '--disable-history'
Press OK to exit the wallet

Wallet is not crashing but giving proper error message, hence closing this ticket.


Refactor source code to ES2015 and remove warnings from SemiStandard

looks good. are we going to use arrow functions?

@oscar-g yes, this is a good feature IMO.


Ignore missing paths during enumeration

It's possible to run into a race condition in which the enumerator lists
lots of repositories and then starts the long process of enumerating through
them. In that time if someone deletes a repo, the enumerator may error out.
Signed-off-by: Huu Nguyen<EMAIL_ADDRESS>

LGTM, except the logrus import.
@caervs @cquon can we carry this one through?

can we carry this one through?

@dmp42 we'd need somebody with push access to @whoshuu's branch, or I can open a separate PR.

Open a new one.

Usurped by #2702


SWARM-1515: Fix RepositoryBuilderMojo.

@mkouba I assume, once in place, you also need another release of the plugin?

ah, ignore my last comment :) it's part of core

Yep, the release and upgrade in https://github.com/wildfly-swarm-prod/wildfly-swarm is necessary.

@kenfinnigan I'll leave that to you


⚠ ClamAV has degraded performance

In c4c5b7c, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 46 ms

Resolved: ClamAV performance has improved in bac81d8.


Add task to enable Einstein Prediction Builder predictions.

Critical Changes
Changes
Adds the EnablePrediction task to change the state of Einstein Prediction Builder predictions from Draft to Enabled.
Issues Closed

I thought I had asked before, but I don't see it now.
Erik: what are the step required to set up an org that can test this? Can you make a minimal CCI repo that represents the test org setup steps?

Moved to internal PR https://github.com/SFDO-Tooling/CumulusCI/pull/2778


Plugin doesn't revert status to online automatically

So, I don'ty know if it's because one of Discord & BD's update broke something, but the plugin doesn't revert the chosen status (Idle, Invisible or DND) to online as it's supposed to do.
It does go from Online to the chosen status though.
I don't know if it's just me or if others are impacted but regardless, I'm merely pointing it out.

EDIT : My bad, I tried tinkering with it and it's working once more. Sorry for the bother

its alright. would you mind sharing what happened?


[tomyumswap] Strategy tomyumswap added

Fixes # .
Changes proposed in this pull request:

New strategy with the name tomyumswap has been added
Based on the token TOMYUM
Minimum voting power needed: 0.1
Voting power and subgraph endpoints have been changed

All test cases are passing

Hi Chaitanya
The same code is working for Pancake (strategy name: cake) and AKS
(strategy name: aks) strategy.
On Mon, 22 Nov, 2021, 9:56 am Chaitanya, @.***> wrote:

@.**** requested changes on this pull request.
In src/strategies/tomyumswap/index.ts
https://github.com/snapshot-labs/snapshot-strategies/pull/176#discussion_r753936903
:

const promises = addresses.map((address) => {

 return fetchVotingPower(address, blockTag, poolAddresses);

}) as ReturnType[];
const votingPowerResults = await Promise.all(promises);

Hi @tomyumswap https://github.com/tomyumswap sending one request per
address won't scale for a large number of votes, please find an alternate
way to get all data for all addresses in a single request
https://github.com/snapshot-labs/snapshot-strategies#checklist-for-adding-a-new-strategy
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/snapshot-labs/snapshot-strategies/pull/176#pullrequestreview-812065857,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AV7YAKSV4WB5NZEHDRKPC5LUNHBBBANCNFSM5IPDFVFA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Probably cake strategy needs to be changed too, on aks strategy, it is not doing for all addresses, 🤔 if we don't change, it won't def. scale for larger votings

Hi Chaitanya
Pls check the latest commit. I have changed the API to include an array of addresses for fetching the voting power. This should fix the scalability issue of the addresses.


Display config for all proof types

depends on https://github.com/keybase/keybase/pull/3108

ran tests locally, merged to avoid CI failures since the server changes are in


[BUG] Publisher removing API's in the Product that are not included in the change.

Release version
v.6.0.1/5.1.4
Describe the bug
We have product that has references to the multiple API's. We deploying specific one API from this Product. During deployment Publisher is deleting API references in the product that are not included in the deployment.
Expected behavior
Product has all references from the source not dependent on the API being deployed.
Actual behavior
API's are removed from the Product references.
Reproduction Steps
Create at least 2 APIs that has same product, deploy only one API and product that references both API's.

Hi @kantipenko, I've experienced this behaviour using APIOps < v6.0.1, check here: #245.
You mentioned versions v.6.0.1/5.1.4 so I just wanted to make sure - is this still happening for you with version 6.x.x?

@DSpirit  You are rights. It actually worked with 6.0.1, we tried with one of the 6.0.1 Release candidates.
Now we are seeing another behaviour
Previously it was also extracting Policy, groups, Apis(even in 6.0.1RC)

Now it is only product information

Can we have both all related information to the product but that it is not deleting API's that we did not specify from the product on deployment?
We have product with multiple API, but want to deploy specific API's through environments. Ideally API that we deploy should have all related information..

I have created a new issue to address product groups not being extracted. It's a bug, will push a fix. Closing this issue in favor of that one.


wip: bypass native password modal on refresh

Description

Notice

[ ] Have you followed the guidelines in our Contributing guide?
[ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)

closes #
Risk

Testing

Engineering

Operations

Screenshots (if applicable)

Current dependencies on/for this PR:

develop

PR #3923   👈

This comment was auto-generated by Graphite.


Cannot load 3rd party kexts

Describe the issue
3rd party kexts, like macFUSE or the kext I am working on personally, won't load.  After reboot I get a pop-up message that says "System Extension Error", "An error occurred with your system extensions during startup and they need to be rebuilt before they can be used. Please go to Security & Privacy System Preferences to re-enable them."
If I go to Preferences and do as it asks, after reboot the same message appears.
SIP is disabled, developer mode is on, etc.
The same kexts work fine on bare metal.
Configuration

UTM Version: 3.2.3 (57)
Host OS Version: macOS 12.3.1
Intel or Apple Silicon? Apple M1 Max

Logs
Output of log show --debug | grep -i macfuse:
2022-05-16 10:36:20.878345-0700 0x263      Default     0x0                  100    0    kernelmanagerd: validating extension at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:36:20.889780-0700 0x263      Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] checking bundle path /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext for Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:36:20.889807-0700 0x263      Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] checking unstaged bundle path /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext for Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:36:20.961129-0700 0x54c      Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] canLoadKernelExtension - direct, evaluate, Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:36:20.962087-0700 0x54c      Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Kernel Extension ALLOWED: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	/Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext (new)
		/Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:36:21.282160-0700 0x54c      Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] canLoadKernelExtension - direct, evaluate, Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:36:21.282557-0700 0x54c      Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Kernel Extension ALLOWED: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:36:21.282857-0700 0x54c      Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Triggered kext upgrade: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:36:21.282978-0700 0x54c      Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Blocking upgrade dialog for kext that is already pending upgrade: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:36:22.559237-0700 0x2e5      Default     0x0                  100    0    kernelmanagerd: validating extension at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:36:22.560082-0700 0x2e5      Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] checking bundle path /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext for Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:36:22.560102-0700 0x2e5      Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] checking unstaged bundle path /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext for Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.432792-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] loadExtensions(withPaths:withIdentifiers:withPersonalityNames:withDependencyAndFolderPaths:options:withReply:): Loading paths: ["/Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext"], identifiers: [], personalityNames: []
2022-05-16 10:42:44.493065-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: validating extension at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.493840-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] checking bundle path /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext for Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.493858-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] checking unstaged bundle path /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext for Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.509628-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: validating extension at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.510237-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: [com.apple.kernelmanagerd.logging:LoadRequestResolution] At least one of the request extensions / bundled extensions io.macfuse.filesystems.macfuse are NOT in the exclude list. Will do full resolution of load request
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.520349-0700 0x184f     Default     0x0                  100    0    kernelmanagerd: validating extension at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.531168-0700 0x1953     Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] canLoadKernelExtension - direct, evaluate, Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:42:44.531854-0700 0x1953     Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Kernel Extension ALLOWED: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	/Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	Kext io.macfuse.filesystems.macfuse v2085.20 in executable kext bundle io.macfuse.filesystems.macfuse at /Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext (new)
		/Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
	/Library/Filesystems/macfuse.fs/Contents/Extensions/12/macfuse.kext
2022-05-16 10:42:44.994916-0700 0x184d     Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] canLoadKernelExtension - direct, evaluate, Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:42:44.995262-0700 0x184d     Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Kernel Extension ALLOWED: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:42:44.995521-0700 0x184d     Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Triggered kext upgrade: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:42:44.995636-0700 0x184d     Default     0x0                  180    0    syspolicyd: [com.apple.syspolicy.kext:default] Blocking upgrade dialog for kext that is already pending upgrade: Kext (3T5GSNBU6W, io.macfuse.filesystems.macfuse) version: 25d8ea0d2b621051e8900f62f6613c904076c8f9
2022-05-16 10:42:44.995862-0700 0x184f     Error       0x0                  100    0    kernelmanagerd: failed to load/rebuild extensions: Extension with identifiers io.macfuse.filesystems.macfuse not approved to load. Please approve using System Preferences.
2022-05-16 10:42:45.006380-0700 0x1961     Error       0x0                  644    0    load_macfuse: (KernelManagement) Remote encountered error: Error Domain=KMErrorDomain Code=27 "Extension with identifiers io.macfuse.filesystems.macfuse not approved to load. Please approve using System Preferences." UserInfo={NSLocalizedDescription=Extension with identifiers io.macfuse.filesystems.macfuse not approved to load. Please approve using System Preferences.}
2022-05-16 10:42:45.006466-0700 0x1961     Error       0x0                  644    0    load_macfuse: (KernelManagement) Extension load failed: <private>

Same root cause as #3526

This bug still exists in macOS 13.0 Beta (build 22A5266r), running as both host and guest. I tested with UTM 3.2.4.
So Apple hasn't fixed it in Ventura -- at least not yet.

This bug (Apple's bug) still exists in macOS 13.0 Beta 2 (build 22A5286j).

Apple's bug still exists in macOS 13.0 beta 3 (build 22A5295h).

Perhaps Applespecified it that way. They have moving away from kexts for awhile.

Apple's bug still exists in macOS 13.0 beta 3 (build 22A5295h).

Ah just came across this problem myself, sadly after I posted about it on the dev forums. Oh well, maybe I'll get a reply.

Still doesn't work in Ventura beta6 and UTM 4.0.2. Not sure this ticket should be closed?

I reported the inability of enabling kexts on machines running via virtualization.framework as to Apple as FB10145502 (third party kexts cannot be used in Virtualization framework apps) a bit ago and as yet have heard nothing, for what it's worth

To follow up on this, using bputil from a terminal in recovery mode gives what's probably the most useful hint: Error Domain 401, "Failed to create local policy", underlying error 0x600001f64150, domain=com.apple.bootpolicy, code=2, "SEP communication (2)"
So it's almost certainly going to be that Virtualization.framework neither provides access to the real SEP nor emulates one fully, and so, until Apple decides to properly support Virtualization, we're just sad.

Ah I only shared the log output in the Apple Dev Forums; the only lines I see are:
: Kcgen roundtrip failed with: Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM
: Kcgen roundtrip failed checkpoint saveAuxkc: status:error fatalError:Optional("Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM")
: Kcgen roundtrip failed: missing last checkpoint or errors found
: Deleting Preboot content

but could be unrelated.

I've made some progress debugging this problem. But I haven't, unfortunately, found a workaround. And what I have found indicates it happens at a very low level, and is almost certainly the result of a deliberate policy decision by Apple.
After much digging around in "kmutil log show" logs, I ended up in more or less the same place as @lundman. As best I can tell, the error he found (BOOTPOLICY_ERROR_ACM) is the key to this bug. This doesn't happen on bare metal, on either macOS 12 or 13.
One of the things I first noticed is that there isn't any "auxiliary kext collection" at all in macOS 12 and 13 VMs (created using the Virtualization framework). (Note the output from kmutil inspect.) This despite the fact that one of its would-be components is an Apple-signed kext in /Library/Apple/System/Library/Extensions -- RemoteVirtualInterface.kext. (This is also not initially loaded on bare metal. It gets loaded along with your first third-party kext.) A necessary step in creating one is to create a "linked manifest" -- a file ending in *.im4m that gets installed to a LocalPolicy subdirectory of /System/Volumes/iSCPreboot. This happens with no problems on bare metal. But in a VM you see the following errors:
macOS 13
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: bootpolicy_create_linked_manifest: entry
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: SEP command 33 (v7) returned 7: bpbc 5150, 0, 0, 0
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: assert: bpe == 0  (/AppleInternal/Library/BuildRoots/0d3d51bd-555a-11ed-88cc-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/BootPolicy/dylib/dylib.c:2437)
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: bootpolicy_create_linked_manifest: exit: ACM (7)
kcgend: Could not createLinkedManifest and/or update auxKC receipt: Boot policy error: Error creating linked manifest: code ACM (7)

macOS 12
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: bootpolicy_create_linked_manifest: entry
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: SEP command 33 (v7) returned 7: bpbc 4177, 0, 0, 0
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: assert: bpe == 0  (/AppleInternal/Library/BuildRoots/d23df31a-1fa6-11ed-91f3-e2c7c6032f02/Library/Caches/com.apple.xbs/Sources/BootPolicy/dylib/dylib.c:2558)
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: bootpolicy_create_linked_manifest: exit: ACM (7)
...
kcgend: persisting error: Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM
...
kernelmanagerd: Kcgen roundtrip failed with: Boot policy error: Error creating linked manifest: code BOOTPOLICY_ERROR_ACM

So a "linked manifest" is an expression of "local policy", and also of "boot policy". I'd guess "SEP" stands for something like "Secure Enclave Proxy". It's accessed through several software layers (including /usr/lib/libbootpolicy.dylib), the lowest of which seems to be /System/Library/Extensions/BootPolicy.kext. I don't know if the Virtualization frameworks' VMs just emulate Apple's "Secure Enclave" (I suspect they do). But, unlike @dariaphoebe, I had no problems communicating with the "SEP" in my VMs, on either macOS 12 or 13.
The "SEP command" that fails in VMs is "SEP command 33". It always succeeds on bare metal (at least in my experience). I see no references to it online. Over time I'll try to learn more about it. If Virtualization framework VMs emulate the "Secure Enclave", it should be possible, at least in principle, to mess with that emulation, to make "command 33" succeed. I'm sure this won't be easy, though, and I may never learn how to do it.
Someone at Apple must have made a deliberate decision to make "command 33" fail in Virtualization framework VMs. That sits oddly with other indications that they intended to support third-party kexts in these VMs -- for example they let you allow third-party kexts in the Recovery Partition. But Apple is a large company, and this won't be the first time the left hand doesn't know what the right hand is doing.
My logs have lots of entries headed "kcgend:". It's completely undocumented. But as best I can tell it's what actually writes "linked manifests" and kernel caches as your Mac boots. It's not meant to run in a VM (the result of another deliberate decision by someone at Apple). But I was able to find a way around this: Just run sudo /usr/libexec/kcgend once in a VM, and it will always run on subsequent boots. Be aware that this will reboot your machine. When I first tried this trick I waited with bated breath for the reboot to finish -- only to discover that this wasn't the workaround I'd hoped it would be :-(

I've learned that "SEP" means "Secure Enclave Processor" ... probably.

After more digging around, I discovered there are three kernel extensions that can receive "SEP commands". (These are the only three kexts that have readFromSEPBuffer() commands.)
BootPolicy.kext
AppleCredentialManager.kext
AppleSSE.kext

Then I wrote a dtrace script to trace when any of them receives a SEP command.
SEPCommand.d.txt
Rename this file from SEPCommand.d.txt to SEPCommand.d, then run sudo dtrace -s SEPCommand.d in a machine that hosts UTM VMs that use the Virtualization framework. You may first need to turn off SIP (with csrutil disable while booted into the Recovery Partition).
The easiest way to trigger SEP commands to BootPolicy.kext is to check for macOS updates (in the System Preferences panel). The easiest way for AppleCredentialManager.kext is to run Safari.
But you don't see anything when you run or reboot a UTM VM (that uses the Virtualization framework). This tells me that Apple's implementation of the Secure Enclave in the Virtualization framework is an emulation -- possibly not a complete one. It doesn't call out to the Secure Enclave Processor in the host.

I've found out what "SEP command 33" is -- _command_create_linked_manifest() in AppleVPBootPolicy.kext. There's an array named _command_functions(), and a pointer to _command_create_linked_manifest() exists in it at offset 33 X 8.
I've also found where _command_create_linked_manifest() fails, I think. My command of ARM64 assembler is a bit wobbly. But as best I can tell, _command_create_linked_manifest() only returns 7 (== BOOTPOLICY_ERROR_ACM) if its call to _validate_acm_context() fails.
So next I need to figure out why the call to _validate_acm_context() fails.

I've discovered some information about why _validate_acm_context() fails. But its very cryptic, and I can't really interpret it:
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: bootpolicy_create_linked_manifest: entry
kernel: (AppleVPBootPolicy) BootPolicy: creating linked manifest
kernel: (AppleVPBootPolicy) BootPolicy: loading misc storage
kernel: (AppleVPCredentialManager) ACM: verifyPolicy: Verifying BootPolicy (V) on CS[100] (preflight=NO, secure=YES, checkKeybagUUID=NO).
kernel: (AppleVPCredentialManager) ACM: dumpVerificationStatus: BootPolicy (V) on CS[100] *NOT* SATISFIED: 01R -> R.
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: SEP command 33 (v7) returned 7: bpbc 4177, 0, 0, 0
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: assert: bpe == 0  (/AppleInternal/Library/BuildRoots/d23df31a-1fa6-11ed-91f3-e2c7c6032f02/Library/Caches/com.apple.xbs/Sources/BootPolicy/dylib/dylib.c:2558)
kcgend: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: bootpolicy_create_linked_manifest: exit: ACM (7)

There are many other cases where _validate_acm_context() succeeds, even in a VM, even while booting. Compare this, picked more or less at random:
syspolicyd: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: reading blessed macOS local policy /System/Volumes/iSCPreboot/BE2ACA7D-8B5E-4E89-88F2-EE05D5354D88/LocalPolicy/1A59E46C2D579AD45C6282DE32E7B3989AD35CB73D301FD7FFE22D9F11454C59683535F41C9822598EE74081553DC5BA.img4
kernel: (AppleVPBootPolicy) BootPolicy: updating local policy User Authorised Kext List
kernel: (AppleVPBootPolicy) BootPolicy: loading misc storage
kernel: (AppleVPCredentialManager) ACM: verifyPolicy: Verifying BootPolicy (V) on CS[118] (preflight=NO, secure=YES, checkKeybagUUID=NO).
kernel: (AppleVPCredentialManager) ACM: dumpVerificationStatus: BootPolicy (V) on CS[118] SATISFIED: 01S -> S.
kernel: (AppleVPBootPolicy) BootPolicy: Local policy signing
kernel: (AppleVPBootPolicy) BootPolicy: AP hacktivation signing
syspolicyd: (libbootpolicy.dylib) [com.apple.BootPolicy:Library] BootPolicy: SEP command 23 (v7) returned 0

For what it's worth, all the failures happen when the call to _validate_acm_context() is made (ultimately) from kcgend. Calls from syspolicyd, for example, work just fine. I noted above that I had to play a hackish trick to get kcgend to run at startup on a Virtualization framework VM. Maybe it's not working entirely correctly. I'll keep playing with it. But at this point I'm not very optimistic I'll be able to do anything about its problems.
"SEP command 23" is _command_update_local_policy_uakl() in AppleVPBootPolicy.kext.
Above I theorized that these failures are the result of a deliberate policy decision by someone at Apple. I still think that. But now I believe I focused too much on the "SEP command 33" failures themselves. They may be just accidental. Now I suspect that the crucial decision was to not allow kcgend to run in Virtualization framework VMs. I found a way to make it run. But (it seems) I haven't yet been able to make it run correctly.
My logs above came from the output (on a VM) of log show --predicate 'eventMessage contains "BootPolicy"'. A lot of the lines are missing from the output of kmutil log show. I'm discovering how much more useful log show is than the Console app's GUI. Previously I thought most of the output you see when you start streaming in the Console app is simply lost if you're not streaming. I couldn't have been more wrong.
Interestingly, I get no output from running log show --predicate 'eventMessage contains "hacktivation"' on my macOS host. I'm not entirely sure what to think about that.

I've found some potentially useful stuff. The AuxilaryStorage image appears to be an iBoot1 bootloader. According to https://github.com/AsahiLinux/docs/wiki/SW:Boot#stage-1-llbiboot1, it reads a local boot policy from /<volume-group-uuid>/LocalPolicy/<policy-hash>.img4, which contains (among other things) a smb2 key, described as bool: Third-party kernel extensions enabled.
According to that wiki page, it should be in "metadata keys", but although running strings on that file (in the VM) does have some of those keys (such as vuid and lobo), I can't find the other keys (like smb* or sip*).
I'm thinking of asking the how to get these metadata keys on the Asahi IRC, but I'm not sure which channel to use (as they don't have an offtopic channel). The closest I can think of is #asahi-alt - For discussion and support for unofficial third-party distro ports.

Yes, @ktprograms, it'd be interesting to know the contents of the local boot policy file in a VM that's been set up to support loading third party kexts (while booted into the Recovery Partition). I'd bet the bool: Third-party kernel extensions enabled key is set to true, and that it makes no difference -- that "SEP command 33" still fails.
I've got some more ideas about how to make kcgend do what it should in a VM. I'll report back in the next few days whether or not they worked out.
They're based on two observations, one of which is a new discovery:

`/System/Library/LaunchDaemons/com.apple.kcgend.plist has the following two keys:

<key>RunAtLoad</key>
<true/>
<key>LimitLoadToHardware</key>
<dict>
    <key>osenvironment</key>
        <array>
            <string>kcgen</string>
        </array>
</dict>

As best I can tell (from xnu kernel source code), osenvironment is stored in the "device tree" under /chosen. I want to figure out how to read the "device tree", and how to manipulate this property in it.

There's a AppleARMWatchdogTimer.kext that loads on bare metal, but not in a VM (even sudo kmutil load -p /System/Library/Extensions/AppleARMWatchdogTimer.kext won't make it load). It's used to create a "watchdog monitor" on bare metal, which logs messages to kmutil log show with the format kcgend: Watchdog .... This doesn't happen in a VM. The messages from kernelmanagerd and kcgend are grouped differently (in kmutil log show output) on bare metal and in VMs. I wonder if the watchdog monitor is used to coordinate the actions of kernelmanagerd and kcgend. The problems with kcgendon VMs, where this monitor doesn't operate, may be due to these two daemons stomping on each other's work. With luck, this problem will be solved by figuring out issue 1.

The page at https://github.com/AsahiLinux/docs/wiki/SW:Boot is very interesting. I notice that it mentions (and at least partially defines) "AP" and "kcOS" -- both of which show up in my research (in logs and in kexts' machine code).

WRT reading the local boot policy: I think bputil can already do this. For example, here's the output of sudo bputil -d on one of my VMs. Notice the value of "3rd Party Kexts Status":
Current local policy:
OS environment:
OS Type                                       : macOS (overriden)
OS Pairing Status                             : Paired
Local Policy Nonce Hash                 (lpnh): 5FC3F6F1403AAEB8EB7694D3F091C82588F2AF5621CC431A4681623BE97BB7FF30AF7E8C5E98CDD1E445F35EB3775139
Remote Policy Nonce Hash                (rpnh): 9C1D68C5C7F10D6E57888625E86AD4DF2D33AE4EE91DBE2112B976D90AA34A474A22102D83C5C26F162D8D1E034B5F11
Recovery OS Policy Nonce Hash           (ronh): 54F5741CEBFC75CB7B01C02968C712A12328B907F470C85DC88D92305542E557264B12BCD72D9BBA148C3298FE10DB3D

Local policy:
Pairing Integrity                             : Valid
Signature Type                                : Other
Unique Chip ID                          (ECID): 0x4D2E2033F0060E0B
Board ID                                (BORD): 0x20
Chip ID                                 (CHIP): 0xFE00
Certificate Epoch                       (CEPO): 0x1
Security Domain                         (SDOM): 0x1
Production Status                       (CPRO): 1
Security Mode                           (CSEC): 1
OS Version                              (love): <IP_ADDRESS>.0,0
Volume Group UUID                       (vuid): BE2ACA7D-8B5E-4E89-88F2-EE05D5354D88
KEK Group UUID                          (kuid): 00000000-0000-0000-0000-000000000000
Local Policy Nonce Hash                 (lpnh): 5FC3F6F1403AAEB8EB7694D3F091C82588F2AF5621CC431A4681623BE97BB7FF30AF7E8C5E98CDD1E445F35EB3775139
Remote Policy Nonce Hash                (rpnh): 9C1D68C5C7F10D6E57888625E86AD4DF2D33AE4EE91DBE2112B976D90AA34A474A22102D83C5C26F162D8D1E034B5F11
Next Stage Image4 Hash                  (nsih): 22954A85F4FDE9B179529AC61EA78C31F00B0E22DF9BE92235BBD80F548B7752FBA4C98D9B31DE9081EAC8EC5DEDCF2A
User Authorized Kext List Hash          (auxp): 0E3CE57A947E28CAADAAF29B9788D4740558B145F696AC594FB17F02CF284EF74B314AFA93A01DECBF7900373C2AC626
Auxiliary Kernel Cache Image4 Hash      (auxi): absent
Kext Receipt Hash                       (auxr): absent
CustomKC or fuOS Image4 Hash            (coih): absent
Security Mode:               Permissive (smb0 && smb1): 1
3rd Party Kexts Status:      Enabled    (smb2): 1
User-allowed MDM Control:    Enabled    (smb3): 1
DEP-allowed MDM Control:     Disabled   (smb4): absent
SIP Status:                  Customized (sip0): 7f
Signed System Volume Status: Disabled   (sip1): 1
Kernel CTRR Status:          Disabled   (sip2): 1
Boot Args Filtering Status:  Disabled   (sip3): 1

Apple actually has pretty decent documentation on the local policy file:
https://support.apple.com/guide/security/contents-a-localpolicy-file-mac-apple-silicon-secc745a0845/web

Here's a progress report ... or better yet a lack-of-progress report.
I pretty quickly discovered that the osenvironment stuff is a red herring: I found kernelmanagerd: kcgen activation settings: [kcgen enabled] messages on both the host and my VMs.
Then I tried to force AppleARMWatchdogTimer.kext to load in my VMs. Like I mentioned above, you can't just load it using kmutil load. In fact the boot kext cache seems to be immutable. You can only add kexts, even Apple-signed ones, to the auxiliary kext cache. But of course that's currently not possible in Virtualization framework VMs.
In principle you should be able to create a custom kext cache (using kmutil create) and load it (in recovery mode) using kmutil configure-boot. The following two links show you how to do this:
https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html
https://github.com/AsahiLinux/macvdmtool/blob/main/README.md
But I couldn't get it to work. I tried adding AppleARMWatchdogTimer.kext to my custom kext cache, creating one with no additional kexts, and even copying the immutable kext cache (from Apple) into /Library/KernelCollections. All failed in more or less the same way: I got a bunch of errors from AppleVPKeyStore.kext, then kernelmanagerd: Kernel requested shutdown. Goodbye!
I suspect Apple's kernel.release.vmapple (the kernel used in Virtualization framework VMs) just doesn't like what I'm trying to do. I could try building a custom kernel, following kernelshaman's instructions. But that would be very time consuming, and might just lead me down another rabbit hole.
I'm giving up on this, at least for the time being.

Since there was "Ask Apple" this week, I did. Quinn confirmed it's not going to work yet, and i'd consider that definitive.

I could try building a custom kernel, following kernelshaman's instructions.

I did this, and managed to get it to work. But yes, it took me down another rabbit hole. I did discover one useful bit of information, though: The failures have nothing to do with the kernel itself (custom or otherwise). In fact the kernel never runs at all when you try to use kmutil configure-boot to load a custom kext cache. I found this out by putting an infinite loop in the custom kernel's code, very early in the boot process. It never gets hit.
This may be some kind of permissions issue (though I loosened them as much as possible). But I think it's more likely that code somewhere in kernelmanagerd simply stops it from working.

Could you test if the loop gets hit on physical hardware?

It does get hit on physical hardware.
To play it safer, I changed the loop to a panic() call. Then my machine kept rebooting until I pressed the power button long enough to turn it off. I didn't see the normal panic screen -- probably the kernel hadn't yet loaded support for it. To restore my machine to its previous state, I booted into recovery mode and ran bputil -f. This gets rid of the custom kernel cache and turns on full security.
So kernelmanagerd (or something called via kernelmanagerd) only prevents booting from a custom kernel cache in a Virtualization framework virtual machine. I'll dig into this further and try to find out exactly how.
I didn't do a full test of my custom kernel (and its custom kext cache) on physical hardware. I don't want to risk harming my machine. The logical place to do this kind of test is in a virtual machine. Clearly Apple has gone to some trouble to prevent tests of custom kernels and third party kernel extensions from working in a Virtualization framework VM. It's rather hard to understand why :-(

Thanks for testing that. I wonder if the VM's bootloader is somehow hardcoding "allowed" kernels. Do you think trying to boot a kernel with a different version that was used in another VM might work? (You could check with uname -a, I think).
Also, is kernelmanagerd used for creating bootable kernels, or does it do stuff at boot time? If it only creates the kernels, as long as it's "valid", I think it would be a bootloader problem.

I doubt that kernelmanagerd hardcodes allowed kernels (though kmutil create (which works through kernelmanagerd) does insist (on macOS 13) that you have a KDK installed that matches the currently running kernel). And if it does, it shouldn't be hard to find the code that implements this restriction.
kernelmanagerd doesn't create kernels. It works with whatever kernel is "installed" -- the one that the boot kext collection is linked to. And as best I can tell it also doesn't create boot kext collections. Apple makes those in some other way, then puts them in the "right" location (which you can see by running kmutil inspect). Once installed in this way, a boot kernel collection is immutable, at least by the "normal" activity of macOS. What can be changed is the auxiliary kext collection. But, as we've discovered, this doesn't work in a Virtualization framework VM.
From kernelmanagerd's man page and from what I've been able to discover, kernelmanagerd manages the kernel-specific part of rebooting -- from making requested (staged) changes to the auxiliary kext collection to triggering a kernel's reloading. As we've seen, it can abort this process and log the (misleading) error message kernelmanagerd: Kernel requested shutdown.
Over the next few days (and possibly weeks), I'm going to dig into kernelmanagerd and its friends, to find as much as I can about why they behave differently on physical hardware and in a Virtualization framework VM.

Exactly the same nonsense still happens on the first macOS 14 beta (build 23A5257q). In a macOS 14 guest (running on a macOS 14 host), after using kmutil load -p /path/to/extension, you keep getting the same "System Extension Error" every time you reboot and log in. Allowing the extension to load in the Security & Privacy system preferences panel makes no difference.
And once again, the same kernel extension loads just fine on the host.
I tested using UTM 4.2.5 (the current version).
I've made some progress reverse engineering the com.apple.Virtualization.VirtualMachine process on an Intel box running macOS 13. The VM runs Linux. But (so far at least) none of what I've learned is relevant here.

This is a problem with SEP (or, more exactly, with its s/w replacement made specifically for .vmapple image because there is no real SEP). Kernel manager and boot policy app tries to query some sort of permissions to build a manifest for auxiliary kernel cache (AuxKC) using a special SEP command and during that an error is thrown which makes AuxKC building process to fail, thus any kernel extension you have approved via "Privacy & Security" are not incorporated into auxiliary kernel cache during reboot process when kcgend daemon is started.
I was able to circumvent it by patching this check, after that this manifest was created properly and kcgend generated AuxKC properly, but there was another error with failed manifest's signature check in iBoot, so a second patch was needed.
Sadly, the method with patching requires a lot of effort starting at AVPBooter and ending with kernel and some of system apps. I'm not sure if you can do it without patching and would like to know if there is any way to do that in a such way.

Very interesting, @flatz, that you were able to patch a bunch of system files. How did you do it? I assume you had to lower the security settings, but which values did you pick?

@steven-michaud Unfortunately you can't just lower security settings to do that. As I told earlier I've started from the Root of Trust, by patching ROM image (AVPBooter binary from Virtualization.fw) to skip signature checks of iBSS/iBEC, ramdisk, kernel, etc.

So it looks like the only file you patched is AVPBooter.vmapple2.bin in the Virtualization framework.
What do you mean by "patch"?
Did you change the file on disk? If so, how did you manage that? Did you boot into Recovery mode and change it there? Did you also need to re-codesign the Virtualization framework?
Or did you change the file's image after it was loaded into memory? If so, how did you do that?

I've found out on my own how to change macOS system files: How to Defang macOS System Protections.
I made a trivial change to AVPBooter.vmapple2.bin (I messed up the spelling of one word in one of its error messages). Then I worked through the "Defang" document up to the end of its "Unlock the Boot Volume" section. I didn't need to make any of the suggested further modifications under "Running Modified Code". UTM worked just fine. I tested with the first macOS 14 beta as both host and guest.
I'll be playing with AVPBooter.vmapple2.bin to see what I can find out.
Thanks, @flatz, for pointing out that it's an interesting target for investigation.

I've figured out how to use third-party kexts on Virtualization framework macOS clients! Please check it out.

Above I talked about custom kext caches not working in macOS guest VMs. My workaround for third-party kexts doesn't directly address this problem. But if Stage 2 iBoot (iBoot.img4) no longer checks the kernel cache's "digest", you can substitute any kernel cache you want, including a custom kernel cache built using kmutil create. Some will make your VM unbootable, or have other undesirable effects. But you can make whatever changes you please to the kernel cache.
I used kmutil create to create a custom kext cache modeled exactly on the original (with exactly the same kernel extensions). That worked fine, not surprisingly. Then I tried adding AppleARMWatchdogTimer.kext, to test out my theory that its absence was causing timing problems. It made my VM unbootable :-( This doesn't disprove my theory. But it does show that Apple had a reason not to include AppleARMWatchdogTimer.kext in its VMs' kernel cache.

Apple still hasn't fixed this bug, but I've discovered a sign they might be working on it.

Is anyone here at Apple's WWDC? If so, could you ask about this issue? It'd be interesting to hear some kind of official response. The WWDC started yesterday (2024-06-10) and runs through Friday (2024-06-14).
I haven't yet tested with macOS 15 (Sequoia), but should be able to in the next few days.

I've run macMini with Sequoia, and latest UTM with VM as Sonoma - that does not work.   Just need to update VM to Sequoia as well.

My Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs document works as-is on a macOS 15 guest running on a macOS 15 host! With the alterations it describes, the guest can load third party kernel extensions.


Fix build numbers so they always end with +D/+R

@AArnott,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Codecov Report

Merging #27 into v15.3 will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##            v15.3      #27   +/-   ##
=======================================
  Coverage   83.85%   83.85%
=======================================
  Files          70       70
  Lines        5338     5338
  Branches      831      831
=======================================
  Hits         4476     4476
  Misses        635      635
  Partials      227      227

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f5950d1...29764fa. Read the comment docs.


Fix bintray upload problem

Thing left to do:

[x] Create opentracing-hprose bintray package
[ ] Test locally with manual bintray credentials
[ ] Update project depending on previous step.

I tested running this from my local machine (see https://github.com/opentracing/opentracing-java/pull/90) and got the same error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project opentracing-hprose: Failed to deploy artifacts: Could not transfer artifact io.opentracing.contrib:opentracing-hprose:jar:1.0.0-20170315.214614-1 from/to jfrog-snapshots (http://oss.jfrog.org/artifactory/oss-snapshot-local): Failed to transfer file: http://oss.jfrog.org/artifactory/oss-snapshot-local/io/opentracing/contrib/opentracing-hprose/1.0.0-SNAPSHOT/opentracing-hprose-1.0.0-20170315.214614-1.jar. Return code is: 401, ReasonPhrase: Unauthorized. -> [Help 1]

Probably we should ask oss.jfrog.org permissions for this bintray package.

I issued an Request to include the package 'opentracing-hprose' in 'jcenter' in bintray.
Let's hope the permission comes soon.

Closing, bintray upload now works.


tdx-encrypted-tdvm-image.md typing error

https://github.com/intel/confidential-computing-zoo/tree/main/cczoo/tdx-encrypted-tdvm-image
There is a typing error

https://github.com/intel/confidential-computing-zoo/tree/main/cczoo/tdx-encrypted-tdvm-image There is a typing error

Thanks Kun for the comment.
Fixed.


Ability to use CSS variable for color

If I can use my CSS variable in color props, that would be good, since I have a lot of theme in the application

Hello, you can do something like that for exemple :
CSS:
:root {
  --nprogress-color: red;
  --nprogress-height: 20px;
}

JS:
<ProgressBar
   height="var(--nprogress-height)"
   color="var(--nprogress-color)"
/>


Tracking: HSTS header on web management endpoint

Make sure the appropriate HSTS headers are set on the web management portal endpoints.

So I was reading the Kubernetes issue for this same problem, and they turned up a really interesting concern: https://github.com/kubernetes/kubernetes/pull/96502#issuecomment-734847096
Basically the HSTS header is set at a node level, but other services on the same node may intentionally offer unencrypted services, so universally turning this on could conflict with system services.


"Switch network" alert if user is on the wrong network

Is your feature request related to a problem? Please describe.
When users visit olympusdao.finance, connects their wallet but are connected to a wrong network, the UI bugs out and nothing is showing.
Describe the solution you'd like
When a user connects his wallet, we should check if he's on the correct network. If not, prompt a pop up telling the user he needs to connect to Ethereum mainnet, and allow him to click a button in the pop up saying "Switch Network".
Clicking the button should trigger the network change in the user's wallet provider.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context

I would like to contribute to OlympusDAO and I think this task is something I could start with.

The issue with MetaMask's wallet_addEthereumChain RPC is that it can only be used to add and switch to networks that are not listed within the user's MetaMask. For this reason we can't use it to prompt the user to change their network to mainnet, since that's already listed. However, there is a draft EIP (https://github.com/ethereum/EIPs/pull/3326) which would enable this. In the meantime the best we could do is show a suggestion to switch networks (and set the provider to undefined if the user is on the wrong network so that no errors would arise due to failed fetches).

We'd like to display the error message as a MaterialUI snack bar: https://material-ui.com/components/snackbars/

Transition: Throw in a bottom right, slide transition.
Content: "You're using an unsupported network. Please switch to Ethereum mainnet"

Discord link: https://discord.com/channels/838651642190495804/838654073464946758/863153664641990696


HDInsight LLAP with Enterprise Security Package

I don't see any documentation about how/if this works with HDInsight LLAP with Enterprise Security Package.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8abcbc42-62d5-e0a0-c57f-db29c1c3e3fd
Version Independent ID: 1f52fc40-7a28-f2cd-77a5-000821ccc2b7
Content: Apache Hadoop & Visual Studio Data Lake Tools - Azure HDInsight
Content Source: articles/hdinsight/hadoop/apache-hadoop-visual-studio-tools-get-started.md
Service: hdinsight
GitHub Login: @hrasheed-msft
Microsoft Alias: hrasheed

@haveyoumetcp Yes, it works with Visual Studio tools.
The following table shows the recommended access methods for each cluster type:

Reference: https://docs.microsoft.com/en-us/azure/hdinsight/domain-joined/apache-domain-joined-manage#access-the-clusters-with-enterprise-security-package
Hope this helps.

Yes, I see that.  But, I don't see any documentation on how to actually connect to an LLAP cluster with EST via Visual Studio. Because it's apparently different than a cluster without LLAP, as that's working for me.  But, the LLAP cluster without ESP is not working in Visual Studio.

Hello @haveyoumetcp. Just to clarify: Are you trying to connect to an Interactive Query ESP cluster with Visual studio? Which HDInsight version are you using? Could you share more details about whats happening when you say its not working? Are you getting a specific error message?

@haveyoumetcp  Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.

It's been 3 days and you closed this?  Some people don't work on weekends...


Rewritten Path and Domain in Set-Cookie headers should be corrected by Envoy

Description:
Using host or path rewrite rules in conjunction with Cookies with Domain or Path attributes can result in what can be considered incorrect behavior.
Given a route config that matches the prefix /foo to /bar an upstream server will likely return a Set-Cookie header with the Path attribute set to /bar<rest-of-match>. However this may be an internal-only path, an implementation detail, etc. so the browser/client etc. then sending that cookie will not send it for requests on the path /foo but instead /bar. This is a similar issue for rewriting the Host/:authority header, the Set-Cookie header may return the rewritten host in the Domain attribute.
It would be useful to have (maybe as an optional configuration) Envoy reverse this for Set-Cookie headers automatically, e.g. for each Set-Cookie header returned by an upstream, undo the rewritten Path or Domain elements to what the original request specified. This could be a RouteAction or maybe evenRouteConfiguration level option. Note this is not a request for generic HTTP header rewriting functionality, though the implementation of such a feature might be made easier by implementing what this issue asks for.
Meta Question
It seems this sort of thing is already somewhat possible in Envoy with Lua filters, e.g. have an on-request function in Lua that captures the original path/:authority header before it is rewritten in the router and save it in dynamic metadata and then an on-response function that uses that saved information to reverse the rewritten content (though would have to be configured at the filter chain level and not route level so might have some drawbacks). From searching through Envoy issues it seems this kind of problem is generally responded to with a suggestion to use Lua by the community. Is there a codified stance the Envoy community has taken regarding this and when a feature will not be considered to add to Envoy? Or is it a matter of people getting things to work with Lua and not pursuing further changes to Envoy?

Note I (or some other resources at my company) am willing to work on this if it is deemed an acceptable change by the community

I've added cors because of the implications of wrong domain headers, and http since I didn't see another reverse-proxy topic that seemed more focused. Design proposal would be helpful.
Since the lua solution is solved with appropriate examples, but is more heavy-weight than required 99% of the time, could you tune this ticket to non-lua, baked-in solutions for envoy? A different "Better examples for set-cookie rewriting in lua" would solve the different ask in this ticket.

cc @alyssawilk for thoughts on this.

I think in-house with similar features we usually expect the upstream to be rewrite-aware, but I think it'd be nice to have it as a configurable feature in Envoy.  I think it's going to be a tricky one to not miss corner cases for though. @sunjayBhatia if you're up for picking it up please feel free to /assign it my way.
cc @antoniovicente as well.

I should be able to get started on this next week, I'll write up a little design here and get some feedback before coding it up fully

Looking with a bit more detail about how this would work, I'm on board with @jnehlmeier's suggestion about Envoy not trying to figure out the correct path/domain attributes to rewrite a cookie with and requiring an explicit configuration by users how they want their Set-Cookie header to look for a couple reasons:

The host/path can be rewritten in a fair few places, Lua filter, RouteAction, RedirectAction, etc. so getting the edge cases correct are not straightforward

Especially with filters that could modify request attributes, seems like Envoy offering an automatic rewrite would be more pain than it is worth, it won't cover everyone's use cases
It might be more repetition, but requiring users to explicitly configure what rewrite they want feels easier/cleaner for Envoy

The use case in https://github.com/envoyproxy/envoy/issues/15612#issuecomment-810373264
We can offer more advanced features in the future if we choose to expose some variables as part of rewrites
We have all the existing code to do advanced request attribute rewriting (path rewrites), repurposing/abstracting common code for header manipulation is possible

We can offer more advanced features in the future if we choose to expose some variables as part of rewrites

Not sure which variables you have in mind but variables for domain and path are required from the beginning when cookie rewrites are configured manually by the user.
Otherwise cookie rewrites are not possible for virtual hosts with multiple domains and/or domain wildcards and also not possible for prefix and regex based path rewrites / redirects.

We can offer more advanced features in the future if we choose to expose some variables as part of rewrites

Not sure which variables you have in mind but variables for domain and path are required from the beginning when cookie rewrites are configured manually by the user.
Otherwise cookie rewrites are not possible for virtual hosts with multiple domains and/or domain wildcards and also not possible for prefix and regex based path rewrites / redirects.

Yeah, was mostly thinking about rolling things out incrementally and possibly not covering wildcard domains etc. initially but yes, to cover those sorts of use cases you are correct, a user configuring routing themselves would need access to original host/path variables


Reliability fixes

Have HA publish temperature setpoint as int instead of float. This fixes some HA cards that publish temperature as float
Improve the MQTT Client robustness by wrapping the on message handler with try catch. This avoids crashing the main Paho's loop

Hi Giovanni,
thank you very much for investing so much time and effort into the Gateway. It's really amazing what you have contributed over the last weeks. When are we ready to release version 0.3.0? Is there anything left that needs to be done for this release? I would like to verify if we have solved issue #27. So far it looks quite good.
Kind regards,
Thomas

Hey,
Glad to have helped. The plan was to leave for the holidays with most of the integration working :)
I think there may be a bug in the decoder still. I'll file a ticket for that and after is solved I think 0.3.0 is ready to go :)


Adding user functionality

Add functionality to add user into the main database...
The form and the underlaying database structure is done just implement the logic in the AddNewUser.java class

Added and Fixed! 🌟👍
Commit: #cf2a3d1

✨ Added: [Dependent] functionality for admin accounts.
🔧 Fixed: Minor bugs and issues.

🚀 In Progress:

Working on the Manage Users feature to fix issue #7.

✨ New User Form:

Stay tuned for more updates! 💻🌈