[BUG] - User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group

Describe the bug
I followed the Readme as close as possible and trying to get a wildcard cert from LetsEncrypt Staging. It seems that that might be some permission issue and feel I missed something. I installed it via: https://artifacthub.io/packages/helm/vultr/cert-manager-webhook-vultr. I get a PresentError due to Forbidden error.
To Reproduce
Steps to reproduce the behavior:

Installed cert-manager

Applied "vultr-credentials" via command line with a Base64 Encoded version of my API Key - I even tried to allow all IPV4 Addresses on this API Key as I am not sure what subnet or IP address to limit for this.

helm install cert-manager-webhook-vultr vultr/cert-manager-webhook-vultr --version 1.0.0

Applied all the examples as one file, including: ClusterIssuer, Role, RoleBinding and Certificate

kubectl get certificate

NAME                                     READY   SECRET                                   AGE
cert-manager-webhook-vultr-ca            True    cert-manager-webhook-vultr-ca            79m
cert-manager-webhook-vultr-webhook-tls   True    cert-manager-webhook-vultr-webhook-tls   79m
staging-mmgm-com                         False   staging-mmgm-tls                         76m

kubectl describe challenges

Status:
  Presented:   false
  Processing:  true
  Reason:      vultr.acme.mmgm.com is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group "acme.mmgm.com" at the cluster scope
  State:       pending
Events:
  Type     Reason        Age              From          Message
  ----     ------        ----             ----          -------
  Normal   Started       9s               cert-manager  Challenge scheduled for processing
  Warning  PresentError  4s (x3 over 9s)  cert-manager  Error presenting challenge: vultr.acme.mmgm.com is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group "acme.mmgm.com" at the cluster scope

Expected behavior
Certificate to be issued.
Additional context
I have not applied my ingress yaml yet as I we hoping to see the Certificate to be issued. Anything you can point me to as to what I am missing, would be greatly appreciated.
Add any other context about the problem here.

@tzarger
Hey there, I would recommend install by following the readme https://github.com/vultr/cert-manager-webhook-vultr#installation
I believe we may need to update artifacthub.
Also, you are correct that appears to have invalid RBAC definitions set

Hello @ddymko I am pretty sure some progress, but not entirely sure everything is working.

Should there be any DNS entries added to prove DNS Ownership?
I have not received any emails at the email I added in the ClusterIssuer
It appears the OrderCreated was the last status and not sure how long that should take.

kubectl get secrets -n cert-manager
NAME                                               TYPE                                  DATA   AGE
cert-manager-cainjector-token-vrjt7                kubernetes.io/service-account-token   3      61m
cert-manager-token-5t4rw                           kubernetes.io/service-account-token   3      61m
cert-manager-webhook-ca                            Opaque                                3      61m
cert-manager-webhook-token-zdvtz                   kubernetes.io/service-account-token   3      61m
cert-manager-webhook-vultr-ca                      kubernetes.io/tls                     3      51m
cert-manager-webhook-vultr-token-rmc4v             kubernetes.io/service-account-token   3      27m
cert-manager-webhook-vultr-webhook-tls             kubernetes.io/tls                     3      51m
default-token-628tw                                kubernetes.io/service-account-token   3      61m
letsencrypt-staging                                Opaque                                1      35m
sh.helm.release.v1.cert-manager-webhook-vultr.v1   helm.sh/release.v1                    1      27m
vultr-credentials                                  Opaque                                1      59m

kubectl get certificate
NAME               READY   SECRET             AGE
staging-mmgm-com   False   staging-mmgm-tls   32m

kubectl describe certificaterequest
Status:
  Conditions:
    Last Transition Time:  2022-06-03T02:26:42Z
    Message:               Certificate request has been approved by cert-manager.io
    Reason:                cert-manager.io
    Status:                True
    Type:                  Approved
    Last Transition Time:  2022-06-03T02:26:42Z
    Message:               Waiting on certificate issuance from order default/staging-mmgm-com-l4gvt-3136983109: "pending"
    Reason:                Pending
    Status:                False
    Type:                  Ready
Events:
  Type    Reason           Age   From          Message
  ----    ------           ----  ----          -------
  Normal  cert-manager.io  32m   cert-manager  Certificate request has been approved by cert-manager.io
  Normal  OrderCreated     32m   cert-manager  Created Order resource default/staging-mmgm-com-l4gvt-3136983109

As you can see, 32 minutes and waiting. So I am wonder if there is something missing. Maybe the API Token is not working from the vultr-credentials secret?
By the way, I also had a friend send me an email to confirm the email was working.
If you need to see the ClusterIssue, Role, RoleBinding or Certificate, let me know how to send privately, I do not want me email and domain info posted here in public.
Thank you.

@ddymko So it seems that main issue may be the fact that the DNS entries in domain is never updated with _acme-challenge and when running the test I receive this error:
bash scripts/fetch-test-binaries.sh
TEST_ASSET_ETCD="/Users/troy/.kube/cert-manager-webhook-vultr-master/_out/kubebuilder/bin/etcd" TEST_ASSET_KUBE_APISERVER="/Users/troy/.kube/cert-manager-webhook-vultr-master/_out/kubebuilder/bin/kube-apiserver" TEST_ASSET_KUBECTL="/Users/troy/.kube/cert-manager-webhook-vultr-master/_out/kubebuilder/bin/kubectl" \
	go test -v .
=== RUN   TestRunsSuite
=== RUN   TestRunsSuite/Conformance
=== RUN   TestRunsSuite/Conformance/Basic
=== RUN   TestRunsSuite/Conformance/Basic/PresentRecord
    util.go:71: error creating test fixtures: exit status 1
=== RUN   TestRunsSuite/Conformance/Extended
=== RUN   TestRunsSuite/Conformance/Extended/DeletingOneRecordRetainsOthers
    suite.go:73: skipping test as strict mode is disabled, see: https://github.com/jetstack/cert-manager/pull/1354
--- FAIL: TestRunsSuite (25.27s)
    --- FAIL: TestRunsSuite/Conformance (14.27s)
        --- FAIL: TestRunsSuite/Conformance/Basic (3.36s)
            --- FAIL: TestRunsSuite/Conformance/Basic/PresentRecord (3.36s)
        --- PASS: TestRunsSuite/Conformance/Extended (0.00s)
            --- SKIP: TestRunsSuite/Conformance/Extended/DeletingOneRecordRetainsOthers (0.00s)
FAIL
FAIL	github.com/vultr/cert-manager-webhook-vultr	25.640s
FAIL
make: *** [test] Error 1

@tzarger
I will have to go through this because I know this plugin works as I have used it in the past and written a post about it
https://www.dymko.me/posts/automatic-dns-tls-on-kubernetes/
We'll look through it

Taking a quick look it looks like your RBAC definitions are causing the problems
vultr.mmgm.com is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group "mmgm.com" at the cluster scope

@ddymko I have copied the RBAC definitions multiple times from the GitHub Repo and now from your article above.
If I don't change the groupName in the cluster issuer, I get this (DNS updated as well)
  Presented:   true
  Processing:  true
  Reason:      Waiting for DNS-01 challenge propagation: DNS record for "mmgm.com" not yet propagated
  State:       pending
Events:
  Type     Reason        Age                From          Message
  ----     ------        ----               ----          -------
  Normal   Started       58s                cert-manager  Challenge scheduled for processing
  Warning  PresentError  58s (x2 over 58s)  cert-manager  Error presenting challenge: the server is currently unable to handle the request (post vultr.acme.vultr.com)
  Normal   Presented     52s                cert-manager  Presented challenge using DNS-01 challenge mechanism

if I change the group name to something like: groupName: acme.mmgm.com I get this (and no DNS update)
Status:
  Presented:   false
  Processing:  true
  Reason:      vultr.acme.makemoneygrowmoney.com is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group "acme.makemoneygrowmoney.com" at the cluster scope
  State:       pending
Events:
  Type     Reason        Age                From          Message
  ----     ------        ----               ----          -------
  Normal   Started       17s                cert-manager  Challenge scheduled for processing
  Warning  PresentError  12s (x3 over 17s)  cert-manager  Error presenting challenge: vultr.acme.mmgm.com is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group "acme.mmgm.com" at the cluster scope

This is my RBAC in either case:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cert-manager-webhook-vultr:secret-reader
  namespace: cert-manager
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["vultr-credentials"]
  verbs: ["get", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cert-manager-webhook-vultr:secret-reader
  namespace: cert-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cert-manager-webhook-vultr:secret-reader
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: cert-manager-webhook-vultr

I noticed in the Role the apiGroup is an array of 1 empty string:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["vultr-credentials"]
  verbs: ["get", "watch"]

In the RoleBinding, roleRef has an apiGroup:
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cert-manager-webhook-vultr:secret-reader

In the RoleBinding the subjects again has no apiGroup:
subjects:
- apiGroup: ""
  kind: ServiceAccount
  name: cert-manager-webhook-vultr
  ```

  Since the permission is in the apiGroup:
  `  Warning  PresentError  12s (x3 over 17s)  cert-manager  Error presenting challenge: vultr.acme.mmgm.com is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "vultr" in API group "acme.mmgm.com" at the cluster scope`

  Could this be the issue?

@ddymko Started with a new cluster and saw this messsage:
  Warning  PresentError  79s (x5 over 2m24s)  cert-manager  Error presenting challenge: {"error":"Unable to add record: New records TTL (60) must match the TTL of existing records (120)","status":400}
It appears that the lowest TTL is 120 ... Is this something that should be adjusted? Or will this be an issue in the future when renewing the certificate?
Beyond that, I deleted my cluster, deleted the _acme-challenge DNS entry did the steps and everything worked this time.
It seems groupName: acme.vultr.com does work... in your repo, it says it should be my domain... but that seems to have the User permission issue.
Here is my latest ClusterIssuer and RBAC:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email<EMAIL_ADDRESS>    server: "https://acme-staging-v02.api.letsencrypt.org/directory"
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
      - dns01:
          webhook:
            groupName: acme.vultr.com
            solverName: vultr
            config:
              apiKeySecretRef:
                key: apiKey
                name: vultr-credentials
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email<EMAIL_ADDRESS>    server: "https://acme-v02.api.letsencrypt.org/directory"
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - dns01:
          webhook:
            groupName: acme.vultr.com
            solverName: vultr
            config:
              apiKeySecretRef:
                key: apiKey
                name: vultr-credentials
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cert-manager-webhook-vultr:secret-reader
  namespace: cert-manager
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["vultr-credentials"]
    verbs: ["get", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cert-manager-webhook-vultr:secret-reader
  namespace: cert-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cert-manager-webhook-vultr:secret-reader
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: cert-manager-webhook-vultr

@tzarger
@ErwinsExpertise can you look into this issue and see if you can find what the problem might be

@tzarger
It's good to hear that it is working for you now. Can you share where you see the groupname should be your domain so we can clarify that?
I went ahead and tested myself and was able to create a wildcard cert without any issues using the following manifests:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email<EMAIL_ADDRESS>    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: letsencrypt-staging
    solvers:
    - dns01:
        webhook:
          groupName: acme.vultr.com
          solverName: vultr
          config:
            apiKeySecretRef:
              key: apiKey
              name: vultr-credentials
---

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cert-manager-webhook-vultr:secret-reader
  namespace: cert-manager
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["vultr-credentials"]
  verbs: ["get", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cert-manager-webhook-vultr:secret-reader
  namespace: cert-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cert-manager-webhook-vultr:secret-reader
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: cert-manager-webhook-vultr
---

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: staging-cert-example-com
spec:
  commonName: "*.domain.com"
  dnsNames:
  - "*.domain.com"
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  secretName: example-com-tls

As far as the TTL message is concerned we can look into adjusting this value to 120. A value of 60 seems to work without issue for me, however, we can look into that more.
The value is set in the code here:
https://github.com/vultr/cert-manager-webhook-vultr/blob/260d09a930f2d58558c11081ba4540925b6530fa/main.go#L93

@ErwinsExpertise Couple things:

I was able to get it to work using a groupName: acme.vultr.com but not using my domain as mentioned in the code here:

# The GroupName here is used to identify your company or business unit that
# created this webhook.
# For example, this may be "acme.mycompany.com".
# This name will need to be referenced in each Issuer's `webhook` stanza to
# inform cert-manager of where to send ChallengePayload resources in order to
# solve the DNS01 challenge.
# This group name should be **unique**, hence using your own company's domain
# here is recommended.
groupName: acme.vultr.com

the TTL of 60 error only occured when I used the vultr groupName and then started over after the _acme-challenge was present. Stangely enough, I saw this only one time, so not sure I can reproduce this. I only mentioned it, to make sure there would be no issues on renewal of the LetsEncrypt certificate.

I see in your code above, you also used groupName: acme.vultr.com ... so is that the right way?


Multiplexing on Host: headers ("virtualhosts")

I figured I'd ask here to see if there's a cleaner way of multiplexing based on host headers to different muxes.
Context (ignore the scratchy code—just a dev env):
	ember := http.FileServer(http.Dir("/path/to/ember-go/dist/"))
	r := web.New()
	// r.Use(...)
	// The Ember entry point
	r.Get("/", ember)
	// Static assets
	r.Get("/assets/*",
		http.StripPrefix("/assets/",
			http.FileServer(http.Dir("/path/to/ember-go/dist/assets"))))
	// Force Goji to serve 404's to Ember so that directly browsing to a route allows Ember to handle it
	r.NotFound(func(w http.ResponseWriter, r *http.Request) {
		http.ServeFile(w, r,
			"/path/to/ember-go/dist/index.html")
	})

In addition:

I have my API endpoints on a sub-router that's handling /api/* — this is the one I'd like to change to handle requests for api.domain.com rather than a /api/ prefix.
The NotFound function on the sub-router returns a 404 with a JSON body
Can't use nginx to serve the static content directly as this will be hosted on Heroku, so Go needs to do the work.

I'd like to instead pass requests to muxes based on the Host header - e.g. www.domain.com and api.domain.com — similar to how gorilla/mux has a .Host() method as part of its router - e.g.

The root handles all incoming requests for /*
Based on the Host: header it passes the request (with the request URI intact) to the relevant sub-router
Calls its own 404 if no matches.

The obvious (to me) way would be to:

Write some middleware of the form func(c *web.C, h http.Handler) http.Handler that sits on the "root" router
Manually construct a map[string]http.Handler mapping hosts to routers
Checks the Host: .... header (after cleaning it)
Call the appropriate sub-router based on the map lookup e.g. _, exists := hosts[cleanHost(r.Host)].ServeHTTPC(c, w, r) else 404 if !exists.

But that seems pretty clunky. Is there a way that mux.Router could be used here? The important part is serving both on the same HTTP(S) port.

One way to do this is to mix-and-match muxes. For instance, the stdlib mux does a pretty good job of dealing with Host headers; you could split things out there.
Depending on your use case, you can also write a custom Pattern type that additionally matches against the Host header.
This is of course also possible to do with middleware, although I don't think that abstraction maps particularly well to this scenario.

Thanks for the reply @zenazn
Forgot that http.ServeMux did that by default. I've been trying to lean on Goji's mux as the "root" mux so it can apply top-level middleware (the recoverer & CSRF) but repeating the middleware on each of the sub-routers might not be a huge problem.
Right now I have the below (using closures for quick testing):
	hosts := map[string]http.Handler{
		"localhost:8000":     static,
		"api.localhost:8000": api,
	}

	mux := func(w http.ResponseWriter, r *http.Request) {
		if vh, exists := hosts[r.Host]; exists {
			vh.ServeHTTP(w, r)
			return
		}
		w.WriteHeader(404)
		return
	}

	r.Handle("/*", mux)
        // Sub-router routes below below...

I'm likely to adapt the above to match on Host explicitly first — api.domain.com — and if there's no matching host <=> router relationship, pass all requests to the fallback router (which would be the static router). Would save having to explicitly lay out all of the hosts you want to handle—pass the api ones to the API router, and pass the rest to the default.
Is there a concise or more flexible way to achieve this with Pattern?

Sorry—didn't notice your final question here. There's not a great way to do this with zenazn/goji's Pattern, but as I'm sure you've figured out by now goji.io's Pattern is much more flexible here and is a far better fit for this use case.


Clean up MshObject.cs and MshMemberInfo.cs

PR Summary
Separated from #7413
Clean up MshObject.cs and MshMemberInfo.cs.
/cc @iSazonov @powercode
PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Change is not breaking
[ ] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

User-facing changes

[x] Not Applicable
OR
[ ] User-facing Documentation needed

[ ] Issue filed - Issue link:

Testing - New and feature

[ ] Not Applicable or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

[x] Add [feature] if the change is significant or affects feature tests

Looks great! Ship it!

@daxian-dbw  Can you resolve conflicts?

The PR has been rebased. @iSazonov maintainers can push changes to PRs, so please feel free to address the issues that are brought up in your comments.

@SteveL-MSFT @daxian-dbw New commits was added.

Looks like we just need to resolve the merge conflict

The code needs to be rebased to fix the test issue on linux as well

Rebased.

CI Appveyor temporary failed.


Add support for MSBuild Core (and thus dotnet build)

Work in progress in the netstandard branch.
Blocked by:

[x] https://github.com/NuGet/Home/issues/4655 Package dependencies are not suppressible when they come from a P2P reference

Remaining work:

[ ] Copy CoreCLR dependencies locally in the package.

Will this fix https://github.com/AArnott/CodeGeneration.Roslyn/issues/12, i.e. enable this NuGet package to be used in Visual Studio 2017?

I doubt it. And FWIW the currently released version works (for me) in VS 2017 just fine.

@AArnott Thanks for the response. I tried to open my project using v0.1.57 in VS2017 today and I also experience https://github.com/AArnott/CodeGeneration.Roslyn/issues/12. I guess I will have to dig into the issue to understand what is causing the problem.


traceroute doesn't work from behind NAT

local> trace <IP_ADDRESS>
trace route to <IP_ADDRESS> (<IP_ADDRESS>), 30 hops max
1  <IP_ADDRESS> 0.126 ms 0.071 ms 0.045 ms
2  <IP_ADDRESS> 0.234 ms 0.150 ms 0.205 ms
3  * * *
4  * * *
5  * * *
6  * * *
7  * * *
8  * * *
9  * * *
10 * * *
11 * * *
12 * * *
...

It's reproducible under Linux:
Doesn't work:
traceroute <IP_ADDRESS>

Working:
traceroute -I <IP_ADDRESS>

"mtr" for example is using ICMP as default.
Is it possible to use ICMP packets for traceroute (as default or via a configurable option)?

mylg.trace supports -I (ICMP) did you try it?
local> trace <IP_ADDRESS> -I

Its working better with "-I".
Can you please update the help site of trace?
local> trace help

    usage:
          trace IP address / domain name [options]
    options:
          -r             Real-time response time at each point along the way
          -n             Do not try to map IP addresses to host names
          -nr            Do not try to map IP addresses to ASN,Holder (RIPE NCC)
          -m MAX_TTL     Specifies the maximum number of hops
          -4             Forces the trace command to use IPv4 (target should be hostname)
          -6             Forces the trace command to use IPv6 (target should be hostname)
    Example:
          trace <IP_ADDRESS>
          trace freebsd.org -r

@bjoernbessert -I still not implement with real-time that's why I didn't add to help but it should be worked w/ regular trace command.


[BYOC] Don't annotate constants

By annotating constants, constant values were frequently getting passed between partitions. As constant values aren't really data flow paths, it's better to not annotate them at all so they remain with the functions that consume them.

Thanks for the PR, so the idea is to create the following graph if I understand correctly:
input
  |
begin
  |
 op -- const
  |
end

where the constant input for op is not annotated.
Since now we require AnnotateTarget to annotate every op, and AnnotateRegionSet, MergeCompilerRegion also require every op to be annotated, could you elaborate if this change works for all of them? At first glance, it might be a problem for AnnotateRegionSet to identify the region inputs.
Following the above concern, could you also add this case to the unit tests of other passes so that we can make sure the rest passes are covered.

Does this affect the end result of the partitioned subgraphs at all? Will the constant still be included in the subgraph?

The motivation for this is that when you have a supported operator, such as qnn.concatenate, which takes some constant inputs, those constant nodes are annotated as 'default'. So the operator itself goes into a target partition whereas the constants remain in the default partition and are then passed to the target partition as arguments.
To see this, I can continue to run the test case through partition graph both with and without this change. Without the change:
def @const_test_0(%const_test_0_i0: (Tensor[(10, 10), uint8], Tensor[(10, 10), uint8]), %const_test_0_i1: (float32, float32), %const_test_0_i2: (int32, int32), Inline=1, Compiler="const_test", global_symbol="const_test_0", Primitive=1) -> Tensor[(10, 20), uint8] {
  qnn.concatenate(%const_test_0_i0, %const_test_0_i1, %const_test_0_i2, 0f /* ty=float32 */, 1 /* ty=int32 */, axis=1) /* ty=Tensor[(10, 20), uint8] */
}

def @main(%a: Tensor[(10, 10), uint8], %b: Tensor[(10, 10), uint8]) -> Tensor[(10, 20), uint8] {
  %0 = abs(%a) /* ty=Tensor[(10, 10), uint8] */;
  %1 = (%0, %b);
  %2 = (0f /* ty=float32 */, 0f /* ty=float32 */);
  %3 = (1 /* ty=int32 */, 1 /* ty=int32 */);
  @const_test_0(%1, %2, %3) /* ty=Tensor[(10, 20), uint8] */
}

With the change:
def @const_test_0(%const_test_0_i0: (Tensor[(10, 10), uint8], Tensor[(10, 10), uint8]), Inline=1, Compiler="const_test", global_symbol="const_test_0", Primitive=1) -> Tensor[(10, 20), uint8] {
  %0 = (0f /* ty=float32 */, 0f /* ty=float32 */);
  %1 = (1 /* ty=int32 */, 1 /* ty=int32 */);
  qnn.concatenate(%const_test_0_i0, %0, %1, 0f /* ty=float32 */, 1 /* ty=int32 */, axis=1) /* ty=Tensor[(10, 20), uint8] */
}

def @main(%a: Tensor[(10, 10), uint8], %b: Tensor[(10, 10), uint8]) -> Tensor[(10, 20), uint8] {
  %2 = abs(%a) /* ty=Tensor[(10, 10), uint8] */;
  %3 = (%2, %b);
  @const_test_0(%3) /* ty=Tensor[(10, 20), uint8] */
}

I'll add this as an explicit test case in the graph partition tests.
The reason I've chosen to just omit the annotations for constants altogether is that the requirement for MergeCompilerRegions is that all dataflow paths are annotated. As constants are just data with no flow, they can be considered as part of the operator which consumes them for the purposes of the analysis done in that pass.

Ah I see your point. I missed considering that ConstantNode is also a "node" instead of a "var". I agree with you that constant nodes should follow the target of the consuming node and the output you illustrated makes sense to me. On the other hand, it'd be better to maintain the statement of "annotating every node" of AnnotateTarget pass. A similar case is TupleNode. We determine the target of tuple nodes by looking at its arguments, and still annotate it whatever it should be at the TVM or other targets. In summary, I'd prefer the following annotation generated by AnnotateTarget:
input
  |
begin
  |
 op -- begin -- end -- const -- begin
  |
 end

instead of
input
  |
begin
  |
 op -- const
  |
 end

This can naturally guarantee that this change will not affect the rest BYOC flow.

I'm concerned that if we enforce annotation of individual constant nodes, MergeCompilerRegions will become a necessary pass to run. Otherwise we'll generate partitions just containing a ConstantNode.

It looks like the refactor to AnnotatedRegionSet has broken this 'fix'. I think the important property to maintain is that 'every node should belong to a region' rather than 'every node should be annotated'. By this, I mean that the following structure:
input
  |
begin
  |
 op -- const
  |
end

while not explicitly annotating every node, should imply that the const is still in a region, just not as an input or output. If we put a begin before the const, that implies there's a data flow path into the region through the constant, which isn't strictly true. Maybe this would require a slightly modified traversal in AnnotatedRegionSet to handle this case.

I'm concerned that if we enforce annotation of individual constant nodes, MergeCompilerRegions will become a necessary pass to run. Otherwise we'll generate partitions just containing a ConstantNode.

It seems to me that if someone runs AnnotateTarget, there's no reason for her to skip MergeCompilerRegions. If she wrote her own annotate pass, then those cases should be handled by herself. Originally I was even thinking to put AnnotateTarget and MergeCompilerRegions in the same pass, but we have consented to have separate passes so that we can have higher flexibility for the merging algorithm.

It looks like the refactor to AnnotatedRegionSet has broken this 'fix'. I think the important property to maintain is that 'every node should belong to a region' rather than 'every node should be annotated'.

I agree with you that every node should belong to a region is an important property we should maintain in AnnotateRegionSet. I'm just concerned about treating ConstantNode as a special node. In this case, we have to be aware of ConstantNode everywhere in this infra, or we will easily introduce bugs when improving one of them otherwise. I'm hoping that we could deal with the constant nodes in AnnotateTarget pass and let other passes treat constant nodes as other nodes. For example like you have illustrated, this graph requires AnnotateRegionSet to deal with consant nodes:
input
  |
begin
  |
 op -- const
  |
end

but for this graph, AnnotateRegionSet can treat const as a normal op:
input
  |
begin
  |
 op -- begin -- end -- const -- begin
  |
 end

I don't think we'd need to treat constant nodes differently everywhere else. At least, in the previous version of AnnotatedRegionSet using the recursive pattern, the constants ended up in the correct region without explicit annotation. I think with the new traversal order though this no longer happens.
The main reason I'm not in favour of explicitly annotating a constant with begin/end is that it implies there is a data flow path in through the constant. We're then violating a different assumption which is that all begin/end annotations cut across data flow.

I guess annotating backend-independent nodes (such as ConstantNodes, TupleGetItem, Tuple, etc) are not trivial and they are special cased. Some (e.g., TupleGetItem node) should look at parents while others (ConstantNode) should look at children. Looking at children, is quite tricky with current infrastructure (correct me, If I am wrong).
Therefore, IMHO I would prefer the current A1) :
input
|
begin
|
op -- begin -- const
|
end
OR the proposed A2) :
input
|
begin
|
op -- const
|
end
I fail to see the value addition of annotating constant nodes as they should strictly belong to the region as the op and should not be considered to be merged with other regions. In that light A2 seems appropriate.
However, the current status A1 (though it has an unnecessary boundary) should work in principle as we bind constants (@zhiics ) if they are a input to a region. But I guess, it might fail to recognize constant tuples. We could fix that as an alternative solution.
Additionally, @comaniac 's proposal makes it compulsory to run MergeCompilerRegions to avoid creating primitive functions with just a constant node in it. Since we do not have control over the output region size of MergeCompilerRegions (yet), we might not want to annotate constants (yet).

Sorry for the late followup. I think both points make sense at certain aspect. @manupa-arm I also think A1 should have been covered by binding constants. Fixing constant tuples looks like a good way to fix this problem though, and it would be helpful to other passes in Relay as well.

Explicitly binding constant tuples in the partitioner seems like a reasonable solution. It would involve generalizing BindParamsByName to work with tuples though (unless we wrote a custom version of that just for partition_graph).

@mbaret yeah, I think that's generally good though. We can probably send an RFC to see if we would break some other things. Otherwise, we can only bake it into partitioning.

@manupa-arm I agree with your point and I prefer A1 more because it at least maintains annotation pairs. A1 actually treats a constant node as an input variable when annotating a target. It seems reasonable to me.

I've implemented an alternative fix in partition_graph.cc by explicitly binding constant tuples (see 5476). This means this PR is no longer required.


Likes Tab No Longer Working

Version of OldTweetDeck 3.6.4 (Chrome) is no longer able to sync Likes of any user (could be due to recent X changes). Am not able to see my own likes, nor likes of anyone I follow even if they've made their likes public again (loading screen keeps syncing). Re-downloaded/re-installed with no fixes; other software up-to-date and other columns working perfectly/syncing correctly.

Same with Firefox

I am having the same issue, but i do not think the reason is the last changes made on Twitter, cause after twitter hide likes it wass still working, even 9 hours ago was still working.

its gone forever

Even for our own likes?


pre-alloc optimized xw product output vector

@arnej27959 please review
@lesters FYI

please have another look


feat: Implement retrying of failed requests

APIClient now accepts Retry configuration where number of retries and backoff can be configured.
The backoff mechanism is inspired by urllib3

Codecov Report

Merging #25 (39e7263) into main (34539df) will decrease coverage by 0.09%.
The diff coverage is 96.49%.

:exclamation: Current head 39e7263 differs from pull request most recent head a17c459. Consider uploading reports for the commit a17c459 to get more accurate results

@@            Coverage Diff             @@
##             main      #25      +/-   ##
==========================================
- Coverage   99.10%   99.00%   -0.10%
==========================================
  Files          28       28
  Lines        1556     1613      +57
==========================================
+ Hits         1542     1597      +55
- Misses         14       16       +2

Impacted Files
Coverage Δ

rossum_api/api_client.py
97.23% <96.42%> (-0.15%)
:arrow_down:

tests/test_api_client.py
99.52% <96.55%> (-0.48%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Reading templateOptions.options

I have created my own type of formlydesigner specific to my webapp. When I try to edit a FormlyFieldConfig, I am having issues reading the templateoptions.options of a select type field.
console.log shows the options as an array, but typescript is screaming at me saying it is an Observable<any[]>. So, when I try to foreach and map, it then shows each object as any[] despite console.log showing it as a single object.
What is a good way to read the options from templateoptions.options in order to populate my custom components?

options can be an array or an instance of Observable:

Option 1: define the expected type (when you're sure that the passed options is always an array).

(field.templateOptions.options as any[]).map(v => ...)

Option 2: check if the passed options is an array first.

if (Array.isArray(field.templateOptions.options)) {
  field.templateOptions.options.map(v => ...)
}


Remove references to log4cxx

Client host fails because there are not Rolling sync with the new changes in https://github.com/ros2/rmw/pull/313


Update the RenderableVNC class to use the  compon…

This PR uses the  component instead of using an html page, so that we don't have to test and maintain the VNC pipeline that goes through vnc_envision.html.  The 'connect_to_running_ens' option on the component should allow this.
Mike, I looked at the logic in RenderableEVSN to figure out where to add the optional query.  But I am not sure how to test this inside AnsysLab.
Mario, maybe the RenderableVNCAngular class can use the component too.

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (a2ddc2f) 80.26% compared to head (c841fc3) 79.80%.
Report is 2 commits behind head on main.

Files
Patch %
Lines

src/ansys/pyensight/core/renderable.py
85.71%
1 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #337      +/-   ##
==========================================
- Coverage   80.26%   79.80%   -0.47%
==========================================
  Files          18       18
  Lines        2554     2570      +16
  Branches      450      412      -38
==========================================
+ Hits         2050     2051       +1
- Misses        350      360      +10
- Partials      154      159       +5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@david-bremer well more than using directly the ansys-nexus-viewer component like you are doing there, I should use a similar approach to call the web-en-sight component, that is the angular component.
Right now, like it was done for the VNC renderable, I am building a URL to the HTML contained in ansys/nexus/angular. I could generate one on the fly. The only issue is that I am passing to that URL the information that then I use inside of the angular component to pass to the ansys-nexus-viewer component. I need to find a way to pass these info downstream, since I cannot pass directly attributes to the angular component as of today


Document how to use the src/sql/archive_config scripts in README.org or remove

Machiavelli uses whole DB back ups now.  Discuss the pros/cons of each solution especially related to GCP.

Instead remove from repo


qemu-bridge-helper is no suitable-bridge helper

My Config:
# Enable virtualisation
  virtualisation.libvirt.swtpm.enable = true;
  virtualisation.libvirt.enable = true;
  virtualisation.libvirt.connections."qemu:///session" = {
    # Add pools
    pools = [
      {
        definition = nixvirt.lib.pool.writeXML {
          name = "default";
          uuid = "a72e2000-ab5e-4c2d-a571-daf611fc77a4";
          type = "dir";
          target = {
            path = "/home/sven/.local/share/libvirt/images";
          };
        };
        # Workaround untils https://github.com/AshleyYakeley/NixVirt/issues/16 is fixed
        volumes = [
          {
            definition = nixvirt.lib.volume.writeXML {
              name = "win10.qcow2";
              capacity = { count = 250; unit = "GiB"; };
              target = {
                format = { type = "qcow2"; };
              };
            };
          }
        ];
        active = true;
      }
      {
        definition = nixvirt.lib.pool.writeXML {
          name = "isos";
          uuid = "5217ddb8-29c2-4a4d-b976-73b9dde59e43";
          type = "dir";
          target = {
            path = "/home/sven/.local/share/libvirt/isos";
          };
        };
        active = true;
      }
      {
        definition = nixvirt.lib.pool.writeXML {
          name = "qemu";
          type = "dir";
          uuid = "be62978e-1d96-466f-bfaf-acc01c9fae76";
          target = {
            path = "/home/sven/.local/share/libvirt/qemu";
          };
        };
        active = true;
      }
    ];

    # Add windows Domain
    domains = lib.mkIf (systemConfig.networking.hostName == "Shi") [
      {
        definition = nixvirt.lib.domain.writeXML (nixvirt.lib.domain.templates.windows
          {
            name = "win10";
            uuid = "a9329510-9185-4849-a4ed-0b52aa2f4d47";
            memory = { count = 24; unit = "GiB"; };
            storage_vol = /home/sven/.local/share/libvirt/images/win10.qcow2;
            nvram_path = /home/sven/.local/share/libvirt/qemu/win10.nvram;
          }
        );
      }
    ];
  };
}

My issue:
Libvirt gives the following error if I try to start my VM:
Error starting domain: 'qemu-bridge-helper' is not a suitable bridge helper: No such file or directory

Traceback (most recent call last):
  File "/nix/store/y8jzspc7yx8rasxihxs0r6b127vqg48a-virt-manager-4.1.0/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/nix/store/y8jzspc7yx8rasxihxs0r6b127vqg48a-virt-manager-4.1.0/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
    callback(*args, **kwargs)
  File "/nix/store/y8jzspc7yx8rasxihxs0r6b127vqg48a-virt-manager-4.1.0/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/y8jzspc7yx8rasxihxs0r6b127vqg48a-virt-manager-4.1.0/share/virt-manager/virtManager/object/domain.py", line 1402, in startup
    self._backend.create()
  File "/nix/store/8bxq9gk9mpsl62my6nd141cjnn9p08qq-python3.11-libvirt-10.0.0/lib/python3.11/site-packages/libvirt.py", line 1379, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: 'qemu-bridge-helper' is not a suitable bridge helper: No such file or directory

Is this NixOS config or Home Manager?

Home-manager

Seem that this has fixed it self. :)


dbt command does not found in multi-root workspace folder

Expected behavior
In our project, we use multi-root workspace.
The folder structure of this project is like this.
.
├── project.code-workspace
├── dbt
│  ├── dbt_project.yml
│  ├── pyproject.toml
│  ├── ...
├── other-workspace-folder
│  ├── ...

As a priority for resolving dbt command paths, we expect the following,

If it is set, the settings of dbt-power-user extension.
Python virtual environment on each workspace folder like dbt folder.
Python virtual environment on workspace root.
Global environment ($PATH)

Actual behavior
The path finding for dbt power user is as follows:

If it is set, the settings of dbt-power-user extension.
Python virtual environment on workspace root.

This python enviroment is determined the first time VSCode is opened, so moving workspace folders does not change the Python virtual environment settings.
Steps To Reproduce

use multi-workspace.
add sample workspace-folder and create dbt project on it.
create python virtual environment on the sample workspace.
set python virtual environment on the sample workspace.

install dbt-core in the python virtual environment on the sample workspace.
open sample/dbt_project.yml

Log output/Screenshots
https://private-user-images.githubusercontent.com/47286750/289367172-311f6d7f-f917-469a-8a9a-4f547416d3ba.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDIyNjEzOTEsIm5iZiI6MTcwMjI2MTA5MSwicGF0aCI6Ii80NzI4Njc1MC8yODkzNjcxNzItMzExZjZkN2YtZjkxNy00NjlhLThhOWEtNGY1NDc0MTZkM2JhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzEyMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMjExVDAyMTgxMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFjZTQ1MjYzOGExMDdhYzM4MjM1Zjk2NjNhMzI1NzA3MmYwZDRkMGIzMjFiNWRmOWY0MThiZjhlZjllY2U0N2QmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.S7E_pLOh74ScMYS_NFfMVzTlBsqXSUiaTFTZdnNSJ7Q
Operating System
Mac OS 14.1.1（23B81）
dbt version
1.7.3
dbt Adapter
1.7.0
dbt Power User version
v0.27.3
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

I'd also like to get DBT working for multi-root workspaces. I recently sent a message to the SQLfluff (Slack) thread asking if this tool was also compatible with multi-root workspaces.
In my case, the first folder of my workspace is not a DBT folder and the extension does not seem to be able to pick my python at all; and indicates no DBT is available.
I have another workspace with just my DBT and everything works well, but it is tedious to switch workspaces. I've tried using folder-specific settings but most are greyed out.

How do you associate a python environment with a specific workspace folder. Can you provide reproduction steps?

I have created a simple project so that this issue can be examined.
https://github.com/yassun7010/vscode-multi-workspace-dbt-sample/tree/main
Indeed, it seems that features such as linage are not working.
I will take some time over the weekend to find the cause and look into it.

@yassun7010: we had to disable the code you submitted because now it has to go through the dbtCoreIntegration class. It seems a bit tricky as well. What if next some user has 2 workspaces with different dbt versions. :)
Current workaround is maybe to associate the python interpreter with the first workspace folder.

Ah, I see.
This extension determines one dbt environment at activation, so it will not work if you have multiple dbt projects.
To support multiple workspaces, for each command or event, the extension needs to find out which workspace dbt is installed in.
For the repository root, workspace.workspaceFolders is
[rootPath], and each workspace folder will contain [folderPath, rootPath]. So the dbt detection logic can be common.
vscode-shandy-sqlfmt does this by examining the dbt execution path.

Anybody working on this issue?
Perhaps the dbt detection mechanism needs to be changed significantly.

@yassun7010 Not sure why I didn't get a notification for this comment.
No one is working on it yet.

Certainly that was not necessary.
I would consider modifying the code, and I think a major modification is needed to properly handle multi-workspace.
No one is currently working on this issue.

@yassun7010 let us know if you need help making these changes

@yassun7010 : Wouldn't it be just easier to use dbtPythonOverride and set it to your preferred python interpreter? This would not require any code changes on your part.

@mdesmet
Our team could not adopt it.
The reason is that the path to the virtual environment is different for each user,
and .vscode/settings.json or $PROJECT.code-workspace files are maintained in Git.

I just wanted to add a small note regarding what was said above :
When using multi-root workspace folders, if the first folder is a DBT project, then the extension seems to work fine for that project. So in your VS Code Project definition, re-ordering folders to put the DBT project as the first folder fixes things for the said DBT environment.
It's a quite nice workaround unless you have other projects/plugins that are not multi-root compatible like DBT power user

Ah, I was unaware of this because I had gotten into the convention of routing the first workspace.
That is certainly a valid method if only vscode-dbt-power-user does not support multi-workspace.

It's a quite nice workaround unless you have other projects/plugins that are not multi-root compatible like DBT power user, or unless you have multiple DBT projects with different configs

Well it seems like I called the devil. A few weeks after writing this message we decided to start an overhaul of our dbt-models and decided to start a new DBT project... so now we're in the configuration where we have 2 DBT projects to maintain, and the extension does not work so well with that....
Anyways I'm hoping there will progress on this. Unfortunately I don't think I can help too much apart from giving feedback at the moment.


$.ajax is not a function

Once I  get the base64 image string in the   main.js  file, the console throws   "$.ajax is not a function" error message.
How do you actually initialize jQuery Ajax  post to server? The documentation is not clear

I found out it was because the code included in demo is the jQuery light  version that precludes ajax.
Replaced  https://code.jquery.com/jquery-3.3.1.slim.min.js  with  the full jQuery version   ( https://code.jquery.com/jquery-3.3.1.min.js  )  and that worked

jemoeder werkt niet

haha lol scrub noscope360 m8 lmao roflllll

I found out it was because the code included in demo is the jQuery light version that precludes ajax.
Replaced https://code.jquery.com/jquery-3.3.1.slim.min.js with the full jQuery version ( https://code.jquery.com/jquery-3.3.1.min.js ) and that worked

Replaced but still recieving the same issue


Terraform plan always shows changes

When running Plan and Apply on terraform from https://github.com/privacysandbox/aggregation-service/blob/main/docs/gcp-aggregation-service.md#set-up-your-deployment-environment, Aggregation Service is deployed successfully. But when terraform plan is executed again it is not empty (and should be).
Most probably the issue is with module.job_service.module.autoscaling.google_cloudfunctions2_function.worker_scale_in_cloudfunction. Two fields are missing there: docker_repository and environment_variables.LOG_EXECUTION_ID
as a result terraform updates  worker_scale_in_cloudfunction wach time when it is run

Hi @sebastianrtb, thanks for the feedback. There's a known issue with the GCP Terraform provider, and once that's resolved, you won't see docker_repository changing in place each time terraform apply is run. For the LOG_EXECUTION_ID, we're still investigating how to prevent this from changing for each apply so I'll have to follow up with you on that.

thank you

Hi @sebastianrtb, I have an update on LOG_EXECUTION_ID. We're going to roll out a fix for this in one of our upcoming releases, but in the meantime, you can prevent LOG_EXECUTION_ID from popping up in each terraform plan by adding LOG_EXECUTION_ID  = true to two of the terraform modules:

autoscaling/workerscalein.tf in service_config > environment_variables (example below)

...
service_config {
    # Only one instance should run to control instance group scale-in
    max_instance_count            = 1
    timeout_seconds               = 180
    available_memory              = "${var.autoscaling_cloudfunction_memory_mb}M"
    service_account_email         = var.worker_service_account
    ingress_settings              = "ALLOW_ALL" # Otherwise, it cannot be triggered by Cloud Scheduler.
    vpc_connector                 = var.vpc_connector_id
    vpc_connector_egress_settings = var.vpc_connector_id == null ? null : "ALL_TRAFFIC"
    environment_variables = {
      PROJECT_ID                  = var.project_id
      REGION                      = var.region
      SPANNER_INSTANCE_ID         = var.metadatadb_instance_name
      SPANNER_DATABASE_ID         = var.metadatadb_name
      MANAGED_INSTANCE_GROUP_NAME = google_compute_region_instance_group_manager.worker_instance_group.name
      TERMINATION_WAIT_TIMEOUT    = var.termination_wait_timeout_sec
      ASG_INSTANCES_TTL           = var.asg_instances_table_ttl_days
      LOG_EXECUTION_ID            = true
    }
  }
...

frontend/main.tf in service_config > environment_variables (example below)

...
service_config {
    min_instance_count            = var.frontend_service_cloudfunction_min_instances
    max_instance_count            = var.frontend_service_cloudfunction_max_instances
    timeout_seconds               = var.frontend_service_cloudfunction_timeout_sec
    available_memory              = "${var.frontend_service_cloudfunction_memory_mb}M"
    service_account_email         = google_service_account.frontend_service_account.email
    vpc_connector                 = var.vpc_connector_id
    vpc_connector_egress_settings = var.vpc_connector_id == null ? null : "ALL_TRAFFIC"
    environment_variables = {
      PROJECT_ID             = var.project_id
      INSTANCE_ID            = var.spanner_instance_name
      DATABASE_ID            = var.spanner_database_name
      PUBSUB_TOPIC_ID        = var.job_queue_topic
      PUBSUB_SUBSCRIPTION_ID = var.job_queue_sub
      JOB_METADATA_TTL       = var.job_metadata_table_ttl_days
      JOB_VERSION            = var.job_version
      LOG_EXECUTION_ID       = true
    }
  }
...

thank you for quick fix ;)

@sebastianrtb Just following up here, the fix for this was rolled out in Aggregation Service version 2.8.0. Once you upgrade to that version or later, you will not need to manually add the log_execution_id to your Terraform.

thats great news, thank you again


[FEATURE REQUEST] Refactor Grafana upgrade role

Is your feature request related to a problem? Please describe.
We keep grafana_version in two places (Grafana role defaults and hardcoded in grafana upgrade task)
Describe the solution you'd like
We need to refactor these roles to use in upgrade role grafana_version from grafana defaults. This should not be hardcoded in upgrade task.

Tested together with #2699.


Remove deploader, depend on MCMP fixes #3991

As stated in issue #3991 support for deploader is being deprecated.
So, I removed deploader, all references to it, and added a required-after on MCMultiPart.

Tested it with and without MCMultiPart installed.
Without it gives the forge missing mod page on startup.
With starts and runs as supposed.

Thanks!


could author please push the code?

Everyone is looking forward to the repo
From the time you created this repo, the community has been following it and hoping to release an exciting alpha version as soon as possible : ).
How can I duplicate it if you don't release it?
Need to repost it to CSDN immediately, @delongchen  will become a star of the community in no time !!!

now you can see the code, and CSDN gck


Release/1.1.2

Coverage decreased (-0.008%) to 99.375% when pulling 399c13718fba43d71aa3c7b7f80e7d7dd2b807e9 on release/1.1.2 into 8cc20a19d01b18b83e18e54aabb416c8dedabfde on master.


Remove the overwrite option from github workflow add command

I got a good comment in #5763

About the --overwrite option: is that really necessary? It seems to me that when you deploy an application using ci/cd, it should always overwrite the existing package...

It makes total sense that a CI/CD pipline is rather made to continuously redeploy and overwrite existing package with new one.
We should remove this option from the command and make overwrite default.
This is a breaking change and should target v8

Good point. Let's deprecate it already, with a warning, and remove in v8.

@waldekmastykarz good point. I added an issue for that

Since we have v8 branch we may now start working on it!
Let's get this 🚢
and Let's put CLI to the 🚀🌙

Can I work on it?

PR merged


Upgrade dependencies, upgrade version

This is a major upgrade to reflect new versions of play, bootstrap, akka, curator, jackson, json4s, kafka, etc.  Play framework deprecated it's Crypto functionality so there were changes to basic auth and basic auth with LDAP.
Need further testing to ensure UI elements are working correctly with bootstrap 4.  Also need help testing LDAP.  Volunteers welcome!

@linehrr can you help test this ?

@dpavlov can you help test LDAP functionality?  I had to make changes due to deprecation of Crypto in Play framework.

@patelh

[info] *** 1 SUITE ABORTED ***
[error] Error: Total 164, Failed 0, Errors 1, Passed 163
[error] Error during tests:
[error] 	controller.api.TestKafkaStateCheck
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 286 s, completed Mar 27, 2019 11:13:49 PM
/home/travis/.travis/functions: line 524:  4660 Terminated              travis_jigger "${!}" "${timeout}" "${cmd[@]}"
The command "travis_wait 30 ./sbt clean coverage assembly" exited with 1.

looks like CI is erroring out, I can for sure test this, after CI passes, I will test the non-LDAP part of it. I don't have LDAP setup so I can't test that.

@linehrr can you try this in your dev environment?

@patelh ok testing it. will report bugs if found.
for other ppl who wants to test this their envs, you can now easily deploy this new version via the new docker image I just published here:
https://hub.docker.com/r/kafkamanager/kafka-manager/tags
tag: <IP_ADDRESS>-beta

@patelh
Old kafka: 1.x

New kafka: 2.x

looks like the red alarming highlights are missing in the new version.

Thanks @linehrr , I fixed the highlighting.  Also upgrade the kafka dependency to 2.2.0, can you please try this again and see if you have any issues with kafka based consumer offsets.

@patelh  I am seeing highlighting now.
re: consumer offsets, what was the thing that you wanted to test? I can see those consumer offsets and lags etc. so I don't know exact how this would be tested.

@patelh we've used this version for several days, no apparent issue came up. we tried creating topic, deleting topic and inspecting topic, everything looks good so far.

@linehrr yea just wanted you to confirm you aren't seeing any other functionality which is broken.  Thanks!  I am ready to merge but need to get someone to test LDAP, otherwise it could be broken.  If I can't get anyone to test LDAP in a few days, will merge this.  It needs to be merged, started on these changes July, 2018.

sounds good, once merged can we cut a release, I will release the docker
image accordingly so we can get more ppl on the newest version.
R
On Mon, Apr 8, 2019 at 5:37 PM patelh<EMAIL_ADDRESS>wrote:

@linehrr https://github.com/linehrr yea just wanted you to confirm you
aren't seeing any other functionality which is broken. Thanks! I am ready
to merge but need to get someone to test LDAP, otherwise it could be
broken. If I can't get anyone to test LDAP in a few days, will merge this.
It needs to be merged, started on these changes July, 2018.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yahoo/kafka-manager/pull/627#issuecomment-481017740,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABEnEw1CQmJsHIa590wgL-7Mhh-Kw7Etks5ve7aCgaJpZM4cO30i
.

--
Best Regards
Ryne

@dpavlov Please help test LDAP, if we don't hear from you, we'll have to risk it and merge this PR.

Hi @patelh i will try to test LDAP today and let you know about results

I've tests LDAP manually today. From my point of view it is working as expected

Thanks @dpavlov


Sqlite

That's good, but why there are code, which is not connected to Our db?

What is our db?

Medivac


File location represented by hostname in addition to filesystem path.

Fixes #11

TODO:
log():

parents to locationStrings
add location based on path


Irsa passRole should accept a list instead of ony one since we can have different node roles for each nodeClass

Description
Since v0.32 Karpenter allows to specify a different role for each nodeClass.
Since this means that you may have >1 role, the IRSA should have the Passrole action to all necesary resources.
Right now, if you set create_iam_role=false, the module only allows to pass a single roleArn for this.
iam_role_arn should be a list so IRSA can pass any of those

[X] ✋ I have searched the open/closed issues and my issue is not listed.

⚠ Note
Before you submit an issue, please perform the following first:

Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
Re-initialize the project root to pull down modules: terraform init
Re-attempt your terraform plan or apply and check if the issue still persists

Versions

Module version [Required]: 19.20

Terraform version: 1.5.7

Reproduction Code [Required]
module "karpenter" {
  source  = "terraform-aws-modules/eks/aws//modules/karpenter"
  version = "19.20.0"

  cluster_name = var.cluster_name

  irsa_oidc_provider_arn          = var.cluster_oidc_provider_arn
  irsa_namespace_service_accounts = ["karpenter:karpenter"]
  irsa_use_name_prefix            = var.irsa_use_name_prefix

  # Since Karpenter is running on an EKS Managed Node group,
  # we can re-use the role that was created for the node group
  create_iam_role = false
  iam_role_arn    = var.cluster_karpenter_controller_nodegroup.iam_role_arn

  # In v0.32.0/v1beta1, Karpenter now creates the IAM instance profile
  # so we disable the Terraform creation and add the necessary permissions for Karpenter IRSA
  enable_karpenter_instance_profile_creation = true

  tags = var.tags
}

Expected behavior
iam_role_arn should be a list so IRSA can pass any of those
Actual behavior
iam_role_arn only accepts a single role, thus nodes defined in many NodeClasses can only assume a single role
Terminal Output Screenshot(s)

Additional context

for this, we will not be deviating from the upstream guidance provided by Karpenter. We have numerous issues with having a policy that differed from what Karpenter recommended and therefore we want to simply align and consume the policy they recommend - this does not accept a list of roles so we will continue with a single pass role https://github.com/aws/karpenter-provider-aws/blob/a7d87efb1f22eeed5b9329255abb7d03d009a220/website/content/en/preview/getting-started/getting-started-with-karpenter/cloudformation.yaml#L189


Failed to build on ios, caused by swift compiler

Current Behavior
I already followed the ios installation and configuration as on video. But still the output of build is:
CompileSwift normal x86_64 /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift
    cd /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerViewManager.swift /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoPlayer.swift /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerView.swift /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/VideoProcessingGPUFilters.swift /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoTrimmer/RNVideoTrimmer.swift -primary-file /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift -emit-module-path /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/Objects-normal/x86_64/RNVideoProcessingManager~partial.swiftmodule -emit-module-doc-path /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/Objects-normal/x86_64/RNVideoProcessingManager~partial.swiftdoc -serialize-diagnostics-path /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/Objects-normal/x86_64/RNVideoProcessingManager.dia -emit-dependencies-path /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/Objects-normal/x86_64/RNVideoProcessingManager.d -emit-reference-dependencies-path /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/Objects-normal/x86_64/RNVideoProcessingManager.swiftdeps -target x86_64-apple-ios7.0-simulator -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.1.sdk -I /Users/trias/Projects/myapp-project/ios/build/Build/Products/Debug-iphonesimulator -F /Users/trias/Projects/myapp-project/ios/build/Build/Products/Debug-iphonesimulator -g -module-cache-path /Users/trias/Projects/myapp-project/ios/build/ModuleCache.noindex -swift-version 3 -enforce-exclusivity=checked -Onone -serialize-debugging-options -Xcc -I/Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/RNVideoProcessing-generated-files.hmap -Xcc -I/Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/RNVideoProcessing-own-target-headers.hmap -Xcc -I/Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/RNVideoProcessing-all-non-framework-target-headers.hmap -Xcc -ivfsoverlay -Xcc /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/all-product-headers.yaml -Xcc -iquote -Xcc /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/RNVideoProcessing-project-headers.hmap -Xcc -I/Users/trias/Projects/myapp-project/ios/build/Build/Products/Debug-iphonesimulator/include -Xcc -I/Applications/Xcode.app/Contents/Developer/Toolcha
ins/XcodeDefault.xctoolchain/usr/include -Xcc -I/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/../../../React -Xcc -I../../react-native/React -Xcc -I../../react-native/React/UIUtils -Xcc -I../../react-native/React/CxxModule -Xcc -I../../react-native/React/CxxBridge -Xcc -I../../react-native/React/Inspector -Xcc -I../../react-native/React/DevSupport -Xcc -I../../react-native/React/Fabric -Xcc -I../../react-native/React/Profiler -Xcc -I../../react-native/React/CxxUtils -Xcc -I../../react-native/React/Modules -Xcc -I../../react-native/React/Views -Xcc -I../../react-native/React/Base -Xcc -I../../react-native/React/Fabric/Mounting -Xcc -I../../react-native/React/Fabric/Surface -Xcc -I../../react-native/React/Views/SafeAreaView -Xcc -I../../react-native/React/Views/ScrollView -Xcc -I../../react-native/React/Base/Surface -Xcc -I../../react-native/React** BUILD FAILED **

The following build commands failed:
        CompileSwift normal x86_64 /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerViewManager.swift
        CompileSwift normal x86_64 /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoPlayer.swift
        CompileSwift normal x86_64 /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerView.swift
        CompileSwift normal x86_64 /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoTrimmer/RNVideoTrimmer.swift
        CompileSwift normal x86_64 /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift
        CompileSwiftSources normal x86_64 com.apple.xcode.tools.swift.compiler
(6 failures)

An error was encountered processing the command (domain=IXUserPresentableErrorDomain, code=1):
This app could not be installed at this time.
Failed to chmod /Users/trias/Library/Developer/CoreSimulator/Devices/F51FAC52-C403-498B-8898-ECAFCAC70EA5/data/Bundle/Application/95D4C6E4-4A86-4B57-AB4A-FF34C7113A6C/MyAppProject.app/MyAppProject : No such file or directory
Failed to chmod /Users/trias/Library/Developer/CoreSimulator/Devices/F51FAC52-C403-498B-8898-ECAFCAC70EA5/data/Bundle/Application/95D4C6E4-4A86-4B57-AB4A-FF34C7113A6C/MyAppProject.app/MyAppProject : No such file or directory
Underlying error (domain=MIInstallerErrorDomain, code=4):
        Failed to chmod /Users/trias/Library/Developer/CoreSimulator/Devices/F51FAC52-C403-498B-8898-ECAFCAC70EA5/data/Bundle/Application/95D4C6E4-4A86-4B57-AB4A-FF34C7113A6C/MyAppProject.app/MyAppProject : No such file or directory
org.reactjs.native.example.MyAppProject: 27207
/Fabric/Mounting/MountItems -Xcc -I../../react-native/React/Fabric/Mounting/ComponentViews -Xcc -I../../react-native/React/Base/Surface/SurfaceHostingView -Xcc -I../../react-native/React/Fabric/Mounting/ComponentViews/ScrollView -Xcc -I../../react-native/React/Fabric/Mounting/ComponentViews/ActivityIndicator -Xcc -I../../react-native/React/Fabric/Mounting/ComponentViews/View -Xcc -I../../react-native/React/Fabric/Mounting/ComponentViews/Root -Xcc -I../../react-native/React/Fabric/Mounting/ComponentViews/Text -Xcc -I/Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/DerivedSources/x86_64 -Xcc -I/Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/DerivedSources -Xcc -DDEBUG=1 -Xcc -working-directory/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios -import-objc-header /Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessing-Bridging-Header.h -pch-output-dir /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/PrecompiledHeaders -pch-disable-validation -module-name RNVideoProcessing -o /Users/trias/Projects/myapp-project/ios/build/Build/Intermediates.noindex/RNVideoProcessing.build/Debug-iphonesimulator/RNVideoProcessing.build/Objects-normal/x86_64/RNVideoProcessingManager.o -index-store-path /Users/trias/Projects/myapp-project/ios/build/Index/DataStore -index-system-modules
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:11:33: error: use of undeclared type 'RCTViewManager'
class RNVideoProcessingManager: RCTViewManager {
                                ^~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:10:2: error: only classes that inherit from NSObject can be declared @objc
@objc(RNVideoProcessingManager)
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:13:34: error: use of undeclared type 'UIView'
   @objc override func view() -> UIView! {
                                 ^~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:17:25: error: method does not override any method from its superclass
    @objc override func constantsToExport() -> [AnyHashable: Any] {
          ~~~~~~~~      ^
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:26:25: error: method does not override any method from its superclass
    override class func requiresMainQueueSetup() -> Bool {
    ~~~~~~~~            ^
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerViewManager.swift:9:29: error: use of undeclared type 'RCTViewManager'
class RNTrimmerViewManager: RCTViewManager {
                            ^~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerViewManager.swift:8:2: error: only classes that inherit from NSObject can be declared @objc
@objc(RNTrimmerViewManager)
~^~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoPlayer.swift:12:22: error: use of undeclared type 'RCTView'
class RNVideoPlayer: RCTView {
                     ^~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoPlayer.swift:11:2: error: only classes that inherit from NSObject can be declared @objc
@objc(RNVideoPlayer)
~^~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerView.swift:10:22: error: use of undeclared type 'RCTView'
class RNTrimmerView: RCTView, ICGVideoTrimmerDelegate {
                     ^~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerView.swift:10:31: error: use of undeclared type 'ICGVideoTrimmerDelegate'
class RNTrimmerView: RCTView, ICGVideoTrimmerDelegate {
                              ^~~~~~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNTrimmerView/RNTrimmerView.swift:9:2: error: only classes that inherit from NSObject can be declared @objc
@objc(RNTrimmerView)
~^~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:19:26: error: use of unresolved identifier 'AVLayerVideoGravityResizeAspect'
            "ScaleNone": AVLayerVideoGravityResizeAspect,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:20:28: error: use of unresolved identifier 'AVLayerVideoGravityResize'
            "ScaleToFill": AVLayerVideoGravityResize,
                           ^~~~~~~~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:21:31: error: use of unresolved identifier 'AVLayerVideoGravityResizeAspect'
            "ScaleAspectFit": AVLayerVideoGravityResizeAspect,
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/trias/Projects/myapp-project/node_modules/react-native-video-processing/ios/RNVideoProcessing/RNVideoProcessingManager.swift:22:32: error: use of unresolved identifier 'AVLayerVideoGravityResizeAspectFill'
            "ScaleAspectFill": AVLayerVideoGravityResizeAspectFill
                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Installing build/Build/Products/Debug-iphonesimulator/MyAppProject.app
Launching org.reactjs.native.example.MyAppProject

Expected Behavior
It should be compiled successfully.
Your Environment
macOS 10.14
Xcode 10.1

software
version

react-native-video-processing
1.20.0

react-native
0.56

node
8.11.1

Please check your Swift version. We're supporting Swift 3+. But seems like there's the other issue. The compiler can't find RCTViews, ViewManager etc. Please check your Build Files

@shahen94 Hi, I already changed all the swift version from 3, 4, and 4.2. But still doesn't work.
Do you have a sample application that can be test?

private projects only. The issue is not related to your swift compiler. Check your Build files.

I am having the same issue

same

Same


Potential different .NET 4.5 apps can override/damage settings of each other

Within the settings implementation of "src/Plugin.Settings.Net45/Settings.cs" you use the framework function "IsolatedStorageFile.GetUserStoreForAssembly()" to get the isolated storage settings target. According to the .NET docs this may cause issues because if one use your .NET implementation assembly within different applications will always uses the same isolated store when using this method (see remarks section).
Maybe it is better to use the call .GetUserStoreForDomain() to separate settings of different apps using the same (your) impl. assembly.
What do you think?

Are you sure?
It says: Different assemblies running within the same application domain always have distinct isolated stores.
always have distinct... @Clancey implemented it

Yes, different assemblies. But if one use -> your <- implementation (dll/assembly) in different apps it is always the same assembly: -> your <- assembly implementing the Store access. If -> I am <- call the method at Store in different assemblies, I will have distinct stores, yes.

Yes, you seem correct.  You should send a pull request changing it to GetUserStoreForApplication Method ()

Ahhhh interesting :) good one, the wording is very strange on there :) will change in the next v3 change.


Add another disk to the VM

How can I add a new disk to the VM, apart from the current disk?
My case: Create a VM and make it have two disks at startup.
main.tf:

...

resource "xenorchestra_vm" "bar" {
    memory_max =<PHONE_NUMBER>
    cpus  = 1
    cloud_config = "${xenorchestra_cloud_config.bar.template}"
    name_label = "VM-build-Terraform"
    name_description = "VM criada a partir do terraform - usando provider Xen Orchestra"
    template = "${data.xenorchestra_template.template.id}" #posso tambem colocar o uuid do template
    network {
      network_id = "${data.xenorchestra_pif.pif.network}"
    }

    disk {
      sr_id = "0c931761-7885-52c3-13ba-3dddd4dd84b2"
      name_label = "Debian10-terraform"
      size =<PHONE_NUMBER>
    }

   #other disk here
    disk {
      sr_id = "0c931761-7885-52c3-13ba-3dddd4dd84b2"
      name_label = "Debian10-terraform-1"
      size =<PHONE_NUMBER>0
    }
}

it does not fail after terraform apply, however, the other disk is neither created nor attached to the VM.
this is normal? Any ideas / suggestions here?

@marlluslustosa I have never tested that functionality out before with the xenorchestra_vm resource. The documentation should do a better job explaining what is / isn't tested or supported (which I can update through this as well).
I will need to take a look at how we can enhance the resource to take into account all disks, make sure they are attached to the VM and that terraform can correctly manage the state of the resource once it's created (showing a "diff" when a disk has been detached, etc).

This will be fixed in v0.9.0 of the provider (released later today).

thanks for the effort!


[Enhancement] Raw data receive prefix parser should pass info to command callback

A common idiom across all modem ports seems to be that receiving raw data via
_Cellular_TimeoutAtcmdDataRecvRequestWithCallback(), currently mostly used in Cellular_SocketRecv()
requires duplication of the prefix parsing in the actual prefix handler, example from BG96
https://github.com/FreeRTOS/FreeRTOS-Cellular-Interface-Reference-Quectel-BG96/blob/c33a3628e015f88df24b21160ba9dbdabc781da1/source/cellular_bg96_api.c#L1690-L1774
and then in the callback itself, since the callback usually refers to information and / or performs checks based on info found in prefix
https://github.com/FreeRTOS/FreeRTOS-Cellular-Interface-Reference-Quectel-BG96/blob/c33a3628e015f88df24b21160ba9dbdabc781da1/source/cellular_bg96_api.c#L1277-L1342

We could add pData, pDataLen argument to prefix parser prototype, though the interface is a bit unwieldy as it is
https://github.com/FreeRTOS/FreeRTOS-Cellular-Interface/blob/d136151a7efefca0df6836e248d30e16b6422a69/source/include/common/cellular_common.h#L192-L196
We might repurpose the pCallbackContext, which is currently being set to CellularContext_t within _Cellular_TimeoutAtcmdDataRecvRequestWithCallback()
AFAIK there aren't many good use cases to have access to the whole library context within callbacks as you're in the reader thread at the time of the callbacks invocation, during which one cannot use the library functions.
Currently one could use it to refer to a member of pModuleContext, but that is also a very roundabout way to pass info.

Upon further study, I've determined that I can use the current interface as it is.
_Cellular_TimeoutAtcmdDataRecvRequestWithCallback() sets up the pDataPrefixCBContext, which is provided to prefix handler.
One can set up the pCallbackContext (prefix handler context) and atReq->pData (parser user data) to point to same object and use that to pass info between them.


Unable to execute OpenCL programs (v17.06)

I am trying to build and run the OpenCL examples, namely cl_convolution.cpp but the program crashes on the line   CLScheduler::get().default_init(); with the following output:
*** Error in `./cl_convolution': free(): invalid pointer: 0x0000000020e4bb08 ***
Aborted (core dumped)

The device is an RK3399 with 64-bit ARM processors and a Mali T860 GPU running Ubuntu 16.04. I tried compiling using the following commands:
aarch64-linux-gnu-g++ cl_convolution.cpp utils/Utils.cpp -std=c++14 -lOpenCL -l arm_compute -o cl_convolution
Also tried -lmali instead of '-lOpenCL', with the same problem. How can I fix this? Thanks

I even tried with a minimal example as follows:

#include "arm_compute/runtime/CL/CLFunctions.h"
#include "arm_compute/core/Types.h"
#include "utils/Utils.h"
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/time.h>
#include <stdarg.h>
#include <iostream>

using namespace arm_compute;

int main(int argc, const char **argv)
{
printf("Begin\n");

CLTensor A;
const TensorShape a_shape(5,5);

A.allocator()->init(TensorInfo(a_shape, 1, DataType::F32));

printf("allocating A...\n");
A.allocator()->allocate();
printf("allocation successful.\n");

return 0;

}

With the following error outputs:
_Begin
allocating A..._
terminate called after throwing an instance of 'cl::Error'
what():  clCreateBuffer
Aborted (core dumped)

In the second case you need to have ARM_COMPUTE_CL defined a the top of your file in order for the CL exceptions to be handled correctly (At the moment it's impossible to tell what's wrong in clCreateBuffer)
We've got several of these boards in the office and they work fiine, can you try with the compilers we recommend in the documentation and using c++11 instead of 14 ?

@AnthonyARM could you please clarify what you mean by "the compilers recommended in the docs"? In the docs, it says to compile using aarch64-linux-gnu-g++, which is what I do here. I also tried arm-linux-gnueabihf-g++, but I'm guessing this creates a 32-bit program, which I can't run on the current machine.
As for the second case, I defined ARM_COMPUTE_CL and the error message doesn't seem to change to include additional detail on the error. Also tried with c++11, but the problem persists.
I wonder if I need to apt-get upgrade or something or do something else to the system? But I'm guessing not, because standard OpenCL programs work with no issues. I'm also not sure where I got the aarch64 compiler, and doing apt-get install g++-aarch64-linux-gnu tells me  Package 'gcc-aarch64-linux-gnu' has no installation candidate.
Did you have to set up anything on the device before compiling ACL programs?

It seems this might be GPU driver related issue. These errors come up whenever a section of code is reached that targets OpenCL to MIDGARD (for example _target(GPUTarget::MIDGARD)). And I can't seem to find a guide to help with installing/updating the Mali driver.

Installed the driver from https://github.com/rockchip-linux/libmali, but still same problem.

Going a little deeper, the error seems to occur at the line (CLScheduler.h):
    CLKernelLibrary::get().init("./cl_kernels", cl::Context::getDefault(), cl::Device::getDefault());
I wonder if the first argument is causing the error, since the folder cl_kernels does not exist anywhere in the headers folders and it's only found in the source folders. But putting the folder in the same directory as the executable doesn't fix it either.

If you've compiled the library with embed_kernels=1 then this parameter is irrelevant and it doesn't matter that this folder doesn't exist.
What happens if you do cl::Context::getDefault() and cl::Device::getDefault() on their own before calling the init() method ?

I didn't compile my own binaries; I'm using the ones provided on this page. What is the value  of embed_kernels in the provided binaries?
Standard OpenCL programs run fine, even with the C++ wrapper. So calling cl::Context::getDefault() and other OpenCL functions does not produce any errors.

embed_kernels means the sources for the OpenCL kernels are embedded inside the library binary instead of read from file at runtime.
That's why I'm saying that having a cl_kernels folder shouldn't make a difference.
All libraries uploaded on Github have been compiled with embed_kernels=1 therefore I don't think that's where your problem comes from .
Can you try to catch the cl::Error in your code in order to print the error number ? (Like here)
If you say you manage to create your own context and device then you could try to manually call the init functions of the CLKernelLibrary and CLScheduler with them (Like default_init() does)

Thank you for your help. In the second code, the error was -34 (invalid context). But I've already built the library before seeing your answer, and using the self-built library for some reason resolves all the errors (cl_convolution runs without errors). Is there a list of all build options used for the provided binaries on this page? I wonder what was causing the error. I changed two options from the default configuration before building: target arch to arm64-v8a, and of course embeded_kernels=True.

Glad it works !
You can use strings libarm_compute.so | grep arm_compute_version to find out what parameters were used.
It's odd that the pre-built binaries don't work for you because we test them on some boards before we release them and they worked then.

I'm not exactly sure what was causing the errors, since the build options seem to be about the same. But I guess the issue is resolved now.

I had exact same problem when I built the ARM compute library with toolchain 4.9 and the application with toolchain 5.4. After compile the library and application with same toolchain, it works.


Add desired census features to db and  forward with representative points

Based on explorations in #290, only the desired census features will be added to the db, since adding all features with non-descriptive column names might confuse or bloat the db.
This data should be made available as json in the representative points sent to the frontend.

The raw data now lives in Postgres under tables named acs_dp_05 &c. These tables can be joined together to create a table containing only the columns that we care about (with null handling / human-readable names).

I edited the above suggested features to reflect based on other trusted sources that report on demographic data such as KFF and CMS. Examples here:

https://data.cms.gov/Marketplace-Eligible-Uninsured-People-for-Outreach/The-Number-of-Estimated-Eligible-Uninsured-People-/pc88-ec56
https://www.kff.org/state-category/demographics-and-the-economy/population/

One thing that is confusing is that most sources separate age by 0-18, 19-25, 26-34, 35-54, 55-65, 65+ but DP05 above groups ages differently. @tetraptych is there a different source that everyone is using for age?

For some reason my updates in the comment above didn't change. Just redid them-- the race category should factor in Latino/Hispanic Ethnicity now.


[MM-33238] Check for admin URL when toggling back bar

Summary
When switching between tabs, we check if the back bar should be displayed. Unfortunately, we didn't account for the admin_console route and you would see the back displayed when switching back to a tab on the admin_console.
Now we check for the admin_console route specifically.
NOTE: Are there any other routes we should be checking for that don't need the back button?
Issue link
https://mattermost.atlassian.net/browse/MM-33238

NOTE: Are there any other routes we should be checking for that don't need the back button?

maybe the plugin marketplace?
not sure if plugins might also have problems with that (like incident response)

NOTE: Are there any other routes we should be checking for that don't need the back button?

maybe the plugin marketplace?
not sure if plugins might also have problems with that (like incident response)

Plugin Marketplace is a full-screen modal so it doesn't have its own route.


[stable/datadog] Make the seccomp profile for system-probe optional

What this PR does / why we need it:
The system-probe container currently has a specific seccomp profile.
This seccomp profile currently misses some syscalls that are necessary to
exec inside the container.
Concretely, attempting to exec inside the container produces this error:
$ kubectl exec -ti datadog-fswnc -c system-probe /bin/bash
shell-init: error retrieving current directory: getcwd: cannot access parent directories: Operation not permitted
bash: initialize_job_control: getpgrp failed: Operation not permitted
command terminated with exit code 1

If we add setpgrp to the seccomp profile, we get:
$ kubectl exec -ti datadog-kbg97 -c system-probe /bin/bash
shell-init: error retrieving current directory: getcwd: cannot access parent directories: Operation not permitted
I have no name!@datadog-kbg97:.$ exit

If we add getcwd, we get:
$ kubectl exec -ti datadog-7b7lf -c system-probe /bin/bash
I have no name!@datadog-7b7lf:/$ exit

If we add geteuid and geteuid32, we get:
$ kubectl exec -ti datadog-c42rb -c system-probe /bin/bash
/bin/bash: cannot set uid to -1: effective uid 0: Invalid argument
/bin/bash: cannot set gid to -1: effective gid -1: Invalid argument
bash-5.0$ exit

If we get getgid and getgid32, we get:
$ kubectl exec -ti datadog-tp4qd -c system-probe /bin/bash
/bin/bash: cannot set uid to -1: effective uid 0: Invalid argument
bash-5.0$ exit

etc.
By default, the seccomp profile will be applied so that the system-probe agent is hardened but this PR adds an option to completely remove the seccomp profile in order to ease debugging.
The removal of the seccomp profile has been tested on GKE, both with Ubuntu and with Container-Optimized OS and both with docker and containerd.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

Hi @L3n41c ,
To add e2e tests in the CI, you can provide custom "values.yaml" files. to validate the deployment with and without the seccomp profile.
put your new values.yaml file into this folder: https://github.com/helm/charts/tree/master/stable/datadog/ci

/ok-to-test

/assign clamoriniere

/lgtm


Prepare-Release fails with "The property 'Count' cannot be found on this object"

D:\github\azure\net [master ≡]> .\eng\common\scripts\Prepare-Release.ps1 Azure.Core core

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

PropertyNotFoundException: D:\github\azure\net\eng\scripts\Language-Settings.ps1:25
Line |
  25 |      if ($projectPaths.Count -gt 1)
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~
     | The property 'Count' cannot be found on this object. Verify that the property exists.

cc @weshaggard

thanks for filing the issue. I have a PR put to resolve this https://github.com/Azure/azure-sdk-for-net/pull/19686

Fixed with https://github.com/Azure/azure-sdk-for-net/pull/19651


WSA messes up the default browser to open a link in Windows

Steps to reproduce
Install an android program that has a "open link in app" support, and go to Android settings (wsa://com.android.settings),  "Apps", find the installed app, and go to "Open by default" and then make sure "Open supported link" option is enabled (which should be enabled by default).

Then in windows, open a web link that can be opened in that app:

The following behaviors will also apply when open a link from WSA using "Windows default preference", making it impossible to open the link in Windows browsers without manually copy and paste the link to browser address bar.
✔ Expected Behavior
The default web browser of Windows system should be opened and link should be opened in the windows browser, or at least there should be an option to let user choose how to open the link.
❌ Actual Behavior
There can be 2 cases of the actual behavior:

For some links, it will be open directly in the android app, regardless of whether WSA is running or not. If WSA is not running, it will be started. I think this may also explain #342
For some links, a popup windows will show to choose an app to open the link. Possible choices containing WSA and default browser:

If "Windows Subsystem for Android" is chosen, the behavior will be same as 1.
If the default browser is chosen, the link sometimes will be opened in Windows browser correctly, but sometimes will be opened in its source code file like this:

Besides, there's no way to change whether to open the link in Android app through windows settings (Windows 11 Settings -> Apps -> Default apps -> Find the Android app -> Click any of the link types, there will be no other option to choose):

Other Software
The list of apps that I have tested:

Amazon Shopping
eBay
Walmart
Facebook
Linkedin
Instagram

Please specify the version of Windows Subsystem for Android
2306.40000.1.0

I wonder if "Windows default apps" app in wsa caused this

Duplicate of #342


Error with Azure OpenAI embeddings

Describe the bug
The command 'memgpt load directory ...' fails with the following error:
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Verified embedding deployment, endpoint URL and keys are correct and set in the environment and MemGPT config.
Was able to resolve by updating the embedding_model() function in embeddings.py to use the AzureOpenAIEmbedding instead of OpenAIEmbedding (when the endpoint is Azure).
return AzureOpenAIEmbedding(
    model=model,
    deployment_name=deployment,
    api_key=config.azure_key,
    azure_endpoint = config.azure_endpoint,
    api_type="azure",
    api_version=config.azure_version,
)

Please describe your setup

[x] MemGPT version

What is the output of memgpt version? (eg "0.2.4"): 0.2.6

[x] How did you install memgpt?

pip install pymemgpt? pip install pymemgpt-nightly? git clone?: git clone

[x] Describe your setup

What's your OS (Windows/MacOS/Linux)?: MacOS
How are you running memgpt? (cmd.exe/Powershell/Anaconda Shell/Terminal): Terminal

I'm not able to find a class named AzureOpenAIEmbedding() Setup description:

memgpt version: 0.2.6
Installation method: git clone followed by pip install pymemgpt
Linux OS, running on bash inside VScode, with conda env

It should be in the llama_index.embeddings package and you will need to import it in the embeddings.py file. I have llama_index v0.9.10.
from llama_index.embeddings import OpenAIEmbedding, AzureOpenAIEmbedding

Could do a PR, but not sure if this is the right way to fix this problem or just a workaround.

I think raise a PR for this, I was playing with langchain earlier and pretty sure I had to use AzureOpenAIEmbedding instead of OpenAIEmbedding with azure_endpoint instead of api_base.
Thanks for posting this - it got memgpt working for me 👍

I think raise a PR for this, I was playing with langchain earlier and pretty sure I had to use AzureOpenAIEmbedding instead of OpenAIEmbedding with azure_endpoint instead of api_base.
Thanks for posting this - it got memgpt working for me 👍

No problem will do a PR.
By the way, next problem I ran into after fixing this one, was quota limit errors when creating embeddings for more than a few files. I had to update the embedding_retry_decorator in llama_index.embeddings.openai. This is probably more specific to the quota limits and number of files I am working with, so likely not an issue for most.


MRG: Fix add_channels behavior

Closes #5320.
@ckiefer0 can you see if this fixes your problem?
@agramfort in the long run we should get rid of info['buffer_size_sec'], it is really a property of the Raw instance and not stored in FIFFB_MEAS_INFO block anyway.

The problem of duplicate channels in raw.info['bads'] has been fixed, however, I still have duplicate entries in raw.info['dig']. Maybe it's only a problem with mne v0.15 since I am not sure which version of mne @agramfort used to replicate the issue?

thx a lot @larsoner
@ckiefer0  you can now use the current master branch
to install it easily:
pip install -U https://api.github.com/repos/mne-tools/mne-python/zipball/master


Problem with @ng-toolkit/_utils

I really don't know what's the issue. Probably @ng-toolkit/_utils it's not published in the proper way. Unfortunately, all the packages which depend on those, are unable to properly consume this package, which leads to:
Error: Cannot find module '@ng-toolkit/_utils/testing

The problem doesn't occur on my local environment (by running ./scripts/deploy_verdaccio.sh), it occurs only on the Travis CI.

What if you publish the @ng-toolkit/_utils only by using travis in order to see what's being published on npm?

I am not sure if I got your point. I checked also publication from the master branch (that goes directly to the npm, not to Verdaccio). From Travis it is not working (the npm test fails, because it can't load @ng-toolkit/_utils
I run the publish script from my machine - worked without any problem...

Looks like Travis is using some strange way of installing dependencies, or we have bad setup for node... No idea...


Docs: style clipboard button on :focus, not just :hover

current look (when focused in Chrome)

after this change

Please do not merge yet. I want @mdo to have a look.