Customizing document unclear or incorrect?

Hi!
With reference to the doc

Question about the document, where it specifies this file and in the kustomization file it is supposed to generate there's a different  filename.

Via the above method, it seems possible to add strategic merge, however, I'd like to configure something like command line arguments (e.g. --concurrent=6 etc), but that seems only possible with json patch:

    - op: add
      path: /spec/template/spec/containers/0/args/0
      value: --concurrent=6

Is there a way to add patchesJson6902 with the current customization implementation?

@kingdonb  Assign to me, please

In reference to (2), and I'm not exactly sure if this helps or if it doesn't, it is possible to use patches directive in Flux Kustomization to apply a json6902 style patch, as in this example:
https://github.com/kingdonb/bootstrap-repo/blob/86f38e8c2457c5024f02be21653525c53ca1e5ba/clusters/gke-geekingdon/flux-system/kustomization.yaml#L6
@cmharden is looking into the docs issue 👍 thanks for your report!

Thanks @kingdonb .
From my understanding, this isn't possible with the current implementation of the patch_names argument in the  flux_sync data source.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-sync.yaml
- gotk-components.yaml
patchesStrategicMerge:  # This and the following line are added by the `flux_sync` data source by `patch_names`
- patch-sops.yaml

While I could manage the kustomize.yaml it doesn't seem to be the intention of the customize-flux document and implementation, as far as I understand it.

The kustomize.yaml content will be generated by the flux_sync data source to include patch files, using a patchesStrategicMerge.

In my opinion what could be a solution is to add a new attribute, e.g. patches
locals {
  #
  # Rename the previous patches to:
  #
  strategic_merge_patches = {
    psp  = file("./psp-patch.yaml")
    sops = <<EOT
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kustomize-controller
  namespace: flux-system
  annotations:
    iam.gke.io/gcp-service-account: ${data.google_service_account.flux_sops.email}
EOT
  }

  #
  # This one is new:
  #
  patches = <<EOT
- target:
    group: apps
    version: v1
    kind: Deployment
    name: kustomize-controller
    namespace: flux-system
  patch: |-
    # this one is needed because it is adding an argument to an existing list of arguments
    - op: add
      path: /spec/template/spec/containers/0/args/0
      value: --concurrent=6
    # these are just examples, these could be configured via strategic merge
    - op: replace
      path: /spec/template/spec/containers/0/resources/limits/cpu
      value: "2"
    - op: replace
      path: /spec/template/spec/containers/0/resources/limits/memory
      value: "2Gi"
EOT
}

...etc...

data "flux_sync" "main" {
  target_path   = var.target_path
  url                  = var.clone_url
  patch_names = keys(local.strategic_merge_patches)   # renaming of the local.patches
  patches          = local.patches                    # add a new attribute for patches
}

The result would be:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-sync.yaml
- gotk-components.yaml
patchesStrategicMerge:
- patch-sops.yaml
- patch-psp.yaml
patches:
- target:
    group: apps
    version: v1
    kind: Deployment
    name: kustomize-controller
    namespace: flux-system
  patch: |-
  - op: add
    path: /spec/template/spec/containers/0/args/0
    value: --concurrent=6
  - op: replace
    path: /spec/template/spec/containers/0/resources/limits/cpu
    value: "2"
  - op: replace
    path: /spec/template/spec/containers/0/resources/limits/memory
    value: "2Gi"

I'm not sure if I was clear, the Flux kustomization has one patches directive, and you can use it to patch command line arguments into commands as if they were json6902 patches.
I don't know exactly how someone would invoke that directive in Terraform, but I've just gone back and re-read my example, and it does not do what I thought it did, so it's now clearer how you got the idea that I was suggesting something I didn't mean to. I think you're right that it isn't possible to manipulate the SIG-CLI kustomization.yaml during bootstrap. But you should be able to update the Flux kustomization for flux-system which has a similar directive you can use for patching.
I believe the Flux Kustomization's spec.patches can accept patches in all of the same forms as SIG-CLI's? I could be wrong about that, but I am fairly sure that it's at least as capable, though you might have to provide the patches in a slightly different form. Here's what I mean: https://fluxcd.io/docs/components/kustomize/kustomization/#patches
So if you can use Terraform provider to manipulate the Flux Kustomization spec with a patch, you can patch it to include this content (the additional concurrency argument) and get your command line arguments in a row. (At least I think that should work...)

Thanks @kingdonb . I think I see what you mean:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-sync.yaml
- gotk-components.yaml
patchesStrategicMerge:
#
# This would be a `kustomize.toolkit.fluxcd.io/v1beta2` with patches for any of the Deployments in `gotk-components.yaml`
- patch-gotk-components.yaml

I'd have to try it.
Just on top my head, I wonder if this would work, for 2 reasons:

If this is the first ever installation of gotk-components.yaml the CRD Kustomiztion from kustomize.toolkit.fluxcd.io/v1beta2 is still unknown in the cluster. Chicken egg problem?
Subsequent runs of Terraform on resource "kubectl_manifest" ... may see that the Deployments from gotk-components.yaml have been modified on the server and try to revert (because the Kustomization patch would have taken place on the server).

the CRD Kustomiztion from kustomize.toolkit.fluxcd.io/v1beta2 is still unknown in the cluster. Chicken egg problem?

Since Flux 0.18, the Kustomize Controller does reconciliation in a single step. I don't think this was ever a chicken-egg problem with prior Flux versions because it would naturally cause problems in a lot of scenarios for bootstrap users, but:
https://github.com/fluxcd/kustomize-controller/blob/8d61ff76d327ec76381f5f7fe3e9f0b722d75cf7/controllers/kustomization_controller.go#L682-L687
Kustomization now does a server-side apply in two stages, so that validation will succeed when a new CRD is being used. 👍
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
...
patchesStrategicMerge:
...

This is not actually what I was thinking, but if it works it's even better. I was suggesting a patch that modifies flux-system Kustomization so that it has a patches directive with the json6902-structured content in a patches.[]patch, that adds the flags to source-controller or any other controllers. I think we were trying to avoid using patchesStrategicMerge because it doesn't make it very easy to add a single field to an array at any arbitrary YAML address like the JSON patch.
The point is though, that Flux's patches directive is meant to be a utility of last resort. If you can use the Kustomize overlay instead, it is more flexible and you should do it. 👍

@kingdonb thanks again.
I went ahead with your suggestion to patch the flux-system kustomization.
It does apply the patches. However, and as I feared, each time I run Terraform, although I made no additional changes, I get the following:
Terraform will perform the following actions:

  # module.flux.kubectl_manifest.install["apps/v1/deployment/flux-system/helm-controller"] will be updated in-place
  ~ resource "kubectl_manifest" "install" {
        id                      = "/apis/apps/v1/namespaces/flux-system/deployments/helm-controller"
        name                    = "helm-controller"
      ~ yaml_incluster          = (sensitive value)
        # (12 unchanged attributes hidden)
    }

  # module.flux.kubectl_manifest.install["apps/v1/deployment/flux-system/kustomize-controller"] will be updated in-place
  ~ resource "kubectl_manifest" "install" {
        id                      = "/apis/apps/v1/namespaces/flux-system/deployments/kustomize-controller"
        name                    = "kustomize-controller"
      ~ yaml_incluster          = (sensitive value)
        # (12 unchanged attributes hidden)
    }

  # module.flux.kubectl_manifest.install["apps/v1/deployment/flux-system/source-controller"] will be updated in-place
  ~ resource "kubectl_manifest" "install" {
        id                      = "/apis/apps/v1/namespaces/flux-system/deployments/source-controller"
        name                    = "source-controller"
      ~ yaml_incluster          = (sensitive value)
        # (12 unchanged attributes hidden)
    }

So, basically it reverts it back to the original specs. The pods start a rolling upgrade. Then a few minutes later the patch on the flux-system kustomization is picked up and applied and thus another rolling upgrade.
This is the situation I thought would happen, hence why my original idea was to add patches on the kustomize.config.k8s.io/v1beta1 :
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-sync.yaml
- gotk-components.yaml
patchesStrategicMerge:
- patch-sops.yaml
- patch-psp.yaml
patches:
- target:
    group: apps
    version: v1
    kind: Deployment
    name: kustomize-controller
    namespace: flux-system
  patch: |-
  - op: add
    path: /spec/template/spec/containers/0/args/0
    value: --concurrent=6
  - op: replace
    path: /spec/template/spec/containers/0/resources/limits/cpu
    value: "2"
  - op: replace
    path: /spec/template/spec/containers/0/resources/limits/memory
    value: "2Gi"

Hi, @nuriel77

We came up with this "great thing,"™ I honestly don't really like it but it's a work-around for the current limitation, which I think I've understood from terraform provider is that users can only customize it through individual strategic merge patches:
https://github.com/kingdonb/bootstrap-repo/commit/3661cdf24071c500662c216940a2a4a634e7e826
So we wanted to append a single arg to an array of args in the container, so we really can't use a strategic merge patch for that, since there is no appending to arbitrary arrays, (of course as it is a strategic merge, you can append to the containers array, adding more containers, but that is no arbitrary array, and there is no such facility for appending args that I know of.)
If anyone knows a way to append to a list of args adding new elements directly into an existing array with a strategicMergePatch, please "lay it on me," but... instead! we wrote a JSON6902 patch to apply, since appending to arrays there is straightforward.
But due to the Terraform limitation, we wrote it as a JSON6902 patch, in the form of a Strategic Merge Patch, to be applied through the Flux Kustomization patches directive.
I'm afraid this is not exactly clear from the snip @cmharden posted above, but it's the same example, my link is just presented in a slightly different context, not in Terraform, but perhaps more clearly showing that the resource above is meant to be used as a Strategic Merge Patch. Hope this helped!

@kingdonb @cmharden first I'd like to thank you for your responses and taking the time to follow up on this thread.
@cmharden I have been using exactly that spec you've shared.
I think I have failed to explain the problem with this approach, and would like to try to explain it here again, and why I previously suggested that we do the JSON6902 patch in the kustomization.yaml rather than via a strategic merge patch.
The steps

Add the patch to terraform (exactly the one you've shared @cmharden although I've patched helm-controller but that doesn't matter)
Run terraform plan and apply - the files get applied in github, consequently the helm-controller deployment it patched and the pod is upgraded to use the new spec.

So far all good, however:
3. Now I run terraform plan again, the plan shows that it wants to update the deployment spec of helm-controller. This is not a desired state because it doesn't adhere to being idempotent.
4. I run terraform apply, the helm-controller deployment is reverted to its original state. Then a few seconds later the kustomization patch kicks in and applies it self. This results in two rolling upgrades.
Some output
We start with the original manifest deployed, this is taken from the running pod's spec (kubectl get po -l app=helm-controller -oyaml):
  spec:
    containers:
    - args:
      - --events-addr=http://notification-controller.flux-system.svc.cluster.local/
      - --watch-all-namespaces=true
      - --log-level=info
      - --log-encoding=json
      - --enable-leader-election

The first-ever plan with the patch @cmharden shared (albeit for helm-controller, not kustomization controller):
Terraform will perform the following actions:

  # module.flux.github_repository_file.kustomize will be updated in-place
  ~ resource "github_repository_file" "kustomize" {
      ~ content             = <<-EOT

            apiVersion: kustomize.config.k8s.io/v1beta1
            kind: Kustomization
            resources:
            - gotk-sync.yaml
            - gotk-components.yaml
          + patchesStrategicMerge:
          + - patch-main.yaml
        EOT
        id                  = "gitops/clusters/dev-beta/flux-system/kustomization.yaml"
      ~ overwrite_on_create = false -> true
        # (8 unchanged attributes hidden)
    }

  # module.flux.github_repository_file.patches["main"] will be created
  + resource "github_repository_file" "patches" {
      + branch              = "dev-beta"
      + commit_author       = (known after apply)
      + commit_email        = (known after apply)
      + commit_message      = (known after apply)
      + commit_sha          = (known after apply)
      + content             = <<-EOT
            ---
            apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
            kind: Kustomization
            metadata:
              name: flux-system
              namespace: flux-system
            spec:
              patches:
                - patch: |-
                    - op: add
                      path: /spec/template/spec/containers/0/args/0
                      value: --concurrent=6
                    - op: replace
                      path: /spec/template/spec/containers/0/resources/limits/cpu
                      value: "2"
                    - op: replace
                      path: /spec/template/spec/containers/0/resources/limits/memory
                      value: "2Gi"
                  target:
                    group: apps
                    version: v1
                    kind: Deployment
                    name: helm-controller
                    namespace: flux-system
        EOT
      + file                = "clusters/dev-beta/flux-system/patch-main.yaml"
      + id                  = (known after apply)
      + overwrite_on_create = false
      + repository          = "gitops"
      + sha                 = (known after apply)
    }

Plan: 1 to add, 1 to change, 0 to destroy.

We see the change being applied:
NAME                                       READY   STATUS              RESTARTS   AGE
helm-controller-5648bfdd55-bxrf7           0/1     ContainerCreating   0          2s
helm-controller-f76d67496-zt25t            1/1     Running             0          4d15h
kustomize-controller-7977b87f7b-5nhxf      1/1     Running             0          4d15h
notification-controller-656cdc97bc-6x7kj   1/1     Running             0          4d15h
source-controller-68d4f75576-844vk         1/1     Running             0          4d15h

Change is in the arguments on the new pod:
    - args:
      - --concurrent=6   ### \o/
      - --events-addr=http://notification-controller.flux-system.svc.cluster.local/
      - --watch-all-namespaces=true
      - --log-level=info
      - --log-encoding=json
      - --enable-leader-election

So far so good.
But, now if I run terraform plan again it wants to update the deployment spec for helm-controller:
Terraform will perform the following actions:

  # module.flux.kubectl_manifest.install["apps/v1/deployment/flux-system/helm-controller"] will be updated in-place
  ~ resource "kubectl_manifest" "install" {
        id                      = "/apis/apps/v1/namespaces/flux-system/deployments/helm-controller"
        name                    = "helm-controller"
      ~ yaml_incluster          = (sensitive value)
        # (12 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

This is because the module.flux.kubectl_manifest.install is trying to apply its contents, it has not idea about the kustomization patch of the flux-system kustomization.

We are experiencing the same issue as @nuriel77 , the provider creates kubectl_manifest resources, which are not applying the provided customization

Thank you, @nuriel77, for the thorough explanation that definitely helps.
@kingdonb It seems like a defect to me, the example uses patches as a variable name, implying that patches: will be used rather than assuming that all patches are a strategic merge.  I wasn't able to find a workaround to patch an unstructured array using the current design.
As @nuriel77   mentioned, an option for fixing the issue would be to add a patchJson6902 as an additional resource.
Alternatively, the patches field could be implemented as described here, rather than explicitly defining the patch type as patchStrategicMerge or patchJson6902
Imho, the former seems like an easier fix and the later more complex, but potentially a cleaner approach.

Hello and thank you for bringing this up. My use case is simply to enable the AWS ECR auto login as described here.
Since it is container args that need to be edited the patchStrategicMerge will not work.

Issue itself
I have achieved it by creating ./clusters/cluster1/kustomization.yaml file with content:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - flux-system
  - my_other_manifests
  - some_other_manifest.yaml
patches:
  # enable Native AWS ECR Auto-Login https://fluxcd.io/docs/guides/image-update/#using-native-aws-ecr-auto-login
  - patch: |-
      - op: add
        path: /spec/template/spec/containers/0/args/0
        value: --aws-autologin-for-ecr
    target:
      kind: Deployment
      name: "image-reflector-controller"

If you create such file in TF or create it manually it's up to you.
This won't fix TF reporting changes in plans, but at least it's much easier to manage, in my opinion.
To avoid the TF reporting changes I have used the ignore_fields parameter in the kubectl_manifest resource to ignore the changes done by Kustomize.
resource "kubectl_manifest" "install" {
  for_each   = { for v in local.install : lower(join("/", compact([v.data.apiVersion, v.data.kind, lookup(v.data.metadata, "namespace", ""), v.data.metadata.name]))) => v.content }
  depends_on = [kubernetes_namespace.flux_system]
  yaml_body  = each.value
  ignore_fields = ["spec.template.spec.containers.0.args"]
}

It's not an ideal solution, but it works for now.
Some thoughts
Adding anything to flux_sync resource won't matter, because the manifests come from flux_install resource. flux_install generates the YAML manifests without any changes/patches applied to them, and such original manifests are consumed by kubectl_manifest resource. The changes will be reported in TF plan until you provide patched manifests to kubectl_manifest so it won't try to update/revert it. I have never used the flux_sync's patch_names param, but it seems to me the entire idea is flawed. Specifying it in flux_sync will cause double roll-out, once by kubectl_manifest using original manifests, and couple seconds later by Flux itself using Kustomize no matter what I do.
In order to really provide a solution to customize Flux deployments the flux_install needs to provide ready-to-go manifests to kubectl_manifest. Have I missed something here?

@balonik Hi.
I am glad there is interest in this issue and that also others are facing the same challenges. I hope it will help to expedite a solution.

It's not an ideal solution, but it works for now.

Thanks for sharing.

Adding anything to flux_sync resource won't matter, because the manifests come from flux_install resource.

It currently works (the patching), I am not sure what do you mean that it doesn't matter?
Only problem is that it results in the rolling upgrade as mentioned in this issue.
From what I understand the original implementation of the customization in this module expects the patch_names attribute to be part of flux_sync. That is because flux_sync is the one that manages the target file kustomization.yaml
By default it looks like this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-sync.yaml
- gotk-components.yaml

The patching has to be done in this file, not in gotk-components.yam itself (which, as you said, is managed by flux_install).
If you check terraform state and search for the following and for kustomization.yaml I believe it will make more sense:
   {
      "module": "module.flux",
      "mode": "data",
      "type": "flux_sync",
      "name": "main",
      "provider": "provider[\"registry.terraform.io/fluxcd/flux\"]",
      "instances": [
      ...

My problem with the approach of setting it in flux_sync. There are major differences between the Flux CLI and TF provider.
Flux CLI:

it is not expected that you run Flux bootstrap via Flux CLI every day
Flux CLI detects existing version and recognizes an upgrade, changing only flux-system/gotk-components.yaml in GIT
no changes to other base files and no apply to cluster during upgrade. It lets Flux to update itself

TF provider:

TF is often part of CI/CD pipelines that can run every day/hour/minute
following TF provider guides we don't detect existing setup and do a full install/bootstrap every apply
every base file is regenerated and pushed to GIT every apply
Flux manifests are applied to cluster via kubectl every apply

These are the major reasons why I think following the Flux recommendations for customization cannot be applied in this TF provider.
I will explain using simple example.
There are basically 2 main stages, the install and sync. In the install stage we generate Flux manifests using flux_install and apply them to k8s cluster using kubectl_manifest. When flux_install generates manifests it defines the state of the Flux deployment to be green. And as such (green) is deployed to the cluster.
In the following sync stage we apply customizations and indirectly define that the deployment should be red. Flux will pickup this definition and apply it to the cluster. Now the state in the cluster is red.
With this approach we have never ending changes, because we define state of a single resource in 2 places. Each stage defines it's own desired state. That's just wrong.
In ideal situation, the manifests applied to the cluster should already be red, so Flux won't do any changes on reconciliation.

@balonik I see what you mean

@balonik is there no way in Terraform to have a single kubectl_manifest that would contain all three:

sync
install
patches (optionally if required)

At the moment my code looks like this:
resource "kubectl_manifest" "install" {
  for_each   = { for v in local.install : lower(join("/", compact([v.data.apiVersion, v.data.kind, lookup(v.data.metadata, "namespace", ""), v.data.metadata.name]))) => v.content }
  depends_on = [kubernetes_namespace.flux_system]
  yaml_body  = each.value
}

resource "kubectl_manifest" "sync" {
  for_each   = { for v in local.sync : lower(join("/", compact([v.data.apiVersion, v.data.kind, lookup(v.data.metadata, "namespace", ""), v.data.metadata.name]))) => v.content }
  depends_on = [kubernetes_namespace.flux_system]
  yaml_body  = each.value
}

What I am getting at is if it can be done via Terraform that the entire manifest will be applied to the cluster as one and not in 2  "kubectl_manifest" steps?

Fwiw, I believe it won't even matter if there was a single stage of kubectl_manifest install. Ideally, a kustomize build would have to be done in terraform and then kubectl_manifest to apply the result. That way terraform will not see a diff in subsequent runs.
I had a quick look at kustomize provider but couldn't understand how to get that into the mix.
If anybody agrees with the direction of performing the kustomize build in terraform and then applying the resultant yaml to the cluster, and happen to think of a solution how to get it done, please do share.

I have found that provider as well and even raised FR to support our use-case https://github.com/kbst/terraform-provider-kustomization/issues/163, where I got pointed to another issue from 2021 in which they declined the idea.
But it should be still doable with local_file, where you can save the generated manifests from flux_install and flux_sync as files during plan/apply execution, have a kustomization.yaml as file in repository and use them with kustomize_build to generate final manifests for kubectl_manifest.
I am not a fan of this approach, because I ran TF in ephemeral k8s pods, so any generated local files would be as change in each TF plan/apply. But if you are running it from (as example) workstation it could work.

Hi @username 👋
I hope you are doing well! As part of our ongoing effort to maintain and improve the quality of our project, I've been reviewing open issues and came across the one you've reported. First off, thank you for taking the time to contribute by reporting this issue; your input is crucial to us.
Upon reviewing the details of your issue, I noticed that it involves the use of a resource or feature that has yet to be supported since the 1.0.0 release of our project, which was approximately 9 months ago. This might be a key factor in the challenges you're experiencing. We understand that changes and deprecations can impact your work, and we're here to help navigate these transitions. If there are specific reasons you've continued using this unsupported resource or if there's any way we can assist in migrating to a supported alternative, please let us know.
Additionally, to ensure the efficient management of our issue tracker and to focus on issues that are actively affecting our community, we have implemented a policy for issues that remain inactive. If there is no activity on this issue within the next 3 weeks, we will consider the issue inactive and close it for you. This doesn't mean your issue is not important to us, but rather that we aim to keep our focus on actively pursued concerns. Of course, if the issue continues or if you have further updates in the future, feel free to reopen the issue or create a new one.
Thank you once again for your contribution to our project. Your feedback not only helps us improve but also supports the broader community in overcoming similar challenges. We look forward to hearing from you and hope to resolve any outstanding concerns together.
Best regards,
Steve

To customize the flux_bootstrap_git resource please take a look at the following example:

https://github.com/fluxcd/terraform-provider-flux/tree/main/examples/github-with-customizations
https://github.com/fluxcd/terraform-provider-flux/tree/main/examples/github-with-inline-customizations


Works with lua 5.4 but incorrectly shows the Lua version number on startup

When I run croissant it displays Lua 5.3, but when I ask Lua what version is actually running, it says Lua 5.4:
$ sudo luarocks --lua-version 5.4 install croissant
...
$ croissant
🥐  Croissant 0.0.1 (C) 2019 Benoit Giannangeli
Lua 5.3 Copyright (C) 1994-2018 Lua.org, PUC-Rio
→ _VERSION
"Lua 5.4"
→

Yes, croissant uses a trick to find out the flavor of lua being run. It has not been updated for Lua 5.4 yet.

Made a pull request for this:
https://github.com/giann/croissant/pull/13

Why does croissant use these tricks to determine version? Wouldn't it be better to simply check the _VERSION global?

Probably because _VERSION is just a (global) variable which is easily overwritten?
🚀 ❯ croissant
🥐  Croissant 0.0.1 (C) 2019 Benoit Giannangeli
Lua 5.3 Copyright (C) 1994-2018 Lua.org, PUC-Rio
→ _VERSION
"Lua 5.4"
→ _VERSION = 'FakeLua 5'
"FakeLua 5"
→ _VERSION
"FakeLua 5"

see http://lua-users.org/lists/lua-l/2016-05/msg00297.html for the original code + rationale

True, but if someone overwrites _VERSION then he's probably testing things. If you don't use _VERSION then he can't do the test he wants.

I don't know what else luaVersion() is being used for other than to report the true Lua version which croissant is running and displaying this value when it starts up.
What I do know is: this hacky-looking code tests for subtleties unique to each Lua version, then uses this to provide a definite Lua version vs having to rely on a variable which might get overwritten anytime.


New package: ChemistryFeaturization v0.1.0

Registering package: ChemistryFeaturization
Repository: https://github.com/aced-differentiate/ChemistryFeaturization.jl
Created by: @rkurchin
Version: v0.1.0
Commit: fc6579c625629182045eb8169ccd1a700424f0b1
Reviewed by: @rkurchin
Reference: https://github.com/aced-differentiate/ChemistryFeaturization.jl/commit/fc6579c625629182045eb8169ccd1a700424f0b1#commitcomment-45009497

This issue is because I'm currently depending on the master branch of GraphPlot.jl. I've opened an issue over there asking for a new release which should resolve the errors here...


Implemented  Error Popups on Chat Preview

introduces an error pop-up notification  whenever a request is made  and encounters any backend errors.
Changes Made:

Implemented an error pop-up for error
Ensured that the notification is displayed clearly to enhance user experience and debugging.

@sethumadhavan-k, attach a loom video


DETRs with Hybrid Matching

In paper, "query denoising and "Hybrid matching" are kept to accelerate convergence and improve performance". Does Hybrid matching refer to the H-DETR paper: https://arxiv.org/pdf/2207.13080.pdf

I realize that it refer to the hybrid matching in Mask-DINO 😅.


fix: fixed ga tracking only in production and track when navigate bet…

fixes #237

Pull Request Test Coverage Report for Build 1150

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.7%) to 73.209%

Totals

Change from base Build 1122:
0.7%

Covered Lines:
580

Relevant Lines:
759

💛  - Coveralls


Running in Pure Javascript Enviornment

Hi,
I am able to sucessfully run lovefield in browser.
When I tried to run lovefield in pure Javascript Environment with in-memory store type, with  lf.schema.DataStoreType.MEMORY
single line of code, written below,  generated error.
var lf = require('lovefield');
Error::
node_modules/lovefield/dist/lovefield.min.js:1 (function (exports, require, module, __filename, __dirname) { if(!self.window){window=self;}                                                                   ^ ReferenceError: self is not defined

Did some tweaking in lovefield/dist/lovefield.js
added this at the beginning of file
var window = {   navigator:{} } var self = {};
Added this after line 3
goog.global = this;   //this is line 3
goog.global.setTimeout = setTimeout;  //I added this

Now it works fine on Node environment with DataStoreType.MEMORY.
Can anyone check the source file and update them, making  sure that this doesn't break anything, and generate the output files that fixes the issue.

There is already a script for running Lovefield in Node, see https://github.com/google/lovefield/blob/master/tools/node_bootstrap.js.

Close due to lack of activity.


Forge does not create working projects under Linux

When working in Linux I get lot's of "Input string was not in a correct format" when working with forge.
You can reproduce with:
forge new project -t console -n ForgeTest --folder ForgeTest

this outputs on my machine:
Forge should be run from solution/repository root. Please ensure you don't run it from folder containing other solutions

Do You want to continue? [Y/n]

Generating project...
Creating /coding/ForgeTest/ForgeTest
/coding/Languages/Fsharp/Forge/temp/bin/../templates/console/App.config => /coding/ForgeTest/ForgeTest/App.config
/coding/Languages/Fsharp/Forge/temp/bin/../templates/console/ApplicationName.fs => /coding/ForgeTest/ForgeTest/ApplicationName.fs
/coding/Languages/Fsharp/Forge/temp/bin/../templates/console/ApplicationName.fsproj => /coding/ForgeTest/ForgeTest/ApplicationName.fsproj
/coding/Languages/Fsharp/Forge/temp/bin/../templates/console/paket.references => /coding/ForgeTest/ForgeTest/paket.references
mono  "/coding/.paket/paket.exe" init
Paket version 4.8.7
Saving file /coding/paket.dependencies
/coding/.paket/paket.targets
Downloading file from https://github.com/fsprojects/Paket/releases/download/4.8.7/paket.targets to /coding/.paket/paket.targets
4 seconds - ready.
mono  "/coding/.paket/paket.exe" add nuget FSharp.Core --no-install
Ignoring the following unknown argument(s): nuget, FSharp.Core, --no-install
Checking Paket version (version add requested)...
Input string was not in a correct format. (Github - cached)
mono  "/coding/.paket/paket.exe" install
Checking Paket version (version install requested)...
Input string was not in a correct format. (Github - cached)
mono  "/coding/.paket/paket.exe" add nuget FAKE
Ignoring the following unknown argument(s): nuget, FAKE
Checking Paket version (version add requested)...
Input string was not in a correct format. (Github - cached)
Done!

Expected Behavior
get a working console project
Current Behavior
get the same kind of errors when doing a ./build.sh
Your Environment
Mono <IP_ADDRESS>
F# 4.1
dotnet-core:
Product Information:
Version:            1.0.1
Commit SHA-1 hash:  005db40cd1
Runtime Environment:
OS Name:     linuxmint
OS Version:  18.1
OS Platform: Linux
RID:         ubuntu.16.04-x64
Base Path:   /usr/share/dotnet/sdk/1.0.1

maybe this is an deeper issue as I cannot rebuild Forge anymore on my system as well - there I get a MethodNotSupported Exception with FAKE:

after some discussion on the FAKE issue and a manual paket update the errors are now as listed in the attached file:
error.txt


Add initial github settings.yml file

Signed-off-by: Ryan Beck-Buysse<EMAIL_ADDRESS>

Documentation for settings files: https://github.com/apps/settings


fixed integer value auto completer

Please ignore at the moment. Creating this pr to run the ci test.

This clearly fixes a bug; the new test case doesn't work for Postgres (and never had), because we use the ILIKE operator on a numeric type.

make sense :+1:

I like this solution, It's not beautiful, but better than dropping auto-completing altogether. If you update the spec description, we can go ahead and merge IMHO.

@wvanbergen fixed the test description.

@wvanbergen  @abenari thanks!

Thanks guys, any chance to see a minor update with this one? If not we are going to backport, just let me know. Thanks!

I was hoping to get #122 in for the next release, but it doesn't look like I will have time any time soon to continue to work on that. SO I will prepare a release now.

That's fantastic, saved me time. You're hero! THanks.

Version 3.2.0 has been released!

Is my understanding of this patch correct: it adds the CAST for all columns and all databases, therefore resulting queries always look like:
WHERE (CAST column AS CHAR(50) LIKE 'something%')

Therefore indicies cannot be used (for prefixed-LIKE operator)?

@lzap yep, all queries for value auto completion, are casting.
You can improve that by using cast only if field.numerical?
Will be happy to get a patch for that :)


Upgrade to Doorkeeper 5.5.0.

5.5.0.rc2 doesn't seem to have any problems, let's see if 5.5.0 still does? It should still break the events page (or at least cause database constraint violation errors), but maybe I'm just crazy.

Yup, 5.5.0 is busted.


⚠ Adacta-fintech.com has degraded performance

In 59d5e49, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 2027 ms

Resolved: Adacta-fintech.com performance has improved in cc80949.


Put all component voltage sources behind MOSFETs

Debugging some voltage stuff is causing issues when they are all eating power at boot. Being able to programmatically turn on and off their power is going to prove useful.

For anyone coming across this issue in the future, this was a major mistake and the sole reason why there will be a revision 7.


Crash when cancel Group by

Description
app crshed when cancel group by in this view

Steps To Reproduce

open any folder
set Group by
click any group titile and go into the view like this

cancle Goup by
app crashed

Requirements
The application run normally
Files Version
<IP_ADDRESS>
Windows Version
10.0.26120.770
Log File
debug.log

Thanks for the report, can you see if this is an issue in the preview version. https://files.community/download

Thanks for the report, can you see if this is an issue in the preview version. https://files.community/download

Yes,app still crash
debug.log

Thanks for checking

This will be fixed in the next release


Start storing data and serving views in response to streaming events instead of cumulatively at the end

larger builds can take a long time to serialize to the database.  Can we alleviate this by batching writes as they come in instead of waiting until the very end?  some data isn't supplied until the build is complete, but we could show targets as their accumulated and build log output, etc.

a little additional discussion suggests we want to look at a message queue to help us achieve this.  maybe roll in something like kafka to be deployed alongside bbportal


add loot spawning after breaking under chest

also fix ItemsToDropoOffset -> ItemsToDropOffset

sorry for the include spam I was tired and copied everything from the file I needed

this so scufed bro ill merge it maybe im thinking about it

this so scufed bro ill merge it maybe im thinking about it

I know bro but that's how I think unfortunately. Feel free to make a followup push to remove the unnecessary stuff lol

I think the actual func is abuildingcontainer::bp_spawnloot


tsc fails on fresh unstable-vite install due to remix & vite typescript conflicts

Reproduction
Compiling a fresh install of unstable-vite remix as follows:
npx create-remix@latest --template remix-run/remix/templates/unstable-vite vitetest
cd vitetest
# Using Typescript (tsc) 5.3.2
npx tsc

results in 49 typescript errors due to module definition conflicts between remix and vite:
Found 49 errors in 4 files.

Errors  Files
    23  node_modules/@remix-run/dev/dist/modules.d.ts:3
     2  node_modules/@remix-run/react/dist/browser.d.ts:30
    23  node_modules/vite/client.d.ts:8
     1  node_modules/vite/types/importMeta.d.ts:17

System Info
NOTE: Tested with Typescript 5.3.2

System:
    OS: macOS 14.2.1
    CPU: (8) arm64 Apple M2
    Memory: 4.35 GB / 24.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.3.1 - ~/.nvm/versions/node/v20.3.1/bin/node
    Yarn: 1.22.21 - ~/.nvm/versions/node/v20.3.1/bin/yarn
    npm: 9.6.7 - ~/.nvm/versions/node/v20.3.1/bin/npm
  Browsers:
    Chrome: 120.0.6099.129
    Safari: 17.2.1
  npmPackages:
    @remix-run/dev: ^2.4.1 => 2.4.1
    @remix-run/node: ^2.4.1 => 2.4.1
    @remix-run/react: ^2.4.1 => 2.4.1
    @remix-run/serve: ^2.4.1 => 2.4.1
    vite: ^5.0.0 => 5.0.11

Used Package Manager
npm
Expected Behavior
Compiling a fresh install of unstable-vite with npx tsc should complete successfully.
npx create-remix@latest --template remix-run/remix/templates/unstable-vite vitetest
cd vitetest
# NOTE: Using Typescript (tsc) 5.3.2
npx tsc

Actual Behavior
Found 49 errors in 4 files.

Errors  Files
    23  node_modules/@remix-run/dev/dist/modules.d.ts:3
     2  node_modules/@remix-run/react/dist/browser.d.ts:30
    23  node_modules/vite/client.d.ts:8
     1  node_modules/vite/types/importMeta.d.ts:17

There are basically just 2 errors: 1 instance of the following type declaration mismatch:
node_modules/@remix-run/react/dist/browser.d.ts:30:9 - error TS2687: All declarations of 'hot' must have identical modifiers.
node_modules/@remix-run/react/dist/browser.d.ts:30:9 - error TS2717: Subsequent property declarations must have the same type.  Property 'hot' must be of type 'ViteHotContext | undefined', but here has type 'any'.

30         hot: any;
           ~~~

and 23 instances of duplicate module definitions:
node_modules/@remix-run/dev/dist/modules.d.ts:3:20 - error TS2300: Duplicate identifier 'asset'.

3     export default asset;
                     ~~~~~

  node_modules/vite/client.d.ts:129:18
    129   export default src
                         ~~~

Thanks for raising this issue!
Looks like this isn't fixable without a breaking change to @remix-run/dev, forcing all consumers of the existing esbuild-based compiler to update their TypeScript configuration. Instead, as a short term fix for Vite consumers, I'm enabling the skipLibCheck flag in the Vite templates in this PR: #8447.
The type clash with @remix-run/react should be fixable by removing the global type augmentation since it's not part of the public API so I might do this as a follow-up PR.


Fixing style exports

Description
This fixes a small bug that had the stylesheets be exported as SCSS instead of the usual CSS, which leaned a bit more on the consumer being able to load sass files v. just being able to load stylesheets in general.
This restores the import behaviour seen in the README and addresses what might have been seen in #325 .
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

:tada: This PR is included in version 8.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Implied repos should replace property placeholders

I have come across a remote repository created by implied-repos add-on with this configuration
{
  "type" : "remote",
  "key" : "remote:i-oracle-repository",
  "description" : "Implicitly created repo for: Oracle Repository (oracle.repository) from repository declaration in POM: org.apache.openjpa:openjpa-persistence-jdbc:2.4.0",
  "metadata" : {
    "implied_by_stores" : "{\n  \"items\" : [ \"remote:central\" ]\n}",
    "changelog" : "Disabling indefinitely via UI"
  },
  "disabled" : true,
  "port" : 0,
  "doctype" : "remote",
  "name" : "i-oracle-repository",
  "path_style" : "plain",
  "allow_snapshots" : false,
  "allow_releases" : true,
  "url" : "${oracle.maven.repo}",
  "timeout_seconds" : 5,
  "nfc_timeout_seconds" : 0,
  "is_passthrough" : false,
  "cache_timeout_seconds" : 0,
  "proxy_port" : 0
}

Obviously the placeholder ${oracle.maven.repo} was not replaced by its value which is specified in the same profile.

@pkocandr
Hi, could you help to provide the input.pom example for me? I try to do a unit-test for impiled repos expression replace (for main reporitories), but it passed, seem wrong place I've located.

This particular pom.xml is available at http://central.maven.org/maven2/org/apache/openjpa/openjpa-persistence-jdbc/2.4.0/openjpa-persistence-jdbc-2.4.0.pom and the profile id is "test-oracle".

The problem is probably not related to implied repos and is maybe a problem in galley that it does not replace placeholders for repository definitions in a profile, but it should of course work for implied repos too.

I believe the MavenPomReader should already be doing property replacement for these things. I wonder if the property itself was originally specified externally, like in the settings.xml or on the command line? FWIW I found this comment in the POM:
Example oracle profile. You can use this profile if you:
                1) have the Oracle artifacts installed in a local repo and
                supply the URL:
	            -Doracle.maven.repo=http://my.local.repo
                2) have a copy of the Oracle driver and run the following
                command:
                    mvn install:install-file -Dfile=${path to ojdbc.jar} \
                                             -DgroupId=com.oracle \
                                             -DartifactId=jdbc-driver \
                                             -Dversion=10g \
                                             -Dpackaging=jar

                You must also set the following properties:
                    -Dopenjpa.oracle.url
                    -Dopenjpa.oracle.username
                    -Dopenjpa.oracle.password

                Optionally, you can override the default Oracle groupId and
                version by also supplying the following properties:
                    -Doracle.groupid=com.oracle
                    -Doracle.version=10g

If we can't resolve the property, I think the other thing we can really do is to test that it's a valid URL. We could start by just constructing a java.net.URL instance with it and see if it throws a malformed url exception. Beyond that, we could use a simplified form of the approach autoprox uses to validate repository URLs:
https://github.com/Commonjava/indy/blob/master/addons/autoprox/common/src/main/java/org/commonjava/indy/autoprox/data/AutoProxDataManagerDecorator.java#L167

Yeah, the validation would help in cases when the property value is not available, but in this case there is a value specified in the properties of the profile
<oracle.maven.repo>http://not.a.real.repository</oracle.maven.repo>

So here the validation would pass while the repo url is still not any real url. But the problem is mainly the placeholoder replacement that does not happen in this case. It seems to be related to an issue we encountered before https://github.com/Commonjava/galley/pull/19

Strange...that looks like it should work. I think we need to add this as an issue to galley-maven, and use that POM as a basis for a test case.
In any case, you're right that the value above would pass URL construction. That's what I was mentioning before about using something more like autoprox validation to actually perform a HEAD request to ensure the server is reachable.


semantic-rs fails if git user.name and user.email aren't configured globally

Given the case there's no user.name and user.email configured globally just for the current repository.
Then semantic-rs fails with the following error:
vagrant@vagrant-ubuntu-trusty-64:~/avm$ semantic-rs -w
semantic.rs 🚀
Analyzing your repository
Current version: 0.6.0
Analyzing commits
Commits analyzed. Bump will be Minor
New version: 0.7.0
Writing Changelog
Committing `Cargo.toml` and `Changelog.md` failed: "Config value \'user.name\' was not found"

I got some code, it's just not pretty, but will work

#57


fpga: Install content in /usr/local for the nlb-demo

OPAE-NLB-demo: install content in /usr/local in order to get clean scan results from BDBA
Addresses https://github.com/intel/intel-device-plugins-for-kubernetes/issues/227
Signed-off-by: Hector Augusto Garcia Baleon<EMAIL_ADDRESS>

Codecov Report

Merging #239 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #239   +/-   ##
=======================================
  Coverage   70.64%   70.64%
=======================================
  Files          16       16
  Lines        1659     1659
=======================================
  Hits         1172     1172
  Misses        450      450
  Partials       37       37

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf2bc7f...9c70701. Read the comment docs.

@bart0sh @rojkov ready to merge?


Custom Colors as Scenes button?

Thanks for bringing this plugin to homebridge. It’s been working great and I am using it with a Glide right now as a potential replacement to the buggy Lifx Beam.
I am wondering if it’s possible to “call” a color mode with a particular color pattern that’s been saved under ‘my color’?
I am able to set DIY scenes as per the guide, and the ‘Color Mode’ also appear in the Eve app, but the DIY scenes are dynamic . The colour mode works but only sets the last used color.
Am I missing something or is that not available at the moment?

Hi @WBWBW
The purpose of the “Colour Mode” button is simply to revert back from any set scene/music mode/diy back to the previous colour.
You can create HomeKit scenes to select any colour you’d like.
When you mention “colour pattern”, what do you mean by this? different colours along the same strip? Is this not considered a DIY?

Yes I meant different colors at the same time. I’ll look again at the DIY but it seems those scenes are all “dynamic” and have some animation in the scene setting. Whereas the one under “Colour” are fixed without animation.

Could you do me a favour, put the plugin into debug mode from the plugin settings, and then use the govee app to turn the strip into some “segment” colour (make sure strip is connected via wifi rather than bluetooth if it supports it).
The logs should show an incoming message when you change to such a segmented colour.
Can you post the logs here that appear?
Thanks

Turns out the AWS code does appear. I missed it first because it doesn’t appear when just changing to “colour” but once you’ve saved some pattern and select one of those pattern it does show up in the logs.

If you install the beta version of the plugin, I have added more fields to add a code to, the are called “Segmented”… “Segmented 2”… etc


[e2e]: go tag main config-ldflags-noassets slsa3

Repo: https://github.com/slsa-framework/example-package/tree/v14.0.24
Run: https://github.com/slsa-framework/example-package/actions/runs/7706996942
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Trigger: push
Branch: v14.0.24
Date: Tue Jan 30 06:10:03 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/v14.0.25
Run: https://github.com/slsa-framework/example-package/actions/runs/7721910828
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Trigger: push
Branch: v14.0.25
Date: Wed Jan 31 06:10:12 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/v14.0.26
Run: https://github.com/slsa-framework/example-package/actions/runs/7736997165
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-noassets.slsa3.yml
Trigger: push
Branch: v14.0.26
Date: Thu Feb  1 06:13:32 UTC 2024
Tests are passing now. Closing this issue.


Cache MD5 sum calculations

Having every query recalculate the MD5 sum of each build is time consuming and wasteful.  Cache the results and use the cache on future queries.

Hi,
thank you very much for the MR. I approve the overall idea, but I don't really approve the way it's done. We already have implemented Memcached, therefore I'll focus on using that layer to boost the speed.
Also, since the cache is not mandatory ( the software works, but it's not just fast ), the die() logic wouldn't fit at all.
I'll work on a smarter patch ( which would involve few lines of code ) that does this :)
Best regards.

After a quick inspection I found that I am already doing a big cache of the whole blob, therefore another layer of cache only for MD5 would be too much :)
See https://github.com/julianxhokaxhiu/LineageOTA/blob/2.0/src/Helpers/Builds.php#L134
Just remember to enable memcached support.

Well, then, something is not working the way it is supposed to.  With my patch a fetch from /api/v1/d852/unofficial/b8ce487c84 is instant.  Without it it takes 16 seconds as it has to read each image from disk to md5sum it.
I have memcached installed and running.

Is the configuration set properly?

I believe so.  Is there anything one needs to do other than to have memcached running on the port that CmOta()'s memcached.port expects?
In any case, this is what my configuration looks like:
    $app = new CmOta();
    $app
    ->setConfig( 'basePath', 'http://'.$_SERVER['HTTP_HOST'].dirname($_SERVER['SCRIPT_NAME']) )
    ->setConfig( 'memcached.host', 'localhost' )
    ->setConfig( 'memcached.port', 11211 )
    ->enableMemcached()
    ->run();

and memcached is running:
memcach+ 26711     1  0 Feb04 ?        00:04:34 /usr/bin/memcached -p 11211 -u memcached -m 64 -c 1024 -l <IP_ADDRESS>,::1

Well, your merge request made me rethink about memcached support, and now I dropped it in favor of APCu which just works if the extension is installed and enabled.
Therefore can you please retry now? Docker build with APCu support is there too. So feel free to try it out :)


Fix typos discovered by codespell

python3 -m pip install codespell ; codespell -w

[x] Chart version not bumped (the versions are all bumped and released at the same time)
[x] README.md updated with any new values or changes
[ ] Updated template tests in ${CHART}/tests/*.py
[ ] Updated integration tests in ${CHART}/examples/*/test/goss.yaml

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?


History operations

Be able to:

[x] Browse history of a repo
[ ] Revert the repo to some previous time point

Currently I don't need and can't easily test these operations, but they will probably eventually be wanted.

It's possible to load all history and iterate through all states (I think unordered). Not sure at the moment what other kind of API is wanted.


Unable to deploy k8s objects

I get the below error (reported in the Execution UI page) when trying to instantiate an assembly
No lifecycleType mapping for transition Install infrastructure type Kubernetes
I on-boarded the driver with the below command parameters:
lmctl lifecycledriver add --type Kubernetes --url http://kubedriver:8294 dev

I can access driver's swagger UI via its NodePort.
I attache below my resource files
Definition/lm/resource.yaml
description: descriptor for k8s1
lifecycle:
  Install:
    drivers:
      Kubernetes:
        infrastructure_type:
          - '*'
default-driver:
  Kubernetes:
    infrastructure_type:
      - '*'

Lifecycle/Kubernetes/kegd.yaml
cleanupOn: Uninstall
compose:
  - name: Install
    deploy:
      - objects:
          file: obj.yaml

Lifecycle/Kubernetes/objects/obj.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: example-a
data:
  dataA: valueA
  dataB: valueB
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: example-b
data:
  dataA: valueA
  dataB: valueB
  dataC: valueC

Thanks.

Hi,
Can you please point me to an end to end example that starts from defining Definition/lm/resource.yaml up to Lifecycle/Kubernetes/kegd.yaml ?
Thanks.

Hi,
What version of LM are you using? As the kubedriver cannot be onboarded as a lifecycledriver - it must use the new type in LM 2.2+, named resourcedriver.
Essentially, this means the Kubernetes driver only supports LM 2.2+
As for docs, this is the best we have right now:
https://github.com/accanto-systems/kubernetes-driver/blob/master/docs/user-guide/building-a-resource.md

Also, once you have LM 2.2+, you should install lmctl 2.5.0 as this version can generated a starter Kubernetes project for you: lmctl project create --type Resource --param driver kubernetes

Hi,
I see. Thanks. My LM is 2.1.0. From where can I obtain 2.2+? I am using AIO. I currently have: lm-docker-source-2.1.0 and lm-helm-charts-2.1.0 packages.
Thanks.


test_object_copy_to_itself should succeed

AWS-S3 allows copying an object to itself, even without a change in
metadata.

Actually, Amazon specifies rejecting self-copy without replace:

You cannot copy an object to itself unless the MetadataDirective header is specified and its value set to REPLACE.

As above, testing reveals a different behavior.  Should we align the test with the specification or the implementation?

@andrewgaul @alfredodeza is rgw accepts, and aws accepts? We can test both behaviors, mark 'fails_on_rgw', and 'fails_on_aws' where appropriate. Not sure I like this solution.


Fix assertion when creating a removable media drive with an empty filename

Must have been a very old bug. Fixed:
>piscsi -id 0 -t SCRM ""
+----+-----+------+-------------------------------------
| ID | LUN | TYPE | IMAGE FILE
+----+-----+------+-------------------------------------
|  0 |   0 | SCRM | NO MEDIUM
+----+-----+------+-------------------------------------

@rdmark No, amything that has to do with the command line is hard or impossible to unit test, but https://github.com/PiSCSI/piscsi/issues/1295 might help with that.


DataManagementApi: change all component tests to use rest-assured

Feature Request
All controller component tests for REST API controllers should use rest-assured instead of spinning up Jetty/Jersey in the test setup.
Which Areas Would Be Affected?
DataManagementApi tests
Why Is the Feature Desired?
to achieve consistency and improve readability of test assertions.
Solution Proposal
use rest-assured. some tests already do this.
Type of Issue
improvement
Checklist

[x] assigned appropriate label?
[ ] Do NOT select a milestone or an assignee!

it appears all DataManagementApi integration tests already use rest-assured. this can be closed.


Adding user_impersonation for consumed services

When a service is consumed by another service, we need to ensure that the scope is available for user impersonation.
Updating the microservice code so that if a service has http consumers then a user impersonation scope is set to enable this.

Service to service might not always be on behalf of and need these permissions. I'm going to spend a little more time thinking about this one.
Have you tried running this multiple times to see how it reacts with the application? I believe when I did this before the app and the permission were competing with each other. (If this is happening it can be solved with an ignore flag on the app, but we might need bigger changes if using the ignore)

@Matthewsre, good points about the on behalf of not being required.  One thought is to update the logic to only add the user_impersonation scope when require_auth = true and the service has at least 1 consumer?  That would be more specific

Abandoning PR; new one incoming with less noise


The User object is null in Edge Function runtime with a valid authorization token

Bug report
The user object is null in Edge Function runtime with a valid authorization token.
Describe the bug
I'm trying to get the User object with a valid Authorization token to perform an insert on a specific table that contains the user id; the user object is always at runtime, even with a valid authorization token.
To Reproduce

Create an Edge function and deploy it

// Get the authorization header from the request.
const authHeader = req.headers.get('Authorization').replace("Bearer ","")
// Client now respects auth policies for this user
supabaseClient.auth.setAuth(authHeader)
// set the user profile
const user = supabase.auth.user()

console.log(user);
>> null

Expected behavior
Returns the current Auth user with UUID.

Getting the same result from copy pasting sample code from the RESTful service.
Function call runs, but the user object is null and I can't access the database with my supabase client (assuming RLS blocks queries because the user is not authenticated).
myfunction/index.ts

import { serve } from<EMAIL_ADDRESS>import { createClient } from "https://esm.sh/<EMAIL_ADDRESS>
serve(async (req) => {
  const { url, method } = req;

  try {
    const supabaseClient = createClient(
      // Supabase API URL - env var exported by default.
      Deno.env.get("SUPABASE_URL") ?? "",
      // Supabase API ANON KEY - env var exported by default.
      Deno.env.get("SUPABASE_ANON_KEY") ?? "",
      // Create client with Auth context of the user that called the function.
      // This way your row-level-security (RLS) policies are applied.
      {
        global: {
          headers: { Authorization: req.headers.get("Authorization")! },
        },
      }
    );

    const {
      data: { user },
    } = await supabaseClient.auth.getUser();

    // Null here
    console.log("USER : ", user);

    return new Response(JSON.stringify({ user: user }), {
      headers: { "Content-Type": "application/json" },
      status: 200,
    });
  } catch (error) {
    return new Response(JSON.stringify({ error: error.message }), {
      headers: { "Content-Type": "application/json" },
      status: 400,
    });
  }
});

Sample node app:
import { createClient } from '@supabase/supabase-js'

const supabase = createClient(
  'LOCAL HOST',
  'SUPABASE_PUBLIC_KEY'
)

const { data: userData, error: userError } = await supabase.auth.signInWithPassword({
  email<EMAIL_ADDRESS>  password: 'mypassword',
})

// Exists here
console.log("User Data: ", userData)

const { data, error } = await supabase.functions.invoke('myfunction')

Or maybe this is the issue... =/

No storage option exists to persist the session, which may result in unexpected behavior when using auth.
If you want to set persistSession to true, please provide a storage option or you may set persistSession to false to disable this warning.

Hi everyone, we have a guide here on how to set up the auth context in your edge function correctly: https://supabase.com/docs/guides/functions/auth#auth-context
@Qw4z1 are you still facing this issue? You need to set persistSession to false when you create the client in your sample node app
As for your code in your edge function, did you check if req.headers.get("Authorization") returns a user's JWT? You should also be setting persistSession to false when you create the client here.

Followed the guide, verified auth header returns JWT, set persistSession to false. Still getting null when calling getUser() and the missing sub claim error
// Create a Supabase client with the Auth context of the logged in user.
    const supabaseClient = createClient<Database>(
      // Supabase API URL - env var exported by default.
      Deno.env.get("SUPABASE_URL") ?? "",
      // Supabase API ANON KEY - env var exported by default.
      Deno.env.get("SUPABASE_ANON_KEY") ?? "",
      // Create client with Auth context of the user that called the function.
      // This way your row-level-security (RLS) policies are applied.
      {
        global: {
          headers: { Authorization: req.headers.get("Authorization")! },
        },
        auth: {
          persistSession: false,
        },
      },
    );
    const { data: { user }, error } = await supabaseClient.auth.getUser();
    console.log(user, error); // returns null

The error
[Info] null AuthApiError: invalid claim: missing sub claim
    at le (https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:5282)
    at eventLoopTick (ext:core/01_core.js:64:7)
    at async Ie (https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:6069)
    at async h (https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:5806)
    at async https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:26219
    at async m._useSession (https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:25008)
    at async m._getUser (https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:26136)
    at async https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:25999
    at async https://esm.sh/v135/@supabase/gotrue-js@2.57.0/esnext/gotrue-js.mjs:2:24294 {
  __isAuthError: true,
  name: "AuthApiError",
  status: 401
}

I am having the same error, but was able to get the user as follows
  const supabaseClient = createClient(
    Deno.env.get("SUPABASE_URL") ?? "",
    Deno.env.get("SUPABASE_ANON_KEY") ?? "",
    {
      global: {
        headers: { Authorization: req.headers.get("Authorization") ?? "" },
      },
    },
  );

  const jwt = req.headers.get("Authorization")!.split(" ")[1];
  const { data: { user }, error } = await supabaseClient.auth.getUser(jwt);

However, this is a different behavior from the documentation and this issue should be corrected

@Chipsnet thanks for figuring it out! In fact, the current behavior is correct and you have to pass the JWT token to getUser.
It's documentation says "Gets the current user details if there is an existing session.". Apparently, there's no session in the edge environment so it's required.  Edge docs have to be fixed though.

This is super annoying and is a recent breaking change which has had serious consequences on our production services. Please FIX DOCS ASAP!

Setting authorization headers on new clients does not work as documented. The only solution to using clients within edge function is the undocumented solution identified by @Chipsnet

Hey guys, any update on this issue?
I tested everything above + https://github.com/supabase/supabase/blob/master/examples/edge-functions/supabase/functions/select-from-table-with-auth-rls/index.ts, but still getting null user.
What I'm trying to do is to trigger webhook with Edge Function, to update table based on any operation in another table.

any update on this issue?

Any update on this ?


Last sale (in seconds since Unix epoch): undefined

Anyone else have the issue of the bot suddenly stop posting tweet of sales only message seen in logs every 1-2 mins
Last sale (in seconds since Unix epoch): undefined
update: twitter seems to suspend accounts thinking its spam

Can this be resolved by reducing the interval it checks?

Last sale (in seconds since Unix epoch): undefined just means a previous sale hasn't happened yet.
Make sure you've got a phone number attached to your Twitter bot account + you aren't polling for changes too often (default in my example is every min)


Feature reuqest: update PettingZoo version

Hi, would it be possible to update this repo to use the most recent version of PettingZoo? This project is listed in PettingZoo's third-party-environments, but we only want to include environments which work with the current version.
If you need any help working out issues due to different versions feel free to ask, there were some breaking changes in version 1.2, so it requires a bit of code changes to adapt. The previous API returned done in the step() function, whereas the new one returns truncated and terminated (matching gymnasium). There is a migration guide for gymnasium explaining the changes further, the steps should be basically the same (we're working on making resources for updating old PettingZoo repositories as well): https://gymnasium.farama.org/content/migration-guide/

Hi, thanks for raising this issue. We updated the env to the newest pz version.

Hi, thanks for raising this issue. We updated the env to the newest pz version.

Hi, apologies for the bad timing but there’s currently a API bug which warrants a new release, I’ll post here when it is fixed but you will want to move to that version instead.

@axelbr Hey sorry for the confusion but the version 1.22.4 which you have in the requirements.txt was removed from pypi due to some API bugs, we're working on 1.23 and should get that out relatively soon but for now you should change it to 1.22.3.


Modsecurity blokcs after the exclusions

Hi,
I have tried to exclude the string scrip%u0074 from blocking. I added SecRule BEFORE the rules (../rules/.conf) or SecRuleUpdateTargetById AFTER the rules (../rules/.conf) and SecRuleUpdateTargetById AFTER the rules (../rules/*.conf), but I was still blocked.
My secure2.conf file:
<IfModule security2_module>
SecDataDir /var/cache/modsecurity
SecRule ARGS:name "@contains scrip%u0074"\
"id:1234567,phase:1,pass,t:none,t:urlDecodeUni,logdata:'Bypass script',ctl:ruleRemoveByTag=attack-xss;!ARGS:name,ctl:ruleRemoveById=920220;!ARGS:name"
IncludeOptional /etc/modsecurity/*.conf
IncludeOptional /etc/apache2/modsecurity-crs/coreruleset-3.3.0/crs-setup.conf
IncludeOptional /etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/*.conf
SecRuleUpdateTargetById 920220 !ARGS:name
SecRuleUpdateTargetByTag "attack-xss" "!ARGS:name"
</IfModule>
Additionally, I have checked the installed version of modsececurity:
ModSecurity for Apache/2.9.5 (http://www.modsecurity.org/) configured.
ModSecurity: APR compiled version="1.7.0"; loaded version="1.7.0"
ModSecurity: PCRE compiled version="8.39 "; loaded version="8.39 2016-06-14"
ModSecurity: LUA compiled version="Lua 5.1"
ModSecurity: YAJL compiled version="2.1.0"
ModSecurity: LIBXML compiled version="2.9.12"

Thank you for opening this issue here.
Could you please add the logs (error.log) so that I see the triggered rules?
And also a curl call to reproduce your problem?
Thank you.

I insert in the browser URL for the name parameter the value scrip%u0074 and, with rules defined above  got such errors:
[Sat Jun 29 06:21:42.823832 2024] [:error] [pid 1175:tid<PHONE_NUMBER>21120] [client <IP_ADDRESS>:38828] [client <IP_ADDRESS>] ModSecurity: Warning. Invalid URL Encoding: Non-hexadecimal digits used at REQUEST_URI. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "371"] [id "920220"] [msg "URL Encoding Abuse Attack Attempt"] [data "/dvwa/vulnerabilities/xss_r/?name=scrip%u0074"] [severity "WARNING"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/255/153/267/72"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "Zn_gNsrsQpRb7rS4ziQduwAAAFI"]
[Sat Jun 29 06:21:42.824030 2024] [:error] [pid 1175:tid<PHONE_NUMBER>21120] [client <IP_ADDRESS>:38828] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "^[\\\\d.:]+$" at REQUEST_HEADERS:Host. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "735"] [id "920350"] [msg "Host header is a numeric IP address"] [data "<IP_ADDRESS>"] [severity "WARNING"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [tag "PCI/6.5.10"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "Zn_gNsrsQpRb7rS4ziQduwAAAFI"]
[Sat Jun 29 06:21:42.825027 2024] [:error] [pid 1175:tid<PHONE_NUMBER>21120] [client <IP_ADDRESS>:38828] [client <IP_ADDRESS>] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "93"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 6)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "Zn_gNsrsQpRb7rS4ziQduwAAAFI"]
[Sat Jun 29 06:21:42.825431 2024] [:error] [pid 1175:tid<PHONE_NUMBER>21120] [client <IP_ADDRESS>:38828] [client <IP_ADDRESS>] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/RESPONSE-980-CORRELATION.conf"] [line "91"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 6 - SQLI=0,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 6, 0, 0, 0"] [ver "OWASP_CRS/3.3.0"] [tag "event-correlation"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "Zn_gNsrsQpRb7rS4ziQduwAAAFI"]

As can be seen, the first log is info about blocking with rule 920220, which was excluded in my defined rules.

@msepczuk Rule 920220 is matching against REQUEST_URI_RAW, not ARGS so you need to modify your exclusion rule. Also, do not prefix variable with !, this is incorrect syntax. Finally, using t:urlDecodeUni is probably going to do problems, too.

Thank you for providing the logs. Yes, I incorrectly assumed that you are blocked by an argument name.
As azurit said, you have to remove the REQUEST_URI_RAW. But as this REQUEST_URI_RAW is the only target that is checked by this rule, you can disable it entirely either unconditionally with:
SecRuleRemoveById 920220 (AFTER the CRS rules include).
Or your suggested idea with your rule and the ctl action BEFORE the CRS rules.
But here you're also mixing various possibilities.
To disable a rule entirely with ctl, use: ctl:ruleRemoveById=920220 .
Important detail: this is not followed by a target ;ARGS=name.
Please have a look at these details and see https://coreruleset.org/docs/concepts/false_positives_tuning/#rule-exclusions.

OK, I simplify the rule:
<IfModule security2_module>
SecDataDir /var/cache/modsecurity

SecRule REQUEST_URI_RAW "@contains scrip%u0074"\
"id:1234567,phase:1,pass,t:none,t:urlDecodeUni,logdata:'Bypass script',ctl:ruleRemoveById=920220"

IncludeOptional /etc/modsecurity/*.conf
IncludeOptional /etc/apache2/modsecurity-crs/coreruleset-3.3.0/crs-setup.conf
IncludeOptional /etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/*.conf

</IfModule>

I am aware that here the problem is related to the t:urlDecodeUni But without that, a value scrip%u0074 is literally put on the page, but I want to change %u0074 to char t (putting script  on the page)

@msepczuk Transformation t:urlDecodeUni is converting the data only inside modsecurity, it has no effect on what data are send to the application. Your exclusion rule will probably doesn't work as scrip%u0074 inside REQUEST_URI_RAW is first converted to script and then it is compated with @contains scrip%u0074 which is no longer true.

By the way, you should put your exclusion rule into file /etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.

OK, I will put my rule in the file you indicated, thanks for the advice! So I went back to the beginning- I want to bypass ModSecurity detection for one particular character (for example, Unicode encoded char). Still, after bypassing, the string should be put on the page in the decoded version. Is it possible with ModSecurity?

No, modsecurity is NOT modifing the data, this is something your web server or application is supposed to do.

One more thing in the context of rule 941100 (the rule from the XSS rules file). I try to bypass this by:
SecRule ARGS_NAMES "@contains scrip%u0074" \
    "id:12345678,phase:1,pass,logdata:'Bypass script 2', ctl:ruleRemoveById=941100"

But it is blocked by the rule 941100.

Can you please provide a log entry for 941100?

ARGS_NAMES is automatically decoded by web server so modsecurity sees decoded value (as you can see in the log) - use <script in the exclusion rule. Alternatively, match it against REQUEST_URI_RAW.

Are you really using CRS v3.3.0?

Such rules:

Generate such logs:

@msepczuk - based on your log snippet (next time please share a text based content) your URI is not what you set in your exclusion.
Please see the log lines' [uri] fields.
I think you should try something similar:
SecRule REQUEST_URI "@begisWith /dvwa/vulnerabilities/xss_r/" \
    "id:12345678,\
    phase:1,\
    t:none,\
    pass,\
    ctl:ruleRemoveById=941100,\
    ctl:ruleRemoveById=941160,\
    ..."

Also: please do not create any exclusion against rules 920NNN - especially your URI does not contain any mentioned character in rule 920220.

@airween This is only for experiment purposes (this exclusion), not for accurate detection. I want to exclude some name parameter values from detection and blocking. Based on your proposal:
SecRule REQUEST_URI "@beginsWith /dvwa/vulnerabilities/xss_r/"\
	"id:12345678,phase:1,pass,t:none,logdata:'Bypass script 12345678',ctl:ruleRemoveById=941100c,tl:ruleRemoveById=941160"

Got such logs:
[Mon Jul 01 08:54:21.586008 2024] [:error] [pid 4554:tid<PHONE_NUMBER>68352] [client <IP_ADDRESS>:33640] [client <IP_ADDRESS>] ModSecurity: Warning. detected XSS using libinjection. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "55"] [id "941100"] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data found within ARGS:name: <script"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoKm_ZF_-Dw3x_xw4F3QTQAAAEA"]
[Mon Jul 01 08:54:21.586220 2024] [:error] [pid 4554:tid<PHONE_NUMBER>68352] [client <IP_ADDRESS>:33640] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "(?i:(?:<\\\\w[\\\\s\\\\S]*[\\\\s\\\\/]|['\\"](?:[\\\\s\\\\S]*[\\\\s\\\\/])?)(?:on(?:d(?:e(?:vice(?:(?:orienta|mo)tion|proximity|found|light)|livery(?:success|error)|activate)|r(?:ag(?:e(?:n(?:ter|d)|xit)|(?:gestur|leav)e|start|drop|over)|op)|i(?:s(?:c(?:hargingtimechange ..." at ARGS:name. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "209"] [id "941160"] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: <script found within ARGS:name: <script"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoKm_ZF_-Dw3x_xw4F3QTQAAAEA"]
[Mon Jul 01 08:54:21.586831 2024] [:error] [pid 4554:tid<PHONE_NUMBER>68352] [client <IP_ADDRESS>:33640] [client <IP_ADDRESS>] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "93"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 13)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoKm_ZF_-Dw3x_xw4F3QTQAAAEA"]
[Mon Jul 01 08:54:21.587198 2024] [:error] [pid 4554:tid<PHONE_NUMBER>68352] [client <IP_ADDRESS>:33640] [client <IP_ADDRESS>] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/RESPONSE-980-CORRELATION.conf"] [line "91"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 13 - SQLI=0,XSS=10,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 13, 0, 0, 0"] [ver "OWASP_CRS/3.3.0"] [tag "event-correlation"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoKm_ZF_-Dw3x_xw4F3QTQAAAEA"]

And finally, as I understand, Your rule is related to any parameter's value with indicated URI, but my goal is to bypass, for example, <script, but <SVG... should still be blocked.

@msepczuk You have typo in that exclusion rule, see (c at the end):
ctl:ruleRemoveById=941100c

@msepczuk:

see what @azurit wrote above
your rule contains this action: logdata:'Bypass script 12345678' - have you seen this message in the log?

If not, then it means your (exclusion) rule does not trigger, so your exclusions don't work as you expect.

Sorry, you are right. I corrected rule in the file but not here (improved now). I can't see this message, so as you mentioned something is wrong with this definition of rule. I checked log message related to these two rules and lack of bypass communicate
Rules:
SecRule REQUEST_URI "@beginsWith /dvwa/vulnerabilities/xss_r/"\
	"id:12345678,phase:1,pass,t:none,logdata:'Bypass script 12345678',ctl:ruleRemoveById=941100,ctl:ruleRemoveById=941160"

SecRule REQUEST_URI_RAW "@contains <scrip\u0074"\
	"id:1234567,phase:1,pass,t:none,t:urlDecodeUni,logdata:'Bypass script 1234567',ctl:ruleRemoveById=920220"

And Logs:
[Mon Jul 01 14:58:49.833466 2024] [:error] [pid 6133:tid<PHONE_NUMBER>37824] [client <IP_ADDRESS>:34240] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "^[\\\\d.:]+$" at REQUEST_HEADERS:Host. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "735"] [id "920350"] [msg "Host header is a numeric IP address"] [data "<IP_ADDRESS>"] [severity "WARNING"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [tag "PCI/6.5.10"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoL8aYe2cZOofZlPge5aywAAAAw"]
[Mon Jul 01 14:58:49.834003 2024] [:error] [pid 6133:tid<PHONE_NUMBER>37824] [client <IP_ADDRESS>:34240] [client <IP_ADDRESS>] ModSecurity: Warning. detected XSS using libinjection. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "55"] [id "941100"] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data found within ARGS:name: <script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoL8aYe2cZOofZlPge5aywAAAAw"]
[Mon Jul 01 14:58:49.834091 2024] [:error] [pid 6133:tid<PHONE_NUMBER>37824] [client <IP_ADDRESS>:34240] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "(?i)<script[^>]*>[\\\\s\\\\S]*?" at ARGS:name. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "92"] [id "941110"] [msg "XSS Filter - Category 1: Script Tag Vector"] [data "Matched Data: <script> found within ARGS:name: <script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoL8aYe2cZOofZlPge5aywAAAAw"]
[Mon Jul 01 14:58:49.834242 2024] [:error] [pid 6133:tid<PHONE_NUMBER>37824] [client <IP_ADDRESS>:34240] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "(?i:(?:<\\\\w[\\\\s\\\\S]*[\\\\s\\\\/]|['\\"](?:[\\\\s\\\\S]*[\\\\s\\\\/])?)(?:on(?:d(?:e(?:vice(?:(?:orienta|mo)tion|proximity|found|light)|livery(?:success|error)|activate)|r(?:ag(?:e(?:n(?:ter|d)|xit)|(?:gestur|leav)e|start|drop|over)|op)|i(?:s(?:c(?:hargingtimechange ..." at ARGS:name. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "209"] [id "941160"] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: <script found within ARGS:name: <script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoL8aYe2cZOofZlPge5aywAAAAw"]
[Mon Jul 01 14:58:49.834595 2024] [:error] [pid 6133:tid<PHONE_NUMBER>37824] [client <IP_ADDRESS>:34240] [client <IP_ADDRESS>] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "93"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 18)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoL8aYe2cZOofZlPge5aywAAAAw"]
[Mon Jul 01 14:58:49.834827 2024] [:error] [pid 6133:tid<PHONE_NUMBER>37824] [client <IP_ADDRESS>:34240] [client <IP_ADDRESS>] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/RESPONSE-980-CORRELATION.conf"] [line "91"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 18 - SQLI=0,XSS=15,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 18, 0, 0, 0"] [ver "OWASP_CRS/3.3.0"] [tag "event-correlation"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoL8aYe2cZOofZlPge5aywAAAAw"]

@msepczuk where did you put these exclusions? Are you sure those are before the other rules?

I put them in REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example . After each change I have performed apache2 restart.

Files with .example extension aren't processed.  At least now logs included my 3 rules:
[Tue Jul 02 12:32:05.820493 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Warning. String match "<scrip\\\\u0074>" at REQUEST_URI_RAW. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf"] [line "80"] [id "1234567"] [data "Bypass script 1234567"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]
[Tue Jul 02 12:32:05.820565 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Warning. String match "/dvwa/vulnerabilities/xss_r/" at REQUEST_URI. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf"] [line "83"] [id "12345678"] [data "Bypass script 12345678"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]
[Tue Jul 02 12:32:05.820591 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Warning. String match "<scrip\\\\u0074>" at ARGS:name. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf"] [line "86"] [id "123456789"] [data "Bypass script 123456789"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]
[Tue Jul 02 12:32:05.822241 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "^[\\\\d.:]+$" at REQUEST_HEADERS:Host. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "735"] [id "920350"] [msg "Host header is a numeric IP address"] [data "<IP_ADDRESS>"] [severity "WARNING"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [tag "PCI/6.5.10"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]
[Tue Jul 02 12:32:05.822841 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "(?i)<script[^>]*>[\\\\s\\\\S]*?" at ARGS:name. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "92"] [id "941110"] [msg "XSS Filter - Category 1: Script Tag Vector"] [data "Matched Data: <script> found within ARGS:name: <script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]
[Tue Jul 02 12:32:05.823454 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "93"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 8)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]
[Tue Jul 02 12:32:05.823863 2024] [:error] [pid 4099:tid<PHONE_NUMBER>39360] [client <IP_ADDRESS>:60706] [client <IP_ADDRESS>] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/modsecurity-crs/coreruleset-3.3.0/rules/RESPONSE-980-CORRELATION.conf"] [line "91"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 8 - SQLI=0,XSS=5,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 8, 0, 0, 0"] [ver "OWASP_CRS/3.3.0"] [tag "event-correlation"] [hostname "<IP_ADDRESS>"] [uri "/dvwa/vulnerabilities/xss_r/"] [unique_id "ZoQrhQ921rv2Qg4yKhZJEgAAAAA"]

I have added one rule against this id: 941110 (

@msepczuk You are doing the same mistakes all the time (like searching for encoded value but using t:urlDecodeUni). You really need to think what you are doing, everything needed was already told to you.

What else do you need? What exactly is not working for you?

@msepczuk You need to pay more attention on what you are doing and reading - you have another typo-like error in your rule 123456789: you are removing rule 941100 but the one which was triggered is 941110.

Oh my mistake. Actually, a typo and it's working now. Thank you for your help and patience!

Please close this issue if you don't need other help.


Ignore arrow keys when event.metaKey.

Currently, impress.js breaks keyboard shortcuts for the browser's Back (⌘←) and Forward (⌘→) because it swallows all keyup events without first checking event.metaKey. You'll probably want to do some filtering around impress.js:425. For comparison, see my implementation in stack.js.

You are probably right... but there is more than just metaKey. On other systems (or browsers) there may be different shortcuts using shiftKey ctrlKey or altKey. They should all be detected I guess.
I will look into it, thanks for suggestion!

On the other hand... during the presentation arrow keys actually do more or less whan back/forward would do.
And you should be able to use [Backspace] key to move back (it seems to work both on Windows and Mac).

Isn't this bug fixed by this commit: https://github.com/impress/impress.js/commit/d8995aa3194f6fdbcc8867626d52f913d0c52882

Isn't this bug fixed by this commit: d8995aa

I guess I didn't closed this in the first triage because we need to check it to make sure. Since that code doesn't have test coverage, I am not sure if we can consider this as "really" fixed.

This is now fixed in master branch.


Compare to different GPUs (RTX 4060Ti and GTX 1660Ti) performance

Issue type
Performance
Have you reproduced the bug with TensorFlow Nightly?
No
Source
binary
TensorFlow version
tf 2.2.3/tf2.3/ tf2.5/tf2.10/tf2.12
Custom code
Yes
OS platform and distribution
Ubuntu 18.04
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Hello,
I'm comparing the performance of two GPUs and the differences between different TensorFlow versions.
I plan to upgrade from GTX 1660 Ti to RTX 4060Ti. I expected the inference speed to be faster after the upgrade, but surprisingly it wasn't.
Below are the results of inferring different models and batch sizes. I used Docker to test different versions of TF.
Due to cuda version, it's hard to compare almost tf versions.
The number is average inference ms
GTX 1660 Ti

RTX 4060 Ti

GTX 1660 Ti vs RTX 4060 Ti (w/ same tf version)

From the chart above, I've drawn a few conclusions:

The higher the TF version, the longer the inference time... why???
I expected the 4060Ti to be faster than the 1660Ti for the same TF version, but this is only observed when bs=8/16, while my use case is for a single image.

I suspected it might be a card issue, but when I tested the PyTorch version of YOLOv5, the 4060Ti was indeed faster.
I wonder if any seniors can give me some clues or help me understand this, thank you.
Is there anything else you'd like me to help with regarding this translation or the content of your message?
Thanks,
KT
Standalone code to reproduce the issue
https://colab.research.google.com/drive/1e_lXPvQ82lCuOyFs9xK8oztagNwxFfkd?usp=sharing

Relevant log output
No response

Hi @tom2002965 ,

Here i observed version mismatch. Could you please check with compatibility versions? And Please use latest version for better results. let us know whether the issue persists with recent TF versions.

Thank you!

Hi @Venkat6871 ,
Very thank for your response.

All the tf/cuda/cudnn are from docker which is tensorflow provided.
Now my gpu drive of RTX 4060Ti is 535.183.01 . I have to update driver to test latest version . So I test tf-2.14 first. The result is below.
2.14 is faster than 2.10/2.12, but it still slower than tf 2.5.

Thanks.

Hi @tom2002965 ,
Apologies for the delay, and thank you for your patience. It appears there are still compatibility issues with TensorFlow 2.14.0. The compatible version of cuDNN is 8.7. Could you please double-check all the compatible versions and verify with the latest ones? If you continue to face issues, please let us know. For your convenience, here is the documentation to help verify the compatible versions.
Thank you!


always return true

i am using laravel 5.2 always return true even if wrong email address provided

/ping @olsgreen ;)

@Grald Are you using the validator or trying to use it via DI or a facade?

I'm seeing this, too, in Laravel. I've tried using it via DI and by plain instantiation (new EmailChecker).

@josephsong Which version of Laravel are you using?

Actually, Lumen v5.3.3.

Well, it is funny. If I use yopmail.com as the domain for the test email, it does return false on the isValid call. But for any others I've tried from the list in this package, it still returns true.

It seems that any of the domains in tests/EmailChecker/Tests/Adpater/BuiltInAdapterTest.php do come back as false. But any others in res/throwaway_domains.txt come back as true.
I had just successfully implemented this package in WordPress, and all domains in the throwaway_domain.txt are marked as invalid. Seems to be something specific to Laravel/Lumen.


Questions about SIFT+LightGlue

Thank you for your great work!
I would like to confirm the following things about your SIFT+LightGlue.

Which type of SIFT is used for training of the published SIFT+LightGlue weight, OpenCV or PyColmap?
Is there a performance gap between OpenCV SIFT and PyColmap one for the published SIFT+Lightglue weight.
In the glue-factory implementation, PyColmap SIFT scores are calculated as the combination of DoG (all values are 1?) scores and scales while OpenCV scores as just DoG scores. Which can we get better results by?

The questions above are answered in LightGlue repository.
Thank you very much.


Opening calculator on execution of script

When trying to execute of a script, gcalc opens up
Currently using Pop!Os on ROG GL503GE

That is strange.  I'm guessing that the keyboard color doesn't change?

I have the exact same issue! Using Linux Mint 19.2 Cinnamon on an Asus ROG Strix GL503VD.
This happens on all commands except rogauracore black. The command does still work, it just also opens the calculator, after changing the keyboard color.

Same issue on Manjaro. Asus ROG Strix GL503VD.

A second or so after running the command and changing the keyboard colour there comes an input event for the calculator button from my keyboard. I haven't found any consistent difference between me actually pressing the (fn+)calculator button and the input event form this program with xinput test-xi2. If it's useful for anyone trying to debug this I can post the logs of course.
For anyone still having this issue, a workaround to make it stop opening the calculator is just removing the Calculator button as a shortcut to open the calculator in your keyboard settings - I've changed it to Shift+Calculator and it works just fine, without rogauracore opening it.

@CYPHER0041 also reports this issue, also on a GL503GE.


Refactor AgentFindByToken

followup of #1890

Codecov Report
Patch coverage: 100.00% and project coverage change: -0.02 :warning:

Comparison is base (b57b6f2) 39.23% compared to head (07a10f0) 39.22%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1891      +/-   ##
==========================================
- Coverage   39.23%   39.22%   -0.02%
==========================================
  Files         177      177
  Lines       10706    10704       -2
==========================================
- Hits         4201     4199       -2
  Misses       6212     6212
  Partials      293      293

Impacted Files
Coverage Δ

server/store/datastore/agent.go
50.00% <100.00%> (-4.17%)
:arrow_down:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


UQI v2

@nidhi-nair I'm closing this since @ajinkyakulkarni informed me that we're not prioritising this.


[C++][Parquet] replace shared_ptr to unique_ptr in Bloom filter buffer allocation

Reporter: Junjie Chen / @chenjunjiedada
Assignee: Junjie Chen](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=junjie) / @chenjunjiedada
Note: This issue was originally created as PARQUET-1377. Please see the migration documentation for further details.

Gang Wu / @wgtmac:
Bloom filter implementation has been refactored already.


Update README.md

This is a documentation fix to work with the latest pull request that was merged into the python-preview branch.

Thank you for finding this! This and other issues are being solved here! https://github.com/microsoft/semantic-kernel/pull/317

Ah, this is the top-level README (we fixed up the one in the ./python subdir). I'm not quite sure what we should do with the top-level, it'll (eventually) need to get merged w/ the top-level readme in the main branch which could be tricky.
One proposal: take the current top-level README in main, paste it here again, and make minimal edits to add a "python" getting started section? (That matches our ./python/README.md) Or even just a small edit (for now) to link to the appropriate Python readme?
Open to thoughts!

@ChinmaySheth we're about to rewrite the main README and to avoid conflicts it's probably better not touching this file for now.