Conversion webhook incompatible with Flux GitOps management of nodepool manifests

Description
Observed Behavior:
When flux is managing the manifests for nodepools, the following is observed. I am starting from version 0.37.2
Update controller to v1.0.1, and then apply updated iam policy. Next update the manifests for ec2nodeclasses and nodepools to the v1 api version in git.
Flux fails to apply the nodepool manifests with the following error
karpenter-nodepools             main@sha1:1eb82518      False           False   NodePool/default dry-run failed: failed to prune fields: failed add back owned items: failed to convert pruned object at version karpenter.sh/v1: conversion webhook for karpenter.sh/v1beta1, Kind=NodePool failed: Post "https://karpenter.karpenter.svc:8443/conversion/karpenter.sh?timeout=30s": EOF

This can be reproduced by doing a dry run of the same manifest with Kubectl
$ kubectl apply --server-side=true -f flux/infra/karpenter-nodepools/default.yaml --dry-run=server --field-manager kustomize-
Error from server: failed to prune fields: failed add back owned items: failed to convert pruned object at version karpenter.sh/v1: conversion webhook for karpenter.sh/v1beta1, Kind=NodePool failed: Post "https://karpenter.karpenter.svc:8443/conversion/karpenter.sh?timeout=30s": EOF

The Karpenter controller logs the following error when this occurs
{
  "level": "ERROR",
  "time": "2024-08-23T20:03:38.118Z",
  "logger": "webhook",
  "message": "http: panic serving <IP_ADDRESS>:49368: runtime error: invalid memory address or nil pointer dereference\ngoroutine 10304 [running]:\nnet/http.(*conn).serve.func1()\n\tnet/http/server.go:1903 +0xb0\npanic({0x2225100?, 0x4734a10?})\n\truntime/panic.go:770 +0x124\nsigs.k8s.io/karpenter/pkg/apis/v1.(*NodeClaimTemplate).convertFrom(0x40034c9310, {0x2f1bb28, 0x4004dfbe60}, 0x4000f6d708)\n\tsigs.k8s.io/karpenter@v1.0.0/pkg/apis/v1/nodepool_conversion.go:181 +0x188\nsigs.k8s.io/karpenter/pkg/apis/v1.(*NodePoolSpec).convertFrom(0x40034c9310, {0x2f1bb28, 0x4004dfbe60}, 0x4000f6d708)\n\tsigs.k8s.io/karpenter@v1.0.0/pkg/apis/v1/nodepool_conversion.go:145 +0xe8\nsigs.k8s.io/karpenter/pkg/apis/v1.(*NodePool).ConvertFrom(0x40034c9208, {0x2f1bb28?, 0x4004dfbe60?}, {0x2efd390?, 0x4000f6d600})\n\tsigs.k8s.io/karpenter@v1.0.0/pkg/apis/v1/nodepool_conversion.go:121 +0x124\nknative.dev/pkg/webhook/resourcesemantics/conversion.(*reconciler).convert(0x400098d8c0,{0x2f1bb28, 0x4004dfbc50}, {{0x400627e000, 0x297, 0x2c0}, {0x0, 0x0}}, {0x400498e880, 0xf})\n\tknative.dev/pkg@v0.0.0-20231010144348-ca8c009405dd/webhook/resourcesemantics/conversion/conversion.go:137 +0x119c\nknative.dev/pkg/webhook/resourcesemantics/conversion.(*reconciler).Convert(0x400098d8c0, {0x2f1bb28?, 0x4004dfbad0?}, 0x4001f5ab40)\n\tknative.dev/pkg@v0.0.0-20231010144348-ca8c009405dd/webhook/resourcesemantics/conversion/conversion.go:57 +0x174\nknative.dev/pkg/webhook.New.conversionHandler.func5({0x2f0c658, 0x4000bbf7a0}, 0x40087699e0)\n\tknative.dev/pkg@v0.0.0-20231010144348-ca8c009405dd/webhook/conversion.go:66 +0x24c\nnet/http.HandlerFunc.ServeHTTP(0x40000b7400?, {0x2f0c658?, 0x4000bbf7a0?}, 0x1d01d10?)\n\tnet/http/server.go:2171 +0x38\nnet/http.(*ServeMux).ServeHTTP(0x4004dfb800?, {0x2f0c658, 0x4000bbf7a0}, 0x40087699e0)\n\tnet/http/server.go:2688 +0x1a4\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP(0x40000b7380, {0x2f0c658, 0x4000bbf7a0}, 0x40087699e0)\n\tknative.dev/pkg@v0.0.0-20231010144348-ca8c009405dd/webhook/webhook.go:310 +0xc4\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP(0x400063a930, {0x2f0c658, 0x4000bbf7a0}, 0x40087699e0)\n\tknative.dev/pkg@v0.0.0-20231010144348-ca8c009405dd/network/handlers/drain.go:113 +0x158\nnet/http.serverHandler.ServeHTTP({0x2ef71d0?}, {0x2f0c658?, 0x4000bbf7a0?}, 0x6?)\n\tnet/http/server.go:3142 +0xbc\nnet/http.(*conn).serve(0x40036539e0, {0x2f1bb28, 0x4000a5ec60})\n\tnet/http/server.go:2044 +0x508\ncreated by net/http.(*Server).Serve in goroutine 287\n\tnet/http/server.go:3290 +0x3f0\n",
  "commit": "62a726c"
}

Expected Behavior:
GitOps tooling like flux should successfully apply the nodepool manifest from
git.
Reproduction Steps (Please include YAML):
Versions:

Chart Version: 1.0.1

Kubernetes Version (kubectl version): 1.30 (EKS)

Flux Version: 2.3.0

Manage Karpenter's installation with flux, I am installing to the karpenter namespace.

Update to version 1.0.1 from 0.37.2

---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: karpenter-crd
  namespace: flux-system
spec:
  chart:
    spec:
      chart: karpenter-crd
      reconcileStrategy: ChartVersion
      sourceRef:
        kind: HelmRepository
        name: karpenter
      version: 1.0.1
  install:
    createNamespace: true
  upgrade:
    remediation:
      retries: 1
  interval: 10m0s
  releaseName: karpenter-crd
  targetNamespace: karpenter
  storageNamespace: karpenter
  values:
    webhook:
      enabled: true
      serviceNamespace: karpenter
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: karpenter
  namespace: flux-system
spec:
  chart:
    spec:
      chart: karpenter
      reconcileStrategy: ChartVersion
      sourceRef:
        kind: HelmRepository
        name: karpenter
      version: 1.0.1
  install:
    createNamespace: true
    crds: Skip # handled by karpenter-crd chart
  upgrade:
    crds: Skip # handled by karpenter-crd chart
    remediation:
      retries: 1
  interval: 10m0s
  releaseName: karpenter
  targetNamespace: karpenter
  storageNamespace: karpenter
  values:
    webhook:
      enabled: true
...

modify the node pool manifest to api version v1

--- a/flux/infra/karpenter-nodepools/default.yaml
+++ b/flux/infra/karpenter-nodepools/default.yaml
@@ -1,4 +1,4 @@
  1 ⋮    │-apiVersion: karpenter.sh/v1
    ⋮  1 │+apiVersion: karpenter.sh/v1beta1
  2 ⋮  2 │ kind: NodePool
  3 ⋮  3 │ metadata:
  4 ⋮  4 │   name: default
@@ -14,10 +14,10 @@ spec:
 14 ⋮ 14 │   template:
 15 ⋮ 15 │     # metadata: {}
 16 ⋮ 16 │     spec:
 17 ⋮    │-      expireAfter: 336h0m0s
    ⋮ 17 │+      # expireAfter: 336h0m0s
 18 ⋮ 18 │       nodeClassRef:
 19 ⋮    │-        group: karpenter.k8s.aws
 20 ⋮    │-        kind: EC2NodeClass
    ⋮ 19 │+        # group: karpenter.k8s.aws
    ⋮ 20 │+        # kind: EC2NodeClass
 21 ⋮ 21 │         name: bottlerocket
 22 ⋮ 22 │       requirements:
 23 ⋮ 23 │         - key: karpenter.sh/capacity-type
diff --git a/flux/infra/karpenter/karpenter-helm.yaml b/flux/infra/karpenter/karpenter-helm.yaml
index 7bb956f..28457e3 100644
--- a/flux/infra/karpenter/karpenter-helm.yaml
+++ b/flux/infra/karpenter/karpenter-helm.yaml
@@ -12,7 +12,7 @@ spec:
 12 ⋮ 12 │       sourceRef:
 13 ⋮ 13 │         kind: HelmRepository
 14 ⋮ 14 │         name: karpenter
 15 ⋮    │-      version: 1.0.1
    ⋮ 15 │+      version: 0.37.2

workaround
In order to proceed I had to clear the managed fields on the nodepool objects
Here are the fields prior to wiping them out
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  annotations:
    compatibility.karpenter.sh/v1beta1-nodeclass-reference: '{"name":"bottlerocket"}'
    karpenter.sh/nodepool-hash: "3179949643060066316"
    karpenter.sh/nodepool-hash-version: v3
  creationTimestamp: "2024-08-23T00:07:33Z"
  generation: 1
  labels:
    kustomize.toolkit.fluxcd.io/name: karpenter-nodepools
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  managedFields:
  - apiVersion: karpenter.sh/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          f:kustomize.toolkit.fluxcd.io/name: {}
          f:kustomize.toolkit.fluxcd.io/namespace: {}
      f:spec:
        f:disruption:
          f:consolidationPolicy: {}
        f:limits:
          f:cpu: {}
          f:memory: {}
        f:template:
          f:metadata: {}
          f:spec:
            f:nodeClassRef:
              f:name: {}
            f:requirements: {}
    manager: kustomize-controller
    operation: Apply
    time: "2024-08-23T00:07:33Z"
  - apiVersion: karpenter.sh/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations: {}
    manager: karpenter
    operation: Update
    time: "2024-08-23T00:07:33Z"
  - apiVersion: karpenter.sh/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:resources:
          .: {}
          f:cpu: {}
          f:ephemeral-storage: {}
          f:memory: {}
          f:pods: {}
    manager: karpenter
    operation: Update
    subresource: status
    time: "2024-08-23T17:50:17Z"
  - apiVersion: karpenter.sh/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:karpenter.sh/nodepool-hash: {}
          f:karpenter.sh/nodepool-hash-version: {}
    manager: karpenter
    operation: Update
    time: "2024-08-23T21:14:17Z"
  - apiVersion: karpenter.sh/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        f:conditions: {}
        f:resources:
          f:hugepages-1Gi: {}
          f:hugepages-2Mi: {}
          f:hugepages-32Mi: {}
          f:hugepages-64Ki: {}
          f:nodes: {}
    manager: karpenter
    operation: Update
    subresource: status
    time: "2024-08-23T21:14:22Z"
  name: default
  resourceVersion: "456500391"
  uid: b999755c-b788-4a58-baec-4476a7eb03f7

I then ran the following command to clear the managed fields
$ kubectl proxy &
$ curl -X PATCH -H 'Content-Type: application/merge-patch+json' -d '{"metadata":{"managedFields": [{}]}}' http://<IP_ADDRESS>:8001/apis/karpenter.sh/v1beta1/nodepools/default

Flux was then able to successfully reconcile the manifest after the removal of the managed fields.
For comparison, here is what the managed fields look like after the reconcilation
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  annotations:
    compatibility.karpenter.sh/v1beta1-nodeclass-reference: '{"name":"bottlerocket"}'
    karpenter.sh/nodepool-hash: "11983763225509541388"
    karpenter.sh/nodepool-hash-version: v3
  creationTimestamp: "2024-08-23T00:07:33Z"
  generation: 2
  labels:
    kustomize.toolkit.fluxcd.io/name: karpenter-nodepools
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  managedFields:
  - apiVersion: karpenter.sh/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          f:kustomize.toolkit.fluxcd.io/name: {}
          f:kustomize.toolkit.fluxcd.io/namespace: {}
      f:spec:
        f:disruption:
          f:consolidationPolicy: {}
        f:limits:
          f:cpu: {}
          f:memory: {}
        f:template:
          f:spec:
            f:expireAfter: {}
            f:nodeClassRef:
              f:group: {}
              f:kind: {}
              f:name: {}
            f:requirements: {}
    manager: kustomize-controller
    operation: Apply
    time: "2024-08-23T21:18:48Z"
  - apiVersion: karpenter.sh/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:compatibility.karpenter.sh/v1beta1-nodeclass-reference: {}
          f:karpenter.sh/nodepool-hash-version: {}
        f:labels:
          .: {}
          f:kustomize.toolkit.fluxcd.io/name: {}
          f:kustomize.toolkit.fluxcd.io/namespace: {}
      f:spec:
        .: {}
        f:disruption:
          .: {}
          f:budgets: {}
          f:consolidateAfter: {}
          f:consolidationPolicy: {}
        f:limits:
          .: {}
          f:memory: {}
        f:template:
          .: {}
          f:metadata: {}
          f:spec:
            .: {}
            f:nodeClassRef:
              .: {}
              f:group: {}
              f:kind: {}
              f:name: {}
            f:requirements: {}
    manager: before-first-apply
    operation: Update
    time: "2024-08-23T21:18:48Z"
  - apiVersion: karpenter.sh/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:karpenter.sh/nodepool-hash: {}
    manager: karpenter
    operation: Update
    time: "2024-08-23T21:18:48Z"
  - apiVersion: karpenter.sh/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        f:resources:
          f:cpu: {}
          f:ephemeral-storage: {}
          f:memory: {}
          f:nodes: {}
          f:pods: {}
    manager: karpenter
    operation: Update
    subresource: status
    time: "2024-08-23T21:19:34Z"

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

We had a very similar issue after upgrading from 0.36.3 to 0.36.4. It looks related to #6849. The namespace was somehow ended up to kube-system although .Release.Namespace is karpenter and nothing else changed apart from the patch version bump

@dstrants - This is due to karpenter chart packaging static CRDs which hardcode the namespace / service name for the conversion webhook. You might want to follow https://github.com/aws/karpenter-provider-aws/issues/6847

We had a very similar issue after upgrading from 0.36.3 to 0.36.4. It looks related to #6849. The namespace was somehow ended up to kube-system although .Release.Namespace is karpenter and nothing else changed apart from the patch version bump

From what I can tell, these are not related issues. The webhook endpoint in the author's error is correctly referencing the karpenter namespace in which they deployed their controller, and the Karpenter controller is logging a failure for the conversion.
I'm receiving the same issue in my cluster, controlled through FluxCD and using the the karpenter and karpenter-crd Helm charts. I've also tried updating my manifest of the NodePool to match exactly what was converted on the cluster to no avail.

@booleanbetrayal and @cmartintlh you are both right. #6847 is more relevant to what we are facing! Thanks for pointing out 🙏🏼

From looking at the calls made by the API server to the conversion webhooks on a server side apply, I see that the an empty nodepool spec is being sent to the conversion webhooks. This results in a nil pointer dereference on the nodeClassRef when we try converting that component. We should be able to mitigate this from the webhook side by checking the object first https://github.com/kubernetes-sigs/karpenter/blob/b69e975128ac9a511542f9f1d245a6d4c3f91112/pkg/apis/v1/nodepool_conversion.go#L179

@engedaam thank you for your reply. At the moment, we paused the upgrade to karpenter v1.0.0 in all our environments. Only a test environment is based on that version now. Can we assume that when v1.1.0 is available and the conversion webhook is removed, we can upgrade successfully in two steps to version v1.1.0? We plan to upgrade our environments that are now based on version v0.37.1 by following these steps:

Upgrade to v1.0.0. In this step, all resources will be converted by the webhook as expected, but Terraform will not be able to manage them
Upgrade to v1.1.0. In this step the webhook doesn't exist anymore, so Terraform should be able to accomplish the apply operation.

Does it make sense?

@Giaco9NN Yeah, that makes sense. A good workaround here is also to use client side apply. Are you able to use that with Terraform? Flux offers the ability use client side which should help mitigate this issue: https://fluxcd.io/flux/faq/#why-are-kubectl-edits-rolled-back-by-flux

Same behavior with argocd,
I've upgraded the Karpenter CRD + controller successfully.
and only then I've converted the manifest files to API v1 with the new places for kubelet and expireAfter
when argo tried to sync i got the following error:
"failed to prune fields: failed add back owned items: failed to convert pruned object at version karpenter.sh/v1: conversion webhook for karpenter.sh/v1beta1, Kind=NodePool failed: Post "https://karpenter.karpenter.svc:8443/conversion/karpenter.sh?timeout=30s": EOF"
Anyone with argocd have solved it ?

@Sagiil  you should consider following https://github.com/aws/karpenter-provider-aws/issues/6847
There's a long thread of various issues with the upgrade to v1 manifests.

@engedaam I tried to leverage the field_manager functionality offered by the Terraform provider. This is the code:
field_manager {
  force_conflicts = true
  name            = "before-first-apply"
}```

This way, I can modify the node pool. I aim to find a way to remove this field_manager because it's unclear to me what it exactly does.
I think the only way is to wait until version v1.1.0.

So we added a fix to the nodeclassRef to not panic on a nil deference to gracefully handle partial objects from the API server during Server side apply. The fix will be included as in an up coming patch release https://github.com/kubernetes-sigs/karpenter/pull/1669. Until then, for those who can please try this snapshot to confirm the fix:
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin<PHONE_NUMBER>62.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-fdf403caa0dce1c77bbbc0bda6863be017d118a7" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

Cool.  I presume this will be backported to a 1.0.x?  I think 1.1 removed the conversion webhooks.

Cool. I presume this will be backported to a 1.0.x? I think 1.1 removed the conversion webhooks.

Yes, we plan on backporting the fix to all version that contain the conversion webhooks.

Closing out this issue as the fix was merged https://github.com/kubernetes-sigs/karpenter/pull/1669


solve multi-thread problem of StatsItemSet#getAndCreateStatsItem

Fix #515
What is the purpose of the change
to solve multi-thread problem of StatsItemSet#getAndCreateStatsItem.
Brief changelog

use putIfAbsent instead  #of put method of ConcurrentHashMap and
if the prev is not null then use get method
to avoid multi-thread problem.

Verifying this change
test case:
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.rocketmq.broker;

import org.apache.rocketmq.common.ThreadFactoryImpl;
import org.apache.rocketmq.store.stats.BrokerStatsManager;
import org.junit.Before;
import org.junit.Test;

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import static org.apache.rocketmq.store.stats.BrokerStatsManager.TOPIC_PUT_NUMS;

public class BrokerStatsManagerTest {

    private BrokerStatsManager brokerStatsManager;
    private ThreadPoolExecutor executor;
    @Before
    public void init() {
        brokerStatsManager = new BrokerStatsManager("DefaultCluster");
        executor = new ThreadPoolExecutor(100, 200, 10, TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(1000), new ThreadFactoryImpl("testMultiThread"));
    }

    @Test
    public void testBrokerStatsManager() throws InterruptedException {

        for(int i =0; i < 10000; i++) {
            executor.submit(new Runnable() {
                @Override
                public void run() {
                    brokerStatsManager.incTopicPutNums("topicTest", 2, 1);
                }
            });
        }
        Thread.sleep(5000);
        System.out.println(brokerStatsManager.getStatsItem(TOPIC_PUT_NUMS,"topicTest").getValue());
    }
}

@xiangwangcheng Could you change your destination repo to develop branch?

@xiangwangcheng Could you change your destination repo to develop branch?

have changed to develop branch

@xiangwangcheng could you add your test code in your pr? just like your issue https://github.com/apache/rocketmq/issues/515

Coverage decreased (-0.1%) to 43.078% when pulling fcae03e73e374078eabde6523f15f35fac730d41 on xiangwangcheng:statsitemset-multi-thread into edf97032e4cba0f593f46a212f7a06899a220a9c on apache:develop.


Schedule django management command

It's so easy to run zappa manage migrate
It would be awesome to do
events:
  - manage: migrate
    expression: rate(1 minute)

Note: not actually for the migrate command, but things like django-post_office's send_queued_mail

To fix it, I need to create a function to run every command, like this:
# crons.py

def migrate():
    from django.core.management import call_command
    call_command('migrate')

# zappa_settings.yaml

events:
  - function: "crons.migrate"
    expression: rate(1 minute)

But this won't run through the Django start up mechanism will it?

Will! This works fine.
I has the same question on my mind, but I try it and works fine.
In my action, I have models and cache used 💃

Thanks @ebertti this was really useful.

To save writing a function for each commend, I've come up with this hack that some might find helpful.
In a file called zappa_commands.py, I have:
class Runner:
    def __getattr__(self, attr):
        from django.core.management import call_command
        return lambda: call_command(attr)

import sys
sys.modules[__name__] = Runner()

I can then do the following in zappa_settings.json:
{
  "function": "zappa_commands.some_command",
  "expression": "rate(2 hours)"
}

This does a few things:

Turn the whole module in to a class. This is what sys.modules[__name__] = Runner()
is doing.

Overwrite __getattr__ on the class. Remember this is now __getattr__ on the file/module because the file is the class.

For any attr, return a python lambda that when called will return a
function that can be called by AWS Lambda.

I'm not especially proud of this, but it's working and I think it reasonable in the context of gluing commands to something I can call on a schedule. I still can't pass any arguments to the command, but that's ok in my case.

Please add official document on Django schedule command. It is very useful knowledge

thanks to @symroe for the handy workaround!
as a tidier alternative to instantiating a class and fiddling with sys.modules... in Python 3.7+ we can use a module-level __getattr__
so create a module e.g. django_commands.py like:
def __getattr__(self, attr):
    from django.core.management import call_command, get_commands
    if attr in get_commands():
        return lambda: call_command(attr)
    else:
        raise AttributeError(f"Django command {attr!r} not found")

(...I suspect the django import could be at the top level but haven't tested moving it yet)
and then in zappa settings:
{
  "function": "django_commands.some_command",
  "expression": "rate(2 hours)"
}

Hi! Its been working for a while, but today it lost all commands... for some reason it doesn't see any other then core one... Does anyone faced this issue?


python311Packages.ray: 2.7.0 -> 2.9.0; fix build

Description of changes
fix build:
https://hydra.nixos.org/build/246120529
upgrade to v 2.9.0. drop python3.9 due to unsupported dependencies (at least django)
https://github.com/ray-project/ray/releases/tag/ray-2.9.0
https://github.com/ray-project/ray/compare/ray-2.7.0...ray-2.9.0

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

I think it has been fixed in 2.9.0. https://github.com/ray-project/ray/pull/41376
Can you upgrade to a newer version?

Result of nixpkgs-review pr 280882 run on x86_64-linux 1

2 packages built:

python311Packages.ray
python311Packages.ray.dist

@ofborg build python310Packages.ray


compiler: add HAVE_EXPRESSION_STATEMENT macro for gcc

Summary
compiler: add HAVE_EXPRESSION_STATEMENT macro for gcc
this will open fast version of div_const when meet 64/32 when use gcc compiling
Impact
div_const in
include/nuttx/lib/math32.h
Testing
CI

[Experimental Bot, please feedback here]
No, this PR does not fully meet the NuttX requirements as described. While it provides a summary of the change, it lacks crucial details. Here's a breakdown:
Missing Information:

Summary:

Why is this change necessary?  What problem does it solve? Is the current division slow? Are there benchmarks showing the improvement? Just stating it "opens a fast version" isn't sufficient.
How exactly does it work?  What is HAVE_EXPRESSION_STATEMENT and how does its presence affect div_const? What code paths are changed? Provide a snippet if possible.
Related Issues: Are there any related NuttX or NuttX Apps issues this PR addresses?

Impact: The impact section is severely lacking. It only mentions the affected file.  Address all the impact points:

New/Changed Feature? Is this an optimization of an existing feature or a new one?
Impact on user, build, hardware, documentation, security, compatibility:  Even if the answer is "NO" for each, explicitly state it. If there is an impact, describe it.  For example, the build impact is likely YES as a new macro is introduced.

Testing:

Build Host(s): Be specific.  "CI" isn't enough. List the actual CI environments used (e.g., "GitHub Actions: Ubuntu Latest, macOS Latest").
Target(s): What architectures and boards were tested?  "sim", "RISC-V", "ARM" are too broad. Specify the specific simulator, RISC-V board, and ARM board.
Logs:  Providing "before" and "after" logs is crucial.  These should demonstrate the performance improvement or the corrected behavior.  Empty code blocks are unacceptable.

Example of Improved Summary & Impact:
## Summary

This PR introduces the `HAVE_EXPRESSION_STATEMENT` macro to enable a faster version of `div_const` when compiling with GCC.  Currently, the 64/32 division is inefficient [link to benchmark/profiling data if available]. This macro allows the compiler to optimize the division operation, resulting in a performance improvement [quantify the improvement if possible]. This change modifies the `div_const` function in `include/nuttx/lib/math32.h`.

## Impact

* Is new feature added? NO (optimization of existing feature)
* Is existing feature changed? YES (`div_const` performance improved)
* Impact on user: NO
* Impact on build: YES (new macro `HAVE_EXPRESSION_STATEMENT` needs to be handled by the build system)
* Impact on hardware: NO
* Impact on documentation: YES (documentation should be updated to explain the new macro and its impact on `div_const`)
* Impact on security: NO
* Impact on compatibility: NO
* Anything else to consider:  [mention anything relevant, e.g., potential impact on code size]

By providing the missing details and explicitly addressing each point in the template, the PR will be much clearer and easier to review, ultimately increasing the chances of it being accepted.


Add description for initializer dtype

Addressing #4897

Travis error looks unrelated.

What are "initializers" and "initializer", respectively?
I think there is no such thing as "dtype" argument in initializers.
Also, this describes how to use dtype from the caller's perspective. Instead, we should describe the requirements of an initiaizer. In other words we should mention the intrinsic dtype attribute of initializer objects.

@niboshi Edited accordingly. If this isn't what you're looking for, please provide example text for what you'd like it to say.
Thanks.

How about this? This is what I suggested previously (with some edit).

If an initializer object has dtype attribute, the initializer can assume that the array to feed the data into has that dtype. If the required dtype, depending on the context where the initializer is used, does not match the dtype attribute, Chainer will report an error.

"Base class" section is broken. Please fix it.
Also, some words have to be marked up correctly as identifiers: "dtype attribute" (there are two instances).

Admended. Please confirm. @niboshi

@niboshi Thanks! Done.

jenkins, test this please

Jenkins CI test (for commit 30cf44de08666b0ed1de8abc383de7fadf10d4c2, target branch master) succeeded!

Ok, now with double graves! Please merge, @niboshi .

jenkins, test this please

Jenkins CI test (for commit 7cd102cf5b8814cf35708cc0860c6922b7fc9745, target branch master) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

LGTM!


Use ContractScan for the deployment link

Links to ContractScan which shows the list of all/most deployments of the account factory.

Disclaimer: I am the author of the tool

🟡 Heimdall Review Status

Requirement
Status
More Info

Reviews

🟡
0/1

Denominator calculation

Show calculation

1 if user is bot
0

1 if user is external
0

From .codeflow.yml

1

Additional review requirements

Show calculation

Max
0

0

From CODEOWNERS

0

Global minimum
0

Max

1

1

1 if commit is unverified
0

Sum
1


update method signature of getAvailableIncludes and getDefauttIncludes

this pr is for small fix for this fatal error:

Declaration of Flugg\Responder\Transformers\Concerns\OverridesFractal::getAvailableIncludes() must be compatible with League\Fractal\TransformerAbstract::getAvailableIncludes(): array
Declaration of Flugg\Responder\Transformers\Concerns\OverridesFractal::getDefaultIncludes() must be compatible with League\Fractal\TransformerAbstract::getDefaultIncludes(): array

PR https://github.com/flugg/laravel-responder/pull/190  is better, covers more problems

Please merge pr #190


New command: riyaz new

It should:

help the course developer to understand where each file goes, and how the settings need to be specified
give user a project template to fill in the blanks or swap out pieces

Approach:

Add a template/ folder in sample-data/ that has files for chapters, lessons, authors, and course.yml
When user runs riyaz new {course-dir}:

Create directory {course-dir}, copy contents of template/ into {course-dir}/. If directory exists, an error should be raised.
Load {course-dir}/course.yml into memory, change value of course: {course-dir} to the title-fied version of {course-dir}.

The sample data loaded should have some demonstration of how to use the features provided by riyaz.

Wouldn't it be better to use cookie-cutter for this?
You can look at an example at:
https://github.com/pipalacademy/python-practice-problems/tree/main/cookiecutter-problem

It should:
* help the course developer to understand where each file goes, and how the settings need to be specified

* give user a project template to fill in the blanks or swap out pieces

Shouldn't the goal just be:

create a new empty course with all required files from a predefined template

Cookie cutter makes sense.

Shouldn't the goal just be:
create a new empty course with all required files from a predefined template

Yeah, okay 👍 . We don't need to make the template content serve as documentation.

Should we have cookiecutter as a dependency and run it from python when riyaz new is run, or let the user run cookiecutter on their own?


Math Domain Error

Using flatlib input:
date = Datetime('1990/06/11', '07:31', '-05:00')
pos = GeoPos('71N17', '157w47')
chart = Chart(date, pos, hsys='Whole Sign', IDs=const.LIST_OBJECTS)
mars = chart.get(const.MARS)
marshouse = chart.houses.getObjectHouse(mars)
print(marshouse)

I get output:
"Error: function terminated. Recommended action: inspect logs for termination reason. Details:
math domain error"

Date 1990/07/11 has the same result, but 1990/05/11 or 1990/08/11 produced the expected output.  Any ideas?
ty!

hi! Was this addressed/fixed?

It is still not fixed, reopening...

Just checked a little bit better this bug, and these are the kind of things that are hard to fix because of the high latitude. Basically, to compute PoF we need to know if this is a day or night chart. And at these latitudes, the function to compute the diurnal arcs fails. I'll leave this bug open, but I do not have an immediate solution for it..

TY.
Some questions:

is there a workaround?
Do you know at what LAT the failure occurs? Is it 'every high lat' or just certain lat/lon combos?
Why is it failing on one date, but not another for the same lat/lon?


Create new file from a list of customizable templates straight from the folder context menu

Description of the new feature / enhancement
Create new file or folder from a list of customizable templates straight from the folder context menu
Steps to create a new object

In a folder (incl. the Desktop) right click / bring up the context menu
From a new submenu "Create" (or some other name)
Select the template of your choice
After clicking, the new file/folder/object is copied from the template and file object is entered into rename mode

Steps to configure the list of templates

In a folder (incl. the Desktop) right click / bring up the context menu
From a new submenu "Create" (or some other name)
Click the "Open Templates" folder menu item

Other options

Remove or Replace or integrate the existing "New" submenu items
Prompt/wizard to help me customize the object to be created. e.g. add support for replacement of certain tokens in plain text - can be made more advanced over time with customizable wizards on create etc.

I implemented this many years ago, and love to implement/partner on a version as part of PowerToys. See also https://www.onegreatworld.com/Download/Windows/My Templates/
Scenario when this would be used?
When creating new objects, the existing New context menu is practically useless (sorry! :-) )

I can't easily configure the list of items
I can't configure multiple of the same object type
I can't template folders

Supporting information
https://www.onegreatworld.com/Download/Windows/My Templates/

Sounds like you want specific files in the  c:\Windows\ShellNew folder. Have you tried placing new files here? They should be used as the template when you click "New >" in the menu.

Hi Jay-o-Way,
Thank you for having a look!
AFAIK, I'm don't think you can customize the existing "New" menu without using the registry. e.g. you copy your file template into a folder that windows have access to, and then you add a FileName key under the .ext/... /ShellNew.
Note: Even when using the registry there are a couple of limitations of the existing "New" functionality:

You can't have multiple versions of the same file type, e.g. if you wanted multiple Word document.docx templates or Markdown.md templates you couldn't do that.

You can't template folders, e.g. let's say that you have a commonly used "New business template folder" that includes Outreach.docx, Presentation.pptx, etc. you wouldn't be able to template that folder and have all those nested objected copied as well.

With the proposed way, there will be a per user specific "New+ Template" folder, and users will be able to just drop files and folders into that folder. 2) And then when want to create a file or folder, they right click then we just render all those files in a "New+" context menu and then New+ will just copy the file/folder over to the destination folder. 3) Similarly, if they want to remove a template from the New+ menu, they just remove the file/folder from the "New+ Template" folder

Kind regards,
Christian
Here a screenshot of what I mean:

You've got a point there.

@Jay-o-Way I've created an early spec and made it available here https://docs.google.com/document/d/10R9UiUanGkTqnGfuBNuymJYlOhilfaKMextq86Slzh0/edit
Do you know if there might be interest in making something like this part of PowerToys or do you already have good signal that is likely not the case?
Either way, thanks again!
Best regards, Christian

A short video of a very early version of New+ running on my local dev box
https://github.com/microsoft/PowerToys/assets/14098831/ee523991-9b79-4dc2-8c97-7bd01e5b2133

I'd be curious to know what you have in mind for settings here.
I would say turning off/replacing the existing "New..." menu would be a non-goal for this work (can always be addressed elsewhere/separately if needed).
For me, the biggest open question is where the templates folder is kept & can the user change or specify the path? Essentially it sounds like we would be duplicating files/folders from that path into the current context/directory. Do we overcomplicate things for the user by having them need to think about where on their device a template is stored and should we instead have the settings for this feature contain some sort of "Upload" that can handle the addition of new templates and show all existing templates in the settings (so that the user doesn't need to navigate to some templates folder in File Explorer)?

Some user cans for consideration...

Users can enable/disable New+ (Off by default)
Users can reorder items in the New+ list
Users can add new items/templates to their New+ list of options
Users can remove items/templates from their New+ list of options
Users can edit existing items/templates in their New+ list of options
Users can specify an icon for each of the items/templates in their New+ list of options
Users can engage in renaming their newly created items directly after creation by New+

New+ should also only show up as an item if a right-click is made in File Explorer AND has no selection.

Some user cans for consideration...

Users can enable/disable New+ (Off by default)
Users can reorder items in the New+ list
Users can add new items/templates to their New+ list of options
Users can remove items/templates from their New+ list of options
Users can edit existing items/templates in their New+ list of options
Users can specify an icon for each of the items/templates in their New+ list of options
Users can engage in renaming their newly created items directly after creation by New+

New+ should also only show up as an item if a right-click is made in File Explorer AND has no selection.

Thanks a lot @ethanfangg! Really greatly appreciate your thoughts and feeback, and for sharing these use cases and previous thoughts.
I'm building out the initial spec here including your feedback. https://docs.google.com/document/d/10R9UiUanGkTqnGfuBNuymJYlOhilfaKMextq86Slzh0/edit (please let know if you can't access/comment etc. )
Re the templates/items themselves, they would by default be stored under the user's Document folder. And there would just be one copy of the template per user per machine.
One drawback with storing the templates as regular Windows Explorer items is that the user can't easily reorder items other than naming items alphabetically. We could potentially introduce a new folder view type or use could just name them 1. ... 2. .... and we can strip the numbering (as an example for Arabic numerals sorted language)
Happy to provide an offline demo etc. of a very early implementation. LMK.
All the best,
Christian

Latest version

@cgaarden I see you link to the onegreatworld website, but would you consider a GitHub repository? This would be very useful for people who want to look into how it works, and people who want to help you etc. You can also place docs/specs here, instead of Google.

@cgaarden I see you link to the onegreatworld website, but would you consider a GitHub repository? This would be very useful for people who want to look into how it works, and people who want to help you etc. You can also place docs/specs here, instead of Google.

@Jay-o-Way just to confirm. Are you suggesting that I create a new public GitHub repository with the latest code and spec before possible PowerToys repository inclusion? And then collab in the new separate repo until the spec and toy has been further developed? And then at that later point we consider inclusion in PowerToys?
Thank you!

@cgaarden Well, that is exactly what I thought of 🙂 But that's mostly because I don't know how or "where" you currently build your app, and how much it's finished. Or, how much it's compatible with PowerToys, if that's what you're going for. But there have been other new applets in the past that didn't have this either, so it's not required. It's all up to you.

Wondering why in the final implementation is chosen to not support Windows 10, aldo the tool linked in this issue works for Windows 7 and higher...

This issue has been worked on during the 0.85 development cycle. Please update to the latest release.

Wondering why in the final implementation is chosen to not support Windows 10, aldo the tool linked in this issue works for Windows 7 and higher...

Hi @jerone , This tool has been a community contribution, where it was chosen to use the new msix context menus for Windows 11, which are not supported in Windows 10. A community contribution to also support Windows 10 would be accepted. Hope this helps in understanding.


Updated Bundle

Bundle has been updated with most up to date docker image. Once this PR is approved, the kepler operator v0.4 should be completed. Note that the Operator sdk version in the bundle went from 1.25 to 1.23 after updating the bundle. Will this be an issue?

I just copy and pasted the icon into clusterserviceversion.yaml in bundle and config manifest. Not sure if there was a programatic way of doing it.

@husky-parul I noticed that github actions are still failing because of the username password missing. Is there anything that can be done to resolve that?

Add repository name
Add CREATED AT
Add channel as v1alpha1
Add quay.io image link under container image
Add https://github.com/sustainable-computing-io/kepler-operator in created at
Need Introduction, Documentation, Contributing and License information. Follow sample csv here https://operatorhub.io/operator/openshift-qiskit-operator

You can preivew the csv here https://operatorhub.io/preview

@husky-parul I noticed that github actions are still failing because of the username password missing. Is there anything that can be done to resolve that?

The checks have passed. Once the PR is merged the workflow will build and push the image.


added sync feature

Added a sync button to update the local DB with the updated information from the API.

will review this at a later date and open a new PR.


Protect Game Resources

Godot version:
3.0.4
OS/device including version:
Windows 10
Perhaps all supported platforms.
Issue description:
Protect the source code and resources of the game.
Currently it's easy to "decompile" / extract all resources including scripts.
Update: GD-Scripts are binary in pck-file. But the scenes are stored in plain text and editable.
Steps to reproduce:
e.g. Export as Windows Desktop (exe). Tested with the classic Godot (not mono)
You'll get following files:

YourGame.exe
YourGame.pck (package of all resources. Not encrypted.)

The same issue may affect all other platforms. Including Android and iOS. All assets and scripts are visible to anyone with a little knowledge of files.

I know you can encrypt the script bytecode by compiling godot from source with the SCRIPT_AES256_ENCRYPTION_KEY environment variable set during compilation: https://github.com/godotengine/godot/blob/8bf5b502223d48ee1b7b88ab8613be7bacbb63d2/core/SCsub#L30

Related to #8614.

BTW, I'm not sure why this is much of a problem. It's not exactly easy to extract a PCK if the person in question is not aware of Godot. And a dedicated person would be able to extract them no matter what you do..

And even if you are able to extract them, all the scripts are (at least by default) compiled, and any scene files are converted to .scn which itself is binary.
Not only that, I'm not sure why it matters so much that people can access the files. Anyone who wants to cheat will cheat, and anyone who wants to mod a game will mod. After they download the game, that copy is theirs to screw around with, and generally do whatever with (within the bounds of fair use, of course). When they do that, it's their own fault if the game screws up, or if they get banned from that game (if online) for cheating -- not the dev's fault.

Relax 😄 What's up guys? Just reported the fact that the pck file displays the source code as plain text and the most resources are extractable. For example with the Dragon Unpacker Hyper Ripper.
For me it's not a problem. Currently I'm switching to another engine. This is not the single reason.
But whatever ... this does not change the fact that the developers sources are insecure. For an open source project, it's ok. But not for proprietary projects.
I do not understand some amusing reactions about this issue. You can view the file with a hex-editor like HxD and scroll the file or search for source code. And some binary or textures (e.g. png).
In my opinion it's not the worst idea to protect this data. It's not a trivial issue, I know. But for commercial projects this could be a KO issue. - Have a nice day. :)

the pck file displays the source code as plain text

I just checked. Exported .tscn (scene) files are indeed plain text, but .gd (GDscript) files are converted to binary. Sure you can read how a scene is constructed, but to read the source code you'd have to decompile it -- which is a pain to do and most people don't seem to want to go through that.

most resources are extractable. For example with the Dragon Unpacker Hyper Ripper.

Resources exported by Unreal Engine, Game Maker Studio, and Unity are also all extractable. Also, just tried this Hyper Ripper myself, wasn't able to extract any scene or source code files. I could extract my application's icon (the only graphic I had in my project) however -- but people have been extracting sprites, models, textures, and sounds from games for years.

The exported project is just a runner for the sources package.

So are Game Maker Studio projects. I believe Unreal Engine works the same too? Unsure about Unity.

Ok, you're right. My fault. The .gd scripts are not stored in plain text. Sorry.
But in my opinion there are too much plain text in this file. And the assets too.
btw. the Hyper Ripper only extract known formats like png. It can't extract Godot formats like scenes or scripts.
I checked this again. In my case I have a script with a lot of exports to adjust parameters of a node.
It looked like script sources. 😄  Now I see it's the scene indeed.
But I be able to change some parameters, store the file (pck) and see the results. It works. You can change anything in the scene including parameters provided from scripts.
Ok, you can say, I don't care what people do with my software. Change the sources whatever. They could use a Trainer (cheating). It's not completely wrong. In multiplayer games, the developer must never trust the client. Whatever ...
I have a bad feeling about these public sources in the package file. The editiable scenes in exported projects. It's like ini-files or settings you never provided.
I just wanted to report this issue after viewed the file and found some plain text ressources like the scenes and other resources like meshes (e.g. PoolVector3Array) and other assets.
I think it's ok for amateur projects. Maybe for a tiny indie-game too. And AAA-Studios maybe never use Godot. Unfortunately. But in my opinion it's not a bad idea to protect as much as possible. Why to provide editable sources? I think this is a topic for proprietary commercial projects.
You can close this issue if you want. Or focus to the open scenes and resources. And protect it by obfuscating, hashes or an encryption. Just to reduce the hackers. Nothing is safe. That's right. But is this a reason to give up? The gaming industry should remove DRM and other protecting software, because people will hack this. Right? 😄

3.1 will have an experimental option to convert all text resources to
binary on export
On Wed, Jun 27, 2018 at 8:46 PM Dominik Geng<EMAIL_ADDRESS>wrote:

Ok, you're right. My fault. The .gd scripts are not stored in plain text.
Sorry.
But in my opinion there are too much plain text in this file. And the
assets too.
btw. the Hyper Ripper only extract known formats like png. It can't
extract Godot formats like scenes or scripts.
I checked this again. In my case I have a script with a lot of exports to
adjust parameters of a node.
It looked like script sources. 😄 Now I see it's the scene indeed.
But I be able to change some parameters, store the file (pck) and see the
results. It works. You can change anything in the scene including
parameters provided from scripts.
Ok, you can say, I don't care what people do with my software. Change the
sources whatever. They could use a Trainer (cheating). It's not completely
wrong. In multiplayer games, the developer must never trust the client.
Whatever ...
I have a bad feeling about these public sources in the package file. The
editiable scenes in exported projects. It's like ini-files or settings you
never provided.
I just wanted to report this issue after viewed the file and found some
plain text ressources like the scenes and other resources like meshes (e.g.
PoolVector3Array) and other assets.
I think it's ok for amateur projects. Maybe for a tiny indie-game too. And
AAA-Studios maybe never use Godot. Unfortunately. But in my opinion it's
not a bad idea to protect as much as possible. Why to provide editable
sources? I think this is a topic for proprietary commercial projects.
You can close this issue if you want. Or focus to the open scenes and
resources. And protect it by obfuscating, hashes or an encryption. Just to
reduce the hackers. Nothing is safe. That's right. But is this a reason to
give up? The gaming industry should remove DRM and other protecting
software, because people will hack this. Right? 😄
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/19790#issuecomment-400865122,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z21AgTVlYBM0NTg6I0lT8cSYBDStZks5uBBlCgaJpZM4U4iQ1
.

If you're worried about piracy, this kind of obfuscation won't help you. You'd need some actual DRM protection, which Godot won't provide (since there are many ways to do it, and it's not a vital thing). So you need a third-party tool for this.
If you're worried about copyright, the law enforcement might be a better bet than trying to obfuscate the assets in your game.
It's not a matter of open/closed project. Any dedicated person will be able to extract the assets and change the base game. Some companies use this to their advantage (see Counter-Strike and Team Fortress for example). The mod scene is quite big and many players might buy your game just to apply the mods.
Suppose we provide support for encrypting the assets. The key must be stored in the executable, since they need to be unencrypted to be used by the game. Considering that Godot is open source, anyone can check out what is the encryption algorithm and where such key is stored. With that information wouldn't be so hard to break through the encryption anyway. You could change the Godot source to modify the algorithm/key location (but then, you already can change the source to add this).
I agree with binary conversion (and that was always in the plan) since it makes everything faster to load. But in terms of protection there's not that much gain, only a false sense of security. Obfuscation is not really security.
I think this is an overreaction and other engines might not offer that much protection you're thinking they do.

That's absolutely correct. But this is not exactly my intention. I just don't like editable plain text in built applications. For example the scenes. As shown on the gif of my previous post.
It's not alone a security or privacy question. It's more about preventability. How obfuscation keeps some of the people away.
But as reduz wrote, 3.1 will bring some changes? Is this documented? Any links? Thanks.
If I am still alone with this opinion, it is regrettable. But maybe I'm not able to articulate correctly in english. I see many confused- and laugh- emoji without a text message about the reason. That's pity.
Godot is a great game engine. I still think that this is a important issue.
I've created this ticket with good intention to improve this software. I never thought that this topic met with so much opposition. I'm a bit disappointed. 😄 It's ok, I'm alone with this opinion.
But this issue and conversation helped me alot for my decisions. Thanks for it.

@domske I don't think that it is that it is not wanted, but rather that this is as impossible topic.  If the resources have to be used, textures unpacked into memory, models transferred to the GPU, etc... etc... then there are always ways to get at the information since it is already local.  The top end DRMs for games run the entire things in virtual machines with layers upon layers of obfuscation and slowdown and they still get broken.  It is an impossible thing if the resources are needed to be used at all.

I know. This is a never ending story. A cold war between a publisher and pirates. 😄 But a necessity.
Whatever.. This is too much. I never wanted such highly (impossible) security. Just a prevention of publishing editable plaintext that could be prevented. Just step by step. For the first step focus the scenes and the fact that everyone can edit it in a simplest way. It's not my intention to prevent all modifications or asset extractions by hackers. Because this is impossible. But you can reduce the amount of them. For example publishing a game in javascript in best clean code with comments vs obfuscated code. That the only reason to obfuscate javascript. To reduce the potentially pirates. The same with the Godot pck-file. The guys who reads the file and found nothing than ugly binary code will close the file again in a disappointed way. Of course, a small percentage will continue and see it as a challenge. 😸

That the only reason to obfuscate javascript. To reduce the potentially pirates.

Not really. "Uglifying" the code is used to make it more compact, reducing download time / bandwidth usage. It's not very hard to undo it (just put in a code formatter, like the browser console provides).

How to hide the CMD window on published games?

It is not shown if you exported without debug.

we can't protect our project perfectly but want to prevent to hack in 2 minutes without special knowledge.
I think that's a reasonable request.

Sure, and using binary resources would be enough for that. I don't know why @domske has such a pessimistic view considering that's already established that this will be done.

@volzhs Thanks, you understood me. 😃
@vnen Of course "minifying" is for download and bandwidth optimization. But also for obfuscating. That's why the word "uglifying" is also used for this. There are different modes to do this. A strong mode removes variable names and comments, etc.
You mean that there is no difference between clean-code and uglified code for hackers? In both cases the amount of hackers is equal? Regardless of formatters, uglified code does not only remove all comments, spaces and the pretty format. It also replaces variable names and in best case it ofuscate the context.
This code is definitely less attractive to hackers. It tooks longer and need more reverse engineering. And sometimes a reason to give up. What's why uglifying exists, too. Not only for download reason.
It's about hacking in 2 minutes without special knowledge and effort.
Do not see this in an extreme contrast. "Secured": YES or NO. It's more about "Secured:" NO, ABIT, BETTER, ALMOST, RESERVED. And the pck-file has the state ABIT and the goal is BETTER. 😄
Just improve it step by step. At first focus the open scenes in the pck-file.
There are many opportunities to do this. I can't say that's the best solution. I just don't like editable plain-text in published binary files like the pck-file.
It's a bad attitude to just leave it open just because you think it's going to be hacked anyway. If so, we could just build a runner for the project sources and publish this to the customers. No offense.
I have no pessimistic view on this. I don't know the changes of Godot in the future. I only reported the actual state. That we all can consider about this issue. If this is already established, that's great.
I don't care about to close this ticket for the reason: "we will work on it, but not especially on this issue. It will implemented successively." Fine :) My intention was just to draw attention to this. And discuss options.

I'm closing this since DRM won't be added and conversion to binary is already tracked by #12452.

I don’t know how we pointed to DRM. 😄 Maybe my fault. This wasn't the topic. Ok, I missed this ticket #12452. It's related to this issue, that’s correct. What a misunderstanding of my ticket. ^^ Ok, not for all. Whatever ... I‘m look forward to version 3.1.

I agree with @domske on this one. I've run some tests, and:

Making my local variables' names longer made the exported .gdc files larger
Adding more comments made the exported .gdc files larger

These are both very surprising. I don't understand why local variable names (variables that are defined within the body of a function) affect the size of exported .gdc files. Comments affecting the size of exported .gdc files is even more mind boggling. I'll get back to these, but first:
A hacker will hack, yes. I used to hack games when I was 14, to make heroes invincible, levels impossible to fail, etc. But the code I hacked was never in plain sight in front of me; the variable names were never visible, and I never saw the name of a single function. All that was visible to me was machine code -- disassembled assembly code to be precise -- with addresses for variables. So, I wouldn't have been able to extract any of that code and repurpose it without spending more time than it's worth (not that I would have regardless, but there are lots of people in the world who would). Even if some of you disagree, I see a lot of value in having the same level of "protection by obfuscation" in the commercial products that I create. In other words, I don't want my algorithms to be visible in their original form after Godot loads them into memory. Some of you say converting to binary is enough for that, but I disagree. Even if it Godot weren't open source (which makes the dissection of any generated binaries much easier), my 14 year old self would have been able to see all that content using a basic debugger. I sure won't be releasing any commercial software that allows easy access to my class, function, and variable names, let alone my comments. And I also respectfully disagree with the "law enforcement might be a better bet" comment. What indie developer has the time or resources to sue someone across the world? Would you even have the time to dissect competitors' code to see if they stole from you? Even if you were to win a lawsuit, what do you think are your chances to successfully collect the money for the damages from an individual on the other side of the world? Prevention is your best bet.
This is not about whether someone can hack and modify your code. It's about protecting your algorithms and intellectual property, such that the time and effort required to reverse engineer would be so high that the people with that skill wouldn't waste their time trying to understand your obfuscated code. In other words, make it difficult enough so that the people who have the skill would find it easier to build their own thing than to steal yours, and the ones without the skill would be limited to minor hacks. Such minor hacks are fun for the kids who can do them anyway, and have no financial harm to the developer as long as they don't distribute a version of your code that bypasses its copy-protection mechanisms. And pushing competitors to come up with their own algorithms is better for everyone, including users; more ideas and more innovation results in more interesting products.
Acknowledging the problem is the first step. Once the problem is acknowledged:

The solution for comments should be simple. They simply should not be in an exported binary.

The solution for local variable names should be simple too. Unless Godot allows dynamic lookup of them, they simply shouldn't be in an exported binary either. If Godot does allow dynamic lookup of local variables, then they can be treated under the same umbrella as other identifiers that I'll cover in item 3 below .

The solution for dynamically accessible identifiers such as nodes, functions, and member variables isn't simple at all, and cannot be done without limiting some of the dynamic features of gdscript. For example, the fact that you can form a string at runtime (e.g. by concatenating two others) and look up a node instance via the string you've just created means that Godot cannot simply obfuscate node names without breaking that dynamic feature (I believe similar dynamic lookup features exist for functions and member variables too, so the same applies to them). However, smart solutions can easily be created that both obfuscate and are able to handle such cases. For example, identifiers could be obfuscated by default, on export, but with the ability to bypass the obfuscator for specific identifiers when desired (e.g. those that start with a specific, reserved prefix).

The solutions can be discussed, but the first step is to acknowledge the problem with a more open mind, without dismissing the problem and deferring it to law enforcement.
P.S. As a bonus, obfuscating identifier names should also help improve performance. For identifiers that don't need to be looked up by dynamically created strings (i.e. most identifiers), making their names as short as possible should speed up any code that does look up things by name (i.e. gdscript and Godot in general).

Good news; I did another test this morning, and it seems the size increase due to comments is just a bug. From what I can see, comments are always converted to the same binary content; when I change the content of the comments without changing their size, the created .gdc file remains the same, which means the information within the comments is lost. So scrap item 1 from my list for the purposes of this discussion, as it needs to be a different issue entry on its own.
Item 3 remains.

@domske

Relax 😄 What's up guys? Just reported the fact that the pck file displays the source code as plain text and the most resources are extractable. For example with the Dragon Unpacker Hyper Ripper.
For me it's not a problem. Currently I'm switching to another engine. This is not the single reason.
But whatever ... this does not change the fact that the developers sources are insecure. For an open source project, it's ok. But not for proprietary projects.
I do not understand some amusing reactions about this issue. You can view the file with a hex-editor like HxD and scroll the file or search for source code. And some binary or textures (e.g. png).
In my opinion it's not the worst idea to protect this data. It's not a trivial issue, I know. But for commercial projects this could be a KO issue. The exported project is just a runner for the sources package.
Have a nice day. :)

i just tested this actually.  i opened my .pck file in visual studio code, and it had over 100k lines, but i searched for a variable that i had in my script, and the entire script was in there lol
maybe once this is merged, and you do compiled under script, this solution and #24716 will be fixed?

Game asset sites also require their assets to be sufficiently protected when incorporated into an app. So we are legally required to provide reasonable protection of any purchased assets we use.
I still don't understand why this is issue is closed, and disagree with the cheered obfuscation is not protection, etc. comments above. By the same logic, do you not put locks on your house doors because people who are willing enough will find a way in anyway? I don't know about you, but I like to lock my doors, and I am legally required to do so for the assets I use.
In summary:

Assets need to be reasonably protected
Scripts need to be obfuscated (see item 3 in my earlier list above)

By the same logic, do you not put locks on your house doors because people who are willing enough will find a way in anyway? I don't know about you, but I like to lock my doors, and I am legally required to do so for the assets I use.

I don't think the comparison with locks makes sense, as locks are not necessarily unfair (you could see it as a parallel to encrypting your own files for privacy's sake).

Wondering why this has been closed. As far as I understand it, #24582 only partially fixes this issue by only encrypting scripts. Scenes and assets are still not encrypted - this is not strictly a matter of DRM, as @ArdaE has pointed out this could prevent the use of a good number of assets (even some otherwise freely available ones) with Godot. This isn't DRM, merely allowing Godot to extend legally-required protections when necessary.

FWIW, I agree 100% with ArdaE: assets need to be reasonably protected & scripts need to be obfuscated. I've been evaluating Godot, and this is a deal-killer for me personally. I am only writing this because business owners (like restaurants, for example) rarely get feedback that tells them why people are not coming. They'll get negative feedback from existing customers, but they never find out why people don't come in the first place.
One thing that is often mentioned about Godot is that it doesn't have as 'sexy' a gallery of Godot games that some other engines have. I'm not going to say it's all because of obfuscation, but serious game developers and artists -- the kind who create games that make great videos -- do care about it.
I hate to write a negative comment, because there is really a lot to praise about Godot. I would love to use it. But if "locking my front door" is next to impossible, I simply can't.

I've been evaluating Godot, and this is a deal-killer for me personally.

Do other game engines have any built-it "assets protection"?

BTW, I'm not sure why this is much of a problem. It's not exactly easy to extract a PCK if the person in question is not aware of Godot. And a dedicated person would be able to extract them no matter what you do..

It's about raising stealing cost. higher costs means more secure. every password in the world CAN be break, it is all about how much it cost.

And even if you are able to extract them, all the scripts are (at least by default) compiled, and any scene files are converted to .scn which itself is binary.
Not only that, I'm not sure why it matters that people can access the files. Anyone who wants to cheat will cheat, and anyone who wants to mod a game will mod. After they download the game, that copy is theirs to screw around with, and generally do whatever with (within the bounds of fair use, of course). When they do that, it's their own fault if the game screws up, or if they get banned from that game (if online) for cheating -- not the dev's fault.

it's like locks, any lock can be broken, it's all about how much the stealler wish to pay. house owner wish to raise the costs, surely.

You seen keen to tell us that a lock would be effective. Tell us, then: How would we implement such a lock?
If we implement it into the engine, then people can examine the source code (which is far easier than you're making it out to be) and figure out how we store the key, how to access it, and what encryption method(s) we use. You might call it hard to crack, but I guarantee you that if we were to implement it into the engine proper, it wouldn't take long before someone copies the decryption code from the engine and uses it for a "Godot game decrypter"... making the lock effectively useless because we offer the keys for free
You are much better off learning C++ and implementing your own asset protection system, than us doing the work for you. Then, they would have to figure it out from scratch because the code wouldn't be public.

In my country there is a saying "Locks from kind people". Any lock can be broken, it's a matter of time, but every house has a lock. The fact is that the better the lock, the longer it takes to break it, and the time and effort may not pay off. Therefore, thieves are more likely to steal things from poor houses with bad locks than from rich houses with good locks.
If you implement the ability to encrypt all files as scripts are encrypted now, users will be grateful. And in general, if this is a project and with the source code someone can make such a plug-in for Godot and all that remains is to implement it into the project. I would, but unfortunately I don't know C ++.

@dbrova15

The fact is that the better the lock, the longer it takes to break it, and the time and effort may not pay off.

This may hold true in the real world, where you don't see the locks and tumblers within a real lock, making it hard to know how to lock-pick -- but Godot is open-source. That doesn't work here.
That's a problem, because it's the equivalent of a lock being transparent -- if you can see how far you need to push each pin up, you don't need to worry about whether you've pushed a pin too far up or not. You know the key to that door now -- you can just create your own copy of the key.
See, Godot being open-source means that if Godot were to include some sort of resource protection, it would need to have that code included with the engine's source code. The source code being openly available (you don't even need to login to GitHub to download a copy), means that anybody could search the code-base for the part that encrypts/decrypts resources, and reverse engineer it. Then they can use that knowledge to build a program that will automatically search out the encryption key -- and use that to decrypt the resources.
All that might sound hard, but I'll be honest: It's really not. It's really, genuinely, not that hard. Honestly, it's harder coming up with how those locks would work, than it is to break the locks. And even if it were hard to break the locks, all it takes is one person offering that decryption program to the world -- now everyone could buy a machine that looks through those transparent locks, and spits out a key that can be used to open that lock.
I'm not trying to say that we aren't sympathetic to developers wanting to protect their game's resources -- we are! But we can't include resource protection within the engine, unless game developers are okay with the drawback that someone could access the game's resources anyways... and I think that drawback effectively negates including resource protection in the first place.

@dbrova15
See #38308 for the full PCK encryption implementation.

Security through obscurity is not security.
And as for PCK, someone could just extract the key as it can't be encrypted to be able to be used.

I think this issue goes beyond security, it is also about keeping original source code as secret.
An example of what I mean is this: I can drink a Coca-cola and it will taste exactly like the original formula (running compiled game), I can even see ingredients in its label (reading the "decompiled" code), but my tongue cannot "decompile" proportions and recipe. They are kept as secret (original source code) and no matter what, the only way I could get that information is by Cocacola company releasing it.
Having comments removed from the compiled code do basically nothing to protect original source code. Well-written source codes are designed to be as readable as possible by programmers, this mean having a good architectural design of classes, variables and methods, and of course, names of those things, because they need to be declarative enough. A well-written source code will be substantially easier to read and understand even without comments provided you have the original variables, classes and method names.
Source codes have intangible value for companies. Released commercial games source codes are extremely rare, because almost always they are kept as secret. So actually releasing a compiled game where its code is basically the source code without comments, is almost the same as releasing the source code, with the barrier of the decompiling task.
The main problem with godot, and I think its the main reason on why main Godot engine developers refuses to implement some kind of obfuscation, is because all variables and methods can be referenced with strings, and that feature surely breaks with a compiler that mangles function and variables names.

BTW, I'm not sure why this is much of a problem. It's not exactly easy to extract a PCK if the person in question is not aware of Godot. And a dedicated person would be able to extract them no matter what you do..

I am very disappointed with your answers. This is a very irresponsible behavior. You believe that there will be thieves invading your home, so it doesn't matter if there is no lock, because locks cannot prevent others from invading.
Because if a thief does this, there is naturally a law that will sanction the thief, so it is only necessary to simply close the door.
What I want to say is that open source is not always a good thing, it will let everyone see the bottom of the code, but we individual developers do not necessarily need these things, because the bottom source code is not what I pursue. Optimizing the engine is your job, and developers should focus on developing and designing games. Because we don't have that much time to modify the underlying code and test it. Because it wastes three months or even a year.
I have returned to C++, DX11, and bullet, and have used the compilation method to create games, because your engine is very unsafe, and it cannot allow me to devote myself to making games.
Even if you think that C++compiled games can be decompiled, even if you think that confused code can be seen by professional people about its functionality and design, it's better than completely exposing yourself.
In addition, what we need is not DRM, but rather not allowing bad people to easily obtain the source code and modify it to make money by re creating new games, which is an insult to the original author. They robbed the author of his wisdom. If GODOT cannot be protected, why should I use it? If I were to create an engine, I would never open its source code.

It's not exactly "easy" to obtain source code because PCK contrary to what you seem to think does NOT contain pure source code (at least not in 3.x).
If you state that Godot can not be protected, well, the same applies to UE4 and Unity... therefore no engine is "responsible" or "safe"

It's not exactly "easy" to obtain source code because PCK contrary to what you seem to think does NOT contain pure source code (at least not in 3.x).
If you state that Godot can not be protected, well, the same applies to UE4 and Unity... therefore no engine is "responsible" or "safe" to you

Why do you mention unity and ue, which are related to this matter?
And Unity has always made efforts to avoid extraction, but you should know that while using scripts is convenient because you don't need to compile your code over and over again, "interpretive" games are the least secure. No matter how hard you try to guard against all this.
I don't think it can be compared to Godot, which does nothing at all.
If you think that Unity is also bad, so it's entirely possible for Godot not to add anti-theft measures, and this theory of who is worse is untenable. Bad places should not be learned from others, otherwise Godot has no advantage. Free+open source is no longer an advantage.
I never said I would use them to make games in the future. I always said I would use my own engine, so if Godot fails to meet people's expectations, it's natural to be abandoned. No one would risk using a very unsafe engine just to speed up game development.
This also includes RPG Maker and Pixel Game Maker MVs, which I have purchased, but I never use to develop games. There is also a game built by GM8.0. I once extracted it and placed the modified file in the original directory. It actually ran successfully! This is terrible.
Ue4 and godot are the same open source engines, and I won't use tools that tell bad guys about the internal structure of locks in their homes.
I have wasted a year researching the use of Godot. Currently, I only use it to write or test some game features. If I want to actually create or release a game, it will not be adopted at all.


Xcode 8 beta 6 compatibility

@JohnSundell I converted the swift 3 codebase to the latest version (shipped with beta 6) on a local branch but I'm not sure how to handle the test migration.
Due to https://github.com/apple/swift-evolution/blob/master/proposals/0072-eliminate-implicit-bridging-conversions.md it's harder to create [String: AnyObject] with array literals because things like 47 are not implicitly convertible to NSNumber anymore. This is something that was looked into before implementing the proposal: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160502/016644.html
So for instance, the following code will not compile: https://github.com/JohnSundell/Unbox/blob/swift3/Tests/UnboxTests.swift#L93-L96 because 27 cannot be implicitly converted to AnyObject (here: NSNumber).
I'm not too sure how to work around this without a massive amount of boilerplate, like this:
[
    "dictionary" : [
        "FAIL" : 59 as NSNumber
    ] as [String: AnyObject]
]

Any idea?

Hi @ldiqual, Unbox has now been updated to support Xcode 8 beta 6! :) UnboxableDictionary is now typed as [String : Any] to conform to the new implicit conversion rules.


application/jose+json is not supported

What is not working as expected?
After integrating notation-go into a Kubernetes admission controller and trying to verify the image ghcr.io/deislabs/ratify/notary-image:signed, verification fails with unable to parse the digital signature, error : signature envelope format with media type \"application/jose+json\" is not supported.
What did you expect to happen?
notation-go should support signature formats it itself created and successfully validate the signature of ghcr.io/deislabs/ratify/notary-image:signed.
How can we reproduce it?
The image ghcr.io/deislabs/ratify/notary-image:signed was used and for verification the following certificate:
-----BEGIN CERTIFICATE-----
MIIDQzCCAiugAwIBAgIUDxHQ9JxxmnrLWTA5rAtIZCzY8mMwDQYJKoZIhvcNAQEL
BQAwKTEPMA0GA1UECgwGUmF0aWZ5MRYwFAYDVQQDDA1SYXRpZnkgU2FtcGxlMB4X
DTIzMDYyOTA1MjgzMloXDTMzMDYyNjA1MjgzMlowKTEPMA0GA1UECgwGUmF0aWZ5
MRYwFAYDVQQDDA1SYXRpZnkgU2FtcGxlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEAshmsL2VM9ojhgTVUUuEsZro9jfI27VKZJ4naWSHJihmOki7IoZS8
3/3ATpkE1lGbduJ77M9UxQbEW1PnESB0bWtMQtjIbser3mFCn15yz4nBXiTIu/K4
FYv6HVdc6/cds3jgfEFNw/8RVMBUGNUiSEWa1lV1zDM2v/8GekUr6SNvMyqtY8oo
ItwxfUvlhgMNlLgd96mVnnPVLmPkCmXFN9iBMhSce6sn6P9oDIB+pr1ZpE4F5bwa
gRBg2tWN3Tz9H/z2a51Xbn7hCT5OLBRlkorHJl2HKKRoXz1hBgR8xOL+zRySH9Qo
3yx6WvluYDNfVbCREzKJf9fFiQeVe0EJOwIDAQABo2MwYTAdBgNVHQ4EFgQUKzci
EKCDwPBn4I1YZ+sDdnxEir4wHwYDVR0jBBgwFoAUKzciEKCDwPBn4I1YZ+sDdnxE
ir4wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAgQwDQYJKoZIhvcNAQEL
BQADggEBAGh6duwc1MvV+PUYvIkDfgj158KtYX+bv4PmcV/aemQUoArqM1ECYFjt
BlBVmTRJA0lijU5I0oZje80zW7P8M8pra0BM6x3cPnh/oZGrsuMizd4h5b5TnwuJ
hRvKFFUVeHn9kORbyQwRQ5SpL8cRGyYp+T6ncEmo0jdIOM5dgfdhwHgb+i3TejcF
90sUs65zovUjv1wa11SqOdu12cCj/MYp+H8j2lpaLL2t0cbFJlBY6DNJgxr5qync
cz8gbXrZmNbzC7W5QK5J7fcx6tlffOpt5cm427f9NiK2tira50HU7gC3HJkbiSTp
Xw10iXXMZzSbQ0/Hj2BF4B40WfAkgRg=
-----END CERTIFICATE-----

Here a shortened version of the code I used (took away lots of the error handling and surrounding logic):
type NotaryV2Validator struct {
	Name        string
	Type        string
	Trust_store truststore.X509TrustStore
}

func (nv2v *NotaryV2Validator) ValidateImage(
	ctx context.Context,
	image name.Reference,
	args policy.RuleOptions,
) (string, error) {
	img, tag := transformImage(image)

	policy, _ := nv2v.setUpTrustPolicy(img, args)
	ver, _ := verifier.New(policy, nv2v.Trust_store, nil)
	remoteRepo, _ := remote.NewRepository(img + tag)
	remoteRegistry := registry.NewRepository(remoteRepo)

	desc, _ := remoteRegistry.Resolve(nv2_ctx, img+tag)
	exampleVerifyOptions := notation.VerifyOptions{
		ArtifactReference:    fmt.Sprintf("%s@%s", img, desc.Digest.String()),
		MaxSignatureAttempts: 50,
	}

	digest, _, err := notation.Verify(nv2_ctx, ver, remoteRegistry, exampleVerifyOptions)
	if err != nil {
		return "", fmt.Errorf("failed to verify image: %s", err)
	}

	return string(digest.Digest), nil
}

func (nv2v *NotaryV2Validator) setUpTrustPolicy(
	image string,
	args policy.RuleOptions,
) (*trustpolicy.Document, error) {
	// imtr stores the certificates to be used for verification
	imtr := nv2v.Trust_store.(*InMemoryTrustStore)
	// selects all the certificates to use, by an overarching policy
	trs, _ := common.GetTrustRoots(args.TrustRoots, imtr.trust_roots, true)

	return &trustpolicy.Document{
		Version: "1.0",
		TrustPolicies: []trustpolicy.TrustPolicy{
			{
				Name:           "default",
				RegistryScopes: []string{image},
				SignatureVerification: trustpolicy.SignatureVerification{
					VerificationLevel: trustpolicy.LevelStrict.Name,
				},
				TrustStores: utils.Map[common.TrustRoot, string](
					trs,
					func(tr common.TrustRoot) string {
						return fmt.Sprintf("ca:%s", tr.Name)
					},
				),
				TrustedIdentities: []string{"*"},
			},
		},
	}, nil
}

Describe your environment
Running inside a distroless/base-debian11 container. The code was compiled to a binary using Go 1.19. The container itself is running inside a Kubernetes cluster as an admission controller.
What is the version of your notation-go Library?
github.com/notaryproject/notation-go v1.0.0

@phbelitz I'm trying to troubleshoot the issue, could you try adding a single line in your code under import:
_ "github.com/notaryproject/notation-core-go/signature/jws".
(Note the underscore _ at the front of the import, since you are not using jws package, it won't stand without this underscore.) Let's see if this could fix the issue. Thanks.

@Two-Hearts yes, awesome. I guess i should import the _ "github.com/notaryproject/notation-core-go/signature/cose" aswell? Wasn't aware that those are necessary. Closing this as the problem was fixed.

I guess i should import the _ "github.com/notaryproject/notation-core-go/signature/cose" aswell?

@phbelitz Yes, since we support two signature formats at the moment. You should also import COSE as well.


Only false and nil should be considered false

Consider this Mirah code
if 0
	puts "true"
else
	puts "false"
end

According to Ruby semantics, this code should print true.
According to Java semantics, this code should not compile.
It is expected that this could should print true (as Mirah should feel like Ruby as much as possible).
However, it actually prints false.

Interesting. I would have expected it to be true as well. Are you looking at it?

The culprit seems to be here: https://github.com/mirah/mirah/blob/0af6095bb85e63d4dd190b9eeb7e1c9dca6209e8/src/org/mirah/jvm/compiler/condition_compiler.mirah#L64
The question is: What should happen in case of
if somemethod
	puts "true"
else
	puts "false"
end

def somemethod
	puts "somemethod called"
	0
end

Should it output just
somemethod called
true

or just
true

?

Maybe we should emit a warning if the result of if a is that a is always considered true due to it being a type being promoted to true regardless of value, because these statements make no sense if the type of the expression is actually known. They make only sense if the type of the expression is unknown, which may only happen when type inference is so strong that types are not so visible anymore. But maybe we should not emit a warning.

Fixed.
No warnings are emitted.
Code in conditionals is always executed, even if the result is known in advance.


Interrupt service routines for Timers?

This might be well beyond the scope of this library, but I was wondering if there is a way to set interrupt service routines for Timer match events. Looking at the BCM2711 and BCM2835 datasheets, it looks like there are 4 system timers which can fire interrupts on match, but I'm not sure how to determine which ones the operating system is using. Is that something that could fall under the scope of this library, or should it be a separate project?

Thanks for the suggestion! While i'm not against adding support for timer interrupts, this might be better suited as a separate project. One of the goals I have for RPPAL is easy peripheral access without needing superuser privileges. As far as I'm aware, the only registers we can access directly without those privileges are in the GPIO section. I'm only somewhat familiar with (user space) timer interrupt support in Linux. I imagine there are already other crates out there that offer support for those.
If you come across anything that would make these a better fit for RPPAL I'd love to hear it though and we can revisit this.


Collision avoidance pos ctrl

Use range sensor measurements to avoid collisions in position control mode.
Range sensor data is read in and transformed into velocity constraints which prevent the vehicle from approaching any obstacle closer than a specified distance. Only the velocity component towards the obstacles is limited all other velocity components of the stick input are preserved and executed.
SITL tested, flight tests pending

This is awesome! Do we want this directly in the position controller though? Can't we split this same functionality into a separate low-level avoidance module like I did in my branch?
This also allows slightly more advanced behaviour in the future if we decide to implement it. (e.g reciprocal velocity avoidance).

Link to branch : https://github.com/mhkabir/Firmware/commit/883ed610639b6b68a1acbf04e45fabe3aa255841

I've been wanting to create an automotive radar based distance sensor. This would be a great combo.

I still don't understand why this needs to live within the multicopter position controller, as a library or otherwise.
Can't obstacle avoidance be it's own module, which  in the future is extended, for example, with a more complex algorithm, or avoidance for geofences, or even with strategies for fixedwings sharing airspace with manned aviation. Building the avoidance module with a similar module structure as land detector at this stage would future proof it and avoid adding crud to the position controller, most of which was cleaned up thanks to FlightTasks.
Fundamentally, I think avoidance has no place inside the controller.
@dagar @LorenzMeier @bkueng thoughts?

Can't obstacle avoidance be it's own module, which in the future is extended, for example, with a more complex algorithm

I thought that was one of the main intentions with FlightTasks, to provide a clean interface for things like obstacle avoidance?

As a start simply keeping the avoidance logic in a separate library could help work towards being a full module applicable to all vehicle types.

we flew some tests today, it behaved reasonable. It was a bit wobbly, and also it sometimes approached a bit closer than the specified distance, if the throttle was kept towards the obstacle. This might also be due to the sensor measurements. We used the local_planner to generate range data form 3 realsense cameras. I did not have time yet to look at the logs but I put them here for anyone who is interested:
Here a  video, where the drone was commanded to go forward at full throttle towards the tree, max velocity was set to 5m/s.
Here some flight logs:
https://logs.px4.io/plot_app?log=f2e4d86f-46cc-462b-b01f-9d4c021cd4ec
https://logs.px4.io/plot_app?log=2c38ece0-2038-4b5f-a8d4-b96cd5db8d60
https://logs.px4.io/plot_app?log=f3cfbff0-ea89-4c36-841d-52071d1e3e90

and also it sometimes approached a bit closer than the specified distance if the throttle was kept towards the obstacle.

I observe this with my approach as well. We might need to pursue some kind of "reciprocal velocity" approach to combat this.

I'm already doing that. The velocity goes negative if you get ranges, which state that you are too close to an obstacle.

➤ Wrike Bot commented:
@Tanja Baumann All subtasks have been completed or deferred, you might want to complete this task as well.

closing this because got replaced by #10785


feat: use future annotations for same class type

Is your feature request related to a problem? Please describe
This issue is about cleaning the type hinting in the repository. Currently, when a method (inside a given class) returns an object whose type is the class itself (e.g. in Environment), the return type is given with quotes:
class Environment:

    ...

    def unwrapped(self) -> "Environment":
        ...

According to this thread, this is needed for Python<3.7, but can be removed with future annotations from Python 3.7+.
Describe the solution you'd like
Since Jumanji supports Python>=3.8, we could get rid of these quotes throughout the repository and use future annotations instead.

I can take this as a first issue :).


[FEATURE] SME Verification

As an SME I should be able to

Submit the required information for my business to be verified.
As an Admin I should be able to
Verify a user account with the information provided during registration
Decline verification to an account with incomplete details
View all verified accounts
View all declined accounts
Edit a previously declined account with new information

Information to verify an account can just be:

Founder ID card, verify d ID

Certificate of business name registration:
verifying the RC number

In the future others like visitation,  and proof of concept may come in.....

A field, rc, has been added to the data base scheme of smeProfile.....the rc is unique and IdCard field on userProfile
For now they can just be a simple unique  number of the docs.


Cannot Barcode Less than 6 characters

Barcode feature will not scan a barcode with less than 6 characters.

Duplicate of #12.


fix: add the operator image env

CONTROLLER_IMG need to be set in generate-manifest in case the value can't be correctly inferred when the branch name of the PR can't be parsed from the SHA value.
Signed-off-by: Hai He<EMAIL_ADDRESS>

/ok-to-merge

/ready-to-test


Widgets data not refreshing properly.

Hello,
I created a dashboard and I am feeding those with data pulled from a DB. I am facing a problem with my list widgets:
For example, if i have 3 items on my DB, the list widget will show the 3 items. However, If I drop one item from the DB, the widget is not refreshing by itself. I hit refresh on my page, CTRL+R, and open in new tab. Cleared cache, opened in incognito, but nothing seems to refresh the widget.
Anyway, I am using Chromium Browser within Raspbery Pi 2.
Problem with my query? Not really. The query is taking the correct number of items from the DB, before/after droping/inserting items.
I also noticed that if I have 3 items on my DB and insert a fourth one, the widget is refreshing properly, changing from "3" to "4". The problem is when I have 3, and the number of items is reduced. It seems some kind of cache is being kept.
The only solution I found was stop and start dashing. Obviously I don't want to do this all the time.
Would appreciate some help.
Thanks.
Luis

Please direct support requests to Stack Overflow: https://stackoverflow.com/questions/tagged/dashing


Add annotations field to Pod Security Policy configurations UI

Rancher versions:
rancher/rancher: 2.0.8
Some PSP functionality is configured through annotations.  For example: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#apparmor
Can we add annotations to the PSP configuration UI?

@davidnuzik Please try to add it in Global level menu > Security > Pod Security Policies > Add Policy
There is a Labels & Annotations section in new/edit policy page.


BLANK_LINE_BETWEEN_PROPERTIES, WRONG_ORDER_IN_CLASS_LIKE_STRUCTURES

Describe the bug
Internal error has occurred in rule [class-like-structures]. Please make an issue on this bug at https://github.com/cqfn/diKTat/.
As a workaround you can disable these inspections in yml config: <[BLANK_LINE_BETWEEN_PROPERTIES, WRONG_ORDER_IN_CLASS_LIKE_STRUCTURES]>.
Not entirely sure where this is coming from, can anyone shed light?
Expected behavior
Observed behavior
Steps to Reproduce
Environment information

diktat version: <IP_ADDRESS>
build tool (maven/gradle): gradle 7.3.1
how is diktat run (CLI, plugin, etc.): CLI
kotlin version: 1.7.20
operating system: MacOS
link to a project (if your project is public):

Hey! Thank you for this report! Any small reproducer?

May be at least some code snippet?

Hi @vishaantiwariepriceline, could you please provide the code to reproduce the error?


Removed Promiseable

I've been wanting to do this for sometime.

Pull Request Test Coverage Report for Build 873

14 of 15 (93.33%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 58.708%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

libraries/botbuilder-core/src/transcriptLogger.ts
11
12
91.67%

Totals

Change from base Build 864:
0.0%

Covered Lines:
1923

Relevant Lines:
3101

💛  - Coveralls


Install yum-plugin-security to ensure --sec-severity works on rhel6

When installing errata on RHEL6, the yum option '--sec-severity' will not work unless the 'yum-plugin-security'  package is installed.  This appears to be just a part of yum in RHEL7.  This will add the package if needed when installing errata.

I read the previous issue - The behavior you describe where installing the security plugin simply upgrades yum -  I can reproduce in yum 3.4.3 on rhel7.1 but not in yum 3.2.29 on rhel 6.7.  On 6.7 there is an actual noarch package 'yum-plugin-security' in the 'rhel-6-server-rpms' repo.  Looks to me like this might be needed in rhel6.  Thus the PR ;)

@civik you are correct! I swore I tested this on RHEL6, but I think I may have started a RHEL7 instance by accident.
I appreciate you putting this PR together. Could you help with two things before I merge-and-release?

Change your yum_package to use an only_if condition rather than wrapping it in an if block - this will address the Foodcritic FC023 warning and be more "chef like" :)
Add some ChefSpec tests to prevent future regressions. I can help with this if you're not familiar, but basically, we should ensure the existing chefspec runner assumes RHEL7 and ensures the package isn't attempted to be installed, and then add another chefspec runner for RHEL6 that ensures the package is installed.

Thanks again for the contribution - let me know how I can help.

Also, thanks for already addressing the rubocop errors :+1:

Working on it now.  The rspec force is not strong with me but I can usually copypasta something in that works using existing as an example.

@civik give it your best shot, and I'll help you get it over the finish line.

Looks good I think, looking forward to your feedback!

This looks great!
A few things:

Let's back out the version bump. I'll take care of that during the release process.
Can we add the RHEL7 test back in, but just not add a new chef runner?

context 'when the host is not rhel6' do
  it 'does not install the yum-security-plugin' do
    # test here
  end
end

context 'when the host is rhel6' do
  let(:chef_run) ...
  # your existing i-expect-chef-to-yum-install test here
end

We can go back and add a RHEL7-specific Chefspec runner once a new version of Fauxhai is released that contains the RHEL7 mocked data in the gem (vs. Fauxhai fetching it via HTTP and running into permissions problem like you experienced in Travis).
Let me know if you have any questions - but we're definitely close to merging this!

Also, when you've made the above changes, can you squash into a single commit?  Let me know if you're unclear how to do that.

Can you handle the squash?

Yup. I'll take care of it. Probably tomorrow.

Rebased / cleaned-up in 417aef83b and merged to master. I'll release a new version on the supermarket shortly.  Thanks!

v0.3.0 has been released to the supermarket this morning.


matches! macro always returns true, even for the wrong thiserror enum variant

Wondering if anyone knows why matches!(some_expr.unwrap_err(), WrongVariant) always returns true?
If I do the following:
match some_expr.unwrap_err() {
  ref v @ ref WrongVariant => assert!(false, format!("the actual variant is {v:?}")),
  RightVariant => assert!(true)
}

It just follows the first branch, regardless of what the actual Variant is... How could that be?

Somehow the compiler let me use WrongVariant but nothing was interpreting it as a variant of my enum... confusing. Once I used the fully qualified variant path everything worked. Sorry for the noise!


Changing passwords

What is the best practice for changing default passwords?
Going into container and settings env vars?

yes