Go Operator Unstructured Values for Helm Charts

Bug Report
Hey I am having issues defining an API struct with an unstructured component. This is primarily for handling in arbitrary overrides the user may give to things like helm charts. As we don't necessarily know or want to know all possible overrides.
What did you do?
I defined an API resource like so:
package v1alpha1

import (
  metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

//+kubebuilder:validation:Optional

type AkSpec struct {

  //+kubebuilder:validation:Schemaless

  Values map[string]interface{} `json:"values,omitempty"`
  // Values unstructured.Unstructured `json:"values,omitempty"`
}

// AkStatus defines the observed state of Ak
type AkStatus struct {}

//+kubebuilder:object:root=true
//+kubebuilder:subresource:status

// Ak is the Schema for the aks API
type Ak struct {
  metav1.TypeMeta   `json:",inline"`
  metav1.ObjectMeta `json:"metadata,omitempty"`

  Spec   AkSpec   `json:"spec,omitempty"`
  Status AkStatus `json:"status,omitempty"`
}

//+kubebuilder:object:root=true

// AkList contains a list of Ak
type AkList struct {
  metav1.TypeMeta `json:",inline"`
  metav1.ListMeta `json:"metadata,omitempty"`
  Items           []Ak `json:"items"`
}

func init() {
  SchemeBuilder.Register(&Ak{}, &AkList{})
}

Then I make / generate the manifests.
What did you expect to see?
A generic un-validated schema to be generated.
What did you see instead? Under which circumstances?
make[1]: Entering directory '/home/archer/git/authentik-manager/operator'
test -s /home/archer/git/authentik-manager/operator/bin/controller-gen || GOBIN=/home/archer/git/authentik-manager/operator/bin go install<EMAIL_ADDRESS>/home/archer/git/authentik-manager/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
gitlab.com/GeorgeRaven/authentik-manager/operator/api/v1alpha1:-: name requested for invalid type: interface{}
gitlab.com/GeorgeRaven/authentik-manager/operator/api/v1alpha1:-: invalid map value type: interface{}
Error: not all generators ran successfully
run `controller-gen object:headerFile=hack/boilerplate.go.txt paths=./... -w` to see all available markers, or `controller-gen object:headerFile=hack/boilerplate.go.txt paths=./... -h` for usage
make[1]: *** [Makefile:94: generate] Error 1
make[1]: Leaving directory '/home/archer/git/authentik-manager/operator'
make: *** [Makefile:95: build] Error 2

Environment
Operator type:
/language go
Kubernetes cluster type:
Minikube, and Vanilla
$ operator-sdk version
operator-sdk version: "unknown", commit: "unknown", kubernetes version: "unknown", go version: "go1.19.4", GOOS: "linux", GOARCH: "amd64"
$ go version (if language is Go)
1.20
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"8f94681cd294aa8cfd3407b8191f6c70214973a4", GitTreeState:"clean", BuildDate:"2023-01-18T15:58:16Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Additional context
I am trying to create a generic un-validated unstructured filed to hold values for helm charts applied by the Go based operator. As a sort of hybrid operator without using the hybrid operator plugin.

@DreamingRaven Would you mind sharing the CR that you used that caused this error?

Hey @everettraven, sure, I was only testing with some gaff CR, I can create a more elaborate CR if necessary. I am just trying to create a CRD that accepts a CR which is both unstructured and unvalidated in its values field that could contain map[string]interface{} types for helm actions.
apiVersion: akm.goauthentik.io/v1alpha1
kind: Ak
metadata:
  labels:
    app.kubernetes.io/name: ak
    app.kubernetes.io/instance: ak-sample
    app.kubernetes.io/part-of: operator
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: operator
  name: ak-sample
  namespace: auth
spec:
  values:
    nothing: nothing

Here is a much more concrete example where the values would point to a much more complex set of values:
apiVersion: akm.goauthentik.io/v1alpha1
kind: Ak
metadata:
  labels:
    app.kubernetes.io/name: ak
    app.kubernetes.io/instance: ak-sample
    app.kubernetes.io/part-of: operator
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: operator
  name: ak-sample
  namespace: auth
spec:
  values:
    global:
      domain:
        # the base domain which should be used for authentication
        base: example.org
        # the domain where authentik ingress will listen
        full: auth.example.org
        # the domain where the ldap ingress will listen
        ldap: ldap.example.org
      admin:
        # the administrative user you want in ldap and the entire SSO
        # ATM does very little
        name: somebody
        email<EMAIL_ADDRESS>
    # this assumes external SMTP server that has username and password
    # creds you can log in with.
    # from=the user all emails are going to come from
    # host=the location of SMTP server to authenticate to
    # username=the user to authenticate to SMTP server with
    smtp:
      # enable me to allow users to reset their own passwords etc
      # if diabled also disabled smtp password file env value
      enabled: false
      username<EMAIL_ADDRESS>      port: 587
      host: smtp.gmail.com
      from<EMAIL_ADDRESS>
    secret:
      generate: true
      randLength: 30
      # you probably dont want to change this name as you will have to change
      # it everywhere in subcharts
      name: auth

    ingress:
      name: auth
      enable: true
      clusterIssuer: letsencrypt-staging # must match cert-manager cluster issuer
      maxBodySize: 32M
      authCertSecret: authentik-cert
      ldapCertSecret: ldap-cert
      # HTTP STRICT TRANSPORT SECURITY
      # hsts:
      #   enable: false
      tls:
        enable: true

    # postgresql DEPENDENCY CHART overrides
    # https://github.com/bitnami/charts/tree/master/bitnami/postgresql/#parameters
    postgresql:
      image:
        registry: docker.io
        repository: bitnami/postgresql
        tag: 15
      # nameOverride: pgsql
      fullnameOverride: postgres
      primary:
        persistence:
          enabled: false
      readReplicas:
        persistence:
          enabled: false
      global:
        postgresql:
          service:
            ports:
              postgresql: 5432
          auth:
            existingSecret: auth
            database: authentik
            username: authentik
            secretKeys:
              adminPasswordKey: postgresPassword
              userPasswordKey: postgresUserPassword
              replicationPasswordKey: postgresReplicationPassword
      # diagnosticMode:
      #   enabled: true
      #   command: ["sleep"]
      #   args: ["infinity"]

    redis:
      fullnameOverride: redis
      image:
        registry: docker.io
        repository: bitnami/redis
        tag: "7.0"
      architecture: replication
      auth:
        enabled: true
        sentinel: false
        existingSecret: auth
        existingSecretPasswordKey: redisPassword
      replicaCount: 5
      master:
        service:
          ports:
            redis: 6379
        persistence:
          enabled: false
          size: 8Gi
      replica:
        persistence:
          enabled: false
          size: 8Gi

    ldap:
      enabled: false
      persistence:
        enabled: false
        name: ldap-pvc
        accessMode: ReadWriteOnce
        size: 8Gi
        mountPath: /data
        # mountPath: /bitnami/openldap
      deployment:
        imagePullPolicy: Always
        image: "docker.io/nitnelave/lldap:v0.4.1-alpine"
        name: ldap
        replicas: 1
        env: # statically defined environment variables can be as many as desired
        - name: LDAP_ADMIN_USERNAME
          value: "admin"
      secrets:
        secretName: auth
        basePath: /secrets
        lookup:
        - env: LLDAP_JWT_SECRET
          key: authJwtToken
        - env: LLDAP_LDAP_USER_PASS
          key: ldapAdminPassword
      service:
        name: ldap
      ports:
      - name: ldap
        containerPort: 3890
        servicePort: 3890
        protocol: TCP
      - name: http
        containerPort: 17170
        servicePort: 80
        protocol: TCP
      labels:
      - key: type
        value: auth
      - key: app
        value: ldap

    authentik:
      enabled: true
      # domain: <BASE_DOMAIN>
      config:
        name: authentik-config
        path: /config
        file: configuration.yml
        noteFile: /notification.txt
        generate: true
      # authentik expects its secrets as files in the same directory
      # so we tell kube where to get secret data and where to put it
      secrets:
        basePath: /secrets
        lookup:
        # this lookup will mount a secret data to a file basePath/files
        # this will also then create an env variable pointing to this secret file
        # thus we have no env variables with secrets in them but files are permed
        # https://www.authentik.com/integration/kubernetes/secrets/
        - key: postgresUserPassword
          file: postgresql-pass
          env: AUTHENTIK_POSTGRESQL__PASSWORD
        - key: redisPassword
          file: redis-pass
          env: AUTHENTIK_REDIS__PASSWORD
        - key: authJwtToken
          file: secret-key
          env: AUTHENTIK_SECRET_KEY
        # - key: sessionSecret
        #   file: SESSION_SECRET
        #   env: AUTHENTIK_SESSION_SECRET_FILE
        # - key: postgresauthentikPassword
        #   file: STORAGE_PASSWORD
        #   env: AUTHENTIK_STORAGE_POSTGRES_PASSWORD_FILE
        # - key: authStorageEncryptionKey
        #   file: STORAGE_ENCRYPTION_KEY
        #   env: AUTHENTIK_STORAGE_ENCRYPTION_KEY_FILE
        # - key: ldapAdminPassword
        #   file: LDAP_PASSWORD
        #   env: AUTHENTIK_AUTHENTICATION_BACKEND_LDAP_PASSWORD_FILE
        - key: smtpPassword
          file: smtp-pass
          env: AUTHENTIK_EMAIL__PASSWORD
      deployment:
        imagePullPolicy: Always
        image: "ghcr.io/goauthentik/server:2023.2.2"
        name: authentik
        replicas: 3
        env: # statically defined environment variables can be as many as desired
        - name: AUTHENTIK_LISTERN__HTTP
          value: <IP_ADDRESS>:9000
        - name: AUTHENTIK_LISTERN__HTTPS
          value: <IP_ADDRESS>:9443
        - name: AUTHENTIK_LISTERN__LDAP
          value: <IP_ADDRESS>:3389
        - name: AUTHENTIK_LISTERN__LDAPS
          value: <IP_ADDRESS>:6636
        - name: AUTHENTIK_LISTERN__METRICS
          value: <IP_ADDRESS>:9300
        - name: AUTHENTIK_LISTERN__DEBUG
          value: <IP_ADDRESS>:9900
        - name: AUTHENTIK_LOG_LEVEL
          value: info
        - name: AUTHENTIK_DISABLE_UPDATE_CHECK
          value: false
        - name: AUTHENTIK_ERROR_REPORTING
          value: false
        - name: AUTHENTIK_AVATARS
          value: gravatar
        - name: AUTHENTIK_DEFAULT_USER_CHANGE_NAME
          value: true
        - name: AUTHENTIK_DEFAULT_USER_CHANGE_EMAIL
          value: true
        - name: AUTHENTIK_DEFAULT_USER_CHANGE_USERNAME
          value: true
        - name: AUTHENTIK_GDPR_COMPLIANCE
          value: true
        - name: AUTHENTIK_DEFAULT_TOKEN_LENGTH
          value: 60
        - name: AUTHENTIK_IMPERSONATION
          value: true
        # - name: AUTHENTIK_FOOTER_LINKS
        #   value: '[{"name": "Link Name","href":"https://goauthentik.io"}]'
        - name: AUTHENTIK_WEB__WORKERS
          value: 2
        - name: AUTHENTIK_WEB__THREADS
          value: 2
      service:
        name: authentik
      ports:
      - name: http
        containerPort: 9000
        servicePort: 80
        protocol: TCP
      - name: https
        containerPort: 9443
        servicePort: 443
        protocol: TCP
      - name: ldap
        containerPort: 3389
        servicePort: 389
        protocol: TCP
      - name: ldaps
        containerPort: 6636
        servicePort: 636
        protocol: TCP
      - name: metrics
        containerPort: 9300
        servicePort: 300
        protocol: TCP
      - name: debug
        containerPort: 9900
        servicePort: 900
        protocol: TCP
      labels:
      - key: type
        value: auth
      - key: app
        value: authentik
      # list blueprint configmap names in this namespace to automount to worker instances
      # https://goauthentik.io/developer-docs/blueprints/
      blueprints: []
      # - name: some-default-blueprint
      #   configMap:
      #     name: example-custom-blueprint-configmap
      #     key: my-default-blueprint
      #   dest: /blueprints/default/some-default-blueprint.yaml

    # Authentik Manager
    akm:
      operator:
        # you probably dont want AKM to be loaded by this chart but we leave this option here for you
        # you should probably load the operator seperately and without this chart installed so it can
        # manage everything
        enabled: false

    # postgres admin dashboard
    pgadmin:
      enable: false
      persistence:
        enable: false
        name: pgadmin-pvc
        accessMode: ReadWriteOnce
        size: 4Gi
        mountPath: /var/lib/pgadmin/data
      deployment:
        imagePullPolicy: Always
        image: "docker.io/dpage/pgadmin4:latest"
        name: pgadmin
        replicas: 1
        env: # statically defined environment variables can be as many as desired
        - name: PGADMIN_LISTEN_PORT
          value: 80
      secrets:
        secretName: auth
        lookup:
        - env: PGADMIN_DEFAULT_PASSWORD
          key: pgAdminPassword
      service:
        name: pgadmin
      ports:
      - name: http-port
        containerPort: 80
        servicePort: 80
        protocol: TCP
      labels:
      - key: type
        value: auth
      - key: app
        value: pgadmin
      # OPTIONAL VOLUMES mount at /var/lib/pgadmin

@DreamingRaven Thanks for sharing, that provided me with some good context :). I was able to do a little bit of digging and according to this controller-tools issue, it seems like just using a raw map[string]interface{} hasn't worked for quite some time due to some of the ways that controller-gen handles this scenario.
After reading through that issue thread a bit, it looks like you can try a couple different approaches:

Use the runtime.RawExtension or json.RawMessage types in place of map[string]interface{}. I think both of these are initially of type []byte but can be unmarshalled to the map[string]interface{} in your reconciliation logic.
Break the mapping up into more than a single type definition like. I think breaking a map[string]interface up would look something like:

type AkSpec struct {

  //+kubebuilder:validation:Schemaless

  Values map[string]Value `json:"values,omitempty"`
  // Values unstructured.Unstructured `json:"values,omitempty"`
}

type Value struct {
  Field interface{}
}

I hope this helps a bit!

Wow thanks @everettraven I was not expecting you to have gone as deep as you have, and as quickly as you have!
I quickly tried your proposed split solution (2.) since that looks like the cleanest solution, however that results in the following error which I believe is the same:
> make generate manifests
test -s /home/archer/git/authentik-manager/operator/bin/controller-gen || GOBIN=/home/archer/git/authentik-manager/operator/bin go install<EMAIL_ADDRESS>/home/archer/git/authentik-manager/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
/home/archer/git/authentik-manager/operator/api/v1alpha1/ak_types.go:31:2: invalid field type: interface{}
Error: not all generators ran successfully
run `controller-gen object:headerFile=hack/boilerplate.go.txt paths=./... -w` to see all available markers, or `controller-gen object:headerFile=hack/boilerplate.go.txt paths=./... -h` for usage
make: *** [Makefile:94: generate] Error 1

I am trying all the other approaches in 1. now too, I will report back shortly.

Hey @everettraven here are my results of json.RawMessage:
//+kubebuilder:validation:Optional

// AkSpec defines the desired state of Ak
type AkSpec struct {

	//+kubebuilder:validation:Schemaless
	//+kubebuilder:pruning:PreserveUnknownFields

	Values json.RawMessage `json:"values,omitempty"`
}

[1/2] STEP 12/12: RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go
# gitlab.com/GeorgeRaven/authentik-manager/operator/controllers
controllers/ak_controller.go:99:88: crd.Spec.Values.UnstructuredContent undefined (type "encoding/json".RawMessage has no field or method UnstructuredContent)
Error: building at STEP "RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go": while running runtime: exit status 1
make: *** [Makefile:98: build] Error 1

And runtime.RawException
//+kubebuilder:validation:Optional

// AkSpec defines the desired state of Ak
type AkSpec struct {

	//+kubebuilder:validation:Schemaless

	Values runtime.RawExtension `json:"values,omitempty"`
}

[1/2] STEP 12/12: RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go
# gitlab.com/GeorgeRaven/authentik-manager/operator/controllers
controllers/ak_controller.go:99:88: crd.Spec.Values.UnstructuredContent undefined (type "k8s.io/apimachinery/pkg/runtime".RawExtension has no field or method UnstructuredContent)
Error: building at STEP "RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go": while running runtime: exit status 1
make: *** [Makefile:98: build] Error 1

Both seem to encounter the same issue during builds. Please correct me if I am doing anything glaringly wrong with your suggestions.

I quickly tried your proposed split solution (2.) since that looks like the cleanest solution, however that results in the following error which I believe is the same:

@DreamingRaven It looks like I missed that the issue I referenced mentioned adding these annotations above the field definition of type interface{}:
	// +kubebuilder:pruning:PreserveUnknownFields
	// +kubebuilder:validation:Schemaless

I'm not sure whether or not that is the problem or would resolve the error you encountered attempting solution 2 as I've never personally attempted something like this (and haven't had enough time today to throw together a simple reproducer)

Both seem to encounter the same issue during builds. Please correct me if I am doing anything glaringly wrong with your suggestions.

So the only thing I noticed right away is that the error is coming from your controller and stating that the Spec.Values.UnstructuredContent field doesn't exist. This likely means that you updated the spec but didn't update your controller code to work with either the json.RawMessage or runtime.RawExtension  types. As far as I am aware neither of those types have a field called UnstructuredContent (json.RawMessage is actually just a []byte).
I think updating your controller code to handle the field type change should resolve that specific error. No promises you won't run into others, but if you do feel free to ping me here :)

@everettraven You aren't the hero I deserve but certainly the hero I needed. You are completely right I dont know why I was so blind to not see the error message was pointing to the controller :see_no_evil: .
It works!
The final CRD Spec API:
// AkSpec defines the desired state of Ak
type AkSpec struct {

	// +kubebuilder:pruning:PreserveUnknownFields
	// +kubebuilder:validation:Schemaless

	// Values is the helm chart values map to override chart defaults. This is often further adapted by the controller
	// to add additional resources like declarative blueprints into the deployments. Values is a loose, and unstructured
	// datatype. It will not complain if the values do not override anything, or do anything at all.
	Values json.RawMessage `json:"values,omitempty"`
}

The reconciliation of this CRD (needs a little cleanup):
package controllers

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"net/url"
	"os"
	"path/filepath"

	"github.com/alexflint/go-arg"
	"github.com/go-logr/logr"
	"k8s.io/apimachinery/pkg/api/errors"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/schema"
	"k8s.io/apimachinery/pkg/types"
	"k8s.io/apimachinery/pkg/util/validation/field"
	"k8s.io/cli-runtime/pkg/genericclioptions"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/rest"
	ctrl "sigs.k8s.io/controller-runtime"
	"sigs.k8s.io/controller-runtime/pkg/client"
	klog "sigs.k8s.io/controller-runtime/pkg/log"

	"helm.sh/helm/v3/pkg/action"
	"helm.sh/helm/v3/pkg/chart"
	chartLoader "helm.sh/helm/v3/pkg/chart/loader"
	"helm.sh/helm/v3/pkg/release"

	akmv1a1 "gitlab.com/GeorgeRaven/authentik-manager/operator/api/v1alpha1"
	"gitlab.com/GeorgeRaven/authentik-manager/operator/utils"
)

// AkReconciler reconciles a Ak object
type AkReconciler struct {
	client.Client
	Scheme *runtime.Scheme
}

//+kubebuilder:rbac:groups=akm.goauthentik.io,resources=aks,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=akm.goauthentik.io,resources=aks/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=akm.goauthentik.io,resources=aks/finalizers,verbs=update

// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
//
// For more details, check Reconcile and its Result here:
// -<EMAIL_ADDRESS>func (r *AkReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	l := klog.FromContext(ctx)

	// Parsing options to make them available TODO: pass them in rather than read continuously
	o := utils.Opts{}
	arg.MustParse(&o)

	actionConfig, err := r.GetActionConfig(req.NamespacedName.Namespace, l)
	if err != nil {
		return ctrl.Result{}, err
	}

	// GET CRD
	crd := &akmv1a1.Ak{}
	err = r.Get(ctx, req.NamespacedName, crd)
	if err != nil {
		if errors.IsNotFound(err) {
			// Request object not found, could have been deleted after reconcile request.
			 For additional cleanup logic use finalizers.
			// Return and don't requeue
			l.Info("Ak resource reconciliation triggered but disappeared. Checking for residual chart for uninstall then ignoring since object must have been deleted.")
			_, err := r.UninstallChart(req.NamespacedName, actionConfig)
			if err != nil {
				return ctrl.Result{}, err
			}
			return ctrl.Result{}, nil
		}
		// Error reading the object - requeue the request.
		l.Error(err, "Failed to get Ak resource. Likely fetch error. Retrying.")
		return ctrl.Result{}, err
	}
	l.Info(fmt.Sprintf("Found Ak resource `%v` in `%v`.", crd.Name, crd.Namespace))

	// Helm Chart Identification
	u, err := url.Parse(fmt.Sprintf("file://workspace/helm-charts/ak-%v.tgz", o.SrcVersion))
	if err != nil {
		return ctrl.Result{}, err
	}

	// Helm Install or Upgrade Chart
	var vals map[string]interface{}
	err = json.Unmarshal(crd.Spec.Values, &vals)
	if err != nil {
		return ctrl.Result{}, err
	}
	_, err = r.UpgradeOrInstallChart(req.NamespacedName, u, actionConfig, vals)
	if err != nil {
		return ctrl.Result{}, err
	}

	return ctrl.Result{}, nil
}

// UpgradeOrInstallChart upgrades a chart in cluster or installs it new if it does not already exist
// ulr format is [scheme:][//[userinfo@]host][/]path[?query][#fragment] e.g file://workspace/helm-charts/ak-0.1.0.tgz"
func (r *AkReconciler) UpgradeOrInstallChart(nn types.NamespacedName, u *url.URL, a *action.Configuration, o map[string]interface{}) (*release.Release, error) {
	// Helm List Action
	listAction := action.NewList(a)
	releases, err := listAction.Run()
	if err != nil {
		return nil, err
	}

	toUpgrade := false
	for _, release := range releases {
		// fmt.Println("Release: " + release.Name + " Status: " + release.Info.Status.String())
		if release.Name == nn.Name {
			toUpgrade = true
		}
	}

	c, err := r.LoadHelmChart(u)
	if err != nil {
		return nil, err
	}

	fmt.Println(o)

	var rel *release.Release
	if toUpgrade {
		// Helm Upgrade
		updateAction := action.NewUpgrade(a)
		rel, err = updateAction.Run(nn.Name, c, o)
		if err != nil {
			return nil, err
		}

	} else {
		// Helm Install
		installAction := action.NewInstall(a)
		installAction.Namespace = nn.Namespace
		installAction.ReleaseName = nn.Name
		rel, err = installAction.Run(c, o)
		if err != nil {
			return nil, err
		}
	}
	return rel, nil
}

func (r *AkReconciler) UninstallChart(nn types.NamespacedName, a *action.Configuration) (*release.UninstallReleaseResponse, error) {
	uninstallAction := action.NewUninstall(a)
	releaseResponse, err := uninstallAction.Run(nn.Name)
	if err != nil {
		return nil, err
	}
	return releaseResponse, nil
}

// GetActionConfig Get the Helm action config from in cluster service account
func (r *AkReconciler) GetActionConfig(namespace string, l logr.Logger) (*action.Configuration, error) {
	actionConfig := new(action.Configuration)
	var kubeConfig *genericclioptions.ConfigFlags
	config, err := rest.InClusterConfig()
	if err != nil {
		return nil, err
	}
	// Set properties manually from official rest config
	kubeConfig = genericclioptions.NewConfigFlags(false)
	kubeConfig.APIServer = &config.Host
	kubeConfig.BearerToken = &config.BearerToken
	kubeConfig.CAFile = &config.CAFile
	kubeConfig.Namespace = &namespace
	if err := actionConfig.Init(kubeConfig, namespace, os.Getenv("HELM_DRIVER"), log.Printf); err != nil {

	}
	return actionConfig, nil
}

// Get Connection Client to Kubernetes
func (r *AkReconciler) GetKubeClient() (*kubernetes.Clientset, error) {
	config, err := rest.InClusterConfig()
	if err != nil {
		return nil, err
	}
	// creates the clientset
	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		return nil, err
	}
	return clientset, nil
}

// GetHelmChart loads a helm chart from a given file as URL
func (r *AkReconciler) LoadHelmChart(u *url.URL) (*chart.Chart, error) {
	// fmt.Println("Scheme:", u.Scheme)
	// fmt.Println("Opaque:", u.Opaque)
	// fmt.Println("User:", u.User)
	// fmt.Println("Host:", u.Host)
	// fmt.Println("Path:", u.Path)
	// fmt.Println("RawPath:", u.RawPath)
	// fmt.Println("ForceQuery:", u.ForceQuery)
	// fmt.Println("RawQuery:", u.RawQuery)
	// fmt.Println("Fragment:", u.Fragment)
	// fmt.Println("RawFragment:", u.RawFragment)

	// GET HELM CHART
	if u.Scheme != "file" {
		err := errors.NewInvalid(
			schema.GroupKind{
				Group: "akm.goauthentik.io",
				Kind:  "Ak",
			},
			fmt.Sprintf("Url scheme `%v` != `file`, unsupported scheme.", u.Scheme),
			field.ErrorList{})
		return nil, err
	}
	// load chart from filepath (which is part of host in url)
	path, err := filepath.Abs(u.Host + u.Path)
	if err != nil {
		return nil, err
	}
	_, err = utils.Exists(path)
	if err != nil {
		return nil, err
	}
	return chartLoader.Load(path)
}

// SetupWithManager sets up the controller with the Manager.
func (r *AkReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&akmv1a1.Ak{}).
		Complete(r)
}

Output of the controller when I apply the CR
2023-03-08T08:50:47Z	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": ":8080"}
2023-03-08T08:50:47Z	INFO	setup	starting manager
2023-03-08T08:50:47Z	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
2023-03-08T08:50:47Z	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
I0308 08:50:47.005932       1 leaderelection.go:248] attempting to acquire leader lease auth/d460f2c2.goauthentik.io...
I0308 08:50:47.009845       1 leaderelection.go:258] successfully acquired lease auth/d460f2c2.goauthentik.io
2023-03-08T08:50:47Z	DEBUG	events	authentik-manager-dd6786489-gfh8w_3a173226-9e07-4e9d-9f78-260906c25b75 became leader	{"type": "Normal", "object": {"kind":"Lease","namespace":"auth","name":"d460f2c2.goauthentik.io","uid":"ef75f7c6-51b3-4f8e-9a71-e613c48a2267","apiVersion":"coordination.k8s.io/v1","resourceVersion":"553"}, "reason": "LeaderElection"}
2023-03-08T08:50:47Z	INFO	Starting EventSource	{"controller": "ak", "controllerGroup": "akm.goauthentik.io", "controllerKind": "Ak", "source": "kind source: *v1alpha1.Ak"}
2023-03-08T08:50:47Z	INFO	Starting Controller	{"controller": "ak", "controllerGroup": "akm.goauthentik.io", "controllerKind": "Ak"}
2023-03-08T08:50:47Z	INFO	Starting workers	{"controller": "ak", "controllerGroup": "akm.goauthentik.io", "controllerKind": "Ak", "worker count": 1}
2023-03-08T08:54:05Z	INFO	Found Ak resource `ak-sample` in `auth`.	{"controller": "ak", "controllerGroup": "akm.goauthentik.io", "controllerKind": "Ak", "Ak": {"name":"ak-sample","namespace":"auth"}, "namespace": "auth", "name": "ak-sample", "reconcileID": "8c7b7047-d980-4ddb-8ae7-7a5a6bc22edd"}
map[nothing:nothing]
2023/03/08 08:54:06 creating 23 resource(s)

Again thanks @everettraven! Hopefully this helps out anyone who has the same issue! Here is a link to my full implementation in case anyone needs this later down the line https://gitlab.com/GeorgeRaven/authentik-manager


Avoid utf-8 conflicts in Digest files

Use +# -- encoding: binary -- encoding comment to silence
utf-8 conflicts in several Digest files

Looks good, does this help parser deal with these files?

Yes parser approves 😃

Thanks!


Implemented wildcard includes for ipsec/strongswan

So this is the PR mentioned here
https://github.com/opnsense/core/pull/3298
It just adds wildcard includes to
ipsec.conf
ipsec.secrets
strongswan.conf
Best regards
Rainer

does this require empty stub files or is it ok for strongswan if no files are present?

It complains for some but does start...
...
-rw-r--r--   1 root     wheel       386 Mar 11 09:36 strongswan.mobile.conf
-rw-------   1 root     wheel        49 Mar 11 09:38 ipsec.mobile.secrets
-rw-r--r--   1 root     wheel      1178 Mar 11 09:45 ipsec.mobile.conf
-rw-r--r--   1 root     wheel       891 Mar 11 09:46 strongswan.conf
-rw-------   1 root     wheel        88 Mar 11 09:46 ipsec.secrets
-rw-r--r--   1 root     wheel       968 Mar 11 09:46 ipsec.conf
-rw-r--r--   1 root     wheel     17089 Mar 11 09:46 filter_tables.conf
root@opnsense01:/usr/local/etc # mv *.mobile.* /root/
root@opnsense01:/usr/local/etc # ipsec stop
Stopping strongSwan IPsec...
root@opnsense01:/usr/local/etc # ipsec start
no files found matching '/usr/local/etc/strongswan.*.conf'
Starting strongSwan 5.7.2 IPsec [starter]...
no files found matching '/usr/local/etc/ipsec.*.conf'
no netkey IPsec stack detected
no KLIPS IPsec stack detected
no known IPsec stack detected, ignoring!
root@opnsense01:/usr/local/etc #

ok, thanks for the clarification :)

Thanks for being so quick - this is really cool!!!

This actually solves
https://github.com/opnsense/core/issues/3295
and
https://github.com/opnsense/core/issues/3300

@stumbaumr can you test https://github.com/opnsense/core/commit/a4d157db0bc622c68af6da717c311499d8362691 ? I've dropped the files in directories, which is more aligned with the standard samples that are shipped with the software on our end.
e.g. /usr/local/etc/strongswan.conf.sample

@AdSchellevis I created comments in your PR - there seems to be a mixup with directory names...

@stumbaumr oops, my mistake, it seems that I mixed them up indeed. take 2 --> https://github.com/opnsense/core/commit/dfd48d29349688d25b102b928ccbd509aa516f9a

@AdSchellevis ok, now it works as intended.

@stumbaumr ok, thanks for confirming


[5.9] Config option to allow caching of null values

@cvsouth - you need to put a detailed explanation of what this PR actually does (beyond the title in your PR), and the reasons why you think this belongs in the Laravel core.
Otherwise it is unlikely to be accepted by Taylor when he sees this, and your PR will be closed.

It is a limitation of Laravel cache that you are not supposed to cache null at all. IMO that is fine, because you can wrap a nullable value in an array or do something similar:
Cache::forever('test', [$nullable]);
Cache::forever('test', $nullable ?? false);

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.

Laravel's cache implementation conforms to PSR-16. null values can already be stored right now with Laravel's cache implementation. The fact that you always get back false for null values is due to the following:

The null value (although it will not be distinguishable from a cache miss when reading it back out).

See https://www.php-fig.org/psr/psr-16/#14-data

Hi @taylorotwell, @driesvints, @halaei and @laurencei.
I've read over PSR-16 and see that it does indeed say for a null value to be indistinguishable from a cache miss. I created the package cvsouth/cache-null which enables support for caching null values.
https://github.com/cvsouth/cache-null
Without the package we can wrap the nullable value in an array or convert it to false but while that is annoying I think the issue is broader as developers that make the mistake of thinking they can cache null values may not get any type of Exception or unexpected behaviour. Methods like remember and rememberForever simply recalculate the value every time null was previously stored instead.


Synchronous database drivers may lead to timeouts

I decided to fork your code and found PyMongo as a dependency, pretty bad, I recommend switching over to motor (pip install motor), it's very easy to convert code from PyMongo to motor as the methods are pretty much the same

Also remove mysql with aiomysql

Also remove requests with aiohttp


Set of changes

Currently houses:
Update to scalar share denomination labels
(Structured this PR in this way in-case I have additional changes to push before it actually gets merged)

Coverage remained the same at 74.812% when pulling 1f5a1e864cd13e1fa1f1cf49249c76153b9f049a on 1254-changes-and-fixes into 62a8adfcf8dcf501100080328fd063b4dbffc54a on master.

Coverage increased (+0.02%) to 74.829% when pulling dc8b4877e78bfe29bfdeeaa456cceb55a3ca921a on 1254-changes-and-fixes into 62a8adfcf8dcf501100080328fd063b4dbffc54a on master.

Coverage increased (+0.02%) to 74.829% when pulling dc8b4877e78bfe29bfdeeaa456cceb55a3ca921a on 1254-changes-and-fixes into 62a8adfcf8dcf501100080328fd063b4dbffc54a on master.


Automatically turn the washing machine plug on/off ⚡🔌

*** draft
Share my automation that turns my washing machine plug on and off based on washing machine activity.
No idle power wasted and safeguards against an issue we had in the UK where some appliances suffered from an electrical fault and caught fire when they were not in use.

Related: #143


[Feature Request] add clean-up action that triggers when template is used

Hi!
Last year i used a Kotlin project template from Jetbrains for the Advent of Code. It had a nice feature:
After creating my project from the template it automatically committed a clean-up change, that exchanged the template readme with a readme that was more tailored for the project.
Are there any plans to add such thing to this project template? Here is the workflow they used in their project template on github.
Some things are always bothering me, when i setup my project through the template:
Static changes
Following things i can think of that always have to change:

All Badges from the README.md are broken as they point to the project template --> that should be easily fixed with the workflow --> Fixed in #3
The README.md contains sections that are no longer needed when the template was created --> Fixed in #3
Rename the "MyProject" to the name of the new github repository --> Fixed in #3

Dynamic/User changes
Following things could be changed sometimes according to what the user wants. Those are probably not easy, maybe impossible to do:

Removing the unwanted examples or dependencies is sometimes anoying because removing one example cannot be done from github directly without branching or multiple commits --> Deprecated because of new template project

For private projects i'm sometimes not interested in MacOS. Maybe it is possible to disable one or many of the CI Matrix entries? --> not sure if this is sensibel to do

Apply for help
If this has any chances of beeing excepted i could invest some time in creating the worklfow for at least "fixing" the links of the build badges?

@lefticus thanks for merging #3!
My spare time is and was a bit limited to add the remaining points.
For the record the following point remains (as i saw you already have applyed some changes yourself):

add step that configures cmake to check the janitor has not broken anything
make janitor able to maintain project templates and create them (see comment above)

I was ready to see it moving forward. I'll keep updating the details as I test out the template for my own tests.


Add TableTaskConfig into TableConfig for task config

Added TableConfigJSONBuilder to build JSON format table config
Modify tests to use the new TableConfigJSONBuilder to build table config

Codecov Report

Merging #1482 into master will increase coverage by 8.3%.
The diff coverage is 85.24%.

@@            Coverage Diff            @@
##           master    #1482     +/-   ##
=========================================
+ Coverage   59.61%   67.91%   +8.3%
=========================================
  Files         788      788
  Lines       38582    38551     -31
  Branches     5033     5033
=========================================
+ Hits        22999    26182   +3183
+ Misses      13959    10658   -3301
- Partials     1624     1711     +87

Impacted Files
Coverage Δ

...m/linkedin/pinot/common/config/IndexingConfig.java
67.27% <ø> (-1.7%)
:arrow_down:

...n/config/SegmentsValidationAndRetentionConfig.java
62.5% <ø> (-2.89%)
:arrow_down:

...controller/helix/ControllerRequestBuilderUtil.java
95% <0%> (+11.09%)
:arrow_up:

.../linkedin/pinot/common/config/TableTaskConfig.java
50% <50%> (ø)

...controller/helix/core/minion/PinotTaskManager.java
70.17% <77.77%> (+48.17%)
:arrow_up:

...t/common/config/helper/TableConfigJSONBuilder.java
89.74% <89.74%> (ø)

.../com/linkedin/pinot/common/config/TableConfig.java
84.21% <91.66%> (+0.46%)
:arrow_up:

...om/linkedin/pinot/transport/netty/NettyServer.java
81.05% <0%> (-2.11%)
:arrow_down:

.../resources/BasePinotControllerRestletResource.java
87.75% <0%> (ø)
:arrow_up:

... and 227 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c826ec...f5d010b. Read the comment docs.


python read AVRO topic has error

What issue do you find in Pulsar docs?
https://pulsar.apache.org/docs/en/client-libraries-python/
from the doc, i can not read topic mess by use python,
spark:3.1.2
scala:2.12
python:3.7
pulsar:2.8.0
spark-plusar:
org.apache.pulsar
pulsar-spark
2.8.0
python-pulsar:
python3 -m pip install pulsar-client=='2.8.1'
python3 -m pip install pulsar-client[avro]

first, in spark scala, i create a topic schema:
object SchemaUtils {
class Users () {
var name: String = null
var age: Int = -1
def this(name: String, age: Int) {
  this()
  this.name = name
  this.age = age
}
override def toString: String = s"{name: $name, age: $age}"
def getName(): String = {
    return name;
}
def getAge() : Int = {
    return age;
}

}
}

second, in spark scala, i send a message into topic, use the Users schema:
import spatio.topics.SchemaUtils.Users
import org.apache.pulsar.client.api.{PulsarClient, Schema}
object clientProducer {
def main(args: Array[String]): Unit = {
val user = new Users("werewr", 35)
println(user)
val service_url_tn = "****"
val topic = "persistent://st/forever/scala-Users-topic"
val client = PulsarClient.builder()
.serviceUrl(service_url_tn)
.build()
val producer =  client.newProducer(Schema.AVRO(classOf[Users])).topic(topic).create()
producer.send(user)
println("send success" + user.toString())
}
}
third, the messge has send success,
./bin/pulsar-admin schemas get persistent://st/forever/scala-Users-topic
{
"version": 0,
"schemaInfo": {
"name": "scala-Users-topic",
"schema": {
"type": "record",
"name": "Users",
"namespace": "spatio.topics.SchemaUtils$",
"fields": [
{
"name": "name",
"type": [
"null",
"string"
]
},
{
"name": "age",
"type": "int"
}
]
},
"type": "AVRO",
"properties": {
"__alwaysAllowNull": "true"
}
}
}
four, in spark scala, i read the topic, and get the message,
object clientConsumer {
implicit val formats = Serialization.formats(NoTypeHints)
def main(args: Array[String]): Unit = {
val service_url_tn = "****"
val client = PulsarClient.builder()
.serviceUrl(service_url_tn)
.build()
val topic = "persistent://st/forever/scala-Users-topic"
val consumer =  client.newConsumer(Schema.AVRO(classOf[Users]))
      .topic(topic)
      .subscriptionName("read-my-topic")
      .subscribeAsync().join()

var cnt = 0
while (true) {
    val user = consumer.receive()
    println(write(user.getValue)) // 将类转为json输出
    cnt +=1
    println(cnt)
}

}
}
and get :
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
{"name":"werewr","age":35}
1
five, so in python, i want to read the topic, but get error,
class Users(Record):
name = String()
age = Integer()
and read topic:
from pulsar.schema import AvroSchema
import pulsar
from pulsar import MessageId
from pulsar import InitialPosition
import schemaProduce
import pulsar_topics
client = pulsar.Client('****')
topic = "persistent://st/forever/scala-Users-topic"
msg_id = MessageId.earliest
topic_schema = AvroSchema(pulsar_topics.Users)
print(topic_schema, type(topic_schema))
consumer = client.subscribe(topic = topic, subscription_name='python-read', schema = topic_schema, initial_position=InitialPosition.Earliest)
while True:
msg = consumer.receive()
print(msg.data())
print(msg.value())
consumer.close()
client.close()
but get the erro:
b'\x02\x0cwerewrF'
Traceback (most recent call last):
File "***/schemaConsume.py", line 38, in
print(msg.value())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pulsar/init.py", line 180, in value
return self._schema.decode(self._message.data())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pulsar/schema/schema_avro.py", line 70, in decode
d = fastavro.schemaless_reader(buffer, self._schema)
File "fastavro/_read.pyx", line 976, in fastavro._read.schemaless_reader
File "fastavro/_read.pyx", line 988, in fastavro._read.schemaless_reader
File "fastavro/_read.pyx", line 659, in fastavro._read._read_data
File "fastavro/_read.pyx", line 516, in fastavro._read.read_record
File "fastavro/_read.pyx", line 651, in fastavro._read._read_data
File "fastavro/_read.pyx", line 436, in fastavro._read.read_union
IndexError: list index out of range

so, how read the topic message?

test send the message:
in scala:
val user = new Person("werewr", 35)
producer.send(user)
in python:
producer.send(Person(name="wtr", age=23)
and python read, in schema/schema_avro.py：
    def decode(self, data):
        print(data, type(data), )
        buffer = io.BytesIO(data2=)
        d = fastavro.schemaless_reader(buffer, self._schema)
        return self._record_cls(**d)

get result:
from scala send message:
('\x02\x0cscala1F', <type 'str'>)
from python send message:
('\xe2\n\x02\x0cscala1', <type 'str’>)
so, they are not the same, there must has decode and encode,

It looks like scala and python have different behavior in encoding message. I will have a test of this case.

@gaoran10 as we discussed just now, can you help look into this issue? And then we can verify whether it is a doc-related issue, thanks.

Closed as stale.
The development of the Python client has been permanently moved to https://github.com/apache/pulsar-client-python. If it's still relevant, please open an issue there.


robyn_allocator scenario- max_response_expected_spend giving issues

Project Robyn
I successfully ran the robyn allocator for scenario max_historical_response but when I am doing the same for max_response_expected_spend, my initial and optimized spends are the same for all media channels,

Please let me know what is happening here.
Thanks.

Hi - this is usually due to the optimizer not finding any place to actually optimize the spend. Could you try broadening the ranges for the budget allocation for each channel? The channel_constr_low and channel_constr_up params are what you're looking for.

I too am having problems.
I want to create a scenario changing 1 year's worth of spending.   2021 spending is 26,166,517 while my expected spend for 2022 would be 34,000,000 .
The allocator calculates this to have an expected spend delta of -0.1354.

channels
date_min
date_max
periods
constr_low
constr_up
histSpend
histSpendTotal
initSpendUnitTotal
initSpendUnit
initSpendShare
initResponseUnit
initResponseUnitTotal
initRoiUnit
expSpendTotal
expSpendUnitTotal
expSpendUnitDelta
optmSpendUnit
optmSpendUnitDelta
optmSpendUnitTotal
optmSpendUnitTotalDelta
optmSpendShareUnit
optmResponseUnit
optmResponseUnitTotal
optmRoiUnit
optmResponseUnitLift
optmResponseUnitTotalLift

channel 1
1/3/2021
12/26/2021
52 weeks
0.7
1.5
1792374
$  26,166,517.24
754171.6
$    52,716.88
0.0699
186.349
487972.8
0.003535
$  34,000,000.00
652054.7945
-0.135402608
36901.81
-0.3
652054.8
-0.1354
0.056593
64.40688
415760.2
0.001745
-0.65437
-0.14798

channel 2
1/3/2021
12/26/2021
52 weeks
0.7
1.5
15258415
$  26,166,517.24
754171.6
$  526,152.26
0.697656
449646.8
487972.8
0.854594
$  34,000,000.00
652054.7945
-0.135402608
492441.3
-0.064070807
652054.8
-0.1354
0.755215
402476.2
415760.2
0.817308
-0.10491
-0.14798

channel 3
1/3/2021
12/26/2021
52 weeks
0.7
1.5
9115728
$  26,166,517.24
754171.6
$  175,302.46
0.232444
38139.65
487972.8
0.217565
$  34,000,000.00
652054.7945
-0.135402608
122711.7
-0.3
652054.8
-0.1354
0.188192
13219.58
415760.2
0.107729
-0.65339
-0.14798

@tgtod002 could you make sure you are on the latest version of Robyn? I believe this was fixed in a previous update, but please let me know if that is not the case.

@NoufaMukhtar could you share screenshots of your response curves as well as the graph outputs for the two scenarios of budget allocation? Feel free to blind any numbers that you do not want to share.

@kyletgoldberg I updated to the latest version of Robyn 3.6.4.   I am still getting negative values.
I used 2021 spend which was  26,266,517,  I am trying to do a scenaior of bringing up the spend 34M .  The allocator comes up with a negative expected spend of -0.1345.

channels
date_min
date_max
periods
constr_low
constr_up
histSpend
histSpendTotal
initSpendUnitTotal
initSpendUnit
initSpendShare
initResponseUnit
initResponseUnitTotal
initRoiUnit
expSpendTotal
expSpendUnitTotal
expSpendUnitDelta

MC1
1/3/2021
12/26/2021
52 weeks
0.7
1.5
1792374
26166517
754171.6
52716.88
0.0699
241.9327
4289190
0.004589
34000000.00
652054.8
-0.1354

mc2
1/3/2021
12/26/2021
52 weeks
0.7
1.5
15258415
26166517
754171.6
526152.3
0.697656
1383827
4289190
2.630089
34000000.00
652054.8
-0.1354

mc3
1/3/2021
12/26/2021
52 weeks
0.7
1.5
9115728
26166517
754171.6
175302.5
0.232444
2905121
4289190
16.57205
34000000.00
652054.8
-0.1354

@tgtod002 are there periods where the spend across channels is 0 in the historical data? the hist/init spend data only looks at non-zero spend, so if there are zeroes it may be causing the behavior.

@tgtod002 hmm... would it be possible to send me the .RDS file with the saved model results to debug further? i think i know where the problem is but it's a bit hard to debug this way. my email is<EMAIL_ADDRESS>

@kyletgoldberg Just checking in.  I sent you the .RDS file last week.   Were you able to receive it?

@tgtod002 yes thank you - I'm working on it. Will get back to you asap

@kyletgoldberg Thanks for the update.   Will try playing with lower channel_contr_low.

@kyletgoldberg .  Some suggestion for a possible enhancement...Currently the spreadsheets for the allocator provide response data at unit level (unit response, optimized response).   What would help is to be able to provide response data at the total spend time - For example, if I am allocating $50M over 52 weeks.  What I would like to also see is what I expect my response levels to be over the whole time period (52 weeks)...Right now, because of the way the unit levels are calculated (on non zeroes).  it is difficult for me to see what I expect to see as responses for each media channels.
Hope that makes sense

noted - thanks for the feedback

Is there any enhancement plan for this?
If not, I can create a function. I already created the function and am using it, but need to do linting and adding some error messages.

@Tomobay That's great!   Do you mind sharing the function you created?

Hi,  Is there an update on resolving this?

You can use
https://github.com/Tomobay/Robyn
robyn_allocator_historical(
new_budget_ratio = 1, # new budget
channels, # channels to optimize
channel_constr_low = 1, # lower band
channel_constr_up = 1, # upper band
InputCollect, # Input Collect
OutputCollect, # OutputCollect
select_model, # selected_model
date_min = NULL, # starting date to optimize
date_max = NULL # end date to optimize
)
I will try to test and document it to merge it to orginal

Please reopen if this issue reoccurs.


feat: use env variable in yaml config files

Description
We added the ability to use environment variables in yaml config files. This allows to use environment variables in the config files of the ocis services which will be replaced by the actual value of the environment variable at runtime.
Example:
web:
  http:
    addr: ${SOME_HTTP_ADDR}

This makes it possible to use the same config file for different environments without the need to change the config file itself. This is especially useful when using docker-compose to run the ocis services. It is a common pattern to create an .env file which contains the environment variables for the docker-compose file. Now you can use the same .env file to configure the ocis services.
Related Issue

Fixes <issue_link>

Motivation and Context

How Has This Been Tested?

test environment:
test case 1:
test case 2:
...

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Technical debt
[ ] Tests only (no source changes)

Checklist:

[ ] Code changes
[ ] Unit tests added
[ ] Acceptance tests added
[ ] Documentation ticket raised:

@mmattel @dragonchaser @kobergj
I suggest to hold this and not commit it to the 5.0.0 release.

@kobergj @dragonchaser I think we should merge this for OCIS v5. Existing yaml files will not contain any ${ENV_VAR} replacements or they would likely be failing anyways. If an admin wants to use them he has to check the env vars anyway. So this should go in sooner rather than later.

CI complains about a changlog issue


DSD-865: Reservoir branding

Fixes JIRA ticket DSD-865
This PR does the following:

Adds Reservoir branding to Storybook.
Updates all references to "design system" in the Storybook documentation to include "Reservoir" in the name of the design system.
Updates language on the Welcome page.

How has this been tested?

n/a

Checklist:

[x] I have updated the Storybook documentation accordingly.
[x] I have added relevant accessibility documentation for this pull request.
[x] All new and existing tests passed.

Front End Review:

[ ] View the example in Storybook

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9

Tugboat has finished building the preview for this pull request!
Link:

https://pr922-bin804gtjifjbmgnazrd3nuseuibkkh5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/624309c29413e8006b2ca6a9


AWS SSO documentation should describe SSO resource, not legacy IAM integration

The "Azure Active Directory single sign-on (SSO) integration with Amazon Web Services (AWS)" documentation page describes how to set up SAML authentication with AWS IAM.
This scenario is no longer the preferred solution, as AWS provides a much more sophisticated "Single Sign-On" resource now: https://aws.amazon.com/single-sign-on/
Please update the documentation to describe Enterprise App setup using this resource. The corresponding AWS documentation is here: https://docs.aws.amazon.com/singlesignon/latest/userguide/scim-profile-saml.html

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8fa5814d-91e2-2f75-77e7-62a8597f340c
Version Independent ID: 458e93d7-a180-4c2f-5077-9b9d124d726a
Content: Tutorial: Azure Active Directory single sign-on (SSO) integration with Amazon Web Services (AWS)
Content Source: articles/active-directory/saas-apps/amazon-web-service-tutorial.md
Service: active-directory
Sub-service: saas-app-tutorial
GitHub Login: @jeevansd
Microsoft Alias: jeedes

@onitake
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

AFAIK, the old IAM-based SAML integration is still supported and used by some organizations. So it may be better to keep the documentation - but provide a new document that accurately describes how to integrate with AWS SSO. The title of the present article is a bit misleading.

Here's a better article from AWS: https://docs.aws.amazon.com/singlesignon/latest/userguide/azure-ad-idp.html

@onitake Azure AD does not support AWS SSO. It is a different product. So we do not recommend to use this approach as describe in the article you have given above. I will add this as a note in our main article.

#please-close

@onitake Azure AD does not support integration with AWS SSO. It is a different product. So we do not recommend to use this approach as describe in the article you have given above. I will add this as a note in our main article.

That makes no sense. You're supporting an older product (AAD integration via AWS IAM), but not the one recommended and supported by the vendor (AWS SSO)?
In any case, my biggest issue with the article is that it names AWS Single Sign-On in the title, but doesn't cover the AWS product of that name. If you're not going to provide documentation that covers the actual AWS SSO setup, then please at least make it clear that you're covering the integration via AWS IAM.


Ability to affect the logger context

Q
A

Bug?
no

New Feature?
yes

The PSR logger interface specifies a context to provide extra information for the message. Since the current implementation is bounded to a predefined context it is not possible to affect this. We could use a message factory or something to generate messages in a custom way as solution.

do you have some example use cases for when a specific context would be useful? and do you mean that the application should be able to add context, or that this factory would add more context?

For example, custom details like: hostname, client name, server name, api version, uuid, ... such details are useful for a more advanced logging. Yes the application should be able to add context, but we could also solve this with a factory. Which approach do you prefer?

don't we log the whole request and response with all headers in the context?

Do you suggest that any relevant details should be passed by the Request or Response in manner to receive such information?

sorry, i was confused then. host name, client and server names, api version are often part of the request and response. but if you mean application (non-http) information, then of course that should not go into the http requests.
do you need to log those information only with logs of the logger plugin, or elsewhere? if you want to log them with each log entry, maybe using a monolog processor would be the best solution?

no worries, glad you found a good solution. the fewer things we add to the plugins, the easier they will be to understand, use and maintain ;-) but don't hesitate to ask again if you need something. and when changing the code of the plugins is the best solution, we also will do that.


Refactor.replace did

@elwinschmitz

I tested a full happy flow of this
not all sidepaths though like the specific change FSP workflows etc. It should be pretty straightforward though right..
I imposed a length of (29,36) on referenceId, where 29 is the (minimum) length of (old) dids, while 36 is the length of (new) uuids.
BTW: I have now imposed this same constraint at like 10 different places, instead of at one place and then referenced that in all .dto files (suggestion welcome)
I came across an error with 'whatsappPaymentMultiple' not being available in my seeded validation program, so I thought it a good idea to impose a fallback to the single message
I also came across (many) issues relating to the handleProof component, but that will be handled in the other subtask.
I haven't waited for check results yet ;)


tweak CI

@homunkulus r+

:pushpin: Commit 7efe470 has been approved by japaric

:hourglass: Testing commit 7efe470 with merge 7efe470...

:broken_heart: Test failed - status-travis

@homunkulus r+

:pushpin: Commit 25bf0d7 has been approved by japaric

:hourglass: Testing commit 25bf0d7 with merge 25bf0d7...

:broken_heart: Test failed - status-travis

:pushpin: Commit 56b1f55 has been approved by japaric

:hourglass: Testing commit 56b1f55 with merge 56b1f55...

:broken_heart: Test failed - status-travis

@homunkulus r+

:pushpin: Commit bb04da3 has been approved by japaric

:hourglass: Testing commit bb04da3 with merge bb04da3...

:sunny: Test successful - status-travis
Approved by: japaric
Pushing bb04da347e19246592ce6d2acacfc455343af0a1 to master...


Bump Swift version to Swift 4.2.1

This PR bumps the swift version number to 4.2.1  now that the branch is open for post-4.2 changes.

Resolves rdar://problem/43124673

Ugh, wrong branch.


[Testing] Upgrade hadolint to 2.10.0

The current version of hadolint (1.11.1) is only suitable for linting Linux commands in Dockerfile. It fails to recognize Windows command syntax.
HADOOP-18133 adds Dockerfile for Windows. Thus, it's essential to upgrade hadolint to 2.10.0 which has the ability to recognize Windows command syntax.

Description of PR
How was this patch tested?
For code changes:

[ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/1/console in case of problems.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
53m  4s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
yamllint
0m  0s

yamllint was not available.

+0 :ok:
hadolint
0m  0s

hadolint was not available.

+0 :ok:
shellcheck
0m  0s

Shellcheck was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 50s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
34m 59s

trunk passed

+1 :green_heart:
mvnsite
22m 28s

trunk passed

+1 :green_heart:
shadedclient
96m 18s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
27m 15s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
20m 59s

the patch passed

+1 :green_heart:
shadedclient
43m 53s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
24m 26s

root in the patch passed.

+1 :green_heart:
asflicense
1m 14s

The patch does not generate ASF License warnings.

248m 46s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4950

Optional Tests
dupname asflicense codespell detsecrets yamllint hadolint shellcheck shelldocs mvnsite unit shadedclient

uname
Linux 5f3de747c9ed 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1750ea072e5abc66cc71430f09eeb66c8ba11f0a

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/1/testReport/

Max. process+thread count
529 (vs. ulimit of 5500)

modules
C: hadoop-client-modules/hadoop-client-check-invariants . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/1/console

versions
git=2.9.5 maven=3.6.3

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/1/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/3/console in case of problems.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
19m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
yamllint
0m  0s

yamllint was not available.

+0 :ok:
shellcheck
0m  0s

Shellcheck was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+0 :ok:
hadolint
0m  0s

hadolint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
20m 16s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 35s

trunk passed

-1 :x:
mvnsite
7m 29s
/branch-mvnsite-root.txt
root in trunk failed.

+1 :green_heart:
shadedclient
82m 38s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 38s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
26m 23s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-1 :x:
mvnsite
6m 59s
/patch-mvnsite-root.txt
root in the patch failed.

+1 :green_heart:
shadedclient
29m 18s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
22m  1s

root in the patch passed.

+1 :green_heart:
asflicense
1m 32s

The patch does not generate ASF License warnings.

184m 29s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4950

Optional Tests
dupname asflicense codespell detsecrets yamllint shellcheck shelldocs hadolint mvnsite unit shadedclient

uname
Linux 6047bcfbe6af 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 59854e0c77bc5e2f0068f6f9315bf3d30d66d663

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/3/testReport/

Max. process+thread count
630 (vs. ulimit of 5500)

modules
C: hadoop-client-modules/hadoop-client-check-invariants . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4950/3/console

versions
git=2.27.0 maven=3.6.3

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Design of header changes

As of now, the header looks quite simple and plain. Design and implement a new header with other important information. You can take inspiration from other websites and re-design it.

I would like to Work on this issue.
Can you Tell Which header you are talking about is it of Website or The PDF's header which is generated ..?

The header of the website with the GIF and the name. It looks okay as of now. Try finding some inspirations for style and color pallettes.

I would like to work on this issue. I want to add animation to the header. As well as various text styling like giving it a light neon look, flickering or brick look.

Can I work on this?


[Maintainership] Add GitHub labels for triage and awaiting_response

Housekeeping

[X] I am a maintainer of dbt-common

Short description
Add GitHub labels for triage and awaiting_response
Acceptance criteria
Labels exist here: https://github.com/dbt-labs/dbt-common/labels
Suggested Tests
N/A
Impact to Other Teams
None.
Will backports be required?
No
Context
No response

Hopefully that wasn't too obnoxious.

Hopefully that wasn't too obnoxious.

No better proof that the labels are there!


[Bug] Dependency name is too long / file not found

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
We've faced some too-long file paths in dbt project - looks like DBT core incorrectly handled it. The full path with the following prefix should not exceed 260 symbols.
Example
C:\Users\<USER>\AppData\Local\Temp\dbt-downloads-qdbfjp57\5578bf4e628e11aeddd9aca27cc791fce\tests\telescope\skill_link\skills\skills_by_applicant_id\sources\test_sl_applicant_skill_from_skill_update_flat\test_sl_applicant_skill_from_skill_update_flat__applicant_sources.sql

In this case dbt raise FileNotFoundError
Expected Behavior
The long file path is handled correctly within dbt without the need to shorten the file name.
Steps To Reproduce

Create a full file path with a length of more than 260 characters within the dbt project.
Run dbt compile

In our case, we created dbt unit test and during dbt test, we got an error  FileNotFoundError
Relevant log output
[0m20:09:06.573317 [error] [MainThread]: Encountered an error:
[WinError 3] The system cannot find the path specified: 'C:\\Users\\SIARHE~1\\AppData\\Local\\Temp\\dbt-downloads-aonydoz4\\5578bf4e628e11aedd9aca27cc791fce\\tests\\telescope\\skill_link\\skills\\skills_by_applicant_id\\sources\\test_sl_applicant_skill_from_skill_update_flat\\test_sl_applicant_skill_from_skill_update_flat__applicant_sources.sql'
[0m20:09:06.576334 [error] [MainThread]: Traceback (most recent call last):
  File "C:\Soft\Python311\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\SIARHE~1\\AppData\\Local\\Temp\\dbt-downloads-aonydoz4\\5578bf4e628e11aedd9aca27cc791fce\\tests\\telescope\\skill_link\\skills\\skills_by_applicant_id\\sources\\test_sl_applicant_skill_from_skill_update_flat\\test_sl_applicant_skill_from_skill_update_flat__applicant_sources.sql'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Soft\Python311\Lib\site-packages\dbt\cli\requires.py", line 138, in wrapper
    result, success = func(*args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Soft\Python311\Lib\site-packages\dbt\cli\requires.py", line 101, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Soft\Python311\Lib\site-packages\dbt\cli\requires.py", line 201, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Soft\Python311\Lib\site-packages\dbt\cli\requires.py", line 247, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Soft\Python311\Lib\site-packages\dbt\cli\main.py", line 447, in deps
    results = task.run()
              ^^^^^^^^^^
  File "C:\Soft\Python311\Lib\site-packages\dbt\task\deps.py", line 217, in run
    self.lock()
  File "C:\Soft\Python311\Lib\site-packages\dbt\task\deps.py", line 193, in lock
    with downloads_directory():
  File "C:\Soft\Python311\Lib\contextlib.py", line 144, in __exit__
    next(self.gen)
  File "C:\Soft\Python311\Lib\site-packages\dbt\deps\base.py", line 43, in downloads_directory
    system.rmtree(DOWNLOADS_PATH)
  File "C:\Soft\Python311\Lib\site-packages\dbt_common\clients\system.py", line 698, in rmtree
    return shutil.rmtree(path, onerror=chmod_and_retry)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Soft\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Soft\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Soft\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Soft\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  [Previous line repeated 5 more times]
  File "C:\Soft\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Soft\Python311\Lib\site-packages\dbt_common\clients\system.py", line 642, in chmod_and_retry
    os.chmod(path, stat.S_IREAD | stat.S_IWRITE)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\SIARHE~1\\AppData\\Local\\Temp\\dbt-downloads-aonydoz4\\5578bf4e628e11aedd9aca27cc791fce\\tests\\telescope\\skill_link\\skills\\skills_by_applicant_id\\sources\\test_sl_applicant_skill_from_skill_update_flat\\test_sl_applicant_skill_from_skill_update_flat__applicant_sources.sql'

[0m20:09:06.578314 [debug] [MainThread]: Command `dbt deps` failed at 20:09:06.578314 after 40.41 seconds

Environment
- OS: Windows
- Python: 3.11
- dbt: 1.8

Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context

Adapter dbt-athena

Thanks for reaching out @eg-tan !
Which version of Windows are you using? Are you using LongPathsEnabled registry setting, by any chance?
From digging into this a little more, it sounds like Windows has traditionally enforced a MAX_PATH limit of 260 characters for file paths. But starting with Windows 10 and Windows Server 2016, it became possible to remove this limitation by changing the system settings. Specifically, by enabling the "LongPathsEnabled" registry setting or modifying the Enable NTFS long paths group policy setting.

Got it, thanks.
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force from admin worked for me


Word-level confidence

Hi there,
Is that possible to get a word-level confidence in recognition response? If so, how can this be achieved? If not, maybe I can contribute by adding it (but I may need some advice along the way).
Thanks!
Ariel

Implemented in gst-kaldi-nnet2-online. Now, word confidence scores can be included with word alignment information in the full results.

Great, thank you!


ontology metadata: tracking changes to which terms are mireoted, and why

From<EMAIL_ADDRESS>on August 17, 2009 12:23:38
If an ontology mireots a term and then decides not to, what should be said. Obsoleting the
mireoted term is clearly wrong. Rather it is the use of the term in the importing ontology that needs
to be recorded.
Original issue: http://code.google.com/p/information-artifact-ontology/issues/detail?id=49

From<EMAIL_ADDRESS>on August 17, 2009 17:19:48
As the mireoted term still exist in the source ontology, do you have an example of
why we may want to record the fact that we stopped using it?
Is it a technical issue, as if IAO stops mireoting (e.g. from OBI) the end user needs
to import OBI on its own?
Or is it to explain why we chose to not mireot anymore a term: in this case the
reference the user made to the external term is probably still the one he intended to
make, and I am not sure we need to record the change in the target ontology?


OCM-7289 | feat: revert maxUnavailable and maxSurge for NodePool

Revert commit 2140 (create/update of maxUnavailable and maxSurge for NodePool)

/lgtm


Add internal flag to disable property upgrades

We can now pass "org.gradle.internal.disable.property.upgrades=true" to as system property to disable property upgrades.
We now add interface BytecodeUpgradeInterceptor to interceptors that we need to filter when a flag is set.
We can probably do things a bit nicer in https://github.com/gradle/gradle/issues/26396, but it needs a bit more extra work.
Part of https://github.com/gradle/gradle-private/issues/4001

@bot-gradle test this

I've triggered the following builds for you. Click here to see all build failures.

PullRequestFeedback build

Let's close this as with recent changes we'd do this another way.


Set width for evaluation form and supporting component containers to 920px

Summary
This does what the title says and aligns the first page of evaluation reports to this width assumption.
Setup to Run Your Code

💻 You will need to use three separate terminals to test this locally.
Terminal 1
Start the Storybook locally.
make storybook

Terminal 2
Start the UI locally.
make client_run

Terminal 3
Start the Go server locally.
make server_run

Additional steps

Access the office app
Login as a QAE/CSR
Navigate to the EVLRPT move (local/ephemeral) or another move (other deployed environments)
Click on quality assurance
Create or edit a report
View the form for the report

Frontend

[ ] User facing changes have been reviewed by design.
[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] There are no new console errors in the browser devtools
[ ] There are no new console errors in the test output
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.

Screenshots

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against 8bc475717cb0c3a90f51b75b93efcf44279fbf85


[Documentation] Sending reserved characters without encoding in query params

Description:
As explained in [1] there could be occurrences where characters such as (";", "/", "?", ":", "@", "=" and "&" ) should be able to be passed as reserved characters instead of encoding. In order to do this, we need to add  '+' as a prefix when adding {query.param} to a uri-template.
Affected Product Version:
EI-6.1.1
Steps to reproduce:

Enable wire logs in integrator profile.
Create and deploy the API given in [1].
Invoke API as given in [1].
In debug logs, it could be noted that the request sent out from EI has encoded the reserved character. ("GET ?symbol=some%3Avalue HTTP/1.1[\r][\n]")
Update the endpoint of the API as below.

 <endpoint>
       <http method="GET" uri-template="http://abc.com?symbol={+query.param.type}"/>
  </endpoint>

Invoke the API again and it could be noted that the reserved character is not encoded.
(GET ?symbol=some:value )

Related Issues:
https://wso2.org/jira/browse/ESBJAVA-3435
[1] https://wso2.org/jira/browse/ESBJAVA-3435

This is achievable using use “*” to define the URL path when creating the API, It's documented in "REST APIs" section in MI docs.


issues downloading data

Hi, the link https://yadi.sk/d/NJSeeOk5bYm7V offers the data to download using a service called Yandex. The zipped archive seems to be 35GB, but download stops after 4.5GB. I've tried on multiple connections (including FIOS), so I'm pretty sure its not the connection.
I was wondering if its possible to split the folders into separate ZIPs or offer some other means to download? Thanks.

Hi, the interface is indeed quite strange. I would recommend you the following options:

try to download not the whole dataset at once but for specific date. For example https://yadi.sk/d/NJSeeOk5bYm7V/Video/20140917 . The same for data other than video.
register on yandex, install their yandex.disk client and save it to your disk. Then it should manage somehow to sync the data from cloud to your PC
also Yandex.Disk supports webdav https://yandex.com/support/disk/webdav.html , so maybe it's possible to navigate easily using it instead

Thanks - I'm going ahead with individual downloads. Unfortunately, yandex disk only gives us 10GB which is not sufficient to clone your data and webdav only works with your own acct (so I can't use it to download yours).
Thank you for your excellent work!

As I remember they recently announced that shared folders will no longer
consume your own space. But I have never checked it myself.
On Wed, Mar 28, 2018, 16:34 Arjun Roychowdhury<EMAIL_ADDRESS>wrote:

Closed #3 https://github.com/blindmotion/normalizer/issues/3.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/blindmotion/normalizer/issues/3#event-1545748464, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABZppb9RRHB6_9cRQz4UyjJgkAxwMeSTks5ti59cgaJpZM4S89FS
.


Keep develop up to date with master's mac build changes

please commit to develop first

TBH I told him to pr to master, my bad.


Revert "CB-19604 Disable markplace related tests for Azure until they…

… are fixed"
This reverts commit e76ff1042b959a54b2b3a712447090727feee497.
See detailed description in the commit message.

Can you please change the branch to CB-2.71.0 ?


fix(TDI-42653): reduce REST API calls

What is the current behavior? (You can also link to an open issue here)
https://jira.talendforge.org/browse/TDI-42653
What is the new behavior?
Assign service client when connection is validated to prevent another API call to get an already acquired access token.
Provide a class to cache and share acquired access tokens.
Please check if the PR fulfills these requirements

[X] The commit message follows Talend standard
[X] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features) ?
[X] The code coverage on new code >75%
[X] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[X] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[X] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Pull request test result: success :star:
Sonar results

Pull request test result: failure :x:

Pull request test result: success :star:
Sonar results

instance could be initialized lazily, but that is not critical. It looks ok if MarketoAccessTokenPool instance is not going to be used in multithreaded environment. Otherwise, the code should be rewritten.

That's not the case so should be ok.

Pull request test result: success :star:
Sonar results


How to download API_KEY from http://www.wunderground.com?

Hi, I've registered new API KEY but can't find the way to download it.
Please tell me how.

@ruddfawcett
Thanks!!


Defining enums

Hi, so JSON schema supports like
{
  "type": "string",
  "enum": ["red", "green", "blue"]
}

And so does Pydantic
from enum import Enum

class FruitEnum(str, Enum):
    APPLE = 'apple'
    BANANA = 'banana'
    ORANGE = 'orange'

class ToolEnum(IntEnum):
    HAMMER = 1
    SCREWDRIVER = 2
    WRENCH = 3

from pydantic import BaseModel

class FruitBasket(BaseModel):
    fruit: FruitEnum
    quantity: int

class ToolBox(BaseModel):
    tool: ToolEnum
    brand: str

Does Shale support something like this today? If not, what would be the way to go about adding support?

Ruby doesn't have enums, you would normally use symbols instead.
You can use symbols in Shale like this:
class SymbolType < Shale::Type::Value
  def self.cast(value)
    value&.to_sym
  end
end

class Person < Shale::Mapper
  attribute :color, SymbolType
end

person = Person.from_json(<<~END)
{
  "color": "red"
}
END

p person.color
# => :red

Or you could use some abstraction like this, but in my opinion it doesn't add any value and just makes things confusing:
module Color
  RED = :red
  GREEN = :green
  BLUE = :blue
end

class ColorType < Shale::Type::Value
  def self.cast(value)
    case value&.to_s
    when 'red'
      Color::RED
    when 'green'
      Color::GREEN
    when 'blue'
      Color::BLUE
    end
  end
end

class Person < Shale::Mapper
  attribute :color, ColorType
end

person = Person.from_json(<<~END)
{
  "color": "red"
}
END


refactor: consolidate pallet event names

Description
Event names should be in the style of AddLiquidity instead of LiquidityAdded.
Actual solution
There is a mix of these styles. Some pallets use Added, some Add and some have both.
Expected solution
One style of event names everywhere.

Hi, shouldn't event names be in past-tense, as something that happened? I think that's the intention and convention acc. to the pallet documentation. That would be in-line with general DDD convention, where events name reflect business logic. Since naming can get tricky, I'm happy to track down all the events and document them. Would that be helpful?
// Events are a simple means of reporting specific conditions and circumstances
// that have happened that users, Dapps and/or chain explorers would find
// interesting and otherwise difficult to detect.

https://substrate.dev/docs/en/knowledgebase/runtime/pallets#skeleton-of-a-pallet

Hi, shouldn't event names be in past-tense, as something that happened? I think that's the intention and convention acc. to the pallet documentation. That would be in-line with general DDD convention, where events name reflect business logic. Since naming can get tricky, I'm happy to track down all the events and document them. Would that be helpful?
// Events are a simple means of reporting specific conditions and circumstances
// that have happened that users, Dapps and/or chain explorers would find
// interesting and otherwise difficult to detect.

https://substrate.dev/docs/en/knowledgebase/runtime/pallets#skeleton-of-a-pallet

Yes, after some discussions, we go with the past tense in form of AddedLiquidity. Details above already updated.

Thanks, once changed, would that be a breaking change? Anything relies on these events?

Thanks, once changed, would that be a breaking change? Anything relies on these events?

Yes. Anything that changes data structures that are put into blockchain is breaking change.


I have trouble when i click sell products

Fatal error: Uncaught Error: Class 'Config' not found in C:\xampp\htdocs\umkm3\assets\seller\index.php:22 Stack trace: #0 {main} thrown in C:\xampp\htdocs\umkm3\assets\seller\index.php on line 22

You need add missing file config.php in system/library folder. Just copy from original opencart.


MFA:Gauth: Fix potential TOTP secret key leakage due to HTTP request logging

Details
While deploying CAS for my employer, I skimmed through the load balancer's
logs and that's how I stumbled upon the issue described below. In short, the user's
TOTP secret key is logged as it's sent as a GET parameter during the registration
process. Most of the times every middleware that terminates TLS (such as reverse proxies,
load balancers and so on) tend to also log GET parameters. CAS users could unknowingly
writing TOTP secret keys to disk, and probably ship those logs to other machines (ELK stacks and so on).
An adversary could potentially steal those just by having access to logs somewhere
other than the CAS hosts themselves.
Brief description of changes applied
An HTTP endpoint [0] currently serves the purpose of grabbing the user's
TOTP secret key and produce the required QR code that in turn gets
scanned by the user's phone, as shown below:
https://github.com/apereo/cas/blob/acdc6cfd9e919e7feb54ad1ad0cbea44db88733d/support/cas-server-support-otp-mfa-core/src/main/java/org/apereo/cas/otp/web/flow/rest/OneTimeTokenQRGeneratorController.java#L28-L33
A security issue arises in the Google Authenticator registration
template (below) because the aforementioned HTTP endpoint is sourced in an
img tag, passing the user's secret key as a GET parameter. This poses
an immediate threat to the confidentiality of the key.
GET parameters get logged in proxies (e.g. Load balancers, reverse
proxies etc), hence CAS users might be unknowingly logging these.
https://github.com/apereo/cas/blob/c839b62772078bf347c45e67bb2b51d0aa622ff0/support/cas-server-support-thymeleaf/src/main/resources/templates/casGoogleAuthenticatorRegistrationView.html#L108
This PR attempts to provide the registration template with the QR code image, base64 encode
it and embed it using the base64:image/png scheme.
NOTE: This is a first draft to discuss the matter, see if you're interested in me merging this one.
Nevertheless, this PR is working as-is, despite lacking the two points below.
There are certainly two things missing here:

Cleaning up the existing /otp/qrgen endpoint
Potentially add more tests

Any possible limitations, side effects, etc
I've verified that this endpoint is only used there, so the shouldn't be any side-effects.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Linos Giannopoulos seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

I just got the time to review what you've said, excuse my late response

In principal, this looks good.
* Please fix all build failures.

There should be none now

* Be sure to add/update tests

I've updated the tests, where needed, but I am not sure which part would need more testing, meaning that the QR generation class already existed and it's quite hard to test that the produced QR code is in fact a valid QR code, and it also contains the values we need it to.

* Make sure you post the same PR for the master branch.

I'll open another PR for master shortly. Since this is a security fix, should we also backport it for 6.1.X? I am not sure how we could go about doing this, since my other PR for 6.1.X got automatically closed due to 6.1.X being on SPM mode.

Thanks much for the changes. Will review.

I'll open another PR for master shortly. Since this is a security fix,

I don't really see this as a "security fix". Security issues are expected to be reported as such:
https://apereo.github.io/cas/developer/Sec-Vuln-Response.html
Note the big yellow box.
So no backports.

Thanks much for the changes. Will review.

I'll open another PR for master shortly. Since this is a security fix,

I don't really see this as a "security fix". Security issues are expected to be reported as such:
https://apereo.github.io/cas/developer/Sec-Vuln-Response.html

Right, that's a misstep on my part, but I still consider this one security issue that deserves a notification to the users,
since valid TOTP seeds can still be on their servers long after the fix is deployed.

Note the big yellow box.
So no backports.

**CAS - Vulnerability Response**CAS - Enterprise Single Sign-On for the Web

Since the information and details are already public, I am not sure how a notification can be handled without causing panic, specially when taking into account the fact that no one I know on the dev team has available cycles to cut a patch for 6.1.x. I'll have to discuss that with the sec team to see what might be the best approach, and whether anyone has any availability or interest to actually merge and accept a patch for 6.1. If not, it's m00t as our normal process cannot apply.
I do appreciate your time and contributions; truly. In the future if you find something deemed as a security issue severe enough to merit an actual fix, please share that information privately so the process can be followed. Look for the appropriate process, and if you don't find one, please ask before stepping into action.
I'll send a note to the security team to see what happens.
PS You don't have to do a PR for master; you're welcome if you wanted to spare the time but this should be easily cherry-pickable.

Message sent. Stand by please.

Since the information and details are already public, I am not sure how a notification can be handled without causing panic, specially when taking into account the fact that no one I know on the dev team has available cycles to cut a patch for 6.1.x. I'll have to discuss that with the sec team to see what might be the best approach, and whether anyone has any availability or interest to actually merge and accept a patch for 6.1. If not, it's m00t as our normal process cannot apply.
I do appreciate your time and contributions; truly. In the future if you find something deemed as a security issue severe enough to merit an actual fix, please share that information privately so the process can be followed. Look for the appropriate process, and if you don't find one, please ask before stepping into action.
I'll send a note to the security team to see what happens.
PS You don't have to do a PR for master; you're welcome if you wanted to spare the time but this should be easily cherry-pickable.

You're absolutely right about this one, I rushed into opening a PR for this one (this would probably be simple issue on Github if you had them open, with the same repercussions of course). I'll keep it in mind, in case I stumble upon something else. Regardless of whether you opt to backporting this to 6.1, I think that after pushing the fix, a notification of sorts should be sent out to users (through the mailing list?), urging for updates and further action on their side, e.g. wiping logs and/or forcibly resetting all users' TOTP keys.

In discussing this with relevant teams, we have decided to proceed with a release this week with follow-up announcements where needed. @leleuj might be in touch to review and handle the release later in the week. Given the body the patch, the PR history/convos have been "sanitized" to make sure we can follow the usual process as much as it can be done. We will handle the port to the main/master branch as well.
As always, thank you for your patience and contribution.

Alright, I'll be available in case more changes are needed here. The 2 failed builds might contain some flaky test, possibly? I've ran them locally and they seem to pass, and I cannot get a pointer as to what I could do to fix them. All in all, they seem unrelated to my changes


Staging to production

Demo event version

Codecov Report
Base: 47.54% // Head: 33.01% // Decreases project coverage by -14.52% :warning:

Coverage data is based on head (f1804c2) compared to base (c250e9b).
Patch coverage: 5.45% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##             main      #12       +/-   ##
===========================================
- Coverage   47.54%   33.01%   -14.53%
===========================================
  Files           6       11        +5
  Lines          61      106       +45
  Branches       12       21        +9
===========================================
+ Hits           29       35        +6
- Misses         32       71       +39

Impacted Files
Coverage Δ

src/components/Frontpage.jsx
0.00% <0.00%> (ø)

src/components/Report.jsx
0.00% <0.00%> (ø)

src/components/ReportList.jsx
0.00% <0.00%> (ø)

src/components/inventory/InventoryReport.jsx
0.00% <0.00%> (ø)

src/services/inventory-service.js
0.00% <0.00%> (ø)

src/services/user-service.js
0.00% <0.00%> (ø)

src/components/inventory/InventoryForm.jsx
75.00% <50.00%> (-5.00%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


vmlatency, checkup: Add OwnerReference to VMIs

Currently, if a problem occurs during the teardown process, one or more VMIs could stay in the namespace as leftovers.
Add OwnerReference to source and target VMIs, in order to use the automatic K8s garbage collector, so they will be garbage-collected when the user-supplied ConfigMap is deleted.
Signed-off-by: Orel Misan<EMAIL_ADDRESS>

Change: Updated commit message and PR description

@orelmisan how did you verify that it actually works? (I am asking because we don't have e2e tests yet).

I've tested it locally:

Disabled the teardown.
Listed the VMIs to see they are still there.
Removed the user-supplied ConfigMap,
Listed the VMIs to see they are removed.

Change: Get the ownerReference as an input to checkup.New(), so checkup will not care what object is in the owner reference.
@ormergi could you please take another look?

Change: Removed unused explicit field initialization.

I prefer how it was before, the UID should be explicitly defined in the checkup object.
Taking it from another object is actually moving the responsibility of the UID to the checkup.

Also, IMO having the code more compact doesn't worth it in this case.
The amount of parameters you need to pass to functions (checkup.New and newLatencyCheckVmi) is reduced but it doesn't improve readability.

The UID is used in two places:

OwnerReference.
Value of the label that is used for podAntiAffinity.

Why is it important to explicitly get it?
The value is the same.
Several interfaces where cleaned, why is it a bad thing in your opinion?

Rebase

I prefer how it was before, the UID should be explicitly defined in the checkup object.
Taking it from another object is actually moving the responsibility of the UID to the checkup.

Also, IMO having the code more compact doesn't worth it in this case.
The amount of parameters you need to pass to functions (checkup.New and newLatencyCheckVmi) is reduced but it doesn't improve readability.

The UID is used in two places:
1. OwnerReference.

2. Value of the label that is used for podAntiAffinity.

Why is it important to explicitly get it? The value is the same.
Several interfaces were cleaned, why is it a bad thing in your opinion?

The cleanup came on the expense of an nontrivial dependency.
The skeleton has explicitly used the UID of the configmap, but it is a generic decision.
The suggested change says that the UID used is up to the checkup package (not the mainflow skeleton) itself to decide and the decision is to take it from the owner object.
Considering that other checkups will be written, we can either provide a default UID (of the configmap) through a skeleton lib or not provide one at all. If we provide it, the checkup writer could still override it.
If we will not provide it, it means the responsibility is completely on the checkup writer and we cannot assist her with this thing.
I prefer we do provide the UID from above explicitly, and it is up to the specific checkup to use it or use something else.
And as I commented before, the owner object as an input is the real problem IMO.

In an offline meeting with @EdDev we've concluded that its more logical to use the checkup's Job underlying Pod as the owner, because it is most probably be removed, unlike the user-supplied ConfigMap which could linger.
One option discussed to get the Pod's name and UID is using downward API as environment variables.

Change: Create the ownerReference inside the checkup, based on the Pod's UID and name.
Change 2: Added a test to check that ownerReference is not created when Pod's UID is unknown.

Second commit:

in order to get the Pod's name

I guess you meant ID not name, right?

Yes, thank you.

Changed the second commit's message in order to answer @EdDev's comment.

Change: Inserted the Pod's name and UID to the checkup's config.
This is an answer to @EdDev's comment.

Change: Aligned the order of the Pod name and UID to be uniform.

Change: Fixed a bug in TestCheckupSetupShouldNotCreateOwnerReferenceWhenPodUIDIsEmpty().


The launcher has wrong scaling with highDPI

Sorry to make another issue...
I tried the highDPI but the scaling of the fonts is a little bit off:

Any idea why ?

Sorry I don't really understand with that

I think the reason is that Godot Project Manager doesn't support HiDPI properly.

oh yea this might be the reason. So it is not on your site :)
I think you can close.
TY

@toger5  You can test with latest commit of godot


UDP Port Forwarding in the Nat Module

Should there be an ability to specify UDP port forwarding in the nat module? https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/nat.nix
It would necessitate an extra property in the forwarding list specification, like protocol = "udp".

@tnias Has done this!