add oauth-proxy to rawdeployments if odh auth label is present

What this PR does / why we need it:
This PR adds the following :

If the isvc has label "security.opendatahub.io/enable-auth" = "true"
-- an oauth proxy container is added to the deployment
-- http port is replaced by https port in the service
-- "service.beta.openshift.io/serving-cert-secret-name" is added to the service to allow creation of the tls secret
Modified the kserve raw deployment spec to use the kserve-sa service account (previously no SA was specified, the default SA was used)
This is needed so that oauth proxy can use the kserve-sa service account to perform auth delegation.
The service account, CRB and Route are created in https://github.com/opendatahub-io/odh-model-controller/pull/274

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes # https://issues.redhat.com/browse/RHOAIENG-10291, https://issues.redhat.com/browse/RHOAIENG-13444
TEST WITH: https://github.com/opendatahub-io/odh-model-controller/pull/274
Type of changes
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Feature/Issue validation/testing:

Deploy ODH/RHOAI:
Install the DSC (kserve spec below) :

    kserve:
      defaultDeploymentMode: RawDeployment
      devFlags:
        manifests:
          - contextDir: config
            sourcePath: overlays/odh
            uri: 'https://github.com/VedantMahabaleshwarkar/kserve/tarball/devflags'
          - contextDir: config
            sourcePath: ''
            uri: 'https://github.com/VedantMahabaleshwarkar/odh-model-controller/tarball/devflags'
      managementState: Managed
      serving:
        ingressGateway:
          certificate:
            type: OpenshiftDefaultIngress
        managementState: Managed
        name: knative-serving

To test enabling route for a particular ISVC
--  Create any kserve isvc+SR
-- isvc should have annotation : "serving.kserve.io/deploymentMode" : RawDeployment
-- isvc should have label : "networking.kserve.io/visibility": "enable-route"

To test route with auth:
--  Create any kserve isvc+SR
-- isvc should have annotation : "serving.kserve.io/deploymentMode" : RawDeployment
-- isvc should have label: "security.opendatahub.io/enable-auth" = "true"
-- isvc should have label: "networking.kserve.io/visibility": "enable-route"

To test Inference without auth:
-- remove isvc label "security.opendatahub.io/enable-auth" = "true"

Now inference should work without token

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

[x] Have you added unit/e2e tests that prove your fix is effective or that this feature works?
[x] Has code been commented, particularly in hard-to-understand areas?
[x] Have you made corresponding changes to the documentation?

Release note:

Added oauth-proxy authentication for Kserve RawDeployment Routes

Re-running failed tests

/rerun-all - rerun all failed workflows.
/rerun-workflow <workflow name> - rerun a specific failed workflow. Only one workflow name can be specified. Multiple /rerun-workflow commands are allowed per comment.

tests passing locally, the failures look like test infra failures
/retest-required

/retest-required

not sure what's happening with the tests, they are passing locally :(

the e2e raw test failure is because https://github.com/opendatahub-io/kserve/blob/master/test/scripts/openshift-ci/run-e2e-tests.sh needs to be updated with the new behavior

Manual testing
Since controller functionality is the concern, all tests are using this gist as a reference, which is using a SKLearn runtime with a basic model. Despite the SKLearn runtime is not an ODH-supported one, we still should be able to verify correctness of controller functionality.
Testing is done by deploying kserve-controller from this PR, and also odh-model-controller from PR opendatahub-io/odh-model-controller#274. ODH setup is done with a custom build of odh-operator. The setup is a standard Serverless setup. For the testings, the deploymentMode annotation is used to switch the mode to Raw.
Quick regression testing for Serverless mode
:green_circle: OK
Follow the mentioned gist as is: https://gist.github.com/israel-hdez/af374562ef9e5b9d80890aa6f0bce20d
Deploy a slim InferenceService in Raw deployment mode
:red_circle: Fails
Follow the gist, but use these annotations in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"

:green_circle: The model seems to deploy fine
:green_circle: SVC looks OK
:green_circle: No oauth-proxy container
:green_circle: No RoleBinding
:green_circle: No Route
:red_circle: Bad endpoint is reported
oc get isvc sklearn-v2-iris-test1
NAME                    URL                                                          READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION   AGE
sklearn-v2-iris-test1   https://sklearn-v2-iris-test1-kserve-raw-tests.example.com   True                                                                  10m
# ---
oc get isvc sklearn-v2-iris-test1 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test1-predictor.kserve-raw-tests.svc.cluster.local # OK
components:
  predictor:
    url: http://sklearn-v2-iris-test1-predictor-kserve-raw-tests.example.com # Wrong
url: https://sklearn-v2-iris-test1-kserve-raw-tests.example.com # Wrong

Deploy a Raw InferenceService with explicit private label
:red_circle: Fails
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "cluster-local"

:green_circle: The model seems to deploy fine
:green_circle: SVC looks OK
:green_circle: No oauth-proxy container
:green_circle: No RoleBinding
:green_circle: No Route
:red_circle: Bad endpoint is reported
oc get isvc sklearn-v2-iris-test3
NAME                    URL                                                          READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION   AGE
sklearn-v2-iris-test3   https://sklearn-v2-iris-test3-kserve-raw-tests.example.com   True                                                                  73s
# ---
oc get isvc sklearn-v2-iris-test3 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test3-predictor.kserve-raw-tests.svc.cluster.local # OK
components:
  predictor:
    url: http://sklearn-v2-iris-test3-predictor-kserve-raw-tests.example.com # Wrong
url: https://sklearn-v2-iris-test3-kserve-raw-tests.example.com # Wrong

Deploy a Raw InferenceService with explicit exposed label
:red_circle: Fails
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "exposed"

:green_circle: The model seems to deploy fine
:red_circle: Reconcile error in odh-model-controller:
2024-12-04T21:43:14Z	ERROR	Reconciler error	{"controller": "inferenceservice", "controllerGroup": "serving.kserve.io", "controllerKind": "InferenceService", "InferenceService": {"name":"sklearn-v2-iris-test4","namespace":"kserve-raw-tests"}, "namespace": "kserve-raw-tests", "name": "sklearn-v2-iris-test4", "reconcileID": "cb88aa95-d15f-4c17-bca8-8fc409cfe671", "error": "2 errors occurred:\n\t* clusterrolebindings.rbac.authorization.k8s.io \"kserve-raw-tests-default-auth-delegator\" already exists\n\t* Route.route.openshift.io \"sklearn-v2-iris-test4\" is invalid: spec.port.targetPort: Required value\n\n"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:227

:red_circle: Reconcile error in kserve-controller (surely a consequence of the previous):
{
  "level": "error",
  "ts": "2024-12-04T21:37:26Z",
  "msg": "Reconciler error",
  "controller": "inferenceservice",
  "controllerGroup": "serving.kserve.io",
  "controllerKind": "InferenceService",
  "InferenceService": {
    "name": "sklearn-v2-iris-test4",
    "namespace": "kserve-raw-tests"
  },
  "namespace": "kserve-raw-tests",
  "name": "sklearn-v2-iris-test4",
  "reconcileID": "75d4b916-3760-4e1c-879d-10a54ae7a20a",
  "error": "fails to reconcile ingress: Route.route.openshift.io \"sklearn-v2-iris-test4\" not found",
  "errorVerbose": "Route.route.openshift.io \"sklearn-v2-iris-test4\" not found\nfails to reconcile ingress\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:247\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224"
}

Deploy a slim Raw InferenceService with auth enabled
:red_circle: Fails
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "exposed"

:red_circle: The model does not deploy; kserve-controller reconcile error:
{
  "level": "error",
  "ts": "2024-12-04T21:50:14Z",
  "msg": "Reconciler error",
  "controller": "inferenceservice",
  "controllerGroup": "serving.kserve.io",
  "controllerKind": "InferenceService",
  "InferenceService": {
    "name": "sklearn-v2-iris-test5",
    "namespace": "kserve-raw-tests"
  },
  "namespace": "kserve-raw-tests",
  "name": "sklearn-v2-iris-test5",
  "reconcileID": "cdb5c0a4-1c92-47ca-b6cd-a598521c7894",
  "error": "fails to reconcile component: fails to create NewRawKubeReconciler for predictor: invalid character '}' looking for beginning of object key string",
  "errorVerbose": "invalid character '}' looking for beginning of object key string\nfails to create NewRawKubeReconciler for predictor\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components.(*Predictor).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components/predictor.go:342\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695\nfails to reconcile component\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224"
}

Deploy a Raw InferenceService with auth enabled and explicit private label
:red_circle: Fails
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "cluster-local"
    security.opendatahub.io/enable-auth: "true"

:red_circle: The model does not deploy; kserve-controller reconcile error:
{
  "level": "error",
  "ts": "2024-12-04T22:00:43Z",
  "msg": "Reconciler error",
  "controller": "inferenceservice",
  "controllerGroup": "serving.kserve.io",
  "controllerKind": "InferenceService",
  "InferenceService": {
    "name": "sklearn-v2-iris-test6",
    "namespace": "kserve-raw-tests"
  },
  "namespace": "kserve-raw-tests",
  "name": "sklearn-v2-iris-test6",
  "reconcileID": "6f55c5df-0843-4cea-9e5a-23fbb715445a",
  "error": "fails to reconcile component: fails to create NewRawKubeReconciler for predictor: invalid character '}' looking for beginning of object key string",
  "errorVerbose": "invalid character '}' looking for beginning of object key string\nfails to create NewRawKubeReconciler for predictor\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components.(*Predictor).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components/predictor.go:342\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695\nfails to reconcile component\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224"
}

Deploy a Raw InferenceService with auth enabled and route enabled
:red_circle: Fails
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "exposed"
    security.opendatahub.io/enable-auth: "true"

:red_circle: The model does not deploy; kserve-controller reconcile error:
{
  "level": "error",
  "ts": "2024-12-04T22:03:05Z",
  "msg": "Reconciler error",
  "controller": "inferenceservice",
  "controllerGroup": "serving.kserve.io",
  "controllerKind": "InferenceService",
  "InferenceService": {
    "name": "sklearn-v2-iris-test7",
    "namespace": "kserve-raw-tests"
  },
  "namespace": "kserve-raw-tests",
  "name": "sklearn-v2-iris-test7",
  "reconcileID": "cf46d9e9-ee52-48dc-a93c-47a17e023244",
  "error": "fails to reconcile component: fails to create NewRawKubeReconciler for predictor: invalid character '}' looking for beginning of object key string",
  "errorVerbose": "invalid character '}' looking for beginning of object key string\nfails to create NewRawKubeReconciler for predictor\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components.(*Predictor).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components/predictor.go:342\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695\nfails to reconcile component\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1695",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.1/pkg/internal/controller/controller.go:224"
}

Other notes

I just realized we are using a ClusterRoleBinding in odh-model-controller. I greatly suggest to use a namespaced RoleBinding.

/test images

Manual testing - round 2
(See previous test round for background)
Quick regression testing for Serverless mode
:green_circle: OK
Follow the mentioned gist as is: https://gist.github.com/israel-hdez/af374562ef9e5b9d80890aa6f0bce20d
Deploy a slim InferenceService in Raw deployment mode
:yellow_circle: OK, needs follow-up PR for fixes
Follow the gist, but use these annotations in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"

:green_circle: The model seems to deploys fine
:yellow_circle: Inference works (see the reason of the yellow at the end)
:green_circle: SVC looks OK
:green_circle: No oauth-proxy container
:yellow_circle: ClusterRoleBinding created... I though this would be created only when needed.
:green_circle: No Route
:yellow_circle: The Status is not right. But OK for now.
oc get isvc sklearn-v2-iris-test1 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test1-predictor.kserve-raw-tests.svc.cluster.local # Schema should be plain-text HTTP
components:
  predictor:
    url: http://sklearn-v2-iris-test1-predictor-kserve-raw-tests.example.com # Wrong
url: http://sklearn-v2-iris-test1-predictor.kserve-raw-tests.svc.cluster.local # OK

Deploy a Raw InferenceService with explicit private label
:yellow_circle: OK, needs follow-up PR for fixes
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "cluster-local"

:green_circle: The model seems to deploys fine
:yellow_circle: Inference works (see the reason of the yellow at the end)
:green_circle: SVC looks OK
:green_circle: No oauth-proxy container
:yellow_circle:  ClusterRoleBinding created... I though this would be created only when needed.
:green_circle: No Route
:yellow_circle: The Status is not right. But OK for now.
oc get isvc sklearn-v2-iris-test2 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test2-predictor.kserve-raw-tests.svc.cluster.local # Schema should be HTTP
components:
  predictor:
    url: http://sklearn-v2-iris-test2-predictor-kserve-raw-tests.example.com # Wrong
url: http://sklearn-v2-iris-test2-predictor.kserve-raw-tests.svc.cluster.local # OK

Deploy a Raw InferenceService with explicit exposed label
:red_circle: Fails
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "exposed"

:green_circle: The model seems to deploys fine
:red_circle: Inference doesn't work. Route returns application not available error page.
:green_circle: SVC looks OK
:green_circle: No oauth-proxy container
:yellow_circle:  ClusterRoleBinding created...
:red_circle: Route created; it doesn't work
:yellow_circle: The Status is not right. But OK for now.
oc get isvc sklearn-v2-iris-test3 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test3-predictor.kserve-raw-tests.svc.cluster.local # Schema should be HTTP
components:
  predictor:
    url: http://sklearn-v2-iris-test3-predictor-kserve-raw-tests.example.com # Wrong
url: https://sklearn-v2-iris-test3-kserve-raw-tests.apps-crc.testing # OK

Deploy a slim Raw InferenceService with auth enabled
:yellow_circle: OK, needs follow-up PR for fixes
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    security.opendatahub.io/enable-auth: "true"

:green_circle: The model deploys fine
:green_circle: Auth works
:green_circle: Inference works
:green_circle: SVC looks OK
:green_circle: oauth-proxy container is present
:green_circle:  ClusterRoleBinding created...
:green_circle: Route is absent
:yellow_circle: The Status is not right. But OK for now.
oc get isvc sklearn-v2-iris-test4 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test4-predictor.kserve-raw-tests.svc.cluster.local:8443 # OK
components:
  predictor:
    url: http://sklearn-v2-iris-test4-predictor-kserve-raw-tests.example.com # Wrong
url: http://sklearn-v2-iris-test4-predictor.kserve-raw-tests.svc.cluster.local:8443 # Schema should be HTTPS.

Deploy a Raw InferenceService with auth enabled and explicit private label
:yellow_circle: OK, needs follow-up PR for fixes
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "cluster-local"
    security.opendatahub.io/enable-auth: "true"

:yellow_circle: Same results as previous.
Deploy a Raw InferenceService with auth enabled and route enabled
:yellow_circle:  OK, needs follow-up PR for fixes
Follow the gist, but use these metadata in the InferenceService:
  annotations:
    serving.kserve.io/deploymentMode: "RawDeployment"
  labels:
    networking.kserve.io/visibility: "exposed"
    security.opendatahub.io/enable-auth: "true"

:green_circle: The model deploys fine
:green_circle: Auth works
:green_circle: Inference works
:green_circle: SVC looks OK
:green_circle: oauth-proxy container is present
:green_circle:  ClusterRoleBinding created...
:green_circle: Route is present
:yellow_circle: The Status is not right. But OK for now.
oc get isvc sklearn-v2-iris-test6 -o yaml | yq '{"address": .status.address, "components": .status.components, "url": .status.url}'
address:
  url: https://sklearn-v2-iris-test6-predictor.kserve-raw-tests.svc.cluster.local:8443 # OK
components:
  predictor:
    url: http://sklearn-v2-iris-test6-predictor-kserve-raw-tests.example.com # Wrong
url: https://sklearn-v2-iris-test6-kserve-raw-tests.apps-crc.testing # OK

Other notes

I greatly suggest to use a namespaced RoleBindings instead of ClusterRoleBindings.
We did #280 for a specific need, but it has an impact about how requests flow. I didn't succeed on doing inference requests using the port in the Service for the ISVC that were private and had to use the pod port. This is leading to the failing Route in the red test. With this, I want to tell that we are not reporting the right port to use when auth is turned off. This is a bug introduced in #280 that we need to fix for GA.


set animate direction

in current implementation on library, animation control is top to bottom. how can i set direction to animate from bottom to top?
Thanks

library expose a function named showOverlay
https://github.com/boyan01/overlay_support/blob/5032ebf3c7347a326b5b75c3667abcb74854c98b/lib/src/overlay.dart#L50
showSimpleNotification, showNotification and toast all used it to achieve the effect.  you can use this function to implement the custom behavior you want.

@boyan01
Thanks, i'm not sure to understand me whats animation controller on this class, whats in mind animations are tween

you could use google flutter tween to get more info


fix stacking of async sagas execution on url change

I had a problem when Saga was lunched several times if you navigate through the app and return to container with saga who would send request to server.
So basically my request ran as many times as i entered this specific url and its saga was passed to injectAsyncSagas.

This is the same solution as @kraffslol suggested in #390, which we didn't merge because @justingreenberg's #399 made more sense. This is fixed if you pull right now!
Thanks so much for taking the time to submit this PR though!

Just running into the duplicate saga execution issue myself today and I'm looking at the various issues posted around them.
Is the idea here that upon changing routes, the saga will 'unbind' itself from the store? That way when the route is hit again, re-injecting the same saga will not cause any duplication?


[WM-2554] pact verification GitHub action

Jira: https://broadworkbench.atlassian.net/browse/WM-2554
What:
Adds github actions that runs VerifyServicePacts (see this PR)
Why:

This is part of the normal provider verification process that most teams are adopting. It's in a separate PR because I'm hoping @ichengchang can weigh in, as the resident expert on this sort of thing

This repository is missing the pact broker credentials.
Please follow this PR as an example to terraform the pact broker credentials for terra-external-credentials-manager repo. You will need a PR to update the file https://github.com/broadinstitute/terraform-ap-deployments/blob/master/github/tfvars/databiosphere-terra-external-credentials-manager.tfvars and have DevOps approve the PR.

Let's update the provider name in https://github.com/DataBiosphere/terra-external-credentials-manager/blob/dev/service/src/test/java/bio/terra/externalcreds/pact/VerifyServicePacts.java#L51 from ecm to externalcreds to match the helm chart name.

Remove the following properties
https://github.com/DataBiosphere/terra-external-credentials-manager/blob/dev/service/build.gradle#L102
https://github.com/DataBiosphere/terra-external-credentials-manager/blob/dev/service/build.gradle#L102
and replace them with
systemProperty 'pactbroker.url', System.getenv('PACT_BROKER_URL')

Also please take at look at
https://github.com/DataBiosphere/terra-billing-profile-manager/blob/main/service/src/test/java/bio/terra/profile/pact/provider/BPMProviderTest.java#L87
https://github.com/DataBiosphere/terra-billing-profile-manager/blob/main/service/src/test/java/bio/terra/profile/pact/provider/BPMProviderTest.java#L111-L123
and see if you can add these to https://github.com/DataBiosphere/terra-external-credentials-manager/blob/dev/service/src/test/java/bio/terra/externalcreds/pact/VerifyServicePacts.java

I think this is erroring out now because the pact from CBAS was not properly published. I'm working on that now


Upgrading to react-router 4

Title of change
Upgrading to react-router 4
Checklist

[ ] Unit tests written
[x] Manually tested
[x] New dependencies were added with yarn
[x] Prettier & ESLint were run

Screenshot
insert screenshot if needed

@sabeurthabti wanted to get your thoughts on how to handle the server-side stuff and code splitting. So far, this is a pain

Deployed to: http://staging.quran.com:32874

@mmahalwy I will be looking into this tomorrow iA

@sabeurthabti okay, this is working. I don't want to make this PR any larger and feel like a follow up PR can follow to polish. I think this is a good place to stop

Tbh I have not done much server-rendering recently, so I wouldn't have had much input.
Glad it is working. Let's get this merged in .

rebuild

@ahmedre I think jenkins died again :I

rebuild

thanks for the heads up - should be good now

Deployed to: http://staging.quran.com:32875


Cleaner aggregate errors API

@smarterclayton could you be talked into splitting the filter and flatten logic into two calls?  This feels like a more generic API.

Sure, I only did it as one because of the complexity of flatten already (filter reduced the space).

I like this.

Does that count as LGTM?

LGTM

Thanks, rebased and everything is golden.
----- Original Message -----

Does that count as LGTM?

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/pull/3353#issuecomment-69360337


Bumped Version 2.2.1 In preparation for release

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Working on the manual backport now


feat: Docker build

Sharing here, may be of interest to those trying to build as a docker image to run in kubernetes or similar. ALSO updated to work with some of the latest go packages & such.  NOT tested, and more work needed to build different ARCH, but... may be of use...

Also golang upgrades, change linter off of the deprecated linter, fixed travis CI to work with all the above.


Fix call to sortmergeterms to use ITensorMPS

Fixes issue where sortmergeterms is now inside the ITensorMPS module within ITensors.

Tests are failing saying ITensorMPS not recognized. I suppose this is because the introduction of ITensorMPS is not included in the latest numbered release of ITensors.jl.

Right, we'll have to register a new version of ITensors.jl, and also we'll want to bump the compat version of ITensors in ITensorChemistry to enforce that the latest version of ITensorChemistry requires versions of ITensors that have the ITensorMPS submodule.

Can you also bump the compat entry for ITensors to v0.3.58? I'm registering it now.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 86.31%. Comparing base (2e2e86c) to head (ac6b801).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main       #9      +/-   ##
==========================================
+ Coverage   86.15%   86.31%   +0.15%
==========================================
  Files           6        6
  Lines         260      263       +3
==========================================
+ Hits          224      227       +3
  Misses         36       36

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Travis does not pick up wheel for scipy 1.3.1 for Python 3.7

Not really sure what is going on, but CI on the geopandas project started to fail since scipy 1.3.1 release.
We are creating a conda environment (based on this yaml file: https://github.com/geopandas/geopandas/blob/master/ci/travis/37-latest-defaults.yaml), where mapclassify is installed via pip, and this requires scipy.
In the travis log (https://travis-ci.org/geopandas/geopandas/jobs/570422723), there is:
Collecting scipy>=0.11 (from mapclassify->-r /home/travis/build/geopandas/geopandas/ci/travis/condaenv.553d61zw.requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/ee/5b/5afcd1c46f97b3c2ac3489dbc95d6ca28eacf8e3634e51f495da68d97f0f/scipy-1.3.1.tar.gz (23.6MB)

so for some reason it is downloading the source release (scipy-1.3.1.targ.gz), while there clearly are already wheels uploaded on pypi (manylinux wheels for python 3.7: https://pypi.org/project/scipy/1.3.1/#files)
This only seems to happen for python 3.7, as one of the other environments with python 3.6 which also tries to install scipy using pip correctly gets the wheel (eg https://travis-ci.org/geopandas/geopandas/jobs/570422722#L490)

Looking at the yaml file there's no entries for numpy or scipy. From the logfile numpy gets installed in the conda environment but something seems to go wrong in the install, because it's not importable. It might be worth explicitly using both scipy and numpy in the environment file.
I don't think this is an issue with scipy per-se, but an issue with the CI setup that's being used.  As such, I'm going to close the issue.
One way of investigating further (apart from hammering the CI with trying things) is to use Docker and a Xenial image to try and repeat the problems you're seeing. Also, try and install scipy directly with pip and see if there's an issue.

From the logfile numpy gets installed in the conda environment but something seems to go wrong in the install, because it's not importable

AFAIK, this is not the numpy from conda, it is the numpy installed in the temporary build environment of pip when trying to build a wheel for scipy.

It might be worth explicitly using both scipy and numpy in the environment file.

Yes, sure, that is a solution to fix our CI problem (ensuring conda already installed it, so the pip install of mapclassify does not need to install scipy).
But this still seems like a general problem to me?

I can reproduce this locally on Ubuntu 18.04, in a new conda environment.
Recreating the environment as done on travis (but without the pip install of scipy step within the creation of the conda environment):
$ conda env create --file=ci/travis/37-latest-defaults.yaml
....
$ conda activate test2
$ which pip
/home/joris/miniconda3/envs/test2/bin/pip
$ pip install scipy
7.4.0
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/ee/5b/5afcd1c46f97b3c2ac3489dbc95d6ca28eacf8e3634e51f495da68d97f0f/scipy-1.3.1.tar.gz (23.6MB)
     |████████████▍                   | 9.1MB 2.2MB/s eta 0:00:07^C
ERROR: Operation cancelled by user

so also here it tried to download the source archive and not the wheel.
Is there a way to check manylinux compatibility of a system (in this specific environment) ?

What do you get for python -c "import distutils.util; print(distutils.util.get_platform())" ?

(test2 is the name of the conda env that recreated the travis one)
(test2) joris@joris-XPS-13-9380:~/scipy$ which python
/home/joris/miniconda3/envs/test2/bin/python
(test2) joris@joris-XPS-13-9380:~/scipy$ python -c "import distutils.util; print(distutils.util.get_platform())"
linux-x86_64

Creating a minimal conda env with just python 3.7 and pip (conda create -n test-scipy-wheel python=3.7 pip) works fine, doing a pip install scipy gets the wheel. So something in the other environment somehow influences that pip does not want to install a wheel for scipy .. Any idea how another installed package can influence this?

I noticed this warning in your log file : https://travis-ci.org/geopandas/geopandas/jobs/570422723#L378 - is it relevant?

I noticed this warning in your log file : https://travis-ci.org/geopandas/geopandas/jobs/570422723#L378 - is it relevant?

Normally not, it's something I should solve, but it is just conda saying that you should add pip explicitly instead of relying on conda doing it for you.
But I pinned it down further to a difference between defaults and conda-forge ...
Creating a minimal env with defaults, then installing scipy with pip gets the source tar.gz:
(base) joris@joris-XPS-13-9380:~/scipy$ conda create -n test-scipy-wheel2 python=3.7 pip -c defaults
...
(base) joris@joris-XPS-13-9380:~/scipy$ conda activate test-scipy-wheel2
(test-scipy-wheel2) joris@joris-XPS-13-9380:~/scipy$ conda list
# packages in environment at /home/joris/miniconda3/envs/test-scipy-wheel2:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
ca-certificates           2019.5.15                     1
certifi                   2019.6.16                py37_1
libedit                   3.1.20181209         hc058e9b_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 9.1.0                hdf63c60_0
libstdcxx-ng              9.1.0                hdf63c60_0
ncurses                   6.1                  he6710b0_1
openssl                   1.1.1c               h7b6447c_1
pip                       19.1.1                   py37_0
python                    3.7.4                h265db76_0
readline                  7.0                  h7b6447c_5
setuptools                41.0.1                   py37_0
sqlite                    3.29.0               h7b6447c_0
tk                        8.6.8                hbc83047_0
wheel                     0.33.4                   py37_0
xz                        5.2.4                h14c3975_4
zlib                      1.2.11               h7b6447c_3
(test-scipy-wheel2) joris@joris-XPS-13-9380:~/scipy$ which pip
/home/joris/miniconda3/envs/test-scipy-wheel2/bin/pip
(test-scipy-wheel2) joris@joris-XPS-13-9380:~/scipy$ pip install scipy
7.4.0
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/ee/5b/5afcd1c46f97b3c2ac3489dbc95d6ca28eacf8e3634e51f495da68d97f0f/scipy-1.3.1.tar.gz (23.6MB)
     |█████▎                          | 3.9MB 425kB/s eta 0:00:47^C
ERROR: Operation cancelled by user

Doing exactly the same with conda-forge gets the wheel:
(base) joris@joris-XPS-13-9380:~/scipy$ conda create -n test-scipy-wheel3 python=3.7 pip -c conda-forge
...
(base) joris@joris-XPS-13-9380:~/scipy$ conda activate test-scipy-wheel3
(test-scipy-wheel3) joris@joris-XPS-13-9380:~/scipy$ conda list
# packages in environment at /home/joris/miniconda3/envs/test-scipy-wheel3:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
bzip2                     1.0.8                h516909a_0    conda-forge
ca-certificates           2019.6.16            hecc5488_0    conda-forge
certifi                   2019.6.16                py37_1    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc-ng                 9.1.0                hdf63c60_0
libstdcxx-ng              9.1.0                hdf63c60_0
ncurses                   6.1               hf484d3e_1002    conda-forge
openssl                   1.1.1c               h516909a_0    conda-forge
pip                       19.2.1                   py37_0    conda-forge
python                    3.7.3                h33d41f4_1    conda-forge
readline                  8.0                  hf8c457e_0    conda-forge
setuptools                41.0.1                   py37_0    conda-forge
sqlite                    3.29.0               hcee41ef_0    conda-forge
tk                        8.6.9             hed695b0_1002    conda-forge
wheel                     0.33.4                   py37_0    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
zlib                      1.2.11            h516909a_1005    conda-forge
(test-scipy-wheel3) joris@joris-XPS-13-9380:~/scipy$ which pip
/home/joris/miniconda3/envs/test-scipy-wheel3/bin/pip
(test-scipy-wheel3) joris@joris-XPS-13-9380:~/scipy$ pip install scipy
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/94/7f/b535ec711cbcc3246abea4385d17e1b325d4c3404dd86f15fc4f3dba1dbb/scipy-1.3.1-cp37-cp37m-manylinux1_x86_64.whl (25.2MB)
     |███▉                            | 3.0MB 3.4MB/s eta 0:00:07^C
ERROR: Operation cancelled by user

OK, and I see, the main difference between both envs is probably the difference in pip version: 19.1.1 (on defaults, where it fails) vs 19.2.1 (on conda-forge, where it succeeds)

So it is probably not related to the scipy 1.3.1 release, as with the older pip also installing older scipy downloads the source (it just coincidentally started being a problem for some reason at the same moment that scipy 1.3.1 was out). Sorry for the noise!

So apparently this is due to Python 3.7.3 -> 3.7.4 update (and conda-forge is still at 3.7.3): https://github.com/conda/conda/issues/9084, https://github.com/tensorflow/tensorflow/issues/31523
Not sure if this is an issue with the Python provided by Anaconda or with Python 3.7.4 itself


CI/CD: producing long running traces

Area(s)
area:cicd
Is your change request related to a problem? Please describe.
This is not strictly a semantic-convention discussion, but it's come about because of trying to produce traces for CI/CD and the sem-conv CI/CD WG is the closest place for a home for this discussion.
The problem I am trying to solve is producing spans of large durations, from a kubernetes controller which may restart.
I'm specifically trying to solve this in argo-workflows, and I gave a talk at argocon (slides) which gives a lot of context to this. Argo Workflows can and is used as a CI/CD tool, but also for many other batch and machine learning scenarios where traces of the flow of a workflow would be useful. I'm pretending that the trace is the lifespan of a workflow, which may be several hours (some run for days). There are child spans representing the phase of the workflow and the individual nodes within the DAG that is being executed.
Argo Workflows
The workflow controller is best thought of here as a kubernetes operator, in this case running something not that far removed from a kubernetes job, just that the job is a DAG rather than a single pod. In the usual kubernetes controller manner, this controller is stateless, and can therefore restart at any time. Any necessary state is stored in the workflow Custom Resource. This is how the controller currently works.
I've used the GO SDK to add OTLP tracing to Argo Workflows, which works unless the controller restarts.
Ideas
Delayed span transmission.
My initial thought
My problem seemed to be that the opentelemetry SDK does not allow me to resume a Span. I'm using the golang SDK, but this is fundamental to the operation of spans. Once a Tracer is shutdown it will end all spans, and that's the end of my trace and spans. The spans will get transmitted as ended.
I therefore supposed you could put the SDK or a set of traces/spans into a mode where the end of the span didn't get transmitted at shutdown, and instead the span could be stored outside of the SDK to be transmitted later by the resumed controller once it started up. The SDK could then also facilitate "storing a span to a file".
This is possibly implementable right now with enough hacking, I haven't managed to get time for it.
Links
I could use span links and create multiple traces for a single workflow. This would crudely work, but I'd argue against it unless the presentation layer can effectively hide this from users.
The current UI for argo-workflows already has a basic view showing a timeline of a workflow for

This won't display or concern the user with controller restarts.
The target audience for these spans is probably somewhat less technical than the existing audience for http microservice tracing. Having to explain why their trace is in multiple parts and that they'll just have to deal with it isn't ideal. Span Metrics are a valuable tool here, and they'll be much more complicated or impossible in some cases.
It may be that the presentation layer can hide this - I have limited exposure to the variety of these in the market and how they deal with links.
Events
We could emit events for span start and end, and make it the "collectors" problem to correlate these and create spans. This is how Github Action tracing works - thanks to @adrielp for telling me about this.
For this to work something has to retain state - either the "End Span" event contains enough to construct the whole span (e.g. Start time) or the collector has to correlate a start event and end event, so the start event needs storage. The collector storing state would be wrong - in a cloud native environment I'd not even expect the same collector to receive the end as received the start. I'm trying not to be opinionated on how you configure your collector, but maybe we have to be.
Protocol changes
A different approach to delaying span transmission, but with a similar goal.
We could change the protocol to allow span end to be "ended due to shutdown", and then allow a future span end for the same span_id to end it properly. This probably just pushes the problem onwards to the collector or the storage backend to do correlation in a similar way to events, so isn't an improvement.
Describe the solution you'd like
I don't work in the telemetry business, and so I'm sure I'm missing other prior art.
I'm open to any mechanism to solve this, and would prefer we came up with a common pattern for this and other longer span/restartable binary problems. I believe these problems will also be there in some of the android/mobile and browser implementations, to which I have little visibility or understanding. Some of these proposed solutions may not work there, so coming up with a common solution which works for my use case and these would be ideal.
I hope this sparks a useful discussion.

When the controller restarts emitting spans that do not match the actual duration of the tasks they represent and linking them using span links due to an inability of having a common parent span is suboptimal.
I have also observed this issue in the Jenkins OpenTelemetry plugin.
As mentioned in the CICD SIG call of the 28th November 2024, this seems to be a limitation of the OpenTelemetry Go SDK and might be shared by other OpenTelemetry SDKs.
The requirements necessary to solve the issue are:

the ability to create spans with custom start times (that might be before the start of the process creating them)
stable span and trace IDs so that spans emitted before the controller restarts can still be displayed as child spans of the parent trace emitted by a later controller

According to specification for the SDKs https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#span-creation there should be an optional parameter for Start timestamp when creating spans. Maybe that could allow setting a start time before the start of the process using the SDK.
Checking the Golang SDK, this should be possible when creating spans using the trace.SpanStartOption:
https://github.com/open-telemetry/opentelemetry-go/blob/b4a91a210c865374f5c72973eeb450036290366f/sdk/trace/tracer.go#L29
In the implementation of the Github actions event receiver the opentelemetry-collector receives the Github actions events via webhook. The events have started_at and completed_at attributes which are mapped to the span start/end timestamps respectively:
https://github.com/krzko/opentelemetry-collector-contrib/blob/feat-add-githubactionseventreceiver/receiver/githubactionsreceiver/trace_event_handling.go#L199
This event receiver seems to use the collector internal data structures https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata and not the opentelemetry-go sdk. As such the Github actions event receiver can directly manipulate all span properties without restrictions from the SDK API.

As long as you can store a trace ID, span ID, and start time, you can probably just generate the entire span when it ends.
You can set a custom start time for spans using the https://pkg.go.dev/go.opentelemetry.io/otel/trace#WithTimestamp option on start.
The only remaining thing you would need to be able to do is to set the trace and span id.  The only way I can think of to do this today is by providing a custom IDGenerator via https://pkg.go.dev/go.opentelemetry.io/otel/sdk/trace#WithIDGenerator.  The IDGenerator has access to the context, which you could use to pass the trace ID and span ID you want to use.  Don't pass it in using trace.ContextWithSpanContext, since then it will be used as the parent context.  You will want to write your own ContextWithHardCodedSpanContext (or similar) to pass it in explicitly through the context.
As for prior art, I and others in the K8s instrumentation SIG explored storing the span context in object annotations as part of  https://github.com/kubernetes/enhancements/pull/2312, but didn't move forward with the proposal.

Notes from SemCon meeting 2024-12-09
There are 2 ways to think about tracing:

2 events: start+stop events
1 event containing both start+stop times or start time+duration

When tracing was designed (leading to the dapper paper?) were mostly around short running operations.
So in OpenTelemetry traces use the start time + duration model.
Long running traces can be problem:

for tail sampling
for post-processing, complete traces. Sometimes there's there's a desire to do some analysis based on
a full trace, and often those are based on assumptions about kind of the maximum length that a trace can realistically be to decide like how long to delay before looking back and processing kind of on an iterative basis.

There's similar concerns also with people who like to mark errors on traces. And so if you have like a message pass queue in the middle, and the message pass queue errors. You'd mark the whole traces in there.
The other thing is with CICD: I don't think you have a large trace problem with this. You just have a long trace problem. Right?
They just take a long time because some builds can be. At least, if you're doing the horrible build for ARM emulation thing we were doing in Github could be like 4h, for no reason outside of emulating your ARM. Build right, and you have to deal with that. But the trace isn't any bigger. So I think you actually have an interesting use case here with long running traces where you're not pushing against concerns with like tail sampling, where we have to keep giant amounts of data in memory for very long amount of time.
You're also not pushing against some of that post-processing things. If the size of your trace is still reasonable because of the domain that you're in. But you are running into problems where the domain you're in and the design of the SDK are not aligned
right? So you might need to either use a different SDK, or get some kind of extension to our sdks and Apis for this particular workflow. I think both of the like. I think we could probably agree that those are true, but it might be useful to write those down
and have that ready when you come to the spec meeting, because again, when you come to the spec meeting, the kind of concerns and thoughts people are going to have are general tracing, so like how expensive is to process a trace. You know what's the overall size of the trace? If I do tail sampling, how much memory do I have to allocate to keeping these traces to tail, sample. Those are the concerns that went into the tracing design. And so just having something to kind of talk about, that, to address it.
I thought that it might be possible even to solve with current sdks (we can start a trace or span with an earlier start time and that it should be feasible to pass a given context like for http header passing context).
You. You probably can. Yeah, it's probably just not ergonomic. You definitely could do it from Java. I know that I'm pretty sure you can do it from go as well.
Okay, this is also an interesting discussion. You're gonna bring it up in the specification call. I think that's the right place to have that discussion.

Note, that this is a spec issue and not a semconv issue, there are existing discussions on that matter:
https://github.com/open-telemetry/opentelemetry-specification/issues/373,
https://github.com/open-telemetry/opentelemetry-specification/issues/2692
As per my initial thoughts on #1688 I think that SIG CICD SemConv could be a driving force behind that, so please review those existing PRs, especially 373 by @arminru and the comment by @xBis7 from Oct 15th at the very bottom


fix(deps): fix dependency toward wal-e to avoid issues with Python mi…

fix dependency toward wal-e to avoid issues with Python minimal requirement

Thanks. Looks good to me 👍

Thanks @remialvado  for the pull request!
Per the CONTRIBUTING.md file displayed when you created this pull request, we need to add you to the list of approved contributors for the Mattermost project.
Please help complete the Mattermost contribution license agreement?
This is a standard procedure for many open source projects. Your form should be processed within 24 hours and reviewers for your pull request will be able to proceed.
Please let us know if you have any questions.
We are very happy to have you join our growing community!

OK, it's done :)
On Sat, Oct 8, 2016 at 11:51 AM, Yi EungJun<EMAIL_ADDRESS>wrote:

Thanks @remialvado https://github.com/remialvado for the pull request!
Per the CONTRIBUTING.md
https://github.com/mattermost/platform/blob/master/CONTRIBUTING.md#submitting-a-pull-request
file displayed when you created this pull request, we need to add you to
the list of approved contributors
https://docs.google.com/spreadsheets/d/1NTCeG-iL_VS9bFqtmHSfwETo5f-8MQ7oMDE5IUYJi_Y/pubhtml?gid=0&single=true
for the Mattermost project.
Please help complete the Mattermost contribution license agreement
http://www.mattermost.org/mattermost-contributor-agreement/?
This is a standard procedure for many open source projects. Your form
should be processed within 24 hours and reviewers for your pull request
will be able to proceed.
Please let us know if you have any questions.
We are very happy to have you join our growing community!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mattermost/mattermost-docker/pull/54#issuecomment-252415597,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHaO5dj8CIpCu_OcZE7q6Hj14beOlKIks5qx2esgaJpZM4KQ_yf
.

Thanks @remialvado for your contribution and @xcompass for the review!


2022.11

Coverage remained the same at 92.902% when pulling 34f24688676199fde4dd467f0e1e274eb0a160f1 on antgonza:2022.11 into 208a7f149c3277be246a5d933d272a9d4d91b6e3 on qiita-spots:dev.

Thank you for the review @justinshaffer


Add an updateLock step to alter resources in pipelines

This PR proposes to add a new updateLock step which can be used to alter the resource configuration. Maybe updateResource would be better, but I thought users of this plugin would find updateLock more intuitive.
With this change, the following can be altered:

adding labels
removing labels
setting labels (replacing)
setting the note
creating new locks
deleting existing locks

New locks are created persistent (non-ephemeral). The use case for this can be having a job that would configure the locks based on an external source (ex. scm).  I have not yet figured out how the sync could remove invalid locks (we will probably need a way to list locks so the ones removed from scm can be deleted - open for discussion)
Deleted locks are set to ephemeral if currently locked.
Once #299 is in, I will submit another PR to allow properties to be altered as well.

[x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
[x] Ensure that the pull request title represents the desired changelog entry
[x] Please describe what you did
[X] Link to relevant issues in GitHub or Jira
[x] Link to relevant pull requests, esp. upstream and downstream changes
[x] Ensure you have provided tests - that demonstrates feature works or fixes the issue

I just realized the org.jenkins.plugins.lockableresources.LockableResourcesManager can be imported in groovy - so I think this step is not necessary :D

I take this back, working straight on the org.jenkins.plugins.lockableresources.LockableResourcesManager is painful,

I worked a bit more on this - so the original pipeline looks like this - using the LockableResourcesManager directly:
import org.jenkins.plugins.lockableresources.LockableResourcesManager as LRM
script {
  Map resources = readYaml(file: 'resources.yaml')

  resources.each { k, v ->
    LRM.get().createResource(k)
    def r = LRM.get().fromName(k)
    r.setEphemeral(false)
    if (v.labels) {
      r.setLabels(v.labels.join(" "))
    }
    else {
      r.setLabels("")
    }
  }

  def allResources = LRM.get().getResources();
  List toBeDeleted = []
  for (resource in allResources) {
    if (resources.keySet().contains(resource.getName()) == false) {
      toBeDeleted += resource
    }
  }

  for (toDelete in toBeDeleted) {
    LRM.get().getResources().remove(toDelete)
  }

  LRM.get().save()
}

With this propose change, I am simplifying it down to
script {
  Map resources = readYaml(file: 'resources.yaml')

  resources.each { k, v ->
    updateLock(resource:k, createResource:true, setLabels:(v.labels?:[]).join(" "))
  }

  def allResources = LRM.get().getResources();
  List toBeDeleted = []
  for (resource in allResources) {
    if (resources.contains(resource.getName()) == false) {
      toBeDeleted += resource
    }
  }

  for (toDelete in toBeDeleted) {
    updateLock(resource:k, deleteResource:true)
  }
}

After this change, the step I'd be missing would be a findLocks step - then I could go
script {
  Map resources = readYaml(file: 'resources.yaml')

  resources.each { k, v ->
    updateLock(resource:k, createResource:true, setLabels:(v.labels?:[]).join(" "))
  }

  for (resource in findLocks()) {
    if (resources.contains(resource.nane) == false) {
      updateLock(resource:k, deleteResource:true)
    }
  }
}

Any update on this? this feature will be super useful for our use case... in fact, many PR requests by @gaspardpetit are exactly what I am looking for to fully adopt this plugin.


Can Apache Sorl be integrated with NLTK?

If not, is there another way around?

This question (like the one you've asked on Stack Overflow, which I've voted to close) is too vague to be answerable. Solr is a search tool and NLTK is a collection of NLP tools; it's unclear what it would even mean to 'integrate' them. I suggest that a maintainer simply close this issue.

Closing off-topic issues to clean up the issue tracker. Please feel free to reopen or create PR if you would like to contribute to Solr integration.


Changing "Harbor" text in top left corner possible?

I want to change the text in the top left corner from "Harbor" to something else.
I'm aware I can change the look and feel via the setting.json file, as documented here, but alas the file in the documentation and the actual file differ. (E.g. product is now project, different entries, ...)
The file in the documentation has a name entry that contains Harbor, while the actual file is missing this. I tried changing all the values in the actual file, and while the changes work (e.g. the text above the login form, the logo, etc.), none of these entries change the "Harbor" text in the top left corner. I also tried adding "name": "<Harbor changed>" into various sections of the setting.json file, but it's never applied either.
Can this be changed somewhere else? Or has this functionality been removed? Why are the files in the documentation and the file in the source code different? Am I missing something? Is there a workaround for me in the meantime? :confused:
The easiest way would be to have a dedicated entry for this in the setting.json file.

PS:

Also, please add an option to remove the ™ symbol after the text above the login form, i.e. the value in appTitle. Or remove it by default if the value is changed from VMware Harbor. But that's not as important right now.
Previous issue regarding UI changes and setting.json: https://github.com/vmware/harbor/issues/5329

I just found out that Harbor was originally contained in projectName, as seen in this commit: https://github.com/vmware/harbor/commit/7845405862a1f762d35b60768cdad039fba96e99#diff-a5f003b3824fc73f8c271f00d67fb8b0.
But the effect of changing this value is only reflected in the "About" popup. In this popup, Harbor changes to whatever the value of projectName is. But Harbor in the top left corner stays the same no matter what.
So my newest suspicion is that the change is just not applied to the UI field in the top left corner for whatever reason, maybe some bug.

Could it be that the error lies here?
https://github.com/vmware/harbor/commit/7845405862a1f762d35b60768cdad039fba96e99#diff-4d34aee432b5db1e634da04279b6bbe9L6
Since this is apparently where that title is set.

Shouldn't this be project instead of projects?
https://github.com/vmware/harbor/blob/8f9e2644075744e091b91d7a5fdab73973c28b50/src/ui_ng/src/app/base/navigator/navigator.component.ts#L62
Since the naming convention changed in https://github.com/vmware/harbor/commit/7845405862a1f762d35b60768cdad039fba96e99 everywhere from projects to project. Including the setting.json file.


Sorting with .populate() option

Hi, is it possible to apply sorting with associations ? For example if we have the following:
model employeeModel
{...
attributes: {
id: {
type: 'integer',
primaryKey: true,
autoIncrement: true,
required: true
},
companyId: {
model: 'companyModel'
type: 'integer',
required: true,
defaultsTo: 0
}
}
}
What I need as results is a list of employees sorted by the company name, where the company's name is in the 'companyModel'. The following does not work:
model.find({
where: {},
sort: 'companyId.companyName'
})
.populate('companyId')
.then(function (result) {
callback(null, {rows: result});
})
.catch(function (err) {
// an error occurred
return callback(err);
});
Is it possible to sort the query results by a populated field ?
Thanks.

Currently there is no way to do this.  @atiertant, will deep-populate help with this?

@devinivy no this would be is a deepCriteria
this is that feature request https://github.com/balderdashy/waterline/issues/266
but a deep-populate would be the first step ...

Should not be closed


Invalid require in README

There's a simple typo in (extra 'o' in asciidoctor)
require 'asciidocotor/reducer/include_mapper/extension'

Thanks for the heads up! Fixed with 17086dbdddd5c554638291f766c576c619cdde7c.


Pagination

@dohooo how to show the dot at the bottom view? like below

)

Is it possible to show it on stack carousel?

Is it possible to show it on stack carousel?

You can refer to the code here and do it yourself. And I have answered the similar question here


libStorage.d.ts - Cannot find module '@aws-sdk/client-s3' or its corresponding type declarations

Checklist

[x] I have read Caveats documentation and didn't find a solution for this problem there.

Bug description
node_modules/aws-sdk-client-mock/dist/types/libStorage.d.ts(8,95): error TS2307: Cannot find module '@aws-sdk/client-s3' or its corresponding type declarations.

Environment

Node version: v16.14.2
Testing lib and version: Jest v27.4.6
Typescript version: v4.4.4
AWS SDK v3 Client mock version: 0.6.2
AWS JS SDK libs and versions: NA

@JasperHub1 please provide a repo with reproduction, including the command you use to run the test.

@m-radzikowski, sorry could't share due to its private repo
the work around is to use skipLibCheck option
Also another option was to install @aws-sdk/client-s3 to project as it's having peer dependencies
Saw you opened this PR https://github.com/m-radzikowski/aws-sdk-client-mock/pull/57 before, might related to this issue

I have the same issue using the lib with typescript. Does anyone have any definitive solution?

I've just released a v1.0.0 that removes the @aws-sdk/client-s3 from peer dependencies. I believe it should solve the problem.
Otherwise, please reopen / open a new issue, providing a simple, small repository with your configuration that will reproduce the issue.

Using the new version (v1.0.0) I'm getting an error related to NativeAttributeBinary type:
node_modules/@aws-sdk/util-dynamodb/dist-types/models.d.ts:19:86 - error TS2304: Cannot find name 'File'.

19 export declare type NativeAttributeBinary = ArrayBuffer | Blob | Buffer | DataView | File | Int8Array | Uint8Array | Uint8ClampedArray | Int16Array | Uint16Array | Int32Array | Uint32Array | Float32Array | Float64Array | BigInt64Array | BigUint64Array;
                                                                                ~~~~

Found 1 error in node_modules/@aws-sdk/util-dynamodb/dist-types/models.d.ts:19

We upgraded to v1.0.0 and no longer having the peer dependency issue. 👏  Thanks

Using the new version (v1.0.0) I'm getting an error related to NativeAttributeBinary type:

@studiojms this doesn't seem like a mocking library issue. See https://github.com/aws/aws-sdk-js-v3/issues/2896
But if you are confident it happens only when the mocking library in v1.0.0 is added to the project, please open a new issue, preferably with a repo reproducing the issue.


Upgrade from 124 to 126 causes crash depending on Preferences

System Details

OS: Arch
Thorium Version 126.0.6478.231

Problem
Upgrading from 124.X to 126.X causes a crash on boot if the Preferences file in the profile contains the string "theme":{"id":"","system_theme":1}, inside the extensions object. I'm guessing this is the setting that controls the experimental Thorium theme, which I had enabled in 124. Removing the string from the Preferences file causes the upgrade to succeed without further problems.
Additional Notes
Mostly putting this here so it can hopefully be automatically fixed in future releases, or at least so that others running into the same problem can find it and know how to solve it without needing to debug for a few hours like I ended up doing. 🙃

@Thulinma This is not the reason for the crash, but you're close. Thanks for letting me know, as I was wondering if the code change would cause crashes on other systems.
Also, what experimental thorium theme in M124 are you talking about.

Ah yes, it was just a guess. Glad that you know what it is, at least! 👍
I.... don't recall what exact release had this, but some releases ago there was a theme of some sort mentioned in the "Thorium Experiments". I only noticed it was gone in the latest version but that may have actually disappeared much longer ago. I don't check the experiments list all that often. Sorry for any confusion! 😅

@Thulinma I think I fixed this here > https://github.com/Alex313031/thorium/commit/5b41ddba80b220e63ceb51c69466aa67488f25e9


Playground: dependencies for jsxImportSource are not installed

TS team decided to use /* @fooImportSource */ syntax for importing JSX types (otherwise if JSX types are imported with import syntax then the JSX expressions do not see the JSX namespace), but the playground is unaware that it needs to import the dependencies for that sort of syntax:
/* @jsxImportSource solid-js */
const d = <div /> // Error: interface JSX.IntrinsicElements is not defined, because playground does not install solid-js

playground example
If we could instead rely on import for importing JSX types (as would be intuitive and aligned with EcmasScript standards and conventions, instead of having to use new @fooImportSource syntax), the playground example (which will install solid-js based on the import statement) would just work already.

I also tried placing foo-runtime in a package, and then importing it with /* @fooImportSource foo-package */ hoping it would import foo-package/foo-runtime but that didn't work.
I think we should stick with ES Module semantics anyway.

The compile-time semantics are built to replicate the source of truth for JSX parsing (aka the babel plugin) - but sure, I'd accept a PR adding the module in fooImportSource to the ATA in the playground 👍🏻

@orta Not sure if you understood my OP. My last comment was intended as a joke, but the actual comment in the OP /* @jsxImportSource package-name */ is what isn't working (just kidding about supporting @anythingImportSource).
So basically, in the OP playground shows a type error when we expect not to see an error.

TypeScript is doing the wrong thing with JSX, in my opinion. We should be able to use import {JSX} from 'anywhere' syntax instead of @jsxImportSource anywhere syntax, and it should just work but it doesn't.

@orta One more example to make the bug clear:
https://www.typescriptlang.org/play?jsx=1#code/PQKhAIAECsGcA8CSBbADgewE4BcDK6BXTAYwFNxZ0AbASwBMBaOcEYAKBrS23AG8ApXAA0AvuABmmdMnABySrUZxZAbjZtsAT1Tlx6dOAC84QUIB0iAHbZMNS7BrEAolVLJS12OGDBw6ANYANOCWBqSYUpjBpuA0XnYUxOg64AAW4aRm6nSkxFQAhpjkyOh0BK5yCvRMsLJ8bOAh+e6wqPlkJsL1jY122OHi7eRWNnYOzq7unt09jbJ66Aykkx7YsgBcfAublgTIAEbhIg09x43Hx2xJ9jzLRuAAFCc+4ABCBDwxcSHoPAmwSRS6SKAH4TgAeBZLFbWAB8AAlllQDAB3LBUOjg4BQ5ZuVawtgAShUQA
Two things:

playground does not correctly discover solid-js/jsx-runtime, thus it has an error
If JSX (TSX) had a better design (i.e. none of this jsxImportSource with hard-coded /jsx-runtime.* file path stuff), and instead allowed it to work with the intuitive import {JSX} from 'solid-js', then
a. The playground example would just work, just as anyone familiar with ES Module syntax would expect.
b. The code would be DRY instead of WET, because as you see JSX has to be imported twice for things to work,

once for non-JSX code within the file (using import {JSX})
and once for JSX code within the same file (using @jsxImportSource)

The compiler follows the JSX babel implementation, so if that changes so will we. The website issues isn't the place for discussing that.
That said this is the website repo, and I'm still OK with what I said earlier. I'd accept a PR for where ATA would handle this case but given that most examples would import the module for its  API anywau, it'd get picked up by that.

but given that most examples would import the module for its API

Not sure if you mean import {JSX} by that, but that doesn't work for JSX markup (this issue).
I made another example, that shows that even if solid-js exists and I can import its APIs, @jsxImportSource still doesn't see the package: playground.
This JSX stuff is difficult to learn, and by it not working in playground may give users a false sense of that approach being wrong.

but given that most examples would import the module for its API

Not sure if you mean import {JSX} by that, but that doesn't work for JSX markup (this issue).
I made another example, that shows that even if solid-js exists and I can import its APIs, @jsxImportSource still doesn't see the package: playground.
This JSX stuff is difficult to learn, and by it not working in playground may give users a false sense of that approach being wrong.

Looks like the problem has been partially fixed, and now my previous comment's example works. @jsxImportSource will work so long as an import statement also imports the package, that way the package will be installed locally by the import statement and hence the @jsxImportSource machinery will see the package.
playground example
If you comment that line out (and refresh), then the @jsxImportSource stops working:
playground example
The separate import is still required for @jsxImportSource to work... but it works now! :tada:


GitHub "About" shows link to old .org domain

It should refer to the new .dev domain now

Hi Dale! Always glad to hear from you.
You're right. I know of at least 2 other things to fix. Keep noting them & we'll work them off. We didn't plan to change domains originally, so it's been a challenge to find all the places to fix.

Well, this problem is also fixed. There may be others. Let me know as you find them.
I do know we have some old URLs in the translations. I'll start by begging the translators. We're also using the old domain for sending email, intentionally; that will be addressed separately.


Add Merkle Mountain Range

Add Merkle Mountain Range. It should be done using the merkle tree backends.

@MauroToscano would like to take this up


Fix RTD by removing m2r package

@jeff231li This seems to fix the RTD failing.

Codecov Report

Merging #132 into master will not change coverage.
The diff coverage is n/a.

awesome!


Realm Backtraces Need Line Numbers

Currently Realm backtraces don't report line numbers. We can fix this either using a gdb script or by using addr2line. Not sure about support for Mac OSX.

I think we have this now? @eddy16112

yes, it is supported when libdw is enabled.


question: how support with  typeorm ^0.3.17

how support with  typeorm ^0.3.17
createConnection' is deprecated
useContainer' is deprecated

please update


webhdfs: implement cp_file/rm_file

Implements cp_file (copying through the user) and rm_file. Also removes a logging call, since it might contain non-string arguments.

I think it makes sense, creating https://github.com/intake/filesystem_spec/issues/763. Btw @martindurant if could you make a new release with all new hdfs enhancements, it would be really cool.

if could you make a new release

On my list!


Errors when running without an valid Kerberos ticket

If import is started without an valid Kerberos ticket, you will get an the following error.
19/04/12 04:49:56 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
19/04/12 04:49:56 INFO retry.RetryInvocationHandler: java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

Add Kerberos initialization based on Kerberos information in configuration file. As a work around, make sure that you run kinit and get a valid ticket before starting.

A 'kinit -s' is no executed at the beginning of the 'import' command and checks if there is a valid ticket.


Filtering Groups with pagination in SCIM2 fails when the groups are created in secondary userstore

Suggested Labels
Affected : 5.8.0- beta
Severity: Major
Priority: Highest
Component: SCIM2
Type-Bug
Steps to reproduce

Create a secondary user store
Create 2 roles as SECONDARY/secondaryrole1, SECONDARY/secondaryrole2 in the secondary userstore
Try to filter the roles in the secondary user store with the pagination and startIndex

Sample Request
curl -v -k --user admin:admin 'https://is580cluster2.qa.com:443/scim2/Groups?domain=SECONDARY&filter=displayName+eq+secondaryrole1&startIndex=1&count=1'
Response
{"totalResults":0,"startIndex":1,"itemsPerPage":0,"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"]}

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


9.12.0 not appearing on Packagist

https://packagist.org/packages/m4tthumphrey/php-gitlab-api does not have https://github.com/m4tthumphrey/php-gitlab-api/releases/tag/9.12.0 yet.
GitHub changed how hooks need to be implemented, which kicked in at the end of January. https://twitter.com/packagist/status/1037035587453235201
Thanks for tagging the release!

cc @m4tthumphrey could you set up the GitHub app or transfer me the repo?

I'll sort it tomorrow @m1guelpf

Think I've sorted it @m1guelpf


Add BottomSheetHeader for the BottomSheet

Table of Contents
Link to GitHub issues it solves
closes #96
Description
Add BottomSheetHeader for the BottomSheet
Contributing has more information and tips for a great pull request.

@zeobot set lc to pass


support distributed deadlock detector

the distributed deadlock detect mechanism implemented in tikv works as follows:
After the start of tikv-server, the deadlock detecting thread running with Detector.run method, this thread is responsible for handling deadlock detection tasks as well as cleanup tasks.
Initially, there will be one leader tikv electing using pd-server,  which will handle the tasks for the whole cluster.
The processing  data stream:

acquire_pessimistic_lock ->  Error::KeyIsLocked
The request is added to the waiter_manager as a lock waiter and the wait event is triggered.
Then on_wait_for_lock handler will handle the wait event, in which the deadlock detection will be executed, this is executed in the LockManager.wait_for, the detection is only used for the non-first lock.
In handle_detect, the detection worker will decide to process locally or to send an RPC request to the detection leader.
The lock wait is described as an edge like  txnTs(node) -> lockTs(node), the deadlock detection is to check if there is another edge lockTs -> txnTs, which results in loop.
In do_detect, the util will use the wait_for_ts as the edge start, try to find one existing edge ending with the txn_ts, if found there will be deadlock and report deadlock error.
The cleanup will handle wakeup event if that the lock is already acquired, and the waiting edge has also an ttl.


Home Assistant non responsive - Event loop is closed

Home Assistant release with the issue: 0.87.1

Operating environment (Hass.io/Docker/Windows/etc.): Raspbian with VirtualEnv

Description of problem:
Home Assistant is non responsive to any call from web browser or third party software (e.g. Homekit).
Restart of HA does not necessary solve the issue.
Reboot of Raspberry does not necessary solve the issue.
Issue can occur also after a while that the HA is running.
Logs show error related to Runtime Error: Event loop is closed.
I attach to this issue traceback regarding one example after a restart, in this case related to components/rpi_gpio.py, but I already experienced same issue related to Homekit component.
In the Homekit case, HA was running fine (I think), and a request from Homekit to HA to open_cover broke everything with a similar traceback.
Traceback (if applicable):
2019-02-18 06:19:24 ERROR (SyncWorker_10) [concurrent.futures] exception calling callback for <Future at 0x6da87250 state=finished raised RuntimeError>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/rpi_gpio.py", line 31, in setup
    hass.bus.listen_once(EVENT_HOMEASSISTANT_START, prepare_gpio)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/core.py", line 592, in listen_once
    self._hass.loop, self.async_listen_once, event_type, listener,
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/util/async_.py", line 202, in run_callback_threadsafe
    loop.call_soon_threadsafe(run_callback)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 621, in call_soon_threadsafe
    self._check_closed()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 358, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 324, in _invoke_callbacks
    callback(self)
  File "/usr/local/lib/python3.6/asyncio/futures.py", line 414, in _call_set_state
    dest_loop.call_soon_threadsafe(_set_state, destination, source)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 621, in call_soon_threadsafe
    self._check_closed()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 358, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Additional information:

Could  you post your full configuration and included your logs several lines before the exception?
This error is not necessary related with components/rpi_gpio.py, it more likes some other guys wrongly closed event loop.

I don't have logs right now cause I restarted HA after it happened.
As soon as it happen again I will post logs.
I can say now that after the restart, every time I see this line of log
Ended unfinished session (id=335 from 2019-02-18 08:44:22.084358) 18 febbraio 2019, 09:54 components/recorder/__init__.py (WARNING)
For the configuration I have quite a big configuration, splitted in several file.
Do you need it entirely, or you just need a sum up of which components I'm using?

components list is fine, and custom components list as well.

Below is the list.
Plus I have a transmission custom component which is the same as the last PR is going to be released in next version.
Do you see anything suspitious?
Should I enable some logs to have more information next time it will occur?
binary_sensor:

template

cameras:

local_file
mjpeg

covers:

template

lights:

mqtt

weather:

darksky

sensors:

yr
systemmonitor
command_line
uptime
template
trakt
filesize

switches:

rpi_gpio

climate:

daikin

general configuration:

mqtt
frontend
google_assistant
lovelace
config
api
http
updater
discovery
conversation
history
logbook
map
sun
tts
cloud
ios
panel_iframe
homekit
zone
rest_command
telegram_bot
shell_command
transmission
input_boolean

notify:

telegram

Happened again this evening, i told Siri to close a cover, and it failed in doing it saying "I didn't receive any response from your device".
Previously (5 min before) I used a Shortcut in IOS to do a call on the api-event of HA from Siri (i'm trying to open a gate from the earphone of my motocycle helmet). It failed in doing so, but I don't know why, I still have to fully analize it.
Here's the log retrieved:
2019-02-18 12:55:57 WARNING (updater) [telegram.vendor.ptb_urllib3.urllib3.connectionpool] Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<telegram.vendor.ptb_urllib3.urllib3.connectionpool.HTTPSConnectionPool object at 0x648e7510>, 'Connect timed out. (connect timeout=5.0)')': /bot649570830:AAFKOwFgDok69IF7Bfexxl06yHsXVIn2CSo/getUpdates 2019-02-19 00:34:28 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last):   File "/srv/homeassistant/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 275, in data_received     messages, upgraded, tail = self._request_parser.feed_data(data)   File "aiohttp\_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data aiohttp.http_exceptions.BadStatusLine: invalid HTTP method 2019-02-19 04:31:56 WARNING (updater) [telegram.vendor.ptb_urllib3.urllib3.connectionpool] Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<telegram.vendor.ptb_urllib3.urllib3.connectionpool.HTTPSConnectionPool object at 0x648e7510>, 'Connect timed out. (connect timeout=5.0)')': /bot649570830:AAFKOwFgDok69IF7Bfexxl06yHsXVIn2CSo/getUpdates 2019-02-19 08:18:17 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-19 08:18:48 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-19 08:28:42 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-19 08:28:52 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-19 10:15:17 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-19 10:15:17 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-19 10:40:22 ERROR (MainThread) [homeassistant.core] Error doing job: Fatal read error on socket transport Traceback (most recent call last):   File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 723, in _read_ready     data = self._sock.recv(self.max_size) TimeoutError: [Errno 110] Connection timed out 2019-02-20 10:59:09 ERROR (updater) [homeassistant.core] Error doing job: Task was destroyed but it is pending! 2019-02-20 22:15:21 ERROR (Thread-272) [pyhap.hap_server] Exception in set_characteristics: Event loop is closed Traceback (most recent call last):   File "/srv/homeassistant/lib/python3.6/site-packages/pyhap/hap_server.py", line 522, in handle_set_characteristics     self.client_address)   File "/srv/homeassistant/lib/python3.6/site-packages/pyhap/accessory_driver.py", line 624, in set_characteristics     char.client_update_value(cq[HAP_REPR_VALUE])   File "/srv/homeassistant/lib/python3.6/site-packages/pyhap/characteristic.py", line 213, in client_update_value     self.setter_callback(value)   File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/homekit/accessories.py", line 48, in wrapper     dt_util.utcnow() + timedelta(seconds=DEBOUNCE_TIMEOUT))   File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/event.py", line 29, in factory     hass.loop, ft.partial(async_factory, *args, **kwargs)).result()   File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/util/async_.py", line 202, in run_callback_threadsafe     loop.call_soon_threadsafe(run_callback)   File "/usr/local/lib/python3.6/asyncio/base_events.py", line 621, in call_soon_threadsafe     self._check_closed()   File "/usr/local/lib/python3.6/asyncio/base_events.py", line 358, in _check_closed     raise RuntimeError('Event loop is closed') RuntimeError: Event loop is closed

Something more that may help: temperature sensors were not updated from 12h before this incident, they read from outside so they should have red different values. Probably HA was not responsive already at that time. Last sensor update at around 11:51 in the morning.

Happened again today, only line of logs is this one
2019-02-21 06:26:13 ERROR (updater) [homeassistant.core] Error doing job: Task was destroyed but it is pending!
I'll try to remove the updater component, and check if that error will not show anymore at least.

Can you turn on more detail logs? Looks like you are not log INFO level message, therefore it is very hard to determined which component is last working one (and maybe the problem one)
logger:
  default: info

You can also try to change it to debug level if we still cannot get valuable information.

Ok I’ll setup logger to info and restore updater component.

Hi @awarecan here I am again.
After several days of "everything working fine" this morning HA was stucked again.
I noticed because I was trying to open covers using SIRI and I got no answer from HA.
Here the last ours of logs, where you can find some error.