[BUG] Buidconfig did not work - can mount a bunch of default openshift ressources

Hi,
I tried to run a build on microshift and it failed. It might be possible a build make not sense on microshift but its for me the only to build an arm container image on my Pi4.
What happened:
$ oc describe builds/openshift-examples-1
...
Events:
  Type		Reason		Age			From			Message
  ----		------		----			----			-------
  Normal	Scheduled	66s			default-scheduler	Successfully assigned build-example/openshift-examples-1-build to microshift
  Warning	FailedMount	65s			kubelet			MountVolume.SetUp failed for volume "builder-dockercfg-trjjx-push" : failed to sync secret cache: timed out waiting for the condition
  Warning	FailedMount	65s			kubelet			MountVolume.SetUp failed for volume "builder-dockercfg-trjjx-pull" : failed to sync secret cache: timed out waiting for the condition
  Warning	FailedMount	65s			kubelet			MountVolume.SetUp failed for volume "build-ca-bundles" : failed to sync configmap cache: timed out waiting for the condition
  Warning	FailedMount	2s (x8 over 66s)	kubelet			MountVolume.SetUp failed for volume "node-pullsecrets" : hostPath type check failed: /var/lib/kubelet/config.json is not a file
  Warning	FailedMount	1s (x7 over 65s)	kubelet			MountVolume.SetUp failed for volume "build-ca-bundles" : configmap references non-existent config key: service-ca.crt

What you expected to happen:
Run build
How to reproduce it (as minimally and precisely as possible):
oc new-project build-example
oc apply -k https://github.com/openshift-examples/web-deployment.git/overlay/simple-build-deploy/

Anything else we need to know?:
Environment:

Microshift version (use microshift version):

MicroShift Version: 4.8.0-0.microshift-unknown
Base OKD Version: 4.8.0-0.okd-2021-10-10-030117

Hardware configuration: Pi4
OS (e.g: cat /etc/os-release):

NAME="Fedora Linux"
VERSION="35.20211212.1 (IoT Edition)"
ID=fedora
VERSION_ID=35
VERSION_CODENAME=""
PLATFORM_ID="platform:f35"
PRETTY_NAME="Fedora Linux 35.20211212.1 (IoT Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:35"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f35/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=35
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=35
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="IoT Edition"
VARIANT_ID=iot
OSTREE_VERSION='35.20211212.1'

Kernel (e.g. uname -a): Linux microshift 5.15.6-200.fc35.aarch64 #1 SMP Wed Dec 1 13:25:57 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
Others:

Relevant Logs

more details:
$ oc describe pods openshift-examples-1-build | tail
Events:
  Type     Reason       Age                  From               Message
  ----     ------       ----                 ----               -------
  Normal   Scheduled    2m20s                default-scheduler  Successfully assigned build-example/openshift-examples-1-build to microshift
  Warning  FailedMount  2m19s                kubelet            MountVolume.SetUp failed for volume "builder-dockercfg-trjjx-push" : failed to sync secret cache: timed out waiting for the condition
  Warning  FailedMount  2m19s                kubelet            MountVolume.SetUp failed for volume "builder-dockercfg-trjjx-pull" : failed to sync secret cache: timed out waiting for the condition
  Warning  FailedMount  2m19s                kubelet            MountVolume.SetUp failed for volume "build-ca-bundles" : failed to sync configmap cache: timed out waiting for the condition
  Warning  FailedMount  17s                  kubelet            Unable to attach or mount volumes: unmounted volumes=[node-pullsecrets build-ca-bundles], unattached volumes=[build-system-configs build-blob-cache kube-api-access-xgzmd node-pullsecrets builder-dockercfg-trjjx-push container-storage-root buildworkdir builder-dockercfg-trjjx-pull build-ca-bundles build-proxy-ca-bundles buildcachedir]: timed out waiting for the condition
  Warning  FailedMount  12s (x9 over 2m20s)  kubelet            MountVolume.SetUp failed for volume "node-pullsecrets" : hostPath type check failed: /var/lib/kubelet/config.json is not a file
  Warning  FailedMount  11s (x8 over 2m19s)  kubelet            MountVolume.SetUp failed for volume "build-ca-bundles" : configmap references non-existent config key: service-ca.crt

$ oc get pods  openshift-examples-1-build  -o yaml | grep -A70 volumes
  volumes:
  - hostPath:
      path: /var/lib/containers/cache
      type: ""
    name: buildcachedir
  - emptyDir: {}
    name: buildworkdir
  - hostPath:
      path: /var/lib/kubelet/config.json
      type: File
    name: node-pullsecrets
  - name: builder-dockercfg-trjjx-push
    secret:
      defaultMode: 384
      secretName: builder-dockercfg-trjjx
  - name: builder-dockercfg-trjjx-pull
    secret:
      defaultMode: 384
      secretName: builder-dockercfg-trjjx
  - configMap:
      defaultMode: 420
      name: openshift-examples-1-sys-config
    name: build-system-configs
  - configMap:
      defaultMode: 420
      items:
      - key: service-ca.crt
        path: certs.d//ca.crt
      name: openshift-examples-1-ca
    name: build-ca-bundles
  - configMap:
      defaultMode: 420
      items:
      - key: ca-bundle.crt
        path: tls-ca-bundle.pem
      name: openshift-examples-1-global-ca
    name: build-proxy-ca-bundles
  - emptyDir: {}
    name: container-storage-root
  - emptyDir: {}
    name: build-blob-cache
  - name: kube-api-access-xgzmd
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace
      - configMap:
          items:
          - key: service-ca.crt
            path: service-ca.crt
          name: openshift-service-ca.crt
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2021-12-22T14:34:40Z"
    message: 'containers with incomplete status: [git-clone manage-dockerfile]'
    reason: ContainersNotInitialized
    status: "False"

The build controller assumes a lot of the referenced Secrets and ConfigMaps are created by other controllers/operators. In microshift it appears that many of these have been disabled or removed entirely.

same issue here,
Events:
  Type		Reason		Age			From			Message
  ----		------		----			----			-------
  Normal	Scheduled	6m20s			default-scheduler	Successfully assigned website/bolt-image-1-build to microshift.groupe-speleo-quercy.fr
  Warning	FailedMount	4m18s			kubelet			Unable to attach or mount volumes: unmounted volumes=[node-pullsecrets build-ca-bundles], unattached volumes=[buildcachedir builder-dockercfg-qw9xh-pull build-blob-cache node-pullsecrets build-system-configs build-ca-bundles kube-api-access-f89ff buildworkdir build-proxy-ca-bundles builder-dockercfg-qw9xh-push container-storage-root]: timed out waiting for the condition
  Warning	FailedMount	2m2s			kubelet			Unable to attach or mount volumes: unmounted volumes=[node-pullsecrets build-ca-bundles], unattached volumes=[build-proxy-ca-bundles build-blob-cache node-pullsecrets builder-dockercfg-qw9xh-push build-system-configs build-ca-bundles container-storage-root buildworkdir buildcachedir builder-dockercfg-qw9xh-pull kube-api-access-f89ff]: timed out waiting for the condition
  Warning	FailedMount	9s (x11 over 6m21s)	kubelet			MountVolume.SetUp failed for volume "build-ca-bundles" : configmap references non-existent config key: service-ca.crt
  Warning	FailedMount	9s (x11 over 6m21s)	kubelet			MountVolume.SetUp failed for volume "node-pullsecrets" : hostPath type check failed: /var/lib/kubelet/config.json is not a file

does this mean that BuildConfigs cannot be used in MicroShift [and should be removed]?

I think the unattached volumes may be a side effect of the MountVolume.SetUp failures.
MountVolume.SetUp failed for volume "node-pullsecrets"
MicroShift has relocated the kubelet's config from /var/lib/kubelet to /var/lib/microshift/resources/kubelet/config/config.yaml. Copying and converting the file to /var/lib/kubelet seems to satisfy that error.
MountVolume.SetUp failed for volume "build-ca-bundles"
Unfortunately, it looks like the openshift-examples-1-ca configMap (ref'd by the build-ca-bundles volume) is derived from a configMap in the openshift-config namespace.  Since MicroShift doesn't embed the openshift-config-controller, the namespace, and thus the config map don't exist.  I've tried editing the openshift-exampled-1-ca configMap with the metadata.annotations."service.beta.openshift.io/inject-cabundle"="true", per docs.  It does inject a CA bundle to the configMap, but by then it looks like the pod has been dropped from the scheduling queue.
Still looking to see if it's possible to hack a way around this, perhaps by creating the openshift-config NS and generating the configMap there.  openshift- prefix NS are prohibited, so I don't know how successful I'm going to be here.

openshift- prefix NS are prohibited, so I don't know how successful I'm going to be here.

I noticed that projects are prohibited, but not namespaces :#

(I'm not suggesting it's a good idea, just that it works ^^)
[tested above on MicroShift, and I know it behaves the same way in OpenShift]

Ah, right.  Makes sense, since projects are managed by a different controller.  I'll post if I find a more elegant solution.

Just for information:
$ oc new-project openshift-test
Error from server (Forbidden): project.project.openshift.io "openshift-test" is forbidden: cannot request a project starting with "openshift-"
$ oc adm new-project openshift-test
Created project openshift-test
$

oc adm is the key, for normal project requests names starting with openshift- is forbidden.

Thanks @rbo .  Using that, here's a hacky little recipe that gets us a little closer:

Create the required project name:
$ oc adm new-project openshift-config
Convert the existing kubelet config to json and write to expected path:

CONFIG=$(
python -c '
import sys, yaml, json
json.dump(yaml.load(stream=sys.stdin, Loader=yaml.Loader), sys.stdout, indent=4)
' <<< "$(podman exec -it microshift cat /var/lib/microshift/resources/kubelet/config/config.yaml)"
)
podman exec -it microshift echo "$CONFIG" > /var/lib/kubelet/config.json

Create the images.config.openshift.io object to tell the build-controller where to clone the ca bundle from.  Necessary because the build controller hard-codes the openshift-config namespace for the configMap, but derives the cm's name from this Image config.

cat <<EOF | oc apply -f -
apiVersion: config.openshift.io/v1
kind: Image
metadata:
  name: cluster
spec:
  additionalTrustedCA:
    name: "openshift-service-ca.crt"
EOF

Execute the build described above:

oc new-project build-example
oc apply -k https://github.com/openshift-examples/web-deployment.git/overlay/simple-build-deploy/

Wonder why thy build image is tagged "unknown", causing the pod to keel over with ImagePullBackOff because the tag doesn't exist.

❯ oc get pods
NAME                                READY   STATUS                  RESTARTS   AGE
openshift-examples-1-build          0/1     Init:ImagePullBackOff   0          52m
opensift-examples-7bf88f65c-5n6jx   1/2     Running                 0          52m
❯ oc get pods openshift-examples-1-build -o yaml | yq ".spec.containers[] | .image"
quay.io/openshift/origin-docker-builder:unknown

@rootfs any ideas on the above?

On Fedora IoT & Raspberry Pi 4, RPM installed microshift:
[root@microshift ~]# curl -L -o yq https://github.com/mikefarah/yq/releases/download/v4.21.1/yq_linux_arm64
[root@microshift ~]# chmod +x yq
[root@microshift ~]# ./yq -o json /var/lib/microshift/resources/kubelet/config/config.yaml > /var/lib/kubelet/config.json

@copejon I guess you missed the step: oc annotate cm/openshift-examples-1-ca service.beta.openshift.io/inject-cabundle=true  in comment https://github.com/redhat-et/microshift/issues/526#issuecomment-1061217866
I have the same problem right now:
$ oc get pods
NAME                                READY   STATUS                  RESTARTS   AGE
openshift-examples-2-build          0/1     Init:ImagePullBackOff   0          4m36s
opensift-examples-7bf88f65c-xxmt4   0/2     CrashLoopBackOff        12         8m55s
$ oc get pods -l openshift.io/build.name=openshift-examples-2 -o json  | jq '.items[] | .spec.containers[] | .image'
"quay.io/openshift/origin-docker-builder:unknown"
$ oc get pods -l openshift.io/build.name=openshift-examples-2 -o json  | jq '.items[] | .spec.initContainers[] | .image'
"quay.io/openshift/origin-docker-builder:unknown"
"quay.io/openshift/origin-docker-builder:unknown"

On a normal OpenShift env.
$ oc get pods -l openshift.io/build.name=openshift-examples-1 -o json  | jq '.items[] | .spec.containers[] | .image'
"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:913845b126727a043a295100251ee1d4076f565a1776608e3d7c3b9d389e86fd"
$ oc get pods -l openshift.io/build.name=openshift-examples-1 -o json  | jq '.items[] | .spec.initContainers[] | .image'
"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:913845b126727a043a295100251ee1d4076f565a1776608e3d7c3b9d389e86fd"
"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:913845b126727a043a295100251ee1d4076f565a1776608e3d7c3b9d389e86fd"

It looks like there some information missing which Image have to use for the build.

On a normal OpenShift env, the builderImage is here configured:
$ oc get cm openshift-controller-manager-images  -n openshift-controller-manager-operator -o yaml
apiVersion: v1
data:
  builderImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:913845b126727a043a295100251ee1d4076f565a1776608e3d7c3b9d389e86fd
  deployerImage: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:234505faa01bb8741ee1fddf86a1a3c359dd7d67221c7eda9307e5e37d90a2f5
kind: ConfigMap
metadata:
  annotations:
    include.release.openshift.io/ibm-cloud-managed: "true"
    include.release.openshift.io/self-managed-high-availability: "true"
    include.release.openshift.io/single-node-developer: "true"
  creationTimestamp: "2022-01-20T19:24:28Z"
  name: openshift-controller-manager-images
  namespace: openshift-controller-manager-operator
  ownerReferences:
  - apiVersion: config.openshift.io/v1
    kind: ClusterVersion
    name: version
    uid: f73ded6e-2055-49d2-8c5f-a0e1cf92139c
  resourceVersion: "1337"
  uid: 4bc50cc2-58f5-4cb7-b261-57e137002dec

I will try to add the missing configuration into /var/lib/microshift/resources/openshift-controller-manager/config/config.yaml but it looks like I can not change the configuration, it will change do default: https://github.com/redhat-et/microshift/blob/df2566b1668015d195501d246562844d4bda7945/pkg/controllers/openshift-controller-manager.go#L87-L101
Here the configuration from normal OpenShift env:
$ oc get -n openshift-controller-manager cm/config -o jsonpath="{.data.config\.yaml}" | jq
{
  "apiVersion": "openshiftcontrolplane.config.openshift.io/v1",
  "build": {
    "buildDefaults": {
      "resources": {}
    },
    "imageTemplateFormat": {
      "format": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:913845b126727a043a295100251ee1d4076f565a1776608e3d7c3b9d389e86fd"
    }
  },
  "deployer": {
    "imageTemplateFormat": {
      "format": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:234505faa01bb8741ee1fddf86a1a3c359dd7d67221c7eda9307e5e37d90a2f5"
    }
  },
  "dockerPullSecret": {
    "internalRegistryHostname": "image-registry.openshift-image-registry.svc:5000"
  },
  "ingress": {
    "ingressIPNetworkCIDR": ""
  },
  "kind": "OpenShiftControllerManagerConfig"
}

Docker builder image: registry.redhat.io/openshift4/ose-docker-builder:v4.9.0 is available for ARM but only with an red hat login.

@rbo @copejon  did you guys ever managed to get buildconfig/build working ? If yes can you share your workaround ? Thanks

@rbo @copejon there is a public image available quay.io/openshift/origin-docker-builder:4.9.0 however how could be instruct microshift to use this docker builder image instead of quay.io/openshift/origin-docker-builder:unknown ?

@ksingh7 quay.io/openshift/origin-docker-builder:4.9.0 is not available for ARM, this is at least for me not useable. I did not spent anymore time in this because if will be disable in the future #633

I believe building in MicroShift is out of scope for the project. The goal is to be lightweight and provide all necessary APIs to maintain/run workloads, but building of the workloads should be done somewhere else.
@fzdarsky , WDYT ^


Fix resolution issues with aggregating projects.

Fixes #21

Is there any guarantee that the one you request is always last?

@sjrd You'd probably need to direct that query to the SBT maintainers :) It's the observable behavior that I see in my own testing. Much like any output from SBT, it's possible a newer version could alter the output format.
One could attempt to parse the output, matching the project identifier in an attempt to make sure the appropriate path is collected? Might be a bit overkill?

@sjrd Baking the workaround directly in seems to be a reasonable approach, so I did that.
Attempting to parse quickly proves overly tricky -- too many edge cases with the difference between input and output of tasks re: implicit axes (e.g. "fastLinkJSOutput" and "root / fastLinkJSOutput" tasks come out as "Compile / fastLinkJSOutput")
I am unaware of any cases where setting aggregate on a key should fail. I also couldn't think of any use case for needing the aggregation to happen when attempting to retrieve the value.
Thoughts?


Add shuffle perturbation

shuffle all but the initial and final character

@parantak, Is this from some paper? Please cite the source, @rajaswa this is a shuffle, swap swaps two adjacent characters. I am doubtful if shuffle can be considered as a unit attack because it involves a lot of change for the word, also the edit distance would heavily change for words of different lengths

https://arxiv.org/pdf/1711.02173.pdf
@someshsingh22
@rajaswa
I might be wrong. But I believe shuffling all but the initial and final character could be perceived as a character perturbation, as it would, in essence, cause a visual disruption but would still be perceptible by humans.
Original sentence: "According to a study from Cambridge University, it doesn’t matter which order letters in a word are"
Shuffled sentence: " According to a study from Cambridge Unvreisity, it doesn't matter which order letters in a word are "
You could go on to shuffle all the words in here, and this sentence would still be comprehensible.

@parantak I believe Somesh is right. We can;t completely shuffle it se=ince the semantics would be lost. Stuff like swap and replacing with nearest keyboard character(which I believe you initiated) are good.

I am not saying a shuffle would be wrong, it's just that I haven't encountered literature that mentions it, so this is in general, that if there's something that is not present in the project section or issues, just mention the source, otherwise we can add it as a dev feature from our own side and mention it in a note

The thing with adversarial attacks is that it should be perceptible to humans, which is I believe this perturbation is. Besides, I went through two papers where this attack was implemented, so I guess it adds some credibility. I am no expert, though, perhaps @rajaswa or @someshsingh22 could throw some light.
Link: https://arxiv.org/pdf/1711.02173.pdf, the paper where shuffle was used as synthetic noise. They weren't very successful when it came to dealing with the noise, but that was back in 2017.

@parantak  That is fine I guess, as long as there is literature supporting it.
Looks fine to me add the unit test and @rajaswa will merge it.

@parantak  this seems credible, but this makes swap function redundant
https://github.com/SforAiDl/decepticonlp/blob/e5ded5c025292889e7e994540bf6addcdf82424f/decepticonlp/transforms/perturb.py#L23
It'd be good if the two can be merged, with an extra parameter of % internal characters of a word  to to be swapped

@parantak I think @rajaswa meant to combine swap and midshuffle, pass an argument like mid=False. Also pull to the latest commit before implementing

yeah, I got that. I opened an issue for the same.


Adds HTTP TaskTimeouts

Adds ReadTimeout and connection timeout to HTTP Tasks.
We take the parameters from HTTP input. If they are not present,  then timeouts from config are used.
I have removed the input parameter in the RestManager.getClient method as it is an unused parameter.
RestManager.getClient uses threadlocal to create clients as we are modying the client object from various threads currently.

Pull Request Test Coverage Report for Build 2431

18 of 20 (90.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.1%) to 68.628%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

contribs/src/main/java/com/netflix/conductor/contribs/http/HttpTask.java
9
11
81.82%

Totals

Change from base Build 2428:
0.1%

Covered Lines:
9133

Relevant Lines:
13308

💛  - Coveralls


Proposal: register controller-manager & scheduler in etcd for componentStatuses

Currently, the ComponentStatuses code is using hardcoded <IP_ADDRESS> IPs and default ports when performing healthchecks on the controller-manager & scheduler:
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/master/master.go#L700
This means that k8s assumes the core components are all on the same host and networking stack (most deployments are using docker host networking to achieve this). However, this is asynchronous when compared with the existing ability to configure the apiserver endpoint on the other components. It's currently possible to manually deploy each component in its own docker container, without host networking, or even on different machines/vms and have everything work except componentStatuses.
To make this relationship more formal, and avoid triangular configuration dependencies, I propose that core components register themselves in etcd, and use it as a sort of service or cluster discovery mechanism.
Proposal A
Phase 1:

controller-manager registers itself when started
scheduler registers itself when started
ComponentStatuses reads the etcd registry to find the controller-manager & scheduler (or generically all core components in an etcd dir)

Phase 2:

apiserver registers itself when started
controller-manager looks up apiserver in etcd instead of by argument/configuration
scheduler looks up apiserver in etcd instead of by argument/configuration

I think this both makes kubernetes more flexible and (hopefully) reduces manual configuration.
Caveats:

when using service discovery, the components need to be configured with both the local IP/port to listen on on and the remote IP/hostname/dns/port to advertise, which may not be the same.

Blocking #10049 (unless it switches to docker host networking)

Two Points were made in conversations:

@thockin mentioned that the google team likely doesn't have a bunch of time to evaluate or help with a large change in this area. So the smallest change that solves the problem without preventing future changes is prefered.
@sttts mentioned that scheduler & controller-manager had moved away from talking to etcd directly in favor of using a Storage abstraction through the apiserver

Given these, I'll make a different proposal.
Proposal B

add a component api endpoint for other components to register themselves.

store the component data in etcd under registry/components/<component-type>/<uuid>.
use uuid to identify component instances (for a future when multiple exist)

controller-manager registers itself to apiserver
scheduler registers itself to apiserver
ComponentStatuses endpoint reads from etcd to find all the components under registry/components/ and checks their /healthz endpoints

One outstanding question:

Is it sufficient to register on component startup or do we need to regularly heartbeat?

For the "simplest solution" we could hardcode the uuid of the components, assuming exactly one instance per type. This way the uuid would never change. Re-registering after component restart would be sufficient to update the IP if it ever changes.
For the "ideal solution", in order to support multiple component instances, we would need lazily generated uuids for each component instance (disk caching would allow faster recovery, without waiting for a TTL). If the component's machine (and local disk) disappears, a replacement component would be expected to generate a new uuid (and may register with a different IP). In this scenario we would need either heart beating (component -> apiserver) or regular health checking (apiserver -> component). Etcd also has a TTL concept that we could use (that requires regular updates/confirmations), if we wanted to expose it through the generic storage abstraction.

In order to be more similar to the existing service rest registry, it probably makes sense to have the apiserver generate unique names (ex: scheduler-xxxxx) and return them to the component upon registration, if it doesn't provide one of its own.

I am interested in this topic, but need time to think through the ramifications and lifecycle aspects of managing this in etcd.
From a security standpoint, the only component that should speak to etcd directly is the kube-apiserver so the only viable option right now for me is Proposal B.

I'm playing around with implementing Proposal B atm. I'll comment on any other issues that I come across before it makes it to a PR.

I have been mulling the concept of initializers - which in concept, are just clients to the kube-apiserver that fulfill many of the use cases we use admission controllers for today, but avoid having to recompile the binary, etc.
Right now, componentStatus is really a fixed list of endpoints, so materializing it to make it more dynamic is a +1 in my view.  What I am struggling to think through is if what I want is just limited to the component object you loosely describe here, or if I want to add richer metadata around components/clients of kube-apiserver in general to support more of a management console.

I'm definitely open to adding more functionality around the component registration in the future. So it's worth exploring what that looks like to make sure we're forwards compatible, but I don't want to add much more than url registering atm. I am planning to register a root url with optional path tho, to support the use of proxies between components. It should also work for api requests if there ever are any other than just /healthz.

So, having attempted Proposal B, I'm running into a lot of changes for ComponentStatuses, now that there's a legit Component Storage and Registry. Unfortunately, this leaves etcd out, because it's not really a component of k8s and shouldn't be expected to register itself or heartbeat to the apiserver. I could hardcode in a hack to combine the concepts for reverse compatibility with the existing ComponentStatuses, but I'm leaning towards excluding etcd instead for simplicity.
Does the user of the ComponentStatuses endpoint need to know if etcd is healthy? If etcd is completely hosed, I would expect that the apiserver would stop responding as healthy on its /healthz (which also isn't included in /componentstatuses).
Currently, the only user of this endpoint that I know of is the cluster verification script used in kube-up. For that purpose I expect that one or more of the components would have failed to come up if etcd was borked. So it doesn't technically need to know, as long as the /healthz endpoints are assumed to be transitive (which they normally are for databases, in my experience).

If we do decide we still want the etcd healthz aggregation, but can accept than not being in /componentstatuses, then we could add a /storage/status or similar.
I'd also like to change /componentstatuses to /component/status to match the new /component endpoint.

@karlkfi GKE uses the /componentstatuses endpoint quite heavily. Please do not make any non-backwards compatible changes to it now that the v1 API is released. We can certainly talk about adding a new status endpoint with richer stats, but please make sure you have buy in from @bgrant0607 before heading too far down that road.

@roberthbailey Can you succinctly describe the GKE usage?
I'm not planning to break it, just trying to ascertain what's intended and desired (use cases) to know the different between "break" and "fix".
Having spent a a day in the componentstatuses code, I'm not particularly impressed with it's design or implementation. I don't mean to offend, but it seems rather haphazardly thrown together and doesn't follow the patterns of some of the other more well-factored endpoints. I'm hoping to make it more consistent and robust, but I need to know exactly how it's being used. Reverse compatibility is often a game of not breaking the users, rather than not making publicly exposed changes. I can probably put in a new endpoint and deprecate the old if it comes to that.

@fabioy can probably give a better succinct explanation than I can but we check the component status on cluster creation (just like the shell scripts) to verify that all of the master bits are reporting that they are healthy. We also continuously monitor the component status to detect when the master components become unhealthy so that they can be repaired. In both cases what we want is a single endpoint that rolls up the overall health status of the master.

Because of the need to do periodic health monitoring, a single endpoint that can gather all the data internally and report it is preferable. This includes doing health checks on etcd and any other new component.
Reading through the design proposed, it also raises the concern that the api server needs to have some notion of what components it should expect to be registered in etcd. Supposing that an etcd health check is followed by other component checks, would, in the degenerate case, a healthy etcd with no master components registered return an "all ok" from this call? That seems wrong. (You could argue that the client could do a per-component check, but then why would the client have to know what components the master should have?). This needs to be addressed.
Also, +1 on the haphazard way that "componentstatuses" was implemented (I took the original "/validate" endpoint and did the thinnest veneer of an API wrapper around it). It certainly needed some more love and attention.

The way I've implemented it so far, the apiserver doesn't need to know what components to expect to function. It just looks for everything under /registry/components/which would include /registry/components/scheduler-xxxxx and /registry/components/controller-manager-xxxxx after they've registered.
Of course, the user wouldn't know that components were unhealthy until after they had registered. So if they didn't register the cluster may look healthy (and smaller than normal). I don't see a way around that, other than configuring the number and type of components to expect on the apiserver (e.g. --components=controller-manager:2,scheduler:1), but that ruins dynamism and resizability...

A compromise would be to have a pre-baked minimum set of expected components and counts (presumably 1 is sufficient). So, it'd expect at least 1 etcd, 1 controller-manager, 1 scheduler, etc. Having more that are healthy would be ok, but at least 1 that is healthy is expected.
In this design, other components could register and also participate in health checks, but the pre-baked list functions as the minimally viable set for k8s to operate.

Presumably the initial deployment would know how many of each component to look for (e.g. the kube-up script will have a variable to launch 2 apiservers, so it should wait to say the cluster is ready until it seems that 2 apiservers have registered themselves). Monitoring the cluster after the initial deployment is a bit more complex, especially if we expect the components to be dynamically scaled up and down.

Catching up...
The intent when /componentstatuses was created was to add a component registration API. I'd like to also use it for addons, like DNS. I'll read the proposal, thanks.

Please see https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/devel/api-conventions.md and https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/devel/api_changes.md regarding how to change the API.
The resource collection would be /components, following our standard conventions. The etcd registry paths would also follow standard API conventions. I can't think of a reason why the API should be namespaced.
The implementation will need to go into the new experimental API group initially, so we can work out the kinks before promoting it.
cc @thockin for review
cc @kelseyhightower since he might be interested in bootstrapping (e.g., cert generation)
A quick thought for now: We should use Service/Endpoints to keep track of where components are. This wouldn't preclude running the components outside the cluster, but would make them easy to track when self-hosted. This would also make it pretty easy to determine which components should exist and are healthy, and would make the names of individual instances irrelevant. The component registration API would refer to the service, via ObjectReference.
I'll drop some more background in #246.

Another detail, in case you weren't aware: It's possible to map services to external entities by specifying a null selector and then setting Endpoints manually. See https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/user-guide/services.md#services-without-selectors for more details.

Yeah, just found devel/api_changes earlier today. Wish I'd seen that before I started coding. :)
Given the desire for reverse compatibility, I think I'm likely going to have to leave /componentstatuses mostly the way it is, deprecated, and add a new /component/status with the new registration semantics that users can migrate to.
So it seems like the two use cases /componentstatuses is used for are:

Check that a newly deployed cluster is up and healthy
Check that a running cluster is still up and healthy

Here's the catch: k8s doesn't know what a healthy k8s looks like, and we don't know now or at compile time what a deployed healthy k8s might look like, because k8s doesn't deploy itself.
Options:

Hard code minimum healthy cluster topology
Have the deployer register the expected components

I'm leaning towards option 2, but not via config, because that creates a configuration dependency cycle. If we have the deployment script wait until apiserver is healthy and then manually register all deployed components, then it can hit /component/status to check full cluster health. Any post-deployment topology changes would require re-registration.
There is another outstanding question, however: Does healthy mean "completely healthy" or does it mean "at least minimally healthy"?
Usually, if an HA dependency loses a node but still holds quorum, something that depends on it doesn't care, and keeps functioning and reporting itself as healthy. This is transitive health. In the case of notification, for example, I don't want to be notified if the cluster can automatically recover, but I probably do want to record it for metrics and logs.
For example, the apiserver /healthz endpoint shouldn't go unhealthy if 1 of 3 etcd nodes dies. But right now that would show up on the /componentstatuses. Is that required information or do we just want to know that etcd as a unit is functional?

@zmerlynn who has been thinking about some deployment changes recently as well.

Looks like /component/status doesn't work as a dynamic read-only endpoint when there's a /component, because it's not a proper subresource the same way other status endpoints are (e.g. pods/status & nodes/status).
Options:

Make it a real read/write subresource, with a cron or something that makes the health checks and updates the storage (etcd via apiserver)
Change the endpoint to /componentStatus so it's not parsed as a subresource.

named resources become /componentStatus/<name>

Change the subresource logic in api_installer's registerResourceHandlers

It occurs to me that Option 1 might also work as a caching layer, which we may need anyway. Without caching, the on-demand health checking is a denial of service liability. Once it's parallelized, it would convert a single incoming request into multiple concurrent outgoing requests, all internal to the cluster network.
That said, externalizing the health checks into yet another component doesn't sound like the best option either.
What if the components just heartbeat?
The scheduler and controller-manager could just continually re-register themselves, using the unique name they acquired when they first registered.
Etcd would have to be health checked by the apiserver, continuously.
Any AddOns we want to register would also have to heartbeat.
This has the added benefit that it simplifies the endpoint to be just etcd backed storage, instead of a read-only endpoint with logic. It also sets the network load at an admin configurable rate, rather than one that responds multiplicatively to external usage.

Component registration is required for clients to introspect which API groups are installed on the apiserver.  Technically all API groups are also services, so there is some overlap in behavior and intent.  However, components are not obviously namespaced (at least today).

There's nothing obviously wrong with having two separate rest endpoints for status and components, /componentstatus represents a particular view of /components

This is getting to be a bit tricky, because I think there are 2 somewhat
overlapping goals.

One is the extensibility and self-registration goal, where the location
of the component and number of instances can vary. Having a r/w endpoint,
with a simple registration format, seems like a good thing. I think the
design addresses this.

The second is the ability to monitor/health ping them. Here, the
requirements (or at least, my desire for them) are:

A single call to obtain all relevant information to make an assessment.
Multiple calls only invite more complexity, because of timeouts, data skew
and incomplete data issues. It's much simpler for any monitoring system if
it's a single call.
Some way to detect that all necessary components have registered (or at
least one of them). A simple way I can see of doing that is, during the
"health check" call, to detect if the necessary set is met, and if not, add
artificial entries for those components that haven't registered (i.e. if no
scheduler has registered, then add an entry for a scheduler with an error
of "NOT REGISTERED" or something like that).
The question of whether the information should be of the health of all or
the minimal set was raised. I think a cluster is considered healthy if the
minimum set is available and healthy (i.e. if an HA configuration is run,
the master is healthy if the minimum set for each component declares itself
to be healthy). Bonus if some format can be devised where this data, plus
the status of the other components (maybe marked as optional or extra
instance) can be conveyed in a single call.

I think there's some value in having the api-server poll the other
components, cache it and just return the data on calls (the DOS risk was
exactly one that we envisioned). The push model is ok, but there are gaps
(as the etcd thing shows) and is a bit more complex for clients to follow
(health endpoints are easy and maybe useable in other scenarios).
Anyway, given these overlapping requirements, unless a good r/w API
proposal emerges, I think I tend to favor two separate rest endpoints.
On Mon, Aug 3, 2015 at 9:49 PM, Clayton Coleman<EMAIL_ADDRESS>wrote:

There's nothing obviously wrong with having two separate rest endpoints
for status and components, /componentstatus represents a particular view of
/components
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11908#issuecomment-127477627
.

--
-- Fabio Yeon

cc @rrati - re: HA registration.

I've actually moved my experimental implementation towards having a pretty simple r/w /components endpoint on the apiserver (wrapping etcd) that the components just heartbeat registration updates to (create the first time to get a unique uid/name). Each update bumps the "last seen" to the current apiserver timestamp. To play with this, since the client had to be updated anyway, I exposed the r/w capabilities in kubectl. So in addition to checking the "last seen" from /components on the apiserver, you can also use kubectl with it's fancy formatting/filtering flags. I probably won't leave write capabilities enabled in kubectl, but read makes sense to keep.
I haven't quite worked out the best way to get from "last seen" to healthy/unhealthy yet. Someone has to pick a timeout, either the client (api parameter?) or the apiserver (config?).
We're likely going to need a read caching layer. It could be in-memory, but I'm beginning to suspect that we need etcd caching clients on every apiserver node, to handle caching externally.

I'm not sure every component needs heartbeating - isn't that what the node
is supposed to be doing with ready checks today?  And endpoint lists are
already the available backends.  If you wanted to health check a component
running on the cluster, the endpoints list is our best representation
On Aug 5, 2015, at 6:06 PM, Karl Isenberg<EMAIL_ADDRESS>wrote:
I've actually moved my experimental implementation towards having a pretty
simple r/w /components endpoint on the apiserver (wrapping etcd) that the
components just heartbeat registration updates to (create the first time to
get a unique uid/name). Each update bumps the "last seen" to the current
apiserver timestamp. To play with this, since the client had to be updated
anyway, I exposed the r/w capabilities in kubectl. So in addition to
checking the "last seen" from /components on the apiserver, you can also
use kubectl with it's fancy formatting/filtering flags. I probably won't
leave write capabilities enabled in kubectl, but read makes sense to keep.
I haven't quite worked out the best way to get from "last seen" to
healthy/unhealthy yet. Someone has to pick a timeout, either the client
(api parameter?) or the apiserver (config?).
We're likely going to need a read caching layer. It could be in-memory, but
I'm beginning to suspect that we need etcd caching clients on every
apiserver node, to handle caching externally.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11908#issuecomment-128163580
.

I'm mildly confused by your comment, @smarterclayton.
By "component" I mean "core component" (e.g. apiserver, scheduler, and controller-manager).
The ready-check (aka probe) that I know of is just the nodes (aka kubelets) probing the pods running on their machine.
I assume there's also some sort of ready-check for the nodes, as kubectl get nodes reveals them as "ready", but I don't know if/how that's updated regularly after initial registration.
My understanding of "endpoints" is that they represent service endpoints, not core component endpoints. The apiserver and scheduler just happen to have services in front of them to make it easier for pods to communicate with them.
Are you saying we should use the service endpoints to probe the health of the core components?
I suppose if we had some sort of built-in Load Balancer it could aggregate the health status of the endpoints it proxies to, but that still leaves needing another aggregation in front to collect the components' health. It would also mean that you couldn't check the health of individual component instances if the Load Balancers themselves were unhealthy.
It might be possible for /componentstatuses to search for named (hardcoded) service endpoints, but that seems a little roundabout. It would seem more logical for the core component services to be registered using the proposed /component registration.

On Aug 5, 2015, at 7:43 PM, Karl Isenberg<EMAIL_ADDRESS>wrote:
I'm mildly confused by your comment, @smarterclayton
https://github.com/smarterclayton.
By "component" I mean "core component" (e.g. apiserver, scheduler, and
controller-manager).
The ready-check (aka probe) that I know of is just the nodes (aka kubelets)
probing the pods running on their machine.
I assume there's also some sort of ready-check for the nodes, as kubectl
get nodes reveals them as "ready", but I don't know if/how that's updated
regularly after initial registration.
Each node heartbeats its /node, and also updates individual pods as their
status changes.  The interval is frequent.
My understanding of "endpoints" is that they represent service endpoints,
not core component endpoints. The apiserver and scheduler just happen to
have services in front of them to make it easier for pods to communicate
with them.
Are you saying we should use the service endpoints to probe the health of
the core components?
I suppose if we had some sort of built-in Load Balancer it could aggregate
the health status of the endpoints it proxies to, but that still leaves
needing another aggregation in front to collect the components' health. It
would also mean that you couldn't check the health of individual component
instances if the Load Balancers themselves were unhealthy.
It might be possible for /componentstatuses to search for named (hardcoded)
service endpoints, but that seems a little roundabout. It would seem more
logical for the core component services to be registered using the proposed
/component registration.
Down the road most of our components will be expected to run on the
infrastructure.  Also, extensions and plugins would be running on the
infrastructure as well.  In those cases, endpoints or pod status are
heartbeat (well, mostly).
I was referring to /components specifically - I think the most important
use case is to allow registration of new elements to the cluster, but what
isn't clear is why components are better than services in terms of
registration.
So a writable endpoint is fine, but we should consider whether we heartbeat
certain elements differently (those that are already providing that
outcome).
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11908#issuecomment-128184777
.

Even if you run all the components in a kubelet as a static pod, that kubelet isn't reporting those pods back to the apiserver. That kubelet isn't even part of the cluster, it's a special bootstrap node that doesn't have anyone monitoring it. So the core components (running in containers on that bootstrap kubelet) are probed for health and resurrected, but they're not reporting their health to the apiserver. There is no apiserver in the "bootstrap cluster", just a kubelet.

Is there a way to query the pod status directly on the bootstrap kubelet?
Even if their were, there's no bootstrap kubelet for some providers, namely Mesos, where the core components are either external or run in Marathon.

Proof of Concept PR: #12324

On Aug 5, 2015, at 9:43 PM, Karl Isenberg<EMAIL_ADDRESS>wrote:
Even if you run all the components in a kubelet as a static pod, that
kubelet isn't reporting those pods back to the apiserver.
Yes it is - they go to kube-system.  Unless you mean something else.
That kubelet isn't even part of the cluster, it's a special bootstrap node
that doesn't have anyone monitoring it. So the core components (running in
containers on that bootstrap kubelet) are probed for health and
resurrected, but they're not reporting their health to the apiserver. There
is no apiserver in the "bootstrap cluster", just a kubelet.
Are you assuming a bootstrap cluster?  It wasn't something we have been
assuming up until now.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11908#issuecomment-128207099
.

On Aug 5, 2015, at 9:45 PM, Karl Isenberg<EMAIL_ADDRESS>wrote:
Is there a way to query the pod status directly on the bootstrap kubelet?
There are status Apis yes, although they don't closely mirror their
apiserver counterparts yet.
Even if their were, there's no bootstrap kubelet for some providers, namely
Mesos, where the core components are either external or run in Marathon.
Like I said before, some components are special, I don't think it's
unreasonable to have them bootstrap.  But I don't think bootstrap is the
dominant reason to add /components - it would be for clients to discover
remote elements.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11908#issuecomment-128207428
.

Looking at giving the Component api a Status filed, similar to Node.Status. Can anyone explain why NodeStatus.Conditions is an array of NodeCondition and not just the last known condition? Does it store all historical conditions?

It does.  Conditions is a general convention api-conventions.md
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/devel/api-conventions.md#typical-status-properties
rather than rely on an enumeration, we prefer to represent possible
conditions.  Allows future conditions to be added in a backwards compatible
way without breaking clients.
On Thu, Aug 6, 2015 at 3:00 PM, Karl Isenberg<EMAIL_ADDRESS>wrote:

Looking at giving the Component api a Status filed, similar to
Node.Status. Can anyone explain why NodeStatus.Conditions is an array of
NodeCondition and not just the last known condition? Does it store all
historical conditions?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11908#issuecomment-128474520
.

--
Clayton Coleman | Lead Engineer, OpenShift

The Conditions array doesn't store "all historical conditions" in the sense of a time series of conditions. It stores the last-known observation of each of a variety of types of conditions, with limited historical information (e.g., time of last status change of each condition).

Closing this issue. Moving conversation to #13216.


Homepage

We could have some cards like:

"most popular concerts" based on how many people have said "i was there"
most recent concerts added
most recent venues added
most recent artists added

This is done


Emulate the CPU (#1)

Implement all of the CPU's instructions and CPU modes, etc:
https://problemkaputt.de/gbatek.htm#armcpureference && https://problemkaputt.de/gbatek.htm#gbatechnicaldata
Finally, include some basic file loading in order to receive instructions from binary.

No, uh, it doesn't actually emulate the CPU yet...
I was just writing some parsing for instruction input.
af86f2ff7624cbee9404cfda2c2484c6645e8c88, so far, is the latest commit and has added six of the sixty possible instructions in Arm7TDMI (for the GBA, as far as I know. I may have miscounted sixty, but that's the ballpark). You can see all of them here: https://problemkaputt.de/gbatek.htm#arminstructionsummary

Also, sorry about requesting a review, I didn't mean to do that until the changes were done.

ah, alright

I'm so lazy I wish I weren't so I could do this

yeah


when trying to compile typescript file with nact, I receive error saying dispatch accepts only 2 arguments

Expected Behavior
When compiling a typescript file that calls dispatch from nact with 3 arguments, I should not receive this error:
actor.ts:39:34 - error TS2554: Expected 2 arguments, but got 3.
39   dispatch(ctx.sender, ctx.name, ctx.self);

Current Behavior
I receive the error when compiling:
actor.ts:39:34 - error TS2554: Expected 2 arguments, but got 3.
39   dispatch(ctx.sender, ctx.name, ctx.self);

Possible Solution
Update typing to allow for a 3rd optional argument
Steps to Reproduce (for bugs)

Add the code from ping pong tutorial:

import {start,dispatch, spawnStateless} from 'nact';
const delay = (time) => new Promise((res) => setTimeout(res, time));

const ping = spawnStateless(system, async (msg, ctx) =>  {
  console.log(msg);
  // ping: Pong is a little slow. So I'm giving myself a little handicap :P
  await delay(500);
  dispatch(ctx.sender, ctx.name, ctx.self);
}, 'ping');

const pong = spawnStateless(system, (msg, ctx) =>  {
  console.log(msg);
  dispatch(ctx.sender, ctx.name, ctx.self);
}, 'pong');

dispatch(ping, 'begin', pong);

Compile with tsc

Context
I'm trying to use nact with typescript
Your Environment

Version used: nact 7.3.1
Environment name and version (e.g. Node v8.8) node 14.3.0
Operating System and version: macos catalina

I've updated the docs @zebralight to reflect the new signatures


Support custom woodwork types in components

Closes #1662. Another large-ish PR but not too much logic change--just replacing the final line to convert to WW to new util method, and writing a lot of tests. I initially wanted to iterate through all the components and test like the pattern we already have in test_components.py but realized that each component has its own nuanced behavior, so decided against it and added the tests to their individual files. The parameterization is similar in many cases though. 😬

Moves Woodwork-related util methods to woodwork_utils.py and move tests to test_woodwork_utils.py
Add private utility method to help best reassign logical types after transformation.

Notes on compatible and incompatible types. Mostly for my own understanding, but also helped inform test cases.
Datetime

Datetime --> Double: errors
Datetime --> Integer: errors
Datetime --> Categorical: works
Datetime --> Natural Language: works

Categorical

Categorical --> Datetime: errors
Categorical --> Double: works, sometimes. ("1", "2" okay, "a", "b" not okay)
Categorical --> Natural Language: works
Categorical --> Integer: errors

Integer

Integer --> Datetime: works... but isn't great lol (literally .000000001 ms)
Integer --> Double: works
Integer --> Categorical: works
Integer --> Natural Language: works

Double

Double --> Datetime: works... but isn't great lol (literally .000000001 ms)
Double --> Integer: works sometimes (if possible to convert)
Double --> Categorical: works
Double --> Natural Language: works

**Natural Language **

Natural Language --> Datetime: errors

Natural Language --> Double: works, sometimes

Natural Language --> Categorical: works

Natural Language --> Integer: works, sometimes

There's no good way of confirming when it is safe to convert floats to category or when it is not without trying. For example: can convert ['1', '2'] (categories) to float, but not ['a', 'b']. Hence, current implementation will try to set the type. If it fails, then don't do anything. If it succeeds, add to logical type dictionary to pass along to the initialization of Woodwork DataTable, so that Woodwork does not have to do type inference, saving us some time.

Right now, the weird situations aren't issues because we don't have any transformers that create categorical or natural language columns. I can see how components we add that do this can handle this manually, by specifying and keeping track of what columns have been changed. But this is certainly something to be wary of, and not ideal. Unfortunately, to support keeping track of user-specified overrides, I don't think this is something we can disallow since the conversion is possible (int --> string is a valid conversion), so we can only be mindful of when we're converting types.

Look into: featuretools component which changed the columns of the original input DF?

@thehomebrewnerd @tamargrey Added you guys because I thought it'd be great if you guys could take a look at _retain_custom_types_and_initalize_woodwork in woodwork_utils.py if you get the chance! 😁

@tamargrey @thehomebrewnerd thank you for your comments, they're super helpful!! Nate, I shelved the suggestion to override Woodwork's default type system in https://github.com/alteryx/evalml/issues/1796, but I'm really excited to see if we can implement that and manage to get better performance numbers from it 😁


Add TLS to Service Monitor Endpoint

Allow for setting up the TLS connection in the service monitor when TLS is enabled on the cluster.

Just curious as to why this was closed without merging?


Add tormol to the list of authors

He contributed significantly to the library, in particular the no_std feature and the conversion traits.
@tormol: Do you want to be listed in the authors list?

Yes :)


BIOS submodule URL uses SSH, rather than HTTPS

Currently, the BIOS submodule is referenced as an SSH address:
[submodule "lambdasoc/software/bios"]
	path = lambdasoc/software/bios
	url = git@github.com:lambdaconcept/lambdasoc-bios

This means that the submodule can only be pulled down if a GitHub-associated SSH key is available; which is somewhat stymieing when using lambdasoc in CI environments.
Would it be possible to change this to HTTPS? I'd happy submit a PR. :)

Thanks! I forgot to update it when we published the repo.


Performed user engagement analysis with unit conversions and data agg…

…regation

Extracted relevant columns for user engagement analysis from the telecom dataset.
Renamed byte-related columns to reflect conversion from bytes to megabytes.
Converted session duration and activity duration columns from milliseconds to seconds.
Aggregated user engagement metrics, including session frequency, session duration, and total data traffic (download and upload) per user.

.


Add 'type' parameter to serializable_hash

The resource object must now contain a 'type' member
http://jsonapi.org/format/#crud-creating
I figured splitting the pull requests would be a good idea. Then I discovered reality.

See PR #11


Add VM concurrency support

Fixes #2016
This PR adds a new flag, VMConcurrencyMode and changes cairoVMExecute.

Tested by running ./build/juno --http --vm-concurrency-mode and requesting
{
	"jsonrpc":"2.0",
	"method":"starknet_traceTransaction",
	"params":["0x5634f2847140263ba59480ad4781dacc9991d0365145489b27a198ebed2f969"],
	"id":1
}

Response body
{
  "jsonrpc": "2.0",
  "result": {
    "type": "INVOKE",
    "execute_invocation": {
      "contract_address": "0x31c9cdb9b00cb35cf31c05855c0ec3ecf6f7952a1ce6e3c53c3455fcd75a280",
      "calldata": [
        "0x20cfa74ee3564b4cd5435cdace0f9c4d43b939620e4a0bb5076105df0a626c6",
        "0x5aee31408163292105d875070f98cb48275b8c87e80380b78d30647e05854d5"
      ],
      "caller_address": "0x0",
      "result": [],
      "calls": [
        {
          "contract_address": "0x20cfa74ee3564b4cd5435cdace0f9c4d43b939620e4a0bb5076105df0a626c6",
          "calldata": [
            "0x5aee31408163292105d875070f98cb48275b8c87e80380b78d30647e05854d5",
            "0x7e5"
          ],
          "caller_address": "0x31c9cdb9b00cb35cf31c05855c0ec3ecf6f7952a1ce6e3c53c3455fcd75a280",
          "result": [],
          "calls": [],
          "events": [],
          "messages": [],
          "execution_resources": { "steps": 25 }
        },
        {
          "contract_address": "0x20cfa74ee3564b4cd5435cdace0f9c4d43b939620e4a0bb5076105df0a626c6",
          "calldata": [
            "0x5aee31408163292105d875070f98cb48275b8c87e80380b78d30647e05854d5"
          ],
          "caller_address": "0x31c9cdb9b00cb35cf31c05855c0ec3ecf6f7952a1ce6e3c53c3455fcd75a280",
          "result": ["0x7e5"],
          "calls": [],
          "events": [],
          "messages": [],
          "execution_resources": { "steps": 31 }
        }
      ],
      "events": [],
      "messages": [],
      "execution_resources": { "steps": 178 }
    }
  },
  "id": 1
}

The current problem is this bug. It happened in sequential mode.
My hypotheses were:

Transaction preparation loop:
In the updated code, transactions are first prepared and stored in a Vec<Transaction> before being executed. This is different from the old version, where each transaction was processed immediately after preparation. This change in flow can cause the error.
State management:
In the old code, the CachedState is modified transactionally and committed after each transaction. In the new code, transactions are prepared, executed in a batch, and then the state is committed. Delaying the commit of state changes until all transactions are executed might lead to issues.
The charge_fee and validate are hardcoded as true in the transaction executor, this might cause the error. I changed them both to false, it doesn't help.

To make it work in my branch:

in vm/rust run make clean
here and here validate: false
in vm/rust run make all
make juno
./build/juno --http --network sepolia  and ran a test query
./build/juno --http --network sepolia --vm-concurrency-mode   and ran a test query
test query:

{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "starknet_estimateFee",
    "params": {
        "request": [
            {
            "type": "INVOKE",
            "sender_address": "0xf9e998b2853e6d01f3ae3c598c754c1b9a7bd398fec7657de022f3b778679",
            "calldata": [
                "0x1",
                "0x41a78e741e5af2fec34b695679bc6891742439f7afb8484ecd7766661ad02bf",
                "0x1987cbd17808b9a23693d4de7e246a443cfe37e6e7fbaeabd7d7e6532b07c3d",
                "0x4",
                "0x16342ade8a7cc8296920731bc34b5a6530f5ee1dc1bfd3cc83cb3f519d6530a",
                "0x77295b07f4b1d4e4fdb18b26faf95d5bdb0087da065b7b0b49d0470d9cbf852",
                "0x1",
                "0x0"
            ],
            "version": "0x100000000000000000000000000000001",
            "signature": [],
            "nonce": "0x2a91",
            "max_fee": "0x0"
            }
        ],
        "block_id": "pending",
        "simulation_flags": [
            "SKIP_VALIDATE"
        ]
    }
}

Pathfinder team encountered the same problem

It would be good to add a test for the concurrency mode if possible

I tried to measure performance by running multiple transactions and found a new bug.
The error is Transaction with no fee transfer info must have zero fee.
This happens here only in concurrency mode. According to blockifier, in both modes L1HandlerTransaction must have fee_transfer_call_info: None , i.e. problem in tx_execution_info.receipt.fee, it's not equal to 0.

Also curious what the performance of simulating a block in concurrency mode is as well.

I roughly measured the performance difference. I used a slice of different txs from sepolia block №84662 with length 89.
I measured the time of cairoVMExecute execution.
Results:

Original version: 100%
Concurrency support: sequential mode: ~99,59%
Concurrency support: concurrent mode: ~115,76%

I'm trying to get a profile of each version, but I get a weird result
I get ExternalCode only in concurrent mode


Add pagination to browse page

Currently, the browse page only shows 25 memes at a time, it would be better is there were a pagination system, so we can organize and show more memes

Hi @Dharmik48 I would like to contribute by working on the pagination system.
Can you please assign this issue to me?

Alright, I will assign this to you🚀

Closed with #10


Update Xcode 8.0 beta 6 (8S201h): Swift 3 UIColor & DispatchQueue

Update UIColor syntax
Update DispatchQueue

thank you for submitting this, but there was a more complete PR which I went with. Now it's all merged in master thanks again


Hardcoded values in the about page

Hi, i am very new to hugo so maybe i am missing something but if i use the About laytout i get hardocded values for instagra, mail and website.
https://github.com/binokochumolvarghese/lightbi-hugo/blob/master/layouts/about/about.html#L16-L33
is this expected ? is there a way to override those with my values ?
thanks

I just removed the hardcoded values from the page. Now you can use the the about page for the content and socail media links.
Hope this will solve your issue.

thank you :+1:


Added support for Swift Package Manager

Thanks for the PR! It'd be really nice to be able to support SPM. A few remarks/questions:

do we need to move the files to Sources? I do see other projects having a 'Source' folder or just sticking with their old naming. Do you know more about that?
I think the Podspec also needs to be changed. Can you do that?
I've merged another PR in the meanwhile so there are some conflicts. Can you try to rebase your branch?

Thanks again!

It's not necessarily needed to move the files into the Sources directory. Apple uses this layout in pretty much all their SPM documentation though, so I figured it would be best practice to do so. But if you don't like this, we can keep this the old way.

Thanks again for the PR. I've merged it in! (Sorry for the delay!)
I've also added you as a collaborator of this repo. Feel free to help maintaining the project. For bigger changes, we prefer a PR.
Thanks!


program loop with password and capcha

I received the normal amazon password and capcha screen while the program was running...I attempted to enter the password and capcha but puppeteer kept trying to enter the password screwing up what I was typing.  The console showed:
Rudi, you didn't win
giveaway 5 did not exist?
giveaway 6 did not exist?
Whoa, got to re-enter password!
giveaway 7 did not exist?
Whoa, got to re-enter password!
giveaway 8 did not exist?
Whoa, got to re-enter password!
giveaway 9 did not exist?
Whoa, got to re-enter password!

whoops, think I broke this in my latest release, sorry. Working on fix...

This program is really growing on me...won at 9:24pm and again at 10:02pm.  :D
I am noticing though the program sitting on a switch account screen and not doing anything....I've clicked on the right account and it continues right on...was worried it would time out if I waited any longer.  Might just be my setup but the double box doesn't get handled...then again I'm probably completing things slower as it waits on my net.


no way to not sync newly installed Extensions by default

If the Settings Sync is On (with Extensions sync there),
the newly installed extensions get synced by default, and there is no way to change it.
That is, the "Sync This Extension" tick is always on by default:

While it might be not desired behavior per se, it also creates additional problems as:

if Extension Pack is being installed (as on the screenshot), even if we disable the "Sync This Extension" tick before - all the extensions it includes will be synced upon the installation.
even if we manually disable every newly installed extension from syncing after, one by one - they might be already synchronized at the moment, and keep be installed remotely after we managed to disable their sync.

As a solution, there should be an option to keep the "Sync This Extension" tick off by default.

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: Version: 1.85.1
OS Version: Browser: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0

Steps to Reproduce:

make sure the Settings Sync is On
open Settings context menu of any new Extension, as on the screenshot
note the "Sync This Extension" tick is always set by default

@sandy081

/duplicate

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!

@sandy081 sorry, duplicate of what?

Can you please search in our issues to find it. Thanks.

@sandy081 I couldn't find anything like that.
Until it's found, could we reopen the issue, please?
I spent efforts to file it.

Not quite it, but similar:
https://github.com/microsoft/vscode/issues/197425


Python GEODATA

Description
A comprehensive description about the Pull Request
Reference Issue number #issue_no
issue_no must refer to the issue related to this PR, visit here
Type of change
Choosing one or more options from the following as per the nature of your Pull request.
NOTE: Check boxes using [x]

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Documentation Update

Checklist:

[x] I have named my files and folder, according to this project's guidelines.
[x] My code follows the style guidelines of this project.
[x] My Pull Request has a descriptive title. (not a vague title like Update index.md)
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have created a helpful and easy to understand README.md, with problem description and my name.
[x] I have included a requirements.txt file (if external libraries are required.)
[ ] My changes do not produce any warnings.
[x] I have starred this repository.
[x] I have added a working sample/screenshot of the script.
[x] I have checked for trailing spaces in file names and none have them.

This is a great repo @pratik-choudhari also thankyou for helping

@DishantIsrani I am not sure if I understand the contribution, it's significance and application. Firstlt, this problem is not in the issues so first please create an issue and I will assign you there. Secondly, Format your readme using the markdown syntax, just a text file makes it ambiguous.

@DishantIsrani Make the .txt file .md file

Done! Thanks @pratik-choudhari

@DishantIsrani You have said that you included requirements file but can't see one.

Updates the files! @pratik-choudhari


Won't install on HOOBS 3

I try to install this plugin by HOOBS 3-versie. 3.1.10
But it doesn't show up in installed plugins
I use a Raspberry 3b

before trying to install run: sudo journalctl -f -u hoobs from the terminal then try to install and you will see It gives error: "undefined ls-remote -h -t git://github.com/tillbaks/node-eiscp.git".
The error can be fixed by installing git:
sudo apt-get install git
After that install the plugin, and it will work. I just installed it 3 minutes ago runing latest version of HOOBS

This solves the problem, thanks


Feature/#55

对应的 Issue
Issue(#55)
修改内容
完成 query-normalizr 功能
发布计划（选填）
可随时发布
其他（选填）
提供 normalize 和 denormalize 方法，比较完善的单元测试，基本包含现在定义的所有情况，koa中间件没有测试，放到issue #56  中做集成测试。

Codecov Report

Merging #57 into master will increase coverage by 5.82%.
The diff coverage is 92.22%.

@@            Coverage Diff             @@
##           master      #57      +/-   ##
==========================================
+ Coverage   58.33%   64.16%   +5.82%
==========================================
  Files          30       30
  Lines         384      466      +82
  Branches      113      154      +41
==========================================
+ Hits          224      299      +75
- Misses        127      134       +7
  Partials       33       33

Impacted Files
Coverage Δ

packages/query-normalizr/src/lib/denormalize.js
91.3% <91.3%> (ø)

packages/query-normalizr/src/lib/normalize.js
93.18% <93.18%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a9b35f5...5fd77fa. Read the comment docs.


[SPARK-33457][PYTHON] Adjust mypy configuration

What changes were proposed in this pull request?

This pull request:

Adds following flags to the main mypy configuration:

strict_optional
no_implicit_optional
disallow_untyped_defs

These flags are enabled only for public API and disabled for tests and internal modules.
Additionally, these PR fixes missing annotations.
Why are the changes needed?

Primary reason to propose this changes is to use standard configuration as used by typeshed project. This will allow us to be more strict, especially when interacting with JVM code. See for example https://github.com/apache/spark/pull/29122#pullrequestreview-513112882
Additionally, it will allow us to detect cases where annotations have unintentionally omitted.
Does this PR introduce any user-facing change?

Annotations only.
How was this patch tested?

dev/lint-python.

FYI @Fokko  @HyukjinKwon
Still work in progress, any suggestions about desired flags (I considered show_error_codes, for clarity, and disallow_any_generics).

Test build #131122 has started for PR 30382 at commit 4de37b2.

My only concern is the maintenance cost for being strict. If it doesn't add non-trivial overhead, I am good to be more strict on the type hints.

My only concern is the maintenance cost for being strict. If it doesn't add non-trivial overhead, I am good to be more strict on the type hints.

For the first two options - we are already covered, as these were used in pyspark-stubs.
The third one required some fixes, but these are mostly things that should be there. In all other cases, we can either ignore the whole module (as with tests and such) or specific lines (for example I considered ignoring callJavaFunc as these annotations are hardly useful and  functions are not exactly public).
So overall, I don't think there's much complexity added and using these, gives us more confidence that annotations are in a good shape.

Test build #131122 has finished for PR 30382 at commit 4de37b2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131122/
Test PASSed.

Test build #131124 has started for PR 30382 at commit c42fe59.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35725/

Test build #131124 has finished for PR 30382 at commit c42fe59.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131124/
Test PASSed.

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35725/

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35725/
Test FAILed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35727/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35727/

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35727/
Test FAILed.

Test build #131515 has started for PR 30382 at commit 426c7bb.

Test build #131515 has finished for PR 30382 at commit 426c7bb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131515/
Test PASSed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36119/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36119/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36119/
Test PASSed.

@zero323, let me know when you think this is ready. Looks fine to me.

Test build #131547 has started for PR 30382 at commit 9182a98.

Test build #131547 has finished for PR 30382 at commit 9182a98.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class PipelineReader(MLReader[Pipeline]):
class PipelineModelReader(MLReader[PipelineModel]):

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131547/
Test PASSed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36150/

@zero323, let me know when you think this is ready. Looks fine to me.

Thanks @HyukjinKwon. I went over all violations of disallow_any_generics and these are mostly intended (little or no additional value gained by addressing these). So I think it ready to review.

Looks good!

Thanks for reviewing @Fokko!

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36150/

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36150/
Test FAILed.

Test build #131599 has started for PR 30382 at commit 9182a98.

Test build #131599 has finished for PR 30382 at commit 9182a98.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class PipelineReader(MLReader[Pipeline]):
class PipelineModelReader(MLReader[PipelineModel]):

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131599/

retest this please

Test build #131614 has started for PR 30382 at commit 9182a98.

Test build #131614 has finished for PR 30382 at commit 9182a98.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class PipelineReader(MLReader[Pipeline]):
class PipelineModelReader(MLReader[PipelineModel]):

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131614/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36200/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36214/

Merged to master.

Thanks @Fokko  and @HyukjinKwon!


Dev lisa

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 37.11340% with 61 lines in your changes missing coverage. Please review.

Please upload report for BASE (dev@7808536). Learn more about missing BASE report.

Files with missing lines
Patch %
Lines

src/pyinla/core/inla.py
13.79%
50 Missing :warning:

src/pyinla/models/regression.py
42.85%
4 Missing :warning:

src/pyinla/prior_hyperparameters/gaussian.py
0.00%
4 Missing :warning:

src/pyinla/likelihoods/binomial.py
80.00%
1 Missing :warning:

src/pyinla/likelihoods/gaussian.py
66.66%
1 Missing :warning:

src/pyinla/models/spatio_temporal.py
66.66%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@          Coverage Diff           @@
##             dev      #21   +/-   ##
======================================
  Coverage       ?   50.16%
======================================
  Files          ?       25
  Lines          ?      620
  Branches       ?        0
======================================
  Hits           ?      311
  Misses         ?      309
  Partials       ?        0

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


TC-Gen: Create a 0.1 degree lat/lon data file for hurricane basins.

This task was split off from the original TC-Gen development issue:
https://github.com/NCAR/MET/issues/1127
TC-Genesis forecasts include a lat/lon but do not specify the basin to which that lat/lon belongs.  Therefore, filtering by basin (e.g. basin = [ "AL" ];) is not useful.  Instead, we need to filter by masking region.  However, we should really provide a standard definition of the basins by creating a global NetCDF 0.1 degree lat/lon file.  Store an integer for each grid point that is a flag value defining the hurricane basin to which that belongs.

See attached commands for generating these regions:
gen_rsmc_basins.txt
Resulting RSMC regions:

Resulting BASIN regions:

Still need to combine them into a single NetCDF and add metadata.

References for RMSC and BASIN definitions from Jonathan Vigh:
Below, please find the relevant information and source info (in a format that you can stick in the documentation or copy into the GitHub issue), followed by answers to your specific questions.
Key for information provided: WMO region (responsible organization - basins), location of document where Area of Responsibility information can be found (followed by the link to the document and any relevant excerpts of graphics).
Region I (RSMC La Reunion - South-West Indian Ocean), page 1 of ANNEX:
https://www.wmo.int/pages/prog/www/tcp/documents/OperationalPlan_TCP-12_WMO-1178_2016_En.pdf
"The area of responsibility of the RSMC includes the tropical and subtropical areas of the South-West Indian Ocean from the Equator to 40°S and west of 90°E to Africa (therefore including the Mozambique Channel)."
Region II (RSMC New Delhi - Bay of Bengal and Arabian Sea), page II-9 of:
https://www.wmo.int/pages/prog/www/tcp/documents/PTC-OperationalPlan_TCP-21_Edition2018.pdf
"The area of responsibility for the issue of tropical cyclone advisories by RSMC Tropical Cyclones, New Delhi cover sea areas of north Indian Ocean between long. 45° E to 100° E."
Region III (RSMC Tokyo - Western Pacific): I could not find any basin definition information in their document, but I think 100° E to 180° E should suffice.
https://www.wmo.int/pages/prog/www/tcp/documents/TCP-23EDITION2019.pdf
Region IV (RSMC Miami - Atlantic / Eastern Pacific), page 2-1 of:
https://www.wmo.int/pages/prog/www/tcp/documents/RA-IV-HC-41-OperationalPlan_en_final.pdf
"The area of responsibility of RSMC Miami for issuing tropical and subtropical cyclone advisories is the North Atlantic Ocean, the Caribbean Sea, Gulf of Mexico, North Pacific Ocean eastward from 140ºW."
Region V: (RSMC Nadi and a variety of Tropical Cyclone Warning Centers (TCWCs), as listed below - Southwest Pacic and South-West Indian Ocean), page II-2 of:
https://www.wmo.int/pages/prog/www/tcp/documents/RAV_OpPlan_TCP24_2018_Final.pdf

Description of sub-regions.

In summary, here are the answers to your specific questions:

Is the boundary between La Reunion and Australia really LON=90E? (Looks more like 88)  Yes, LON=90E.
Is the southwest latitude of Australia really LAT=38S?   No, LAT=36S
Is the southeast latitude of Australia really LAT=35S?    No, LAT=32S
Is the boundary between Nadi and Wellington really LON = 25S?   Yes

Final version of this file was committed to the develop branch and unit_tc_gen.xml uses this data to filter genesis by basin.


Click to make stay until unclicked -- how to accomplish, example from my code that didn't work provided

I wanted to do a "click to stay" method but couldn't figure out how to do it, even using something with a javascript global.  Example:
SCRIPT type="text/javascript"
var tipster1_stay=false;
/SCRIPT
.... then in ready functiuon:
$("#tipster1").tooltipster({animation:"fade",maxWidth:"90%",content:$('my cool html content')});
$("#tipster1").click(function(e){
if ( tipster1_stay ) {
$("#tipster1").tooltipster("option","autoClose","true");
tipster1_stay=false;
} else {
$("#tipster1").tooltipster("option","autoClose","false");
tipster1_stay=true;
}
});

Hello, please head over to stackoverflow or another help forum for implementation help. Thank you.

wow the warm feeling

I wish I had more time to offer to all users world wide, unfortunately I'm very busy with my work. I can only provide bug support. All tools to accomplish what you want are provided in Tooltipster, I hope you can eventually get what you want. Thank you.


[CHK-3513] feat: handle psp 404 error code

List of Changes

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

please a quick description, and maybe a screenshot of the new error message might be helpful


Add control on delay and interval on "On XX Button" block

Is your feature request related to a problem? Please describe.
When using the block to control key presses, there is an option called "repeat".
Today, there is a delay before the action inside the block is run which in my case causes an unwanted latency in command.
Describe the solution you'd like
I'd like to have advanced settings (with a + sign) that allows me to set delay and interval.
Describe alternatives you've considered
There is already repeatDelay and repeatInterval when using Javascript.
But I need to be able to do this with blocks for younger kids.
Additional context
I use the "On XX Button" to control very simple platform game and a delay is not something very nice for the gameplay.

Is there a workaround with blocks?
It's really weird to have a player that does not move as soon you hit a key.

@titimoby can you share your program? I don't understand this issue.

First is the block version. When you press left or right, there is a delay before the sprite moves.
Block version
Second is the typescript version where repeatDelay is set to 0 because the API exists.
The sprite moves as soon as you press right or left arrow key.
Typescript version


Add API and fix filter list bug

Fix the potential bug caused by the missing filter.
Add gas price API

Codecov Report

Merging #470 (9523d56) into develop (9eb8f8c) will increase coverage by 9.95%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #470      +/-   ##
===========================================
+ Coverage    76.09%   86.05%   +9.95%
===========================================
  Files           81       49      -32
  Lines         3037     1936    -1101
  Branches       469      307     -162
===========================================
- Hits          2311     1666     -645
+ Misses         726      270     -456

Flag
Coverage Δ

batch-submitter
?

contracts
86.05% <ø> (ø)

core-utils
?

data-transport-layer
?

message-relayer
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ckages/data-transport-layer/src/utils/constants.ts

...kages/batch-submitter/src/batch-submitter/index.ts

.../l1-ingestion/handlers/sequencer-batch-appended.ts

packages/core-utils/src/fees.ts

...ckages/data-transport-layer/src/utils/contracts.ts

...ackages/batch-submitter/src/utils/tx-submission.ts

.../src/services/l2-ingestion/handlers/transaction.ts

packages/message-relayer/src/relay-tx.ts

...itter/src/batch-submitter/state-batch-submitter.ts

...h-submitter/src/batch-submitter/batch-submitter.ts

... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9eb8f8c...9523d56. Read the comment docs.


home-assistant: 0.72.1 -> 0.73.0

Motivation for this change
https://www.home-assistant.io/blog/2018/07/06/release-73/
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

cc @peterhoeg

Success on aarch64-linux (full log)
Attempted: home-assistant
Partial log (click to expand)
tests/components/test_init.py ................                           [ 71%]
tests/components/test_introduction.py .                                  [ 71%]
tests/components/test_logger.py ......                                   [ 74%]
tests/components/test_script.py .....                                    [ 77%]
tests/components/test_shell_command.py ........                          [ 81%]
tests/components/test_system_log.py ................                     [ 90%]
tests/components/test_websocket_api.py ..................                [100%]

========================= 187 passed in 23.18 seconds ==========================
/nix/store/xphfnza6xzz5pjcmgm7pm90z3dzcbm2d-homeassistant-0.73.0

Success on x86_64-linux (full log)
Attempted: home-assistant
Partial log (click to expand)
tests/components/test_init.py ................                           [ 71%]
tests/components/test_introduction.py .                                  [ 71%]
tests/components/test_logger.py ......                                   [ 74%]
tests/components/test_script.py .....                                    [ 77%]
tests/components/test_shell_command.py ........                          [ 81%]
tests/components/test_system_log.py ................                     [ 90%]
tests/components/test_websocket_api.py ..................                [100%]

========================== 187 passed in 5.91 seconds ==========================
/nix/store/gs1ca7j3mhnfr6idra72zvxi73wcshlq-homeassistant-0.73.0


[Bug] Pull Request reviewer duplicates

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue
Currently the reviewers column in github__pull_requests does not account for duplicate/removed reviewers. E.g. it just joins all reviewers based on the the string aggregated users from the stg_github__pull_request_review model.
This aggregation seems to be inherited upstream through the int_github__pull_request_reviewers model and therefore, this model should be updated to be like so instead. I don't actually think in the example branch that we even need to leverage pull_request_review anymore but should just be able to just use pull_request_reviewer_history .
Relevant error log or model output
This is the sample of the data as it currently stands:

 split_reviewers as (

  select
    reviewers,
    split_reviewers,
    count(distinct issue_id)
  FROM `dbt-package-testing.fivetran_transformations_github.github__pull_requests` ,
  unnest(split(reviewers, ', ')) split_reviewers
  group by 1,2
  having count(distinct issue_id) > 1
)

select * from split_reviewers

Some rows literally look like this:
```fivetran-joemarkiewicz, fivetran-joemarkiewicz, fivetran-joemarkiewicz```

Presumably because the `pull_request_review` seems like an event based table so aggregating all the users there will yield duplicates.

Expected behavior
The aggregated reviewers column should be de-duplicated with only one instance per unique reviewer.
So instead of:
fivetran-joemarkiewicz, fivetran-joemarkiewicz, fivetran-joemarkiewicz
It would be
fivetran-joemarkiewicz
dbt Project configurations
n/a
Package versions
n/a
What database are you using dbt with?
bigquery
dbt Version
1.3
Additional Context
No response
Are you willing to open a PR to help address this issue?

[X] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[ ] No.

Hi @fivetran-sheringuyen could you expand on the suggestion to use  pull_request_reviewer_history instead of/in addition to pull_request_review? I created a change in this PR that fixes the duplicate issue, but I'm suspecting I'm missing a bigger piece of the puzzle.


Support ROS2 build

@koide3
I ported ndt_omp to ROS2 and set up a workflow for CI. https://github.com/tier4/ndt_omp/pull/5
I would like to create a pull request to this repository, but before that I would like to discuss how to manage branches.
In our setup, we support both ROS1 and ROS2 builds. Therefore, we can support both in the master branch.
However, some repositories (e.g. https://github.com/ros/diagnostics) have separate branches like master and ros2-devel. This simplifies the release process. Which pattern is better for you to manage?

@koide3 Hi, I created a PR to be able to build both ROS1 and ROS2 on the master branch.
If you want to change the branch configuration, please leave a comment. I will recreate the PR.

Thanks a lot for your contribution. I prefer supporting both ROS1 and ROS2 on the master branch, and your PR looks fine. I'll review and merge the PR later.


#149034 Select all when shift click in outline

This pull request adds the feature requested with #149034.

Two instances of let need changing to const in order to fix Hygiene & Layering errors.

Done, applied changes


update node driver registrar to 2.9.2

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/retest

/retest

/approve


Download and use vault enterprise for testing

It runs for 30 min without a license. We can use it to test the few enterprise vault integrations we have.
@mp911de set this up for spring vault so we should collaborate.

Ideally, copy scripts and the OpenSSL conf from spring-vault (except k8s scripts) and then launch start.sh which will generate certificates, download, and launch Vault.
Alternatively, launch each script individually.
install_vault.sh can be customized either via command line args or environment variables (VAULT_VER, EDITION, see the matrix build for examples).


Fix and align analyzers

Hey @cmgrd I have created the issue as requested, please update with information if necessary. :)


Extension causes high cpu load

Issue Type: Performance
Extension Name: vscode-database
Extension Version: 2.0.3
OS Version: Windows_NT x64 10.0.17134
VSCode version: 1.32.3

:warning: Make sure to attach this file from your home-directory: C:\Users\Emamuzo\bajdzis.vscode-database-unresponsive.cpuprofile.txt :warning:
Find more details here: https://github.com/Microsoft/vscode/wiki/Explain:-extension-causes-high-cpu-load

How many connections do you have saved in database.connections in vscode settings?


image-builder release-channel support for 1-25

image-builder utility works perfectly fine for building ubuntu ovas for k8s version 1.23 using release channel 1-23.
However, when I am trying to create the ova for 1.25 version via image-builder utility using 1-25 as the release channel., I get an error.
Command to create (same works perfectly with 1-23 release channel and am able to create the ova for 1.23)
image-builder build --os ubuntu --hypervisor vsphere --release-channel 1-25 --vsphere-config $HOME/vsphere-connection.json

Error
Creating builder config
2023/08/04 10:32:01 release-channel should be one of 1-20, 1-21, 1-22, 1-23

Documentation says that 1.25 version ovas are supported via 1-25 release channel. That does not seem to be true.
EKS Anywhere version
ubuntu@eksa-admin-machine:~$ eksctl anywhere version
v0.16.2

Image-builder version
image-builder@eksa-admin-machine:~$ image-builder version
0.1.0

Can you kindly guide.
thanks,
Ambar.

Ok, i got the issue. I had to upgrade the image-builder utility itself to accomodate the newer release channel of 1-25
After upgrading the image-builder from 0.1.0 to 0.1.2
image-builder@eksa-admin-machine:~$ image-builder version
0.1.2

The template is successfully generated.
make: Leaving directory '/home/image-builder/eks-anywhere-build-tooling/projects/kubernetes-sigs/image-builder'
2023/08/04 12:32:07 Image Build Successful
 Please find the output artifact at /home/image-builder/ubuntu.ova
2023/08/04 12:32:07 Moving artifacts from build directory to current working directory
2023/08/04 12:32:07 Cleaning up cache build files
2023/08/04 12:32:07 Build Successful. Output artifacts located at current working directory
[04-08-23 12:32:08] Uploading ubuntu-2004-kube-v1.25.11.ovf... OK
[04-08-23 12:33:00] Uploading ubuntu-2004-kube-v1.25.11-1.vmdk... OK


lnmp环境配置完成后访问，页面一片空白···

server
{
listen 80;
#listen [::]:80;
server_name v3.xxxx.com;
index index.html index.htm index.php default.html default.htm default.php;
root  /home1/ss-panel/public;
    include none.conf;
    #error_page   404   /404.html;

    location / {
               try_files $uri $uri/ /index.php$is_args$args;
               }

    location ~ [^/]\.php(/|$)
    {
        # comment try_files $uri =404; to enable pathinfo
        try_files $uri =404;
        fastcgi_pass  unix:/tmp/php-cgi.sock;
        fastcgi_index index.php;
        include fastcgi.conf;
        #include pathinfo.conf;
    }

    location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$
    {
        expires      30d;
    }

    location ~ .*\.(js|css)?$
    {
        expires      12h;
    }

    access_log  /home/wwwlogs/v3.xxxx.com.log  access;
}

看下nginx和php的log

2016/01/31 10:51:12 [error] 21512#0: *3 FastCGI sent in stderr: "PHP message: PHP Warning:  require(): open_basedir restriction in effect. File(/home1/ss-panel/bootstrap.php) is not within the allowed path(s): (/home1/ss-panel/public:/tmp/:/proc/) in /home1/ss-panel/public/index.php on line 7
PHP message: PHP Warning:  require(/home1/ss-panel/bootstrap.php): failed to open stream: Operation not permitted in /home1/ss-panel/public/index.php on line 7
PHP message: PHP Fatal error:  require(): Failed opening required '/home1/ss-panel/public/../bootstrap.php' (include_path='.:/usr/local/php/lib/php') in /home1/ss-panel/public/index.php on line 7" while reading response header from upstream, client: <IP_ADDRESS>, server: v3.1122.lol, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/tmp/php-cgi.sock:", host: "v3.1122.lol"

location ~ [^/].php(/|$)这里添加下
fastcgi_param  PHP_VALUE  "open_basedir=$document_root:/tmp/:/proc/:/home1/ss-panel/";

OK了，多谢·


Can't replace non-word characters

Minimal reproduction: https://stackblitz.com/edit/typescript-unaxav?file=replace.ts
It appears that the replace function isn't triggered, so the problem is not in the data field.

In my use case, it happens because Rules aren't taken into account.
Adding:
case 'Rule':
  node.selector = node.selector.replace(regex, replacement);
  break;

fixes it.
@mzdr, @dschenk : If this is outside the scope of the package, I'll create a fork, otherwise I'll be happy to provide a PR with additional test cases.

Hi @doplumi
I think it makes sense to include non-word characters as well for replacement. If you can provide a PR I will merge it.
Thanks for opening this issue and providing insights and a possible solution. Much appreciated!