Grid

Hi, I would like to have progressview with span size of 1 when my grid layout has 3 column. I attempted to use this code but it is not impossible. Please help me to resolve it.
gridLayoutManager.setSpanSizeLookup(new GridLayoutManager.SpanSizeLookup() {
            @Override
            public int getSpanSize(int position) {
                return adapter.isHeader(position) ? gridLayoutManager.getSpanCount() : 1;
            }
 });

and
gridLayoutManager.setSpanSizeLookup(new GridLayoutManager.SpanSizeLookup() {
            @Override
            public int getSpanSize(int position) {
 switch(adapter.getItemViewType(position)){
                    case adapter.TYPE_HEADER:
                        return 1;
                    case adapter.TYPE_ITEM:
                        return gridLayoutManager.getSpanCount(); //number of columns of the grid
                    default:
                        return 1;
 }
  }
 });

Thanks and Best Regards

Hi!
Thank you for reporting this.
I'm not sure that this code must be in the library because implementations of GridLayoutManager.SpanSizeLookup may be complex. So it's up to a client to resolve span's size.
Here is an example of how you can do it:
final EndlessRecyclerView list = ...
final GridLayoutManager manager = new GridLayoutManager(v.getContext(), 3);

// we want progress view to fill entire row
manager.setSpanSizeLookup(new GridLayoutManager.SpanSizeLookup() {
    @Override
    public int getSpanSize(int position) {
        return list.isRefreshing() && position == manager.getItemCount() - 1 ?
                manager.getSpanCount() : 1;
    }
});

hi, thanks for answer but the list does not have isRefreshing() . ?!

hi again, so sorry, it is not working and also new version in grid mode when scrolling down 3 times, it comes back to first row, scroll up. I think it is a bug however in version 1.0 the scrolling is work nice.
At the end, this code not working for span size of loading.

it is not working and also new version in grid mode when scrolling down 3 times, it comes back to first row, scroll up

I'm going to need more details on this issue. Have you tried to run sample application on your device? Does it work the same? What device do you use? What Android version does it run?
Could you also give a snippet of how you use this view?
I'm guessing that this could happen if you replace adapter when data is being refreshed.

At the end, this code not working for span size of loading.

Yet again, on my device the sample app that comes with the library works fine:

in my adapter i have also extra header.
 @Override
    public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
        ctx = parent.getContext();
        if (viewType == TYPE_HEADER) {
            View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.header_recyclerview_new_channels, parent, false);
            return new ChannelSliderHeaderViewHolder(view);
        } else if (viewType == TYPE_ITEM) {
            View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.subscribe_layout, parent, false);
            return new ChannelViewHolder(view);
        }
        throw new RuntimeException("there is no type that matches the type " + viewType + " + make sure your using types correctly");
    }

and also
@Override
    public int getItemViewType(int position) {
        if (isPositionHeader(position))
            return TYPE_HEADER;
        return TYPE_ITEM;
    }

    private boolean isPositionHeader(int position) {
        return position == 0;
    }

    public boolean isHeader(int position){
        if(position==0){
            return true;
        } else {
            return false;
        }
    }

Please review and help me. thanks

I don't see any logical problems here. The header itself can not be a problem.

hi again
I use this code and the result is a video that you can see.
http://www.myscreen.my/img/device-2016-03-16-channels.mp4
 return list.isRefreshing() && position == gridLayoutManager.getItemCount() - 1 ?
                        gridLayoutManager.getSpanCount() : 1;

Ok. I see two problems here.
First, method gridLayoutManager.getItemCount() is going to return you number of items in your adapter. That means you don't need to subtract 1 to get position of a progress view because it will be equals to a number of items. I had the same mistake when I posted my first comment (I edited it later). This will give you proper span size.
Secon, you need to specify progress view height explicitly in your layout file. This way it won't fill entire screen during scrolling.
However it's still a mistery to me why your list scrolls up.

it works but your new version after loading it scrolls up in grid and in list the last 2 or 3 rows disappear. ;( please help

How do you update the list after loading?

add data to adapter then notifydatachanged then
                        handler.postDelayed(new Runnable() {
                            @Override
                            public void run() {
                                list.setRefreshing(false);
                                loading = false;
                            }
                        }, DELAY);

This doesn't give me much. In order to solve this problem I'm going to need to see more code.


Force running all system pods on the control plane nodes

Hi
I have a running Kubernetes cluster in AWS, with

3 etcd nodes
2 control plane nodes
3 worker nodes distributed in 3 worker pools (one pool in each AZ, with one node in each pool).

When I'm testing auto scaling of the worker node pools I see that some system critical pods are running on the worker nodes. Once in a while when a worker node is terminated by AWS, the critical pods are then terminated and redeployed to any running worked node. Not a big problem, but for example heapster statistics will not be available for the short period of time it takes Kubernetes to restart the pod on a running node.
Any reason in particular that these pods are not run on the control plane nodes? And can I force them to run on the control plane nodes by modifying the userdata-controller-file before running kube-aws up?

heapster
kube-dns
kube-dns-autoscaler
kube-rescheduler
kubernetes-dashboard

NAME                                                                READY     STATUS    RESTARTS   AGE       IP            NODE
heapster-v1.3.0-76786035-9qq4g                                      2/2       Running   0          14m       <IP_ADDRESS>   ip-10-1-44-196.eu-west-1.compute.internal
kube-apiserver-ip-10-1-43-150.eu-west-1.compute.internal            1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-43-150.eu-west-1.compute.internal
kube-apiserver-ip-10-1-44-191.eu-west-1.compute.internal            1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-44-191.eu-west-1.compute.internal
kube-controller-manager-ip-10-1-43-150.eu-west-1.compute.internal   1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-43-150.eu-west-1.compute.internal
kube-controller-manager-ip-10-1-44-191.eu-west-1.compute.internal   1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-44-191.eu-west-1.compute.internal
kube-dns-3816048056-5tpmj                                           4/4       Running   0          1h        <IP_ADDRESS>   ip-10-1-43-42.eu-west-1.compute.internal
kube-dns-3816048056-bw11s                                           4/4       Running   0          2h        <IP_ADDRESS>   ip-10-1-45-239.eu-west-1.compute.internal
kube-dns-autoscaler-1464605019-k3s5k                                1/1       Running   0          14m       <IP_ADDRESS>   ip-10-1-44-196.eu-west-1.compute.internal
kube-proxy-ip-10-1-43-150.eu-west-1.compute.internal                1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-43-150.eu-west-1.compute.internal
kube-proxy-ip-10-1-43-42.eu-west-1.compute.internal                 1/1       Running   0          1h        <IP_ADDRESS>    ip-10-1-43-42.eu-west-1.compute.internal
kube-proxy-ip-10-1-44-191.eu-west-1.compute.internal                1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-44-191.eu-west-1.compute.internal
kube-proxy-ip-10-1-44-196.eu-west-1.compute.internal                1/1       Running   0          24m       <IP_ADDRESS>   ip-10-1-44-196.eu-west-1.compute.internal
kube-proxy-ip-10-1-44-236.eu-west-1.compute.internal                1/1       Running   0          6m        <IP_ADDRESS>   ip-10-1-44-236.eu-west-1.compute.internal
kube-proxy-ip-10-1-45-239.eu-west-1.compute.internal                1/1       Running   0          2h        <IP_ADDRESS>   ip-10-1-45-239.eu-west-1.compute.internal
kube-rescheduler-3155147949-0rm2p                                   1/1       Running   0          1h        <IP_ADDRESS>    ip-10-1-43-42.eu-west-1.compute.internal
kube-scheduler-ip-10-1-43-150.eu-west-1.compute.internal            1/1       Running   0          12h       <IP_ADDRESS>   ip-10-1-43-150.eu-west-1.compute.internal
kube-scheduler-ip-10-1-44-191.eu-west-1.compute.internal            1/1       Running   1          12h       <IP_ADDRESS>   ip-10-1-44-191.eu-west-1.compute.internal
kubernetes-dashboard-v1.5.1-lpnbb                                   1/1       Running   0          1h        <IP_ADDRESS>   ip-10-1-43-42.eu-west-1.compute.internal

@paalkr Hi, thanks for trying kube-aws!
AFAIK, these pods are scheduled to worker nodes by default in Kubernetes.
I partly agree to you though - actually, I'm running tiller on controller nodes.
However, as of today, I personally believe that running pods like kube-dns on controller nodes wouldn't be a good idea.
Controller nodes can't be auto-scaled easily due to the --apiserver-count param required by apiserver. If you've scaled out your worker nodes considerably, kube-dns auto-scaled by kube-dns-autoscaler would easily outgrow controller nodes. #499 is a related issue about the --apiserver-count param.

Also, you can definitely run any "system" pods on controller nodes by adding appropriate tolerations to tolerate taints associated only to controller nodes.
In k8s 1.6, the toleration look like:
https://github.com/kubernetes-incubator/kube-aws/blob/master/core/controlplane/config/templates/cloud-config-controller#L713-L717
In k8s 1.5, it is in the annotations field instead.

Hi
Thanks for the feedback. I see that scaling the dns service beyond the number of controller plane nodes can be necessary when scaling out a cluster to a certain level. My main concern is heapster, dashboard and reschduler that by default only runs in one pod and ar not scaled horizontally.
I will try to modify the controller user data adding the toleration for heapster and see how that work. I guess this will do the trick?
  - path: /srv/kubernetes/manifests/heapster-de.yaml
    content: |
        apiVersion: extensions/v1beta1
        kind: Deployment
        metadata:
          name: heapster-v1.3.0
          namespace: kube-system
          labels:
            k8s-app: heapster
            kubernetes.io/cluster-service: "true"
            version: v1.3.0
        spec:
          replicas: 1
          selector:
            matchLabels:
              k8s-app: heapster
              version: v1.3.0
          template:
            metadata:
              labels:
                k8s-app: heapster
                version: v1.3.0
              annotations:
                scheduler.alpha.kubernetes.io/critical-pod: ''
            spec:
              tolerations:
              - key: "CriticalAddonsOnly"
                operator: "Exists"
              - key: "node.alpha.kubernetes.io/role"
                operator: "Equal"
                value: "master"
                effect: "NoSchedule"
              containers:
                - image: gcr.io/google_containers/heapster:v1.3.0
                  name: heapster
                  livenessProbe:
                    httpGet:
                      path: /healthz
                      port: 8082
                      scheme: HTTP
                    initialDelaySeconds: 180
                    timeoutSeconds: 5
                  resources:
                    limits:
                      cpu: 80m
                      memory: 200Mi
                    requests:
                      cpu: 80m
                      memory: 200Mi
                  command:
                    - /heapster
                    - --source=kubernetes.summary_api:''
                - image: gcr.io/google_containers/addon-resizer:1.6
                  name: heapster-nanny
                  resources:
                    limits:
                      cpu: 50m
                      memory: 90Mi
                    requests:
                      cpu: 50m
                      memory: 90Mi
                  env:
                    - name: MY_POD_NAME
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.name
                    - name: MY_POD_NAMESPACE
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.namespace
                  command:
                    - /pod_nanny
                    - --cpu=80m
                    - --extra-cpu=4m
                    - --memory=200Mi
                    - --extra-memory=4Mi
                    - --threshold=5
                    - --deployment=heapster-v1.3.0
                    - --container=heapster
                    - --poll-period=300000
                    - --estimator=exponential

@paalkr LGTM 👍
Let me also add that, you should ensure system pods scheduled to conroller nodes to have:

Pod anti-affinities, not to schedule 2 equivalent pods to the same node
Have 2 or more replicas if it is designed to work like that

So that a rolling-update of controller nodes won't take down your system services.

Thanks, good point. I'll add anti-afinity to the config as well.
Will heapster, dashboard and rescheduler run nicely when scaled to two pods each, or will they make trouble for each other?

When I originally added the rescheduler it didn't play nice with more than one. That may be fixed now we unblocked port 443 between controllers. I can align it to the scheduler as part of the existing rescheduler issue.

Thanks
@mumoshu , even without anti-affinity and two replicas a rolling update for the controller nodes wont be any worse to the system than the current situation? Like if the worker node that happens runs heapster, rescheduler or dashboard is terminated by AWS. Kubernates will just move the pods to the controller node not being updated, right?
Will this anit-affinity and scaling setting work for heapster?
  - path: /srv/kubernetes/manifests/heapster-de.yaml
    content: |
        apiVersion: extensions/v1beta1
        kind: Deployment
        metadata:
          name: heapster-v1.3.0
          namespace: kube-system
          labels:
            k8s-app: heapster
            kubernetes.io/cluster-service: "true"
            version: v1.3.0
        spec:
          replicas: 2
          selector:
            matchLabels:
              k8s-app: heapster
              version: v1.3.0
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                  - key: k8s-app
                    operator: In
                    values:
                    - heapster
                topologyKey: kubernetes.io/hostname
          template:
            metadata:
              labels:
                k8s-app: heapster
                version: v1.3.0
              annotations:
                scheduler.alpha.kubernetes.io/critical-pod: ''
            spec:
              tolerations:
              - key: "CriticalAddonsOnly"
                operator: "Exists"
              - key: "node.alpha.kubernetes.io/role"
                operator: "Equal"
                value: "master"
                effect: "NoSchedule"
              containers:
                - image: gcr.io/google_containers/heapster:v1.3.0
                  name: heapster
                  livenessProbe:
                    httpGet:
                      path: /healthz
                      port: 8082
                      scheme: HTTP
                    initialDelaySeconds: 180
                    timeoutSeconds: 5
                  resources:
                    limits:
                      cpu: 80m
                      memory: 200Mi
                    requests:
                      cpu: 80m
                      memory: 200Mi
                  command:
                    - /heapster
                    - --source=kubernetes.summary_api:''
                - image: gcr.io/google_containers/addon-resizer:1.6
                  name: heapster-nanny
                  resources:
                    limits:
                      cpu: 50m
                      memory: 90Mi
                    requests:
                      cpu: 50m
                      memory: 90Mi
                  env:
                    - name: MY_POD_NAME
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.name
                    - name: MY_POD_NAMESPACE
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.namespace
                  command:
                    - /pod_nanny
                    - --cpu=80m
                    - --extra-cpu=4m
                    - --memory=200Mi
                    - --extra-memory=4Mi
                    - --threshold=5
                    - --deployment=heapster-v1.3.0
                    - --container=heapster
                    - --poll-period=300000
                    - --estimator=exponential

@paalkr Sorry my explanation was'nt complete.

If you'd like to endure just rolling updates of nodes, setting maxUnavailable to zero would be enough
If you'd like to endure both rolling updates and sudden death of nodes, at least having 2 or more replicas, and optionally setting maxUnavailable to zero would be enough

I see. I tried to just do an in place update of the heapster deployment by
kubectl replace -n kube-systen -f heapster-deployment.yaml
But I did get an error message. So I must have something wring in my deployment definition.
error validating data: found invalid field affinity for v1beta1.DeploymentSpec; if you choose to ignore these errors, turn validation off with --validate=false
versions
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.2", GitCommit:"477efc3cbe6a7effca06bd1452fa356e2201e1ee", GitTreeState:"clean", BuildDate:"2017-04-19T20:33:11Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.1+coreos.0", GitCommit:"9212f77ed8c169a0afa02e58dce87913c6387b3e", GitTreeState:"clean", BuildDate:"2017-04-04T00:32:53Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd6

content of heapster-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: heapster-v1.3.0
  namespace: kube-system
  labels:
    k8s-app: heapster
    kubernetes.io/cluster-service: "true"
    version: v1.3.0
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: heapster
      version: v1.3.0
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: k8s-app
            operator: In
            values:
            - heapster
        topologyKey: kubernetes.io/hostname
  template:
    metadata:
      labels:
        k8s-app: heapster
        version: v1.3.0
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      tolerations:
      - key: "CriticalAddonsOnly"
        operator: "Exists"
      - key: "node.alpha.kubernetes.io/role"
        operator: "Equal"
        value: "master"
        effect: "NoSchedule"
      containers:
        - image: gcr.io/google_containers/heapster:v1.3.0
          name: heapster
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8082
              scheme: HTTP
            initialDelaySeconds: 180
            timeoutSeconds: 5
          resources:
            limits:
              cpu: 80m
              memory: 200Mi
            requests:
              cpu: 80m
              memory: 200Mi
          command:
            - /heapster
            - --source=kubernetes.summary_api:''
        - image: gcr.io/google_containers/addon-resizer:1.6
          name: heapster-nanny
          resources:
            limits:
              cpu: 50m
              memory: 90Mi
            requests:
              cpu: 50m
              memory: 90Mi
          env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          command:
            - /pod_nanny
            - --cpu=80m
            - --extra-cpu=4m
            - --memory=200Mi
            - --extra-memory=4Mi
            - --threshold=5
            - --deployment=heapster-v1.3.0
            - --container=heapster
            - --poll-period=300000
            - --estimator=exponential

Sorry, didn't mean to close the issue ;) I just hit the wrong comment button.
@mumoshu , adding the proper toleration to the heapster pod spec will only allow the pod to run on the controller, but not force the pod to run only on the controller. For this I have to add a nodeAffinity to attract the pod to any of the controller nodes.

I really can't get this right...
I thought I've had created the correct affinity and tolerations now, but the heapster pods always launches on the worker nodes. How can I debug this further?
My heapster depolyment definition
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: heapster-v1.3.0
  namespace: kube-system
  labels:
    k8s-app: heapster
    kubernetes.io/cluster-service: "true"
    version: v1.3.0
spec:
  replicas: 2
  # selector:
    # matchLabels:
      # k8s-app: heapster
      # version: v1.3.0
  template:
    metadata:
      labels:
        k8s-app: heapster
        version: v1.3.0
      # annotations:
        # scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      tolerations:
      - key: "CriticalAddonsOnly"
        operator: "Exists"
      - key: "node.alpha.kubernetes.io/role"
        operator: "Equal"
        value: "master"
        effect: "NoSchedule"
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: k8s-app
                operator: In
                values:
                - heapster
            topologyKey: kubernetes.io/hostname
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kube-aws.coreos.com/role
                operator: NotIn
                values:
                - worker
      containers:
        - image: gcr.io/google_containers/heapster:v1.3.0
          name: heapster
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8082
              scheme: HTTP
            initialDelaySeconds: 180
            timeoutSeconds: 5
          resources:
            limits:
              cpu: 80m
              memory: 200Mi
            requests:
              cpu: 80m
              memory: 200Mi
          command:
            - /heapster
            - --source=kubernetes.summary_api:''
        - image: gcr.io/google_containers/addon-resizer:1.6
          name: heapster-nanny
          resources:
            limits:
              cpu: 50m
              memory: 90Mi
            requests:
              cpu: 50m
              memory: 90Mi
          env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          command:
            - /pod_nanny
            - --cpu=80m
            - --extra-cpu=4m
            - --memory=200Mi
            - --extra-memory=4Mi
            - --threshold=5
            - --deployment=heapster-v1.3.0
            - --container=heapster
            - --poll-period=300000
            - --estimator=exponential

A controller node spec
- apiVersion: v1
  kind: Node
  metadata:
    annotations:
      node.alpha.kubernetes.io/ttl: "0"
      volumes.kubernetes.io/controller-managed-attach-detach: "true"
    creationTimestamp: 2017-04-19T22:36:35Z
    labels:
      beta.kubernetes.io/arch: amd64
      beta.kubernetes.io/instance-type: t2.medium
      beta.kubernetes.io/os: linux
      failure-domain.beta.kubernetes.io/region: eu-west-1
      failure-domain.beta.kubernetes.io/zone: eu-west-1b
      kubernetes.io/hostname: ip-10-1-44-191.eu-west-1.compute.internal
    name: ip-10-1-44-191.eu-west-1.compute.internal
    namespace: ""
    resourceVersion: "126535"
    selfLink: /api/v1/nodesip-10-1-44-191.eu-west-1.compute.internal
    uid: a4cdc10a-2550-11e7-a0cd-02d5584ffffb
  spec:
    externalID: i-02756bb5346d3299d
    providerID: aws:///eu-west-1b/i-02756bb5346d3299d
    taints:
    - effect: NoSchedule
      key: node.alpha.kubernetes.io/role
      timeAdded: null
      value: master
  status:
    addresses:
    - address: <IP_ADDRESS>
      type: InternalIP
    - address: <IP_ADDRESS>
      type: LegacyHostIP
    - address: ip-10-1-44-191.eu-west-1.compute.internal
      type: InternalDNS
    - address: ip-10-1-44-191.eu-west-1.compute.internal
      type: Hostname
    allocatable:
      cpu: "2"
      memory: 3947136Ki
      pods: "110"
    capacity:
      cpu: "2"
      memory: 4049536Ki
      pods: "110"
    conditions:
    - lastHeartbeatTime: 2017-04-20T18:10:11Z
      lastTransitionTime: 2017-04-19T22:36:35Z
      message: kubelet has sufficient disk space available
      reason: KubeletHasSufficientDisk
      status: "False"
      type: OutOfDisk
    - lastHeartbeatTime: 2017-04-20T18:10:11Z
      lastTransitionTime: 2017-04-19T22:36:35Z
      message: kubelet has sufficient memory available
      reason: KubeletHasSufficientMemory
      status: "False"
      type: MemoryPressure
    - lastHeartbeatTime: 2017-04-20T18:10:11Z
      lastTransitionTime: 2017-04-19T22:36:35Z
      message: kubelet has no disk pressure
      reason: KubeletHasNoDiskPressure
      status: "False"
      type: DiskPressure
    - lastHeartbeatTime: 2017-04-20T18:10:11Z
      lastTransitionTime: 2017-04-19T22:36:35Z
      message: kubelet is posting ready status
      reason: KubeletReady
      status: "True"
      type: Ready
    daemonEndpoints:
      kubeletEndpoint:
        Port: 10250
    images:
    - names:
      - quay.io/coreos/hyperkube@sha256:1c8b4487be52a6df7668135d88b4c375aeeda4d934e34dbf5a8191c96161a8f5
      - quay.io/coreos/hyperkube:v1.6.1_coreos.0
      sizeBytes: 664861472
    - names:
      - gcr.io/google_containers/heapster@sha256:3dff9b2425a196aa51df0cebde0f8b427388425ba84568721acf416fa003cd5c
      - gcr.io/google_containers/heapster:v1.3.0
      sizeBytes: 68105973
    - names:
      - gcr.io/google_containers/addon-resizer@sha256:ba506f5f21356331d92141ee48fc4945fd467ec6010364ae970342de5477272c
      - gcr.io/google_containers/addon-resizer:1.6
      sizeBytes: 48784610
    - names:
      - gcr.io/google_containers/pause-amd64@sha256:163ac025575b775d1c0f9bf0bdd0f086883171eb475b5068e7defa4ca9e76516
      - gcr.io/google_containers/pause-amd64:3.0
      sizeBytes: 746888
    nodeInfo:
      architecture: amd64
      bootID: e66ecfc6-6231-41c0-9f5a-320feda7f400
      containerRuntimeVersion: docker://1.12.6
      kernelVersion: 4.9.16-coreos-r1
      kubeProxyVersion: v1.6.1+coreos.0
      kubeletVersion: v1.6.1+coreos.0
      machineID: 8e025a21a4254e11b028584d9d8b12c4
      operatingSystem: linux
      osImage: Container Linux by CoreOS 1298.7.0 (Ladybug)
      systemUUID: EC238E36-080F-BAFB-608E-8C11B6B2F37E
- apiVersion: v1
  kind: Node
  metadata:
    annotations:
      kube-aws.coreos.com/securitygroups: k8sprod-prerequisites-WorkerSecurityGroup-6O3GXX71Z193,k8sprod-Controlplane-1RJHQ7DSHBPTR-SecurityGroupWorker-FAESBHBU1F3T
      node.alpha.kubernetes.io/ttl: "0"
      volumes.kubernetes.io/controller-managed-attach-detach: "true"
    creationTimestamp: 2017-04-20T10:32:34Z
    labels:
      beta.kubernetes.io/arch: amd64
      beta.kubernetes.io/instance-type: t2.large
      beta.kubernetes.io/os: linux
      failure-domain.beta.kubernetes.io/region: eu-west-1
      failure-domain.beta.kubernetes.io/zone: eu-west-1b
      kube-aws.coreos.com/autoscalinggroup: k8sprod-T2largeB-1ANR4S4CPBPZ7-Workers-11MVW8HICN4JB
      kube-aws.coreos.com/launchconfiguration: k8sprod-T2largeB-1ANR4S4CPBPZ7-WorkersLC-169AP7153B6DN
      kube-aws.coreos.com/role: worker
      kubernetes.io/hostname: ip-10-1-44-236.eu-west-1.compute.internal
    name: ip-10-1-44-236.eu-west-1.compute.internal
    namespace: ""
    resourceVersion: "126539"
    selfLink: /api/v1/nodesip-10-1-44-236.eu-west-1.compute.internal
    uid: aaa61ee8-25b4-11e7-9b85-06433a3e9fe9
  spec:
    externalID: i-0c166cdaa4e7002e7
    providerID: aws:///eu-west-1b/i-0c166cdaa4e7002e7
  status:
    addresses:
    - address: <IP_ADDRESS>
      type: InternalIP
    - address: <IP_ADDRESS>
      type: LegacyHostIP
    - address: ip-10-1-44-236.eu-west-1.compute.internal
      type: InternalDNS
    - address: ip-10-1-44-236.eu-west-1.compute.internal
      type: Hostname
    allocatable:
      cpu: "2"
      memory: 8075900Ki
      pods: "110"
    capacity:
      cpu: "2"
      memory: 8178300Ki
      pods: "110"
    conditions:
    - lastHeartbeatTime: 2017-04-20T18:10:13Z
      lastTransitionTime: 2017-04-20T10:32:34Z
      message: kubelet has sufficient disk space available
      reason: KubeletHasSufficientDisk
      status: "False"
      type: OutOfDisk
    - lastHeartbeatTime: 2017-04-20T18:10:13Z
      lastTransitionTime: 2017-04-20T10:32:34Z
      message: kubelet has sufficient memory available
      reason: KubeletHasSufficientMemory
      status: "False"
      type: MemoryPressure
    - lastHeartbeatTime: 2017-04-20T18:10:13Z
      lastTransitionTime: 2017-04-20T10:32:34Z
      message: kubelet has no disk pressure
      reason: KubeletHasNoDiskPressure
      status: "False"
      type: DiskPressure
    - lastHeartbeatTime: 2017-04-20T18:10:13Z
      lastTransitionTime: 2017-04-20T10:32:44Z
      message: kubelet is posting ready status
      reason: KubeletReady
      status: "True"
      type: Ready
    daemonEndpoints:
      kubeletEndpoint:
        Port: 10250
    images:
    - names:
      - xxx.dkr.ecr.eu-west-1.amazonaws.com/ags/105@sha256:20f391bc99458c7bf926f2ecee5bda3db34f0781e45f2969730bd3bacf74cad2
      -<PHONE_NUMBER>93.dkr.ecr.eu-west-1.amazonaws.com/ags/105:GeomapAdmin_1.0.1
      sizeBytes:<PHONE_NUMBER>
    - names:
      - quay.io/coreos/hyperkube@sha256:1c8b4487be52a6df7668135d88b4c375aeeda4d934e34dbf5a8191c96161a8f5
      - quay.io/coreos/hyperkube:v1.6.1_coreos.0
      sizeBytes: 664861472
    - names:
      - gcr.io/google_containers/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb
      - gcr.io/google_containers/echoserver:1.4
      sizeBytes: 140366210
    - names:
      - quay.io/coreos/awscli@sha256:712772e2329b24c203462a72f967a330621d2024b5a5a3545b0bb46dc12efd16
      - quay.io/coreos/awscli:master
      sizeBytes: 97498295
    - names:
      - gcr.io/google_containers/heapster@sha256:3dff9b2425a196aa51df0cebde0f8b427388425ba84568721acf416fa003cd5c
      - gcr.io/google_containers/heapster:v1.3.0
      sizeBytes: 68105973
    - names:
      - gcr.io/google_containers/addon-resizer@sha256:ba506f5f21356331d92141ee48fc4945fd467ec6010364ae970342de5477272c
      - gcr.io/google_containers/addon-resizer:1.6
      sizeBytes: 48784610
    - names:
      - gcr.io/google_containers/cluster-proportional-autoscaler-amd64@sha256:5a3bdd25a5b0f7f8f285e8ff8f4402cf86ddfdfa537e9f053c77c5f043821f70
      - gcr.io/google_containers/cluster-proportional-autoscaler-amd64:1.0.0
      sizeBytes: 48155586
    - names:
      - gcr.io/google_containers/defaultbackend@sha256:ee3aa1187023d0197e3277833f19d9ef7df26cee805fef32663e06c7412239f9
      - gcr.io/google_containers/defaultbackend:1.0
      sizeBytes: 7510068
    - names:
      - gcr.io/google_containers/pause-amd64@sha256:163ac025575b775d1c0f9bf0bdd0f086883171eb475b5068e7defa4ca9e76516
      - gcr.io/google_containers/pause-amd64:3.0
      sizeBytes: 746888
    nodeInfo:
      architecture: amd64
      bootID: ab00ea02-217d-47b4-8f78-96f98a937717
      containerRuntimeVersion: docker://1.12.6
      kernelVersion: 4.9.16-coreos-r1
      kubeProxyVersion: v1.6.1+coreos.0
      kubeletVersion: v1.6.1+coreos.0
      machineID: 8e025a21a4254e11b028584d9d8b12c4
      operatingSystem: linux
      osImage: Container Linux by CoreOS 1298.7.0 (Ladybug)
      systemUUID: EC21F8F9-0AE4-8B0C-425B-5C910B0A5CBB

and a worker node spec
- apiVersion: v1
  kind: Node
  metadata:
    annotations:
      kube-aws.coreos.com/securitygroups: k8sprod-prerequisites-WorkerSecurityGroup-6O3GXX71Z193,k8sprod-Controlplane-1RJHQ7DSHBPTR-SecurityGroupWorker-FAESBHBU1F3T
      node.alpha.kubernetes.io/ttl: "0"
      volumes.kubernetes.io/controller-managed-attach-detach: "true"
    creationTimestamp: 2017-04-20T08:21:16Z
    labels:
      beta.kubernetes.io/arch: amd64
      beta.kubernetes.io/instance-type: t2.large
      beta.kubernetes.io/os: linux
      failure-domain.beta.kubernetes.io/region: eu-west-1
      failure-domain.beta.kubernetes.io/zone: eu-west-1c
      kube-aws.coreos.com/autoscalinggroup: k8sprod-T2largeC-17Z2CTVC1TJGE-Workers-4H8RK5JLPXND
      kube-aws.coreos.com/launchconfiguration: k8sprod-T2largeC-17Z2CTVC1TJGE-WorkersLC-1F7XCJ5EQBU7X
      kube-aws.coreos.com/role: worker
      kubernetes.io/hostname: ip-10-1-45-239.eu-west-1.compute.internal
    name: ip-10-1-45-239.eu-west-1.compute.internal
    namespace: ""
    resourceVersion: "126528"
    selfLink: /api/v1/nodesip-10-1-45-239.eu-west-1.compute.internal
    uid: 531103c6-25a2-11e7-a0cd-02d5584ffffb
  spec:
    externalID: i-0aa0e5fad1ec73293
    providerID: aws:///eu-west-1c/i-0aa0e5fad1ec73293
  status:
    addresses:
    - address: <IP_ADDRESS>
      type: InternalIP
    - address: <IP_ADDRESS>
      type: LegacyHostIP
    - address: ip-10-1-45-239.eu-west-1.compute.internal
      type: InternalDNS
    - address: ip-10-1-45-239.eu-west-1.compute.internal
      type: Hostname
    allocatable:
      cpu: "2"
      memory: 8075900Ki
      pods: "110"
    capacity:
      cpu: "2"
      memory: 8178300Ki
      pods: "110"
    conditions:
    - lastHeartbeatTime: 2017-04-20T18:10:06Z
      lastTransitionTime: 2017-04-20T08:21:16Z
      message: kubelet has sufficient disk space available
      reason: KubeletHasSufficientDisk
      status: "False"
      type: OutOfDisk
    - lastHeartbeatTime: 2017-04-20T18:10:06Z
      lastTransitionTime: 2017-04-20T08:21:16Z
      message: kubelet has sufficient memory available
      reason: KubeletHasSufficientMemory
      status: "False"
      type: MemoryPressure
    - lastHeartbeatTime: 2017-04-20T18:10:06Z
      lastTransitionTime: 2017-04-20T08:21:16Z
      message: kubelet has no disk pressure
      reason: KubeletHasNoDiskPressure
      status: "False"
      type: DiskPressure
    - lastHeartbeatTime: 2017-04-20T18:10:06Z
      lastTransitionTime: 2017-04-20T08:21:26Z
      message: kubelet is posting ready status
      reason: KubeletReady
      status: "True"
      type: Ready
    daemonEndpoints:
      kubeletEndpoint:
        Port: 10250
    images:
    - names:
      - xxx.dkr.ecr.eu-west-1.amazonaws.com/ags/105@sha256:20f391bc99458c7bf926f2ecee5bda3db34f0781e45f2969730bd3bacf74cad2
      -<PHONE_NUMBER>93.dkr.ecr.eu-west-1.amazonaws.com/ags/105:GeomapAdmin_1.0.1
      sizeBytes:<PHONE_NUMBER>
    - names:
      - quay.io/coreos/hyperkube@sha256:1c8b4487be52a6df7668135d88b4c375aeeda4d934e34dbf5a8191c96161a8f5
      - quay.io/coreos/hyperkube:v1.6.1_coreos.0
      sizeBytes: 664861472
    - names:
      - gcr.io/google_containers/nginx-ingress-controller@sha256:995427304f514ac1b70b2c74ee3c6d4d4ea687fb2dc63a1816be15e41cf0e063
      - gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.3
      sizeBytes: 121204435
    - names:
      - quay.io/coreos/awscli@sha256:712772e2329b24c203462a72f967a330621d2024b5a5a3545b0bb46dc12efd16
      - quay.io/coreos/awscli:master
      sizeBytes: 97498295
    - names:
      - gcr.io/google_containers/heapster@sha256:3dff9b2425a196aa51df0cebde0f8b427388425ba84568721acf416fa003cd5c
      - gcr.io/google_containers/heapster:v1.3.0
      sizeBytes: 68105973
    - names:
      - gcr.io/google_containers/addon-resizer@sha256:ba506f5f21356331d92141ee48fc4945fd467ec6010364ae970342de5477272c
      - gcr.io/google_containers/addon-resizer:1.6
      sizeBytes: 48784610
    - names:
      - gcr.io/google_containers/kubedns-amd64@sha256:3d3d67f519300af646e00adcf860b2f380d35ed4364e550d74002dadace20ead
      - gcr.io/google_containers/kubedns-amd64:1.9
      sizeBytes: 46998769
    - names:
      - gcr.io/google_containers/dnsmasq-metrics-amd64@sha256:4063e37fd9b2fd91b7cc5392ed32b30b9c8162c4c7ad2787624306fc133e80a9
      - gcr.io/google_containers/dnsmasq-metrics-amd64:1.0
      sizeBytes: 13998769
    - names:
      - gcr.io/google_containers/exechealthz-amd64@sha256:503e158c3f65ed7399f54010571c7c977ade7fe59010695f48d9650d83488c0a
      - gcr.io/google_containers/exechealthz-amd64:1.2
      sizeBytes: 8374840
    - names:
      - gcr.io/google_containers/defaultbackend@sha256:ee3aa1187023d0197e3277833f19d9ef7df26cee805fef32663e06c7412239f9
      - gcr.io/google_containers/defaultbackend:1.0
      sizeBytes: 7510068
    - names:
      - gcr.io/google_containers/kube-dnsmasq-amd64@sha256:a722df15c0cf87779aad8ba2468cf072dd208cb5d7cfcaedd90e66b3da9ea9d2
      - gcr.io/google_containers/kube-dnsmasq-amd64:1.4
      sizeBytes: 5126001
    - names:
      - gcr.io/google_containers/pause-amd64@sha256:163ac025575b775d1c0f9bf0bdd0f086883171eb475b5068e7defa4ca9e76516
      - gcr.io/google_containers/pause-amd64:3.0
      sizeBytes: 746888
    nodeInfo:
      architecture: amd64
      bootID: f816b350-c0f5-40b1-a836-e011c02b2f78
      containerRuntimeVersion: docker://1.12.6
      kernelVersion: 4.9.16-coreos-r1
      kubeProxyVersion: v1.6.1+coreos.0
      kubeletVersion: v1.6.1+coreos.0
      machineID: 8e025a21a4254e11b028584d9d8b12c4
      operatingSystem: linux
      osImage: Container Linux by CoreOS 1298.7.0 (Ladybug)
      systemUUID: EC2905EE-B013-15FA-984A-C828014882A1

Hmm, sorry for spamming with questions and comments
To me it seems like my deployment is ignoring all my tolerations and affinity configuration. I tried to use nodeAffinity to force heapster to run on a specific worker, but still a random worker is picket by the scheduler
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              # - key: kube-aws.coreos.com/role
                # operator: NotIn
                # values:
                # - worker
              - key: kubernetes.io/hostname
                operator: In
                values:
                - ip-10-1-45-239.eu-west-1.compute.internal

Using the good old nodeSelector does work though, and heapster ends up on the specified node.
      nodeSelector:
        kubernetes.io/hostname: ip-10-1-45-239.eu-west-1.compute.internalal

But if I try to use nodeSelector to force the heapster pod to a controller node the scheduler complains about the taint not being accepted. Even though I have added this tolerations to my deployment file
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      - key: 'node.alpha.kubernetes.io/role'
        operator: Equal
        value: master
        effect: NoSchedule

Quick comment after looking at your above example yaml - shouldn't
nodeAffinity in pod spec rather than deployment spec?(does it even pass the
validation when in deployment spec?
2017年4月21日(金) 7:08 paalkr<EMAIL_ADDRESS>
Hmm, sorry for spamming with questions and comments
To me it seems like my deployment is ignoring all my tolerations and
affinity configuration. I tried to use nodeAffinity to force heapster to
run on a specific worker, but still a random worker is picket by the
scheduler
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          # - key: kube-aws.coreos.com/role
            # operator: NotIn
            # values:
            # - worker
          - key: kubernetes.io/hostname
            operator: In
            values:
            - ip-10-1-45-239.eu-west-1.compute.internal

Using the good old nodeSelector does work though, and heapster ends up on
the specified node.
  nodeSelector:
    kubernetes.io/hostname: ip-10-1-45-239.eu-west-1.compute.internalal

But if I try to use nodeSelector to force the heapster pod to a controller
node the scheduler complains about the taint not being accepted. Even
though I have added this tolerations to my deployment file
  tolerations:
  - key: CriticalAddonsOnly
    operator: Exists
  - key: 'node.alpha.kubernetes.io/role'
    operator: Equal
    value: master
    effect: NoSchedule

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-incubator/kube-aws/issues/566#issuecomment-295937194,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABV-RyB5nKxY6Chb3Ao5mGCoFDi9xmzks5rx9dMgaJpZM4NC3c4
.

@mumoshu , my first example had this wrong and it was correctly not passing validation. But I thought I got this right in the latest configuration i posted. I but it in here as well just for reference.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: heapster-v1.3.0
  namespace: kube-system
  labels:
    k8s-app: heapster
    kubernetes.io/cluster-service: "true"
    version: v1.3.0
spec:
  replicas: 2
  # selector:
    # matchLabels:
      # k8s-app: heapster
      # version: v1.3.0
  template:
    metadata:
      labels:
        k8s-app: heapster
        version: v1.3.0
      # annotations:
        # scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      tolerations:
      - key: "CriticalAddonsOnly"
        operator: "Exists"
      - key: "node.alpha.kubernetes.io/role"
        operator: "Equal"
        value: "master"
        effect: "NoSchedule"
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: k8s-app
                operator: In
                values:
                - heapster
            topologyKey: kubernetes.io/hostname
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kube-aws.coreos.com/role
                operator: NotIn
                values:
                - worker
......

@mumoshu , Any suggestions why the tolerations and affinity settings are not respected?

@paalkr AFAICS, kube-aws worker nodes aren't labeled with kube-aws.coreos.com/role by default.
Could you ensure that they're explicitly labeled with appropriate config in cluster.yaml like:
worker:
  nodePools:
  - name: pool1
     nodeLabels:
       kube-aws.coreos.com/role: worker

?

Hi
My cluster.yaml file is modified to properly tag workers, according to your suggestion. So the tag are in place. But still the nodeAffinity does not function as expected. The pods are attracted to the workers, not the controllers.

@paalkr Hi! Could you share me the result of kubectl describe node <one of your worker node> and kubectl desribe node <one of your controller node>?

@mumoshu , thanks for taking time to help out. Highly appreciated!
Please find the output below
Controller:
Name:                   ip-10-1-43-150.eu-west-1.compute.internal
Role:
Labels:                 beta.kubernetes.io/arch=amd64
                        beta.kubernetes.io/instance-type=t2.medium
                        beta.kubernetes.io/os=linux
                        failure-domain.beta.kubernetes.io/region=eu-west-1
                        failure-domain.beta.kubernetes.io/zone=eu-west-1a
                        kubernetes.io/hostname=ip-10-1-43-150.eu-west-1.compute.internal
Annotations:            node.alpha.kubernetes.io/ttl=0
                        volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:                 node.alpha.kubernetes.io/role=master:NoSchedule
CreationTimestamp:      Thu, 20 Apr 2017 00:36:57 +0200
Phase:
Conditions:
  Type                  Status  LastHeartbeatTime                       LastTransitionTime                      Reason                          Message
  ----                  ------  -----------------                       ------------------                      ------                          -------
  OutOfDisk             False   Tue, 25 Apr 2017 08:45:20 +0200         Thu, 20 Apr 2017 00:36:57 +0200         KubeletHasSufficientDisk        kubelet has sufficient disk space available
  MemoryPressure        False   Tue, 25 Apr 2017 08:45:20 +0200         Thu, 20 Apr 2017 00:36:57 +0200         KubeletHasSufficientMemory      kubelet has sufficient memory available
  DiskPressure          False   Tue, 25 Apr 2017 08:45:20 +0200         Thu, 20 Apr 2017 00:36:57 +0200         KubeletHasNoDiskPressure        kubelet has no disk pressure
  Ready                 True    Tue, 25 Apr 2017 08:45:20 +0200         Thu, 20 Apr 2017 00:36:57 +0200         KubeletReady                    kubelet is posting ready status
Addresses:              <IP_ADDRESS>,<IP_ADDRESS>,ip-10-1-43-150.eu-west-1.compute.internal,ip-10-1-43-150.eu-west-1.compute.internal
Capacity:
 cpu:           2
 memory:        4049536Ki
 pods:          110
Allocatable:
 cpu:           2
 memory:        3947136Ki
 pods:          110
System Info:
 Machine ID:                    8e025a21a4254e11b028584d9d8b12c4
 System UUID:                   EC235DB6-B862-7550-8FF8-BB7847BEBAF8
 Boot ID:                       d739a8fb-6f0b-4e1d-baa2-e2a7db373e22
 Kernel Version:                4.9.16-coreos-r1
 OS Image:                      Container Linux by CoreOS 1298.7.0 (Ladybug)
 Operating System:              linux
 Architecture:                  amd64
 Container Runtime Version:     docker://1.12.6
 Kubelet Version:               v1.6.1+coreos.0
 Kube-Proxy Version:            v1.6.1+coreos.0
ExternalID:                     i-0c48af722c785f04b
Non-terminated Pods:            (4 in total)
  Namespace                     Name                                                                            CPU Requests    CPU Limits      Memory Requests Memory Limits
  ---------                     ----                                                                            ------------    ----------      --------------- -------------
  kube-system                   kube-apiserver-ip-10-1-43-150.eu-west-1.compute.internal                        0 (0%)          0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-controller-manager-ip-10-1-43-150.eu-west-1.compute.internal               200m (10%)      0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-proxy-ip-10-1-43-150.eu-west-1.compute.internal                            0 (0%)          0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-scheduler-ip-10-1-43-150.eu-west-1.compute.internal                        100m (5%)       0 (0%)          0 (0%)          0 (0%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits      Memory Requests Memory Limits
  ------------  ----------      --------------- -------------
  300m (15%)    0 (0%)          0 (0%)          0 (0%)
Events:         <none>

Worker:
Name:                   ip-10-1-43-42.eu-west-1.compute.internal
Role:
Labels:                 beta.kubernetes.io/arch=amd64
                        beta.kubernetes.io/instance-type=t2.large
                        beta.kubernetes.io/os=linux
                        failure-domain.beta.kubernetes.io/region=eu-west-1
                        failure-domain.beta.kubernetes.io/zone=eu-west-1a
                        kube-aws.coreos.com/autoscalinggroup=k8sprod-T2largeA-JQVZA9S334BT-Workers-VD3FAJXLV81K
                        kube-aws.coreos.com/launchconfiguration=k8sprod-T2largeA-JQVZA9S334BT-WorkersLC-41PSQQY31I4Y
                        kube-aws.coreos.com/role=worker
                        kubernetes.io/hostname=ip-10-1-43-42.eu-west-1.compute.internal
Annotations:            kube-aws.coreos.com/securitygroups=k8sprod-prerequisites-WorkerSecurityGroup-6O3GXX71Z193,k8sprod-Controlplane-1RJHQ7DSHBPTR-SecurityGroupWorker-FAESBHBU1F3T
                        node.alpha.kubernetes.io/ttl=0
                        volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:                 <none>
CreationTimestamp:      Thu, 20 Apr 2017 11:18:23 +0200
Phase:
Conditions:
  Type                  Status  LastHeartbeatTime                       LastTransitionTime                      Reason                          Message
  ----                  ------  -----------------                       ------------------                      ------                          -------
  OutOfDisk             False   Tue, 25 Apr 2017 08:47:35 +0200         Sat, 22 Apr 2017 06:31:01 +0200         KubeletHasSufficientDisk        kubelet has sufficient disk space available
  MemoryPressure        False   Tue, 25 Apr 2017 08:47:35 +0200         Sat, 22 Apr 2017 06:31:01 +0200         KubeletHasSufficientMemory      kubelet has sufficient memory available
  DiskPressure          False   Tue, 25 Apr 2017 08:47:35 +0200         Sat, 22 Apr 2017 06:31:01 +0200         KubeletHasNoDiskPressure        kubelet has no disk pressure
  Ready                 True    Tue, 25 Apr 2017 08:47:35 +0200         Sat, 22 Apr 2017 06:31:01 +0200         KubeletReady                    kubelet is posting ready status
Addresses:              <IP_ADDRESS>,<IP_ADDRESS>,ip-10-1-43-42.eu-west-1.compute.internal,ip-10-1-43-42.eu-west-1.compute.internal
Capacity:
 cpu:           2
 memory:        8178300Ki
 pods:          110
Allocatable:
 cpu:           2
 memory:        8075900Ki
 pods:          110
System Info:
 Machine ID:                    8e025a21a4254e11b028584d9d8b12c4
 System UUID:                   EC200572-41D5-2244-3240-0FE68293A76F
 Boot ID:                       2d7db8bd-2f04-472b-b46d-a2c0d5f68c5b
 Kernel Version:                4.9.16-coreos-r1
 OS Image:                      Container Linux by CoreOS 1298.7.0 (Ladybug)
 Operating System:              linux
 Architecture:                  amd64
 Container Runtime Version:     docker://1.12.6
 Kubelet Version:               v1.6.1+coreos.0
 Kube-Proxy Version:            v1.6.1+coreos.0
ExternalID:                     i-024daef28f2c111dc
Non-terminated Pods:            (8 in total)
  Namespace                     Name                                                            CPU Requests    CPU Limits      Memory Requests Memory Limits
  ---------                     ----                                                            ------------    ----------      --------------- -------------
  default                       geomapadmin-468890941-xz00f                                     100m (5%)       500m (25%)      350Mi (4%)      0 (0%)
  default                       ingress-nginx-2346665006-0463j                                  50m (2%)        0 (0%)          0 (0%)          0 (0%)
  default                       nginx-default-backend-2003809344-2184w                          10m (0%)        100m (5%)       20Mi (0%)       50Mi (0%)
  kube-system                   heapster-v1.3.0-504010935-3jvsb                                 194m (9%)       194m (9%)       354Mi (4%)      354Mi (4%)
  kube-system                   kube-dns-3816048056-5tpmj                                       260m (13%)      0 (0%)          140Mi (1%)      220Mi (2%)
  kube-system                   kube-proxy-ip-10-1-43-42.eu-west-1.compute.internal             0 (0%)          0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-rescheduler-3155147949-0rm2p                               10m (0%)        0 (0%)          100Mi (1%)      0 (0%)
  kube-system                   kubernetes-dashboard-v1.5.1-lpnbb                               100m (5%)       100m (5%)       50Mi (0%)       50Mi (0%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits      Memory Requests Memory Limits
  ------------  ----------      --------------- -------------
  724m (36%)    894m (44%)      1014Mi (12%)    674Mi (8%)
Events:         <none>

@paalkr Sorry for being late in replying and thanks for the info!
Your configuration seems good. Then, have you by any chance modified node labels after the pods had ben scheduled?
Could you kubectl delete those problematic pods anyway and then observe if they still get re-scheduled to the worker nodes?
If deleting pods doesn't fix your case, could you try to tag controller nodes and give pods node affinities to controller nodes, instead of node anti-affinities to worker nodes?

FYI, experimental.nodeLabels can be used to label only controller nodes.

Thanks @mumoshu . Yes, I noticed the experimental.nodeLabels feature, and applied that wile updating kube-aws to the latest RC. I also deleted the out-of-the-box heapster deployemnt (and by that all Heapster pods), and redeployed Heapster. But still no luck.
Output from one of the controller nodes
Name:                   ip-10-1-45-70.eu-west-1.compute.internal
Role:
Labels:                 beta.kubernetes.io/arch=amd64
                        beta.kubernetes.io/instance-type=t2.medium
                        beta.kubernetes.io/os=linux
                        failure-domain.beta.kubernetes.io/region=eu-west-1
                        failure-domain.beta.kubernetes.io/zone=eu-west-1c
                        kube-aws.coreos.com/role=controller
                        kubernetes.io/hostname=ip-10-1-45-70.eu-west-1.compute.internal
Annotations:            node.alpha.kubernetes.io/ttl=0
                        volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:                 node.alpha.kubernetes.io/role=master:NoSchedule
CreationTimestamp:      Fri, 28 Apr 2017 10:02:13 +0200
Phase:
Conditions:
  Type                  Status  LastHeartbeatTime                       LastTransitionTime                      Reason                          Message
  ----                  ------  -----------------                       ------------------                      ------                          -------
  OutOfDisk             False   Sun, 30 Apr 2017 18:29:04 +0200         Fri, 28 Apr 2017 10:02:13 +0200         KubeletHasSufficientDisk        kubelet has sufficient disk space available
  MemoryPressure        False   Sun, 30 Apr 2017 18:29:04 +0200         Fri, 28 Apr 2017 10:02:13 +0200         KubeletHasSufficientMemory      kubelet has sufficient memory available
  DiskPressure          False   Sun, 30 Apr 2017 18:29:04 +0200         Fri, 28 Apr 2017 10:02:13 +0200         KubeletHasNoDiskPressure        kubelet has no disk pressure
  Ready                 True    Sun, 30 Apr 2017 18:29:04 +0200         Fri, 28 Apr 2017 10:02:13 +0200         KubeletReady                    kubelet is posting ready status
Addresses:              <IP_ADDRESS>,<IP_ADDRESS>,ip-10-1-45-70.eu-west-1.compute.internal,ip-10-1-45-70.eu-west-1.compute.internal
Capacity:
 cpu:           2
 memory:        4049512Ki
 pods:          110
Allocatable:
 cpu:           2
 memory:        3947112Ki
 pods:          110
System Info:
 Machine ID:                    8e025a21a4254e11b028584d9d8b12c4
 System UUID:                   EC21EB49-60D8-2923-FFA2-D2D16B6A97A6
 Boot ID:                       a91ab878-68a4-4a44-ac50-34c7d3211920
 Kernel Version:                4.9.24-coreos
 OS Image:                      Container Linux by CoreOS 1353.7.0 (Ladybug)
 Operating System:              linux
 Architecture:                  amd64
 Container Runtime Version:     docker://1.12.6
 Kubelet Version:               v1.6.2+coreos.0
 Kube-Proxy Version:            v1.6.2+coreos.0
ExternalID:                     i-08934667737e24263
Non-terminated Pods:            (4 in total)
  Namespace                     Name                                                                            CPU Requests    CPU Limits      Memory Requests Memory Limits
  ---------                     ----                                                                            ------------    ----------      --------------- -------------
  kube-system                   kube-apiserver-ip-10-1-45-70.eu-west-1.compute.internal                         0 (0%)          0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-controller-manager-ip-10-1-45-70.eu-west-1.compute.internal                200m (10%)      0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-proxy-ip-10-1-45-70.eu-west-1.compute.internal                             0 (0%)          0 (0%)          0 (0%)          0 (0%)
  kube-system                   kube-scheduler-ip-10-1-45-70.eu-west-1.compute.internal                         100m (5%)       0 (0%)          0 (0%)          0 (0%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits      Memory Requests Memory Limits
  ------------  ----------      --------------- -------------
  300m (15%)    0 (0%)          0 (0%)          0 (0%)
Events:         <none>

Heapster deployment definition
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: heapster-v1.3.0
  namespace: kube-system
  labels:
    k8s-app: heapster
    kubernetes.io/cluster-service: "true"
    version: v1.3.0
spec:
  replicas: 3
  # selector:
    # matchLabels:
      # k8s-app: heapster
      # version: v1.3.0
  template:
    metadata:
      labels:
        k8s-app: heapster
        version: v1.3.0
      # annotations:
        # scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      # nodeSelector:
        # # kube-aws.coreos.com/role: controller
        # kubernetes.io/hostname: ip-10-1-45-239.eu-west-1.compute.internalal
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      - key: 'node.alpha.kubernetes.io/role'
        operator: Equal
        value: master
        effect: NoSchedule
      affinity:
        podAntiAffinity:
        # podAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: k8s-app
                operator: In
                values:
                - heapster
            topologyKey: kubernetes.io/hostname
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kube-aws.coreos.com/role
                operator: In
                values:
                - controller
              # - key: kubernetes.io/hostname
                # operator: In
                # values:
                # - ip-10-1-45-239.eu-west-1.compute.internal
      containers:
        - image: gcr.io/google_containers/heapster:v1.3.0
          name: heapster
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8082
              scheme: HTTP
            initialDelaySeconds: 180
            timeoutSeconds: 5
          resources:
            limits:
              cpu: 80m
              memory: 200Mi
            requests:
              cpu: 80m
              memory: 200Mi
          command:
            - /heapster
            - --source=kubernetes.summary_api:''
        - image: gcr.io/google_containers/addon-resizer:1.6
          name: heapster-nanny
          resources:
            limits:
              cpu: 50m
              memory: 90Mi
            requests:
              cpu: 50m
              memory: 90Mi
          env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          command:
            - /pod_nanny
            - --cpu=80m
            - --extra-cpu=4m
            - --memory=200Mi
            - --extra-memory=4Mi
            - --threshold=5
            - --deployment=heapster-v1.3.0
            - --container=heapster
            - --poll-period=300000
            - --estimator=exponential

List of controller nodes
ip-10-1-45-70.eu-west-1.compute.internal
ip-10-1-43-190.eu-west-1.compute.internal

Heapster only running on workers
heapster-v1.3.0-504010935-7zmr9                                     2/2       Running   0          2d        <IP_ADDRESS>   ip-10-1-43-118.eu-west-1.compute.internal
heapster-v1.3.0-504010935-qjrfx                                     2/2       Running   0          2d        <IP_ADDRESS>   ip-10-1-44-100.eu-west-1.compute.internal
heapster-v1.3.0-504010935-wh2lx                                     2/2       Running   0          2d        <IP_ADDRESS>   ip-10-1-45-91.eu-west-1.compute.internal

@paalkr Sorry for the long silence!
Hmm, that's strange. Would you mind sharing me the output from kubectl get no --show-labels?
Only possible causes in my mind for now are a bug in k8s which makes nodeAffinity broken or a bug in kube-aws which adds uniform labels to all the nodes not only controller but also worker.

@paalkr Any updates on this?
Also - is there any chance you had used an old version of kubectl on your machine (rather than the one run by kube-aws within the cluster) to update the heapster deployment?
An old kubectl would strip all the fields like affinity and tolerations added in k8s 1.6.
If that's the case, I'd suggest you to upgrade kubectl first and test if it works, and then modify cloud-config-controller to instruct kube-aws to do it for you.

I discovered that if I disable the nanny heaspter container it worked. The nanny is responsible for scaling the pod, any reason in particular to use the nanny rather then hpa for scaling heapster?

@paalkr Thanks for the reply!
According to the last changed date of it, nanny seems to use an older version of kubernetes client to communicate with k8s.
Also, how it updates a k8s deployment is problematic - it is using Update instead of Patch. It certainly strips all the recently introduced fields like taints and affinity when it reads and then overwrites the heapster deployment.
So, I'd suggest you to not use addon_resizer, or send a PR to fix the problem.
Also, I guess what you want is VPA rather than HPA as scaling "out" heapster doesn't make sense?
What you want is scaling "up" of heapster, right?
Anyway, VPA is under-development in the kubernetes/autoscaler repo though.

@mumoshu , thanks for the confirmation. I did come to the conclusion that the nanny somehow interfered with my desired goal. And you just confirmed and pinpointed the problem, that the nanny actually is using an old version of the k8s client not compatible with taints and affinity ;)
I'm not sure if I understand what you mean by scaling heapster up instead of out.
Not much documentation yet of the VPA initiative regarding the objective and mechanics, or do I look at the wrong location?

I'm sorry to "bump" this issue. But are there any news in regards to how to solve this problem?

I followed discussion and can't understand why running kube-system nodes on workers is something you try to avoid. It doesn't matter where pods are running, nodes will go down on updates.
You can shorter that period by enabling nodeDrainer, which was updated recently and now evicts pods correctly, so downtime would be only time needed to start a pod on a new node.

@paalkr

I'm not sure if I understand what you mean by scaling heapster up instead of out.

Scaling up here means that you provide more resources(cpu, memory) to the single heapster pod so that heapster can keep correcting metrics even when your cluster gets larger. Nanny and VPA automates this process.
AFAIK heapster doesn't scale by adding replicas - so scaling "out" isn't an option.

@redbaron Thanks for chiming in 👍
I guess @paalkr's original explanation on the problem adds the context:

When I'm testing auto scaling of the worker node pools I see that some system critical pods are running on the worker nodes. Once in a while when a worker node is terminated by AWS, the critical pods are then terminated and redeployed to any running worked node. Not a big problem, but for example heapster statistics will not be available for the short period of time it takes Kubernetes to restart the pod on a running node.

Suppose you want to enable cluster-autoscaling on your cluster(with CA or AWS-native autoscaling), you can keep running pods on worker nodes as long as a deployment has 2 or more replicas and it isn't "critical"(note that, cluster-autoscaler doesn't try to terminate nodes running critical pods). For critical single pods like rescheduler, heapster, kube-dns-autoscaler, cluster-autoscaler, I understand that some people would like schedule them on controller nodes so that (1) they're not affected by cluster-autoscaling to incur downtime when AWS-native autoscaling is enabled on worker nodes (2) k8s cluster-autoscaler is able to delete nodes as there would be no worker nodes running critical pods
@paalkr Do I understand your problem correctly?

Just to be sure:

kube-dns pods should run on worker nodes
heapster deployment should have only 1 replica. Setting replicas: 2 on it doesn't make sense

No, question had nothing about cluster autoscaler or I fail to see it:

Once in a while when a worker node is terminated by AWS,

Once in a while AWS terminates VMs and really doens't care whether your VM is a Nodepool or Controller or Etcd ASG, so moving critical pod from one ASG to another buys you nothing in that regard.
I agree with your argument that there is a use-case to run certain pods on controllers, thanks for clarification.

@redbaron Thanks,

Once in a while AWS terminates VMs and really doens't care whether your VM is a Nodepool or Controller or Etcd ASG, so moving critical pod from one ASG to another buys you nothing in that regard.

Yes, I agree with you. There's no way to avoid a short downtime in such cases! Thanks for the clarification too 👍

@mumoshu , @redbaron
Absolutely correct. EC2 instances will fail, it's just a matter of time. And if that instance happens to be the node running heapster you will be out of statistic while the pod is moved to a running node, and a replacing instance added back to the cluster. My goal here was actually to run at least two instances of heaspter in parallel on different nodes, but I understand now that heaspter won't be particularly happy about having any siblings playing alongside ;)


Add functions for setting only mtime/atime

Please add set_file_mtime() and set_file_atime() that set only one of the values.
In Linux this can be done by passing UTIME_OMIT as the argument that's not needed. See example in libstdc++'s implementation of std::filesystem::last_write_time(): https://github.com/gcc-mirror/gcc/blob/gcc-8_1_0-release/libstdc%2B%2B-v3/src/filesystem/std-ops.cc#L1067

Done in https://github.com/alexcrichton/filetime/pull/40


Stucked at Compiling surel_gacc on Macbook M1 chip

Hi there, when trying to run this repo at MacBook Pro (13-inch, M1, 2020), I found some obstacles:

failed at recover conda environment using conda env create -f environment.yml

failed at compiling surel_gacc

Hope to get help with these issues. Can we run SUREL without c acceleration?

Hello, thank you for your interest in our work. Note that SUREL has not been tested on MacOS.

environment.yml is exported from Ubuntu 20.04, some packages may not work on Mac.
SUREL depends on C extensions. From the error trace, it looks like your environment doesn't have openmp properly configured.

Hi,
Sorry for inserting myself into the conversation 😅
I saw the issue and had remembered facing the same problem on my M1 as well. The fix that worked for me:
You need to update the clang compiler to one that comes with Homebrew.
Prerequisite: Make sure homebrew is installed (https://docs.brew.sh/Installation) on your macbook.
Steps after making sure homebrew is installed:

Install llvm on your homebrew:  brew install llvm
Point to the brew clang compiler (could be different version for you): export CC=/opt/homebrew/Cellar/llvm/13.0.1_1/bin/clang

This should ideally work. Please double-check if you have the clang compiler in this path export CC=/opt/homebrew/Cellar/llvm/13.0.1_1/bin/clang. It could be a different version for you.

@venomouscyanide many thanks! It works.
But another Error raised after run the script python main.py --dataset ogbl-collab --metric hit --num_step 4 --num_walk 200 --use_val

from surel_gacc import run_sample ImportError: dynamic module does not define module export function (PyInit_surel_gacc)


http error: certificate signed by unknown authority (code=500)

Description
When I click the App Templates link its throws an error. Checking the logs it says
2018/04/24 06:24:49 Starting Portainer 1.16.5 on :9000,
2018/04/24 07:13:16 http error: User not found (code=404),
2018/04/24 07:14:23 http error: Get https://raw.githubusercontent.com/portainer/templates/master/templates.json: x509: certificate signed by unknown authority (code=500),
2018/04/24 07:14:35 http error: Get https://raw.githubusercontent.com/portainer/templates/master/templates.json: x509: certificate signed by unknown authority (code=500),
2018/04/24 07:14:51 http error: Get https://raw.githubusercontent.com/portainer/templates/master/templates.json: x509: certificate signed by unknown authority (code=500),
2018/04/24 07:15:54 http error: Get https://raw.githubusercontent.com/portainer/templates/master/templates.json: x509: certificate signed by unknown authority (code=500),
2018/04/24 07:24:17 http error: Get https://raw.githubusercontent.com/portainer/templates/master/templates.json: x509: certificate signed by unknown authority (code=500),

Steps to reproduce the issue:

Deploy Portainer
Click on the App Templates link
Check container logs

Technical details:

Portainer version: 1.16.5
Target Docker version (the host/cluster you manage): 17.12.1-ce
Platform (windows/linux): Alpine Linux 3.7.0
Command used to start Portainer:

docker service create \
--name portainer \
--publish 9000:9000 \
--replicas=1 \
--constraint 'node.role == manager' \
--mount type=bind,src=//var/run/docker.sock,dst=/var/run/docker.sock \
--mount type=bind,src=//opt/portainer,dst=/data \
portainer/portainer \
-H unix:///var/run/docker.sock

Target Swarm version (if applicable): N/A
Browser: Google Chrome Version 65.0.3325

Are you able to reproduce this issue or was it a temporary issue?

It is not temporary. Every time I click I get the same error message.

I am also running into the same issue and i am using minikube for docker deamon and able to access docker endpoint (minikube exposed only ssl way)
Please help on selecting which crt, key has to be uploaded for TLS CA Certificate, TLS Certificate and TLS Key from my below (.minikube directory).

I am also running into the same issue and i am using minikube for docker deamon and able to access docker endpoint (minikube exposed only ssl way)
Please help on selecting which crt, key has to be uploaded for TLS CA Certificate, TLS Certificate and TLS Key from my below (.minikube directory).

portainer ui:

.minikube folder:

Portainer log:

#460

Stale bot, bugger off. This is still an issue.
It may be related to the way docker handles trusted CA certs. Per official docs, the certs need to be placed in /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt (or equivalent).
I'm curious if the portainer docker agent image can correctly add the root CA.

As this is being reported as an issue still, I'll re-open for discussion.
@issmirnov can you please outline your docker version, portainer version, os and reproduction steps as well as whether you are using a reverse proxy with Portainer so that I can have a go at reproducing this?

Hey @itsconquest , thanks for getting back to me! Here's what I've got set up.
CA structure
└── root CA
    ├── intermediate CA: bastion
    │   ├── server.pem (cert)
    │   └── server-key.pem
    └── intermediate CA: warrior
        ├── client.pem (cert)
        └── client-key.pem

So, as you can see I have a common root CA that I use for all devices.
Server Docker setup
# cat /etc/docker/daemon.json
...
  "tlscacert": "root-ca.pem",
  "tlscert": "server.pem",
  "tlskey": "server-key.pem",
...

Versions
Portainer: 1.20.2
Portainer deployment: docker run -d --restart=always -p 8200:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer
Docker: 19.03.10 Community (both server and client , on both machines)
OS: Ubuntu 18.04.4 LTS on both client and server
Reverse proxy
Yes, nginx. Do I need to enable http2 and proxy upgrade params?
Repro Steps

On client machine, install docker + spin up portainer pointing at local docker socket.
On remote server, spin up docker, and update daemon.json with root CA, server cert, server key. Verify firewall allows access to TLS endpoint: docker --tlsverify --tlscacert=root-ca.pem --tlscert=client.pem --tlskey=client-key.pem -H=<IP_ADDRESS>:2376 version
Log in to portainer, try to add a new endpoint while setting up "TLS with server and client verification"
upload client key, client cert, and root CA

I expect this to work, since the server will present a CA signed by intermediate-bastion, which is backed by root-CA. client has root-CA, so can verify.
Note: I tried to upload the client-bundle (with root-ca + intermediate-warrior + client-cert chained), but it did not help.
Note
I was able to use "TLS with Client Verification Only" to connect, but since I have the full PKI setup, I'd like to perform server verification too.

I worked around the issue with this Dockerfile. It will show a warning about "only one certificate" but you can ignore it. I'm not sure if this works with the normal "portainer/agent" image. That one seems tightly protected. The alpine one was easier to modify.
FROM portainer/agent:linux-amd64-1.6.0-alpine

RUN apk update && apk add ca-certificates && rm -rf /var/cache/apk/*

COPY ./my-cert-auth.crt /usr/local/share/ca-certificates/

RUN update-ca-certificates

This is being investigated, thanks for the info. Can anyone reproduce with the latest version of Portainer, v1.24.0?

@issmirnov I talked this over with my team and I don't think we ever supported intermediate certs for docker TLS enabled endpoints..

@itsconquest understood, is it possible to add multiple certs so that the CA chain works?

You should be able to combine root and intermediate certs.
We will add a feature request to support intermediate certs.

Closing this as I have opened a separate feat request

The user account who open the separate feat request in 2020 in now a deleted account. Does portainer support intermediate certs now?


Tokyo Disney Land & Tokyo Disney Sea get wait time was broken

Hi,
The Tokyo disney get wait time function looks something wrong with API, every attraction always return "Closed" type, and the time wrong too.
Maybe it is relate to request of the realtime url?

What version of wdwjs are you using? And at what time did you make the request?
I can't test right now, since it's nearly midnight and the park is legitimately closed :) Will test later today though to try and reproduce

I use the version is 3.0.6, and it was wrong from morning before 8 A.M (Japan timezone).
I clone and try it all my day, the message always said : "運営状況の確認には位置情報の送信が必要です。"
It is means that "You need send your location info for checking attractions status"
I'm pretty sure that yesterday was good.

Aha! That is interesting. I think they might have changed how they check you're inside the park...
When I wrote the code, the webpage would request your geo-location and send your longitude and latitude to an API, which would set a cookie to confirm you're in that spot.
So, I just invent a longitude/latitude pair randomly within the parks grounds and send that and store the returned cookie.
I'm guessing they have changed how this setup works, I'll have a poke tonight (I'm in GMT), or if you can fathom how it's put together, feel free to make a Pull Request (that would be very exciting for me!).
Thank you for the bug report, it makes me really glad when people report things so we can keep this library correct and working!

You know what! They don't change the rule of make sure you are in the park. Because I write a testing script and works good.
So, I think the point might be the SetupAPI function need some fix. When I find it out, then push and make Pull Request.
Have fun tonight :)

Pushed a commit to master that may fix this.
https://github.com/cubehouse/wdwJS/commit/36c3282037b4282b87fb23dfb7d2b64a13e7378a
Let me know if that looks good, I think this was a mistake introduced with the recent addition to the ride opening/closing times being added.

Merged in #17
Thanks :)


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 1.10.145 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


RCTScrollEvent: get all required values injected rather than accessing the scroll view

This PR fixes #15006 by removing all UI API calls from RCTScrollEvent.
-[RCTScrollEvent arguments] can now be called from a background thread.
The Main Thread Checker of Xcode 9 will not any longer produce runtime issues when calling this method.
Test plan

create a React Native (version: this PR) project with a scroll view
open it in Xcode 9
launch it
scroll the scroll view
observe the runtime issues in Xcode. There should not contain "UI API called from background thread"-issues.

I verified my changes on this branch: https://github.com/HeEAaD/Demo-ReactNative-UI-not-on-main-thread/tree/fix

@HeEAaD Thank you!


Update main.cpp

Hello dude. I tried use u project, but lowercase key is not work.
Keyboard chicony KB-5191. I added this for working.

Thanks for the PR!


alias or modify legend format

Currently the graph legend always displays the metric path query. It would be good to modify this so that it can show other things, most typically the host name.
I'd be happy to either use AS or ALIAS or the prometheus data source plugin exposes another field:

We can add ALIAS keyword to grafana. To make it useful, we will need to have dalmatinerdb/dalmatinerdb#56 implemented.

this has been implemented as per https://github.com/dalmatinerdb/dalmatinerdb-datasource/pull/12

yay!


Updated docs 0.8.8

https://micdavis.github.io/DataProfiler/docs/0.8.8/html/index.html

does this not include the chi2-test note withpsi?

It does here: https://micdavis.github.io/DataProfiler/docs/0.8.8/html/profiler.html#profile-differences:~:text=t-test%2C-,chi2-test%2C,-and psi (Popoulation

Will wait on auto-merge for @taylorfturner to examine:
https://github.com/capitalone/DataProfiler/pull/765/commits/24c6573d64f96939163678f3c685565faeb23bba
But I think the chi2-test


UDP: send with source address

If a UDP server is bound to more than a single address (typically because it's bound to the unspecified address) and the system can have more than a single address at an interface (not uncommon at IPv6), then an unconnected UDP server's send function needs a function that allows setting the source address (basically to the .1 of the result of a receive call). Otherwise, the OS will fill in that information -- and if the peer is picky about addresses (as CoAP is), it may guess wrong. (For example, Linux will pick the most recent address).
Adding support for this in a fully reliable way would probably mean adding a trait with an ugly name (UdpServerMultiaddress::send_to_from?). A more compatible, softer but prone-to-silent-misbehavior way forward would be to introduce UdpSever::send_to_from with a default implementation that disregards the from address (which is an OK behavior in the may cases when there is only one legal value anyway). Ideally, that default implementation would go away with the next incompatible version bump. (We can't have Rust show deprecation warnings when relying on a default implementation, can we?)
I'm happy to do a PR for this (just can't do it right now), but it would be good to have some initial feedback first.

Starting to use the embedded-nal server side, I noted that the bind function doesn't even take an address. This eliminates the workaround I'd otherwise have used here (to only ever bind a socket to a single unicast address).

This was fixed from the onset in the async traits which have that address accessible unconditionally.
The issue is low priority for me now because my use cases more and more use async; still, having feature parity would be nice.


Chore/clean up examples and demos

Rip out old examples
Create list of missing examples in project board
Move example index to root (netlify)

[ ] Show global config/settings

[ ] Watch & update on change

[ ] Group tools by baseTool type
[ ] Add a 2nd canvas for testing synchronizers
[ ] Support for different mouseButtonMasks

[ ] Show set tools + interaction paradigm

Preview URL: https://deploy-preview-541--cornerstone-tools.netlify.com/


Compare black, blue, yapf, and autopep8

Analyze which formatting tool is more helpful

Switch to blue


Support for 'source'

A feature like "source xxx.sh" is needed.
I tried "sh.bash("-c", "source xxx.sh")" but the environment variables in xxx.sh weren't imported into the current environment.

I feel strongly that this is outside of the scope of sh. To be more specific, the goal of sh is to make calling commands like calling functions. What you have described is making python more like bash.


Build Error - Failure to find com.yahoo.ycsb:core:jar:0.1.4 in https://repo.maven.apache.org/maven2

通过 Git 更新至最新版本，在干净的环境下执行：
maven clean package -Dmaven.test.skip=true。
没有通过 Build - 看起来仍然是依赖包com.yahoo.ycsb:core:jar:0.1.4 无法通过 maven2  repository 找到。这个包的地址在 - http://kompics.sics.se/maven/repository/ 上，需要在 POM 里面配置该 repository 才行。
具体的出错信息如下：
[INFO] ledger-rpc ......................................... SUCCESS [  0.215 s]
[INFO] contract-maven-plugin .............................. SUCCESS [  1.729 s]
[INFO] consensus-bftsmart ................................. FAILURE [  0.031 s]
[INFO] consensus-mq ....................................... SKIPPED
[INFO] storage-redis ...................................... SKIPPED
[INFO] storage-rocksdb .................................... SKIPPED
[INFO] storage-composite .................................. SKIPPED
[INFO] gateway ............................................ SKIPPED
[INFO] tools-initializer .................................. SKIPPED
[INFO] peer ............................................... SKIPPED
[INFO] sdk-samples ........................................ SKIPPED
[INFO] tools-keygen-booter ................................ SKIPPED
[INFO] tools-initializer-booter ........................... SKIPPED
[INFO] tools-capability ................................... SKIPPED
[INFO] test ............................................... SKIPPED
[INFO] test-consensus-client .............................. SKIPPED
[INFO] test-consensus-node ................................ SKIPPED
[INFO] test-ledger-core ................................... SKIPPED
[INFO] test-integration ................................... SKIPPED
[INFO] deployment ......................................... SKIPPED
[INFO] deployment-gateway ................................. SKIPPED
[INFO] deployment-peer .................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  18.602 s
[INFO] Finished at: 2019-04-24T13:57:02+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project consensus-bftsmart: Could not resolve dependencies for project com.jd.blockchain:consensus-bftsmart:jar:0.9.0-SNAPSHOT: Failure to find com.yahoo.ycsb:core:jar:0.1.4 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :consensus-bftsmart
TiedangdeMacBook-Pro:source junahan$

You need to compile and install another dependent project  bftsmart, which is located at https://github.com/blockchain-jd-com/bftsmart.
You can found a core.jar file in the lib directory under the bftsmart project root directory, and then, install it to your local maven repository with artifact id “com.yahoo.ycsb:core:jar:0.1.4”.


ARROW-10668: [R] Support for the .data pronoun

and tests for the .env pronoun

Can you add a news bullet for this (making sure you're rebased on latest master)?

Codecov Report

Merging #9051 (39a1d7b) into master (2f68741) will decrease coverage by 0.25%.
The diff coverage is 68.42%.

@@            Coverage Diff             @@
##           master    #9051      +/-   ##
==========================================
- Coverage   82.87%   82.61%   -0.26%
==========================================
  Files         201      202       +1
  Lines       49737    50052     +315
==========================================
+ Hits        41219    41351     +132
- Misses       8518     8701     +183

Impacted Files
Coverage Δ

rust/arrow/src/array/equal/mod.rs
91.94% <ø> (ø)

rust/arrow/src/compute/kernels/take.rs
95.21% <ø> (-0.04%)
:arrow_down:

rust/arrow/src/compute/util.rs
98.93% <ø> (ø)

rust/arrow/src/json/reader.rs
81.49% <ø> (ø)

rust/arrow/src/util/test_util.rs
90.90% <ø> (ø)

rust/datafusion/examples/flight_server.rs
0.00% <0.00%> (ø)

rust/datafusion/src/datasource/csv.rs
81.25% <ø> (ø)

rust/datafusion/src/datasource/datasource.rs
100.00% <ø> (ø)

rust/datafusion/src/datasource/empty.rs
70.58% <ø> (ø)

rust/datafusion/src/logical_plan/builder.rs
88.26% <0.00%> (ø)

... and 48 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e57002...39a1d7b. Read the comment docs.

Passing R CI: https://github.com/jonkeane/arrow/actions/runs/454721137


configuration for a subject prefix for email

Added an option to add a subject prefix to the emails sent by solid_errors. Useful when you have multiple apps/environments using solid errors and what to easily filter the email received

Ok, now the empty prefix is handled.
i also have another idea that fits in the way we use solid_errors:
Make subject_prefix enabled by default, with a value of
[#{Rails.application.name.upcase][#Rails.application.env.upcase].
Disable it with config.solid_errors.subject_prefix = "" or override it with your own custom string
What do you think about that?

Ok, now the empty prefix is handled. i also have another idea that fits in the way we use solid_errors: Make subject_prefix enabled by default, with a value of [#{Rails.application.name.upcase][#Rails.application.env.upcase]. Disable it with config.solid_errors.subject_prefix = "" or override it with your own custom string What do you think about that?

Making this the default would be a breaking change. I would rather introduce such a change slowly. So, let's start by providing that as the example in the README for this feature.

Ok i've renamed the option for conformity with other email settings and updated the README


fix(EU): App ID updates

This can't be merged until stamp app ids are updated too.  Dependent on https://github.com/neoPix/bluelinky-stamps/pull/8.
Also will need some tweaks to notification code.

I just initiated the tests with latest stamps generated and tests are still failing for my EU KIA account.
@ZuinigeRijder , you are saying that we should not update Hyundai APP ID, right?

I just initiated the tests with latest stamps generated and tests are still failing for my EU KIA account. @ZuinigeRijder , you are saying that we should not update Hyundai APP ID, right?

Yes, it works without changing the Hyundai App ID. In the Hyundai Bluelink App on Android, the current App ID is used (I sniffed it). Apparently for IOS the Hyundai App ID is different, but the Hyundai Android App ID works perfectly now.


Need to distinguish sequence alterations that are due to assembly from those that are biological

Hi there,
In the Apollo genome editor we have added the capability for curators to indicate sequence alterations (i.e. substitutions, deletions, and insertions). They have the option of indicating whether the source of the alteration is due to a genuine biological difference, or the source of the alteration is due to an artifact in the base-calling/assembly process. For the first we can use the existing SO types, however, but for the second we have a problem because substitution, deletion, and insertion are all child classes of biological region. What we need are equivalent distinctions (child classes) under base_call_error_correction and/or perhaps assembly_error_correction. It's more likely the former, but it may be possible that the error could arise from the assembly. (so another question is what would be the correct parent class if this isn't known).
Help please!
nota bene 1: In Apollo all features are typed using SO
nota bene 2: We're using the dbSNP definition of SNP which is alterations <=50bp "dbSNP does not accept synthetic mutations or variations ascertained from cross-species alignments and analysis. Variations > 50 nucleotides in length should be submitted to the Database of Genomic Structural Variation (dbVAR)" https://www.ncbi.nlm.nih.gov/books/NBK174586/

These are the changes we propose (terms in red are the new terms we would create):

Term: sequence_alteration_artefact
Definition: A sequence_alteration where the source of the alteration is due to an artefact in the base-calling or assembly process.
synonym: sequence alteration artefact
is_a: SO:0000413 ! sequence_difference
Term: indel_artefact
Definition: An indel that is the result of base-calling or assembly error.
synonym: indel artefact
is_a: sequence_alteration_artefact
Term: deletion_artefact
Definition: A deletion that is the result of base-calling or assembly error.
synonym: deletion artefact
is_a: sequence_alteration_artefact
Term: insertion_artefact
Definition: An insertion that is the result of base-calling or assembly error.
synonym: insertion artefact
is_a: sequence_alteration_artefact
Term: duplication_artefact
Definition: A duplication that is the result of base-calling or assembly error.
Synonym: duplication artefact
is_a: insertion_artefact
Term: substitution_artefact
Definition: A substitution that is the result of base-calling or assembly error.
synonym: substitution artefact
is_a: sequence_alteration_artefact
Term: SNV_artefact
Definition: An SNV that is the result of base-calling or assembly error.
synonym: SNV artefact
is_a: substitution_artefact
Term: MNV_artefact
Definition: An MNV that is the result of base-calling or assembly error.
synonym: MNV artefact
is_a: substitution_artefact
We felt that it would make sense to make sequence_alteration_artefact a child of sequence_difference since a sequence alteration could be annotated as a sequence_alteration_artefact due to a possible_base_call_error or possible_assembly_error. In addition to terms for substitution, deletion, insertion and indel artefacts, we also proposed more specific terms like SNV and MNV artefact. Are these useful? Do you need more specific terms like these for all types of sequence alterations? Please make any suggestions about what we have proposed.

I believe those terms are supposed to use "artifact" (US spelling, which I see is preferred according to Karen's 2005 Genome Biology paper). I'm sure "artifact" looks just as wrong to use as "artefact" does to me ;).
-Terence

If there are no objections to the changes I proposed I will add these in.

Hello, just sticking my head in here for an update. :bowtie:
Dear @nicoleruiz have these changes been incorporated?
Adding @nathandunn: follow this ticket to inform this one from the Apollo tracker.

These changes have not been added yet. I can add these in today if you think the terms, definitions and placement in the ontology make sense. Please let me know if there are any additional terms needed.

Hi @nicoleruiz, these terms do clarify the situation we encountered and will be tremendously useful. I agree with terms, definitions, and placement. I don't see the need for additional terms right now. Thank you!
In the future, I suppose, it will also be useful to establish the parallel between the placement for these and the sequence_feature/sequence_comparison/sequence_alteration classes.
FYI: @nathandunn @deepakunni3 @selewis - you all will be looking at these closely when updating and correcting the editing logic to include variant viz. & annotation for the Apollo issue I referenced earlier.

Eek! One more thing from me: @nicoleruiz I agree with @murphyte regarding the switch from 'artefact' to 'artifact'. :bowtie:
Thanks again!

All of these terms have been added.

SO:0002172 sequence_alteration_artifact
SO:0002174 deletion_artifact
SO:0002173 indel_artifact
SO:0002175 insertion_artifact
SO:0002177 duplication_artifact
SO:0002176 substitution_artifact
SO:0002179 MNV_artifact
SO:0002178 SNV_artifact


Fix privilege acquisition to work better with docker

This problem was noted by people trying to make Traffic Server work in docker without complete privileges.  The current code tries to grab all privileges at once.  If any privilege fails, no privilege is granted.  In the standard --privileged case only the CAP_NET_BIND_SERVICE is granted (of the privileges we care about), and for a basic reverse proxy that is the only capability needed,  but the Traffic Server code would try to get all three of CAP_NET_ADMIN, CAP_NET_BIND_SERVICE, CAP_IPC_LOCK and fail.  So the Traffic Server process running in the docker container could not listen on low ports.
This code change attempts to get the capabilities one at a time so that capabilities that can be granted will be granted.

[approve ci]

Cherry picked to 8.1.0

This is also needed when service hardening is performed via systemd, this commit was necessary to be able to bind on port 443 with the following systemd unit:
[Service]
ExecStartPre=/usr/local/sbin/update-ocsp-all
ExecStart=
ExecStart=/usr/bin/traffic_manager --run-root=/srv/trafficserver/tls --nosyslog
Restart=always
RestartSec=1
ExecReload=
ExecReload=/usr/bin/traffic_ctl --run-root=/srv/trafficserver/tls config reload
SuccessExitStatus=15

LimitNOFILE=500000
LimitMEMLOCK=90000

# Security options
ProtectKernelModules=yes
ProtectKernelTunables=yes
PrivateTmp=yes

RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK

CapabilityBoundingSet=CAP_DAC_OVERRIDE CAP_SETGID CAP_SETUID CAP_SYS_PTRACE CAP_FOWNER CAP_NET_BIND_SERVICE
SystemCallFilter=~@keyring @clock @cpu-emulation @obsolete @module @raw-io

# The entire file system hierarchy is mounted read-only, except for the API
# file system subtrees /dev, /proc and /sys
ProtectSystem=strict


chore: backport fixes

these 2 patches are good to have in v3 too

had to refactor the hmr one slightly due to the changes in v4 in the options, but spirit is identical


When reviewing an existing export it would be good to easily see the features selected for export

When I look at an existing export, I don't see any way to see what features are selected. In the few exports I made, there is a "Josm presets.xml" file that I think lists what features are selected, but not all exports have that file.
That would really be helpful as we want to encourage people to re-use existing exports I would think.

There are 258 possible feature selections in the HDM model, so it's a question of how to display this without cluttering the UI....?

I was thinking the same "tree" view (with expand and collapse all options) just not a selectable tree view, basically display only.

Another simpler option is just a text list that gets written to a file, basically a simplified version of the .xml file that gets generated for presets. And it gets listed as one of the file types to download, but it is for informational purposes only. But I really think we need to see what is selected on a job to know if we can reuse it or not.
So I think the tree list view is good, the text file is good, but if either of those are too complicated, how about at least letting us know if the all "HDM Model" or all "OSM Model" options were chosen, as I bet one of those "every item" will be the choice for the vast majority of export jobs.

Perhaps we could have the feature tags selected from the tree saved by default as a 'Custom JOSM Preset (XML)' file provided along with the other export file formats, so the user can check the features from there before re-running, cloning, etc.

Brian has added a button (Features) at the bottom of the Export Summary page, which when clicked lists all the features selected on the tree tag. This is simply a duplicate of the tree selection during the create export phase.


[10.x] Cache ViewName::normalize result for performance

When lots for namespaced views are used repetitively this can end up contributing to a significant chunk of time.
We've seen cases where 17k+ views have been loaded and this has totalled 2s+ of runtime.
The result is constant so no risk is involved.

Hi there. Performance improvement PRs without benchmarks are usually closed. Please provide some detailed benchmarks of before/after.


⚠ Bofrost.lu has degraded performance

In a238fd7, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1530 ms

Resolved: Bofrost.lu performance has improved in 8e20300.