unable to reach from a pod to its hosting kube-node in kubernetes cluster with weave CNI

What you expected to happen?
ping from the pod to its host
What happened?
I have deployed metric server pod on kubernetes cluster which is able to ping all the nods (though collect their metrics) except for the node where the pod is deployed.
also tried kube-state-metrics and get the same result.
$kubectl top node" output
NAME    CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
r2s12   502m         6%     2910Mi          12%
r2s14   111m         1%     1872Mi          23%
r2s13
Anything else we need to know?
kubeadm is used to create the cluster.
below are my nodes:
NAME STATUS ROLES AGE VERSION INTERNAL-IP
node/r2s12 Ready master 7d19h v1.12.1 <IP_ADDRESS>
node/r2s13 Ready 7d19h v1.12.1 <IP_ADDRESS>
node/r2s14 Ready 7d19h v1.12.1 <IP_ADDRESS>
Versions:

$ weave version
root@r2s12:~# weave version
This is WEAVE, Version 4.4 (TeX Live 2015/Debian)
weave: fatal: web file `version.web' not found.

kernal: ubuntu 16.04

$ docker version
Client:
Version: 18.06.0-ce
API version: 1.38
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:11:02 2018
OS/Arch: linux/amd64
Experimental: false

Server:
Engine:
Version: 18.06.0-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:09:05 2018
OS/Arch: linux/amd64
Experimental: false

$ kubectl version
v1.12.1

Logs:
metric server log:

E1122 11:21:31.537391       1 manager.go:102] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:r2s13: unable to fetch metrics from Kubelet r2s13 (<IP_ADDRESS>): Get https://<IP_ADDRESS>:10250/stats/summary/: dial tcp <IP_ADDRESS>:10250: i/o timeout

or, if using Kubernetes:
$ kubectl logs -n kube-system <weave-net-pod> weave

WARN: 2018/11/21 08:06:43.109796 Vetoed installation of hairpin flow FlowSpec{keys: [EthernetFlowKey{src: ae:9a:41:9a:13:07, dst: f6:f7:3a:0c:0d:e4} InPortFlowKey{vport: 1}], actions: [OutputAction{vport: 1}]}
INFO: 2018/11/21 08:56:08.935426 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave
ERRO: 2018/11/21 14:10:39.476161 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (82:45:c0:72:1b:5e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/21 14:28:28.193166 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=fastdp&os=linux&signature=ti5mHxx4N2jhWg%2FW145Y%2Fy%2Fc6s99JbLsGvwbKbYWSyA%3D&version=2.4.1: dial tcp [2607:f8b0:4020:804::2013]:443: connect: network is unreachable
INFO: 2018/11/21 19:29:37.801866 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave
INFO: 2018/11/22 02:11:04.950606 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=fastdp&os=linux&signature=ti5mHxx4N2jhWg%2FW145Y%2Fy%2Fc6s99JbLsGvwbKbYWSyA%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:37146-><IP_ADDRESS>:53: i/o timeout
ERRO: 2018/11/22 06:49:13.099669 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (82:45:c0:72:1b:5e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/22 06:49:13.099760 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (82:45:c0:72:1b:5e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/22 07:16:50.985112 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (82:45:c0:72:1b:5e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/22 07:46:57.978057 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=sleeve&os=linux&signature=ti5mHxx4N2jhWg%2FW145Y%2Fy%2Fc6s99JbLsGvwbKbYWSyA%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:42955-><IP_ADDRESS>:53: i/o timeout
ERRO: 2018/11/22 09:35:58.458456 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (66:fa:18:34:40:83) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/22 09:37:14.782819 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (82:45:c0:72:1b:5e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/22 10:06:23.628020 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (82:45:c0:72:1b:5e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)

-->
Network:

$ ip route
<IP_ADDRESS>/12 dev weave  proto kernel  scope link  src <IP_ADDRESS>
<IP_ADDRESS>/26 dev enp5s4f0  proto kernel  scope link  src <IP_ADDRESS>
<IP_ADDRESS>/24 dev enp5s4f1  proto kernel  scope link  src <IP_ADDRESS>
<IP_ADDRESS>/16 dev docker0  proto kernel  scope link  src <IP_ADDRESS> linkdown

$ ip -4 -o addr
1: lo    inet <IP_ADDRESS>/8 scope host lo\       valid_lft forever preferred_lft forever
4: enp5s4f0    inet <IP_ADDRESS>/26 brd <IP_ADDRESS> scope global enp5s4f0\       valid_lft forever preferred_lft forever
5: enp5s4f1    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global enp5s4f1\       valid_lft forever preferred_lft forever
7: docker0    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global docker0\       valid_lft forever preferred_lft forever
10: weave    inet <IP_ADDRESS>/12 brd <IP_ADDRESS> scope global weave\       valid_lft forever preferred_lft forever

$ sudo iptables-save
# Generated by iptables-save v1.6.0 on Thu Nov 22 13:50:30 2018
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [35:2116]
:POSTROUTING ACCEPT [35:2116]
:DOCKER - [0:0]
:KUBE-MARK-DROP - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-SEP-36FTFJ6TMPCA3MOB - [0:0]
:KUBE-SEP-4V43NLPFJ2DWVK47 - [0:0]
:KUBE-SEP-4WSWFP6HLKGHXKOX - [0:0]
:KUBE-SEP-BMXOE6ZM4WJKSWR6 - [0:0]
:KUBE-SEP-BO2XIYXA55BODR2G - [0:0]
:KUBE-SEP-HBWNXSVBCYIMDOO2 - [0:0]
:KUBE-SEP-HXAB4K5SNE6SOUPY - [0:0]
:KUBE-SEP-JJGE5N6MW2ULLL3L - [0:0]
:KUBE-SEP-JYF4OJ6RSULE4YI6 - [0:0]
:KUBE-SEP-NYOZ5QVVOSD6V4IJ - [0:0]
:KUBE-SEP-OTJ7N2WJ7UGHPRZJ - [0:0]
:KUBE-SEP-PGKBHR7HMQAUHTBJ - [0:0]
:KUBE-SEP-QUGN2CXVUFGE7ELS - [0:0]
:KUBE-SEP-R6L5B3XQ6KJYXB7F - [0:0]
:KUBE-SEP-T5E2YGIFQGNDZUEU - [0:0]
:KUBE-SEP-WRKT4RAHACZUDFS2 - [0:0]
:KUBE-SEP-XKMU5ICJFX5QWQ5D - [0:0]
:KUBE-SEP-XOJ56UW5BYLJRW4Q - [0:0]
:KUBE-SEP-ZAGNJ67UWBNPO4HB - [0:0]
:KUBE-SEP-ZYMC55Z5IBS3BW6M - [0:0]
:KUBE-SERVICES - [0:0]
:KUBE-SVC-7JSRZHB3YWVS5NMG - [0:0]
:KUBE-SVC-AWA2CQSXVI7X2GE5 - [0:0]
:KUBE-SVC-BT6UIQGWTIPVJ3V7 - [0:0]
:KUBE-SVC-CK3M5ECVAI6WWQM4 - [0:0]
:KUBE-SVC-CNCAHLOUVJ7PHAAI - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-GRVIJZ6QHJZF73YT - [0:0]
:KUBE-SVC-IFO32E4YIRUTZPGJ - [0:0]
:KUBE-SVC-LC5QY66VUV2HJ6WZ - [0:0]
:KUBE-SVC-MOX62VNPUJXMPGLS - [0:0]
:KUBE-SVC-NAZP4SD6XLP35COK - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:KUBE-SVC-XGLOHA7QRQ3V22RZ - [0:0]
:WEAVE - [0:0]
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT ! -d <IP_ADDRESS>/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A POSTROUTING -s <IP_ADDRESS>/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -j WEAVE
-A DOCKER -i docker0 -j RETURN
-A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-NODEPORTS -p tcp -m comment --comment "monitoring/grafana:http" -m tcp --dport 30080 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "monitoring/grafana:http" -m tcp --dport 30080 -j KUBE-SVC-AWA2CQSXVI7X2GE5
-A KUBE-NODEPORTS -p tcp -m comment --comment "monitoring/prometheus-k8s:web" -m tcp --dport 32574 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "monitoring/prometheus-k8s:web" -m tcp --dport 32574 -j KUBE-SVC-IFO32E4YIRUTZPGJ
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/kube-ops-view:" -m tcp --dport 31710 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/kube-ops-view:" -m tcp --dport 31710 -j KUBE-SVC-MOX62VNPUJXMPGLS
-A KUBE-NODEPORTS -p tcp -m comment --comment "monitoring/alertmanager-main:web" -m tcp --dport 31315 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "monitoring/alertmanager-main:web" -m tcp --dport 31315 -j KUBE-SVC-NAZP4SD6XLP35COK
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE
-A KUBE-SEP-36FTFJ6TMPCA3MOB -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-36FTFJ6TMPCA3MOB -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:8080
-A KUBE-SEP-4V43NLPFJ2DWVK47 -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-4V43NLPFJ2DWVK47 -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:8443
-A KUBE-SEP-4WSWFP6HLKGHXKOX -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-4WSWFP6HLKGHXKOX -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:80
-A KUBE-SEP-BMXOE6ZM4WJKSWR6 -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-BMXOE6ZM4WJKSWR6 -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:443
-A KUBE-SEP-BO2XIYXA55BODR2G -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-BO2XIYXA55BODR2G -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:8080
-A KUBE-SEP-HBWNXSVBCYIMDOO2 -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-HBWNXSVBCYIMDOO2 -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:9090
-A KUBE-SEP-HXAB4K5SNE6SOUPY -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-HXAB4K5SNE6SOUPY -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:80
-A KUBE-SEP-JJGE5N6MW2ULLL3L -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-JJGE5N6MW2ULLL3L -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:6379
-A KUBE-SEP-JYF4OJ6RSULE4YI6 -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-JYF4OJ6RSULE4YI6 -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:9093
-A KUBE-SEP-NYOZ5QVVOSD6V4IJ -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-NYOZ5QVVOSD6V4IJ -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:443
-A KUBE-SEP-OTJ7N2WJ7UGHPRZJ -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-OTJ7N2WJ7UGHPRZJ -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:53
-A KUBE-SEP-PGKBHR7HMQAUHTBJ -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-PGKBHR7HMQAUHTBJ -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:6443
-A KUBE-SEP-QUGN2CXVUFGE7ELS -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-QUGN2CXVUFGE7ELS -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:443
-A KUBE-SEP-R6L5B3XQ6KJYXB7F -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-R6L5B3XQ6KJYXB7F -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:9093
-A KUBE-SEP-T5E2YGIFQGNDZUEU -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-T5E2YGIFQGNDZUEU -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:8080
-A KUBE-SEP-WRKT4RAHACZUDFS2 -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-WRKT4RAHACZUDFS2 -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:9090
-A KUBE-SEP-XKMU5ICJFX5QWQ5D -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-XKMU5ICJFX5QWQ5D -p udp -m udp -j DNAT --to-destination <IP_ADDRESS>:53
-A KUBE-SEP-XOJ56UW5BYLJRW4Q -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-XOJ56UW5BYLJRW4Q -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:9093
-A KUBE-SEP-ZAGNJ67UWBNPO4HB -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-ZAGNJ67UWBNPO4HB -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:3000
-A KUBE-SEP-ZYMC55Z5IBS3BW6M -s <IP_ADDRESS>/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-ZYMC55Z5IBS3BW6M -p tcp -m tcp -j DNAT --to-destination <IP_ADDRESS>:6443
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/traefik-ingress-service:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/traefik-ingress-service:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-CNCAHLOUVJ7PHAAI
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kube-ops-view-redis: cluster IP" -m tcp --dport 6379 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kube-ops-view-redis: cluster IP" -m tcp --dport 6379 -j KUBE-SVC-BT6UIQGWTIPVJ3V7
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/kubernetes-dashboard: cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/kubernetes-dashboard: cluster IP" -m tcp --dport 443 -j KUBE-SVC-XGLOHA7QRQ3V22RZ
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/traefik-ingress-service:web cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/traefik-ingress-service:web cluster IP" -m tcp --dport 80 -j KUBE-SVC-7JSRZHB3YWVS5NMG
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/grafana:http cluster IP" -m tcp --dport 3000 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/grafana:http cluster IP" -m tcp --dport 3000 -j KUBE-SVC-AWA2CQSXVI7X2GE5
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/prometheus-k8s:web cluster IP" -m tcp --dport 9090 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/prometheus-k8s:web cluster IP" -m tcp --dport 9090 -j KUBE-SVC-IFO32E4YIRUTZPGJ
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kube-ops-view: cluster IP" -m tcp --dport 82 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kube-ops-view: cluster IP" -m tcp --dport 82 -j KUBE-SVC-MOX62VNPUJXMPGLS
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/alertmanager-main:web cluster IP" -m tcp --dport 9093 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/alertmanager-main:web cluster IP" -m tcp --dport 9093 -j KUBE-SVC-NAZP4SD6XLP35COK
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/prometheus-adapter:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "monitoring/prometheus-adapter:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-GRVIJZ6QHJZF73YT
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/metrics-server: cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/metrics-server: cluster IP" -m tcp --dport 443 -j KUBE-SVC-LC5QY66VUV2HJ6WZ
-A KUBE-SERVICES ! -s <IP_ADDRESS>/16 -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/traefik-ingress-service:admin cluster IP" -m tcp --dport 8080 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d <IP_ADDRESS>/32 -p tcp -m comment --comment "kube-system/traefik-ingress-service:admin cluster IP" -m tcp --dport 8080 -j KUBE-SVC-CK3M5ECVAI6WWQM4
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
-A KUBE-SVC-7JSRZHB3YWVS5NMG -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-4WSWFP6HLKGHXKOX
-A KUBE-SVC-7JSRZHB3YWVS5NMG -j KUBE-SEP-HXAB4K5SNE6SOUPY
-A KUBE-SVC-AWA2CQSXVI7X2GE5 -j KUBE-SEP-ZAGNJ67UWBNPO4HB
-A KUBE-SVC-BT6UIQGWTIPVJ3V7 -j KUBE-SEP-JJGE5N6MW2ULLL3L
-A KUBE-SVC-CK3M5ECVAI6WWQM4 -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-36FTFJ6TMPCA3MOB
-A KUBE-SVC-CK3M5ECVAI6WWQM4 -j KUBE-SEP-T5E2YGIFQGNDZUEU
-A KUBE-SVC-CNCAHLOUVJ7PHAAI -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-BMXOE6ZM4WJKSWR6
-A KUBE-SVC-CNCAHLOUVJ7PHAAI -j KUBE-SEP-QUGN2CXVUFGE7ELS
-A KUBE-SVC-ERIFXISQEP7F7OF4 -j KUBE-SEP-OTJ7N2WJ7UGHPRZJ
-A KUBE-SVC-GRVIJZ6QHJZF73YT -j KUBE-SEP-ZYMC55Z5IBS3BW6M
-A KUBE-SVC-IFO32E4YIRUTZPGJ -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-WRKT4RAHACZUDFS2
-A KUBE-SVC-IFO32E4YIRUTZPGJ -j KUBE-SEP-HBWNXSVBCYIMDOO2
-A KUBE-SVC-LC5QY66VUV2HJ6WZ -j KUBE-SEP-NYOZ5QVVOSD6V4IJ
-A KUBE-SVC-MOX62VNPUJXMPGLS -j KUBE-SEP-BO2XIYXA55BODR2G
-A KUBE-SVC-NAZP4SD6XLP35COK -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-JYF4OJ6RSULE4YI6
-A KUBE-SVC-NAZP4SD6XLP35COK -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-R6L5B3XQ6KJYXB7F
-A KUBE-SVC-NAZP4SD6XLP35COK -j KUBE-SEP-XOJ56UW5BYLJRW4Q
-A KUBE-SVC-NPX46M4PTMTKRN6Y -j KUBE-SEP-PGKBHR7HMQAUHTBJ
-A KUBE-SVC-TCOU7JCQXEZGVUNU -j KUBE-SEP-XKMU5ICJFX5QWQ5D
-A KUBE-SVC-XGLOHA7QRQ3V22RZ -j KUBE-SEP-4V43NLPFJ2DWVK47
-A WEAVE -s <IP_ADDRESS>/12 -d <IP_ADDRESS>/4 -j RETURN
-A WEAVE ! -s <IP_ADDRESS>/12 -d <IP_ADDRESS>/12 -j MASQUERADE
-A WEAVE -s <IP_ADDRESS>/12 ! -d <IP_ADDRESS>/12 -j MASQUERADE
COMMIT
# Completed on Thu Nov 22 13:50:30 2018
# Generated by iptables-save v1.6.0 on Thu Nov 22 13:50:30 2018
*filter
:INPUT ACCEPT [5796:4095012]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [6202:5908455]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:KUBE-EXTERNAL-SERVICES - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-FORWARD - [0:0]
:KUBE-SERVICES - [0:0]
:WEAVE-NPC - [0:0]
:WEAVE-NPC-DEFAULT - [0:0]
:WEAVE-NPC-EGRESS - [0:0]
:WEAVE-NPC-EGRESS-ACCEPT - [0:0]
:WEAVE-NPC-EGRESS-CUSTOM - [0:0]
:WEAVE-NPC-EGRESS-DEFAULT - [0:0]
:WEAVE-NPC-INGRESS - [0:0]
-A INPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes externally-visible service portals" -j KUBE-EXTERNAL-SERVICES
-A INPUT -j KUBE-FIREWALL
-A INPUT -i weave -j WEAVE-NPC-EGRESS
-A FORWARD -m comment --comment "kubernetes forwarding rules" -j KUBE-FORWARD
-A FORWARD -i weave -m comment --comment "NOTE: this must go before \'-j KUBE-FORWARD\'" -j WEAVE-NPC-EGRESS
-A FORWARD -o weave -m comment --comment "NOTE: this must go before \'-j KUBE-FORWARD\'" -j WEAVE-NPC
-A FORWARD -o weave -m state --state NEW -j NFLOG --nflog-group 86
-A FORWARD -o weave -j DROP
-A FORWARD -i weave ! -o weave -j ACCEPT
-A FORWARD -o weave -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -j KUBE-FIREWALL
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
-A KUBE-FORWARD -s <IP_ADDRESS>/16 -m comment --comment "kubernetes forwarding conntrack pod source rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-FORWARD -d <IP_ADDRESS>/16 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A WEAVE-NPC -m state --state RELATED,ESTABLISHED -j ACCEPT
-A WEAVE-NPC -d <IP_ADDRESS>/4 -j ACCEPT
-A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-DEFAULT
-A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
-A WEAVE-NPC -m set ! --match-set weave-local-pods dst -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-;rGqyMIl1HN^cfDki~Z$3]6!N dst -m comment --comment "DefaultAllow ingress isolation for namespace: default" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-Rzff}h:=]JaaJl/G;(XJpGjZ[ dst -m comment --comment "DefaultAllow ingress isolation for namespace: kube-public" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-P.B|!ZhkAr5q=XZ?3}tMBA+0 dst -m comment --comment "DefaultAllow ingress isolation for namespace: kube-system" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-tMk6Dh2:+Aq9sz3GtyPLSgQGO dst -m comment --comment "DefaultAllow ingress isolation for namespace: monitoring" -j ACCEPT
-A WEAVE-NPC-EGRESS -m state --state RELATED,ESTABLISHED -j ACCEPT
-A WEAVE-NPC-EGRESS -m state --state NEW -m set ! --match-set weave-local-pods src -j RETURN
-A WEAVE-NPC-EGRESS -d <IP_ADDRESS>/4 -j RETURN
-A WEAVE-NPC-EGRESS -m state --state NEW -j WEAVE-NPC-EGRESS-DEFAULT
-A WEAVE-NPC-EGRESS -m state --state NEW -m mark ! --mark 0x40000/0x40000 -j WEAVE-NPC-EGRESS-CUSTOM
-A WEAVE-NPC-EGRESS -m state --state NEW -m mark ! --mark 0x40000/0x40000 -j NFLOG --nflog-group 86
-A WEAVE-NPC-EGRESS -m mark ! --mark 0x40000/0x40000 -j DROP
-A WEAVE-NPC-EGRESS-ACCEPT -j MARK --set-xmark 0x40000/0x40000
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-s_+ChJId4Uy_$}G;WdH|~TK)I src -m comment --comment "DefaultAllow egress isolation for namespace: default" -j WEAVE-NPC-EGRESS-ACCEPT
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-s_+ChJId4Uy_$}G;WdH|~TK)I src -m comment --comment "DefaultAllow egress isolation for namespace: default" -j RETURN
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-41s)5vQ^o/xWGz6a20N:~?#|E src -m comment --comment "DefaultAllow egress isolation for namespace: kube-public" -j WEAVE-NPC-EGRESS-ACCEPT
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-41s)5vQ^o/xWGz6a20N:~?#|E src -m comment --comment "DefaultAllow egress isolation for namespace: kube-public" -j RETURN
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-E1ney4o[ojNrLk.6rOHi;7MPE src -m comment --comment "DefaultAllow egress isolation for namespace: kube-system" -j WEAVE-NPC-EGRESS-ACCEPT
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-E1ney4o[ojNrLk.6rOHi;7MPE src -m comment --comment "DefaultAllow egress isolation for namespace: kube-system" -j RETURN
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-cKT4=+;yzwiz8x@;C{fjHV6$0 src -m comment --comment "DefaultAllow egress isolation for namespace: monitoring" -j WEAVE-NPC-EGRESS-ACCEPT
-A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set weave-cKT4=+;yzwiz8x@;C{fjHV6$0 src -m comment --comment "DefaultAllow egress isolation for namespace: monitoring" -j RETURN

@hshtel is there a kubelet serving on <IP_ADDRESS>:10250? Is it accessible from other nodes?
You observe same behaviour if  metrics aggregation pod runs in different node as well?

@murali-reddy yes there is a kubelet serving on this and actually I have other nodes running on it such Kubernetes-Dashboard. below is netstat -an | grep 10250 output on r2s13:
tcp6       0      0 :::10250                :::*                    LISTEN
tcp6       0      0 <IP_ADDRESS>:10250    <IP_ADDRESS>:59684    ESTABLISHED
tcp6       0      0 <IP_ADDRESS>:10250    <IP_ADDRESS>:50776    ESTABLISHED
tcp6       0      0 <IP_ADDRESS>:10250    <IP_ADDRESS>:50508    ESTABLISHED
I can ping between these nodes, and I can ping from inside the metric pod to the other nodes (r2s14 and r2s12).
the same behavior occurred when I was running this pod on r2s14.

@hshtel I see exact problem and symptoms reported by you in #3434 as well. If its same issues please close one of the issue.
Are there any network policies applied? Please see in the weave-npc logs if there is any log entry mentioning of dropping of the packets.
Are you able to ping <IP_ADDRESS> from metrics pod?

I have closed the mentioned problem #3434.
I didn't configure any network policies, just installed the weave.
I am not able to ping <IP_ADDRESS> from metrics pod, but I can ping <IP_ADDRESS> and <IP_ADDRESS> from it.
below is a repeated error from weave-npc log, I don't know if it could be related to this problem:
DEBU: 2018/11/23 17:06:08.795615 EVENT UpdatePod {"metadata":{"creationTimestamp":"2018-10-24T11:56:59Z","generateName":"coredns-745bf547c4-","labels":{"k8s-app":"kube-dns","pod-template-hash":"745bf547c4"},"name":"coredns-745bf547c4-grv6s","namespace":"kube-system","resourceVersion":"4524675","selfLink":"/api/v1/namespaces/kube-system/pods/coredns-745bf547c4-grv6s","uid":"e9a3037b-d783-11e8-a2ac-00259e1e2b8c"},"spec":{"containers":[{"image":"k8s.gcr.io/coredns:1.2.2","imagePullPolicy":"IfNotPresent","name":"coredns","ports":[{"containerPort":53,"name":"dns","protocol":"UDP"},{"containerPort":53,"name":"dns-tcp","protocol":"TCP"},{"containerPort":9153,"name":"metrics","protocol":"TCP"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File"}],"dnsPolicy":"Default","nodeName":"r2s13","priority":0,"restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{},"serviceAccount":"coredns","serviceAccountName":"coredns","terminationGracePeriodSeconds":30},"status":{"conditions":[{"lastProbeTime":null,"lastTransitionTime":"2018-10-24T11:57:19Z","status":"True","type":"Initialized"},{"lastProbeTime":null,"lastTransitionTime":"2018-11-23T17:06:00Z","message":"containers with unready status: [coredns]","reason":"ContainersNotReady","status":"False","type":"Ready"},{"lastProbeTime":null,"lastTransitionTime":"2018-11-23T17:06:00Z","message":"containers with unready status: [coredns]","reason":"ContainersNotReady","status":"False","type":"ContainersReady"},{"lastProbeTime":null,"lastTransitionTime":"2018-10-24T11:56:59Z","status":"True","type":"PodScheduled"}],"hostIP":"<IP_ADDRESS>","phase":"Running","podIP":"<IP_ADDRESS>","qosClass":"Burstable","startTime":"2018-10-24T11:57:19Z"}} {"metadata":{"creationTimestamp":"2018-10-24T11:56:59Z","generateName":"coredns-745bf547c4-","labels":{"k8s-app":"kube-dns","pod-template-hash":"745bf547c4"},"name":"coredns-745bf547c4-grv6s","namespace":"kube-system","resourceVersion":"4524691","selfLink":"/api/v1/namespaces/kube-system/pods/coredns-745bf547c4-grv6s","uid":"e9a3037b-d783-11e8-a2ac-00259e1e2b8c"},"spec":{"containers":[{"image":"k8s.gcr.io/coredns:1.2.2","imagePullPolicy":"IfNotPresent","name":"coredns","ports":[{"containerPort":53,"name":"dns","protocol":"UDP"},{"containerPort":53,"name":"dns-tcp","protocol":"TCP"},{"containerPort":9153,"name":"metrics","protocol":"TCP"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File"}],"dnsPolicy":"Default","nodeName":"r2s13","priority":0,"restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{},"serviceAccount":"coredns","serviceAccountName":"coredns","terminationGracePeriodSeconds":30},"status":{"conditions":[{"lastProbeTime":null,"lastTransitionTime":"2018-10-24T11:57:19Z","status":"True","type":"Initialized"},{"lastProbeTime":null,"lastTransitionTime":"2018-11-23T17:06:00Z","message":"containers with unready status: [coredns]","reason":"ContainersNotReady","status":"False","type":"Ready"},{"lastProbeTime":null,"lastTransitionTime":"2018-11-23T17:06:00Z","message":"containers with unready status: [coredns]","reason":"ContainersNotReady","status":"False","type":"ContainersReady"},{"lastProbeTime":null,"lastTransitionTime":"2018-10-24T11:56:59Z","status":"True","type":"PodScheduled"}],"hostIP":"<IP_ADDRESS>","phase":"Running","podIP":"<IP_ADDRESS>","qosClass":"Burstable","startTime":"2018-10-24T11:57:19Z"}}

@hshtel that is not a weave-npc error; just reporting an update received from Kubernetes.
It does appear to say that kube-dns is not ready, which will cause other problems.
Perhaps if you do kubectl -n kube-system describe pod coredns-745bf547c4-grv6s it will tell you more.

@bboreham please notice that only one of the two coredns pod is restarting, and when I hit describe for this pod I get below event "NetworkPlugin cni failed to set up pod "coredns-d5dd54cbf-jxzhc_kube-system" network: unable to allocate IP address: Post". could it be the weave the reason for this problem?
kubectl describe pod coredns-d5dd54cbf-fzhzp -n kube-system

Events:
  Type     Reason                  Age                    From               Message
  ----     ------                  ----                   ----               -------
  Normal   Scheduled               58m                    default-scheduler  Successfully assigned kube-system/coredns-d5dd54cbf-jxzhc to r2s12
  Normal   Created                 53m (x3 over 58m)      kubelet, r2s12     Created container
  Normal   Started                 53m (x3 over 58m)      kubelet, r2s12     Started container
  Normal   Killing                 53m (x2 over 55m)      kubelet, r2s12     Killing container with id docker://coredns:Container failed liveness probe.. Container will be killed and recreated.
  Normal   Pulled                  53m (x3 over 58m)      kubelet, r2s12     Container image "k8s.gcr.io/coredns:1.2.2" already present on machine
  Warning  Unhealthy               17m (x58 over 56m)     kubelet, r2s12     Liveness probe failed: HTTP probe failed with statuscode: 503
  Warning  BackOff                 13m (x73 over 41m)     kubelet, r2s12     Back-off restarting failed container
  Warning  FailedCreatePodSandBox  8m2s                   kubelet, r2s12     Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "cd6a070327f1eee11727e424f57977b0211a1ae2933ff11bda8be714b3332988" network for pod "coredns-d5dd54cbf-jxzhc": NetworkPlugin cni failed to set up pod "coredns-d5dd54cbf-jxzhc_kube-system" network: unable to allocate IP address: Post http://<IP_ADDRESS>:6784/ip/cd6a070327f1eee11727e424f57977b0211a1ae2933ff11bda8be714b3332988: dial tcp <IP_ADDRESS>:6784: connect: connection refused

Yes, it's possible.  Please post the full logs of the weave-net pod on that node.

I will post the log of all weave containers:
weave-net-5fb7n on r2s12:
root@r2s12:~# kubectl logs weave-net-5fb7n -n kube-system -c weave
DEBU: 2018/11/25 11:59:30.203596 [kube-peers] Checking peer "0a:3a:30:bc:0c:e2" against list &{[{f6:b0:8c:c6:13:40 r2s14} {ae:9a:41:9a:13:07 r2s13} {0a:3a:30:bc:0c:e2 r2s12}]}
INFO: 2018/11/25 11:59:30.462430 Command line options: map[db-prefix:/weavedb/weave-net docker-api: ipalloc-range:<IP_ADDRESS>/12 metrics-addr:<IP_ADDRESS>:6782 no-dns:true port:6783 conn-limit:100 datapath:datapath expect-npc:true http-addr:<IP_ADDRESS>:6784 ipalloc-init:consensus=3 nickname:r2s12 host-root:/host name:0a:3a:30:bc:0c:e2]
INFO: 2018/11/25 11:59:30.462515 weave  2.4.1
INFO: 2018/11/25 11:59:30.573333 Re-exposing <IP_ADDRESS>/12 on bridge "weave"
INFO: 2018/11/25 11:59:30.600523 Bridge type is bridged_fastdp
INFO: 2018/11/25 11:59:30.600560 Communication between peers is unencrypted.
INFO: 2018/11/25 11:59:30.662749 Our name is 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 11:59:30.662812 Launch detected - using supplied peer list: [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
INFO: 2018/11/25 11:59:30.662850 Checking for pre-existing addresses on weave bridge
INFO: 2018/11/25 11:59:30.663031 weave bridge has address <IP_ADDRESS>/12
INFO: 2018/11/25 11:59:30.689547 [allocator 0a:3a:30:bc:0c:e2] Initialising with persisted data
INFO: 2018/11/25 11:59:30.689682 Sniffing traffic on datapath (via ODP)
INFO: 2018/11/25 11:59:30.690153 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 11:59:30.690469 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 11:59:30.690634 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 11:59:30.692412 ->[<IP_ADDRESS>:60910] connection accepted
INFO: 2018/11/25 11:59:30.693349 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection ready; using protocol version 2
INFO: 2018/11/25 11:59:30.693612 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/25 11:59:30.693684 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection added (new peer)
INFO: 2018/11/25 11:59:30.693720 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection ready; using protocol version 2
INFO: 2018/11/25 11:59:30.693753 ->[<IP_ADDRESS>:60910|0a:3a:30:bc:0c:e2(r2s12)]: connection shutting down due to error: cannot connect to ourself
INFO: 2018/11/25 11:59:30.693917 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using fastdp
INFO: 2018/11/25 11:59:30.694055 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection added (new peer)
INFO: 2018/11/25 11:59:30.695443 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection shutting down due to error: cannot connect to ourself
INFO: 2018/11/25 11:59:30.702566 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection fully established
INFO: 2018/11/25 11:59:30.703039 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 11:59:30.708581 sleeve ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: Effective MTU verified at 1438
INFO: 2018/11/25 11:59:30.725209 Listening for HTTP control messages on <IP_ADDRESS>:6784
INFO: 2018/11/25 11:59:30.725915 Listening for metrics requests on <IP_ADDRESS>:6782
INFO: 2018/11/25 11:59:31.025688 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 11:59:31.195354 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using sleeve
INFO: 2018/11/25 11:59:31.195410 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection fully established
INFO: 2018/11/25 11:59:31.195724 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 11:59:31.200918 sleeve ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: Effective MTU verified at 1438
INFO: 2018/11/25 11:59:31.400268 [kube-peers] Added myself to peer list &{[{f6:b0:8c:c6:13:40 r2s14} {ae:9a:41:9a:13:07 r2s13} {0a:3a:30:bc:0c:e2 r2s12}]}
DEBU: 2018/11/25 11:59:31.411206 [kube-peers] Nodes that have disappeared: map[]
<IP_ADDRESS>
INFO: 2018/11/25 11:59:31.504239 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave
INFO: 2018/11/25 11:59:31.695958 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using fastdp
INFO: 2018/11/25 11:59:40.812540 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 11:59:41.640670 Discovered remote MAC 1e:e6:50:8e:90:c7 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:01:57.393179 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:44174-><IP_ADDRESS>:6783: read: connection reset by peer
INFO: 2018/11/25 12:01:57.393365 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection deleted
INFO: 2018/11/25 12:01:57.394276 Removed unreachable peer ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:01:57.394546 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:01:57.394989 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:01:57.607997 Discovered remote MAC 86:d2:8d:d2:51:40 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:01:58.431993 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:01:58.432515 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:01:59.292672 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:01:59.292820 Discovered remote MAC 46:d2:1a:4c:d1:bc at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:02:02.925907 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:02:02.926283 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:02:06.923411 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:02:06.923823 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:02:15.909753 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:02:15.910132 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:02:25.337165 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:02:25.337602 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:02:45.790827 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:02:45.791322 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:03:05.871267 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:03:05.871801 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:03:26.666518 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:03:26.667033 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:03:36.410833 ->[<IP_ADDRESS>:47566] connection accepted
INFO: 2018/11/25 12:03:36.411955 ->[<IP_ADDRESS>:47566|ae:9a:41:9a:13:07(r2s13)]: connection ready; using protocol version 2
INFO: 2018/11/25 12:03:36.412140 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/25 12:03:36.412188 ->[<IP_ADDRESS>:47566|ae:9a:41:9a:13:07(r2s13)]: connection added (new peer)
INFO: 2018/11/25 12:03:36.415715 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 12:03:36.415806 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using sleeve
INFO: 2018/11/25 12:03:36.415840 ->[<IP_ADDRESS>:47566|ae:9a:41:9a:13:07(r2s13)]: connection fully established
INFO: 2018/11/25 12:03:36.426425 sleeve ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: Effective MTU verified at 1438
INFO: 2018/11/25 12:03:36.727636 Discovered remote MAC b2:d6:a3:8b:39:f4 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.840942 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.907601 Discovered remote MAC 8e:db:74:3b:32:c9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.915587 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/25 12:03:37.076070 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:37.238501 Discovered remote MAC 06:39:6c:32:63:d9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:37.543427 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:37.687787 Discovered remote MAC 56:73:bb:d8:aa:5e at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:37.988066 Discovered remote MAC 62:23:82:bb:50:12 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:38.371417 Discovered remote MAC 2e:7d:5a:fa:87:42 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:38.609612 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:41.378973 Discovered remote MAC 56:8d:09:aa:75:53 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:41.634729 Discovered remote MAC 6e:0f:9d:8d:c9:11 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:41.807662 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:42.144464 Discovered remote MAC 2a:da:54:5c:d7:bc at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:14:46.960007 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:14:48.970734 Discovered remote MAC 62:23:82:bb:50:12 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:14:54.706640 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:14:54.867055 Discovered remote MAC 6e:0f:9d:8d:c9:11 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:15:46.951828 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:15:46.951900 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using sleeve
INFO: 2018/11/25 12:15:50.750709 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:15:50.750767 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using sleeve
INFO: 2018/11/25 12:29:59.579838 Discovered remote MAC 36:3e:00:1e:85:9b at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:16:23.051230 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:16:24.068425 Discovered remote MAC 86:d2:8d:d2:51:40 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 13:21:04.922449 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:35:19.593865 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 13:51:18.822134 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 14:15:03.795456 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 14:15:05.124976 Discovered remote MAC 62:1b:41:54:15:e0 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 14:17:02.606513 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 15:12:49.771746 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 17:16:57.995725 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 17:53:16.061314 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 18:38:14.545682 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=W1WhIoRt29Cc1K8fWZQrFyXxoFs0Tpek%2F0JCJZ13P3g%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:44799-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 19:44:09.417678 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 20:27:33.786108 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 20:44:08.717660 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 21:31:24.456040 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 22:38:46.577282 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 22:42:37.098054 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 23:21:27.343860 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=W1WhIoRt29Cc1K8fWZQrFyXxoFs0Tpek%2F0JCJZ13P3g%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:38507-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 23:46:38.063092 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 23:49:41.645931 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 01:14:04.868399 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 01:43:56.913972 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 03:23:41.978447 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 04:51:09.439893 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=W1WhIoRt29Cc1K8fWZQrFyXxoFs0Tpek%2F0JCJZ13P3g%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:52338-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/26 05:59:54.814142 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:34:15.310540 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:35:42.318405 Discovered remote MAC 92:6f:93:12:60:89 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:40:41.557942 Discovered remote MAC e2:77:25:ec:9d:dc at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:51:03.466680 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/26 06:52:56.365562 Discovered remote MAC 56:4a:04:2d:c8:d9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 07:02:36.563649 Discovered remote MAC 86:d2:8d:d2:51:40 at f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:36.563710 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:36.564687 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:36.565905 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.039037 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.063932 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.099878 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.137340 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.137558 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.137954 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.138002 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.138071 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.138231 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.138353 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.139662 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.150893 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.154763 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.172357 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.174894 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.196335 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.196375 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.199889 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.199925 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.226463 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.226918 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.227008 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.233638 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.233677 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.247964 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.248087 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.252500 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.345042 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.347752 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:02:37.400911 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:03:07.399534 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
ERRO: 2018/11/26 07:03:09.400633 Captured frame from MAC (86:d2:8d:d2:51:40) to (ae:9a:41:9a:13:07) associated with another peer f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/26 07:58:07.589620 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
WARN: 2018/11/26 08:15:12.713797 Vetoed installation of hairpin flow FlowSpec{keys: [EthernetFlowKey{src: 0a:3a:30:bc:0c:e2, dst: 7a:9d:37:1b:12:90} InPortFlowKey{vport: 1}], actions: [OutputAction{vport: 1}]}
INFO: 2018/11/26 09:05:22.709530 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/26 09:28:04.862555 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 09:57:29.909139 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=W1WhIoRt29Cc1K8fWZQrFyXxoFs0Tpek%2F0JCJZ13P3g%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:48992-><IP_ADDRESS>:53: i/o timeout

weave-net-pwqdv on r2s13:

root@r2s12:~# kubectl logs weave-net-pwqdv -n kube-system -c weave
DEBU: 2018/11/25 12:03:49.994110 [kube-peers] Checking peer "ae:9a:41:9a:13:07" against list &{[{f6:b0:8c:c6:13:40 r2s14} {ae:9a:41:9a:13:07 r2s13} {0a:3a:30:bc:0c:e2 r2s12}]}
INFO: 2018/11/25 12:03:50.342297 Command line options: map[conn-limit:100 expect-npc:true datapath:datapath db-prefix:/weavedb/weave-net http-addr:<IP_ADDRESS>:6784 metrics-addr:<IP_ADDRESS>:6782 nickname:r2s13 port:6783 name:ae:9a:41:9a:13:07 no-dns:true docker-api: host-root:/host ipalloc-init:consensus=3 ipalloc-range:<IP_ADDRESS>/12]
INFO: 2018/11/25 12:03:50.342590 weave  2.4.1
INFO: 2018/11/25 12:03:50.485681 Re-exposing <IP_ADDRESS>/12 on bridge "weave"
INFO: 2018/11/25 12:03:50.515363 Bridge type is bridged_fastdp
INFO: 2018/11/25 12:03:50.515384 Communication between peers is unencrypted.
INFO: 2018/11/25 12:03:50.687670 Our name is ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:50.687789 Launch detected - using supplied peer list: [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
INFO: 2018/11/25 12:03:50.687830 Checking for pre-existing addresses on weave bridge
INFO: 2018/11/25 12:03:50.688216 weave bridge has address <IP_ADDRESS>/12
INFO: 2018/11/25 12:03:50.717278 [allocator ae:9a:41:9a:13:07] Initialising with persisted data
INFO: 2018/11/25 12:03:50.717415 Sniffing traffic on datapath (via ODP)
INFO: 2018/11/25 12:03:50.718942 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:03:50.719753 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:03:50.720000 ->[<IP_ADDRESS>:36832] connection accepted
INFO: 2018/11/25 12:03:50.720120 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:03:50.721021 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection shutting down due to error: cannot connect to ourself
INFO: 2018/11/25 12:03:50.721146 ->[<IP_ADDRESS>:36832|ae:9a:41:9a:13:07(r2s13)]: connection shutting down due to error: cannot connect to ourself
INFO: 2018/11/25 12:03:50.721645 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection ready; using protocol version 2
INFO: 2018/11/25 12:03:50.721835 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/25 12:03:50.721880 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection added (new peer)
INFO: 2018/11/25 12:03:50.722119 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection ready; using protocol version 2
INFO: 2018/11/25 12:03:50.722376 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using fastdp
INFO: 2018/11/25 12:03:50.723021 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection added (new peer)
INFO: 2018/11/25 12:03:50.783426 Listening for HTTP control messages on <IP_ADDRESS>:6784
INFO: 2018/11/25 12:03:50.783874 Listening for metrics requests on <IP_ADDRESS>:6782
INFO: 2018/11/25 12:03:51.012989 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 12:03:51.013134 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using sleeve
INFO: 2018/11/25 12:03:51.013176 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection fully established
INFO: 2018/11/25 12:03:51.013708 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using sleeve
INFO: 2018/11/25 12:03:51.014640 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using fastdp
INFO: 2018/11/25 12:03:51.014723 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 12:03:51.013805 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection fully established
INFO: 2018/11/25 12:03:51.018775 sleeve ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: Effective MTU verified at 1438
INFO: 2018/11/25 12:03:51.023584 sleeve ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: Effective MTU verified at 1438
INFO: 2018/11/25 12:03:51.223742 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/25 12:03:51.341260 [kube-peers] Added myself to peer list &{[{f6:b0:8c:c6:13:40 r2s14} {ae:9a:41:9a:13:07 r2s13} {0a:3a:30:bc:0c:e2 r2s12}]}
DEBU: 2018/11/25 12:03:51.352134 [kube-peers] Nodes that have disappeared: map[]
<IP_ADDRESS>
INFO: 2018/11/25 12:03:51.697725 Discovered remote MAC 86:d2:8d:d2:51:40 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:03:51.853709 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 12:03:52.222972 Discovered remote MAC 3e:32:10:c3:fd:ba at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:03:52.223354 Discovered remote MAC 02:bc:92:43:61:b3 at 0a:3a:30:bc:0c:e2(r2s12)
WARN: 2018/11/25 12:03:53.094009 [allocator]: Delete: no addresses for 5cea538a8a7bceae10deede3a8e9ce20ec4e35901dd21d55a136d37372ddde13
WARN: 2018/11/25 12:03:53.207989 [allocator]: Delete: no addresses for df0e6f7aa1a5a810b2523a8f6dd5ae61d4a4c56931a9d61f2715cf1fa55c7140
WARN: 2018/11/25 12:03:53.345971 [allocator]: Delete: no addresses for a5220c52b87f79396a7b440047e9ae1d485a8f104044712dcde9f57824e4d3d2
WARN: 2018/11/25 12:03:53.405931 [allocator]: Delete: no addresses for 077c7e71ab6a9715394832820b716477f401a22d26575b3a274733f3fdb6dd52
INFO: 2018/11/25 12:03:57.554347 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave
INFO: 2018/11/25 12:04:08.965248 Discovered remote MAC 92:41:2d:35:98:a0 at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:04:29.808134 Discovered remote MAC 46:d2:1a:4c:d1:bc at f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/25 12:16:01.230739 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:16:01.230793 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using sleeve
INFO: 2018/11/25 12:16:01.248862 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:16:01.248890 overlay_switch ->[f6:b0:8c:c6:13:40(r2s14)] using sleeve
INFO: 2018/11/25 12:30:14.002749 Discovered remote MAC 5e:bc:53:01:1e:eb at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 13:21:17.122320 Discovered remote MAC 5e:bc:53:01:1e:eb at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 13:51:30.979549 Discovered remote MAC 5e:bc:53:01:1e:eb at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 14:15:19.179493 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 18:00:21.649811 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=ti5mHxx4N2jhWg%2FW145Y%2Fy%2Fc6s99JbLsGvwbKbYWSyA%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:53473-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 22:45:12.845947 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=ti5mHxx4N2jhWg%2FW145Y%2Fy%2Fc6s99JbLsGvwbKbYWSyA%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:55785-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 23:49:52.979718 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 00:11:07.970784 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 01:44:08.087955 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 03:23:53.017020 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 04:57:39.936327 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=ti5mHxx4N2jhWg%2FW145Y%2Fy%2Fc6s99JbLsGvwbKbYWSyA%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:55778-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/26 06:00:05.637373 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:35:55.096941 Discovered remote MAC 32:1f:40:d2:cc:c3 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:40:54.316859 Discovered remote MAC b2:00:72:68:e1:c7 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:53:09.102351 Discovered remote MAC ba:85:a2:75:75:5a at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 08:11:48.106245 Discovered remote MAC 7a:9d:37:1b:12:90 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 08:14:55.887113 Discovered remote MAC 6a:3e:f3:eb:69:d9 at 0a:3a:30:bc:0c:e2(r2s12)

weave-net-vp2mt on r2s14:
root@r2s12:~# kubectl logs weave-net-vp2mt -n kube-system -c weave
DEBU: 2018/11/23 19:00:07.913037 [kube-peers] Checking peer "f6:b0:8c:c6:13:40" against list &{[{f6:b0:8c:c6:13:40 r2s14} {ae:9a:41:9a:13:07 r2s13} {0a:3a:30:bc:0c:e2 r2s12}]}
INFO: 2018/11/23 19:00:08.140316 Command line options: map[host-root:/host metrics-addr:<IP_ADDRESS>:6782 datapath:datapath expect-npc:true no-dns:true db-prefix:/weavedb/weave-net docker-api: conn-limit:100 port:6783 ipalloc-range:<IP_ADDRESS>/12 name:f6:b0:8c:c6:13:40 nickname:r2s14 http-addr:<IP_ADDRESS>:6784 ipalloc-init:consensus=3]
INFO: 2018/11/23 19:00:08.140394 weave  2.4.1
INFO: 2018/11/23 19:00:08.498688 Bridge type is bridged_fastdp
INFO: 2018/11/23 19:00:08.498727 Communication between peers is unencrypted.
INFO: 2018/11/23 19:00:08.569917 Our name is f6:b0:8c:c6:13:40(r2s14)
INFO: 2018/11/23 19:00:08.570009 Launch detected - using supplied peer list: [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
INFO: 2018/11/23 19:00:08.582700 Checking for pre-existing addresses on weave bridge
INFO: 2018/11/23 19:00:08.589366 [allocator f6:b0:8c:c6:13:40] Initialising with persisted data
INFO: 2018/11/23 19:00:08.589453 Sniffing traffic on datapath (via ODP)
INFO: 2018/11/23 19:00:08.654717 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:00:08.655121 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:00:08.655218 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:00:08.659401 ->[<IP_ADDRESS>:48199] connection accepted
INFO: 2018/11/23 19:00:08.661824 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection ready; using protocol version 2
INFO: 2018/11/23 19:00:08.662916 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/23 19:00:08.662998 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection added (new peer)
INFO: 2018/11/23 19:00:08.664215 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection ready; using protocol version 2
INFO: 2018/11/23 19:00:08.664367 Listening for HTTP control messages on <IP_ADDRESS>:6784
INFO: 2018/11/23 19:00:08.664617 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/23 19:00:08.664803 Listening for metrics requests on <IP_ADDRESS>:6782
INFO: 2018/11/23 19:00:08.664690 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection added (new peer)
INFO: 2018/11/23 19:00:08.666175 ->[<IP_ADDRESS>:6783|f6:b0:8c:c6:13:40(r2s14)]: connection shutting down due to error: cannot connect to ourself
INFO: 2018/11/23 19:00:08.666434 ->[<IP_ADDRESS>:48199|f6:b0:8c:c6:13:40(r2s14)]: connection shutting down due to error: cannot connect to ourself
INFO: 2018/11/23 19:00:08.667122 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection fully established
INFO: 2018/11/23 19:00:08.671945 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using sleeve
INFO: 2018/11/23 19:00:08.671995 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection fully established
INFO: 2018/11/23 19:00:08.672109 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/23 19:00:08.672450 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/23 19:00:08.685289 sleeve ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: Effective MTU verified at 1438
INFO: 2018/11/23 19:00:08.689172 sleeve ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: Effective MTU verified at 1438
INFO: 2018/11/23 19:00:08.957467 Discovered remote MAC 66:3f:bc:d5:39:52 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:00:09.167210 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/23 19:00:09.201493 [kube-peers] Added myself to peer list &{[{f6:b0:8c:c6:13:40 r2s14} {ae:9a:41:9a:13:07 r2s13} {0a:3a:30:bc:0c:e2 r2s12}]}
DEBU: 2018/11/23 19:00:09.212283 [kube-peers] Nodes that have disappeared: map[]
<IP_ADDRESS>
INFO: 2018/11/23 19:00:09.489402 Discovered remote MAC 56:de:64:48:1e:8c at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:00:09.916281 Discovered remote MAC d6:63:8d:87:d1:81 at ae:9a:41:9a:13:07(r2s13)
WARN: 2018/11/23 19:00:10.207461 [allocator]: Delete: no addresses for e40b785c2a5cf4c70247147bf794312bf242fbbf4de675903e5d6c4d1c7aa528
INFO: 2018/11/23 19:00:12.337802 Discovered remote MAC 2a:61:2e:ac:a1:95 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:00:18.766423 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=sleeve&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:37228-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/23 19:00:44.724133 Discovered remote MAC fa:14:4e:2b:20:b4 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:00:49.150106 Discovered remote MAC 0a:72:79:49:9d:ca at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:00:49.248429 Discovered remote MAC c6:27:30:71:11:b4 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:01:40.031645 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:56846-><IP_ADDRESS>:6783: read: connection reset by peer
INFO: 2018/11/23 19:01:40.031766 ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: connection deleted
INFO: 2018/11/23 19:01:40.033883 Removed unreachable peer ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:01:40.035036 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:40.035448 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:41.376787 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:01:42.553439 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:42.553928 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:45.398848 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:45.399315 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:45.717837 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:33903-><IP_ADDRESS>:6783: read: connection reset by peer
INFO: 2018/11/23 19:01:45.717946 ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: connection deleted
INFO: 2018/11/23 19:01:45.717999 Removed unreachable peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:01:45.718726 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:45.719282 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:48.267780 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:48.268161 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:49.285235 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:49.285594 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:49.777666 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:49.778019 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:01:53.727698 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:55.788914 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:01:55.789395 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:02:01.223933 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:02:26.949257 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:02:33.401898 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:02:41.500230 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:02:44.497135 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:02:45.782896 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:02:48.405920 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:03:00.168786 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:03:00.369210 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:03:03.280939 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:03:03.405942 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:03:19.013787 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:03:20.573309 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: no route to host
INFO: 2018/11/23 19:03:29.612105 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:03:29.612466 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:03:42.994017 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:03:42.994345 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:03:52.865859 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:03:52.866373 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:04:20.074091 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:04:20.074623 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:04:26.976617 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/23 19:04:26.977228 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/23 19:04:53.992961 ->[<IP_ADDRESS>:41806] connection accepted
INFO: 2018/11/23 19:04:54.013662 ->[<IP_ADDRESS>:41806|0a:3a:30:bc:0c:e2(r2s12)]: connection ready; using protocol version 2
INFO: 2018/11/23 19:04:54.013817 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/23 19:04:54.013867 ->[<IP_ADDRESS>:41806|0a:3a:30:bc:0c:e2(r2s12)]: connection added (new peer)
INFO: 2018/11/23 19:04:54.015998 ->[<IP_ADDRESS>:41806|0a:3a:30:bc:0c:e2(r2s12)]: connection fully established
INFO: 2018/11/23 19:04:54.059516 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/23 19:04:54.065328 sleeve ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: Effective MTU verified at 1438
INFO: 2018/11/23 19:04:54.116746 Discovered remote MAC 22:9c:cc:3e:f5:20 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:04:54.382816 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:04:54.812775 Discovered remote MAC 4e:24:c0:30:ed:d6 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:04:55.340675 Discovered remote MAC d6:1c:00:d1:67:f6 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 19:04:57.716735 ->[<IP_ADDRESS>:59615] connection accepted
INFO: 2018/11/23 19:04:57.717757 ->[<IP_ADDRESS>:59615|ae:9a:41:9a:13:07(r2s13)]: connection ready; using protocol version 2
INFO: 2018/11/23 19:04:57.717932 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/23 19:04:57.717972 ->[<IP_ADDRESS>:59615|ae:9a:41:9a:13:07(r2s13)]: connection added (new peer)
INFO: 2018/11/23 19:04:57.730265 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using sleeve
INFO: 2018/11/23 19:04:57.730337 ->[<IP_ADDRESS>:59615|ae:9a:41:9a:13:07(r2s13)]: connection fully established
INFO: 2018/11/23 19:04:57.730550 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/23 19:04:57.740951 sleeve ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: Effective MTU verified at 1438
INFO: 2018/11/23 19:04:58.219872 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/23 19:04:58.260179 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:04:58.588569 Discovered remote MAC b2:d6:a3:8b:39:f4 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:04:58.676731 Discovered remote MAC 8e:db:74:3b:32:c9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:04:59.689859 Discovered remote MAC ae:51:5e:c7:a2:b0 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:00.136132 Discovered remote MAC 1e:e6:50:8e:90:c7 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:00.662207 Discovered remote MAC 32:db:65:76:73:3e at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:00.845232 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:01.350811 Discovered remote MAC 7e:32:ae:fc:c4:6a at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:01.501786 Discovered remote MAC 56:5a:fc:e7:3f:f2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:07.114439 Discovered remote MAC de:c7:00:36:74:b1 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:07.341110 Discovered remote MAC da:7d:e7:13:2c:17 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:07.513128 Discovered remote MAC 76:5b:b5:f7:23:ef at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:05:07.763595 Discovered remote MAC 02:05:d7:bd:d7:46 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:32:09.484660 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:32:11.489669 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:53:10.238803 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:53:12.241743 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 19:55:21.208324 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 20:19:23.781747 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/23 20:26:35.426052 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/23 20:26:35.893829 Discovered remote MAC ea:4d:23:65:81:71 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 00:10:24.953520 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 00:10:24.954231 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 00:20:22.982813 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=fastdp&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:60467-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/24 03:36:39.624268 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 03:36:39.624936 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 05:18:37.145448 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=fastdp&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:33572-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/24 05:26:38.301203 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 07:29:29.367124 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 07:29:29.367900 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/24 07:29:31.869668 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/24 07:29:31.869908 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/24 07:30:08.511735 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/24 07:35:08.511782 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/24 07:45:08.511810 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/24 07:50:08.511767 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/24 07:55:08.512027 Captured frame from MAC (da:b3:b8:8e:3c:01) to (4e:1e:e7:13:c3:ea) associated with another peer ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 08:51:26.095527 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 08:51:42.387762 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 10:30:39.439110 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 10:54:28.785409 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=fastdp&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:53906-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/24 12:55:18.784306 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 12:55:18.794046 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 15:20:04.712194 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 15:20:04.712725 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 16:19:38.833147 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=fastdp&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:54544-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/24 20:13:38.436276 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 21:16:25.009838 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/24 21:16:25.010522 Discovered remote MAC 4e:1e:e7:13:c3:ea at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 21:20:29.386749 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 22:24:45.255932 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=fastdp&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:49313-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/24 22:45:54.311317 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/24 23:37:21.264794 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 04:02:06.279166 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=fastdp&flag_network=fastdp&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:40742-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 06:38:48.892390 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 07:05:58.261932 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 07:06:20.467050 Discovered remote MAC 1e:e6:50:8e:90:c7 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 07:06:21.840435 Discovered remote MAC 0a:b9:22:75:d6:68 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 07:06:21.936232 Discovered remote MAC 96:0e:f4:d5:ed:ce at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 07:07:56.158821 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 07:07:56.159792 Discovered remote MAC 32:db:65:76:73:3e at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 07:44:32.446909 Discovered remote MAC ae:5c:9f:d7:55:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 07:45:14.691590 Discovered remote MAC 32:db:65:76:73:3e at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 07:45:14.692783 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 07:46:51.647150 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 08:28:19.999878 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/25 08:28:19.999949 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (32:db:65:76:73:3e) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 08:41:50.128770 Discovered remote MAC 22:2d:45:9d:8d:a1 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 08:41:50.156673 Discovered remote MAC d6:29:4e:49:82:87 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 08:44:26.360686 Discovered remote MAC e2:39:f9:4f:56:7a at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 08:45:46.907678 Discovered remote MAC da:01:c9:5c:b2:e4 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 08:45:47.901561 Discovered remote MAC 2e:28:57:5b:3d:ed at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 08:54:03.823511 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 09:11:38.563662 Discovered remote MAC 7e:17:93:c7:67:70 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 09:11:38.707556 Discovered remote MAC b6:d6:88:56:d0:36 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 09:34:01.355793 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/25 09:34:01.355871 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (de:c7:00:36:74:b1) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/25 09:34:01.356087 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (de:c7:00:36:74:b1) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 09:35:44.730336 Discovered remote MAC 4e:8f:97:0a:62:1c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 09:35:47.887287 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 09:35:47.888265 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 09:36:06.544355 Discovered remote MAC 32:db:65:76:73:3e at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 09:42:00.174783 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave
INFO: 2018/11/25 09:50:26.532691 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 10:23:33.922328 ->[<IP_ADDRESS>:41806|0a:3a:30:bc:0c:e2(r2s12)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:6783-><IP_ADDRESS>:41806: read: connection reset by peer
INFO: 2018/11/25 10:23:33.922438 ->[<IP_ADDRESS>:41806|0a:3a:30:bc:0c:e2(r2s12)]: connection deleted
INFO: 2018/11/25 10:23:33.923917 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 10:23:33.924389 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 10:23:33.925171 Removed unreachable peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 10:23:35.675801 Discovered remote MAC 1e:e6:50:8e:90:c7 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 10:26:17.902832 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 10:26:17.903226 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 10:26:29.635279 ->[<IP_ADDRESS>:60858] connection accepted
INFO: 2018/11/25 10:26:29.651513 ->[<IP_ADDRESS>:60858|0a:3a:30:bc:0c:e2(r2s12)]: connection ready; using protocol version 2
INFO: 2018/11/25 10:26:29.651724 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/25 10:26:29.651769 ->[<IP_ADDRESS>:60858|0a:3a:30:bc:0c:e2(r2s12)]: connection added (new peer)
INFO: 2018/11/25 10:26:29.680883 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 10:26:29.680985 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using sleeve
INFO: 2018/11/25 10:26:29.681055 ->[<IP_ADDRESS>:60858|0a:3a:30:bc:0c:e2(r2s12)]: connection fully established
INFO: 2018/11/25 10:26:29.691688 sleeve ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: Effective MTU verified at 1438
INFO: 2018/11/25 10:26:29.711226 Discovered remote MAC 5e:23:3e:13:86:bc at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 10:26:29.815171 Discovered remote MAC 12:9f:4a:bb:54:d6 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 10:26:29.823001 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 10:26:30.154992 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/25 10:26:31.203947 Discovered remote MAC 3e:f7:ea:af:6b:fd at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 11:57:39.919436 ->[<IP_ADDRESS>:60858|0a:3a:30:bc:0c:e2(r2s12)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:6783-><IP_ADDRESS>:60858: read: connection reset by peer
INFO: 2018/11/25 11:57:39.919574 ->[<IP_ADDRESS>:60858|0a:3a:30:bc:0c:e2(r2s12)]: connection deleted
INFO: 2018/11/25 11:57:39.920904 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 11:57:39.921395 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 11:57:39.921629 Removed unreachable peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 11:59:29.645800 ->[<IP_ADDRESS>:33822] connection accepted
INFO: 2018/11/25 11:59:29.647114 ->[<IP_ADDRESS>:33822|0a:3a:30:bc:0c:e2(r2s12)]: connection ready; using protocol version 2
INFO: 2018/11/25 11:59:29.647318 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using fastdp
INFO: 2018/11/25 11:59:29.647359 ->[<IP_ADDRESS>:33822|0a:3a:30:bc:0c:e2(r2s12)]: connection added (new peer)
INFO: 2018/11/25 11:59:29.650256 ->[<IP_ADDRESS>:33822|0a:3a:30:bc:0c:e2(r2s12)]: connection fully established
INFO: 2018/11/25 11:59:30.154216 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 11:59:30.159926 sleeve ->[<IP_ADDRESS>:6783|0a:3a:30:bc:0c:e2(r2s12)]: Effective MTU verified at 1438
INFO: 2018/11/25 11:59:30.291134 Discovered remote MAC 12:9f:4a:bb:54:d6 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 11:59:30.324920 Discovered remote MAC 02:bc:92:43:61:b3 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 11:59:30.395851 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 12:01:56.341088 ->[<IP_ADDRESS>:59615|ae:9a:41:9a:13:07(r2s13)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:6783-><IP_ADDRESS>:59615: read: connection reset by peer
INFO: 2018/11/25 12:01:56.341218 ->[<IP_ADDRESS>:59615|ae:9a:41:9a:13:07(r2s13)]: connection deleted
INFO: 2018/11/25 12:01:56.342331 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:01:56.342823 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:01:56.344270 Removed unreachable peer ae:9a:41:9a:13:07(r2s13)
ERRO: 2018/11/25 12:02:13.582024 Captured frame from MAC (0a:3a:30:bc:0c:e2) to (da:b3:b8:8e:3c:01) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 12:03:19.867224 ->[<IP_ADDRESS>:6783] attempting connection
INFO: 2018/11/25 12:03:19.867748 ->[<IP_ADDRESS>:6783] error during connection attempt: dial tcp4 :0-><IP_ADDRESS>:6783: connect: connection refused
INFO: 2018/11/25 12:03:35.356571 ->[<IP_ADDRESS>:59212] connection accepted
INFO: 2018/11/25 12:03:35.357985 ->[<IP_ADDRESS>:59212|ae:9a:41:9a:13:07(r2s13)]: connection ready; using protocol version 2
INFO: 2018/11/25 12:03:35.358136 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/25 12:03:35.358173 ->[<IP_ADDRESS>:59212|ae:9a:41:9a:13:07(r2s13)]: connection added (new peer)
INFO: 2018/11/25 12:03:35.361323 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using sleeve
INFO: 2018/11/25 12:03:35.361404 ->[<IP_ADDRESS>:59212|ae:9a:41:9a:13:07(r2s13)]: connection fully established
INFO: 2018/11/25 12:03:35.361657 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2018/11/25 12:03:35.392599 sleeve ->[<IP_ADDRESS>:6783|ae:9a:41:9a:13:07(r2s13)]: Effective MTU verified at 1438
INFO: 2018/11/25 12:03:35.673169 Discovered remote MAC b2:d6:a3:8b:39:f4 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:35.786422 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:35.853131 Discovered remote MAC 8e:db:74:3b:32:c9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:35.860262 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp
INFO: 2018/11/25 12:03:36.021655 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.184132 Discovered remote MAC 06:39:6c:32:63:d9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.334264 Discovered remote MAC 02:05:d7:bd:d7:46 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.488949 Discovered remote MAC da:b3:b8:8e:3c:01 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.633400 Discovered remote MAC 56:73:bb:d8:aa:5e at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:36.933690 Discovered remote MAC 62:23:82:bb:50:12 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:37.316906 Discovered remote MAC 2e:7d:5a:fa:87:42 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:37.555092 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:40.324443 Discovered remote MAC 56:8d:09:aa:75:53 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:40.580211 Discovered remote MAC 6e:0f:9d:8d:c9:11 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:40.753089 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:03:41.090019 Discovered remote MAC 2a:da:54:5c:d7:bc at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:15:30.651666 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:15:30.651716 overlay_switch ->[0a:3a:30:bc:0c:e2(r2s12)] using sleeve
INFO: 2018/11/25 12:15:35.661764 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:15:35.661814 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using sleeve
INFO: 2018/11/25 12:17:48.635010 Discovered remote MAC 06:39:6c:32:63:d9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:25:27.397425 Discovered remote MAC 6e:0f:9d:8d:c9:11 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:25:27.397691 Discovered remote MAC 06:c1:9c:c9:24:69 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:25:29.652369 Discovered remote MAC 62:23:82:bb:50:12 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:25:31.416715 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:25:34.267779 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 12:25:34.615698 Discovered remote MAC 2a:da:54:5c:d7:bc at ae:9a:41:9a:13:07(r2s13)
WARN: 2018/11/25 12:29:57.305287 Vetoed installation of hairpin flow FlowSpec{keys: [EthernetFlowKey{src: f6:b0:8c:c6:13:40, dst: 3e:32:10:c3:fd:ba} InPortFlowKey{vport: 1}], actions: [OutputAction{vport: 1}]}
INFO: 2018/11/25 12:29:58.477752 Discovered remote MAC 36:3e:00:1e:85:9b at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 12:29:58.627400 Discovered remote MAC 5e:bc:53:01:1e:eb at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 13:16:21.864844 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 13:16:22.881559 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:16:59.057760 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:21:01.724534 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:21:01.725209 Discovered remote MAC 5e:bc:53:01:1e:eb at 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/25 13:21:01.725252 Captured frame from MAC (5e:bc:53:01:1e:eb) to (ae:9a:41:9a:13:07) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/25 13:21:01.725279 Captured frame from MAC (5e:bc:53:01:1e:eb) to (ae:9a:41:9a:13:07) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
ERRO: 2018/11/25 13:21:03.727641 Captured frame from MAC (5e:bc:53:01:1e:eb) to (ae:9a:41:9a:13:07) associated with another peer 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 13:51:15.568927 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 13:51:15.569311 Discovered remote MAC 5e:bc:53:01:1e:eb at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 14:15:02.503155 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 14:15:03.758995 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 14:15:03.832594 Discovered remote MAC 62:1b:41:54:15:e0 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 14:17:01.310241 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 14:17:06.433832 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 15:33:17.054144 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:44503-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 17:23:27.405679 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 19:53:51.060282 Discovered remote MAC 06:39:6c:32:63:d9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 21:16:40.993853 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:42805-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/25 22:42:30.365654 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/25 23:49:37.311979 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/25 23:49:37.312272 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 00:10:52.293958 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 01:43:52.370980 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 01:43:52.371609 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 02:48:50.712479 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=wi6Km4EP%2F08mg%2BV3Rb0l4EphyJfKw9rxBoYjJYAd8V4%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:56339-><IP_ADDRESS>:53: i/o timeout
INFO: 2018/11/26 03:23:37.257201 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 03:23:37.257888 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 05:58:38.649468 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 05:59:49.810715 Discovered remote MAC 76:cf:85:bd:f2:e2 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 05:59:49.811430 Discovered remote MAC be:b6:78:c3:7c:3c at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:35:39.252695 Discovered remote MAC 92:6f:93:12:60:89 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:35:39.255639 Discovered remote MAC 32:1f:40:d2:cc:c3 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:36:02.453815 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:40:38.473388 Discovered remote MAC b2:00:72:68:e1:c7 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:40:38.483120 Discovered remote MAC e2:77:25:ec:9d:dc at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 06:52:53.253636 Discovered remote MAC ba:85:a2:75:75:5a at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 06:52:53.268645 Discovered remote MAC 56:4a:04:2d:c8:d9 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 07:02:33.449025 Discovered remote MAC 52:4a:f5:1d:1e:ea at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 07:03:06.285797 Discovered remote MAC ae:9a:41:9a:13:07 at ae:9a:41:9a:13:07(r2s13)
INFO: 2018/11/26 08:00:04.443295 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave
INFO: 2018/11/26 08:11:32.223989 Discovered remote MAC 7a:9d:37:1b:12:90 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 08:14:40.003505 Discovered remote MAC 6a:3e:f3:eb:69:d9 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 09:33:54.178292 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)
INFO: 2018/11/26 10:54:13.095386 Discovered remote MAC 0a:3a:30:bc:0c:e2 at 0a:3a:30:bc:0c:e2(r2s12)

@bboreham knowing that now only one of the two coredns keeps crashing with below error, and actually I am not aware whether the coredns is the main cause for all these problems or the weave:
root@r2s12:~# kubectl logs coredns-d5dd54cbf-kpj45 -n kube-system
E1126 11:50:06.160572       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:355: Failed to list *v1.Namespace: Get https://<IP_ADDRESS>:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp <IP_ADDRESS>:443: i/o timeout

@bboreham do you mean the pods positioning as below:
kubectl get pod --all-namespaces -o wide

NAMESPACE     NAME                                        READY   STATUS    RESTARTS   AGE     IP               NODE    NOMINATED NODE

kube-system   pod/coredns-d5dd54cbf-9zjv9                 1/1     Running   6          6h44m   <IP_ADDRESS>       r2s13   <none>
kube-system   pod/coredns-d5dd54cbf-kpj45                 1/1     CrashLoopBackOff   70         5h23m   <IP_ADDRESS>      r2s12   <none>
kube-system   pod/kube-apiserver-r2s12                    1/1     Running   9          36d     <IP_ADDRESS>   r2s12   <none>
kube-system   pod/metrics-server-7fbd9b8589-ml6jw         1/1     Running   2          5d5h    <IP_ADDRESS>       r2s13   <none>
kube-system   pod/weave-net-5fb7n                         2/2     Running   9          33d     <IP_ADDRESS>   r2s12   <none>
kube-system   pod/weave-net-pwqdv                         2/2     Running   10         33d     <IP_ADDRESS>   r2s13   <none>
kube-system   pod/weave-net-vp2mt                         2/2     Running   17         33d     <IP_ADDRESS>   r2s14   <none>

and output of the top commands:
root@r2s12:~# kubectl top nodes
NAME    CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
r2s12   371m         4%     2193Mi          9%
r2s14   184m         2%     1549Mi          19%
r2s13   <unknown>                           <unknown>               <unknown>               <unknown>

Yes, that's what I needed.

I don't know what's wrong.  We have:
  Warning  FailedCreatePodSandBox  8m2s                   kubelet, r2s12     Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "cd6a070327f1eee11727e424f57977b0211a1ae2933ff11bda8be714b3332988" network for pod "coredns-d5dd54cbf-jxzhc": NetworkPlugin cni failed to set up pod "coredns-d5dd54cbf-jxzhc_kube-system" network: unable to allocate IP address: Post http://<IP_ADDRESS>:6784/ip/cd6a070327f1eee11727e424f57977b0211a1ae2933ff11bda8be714b3332988: dial tcp <IP_ADDRESS>:6784: connect: connection refused

which says kubelet couldn't talk to Weave Net.  However the logs of Weave Net indicate it was running for a long time.  (Need to know the time that command was run to subtract 8m2s to arrive at a time to match in the logs)
We have:
INFO: 2018/11/25 11:59:31.504239 Weave version 2.5.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.5.0/weave

then later:
INFO: 2018/11/25 18:38:14.545682 Error checking version: Get https://checkpoint-api.weave.works/v1/check/weave-net?arch=amd64&flag_docker-version=none&flag_kernel-version=4.4.0-87-generic&flag_network=sleeve&flag_network=sleeve&os=linux&signature=W1WhIoRt29Cc1K8fWZQrFyXxoFs0Tpek%2F0JCJZ13P3g%3D&version=2.4.1: dial tcp: lookup checkpoint-api.weave.works on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:44799-><IP_ADDRESS>:53: i/o timeout

which suggests maybe something broke the network on the node as a whole?  (It's failing to get a response from Google DNS, which is generally reliable)
The other two nodes have those messages too - sometimes it can get through and sometimes it cannot.
This:
INFO: 2018/11/25 12:03:36.915587 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using fastdp

followed by this:
INFO: 2018/11/25 12:15:46.951828 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] fastdp timed out waiting for vxlan heartbeat
INFO: 2018/11/25 12:15:46.951900 overlay_switch ->[ae:9a:41:9a:13:07(r2s13)] using sleeve

also says the underlying network was working at one point then not working so well.
Beyond "things are going wrong" I can't diagnose the problem at this distance.

@bboreham can we get anything from the logs on 2018/11/26 as I have tried several things on 2018/11/25 which may lead to this confusion.

Maybe if you run the experiment again, from a clean slate (i.e. new VMs or at least freshly-rebooted nodes), and supply all the logs from that point, it might be possible to spot what happened.

@bboreham I will redeploy the cluster and check if the problem still exists or not,

I have recreated the cluster and the problem didn't appear. I don't know what I made wrong at the first time and I will close this issue, thanks all.

@hshtel good to hear, thanks for coming back to us.

@bboreham I found the cause for the main problem "unable to reach from a pod to its hosting kube-node"
I have deleted the line "<IP_ADDRESS>   r2s13" from "/etc/hosts" file on the node where the pod is deployed and it worked.
I didn't understand how this caused the problem and I will check this with Metric-server folks.