weaver process using excessive amount of cpu and network

The weaver process on some of our nodes is using an excessive amount of CPU for long periods at a time.  Here's 30 second snapshot of CPU utilization (the server has 2 cores so one core is pegged during this period).  This actually happens for a couple minutes at a time.
Also, please note at the same time we are seeing high CPU usage we are also seeing very high network traffic.  None of our services running on these boxes should use anything even close to the amount of network traffic shown.  Typically usage is 100-300KB/sec, but when weaver is using CPU we see network traffic upwards of 100MB/sec.
----system---- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- ----most-expensive---- -most-expensive-
     time     |usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw |     i/o process      |  cpu process
24-02 20:03:10| 41  39  15   0   0   6|   0     0 | 104M  107M|   0     0 |  30k   50k|weaver        0    52M|weaver        56
24-02 20:03:11| 35  43  13   0   0   9|   0   144k| 108M  110M|   0     0 |  31k   52k|weaver        0    54M|weaver        55
24-02 20:03:12| 44  38  13   0   0   5|   0     0 | 106M  109M|   0     0 |  26k   48k|weaver       16k   54M|weaver        53
24-02 20:03:13| 40  39  13   0   0   8|   0    48k| 104M  107M|   0     0 |  28k   51k|weaver     1409B   52M|weaver        53
24-02 20:03:14| 40  40  13   0   0   8|   0     0 | 107M  109M|   0     0 |  29k   50k|weaver        0    53M|weaver        55
24-02 20:03:15| 46  37  11   0   0   6|   0     0 | 107M  109M|   0     0 |  25k   46k|weaver        0    53M|weaver        54
24-02 20:03:16| 36  43  12   0   0   8|   0    40k| 108M  110M|   0     0 |  30k   52k|weaver        0    54M|weaver        56
24-02 20:03:17| 44  38  12   0   0   5|   0   152k| 107M  109M|   0     0 |  26k   47k|weaver       12k   54M|weaver        52
24-02 20:03:18| 45  36  14   0   0   5|   0    88k| 106M  108M|   0     0 |  26k   48k|weaver        0    53M|weaver        53
24-02 20:03:19| 42  38  12   0   0   8|   0     0 | 106M  108M|   0     0 |  30k   51k|weaver     1409B   52M|weaver        54
24-02 20:03:20| 42  37  13   0   0   7|   0     0 | 108M  111M|   0     0 |  29k   50k|weaver        0    54M|weaver        54
24-02 20:03:21| 35  44  13   0   0   8|   0     0 | 108M  110M|   0     0 |  29k   51k|weaver        0    54M|weaver        57
24-02 20:03:22| 39  41  12   0   0   8|   0    24k| 106M  109M|   0     0 |  28k   51k|weaver        0    53M|weaver        54
24-02 20:03:23| 49  36   9   0   0   6|   0    64k|  94M   96M|   0     0 |  24k   43k|weaver        0    47M|weaver        48
24-02 20:03:24| 68  29   0   0   0   3|   0     0 |  54M   55M|   0     0 |5395    35k|weaver       18k   27M|scout         48
24-02 20:03:25| 69  25   1   0   0   5|   0     0 |  60M   61M|   0     0 |8129    37k|weaver        0    30M|scout         42
24-02 20:03:26| 41  41  12   0   0   6|   0     0 | 107M  109M|   0     0 |  32k   51k|weaver        0    54M|weaver        57
24-02 20:03:27| 41  41  11   0   0   6|   0   140k| 107M  108M|   0     0 |  24k   48k|weaver        0    53M|weaver        52
24-02 20:03:28| 46  35  11   0   0   8|   0   104k| 108M  109M|   0     0 |  26k   47k|weaver     8352B   53M|weaver        56
24-02 20:03:29| 43  41  10   0   0   6|   0     0 |  97M   98M|   0     0 |  25k   46k|weaver        0    48M|weaver        50
24-02 20:03:30| 44  41  10   0   0   5|   0     0 | 101M  103M|   0     0 |  27k   46k|weaver        0    51M|weaver        52
24-02 20:03:31| 38  44  13   0   0   5|   0     0 | 108M  111M|   0     0 |  30k   51k|weaver        0    53M|weaver        54
24-02 20:03:32| 43  38  13   0   0   5|   0     0 | 108M  110M|   0     0 |  28k   52k|weaver     1409B   55M|weaver        55
24-02 20:03:33| 41  44  11   0   0   4|   0   108k| 104M  106M|   0     0 |  29k   52k|weaver        0    52M|weaver        54
24-02 20:03:34| 43  38  12   0   0   7|   0     0 | 108M  110M|   0     0 |  27k   50k|weaver       77B   53M|weaver        54
24-02 20:03:35| 39  42  12   0   0   7|   0     0 | 107M  110M|   0     0 |  28k   50k|weaver        0    54M|weaver        56
24-02 20:03:36| 38  45  12   0   0   6|   0    72k| 110M  112M|   0     0 |  28k   51k|weaver     1407B   54M|weaver        55
24-02 20:03:37| 36  42  13   0   0   9|   0     0 | 105M  107M|   0     0 |  31k   51k|weaver        0    52M|weaver        54
24-02 20:03:38| 39  42  12   0   0   7|   0    96k| 105M  107M|   0     0 |  26k   51k|weaver        0    52M|weaver        52
24-02 20:03:39| 40  38  13   0   0   9|   0     0 | 105M  108M|   0     0 |  29k   50k|weaver        0    52M|weaver        53
24-02 20:03:40| 46  36  12   0   0   7|   0     0 | 112M  114M|   0     0 |  26k   45k|weaver       14k   55M|weaver        56
24-02 20:03:41| 42  38  12   0   0   8|   0     0 | 107M  110M|   0     0 |  28k   52k|weaver        0    53M|weaver        55
24-02 20:03:42| 39  40  14   0   0   7|   0   140k| 109M  112M|   0     0 |  32k   51k|weaver        0    55M|weaver        56

Here are weave logs from the same timeframe:
weave 2015/02/24 20:03:01.072070 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:01.072428 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:01.856588 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:55309: i/o timeout
weave 2015/02/24 20:03:01.856820 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:03:01.857370 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:01.857479 ->[<IP_ADDRESS>:48716] attempting connection
weave 2015/02/24 20:03:01.857556 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:01.857625 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:03:01.859337 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:03:01.864622 ->[<IP_ADDRESS>:51390] connection accepted
weave 2015/02/24 20:03:01.865520 ->[<IP_ADDRESS>:51390] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:01.866127 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:01.869212 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:01.965484 ->[<IP_ADDRESS>:48716] error during connection attempt: dial tcp4 <IP_ADDRESS>:48716: connection refused
weave 2015/02/24 20:03:02.069558 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:02.069586 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:02.069611 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:02.069622 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:02.091428 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:03:02.137979 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:03:02.138161 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:03:02.138612 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:02.138755 ->[<IP_ADDRESS>:48716] attempting connection
weave 2015/02/24 20:03:02.138857 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:03:02.138981 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:02.139964 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:03:02.143524 ->[<IP_ADDRESS>:51393] connection accepted
weave 2015/02/24 20:03:02.143686 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:02.144359 ->[<IP_ADDRESS>:51393] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:02.144731 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:02.145254 ->[<IP_ADDRESS>:56597] connection accepted
weave 2015/02/24 20:03:02.145988 ->[<IP_ADDRESS>:56597] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:02.146032 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:02.159161 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:03:02.159281 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:03:02.159438 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:03:02.159443 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:02.161210 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:03:02.164069 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:02.217245 ->[<IP_ADDRESS>:51396] connection accepted
weave 2015/02/24 20:03:02.257623 ->[<IP_ADDRESS>:51396] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:02.258145 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:03.073536 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:03.073567 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:04.398885 Discovered remote MAC 02:42:c0:a8:04:2f at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:03:05.549208 Discovered remote MAC 02:42:c0:a8:04:30 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:03:05.577444 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:05.577473 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:06.143453 ->[<IP_ADDRESS>:56683] connection accepted
weave 2015/02/24 20:03:06.144649 ->[<IP_ADDRESS>:56683] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:06.144691 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:06.148679 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:03:06.148823 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:03:06.150078 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:03:06.150084 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:06.151373 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:03:06.153468 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:06.153968 ->[<IP_ADDRESS>:51407] connection accepted
weave 2015/02/24 20:03:06.154682 ->[<IP_ADDRESS>:51407] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:06.154992 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:06.573530 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:06.573570 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:07.222338 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:03:07.573554 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:07.573582 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:09.089334 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:09.089477 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:09.096299 ->[<IP_ADDRESS>:36514] connection accepted
weave 2015/02/24 20:03:09.096855 ->[<IP_ADDRESS>:36514] connection shutting down due to error during handshake: Already have connection to 7a:b5:2d:25:d1:cf at <IP_ADDRESS>:55467
weave 2015/02/24 20:03:09.113064 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:55467: i/o timeout
weave 2015/02/24 20:03:09.113224 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:03:09.113642 ->[<IP_ADDRESS>:51106] attempting connection
weave 2015/02/24 20:03:09.113758 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:09.113994 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:09.113894 ->[<IP_ADDRESS>:39140] attempting connection
weave 2015/02/24 20:03:09.115351 ->[<IP_ADDRESS>:51106] error during connection attempt: dial tcp4 <IP_ADDRESS>:51106: connection refused
weave 2015/02/24 20:03:09.117331 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:09.124112 ->[<IP_ADDRESS>:39140] error during connection attempt: dial tcp4 <IP_ADDRESS>:39140: connection refused
weave 2015/02/24 20:03:09.135035 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:03:09.161028 ->[<IP_ADDRESS>:48716] error during connection attempt: dial tcp4 <IP_ADDRESS>:48716: connection refused
weave 2015/02/24 20:03:09.265145 ->[<IP_ADDRESS>:56762] connection accepted
weave 2015/02/24 20:03:09.266014 ->[<IP_ADDRESS>:56762] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:03:09.266416 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:03:09.267726 Discovered remote MAC 02:42:c0:a8:0f:07 at 7a:21:54:68:1a:41
weave 2015/02/24 20:03:09.274689 Discovered remote MAC 02:42:c0:a8:0f:07 at 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:03:09.281272 ->[7a:b5:2d:25:d1:cf]: connection fully established
weave 2015/02/24 20:03:09.281862 EMSGSIZE on send, expecting PMTU update (IP packet was 60082 bytes, payload was 60074 bytes)
weave 2015/02/24 20:03:09.282686 ->[7a:b5:2d:25:d1:cf]: Effective PMTU set to 8921
weave 2015/02/24 20:03:11.591111 ->[<IP_ADDRESS>:56814] connection accepted
weave 2015/02/24 20:03:11.591968 ->[<IP_ADDRESS>:56814] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:11.592674 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:11.592779 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:11.600604 Discovered remote MAC 02:42:c0:a8:0e:05 at 7a:21:54:68:1a:41
weave 2015/02/24 20:03:11.621742 Discovered remote MAC 02:42:c0:a8:0e:05 at 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:11.624726 ->[<IP_ADDRESS>:51414] connection accepted
weave 2015/02/24 20:03:11.625619 ->[<IP_ADDRESS>:51414] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:11.629450 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:11.629472 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:12.279619 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:03:12.280177 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:03:12.280677 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:12.280783 ->[<IP_ADDRESS>:48716] attempting connection
weave 2015/02/24 20:03:12.280866 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:12.280969 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:03:12.285462 ->[<IP_ADDRESS>:51417] connection accepted
weave 2015/02/24 20:03:12.286256 ->[<IP_ADDRESS>:51417] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:12.286966 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:12.288942 ->[<IP_ADDRESS>:48716] error during connection attempt: dial tcp4 <IP_ADDRESS>:48716: connection refused
weave 2015/02/24 20:03:12.292296 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:03:12.296773 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:03:12.298027 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:17.185445 ->[<IP_ADDRESS>:56936] connection accepted
weave 2015/02/24 20:03:17.186217 ->[<IP_ADDRESS>:56936] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:03:23.625999 ->[<IP_ADDRESS>:57077] connection accepted
weave 2015/02/24 20:03:23.627191 ->[<IP_ADDRESS>:57077] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:23.627511 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:23.628780 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:23.633172 ->[<IP_ADDRESS>:51428] connection accepted
weave 2015/02/24 20:03:23.634427 ->[<IP_ADDRESS>:51428] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:23.634583 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:23.641148 ->[<IP_ADDRESS>:51429] connection accepted
weave 2015/02/24 20:03:23.641846 ->[<IP_ADDRESS>:51429] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:23.642485 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:23.642556 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:23.661916 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:03:23.662391 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:03:23.663095 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:23.663264 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:03:23.663447 ->[<IP_ADDRESS>:48716] attempting connection
weave 2015/02/24 20:03:23.663623 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:23.666825 ->[<IP_ADDRESS>:51431] connection accepted
weave 2015/02/24 20:03:23.667611 ->[<IP_ADDRESS>:51431] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:23.668264 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:23.693755 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:03:23.709431 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:24.146354 Expired MAC 02:42:c0:a8:04:0b at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:03:24.146497 Expired MAC 02:42:c0:a8:04:0c at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:03:27.902422 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:03:27.902835 ->[7a:02:0f:ac:19:f4]: connection deleted
weave 2015/02/24 20:03:27.903528 ->[<IP_ADDRESS>:59426] attempting connection
weave 2015/02/24 20:03:27.903704 ->[<IP_ADDRESS>:37885] attempting connection
weave 2015/02/24 20:03:27.903871 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:27.921297 ->[<IP_ADDRESS>:37885] error during connection attempt: dial tcp4 <IP_ADDRESS>:37885: connection refused
weave 2015/02/24 20:03:27.921374 ->[<IP_ADDRESS>:46776] connection accepted
weave 2015/02/24 20:03:27.921998 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:27.922539 ->[<IP_ADDRESS>:46776] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:03:27.922925 ->[7a:02:0f:ac:19:f4]: connection added
weave 2015/02/24 20:03:40.080152 ->[<IP_ADDRESS>:57406] connection accepted
weave 2015/02/24 20:03:40.080813 ->[<IP_ADDRESS>:57406] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:40.081551 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:03:40.081727 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:40.093441 ->[<IP_ADDRESS>:51454] connection accepted
weave 2015/02/24 20:03:40.096164 ->[<IP_ADDRESS>:51454] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:03:40.097874 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Multiple connections to 7a:8b:d8:d8:4a:88 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:03:40.129819 ->[7a:8b:d8:d8:4a:88]: connection fully established
weave 2015/02/24 20:03:40.130144 EMSGSIZE on send, expecting PMTU update (IP packet was 60082 bytes, payload was 60074 bytes)
weave 2015/02/24 20:03:40.130169 ->[7a:8b:d8:d8:4a:88]: Effective PMTU set to 8921
weave 2015/02/24 20:03:52.204139 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:53.203684 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:54.207921 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:56.950292 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:57.922912 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:03:57.923314 ->[7a:02:0f:ac:19:f4]: connection deleted
weave 2015/02/24 20:03:57.924292 ->[<IP_ADDRESS>:37885] attempting connection
weave 2015/02/24 20:03:57.924458 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:03:57.927489 ->[<IP_ADDRESS>:46801] connection accepted
weave 2015/02/24 20:03:57.928331 ->[<IP_ADDRESS>:46801] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:03:57.929000 ->[7a:02:0f:ac:19:f4]: connection added
weave 2015/02/24 20:03:57.929993 ->[<IP_ADDRESS>:37885] error during connection attempt: dial tcp4 <IP_ADDRESS>:37885: connection refused
weave 2015/02/24 20:03:57.947244 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:03:57.949009 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:03:58.947818 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:01.471911 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:02.472124 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:03.472171 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:04.275390 Discovered remote MAC 02:42:c0:a8:04:31 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:04:04.636086 Discovered remote MAC 02:42:c0:a8:04:33 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:04:04.663235 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:04.822977 Discovered remote MAC 02:42:c0:a8:04:32 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:04:05.667035 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:07.864592 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:09.281340 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:56762: i/o timeout
weave 2015/02/24 20:04:09.281592 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:04:09.282335 ->[<IP_ADDRESS>:39140] attempting connection
weave 2015/02/24 20:04:09.282437 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:09.282595 ->[<IP_ADDRESS>:51106] attempting connection
weave 2015/02/24 20:04:09.290593 ->[<IP_ADDRESS>:36562] connection accepted
weave 2015/02/24 20:04:09.291575 ->[<IP_ADDRESS>:36562] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:04:09.292279 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:04:10.289575 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:04:10.289800 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:04:10.291848 ->[<IP_ADDRESS>:51106] error during connection attempt: dial tcp4 <IP_ADDRESS>:51106: connection refused
weave 2015/02/24 20:04:10.293917 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:04:10.306662 ->[<IP_ADDRESS>:36565] connection accepted
weave 2015/02/24 20:04:10.308484 ->[<IP_ADDRESS>:36565] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:04:10.309153 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:04:10.315691 ->[<IP_ADDRESS>:58039] connection accepted
weave 2015/02/24 20:04:10.346478 ->[<IP_ADDRESS>:58039] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:04:10.347095 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:04:10.347118 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: Multiple connections to 7a:b5:2d:25:d1:cf added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:04:10.355307 Discovered remote MAC 02:42:c0:a8:0f:07 at 7a:21:54:68:1a:41
weave 2015/02/24 20:04:10.355379 Discovered remote MAC 02:42:c0:a8:0f:07 at 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:04:11.341501 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:11.343229 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:04:17.203182 ->[<IP_ADDRESS>:59426] error during connection attempt: dial tcp4 <IP_ADDRESS>:59426: connection refused
weave 2015/02/24 20:04:17.203309 ->[<IP_ADDRESS>:48716] error during connection attempt: dial tcp4 <IP_ADDRESS>:48716: connection refused
weave 2015/02/24 20:04:17.215443 ->[<IP_ADDRESS>:58197] connection accepted
weave 2015/02/24 20:04:17.220417 ->[<IP_ADDRESS>:58197] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:04:17.222657 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:04:20.315151 Discovered remote MAC 02:42:c0:a8:04:37 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:04:23.048036 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:24.045548 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:24.146860 Expired MAC 02:42:c0:a8:04:0f at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:04:25.045530 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:26.543545 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:27.541517 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:27.928641 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:04:27.928881 ->[7a:02:0f:ac:19:f4]: connection deleted
weave 2015/02/24 20:04:27.929272 ->[<IP_ADDRESS>:37885] attempting connection
weave 2015/02/24 20:04:27.929424 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:27.929513 ->[<IP_ADDRESS>:60854] attempting connection
weave 2015/02/24 20:04:27.929599 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:27.931217 ->[<IP_ADDRESS>:37885] error during connection attempt: dial tcp4 <IP_ADDRESS>:37885: connection refused
weave 2015/02/24 20:04:27.934233 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:04:27.934743 ->[<IP_ADDRESS>:46828] connection accepted
weave 2015/02/24 20:04:27.935508 ->[<IP_ADDRESS>:46828] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:04:27.935935 ->[7a:02:0f:ac:19:f4]: connection added
weave 2015/02/24 20:04:27.951002 ->[<IP_ADDRESS>:60854] error during connection attempt: dial tcp4 <IP_ADDRESS>:60854: connection refused
weave 2015/02/24 20:04:28.541521 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:28.946960 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:04:30.130134 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:31.129597 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:32.129530 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:34.208640 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:34.917125 ->[<IP_ADDRESS>:58584] connection accepted
weave 2015/02/24 20:04:34.917972 ->[<IP_ADDRESS>:58584] connection shutting down due to error during handshake: Already have connection to 7a:8f:17:40:9e:ce at <IP_ADDRESS>:51743
weave 2015/02/24 20:04:35.205515 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:36.205553 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:37.400786 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:38.397533 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:39.397509 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:40.128278 ->[<IP_ADDRESS>:51501] connection accepted
weave 2015/02/24 20:04:40.128993 ->[<IP_ADDRESS>:51501] connection shutting down due to error during handshake: Already have connection to 7a:8b:d8:d8:4a:88 at <IP_ADDRESS>:57406
weave 2015/02/24 20:04:40.129566 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:57406: i/o timeout
weave 2015/02/24 20:04:40.129713 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:04:40.130330 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:04:40.130347 ->[<IP_ADDRESS>:48716] attempting connection
weave 2015/02/24 20:04:40.130333 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:40.130426 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:40.131728 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:04:40.133796 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:04:40.138378 ->[<IP_ADDRESS>:48716] error during connection attempt: dial tcp4 <IP_ADDRESS>:48716: connection refused
weave 2015/02/24 20:04:40.158195 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:04:40.346707 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:04:40.347142 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:04:40.347696 ->[<IP_ADDRESS>:51106] attempting connection
weave 2015/02/24 20:04:40.348982 ->[<IP_ADDRESS>:51106] error during connection attempt: dial tcp4 <IP_ADDRESS>:51106: connection refused
weave 2015/02/24 20:04:40.350067 ->[<IP_ADDRESS>:36588] connection accepted
weave 2015/02/24 20:04:40.351701 ->[<IP_ADDRESS>:36588] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:04:40.352059 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:04:40.358011 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:04:40.358134 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:04:40.358847 ->[<IP_ADDRESS>:51106] attempting connection
weave 2015/02/24 20:04:40.359364 ->[<IP_ADDRESS>:58726] connection accepted
weave 2015/02/24 20:04:40.359904 ->[<IP_ADDRESS>:58726] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:04:40.360301 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:04:40.362837 ->[<IP_ADDRESS>:51106] error during connection attempt: dial tcp4 <IP_ADDRESS>:51106: connection refused
weave 2015/02/24 20:04:43.389867 ->[<IP_ADDRESS>:51506] connection accepted
weave 2015/02/24 20:04:43.390585 ->[<IP_ADDRESS>:51506] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:04:43.390941 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:04:57.935726 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:04:57.935991 ->[7a:02:0f:ac:19:f4]: connection deleted
weave 2015/02/24 20:04:57.936591 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:57.936754 ->[<IP_ADDRESS>:60854] attempting connection
weave 2015/02/24 20:04:57.936835 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:04:57.937038 ->[<IP_ADDRESS>:37885] attempting connection
weave 2015/02/24 20:04:57.939673 ->[<IP_ADDRESS>:46846] connection accepted
weave 2015/02/24 20:04:57.940466 ->[<IP_ADDRESS>:46846] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:04:57.940994 ->[7a:02:0f:ac:19:f4]: connection added
weave 2015/02/24 20:04:58.493558 ->[7a:b5:2d:25:d1:cf]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:58.493587 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:58.493616 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:59.493514 ->[7a:b5:2d:25:d1:cf]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:59.493651 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:04:59.494069 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:00.493536 ->[7a:b5:2d:25:d1:cf]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:00.493680 ->[7a:8b:d8:d8:4a:88]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:00.494135 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:10.360150 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:05:10.360945 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:05:10.361851 ->[<IP_ADDRESS>:51106] attempting connection
weave 2015/02/24 20:05:10.367619 ->[<IP_ADDRESS>:36644] connection accepted
weave 2015/02/24 20:05:10.368934 ->[<IP_ADDRESS>:51106] error during connection attempt: dial tcp4 <IP_ADDRESS>:51106: connection refused
weave 2015/02/24 20:05:10.370464 ->[<IP_ADDRESS>:36644] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:05:10.371201 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:05:10.868084 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:05:10.868355 ->[7a:b5:2d:25:d1:cf]: connection deleted
weave 2015/02/24 20:05:10.868900 ->[<IP_ADDRESS>:51106] attempting connection
weave 2015/02/24 20:05:10.869844 ->[<IP_ADDRESS>:51106] error during connection attempt: dial tcp4 <IP_ADDRESS>:51106: connection refused
weave 2015/02/24 20:05:10.872604 ->[<IP_ADDRESS>:36648] connection accepted
weave 2015/02/24 20:05:10.873900 ->[<IP_ADDRESS>:36648] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:05:10.874299 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:05:10.875728 ->[<IP_ADDRESS>:59357] connection accepted
weave 2015/02/24 20:05:10.876194 ->[<IP_ADDRESS>:59357] completed handshake with 7a:b5:2d:25:d1:cf
weave 2015/02/24 20:05:10.876554 ->[7a:b5:2d:25:d1:cf]: connection added
weave 2015/02/24 20:05:10.876567 ->[7a:b5:2d:25:d1:cf]: connection shutting down due to error: Multiple connections to 7a:b5:2d:25:d1:cf added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:05:10.891768 ->[7a:b5:2d:25:d1:cf]: connection fully established
weave 2015/02/24 20:05:10.892280 EMSGSIZE on send, expecting PMTU update (IP packet was 60082 bytes, payload was 60074 bytes)
weave 2015/02/24 20:05:10.892303 ->[7a:b5:2d:25:d1:cf]: Effective PMTU set to 8921
weave 2015/02/24 20:05:12.038548 Discovered remote MAC 02:42:c0:a8:04:34 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:12.393095 ->[<IP_ADDRESS>:39140] error during connection attempt: dial tcp4 <IP_ADDRESS>:39140: connection refused
weave 2015/02/24 20:05:12.590475 Discovered remote MAC 02:42:c0:a8:04:35 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:12.790484 Discovered remote MAC 02:42:c0:a8:04:36 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:12.975366 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:05:12.982374 ->[<IP_ADDRESS>:60854] error during connection attempt: dial tcp4 <IP_ADDRESS>:60854: connection refused
weave 2015/02/24 20:05:12.983122 ->[<IP_ADDRESS>:37885] error during connection attempt: dial tcp4 <IP_ADDRESS>:37885: connection refused
weave 2015/02/24 20:05:12.990534 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:05:13.390814 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:05:13.391059 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:05:13.391772 ->[<IP_ADDRESS>:48716] attempting connection
weave 2015/02/24 20:05:13.391872 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:05:13.392021 ->[<IP_ADDRESS>:48383] attempting connection
weave 2015/02/24 20:05:13.392160 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:05:13.406300 ->[<IP_ADDRESS>:51554] connection accepted
weave 2015/02/24 20:05:13.413732 ->[<IP_ADDRESS>:51554] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:05:13.415084 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:05:14.080939 ->[<IP_ADDRESS>:59422] connection accepted
weave 2015/02/24 20:05:14.111336 ->[7a:8b:d8:d8:4a:88]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:05:14.111446 ->[7a:8b:d8:d8:4a:88]: connection deleted
weave 2015/02/24 20:05:14.112465 ->[<IP_ADDRESS>:59422] completed handshake with 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:05:14.112791 ->[7a:8b:d8:d8:4a:88]: connection added
weave 2015/02/24 20:05:14.116828 Discovered remote MAC 02:42:c0:a8:0e:05 at 7a:21:54:68:1a:41
weave 2015/02/24 20:05:14.119626 Discovered remote MAC 02:42:c0:a8:0e:05 at 7a:8b:d8:d8:4a:88
weave 2015/02/24 20:05:15.611279 ->[7a:8b:d8:d8:4a:88]: connection fully established
weave 2015/02/24 20:05:15.611501 EMSGSIZE on send, expecting PMTU update (IP packet was 60082 bytes, payload was 60074 bytes)
weave 2015/02/24 20:05:15.611526 ->[7a:8b:d8:d8:4a:88]: Effective PMTU set to 8921
weave 2015/02/24 20:05:16.302088 ->[<IP_ADDRESS>:59476] connection accepted
weave 2015/02/24 20:05:16.318685 ->[<IP_ADDRESS>:59476] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:05:16.318736 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: Multiple connections to 7a:02:0f:ac:19:f4 added to 7a:8c:a1:e4:43:13
weave 2015/02/24 20:05:16.353354 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: Failed to decode packet: decryption failed; Unable to decrypt UDP packet
weave 2015/02/24 20:05:16.353784 ->[7a:02:0f:ac:19:f4]: connection deleted
weave 2015/02/24 20:05:16.354514 ->[<IP_ADDRESS>:37885] attempting connection
weave 2015/02/24 20:05:16.354620 ->[<IP_ADDRESS>:60854] attempting connection
weave 2015/02/24 20:05:16.357561 ->[<IP_ADDRESS>:46874] connection accepted
weave 2015/02/24 20:05:16.358311 ->[<IP_ADDRESS>:46874] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:05:16.358628 ->[7a:02:0f:ac:19:f4]: connection added
weave 2015/02/24 20:05:16.362781 ->[<IP_ADDRESS>:37885] error during connection attempt: dial tcp4 <IP_ADDRESS>:37885: connection refused
weave 2015/02/24 20:05:16.362844 ->[<IP_ADDRESS>:60854] error during connection attempt: dial tcp4 <IP_ADDRESS>:60854: connection refused
weave 2015/02/24 20:05:16.402120 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:05:16.408781 ->[<IP_ADDRESS>:48716] error during connection attempt: dial tcp4 <IP_ADDRESS>:48716: connection refused
weave 2015/02/24 20:05:16.416215 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:05:16.418201 ->[<IP_ADDRESS>:48383] error during connection attempt: dial tcp4 <IP_ADDRESS>:48383: connection refused
weave 2015/02/24 20:05:17.235033 ->[<IP_ADDRESS>:59495] connection accepted
weave 2015/02/24 20:05:17.235540 ->[<IP_ADDRESS>:59495] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:05:24.147495 Expired MAC 02:42:c0:a8:04:11 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:24.147528 Expired MAC 02:42:c0:a8:04:13 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:24.147541 Expired MAC 02:42:c0:a8:04:12 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:24.147585 Expired MAC 02:42:c0:a8:04:10 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:24.147596 Expired MAC 02:42:c0:a8:04:14 at 7a:87:ef:4c:d7:5e
weave 2015/02/24 20:05:46.358729 ->[7a:02:0f:ac:19:f4]: connection shutting down due to error: failed to establish UDP connectivity
weave 2015/02/24 20:05:46.359143 ->[7a:02:0f:ac:19:f4]: connection deleted
weave 2015/02/24 20:05:46.359770 ->[<IP_ADDRESS>:37885] attempting connection
weave 2015/02/24 20:05:46.359946 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:05:46.360061 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/24 20:05:46.360144 ->[<IP_ADDRESS>:34091] attempting connection
weave 2015/02/24 20:05:46.366337 ->[<IP_ADDRESS>:46898] connection accepted
weave 2015/02/24 20:05:46.367384 ->[<IP_ADDRESS>:46898] completed handshake with 7a:02:0f:ac:19:f4
weave 2015/02/24 20:05:46.367928 ->[7a:02:0f:ac:19:f4]: connection added
weave 2015/02/24 20:05:47.366631 ->[<IP_ADDRESS>:37885] error during connection attempt: dial tcp4 <IP_ADDRESS>:37885: connection refused
weave 2015/02/24 20:05:47.389590 ->[<IP_ADDRESS>:34091] error during connection attempt: dial tcp4 <IP_ADDRESS>:34091: connection refused
weave 2015/02/24 20:05:47.390524 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:21:54:68:1a:41 at <IP_ADDRESS>:6783
weave 2015/02/24 20:05:47.391186 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:43466
weave 2015/02/24 20:05:53.815920 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:54.815964 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:55.830275 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:58.193571 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:59.196209 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact
weave 2015/02/24 20:05:59.496691 ->[7a:02:0f:ac:19:f4]: Cannot forward frame yet - awaiting contact

Is the network traffic to/from port 6783, which weave uses?  And if so, is it UDP (data) or TCP (control)?
Can you get a packet capture?

I was able to narrow things down a bit more.  It appears to be limited to certain interfaces and the traffic does not go over eth0, so it appears to be localized to the instance.  I'm going to do some more digging.
----total-cpu-usage---- --net/eth0--net/veth31c-net/vethwep--net/weave---net/total- -------most-expensive-cpu-process-------
usr sys idl wai hiq siq| recv  send: recv  send: recv  send: recv  send: recv  send|process              pid  cpu read write
  7   5  88   0   0   0|   0     0 :   0     0 :   0     0 :   0     0 :   0     0 |revealcloud          8768 0.1%3914B 685B
 40  44   8   0   0   7|4666B 9089B:  56M  552B: 126B   57M:  56M   57M: 113M  115M|weaver               23750 51%   0   56M
 48  40   6   0   0   6|7291B 8037B:  54M  384B: 116B   55M:  54M   55M: 108M  110M|weaver               23750 49%   0   54M
 58  35   2   0   0   5|  13k   12k:  35M  266B:  42B   36M:  35M   36M:  70M   72M|weaver               23750 35%   0   35M
 66  31   0   0   0   4|7825B 8247B:  27M  164B:  42B   27M:  27M   27M:  53M   55M|scout                1761  47%   0    0
 57  35   3   0   0   4|  13k   18k:  37M  542B: 158B   37M:  37M   37M:  73M   75M|weaver               23750 37%   0   37M
 45  40   7   0   0   7|8867B   40k:  54M  850B: 200B   55M:  54M   55M: 107M  109M|weaver               23750 47%   0   53M
 46  41   7   0   0   5|  99k   13k:  53M 1963B: 126B   54M:  53M   54M: 106M  107M|weaver               23750 48%1405B  52M
 43  43   8   0   0   6|  17k   23k:  55M 9923B:1061B   56M:  55M   56M: 110M  112M|weaver               23750 50%5082B  55M
 47  41   6   0   0   6|  10k 9865B:  48M  820B: 210B   49M:  48M   50M:  97M   99M|weaver               23750 49%   0   51M^

At the moment, there appears to be two interfaces contributing to the network traffic: vethda62a1e which is the interface assigned to the "weave container" and the weave bridge interface (weave create-bridge):
# tcpdump -i weave -c 20
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on weave, link-type EN10MB (Ethernet), capture size 65535 bytes
03:44:18.240224 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 4525
03:44:18.240278 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-13-7.ec2.internal.6783: UDP, length 4525
03:44:18.240389 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:18.240436 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-13-7.ec2.internal.6783: UDP, length 8968
03:44:18.240446 IP ip-192-168-2-254.ec2.internal > ip-192-168-13-7.ec2.internal: udp
03:44:18.240509 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 776
03:44:18.240548 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-13-7.ec2.internal.6783: UDP, length 776
03:44:18.240641 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:18.240692 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-13-7.ec2.internal.6783: UDP, length 8968
03:44:18.240703 IP ip-192-168-2-254.ec2.internal > ip-192-168-13-7.ec2.internal: udp
03:44:18.240776 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 152
03:44:18.240814 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-13-7.ec2.internal.6783: UDP, length 152
03:44:18.240911 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 8968
03:44:18.240965 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-16-41.ec2.internal.6783: UDP, length 8968
03:44:18.240975 IP ip-192-168-2-254.ec2.internal > ip-192-168-16-41.ec2.internal: udp
03:44:18.241043 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 1608
03:44:18.241089 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-16-41.ec2.internal.6783: UDP, length 1608
03:44:18.241195 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 8968
03:44:18.241253 IP ip-192-168-2-254.ec2.internal.6783 > ip-192-168-16-41.ec2.internal.6783: UDP, length 8968
03:44:18.241266 IP ip-192-168-2-254.ec2.internal > ip-192-168-16-41.ec2.internal: udp
20 packets captured
47 packets received by filter
0 packets dropped by kernel

# tcpdump -i vethda62a1e -c 20
tcpdump: WARNING: vethda62a1e: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vethda62a1e, link-type EN10MB (Ethernet), capture size 65535 bytes
03:44:23.473409 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:23.473521 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 8968
03:44:23.473642 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 2024
03:44:23.473748 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 3672
03:44:23.473889 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:23.474132 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 4376
03:44:23.474271 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:23.474548 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 3480
03:44:23.474687 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:23.474820 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 3480
03:44:23.474982 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:23.475109 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 8968
03:44:23.475237 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 1816
03:44:23.475344 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 3480
03:44:23.475486 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 8968
03:44:23.475630 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
03:44:23.475763 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 3896
03:44:23.475882 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 3640
03:44:23.476278 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.40222: UDP, length 8968
03:44:23.476423 IP ip-192-168-2-22.ec2.internal.6783 > ip-192-168-2-254.ec2.internal.57494: UDP, length 8968
20 packets captured
45 packets received by filter
0 packets dropped by kernel

# docker inspect weave | grep 192
        "Gateway": "<IP_ADDRESS>",
        "IPAddress": "<IP_ADDRESS>",

Eventually it settles down.  I'm not sure if it comes back on its own, but it definitely comes back if I manually restart the weave container.

Thanks, it's very helpful that you did that 'docker inspect' - it says that what we're seeing in those captures is traffic between the weave container (<IP_ADDRESS>) and the docker bridge (<IP_ADDRESS>), and then traffic between the docker bridge and other IP addresses like 192-168-13-7.
This traffic should definitely not be on the weave bridge.
"vethda62a1e" looks like a veth created by Docker; the traffic you show is expected there.
It may help to see your iptables rules; the iptables-save command can export them all in one go.

Here are the iptables rules:
https://gist.github.com/bmurphy1976/6434b97da136a5e52a8f

Looking through the rules I also see a holdover from when the weave interface was configured with a bad network:
-A POSTROUTING -s <IP_ADDRESS>/8 ! -o weave -j MASQUERADE
-A WEAVE ! -s <IP_ADDRESS>/8 -o weave -j MASQUERADE
-A WEAVE -s <IP_ADDRESS>/8 ! -o weave -j MASQUERADE
-A WEAVE ! -s <IP_ADDRESS>/16 -o weave -j MASQUERADE
-A WEAVE -s <IP_ADDRESS>/16 ! -o weave -j MASQUERADE

should be <IP_ADDRESS>/16 not /8.  The /8 was a mis-configuration we deployed last week.
I'm tempted to reprovision this instance so it comes up in a known clean state.  I could still be fighting cruft from this earlier misconfiguration, but I'm also hesitant because if the problem goes away I may not be able to reproduce it and it could come back to bite us later.

I configured weave and docker following the instructions in this article: http://sttts.github.io/docker/weave/mesos/2015/01/22/weave.html
root@appcluster01-us-east-1b-host01:~# cat /etc/network/interfaces.d/weave.cfg
auto weave
iface weave inet manual
        pre-up /usr/local/sbin/weave create-bridge
        post-up /usr/local/sbin/weave expose <IP_ADDRESS>/16
        pre-down /usr/local/sbin/weave hide <IP_ADDRESS>/16 && ifconfig weave down
        post-down brctl delbr weave

root@appcluster01-us-east-1b-host01:~# cat /etc/default/docker
DOCKER_OPTS="-r=false -g /mnt/docker -H tcp://<IP_ADDRESS>:4243 -H unix:///var/run/docker.sock -s aufs --bridge=weave --fixed-cidr=<IP_ADDRESS>/25"

OK, that would have been useful to know.

Sorry, I have a lot balls in the air at the moment.  To my knowledge that is the only special about our deployments to these servers.

I'm wondering about this command: weave expose <IP_ADDRESS>/16, which you run from weave.cfg.  This will have the effect of routing all traffic for the 192.168 network onto the weave bridge.
Then, you seem to have other peers connecting from addresses like <IP_ADDRESS> and <IP_ADDRESS>, which are in that network.
Have I got that right?  The 'outside' IP addresses where Weave peers connect, and the 'inside' addresses that you want to use on the Weave network, must be disjoint.

Here is what I did in our cluster.

SERVER
DOCKER
BRIDGE

reserved
<IP_ADDRESS>/25
n/a

A
<IP_ADDRESS>/25
<IP_ADDRESS>

B
<IP_ADDRESS>/25
<IP_ADDRESS>

C
<IP_ADDRESS>/25
<IP_ADDRESS>

The /25 networks were chosen so that the ip addresses docker allocated would not conflict with the ip address assigned to the bridge.
I did this so that I could deterministically manage ip addresses without having to introduce a centralized ip address coordination service (ala dhcp or etcd).
If I understand what you are saying, it might be better for me to do the following:

SERVER
DOCKER
BRIDGE

A
192.168.1.x/24
<IP_ADDRESS>

B
192.168.2.x/24
<IP_ADDRESS>

C
192.168.3.x/24
<IP_ADDRESS>

Or maybe I misunderstand what you mean by disjoint and this isn't disjoint enough.

Two things:

you've got <IP_ADDRESS>/16 on the weave expose.  This /16 envelops all the /25 networks.  Won't work.
you should weave connect using the real IP addresses of the machines where Weave is running.

I pulled the /16 from Stefan's article (linked above) so it sounds like that was bad advice and I have more to understand here.
As for weave connect, I'm using the following "weave launch -password PASSWORD [cname to known server] where cname resolves to either private or public ip address depending on if the server is hosted within the same Amazon account.

I'm reviewing the article, I either misunderstood something or confused it with something else I was reading at the time.  I think I see what you are saying now.  I'm going to review all of our setting from top to bottom and follow up with what I find.

The problem I'm talking about is that Weave thinks it should connect to other peers at addresses like <IP_ADDRESS>; since this is inside your weave expose range the traffic will get routed round in a loop causing your symptoms of excessive CPU and network usage.
I had supposed you were supplying that address via weave connect or weave launch, but your comment about cname suggests not.  It's possible this is a side-effect of something else; can I get the first 50 lines or so of some of the weave logfiles so I can see how they get hold of these addresses?

weave 2015/02/27 16:54:41.147071 Command line options: map[name:7a:21:54:68:1a:41 wait:20 iface:ethwe]
weave 2015/02/27 16:54:41.147169 Command line peers: [appcluster02.CENSORED.com]
weave 2015/02/27 16:54:42.166419 Communication between peers is encrypted.
weave 2015/02/27 16:54:42.167253 Our name is 7a:21:54:68:1a:41
weave 2015/02/27 16:54:42.218374 Sniffing traffic on &{112343 65535 ethwe 4e:82:21:81:08:94 up|broadcast|multicast}
weave 2015/02/27 16:54:42.218434 Discovered our MAC 4e:82:21:81:08:94
weave 2015/02/27 16:54:42.225928 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:42.228421 ->[<IP_ADDRESS>:6783] completed handshake with 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:54:42.229074 ->[7a:87:ef:4c:d7:5e]: connection added
weave 2015/02/27 16:54:42.240050 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:54:42.240254 ->[<IP_ADDRESS>:33902] attempting connection
weave 2015/02/27 16:54:42.240283 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:42.240626 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:42.240791 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:42.240868 Discovered local MAC 02:42:c0:a8:01:1d
weave 2015/02/27 16:54:42.240890 ->[<IP_ADDRESS>:37168] attempting connection
weave 2015/02/27 16:54:42.242278 ->[<IP_ADDRESS>:6783] completed handshake with 7a:8c:a1:e4:43:13
weave 2015/02/27 16:54:42.242572 ->[7a:8c:a1:e4:43:13]: connection added
weave 2015/02/27 16:54:42.247263 ->[7a:8c:a1:e4:43:13]: connection fully established
weave 2015/02/27 16:54:42.248094 EMSGSIZE on send, expecting PMTU update (IP packet was 60082 bytes, payload was 60074 bytes)
weave 2015/02/27 16:54:42.248120 ->[7a:8c:a1:e4:43:13]: Effective PMTU set to 8921
weave 2015/02/27 16:54:42.249826 ->[7a:8c:a1:e4:43:13]: Effective PMTU verified at 8921
weave 2015/02/27 16:54:42.361444 Discovered local MAC 7a:ac:57:fd:13:11
weave 2015/02/27 16:54:42.729650 ->[7a:87:ef:4c:d7:5e]: connection fully established
weave 2015/02/27 16:54:42.730106 EMSGSIZE on send, expecting PMTU update (IP packet was 60082 bytes, payload was 60074 bytes)
weave 2015/02/27 16:54:42.730134 ->[7a:87:ef:4c:d7:5e]: Effective PMTU set to 8921
weave 2015/02/27 16:54:42.734517 ->[7a:87:ef:4c:d7:5e]: Effective PMTU verified at 8921
weave 2015/02/27 16:54:43.238297 Discovered remote MAC 7a:8c:a1:e4:43:13 at 7a:8c:a1:e4:43:13
weave 2015/02/27 16:54:43.239231 ->[<IP_ADDRESS>:33902] error during connection attempt: dial tcp4 <IP_ADDRESS>:33902: connection refused
weave 2015/02/27 16:54:43.239296 Discovered remote MAC 7a:87:ef:4c:d7:5e at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:54:43.241462 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:54:43.241503 ->[<IP_ADDRESS>:37168] error during connection attempt: dial tcp4 <IP_ADDRESS>:37168: connection refused
weave 2015/02/27 16:54:43.243027 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:43.243376 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:44.124888 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:44.128376 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:45.282446 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:45.287740 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:45.491320 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:54:45.492973 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:54:46.997767 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:47.001073 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:47.834777 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:54:47.835958 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:54:48.160116 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:48.164526 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:49.530927 ->[<IP_ADDRESS>:33902] attempting connection
weave 2015/02/27 16:54:49.532112 ->[<IP_ADDRESS>:33902] error during connection attempt: dial tcp4 <IP_ADDRESS>:33902: connection refused
weave 2015/02/27 16:54:49.867231 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:49.870291 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:50.447778 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:54:50.449157 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:54:53.851543 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:54:53.852811 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:54:54.802326 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:54.805658 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:55.455821 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:54:55.459012 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:54:57.553465 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:54:57.554509 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:55:01.465502 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:55:01.473866 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:6783
weave 2015/02/27 16:55:01.670146 ->[<IP_ADDRESS>:33902] attempting connection
weave 2015/02/27 16:55:01.672131 ->[<IP_ADDRESS>:33902] error during connection attempt: dial tcp4 <IP_ADDRESS>:33902: connection refused
weave 2015/02/27 16:55:04.241713 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:55:04.249189 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:55:05.037337 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:55:05.041465 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:8c:a1:e4:43:13 at <IP_ADDRESS>:6783
weave 2015/02/27 16:55:05.963053 Discovered remote MAC 02:42:c0:a8:04:44 at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:55:06.101378 Discovered remote MAC 02:42:c0:a8:04:43 at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:55:06.464400 Discovered remote MAC 02:42:c0:a8:04:45 at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:55:06.585662 Discovered remote MAC 02:42:c0:a8:04:47 at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:55:07.235022 Discovered remote MAC 02:42:c0:a8:04:46 at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:55:07.371076 Discovered remote MAC 02:42:c0:a8:04:48 at 7a:87:ef:4c:d7:5e
weave 2015/02/27 16:55:12.918750 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/27 16:55:12.923749 ->[<IP_ADDRESS>:6783] connection shutting down due to error during handshake: Already have connection to 7a:87:ef:4c:d7:5e at <IP_ADDRESS>:6783
weave 2015/02/27 16:55:13.820416 ->[<IP_ADDRESS>:55303] attempting connection
weave 2015/02/27 16:55:13.821669 ->[<IP_ADDRESS>:55303] error during connection attempt: dial tcp4 <IP_ADDRESS>:55303: connection refused
weave 2015/02/27 16:55:15.880568 ->[<IP_ADDRESS>:33902] attempting connection
weave 2015/02/27 16:55:15.881460 ->[<IP_ADDRESS>:33902] error during connection attempt: dial tcp4 <IP_ADDRESS>:33902: connection refused

OK, the <IP_ADDRESS> and <IP_ADDRESS> addresses are picked up as the 'remote' end of existing TCP connections between peers.  This is, I think, a consequence of the weave expose which sets up an iptables MASQUERADE rule so packets coming in to the network appear to come from a different address.
Ping @sttts: can I ask what is the reason for doing this weave expose?  Your article suggests it is to assign an IP address to the bridge.

I've done some more testing on another simpler cluster that I created from scratch so it has a better chance of not being corrupted by previous configuration errors.
If the weave interface does not have an ip address assigned, Docker refuses to start with this error message: Interface weave has no IP addresses
For anybody else who may be following along and trying to do the same thing in the future, I found another problem.  Configuring the weave interface appears to be a little bit racey.  I had at least one instance where I rebooted the machine and the weave interface appears to have been created and docker started before the ip address was assigned to the interface.  I added sleep 2s to weave.cfg.  I don't know if that helped or not, but I haven't seen this problem since adding that.
weave.cfg:
auto weave
iface weave inet manual
        pre-up /usr/local/sbin/weave create-bridge
        post-up /usr/local/sbin/weave expose <IP_ADDRESS>/16 && sleep 2s
        pre-down /usr/local/sbin/weave hide <IP_ADDRESS>/16 && ifconfig weave down
        post-down brctl delbr weave

At the moment, I have a new cluster using the <IP_ADDRESS> gateway <IP_ADDRESS>/8 network setup I proposed earlier.  I have not throw any kind of real load at it, but for the moment it appears to be behaving.  At the very least the weave output logs are significantly quieter than my other cluster.
weave status still shows a mixture of LAN (<IP_ADDRESS>/8) ip address and weave (<IP_ADDRESS>/16) ip addresses:
weave router 0.9.0
Encryption off
Our name is 7a:5b:f5:54:a8:19
Sniffing traffic on &{23 65535 ethwe 06:50:1f:9b:84:4a up|broadcast|multicast}
MACs:
02:42:c0:a8:03:01 -> 7a:d5:87:cd:79:f5 (2015-02-27 22:04:38.928646919 +0000 UTC)
02:42:c0:a8:01:01 -> 7a:5b:f5:54:a8:19 (2015-02-27 22:04:39.098451247 +0000 UTC)
7a:5b:f5:54:a8:19 -> 7a:5b:f5:54:a8:19 (2015-02-27 22:04:39.186378491 +0000 UTC)
92:8d:39:55:86:34 -> 7a:5b:f5:54:a8:19 (2015-02-27 22:04:39.266277973 +0000 UTC)
02:42:c0:a8:01:02 -> 7a:5b:f5:54:a8:19 (2015-02-27 22:04:43.723063696 +0000 UTC)
06:50:1f:9b:84:4a -> 7a:5b:f5:54:a8:19 (2015-02-27 22:04:38.923209653 +0000 UTC)
Peers:
Peer 7a:5b:f5:54:a8:19 (v4) (UID<PHONE_NUMBER>092232142)
   -> 7a:78:20:61:58:69 [<IP_ADDRESS>:6783]
   -> 7a:d5:87:cd:79:f5 [<IP_ADDRESS>:6783]
Peer 7a:78:20:61:58:69 (v7) (UID<PHONE_NUMBER>549192172)
   -> 7a:5b:f5:54:a8:19 [<IP_ADDRESS>:59970]
   -> 7a:d5:87:cd:79:f5 [<IP_ADDRESS>:6783]
Peer 7a:d5:87:cd:79:f5 (v7) (UID<PHONE_NUMBER>738776459)
   -> 7a:5b:f5:54:a8:19 [<IP_ADDRESS>:58458]
   -> 7a:78:20:61:58:69 [<IP_ADDRESS>:43597]
Routes:
unicast:
7a:5b:f5:54:a8:19 -> 00:00:00:00:00:00
7a:78:20:61:58:69 -> 7a:78:20:61:58:69
7a:d5:87:cd:79:f5 -> 7a:d5:87:cd:79:f5
broadcast:
7a:5b:f5:54:a8:19 -> [7a:78:20:61:58:69 7a:d5:87:cd:79:f5]
7a:78:20:61:58:69 -> []
7a:d5:87:cd:79:f5 -> []
Reconnects:

@bboreham the reason is to assign an IP for hose such that port forwardings -p 32080:80 keep working. In my case, I am running Mesos with Marathon. Neither knows about the internal native ports yet, so things like healthchecks go through the exposed IP via the iptables rules by docker.
In my proof of concept implementation of the article I actually configure the exposed interface of the host as <IP_ADDRESS>/8, covering all sub-networks 10.x.0.0/16 (x!=0), i.e. like @bmurphy1976 does. I didn't see any masquerading of packets at that time. I can check my installation. I still have it as Vagrant machine I think.

Checked my setup, masquerading is setup the same way as in the iptables-save example above. I also see @bmurphy1976's mixed peer IPs in weave status.
@bboreham can you elaborate more on which packets are routed in a loop and why? I don't see this here. tcpdump between the machines only shows udp packets using the private LAN ips. tcpdump on the weave interface only shows the application traffic with the expected container IPs. So nothing unusual here.

I replicated your set-up to confirm what is happening.
As you have seen, Weave has amongst its connections various IP addresses, some of which are 'real' (e.g. <IP_ADDRESS>) and some of which are masqueraded (e.g. <IP_ADDRESS>):
Peer 7a:8f:17:40:9e:ce (v641) (UID<PHONE_NUMBER>4184655389)
   -> 7a:87:ef:4c:d7:5e [<IP_ADDRESS>:6783]
   -> 7a:21:54:68:1a:41 [<IP_ADDRESS>:6783]
   -> 7a:8c:a1:e4:43:13 [<IP_ADDRESS>:6783]
Peer 7a:87:ef:4c:d7:5e (v337) (UID<PHONE_NUMBER>833339324)
   -> 7a:02:0f:ac:19:f4 [<IP_ADDRESS>:41517]
   -> 7a:8f:17:40:9e:ce [<IP_ADDRESS>:55303]

At https://gist.github.com/bmurphy1976/e93080a4a04a368109fe we see Weave connecting with another peer at a masqueraded address:
weave 2015/02/25 01:31:49.327169 ->[<IP_ADDRESS>:6783] attempting connection
weave 2015/02/25 01:31:49.750453 ->[<IP_ADDRESS>:6783] completed handshake with 7a:21:54:68:1a:41
weave 2015/02/25 01:31:49.767076 ->[7a:21:54:68:1a:41]: connection fully established

This is standard Weave behaviour: it tries to connect to any IP address it sees connected elsewhere, and it tries both on the port seen there and on its standard port (6783).
However, the only way it can make an outgoing connection to <IP_ADDRESS> is across the Weave network.  So now we have a loop: every time we try to send data to that peer it needs to be encapsulated and sent across the Weave network, which in turn will get encapsulated, and so on.
In order to get into this state, the peer has to have an existing connection to the Weave network, and it needs to succeed in connecting via the masqueraded address before it connects via the 'real' address.  I needed to set up four peers before I could force this to happen, and it only happens about one in every three attempts.

Within my replica set-up, I tried just giving the weave bridge an IP address without the masquerade rules, and that let me bring up a stable weave network. I don't know if this is sufficient to do what you need with Mesos/Marathon.
The command I used was:
sudo ip addr add dev weave <CIDR>

Thanks for the detailed analysis.
I configured weave now as suggested: without expose, but by assigning only the IP. The setup still works fine. It's not surprising because for internet routing and access to the private LAN ips of the hosts, it is enough to have the remaining masquerading rule:
-A POSTROUTING -s <IP_ADDRESS>/16 ! -o weave -j MASQUERADE

@bmurphy1976 Updated my blog post: http://sttts.github.io/docker/weave/mesos/2015/01/22/weave.html#expose-update-remark

@sttts awesome.  I'm going to redo our test cluster as well and see how that impacts things.

I have tested it in a small Vagrant setup. If you find problems in a bigger setup, of course I am interested :-)

I've managed to test a small cluster and everything looked good.
I'm slowly updating our existing cluster.  I've managed to replace a few nodes in the cluster and those nodes are behaving better.  Tomorrow I plan to replace the last two most troublesome nodes (including the one that had major CPU usage issues).
I should have a good idea in the next day or two how much better things are running in a live and realistic environment (8 servers, 3 high utilization, 4 medium/low utilization).

Ok, I'm going to close these two issues. Feel free to reopen if you get further info.

Our cluster has been fully upgraded and is running at load.   No issues so far, things look great!

Great new! Thanks for pointing out the issue and the support to fix it.


Extend title definitions to beyond parameters in a flow definition

Extend title definitions to beyond parameters in a flow definition
Description of Problem:
Currently in a flow definition the title property of a parameter can be set. Users may want to add bespoke titles in a flow (overriding and native title attribute of the object) for all objects in a flow.
Potential Solutions:

Add the ability to set titles to all object types used as inputs in a flow by extending the current definitions.
Alternatively this could be restricted to just data objects which is where I see the greatest need.

This was addressed in #311 and #334


IRGen: simplify code generation for enum switch statements

This change uses the 'gather bits' functionality of enum payloads
to create a contiguous value to switch over. This allows us to
remove the code that currently attempts to build a switch statement
by comparing each element in the payload in turn.
The downside of this technique is that we may do more work up front
gathering bits and we may also need to compare larger values in some
situations. The upside is that we can remove a lot of complicated
code from IRGen. Also, we pass the responsibility for multi-way
branch generation to LLVM which can make use of a wider range of
switch lowering strategies than IRGen can sensibly support.

@jckarter I think it now makes sense to let LLVM build the switch statement, what do you think?
If LLVM still isn't great at generating switch statements over multi-word integers then we could add one or more pre-processing passes to reduce the size of the mask by e.g. looking for common bit patterns across the case values, removing them from the mask and branching away before the switch if the bit pattern is incorrect. But I haven't had any problems so far while testing this and I think it's better to let LLVM handle that if we can get away with it.

LLVM generates pretty lousy assembly language from our current code generation patterns, so this may in fact end up being better. It'd be interesting to see the impact it has on our benchmark suite.

@swift-ci Please benchmark

@swift-ci Please test

Performance: -O
Code size: -O
Performance: -Osize

Improvement
OLD
NEW
DELTA
RATIO

PrefixCountableRangeLazy
18
15
-16.7%
1.20x (?)

Code size: -Osize
Performance: -Onone

Improvement
OLD
NEW
DELTA
RATIO

DataCountSmall
50
45
-10.0%
1.11x (?)

Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

Build failed
Swift Test OS X Platform
Git Sha - d12de33db4993173c368985b57540e22ef8b02d0

Looks like one test still needs updating for 32-bit:
11:59:14 ******************** TEST 'Swift(iphonesimulator-i386) :: IRGen/enum.sil' FAILED ********************
11:59:14 Script:
11:59:14 --
11:59:14 : 'RUN: at line 3';   /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/bin/swiftc -frontend -target i386-apple-ios7.0  -module-cache-path '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/swift-test-results/i386-apple-ios7.0/clang-module-cache' -sdk '/Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk' -swift-version 4   /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/IRGen/enum.sil -gnone -emit-ir -disable-diagnostic-passes -enable-objc-interop  | '/usr/bin/python' '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/utils/PathSanitizingFileCheck' --sanitize BUILD_DIR='/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64' --sanitize SOURCE_DIR='/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift' --use-filecheck '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/llvm-macosx-x86_64/bin/FileCheck'  /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/IRGen/enum.sil --check-prefix=CHECK --check-prefix=CHECK-32 --check-prefix=CHECK-objc --check-prefix=CHECK-objc-32 --check-prefix=CHECK-objc-32-simulator-true -DWORD=i32
11:59:14 : 'RUN: at line 4';   /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/bin/swiftc -frontend -target i386-apple-ios7.0  -module-cache-path '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/swift-test-results/i386-apple-ios7.0/clang-module-cache' -sdk '/Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk' -swift-version 4   /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/IRGen/enum.sil -gnone -emit-ir -disable-diagnostic-passes -disable-objc-interop | '/usr/bin/python' '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/utils/PathSanitizingFileCheck' --sanitize BUILD_DIR='/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64' --sanitize SOURCE_DIR='/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift' --use-filecheck '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/llvm-macosx-x86_64/bin/FileCheck'  /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/IRGen/enum.sil --check-prefix=CHECK --check-prefix=CHECK-32 --check-prefix=CHECK-native --check-prefix=CHECK-native-32 -DWORD=i32
11:59:14 --
11:59:14 Exit Code: 1
11:59:14
11:59:14 Command Output (stderr):
11:59:14 --
11:59:14
/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/IRGen/enum.sil:897:14: error: CHECK-32: expected string not found in input
11:59:14 // CHECK-32: [[IS_Y:%.*]] = icmp eq i32 %1, -2147483648
11:59:14              ^
11:59:14 <stdin>:1261:12: note: scanning from here
11:59:14 ; <label>:5: ; preds = %entry
11:59:14            ^
11:59:14 <stdin>:1284:2: note: possible intended match here
11:59:14  %4 = inttoptr i32 %1 to i8**
11:59:14  ^
11:59:14
11:59:14 --
11:59:14
11:59:14 ********************

Thanks @jckarter. I've updated the test.

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - d12de33db4993173c368985b57540e22ef8b02d0

@mundaym BTW, if you'd like to get commit access, I think you've more than met the requisite five patches to ask: https://swift.org/contributing/#commit-access


Ensure we clean up CancellationTokenSources

We use CancellationTokenSource quite a bit in RestChannel. It can be hard to work out when to dispose of it, particularly in terms of what gRPC might do for us. However, we really should do so.
There's more info in https://stackoverflow.com/questions/6960520/when-to-dispose-cancellationtokensource.

This was fixed by #661, but I used the wrong issue number :(


After! email not being called

I have the following two lines
(after! helm (message "HELM: This is a test"))
(after! email (message "EMAIL: This is a test"))

But for some reason, the first message is printed while the second is not. I have email in my init.el to be loaded and I can call mu4e functions (so it was loaded correctly). Am I doing something wrong?

I misunderstood the documentation. I corrected this to
(after! mu4e (message "EMAIL: This is a test"))
and it seems to work.


use_arrow=True vs False: different empty dataframe result when no columns fetched

When reading no columns from a file, this results in an empty dataframe with no columns using read_dataframe, you get a different result depending on use_arrow:

if False: you get a DataFrame with 0 rows
if True: you get a DataFrame with the number of rows of the input file, containing only the index.

Not sure what is the expected behaviour. For what it is worth, fiona seems to do the same as use_arrow=False.
Script to reproduce:
import pyogrio

url = "https://github.com/theroggy/pysnippets/raw/main/pysnippets/pyogrio/polygon-parcel_31370.zip"
for use_arrow in [True, False]:
    gdf = pyogrio.read_dataframe(url, use_arrow=use_arrow, columns=[], read_geometry=False)
    print(f"\nuse_arrow={use_arrow}, len(gdf): {len(gdf)}, gdf:\n{gdf}")

relevant output:
use_arrow=True, len(gdf): 46, gdf:
Empty DataFrame
Columns: []
Index: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45]

use_arrow=False, len(gdf): 0, gdf:
Empty DataFrame
Columns: []
Index: []

Otherwise, if we just preemptively return an empty data frame / table, then it may out of sync with a read against the actual data source, which may fail for any number of reasons. I.e., it might be better that we don't succeed when reading no geometry / columns / FID when we'd fail otherwise if any of those are true / non-empty

IMO it would make sense to error instead of returning an empty table, as that's more likely to imply user error that columns was empty

Alternatively, at the end of read_arrow we could detect if the arrow table has no columns after attempting to read it, and then return a truly empty table:

I personally wouldn't do this, as one could argue that the the current result with use_arrow=True is more correct than a "truly" empty table. Because you did read all rows, just no columns ..
But certainly fine with detecting this case and raising an error. I think it will typically indeed be a user error.

I also can't really think of use cases where reading no columns nor a geometry from a file would lead to something useful... so I agree it smells like a user error...
When fid_as_index=True you could return some data even though no columns nor geometry is asked, but it still can't really think of a use for this either...


Prometheus is pulled from community-operators

without this change, prometheus won't install anymore during quickstart or run-ci.

How to test/verify: on a working stf deployment, run a
oc get subscription prometheus -ojson | jq '.spec'

The output I get after this change is
{
  "channel": "beta",
  "installPlanApproval": "Automatic",
  "name": "prometheus",
  "source": "community-operators",
  "sourceNamespace": "openshift-marketplace"
}

I'm not having the same issue with run-ci.yaml. Are you using the latest run-ci.yaml from master? Can I see some logs that show the issue? I have been unable to reproduce this.


introduction video + launchpass site

Today I'm going to try to work on:

script
video mockup

Done!


JDBC Drivers for MariaDB (> v3.0.4) do not allow LOAD DATA LOCAL INFILE if ResultSet fetch size is set

Hi, I experienced exactly the same error on DBeaver 23.1.1 on macOS Big Sur & Windows, MariaDB version "10.5.20-MariaDB-log" on FreeBSD 13, JDBC driver MariaDB Connector/J 3.1.4. The fix mentioned here did not work. Rolling back to MariaDB Connector/J 3.0.4 fixed this issue. The same error occurred for each of the JDBC versions between 3.0.5 and 3.1.4.
Using the latest mySQL JDBC connector for MariaDB works as expected, i.e. it does not show the error described here.

Thanks for the info and testing, hkirschk!
Version 3.2.0 of the MariaDB Connector/J also does not work but rolling back to 3.0.4 does the trick
It should also be noted that the following allows the 3.2.0 driver to work:
Set the ResultSet fetch size parameter to 0

This, I believe, is a bug in DBeaver. DBeaver should detect a LOAD DATA statement and turn off the fetch size for the transaction.
Setting this value can be accomplished through several clicks, which is an annoyance, or running a simple select (such as SELECT 1) and then changing the value in the row limit box and THEN running the LOAD DATA command.
In both instances, it is necessary to remember to put the limit back after the load is done.
Originally posted by @JTMosaic in https://github.com/dbeaver/dbeaver/issues/19115#issuecomment-1755506069

Hello @JTMosaic
Please provide clear steps to reproduce.

Download the attached test file and save it to a known location testdata.csv

Ensure your DBeaver settings (global and for the connection) have a ResultSet fetch size > 0

Connect to a MariaDB server using a driver version > 3.0.4

Create a table in a schema of your choice:

CREATE TABLE loadtest (
row_id int,
value varchar(10));

Adjust the following script to point to the file saved in step 1

into table loadtest
fields terminated by ','
ignore 1 lines;

Run the script. The following error will be returned:

org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error [HY000]: (conn=32110) LOAD DATA LOCAL INFILE asked for file '/testdata' that doesn't correspond to initial query SET STATEMENT SQL_SELECT_LIMIT=1000 FOR load data local infile '/testdata'
into table loadtest
fields terminated by ','
ignore 1 lines. Possible malicious proxy changing server answer ! Command interrupted

Set the ResultSet fetch size to 0 and run the script in step 6 again
Import successful

I reproduced it. Thanks for your report!


NEXT-21659: Allow updating custom and credit line items without ALLOW_PRODUCT_PRICE_OVERWRITES permission, when no prices are being updated

Fix for NEXT-21659: Allow updating custom and credit line items without ALLOW_PRODUCT_PRICE_OVERWRITES permission, when no prices are being updated.
The product line item type only checks for this permission when a price_definition is set in the date -> so only when an actual price change occurs.
Modified both the custom and credit type line items with similar behaviour.
https://issues.shopware.com/issues/NEXT-21659

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hey,
I will close this pull request as it's inactive for a while. Feel free to create a new one, if you have updated your code.
Anyway thanks!


Manager is not showing newly added field in the table but it shows up in the information panel in editing mode

Reported during Manager bug out event
I published a Manager Instant app using a basic web map with two layers.  After publishing, I added a field to one of the layers.  When editing a feature in that layer, I see the new field in the Edit feature pane, but it does not appear in the table.
Test app: https://afmie.maps.arcgis.com/apps/instant/manager/index.html?appid=6db469f8c81844b6b16195c97a3f8fd9

@rachelmitcavish we are tracking this issue here

Verified in devext

@jmhauck , the new field shows up at the table and once I click on Edit button but in the information panel.
Test app: https://solutionsqa.mapsqa.arcgis.com/apps/instant/manager/index.html?appid=c05a0d6c7ee44911a298f58778ef9fe4
Not showing the new field here, but once I click on the Edit button it shows up

@sarahmcdonald7

@azizaparveen if you us the refresh button in the table tools, does the new field appear?

N

@azizaparveen if you us the refresh button in the table tools, does the new field appear?

No, it still does not show up in the information panel, everywhere else yes.

New field not showing up in the Info panel is a known limitation, This dialog is based on the pop-up definition, right now to see the new field we will need to update the pop-up definition, save the map and then reload the map.


CopyInput can render as a Text Copy Button instead of Copy Icon

Problem/Feature
In HSDS v3 we lost the ability to display CopyInput with a text button. See Jira Issue: https://helpscout.atlassian.net/browse/HSDS-159
Solution
Adds an optional buttonLabel prop which defaults to null, but if provided, will be used as the copy button's text in place of a copy icon (which is still the default!)

New Story:

CopyInput "text button"
http://localhost:8900/?path=/story/🧬-components-forms-copyinput--text-button

New Test also added so please check that it passes!

Pull Request Test Coverage Report for Build 3331

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.001%) to 94.576%

Totals

Change from base Build 3328:
0.001%

Covered Lines:
6325

Relevant Lines:
6555

💛  - Coveralls


Web3 version compatibility

Hello! It would be better If you mention about compatible version range of Web3.js in the readme or in release page for each release.
The latest version of Web3.js doesn't work with the library at all.
With a quick test, I would say, from<EMAIL_ADDRESS>to<EMAIL_ADDRESS>is fine with the latest one.
<EMAIL_ADDRESS>and below gives;
core.addProviders is not a function error
<EMAIL_ADDRESS>- 39 has a babel module issue
Cannot find module '@babel/runtime/helpers/classCallCheck'
<EMAIL_ADDRESS>and above has;
(node:91462) UnhandledPromiseRejectionWarning: Error: Node error: {"code":-32600,"message":"invalid json request"}

We should add support for the latest versions of web3.

Doesn't work with<EMAIL_ADDRESS>

Doesn't work with<EMAIL_ADDRESS>Tried with different web3 versions but all give this error.

UnhandledPromiseRejectionWarning: Error: Node error: {"code":-32600,"message":"invalid json request"}

@mdtanrikulu  Could you describe more clearly how you resolve this?

@OliveIT<EMAIL_ADDRESS>and below are ok. Be sure to use the right syntax in package.json, not the one with '^...'. If you need to only resolve name, you should use web3js or ethers. If you need reverse, ethers is the only way to do it with a recent web3js version.
There is a feature ticket: https://github.com/ethereum/web3.js/issues/2683

How are you initializing ensjs? You can now pass a provider object as the first argument, which should remove any dependency on a specific version.

I'm using infura in my tests:
new ENS(new Web3.providers.HttpProvider(`https://mainnet.infura.io/v3/${token}`))

I have this error:
(node:7284) UnhandledPromiseRejectionWarning: Error: Node error: {"code":-32600,"message":"invalid json request"}
    at Function.validate (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/web3-providers/dist/web3-providers.cjs.js:111:18)
    at HttpProvider._callee$ (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/web3-providers/dist/web3-providers.cjs.js:706:61)
    at tryCatch (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/regenerator-runtime/runtime.js:45:40)
    at Generator.invoke [as _invoke] (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/regenerator-runtime/runtime.js:271:22)
    at Generator.prototype.(anonymous function) [as next] (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/regenerator-runtime/runtime.js:97:21)
    at asyncGeneratorStep (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/home/ellis/Documents/Perso/Sources/fireblock/backend_express/node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)
    at process.internalTickCallback (internal/process/next_tick.js:77:7)

@ellis2323 That traceback doesn't tell us a lot because it doesn't include the call site. Are you able to narrow down what call is causing it?

A simple unit test with web3 beta52. It was something like that:
const ENS = require('ethereum-ens')
const Web3 = require('web3')

let provider = new Web3.providers.HttpProvider(`https://mainnet.infura.io/v3/${token}`)
const ens = new ENS(provider);

let address = await ens.resolver('ellis2323.eth').addr()

Looks like this is more of Infura not responding to older version of web3.js.
Here is the sample code.
const Web3 = require('web3')
const Web3Legacy = require('web3legacy')
console.log('web3 version', (new Web3()).version)
console.log('web3 legacy version', (new Web3Legacy()).version)
// Do "export INFURA_PROJECT_ID=YOURPROJECTID" on your terminal
const projectid = process.env.INFURA_PROJECT_ID
const endpoint = 'https://mainnet.infura.io/v3/' + projectid
const provider = new Web3.providers.HttpProvider(endpoint)
const web3 = new Web3(provider)
const web3legacy = new Web3Legacy(provider)
async function call(){
    console.log('latest web3')
    console.log('blocknumber', await web3.eth.getBlockNumber())

    console.log('lagacy web3')
    console.log('blocknumber', await web3legacy.eth.getBlockNumber())
}
call()

The above script shows the same error message
web3 version 1.0.0-beta.55
web3 legacy version 1.0.0-beta.37
latest web3
blocknumber 7732404
lagacy web3
(node:57809) UnhandledPromiseRejectionWarning: Error: Node error: {"code":-32600,"message":"invalid json request"}
    at Function.validate (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/web3-providers/dist/web3-providers.cjs.js:114:18)
    at HttpProvider._callee$ (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/web3-providers/dist/web3-providers.cjs.js:710:61)
    at tryCatch (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/regenerator-runtime/runtime.js:45:40)
    at Generator.invoke [as _invoke] (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/regenerator-runtime/runtime.js:271:22)
    at Generator.prototype.(anonymous function) [as next] (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/regenerator-runtime/runtime.js:97:21)
    at asyncGeneratorStep (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/Users/makoto/work/ens/tmp/ethereum-ens-versions/node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)
    at processTicksAndRejections (internal/process/task_queues.js:86:5)
(node:57809) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:57809) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I am quite sure that this started happening recently as it did not happen back in March when I last tried..
I will raise the issue into Infura and will see what they say.
If they won't support old version of web3, then we may have to upgrade web3 we use to the latest (which I tried but struggled a bit.

I use infura for other calls without problem. It must be related to their code with ens functions.

Ok, raised at https://community.infura.io/t/web3-js-1-0-0-beta-37-call-raises-code-32600-message-invalid-json-request/583

It looks like there might be some strange dependency issues that are causing this. It definitely is sending invalid json. more details on our infura community thread: https://community.infura.io/t/web3-js-1-0-0-beta-37-call-raises-code-32600-message-invalid-json-request/583/3?u=egalano

Could we release a new version not pinned to<EMAIL_ADDRESS>now that<EMAIL_ADDRESS>is out (essentially a LTS version of beta.37)?

Bumping this for support of Web3 1.x


Add list of Metrorail station names and codes to the admin interface

There are only a limited set of Metrorail station names and codes. The admin interface could have that list handy so that someone setting up a screen does not need to make an API request for the list.

Added this feature as part of Issue 13.


MacVim errors on :VimuxTogglePane and doesn't :VimuxSendText but finishes without error

Have you guys tried using MacVim? I can't use:
:call VimuxSendText('echo whatever') it just doesn't appear in the tmux window.
Using :VimuxRunCommand works however. But I don't want a split since MacVim is one GUI and have to run tmux besides.
Using :VimuxTogglePane it also errors out.
Error detected while processing function VimuxTogglePane[6].._VimuxTmux:
line    2:
E484: Can't open file /var/folders/_m/2_2m6b0j34n6g3r5qbdv6b480000gn/T/vgmSOPP/56

If I'm reading this correctly it should be this line 2 in VimuxTogglePane
https://github.com/benmills/vimux/blob/master/plugin/vimux.vim#L95
But it's probably the next line where it calls _VimuxTmux.
I don't know if this could be a Vim-issue. But I'm happy to take a look at it later.
Console vim (mvim -v for MacVim) works with :VimuxTogglePane but GUI doesn't.

So you're using MacVim (GUI) and a terminal window running tmux besides? This doesn't seem to be the intended way of using this plugin. What is your goal?

This doesn't seem to be the intended way of using this plugin.
Thats probably an opinion. I have used it on Linux, WSL and FreeBSD like this without problems - it's probably trivial to get MacVim to work and when I have some days off I will attempt :)
When I execute :VimuxRunCommand in a non-tmux vim and have IRC running in tmux it executes the command there due to :help VimuxConfiguration_use_nearest g:VimuxUseNearest .. Being able to open a tmux and be confident that :VimuxRunCommand attempts to run in the closest tmux unless named is a nice feature and for MacVim users and other people using GUI's without tmux this is desirable.

Linux/WSL/FreeBSD are all OS platforms, and any way you run tmux in them is expected to work. MacVIM is a GUI that doesn't run in tmux, it is not a TUI. I have no idea what the expected result you are looking for is.
I'd happy to accept any contribution than enabled a workflow for your scenario, but without more details about what you expect there doesn't seem to be anything actionable here. I'm guessing if you want to use a GUI version of VIM rather than a TUI version inside Tmux your problem might be better served with vim splits and terminal commands.
If I'm miss-reading the situation feel free to comment with more detail.


S2024 simulation

This PR contains work on:

manual_control.py => manually driving in simulation and getting input sensor data
gem_control.py => through out sensor / actuator control through ros-topics

The code analysis is failing this request, can you comment on why it’s complaining? Are they reasonable and valid or are they false positives?