Network stability improvements

What current issue(s) does this address, or what feature is it adding?
Addresses #325 and #492
There's a couple of issues with the current networking code that I'll only describe at a high level:

nodes get stuck/stop syncing at some point
debugging output is insufficient to investigate the reason of poor stability

How did you solve this problem?

Added health checking of peers based on block/header requests vs actual responses within a given time threshold. Bad peers are disconnected from and moved into the DEAD_ADDRS list.
Added monitoring of connected peers and if we reach 0 we reset and queue up peers. Address selection is sorted from "never connected to" to "recently disconnected from" so we always try to pick a fresh one.
Added data throttling when our request speed exceeds a certain response speed threshold
Move some threading calls into their proper threads (mainly into the reactor) to avoid unexpected behaviour.
Improved the nodes output to give a bit more information on the clients and addresses. i.e.

neo> nodes
Total Connected: 2
Peer 0  /NEO:2.8.0/ -  <IP_ADDRESS>:20333 - IO   0.323453 MB in /   0.009913 MB out
Peer 1  /Neo:2.9.0/ -    <IP_ADDRESS>:20333 - IO   0.131664 MB in /   0.006687 MB out

KNOWN_ADDRS: 174
DEAD_ADDRS: 6
QUEUED_ADDRS: 3

Added a lot of debug output and nicely format them to make analysis easier in the future.
Added 2 "hidden" commands disconnect_node <id> and handles for debugging purposes. They do not show up in the help output of the prompt, but can be used.
Fixed an issue where block persisting would pass, but the notificationDB fails emitting the event which caused the BlockPersist task to cancel and the node gets 'stuck'.

How did you make sure your solution works?
For the past 2 weeks I've been monitoring and adjusting this daily.
Are there any special changes in the code that we should be aware of?

@localhuman I'd say this one is specifically for you because it's notificationDB related which I think you work a lot with? Block 1231519 has one 'malformed' transaction that throws an exception in the NotificationDB when serialising a Neo.Contract.Create event.

test-cases have not been finalised. I've skipped 2 because I wanted to get this PR out so people can start testing if they want.

Known issues

minor - DEAD_ADDRS can contain duplicates. This has little effect on usage or stability.
minor - about 1 in 200 client disconnects can have a connection that doesn't fully close (meaning the TCP socket stays open in LISTENING state shown by netstat). I'm trying to resolve this with the Twisted package authors. This can be a problem if it was the last node connected while having no fresh addresses to pick from. It then prevents reset. I've seen this happen twice over the course of 2 weeks of testing.
major - header and block persistence blocks the main thread/health monitoring threads leading to nodes timing out if processing takes too long. This is something I want to address in a separate PR because it will require extensive testing again given that I don't know what the effect will be if I simply move it into its own thread. The impact can be that we switch out healthy nodes because they time-out or exceed our response threshold. Note that this will not get us in a stuck state, there's always a check that will reset the node if all nodes gone bad.
don't enable AcceptIncomingPeers in your config file. It suffers from the same as point 3. and then some. It's on the todo list after addressing 3.

Please check the following, if applicable:

[ ] Did you add any tests?
[X] Did you run make lint?
[X] Did you run make test?
[X] Are you making a PR to a feature branch or development rather than master?
[X] Did you add an entry to CHANGELOG.rst? (if not, please do)

I hope that some more people can try running an additional node with these changes and share your experiences before we do the final merging.
@jseagrave21 @ThomasLobker @localhuman

Coverage decreased (-1.7%) to 82.204% when pulling 19a1125df1b11656f2a8e23774f47f3678491994 on ixje:network_improv into 1aec5f0d779b3ba2f997926ced64a7d5cd09590b on CityOfZion:development.

@ixje I will start it tonight. Also, we can keep https://github.com/CityOfZion/neo-python/issues/721 open as a warning until resolved.
This looks amazing!

First feedback: I see this error frequently:
[D 181120 04:30:11 NeoNode:454] [002][MAINTAIN][   <IP_ADDRESS>:10333] asking for more blocks 2985303 - 2985311 (8) stale count: 135 BCRLen: 8
[D 181120 04:30:11 NeoNode:642] [002][MAINTAIN][   <IP_ADDRESS>:10333][HEALTH][B] OK - response time 20.76559090614319
[E 181120 04:30:11 InvPayload:62] COULD NOT WRITE INVENTORY HASHES argument should be bytes, buffer or ASCII string, not 'NoneType'

Second error I see regularly
[C 181120 05:31:15 _observer:131]
    Traceback (most recent call last):
      File "/mnt/c/users/seagrave/neo/neo-python/venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred
        result = f(*args, **kw)
      File "/mnt/c/users/seagrave/neo/neo-python/venv/lib/python3.7/site-packages/klein/app.py", line 128, in execute_endpoint
        return endpoint_f(self._instance, *args, **kwargs)
      File "/mnt/c/users/seagrave/neo/neo-python/venv/lib/python3.7/site-packages/klein/app.py", line 227, in _f
        return _call(instance, f, request, *a, **kw)
      File "/mnt/c/users/seagrave/neo/neo-python/venv/lib/python3.7/site-packages/klein/app.py", line 50, in _call
        result = f(*args, **kwargs)
      File "/mnt/c/users/seagrave/neo/neo-python/venv/lib/python3.7/site-packages/neo_python-0.8.3.dev0-py3.7.egg/neo/api/utils.py", line 27, in wrapper
        response_data = gzip.compress(bytes(response_data, 'utf-8'), compresslevel=COMPRESS_FASTEST)
    TypeError: encoding without a string argument

Unfortunately, mine got stuck at 2985978 / 2985978
Logs:
[D 181120 18:38:05 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:05 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7b38>
[I 181120 18:38:05 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7128>
[D 181120 18:38:06 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:06 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7048>
[D 181120 18:38:06 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:06 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c3b670198>
[I 181120 18:38:06 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4ef58160>
[D 181120 18:38:07 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:07 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4367b198>
[I 181120 18:38:07 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4e6643c8>
[D 181120 18:38:08 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:08 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c509392b0>
[I 181120 18:38:08 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392c88>
[D 181120 18:38:09 NodeLeader:37] Failed connecting to <IP_ADDRESS>:20037
[I 181120 18:38:09 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7f98>
[I 181120 18:38:09 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4ef81080>
[D 181120 18:38:10 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:10 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c50983f60>
[I 181120 18:38:10 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c5d51bd68>
[D 181120 18:38:11 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:11 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7b00>
[D 181120 18:38:11 NodeLeader:369] Peer check loop...checking [A:132 D:0 C:0 M:20 Q:20]
[D 181120 18:38:11 NodeLeader:376] Peer count is 0, allow for retries or queued connections to be established 0
[I 181120 18:38:11 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4ef58208>
[I 181120 18:38:12 threading:865] [MainNet] Block 2985978 / 2985978
[I 181120 18:38:12 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392c50>
[D 181120 18:38:13 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:13 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392c50>
[D 181120 18:38:13 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:13 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c43662e48>
[I 181120 18:38:13 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c5dd770f0>
[D 181120 18:38:14 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:14 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c3b7134a8>
[I 181120 18:38:14 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c50939748>
[I 181120 18:38:15 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392908>
[D 181120 18:38:16 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:16 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392908>
[D 181120 18:38:16 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:16 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c50983e48>
[I 181120 18:38:16 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf77b8>
[D 181120 18:38:17 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:17 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c5ddeab70>
[I 181120 18:38:17 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392160>
[D 181120 18:38:18 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:18 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c5ddeada0>
[I 181120 18:38:18 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4f8fe5f8>
[D 181120 18:38:19 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:19 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c5d51b080>
[I 181120 18:38:19 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c39392320>
[D 181120 18:38:20 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:20 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c5ddeac88>
[I 181120 18:38:20 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c393922e8>
[D 181120 18:38:21 NodeLeader:369] Peer check loop...checking [A:132 D:0 C:0 M:20 Q:20]
[D 181120 18:38:21 NodeLeader:376] Peer count is 0, allow for retries or queued connections to be established 1
[I 181120 18:38:27 threading:865] [MainNet] Block 2985978 / 2985978
[D 181120 18:38:31 NodeLeader:369] Peer check loop...checking [A:132 D:0 C:0 M:20 Q:20]
[D 181120 18:38:31 NodeLeader:376] Peer count is 0, allow for retries or queued connections to be established 2

@xxedocxx at first sight what you've posted

[D 181120 18:38:05 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:05 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7b38>
[I 181120 18:38:05 _observer:131] Starting factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7128>
[D 181120 18:38:06 NodeLeader:37] Failed connecting to <IP_ADDRESS>:10333
[I 181120 18:38:06 _observer:131] Stopping factory <neo.Network.NodeLeader.NeoClientFactory object at 0x7f0c4daf7048>

does not look like stuck to me, but like addresses that are either refusing connection because they maxed out on allowed peers or are not responding.

[D 181120 18:38:31 NodeLeader:369] Peer check loop...checking [A:132 D:0 C:0 M:20 Q:20]
[D 181120 18:38:31 NodeLeader:376] Peer count is 0, allow for retries or queued connections to be established 2

There should be a 3rd log message after the above saying:

Peer count 0 exceeded max retries threshold, restarting...

I'll have to check how it works when I'm fully synced. To be fair thinking back at it I believe I only tried out everything while the node was in CATCHUP mode, not in MAINTAIN mode. Although I don't expect that to make much of a difference as in MAINTAIN mode it's just requesting data at a higher interval.

@ixje I restarted the api server right after I made the previous post and got stopped at 2986692 / 2986692
[D 181120 23:19:41 NodeLeader:369] Peer check loop...checking [A:357 D:0 C:0 M:20 Q:20]
[D 181120 23:19:41 NodeLeader:372] Peer count 0 exceeded max retries threshold, restarting...
[D 181120 23:19:41 NodeLeader:131] stop_peer_check_loop, cancel: True
[D 181120 23:19:41 NodeLeader:133] stop_peer_check_loop, calling stop()
[D 181120 23:19:41 NodeLeader:136] stop_peer_check_loop, calling cancel()
[D 181120 23:19:41 NodeLeader:150] stop_check_bcr_loop, cancel: True
[D 181120 23:19:41 NodeLeader:238] Starting up nodeleader
[D 181120 23:19:41 NodeLeader:248] Starting up nodeleader: starting peer and mempool check loops
[D 181120 23:19:41 NodeLeader:121] start_peer_check_loop
[D 181120 23:19:41 NodeLeader:399] Queuing    <IP_ADDRESS>:10333 for new connection [in queue: 1 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 2 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 3 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing  <IP_ADDRESS>:10333 for new connection [in queue: 4 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 5 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 6 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 7 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 8 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing  <IP_ADDRESS>:10333 for new connection [in queue: 9 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 10 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 11 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 12 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing    <IP_ADDRESS>:10333 for new connection [in queue: 13 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing <IP_ADDRESS>:10333 for new connection [in queue: 14 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing  <IP_ADDRESS>:10333 for new connection [in queue: 15 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 16 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing <IP_ADDRESS>:10333 for new connection [in queue: 17 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 18 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 19 connected: 0 maxpeers:20]
[D 181120 23:19:41 NodeLeader:399] Queuing   <IP_ADDRESS>:10333 for new connection [in queue: 20 connected: 0 maxpeers:20]
I hope this helps.

@xxedocxx can you please define what you consider "stopped"? I have a feeling our definitions differ leading to confusion (at least on my side).
What I can tell from those logs is that at some point you lost connection to all your peers. This was detected and it started queueing new connections. Then your shared log stops. Shortly after your shared logs I expect to see new node created followed by connection established if successful.
Connection to peers can get lost for various reasons:

they stopped running their client
they considered us as a bad peer and disconnect us
we disconnect them because because we consider them as a bad peer (based on performance)

A moment of 0 connected peers is not per definition a bad thing. It's just part of a none perfect world.
W.r.t. queuing:

Queuing just means we've lined up new addresses to start connecting to
There's a delay between queuing and actually attempting to establish a connection
Queuing does not guarantee that we can connect. The addresses we know about have been shared by other peers, but without any guarantee that they're valid. We might fail to connect to them because

they've reached their max connection limit or don't accept incoming connections
they've stopped running their client in the mean time

@ixje My bad I should've said "got stuck"
I haven't looked into the code deeply but your explain got me its logic.
I'd love to help if I can.
My node is running on AWS EC2 t2.medium

@ixje my node stopped producing blocks on 2990906. Here is what the log looks like:
[D 181121 15:02:55 NodeLeader:369] Peer check loop...checking [A:2 D:798 C:3 M:50 Q:0]
[I 181121 15:02:55 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:55 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:02:56 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:56 +0000] "POST / HTTP/1.1" 200 40 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:02:56 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:56 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:02:56 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:56 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:02:57 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:56 +0000] "POST / HTTP/1.1" 200 599 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:02:57 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:56 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:02:57 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:57 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:02:57 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:57 +0000] "POST / HTTP/1.1" 200 5608 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:02:57 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:57 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:02:58 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:58 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:02:58 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:58 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:58 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:58 +0000] "POST / HTTP/1.1" 200 46 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "OPTIONS / HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "OPTIONS / HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "OPTIONS / HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "POST / HTTP/1.1" 200 46 "http://switcheoexplorer.tech/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "POST / HTTP/1.1" 200 5608 "http://switcheoexplorer.tech/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "POST / HTTP/1.1" 200 599 "http://switcheoexplorer.tech/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:02:59 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:00 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:02:59 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:00 threading:865] [MainNet] Block 2990906 / 2990906
[I 181121 15:03:00 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:00 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:00 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:00 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:01 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:01 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:01 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:01 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:02 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:01 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:02 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:02 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:02 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:02 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:03 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:03 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:03 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:03 +0000] "POST / HTTP/1.1" 200 46 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:03:03 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:03 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:03 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 46 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "OPTIONS / HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "OPTIONS / HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "OPTIONS / HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 599 "-" "Python/3.6 aiohttp/3.4.4"
[I 181121 15:03:04 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[I 181121 15:03:05 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 46 "http://switcheoexplorer.tech/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:03:05 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 5608 "http://switcheoexplorer.tech/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:03:05 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 83 "-" "Python-urllib/2.7"
[I 181121 15:03:05 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:04 +0000] "POST / HTTP/1.1" 200 599 "http://switcheoexplorer.tech/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
[I 181121 15:03:05 _observer:131] "<IP_ADDRESS>" - - [21/Nov/2018:23:03:05 +0000] "POST / HTTP/1.1" 200 166 "-" "Python-urllib/2.7"
[D 181121 15:03:05 NodeLeader:369] Peer check loop...checking [A:2 D:798 C:3 M:50 Q:0]

Also it is still connected to 3 peers and responding to requests:
{"connected": [{"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}]

I was thinking maybe we should have a blockheight checker tied to a loop and if at a predefined time (e.g. a minute) the blockheight is still the same then NodeLeader resets. What do you think?

don't use the above yet. I've got more stuff incoming.

I identified an issue where we would queue and address that was already in the queue list. If that address would then fail to connect, then only 1 of the 2 instances would be removed. This later on would prevent new peers getting added to the queue because we think it's still full.

Related to the above. I ended up making queue processing part of the peer check loop, instead of processing setup up a "connect to" task once when the address is added to the queue. This should prevent any addresses getting stuck in the queue.

A second issue from which I do not know the root cause is; a new node got created, did not loose connection at any point, yet it would never reach the ProtocolReady state where it starts its data request tasks. I've added a check to make sure that all connected peers are running their tasks unless we're in data throttling mode.

@jseagrave21 had a finding where block heights would not advance. This can happen if nodes that we're connected to are stuck themselves. Pulled his PR and adjusted it such that we not fully reset, but disconnected the peers and allow for trying out different addresses that we know off.

Unrelated: @ixje  even though there aren't any conflicts, could you merge development or https://github.com/CityOfZion/neo-python/pull/724 with this branch to make live testing easier? (So, I don't have to manually implement #724 before testing)

On the current run my node has remained at 0 peers for quite some time. I see peer_check_loop doing its job and restarting NodeLeader, but the node is not connecting any mew peers. However, blockheight is still advancing and I was wondering how that was possible?

My best guess is that blocks were received, they took a long time to process, this caused nodes to timeout (because processing unfortunately is still blocking) but the height advanced.

@ixje So, I figured out what happened with "getpeers" and I don't understand it. Somehow on my most recent pull from your branch, it missed this commit https://github.com/CityOfZion/neo-python/pull/723/commits/d6c35d732f9616b329c42ed2214b65422af41d7f

After my node ran on the latest pull for ~500 min, peers were at 0 for quite awhile but blockheight was still advancing, usually lagging the best blockheight. Internally, every connection to an IP was failing and the traceback was from Twisted and stated the reason for connection failure was "Invalid Argument". Also, I could not access any website from my computer and had to restart my computer before my access was restored. @ixje is this related to the Twisted issue mentioned above?

My node is at ~400 min and seemingly stable right now 🤞 But with only one accept and one connected peer. Is this acceptable node behavior?
[D 181123 23:12:58 NodeLeader:420] Peer check loop...checking [A:1 D:644 C:1 M:25 Q:0]

Or should we put a threshold of connected peers/time limit to go look for new peers, even if connected peers != 0?

for the time being removing reactor.suggestThreadPoolSize(30) from prompt.py seems to resolve the "22: invalid argument" error. Can't say for sure because my Windows machine thought it was a good idea to just install updates unasked 😡

Do you think this would also affect np-api-server? I haven't run np-prompt at all so far for my testing.

My node is stuck at Block 3008831. PeerCheckLoop looks good: Peer check loop...checking [A:2 D:5246 C:2 M:25 Q:1]
And, BlockheightCheck is working. I have seen watched it "Restart NodeLeader" several times now, but blockheight is still not advancing. I feel like the BlockheightCheck needs to issue a hard reset for NodeLeader, as simply disconnecting all the connected peers does not seem to work. Thoughts?

thanks for reporting, we'll have to keep an eye out why this is happening and if it's something we can prevent.

I feel like the BlockheightCheck needs to issue a hard reset for NodeLeader, as simply disconnecting all the connected peers does not seem to work. Thoughts?

Bad idea, reason explained before here (bullet 2): https://github.com/ixje/neo-python/pull/2#issuecomment-440970425

My node is stuck at Block 3015922. Everything appears to be running normally, including headerheight advancing. Again, disconnecting all peers does not seem to solve the problem. I see several of these errors:
[E 181127 02:15:00 InvPayload:62] COULD NOT WRITE INVENTORY HASHES argument should be bytes, buffer or ASCII string, not 'NoneType'

but they appear to be intermittent and not truly related to blockheight not advancing 🤷♂

@ixje the current build is great! My node has been running for about 22 hours now. I have two questions:

Is it acceptable to run a node with only 1 peer for an extended time? I don't think we have to worry about stale blocks because of NEO block finality but I want to make sure.
My DEAD_ADDRS list is now at about 2500 and I am wondering if we should manage this list any differently in the background to ensure the node can run for extended times without getting bogged down potentially?

Good to hear at least it seems more stable now. We don't have to worry about the size of the DEAD_ADDRS list. There are other lists internally that are far bigger than this
It's not necessarily bad to be connected to 1 peer if that is the only one that is stable. It might be a bug somewhere though because I run into the exact same scenario. Sounds suspicious 🤔

Is this acceptable connection behavior?  You should only be connected to an IP + PORT combination once. Will add this to my TODO/unittest list. Thanks!

My node has now been running for >49 hours. The only problem I see now is peers are at maxpeers but they are all the same peer:
"connected": [{"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}, {"address": "<IP_ADDRESS>", "port": 10333}], "unconnected": [{"address": "<IP_ADDRESS>

Another related oddity I found is that the same address has been filling the DEAD_ADDRS list. The DEAD_ADDRS list is now >13,000 and the majority of the addresses is "<IP_ADDRESS>".
I double checked the logic and I think it looks good. I haven't found the bug yet. One (easy-way-out) method would be to implement another check, probably on the same schedule as the memcheck and the blockchaincheck to remove any duplicates in the DEAD_ADDRS and KNOWN_ADDRS lists. Similar action to be taken for Peers but they would also be eliminated with Disconnect()

Thanks for reporting 👍 . Instead of cleaning up I want to address the root cause and prevent them getting into the lists in the first place.

some notes;

acceptance and working with incoming clients does work but it has only been tested shortly.
a higher maxpeers setting equals worse RPC server performance. Something is still causing the reactor (of the Twisted package) to block for long times. Which in turns means RPC requests get no response. More nodes means more times this can occur. To avoid confusion; maxpeers does not mean a limit on simultaneous RPC connections
a low maxpeers is better for syncing up. A high setting is arguably better when you're fully synced up. Why arguably? Because it depends on the quality of the node(s) you're connected to. You just need 1 good node. On the other hand more nodes means more likelihood one of the has the real latest chain state.

:+1: fixed my syncing problem.

Thanks for testing 👍  I would appreciate it if you could let me know if you have any unexpected findings.

@ixje https://github.com/CityOfZion/neo-python/pull/723/commits/b77e48911c7fdd037f1c306de372d4939da12bc8 fixed the previously reported error. My node is running smoothly, error free. I recommend a bigger beta group / merging at this point.
@ThomasLobker have you tested this latest version? It is very good.

@jseagrave21 I'm now testing with 0.8.2 (not development branch) and made some fixes on my own. Well I cannot really call it a fix since it's more a workaround. I am running a couple of nodes in Docker and the nodes are simply crashing anytime that there are no more connected peers. Docker will pick this up and restart the node. Also I've noticed that nodes are running a lot smoother once they are not in catch-up mode anymore.

I have updated my private branch to the latest development branch. It's working fine, but unfortunately I'm still getting peer drops. Currently I am detecting if there are no more connected peers and shutting down the node, in order to let Docker restart the node and try again.
    ]
[D 181218 14:59:33 NeoNode:641] [0][mode 2] IPv4Address(type='TCP', host='<IP_ADDRESS>', port=10333) - On neo Node loop error [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.CancelledError'>:
    ]
[D 181218 14:59:33 NeoNode:641] [0][mode 2] IPv4Address(type='TCP', host='<IP_ADDRESS>', port=10333) -<PHONE_NUMBER> disconnected [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.
    ]
[D 181218 14:59:33 NodeLeader:44] Dropped connection from neo-cli:10333
[I 181218 14:59:33 NodeLeader:49] No more connected peers...```

It cannot ensure that we never reach 0 peers for many reasons, the most straight forward is all connected peers are bad or simple close down.
The main idea behind this PR is that it will not get stuck forever but can recover.

just a note of warning; it can take up to several minutes. It will detect 0 peers and first attempt to queue new ones from its KNOWN_ADDRS list. Now, this list can contain all kinds of dead addresses that we received from other nodes but have not yet validated. This means it can hit a series of dead addresses before it finally finds one it can connect to. If they're all dead it will be recognised as well and will do a full restart after 3 confirmations. On restart it will mark all ever known addresses as valid again. The reason for doing this is because some addresses could have been bad in the past but have recovered now. They might also have new addresses they've learned over time, so we at least want to pick up those before we mark it as a bad node.

Sounds good @ixje
I would say, if we have reached the state of 0 connected peers and it has to search for new peers that are alive, then just start with the configured peers. As an infrastructure maintainer I have control over those peers and if they are down, it will be noticed and fixed.


Any interest in serde support?

A while back I forked this repo for a summer project and one of the things I added to my fork was serde support before I got busy with school and work. If I opened a PR for that would it be welcome? In its present state it may be a bit disruptive to the API since if I recall correctly I had to adjust one of the traits.

Hi @dgunay,
ta is not involved in any kind of data serialization.
Could you please clarify what do exactly mean by serde support?

Just submitted a PR - basically, it allows the user to choose any serialization crate that is compatible with serde (JSON, binary, etc) and then use it to serialize/deserialize the indicators. For my use case, I was serializing machine learning models (which contained various ta::indicators) to disk using a binary format.
I added derived Serialize/Deserialize to the technical indicators to do it automatically.
While it is possible for the user to implement this themselves for remote types, it is an awkward process that requires the user to essentially redefine the struct's shape in their own crate. It is a brittle solution that breaks if the upstream code changes the struct definition in any way.
I would have had to do this for every indicator I wanted to use, but I wanted forward compatibility with any ta::indicators::* so I forked the repository and derived Serialize/Deserialize to accomplish this.


Fix for swarm init/leave race condition

This change serialize the init/close of the libnetwork agent into one single code path.
On swarm init there are several things that are appening in parallel, before the
initialization of the agent was happening thorugh differents code paths depending
from timing.
This change cleans up the entire init logic letting the init happening in the goroutine
clusterAgentInit.
Moby side: https://github.com/moby/moby/pull/32906
Signed-off-by: Flavio Crisciani<EMAIL_ADDRESS>

LGTM

@aboch after a new change on the moby side I realized that there was no reason to change the ListenClusterEvent function so I just restored it. The provider interface again returns a read only channel as the original

Right, because we no longer need to push an event from inside libnetwork (I remember at some point the key ready msg was being sent from inside SetKey)
Still LGTM


Move general glTF helpers to their own libraries

gltfDefaults
ModelCommonMaterials

And publish them on npm so we can use them in gltf-statistics, OBJ2GLTF, etc.

Progress in gltf-pipeline here: https://github.com/AnalyticalGraphicsInc/gltf-pipeline/pull/272

@lilleyse OK to close this?

Yes, fine to close.


Add bower support to main repo

I'd like to use bower to consume the tslib repo. It looks like the bower version of this repo was already created a month ago: https://github.com/Draccoz/tslib. It'd be awesome if we could bring the bower and npm definitions of the repo into the same place, under the Microsoft org. I realize that since there is an existing bower definition, this might cause problems. In that case, is there any way we can set up a hook to submit a PR to the bower repo every time something gets merged to master here?

We would be open to add bower support. @Draccoz would you be willing to give up your repo and help us set up the bower support in this repo?

Sure! Just let me know what exactly should I do. AFAIK bower has no auto unregister mechanism, can be done VIA contacting. So should I just contact them and you guys would take the bower name? I am not sure if I can just change the path assigned in bower. Personally I think the easiest would be unregistering the repo so you can claim it.

I do not have much experience with bower. so really not much help i can offer. but if you have a PR to add a bower.json, i can keep the version up-to-date as i publish to npm.

This repo (react-bootstrap-overlays) had a similar issue:https://github.com/react-bootstrap/react-overlays/issues/26
I'm not sure how careful we want to be (ideally this change is backwards compatible), but by deleting the existing bower registration we might break ppl's builds.

I added a PR (bower.json and html importable modules). Maybe the bower guys could just manually transfer the git link? Maybe it wouldn't break it? You guys know the specific person? If not I will just send them an email.
As for the PR itself, besides of bower.json I've also got there small html modules to allow html5 imports (handy till new spec is out).

The config file is in, thanks @Draccoz. I will will leave this issue open until we figure out the registration migration.

Thanks @mhegazy. Does this also mean that whenever changes are made to this repo, you will submit a corresponding PR to @Draccoz's repo to keep the bower dependency up-to-date? Would that be a good place for a hook to submit the PR?

@billderose It will be automatic as bower:tslib will be bound to this repo. As for this one, guys on bower just answered there should be no issue with our transfer, so we can proceed (especially if version numbers are correct and transfer is from fork to original).
@mhegazy Should I do it now? Want to coordinate it somehow or let's hope nobody takes it and just go whenever :P?

Let me know once it is released and I will try to register. Best time for me is between 10:00 am and 4:00 pm PST.

OK, it's unpublished :). Your turn now. I've installed tslib via bower before that so can test later if everything is fine.

should be registered now. just published<EMAIL_ADDRESS>for both npm and bower.

also readme updated.

Works for me :) had to clean the cache (or force newer version), but so far works perfectly. Thanks for taking the repo off me :P hope it gets even more users now = ).

thanks @Draccoz for the help


Improve directory structure consistency

update the codebase to have the same sub directory structure

detections
baselines
response_tasks

We plan update the whole dir structure


First pass for strand fixes and export timer.

More optimizations coming.
Also there is a fix for constant width hair strands. Before they would actuality be overridden by the base and tip widths.

Yep.  I was thinking of that exactly.  How is the rib gen time difference?  And rendering?  You could output the ribgen time to the engine report I think...


Revert "AMBARI-24270. Agent Status Command Are Randomly Failing With …

…Empty stderr (aonishuk) (#1728)"
This reverts commit 219a23623349bc3715f6c667369477bd32ef1b0b.
What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
How was this patch tested?
(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
Please review Ambari Contributing Guide before opening a pull request.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/3052/
Test FAILed.
Test FAILured.

retest this please

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/3053/
Test FAILed.
Test FAILured.

retest this please

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/3056/
Test PASSed.


GPL-nnn - As a developer I would like to generate multiple HT barcodes for beckman destination plates

User story
As a developer I would like to generate HT barcodes for beckman destination plates.
Who are the primary contacts for this story
Steve / Harry
Acceptance criteria
To be considered successful the solution must allow:

[ ] I should be able to send a request for multiple HT barcodes and receive an array of barcodes
[ ] ...

Dependencies
This story is blocked by the following dependencies:

#<issue_no.>
sanger/#<issue_no.>

Additional context
Add any other context or screenshots about the feature request here.

Results of discussion:

create a new alembic migration to add new sequence 'HT'
create a new config file which links prefixes to sequence. All centres will link to heron sequence and ht will link to ht sequence
modify endpoints so the sequence is selected on the prefix
modify __build_barcode method to remove hexing number
modify barcode method in formatter to format barcode based on prefix i.e. if it is HT don't format


How to determine the type of elements read from a JSON array?

The JSON files I need to read with my program have the form
{   "labels":   [ 0, 0, 1, 1 ],   "vectors":   [  [ 1, 1 ],  [ 2, 2 ],  [ 2, 4 ],  [ 4, 2 ]  ]   }
or
{   "labels":   [ 0, 0, 1, 1 ],   "vectors":   [  "file1",  "file2",  "file3",  "file4"  ]   }
to represent specimen that have a numerical label and numerical data, the second of which may be given in the JSONs themselves or read from files.
The labels are read in the line
std::vector <float> *y =
     new std::vector <float> ( jdesign["labels"].get<std::vector<float>>() );

and for the first form the line
std::vector<std::vector<float>>
     xtest1 ( jdesign["vectors"].get<std::vector<std::vector<intensity>>>() );

does the trick. But if I want to test for the second form (the values are given as file names) and use
std::vector<std::string>
     xtest2 ( jdesign["vectors"].get<std::vector<std::string>>() );

I get:
terminate called after throwing an instance of 'nlohmann::detail::type_error'
what():  [json.exception.type_error.302] type must be string, but is array`

Is there a way to find out the type of the elements of vectors before assigning them to my program variables?
I am using Debian Linux (stretch, kernel 4.9.0-8-amd64) and GCC 8.2.0 and the json library as pulled from the GitHub repository.

You could call type() to get the type.

Thanks! but both examples above are json::value_t::array -- is it possible to test the types of the array elements in some way?

You would need to call it for each element. As the array has a begin/end iterator, you can also use std::all_of together with functions like is_string to check if all elements are strings, for example.

That makes sense, thanks again. But if I understand it correctly then I would always first need to do a get() before I could check the contents of the array, which sounds like a catch-22? I think it is better then to add a variable to the JSON to find out what T should be before calling get().

No, you can access the JSON vector before this:
if (std::all_of(jdesign["vectors"].begin(), jdesign["vectors"].end(), [](const json& el){ return el.is_string(); })
{
   // access as string vector, e.g. like
   std::vector<std::string> foo = jdesign["vectors"];
}

(untested code, but you should get the gist).


Multi-chain Dockerfile

Here we refactor the Dockerfile in order to support chains other than Ethereum. This is achieved by introducing a new CHAIN argument which is used to generate the right set of chain-specific contract bindings and is included on the build
tag list in order to build the keep-ecdsa client with the right host chain implementation.
This pull request also introduces several adjustments needed to make the Dockerfile work with Celo. The celo-blockchain module introduces a dependency on the celo-bls-go library. That library uses some native C code under the hood and links the proper target implementation according to the passed build tags. By default, the glibc implementation is used which doesn't work on the Alpine platform. Alpine uses the musl C standard library implementation so while using keep-ecdsa with Celo, the musl build tag must be explicitly passed. This is achieved by introducing the LIBC env variable which defines the correct C standard library implementation tag. The value of LIBC variable is passed to the Go compiler via APP_BUILD_TAGS and ABIGEN_BUILD_TAGS variables. As the last step of this part, several changes in tests have been made in order to run them with celo build tag as part of a future CI job.
By the way, this change also gets rid of the go-ethereum's abigen installation. This is no longer needed due to changes introduced by #481. This also simplifies the Dockerfile as we don't need to switch between chain-specific abi generators.

Left a couple of comments. Are we expecting CI updates to be in this PR or a separate one?

By the way, this change also gets rid of the go-ethereum's abigen installation.

Glorious 🙌

CI updates will be in the next PR.


Fix typo / grammar from rest-notebook.md

Just modifying doc so I don't file a new issue to JIRA.

@AhyoungRyu Thanks for the review. It's ready for another round of review.

Thanks @HeartSaVioR for the improvement.
Looks good to me!

LGTM :+1:


Add a validation panel for OSM edition

Once the user has made change and validated the OSM tag editor, he should be displayed a modal asking for validation.
This modal could (or not) inform about the whole point as it is going to be in the OSM database.

The repository moved to https://gitlab.com/mapcontrib/mapcontrib.next.


[wp-admin.common.adminbar.user]

Admin bar user options are not being removed.
Intervention: 2.0.0-rc.1
WordPress: 5.6
common.adminbar.user.[howdy, avatar, profile, edit]

Never mind... it just dawned on me that those are options, not an array, and that I should remove the brackets.
common.adminbar.user.howdy

Never mind... it just dawned on me that those are options, not an array, and that I should remove the brackets.
common.adminbar.user.howdy

Yea! I can look into making that clearer in the docs.
You can also pass common.adminbar.user => ['howdy', 'avatar', 'profile', 'edit] if you want to group them.

Yea! I can look into making that clearer in the docs.
You can also pass common.adminbar.user => ['howdy', 'avatar', 'profile', 'edit] if you want to group them.


Problem with new COUNT_TO_WRITE variable

This issue may only occur when cancelled and then resumed. The resume may skip over some pictures in the sequence:
lets say we have 8 files
photo1 to photo8
There are 4 workers in the upload script by default, running concurrently.
So each photo that is uploaded COUNT_TO_WRITE is incremented. Now the user cancels upload because they no longer have the time or they lost internet and need to resume
lets say:
Photo1....Uploaded
Photo2....Uploaded
Photo3...Needs Upload
Photo4....Uploaded
Photo5...Needs Upload
Photo6...Needs Upload
Photo7...Needs Upload
Photo8...Needs Upload
Photo4 got uploaded because it's a smaller size or the OS decided to give it a bit more bandwith. The count in the resume file will be 3 and when it will resume, it will skip over photo3 because the file says it has already uploaded 3 files(but doesn't specify which)

I will investigate this issue. Thanks  for helping 👍


v2.1 client not granting signature against workflow v2.0.0

Using the newest version of the CLI (2.1) I am unable to login to older versions of the workflow platform (tested on v2.0.0). Specifically, no signature is being retained by the client. E.g.:
$ deis login <a v2.0.0 workflow endpoint>
!    WARNING: Client and server API versions do not match. Please consider upgrading.
!    Client version: 2.1
!    Server version: 2.0
username: admin
password:
!    WARNING: Client and server API versions do not match. Please consider upgrading.
!    Client version: 2.1
!    Server version: 2.0
Logged in as admin
$ cat ~/.deis/client.json
{"username":"xxxx","ssl_verify":false,"controller":"<a v2.0.0 workflow endpoint>","token":"","response_limit":0}

I am not having this problem authenticating against v2.1.0 workflow clusters when using v2.1 of the client.

I think this is exactly the reason why we decided to put that big fat warning in there. The client is not guaranteed to work with older releases, and vise versa. That being said, I don't see any reason why this would be broken according to https://github.com/deis/controller/compare/v2.0.0...v2.1.0. The only relevant change I've seen server-side is in https://github.com/deis/controller/commit/46829d34b99edea661075301c847cced26318da8 and that was in 2.0.0-alpha, so I'm at a loss how a 2.1 client would fail to save the token from a 2.0 server. I'll see if I can replicate :)

Ok. I figured out the problem....  When the request returns a version mismatch, it returns that error in addation to the response. When the methods get the error response they immediately bail out and don't attempt to parse the body.
The CLI on the other hand, checks to see if the error is an API mismatch, and if there is, recognizes it is a warning and continues on, even though they're only working with a blank object.
What the SDK should do is implement a method similar to the workflow-cli's checkAPICompatibility which checks if the the error is an api compatibility one before bailing out.


Can this network run on datasets such as S3DIS and Toronto3D

Thank you for your great work.But I want to know that Can this network run on datasets such as S3DIS and Toronto3D

Thank you for your great work.But I want to know that Can this network run on datasets such as S3DIS and Toronto3D

Hi @mooncakehub, though I didn't try LaserMix on these two datasets, I believe it is promising. To the best of my knowledge, both S3DIS and Toronto3D have certain scene priors encoded and it is intuitive to mix along such priors to enhance the training.


updated smart contract with tags and fetching data

updated smart contract

tags support
create tags if not exist. In smart contract context this is needed when we sync data from db. if tag does not exist we create that.
get urls by tag
get urls by user
get all urls
get specific url
get downvote or upvote

Give me 30 minutes I just push some changes for the contract.

Delete some for loops to make gas usage as low as possible
New tags struct to quickly check if the tag exists (avoids huge for loops)

@hackertron please check the new changes and merge it if you agree 😊

Agree with the changes. Just a quick discussion that we did , we can think about

using a default/general tag in the smart contract ( hardcode it as default value) and also on backend side (mongo) . I will check it


[usecase-wrt] Remove some tests due to change to auto tests

Impacted TCs num(approved): New 0, Update 0, Delete 7

The auto tests pr link: https://github.com/crosswalk-project/crosswalk-test-suite/pull/2045

Updated


[rstream] StreamCancel is required

I was trying to create stream with only one value but the stream creation example with source function doesn't compile
a = rs.stream((s) => {
    s.next(1);
    s.done();
});

with the following error Argument of type '(s: Stream<{}>) => void' is not assignable to parameter of type 'StreamSource<{}>'.   Type 'void' is not assignable to type 'StreamCancel'
Solution, return void function with optional StreamCancel interface declaration
a = rs.stream((s): StreamCancel => {
    s.next(1);
    s.done();
    return () => {};
});

It was confusing as documentation says it's optional  If the function returns another function, it will be used for cleanup purposes if the stream is cancelled.

@vorg thanks - just updated the StreamSource type to allow void returns from those functions. the docs and implementation were correct, just the type sig was wrong (incomplete). Now fixed in v2.3.2


Add PWA / offline mode

Work done

added workbox-webpack-plugin to generate pwa manifest files during webpack build
added secure context development mode (https)
added certificate generation script & git ignore for .cert folder
added documentation on how to access secure development mode
tidied up site.webmanifest

Todo

Add a button to make it more obvious to users that they can install for offline use

How can I see this shiny new offline mode?

make sure if you're viewing in dev-mode you go through the docs to view it in a secure context.
closes #7

Looks awesome! two minor callouts but once you commit that i'll go ahead and merge!

@sdennett55 - something I noticed is that a large number of files aren't formatted with prettier. Do you want me to raise another PR for that?

@sdennett55 - something I noticed is that a large number of files aren't formatted with prettier. Do you want me to raise another PR for that?

works for me!

comments resolved 👍

Merged. Thanks @jamiehaywood!


[WIP] nextcloud-client: 2.3.2 -> 2.5.0-rc1

Motivation for this change
This updates the nextcloud-client from 2.3.2 (which was the owncloud-client with a nextcloud theme) to 2.5.0 (which is now an independent fork of the owncloud-client).
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

I'm trying to build it locally, but it's taking a lot of time, whereas the Drone CI builds of nextcloud/desktop are significantly fastler).
Maybe we're compiling more than we need to?

@GrahamcOfBorg build nextcloud-client

#50463

@caugner @eqyiel would you mind testing my patch (#50463)? It seems to properly migrate the patch and you don't have to recompile anymore when enabling gnome-keyring support :)

@Ma27 Thanks, I'm closing my PR in favor of yours (#50463) and will try to test your patch in the next couple of days.


Add node selector option for cms/nginx/mongo deployments

cms:
    nodeSelector:
      disk: ssd

Approved by @malaskowski.


Add missing commits to the master from 2201.1.x

Purpose
$Subject
Approach

Describe how you are implementing the solutions along with the design details.

Samples

Provide high-level details about the samples related to this feature.

Remarks

List any other known issues, related PRs, TODO items, or any other notes related to the PR.

Check List

[ ] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

@madushajg @prabushi 2201.1.x is being synced to master automatically. So, is it required to add this manually?

@madushajg @prabushi 2201.1.x is being synced to master automatically. So, is it required to add this manually?

It was told that we need to manually add all fixes as we can't automatically sync branches now. @keizer619 Please confirm.


New device request: AVATTO Smart Wifi Thermostat WT100

Would be great, if you could add this device. Thank you in advance.
Brand name, model:
Avatto Smart Wifi Thermostat
WT100-EH (most identical device WT100-BH)
Home Assistant Log:
2022-11-18 10:06:18.595 WARNING (MainThread) [custom_components.tuya_local.config_flow] Device matches hydrotherm_dynamic_x8_water_heater with quality of 28%. DPS: {'1': True, '2': 215, '3': 213, '4': 'hot', '5': '1', '8': True, '13': True, '16': 0, '20': -18, '21': 25, '25': '1', '26': False, '31': False, '41': '3', '42': '3', '101': False, '105': 5, '107': 35, 'updated_at':<PHONE_NUMBER>.2266963}

Tuya IoT,
"Get Device Specification Attribute:"
{
  "result": {
    "category": "wk",
    "functions": [
      {
        "code": "switch",
        "dp_id": 1,
        "type": "Boolean",
        "values": "{}"
      },
      {
        "code": "temp_set",
        "dp_id": 2,
        "type": "Integer",
        "values": "{\"unit\":\"℃\",\"min\":0,\"max\":700,\"scale\":1,\"step\":1}"
      },
      {
        "code": "mode",
        "dp_id": 4,
        "type": "Enum",
        "values": "{\"range\":[\"cold\",\"hot\",\"wind\",\"comfortable\",\"energy\",\"auto\",\"holiday\",\"eco\",\"manual\",\"sleep\",\"dry\",\"program\",\"floor_heat\",\"auxiliary_heat\"]}"
      },
      {
        "code": "child_lock",
        "dp_id": 8,
        "type": "Boolean",
        "values": "{}"
      }
    ],
    "status": [
      {
        "code": "switch",
        "dp_id": 1,
        "type": "Boolean",
        "values": "{}"
      },
      {
        "code": "temp_set",
        "dp_id": 2,
        "type": "Integer",
        "values": "{\"unit\":\"℃\",\"min\":0,\"max\":700,\"scale\":1,\"step\":1}"
      },
      {
        "code": "temp_current",
        "dp_id": 3,
        "type": "Integer",
        "values": "{\"unit\":\"℃\",\"min\":0,\"max\":999,\"scale\":1,\"step\":1}"
      },
      {
        "code": "mode",
        "dp_id": 4,
        "type": "Enum",
        "values": "{\"range\":[\"cold\",\"hot\",\"wind\",\"comfortable\",\"energy\",\"auto\",\"holiday\",\"eco\",\"manual\",\"sleep\",\"dry\",\"program\",\"floor_heat\",\"auxiliary_heat\"]}"
      },
      {
        "code": "child_lock",
        "dp_id": 8,
        "type": "Boolean",
        "values": "{}"
      }
    ]
  },
  "success": true,
  "t":<PHONE_NUMBER>114,
  "tid": "ea940bcb672011ed8c7e06bdf2add754"

Thank you. Pls. let me know, if you need further details.

For all who are interested in this device:
I still tested some similar thermostats, which all did not function as expected: Wrong and incomplete manual, not all functions could be set directly on the device (only with tuya app), switching manual, auto, home and away also not selectable directly at the device (only app).
I bought this one (16A, WT100-EH) for my water floor heating, controlled by a 230V valve, NC, not heating valve is closed.
Pos:

all settings could also be done and programmed directly at the device, without app
manual is complete and all settings and parameters are described correctly
settings and programming of schedule times and temperatures could be done directly at the device, without app!
delay of relay could be programmed (minutes delay), ideal for valves
temperature hysteresis for switching 230V out could be programmed 0,1 - 0,5 - 1 - .... °C
output is 16A, 230V as NC = 230V out for heating, could also be programmed vice versa for NO, means 230V out for not heating.
display backlight could be adjusted in high, middle, low and off.
Neg:
front display panel is a little mirrored coated

Bought it from Amazon for a special discount price of €33

Thank you very much for your very fast integration. I am just testing everything and there are several problems. I will come back in a short time. I will collect all problems. In basic it is working well and temperature settings and values are correct.

What I found:

(in the screenshot the selected mode is "Temporary program mode")
But there are several problems, because "dp_id": 4 only has 3 values and this 3 modes could be selected by the following programmed values in the "avatto_wt100_thermostat.yaml":
Program mode is equivalent to

dps_val: hot
value: heat

Temporary program mode is equivalent to

dps_val: wind
value: fan_only

Manual mode is equivalent to

dps_val: cold
value: Cool

Only this 3 modes "Program mode, Temporary program mode and Program mode" are available in the mode settings of the WT100
All 3 modes could be selected by the created climate card by pressing and selcting "Heat", "Cool" and "Fan only"
The other modes are useless.
Child mode is working as well.
Some more information about the not unknown dp_id's:
DPS: {'1': True, '2': 215, '3': 213, '4': 'hot', '5': '1', '8': True, '13': True, '16': 0, '20': -18, '21': 25, '25': '1', '26': False, '31': False, '41': '3', '42': '3', '101': False, '105': 5, '107': 35,
id20:  -18 , is Temperature calibration (offset), but is in real -1.8 °C
id21:   25 , is "Max setting of temperature"
id107:  35 , is "Max temperature limit of external temperature sensor"
id105:   5 , is "Relay delay when changing temperature"
I've tried to change the yaml by my own, but I do not understand, what dps_val and value exactly do, especially which is the one for the symbol and the sending value.
Seams to be dps_val is the command and value the symbol on the card?
Thank you again.

So I tried some programming and the device is now nearly perfect working.
Working perfect:

all "secondary_entities"
climate gauge with actual current temperature and temperature settings

Not included id's, and not important in my opinion:

parameter of weekly schedule (time and temperature settings)
Sound on/off
Frost protection settings
Switching relay between NC and NO
reset

Still needs improvement:

3 icons or symbols in the climate gauge to select the 3 modes: Manual mode, Program mode and Temporary Program mode (Override mode)

I didn't know how to implement these icons and functions for 3 modes.
For now, switching between the modes works well under the 3 dots on the top right.
I also do not know, how to eliminate the field "action" under the 3 dots. I think there is no need for this function. Only selecting between the 3 modes will be enough.
avatto_wt100_thermostat.yaml
avatto_wt100_thermostat.yaml.zip
name: Avatto WT100-EH thermostat
primary_entity:
  entity: climate
  dps:
    - id: 1
      name: hvac_mode
      type: boolean
      mapping:
        - dps_val: false
          value: "off"
        - dps_val: true
          constraint: preset_mode
          conditions:
            - dps_val: cold
              value: "Manual"
            - dps_val: hot
              value: "Program"
            - dps_val: wind
              value: "Program Override"
    - id: 2
      name: temperature
      type: integer
      range:
        min: 0
        max: 700
      mapping:
        - scale: 10
    - id: 3
      name: current_temperature
      type: integer
      readonly: true
      mapping:
        - scale: 10
    - id: 4
      name: preset_mode
      type: string
      mapping:
        - dps_val: cold
          value: "Manual"
        - dps_val: hot
          value: "Program"
        - dps_val: wind
          value: "Program Override"
    - id: 5
      name: unknown_5
      type: string
    - id: 13
      name: unknown_13
      type: boolean
    - id: 26
      name: unknown_26
      type: boolean
    - id: 31
      name: unknown_31
      type: boolean
    - id: 101
      name: unknown_101
      type: boolean
secondary_entities:
  - entity: lock
    name: Child lock
    category: config
    dps:
      - id: 8
        type: boolean
        name: lock
        mapping:
          - dps_val: true
            icon: "mdi:hand-back-right-off"
          - dps_val: false
            icon: "mdi:hand-back-right"
  - entity: select
    name: Temperature sensor
    icon: "mdi:thermometer"
    category: config
    dps:
      - id: 25
        type: integer
        name: option
        mapping:
          - dps_val: 1
            value: Internal
          - dps_val: 2
            value: External
          - dps_val: 3
            value: Both
  - entity: number
    name: Temperature calibration offset
    category: config
    icon: "mdi:arrow-expand-horizontal"
    dps:
      - id: 20
        type: integer
        name: value
        mapping:
        - scale: 10
        range:
          min: -99
          max: 99
  - entity: select
    name: Display backlight
    icon: "mdi:lightbulb"
    category: config
    dps:
      - id: 41
        type: integer
        name: option
        mapping:
          - dps_val: 0
            value: "off"
          - dps_val: 1
            value: low
          - dps_val: 2
            value: middle
          - dps_val: 3
            value: high
  - entity: select
    name: Working day settings
    icon: "mdi:calendar"
    category: config
    dps:
      - id: 42
        type: integer
        name: option
        mapping:
          - dps_val: 1
            value: 5+2
          - dps_val: 2
            value: 6+1
          - dps_val: 3
            value: "7.0"
  - entity: number
    name: Max settings temperature
    category: config
    icon: "mdi:arrow-expand-horizontal"
    dps:
      - id: 21
        type: integer
        name: value
        unit: C
        range:
          min: 15
          max: 95
  - entity: number
    name: External sensor temperature limit
    category: config
    icon: "mdi:arrow-expand-horizontal"
    dps:
      - id: 107
        type: integer
        name: value
        unit: C
        range:
          min: 5
          max: 60
  - entity: number
    name: Temperature hysteresis switch difference
    category: config
    icon: "mdi:arrow-expand-horizontal"
    dps:
      - id: 105
        type: integer
        name: value
        mapping:
        - scale: 10
        range:
          min: 5
          max: 95

I have two questions:
Would it be possible, if you could explain to me, how to program the 3 modes within icons/symbols on the gauge?
Or maybe you could do the adaption.
Is it possible, if the device is locked by "Child lock", that everything is locked and adjustments are disabled.
At the moment, it is possible to change settings even if Child lock is enabled.
Thank you.

This seems almost identical to the Beok TOL47 thermostat, which was added recently, including the quirky use of cold/hot/wind for manual/program/temporary program.  So I have filled in the unknowns from there to add support for

parameter of weekly schedule (time and temperature settings)
Sound on/off
Frost protection settings
Switching relay between NC and NO

There is still one unknown which might be the reset, but that is not present on the Beok thermostat so I cannot confirm - the only other difference is the temperature ranges so I am reasonably confident that these share firmware and probably come from the same original manufacturer just rebranded for different markets, so the mapping of unknowns to the above functions should be correct.
Regarding the symbols, Home Assistant has a fixed list of valid hvac_modes, so you can only use those.  As with the Beok thermostat, Manual maps to heat, but both Program and Temporary Program map to auto in Home Assistant's modes, so it is not possible to distinguish these through the hvac_mode, only through the preset_mode selector (which you can access through the ... menu on the Climate card).
The behaviour of child lock is defined by the device's firmware.  Generally it should lock adjustments from the device itself.  If you are wanting it to lock adjustments made from Home Assistant UI, you will have to do this by scripting your UI.  I don't know if HA provides ways to disable or hide cards from automations, but this is probably what you would need to do, and hide the lock control so it is only visible to admin users, not from a wall tablet for example.


Flutter Android studio plugin - wrong option when it's already Android studio

Steps to Reproduce
As shown in attached image, we are finding "Open in Android studio..." option under Tools => Flutter menu, though it's already Android studio.
Flutter Doctor
/Users/paresh.mayani/Documents/Paresh/Software_Tools/flutter/bin/flutter --no-color doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, v0.1.5, on Mac OS X 10.13.1 17B1003, locale en-IN)
[✓] Android toolchain - develop for Android devices (Android SDK 27.0.3)
[!] iOS toolchain - develop for iOS devices (Xcode 9.2)
✗ libimobiledevice and ideviceinstaller are not installed. To install, run:
brew install --HEAD libimobiledevice
brew install ideviceinstaller
✗ ios-deploy not installed. To install:
brew install ios-deploy
✗ CocoaPods not installed.
CocoaPods is used to retrieve the iOS platform side's plugin code that responds to your plugin usage on the Dart side.
Without resolving iOS dependencies with CocoaPods, plugins will not work on iOS.
For more info, see https://flutter.io/platform-plugins
To install:
brew install cocoapods
pod setup
[✓] Android Studio
[✓] Android Studio (version 3.0)
[✓] Connected devices (2 available)
! Doctor found issues in 1 category.
Process finished with exit code 0

/cc @devoncarew

Thanks for the feedback! @stevemessick can confirm, but I believe this option will actually open the android/ submodule in Android Studio, allowing you to better work on the native side of your app.

Clicking on this option is opening .iml file actually!

That menu item name was misleading. It was changed in https://github.com/flutter/flutter-intellij/pull/1758. It now opens the Android module, not the .iml file. There's more work to improve it for Flutter plugins, in progress.
It would be easier for us to have Android Studio issues reported to the flutter-intellij project. We'll find them here, anyway.

cool to see this fixed! 👍


Correspondence ForSyDe SystemC model & SDF3 graph

@krosvall opened it on 29 feb 2016:

I have compared the plotted ForSyDe model of the JPEG example with the SDF graph produced by f2sdf3 and I don't see how they are the same. There seems to be something wrong in the conversion.

This issue was more complex and was treated in the projects ForSyDe-SystemC and f2sdf3.


Character escaping

Closes #36.
Closes #101.

There is a problem with including tildes in fenced code info strings, resulting in this when running tests.
Example
``` ~
```

BEGIN fencedCode
- src: ./_markdown_test/d41d8cd98f00b204e9800998ecf8427e.verbatim
- infostring: \CCC:7:126
END fencedCode

This part addresses it, though most likely incorrectly.
\catcode"7E=12%  Prevent tildes (U+007E) from being active in programmatic text

Let's add it to tests/support/setup.tex? For pipes, we added a similar command to tests/templates/context/*.m4, which we may want to move to tests/support/setup.tex for simplicity.

Something like this then?
setup.tex
\long\def\TYPE#1{%
  \catcode"7C=12%  Prevent pipes (U+007C) from being active in ConTeXt
  \catcode"7E=12%  Prevent tildes (U+007E) from being active in programmatic text
  \begingroup

There are also these two in plain-setup.tex and markdownthemewitiko_markdown_test.sty currently:
\catcode"09=12%  Prevent tabs (U+0009) from folding into a single space token
\catcode"20=12%  Prevent spaces (U+0020) from folding into a single space token

I suppose I also add them there?

I suppose I also add them there?

Let's just add them there. There is no point in changing category codes at the beginning of the \TYPE command, since by then, the arguments will already have been assigned category codes.


Escape special chars in filename like \t and \n

Basic fix for https://github.com/Peltoche/lsd/issues/335
Might also fix https://github.com/Peltoche/lsd/issues/325

Before it would put a tab instead of \t in the filename. Same for \n.

Looks like this actually breaks how emoji and other weird stuff gets rendered.
These stuff fail to render. Seems like git just prints escaped stuff, but gnu ls prints the actual chars.
("Ｈｅｌｌｏ,ｗｏｒｌｄ!", 24),
("ASCII1234-_", 13),
("File with space", 17),
("制作样本。", 12),
("日本語", 8),
("샘플은 무료로 드리겠습니다", 28),
("👩🐩", 6),
("🔬", 4),

exa seemed to have a pretty good solution for this, so I kinda stole it and adapted for our use case.

Codecov Report

Merging #384 into master will decrease coverage by 0.23%.
The diff coverage is 35.29%.

@@            Coverage Diff             @@
##           master     #384      +/-   ##
==========================================
- Coverage   66.06%   65.83%   -0.24%
==========================================
  Files          19       19
  Lines        1827     1841      +14
==========================================
+ Hits         1207     1212       +5
- Misses        620      629       +9

Impacted Files
Coverage Δ

src/meta/name.rs
88.67% <35.29%> (-3.75%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8ad5ec...7de751b. Read the comment docs.


DOI badge

Codecov Report

Merging #61 into master will decrease coverage by 1.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #61      +/-   ##
==========================================
- Coverage   74.92%   73.91%   -1.02%
==========================================
  Files          18       17       -1
  Lines        1057      920     -137
==========================================
- Hits          792      680     -112
+ Misses        265      240      -25

Impacted Files
Coverage Δ

src/gp/kernels/abstract_kernel.jl
20% <0%> (-20%)
:arrow_down:

src/gp/kernels/scaled_squared_distance.jl
92.85% <0%> (-3.92%)
:arrow_down:

src/abc/io.jl
82.14% <0%> (-3.58%)
:arrow_down:

src/abc/summary_stats.jl
94.59% <0%> (-2.71%)
:arrow_down:

src/gp/gp.jl
85.56% <0%> (-0.15%)
:arrow_down:

src/abc/simulation.jl
100% <0%> (ø)
:arrow_up:

src/gp/kernels/rbf_kernels.jl
100% <0%> (ø)
:arrow_up:

src/abc/emulation.jl
100% <0%> (ø)
:arrow_up:

src/GpABC.jl

src/gp/gp_optimisation.jl
65% <0%> (+0.55%)
:arrow_up:

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3db0b4f...bf592ae. Read the comment docs.


Fe: Document NFT types

We need proper documentation for how to mint each NFT type. The frontend is currently very opaque about it.

Starting this soon, I will need a sample models for each type:

walking pet
flying pet
mount
vehicle
hoverboard
wearable
weapons

walking pet: https://github.com/avaer/dragon-pet
flying pet: https://github.com/avaer/dragon-fly
mount: https://github.com/avaer/dragon-mount
vehicle: https://github.com/avaer/hovercraft
hoverboard: https://github.com/avaer/hoverboard
wearable: https://github.com/avaer/sword
weapons: https://github.com/avaer/pickaxe, https://github.com/avaer/rifle

Update on it. The information on these pages isn't complete, need to know what the best way is to mint things. Would it be through a URL on https://webaverse.com/mint ?
https://docs.webaverse.com/docs/create/sword
https://docs.webaverse.com/docs/create/pickaxe
https://docs.webaverse.com/docs/create/guns
https://docs.webaverse.com/docs/create/pets
https://docs.webaverse.com/docs/create/mounts
https://docs.webaverse.com/docs/create/hoverboards


CORE-[656] Update Dashboard Team layout

Problem solved

Move the Team analytics from overview page to Analytics tab
Move the changelog to "projects" page and make that the default page to show on /team/

#5798  👈 (View in Graphite)
main

How to use the Graphite Merge Queue
Add either label to this PR to merge it via the merge queue:

merge-queue - adds this PR to the back of the merge queue
hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.


Propagate the changes from DPG-Standard#20

Adjust screening-schema.json to reflect the changes introduced in DPGAlliance/DPG-Standard#20

Changes from DPG-Standard are now reflecting in screening-schema.json as documented in commit 3741565.


logout is not working in superset

When i click logout option in dropdown i couldn't able to logout.  Again the same user dashboard was still i can able to see and the user was logged in. Session was not clearing while logout option click.
i have tried using localstorage.clear() was not working. any other way to make it work will be helpful thanks
Expected results
User should able to logout.
Actual results
User once logged in. He can't logout.
How to reproduce the bug

Go to 'Right side top dropdown
Click on 'logout'
User will not able to logout

@SeLvAKuMaR-007 - which version are you running?  I end up having this work properly with my Custom Security Manager using Okta as the OAuth source on v0.33.0rc1.

Any infos for this? I cannot logout either. I am redirected to login and then superset/welcome and i am still loggin.


Update VanGogh.toml

https://github.com/anoma/namada-testnets/pull/1512 https://github.com/anoma/namada-testnets/pull/1684
Description
All previous genesis validators should name their PRs "Update {validator_alias}.toml for tesntet-15" and provide links to previous PRs merged.
If this is an UPDATE for a previous genesis validator
Checks are made against your net_address. If this has changed since the previous testnet, make sure you provide links of previous prs merged from your previous git username.
Thanks in advance!
Checklist before merging

[x] Only one toml is added in this PR
[x] The file being added is indeed a .toml file
[x] The toml being added is to the correct folder, and only to the correct folder
[x] The eth_hot_key and eth_cold_key are present
[x] The email, discord, elements telegram, and twitter fields are present and valid

Successfully validated transactions.toml file for commit hash bb9271d


Deleted ticket counts should include PGTs in the final cumulative count

Closes #1969

Reported deleted ticket counts may not include PGTs in the final cumulative count.  This would not affect TGTs and STs.
Test cases have been updated to add PGTs to the registry after they are created.  The counts are checked to make sure of accuracy.
Related pull request: #1978

Merge with master.


chore: docker build optimization

PR Type

[x] Build-related changes
[x] CI-related changes
What Is the Current Behavior?

npm run synchronize-lazy-components slow in docker build
docker build includes build cache

What Is the New Behavior?

docker build is run in CI mode (disables angular build cache and schematic lint fixes)
read CI variable in schematics from env

Does this PR Introduce a Breaking Change?

[ ] Yes
[x] No
Other Information
AB#74538

In addition to this PR I want to point out what The Angular - Persistent disk cache page states:

The Angular CLI checks for the presence and value of the CI environment variable to determine in which environment it is running.

Which is the explanation why no further changes are included in this PR to implement the claim "Disables angular build caches"


Implement new async thumbnail feature

Implementation of task https://github.com/concrete5/concrete5/issues/9626

This looks really close – I have some feedback, I'll attach it to the separate ticket.

As far as i have understand the docs it is not possible to use the placeholder class for images. Even in the docs page itself's you can see that for the placeholder image of the card an SVG element is used. And this SVG is part of the static HTML code. It is not generated dynamically by JavaScript. That was the reason why i'm generating SVG too.

Hmm, I didn't know that placeholder functionality didn't work for images. Is there any way we could make the temporary gray image something less static than the gray square? Some CSS class that could denote that the thumbnail is loading in? Otherwise I think we're ready to merge this in.

@aembler I have added the bootstrap placeholder classes. Now you have the animation effects applied to the image placeholders.

I have merged this in here – https://github.com/concrete5/concrete5/pull/9921 – had to do a secondary pull request due to conflicts.
This looks great!


Is this project not maintained anymore?

I see that there haven't been commits/ pull requests this year. I open issues and get no answer. Is this project not maintained anymore?

Yes, it's maintained. I consider the project stable, still use it in all my apps and haven't seen anything that can't be avoided. I'm busy with several other projects and have a little time to follow GitHub issues. The fact is that UI debugging requires a lot of monkey testing (multiplied by the number of themes).


Cursor is not moved by arrow keys of keyboard in output panel

VSCode Version: 1.6.0
OS Version: Mac 10.11.6

Steps to Reproduce:

open output panel (cmd+shift+U).
focus at the panel.
press the arrow keys on the keyboard.
cursor is not moved.

It is very inconvenient 😢

October to investigate

Thank you for investigation!
This is not new feature request.
This feature is possible less than 1.6.0, but It became impossible at 1.6.0.

The bottom line issue is that editorTextFocus is not set as a context key when output panel is focused. Thus none of the actions that require editorTextFocus are triggered in the output. There are different ways to solve this:

go back to the old behavior that 'editorTextFocus' is fullfiled if output has focus. Currently 'editorTextFocus' is set for a CodeEditorWidget and since the output is just a StringEditor it does not set this context key nicely any more
just add the or in the when clause for the missing actions so they are also triggered if 'inOutput'

@alexandrudima @jrieken @bpasero  does it intutivly make sense that 'editorTextFocus' is fullfiled if output is focused. If yes we should go with the first soltuion, if not we should go with the second. Though this might break some people - but apperantly not a lot since this is the only complaint thus far.
@takecy as a workaround please add to your keybindings.json (preferences > keyboard shortcuts)
{ "key": "up",                    "command": "cursorUp", "when": "inOutput"},
{ "key": "down",                    "command": "cursorDown", "when": "inOutput"}

@isidorn
Thank you for investigation and show workaround.
I have obtained the desired behavior after i added below to keybindings.json  😄
{
    "key": "up",
    "command": "cursorUp",
    "when": "inOutput"
},
{
    "key": "down",
    "command": "cursorDown",
    "when": "inOutput"
},
{
    "key": "shift+up",
    "command": "cursorUpSelect",
    "when": "inOutput"
},
{
    "key": "shift+down",
    "command": "cursorDownSelect",
    "when": "inOutput"
},
{
    "key": "left",
    "command": "cursorLeft",
    "when": "inOutput"
},
{
    "key": "right",
    "command": "cursorRight",
    "when": "inOutput"
},
{
    "key": "shift+left",
    "command": "cursorLeftSelect",
    "when": "inOutput"
},
{
    "key": "shift+right",
    "command": "cursorRightSelect",
    "when": "inOutput"
},
{
    "key": "shift+alt+left",
    "command": "cursorWordStartLeftSelect",
    "when": "inOutput"
},
{
    "key": "shift+alt+right",
    "command": "cursorWordStartRightSelect",
    "when": "inOutput"
}

@isidorn The editorTextFocus should be set by the code editor widget as soon as it has focus. I'd wait for @alexandrudima to comment

@jrieken and it is properly set right now. But what changed is that this moved to the code editor widget and output is not a code editor widget.

output is not a code editor widget

What? We both talk about this?

Yeah. OutputPanel is a StringEditor which is a BaseTextEditor which is aBaseEditor.
I do not see the CodeEditorWidget class in the hierarchy

It is never in those hierarchies but the BaseEditor just used it

@alexandrudima a gentle reminder to comment once you find time. If not will set this for november since there is a workaround

I haven't done anything specific for the output editor, it is AFAIK a regular CodeEditorWidget, which sets the editorTextFocus key as usual on the passed in IContextKeyService. Perhaps something is wrong with the passed in IContextKeyService?

Thanks, I will investigate.

I've just logged the context and the output editor has no editor context, only the global context.

I think the culprit is 08d96455950f917521665a8fff8019f5a49142a8
Easier test-case: Ctrl+F followed by Escape

@alexandrudima thanks a lot - yes that is the root cause. I was not aware that by adding a scopedContextKeyService I would overwrite the original one. The reason why I did that is that I wanted a context key that is true when only output is focussed.
I can not bind that context key to the editorContextKey since that one will be true whenever any editor is focussed.
And there is no other HTML element I can bind it to.
This particular issue we can easily fix by just reverting my commit - but is there a way for me to also add the key for output that I want.

@alexandrudima if time permits do a code review.
I am doing the same as in repl now and it works just fine. I create the scoped instantiation service and I create the editor using that service. Though I still put my context key on the editor html element but they seem to get somehow nicely merged

Verified that I can use arrow keys in output panel to move up and down.

Thank you for fixing this issue.
works fine!


Symfony3 compatibility

Symfony\Component\DomCrawler\Crawler changes in 3.0 : it doesn't implement anymore \SplObjectStorage so we can't rely anymore on contains method in our tests.
I don't really know this component, if someone have a better idea.. ? ping @jubianchi

You can use "in_array" function instead of "contains"
and it looks like this
->array(iterator_to_array($children))
  ->contains($child)

https://github.com/aliel/AtoumBundle/commit/66e05d0f1a657a1ee81b9d324333ad65ac3b6abd

:+1:

merge possible ?

If everybody is OK, I'll merge this tomorrow and issue an new release.

@jubianchi ok for me :)

OK

Up please ?

https://github.com/atoum/AtoumBundle/releases/tag/1.4.0


TypeScript: too much console log noise in typescript-test-intro tests?

Feedback that there is too much log output in the unit tests for typescript-test-intro, leading to confusion about if the test runs are successful or not.
Evaluate the output to determine if its working as expected. Consider suppressing logs during unit test runs (maybe with a logging lib like the Lambda PowerTools logger).

Although Jest explicitly leaves logging turned on (see this thread), there is a way to turn it off through the CLI using the --silent flag.
Looking at baking this into the run commands or jest config.


add submodule spconv v1.0 (8da6f96)

Add spconv v1.0 as a submodule, which is convenient to use the flag --recursive to clone the whole repository including PCDet and spconv which mentioned in the README.

I hope to leave it up to the user to install the spconv, since there are lots of users who have already installed spconv before.
I think you could update the readme to indicate that if the user doesn't install spconv yet, he could run git submodule update --init --recursive to clone the spconv repo, and then install it.


Issues with Pi-Plate node forcing nodered restarts

Having an issue with nodered, once the relay node is deployed it forces node red to schedule restart and sits there on a constant loop.
I'm about out of ideas on how to get it too run.
I have followed the install guide on the pi plates website, installed pi-plates module through pip.

8 Jul 17:15:29 - [info] [xiaomi-gateway-config:Xiomi Gateway - Amarok] UDP socket listening on <IP_ADDRESS>:9898
8 Jul 17:15:29 - [info] [tcp in:b6f3127a.bdf7e] listening on port 8099
8 Jul 17:15:29 - [red] Uncaught Exception:
8 Jul 17:15:29 - Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
at doWrite (_stream_writable.js:406:19)
at writeOrBuffer (_stream_writable.js:394:5)
at Socket.Writable.write (_stream_writable.js:294:11)
at WorkQueue.do_cmd [as worker] (/home/pi/.node-red/node_modules/pi-plates/BASEplate.js:26:17)
at Immediate. (/home/pi/.node-red/node_modules/vasync/lib/vasync.js:562:6)
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Service RestartSec=100ms expired, scheduling restart.
nodered.service: Scheduled restart job, restart counter is at 2.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
8 Jul 17:15:32 - [info]
Welcome to Node-RED
8 Jul 17:15:32 - [info] Node-RED version: v1.0.3
8 Jul 17:15:32 - [info] Node.js  version: v10.15.2
8 Jul 17:15:32 - [info] Linux 4.19.97-v7l+ arm LE
8 Jul 17:15:34 - [info] Loading palette nodes
8 Jul 17:15:37 - [info] Dashboard version 2.22.1 started at /ui
8 Jul 17:15:38 - [info] Settings file  : /home/pi/.node-red/settings.js
8 Jul 17:15:38 - [info] Context store  : 'default' [module=localfilesystem]
stderr: Traceback (most recent call last):
File "/home/pi/.node-red/node_modules/pi-plates/plate_io.py", line 3, in
import piplates.DAQCplate as DP
ModuleNotFoundError: No module named 'piplates'
code: 1 signal: null

It appears to be looking for a module called piplates, isnt the module called 'pi-plates' ?

I also get the same error when launching module 'piplates' from Python Shell.
">>> import piplates.RELAYplate as RELAY
Traceback (most recent call last):
File "<pyshell#3>", line 1, in
import piplates.RELAYplate as RELAY
ModuleNotFoundError: No module named 'piplates'

"

Yet when i run in Terminal, i can manually turn on relays on the Pi Plate....
pi@raspberrypi:~/.node-red $ python
Python 2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import piplates.RELAYplate as RELAY
RELAY.getID(0)
'Pi-Plate RELAY'
RELAY.relayON(0,1)

As you would expect relay 1 turns on...

At first glance, it looks like you've installed the python 2.7 version of the pi-plates python library.  The Node-RED package (as of the most recent release) requires the python 3 version of the pi-plates library.  Could you try installing the python 3 version (e.g. 'sudo apt install python3-pip' then 'pip3 install pi-plates')?

You my friend have just saved me from going bald!
Thank you so much, really appreciate you taking the time to reply and solve my issue!
For those n00bs who stumble upon this issue aswell. The install needs to be done as below;
Sudo apt update
sudo apt install python3-pip
pip3 install pi-plates
Thanks to mharsch for the solution!

Isn't pip3 already part of the RPi OS image?
On Wed, Jul 8, 2020, 8:26 AM beccsjb<EMAIL_ADDRESS>wrote:

You my friend have just saved me from going bald!
Thank you so much, really appreciate you taking the time to reply and
solve my issue!
For those n00bs who stumble upon this issue aswell. The install needs to
be done as below;
Sudo apt update
sudo apt install python3-pip
pip3 install pi-plates
Thanks to mharsch for the solution!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Harsch-Systems/node-red-contrib-pi-plates/issues/9#issuecomment-655487310,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACACCANTM5MDESEZSSAPQ3TR2RQWJANCNFSM4OUMQGJQ
.

Neither pip nor pip3 are installed by default on the 'lite' version of Raspberry Pi OS.  It looks like they are installed by default when you start with the 'full' image.


Various improvements

Remove executable permission
Use max-base v1.1.1
Avoid creating a Docker layer containing the compressed model file.

@xuhdev did you happen to notice the size difference before/after this PR?

Yes, it's a 300M reduction:
max-image-resolution-enhancer-origin   latest              e252f1e71091        4 seconds ago       2.26GB
max-image-resolution-enhancer          latest              713e8bbe7743        2 minutes ago       1.89GB

Great, thanks a lot @xuhdev!!!!!


hetznerctl not executable

I installed hetzner for python.
root@om:~# pip install hetzner

Collecting hetzner
  Downloading hetzner-0.7.5.tar.gz
Building wheels for collected packages: hetzner
  Running setup.py bdist_wheel for hetzner ... done
  Stored in directory: /root/.cache/pip/wheels/ff/4e/42/3aac160c19e683ce1db4f7f47882187843907cc61939a3df4c
Successfully built hetzner
Installing collected packages: hetzner
Successfully installed hetzner-0.7.5

But the script is not executable:
root@om:~# hetznerctl
-bash: /usr/local/bin/hetznerctl: Keine Berechtigung

root@om:~# ls -l /usr/local/bin/hetznerctl
-rw-r--r-- 1 root root 8327 Feb 26 22:12 /usr/local/bin/hetznerctl

Tried to reproduce this on a clean Ubuntu 17.10 VM and was unable to reproduce this:
root@ubuntu-test:~# pip install hetzner
Collecting hetzner
  Downloading hetzner-0.7.5.tar.gz
Building wheels for collected packages: hetzner
  Running setup.py bdist_wheel for hetzner ... done
  Stored in directory: /root/.cache/pip/wheels/ff/4e/42/3aac160c19e683ce1db4f7f47882187843907cc61939a3df4c
Successfully built hetzner
Installing collected packages: hetzner
Successfully installed hetzner-0.7.5
root@ubuntu-test:~# hetznerctl
Usage: hetznerctl [global options] command [options]

hetznerctl: error: No config file available, specify -u and -p to create a new one.
root@ubuntu-test:~# which hetznerctl
/usr/local/bin/hetznerctl
root@ubuntu-test:~# ls -l /usr/local/bin/hetznerctl
-rwxr-xr-x 1 root root 8327 Nov 18 16:41 /usr/local/bin/hetznerctl
root@ubuntu-test:~#

@guettli: Can you please give more details about your system?

Closing due to inactivity.


Common Issues Section

In the common issues section the 2nd item around "You can't access the application" the resolution indicates "Step 3 Choose API's to access" is not clear as there are multiple Step 3 sections in this document and note describe Choosing API's.
Please clarify this more.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 717d397d-4186-c7de-431d-3260d3f0484a
Version Independent ID: bdd7545b-5970-93e0-5cbb-a59da4ace01b
Content: Configure Power BI integration for workspaces - Finance & Operations | Dynamics 365 | #MSDyn365FO
Content Source: articles/dev-itpro/analytics/configure-power-bi-integration.md
Service: dynamics-ax-platform
GitHub Login: @MilindaV2
Microsoft Alias: milindav

updated the document


Sidebars and edit area not loaded correctly for user with System Admin and Wiki Approval role

In our setup, clicking edit on a wiki page only ok for Administrator.
Any other users even with System Admin and Wiki Approver roles only see blank edit page without sidebars like follow image
bench version output:
education 0.0.1
erpnext 14.18.3
frappe 14.28.2
hrms 15.0.0-dev
wiki 0.0.1
Do we need to set other roles?

This is weird. The sidebar on /edit-wiki doesn't depend on permissions or roles afaik. I tried with both website user and system admin with wiki approval role and it works fine.
This feels like wiki failed to fetch the root sidebar but that too doesn't depend on any roles.
I've created a telegram group for wiki support. Feel free to join the group and we can nail down this issue.

Closing since this isn't relevant after #124


Feature/drag categories

Issue #234
Changes
User can now change categories order:

Added react-beautiful-dnd (See motivation for this below)
Created Cypress tests

Motivation for adding react-beautiful-dnd
It provides an easy and accessible way to use drag and drop in a react app with beautiful animations. This can allow for new functionalities for drag and drop within the app such as adding multiple editor windows, or modifying the layout order.

Nice. Thanks for clean tests 👌