Envoy segfaults when a dynamic listener fails to bind to a port and gRPC access logging is enabled

Envoy segfaults for me when a dynamic listener fails to bind to a port (e.g. because another process is already bound to that port) and gRPC access logging is enabled for that listener. This is happening since at least Envoy 07fa5e0a43b489d907e402d0bcb0ff85d40f823a (Jan 15 2020) and continues today on master (f958d39fcd927603e15460870d36febd27156e05).
I'm not sure if this is truly related to gRPC access logging, but I wasn't able to narrow down the cause further.
Repro steps:
The smallest reproduction I can find is using the envoy.yaml and listeners.yaml posted here:
https://gist.github.com/chriskuehl/a0672bd237216f5b339daa2980fc3f17

Copy envoy.yaml and listeners.yaml from the gist.
Start any process (not envoy) listening on port 25000. This will conflict with the dynamic listener from the config above and cause the segfault. For example you can run nc -l -p 25000.
With the port still occupied, start Envoy with envoy --config-path envoy.yaml; the Envoy worker should segfault immediately upon start (see the gist for output logs).

Note that the listener needs to be defined as a dynamic resource for this to happen. If you comment out the "access_log" settings from the listener's config and try starting Envoy, you will see an error about the port being unavailable, but no segfault.
I tested this using the envoyproxy/envoy-alpine-debug-dev Docker image today with Envoy sha f958d39fcd927603e15460870d36febd27156e05 and linux kernel 4.4.0-1094-aws.
Config:
https://gist.github.com/chriskuehl/a0672bd237216f5b339daa2980fc3f17
Logs:
https://gist.github.com/chriskuehl/a0672bd237216f5b339daa2980fc3f17
Call Stack:
Unfortunately I have not been able to get a backtrace to be printed.

This is the same problem we have had multiple times now where we do a TLS set, capture this, and have it go away before the set completes. I'm going to see if I can figure out a safer way of handling this generically.
(gdb) bt
#0  Envoy::Extensions::AccessLoggers::HttpGrpc::HttpGrpcAccessLog::HttpGrpcAccessLog(std::__1::unique_ptr<Envoy::AccessLog::Filter, std::__1::default_delete<Envoy::AccessLog::Filter> >&&, envoy::extensions::access_loggers::grpc::v3::HttpGrpcAccessLogConfig, Envoy::ThreadLocal::SlotAllocator&, std::__1::shared_ptr<Envoy::Extensions::AccessLoggers::GrpcCommon::GrpcAccessLoggerCache>, Envoy::Stats::Scope&)::$_0::operator()(Envoy::Event::Dispatcher&) const (this=0x55555d7e5e18)
    at source/extensions/access_loggers/grpc/http_grpc_access_log_impl.cc:46
#1  0x0000555558a5576b in _ZNSt3__18__invokeIRZN5Envoy10Extensions13AccessLoggers8HttpGrpc17HttpGrpcAccessLogC1EONS_10unique_ptrINS1_9AccessLog6FilterENS_14default_deleteIS8_EEEEN5envoy10extensions14access_loggers4grpc2v323HttpGrpcAccessLogConfigERNS1_11ThreadLocal13SlotAllocatorENS_10shared_ptrINS3_10GrpcCommon21GrpcAccessLoggerCacheEEERNS1_5Stats5ScopeEE3$_0JRNS1_5Event10DispatcherEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSY_DpOSZ_ (__f=..., __args=...)
    at /opt/llvm/bin/../include/c++/v1/type_traits:3539
#2  0x0000555558a556bb in std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> >::__call<Envoy::Extensions::AccessLoggers::HttpGrpc::HttpGrpcAccessLog::HttpGrpcAccessLog(std::__1::unique_ptr<Envoy::AccessLog::Filter, std::__1::default_delete<Envoy::AccessLog::Filter> >&&, envoy::extensions::access_loggers::grpc::v3::HttpGrpcAccessLogConfig, Envoy::ThreadLocal::SlotAllocator&, std::__1::shared_ptr<Envoy::Extensions::AccessLoggers::GrpcCommon::GrpcAccessLoggerCache>, Envoy::Stats::Scope&)::$_0(Envoy::Event::Dispatcher&)&>(Envoy::Extensions::AccessLoggers::HttpGrpc::HttpGrpcAccessLog::HttpGrpcAccessLog(std::__1::unique_ptr<Envoy::AccessLog::Filter, std::__1::default_delete<Envoy::AccessLog::Filter> >&&, envoy::extensions::access_loggers::grpc::v3::HttpGrpcAccessLogConfig, Envoy::ThreadLocal::SlotAllocator&, std::__1::shared_ptr<Envoy::Extensions::AccessLoggers::GrpcCommon::GrpcAccessLoggerCache>, Envoy::Stats::Scope&)::$_0(Envoy::Event::Dispatcher&)&) (__args=..., __args=...)
    at /opt/llvm/bin/../include/c++/v1/__functional_base:317
#3  0x0000555558a5565b in std::__1::__function::__alloc_func<Envoy::Extensions::AccessLoggers::HttpGrpc::HttpGrpcAccessLog::HttpGrpcAccessLog(std::__1::unique_ptr<Envoy::AccessLog::Filter, std::__1::default_delete<Envoy::AccessLog::Filter> >&&, envoy::extensions::access_loggers::grpc::v3::HttpGrpcAccessLogConfig, Envoy::ThreadLocal::SlotAllocator&, std::__1::shared_ptr<Envoy::Extensions::AccessLoggers::GrpcCommon::GrpcAccessLoggerCache>, Envoy::Stats::Scope&)::$_0(std::__1::allocator<std::__1::allocator>, std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&))>::operator()(std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)) (
    this=0x55555d7e5e18, __arg=...) at /opt/llvm/bin/../include/c++/v1/functional:1540
#4  0x0000555558a5477d in std::__1::__function::__func<Envoy::Extensions::AccessLoggers::HttpGrpc::HttpGrpcAccessLog::HttpGrpcAccessLog(std::__1::unique_ptr<Envoy::AccessLog::Filter, std::__1::default_delete<Envoy::AccessLog::Filter> >&&, envoy::extensions::access_loggers::grpc::v3::HttpGrpcAccessLogConfig, Envoy::ThreadLocal::SlotAllocator&, std::__1::shared_ptr<Envoy::Extensions::AccessLoggers::GrpcCommon::GrpcAccessLoggerCache>, Envoy::Stats::Scope&)::$_0(std::__1::allocator<std::__1::allocator>, std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&))>::operator()(std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)) (this=0x55555d7e5e10,
    __arg=...) at /opt/llvm/bin/../include/c++/v1/functional:1714
#5  0x000055555a7cee0b in std::__1::__function::__value_func<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>::operator()(Envoy::Event::Dispatcher&) const (this=0x55555d7e5e10, __args=...)
    at /opt/llvm/bin/../include/c++/v1/functional:1867
#6  0x000055555a7cbc56 in std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>::operator()(Envoy::Event::Dispatcher&) const
    (this=0x55555d7e5e10, __arg=...) at /opt/llvm/bin/../include/c++/v1/functional:2473
#7  0x000055555a7c328b in Envoy::ThreadLocal::InstanceImpl::Bookkeeper::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_7::operator()(Envoy::Event::Dispatcher&) const (this=0x55555d7e5e10,
    dispatcher=...) at source/common/thread_local/thread_local_impl.cc:101
#8  0x000055555a7c323b in _ZNSt3__18__invokeIRZN5Envoy11ThreadLocal12InstanceImpl10Bookkeeper3setENS_8functionIFNS_10shared_ptrINS2_17ThreadLocalObjectEEERNS1_5Event10DispatcherEEEEE3$_7JSB_EEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSG_DpOSH_ (__f=..., __args=...) at /opt/llvm/bin/../include/c++/v1/type_traits:3539
#9  0x000055555a7c31bb in std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> >::__call<Envoy::ThreadLocal::InstanceImpl::Bookkeeper::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_7(Envoy::Event::Dispatcher&)&>(Envoy::ThreadLocal::InstanceImpl::Bookkeeper::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_7(Envoy::Event::Dispatcher&)&) (__args=...,
    __args=...) at /opt/llvm/bin/../include/c++/v1/__functional_base:317
#10 0x000055555a7c315b in std::__1::__function::__alloc_func<Envoy::ThreadLocal::InstanceImpl::Bookkeeper::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_7(std::__1::allocator<std::__1::allocator>, std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&))>::operator()(Envoy::Event::Dispatcher&) (this=0x55555d7e5e10, __arg=...)
    at /opt/llvm/bin/../include/c++/v1/functional:1540
#11 0x000055555a7c232d in std::__1::__function::__func<Envoy::ThreadLocal::InstanceImpl::Bookkeeper::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_7(std::__1::allocator<std::__1::allocator>, std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&))>::operator()(Envoy::Event::Dispatcher&) (this=0x55555d7e5e00, __arg=...)
    at /opt/llvm/bin/../include/c++/v1/functional:1714
#12 0x000055555a7cee0b in std::__1::__function::__value_func<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>::operator()(Envoy::Event::Dispatcher&) const (this=0x55555dc90e60, __args=...)
---Type <return> to continue, or q <return> to quit---
    at /opt/llvm/bin/../include/c++/v1/functional:1867
#13 0x000055555a7cbc56 in std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>::operator()(Envoy::Event::Dispatcher&) const
    (this=0x55555dc90e60, __arg=...) at /opt/llvm/bin/../include/c++/v1/functional:2473
#14 0x000055555a7ca833 in Envoy::ThreadLocal::InstanceImpl::SlotImpl::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_13::operator()() const (this=0x55555dc90e50)
    at source/common/thread_local/thread_local_impl.cc:214
#15 0x000055555a7ca7dd in _ZNSt3__18__invokeIRZN5Envoy11ThreadLocal12InstanceImpl8SlotImpl3setENS_8functionIFNS_10shared_ptrINS2_17ThreadLocalObjectEEERNS1_5Event10DispatcherEEEEE4$_13JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSG_DpOSH_ (
    __f=...) at /opt/llvm/bin/../include/c++/v1/type_traits:3539
#16 0x000055555a7ca78d in std::__1::__invoke_void_return_wrapper<void>::__call<Envoy::ThreadLocal::InstanceImpl::SlotImpl::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_13&>(Envoy::ThreadLocal::InstanceImpl::SlotImpl::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_13&) (__args=...)
    at /opt/llvm/bin/../include/c++/v1/__functional_base:348
#17 0x000055555a7ca75d in std::__1::__function::__alloc_func<Envoy::ThreadLocal::InstanceImpl::SlotImpl::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_13(std::__1::allocator<std::__1::allocator>, void ())>::operator()() (this=0x55555dc90e50)
    at /opt/llvm/bin/../include/c++/v1/functional:1540
#18 0x000055555a7c993e in std::__1::__function::__func<Envoy::ThreadLocal::InstanceImpl::SlotImpl::set(std::__1::function<std::__1::shared_ptr<Envoy::ThreadLocal::ThreadLocalObject> (Envoy::Event::Dispatcher&)>)::$_13(std::__1::allocator<std::__1::allocator>, void ())>::operator()() (this=0x55555dc90e40) at /opt/llvm/bin/../include/c++/v1/functional:1714
#19 0x0000555558c1b3c5 in std::__1::__function::__value_func<void ()>::operator()() const
    (this=0x7ffff4801050) at /opt/llvm/bin/../include/c++/v1/functional:1867
#20 0x0000555558c1b385 in std::__1::function<void ()>::operator()() const (
    this=0x7ffff4801050) at /opt/llvm/bin/../include/c++/v1/functional:2473
#21 0x000055555a9fbe19 in Envoy::Event::DispatcherImpl::runPostCallbacks (
    this=0x55555d863980) at source/common/event/dispatcher_impl.cc:234
#22 0x000055555a9fbd15 in Envoy::Event::DispatcherImpl::run (this=0x55555d863980,
    type=Envoy::Event::Dispatcher::RunType::Block)
    at source/common/event/dispatcher_impl.cc:203
#23 0x000055555a9cb20a in Envoy::Server::WorkerImpl::threadRoutine (this=0x55555d74bbd0,
    guard_dog=...) at source/server/worker_impl.cc:133
#24 0x000055555a9d27ec in Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4::operator()() const (this=0x55555d6ab0b8) at source/server/worker_impl.cc:99
#25 0x000055555a9d27ad in _ZNSt3__18__invokeIRZN5Envoy6Server10WorkerImpl5startERNS2_8GuardDogEE3$_4JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS8_DpOS9_ (__f=...) at /opt/llvm/bin/../include/c++/v1/type_traits:3539
#26 0x000055555a9d275d in std::__1::__invoke_void_return_wrapper<void>::__call<Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4&>(Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4&) (__args=...)
    at /opt/llvm/bin/../include/c++/v1/__functional_base:348
#27 0x000055555a9d272d in std::__1::__function::__alloc_func<Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4(std::__1::allocator<std::__1::allocator>, void ())>::operator()() (this=0x55555d6ab0b8) at /opt/llvm/bin/../include/c++/v1/functional:1540
#28 0x000055555a9d185e in std::__1::__function::__func<Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4(std::__1::allocator<std::__1::allocator>, void ())>::operator()() (this=0x55555d6ab0b0) at /opt/llvm/bin/../include/c++/v1/functional:1714
#29 0x0000555558c1b3c5 in std::__1::__function::__value_func<void ()>::operator()() const
    (this=0x55555d6ab0b0) at /opt/llvm/bin/../include/c++/v1/functional:1867
#30 0x0000555558c1b385 in std::__1::function<void ()>::operator()() const (
    this=0x55555d6ab0b0) at /opt/llvm/bin/../include/c++/v1/functional:2473
#31 0x000055555b87d0e2 in Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, std::__1::optional<Envoy::Thread::Options> const&)::{lambda(void*)#1}::operator()(void*) const (this=0x55555d6ab0a0, arg=0x55555d6ab0a0)
    at source/common/common/posix/thread_impl.cc:49
#32 0x000055555b87d0b5 in Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, std::__1::optional<Envoy::Thread::Options> const&)::{lambda(void*)#1}::__invoke(void*) (arg=0x55555d6ab0a0) at source/common/common/posix/thread_impl.cc:48
#33 0x00007ffff74136db in start_thread (arg=0x7ffff4815700) at pthread_create.c:463
#34 0x00007ffff713ca3f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

(Thanks for the awesome repro instructions btw!)

I'm going to prototype a change to TLS that I think will make this type of problem impossible: I'm going to cause slot destruction on the main thread to block until any pending callbacks have run on the workers. This has the downside of introducing blocking behavior that could conceivably deadlock. On the upside, it is a rare operation and should make this type of bug impossible. If we do this a lot of the complexity inside TLS around "bookkeeper" can be removed. I've thought about it a bunch and unless we do this I think it's impossible to generically prevent this problem from happening as we can't control what callers capture in their lambdas. cc @envoyproxy/maintainers and @lambdai who I think worked on the bookkeeper fix.

+1 for TLS change.
If TLS could make the destroy in master thread, there might be some code be simplified by maintaining a shared_ptr at main thread. I think I used it beside TLS before bookkeeper.
CC @stevenzzzz who initially introduced bookkeeper. I forgot the very first cases where bookkeeper is used.

I need some more context, does this happen on the envoy quit path?
this doesnt seems like a bookeeper problem,rather a problem tries to resolve :P
Bookkeeper is there to defense the race when a callback on a slotimpl is flying in worker threads while the slotimpl is destructed on the main thread, it achieves this by storing the slotimpl instance in a deferred deletion queue.
when this deferred deletion queue is also deleted on main thread, there is no way for the bookkeeper to resolve this race.

/cc stevenzzzz

I need some more context, does this happen on the envoy quit path?

No, this is during normal operation, for instance during listener/filter chain tear down due to listener add failure, LDS update, etc.

there is no way for the bookkeeper to resolve this race.

Right, that's what I'm trying to fix. :) We have had a variant of this bug at least 3 times if not more. I'm proposing that instead of the Bookkeeper model, we just block the main thread during slot destruction until all pending worker callbacks drain. This will defend against what Bookkeper was implemented for, as well as prevent the entire class of bug where someone captures "this" (or similar) and let's it go out of scope while the callbacks are draining.

is there an integration test that could reproduce this? I am curious how the race could form.
If a worker thread holds a reference to the slot and never let go, we cant block forever in the main thread. I'd say it's a anti pattern to actually hold something that's supposed to be owned(create/teardown) by mainthread only.

is there an integration test that could reproduce this? I am curious how the race could form.

I will make one as part of this fix. It's not that complicated. Basically:

Create slot
Initiate run/update on workers, capturing this.
Delete slot.
Crash. Bookeeper does not help here.

If a worker thread holds a reference to the slot and never let go, we cant block forever in the main thread. I'd say it's a anti pattern to actually hold something that's supposed to be owned(create/teardown) by mainthread only.

This is not how I will implement it. The blocking will only be until the callback runs or has already run. The worker will not be able to control the lifetime. I think this is acceptable, and as an additional benefit all of the bookeeper complexity is no longer needed and can be removed. I will prototype a change this week and we can look at it.

sounds good, thanks(

Can we make the HttpGrpcAccessLog ctor synchronous and block in it until tls has been initialized with thread-local loggers? A bonus is that it would probably resolve all corner cases around initialization...

Can we make the HttpGrpcAccessLog ctor synchronous and block in it until tls has been initialized with thread-local loggers? A bonus is that it would probably resolve all corner cases around initialization...

Unfortunately this is not possible due to the way init works. See linked fix for more discussion on this.

Unfortunately this is not possible due to the way init works. See linked fix for more discussion on this.

Sorry, I don't think I explained that well -- I meant block until callbacks used in set() return. Synchronization in the dtor doesn't require changes in the client code though, so nm.


Native REPL appends empty output

Native REPL appends empty output whereas it could be cleaner without output section as in Interactive window offered by Jupyter:
Instead of:

We could have something like:

dup of https://github.com/microsoft/vscode-python/issues/23520


Boilerplate

I added a boilerplate that may be good to use. It's main benefit is that you can build it from the home directory with just npm start.

in the main directory of boilerplate/
when I run npm install this error comes up:

<EMAIL_ADDRESS>start /home/yizongk/HunterCollege/OpenSourceWeiss/OpenSource-NY/src/boilerplate
concurrently --kill-others-on-fail "cd client && npm start" "cd server && nodemon server.js"

[1] /bin/sh: 1: nodemon: not found
And according to the README, it looks like you still would have to go to boilderplate/server and boilerplate/client and do npm install on each of those directory before you can run the app. I am not sure how this can be built from home direcotry. Or do you mean you can run this from home directory with just npm start?

It looks like we could imitate what you want from src/myapp/
By creating a package.json  in that directory and adding this
"devDependencies": {
"concurrently": "^3.6.1"
}
and
"scripts": {
"test": "echo "Error: no test specified" && exit 1",
"start": "concurrently --kill-others-on-fail "cd client && npm start" "cd server && npm start""
},

Let me know anyones thoughts on this

I went into boilerplate/server and boilerplate/client and followed the instructions that were written in the READMEs. I ran npm install for both of them, and ran npm start for the client, and this is what showed up on the website. Is it supposed to look like this?

It looks like we could imitate what you want from src/myapp/
By creating a package.json in that directory and adding this
"devDependencies": {
"concurrently": "^3.6.1"
}
and
"scripts": {
"test": "echo "Error: no test specified" && exit 1",
"start": "concurrently --kill-others-on-fail "cd client && npm start" "cd server && npm start""
},

@yizongk I don't think we have to run "npm start" in the server folder since it wasn't stated in the readme. However, I did try to run it, and I got an error "module.js:549 throw err;" It could just be on my end, though. Other than that, I'm fine with your changes!

@mxmsunny Yes, that's what it's supposed to show, since none of the content inside of the boilerplate was modified

@mxmsunny  Also, you only need to run npm start in the home directory (where the client directory and server directory are located) and that should start the client and server concurrently.

@yizongk I will add installation with one command script later on

@mxmsunny Also, you only need to run npm start in the home directory (where the client directory and server directory are located) and that should start the client and server concurrently.

That's the point that gave me the error that I mentioned in my first comment.
Here's the command I did:
cd boilerplate/
npm install
cd client/
npm install
cd ../server/
npm install
cd ..               // At this point, my current directory is boilerplate/
npm start      // Give me error

There is also so many other files and dependency, are they related to our project? or can they be removed?

Since there seems to be an error experieced by two people I don't really think we should pull this request. I do think its a good start but the issue has to be tested on your side @Chocolate-Spaghet  to see if it occurs and then see if you can fix it.

@yizongk do npm i -g concurrently nodemon in order to fix the error.

You should add dependency to your package.json and update this PR


Add link to Omnipod support article for macOS uploads that fail because of removable volumes permissions (UPLOAD-789)

Addresses UPLOAD-789.

@krystophv Please have a look at the changes since the last review :pray:


proxychains + (socks & https proxy support)

I'd quite like to see Requests have support for proxy chaining. The actual code is pretty straight forward from what I can make of this implementation: https://github.com/pagekite/PySocksipyChain/
The httplib2 module supports both SOCKS and HTTPS proxies via the same terrible library (sockipy, as I recall). It is very frustrating to work with, so I'd recommend clean implementations of SOCKS and HTTPS proxy support.
So if you can, supporting more than a single instance of a single proxy+protocol would be great. thanks,
gq

While this is quite an edge case, I'd love to see this implemented myself.
I have to wonder, though  —  why not just set up a local proxy that does the chaining for you?

I think I'm going to close this (for now) for being out of scope.
Unless someone else implements, of course. Otherwise, I have no desire to do so :)

The local proxy option is a reasonable alternative, but not always practical. If I get around to requiring this functionality again, I'll do the change myself and send a pull request.
cheers.

That would be quite welcome! (assuming it isn't too difficult to maintain)

from what I've seen, it is basically just iterating over a list of proxies (e.g. (proto, addr, port)) and issuing the correct "proxy_connect_%s'%proto call as you go. At a wild guess (ignoring the actual proxy connection code) something like 10-15 lines diff from now? ...
the proxy connection code is more problematic as it would require either writing SOCKS4/5 + HTTPS proxy support, or trying to find a usable 3rd party module as a dependency. I know of know usable 3rd party modules. (none that I'd want near my code anyway ;)

I'm gonna be cheeky and leave this here, just so I can find it later should I return to this (or if someone else is inspired to try it, heres a reference implementation):
https://github.com/pagekite/PySocksipyChain/blob/master/sockschain/__init__.py#L923


Proposal for the async action problem

There have been a number of issues surrounding the problem of asynchronous (really just impure generally) actions.  But I think the variety of things to be done by these actions are actually relatively small in number, with the overwhelming majority being simple api requests.
Api requests can be trivially described by data: {method, url, headers, body}.  So why are we imperatively triggering these requests when we can produce descriptions that can be executed by middleware?  We can create action creators for these declarative descriptions as well, even with apis identical to, say, the real fetch api.
Except of course that the native fetch api returns a promise.  But what is a promise really, if not a tree of the form: {then: [{success, failure, then}]}, where success/failure too can be declarative, pure descriptions of what should happen in each condition, or curried pure functions that return declarative descriptions.
To state the proposal succinctly: Contain all true side-effects in middleware by producing only declarative descriptions of those side-effects in your action creators.
The benefits of this over redux-thunk or redux-promise are that you no longer have to make mocks, ever, because action creators themselves no longer imperatively trigger side-effects, you can simply ignore requests for pieces of data your tests don't care about.  And probably more importantly, unlike promises and thunks which are completely opaque, these descriptions would be transparent to middleware.  The middleware can interpret (and re-interpret) these descriptions, substantially increasing the power and flexibility of middleware.
Ultimately I think this would allow a pretty interesting effect-middleware ecosystem to evolve and I think the community would be able to come up with some really interesting and powerful applications that wouldn't otherwise be possible.
@gaearon Would love to hear your thoughts on this, and i'd be happy to make some demo middleware if people are interested / don't think this is a terrible idea.
EDIT: I should note that I didn't make a PR for this (as was requested in the closing of the other threads) because this doesn't require any changes to core, it'd just be a change of convention.
EDIT2: As a motivating example, one obvious use-case for this would be a middlware that automatically memoizes API requests for data that already exists on the client.  Or similarly, grouping api requests in 50ms buckets and sending them to be executed as a batch, etc.  There are all kinds of crazy, powerful things you could do with this.
EDIT3: I just discovered the redux-requests package, and it is doing basically what i'm suggesting, except that it hasn't taken the final step and had the middleware actually execute the effect, and in so doing it IMO loses most of the potential benefits of this approach.

I have created a some super super basic prototypes if people want to mess around with them:
redux-fetch
declarative-fetch
redux-fetch is the middleware, and declarative-fetch is an action creator for fetch operations that works (mostly) just like the normal fetch api.  E.g.
function signupUser (user) {
  return post('/user/', {
    method: 'post',
    body: JSON.stringify(user),
    headers: {
      Accept: 'application/json',
      'Content-Type': 'application/json'
    }
  })
  .then(userDidLogin)
  .toJSON()
}

const userDidLogin = createAction('USER_DID_LOGIN')

The toJSON() at the end there is just to turn the declarative-promise back into a plain object.  The userDidLogin action will then get dispatched with the result of the request as its payload.
Also, I made a bunch of design decisions really quickly here prototyping this, so don't read too much into that.  Its very possible that 'fetch' shouldn't be an action type at all, or that I shouldn't be trying so hard to mimic the original APIs or whatever, I certainly haven't thought it all through yet, this is just a PoC for now.
It is also totally untested except for the fact that it has all worked together so far a couple times on my local project - so don't be surprised if something is broken.

Relevant: https://github.com/evancz/elm-effects

A glaring omission of this approach — as far as I understand — is that there is no signal to the UI that the request has been made. I'm also not sure what a failure of the API call would result in.
I've been working on a variation of the approach in the real-world example in the redux repository — now available as redux-api-middleware in npm. It takes care of API calls with the {method, url, headers, body} pattern you mention above.
I'm using this right now and works as I expected it to. I have to write tests for it though, but I don't want to do that until people say it looks like something they might use.

I'm surely not understanding the full discourse here so please take my question with that in mind: What does this provide above and beyond a simple 'agent' or 'client' or 'driver' that allow record/replay of request/response?

@agraboso Ya, it looks like you did the exact same thing I did with redux-fetch.
I'm not sure what you mean though that there is no signal to the UI that the request has been made.  Why can't you simply dispatch multiple actions?
function createUser (user) {
  return (dispatch) => {
    dispatch(REQUEST_START)
    dispatch(fetch('/user', {method: 'post', body: user})
  }
}

Or have the middleware do it, or whatever.  I don't think the loading state here is any more complex than with any other approach really.

@danmartinez101 It offers two primary benefits:

App-level code is side-effect free.  At the very least, this makes it easier to write tests, and easier to write isomorphic code.
API requests become transparent data, that can be reasoned about by middleware.  The most obvious use-case is automatic caching, but there are many others.

@ashaffer @agraboso I think separating api design from your redux actions might be the favorable approach.
So my envisoned approach is

to design a reusable framework independent application specific api based on the whatwg fetch api. Although i really hope that redux/react will be feasable solution for a very long time I don't think it is a good idea to couple api and redux middleware too tightly together.
dispatch you api calls by redux-promise
use a thunk to signal the start of a request/api call/multiple api calls
create higher order functions like submitRequest, submitRequests which combines the above techniques.

What do I miss out?

@speedskater Can you elaborate a little more on what you mean in point 1?

@ashaffer The idea of point one is:
You have some kind of server side API. As pointed out in my comment a RESTful API. You create some API functions based on your REST-Interface. This API functions return promises. For example you will generate an API function like filterTodos('myspecialfilter', SortOrder.ASC)
Based on this basic API functions you can construct simple action creators which return FSA actions (https://github.com/acdlite/flux-standard-action) with the corresponding promise (result of your api call)
In case you have more complex api calls where an action creator triggers multiple api calls (imho, it is important that they solely dependent on the parameters of the action creator) there are two possiblities:

If you are only interested in the final result: Create a compound API function which returns a single Promise and use it in the action creator.
If you are interested in intermediate states like begin, number successful operation, number failed operations or overall finishd things become a bit more difficult, but I think this could be a possible solution:

Use the redux-thunk API to dispatch your actions/promises...
Convert your reducer into a state machine which allows you to decoratively declare what should happen when.

Regarding the last point as far as i know there is currently no project making this an easy task. But i envison something like the following example modeling a session reducer which holds the current logged in state and the corresponding data. States are LOGGED_OUT, LOGGED_IN_WITHOUT_SESSION_CONTEXT, LOGGED_IN, LOGGED_IN_WITH_EXPIRED_PASSWORD.
and state transitions can happen on ACTIONS (successul/failed).
Errors are narrowed by their Exception classes (in the example InvalidPermisssions) and filter functions on the exception (eg. hasInvalidCredentials, isPasswordExpired). The same filter mechanism could be used on the actual payload in the successful case.
Corresponding actions to this reducer would be login(username, password) which tries to login first and load the session context. changepassword(username, oldpw, newpw, newpw2) which tries to change the password and load the session context.
To formulate such a reducer I would envison something like the following code snippet:
LoginStateMachine = {
    //possible state transitions in the LOGGED_OUT state
    LOGGED_OUT: {
        [ succesful(SESSION_CREATE) ]: (previousStateContent, { sessionId }) => {
            return [ LOGGED_IN_WITHOUT_SESSION_CONTEXT, { sessionId } ];
        },
        [ failed(SESSION_CREATE).with(InvalidPermissions).and(hasInvalidCredentials)
]: (previousStateContent, error) => {
            return [ LOGGED_OUT, { error } ];
        },
        [ failed(SESSION_CREATE).with(InvalidPermissions).and(isPasswordExpired) ]:
(previousStateContent, error ) => {
            return [ PASSWORD_EXPIRED, { error } ];
        },
        [ failed() ]: (previousStateContent, error) => {
            return [ LOGGED_OUT, { error } ];
        }
    },
    //possible state transitions after the inital session was created but the load of the session context is still in progress
    LOGGED_IN_WITHOUT_SESSION_CONTEXT: {
        [ succesful(SESSION_CONTEXT_LOAD) ]: (previousStateContent, { sessionContext }) => {
            return [ LOGGED_IN, Object.assign{ previousStateContent, { sessionContext } ];
        }
    },
    //possible state transitions in case the logged in user has an expired password
    PASSWORD_EXPIRED: {
        [ succesful(RENEW_PASSWORD) ] : (previousStateContent, { sessionId }) => {
            return [ LOGGED_IN_WITHOUT_SESSION_CONTEXT ];
        },
        [ failed(RENEW_PASSWORD).with(InvalidPermission).and(hasInvalidCredentials)
]: (previousStateContent, error) => {
            return [ PASSWORD_EXPIRED, { error }]
        }
    },
    //possible state transitions ins case of logged in state
    LOGGED_IN: .....
}

I am planning to create something maybe calling it redux-fsm. But it could take some more time till I have enought time to implement it. If this is interesting for someone I would really like to join forces.
@ashaffer @gaearon Does the proposed solutions make any sense or is it too over engineered?

@speedskater I think your concept of having state machines for dealing with these things is definitely interesting, and could conceivably allieve some of the boilerplate associated with handling these sorts of complicated state transitions with many intermediates.  However, I think it's mostly orthogonal to the purpose of this issue which is to think about declaratively specifying side-effects in middleware, so maybe it'd be best to discuss in a separate issue?

An interesting consequence of this kind of request de-duplication, is that for idempotent requests, loading is no longer a state - it is a materialized view.  This is because anytime the data is not yet here, from the perspective of your ui code, you can just make the request again.
Consider a / route with a different view if you're authenticated users:
router({
  '/': function (state) {
    if (!state.authResolved) {
      dispatch(CHECK_AUTH)
      return (<Spinner />)
    }

    return state.loggedIn ? <App /> : <HomePage />
  }
})

We don't need to know whether a request is in-flight.  The only state to consider is: is the data we want here, or did it error in some way.

We implemented this middleware to handle api actions via axios:
/*
 * Redux middleware that handles making API requests using axios.
 *
 * Actions should be of the form:
 * {
 *   api: {
 *     url: '/url/to/request',
 *     method: 'GET|POST|PUT|DELETE',
 *     data: { ... }
 *   },
 *   pending: String|Object|Function,
 *   success: String|Object|Function,
 *   failure: String|Object|Function
 * }
 *
 * "Callbacks" can be one of three types and are optional.
 * - String: dispatch action with { type: String, payload: ? }
 *   payload will be undefined for pending,
 *   the server data response for success,
 *   or the err for failure
 * - Object: A flux action to dispatch. This could be another API call if you like
 *   as it goes through the middleware.
 * - Function: Will be called with these parameters:
 *   - dispatch: Redux store's dispatch
 *   - getState: function to get the store's state
 *   - payload: See above for information on payload
 */
module.exports = function createApiMiddleware(
  axios = require('axios'),
  Raven = require('raven-js')
) {
  return ({ dispatch, getState }) => {
    function maybeDispatch(action, payload) {
      if (!action) {
        return;
      }

      if (typeof action === 'function') {
        action(dispatch, getState, payload);
        return;
      }

      if (typeof action === 'object') {
        dispatch(action);
        return;
      }

      action = { type: action };

      if (typeof payload !== 'undefined') {
        action.payload = payload;
      }

      dispatch(action);
    }

    return next => action => {
      if (!action.api) {
        return next(action);
      }
      maybeDispatch(action.pending);

      return axios(action.api)
      .then((data) => {
        maybeDispatch(action.success, data);
      })
      .catch((err) => {
        if (err instanceof Error) {
          console.error(err);
          Raven.captureException(err);
        } else {
          console.error('API Error: ', err);
          Raven.captureMessage('API Error', { extra: err });
        }

        try {
          maybeDispatch(action.failure, err);
        } catch(dispatchErr) {
          console.error(dispatchErr);
          Raven.captureException(dispatchErr);
        }
      });
    };
  };
};

const createApiMiddleware = require('app/redux/create_api_middleware');

describe('Api Middleware', () => {
  let axios, apiMiddleware, next, dispatch, getState, Raven;
  const api = {
    url: '/url',
    method: 'GET'
  };

  beforeEach(() => {
    axios = sinon.stub().returnsPromise();
    next = sinon.stub().returns('result');
    dispatch = sinon.stub();
    getState = sinon.stub();
    Raven = {
      captureException: sinon.stub(),
      captureMessage: sinon.stub()
    };

    apiMiddleware = createApiMiddleware(axios, Raven)({ dispatch, getState });
  });

  it('invokes and returns next if the action is not an api action', () => {
    const action = { type: 'FOO' };
    const result = apiMiddleware(next)(action);

    expect(result).to.equal('result');
    expect(next).to.have.been.calledWith(action);
  });

  it('makes the api call if one is specified', () => {
    const action = { api };

    apiMiddleware(next)(action);

    expect(axios).to.have.been.calledWith(api);
  });

  it('dispatches the pending action', () => {
    const pending = 'PENDING';

    const action = {
      api,
      pending
    };

    apiMiddleware(next)(action);

    expect(dispatch).to.have.been.calledWith({ type: pending });
  });

  it('can dispatch already made actions', () => {
    const pending = { type: 'PENDING', payload: 3 };
    const action = {
      api,
      pending
    };

    apiMiddleware(next)(action);

    expect(dispatch).to.have.been.calledWith(pending);
  });

  it('dispatches the success action when axios resolves', () => {
    const success = 'SUCCESS';
    const action = {
      api,
      success
    };
    const payload = 'payload';

    axios.resolves(payload);
    apiMiddleware(next)(action);

    expect(dispatch).to.have.been.calledWith({ type: success, payload });
  });

  it('dispatches the failure action when axios rejects', () => {
    const failure = 'FAILURE';
    const action = {
      api,
      failure
    };
    const err = 'err';

    axios.rejects(err);
    apiMiddleware(next)(action);

    expect(dispatch).to.have.been.calledWith({ type: failure, payload: err });
  });

  it('calls functions when passed as actions', () => {
    const action = {
      api,
      pending: sinon.stub(),
      success: sinon.stub(),
      failure: sinon.stub()
    };

    const payload = 'payload';
    const err = 'err';
    axios.resolves(payload);
    apiMiddleware(next)(action);

    expect(action.pending).to.have.been.calledWith(dispatch, getState);
    expect(action.success).to.have.been.calledWith(dispatch, getState, payload);

    axios.rejects(err);
    apiMiddleware(next)(action);

    expect(action.failure).to.have.been.calledWith(dispatch, getState, err);
  });

  it('sends api errors to raven', () => {
    const failure = 'FAILURE';
    const action = {
      api,
      failure
    };
    const err = 'err';

    axios.rejects(err);
    apiMiddleware(next)(action);

    expect(Raven.captureMessage).to.have.been.calledWith('API Error', { extra: err });
  });

  it('sends exceptions to raven', () => {
    const failure = 'FAILURE';
    const action = {
      api,
      failure
    };
    const err = new Error();

    axios.rejects(err);
    apiMiddleware(next)(action);

    expect(Raven.captureException).to.have.been.calledWith(err);
  });

  it('sends exceptions to raven if failure throws', () => {
    const failureErr = new Error('failure');
    const failure = () => { throw failureErr; };
    const action = {
      api,
      failure
    };
    const err = new Error();

    axios.rejects(err);
    apiMiddleware(next)(action);

    expect(Raven.captureException).to.have.been.calledWith(err);
    expect(Raven.captureException).to.have.been.calledWith(failureErr);
  });
});

@ashaffer You are right. The state machine should be in a separate issue.
But the basic question stays where to put the side effect without bundling the application code to tightly to redux.
Imho a special middleware for requests is not an ideal solution, because its only synthactic sugar on top of thunk middleware.

alt data sources for inspiration http://techblog.trunkclub.com/alt/react/flux/babel/2015/08/13/using-alts-data-sources.html

@matystl @gaearon @jlongster (if you're still interested)
Alright guys.  I've been working on this thing for a little while now, and I think its complete enough to establish the concept:
redux-effects
Its aim is to be a sub-middleware system of redux that handles all side-effects and all sources of non-determinism (e.g. Math.random) via object descriptors.  By using this with redux and a virtual-dom library of some kind, it should be possible to write an entire front-end application without writing a single impure, non-deterministic function anywhere.  Perhaps a good way of thinking of it is virtual-dom for effects.
Because everything is just transparent data, function composition can be used to solve a much greater variety of problems.
The core abstraction of this library is declarative-promise, which allows all of your effects to compose indefinitely.  The return value of your .then is dispatched back into redux, triggering either more effects or ultimately actions that will preserve their results in state.
This all works particularly well with redux-multi, which just allows you to dispatch an array of actions simultaneously (e.g. for loading states):
const userDidLogin = createAction('USER_DID_LOGIN')
const userCreateFailed = createAction('USER_CREATE_FAILED')
const creatingUser = createAction('CREATING_USER')

function createUser (userData) {
  return [
    api.user.create(userData).then(userDidLogin, userCreateFailed),
    creatingUser()
  ]
}

Here's an example file from an application i'm currently working on that uses this pattern:
/**
 * Imports
 */

import {createAction} from 'redux-actions'
import {handleOnce} from 'declarative-events'
import cookie from 'declarative-cookie'
import {bindUrl} from 'declarative-location'
import api from 'lib/api'

/**
 * Action Creators
 */

function initializeApp () {
  return handleOnce('domready', () => [
    appDidLoad(),
    bindUrl(setUrl),
    initializeUser
  ])
}

function initializeUser () {
  return cookie('authToken')
    .then(token => token ? api.user.getCurrentUser(token) : null)
    .then(userDidResolve)
}

function signupUser (user) {
  return api.user
    .createUser(user)
    .then(userDidLogin)
}

const userDidResolve = createAction('USER_DID_RESOLVE')
const userDidLogout = createAction('USER_DID_LOGOUT')
const userDidLogin = createAction('USER_DID_LOGIN')

const setUrl = createAction('SET_URL')

const appDidLoad = createAction('APP_DID_LOAD')
const appDidRender = createAction('APP_DID_RENDER')

function loginUser (credentials) {
  return api.user
    .loginUser(credentials)
    .then(user => [
      userDidLogin(user),
      cookie('authToken', user.token)
    ])
}

/**
 * Exports
 */

export default {
  initializeApp,
  appDidLoad,
  appDidRender,
  signupUser,
  loginUser
}

DOM initialization, url watching, etc..can all be handled in a pure way using this strategy.  This makes all of this completely testable, instrumentable, and isomorphic, without really sacrificing (IMO) any developer ergonomics, aside from having to install the effect middleware.

this is pretty seksi @ashaffer.  well done.

I've made some updates.  redux-effects is no longer its own middleware stack, that package is now solely responsible for effect composition, and can be replaced with other effect composition libraries if you want, as the composition strategy is completely orthogonal to the actual effect execution.
I've also added a non-mutative and simpler composition helper, bind-effect.  It doesn't mutate any data (unlike declarative-promise) and it returns a plain object. Also, all of the action creators are no longer opinionated about composition (i.e. they don't wrap their return values in a declarative-promise anymore).
Now the only requirement for something being an effect middleware is that it return a promise.  If it does that, then redux-effects will let you compose other pure functions around its result.

I've been working on a generator based approach to this called redux-gen. Would love to know what people think.
The control flow in the previous examples seems to mimicking a generator. So if we say that actions return generators or are GeneratorFunctions then we can abstract this form of writing async action creators even further.
import { createStore, applyMiddleware } from 'redux'
import gen from '@weo-edu/redux-gen'
import rootReducer from './reducers/index'
import fetch from 'isomorphic-fetch'

// create a store that has redux-thunk middleware enabled
const createStoreWithMiddleware = applyMiddleware(
  gen()
  fetch
)(createStore);

const store = createStoreWithMiddleware(rootReducer);

// returns [
//  {username: "josh", id: 1},
//  {username: "tio", id: 2},
//  {username: "shasta", id: 3}
// ]
store.dispatch(getUsers())

// Side Effects Middleware

function fetch ({dispatch, getState}) {
  return next => action =>
    action.type === 'FETCH'
      ? fetch(action.payload.url, action.payload.params).then(res => res.json())
      : next(action)
}

// Actions

function getUsers *() {
  var userIds = yield {url: '/users', method: 'GET', type: 'FETCH'}
  return userIds.map(userId => {
    return yield {url: '/user/' + userId, method: 'GET, type: 'FETCH'}
  })
}

In order to avoid writing action creators as actual GeneratorFunction, we can use libraries like yields, which returns functions that return generators. Yields parallels the control flow that can be achieved with GeneratorFunctions, but enforces that the functions are pure. Since the actions creators have no side effects they can easily be tested by iterating of the generator returned by an action.
In the previous example, using this action would be equivalent.
import yields from '@weo-edu/yield'
var getUsers = yields(function () {
  return {url: '/users', method: 'GET', type: 'FETCH'}
}).yields(function (userIds) {
  return userIds.map(userId => {
    return {url: '/user/' + userId, method: 'GET', type: 'FETCH'}
  })
})

@joshrtay The above is really just an alternate composition strategy for redux-effects.  It is compatible with the same effect middleware ecosystem.
The primary disadvantage of using generators is that they hold their own state in an opaque way, and they encourage an impure programming style.  But on the other hand they offer significant syntactic benefits.
What's cool about this approach is that you can use either of our effect composition libraries (or both, if you wanted) without changing anything about the effect middleware or transformation middleware in your stack.  Each feature is responsible for itself and precisely nothing more.

Closing as redux-effects seems to fill the niche for people who are interested in describing effects explicitly.


Could review.docs.microsoft.com become public?

Original GitHub issue https://github.com/dotnet/docs/issues/1827 from @svick
As far as I can tell, CI in this repo publishes each PR to https://review.docs.microsoft.com/, where it presumably looks the same as it will look on https://docs.microsoft.com/. This would be especially useful, since building with docfx locally has issues (the template is very different; getting API reference right is hard; it's slow).
But, it's not accessible to me (and I assume everyone else outside MS): when I try to access it, I get a 403 error (after logging in).
A recent example where I would use this are changes to package.md in https://github.com/dotnet/docs/pull/1825. I suspect it has issues, but it's hard for me to check, since I can't access https://review.docs.microsoft.com/en-us/dotnet/articles/core/packages?branch=pr-en-us-1825 (I found the URL in the publicly accessible CI log).
Another reason why this is annoying is that CI sends emails to the PR author, containing links to https://review.docs.microsoft.com/, effectively saying "hey, come check that your PR looks okay", which the author is unable to do, if they are not from MS.
Since I don't see any reason for this limitation (all the information in the PRs is public anyway), could https://review.docs.microsoft.com/ become public?

@mairaw @svick Thanks for you feedback. After discussed with the team, 1) we do have a future roadmap item to figure out a way to review a PR from a non-Microsoft user that doesn’t require an @microsoft.com address.  2) We’ll look at adding messaging to the PRs to warn people that the review links will not work for them at this time.

Wonder if there was any progress to this? Being able to verify changes would be great since we don't have ways to check the pages until the PR is merged and the changes are reflected to the website.


Stuck after failed firmware backup

Before opening a new issue, please read the following
Are you reporting a successful flash?
No
Do you have a question about configuring your newly flashed device?
No
Did you have issues getting a device to flash?*
Yes
Teckin SP22 USA model (Old Model, bought in June 2019 before switch to non-ESP82* chip).
Logs
smarthack-mqtt.log
pi@raspberrypi:~/tuya-convert/scripts $ cat smarthack-mqtt.log
1576865605: mosquitto version 1.5.7 starting
1576865605: Using default config.
1576865605: Opening ipv4 listen socket on port 1883.
1576865605: Opening ipv6 listen socket on port 1883.
1576865777: mosquitto version 1.5.7 starting
1576865777: Using default config.
1576865777: Opening ipv4 listen socket on port 1883.
1576865777: Opening ipv6 listen socket on port 1883.
^C1576866033: mosquitto version 1.5.7 terminating
1576866047: mosquitto version 1.5.7 starting
1576866047: Using default config.
1576866047: Opening ipv4 listen socket on port 1883.
1576866047: Opening ipv6 listen socket on port 1883.
1576866155: mosquitto version 1.5.7 starting
1576866155: Using default config.
1576866155: Opening ipv4 listen socket on port 1883.
1576866155: Opening ipv6 listen socket on port 1883.
1576866204: New connection from <IP_ADDRESS> on port 1883.
1576866204: New client connected from <IP_ADDRESS> as 35004736b4e62d4c5d1d (c1, k30, u'35004736b4e62d4c5d1d').
1576866204: Will message specified (107 bytes) (r0, q1).
1576866204:     tuya/smart/will
1576866204: Sending CONNACK to 35004736b4e62d4c5d1d (0, 0)
1576866204: Received SUBSCRIBE from 35004736b4e62d4c5d1d
1576866204:     smart/device/in/35004736b4e62d4c5d1d (QoS 0)
1576866204: 35004736b4e62d4c5d1d 0 smart/device/in/35004736b4e62d4c5d1d
1576866204: Sending SUBACK to 35004736b4e62d4c5d1d
1576866209: Received PUBLISH from 35004736b4e62d4c5d1d (d0, q0, r0, m0, 'smart/device/out/35004736b4e62d4c5d1d', ... (80 bytes))
1576866209: Received PUBLISH from 35004736b4e62d4c5d1d (d0, q0, r0, m0, 'smart/device/out/35004736b4e62d4c5d1d', ... (81 bytes))
1576866212: New connection from <IP_ADDRESS> on port 1883.
1576866212: New client connected from <IP_ADDRESS> as fb78bbf1-1d35-445d-8f5a-4a1f31f12150 (c1, k60).
1576866212: No will message specified.
1576866212: Sending CONNACK to fb78bbf1-1d35-445d-8f5a-4a1f31f12150 (0, 0)
1576866212: Received PUBLISH from fb78bbf1-1d35-445d-8f5a-4a1f31f12150 (d0, q0, r0, m0, 'smart/device/in/35004736b4e62d4c5d1d', ... (147 bytes))
1576866212: Sending PUBLISH to 35004736b4e62d4c5d1d (d0, q0, r0, m0, 'smart/device/in/35004736b4e62d4c5d1d', ... (147 bytes))
1576866212: Received DISCONNECT from fb78bbf1-1d35-445d-8f5a-4a1f31f12150
1576866212: Client fb78bbf1-1d35-445d-8f5a-4a1f31f12150 disconnected.
1576866254: Client 35004736b4e62d4c5d1d has exceeded timeout, disconnecting.
1576866254: Socket error on client 35004736b4e62d4c5d1d, disconnecting.
1576866456: mosquitto version 1.5.7 starting
1576866456: Using default config.
1576866456: Opening ipv4 listen socket on port 1883.
1576866456: Opening ipv6 listen socket on port 1883.
1576871173: mosquitto version 1.5.7 starting
1576871173: Using default config.
1576871173: Opening ipv4 listen socket on port 1883.
1576871173: Opening ipv6 listen socket on port 1883.
1576871940: mosquitto version 1.5.7 starting
1576871940: Using default config.
1576871940: Opening ipv4 listen socket on port 1883.
1576871940: Opening ipv6 listen socket on port 1883.
1576872138: mosquitto version 1.5.7 starting
1576872138: Using default config.
1576872138: Opening ipv4 listen socket on port 1883.
1576872138: Opening ipv6 listen socket on port 1883.

smarthack-psk.log
pi@raspberrypi:~/tuya-convert/scripts $ cat smarthack-psk.log
new client on port 443 from <IP_ADDRESS>:41791
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41815
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
^CTraceback (most recent call last):
  File "./psk-frontend.py", line 111, in <module>
    main()
  File "./psk-frontend.py", line 104, in main
    r,_,_ =  select.select(readables, [], [])
KeyboardInterrupt
new client on port 443 from <IP_ADDRESS>:41904
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41913
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41938
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41941
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41962
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:43061
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:43146
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))

smarthack-udp.log
pi@raspberrypi:~/tuya-convert/scripts $ cat smarthack-udp.log
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
^CListening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35004736b4e62d4c5d1d', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35004736b4e62d4c5d1d', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35004736b4e62d4c5d1d', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667

smarthack-wifi.log
pi@raspberrypi:~/tuya-convert/scripts $ cat smarthack-wifi.log
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED 68:c9:0b:eb:b6:f6
wlan0: AP-STA-DISCONNECTED 68:c9:0b:eb:b6:f6
wlan0: interface state ENABLED->DISABLED
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
wlan0: AP-DISABLED
wlan0: CTRL-EVENT-TERMINATING
nl80211: deinit ifname=wlan0 disabled_11b_rates=0
AP closed
Stopping DNSMASQ server...
Configuring AP interface...
RTNETLINK answers: File exists
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...

 I tried cracking one open and soldering to it to flash over FTDI but there was some flubbing and it didnt work right, not sure what went wrong.

Anyway I still have one good one and heard that the new Tuya Convert can handle them now. (these are not the new Teckins without an ESP chip).

I am using a new SD install of raspbian buster lite with a Raspberry Pi 3 B+

I followed the instructions in DigiBlur's new tuya convert video and I got to the point where it said "Successfuly found device on <IP_ADDRESS> and started a backup but it took a long time and i noticed it wasnt showing any transferred bytes so i CTRL-C'd out and rebooted. But now when I plug in my Teckin device I cant get it int othe flashing mode. It looks like the firmware has been modified as when i plug it in the LED blinks (somewhat unevenly, like it's trying to blink steadily but sometimes misses a beat), and eventually it stops blinking entirely!

I tried multiple times to get the ./start_flash to work again but it seems like it cant get the device (smartconfig retires infinitely).

I don't know when the logs are generated, or overwritten, but i attached the logs i have on hand. They may not be the logs with the attempt that failed to do firmware backup

the firmware backup is 0 bytes by the way

Is the device bricked?

[I 191220 18:23:30 web:2246] 206 GET /files/upgrade.bin (<IP_ADDRESS>) 910.55ms

It looks like it successfully flashed the intermediate firmware. It's most likely still running the intermediate firmware, since you said you can't put it into pairing mode. The intermediate firmware serves no function other than backing up and flashing a new image.

I would try restarting start_flash and then plugging the device back in. Open a new terminal and run the firmware_picker directly.

I was able to grab the only device that i cracked open. It seems to be one stock tuya. so I tried the procedure and here is an image of it stuck at the backup: https://imgur.com/a/bcpNqMx
So same exact deal. It can't transfer.
I logged in with another shell while its in this process and here are the logs from this one
smarthack-mqtt.log
pi@raspberrypi:~/tuya-convert-2/scripts $ cat smarthack-mqtt.log
1576865605: mosquitto version 1.5.7 starting
1576865605: Using default config.
1576865605: Opening ipv4 listen socket on port 1883.
1576865605: Opening ipv6 listen socket on port 1883.
1576865777: mosquitto version 1.5.7 starting
1576865777: Using default config.
1576865777: Opening ipv4 listen socket on port 1883.
1576865777: Opening ipv6 listen socket on port 1883.
^C1576866033: mosquitto version 1.5.7 terminating
1576866047: mosquitto version 1.5.7 starting
1576866047: Using default config.
1576866047: Opening ipv4 listen socket on port 1883.
1576866047: Opening ipv6 listen socket on port 1883.
1576866155: mosquitto version 1.5.7 starting
1576866155: Using default config.
1576866155: Opening ipv4 listen socket on port 1883.
1576866155: Opening ipv6 listen socket on port 1883.
1576866204: New connection from <IP_ADDRESS> on port 1883.
1576866204: New client connected from <IP_ADDRESS> as 35004736b4e62d4c5d1d (c1, k30, u'35004736b4e62d4c5d1d').
1576866204: Will message specified (107 bytes) (r0, q1).
1576866204:     tuya/smart/will
1576866204: Sending CONNACK to 35004736b4e62d4c5d1d (0, 0)
1576866204: Received SUBSCRIBE from 35004736b4e62d4c5d1d
1576866204:     smart/device/in/35004736b4e62d4c5d1d (QoS 0)
1576866204: 35004736b4e62d4c5d1d 0 smart/device/in/35004736b4e62d4c5d1d
1576866204: Sending SUBACK to 35004736b4e62d4c5d1d
1576866209: Received PUBLISH from 35004736b4e62d4c5d1d (d0, q0, r0, m0, 'smart/device/out/35004736b4e62d4c5d1d', ... (80 bytes))
1576866209: Received PUBLISH from 35004736b4e62d4c5d1d (d0, q0, r0, m0, 'smart/device/out/35004736b4e62d4c5d1d', ... (81 bytes))
1576866212: New connection from <IP_ADDRESS> on port 1883.
1576866212: New client connected from <IP_ADDRESS> as fb78bbf1-1d35-445d-8f5a-4a1f31f12150 (c1, k60).
1576866212: No will message specified.
1576866212: Sending CONNACK to fb78bbf1-1d35-445d-8f5a-4a1f31f12150 (0, 0)
1576866212: Received PUBLISH from fb78bbf1-1d35-445d-8f5a-4a1f31f12150 (d0, q0, r0, m0, 'smart/device/in/35004736b4e62d4c5d1d', ... (147 bytes))
1576866212: Sending PUBLISH to 35004736b4e62d4c5d1d (d0, q0, r0, m0, 'smart/device/in/35004736b4e62d4c5d1d', ... (147 bytes))
1576866212: Received DISCONNECT from fb78bbf1-1d35-445d-8f5a-4a1f31f12150
1576866212: Client fb78bbf1-1d35-445d-8f5a-4a1f31f12150 disconnected.
1576866254: Client 35004736b4e62d4c5d1d has exceeded timeout, disconnecting.
1576866254: Socket error on client 35004736b4e62d4c5d1d, disconnecting.
1576866456: mosquitto version 1.5.7 starting
1576866456: Using default config.
1576866456: Opening ipv4 listen socket on port 1883.
1576866456: Opening ipv6 listen socket on port 1883.
1576871173: mosquitto version 1.5.7 starting
1576871173: Using default config.
1576871173: Opening ipv4 listen socket on port 1883.
1576871173: Opening ipv6 listen socket on port 1883.
1576871940: mosquitto version 1.5.7 starting
1576871940: Using default config.
1576871940: Opening ipv4 listen socket on port 1883.
1576871940: Opening ipv6 listen socket on port 1883.
1576872138: mosquitto version 1.5.7 starting
1576872138: Using default config.
1576872138: Opening ipv4 listen socket on port 1883.
1576872138: Opening ipv6 listen socket on port 1883.
1576874586: mosquitto version 1.5.7 starting
1576874586: Using default config.
1576874586: Opening ipv4 listen socket on port 1883.
1576874586: Opening ipv6 listen socket on port 1883.
1576874810: New connection from <IP_ADDRESS> on port 1883.
1576874810: New client connected from <IP_ADDRESS> as 35808047b4e62d1d56f8 (c1, k30, u'35808047b4e62d1d56f8').
1576874810: Will message specified (107 bytes) (r0, q1).
1576874810:     tuya/smart/will
1576874810: Sending CONNACK to 35808047b4e62d1d56f8 (0, 0)
1576874810: Received SUBSCRIBE from 35808047b4e62d1d56f8
1576874810:     smart/device/in/35808047b4e62d1d56f8 (QoS 0)
1576874810: 35808047b4e62d1d56f8 0 smart/device/in/35808047b4e62d1d56f8
1576874810: Sending SUBACK to 35808047b4e62d1d56f8
1576874815: Received PUBLISH from 35808047b4e62d1d56f8 (d0, q0, r0, m0, 'smart/device/out/35808047b4e62d1d56f8', ... (80 bytes))
1576874815: Received PUBLISH from 35808047b4e62d1d56f8 (d0, q0, r0, m0, 'smart/device/out/35808047b4e62d1d56f8', ... (81 bytes))
1576874819: New connection from <IP_ADDRESS> on port 1883.
1576874819: New client connected from <IP_ADDRESS> as 8670b8a8-adc7-4239-b19a-7a37dfc8374c (c1, k60).
1576874819: No will message specified.
1576874819: Sending CONNACK to 8670b8a8-adc7-4239-b19a-7a37dfc8374c (0, 0)
1576874819: Received PUBLISH from 8670b8a8-adc7-4239-b19a-7a37dfc8374c (d0, q0, r0, m0, 'smart/device/in/35808047b4e62d1d56f8', ... (147 bytes))
1576874819: Sending PUBLISH to 35808047b4e62d1d56f8 (d0, q0, r0, m0, 'smart/device/in/35808047b4e62d1d56f8', ... (147 bytes))
1576874819: Received DISCONNECT from 8670b8a8-adc7-4239-b19a-7a37dfc8374c
1576874819: Client 8670b8a8-adc7-4239-b19a-7a37dfc8374c disconnected.
1576874861: Client 35808047b4e62d1d56f8 has exceeded timeout, disconnecting.
1576874861: Socket error on client 35808047b4e62d1d56f8, disconnecting.

smarthack-psk.log
pi@raspberrypi:~/tuya-convert-2/scripts $ cat smarthack-psk.log
new client on port 443 from <IP_ADDRESS>:41791
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41815
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
^CTraceback (most recent call last):
  File "./psk-frontend.py", line 111, in <module>
    main()
  File "./psk-frontend.py", line 104, in main
    r,_,_ =  select.select(readables, [], [])
KeyboardInterrupt
new client on port 443 from <IP_ADDRESS>:41904
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41913
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41938
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41941
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:41962
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:43061
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:43146
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))
new client on port 443 from <IP_ADDRESS>:43313
('could not establish sslpsk socket:', SSLError(1, u'[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:727)'))

smarthack-udp.log
pi@raspberrypi:~/tuya-convert-2/scripts $ cat smarthack-udp.log
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
^CListening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35004736b4e62d4c5d1d', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35004736b4e62d4c5d1d', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35004736b4e62d4c5d1d', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35808047b4e62d1d56f8', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}
<IP_ADDRESS> {'ip': '<IP_ADDRESS>', 'gwId': '35808047b4e62d1d56f8', 'active': 2, 'ability': 0, 'mode': 0, 'encrypt': True, 'productKey': 'IVgPyZR1c2OyLlyA', 'version': '3.1'}

smarthack-wifi.log
pi@raspberrypi:~/tuya-convert-2/scripts $ cat smarthack-wifi.log
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED 68:c9:0b:eb:b6:f6
wlan0: AP-STA-DISCONNECTED 68:c9:0b:eb:b6:f6
wlan0: interface state ENABLED->DISABLED
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
wlan0: AP-DISABLED
wlan0: CTRL-EVENT-TERMINATING
nl80211: deinit ifname=wlan0 disabled_11b_rates=0
AP closed
Stopping DNSMASQ server...
Configuring AP interface...
RTNETLINK answers: File exists
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-CONNECTED b4:e6:2d:4c:5d:1d
wlan0: AP-STA-DISCONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...
Configuration file: /dev/stdin
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:28:e7:eb and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: AP-STA-CONNECTED bc:98:df:6b:ba:d2
Attempting to stop wpa_supplicant
Configuring AP interface...
RTNETLINK answers: File exists
Starting DNSMASQ server...
Starting AP on wlan0...

I would try restarting start_flash and then plugging the device back in. Open a new terminal and run the firmware_picker directly.

Should i still connect to the vtrust-flash with my phone even though it's in the intermediate firmware stage?

Well that worked! I was able to flash ESPURNA on one device. Although I meant to hit flash tasmota....
Now i need to find out how to get it onto tamsota

Awesome! Glad it worked. You can flash Tasmota from Espurna.

Should i still connect to the vtrust-flash with my phone even though it's in the intermediate firmware stage?

For future readers, no need to reconnect phone if in intermediate.

So, maybe this was misleading. I actually haven't flashed the original device i posted about (the first one to get stuck). I only flashed the second device (the one i orignally cracked open). I am still attempting to get it to flash but it looks like it can't find the device.

False Alarm, after banging it around and plugging and unplugging a ton it i got the device to be discovered!!!
Now i have two perfectly usable, secure, private devices! Thank you Tuya-Convert maintainers!


Newly opened windows do not inherit style change

When opening a new window of an app specified to be "enhanced" with MFE, it does not automatically acquire the updated style (an Acrylic title bar background, for example); it retains the Windows default style.
Restarting MFE or just reloading the config file does update the style on any already-open windows.

On a less important note, when a window is not maximized it shows the Extend Frame Into Client Area effect only until it is maximized, which then redraws the original style (and the larger-scale translucency does not return if the window is restored to its original size).  Reloading the app or just reloading the config file is the only way to re-apply the effect.

latest release or latest commit?

<IP_ADDRESS>, which I grabbed from master maybe an hour before I posted this issue.

Thanks for the quick fix!


Added Log() to FloatTensor

Adding a binding to PySyft as well.

Hi @saru95, great!
Ooopsy, you forgot to implement the GPU kernel but you declared it. If the current PR gets merged, it will break the build. It's the same story than for Log1p in https://github.com/OpenMined/OpenMined/pull/183
Good work anyway!
Cheers

Let me give it a try. I'll let you know. :)

Thanks!
Feel free also to have a look at current other PRs as Diego pointed out there that in gpu tests you have to move data to the gpu for the actual gpu code to be called! Just saying ;)
Cheers

Hey @saru95
I've fixed the GPU tests file, it contained the wrong functionality. You can check how to implement GPU tests now!
Very sorry for the inconvenient


ZHA: EmberStatus.DELIVERY_FAILED while adding a Ikea lightbulb

Home Assistant release with the issue:

0.93.0
Operating environment (Hass.io/Docker/Windows/etc.):

Hass.io
Component/platform:

ZHA
Description of problem:
I'm getting an EmberStatus.DELIVERY_FAILED exception when I'm trying to add an Ikea lightbulb. The model I'm trying to add is: TRADFRI bulb E27 W opal 1000lm
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

Traceback (if applicable):
2019-05-16 23:22:28 WARNING (MainThread) [zigpy.device] [0x02fb] Requesting Node Descriptor failed: [0x02fb:0:0x0002]: message send failure: EmberStatus.DELIVERY_FAILED
2019-05-16 23:22:47 ERROR (MainThread) [zigpy.device] Failed ZDO request during device initialization: [0x02fb:0:0x0005]: message send failure: EmberStatus.DELIVERY_FAILED
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zigpy/device.py", line 76, in _initialize
    epr = await self.zdo.request(0x0005, self.nwk, tries=3, delay=2)
  File "/usr/local/lib/python3.7/site-packages/zigpy/util.py", line 53, in retry
    r = await func()
  File "/usr/local/lib/python3.7/site-packages/zigpy/device.py", line 129, in request
    expect_reply=expect_reply,
  File "/usr/local/lib/python3.7/site-packages/bellows/zigbee/application.py", line 355, in request
    res = await asyncio.wait_for(req.send, timeout=APS_ACK_TIMEOUT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 416, in wait_for
    return fut.result()
zigpy.exceptions.DeliveryError: [0x02fb:0:0x0005]: message send failure: EmberStatus.DELIVERY_FAILED

Additional information:

Try pairing it again.
What kind of radio are you using?
What kind of hardware are you running hassio on?

Thursday evening I've tried to pair it multiple times without any success. This morning I tried paring it again and now it's added to the ZigBee network. So I don't understand where the exception came from.
If someone else has trouble resetting an Ikea light bulb try this:
https://github.com/athombv/com.ikea.tradfri/issues/4#issuecomment-326731685


Sleep PC suddenly stop working (wont turn off TV) after updating windows 11

Hi bro!
Just found an issue after updating windows 11, sleep wont turn off TV anymore! :(
I have attached the log file for u here
I'm looking forward to your reply and thanks for your work!
William
Log.txt

Hey,
If you look in the log there is an error regarding an SSL handshake during the routine for powering off. I've seen it pop up over the last month or so but it is random. Not sure why the SSL handshake started failing sometimes. Most things point towards changing to webos23 for the TV.
Anyways, I'm working on it so stay tuned

Thanks alot bro!

just an interesting thing I found:
I installed EA app and in app, I tick run a service in background which called "EABackgroundService" showed in Taskmanager and this service if it runs, there is NO issue with sleep pc turns off TV anymore!
The moment I endtask this service, issue went back haha. Wierd right but I believe this service has something to do with SSL

For a temporarly solution, I set this service to automatically run instead of manual (default) so it starts when windows boot up and boom, no more issue but still looking for an official fix!

Hehe, that sounds coincidental!
Keep me updated on how it goes

Hey bro! Been testing it for an hour continously and no issue!

Hi @WilliamLeGod there is a dev build (v3.6.6) to address the SSL handshake issue on Discord if you want to try https://discord.gg/7KkTPrP3fq

sup bro. just tested and didnt work 😦
Uploading Log.txt…

Log.txt


Extension Manager exception handling

WebScriptHook should not crash entirely when one of its extensions throws an exception
This includes handling exceptions that might occur when

Loading extensions from assemblies through reflection
Ticking extensions
Extensions handling web calls

done


error while installing testrpc

Trying to install testrpc. Have installed node.js and the npm package.
Keep on getting this message:
npm ERR! Darwin 15.6.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g" "ethereumjs-testrpc"
npm ERR! node v6.11.0
npm ERR! npm  v3.10.10
npm ERR! path ../lib/node_modules/ethereumjs-testrpc/bin/testrpc
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall symlink
npm ERR! Error: EACCES: permission denied, symlink '../lib/node_modules/ethereumjs-testrpc/bin/testrpc' -> '/usr/local/bin/testrpc'
npm ERR!     at Error (native)
npm ERR!  { Error: EACCES: permission denied, symlink '../lib/node_modules/ethereumjs-testrpc/bin/testrpc' -> '/usr/local/bin/testrpc'
npm ERR!     at Error (native)
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'symlink',
npm ERR!   path: '../lib/node_modules/ethereumjs-testrpc/bin/testrpc',
npm ERR!   dest: '/usr/local/bin/testrpc' }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/Terry/npm-debug.log
npm ERR! code 1
Installing on Mac OS X El capitan
Version 10.11.6

did you read error message?

npm ERR! Please try running this command again as root/Administrator.

How do i run as the root/administrator?

would this solve it?
sudo npm install -g ethereumjs-testrpc

I am running as admin and get the same

Install works w/ node v6.11.0 Didn't work w/ v6.9.1 or v8.1.1

I am having the same problem

I am having the same error, did you fix it?

Closing due to inactivity/age


feat: stop using console2

Motivation
There is no reason to use console2 anymore.

Solution

Further clarification:
console2 was only created because the Hardhat console's (console) incorrect ABI encoding made it so that the calls would not show up / be decoded in traces, as noted in the console2 docs.
Both consoles have the same effect now in Foundry after https://github.com/foundry-rs/foundry/pull/6504, so only one console will exist in a future version.


Bugtool: Add gops output

Add gops output for cilium agent to know what happens to the agent in
case of something wrong.

Fixes #3608
Signed-off-by: Eloy Coto<EMAIL_ADDRESS>

test-me-please


<EMAIL_ADDRESS>decompress error

PS C:\Users\dubey> scoop install twilio
WARN  Purging previous failed installation of twilio.
ERROR 'twilio' isn't installed correctly.
Removing older version (5.2.3).
'twilio' was uninstalled.
Installing 'twilio' (5.2.3) [64bit] from twilio-scoop bucket
Loading twilio-win32-x64.tar.xz from cache
Checking hash of twilio-win32-x64.tar.xz ... ok.
Extracting twilio-win32-x64.tar.xz ... ERROR Exit code was 2!
Failed to extract files from C:\Users\dubey\scoop\apps\twilio\5.2.3\twilio-win32-x64.tar.
Log file:
~\scoop\apps\twilio\5.2.3\7zip.log
Please try again or create a new issue by using the following link and paste your console output:
https://github.com/twilio/scoop-twilio-cli/issues/new?title=twilio%405.2.3%3A+decompress+error
PS C:\Users\dubey>

This looks like a duplicate issue.
Please try running this cmd in Powershell ran as an Admin.


How to use?

How do you even use this? there is no instructions on the page, how are we supposed to know how to load this?

Also you can look for examples, like, in fmodel source code or anywhere else. Just use github search.

Also you can look for examples, like, in fmodel source code or anywhere else. Just use github search.

Uh... Learn C#, that's all. You need to code to use this library.

What do you expect? This is a library, not an app...


Added a development tool to create a filter mask.

Usage:
./tools/create_mask.py --input img1.png img2.png --output mask.png
TODO:
Support other filters in addition to MM_DARK.

Cool!


Improve compatibility with ZoomText and Widows Magnifier magnification functionality

Improve compatibility with ZoomText and Widows Magnifier magnification functionality
it was reported that there is some distortion when using these when using crl +

We can't test with zoomText as it is licensed product costing. But Windows Magnifier is on all Windows machines. Here is a example of distortion mentioned in bug:

@xsrust and I went back to the UOE page and used the Windows Magnifier again and now we noticed similar distortion of text in Window's Edge

https://www.realpointdesign.co.uk/1572/choosing-fonts-for-your-website-for-accessibility/
The above article recommends Verdana for Accessibility:
Our standard recommendations for on-screen typefaces are:
    Verdana (when on screen legibility is of prime importance)
    Trebuchet (when a slightly more interesting style is looked for)
    Arial / Helvetica (the stalwarts of the typeface world)

@xsrust @briri @magdalenadrafiova @sjDCC @dsisu
The question is do we set the default font-family to say
font-family: Verdana, Trebuchet, "Helvetica Neue", Helvetica, Arial, sans-serif;
Feedback requested. I will add some screenshots with this this afternoon.
Currently default font-family  style used by Roadmap comes from Bootstrap:
In the style file  file https://github.com/twbs/bootstrap/blob/v3.4.1/dist/css/bootstrap.css the font-family is set
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
See line 1084 onwards
body {
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-size: 14px;
 ...
}

See line 6068 onwards
.tooltip {
  position: absolute;
  z-index: 1070;
  display: block;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-style: normal;
  font-weight: 400;
...
}

See line 6068 onwards:
.popover {
  position: absolute;
  top: 0;
  left: 0;
  z-index: 1060;
  display: none;
  max-width: 276px;
  padding: 1px;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-style: normal;
  font-weight: 400;
...
}

I'd be happy to change that font to the more accessible one for the Roadmap project.
I'm not entirely sure we need to though, as Helvetica was listed on the accessible fonts from the article linked.

New text scales well for me. Looks good. Closing out

By the way @johnpinto1 I really appreciate all the screengrabs you put in. Helps to know what we're testing :-)


Add password support

Uses this Xtractr library update and adds support for pulling passwords out of paths. The passwords must exist In the item path and be surrounded by {{ and }}. Closes #101.
Docker tag: golift/unpackerr:password(link)

Word great, thank you!


Weather Underground GET request needs a timeout

I think this is where the WeatherStation gets stuck sometimes. It just keeps waiting forever. I guess it would be a good idea to add a timeout here.
https://github.com/squix78/esp8266-weather-station/blob/master/WundergroundClient.cpp
// This will send the request to the server
 client.print(String("GET ") + url + " HTTP/1.1\r\n" +
              "Host: api.wunderground.com\r\n" +
              "Connection: close\r\n\r\n");
 while(!client.available()) {
   delay(1000);
 }

I think it schould be done at all internet requests, not only while wunderground update.

Ander L add a soluce for that , and it work for me
Ander L say :
Hi,
@Wolfgang : could try something like this:
// This will send the request to the server
client.print(String("GET ") + url + " HTTP/1.1\r\n" +
"Host: api.wunderground.com\r\n" +
"Connection: close\r\n\r\n");
int cntr=0;
while(!client.available()&& cntr < 3) {
cntr++;// Make 3 tries before quiting
delay(1000);
}
I've used something similar in other code for making sure the wifi is connected.

Cool, thanks. Just deployed this on my WeatherBug, will report back how it goes. It generally seems to fail at night for me, works fine all day.

@AdySan have you solved the problem by adding the timeout?
I'm using the stuff concerning the Weather Station found here on GitHub. I'm improving the project by means of adding a MQTT communication to the EasyIoT cloud server in order to remotely control a Thermostat.
The MQTT part works fine, but sometimes (expecially during night) I'm having troubles with the OLED, which freezes and stops working (during the update to wunderground) until a RST.
It seems t I can't understand where's the problem.
I've done some checks with WL_CONNECTED before requesting to wunderground, I've even added a timeout as suggested but it still freezes.
For a while I thought to add a softWDT to avoid this problem, in fact as I reset the node the system restart with any problems. Any idea?
WifiConnection code in loop:
`
if (WiFi.status() == WL_CONNECTED) {
if (readyForThermostatUpdate && ui.getUiState().frameState == FIXED) {
  updateThermostat();
}

if (readyForWeatherUpdate && ui.getUiState().frameState == FIXED) {
  updateData(&display);
}

}
else{
Serial.println("Disconnected.");
WiFiConnect();
}
void WiFiConnect() {
Serial.println("***************");
Serial.print("Connecting to ");
Serial.println(AP_SSID);
WiFi.begin(AP_SSID, AP_PASSWORD);
int counter = 0;
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
display.clear();
display.drawString(64, 10, "Connecting to WiFi");
display.drawXbm(46, 30, 8, 8, counter % 3 == 0 ? activeSymbole : inactiveSymbole);
display.drawXbm(60, 30, 8, 8, counter % 3 == 1 ? activeSymbole : inactiveSymbole);
display.drawXbm(74, 30, 8, 8, counter % 3 == 2 ? activeSymbole : inactiveSymbole);
display.display();
counter++;

}
Serial.println("");
Serial.println("WiFi connected");
Serial.print("IP address: ");
Serial.println(WiFi.localIP());
}`
Timeout code used:
`// This will send the request to the server
client.print(String("GET ") + url + " HTTP/1.1\r\n" +
"Host: api.wunderground.com\r\n" +
"Connection: close\r\n\r\n");
int cntr=0;
while(!client.available()&& cntr < 3) {
cntr++;// Make 3 tries before quiting
delay(1000);
}

@stex2005 Thanks for the pointer. Unfortunately adding the timeout hasn't helped as you observed too. I never really considered that the OLED could be freezing. I don't thing it's a problem with the OLED library, since I have other projects using it which are running flawlessly. Perhaps something to do with the UI part of the library.

This is an ongoing problem that I experience regularly and is also reported in https://github.com/squix78/esp8266-weather-station/issues/36 . @stex2005 reports and @AdySan  reports that adding a timeout doesn't fix it, presumably because the call to [client.available()] (https://github.com/kentaylor/esp8266-weather-station/blob/master/ThingspeakClient.cpp#L42) is not returning.
I don't know the solution but can provide some data. I'm confident it doesn't occur if the internet connection is good and the server to which it connects is working correctly. The evidence is that I have one device on a flaky network at https://thingspeak.com/channels/126118 and it fails much more often than one on a good network at home. The device on the flaky network sometimes fails on the Updating time... screen which I haven't seen on the good network. Thingspeak is flaky and fails in different ways. Most of the time this shows up as a gap in the data for both device installations. It can also show up in both devices failing at the Updating thingspeak... screen at the same time. I've recorded the connection times of one simultaneous freeze failure on both devices. The simultaneous identical failure on different networks must be associated with an incorrect response from the Thingspeak server that the ESP8266 can not handle.
Interestingly Thingspeak is called in 2 places in the code, here and here . One is called 5 times more often than the other but the failure always occurs in the code called least often. The difference is that one call waits for a response and continues to call client.available() while it waits while the other calls client.available() once and moves on. So it is something to do with waiting for a response when the response has errors and getting stuck in the client.available() function. The client.available() function maps to a call to the Espressif library at https://github.com/esp8266/Arduino/blob/633e48f3aec5f1c3c11d4498fc90d378d49e6e9f/libraries/ESP8266WiFi/src/include/ClientContext.h#L208 .

Still getting stuck and able to observe that it was stuck [here] (https://github.com/kentaylor/esp8266-weather-station/blob/77891b2892f8cc2d72c3ffd1d6bf90be56fee015/ThingspeakClient.cpp#L32) on one occasion.
Currently the weather station library uses the WiFiClient library for accessing web services. This requires that the http processing logic and request format be constructed in code. There is also a higher level HTTPClient library that could be used for these interactions.

Hi Ken. What is the situation for you on this issue? Did your fix solve it? I am running now my weather station for several weeks without need for manual restart...

I like the idea of using the HTTPClient library. This accesses web services at a higher level of abstraction than the WiFiClient library so for that reason alone I'd recommend the changes in the commit at https://github.com/kentaylor/esp8266-weather-station/commit/f4bdc9630a95ab6425d895cd8a26ebec4fb66f3a.
Since having made that change I haven't seen a failure but I'm not sure if the HTTPClient library has fixed the problem. I previously had a device on a flaky network so I saw the problem often (every few days). That network is now fixed so I can't be sure if the code change helped. I have only seen the problem rarely (less than once per month) on a good network. I've also seen it when Thingspeak gets overloaded and Thingspeak responses become flaky but that was only once. If that happens, every weather station everywhere will fail at roughly the same time and have to be rebooted.

The HTTPClient library has not fixed the problem. I now have a few devices on a Telco WiFi dongle and some have experienced the hanging problem again. I expect that is because TELCO networks are flaky at times.
This means that the problem is not in the Weather Station code and in particular is not related to the way the weather station library uses the WiFiClient library for accessing web services. The problem is further down the stack somewhere within the HTTP or TCP/IP handling.
I've seen it in other applications that use HTTP as well but never in applications that use UDP. If the device can not establish a HTTP connection it will be OK but if it establishes the connection it becomes susceptible to hanging when connectivity is lost part way through the transmission. It will continue to wait for data that the server will never send, even after network connectivity is reestablished. With UDP it doesn't wait which is probably why UDP doesn't cause hanging.