Connection Pooling

Poolboy
Poolboy works by providing exclusive access to a given resource in a "transaction".
Pros:

Dynamic connection allocation is taken care of for you.

Cons:

Connection affinity is required for cursors, which means a transaction may be required for the full length of the cursor. For long running change feeds, this will require one connection per changefeed.
Pipelining is not utilized as each connection is checked out until a response is provided.

While my assumption was to use Poolboy, it seems that there are better approaches available. There are too many tradeoffs to be made for not a whole lot of benefit.
Hand rolled
An ideal connection pool will do the following:

Connect to N clients in a cluster.
Load balance queries between connections.
Route query to master replica for the target table.
If a connection dies, remove that connection from the rotation.
If a query fails because of a server connectivity issue, retry query on another host.

Initial design:
A supervisor tree will be over N connections and a coordinator.
Client processes send queries directly to the connection, just like we currently do. In order to know which connection to contact, the client can request a connection from the coordinator. When the client is finished with the query it informs the coordinator (in order for the coordinator to properly balance connections).
This would require zero changes to Exrethinkdb.Connection. The connection would be dumb to pooling. We could provide a Exrethinkdb.ConnectionPool module with run/next/close that wrap the logic for interacting with the coordinator. The end API for the user would be the same. We will also provide a use Exrethinkdb.ConnectionPool macro. Ideally you would replace use Exrethinkdb.Connection with use Exrethinkdb.ConnectionPool and your application would still work perfectly.
Routing queries to proper replicas can be done fairly well in the coordinator by looking at table("foo_table") |> config |> run. The coordinator can do this on a timer (once every minute is probably sufficient). It can then use this information to route the connection properly. In the event of stale_reads, the coordinator can load balance between non-master connections. We'll have to add table to the Query struct, but that's pretty straightforward.
Exrethinkdb.ConnectionPool.run will report back to the coordinator the success or failure of a query.  The coordinator will also be monitoring the Connection processes. There will also be logic to retry on a different server if a failure occurs (in the event of stale_reads. otherwise, retrying won't be useful until a new master is selected).
I think this custom approach will be fairly simple. It will be designed so that failure leads to a worst case scenario of routing requests to the less optimal replica.

Their is an issue open for Ecto that solves this by "reserving" a worker
https://github.com/elixir-lang/ecto/issues/569 I'd be pretty strongly
against hand rolling because this kind of thing is full of pitfalls. We
could also ping @fishcakez and see if maybe his sbroker might be a stronger
use case here.
On Wed, May 6, 2015 at 11:15 PM, Peter Hamilton<EMAIL_ADDRESS>wrote:

Poolboy
Poolboy works by providing exclusive access to a given resource in a
"transaction".
Pros:

Dynamic connection allocation is taken care of for you.

Cons:

Connection affinity is required for cursors, which means a
transaction may be required for the full length of the cursor. For long
running change feeds, this will require one connection per changefeed.
Pipelining is not utilized as each connection is checked out until a
response is provided.

While my assumption was to use Poolboy, it seems that there are better
approaches available. There are too many tradeoffs to be made for not a
whole lot of benefit.
Hand rolled
An ideal connection pool will do the following:

Connect to N clients in a cluster.
Load balance queries between connections.
Route query to master replica for the target table.
If a connection dies, remove that connection from the rotation.
If a query fails because of a server connectivity issue, retry
query on another host.

Initial design:
A supervisor tree will be over N connections and a coordinator.
Client processes send queries directly to the connection, just like we
currently do. In order to know which connection to contact, the client can
request a connection from the coordinator. When the client is finished with
the query it informs the coordinator (in order for the coordinator to
properly balance connections).
This would require zero changes to Exrethinkdb.Connection. The connection
would be dumb to pooling. We could provide a Exrethinkdb.ConnectionPool
module with run/next/close that wrap the logic for interacting with the
coordinator. The end API for the user would be the same. We will also
provide a use Exrethinkdb.ConnectionPool macro. Ideally you would replace use
Exrethinkdb.Connection with use Exrethinkdb.ConnectionPool and your
application would still work perfectly.
Routing queries to proper replicas can be done fairly well in the
coordinator by looking at table("foo_table") |> config |> run. The
coordinator can do this on a timer (once every minute is probably
sufficient). It can then use this information to route the connection
properly. In the event of stale_reads, the coordinator can load balance
between non-master connections. We'll have to add table to the Query
struct, but that's pretty straightforward.
Exrethinkdb.ConnectionPool.run will report back to the coordinator the
success or failure of a query. The coordinator will also be monitoring the
Connection processes. There will also be logic to retry on a different
server if a failure occurs (in the event of stale_reads. otherwise,
retrying won't be useful until a new master is selected).
I think this custom approach will be fairly simple. It will be designed so
that failure leads to a worst case scenario of routing requests to the less
optimal replica.
--
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32.

I think table affinity is not going to be provided by existing solutions.
Table affinity will cut down on network bottlenecks on the server side.
The ecto solution solves a very different problem of nested transactions.
The solution they use (correct me if I'm wrong) is to permanently checkout
a connection for a long lived process to use. The alternatives proposed are
to provide the worker in the gen_call.
We are dealing with a very different set of constraints. The unit of
exclusivity for a postgres connection is a full query/response or even a
full transaction. For rethinkdb it's a query or a response. Since we can
interleave queries and responses we only need exclusive access in very
small chunks and the current implementation does it very well.
The performance bottleneck is purely server side. We will not have nor want
multiple connections to the same host.
The hand rolled proposal is built on top of independently useful building
blocks. The current connections are already done and work well. The
coordinator is going to be necessary for table affinity in any solution we
come across.
The ideal drop in solution would be something that:

keep track of established connections.
reconnect broken connections.

Number 2 is exactly what a supervisor is for. This is already an out of the
box solution. The only problem we need to solve is how a client find out
which connections currently are alive. I'm fairly certain which_children
will go a long way in solving that.
I'm going to take a very lightweight pass at this and see how difficult it
actually is.
On Thu, May 7, 2015, 8:26 AM Jason S<EMAIL_ADDRESS>wrote:

Their is an issue open for Ecto that solves this by "reserving" a worker
https://github.com/elixir-lang/ecto/issues/569 I'd be pretty strongly
against hand rolling because this kind of thing is full of pitfalls. We
could also ping @fishcakez and see if maybe his sbroker might be a stronger
use case here.
On Wed, May 6, 2015 at 11:15 PM, Peter Hamilton<EMAIL_ADDRESS>wrote:

Poolboy
Poolboy works by providing exclusive access to a given resource in a
"transaction".
Pros:

Dynamic connection allocation is taken care of for you.

Cons:

Connection affinity is required for cursors, which means a
transaction may be required for the full length of the cursor. For long
running change feeds, this will require one connection per changefeed.
Pipelining is not utilized as each connection is checked out until a
response is provided.

While my assumption was to use Poolboy, it seems that there are better
approaches available. There are too many tradeoffs to be made for not a
whole lot of benefit.
Hand rolled
An ideal connection pool will do the following:

Connect to N clients in a cluster.
Load balance queries between connections.
Route query to master replica for the target table.
If a connection dies, remove that connection from the rotation.
If a query fails because of a server connectivity issue, retry
query on another host.

Initial design:
A supervisor tree will be over N connections and a coordinator.
Client processes send queries directly to the connection, just like we
currently do. In order to know which connection to contact, the client
can
request a connection from the coordinator. When the client is finished
with
the query it informs the coordinator (in order for the coordinator to
properly balance connections).
This would require zero changes to Exrethinkdb.Connection. The connection
would be dumb to pooling. We could provide a Exrethinkdb.ConnectionPool
module with run/next/close that wrap the logic for interacting with the
coordinator. The end API for the user would be the same. We will also
provide a use Exrethinkdb.ConnectionPool macro. Ideally you would
replace use
Exrethinkdb.Connection with use Exrethinkdb.ConnectionPool and your
application would still work perfectly.
Routing queries to proper replicas can be done fairly well in the
coordinator by looking at table("foo_table") |> config |> run. The
coordinator can do this on a timer (once every minute is probably
sufficient). It can then use this information to route the connection
properly. In the event of stale_reads, the coordinator can load balance
between non-master connections. We'll have to add table to the Query
struct, but that's pretty straightforward.
Exrethinkdb.ConnectionPool.run will report back to the coordinator the
success or failure of a query. The coordinator will also be monitoring
the
Connection processes. There will also be logic to retry on a different
server if a failure occurs (in the event of stale_reads. otherwise,
retrying won't be useful until a new master is selected).
I think this custom approach will be fairly simple. It will be designed
so
that failure leads to a worst case scenario of routing requests to the
less
optimal replica.
--
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32.

—
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32#issuecomment-99909472.

@hamiltop after talking with @ericmj and @fishcakez they've convinced me that maybe we don't need connection pooling and that I am probably wrong.

Table affinity is what's got me convinced I want to do some level of
connection management. I hacked together a lightweight naive pool and the
code is dead simple.
I skimmed the conversation in irc and I think your previous comment here
about this being a smarter client then something like postgres is accurate
and changes the discussion a bit.
I'll hopefully have my code up soon.
On Thu, May 7, 2015, 9:07 AM Jason S<EMAIL_ADDRESS>wrote:

@hamiltop https://github.com/hamiltop after talking with @ericmj
https://github.com/ericmj and @fishcakez https://github.com/fishcakez
they've convinced me that maybe we don't need connection pooling and that I
am probably wrong.
—
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32#issuecomment-99923747.

https://github.com/hamiltop/exrethinkdb/blob/connection_pool/lib/exrethinkdb/connection/pool.ex
used like:
hosts = [[host: "localhost", port: 28015], [host: "localhost", port: 28015]]
{:ok, conn_pool} = Exrethinkdb.Connection.Pool.start_link([hosts: hosts])
conn = Exrethinkdb.Connection.Pool.get_connection(conn_pool)
Exrethinkdb.Query.table("people") |> Exrethinkdb.Connection.run conn
# alternatively, run from the conn_pool and it will pick a connection for you
Exrethinkdb.Query.table("people") |> Exrethinkdb.Connection.Pool.run conn_pool

It's naive, but accomplishes the basic goals. The optimizations would be in how get_connection works in order to provide table affinity.
As an already-implemented, cursors store the pid of their connection internally and will use that connection when next is called.

@hamiltop,  I really like the ideas that you've laid out for the hand rolled solution because it takes into account the unique characteristics of RethinkDB.  Poolboy, and other technologies are going to make assumptions that are part of a different paradigm.  I also like the fact that you were able to build a working version based on the code that's already been written.  As @jeregrine mentioned, there are bound to be Gremlins lurking in the dark, but that seem to be the case for just about everything in this line of work.  At this point, I can't think of a good reason not to run with your solution.

Some interesting results:
Three configurations:

Single local connection (single)
A Pool of 4 connections to localhost (pool)
A pool of 4 connections to localhost plus 1 connection to a remote host (cluster). The remote host has significant network latency (this is going from my laptop to my iMac over WiFi).

All of these were run with use_outdated: true. Without allowing outdated reads, the cluster mode has no real benefit if you aren't sharding your data. If you shard your data, the results are similar to use_outdated: true in that the clustering has benefit.
The first test was a non-indexed read in a table of 10k records (looking for one record), repeated 40 times across 40 Tasks for a total of 1600 queries and 1.6M reads (non-indexed reads count one read per record). This was very much CPU bound in rethinkdb.
Real results now
{:pool, {24016272, :ok}}
{:single, {23215845, :ok}}
{:cluster, {17805260, :ok}}
Again, reversed
{:cluster, {18998542, :ok}}
{:single, {23591027, :ok}}
{:pool, {23255329, :ok}}

The next test was for an indexed get in the same table, but repeated 4000 times across 40 Tasks for a total of 160000 queries and 160000 reads (one read per get). This was CPU bound on the elixir side.
Real results now
{:pool, {19722294, :ok}}
{:single, {20719733, :ok}}
{:cluster, {35041397, :ok}}
Again, reversed
{:cluster, {30579648, :ok}}
{:single, {20782334, :ok}}
{:pool, {20139856, :ok}}

Conclusions
For heavy non-indexed reads (or anything CPU bound in the DB) clustering is a benefit. For heavy indexed reads (or anything light on CPU in the DB) clustering slows things down (or at best doesn't really help much).
Additionally, fine tuning the pool is important. When I ran the cluster test with 1 local and 1 remote connection it was much slower than when I ran it with 4 local and 1 remote. This was in a very lopsided network, but I've encountered lopsided clusters aplenty in production environments.
I think the takeaway is that a developer will need to weight hosts in the pool according to their requirements. The configuration should be something like:
[
  [host: "db1", port: 28015, weight: 2],
  [host: "db2", port: 28015, weight: 3],
  [host: "db3", port: 28015, weight: 1]
]

Additionally, table affinity is definitely required. Hitting the connection that isn't the master for the table was costly.

The results are exactly what I would expect them to be given the parameters you described.  In my mind this seems to indicate that initial design is working properly.  This will add the flexibility needed to get the most out of RethinkDB and the application.  A couple of things that I was thinking about were pools that had equally weighted hosts.  For example, if you have a RethinkDB cluster in data-center A, and another RethinkDB Cluster in data-center B, it seems logical to have the following configuration:
[
[host: "dca_db1", port: 28015, weight: 0],
[host: "dca_db2", port: 28015, weight: 0],
[host: "dca_db3", port: 28015, weight: 0]
[host: "dcb_db1", port: 28015, weight: 1],
[host: "dcb_db2", port: 28015, weight: 1],
[host: "dcb_db3", port: 28015, weight: 1]
]
This would allow the application servers in data-center A to utilize the local network for best performance, but fall back to the cluster in data-center B.  Given that they are equally weighted in each group, I would expect that the load would be balanced among them.
I have been kicking around the idea of adding tags that could be used to specify things like racks, data-centers, etc, but the weight parameter may cover most cases.  At this point I'm not sure if they would help or hurt, but I thought the idea was at least worth mentioning.
--Nick

weight: 0 would need to be a special case, as otherwise weights are
traditionally applied as P(use this host) = (weight of host)/sum(weights)
which in the case of weight 0 would be 0% probability. We could probably
special case it so that is sum(weights) is 0 the we just round robin.
I did find a bug in some code not pushed yet. :random.uniform has a state
local to each process. The current code (my new code has not been pushed
yet) resulted in all processes using the same round robin ordering of
connections. The tests were very useful in exposing that.
On Fri, May 8, 2015 at 1:29 PM Nick<EMAIL_ADDRESS>wrote:

The results are exactly what I would expect them to be given the
parameters you described. In my mind this seems to indicate that initial
design is working properly. This will add the flexibility needed to get the
most out of RethinkDB and the application. A couple of things that I was
thinking about were pools that had equally weighted hosts. For example, if
you have a RethinkDB cluster in data-center A, and another RethinkDB
Cluster in data-center B, it seems logical to have the following
configuration:
[
[host: "dca_db1", port: 28015, weight: 0],
[host: "dca_db2", port: 28015, weight: 0],
[host: "dca_db3", port: 28015, weight: 0]
[host: "dcb_db1", port: 28015, weight: 1],
[host: "dcb_db2", port: 28015, weight: 1],
[host: "dcb_db3", port: 28015, weight: 1]
]
This would allow the application servers in data-center A to utilize the
local network for best performance, but fall back to the cluster in
data-center B. Given that they are equally weighted in each group, I would
expect that the load would be balanced among them.
I have been kicking around the idea of adding tags that could be used to
specify things like racks, data-centers, etc, but the weight parameter may
cover most cases. At this point I'm not sure if they would help or hurt,
but I thought the idea was at least worth mentioning.
--Nick
—
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32#issuecomment-100353032
.

You have a couple of issues regarding failure handling with the proposed pooling method:
The case where all connections are unavailable isn't handled. This could happen if all connections are :restarting, in which case a client will crash when generating the random integer. Perhaps that is desired.
 In the current implementation connect and handshake occur in the init/1 callback. This means that the supervisor/the pool is blocked and all client processes requesting a worker are blocked too while a single connection process starts.
If one of the rethinkdb servers is down or there is a issue with connecting/sending data, a connection process will fail to connect. This will either result in the supervisor blocking for a significant time while a connection process tries to start (as above) or more likely the supervisor will shutdown because the max_restarts limit was reached because a connection process keeps failing to start in rapid succession.
Finally the poolling strategy does not provide any backpressure or load shedding to client processes beyond the connection processes' message queues. Designing around a minimal number of connection processes that pipeline requests makes this is a potential bottleneck.

Thanks for chiming in @fishcakez
The case for "unable to acquire a connection" has not been addressed at any level yet, but either a "let it crash" philosophy or responding with {:error, msg} should suffice for both the individual connection case and the pool case.
To check my understanding, the init function of a GenServer is run as part of GenServer.start_link and therefore it causes the supervisor to block. Is it a common pattern to send oneself a :connect message to be processed immediately after init? How is this otherwise handled?
How is the max_restarts issue (where one buggy child kills all the other healthy children as it brings down the supervisor) solved elsewhere? By not using supervision?
Backpressure is applied to individual clients through GenServer.call. The client process will block until it receives a response. This does nothing to address an unbounded number of client processes. It also does not deal with load shedding, as stated. Would a simple solution be for an individual connection to keep track of active clients and respond :too_busy? The client should then try another connection or in the case of not using a connection pool just fail. Change feeds would need to be accounted for differently because they don't have the same resource footprint as active connections.
While I'm exploring the idea of a custom pool with table affinity, I'm committed to making Connection compatible with existing connection pooling solutions like poolboy. Not using pipelining isn't the end of the world, and this is how it is done in other languages. This custom pooling is an optimization. Perhaps there's a hybrid solution that can be found.

" We could probably special case it so that is sum(weights) is 0 the we just round robin."
-- That is the behavior that I had in mind with my sample configuration.  Would it be possible to round robin any weights that are the same?  In my example config above, the idea was to round robin the primary cluster, then fail over to another cluster and round robing those connections.  If that's not possible, I suppose the config could look like this, as long as 0 was selected before the higher number weights:
[
[host: "dca_db1", port: 28015, weight: 0],
[host: "dca_db2", port: 28015, weight: 0],
[host: "dca_db3", port: 28015, weight: 0]
[host: "dcb_db1", port: 28015, weight: 1],
[host: "dcb_db2", port: 28015, weight: 2],
[host: "dcb_db3", port: 28015, weight: 3]
]
"This custom pooling is an optimization. Perhaps there's a hybrid solution that can be found."
-- Darn those gremlins :)

The only two extension points I'd like are:

Oversubscribing checkouts and connections so we can take advantage of
pipelining.
Custom pooling strategies (so table affinity can be done).

It may be worth seeing if an existing solution can be adapted. It may also
be useful to dive down the rabbit hole and see how riak does client side
routing.
On Fri, May 8, 2015 at 4:10 PM Nick<EMAIL_ADDRESS>wrote:

" We could probably special case it so that is sum(weights) is 0 the we
just round robin."
-- That is the behavior that I had in mind with my sample configuration.
Would it be possible to round robin any weights that are the same? In my
example config above, the idea was to round robin the primary cluster, then
fail over to another cluster and round robing those connections. If that's
not possible, I suppose the config could look like this, as long as 0 was
selected before the higher number weights:
[
[host: "dca_db1", port: 28015, weight: 0],
[host: "dca_db2", port: 28015, weight: 0],
[host: "dca_db3", port: 28015, weight: 0]
[host: "dcb_db1", port: 28015, weight: 1],
[host: "dcb_db2", port: 28015, weight: 2],
[host: "dcb_db3", port: 28015, weight: 3]
]
"This custom pooling is an optimization. Perhaps there's a hybrid solution
that can be found."
-- Darn those gremlins :)
—
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32#issuecomment-100391625
.

https://github.com/inaka/worker_pool/compare/master...marianoguerra:issue_16_custom_strategies seems to be interesting. I'd rather use Poolboy just for experience and expertise within the Elixir ecosystem.
#1 could probably be accomplished via some hack layer on top of a custom strategy. Something like "when creating a worker, wrap an existing connection if X and Y else start a new connection".

Not using pipelining isn't the end of the world, and this is how it is done in other languages.

You can still pipeline without using pools, either by making the pipelining explicit or by having a Task like API where you can wait for multiple pipelined queries to complete.

Tasks are something I'd like to add anyway.
I don't think I'll be intellectually satisfied without oversubscription,
but we can get to practical and useful without it.
As far as table affinity, anything wrong with N pools, one per server? Then
we can bolt a layer on top pointing the client to the correct pool for a
given query (with fallback to any pool if needed).
On Sat, May 9, 2015, 4:30 AM Eric Meadows-Jönsson<EMAIL_ADDRESS>wrote:

Not using pipelining isn't the end of the world, and this is how it is
done in other languages.
You can still pipeline without using pools, either by making the
pipelining explicit or by having a Task like API where you can wait for
multiple pipelined queries to complete.
—
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/exrethinkdb/issues/32#issuecomment-100467326
.

Re: Tasks
https://github.com/hamiltop/exrethinkdb/issues/33

To check my understanding, the init function of a GenServer is run as part of GenServer.start_link and therefore it causes the supervisor to block. Is it a common pattern to send oneself a :connect message to be processed immediately after init? How is this otherwise handled?

Exactly how you describe, casting a reconnect message to self in init/1 is very common.

How is the max_restarts issue (where one buggy child kills all the other healthy children as it brings down the supervisor) solved elsewhere? By not using supervision?

The fault tolerance in OTP comes about by isolating errors at suitable levels. A supervision tree does this in layers, errors slowly bubble up the supervision tree as processes keep crashing. Hopefully at some point as close to the bottom of the tree as possible the state is fixed and normal service resumes.
Not being able to connect to the database (network partition) is a realistic error and it is reasonable to handle it. Ideally this error is contained so that connections to other databases are not effected. If the other connections are shutdown because the max restart limit is reached the fault has spread further than is necessary.
A circuit breaker or fuse type system (such as https://github.com/jlouis/fuse) can be used to raise an alarm (http://www.erlang.org/doc/man/alarm_handler.html) and throttle retries, possibly backing off between attempts (such as https://github.com/ferd/backoff).

Would a simple solution be for an individual connection to keep track of active clients and respond :too_busy?

Yes.

I'm committed to making Connection compatible with existing connection pooling solutions like poolboy.

You could try a slightly different take on the usual semantics if the max_overflow is set to 0. The poolboy transaction need only be to setup the asynchronous request (or task as in #33). It would not have to remain for the lifetime of the request (i.e. checkin the connection and then await the result). I am unsure what performance would be like but it might be interesting to try as it would not require changing your Connection process.

The original goal of connection pooling pre 1.0.0 was that Exrethinkdb.Connection should work without any modifications with a connection pooling system. I think this discussion has been very useful. I think with a few modifications to Exrethinkdb.Connection, getting poolboy to work is just a matter of following some guidelines around configuration.
Modes of use

Traditional Pool. Check out a connection and check it back in when you are done. For changefeeds and other queries that are tied to specific connections, you need to keep the connection checked out until you close the feed. Otherwise, that connection might be terminated or restarted. You can, however, use that checked out connection in as many processes as you like. This will be especially useful for tasks.
Static Pool. max_overflow is set to 0. When you check out a connection you can check it back in immediately. After checking it in, you are still fully enabled to make requests on that connection. Changefeeds and other queries just work, no need to worry about hogging the connections in the pool.

I think the original goal of solving identified problems and then providing an example of how to use it with poolboy is still the correct approach. The other issue discussed here is table affinity and I'm not going to include smart routing in 1.0.0 for now.
Identified problems

Blocking in init. New issue opened at https://github.com/hamiltop/exrethinkdb/issues/34
Connection failure. New issue opened at https://github.com/hamiltop/exrethinkdb/issues/35
Throttling/Back Pressure/Load Shedding. New issue opened at https://github.com/hamiltop/exrethinkdb/issues/36

Any discussions on those topics should be moved to the issues opened. Any new issues can and should be discussed in this thread.

I'm going to reduce this problem down to a fairly simple one. Table affinity is not necessary with a rethinkdb-proxy. Throttling/Back Pressure/Load Shedding is still a problem, as is handling timeouts properly. It should work with poolboy out of the box, but that has yet to be tested.

RethinkDB.Connection works fine with poolboy right now:
pool_options = [name: {:local, :hello_pool}, worker_module: RethinkDB.Connection, size: 10, max_overflow: 10]
:poolboy.start_link(pool_options, [])
a = :poolboy.checkout(:hello_pool)
RethinkDB.Query.table_list |> RethinkDB.Connection.run(a)

Some advisable defaults:
size == max_overflow
Cursors, changefeeds, and some other queries are tied to a connection process. If that connection is terminated, then the query will fail. By setting them equal, you can be a little loose about mutual exclusion and check a connection back in even though you are still using it. The driver itself won't care, it properly multiplexes requests.
Changefeeds
Since changefeeds are persistent, mixing them into the pool makes for some difficult balancing. It might be worthwhile to create a separate pool for changefeeds in your application.
I may create a RethinkDB.Pool library with theses defaults, but I don't want to make the driver dependent on poolboy.

Removing 1.0.0 milestone. The goal was to have a connection that worked easily with poolboy, which is what we have.

Recently discovered that RethinkDB has core <-> connection affinity. So while the Elixir side scales well with multiple clients, a single connection is going to perform slightly worse on the server side. Connection pooling should be a priority.

Agreed that Connection pooling should be a priority, it is kinda frustrating without this feature

How is it frustrating? Can you provide some examples?
On Mon, Apr 18, 2016, 2:56 AM Leng Sheng Hong<EMAIL_ADDRESS>wrote:

Agreed that Connection pooling should be a priority, it is kinda
frustrating without this feature
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/rethinkdb-elixir/issues/32#issuecomment-211305076

I updated the README to show how to use poolboy out of the box.

I have been working on a adapter for Ecto for a while (:rethinkdb_ecto).
Currently it supports most of the Ecto DSL (including basic support for joins), storage and migrations.
As Ecto 2.0 requires adapters to support DBConnection, I rewrote RethinkDB.Connection to implement the DBConnection behavior. It is almost ready now (SSL support missing, need to rewrite some tests).
Basically, DBConnection uses Connection as the default pool implementation. It also has built-in support for :poolboy and :sbroker. Which means that moving from  Connection to DBConnection should not break things.
Right now, the only thing you have to do to get started is to replace
worker(RethinkDB.Connection, [[name: :foo]])

with
supervisor(RethinkDB.Connection, [[name: :foo]])

To use a different pool implementation (:poolboy for example), we could implement something like this:
defmodule Repo do
  use RethinkDB.Connection, pool: DBConnection.Poolboy
end

If you are interested in using DBConnection, I will get my fork ready (fix a few things) and create a pull request.
Note that i also implemented the new V1_0 handshake (RethinkDB >= 2.3.0) with support for user authentication (SCRAM). I skipped both V0_3 and V0_4 which may be required to connect to older version of RethinkDB.

Awesome! I appreciate the help.
My concerns are:

No required dependency on Poolboy or sbroker. We should make those
optional (shouldn't be hard).
Should work with 2.0.0+
Persistent checkouts from the pool and oversubscription are a priority.
One changefeeds per connection is not going to scale well. We need a
mechanism to oversubscribe. Poolboy doesn't really handle that but sbroker
might. A possible convention might be to create two pools one for
changefeeds and one for normal queries.

Otherwise, please submit the PR! Love the help.
On Tue, Apr 19, 2016, 5:31 AM Mario Flach<EMAIL_ADDRESS>wrote:

I have been working on a adapter for Ecto for a while (:rethinkdb_ecto
https://github.com/almightycouch/rethinkdb_ecto).
Currently it supports most of the Ecto DSL (including basic support for
joins), storage and migrations.
As Ecto 2.0 requires adapters to support DBConnection, I rewrote
RethinkDB.Connection to implement the DBConnection behavior
https://github.com/almightycouch/rethinkdb-elixir/commit/34e53e505026c2764c549d0c31916bd28437494f.
It is almost ready now (SSL support missing, need to rewrite some tests).
Basically, DBConnection uses Connection as the default pool
implementation. It also has built-in support for :poolboy and :sbroker.
Which means that moving from Connection to DBConnection should not break
things.
Right now, the only thing you have to do to get started is to replace
worker(RethinkDB.Connection, [[name: :foo]])
with
supervisor(RethinkDB.Connection, [[name: :foo]])
To use a different pool implementation (:poolboy for example), we could
implement something like this:
defmodule Repo do
use RethinkDB.Connection, pool: DBConnection.Poolboyend
If you are interested in using DBConnection, I will get my fork ready
(fix a few things) and create a pull request.
Note that i also implemented the new V1_0 handshake (RethinkDB >= 2.3.0)
with support for user authentication (SCRAM). I skipped both V0_3 and V0_4
which may be required to connect to older version of RethinkDB.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hamiltop/rethinkdb-elixir/issues/32#issuecomment-211902104

DBConnection does provide pooling (and poolboy and sbroker are optional dependencies) and the behaviour with pooling should be the same as without pooling, it just a case of adding the pool: module option to choose the pool. This means a connection is always limited to one client process.
Ecto 2.0 does not require all adapters to use DBConnection, only the SQL adapters (as DBConnection is required for the (concurrent) sandbox).
I have no idea about changefeeds but I assume they are approximately pubsub. This won't play well with DBConnection unfortunately because the socket is passed to the client process on each call. Postgrex uses a different process to support postgres notifications.
I am not sure what the best strategy for pooling changefeeds is. The simplest strategy might be to use poolboy and only carry out the subscription to a changefeed in the poolboy transaction (checkin/checkout). Then remain subscribed to the changefeed after checking in the process to poolboy. Likely it would be a good idea to use a :fifo queue otherwise one changefeed may end up with all the subscriptions. This would allow pooling and not pooling changefeeds. I am not sure if this will be good enough but hopefully it is. Whereas using sbroker would mean always pooling.

Yeah, I noticed that DBConnection passing the socket to the client process means we can't multiplex queries.
In an ideal world I think a tunable "oversubscription" setting would be what we want. Then maintain two pools, one for changefeeds and one for normal queries. Even in the query case, oversubscription on the connection is efficient.

Shackle is a pooling library that supports multiplexing: https://github.com/lpgauth/shackle. It works quite differently to DBConnection but provides similar abstractions.

Is your fork ready for a discussion of merging into the mainline?

Hi @hamiltop, The fork is ready to be merged without conflics.
All the tests are passing successfully with the exception of 5 tests in test/connection_test.exs which must be rewritten to work with the DBConnection behaviour. Some new test should also be written to test the connection pools.
There are a few implementation details that should be discussed thought:

The connection logic is implemented in the Multiplexer module.

Each connection has two processes, for sending and receiving (see recv_loop/2).
Calling send_recv/3 function blocks until the response is received (or connection timeout) without blocking the connection. This is done using GenServer.call/3 and GenServer.reply/3.
Only the latest authentication method (server >= 2.3) has been implemented. Older authentication methods must be implemented if required.

To use a specific pool backend, pass the module to the :pool option:

Within a supervision tree: supervisor(MyApp.Repo, [[pool: DBConnection.Poolboy]])
For available options, check: DBConnection.start_link/2.
Default is DBConnection.Connection, :poolboy and :sbroker are optional dependencies.

Opened #106 to discuss


Accessing x-wp-total and x-wp-totalpages fails on this particular account

Hi guys we have a woocommerce that is loading the headers x-wp-total and x-wp-totalpages in postman as seen in the sceenshot below:

but when I run the same query via the ruby gem, it omits the headers
request:
api.get('products?page=1&per_page=' + (search_params.page_size || 25), function(err, data, res) {

      console.log('headers', data.headers)

      // return response
      const response = {
        statusCode: 200,
        body: JSON.stringify({
          result: true,
          page_count: parseInt(data.headers['x-wp-totalpages']),
          page_size: (search_params.page_size || 25)
        })
      };

      callback(null, response);
    });

logged response:
{ date: 'Sat, 28 Apr 2018 01:51:48 GMT',
  'content-type': 'text/html; charset=UTF-8',
  'transfer-encoding': 'chunked',
  connection: 'close',
  'set-cookie': [ '__cfduid=d8b13fb621d86965f62633229257cdf721524880308; expires=Sun, 28-Apr-19 01:51:48 GMT; path=/; domain=.robbostackle.com; HttpOnly' ],
  'x-powered-by': 'PHP/7.0.29',
  'cache-control': 'max-age=0',
  expires: 'Sat, 28 Apr 2018 01:51:48 GMT',
  'host-header': '192fc2e7e50945beb8231a492d6a8024',
  vary: 'Accept-Encoding',
  'x-proxy-cache': 'MISS',
  'expect-ct': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
  server: 'cloudflare',
  'cf-ray': '4125e2c73c6c51f7-MEL' }

I know this is probably not a problem with the ruby gem, but I'm thinking along the lines of the customer might need to expose headers based on the user agent or something??
Any input on this?
System satus
WordPress Environment
Home URL: https://www.robbostackle.com
Site URL: https://www.robbostackle.com
WC Version: 3.3.5
Log Directory Writable: ✔
WP Version: 4.9.5
WP Multisite: –
WP Memory Limit: 1,000 MB
WP Debug Mode: –
WP Cron: ✔
Language: en_GB
Server Environment
Server Info: Apache/2.4.29 (Unix) mod_hive/6.6 OpenSSL/1.0.1e-fips mod_fastcgi/2.4.6
PHP Version: 7.0.29
PHP Post Max Size: 500 MB
PHP Time Limit: 600
PHP Max Input Vars: 10000
cURL Version: 7.53.0
OpenSSL/1.0.2k
SUHOSIN Installed: –
MySQL Version: 5.6.36
Max Upload Size: 500 MB
Default Timezone is UTC: ✔
fsockopen/cURL: ✔
SoapClient: ✔
DOMDocument: ✔
GZip: ✔
Multibyte String: ✔
Remote Post: ✔
Remote Get: ✔
Database
WC Database Version: 3.3.5
WC Database Prefix: wpq3_
Total Database Size: 86.64MB
Database Data Size: 59.34MB
Database Index Size: 27.30MB
wpq3_woocommerce_sessions: Data: 0.56MB + Index: 0.02MB
wpq3_woocommerce_api_keys: Data: 0.00MB + Index: 0.01MB
wpq3_woocommerce_attribute_taxonomies: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_downloadable_product_permissions: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_order_items: Data: 0.01MB + Index: 0.01MB
wpq3_woocommerce_order_itemmeta: Data: 0.05MB + Index: 0.04MB
wpq3_woocommerce_tax_rates: Data: 0.00MB + Index: 0.01MB
wpq3_woocommerce_tax_rate_locations: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_shipping_zones: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_shipping_zone_locations: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_shipping_zone_methods: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_payment_tokens: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_payment_tokenmeta: Data: 0.00MB + Index: 0.00MB
wpq3_woocommerce_log: Data: 0.00MB + Index: 0.00MB
wpq3_commentmeta: Data: 0.00MB + Index: 0.00MB
wpq3_comments: Data: 0.02MB + Index: 0.02MB
wpq3_crony_jobs: Data: 0.02MB + Index: 0.00MB
wpq3_crony_logs: Data: 0.02MB + Index: 0.00MB
wpq3_links: Data: 0.00MB + Index: 0.00MB
wpq3_linksync_laidKey: Data: 0.02MB + Index: 0.00MB
wpq3_linksync_log: Data: 0.48MB + Index: 0.00MB
wpq3_options: Data: 0.84MB + Index: 0.07MB
wpq3_pmxe_exports: Data: 0.70MB + Index: 0.00MB
wpq3_pmxe_google_cats: Data: 0.26MB + Index: 0.05MB
wpq3_pmxe_posts: Data: 4.94MB + Index: 2.16MB
wpq3_pmxe_templates: Data: 0.00MB + Index: 0.00MB
wpq3_pmxi_files: Data: 0.01MB + Index: 0.00MB
wpq3_pmxi_history: Data: 0.00MB + Index: 0.00MB
wpq3_pmxi_imports: Data: 1.36MB + Index: 0.00MB
wpq3_pmxi_posts: Data: 5.81MB + Index: 2.24MB
wpq3_pmxi_templates: Data: 0.20MB + Index: 0.00MB
wpq3_postmeta: Data: 32.80MB + Index: 11.13MB
wpq3_posts: Data: 4.01MB + Index: 1.10MB
wpq3_po_plugins: Data: 3.52MB + Index: 5.91MB
wpq3_qss: Data: 0.17MB + Index: 0.05MB
wpq3_redirection_404: Data: 0.45MB + Index: 0.56MB
wpq3_redirection_groups: Data: 0.02MB + Index: 0.03MB
wpq3_redirection_items: Data: 0.02MB + Index: 0.08MB
wpq3_redirection_logs: Data: 0.02MB + Index: 0.08MB
wpq3_sbb_blacklist: Data: 0.20MB + Index: 0.08MB
wpq3_snp_log: Data: 0.02MB + Index: 0.00MB
wpq3_snp_stats: Data: 0.02MB + Index: 0.05MB
wpq3_social_users: Data: 0.00MB + Index: 0.00MB
wpq3_termmeta: Data: 0.76MB + Index: 0.61MB
wpq3_terms: Data: 0.31MB + Index: 0.44MB
wpq3_term_relationships: Data: 0.89MB + Index: 1.77MB
wpq3_term_taxonomy: Data: 0.39MB + Index: 0.62MB
wpq3_usermeta: Data: 0.17MB + Index: 0.09MB
wpq3_users: Data: 0.02MB + Index: 0.02MB
wpq3_wcpdf_invoice_number: Data: 0.02MB + Index: 0.00MB
wpq3_wc_download_log: Data: 0.02MB + Index: 0.03MB
wpq3_wc_point_of_sale_sale_reports: Data: 0.00MB + Index: 0.00MB
wpq3_wc_webhooks: Data: 0.02MB + Index: 0.02MB
wpq3_wd_fb_data: Data: 0.05MB + Index: 0.00MB
wpq3_wd_fb_info: Data: 0.02MB + Index: 0.00MB
wpq3_wd_fb_option: Data: 0.02MB + Index: 0.00MB
wpq3_wd_fb_shortcode: Data: 0.02MB + Index: 0.00MB
wpq3_wd_fb_theme: Data: 0.02MB + Index: 0.00MB
wpq3_woochimp_scheduled_events: Data: 0.00MB + Index: 0.00MB
wpq3_wplc_chat_msgs: Data: 0.02MB + Index: 0.00MB
wpq3_wplc_chat_sessions: Data: 0.02MB + Index: 0.00MB
wpq3_wplc_offline_messages: Data: 0.02MB + Index: 0.00MB
Post Type Counts
attachment: 5284
blocks: 23
bne_testimonials_sg: 1
custom_css: 1
extra_product_tab: 2
featured_item: 8
mc4wp-form: 1
nav_menu_item: 15
page: 25
pos_temp_register_or: 1
post: 9
product: 3368
product_variation: 5066
revision: 8
sg_optimizer_jobs: 5
shop_coupon: 1
shop_order: 48
shop_order_refund: 5
snp_popups: 1
wcst_trigger: 9
woocollections: 4
woocollitems: 10
wpcf7_contact_form: 9
Security
Secure connection (HTTPS): ✔
Hide errors from visitors: ✔
Active Plugins (21)
ManageWP - Worker: by ManageWP – 4.5.0
BNE Testimonials Pro: by Kerry Kline – 2.7.2
Actionable Google Analytics: by Tatvic – CC-V3-2.0 – Not tested with the active version of WooCommerce
Contact Form 7: by Takayuki Miyoshi – 5.0.1
MailChimp for WordPress: by ibericode – 4.2.1
Nextend Social Login: by Nextendweb – 3.0.6
WooCommerce Product Filter: by Mihajlovic Nenad – 6.5.5 – Not tested with the active version of WooCommerce
Redirection: by John Godley – 3.2
SG Optimiser: by SiteGround – 4.0.5
Squirrly SEO 2018 (Briefcase): by Squirrly SEO – 8.3.17
StopBadBots: by Bill Minozzi – 4.77
Transients Manager: by Pippin Williamson – 1.7.4
Woocomerce Brands Pro: by Proword – 4.3.1 – Not tested with the active version of WooCommerce
Woocommerce Variations Table - Grid: by Spyros Vlachopoulos – 1.3.9 – Not tested with the active version of WooCommerce
WooCommerce Extended Coupon Features: by Soft79 – 2.6.2
WooCommerce Composite Products: by SomewhereWarm – 3.13.7 – 3.13.8 is available
WooCommerce Stacked Product Layout: by SomewhereWarm – 1.1.7 – Not tested with the active version of WooCommerce
WooCommerce PayPal Express Checkout Gateway: by WooCommerce – 1.5.3
WooCommerce Pretty Emails: by MB Création – 1.8.6 – Not tested with the active version of WooCommerce
WooCommerce: by Automattic – 3.3.5
WP Rocket: by WP Media – 3.0.4
Settings
API Enabled: ✔
Force SSL: –
Currency: GBP (£)
Currency Position: left
Thousand Separator: ,
Decimal Separator: .
Number of Decimals: 2
Taxonomies: Product Types: composite (composite)
external (external)
grouped (grouped)
simple (simple)
variable (variable)
Taxonomies: Product Visibility: exclude-from-catalog (exclude-from-catalog)
exclude-from-search (exclude-from-search)
featured (featured)
outofstock (outofstock)
rated-1 (rated-1)
rated-2 (rated-2)
rated-3 (rated-3)
rated-4 (rated-4)
rated-5 (rated-5)
WC Pages
Shop base: #63 - /shop/
Basket: #62 - /cart/
Checkout: #61 - /checkout/
My account: #60 - /my-account/
Terms and conditions: #10945 - /terms-conditions/
Theme
Name: Flatsome Child
Version: 3.0
Author URL:
Child Theme: ✔
Parent Theme Name: Flatsome
Parent Theme Version: 3.5.3
Parent Theme Author URL: http://www.uxthemes.com/
WooCommerce Support: ✔
Templates
Overrides: flatsome/woocommerce/archive-product.php
flatsome/woocommerce/cart/cart-empty.php
flatsome/woocommerce/cart/cart.php
flatsome/woocommerce/checkout/form-checkout.php
flatsome/woocommerce/checkout/form-coupon.php
flatsome/woocommerce/checkout/thankyou.php
flatsome/woocommerce/content-product.php
flatsome/woocommerce/content-product_cat.php
flatsome/woocommerce/content-single-product.php
flatsome-child/woocommerce/content-widget-product.php
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/admin-cancelled-order.php version 2.3.0 is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/admin-failed-order.php
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/admin-new-order.php version 2.0.0 is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-completed-order.php version 1.6.4 is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-invoice.php version 2.2.0 is out of date. The core version is 3.3.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-new-account.php
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-note.php version 1.6.4 is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-on-hold-order.php version - is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-processing-order.php version 1.6.4 is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-refunded-order.php version 2.4.0 is out of date. The core version is 2.5.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/customer-reset-password.php
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/email-addresses.php version 2.2.0 is out of date. The core version is 3.2.1
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/email-customer-details.php
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/email-footer.php version 2.0.0 is out of date. The core version is 2.3.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails//emails/themes/email-header-centered.php version 1.0 is out of date. The core version is 2.4.0
/home/globet47/public_html/robbostackle.com/wp-content/plugins/woocommerce-pretty-emails/emails/email-order-items.php version 2.1.2 is out of date. The core version is 3.2.0
flatsome/woocommerce/global/breadcrumb.php
flatsome/woocommerce/global/quantity-input.php
flatsome/woocommerce/global/wrapper-end.php
flatsome/woocommerce/global/wrapper-start.php
flatsome/woocommerce/loop/loop-end.php
flatsome/woocommerce/loop/loop-start.php
flatsome/woocommerce/loop/pagination.php
flatsome/woocommerce/loop/result-count.php
flatsome/woocommerce/loop/sale-flash.php
flatsome/woocommerce/myaccount/form-login.php
flatsome/woocommerce/myaccount/navigation.php
flatsome/woocommerce/notices/error.php
flatsome/woocommerce/notices/notice.php
flatsome/woocommerce/notices/success.php
flatsome/woocommerce/product-searchform.php
flatsome/woocommerce/single-product/photoswipe.php
flatsome/woocommerce/single-product/price.php
flatsome/woocommerce/single-product/product-image.php
flatsome/woocommerce/single-product/product-thumbnails.php
flatsome/woocommerce/single-product/related.php
flatsome/woocommerce/single-product/review.php
flatsome/woocommerce/single-product/sale-flash.php
flatsome/woocommerce/single-product/share.php
flatsome/woocommerce/single-product/short-description.php
flatsome/woocommerce/single-product/tabs/tabs.php
flatsome/woocommerce/single-product/title.php
flatsome-child/woocommerce/single-product/up-sells.php
flatsome/woocommerce/single-product-reviews.php
flatsome/woocommerce/single-product.php
Outdated Templates: ❌Learn how to update
`

SOLVED:
Plugin conflict: Stop-Bad-Bots
blocking specific user agents


Add an output (widget) class

This adds an output widget class and a PyironWrapper to the ProjectBrowser.

The PyironWrapper ensures that all pyiron objects play well with e.g. list_nodes() and stores the appropriate self-representation of an object (i.e. self.plot3d() for structures and self.plot() for murnaghan)
The DisplayOutputGUI may be used like a 'normal' ipython.Output widget, however, also via a display method which allows for automatic type adaptations for an object to be displayed.

I will work on additional tests and update the environment to include everything which is needed in the evening.

I will merge this one now, despite the known issue #6. Due to this issue the example notebook also shows the 'save' way of initializing the ProjectBrowser with a given Project directly. Moreover, since this is the only GUI feature so far the benefit of binding this to the Project is not (yet) that big.


segmentation fault when I convert onnx to trt engine.

Thanks for great work! I have a quick question. I'm using tensorrt 8.5.2 when I convert onnx model to trt engine. However, I have a error like segmentation fault. CLI takes a commend like "trtexec --onnx="model.onnx" --saveEngine="model.engine" --fp16" as mentioned in README.md. Could you help me?

more information pls, where will happen segfault? first check your exported onnx, is it correct? can you infer it with onnxruntime? then go down to tensorrt and try to convert

FYI, I convert it successfully on TRT8.6 and TRT10 @ PC, even TRT <IP_ADDRESS> @ Jetson

ah? The log is simple. I add flag '--verbose' to see the detailed log. It just show segmentation fault. It is wired since dfine-n was converted to tenssort successfully. I use the device orin-nx 16G and a version of Jetpact is 5.1.2. It equipt tenssorrt <IP_ADDRESS> . Following your answer, you convert dfine-s successfully on jetson, right? Could you tell me the detailed information about your spec of jetson device?

ah, It is a different question. I want to make model more lightly because of a expensive computation cost. I think the number of parameter in dfine-n is ok, but I want faster model. One way of this is to chance layer of decoder. It maybe reduce some heads in attention... I cannot ensure a result...Could you give a advice for it?

ah? The log is simple. I add flag '--verbose' to see the detailed log. It just show segmentation fault. It is wired since dfine-n was converted to tenssort successfully. I use the device orin-nx 16G and a version of Jetpact is 5.1.2. It equipt tenssorrt <IP_ADDRESS> . Following your answer, you convert dfine-s successfully on jetson, right? Could you tell me the detailed information about your spec of jetson device?

Jetson Orin NX 8GB, Jetpack 5.1.1, TensorRT v8502, converting original weights (onnx exported using repo's export script except for changing batchsize to 8)

command of converting:
trtexec --directIO --fp16 --onnx=dfine_n_coco.onnx --saveEngine=dfine_n_coco.640x640.b8.engine --minShapes=images:1x3x640x640,orig_target_sizes:1x2 --maxShapes=images:8x3x640x640,orig_target_sizes:8x2 --optShapes=images:8x3x640x640,orig_target_sizes:8x2

ah, It is a different question. I want to make model more lightly because of a expensive computation cost. I think the number of parameter in dfine-n is ok, but I want faster model. One way of this is to chance layer of decoder. It maybe reduce some heads in attention... I cannot ensure a result...Could you give a advice for it?

sorry I haven't got a good GPU to retrain it, reduce heads or reduce layer's width might be a good idea but I have no resource to prove it, actually I prefer replacing some layers with those which supports reparameterization method like RepVGG's blocks (may have a greater parameters but really hardware-friendly: 3x3 conv & 1x1 conv / Linear, BatchNorm, ReLU -> Really fast), especially I hate using GeLU and its variants since it's suck on Jetson and Raspberry Pi and Rockchip, (using these) leads to increasing latency and vram usage

command of converting:
trtexec --directIO --fp16 --onnx=dfine_n_coco.onnx --saveEngine=dfine_n_coco.640x640.b8.engine --minShapes=images:1x3x640x640,orig_target_sizes:1x2 --maxShapes=images:8x3x640x640,orig_target_sizes:8x2 --optShapes=images:8x3x640x640,orig_target_sizes:8x2

Thanks for detailed information. I just want to convert dfine-s to trtengine. Did you also success to convert it?

command of converting:
trtexec --directIO --fp16 --onnx=dfine_n_coco.onnx --saveEngine=dfine_n_coco.640x640.b8.engine --minShapes=images:1x3x640x640,orig_target_sizes:1x2 --maxShapes=images:8x3x640x640,orig_target_sizes:8x2 --optShapes=images:8x3x640x640,orig_target_sizes:8x2

Thanks for detailed information. I just want to convert dfine-s to trtengine. Did you also success to convert it?

yep, same way, just replace n to s

ah, It is a different question. I want to make model more lightly because of a expensive computation cost. I think the number of parameter in dfine-n is ok, but I want faster model. One way of this is to chance layer of decoder. It maybe reduce some heads in attention... I cannot ensure a result...Could you give a advice for it?

sorry I haven't got a good GPU to retrain it, reduce heads or reduce layer's width might be a good idea but I have no resource to prove it, actually I prefer replacing some layers with those which supports reparameterization method like RepVGG's blocks (may have a greater parameters but really hardware-friendly: 3x3 conv & 1x1 conv / Linear, BatchNorm, ReLU -> Really fast), especially I hate using GeLU and its variants since it's suck on Jetson and Raspberry Pi and Rockchip, (using these) leads to increasing latency and vram usage

Great idea. Why I choose this network is mainly because this network is based on transformer. I am searching network using transformer based network. Although I also think GeLu and its is not that good for jetson device, I have to use transformer based network.. That's why I try to change the number of head. Again, thanks for share your insight!

command of converting:
trtexec --directIO --fp16 --onnx=dfine_n_coco.onnx --saveEngine=dfine_n_coco.640x640.b8.engine --minShapes=images:1x3x640x640,orig_target_sizes:1x2 --maxShapes=images:8x3x640x640,orig_target_sizes:8x2 --optShapes=images:8x3x640x640,orig_target_sizes:8x2

Thanks for detailed information. I just want to convert dfine-s to trtengine. Did you also success to convert it?

yep, same way, just replace n to s

When I relpace 'n' to 's', 'trtexec' raise a segmentation fault error. Before the error, I am able to see the following log.
"[W] [TRT] Tactic Device request: 9240MB Available: 6964MB. Device memory is insufficient to use tactic.
Segmentation fault (core dumped)"
then, the segmentation fault error occurs. What I only do before converting it is to convert pth to onnx. I use the convert tools in this repo. of course, A result of Inference  using onnxruntime on onnx filet is correct.. Do you have any idea?


allow skipping leader election

Allow skiping leader election to add some flexibility when developing on hive where we can simultaneously have hive running in a cluster (potentially with some controllers disabled), and running hive locally.

/assign @akhil-rane
/cc @staebler


Crops?

Do we want a table of crops and a typeahead to search for them?
As well as the admin side to add/remove the list?

@jSylvestre I kinda think we should

Populate the react info from the DB...


[Bug Report]: Erratic pasted code content behavoiour upon page save

Describe the Bug
When pasting text from a html pre element (like code samples), the editor goes berserk. It shows everything correctly, but when saving the page, it asks if I'm sure I want to navigate away. If I click yes, the page is saved, but all the other code blocks on the page get doubled somehow: each line in the other code samples becomes a separate code block inside it's main code block...
Steps to Reproduce

Copy a code sample from https://kinsta.com/blog/xmlrpc-php/, but any text inside a pre is the same, from what I've tested.
create a document in bookstack and add a code block with a few lines, save
Paste inside the created page (that already has a code block). (The new code block you add after the existing block(s), on an empty white space.)
Click save.

Expected Behaviour
either copy as text (clean all formatting) or insert as code block
Screenshots or Additional Context
No response
Exact BookStack Version
21.10.3
PHP Version
7.4
Hosting Environment
Ubuntu 20.4 LTS, Apache + Nginx

Thanks for reporting @noxtras.
This has been patched to be part of v21.12.1.
This wasn't caused by any pre blocks, but code blocks on other sites, with certain formatting, could confuse and break our parsing logic which would then break the editor. The patch adds more extensive and safer language parsing.

super


Allow proper APS dictionary updating

There are some notifications which require adding special flags to the aps dictionary, for example "mutable-content": 1 (See #23).
This PR adds that functionality, while also making the tests pass on Travis-CI

Hello, just chasing up on this to see if you've had time to look at this

@melbic Wonder if we can have an update on this :)

Thanks for you work. I merged it and released the version 3.1


devcontainer vscode settings not taking effect in remote container

VSCode Version: 1.71.0 (Universal)
Local OS Version: macOS Monterey 12.5.1
Local chip architecture: Apple Silicon
Reproduces in: Remote - Containers

Maybe I'm not understanding how this is supposed to work . . .
I have a devcontainer.json to which I added some settings:
  "customizations": {
    "vscode": {
      "extensions": [...deleted for brevity...],
      "settings": {
        "rubyTestExplorer.testFramework": "rspec",
        "rubyTestExplorer.logpanel": true,
        "rubyTestExplorer.debugCommand": "rdebug-ide",
        "rubyTestExplorer.filePattern": [
          "*_spec.rb"
        ]
      }
    }

After rebuilding the devcontainer and restarting VSCode I'd expect these settings to be present in my remote container but they are not:
> cat ~/.vscode-server/data/Machine
{}

Aha, I have my home directory in the container mounted to a persistent docker volume, that seems to make the difference . . . is that expected?

Workaround seems to be whenever I need to change the vscode customizations (adding extensions, settings), I have to
rm -rf ~/.vscode-server

and restart vscode.


Update report dependencies endpoint to group by name only

For UI changes https://github.com/konveyor/tackle2-ui/issues/1338 and https://github.com/konveyor/tackle2-ui/issues/1340, the dependency table needs to display dependencies grouped by name only.  It currently displays rows based on name and sha.
Endpoint /analyses/report/dependencies is what is currently being used.  It returns struct:
//
// DepReport REST resource.
type DepReport struct {
	Provider     string   `json:"provider"`
	Name         string   `json:"name"`
	Version      string   `json:"version"`
	SHA          string   `json:"sha"`
	Labels       []string `json:"labels"`
	Applications int      `json:"applications"`
}

Please adjust the response to group dependencies by name only.  The SHA and Version fields are no longer needed.   The Applications count and Labels summary fields should also be adjusted.
Endpoint /analyses/reports/dependencies/applications is used by the application table on a dependency's details view.  This endpoint works as needed, but the response object has the dependency's sha named "rule":
//
// DepAppReport REST resource.
type DepAppReport struct {
	ID              uint   `json:"id"`
	Name            string `json:"name"`
	Description     string `json:"description"`
	BusinessService string `json:"businessService"`
	Dependency      struct {
		ID       uint     `json:"id"`
		Provider string   `json:"provider"`
		Name     string   `json:"name"`
		Version  string   `json:"version"`
		SHA      string   `json:"rule"`
		Indirect bool     `json:"indirect"`
		Labels   []string `json:"labels"`
	} `json:"dependency"`
}

Please rename the SHA field's json name to "sha".

cc @ibolton336

Is the filter on:

version
sha

still needed?

Is the filter on:
* version

* sha

still needed?

I don't think so... At the very least, they will not have a use on the dependencies table.


284 ImageTransformations

Motivation
Fixes #284
Checklist

[ ] Code follows coding conventions held in this repo
[ ] Automated tests have been added
[ ] Tests are passing
[ ] Docs have been updated (if applicable)
[ ] Temporary settings (e.g. variables used during development and testing) have been reverted to defaults

How to test
Check if scenario mentioned in #284 is not reproducible

I am not sure whether we want to release major version v16 because of namespace changes, how much do you rush on this @petrsvihlik?

@Sevitas already did - https://github.com/Kentico/kontent-delivery-sdk-net/releases/tag/16.0.0.beta1

Closing in favor of #300


Updates branding to remove "Python Tools for Visual Studio" name.

Updates branding to remove "Python Tools for Visual Studio" name.
Moves more strings into common resource file.
Removes ML project
Removes IPyTools detection.

:shipit:


Remove Ord and PartialOrd derives where they don't make sense.

Many types that are not intrinsically comparable had derives for Ord and PartialOrd. These have been removed.

Thanks for linking to that doc; I've never seen it before!
I see what you mean about eagerly implementing types, but my interpretation of their guidance is only to implement the traits where they make semantic sense. I suppose my removal of these derives has the consequence that Vecs of these types can't be sorted, nor can BTreeMaps be keyed by these types, but those seem like very narrow use cases.

I'm very split on this. On one hand I agree, it doesn't make much sense to sort some of these times, on the other hand it's not like it's hindering anything to have them.

That's true, it's not hindering anything, but the behavior may be surprising in some circumstances. I actually started this PR after noticing Price had a derive for Ord and that the behavior could be unintuitive to people legitimately wishing to compare prices.

I asked around in the #rust:matrix.org room and the consensus was not to implement Ord and PartialOrd unless the sementics make sense. Note the way it's phrased in the API guidelines:

Consequently, crates that define new types should eagerly implement all applicable, common traits.

I added some impls for Price and Legality but I didn't add one for Color since colors are a cycle.


CASSANDRA-18683: Add PartitionSizeTableProvider for reading the compressed and uncompr…

…essed sizes of all partitions in a table by utilizing the SSTable Index.db files

Committed: https://github.com/apache/cassandra-analytics/commit/014db08a79f00ef0d94e6855779e398c9dc689c1
Jira: https://issues.apache.org/jira/browse/CASSANDRA-18683


Refactor names

Description:
Refactor code:

Rename percent to percentage according spec https://www.w3.org/TR/css-values-4/#percentages
Rename *Identifier to Ident/CustomIdent/DashedIdent as in spec https://www.w3.org/TR/css-values-4/#textual-values
Move some at-rules to own files
base.rs is not contains base types for CSS (we need to move at-rules too, but it will be in future, after we finish refactor at-rules)

BREAKING CHANGE:
Yes
Related issue (if exists):
No

Can you rebase?

Yep, WIP

@kdy1 Done :+1:


How to use NAND Flash device simulator

I add NAND simulation under sim/nsh configuration, when 'nand' is executed, the ‘/dev/nand‘ device will be generated, but when using "mkfatfs /dev/nand" or "echo hello > /dev/nand", some errors will be generated and the program will be killed. I don't know how to use this device, actually I want to try to port the YAFFS2 filesystem based on this emulated NAND, and it has been successfully ported in bare metal, I need some help.
In addition, there are some problems with this NAND driver:

Some debugging and logging must be enabled；
Compilation error under HOST_ARM64;
SPI NAND cannot be supported

And this is my .config
config.txt
Error logs:
NuttShell (NSH) NuttX-12.5.1-RC0
MOTD: username=admin password=Administrator
nsh> nand
nand_ram: [LOWER | initialize]
Driver running!
nand_wrapper: [UPPER 1 | ioctl] Command: 1537, Arg :<PHONE_NUMBER>86160
nand_wrapper: [UPPER 1 | ioctl] Done
Page size: 512
nsh> echo hello > /dev/nand
nand_wrapper: [UPPER 2 | ioctl] Command: 1537, Arg :<PHONE_NUMBER>83416
nand_wrapper: [UPPER 2 | ioctl] Done
nand_wrapper: [UPPER 3 | isbad] Blocks: 0
nand_ram: [LOWER 1 | rawread] Page 0
nand_ram: [LOWER 1 | rawread] Done
nand_ram: [LOWER 2 | rawread] Page 1
nand_ram: [LOWER 2 | rawread] Done
*** stack smashing detected ***:  terminated
已放弃 (核心已转储)

Hi @laoniaokkk did you look at https://nuttx.apache.org/docs/latest/applications/testing/nand_sim/index.html ?
The "stack smashing detected" means you are passing some memory limit.
Maybe @resyfer could give you more details. Sarav, could you please take a look?

Hi @acassis, the execution weirdly stops in-between execution it seems. I am not too knowledgeable about the memory limits in the simulator.
For me the stack trace for mkfatfs on /dev/nand sort of went like this:
mkfatfs.c:mkfatfs():L326
fs_open.c:open():L453
fs_open.c:nx_vopen():L300
fs_open.c:file_vopen():L199
fs_blockproxy.c:block_proxy():L167
bchdev_register.c:bchdev_register():L61
bchlib_setup.c:bchlib_setup():L75
fs_openblockdriver.c:open_blockdriver():L86
fs_mtdproxy.c:mtd_proxy():L164
And in ftl.c, it crashes in between this area. My config:

Disabled:

CONFIG_FTL_READAHEAD
CONFIG_FTL_WRITEBUFFER

Thus FTL_HAVE_RWBUFFER is also disabled. So the entire area has nothing to crash on, and I think, like @acassis  said, it might be due to a memory limit. I am not sure what can prevent this, and I need help on this.
Both mkfatfs and echo commands provided by @laoniaokkk stop at this point.
nand (the device simulator) relies heavily on memory. Currently, the entire memory is allocated in an entire contiguous chunk, which might be a cause. Of course this can be changed to be more lazy in nature.
The sim device is still experimental so a lot of features are missing or might not work in certain circumstances. It has been tested mostly by using littlefs. Thank you for pointing this out!


Clarify updates to G-stage PTE A/D bits for implicit access to VS-stage page tables

As discussed in issue #1103

Sorry messed up my repo. Will fix and resubmit.


Dydx Airdrop  You've receive $2000 🪂🔴

Congratulations! You've been chosen to receive a $2,000 bonus. 🪂🔴
Trading and Launch Rewards are live on dYdX Chain! ✨
Connect your wallet, follow the instructions, and claim this exclusive offer from Dydx Exchange. For GitHub users only—share with friends to make it eligible. Redeem your reward first.

🚀 Claim Your Bonus Now! Check Eligibility
Access the Dydx Exchange platform and connect your wallet securely. Ensure you're using a trusted wallet for the transaction.

🔗 Connect your wallet twice
📋 Register on the platform
💰 Claim your bonus, up to $2,000!

收到了，谢谢哈。

很高兴收到您的邮件


Add integration to have LocalLifecycleOwner/LocalSavedStateRegistryOwner/LocalViewModelStoreOwner at each Composable level

Certain Jetpack libraries such as AndroidX Camera require integration against lifecycleOwners and whatnot, but without setting up a LocalLifecycleOwner, that would use the Activity, and would not respect the removal from the composition.
The problem is that while service creation and activation state is tracked, onStart/onStop are not. However, we can easily ask for LifecycleOwner and observe it ourselves to get this information.
Also, enabling scoped services is completely optional, it's only enabled when setScopedServices() is set to a value that is meaningful. But this makes it tricky to store the ViewModelStoreOwner (although retained objects were added specifically with this feature in mind in the future, same for its Bundleable support).
However, there is no way around this, this must be implemented at some point, as while sure, you can show a Composable as a screen, but you won't be able to interop that easily with AndroidX -- which harms the adoption of this library.

Is there a point in adding LocalViewModelStoreOwner? I thought "the simple stack way" is to use scoped services, not AndroidX ViewModel

@matejdro I forgot to say, thank you for the contribution! I've released the latest changes in 0.10.0.
In the end, we didn't actually introduce a lot of significant breaking changes, and this solves some very long-lasting problems. Now all I need to do is think of a good way to handle https://github.com/Zhuinden/simple-stack/issues/259 to handle android:enableOnBackInvokedCallback so that exiting the app will that fancy gesture animation, but I'll actually try to tackle that sometime next week. I don't like having to call the deprecated onBackPressed. 😅


x/tools/internal/lsp: error handling

We need to think about what the rules for error handling should be in gopls, specifically what kinds of things result in rpc errors vs just logging.
For instance:

errors in producing diagnostics, which happen without the user requesting them, cannot result in an rpc error but should not be invisible. Should they appear as a diagnostic, or just be logged to the client console?
errors in formatting, should they fail the rpc or just result in no edits?
errors in attempting to generate code actions, which happen inside an rpc but without the user asking for them, should they fail the rpc or just log to the client console?

I am currently looking into using xerrors extensively through gopls to both improve the error messages, give us call traces, and also give us the type information to make decisions in the lsp layer about things that happened way down inside the source layer.
Once that is done it will get much easier to think about and experiment with the rules, but we should probably have some kind of overall logic we apply to make these decision.

FWIW my answers would be:

errors in producing diagnostics, which happen without the user requesting them, cannot result in an rpc error but should not be invisible. Should they appear as a diagnostic, or just be logged to the client console?

Appear as a diagnostic.

errors in formatting, should they fail the rpc or just result in no edits?

Just result in no edits.

I agree about diagnostics, but @myitcv, I think we disagree about formatting. I think it's on the client side to make the decision of how to present formatting errors. If we respond with no edits in the case of both a correctly formatted file and a broken file that cannot be formatted, we are not giving the client all of the information it needs. As a user, I know that I would appreciate an error message if I keep trying to format a file that has a syntax error that maybe I didn't notice.

I for one am not clear on where the line between errors and diagnostics starts/ends, so please to continue to consider my answers in that light.
If I have a syntax error in my file, should I expect to see a diagnostics about that? I think the answer is, yes, diagnostics are reported to the client (not least because that's what I'm seeing today). So, assuming the user is looking at those diagnostics, I think it's fair for the server to return no edits for a file with syntax error: there is nothing that can be done, the user needs to fix the syntax error first.
Otherwise, we would need to start encoding detail in the error returned to the client, detail that is already available in the diagnostics.
Or am I missing something?

I just don't think that we can rely on diagnostics for this. VSCode already has a configuration to turn off diagnostics from the language server, so we have no guarantee that just because a user is asking the server for formatting, that the user is also seeing our diagnostics.
Additionally, even though I recently disabled this, I think we should do our best to format files with syntax errors (even if imperfectly), so then that case will become ambiguous. In the case completion or signature help, an error means that we have no results to show the user - so it's fine if we just reply with no results. But with formatting, no results has multiple meanings - either my file is already perfectly formatted or my file is totally broken, and if I've disabled diagnostics, or even if I'm not paying attention and I didn't notice a red squiggle, I can't distinguish between the two.
To be honest, I think I'd prefer to send all of the errors and let the client decide. It'd be nice if we could differentiate between a fundamental failure and an error that could be user-facing. I'm imagining a scenario where you made a typo and are trying to complete "fm." instead of "fmt." -- it might be nice to get a popup that "fm" is undefined, even in addition to the diagnostic.

VSCode already has a configuration to turn off diagnostics from the language server, so we have no guarantee that just because a user is asking the server for formatting, that the user is also seeing our diagnostics.

But didn't you also mention (on our call?) that VSCode ignores errors from calls to the server?

if I've disabled diagnostics, or even if I'm not paying attention and I didn't notice a red squiggle, I can't distinguish between the two.

Good point. But at this point it feels like we're about to start defining an entirely new error mechanism with structure being added to the errors returned by gopls. Unless structured errors (that are above JSON RPC in the "stack") already form part of the LSP spec?
Even if VSCode didn't ignore these errors (assuming my recollection above is correct), it's presumably not going to be able to do anything more than print the string version of the structured error. i.e. there is no logic to "do X in case of structure Y"?
The ResponseError type looks fairly low-level. But again, yours is the greater experience with the LSP spec here so I'll defer 😄

I updated https://github.com/golang/go/wiki/gopls-integrator-FAQ#rpc-response-errors to reflect the current state of affairs regarding this issue. Things seem to have stabilized a bit, so I am thinking that I will close this issue for now, and we can re-open discussion on a case-by-case basis as new problems come up.


Update dotnet core sdk to 2.1.3

I've removed the now obsolete PackageTargetFallback element from the csproj file:
See also: https://stackoverflow.com/questions/45569378/upgrading-to-net-core-2-0-packagetargetfallback-and-assettargetfallback-cannot

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: wgross sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


No codemirror.js

Hi I downloaded codemiror there is now codemirror.js file in libs.

Not a bug - codemirror now uses node to create the codemirror.js file. Do you have npm on your machine?

Download it from NPM or the website, don't clone the git repository.


PERK time integration schemes' documentation

Here is my attempt to document PERK schemes under the section time integration. Since I think the existing information should preface the brief tutorial of PERK integrator, I put everything in one file but created a different section so that the user can still navigate quickly to the Optimized Scheme section by clicking. However, if you wish to make it a folder and separate the content of this one file into two, being time-integrator-overview.md and optimized-schemes.md, I can also do that.

Cool, will take a look next week when I'm back!

Thanks for starting this! I will use this as a starting point and add stuff to it :)

@warisa-r I elaborated a bit. Can you go over this and proof-read/check for typos/bad formulations?

Let' s wait for https://github.com/trixi-framework/Trixi.jl/pull/2008 then we/you can open the PR at the main repo :)

Ready to move on to the main repo!


Any document?!

Hi, I didn't find any documents that describe how to use this bundle, it would be great if you added some instructions to the readme file

Hello Mr. Nejati, I added an example in Readme file, Is it clear or you need more?
Thanks.


[Addition]: Witness stuffer request verification

Hotspot b58 Addresses
1121f8SqCjmPbo4QpBUFw68AxyR6mYQkxFQ4QT6uXJiPmjMK5aHw
Discord Handle
digitalhuevos
Reason(s)
Hot spot is stuffing witness and packet sniffing 1000+beacons

'I've always been a quick earner'
This HS's earnings over the previous few days are 10 times its historical daily earning. Is it possible that the current weirdness with the network has suddenly decided to grant them this massive increase, several days running?

1275 beacons witnessed and you are basically doing 1 every 3-4minutes now seems not only unlikely with the current issues. i would venture to say almost impossible.

Similar behaviour from a very long-serving HS:
https://explorer.helium.com/hotspots/11oybvJjNEG2LBsiNuVgLL4QMcEBeuuPGQi75c47DehhYMMM4Cp
Not sure if that suggests it COULD be a lucky network weirdness (since it is another example), or just that long-serving HS can also be involved in possible gaming

這樣玩下去HNT就要死了

Interestingly, crowdspot's new 'location confidence' feature suggests this HS's location is 'likely abnormal'
https://www.crowdspot.io/hotspots/1121f8SqCjmPbo4QpBUFw68AxyR6mYQkxFQ4QT6uXJiPmjMK5aHw

But I've no idea what this new measure is based on or how accurate it is, so can't put any stock in it. This is a major disadvantage. Presumably if this metric was reliable it could simply be calculated for all HS and then a list presented for further review. Since this doesn't happen, one assumes the measure isn't seen as reliable as yet.
If there was a way to provide feedback on crowdspot, or even ask about how it works, then this kind of issue could be clarified.


Mysterious issue in CI -- wondering if anyone else is experiencing this?

Hey there!
I know nobody likes vague issues without a way of reproducing but my colleague @tgreen7 and I have found that using the latest version of @risingstack/react-easy-state 6.2.0 breaks in CI without an error message. However v6.1.3 works fine for us but has that nasty input bug.
I know a lot has changed between those two versions. Is anyone else experiencing a mysterious break when going from 6.1.3 to 6.2.X ?
I'll update this with more info as we try to dig deeper into it
Thanks!

Can you try v6.2.1? (See this issue for context)

@solkimicreb thanks for the idea. It took us a little while but it turns out it was an issue caused by our tests running electron@59. The version of node bundled with electron@59 apparently didn't have support for the spread operator being used in your code base. We're upgrading our testing framework (cypress) to a higher version which should use a node version compatible with spreads.
Thanks! Hopefully this helps out some other people :)

This is a valid issue, thanks for the investigation!
Object spread is a Node 8+ feature while we should support Node 6+. This was on oversight in the babel config which we will fix in the next release.

@solkimicreb this is great news! Let me know when that new feature is released as we're having issues upgrading cypress and would still love to use react-easy-state. Thanks!

@solkimicreb any movement on this one? Thanks!

:tada: This issue has been resolved in version 6.2.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Seed widget branch with initial structure

Added the 'widget' file to the jupyter sub-module. This file includes a render function which returns a root component containing a placeholder element.

@frreiss @Crushellini


[Op] Fix power_dx

Description
The current derivation of y=x^a is dx = a * x^a / x. This is problematic and results in nan when x=0. This PR changes the logic to dx = a * x ^ (a-1) to avoid such case.
Checklist

[x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage
[x] Code is well-documented

cc @awslabs/raf-reviewer @yzhliu

Thanks @yzhliu


Info: Add version number

Add the version number to the form "Info".
The version number should be determined from the DLL at runtime.

Released with version <IP_ADDRESS>


TerraformExplore

When using the explorer and a resource has a large output I am unable to scroll in the telescope view. Is it just me or is this a bug?

do you mean the preview window?
if so - i'd need to check if there is a limit on how much the preview viewport will allow.
can you tell me how many lines that resource has?

odd i can't even get the preview window outside of this project to scroll up or down... using the default keymaps.
https://github.com/nvim-telescope/telescope.nvim/blob/master/lua/telescope/mappings.lua#L149C22-L149C22


type configurations

Pull Request Test Coverage Report for Build 56

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 52:
0.0%

Covered Lines:
82

Relevant Lines:
82

💛  - Coveralls


Inter Cam HW Sync prevents from RGB/D sync on each device

Hi,
I have managed to have perfect time-synchronized flow (all RGB+Depth at 30 FPS) between two cameras with the realsense viewer app. But when I try to do the same without the app GUI, RGB wants always to be the master over depth (29.8 FPS instead of 30 FPS) and only the depth flows of the two cameras seems time-sync, not the rgb flows since the RGB and depth flow of the same device are not sync. I have tried to launch depth sensor before rgb sensor with sensor.start but without success.
Is there any way to force the Depth sensor as the master over RGB?
Note: I am using hardware sync between the two cameras as explained in the multicamera white paper

If the depth flows of the two cameras are synced and the RGB stream is the master one one of the cameras, then wouldn't that RGB stream also be synced to the depth stream on the other (i.e. slave) camera? Seems like the only issue then would be that the slave camera's RGB stream might be out of sync.

I have checked and the RGB stream is not synced with depth streams whether the camera I look at.
In more details, the INTER-camera sync (one D415 with other one D415) only works for depth streams which are time synced according global timestamps. To have also INTER-camera sync for both RGB streams, the INTRA-camera sync (RGB stream synced with the depth stream of the same D415 camera) has to work for each camera since for INTER-camera sync, it is the depth streams which are connected with the sync trigger clock. So if the depth streams are not master over RGB streams, they do not share the sync trigger clock with RGB.
The problem is that, by default, the RGB stream is the master over depth stream on the same D415 device for INTRA-camera sync. I  could bypass this problem in the realsense-viewer GUI by simply launch the depth sensor before the RGB sensor in order to force the depth to be master for INTRA-camera sync. But I did not succeed when trying to reproduce it with my own code (depth_sensor.start() followed by rgb_sensor.start() with AE priority off and same frame rates).
I think it is the same problem as #2637 which reports (see end of issue) good depth synced but wrong rgb synced. Since I could do it in the viewer GUI (https://www.youtube.com/watch?v=DSm7NYG5bZE&feature=youtu.be) I know there is a way to do it but I do not know how. I will try to do hardware reset and put some sleep time as in #2281.
In the pose #2637, there is this explanation about INTRA and INTER camera sync if I was not clear above:
"Furthermore, since I'm unable to set any flags on RGB, how does the RGB image sensor get hardware synced?
As far as I know D415 allows Depth+Color sync to another Depth+Color. D435 allows Depth to Color sync, and Depth to another Depth sync, but not Color to another Color, so at least one (or both) of the color sensors will not be synced. Perhaps firmware / hardware people will be able to find a solution in future update, but for now it is a limitation. For the D415, setting option on the depth sensor will also propagate to color (assuming regular checklist for sync is green - AE Priority Off and same FPS)"
Thank you for your help!

I solved the problem thanks to  #2281 by adding some hardware_reset and sleep(3) before to setup the sensor. It seems that without the hardware reset, it is not possible to force the depth stream as master. I have now the same as in the realsense-viewer GUI.


fix: fix error check lint errors

Please answer these questions before submitting a pull request

Why submit this pull request?

[X] Bugfix

[ ] New feature provided

[ ] Improve performance

[ ] Backport patches

Related issues
#202

Codecov Report

Merging #204 (b59ed06) into master (f0ed89f) will increase coverage by 0.08%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #204      +/-   ##
==========================================
+ Coverage   47.68%   47.77%   +0.08%
==========================================
  Files          32       32
  Lines        1883     1884       +1
==========================================
+ Hits          898      900       +2
+ Misses        846      845       -1
  Partials      139      139

Impacted Files
Coverage Δ

pkg/seven/state/service_worker.go
0.00% <0.00%> (ø)

pkg/seven/state/solver.go
3.96% <0.00%> (+1.98%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0ed89f...b59ed06. Read the comment docs.

Codecov Report

Merging #204 (b59ed06) into master (f0ed89f) will increase coverage by 0.08%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #204      +/-   ##
==========================================
+ Coverage   47.68%   47.77%   +0.08%
==========================================
  Files          32       32
  Lines        1883     1884       +1
==========================================
+ Hits          898      900       +2
+ Misses        846      845       -1
  Partials      139      139

Impacted Files
Coverage Δ

pkg/seven/state/service_worker.go
0.00% <0.00%> (ø)

pkg/seven/state/solver.go
3.96% <0.00%> (+1.98%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0ed89f...b59ed06. Read the comment docs.

I think Makefile already includes "make lint"
https://github.com/apache/apisix-ingress-controller/blob/master/Makefile#L49-L52
lint:
	golangci-lint run

@tokers PTAL, thanks!

I think Makefile already includes "make lint"
https://github.com/apache/apisix-ingress-controller/blob/master/Makefile#L49-L52
lint:
	golangci-lint run

@tokers PTAL, thanks!

@lianghao208 Well done!

@lianghao208 Well done!


Add colouring (brown) for function/proc names

Where declared, and where used.

Implemented for using a function in an expression, but not yet for the name definition (for func or proc)


Maintainers

Why are these changes needed?
this PR is for appending and reviewing maintainers list. if you are a maintainer and not in the list, please make a commit to the PR by adding your details in the same format.
Thanks

Related issue number

Checks

[ ] I've included any doc changes needed for https://autogen-ai.github.io/autogen/. See https://autogen-ai.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ ] I've made sure all auto checks have passed.

@marklysze can you re-approve this tomorrow ( 20-10-24 ), to add this PR to the merge queue.
I'm waiting on any pending maintainers to append to the list.


Figure out how to use docker hub github thing

https://github.com/features/package-registry

Note: I signed up for the beta and I'm waiting for github to respond.

This is now active. See https://help.github.com/en/articles/about-github-package-registry for documentation.
The upside is that we can push docker images to github. The downside is that the images automatically create git tags, which isn't really what we want.


0.11 pods state unknown then minion unavailable

I had managed produce the issue with GCE Container Engine and also had same issue then deployed Kubernetes cluster using provided script. In case one minion is down (in my tests I stopped the vm) then previously deployed containers are not rescheduled to other minions and state of container is "unknown".
In case pod with unknown state is deleted manually, then pod is scheduled to be deployed on other available minion.
If the minion was started, then all pods with unknown state were started on that minion as well.
In my test cluster I tried to set up redundancy for redis-master service and tried to see if self healing works then vm should not be available.

Thanks for reporting.  How did you stop the vm, just shutting it down?  Have you deleted it explicitly using kubectl delete?
nodecontroller will delete the pods in the node after 5min.  There might be some problem deleting the pods.

Yes, I shut it down using stop functionality from gce command line or from web console. After 30 minutes the pod's state was still unknown on the same host.
For deletion I have used kubectl delete then I deployed my own kubernetes master or then I used google gce preview then:
gcloud preview container pods delete

I can reproduce with Vagrant VMs.

Looks like a namespace issue #4832
Can you verify it works for you?  The stopped node will be detected immediately, but pods on it will only be deleted after 5min.  You can control the time using flag pod_eviction_timeout.
Deleting the node through kubectl delete will leave the pods unknown (if you don't stop the node first).  We should improve this.

I tried the latest master and verified that the related fix was included. Then I rebuild and set up Kubernetes cluster in vagrant, then unfortunately it didn't work. The interesting thing was then I queried kubectl get events, then I noticed that new pod was rescheduled to same node, even then the node was down.
The first schedule, then node was up:
Thu, 26 Feb 2015 22:20:32 +0000   Thu, 26 Feb 2015 22:20:32 +0000   1                   redis-master-controller-gziey   Pod                                                     scheduled           {scheduler }           Successfully assigned redis-master-controller-gziey to <IP_ADDRESS>
The second schedule, then node was down:
Thu, 26 Feb 2015 22:24:47 +0000   Thu, 26 Feb 2015 22:24:47 +0000   1                   redis-master-controller-oh43e   Pod                                                     scheduled           {scheduler }           Successfully assigned redis-master-controller-oh43e to <IP_ADDRESS>
But the result is the I query pods then status for redis-master:
redis-master-controller-oh43e                       redis-master        dockerfile/redis                   <IP_ADDRESS>/             app=redis,name=redis-master                           Unknown

That's odd.  Not sure what's going on here. I failed to create a vagrant cluster myself, posted issue here #4302.  So I'll need more information from your side.
Can you post all events and logs from kube-controller-manager and kube-apiserver?  It would be useful if you can also post the sequence of commands (actions) you've used.  Thanks!

The issue is not feasible now, closing.  Feel free to reopen if it happends again.


"Error connecting to server" when running dbt 1.5.2 from databricks workflow

Describe the bug
As of this morning, when running dbt from our databricks worklfows we are getting failures which seems to be related to connection issues. More specifically it looks like this:

and when viewing the query in the databricks query history we get the below error:
Query could not be scheduled: BAD_REQUEST: Invalid SessionHandle: Session [01ee05f4-c035-152f-9002-79f97bbe0a89] is closed.
Steps To Reproduce
Run dbt-databricks from a databricks workflow using version 1.5.2
Expected behavior
For dbt to be able to run the query against the Databricks sql endpoint
Screenshots and log output

System information
The output of dbt --version:

The operating system you're using:
Databricks job-cluster
The output of python --version:
python version: 3.9.5
Additional context
This is a great package! Let me know if I can provide any more details.

As of today we're getting the same errors.

We're also getting them now with older versions of dbt and dbt-databricks.

We did a few more tests and concluded the following; dbt-databricks is always grabbing the latest and greatest of the databricks-sql-connector, even when we downgrade dbt-databricks. So, what we did was pin the databricks-sql-connector version at 2.5.2 straight in the requirements.txt file. And voila, no more problems!
From this, it seems pretty clear that the hiccup was hiding in the databricks-sql-connector package, especially since it was just updated last night.

Great thank you! Are you able to pin the databricks-sql-connector version inside the datarbicks job as well?

Great thank you! Are you able to pin the databricks-sql-connector version inside the datarbicks job as well?

Yes this is possible!
Within your databricks job configure a new dependent library like this:

Seems to work, thank you very much!

Fix has been published. Version 1.5.3. Thanks again for the issue.


Discord Webhook or Rss Feed

a Discord Webhook or a
Rss feed are a nice idea for this changelog?

I'm agree that coule be a great idea !

Support for webhooks has not been added?🥲

As you can see not yet, but the plugin is open source, if you need this you can make a pull request or find someone who can do it


[DUPLICATE] [editor]: Support ttf in preview

Please add support for .ttf files to the editor preview.
Generated by adamraichu.zip-viewer

Closed as duplicate.


merkle_tree_root may not be constrained

generate_r1cs_witness takes a merkle_tree_root but doesn't do anything with it. This makes me wonder if it's actually constrained by anything?

Yes, the Merkle tree root is constrained as follows:

it is part of the circuit input: https://github.com/Zerocash/libzerocash/blob/master/zerocash_pour_ppzksnark/zerocash_pour_gadget.tcc#L60 and https://github.com/Zerocash/libzerocash/blob/master/zerocash_pour_ppzksnark/zerocash_pour_gadget.tcc#L447 ; and
we generate constraints checking that each of the old coins is in a Merkle tree with root merkle_tree_root: https://github.com/Zerocash/libzerocash/blob/master/zerocash_pour_ppzksnark/zerocash_pour_gadget.tcc#L262 and https://github.com/Zerocash/libzerocash/blob/master/zerocash_pour_ppzksnark/zerocash_pour_gadget.tcc#L279 .

In the witness map, the Merkle tree root can be recomputed given leaf old_coin_commitment_variables[i] and the corresponding Merkle tree authentication path old_coin_authentication_paths[i] (consisting of O(log n) auxiliary hash values) by simply hashing bottom-up. In fact, this is what happens in generate_r1cs_witness: the Merkle tree root is recomputed inside.
When testing libzerocash, we had code that checked if the externally provided merkle_tree_root matches the one computed "inside" from old_coin_commitment_variables[i] and old_coin_authentication_paths[i]. When this debugging aid was removed, the interface still retained the "redundant" root.
Thanks for the good catch! I'll add back the checks as they might be useful for people who want to modify the internals of the Pour gadget.