etcdserver: publish error: etcdserver: request timed out

Hi,
I am running ectd from a docker container. But I get the following errors when I try to bring up a two node cluster. The two nodes are not within the same private network. One is on Google Compute Engine and other is on Digital Ocean.
Did I do something wrong?
I set HostIP as the nodes public ip address.
docker run -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379
--name etcd quay.io/coreos/etcd:v2.0.9
-name etcd1
-election-timeout 500000
-heartbeat-interval 100000
-advertise-client-urls http://${HostIP}:2379,http://${HostIP}:4001
-listen-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
-initial-advertise-peer-urls http://${HostIP}:2380
-listen-peer-urls http://<IP_ADDRESS>:2380
-initial-cluster-token etcd-cluster-1
-initial-cluster etcd0=http://<IP_ADDRESS>:2380,etcd1=http://${HostIP}:2380
-initial-cluster-state new
2015/04/16 14:45:24 etcd: no data-dir provided, using default data-dir ./etcd1.etcd
2015/04/16 14:45:24 etcd: listening for peers on http://<IP_ADDRESS>:2380
2015/04/16 14:45:24 etcd: listening for client requests on http://<IP_ADDRESS>:2379
2015/04/16 14:45:24 etcd: listening for client requests on http://<IP_ADDRESS>:4001
2015/04/16 14:45:24 etcdserver: datadir is valid for the 2.0.1 format
2015/04/16 14:45:24 etcdserver: name = etcd1
2015/04/16 14:45:24 etcdserver: data dir = etcd1.etcd
2015/04/16 14:45:24 etcdserver: member dir = etcd1.etcd/member
2015/04/16 14:45:24 etcdserver: heartbeat = 100000ms
2015/04/16 14:45:24 etcdserver: election = 500000ms
2015/04/16 14:45:24 etcdserver: snapshot count = 10000
2015/04/16 14:45:24 etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
2015/04/16 14:45:24 etcdserver: initial advertise peer URLs = http://<IP_ADDRESS>:2380
2015/04/16 14:45:24 etcdserver: initial cluster = etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380
2015/04/16 14:45:24 etcdserver: start member 6af8416381534b54 in cluster 7d056951ac52f880
2015/04/16 14:45:24 raft: 6af8416381534b54 became follower at term 0
2015/04/16 14:45:24 raft: newRaft 6af8416381534b54 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2015/04/16 14:45:24 raft: 6af8416381534b54 became follower at term 1
2015/04/16 14:45:24 etcdserver: added member af525821409e335 [http://<IP_ADDRESS>:2380] to cluster 7d056951ac52f880
2015/04/16 14:45:24 etcdserver: added local member 6af8416381534b54 [http://<IP_ADDRESS>:2380] to cluster 7d056951ac52f880
2015/04/16 14:45:29 etcdserver: publish error: etcdserver: request timed out
2015/04/16 14:45:34 etcdserver: publish error: etcdserver: request timed out

what is the configuration of etcd0? It seems they cannot reach each other, and no leader is elected

It happens because two members cannot reach each other.
Could you confirm that the two containers can talk using its IP?
Moreover, I see your heartbeat is 100s and election is 500s, which is rather high. The high value may cause some timeout on follower in 2.0 version(we have fixed it in master). Could you adjust it to 1s and 5s?

Both nodes can ping each other.  Here are their configs. Do I need to have a minimum of 3 nodes?
ectd0 config
    docker run -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379 \
    --name etcd quay.io/coreos/etcd:v2.0.9 \
    -name etcd0 \
    -election-timeout 5000 \
    -heartbeat-interval 1000 \
    -advertise-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
    -listen-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
    -initial-advertise-peer-urls http://<IP_ADDRESS>:2380 \
    -listen-peer-urls http://<IP_ADDRESS>:2380 \
    -initial-cluster-token etcd-cluster-1 \
    -initial-cluster etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380  \

ectd0 log
2015/04/17 01:43:25 etcd: no data-dir provided, using default data-dir ./etcd0.etcd
2015/04/17 01:43:25 etcd: listening for peers on http://<IP_ADDRESS>:2380
2015/04/17 01:43:25 etcd: listening for client requests on http://<IP_ADDRESS>:2379
2015/04/17 01:43:25 etcd: listening for client requests on http://<IP_ADDRESS>:4001
2015/04/17 01:43:25 etcdserver: datadir is valid for the 2.0.1 format
2015/04/17 01:43:26 etcdserver: name = etcd0
2015/04/17 01:43:26 etcdserver: data dir = etcd0.etcd
2015/04/17 01:43:26 etcdserver: member dir = etcd0.etcd/member
2015/04/17 01:43:26 etcdserver: heartbeat = 1000ms
2015/04/17 01:43:26 etcdserver: election = 5000ms
2015/04/17 01:43:26 etcdserver: snapshot count = 10000
2015/04/17 01:43:26 etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
2015/04/17 01:43:26 etcdserver: initial advertise peer URLs = http://<IP_ADDRESS>:2380
2015/04/17 01:43:26 etcdserver: initial cluster = etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380
2015/04/17 01:43:26 etcdserver: start member af525821409e335 in cluster 7d056951ac52f880
2015/04/17 01:43:26 raft: af525821409e335 became follower at term 0
2015/04/17 01:43:26 raft: newRaft af525821409e335 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2015/04/17 01:43:26 raft: af525821409e335 became follower at term 1
2015/04/17 01:43:26 etcdserver: added local member af525821409e335 [http://<IP_ADDRESS>:2380] to cluster 7d056951ac52f880
2015/04/17 01:43:26 etcdserver: added member 6af8416381534b54 [http://<IP_ADDRESS>:2380] to cluster 7d056951ac52f880
2015/04/17 01:43:31 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:32 raft: af525821409e335 is starting a new election at term 1
2015/04/17 01:43:32 raft: af525821409e335 became candidate at term 2
2015/04/17 01:43:32 raft: af525821409e335 received vote from af525821409e335 at term 2
2015/04/17 01:43:32 raft: af525821409e335 [logterm: 1, index: 2] sent vote request to 6af8416381534b54 at term 2
2015/04/17 01:43:36 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:38 raft: af525821409e335 [logterm: 1, index: 2, vote: af525821409e335] rejected vote from 6af8416381534b54 [logterm: 1, index: 2] at term 2
2015/04/17 01:43:39 raft: af525821409e335 is starting a new election at term 2
2015/04/17 01:43:39 raft: af525821409e335 became candidate at term 3
2015/04/17 01:43:39 raft: af525821409e335 received vote from af525821409e335 at term 3
2015/04/17 01:43:39 raft: af525821409e335 [logterm: 1, index: 2] sent vote request to 6af8416381534b54 at term 3
2015/04/17 01:43:39 etcdhttp: unexpected error: etcdserver: request timed out
2015/04/17 01:43:40 etcdhttp: unexpected error: etcdserver: request timed out
2015/04/17 01:43:41 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:46 raft: af525821409e335 is starting a new election at term 3
2015/04/17 01:43:46 raft: af525821409e335 became candidate at term 4
2015/04/17 01:43:46 raft: af525821409e335 received vote from af525821409e335 at term 4
2015/04/17 01:43:46 raft: af525821409e335 [logterm: 1, index: 2] sent vote request to 6af8416381534b54 at term 4
2015/04/17 01:43:46 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:46 raft: af525821409e335 [term: 4] ignored a MsgVote message with lower term from 6af8416381534b54 [term: 3]
2015/04/17 01:43:50 received terminated signal, shutting down...
2015/04/17 01:43:51 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:56 etcdserver: publish error: etcdserver: request timed out

etcd1
docker run -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379 \
 --name etcd quay.io/coreos/etcd:v2.0.9 \
 -name etcd1 \
 -election-timeout 5000\
 -heartbeat-interval 1000 \
 -advertise-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
 -listen-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
 -initial-advertise-peer-urls http://<IP_ADDRESS>:2380 \
 -listen-peer-urls http://<IP_ADDRESS>:2380 \
 -initial-cluster-token etcd-cluster-1 \
 -initial-cluster etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380 \
 -initial-cluster-state new

etcd1 log
2015/04/17 01:43:30 etcd: no data-dir provided, using default data-dir ./etcd1.etcd
2015/04/17 01:43:30 etcd: listening for peers on http://<IP_ADDRESS>:2380
2015/04/17 01:43:30 etcd: listening for client requests on http://<IP_ADDRESS>:2379
2015/04/17 01:43:30 etcd: listening for client requests on http://<IP_ADDRESS>:4001
2015/04/17 01:43:30 etcdserver: datadir is valid for the 2.0.1 format
2015/04/17 01:43:30 etcdserver: name = etcd1
2015/04/17 01:43:30 etcdserver: data dir = etcd1.etcd
2015/04/17 01:43:30 etcdserver: member dir = etcd1.etcd/member
2015/04/17 01:43:30 etcdserver: heartbeat = 1000ms
2015/04/17 01:43:30 etcdserver: election = 5000ms
2015/04/17 01:43:30 etcdserver: snapshot count = 10000
2015/04/17 01:43:30 etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
2015/04/17 01:43:30 etcdserver: initial advertise peer URLs = http://<IP_ADDRESS>:2380
2015/04/17 01:43:30 etcdserver: initial cluster = etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380
2015/04/17 01:43:30 etcdserver: start member 6af8416381534b54 in cluster 7d056951ac52f880
2015/04/17 01:43:30 raft: 6af8416381534b54 became follower at term 0
2015/04/17 01:43:30 raft: newRaft 6af8416381534b54 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2015/04/17 01:43:30 raft: 6af8416381534b54 became follower at term 1
2015/04/17 01:43:30 etcdserver: added member af525821409e335 [http://<IP_ADDRESS>:2380] to cluster 7d056951ac52f880
2015/04/17 01:43:30 etcdserver: added local member 6af8416381534b54 [http://<IP_ADDRESS>:2380] to cluster 7d056951ac52f880
2015/04/17 01:43:35 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:38 raft: 6af8416381534b54 is starting a new election at term 1
2015/04/17 01:43:38 raft: 6af8416381534b54 became candidate at term 2
2015/04/17 01:43:38 raft: 6af8416381534b54 received vote from 6af8416381534b54 at term 2
2015/04/17 01:43:38 raft: 6af8416381534b54 [logterm: 1, index: 2] sent vote request to af525821409e335 at term 2
2015/04/17 01:43:40 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:45 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:46 raft: 6af8416381534b54 is starting a new election at term 2
2015/04/17 01:43:46 raft: 6af8416381534b54 became candidate at term 3
2015/04/17 01:43:46 raft: 6af8416381534b54 received vote from 6af8416381534b54 at term 3
2015/04/17 01:43:46 raft: 6af8416381534b54 [logterm: 1, index: 2] sent vote request to af525821409e335 at term 3
2015/04/17 01:43:50 etcdserver: publish error: etcdserver: request timed out
2015/04/17 01:43:50 etcdhttp: unexpected error: etcdserver: request timed out
^C2015/04/17 01:43:51 received interrupt signal, shutting down...

I think the network between these nodes is really unstable. etcd0 receives one of three vote messages and etcd1 receives no vote messages. Could you stat your network condition (etcd0 -> etcd1:2380 & etcd1 -> etcd0:2380)?

Is ping enough?

the stats should include the latency, throughput, miss rate of sending tcp messages to 2380 port on two machines.

Can you kindly tell me how to get those stats?

Hi,
I have change the two nodes to Google Compute. But I get the same error.
etcdserver: publish error: etcdserver: request timed out
When I change the ip address on both nodes to private ip address, the cluster forms correctly.

etcd 0 config
docker run -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379 \
 --name etcd quay.io/coreos/etcd:v2.0.9 \
 -name etcd0 \
 -election-timeout 5000\
 -heartbeat-interval 1000 \
 -advertise-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
 -listen-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
 -initial-advertise-peer-urls http://<IP_ADDRESS>:2380 \
 -listen-peer-urls http://<IP_ADDRESS>:2380 \
 -initial-cluster-token etcd-cluster-1 \
 -initial-cluster etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380 \
 -initial-cluster-state new

etcd0
2015/04/17 07:32:42 etcd: no data-dir provided, using default data-dir ./etcd0.etcd
2015/04/17 07:32:42 etcd: listening for peers on http://<IP_ADDRESS>:2380
2015/04/17 07:32:42 etcd: listening for client requests on http://<IP_ADDRESS>:2379
2015/04/17 07:32:42 etcd: listening for client requests on http://<IP_ADDRESS>:4001
2015/04/17 07:32:42 etcdserver: datadir is valid for the 2.0.1 format
2015/04/17 07:32:43 etcdserver: name = etcd0
2015/04/17 07:32:43 etcdserver: data dir = etcd0.etcd
2015/04/17 07:32:43 etcdserver: member dir = etcd0.etcd/member
2015/04/17 07:32:43 etcdserver: heartbeat = 1000ms
2015/04/17 07:32:43 etcdserver: election = 5000ms
2015/04/17 07:32:43 etcdserver: snapshot count = 10000
2015/04/17 07:32:43 etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
2015/04/17 07:32:43 etcdserver: initial advertise peer URLs = http://<IP_ADDRESS>:2380
2015/04/17 07:32:43 etcdserver: initial cluster = etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380
2015/04/17 07:32:43 etcdserver: start member 7ef0e4b82ead0f5b in cluster ad99329e439954cb
2015/04/17 07:32:43 raft: 7ef0e4b82ead0f5b became follower at term 0
2015/04/17 07:32:43 raft: newRaft 7ef0e4b82ead0f5b [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2015/04/17 07:32:43 raft: 7ef0e4b82ead0f5b became follower at term 1
2015/04/17 07:32:43 etcdserver: added member 6af8416381534b54 [http://<IP_ADDRESS>:2380] to cluster ad99329e439954cb
2015/04/17 07:32:43 etcdserver: added local member 7ef0e4b82ead0f5b [http://<IP_ADDRESS>:2380] to cluster ad99329e439954cb
2015/04/17 07:32:48 etcdserver: publish error: etcdserver: request timed out
2015/04/17 07:32:49 etcdhttp: unexpected error: etcdserver: request timed out
2015/04/17 07:32:49 etcdhttp: unexpected error: etcdserver: request timed out
2015/04/17 07:32:52 raft: 7ef0e4b82ead0f5b is starting a new election at term 1
2015/04/17 07:32:52 raft: 7ef0e4b82ead0f5b became candidate at term 2
2015/04/17 07:32:52 raft: 7ef0e4b82ead0f5b received vote from 7ef0e4b82ead0f5b at term 2
2015/04/17 07:32:52 raft: 7ef0e4b82ead0f5b [logterm: 1, index: 2] sent vote request to 6af8416381534b54 at term 2
2015/04/17 07:32:53 etcdserver: publish error: etcdserver: request timed out
2015/04/17 07:32:58 etcdserver: publish error: etcdserver: request timed out
2015/04/17 07:32:59 raft: 7ef0e4b82ead0f5b is starting a new election at term 2
2015/04/17 07:32:59 raft: 7ef0e4b82ead0f5b became candidate at term 3
2015/04/17 07:32:59 raft: 7ef0e4b82ead0f5b received vote from 7ef0e4b82ead0f5b at term 3
2015/04/17 07:32:59 raft: 7ef0e4b82ead0f5b [logterm: 1, index: 2] sent vote request to 6af8416381534b54 at term 3

etcd 1 config
docker run -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379 \
 --name etcd quay.io/coreos/etcd:v2.0.9 \
 -name etcd1 \
 -election-timeout 5000\
 -heartbeat-interval 1000 \
 -advertise-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
 -listen-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 \
 -initial-advertise-peer-urls http://<IP_ADDRESS>:2380 \
 -listen-peer-urls http://<IP_ADDRESS>:2380 \
 -initial-cluster-token etcd-cluster-1 \
 -initial-cluster etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380 \
 -initial-cluster-state new

** etcd 1 log **
2015/04/17 07:32:45 etcd: no data-dir provided, using default data-dir ./etcd1.etcd
2015/04/17 07:32:45 etcd: listening for peers on http://<IP_ADDRESS>:2380
2015/04/17 07:32:45 etcd: listening for client requests on http://<IP_ADDRESS>:2379
2015/04/17 07:32:45 etcd: listening for client requests on http://<IP_ADDRESS>:4001
2015/04/17 07:32:45 etcdserver: datadir is valid for the 2.0.1 format
2015/04/17 07:32:46 etcdserver: name = etcd1
2015/04/17 07:32:46 etcdserver: data dir = etcd1.etcd
2015/04/17 07:32:46 etcdserver: member dir = etcd1.etcd/member
2015/04/17 07:32:46 etcdserver: heartbeat = 1000ms
2015/04/17 07:32:46 etcdserver: election = 5000ms
2015/04/17 07:32:46 etcdserver: snapshot count = 10000
2015/04/17 07:32:46 etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
2015/04/17 07:32:46 etcdserver: initial advertise peer URLs = http://<IP_ADDRESS>:2380
2015/04/17 07:32:46 etcdserver: initial cluster = etcd0=http://<IP_ADDRESS>:2380,etcd1=http://<IP_ADDRESS>:2380
2015/04/17 07:32:46 etcdserver: start member 6af8416381534b54 in cluster ad99329e439954cb
2015/04/17 07:32:46 raft: 6af8416381534b54 became follower at term 0
2015/04/17 07:32:46 raft: newRaft 6af8416381534b54 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2015/04/17 07:32:46 raft: 6af8416381534b54 became follower at term 1
2015/04/17 07:32:46 etcdserver: added local member 6af8416381534b54 [http://<IP_ADDRESS>:2380] to cluster ad99329e439954cb
2015/04/17 07:32:46 etcdserver: added member 7ef0e4b82ead0f5b [http://<IP_ADDRESS>:2380] to cluster ad99329e439954cb
2015/04/17 07:32:51 etcdserver: publish error: etcdserver: request timed out
2015/04/17 07:32:54 raft: 6af8416381534b54 is starting a new election at term 1
2015/04/17 07:32:54 raft: 6af8416381534b54 became candidate at term 2
2015/04/17 07:32:54 raft: 6af8416381534b54 received vote from 6af8416381534b54 at term 2
2015/04/17 07:32:54 raft: 6af8416381534b54 [logterm: 1, index: 2] sent vote request to 7ef0e4b82ead0f5b at term 2

I don't know why, but the same config worked on digital ocean. I setup two nodes on different data centre. The same etcd config worked. Something wrong with Google Compute.

met the same error, but it turns running when I run all the 3 instances, it need 3 nodes all started, or it can not connect to others and report this error.
It is strange....


Assign Funding Stage

As a user I can assign funding stages to startup investments so that I can track their growth and funding needs accurately.

reviewed and merged.


Add integration tests

Integration tests for HTTP, Unix Domain Socket transports.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 83.99%. Comparing base (9abc3ea) to head (014caf1).
Report is 6 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #90      +/-   ##
==========================================
+ Coverage   79.73%   83.99%   +4.26%
==========================================
  Files          31       29       -2
  Lines        1283     1231      -52
  Branches      166      161       -5
==========================================
+ Hits         1023     1034      +11
+ Misses        252      190      -62
+ Partials        8        7       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Cluster Metadata - Long Term Plan

There exists two primary ways of agreeing on metadata within a Riak cluster.

The ring;

For non-typed buckets, their properties are stored on the ring.
Otherwise the ring is reserved for information about the allocation of partitions within the cluster (and current known status of cluster members).
The ring is gossiped (using riak_core_gossip) and persisted on changes,  and conflicting changes are merged.
Although changes to the cluster membership are made through the claimant, changes to bucket properties bypass the claimant and are made directly.
There may be potential issues with changing bucket properties when planning cluster membership changes
When the ring is updated, the bucket properties are flushed and re-written to an ETS table from where they are read.
After 90s without a ring change, the latest ring has read access optimised using riak_core_mochiglobal (essentially compiling it as a module) - but requests for bucket properties only ever use the ETS table.
Each node sends its ring to another random node every 60s, even if there are no changes, so eveyr node should eventually find out about a new version of the ring, even if the node was down when a ring change was gossiped.

Cluster metadata;

For typed buckets and bucket types, their properties are  stored in cluster metadata.
Configuration for riak_core_security is also stored in cluster metadata.
There are no other uses for cluster metadata.
Cluster metadata is entirely separate to the ring.  It is gossiped using riak_core_broadcast, and stored within a DETS file, with a read-only version promoted to an ETS table.
Reading bucket properties for typed/non-typed buckets follows two different code paths (as one is a read from the ring, and one a read from cluster metadata) - but code paths ultimately lead to an ets:lookup/2 (although in the case of a typed bucket two lookups are required).
If cluster metadata gets out of sync between nodes, the differences are detected by active anti-entropy (using the riak_core_metadata_hashtree module, which then uses hashtree and eleveldb).
Although typed bucket property changes do not use the ring, they are channeled via the claimant node.

Some notes:

Both the use of an ets table as a read-only cache, and the use of riak_core_mochiglobal seem to anachronistic in the context of erlang persistent_term.
The most obvious change is to replace the use of riak_core_mochiglobal with erlang persistent_term for storing the ring (and the ring epoch).  There is already the delayed protection of promote_ring to prevent over-frequent updates.
If riak_core_security is not used, and buckets are either generally un-typed, or limited in number - the riak_core_metadata approach seems excessive.
The riak_core_metadata was added for a reason though - is riak_core_metadata/riak_core_broadcast a more robust and scalable approach than riak_core_ring/riak_core_gossip?
The need for AAE in riak_core_metadata creates a dependency on eleveldb.  eleveldb itself can be removed, but that doesn't eliminate the complexity of the process.  Also, this PR changes the underlying store for hashtree - and so has an impact on those using AAE for KV anti-entropy.
Riak core metadata is subject to hard limits with the 2GB maximum size of DETS table files.
The need for AAE is to ensure eventual consistency (if a node was down and missed broadcasts, it must discover the discrepancy via AAE), but given a 2GB maximum size, perhaps simpler methods than hashtree can be used for comparisons e.g. iterate over keys and comparing directly.
There is the additional complexity of fixups (which is currently only used by riak_repl); fixups allow an application to register a need to filter all bucket property changes before they are applied (in the case of riak_repl this is used to ensure that the riak_repl post-commit hook is applied on every bucket).
The net effect of the different methods for holding cluster metadata is confusing.
There are no known issues with the current setup; it may be confusing, but it works.

Should there be a long-term plan to change this?  To unify or simplify the methods of storing and gossiping information required across the cluster?  Is there sufficient potential reward to justify the risk of any change (particularly wrt to the overheads of testing backwards compatibility).

See also https://github.com/nhs-riak/riak_core/issues/5

Thank you for putting this together. I had some similar thoughts when working around some of this code.
cluster_metadata is an implementation of Plumtree. The riak_core_gossip creates a binary tree and messages a node's immediate children. Plumtree came much later and I'm not sure what initiated the change in gossip algorithm but I'd be interested in benchmarking how Plumtree performs managing the ring.
I'm for keeping cluster_metadata or at least the idea of it. I like keeping the ring small, safe and updated infrequently. I'd even look to move some fields in meta over to cluster_metadata, like default bucket-types. Using the ring as an ad-hoc store means that business logic and usage can get in the way of a core component in Riak. If the ring and metadata share much of the same implementation, then it won't feel overkill.
As an aside, I'm curious how other Riak users use bucket-types and whether we can just remove the CLI and move it to riak.conf.


Failing Test - "Should not return bookmarks for a nonexistent user"

Failing test in testBookmarksAndUserProfile.js
See details below:

Missing check in DAO to ensure that user exists before attempting to access user fields.
recipes.controller was missing try-catch block to catch any errors

Resolved in pull request #35


[dvsim] Force an ordering when constructing command in Deploy object

A Deploy object has an effect by running some command. This command is
constructed as a string in Deploy.construct_cmd and is of the form
make -f ... A=1 B=2 C=3

where the A, B, C parameters are found by iterating over
"mandatory command attributes". These are stored in a dict. We don't
really care about the exact ordering, but it does need to be
consistent (and not depend on Dict's internal hashing).
One way to do this is to make mandatory_cmd_attrs an OrderedDict, but
there are several definitions of mandatory_cmd_attrs, which all need
to be updated correctly. Since we don't care about the order, and only
care that it's consistent, it seems simpler just to iterate over the
attributes in sorted order.
This is related to PR #3111. The inconsistent ordering from iterating over a Dict caused problems there, but I think this needs fixing either way: even if Make doesn't care about the order we give it parameter values, it's rather cleaner to do it the same every time.
I think this approach is a bit simpler than using an OrderedDict as proposed there and moves the job of forcing an ordering to just one line.

Sri wants to do this a different way.


dependabot-updates

Handles dependabot PRs

https://github.com/Esri/solutions-components/pull/733
https://github.com/Esri/solutions-components/pull/734
https://github.com/Esri/solutions-components/pull/735
https://github.com/Esri/solutions-components/pull/736

Does not handle dependabot PR https://github.com/Esri/solutions-components/pull/737 because of type issues. Changed package version of arcgis-js-api to "~4.29.10" so that we only automatically get patch version updates and don't automatically get 4.30.x.
Problems with updating arcgis-js-api to "4.30.8":
[ ERROR ]  TypeScript: src/components/edit-card/edit-card.tsx:329:16
           Conversion of type '{ layer: __esri.Layer;
           geometryUpdatesEnabled: boolean; }' to type
           'LayerInfo' may be a mistake because neither type
           sufficiently overlaps with the other. If this was
           intentional, convert the expression to 'unknown'
           first.Type '{ layer: Layer; geometryUpdatesEnabled:
           boolean; }' is missing the following properties from
           type 'LayerInfo': id, name, type, url

    L328:  const layerInfos = layers.map(layer => {
    L329:    return {
    L330:      layer,

[ ERROR ]  TypeScript: src/components/edit-card/edit-card.tsx:337:9
           Type 'LayerInfo[]' is not assignable to type
           'EditorLayerInfo[]'.Property 'layer' is missing in
           type 'LayerInfo' but required in type
           'EditorLayerInfo'.

    L336:  view: this.mapView,
    L337:  layerInfos,
    L338:  visibleElements: {

[ ERROR ]  TypeScript: src/components/layer-table/layer-table.tsx:1528:5
           Type '(string | number)[]' is not assignable to type
           'number[]'.Type 'string | number' is not assignable
           to type 'number'.Type 'string' is not assignable to
           type 'number'.

   L1527:  protected _getIds(): number[] {
   L1528:    return this._table.highlightIds.toArray();
   L1529:  }

[ ERROR ]  TypeScript: src/components/layer-table/layer-table.tsx:1589:90
           Argument of type 'string | number' is not assignable
           to parameter of type 'number'.Type 'string' is not
           assignable to type 'number'.

   L1588:  const reAdd = this.selectedIds.length > 1 && evt.removed.length === 1;
   L1589:   ids.filter(id => this.selectedIds.indexOf(id) < 0);
   L1590:  this._clearSelection();

[ ERROR ]  TypeScript: src/components/layer-table/layer-table.tsx:1599:11
           Type '(string | number)[]' is not assignable to type
           'number[]'.

   L1598:    // https://github.com/Esri/solutions-components/issues/365
   L1599:    this.selectedIds = ids.reverse();
   L1600:  }

[ ERROR ]  TypeScript: src/components/layer-table/layer-table.tsx:1606:9
           Type 'string | number' is not assignable to type
           'number'.Type 'string' is not assignable to type
           'number'.

   L1605:  this._previousCurrentId = this._currentId;
   L1606:  this._currentId = [...ids].reverse()[0];
   L1607:  if (ids.length === 1) {

[ ERROR ]  TypeScript: src/components/public-notification/public-notification.tsx:329:38
           Property 'autoOpenEnabled' does not exist on type
           'Popup'. Did you mean 'autoCloseEnabled'?

    L328:  if (v?.popup) {
    L329:    this._popupsEnabled = v?.popup.autoOpenEnabled;
    L330:  }

[39:20.1]  build failed in 5.05 s

@MikeTschudi  Why stay at 4.29? Would like to stay up with the latest so when we use in IA apps we don't have new issues popup because they are always at the latest as far as I know.

@MikeTschudi were you thinking we'd work through the errors listed above with an explicit shift to 4.30.x?

@MikeTschudi were you thinking we'd work through the errors listed above with an explicit shift to 4.30.x?

Yes.  I wanted to get the bulk of the dependabot updates through, and then we can work our way through the type changes in 4.30.x so that we can update arcgis-js-api.  I'll leave PR #737 open.


Support Ktor 2.x

Summary
When trying to build a Kotlin Multiplatform project with JS target, I got:
Module "com.apollographql.apollo3:apollo-runtime" has a reference to symbol io.ktor.client.features/HttpTimeout.Feature|null[0]. Neither the module itself nor its dependencies contain such declaration.
If a project uses Ktor 2.0.2 it causes a conflict with the Apollo one that's 1.6.7
Version
3.3.0
Description
In a Kotlin Multiplatform project I got declared:
        val ktor_version = "2.0.2"
        val apollo_version = "3.3.0"

        val commonMain by getting {

            dependencies {
                ...
                implementation("io.ktor:ktor-client-core:$ktor_version")
                implementation("io.ktor:ktor-client-content-negotiation:$ktor_version")
                implementation("io.ktor:ktor-serialization-kotlinx-json:$ktor_version")

                implementation("com.apollographql.apollo3:apollo-api:$apollo_version")
                implementation("com.apollographql.apollo3:apollo-runtime:$apollo_version")
                ...
            }
        }

        val jsMain by getting {

            dependencies {
                ...
                implementation("io.ktor:ktor-client-js:$ktor_version")
                implementation("io.ktor:ktor-http-cio-js:2.0.2")
                ...
            }
        }

Trying to build the module results in Module "com.apollographql.apollo3:apollo-runtime" has a reference to symbol io.ktor.client.features/HttpTimeout.Feature|null[0]. Neither the module itself nor its dependencies contain such declaration.

Thanks a lot for sending this!
Do you know how much change is needed to upgrade to Ktor 2? IIRC we only used Ktor for JS clients so hopefully there isn't a too big impact?

@maribelmai We've updated the dependency, this will be in the next release, in the meantime if you want to try it, you can use the snapshots!

I used the snapshot! looks good 👍
Thanks so much for prioritizing and fixing this one!


Add a new subsection on dealing with unknown information

As discussed in the last monthly meeting, I suggest to add a subsection that explains the use standard values for unknown information.
This PR depends on #4 because it modifies the section that has been moved in this other PR. It has been based on the schema branch to avoid a conflict. Nevertheless, the changes are independent of that PR. It may be rebased onto develop if desired.
Check the result on RtD.

Many thanks @markusstocker for the comments.  I added a change following two of your suggestions.
Note that some of the comments are not related to this PR. I admit that this might be somewhat confusing, but this PR has been made on top of #4, otherwise it would have created a conflict with that other PR. That is why any change from #4 also appears to be a change from this PR as well. #4 in turn has moved a large fraction of the original text into a new section. So this text appears to have been added in #4, but in fact it has been left unchanged. Your comments 1, 3, and 4 relate to text portions that have not been changed in any of my PRs, see my reply above. It would make things easier if we keep the discussions about unrelated suggestions for changes separate.
I'm not able to follow your 2nd comment and I didn't quite understand your last one.
There is one suggestion concerning a formulation detail that may be debatable.

Happy with section 'dealing with unknown information'. Will refer to main co-authors

@markusstocker Happy with the section. Please merge.


Uncaught TypeError: Cannot read property 'select2' of undefined

I'm not really sure how to explain what's happening here, but I'm trying to wrap up my select2 plugin as a reactjs component, the way I do it is I remove all the "" and fill it up when a new dataset is given. When I click the dropdown, I get no error, when I make a new selection, and click the same dropdown to open it again I get the aforementioned error:
Uncaught TypeError: Cannot read property 'select2' of undefined
And I saw that the error is located in this section of the code (found in select.full.js):
             $all.each(function () {
                  var $this = $(this);
                  if (this == $select[0]) {
                      return;
                  }
                  var $element = $this.data('element');
                  $element.select2('close');
              });

If I simply open the dropdown, this == $select[0] is equal to true, and everything is great.. but after I make the selection, and I open the same dropdown, this == $select[0], is false, and $element can't be found. Is it really wise to check the equality of two jquery objects? The functionality on my page is working, and everything seems to be great, but it's just annoying to see this error in my console, when it's not really an error. What are you really checking for here?

I have the problem "Uncaught TypeError: Cannot read property 'select2' of undefined" too,
But I saw that the error is located in this section of the code (found in line 5656 of select.js):
if ($.fn.select2 == null) {
So I add the code before that line
!$.fn && ($.fn = {})
Then the problem solved.

This appears to be a duplicate of #3282.


Bring back Sergio's "Add support for Process stdin stream"

This PR resurrects @sergiocampama's https://github.com/apple/swift-tools-support-core/pull/122, which we required in order to be able to forward standard input in the new swift-driver to the compile jobs it spawns (https://github.com/apple/swift-driver/pull/595).
The change was previously reverted in https://github.com/apple/swift-tools-support-core/pull/143, due to an error seen on Linux.

@swift-ci please test

@swift-ci please test macOS Platform

macOS testing failing with:
15:30:36 [./swift/utils/build-script] ERROR: can't find CMake (please install CMake)

cc @shahmishal

Fixed the PATH issue

@swift-ci please test macOS Platform

Fixed the PATH issue

Thanks!

@sergiocampama @neonichu, Cross-repository tests in https://github.com/apple/swift/pull/36869 all pass, including the Linux smoke tests, which seems to have caused the revert in https://github.com/apple/swift-tools-support-core/pull/143.
Could you please take a look if this is okay to merge? Content-wise, this PR is identical to #122.


Merge changes from collaboration-readme-file into main

I've added the group norms lists, and summaries and i want teammates/ a teammate to review them and merge them

Hello Team
Collaboration is all about bringing different strengths together. When we work as a team, everyone’s unique perspective and skills help us tackle challenges in ways we couldn’t do alone. It’s about trusting each other, communicating openly, and staying focused on the same goal. When we all pull together and support one another, the results can be amazing!
How does that sound? Would you like any changes?😊

The group norms are very well presented and clearly explained.Great job ensuring the norms cover all key aspects of teamwork!


[NPM] Drop empty + preexisting closed connections

What does this PR do?
Currently we maintain a hacky mutex in kernelspace that helps us to prevent tracking the same connection closure twice. The only reason this is needed is to support tracking connection closures that:

we did not observe the handshake for
we did not observe any connection or tcp specific events for
we only saw the closure for.
in this case we flush an empty connection to userspace.

it will save us a lot of code complexity, memory and CPU to stop tracking these events. we can remove the conn_close_flushed map.
Motivation
https://datadoghq.atlassian.net/browse/NPM-3747
Describe how to test/QA your changes
Possible Drawbacks / Trade-offs
Additional Notes

/merge


Implementing your code as is fails

Compilser says that address, command and toggle was not declared in this scope

Have you declared those three variables correctly?
You have to paste the following code right before the if (rc5.read(&toggle, &address, &command)) part:
  unsigned char toggle;
  unsigned char address;
  unsigned char command;

This should solve the problem.
Just look at the example code provided in RC5/examples/RC5Decode/RC5Decode.ino


Ability to have multiple tabs

I should be able to create multiple tabs for multiple queries simultaneously. As much as 10 tabs.

Done in v1.3.0


Update setup_cln.md - simple typo fix

Please add [skip ci] to the commit message of documentation-only commits. That way it skips the resource heavy test suite upon commit. Please git push --force after you've used git commit --amend to add [skip ci] to your commit message.

Please add [skip ci] to the commit message of documentation-only commits. That way it skips the resource heavy test suite upon commit. Please git push --force after you've used git commit --amend to add [skip ci] to your commit message.

Will do - sorry about that!

Good eye! ACK. 667aa4b


Asset creation threading issue.

Description
Asset allocation is currently not thread safe. In practice, I've not seen or heard this being a problem, but in stress tests, it can be consistently reproduced. It's probably one of those annoying things that happens once every 1000 runs.
I believe the core issue is that assets are all added and removed from a single core list when they're created, and this list is not thread safe. Most assets do have thread safety built into their own implementations, but this shared list never accounted for it.

Fixed in #1025


KinematicBody.move_and_collide() affects all KinematicCollisions

Godot version:
Godot_v3.3.1.stable.official
OS/device including version:
Windows 10
64 bit
Issue description:
When move_and_collide returns a KinematicCollision, it always returns the same collision, but changes its values. This causes all collisions to actually be the same object with the same values. This is not intuitive, as it appears that nothing has touched the collisions, but they are all changing.
Steps to reproduce:
Use multiple move_and_collide calls from the same object, make sure they all result in collisions. Check whether all of the returned collisions are the same object.
Check to see step by step. Each variable holds the same object, which is updated by move_and_collide.
Minimal reproduction project:
TestProject.zip

Duplicate of #36290


Fix typo

PR checklist

[ ] Addresses an existing issue: #0000
[ ] New feature, bugfix, or enhancement

[ ] Includes tests

[X] Documentation update
[X] Enable CircleCI for your fork (https://circleci.com/add-projects)

What changes did you make?
Fixed a typo.

Your tests are failing even though your fork appears to pass when merged. I think you might need to clear the cache in CircleCI


chia get_transactions does not return all change transactions, so tx_removals doesn't sum up correctly.

Not sure this project is active anymore but I'm leaving this message as a FYI
I've noticed an issue with chia get_transactions, it doesn't return all the additions/removals. So the script might say you have a transaction with 1.75 xch instead of 8.
Not sure what the purpose of manually adding the individual additions/removals, but commenting out the loop that counted those fixed it for me.

I haven't heard about Chia CLI not returning all the removals properly. Admittedly, I haven't tested it recently.
But for the summing the removals, it is necessary on Outgoing transactions to account for the 'change' back from the transaction. The amount alone can't be used because it includes the change back coins.
For example, if you went to a store and your total was $12, but all you had was two $10 bills, you'd give two $10s to the store and they'd give you back one $5 bill and three $1 bills. If you don't account for the "20" going out, your total would be 4 (12 out minus the 8 change back... which isn't correct. I'm not sure why Chia reports it this way).  You might check some of your Outgoing Transactions to make sure this scenario is accounted for properly.


[enhancement](better method consume) add a TrackExecutionTime annotation to log method exec time

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
add a TrackExecuteTime annotation to log method exec time
Use case
Then we could print a method's exec time by using:
@TrackExecuteTime(threshold=3000, printStack=true)
@Override
public Job pop() {

The threshold is 3000ms, if the execute time > threshold(3000), then should LOG.warn("{} execution {} ms",  method name)
If user set print stack = true, then should also print the method's call stack. Because the method maybe called by many methods the user want to know which method call this method.
Related issues
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I'll do this


Addresses Issue #1055 - External Request Endpoint

Added RequestApiController
Added Tests for controller
Added RequestViewModel
Added RequestCommand components
Added ExternalEndpointAttribute that can be used to avoid
AntoForgeryTokenChecking in tests

Hi Sean, if you add the phrase "Fixes #1055" to the description of the PR, it will automatically close the issue once the PR is merged.

Hey @seanepping you removed WIP, is this good to go?

@tonysurma I believe so.  With the current setup I don't have a means to have the smoke alarm app talk to my local, but they should be able to hit a dev environment?


default parameters for vars

I reinstalled padawan and got a lot of errors about vars not defined also if they are defined.
I saw that in this code there are no default, this patch add a default system for the settings and resolve the issue and now is working again.

In my settings there was the problem that that values wasn't available and I have no idea why.
Anyway this is my settings https://github.com/Mte90/My-Scripts/blob/master/vim/vimrc


Add util flags: gcp-oauth-client-id, gcp-oauth-client-secret

Each GCP Connector that I manage needs to create its own robot account using our system's refresh token, so I need to use the util's handy CLI flags to pass it the options (gcp-user-refresh-token etc) required to create the config file and robot account.
The only flags missing were:

gcp-oauth-client-id
gcp-oauth-client-secret

I've added these and tested it, and can submit a pull request if that's OK.
Cheers,
Nick.

I'm happy to review a pull request.


Implement Pod for tuples

Closes https://github.com/gfx-rs/gfx/issues/1157

Thanks!
@homu r+

:pushpin: Commit 9c1e943 has been approved by kvark

:zap: Test exempted - status


Save user Session on exit

Ask the user if he/she wants to save the session when the application is closed


I am getting this Error while I am trying to connect Cosmosdb with Azure Data bricks using Azure Cosmos DB OLTP Spark 3 connector

I am getting this error while I am trying to connect with cosmos Db from Azure Data Bricks using Azure Cosmos DB OLTP Spark 3 connector
IllegalArgumentException                  Traceback (most recent call last)
 in
10
11 # Connect via azure-cosmosdb-spark to create Spark DataFrame
---> 12 flights = spark.read.format(
13     "cosmos.oltp").options(**readConfig).load()
14 print(flights)
/databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
208             return self._df(self._jreader.load(self._spark._sc._jvm.PythonUtils.toSeq(path)))
209         else:
--> 210             return self._df(self._jreader.load())
211
212     def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,
/databricks/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py in call(self, *args)
1302
1303         answer = self.gateway_client.send_command(command)
-> 1304         return_value = get_return_value(
1305             answer, self.gateway_client, self.target_id, self.name)
1306
/databricks/spark/python/pyspark/sql/utils.py in deco(*a, **kw)
114                 # Hide where the exception came from that shows a non-Pythonic
115                 # JVM exception message.
--> 116                 raise converted from None
117             else:
118                 raise
IllegalArgumentException: Input byte array has incorrect ending byte at 16

@kushagraThapar PTAL

@moderakh  - please take a look at this issue.

@ksubhakar could you provide sample code how to repro this?
also what is the exact version of the spark connector you are using?

Some Configuration Error's on my end, I got them fixed. Thank you


GTypeInterfaces are not in the binding

Some places in the documentation mention interfaces, for example this bit in the docs for IsA<>
/// For instance, since originally `GtkWidget` is a subclass of `GObject` and
/// implements the `GtkBuildable` interface, `gtk::Widget` implements
/// `IsA<glib::Object>` and `IsA<gtk::Buildable>`.

I was looking at where the IsA trait is implemented for the various object types.  Let's look at the docs for the macro that generates it...
/// glib_wrapper! {
///     /// A container with just one child.
///     pub struct Bin(Object<ffi::GtkBin>): Container, Widget, Buildable;

But in the actual bin.rs there is only this:
glib_wrapper! {
    pub struct Bin(Object<ffi::GtkBin>): Container, Widget;

Also, GtkWidget supports Buildable and AtkImplementor (see here), but they are not listed in gtk/src/auto/widget.rs.
Some other interfaces are declared just fine, however.  For example, recent_chooser_widget.rs properly declares Orientable (because of Box), and the RecentChooser iface.
Also, I'm not sure how glib-rs handles calling an interface method on an instance.  For example, GTK+ has this code:
void
gtk_recent_chooser_set_sort_func  (GtkRecentChooser  *chooser,
				   GtkRecentSortFunc  sort_func,
				   gpointer           sort_data,
				   GDestroyNotify     data_destroy)
{
  g_return_if_fail (GTK_IS_RECENT_CHOOSER (chooser));

  GTK_RECENT_CHOOSER_GET_IFACE (chooser)->set_sort_func (chooser,
  							 sort_func,
  							 sort_data,
  							 data_destroy);
}

That GTK_RECENT_CHOOSER_GET_IFACE() expands to a call to g_type_interface_peek(), which returns NULL if the instance is not of a class that supports that interface.  I can't find any calls to g_type_interface_peek() in glib-rs at all :)

Hmmm, maybe the lack of calls to g_type_interface_peek() is because e.g.
impl<O: IsA<RecentChooser> + IsA<glib::object::Object>> RecentChooserExt for O

implements the methods by simply calling the GTK+ functions, and those functions do the type checking?

If the interface impl is missing, first of all check if the interface has bindings at all. In theory the interfaces would be implemented automatically by the code generator if the interface is generated and it is statically known that some class implements it.
For interface methods, it is (currently) required that there is an actual C function like the above that gets the interface and calls the vfunc.

In case it wasn't clear, there are no bindings for GtkBuildable yet

Should this be closed then or is there anything missing?

Any ideas on why there are no bindings for GtkBuildable?  It is in gir-files/Gtk-3.0.gir.
Is gtk-rs/gtk/Gir.toml written by hand?  Because GtkBuildable is missing from there.

Any ideas on why there are no bindings for GtkBuildable? It is in gir-files/Gtk-3.0.gir.

Either we forgot or we add an error. But this I don't have any memories bound to Buildable, I think we forgot. Might be worth adding it.

Is gtk-rs/gtk/Gir.toml written by hand? Because GtkBuildable is missing from there.

It is written by hand.

For the non-sys (the safe, proper bindings), you explicitly have to list all things that should go in and then make sure the output is correct and makes sense manually. That's why it's not taking everything by default.
Most things that are missing right now were just not added because nobody thought of adding them, and nobody needed them yet :) Just send PRs for everything you notice, usually it's not complicated (and ask on IRC if you need help with the tooling).

Ah.  If I add "Gtk.Buildable" to the generate section of gtk/Gir.toml, it indeed lists the interface in the calls to glib_wrapper!(), and creates a gtk/src/auto/buildable.rs.
However, it removes the get_name() and set_name() methods from classes that have them, e.g. gtk_file_filter_get_name() and gtk_widget_get_name().  I'm not sure how to let this interface have its own wrappers without the functions disappearing for other objects.

It's "common" to have both automatically generated code and manual code. Take a look at *ExtManual traits (you need to export them in prelude). Also, once you have update the Gir.toml file, just run make gir and let the magic happens. :)

Thanks, @GuillaumeGomez!  That's what I needed to know.
Closing this issue; I'll file a PR for Buildable in the gtk module.


WIP: 2-cockpit-9: [no-test] Update translations from Fedora Weblate

Update translations from Fedora Weblate

[ ] po-refresh

po-refresh in progress on 2-cockpit-9.
Log: https://logs.cockpit-project.org/logs/po-refresh-1068-20200730-031818/


AKS Azure AvailabilityZone Rule

PR Summary
Fix #880
Added a Azure.AKS.AvailabilityZone option for the following types:

Microsoft.ContainerService/managedClusters

Added the zone mappings from https://docs.microsoft.com/en-us/rest/api/resources/providers/list to providers.json with the regions that support availability zones. Rule configures extra zones with the AZURE_AKS_ADDITIONAL_REGION_AVAILABILITY_ZONE_LIST configuration option
Also did a bit of refactoring and made a ResourceProviderHelper class to share those common methods to read the providers.json file. Let me know if you'd like this to be different. I didn't want to copy the methods over to the Helper class since it seems like these methods could be reused a lot if interacting with provders.json data.
Also added some tests but can add more to avoid regression in the future. There is a couple of edge cases where the zones passed in from the custom configuration could be incorrect, but didn't know how this should be handled. I can probably add a check to make sure valid zones are being passed in.
Let me know what you think of this so far 🙂
PR Checklist

[x] PR has a meaningful title
[x] Summarized changes
[ ] Change is not breaking
[ ] This PR is ready to merge and is not Work in Progress
Rule changes

[x] Unit tests created/ updated
[x] Rule documentation created/ updated
[x] Link to a filed issue
[ ] Change log has been updated with change under unreleased section

Other code changes

[ ] Unit tests created/ updated
[ ] Link to a filed issue
[ ] Change log has been updated with change under unreleased section

/azp run


Fixing dependecy for unit test

New PR created with supporting functional test - https://github.com/almighty/almighty-ui/pull/166


Add timeout opt

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 86.667%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
78

Relevant Lines:
90

💛  - Coveralls


add generation to spec for defaults (#1298)

Checklist

[x] The title of this PR would make a good line in Acorn's Release Note's Changelog
[x] The title of this PR ends with a link to the main issue being address in parentheses, like: This is a title (#1216). Here's an example
[x] All relevant issues are referenced in the PR description. NOTE: don't use GitHub keywords that auto-close issues
[x] Commits follow contributing guidance
[x] Automated tests added to cover the changes. If tests couldn't be added, an explanation is provided in the Verification and Testing section
[x] Changes to user-facing functionality, API, CLI, and upgrade impacts are clearly called out in PR description
[ ] PR has at least two approvals before merging (or a reasonable exception, like it's just a docs change)

#1298

I don't like this approach. I know this is a bit of a corner case, but the solution is very antithetical to how we present acorn

Run kubectl edit appinstance <appinstancename> -n <namespace> to open the configuration file for the app that needs to be updated.

We don't even want the user to know "AppInstance" is a thing. In fact, a normal developer should not have access to appInstance. Can you set generation on App rather than appInstance?
What if we just had an explicit --generation flag with a description like "Setting this to a new value will cause the app to be re-evaluated and potentially redeployed.

closing until issue is confirmed by user


Updated secret permissions for cluster-node-tuning-operator and open…

What this PR does / why we need it:
Updated some more secret permissions to 416 specifically those mounted by
cluster-node-tuning-operator
openshift-route-controller-manager
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #1031
Checklist

[x ] Subject and description added to both, commit and PR.
[x ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

Troubleshooting build errors


Time region support

Fixes #10

@securenotebook Would this work for you?

@securenotebook Would this work for you?

WOW! looks amazing! Exactly what I wanted. Let me work out how to deploy this locally
Thank you so much!

Merging for now, but please let me know if you find any bugs or suggestions for improvement!

Hey marcusolsson, I'm trying to install this lastest master branch to test, however, I'm struggling...  Is there any easy way to do this? grafna-cli is only picking up version 0.5 latest version is 0.6.
I git cloned the source and deployed in my docker container @ /var/lib/grafana/plugins/grafana-hourly-heatmap-panel/ then restarted grafana.
t=2020-09-20T14:44:31+0800 lvl=warn msg="Plugin missing module.js" logger=plugins name="Hourly heatmap" warning="Missing module.js, If you loaded this plugin from git, make sure to compile it." path=/var/lib/grafana/plugins/grafana-hourly-heatmap-panel/src/module.js
I've spent about an hour trying to work this out and failed. I thought I'd just ask the expert.
Or push to grafana plugins ?

Hey! If you clone this from master, you need to build it before Grafana can load it. This should do it:
cd grafana-hourly-heatmap-panel
yarn install
yarn build

Then restart Grafana. Hope that works 👍

It was staring me right in the face!  When i tried to run yarn it wasnt installed so i moved on.... after installing it all good.
First glance, everything is working as expected!
 Thanks!


Move Transforms to Separate Package

Transforms are now capable of generating metadata, which introduces a dependency between smartnoise-sql and smartnoise-synth.  These packages should be able to be independently installed.  A possible solution is to move transforms to a separate package and move metadata into the transforms package.  Then each of the larger packages could include transforms as a dependency.
Splitting this off would be a breaking change for smartnoise-synth, because namespaces would change.  It need not be a breaking change for smartnoise-sql.

Could ideally move the Metadata class at the same time, to make it easier to infer metadata from data, and instantiate transformers from Metadata.  This would then also be a breaking change for smartnoise-sql.


Option to not override default statusbar

As it is right now the plugin overrides any previously set statusbar, I'd rather set my own. Can you make this optional?

Hi @marcusds,
You're very right, I will put a property to disable that behaviour.
Thanks for the feedback! :)

Should be fixed in v2.0.1 !


[WIP] update react-js-buy's shopify-buy dependency use of api

still WIP?

yes. At the moment I'm getting the following error when I try to select a product variant and unsure how to proceed:
Uncaught TypeError: Cannot read property 'Helpers' of undefined at Product.handleOptionChange

I got past this error by passing down client as a prop from App component, to products, to product... not sure if this is the right way. (besides being new to js-buy I'm new to react)

It seems this error existed already, I've reported it #54
Which means, this is ready for review. I poked around the page manually and followed the errors, so I likely overlooked some other necessary changes.

Looks good, thanks!


Team Management APIs

The documentation for the team management states:

NOTE: The APIs needed for teams are not supported yet by GitHub Apps
https://developer.github.com/v3/apps/available-endpoints/

however the official GitHub API documentation states:

The following endpoints are available for use by GitHub Apps. Your app can make the following requests using GraphQL v4 or REST v3 endpoints. For more information, see "GitHub Apps."

so perhaps the APIs got updated? and all that's missing to making this work is the preview headers addition?
unless I'm missing something, I'm happy to make a PR to add preview headers.
cheers.

Can this be reopened? Anyone have any more info on this/if this now works or not?

I belive this is working as expected at this point, and the README is just misleading.
see the code responsible for team management here: https://github.com/probot/settings/blob/master/lib/plugins/teams.js

@ahmadnassri OK, I guess we'll just have to try then.
Have the GitHub APIs been updated or is this comment at the top of that file still relevant?

// NOTE: The APIs needed for this plugin are not supported yet by GitHub Apps

@ahmadnassri OK, I guess we'll just have to try then.
Have the GitHub APIs been updated or is this comment at the top of that file still relevant?
https://github.com/probot/settings/blob/master/lib/plugins/teams.js#L1


add some unit testing for the OIDC token flow

Signed-off-by: Jake Sanders<EMAIL_ADDRESS>

Nice!


How to build Android AAR without opencv output(lines) like mediapipe Facemesh Solutions.

Hi, I need to create similar like Facemesh Solutions aar, will later use as library module in my Android Studio Project, so that i can able to modify or customize the solutions core files which is in mediapipe/java folder.

I can be able to generate Android AAR, but its building with OpenCV Library files. How can I build without OpenCV library similar like Solutions AAR locally.

currently I'm using below one to create Android AAR for face mesh.

load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")

mediapipe_aar(
    name = "mediapipe_face_mesh",
    calculators = ["//mediapipe/graphs/face_mesh:mobile_calculators"],
)

I just saw the source code inside the folder java/com/google/mediapipe/** .
when I try to include these as library module getting some files not found, for example com.google.mediapipe.proto.GraphTemplateProto.CalculatorGraphTemplate. please let me know should I include any more files in different places.

You need to run protoc from protocol-buffers and generate java or kotlin code based what you wanna use it.
File you need (make sure you have other import files as well
https://github.com/google/mediapipe/blob/master/mediapipe/framework/tool/calculator_graph_template.proto
CalculatorGraphTemplate is here
Example use on my own case (different one but it should show the main idea)
protoc --java_out=/home/onuralps/protobuff_outputs mediapipe/modules/face_geometry/protos/face_geometry.proto
protoc --java_out=/home/onuralps/protobuff_outputs mediapipe/modules/face_geometry/protos/mesh_3d.proto
protoc --java_out=/home/onuralps/protobuff_outputs mediapipe/framework/formats/matrix_data.proto

@onuralpszr Thanks, i will check this. you know how can we build AAR without opencv output/library files.?

I only change openCV version 3 or 4 based on the solution or project I was working on it.  Plus, openCV used very widely in project I don't think it is possible (at least I never tried  to remove openCV completely from what I compile. Maybe someone else knows it)
Technically openCV version is set in here

"--define EXCLUDE_OPENCV_SO_LIB=1" They mentioned here -mediapipe_aar.bzl
Finally openCV Lib files not included, but unfortunately it makes app crashes (possibly because of OpenCv Jni files not found) 👎🏼

Did you try to include your own openCV ? Also you are  correct, I forget that option in bzl file. I checked my own project and I remember that I include openCV4 myself instead of what they offer but problem is internally in some solutions it does crash because difference between cv3 and cv4 and how is it code inside mp source

Yeah you are right, i also noticed that they used openCV3 for some usecase and openCV4 for some . for facemesh they >using openCV3, we can check here - MainActivity.java

Yes that is why there are 2 cv configs (cv4 and cv3) but in some cases you can use cv4 instead of cv3 even project asked to use cv3 (not %100 guarantee and you may wanna check everything work as intended)
(I tested that too)

However, I believe that it would work if we use OpenCV 3 or 4 depending on the use case.

It does work (I tested)

Hi @wh173d3v11,
We are no longer maintaining the legacy facemesh solution, as it has been integrated into the new Face Landmarker task API, as explained in our documentation.
We intend to explore and implement our upgraded Face Landmarker in Android, following the steps outlined in our documentation.
Additionally, we kindly request you to inform us of any errors encountered during the exploration process.
Thank you


Add unit tests and tweak a few files

Type of change (place an x in the [ ] that applies)

[x] New sample
[ ] New feature
[ ] Bug fix
[x] Documentation

Summary
This pull request adds the first unit test to this repo and also improves a few files in the same way with https://github.com/slack-samples/deno-starter-template/pull/14
Requirements (place an x in each [ ] that applies)

[x] I’ve checked my submission against the Samples Checklist to ensure it complies with all standards
[x] I have ensured the changes I am contributing align with existing patterns and have tested and linted my code
[x] I've read and agree to the Code of Conduct

Thanks for the reviews!


Provide Docker image as non root

Discussed in https://github.com/meilisearch/MeiliSearch/discussions/1756

Originally posted by igaul October  1, 2021
Hi,
I'd like to request the official docker image run with a non root user, so the final image of Dockerfile would be something like this (what I am using)
`# Run
FROM    alpine:3.14
ARG     USER=meiliuser
ENV     HOME /home/$USER
ENV     MEILI_HTTP_ADDR <IP_ADDRESS>:7700
download runtime deps as root and create $USER
RUN apk add -q --no-cache libgcc tini curl
&& adduser -D $USER
WORKDIR $HOME
USER $USER
copy file as $USER to $HOME
COPY  --from=compiler /meilisearch/target/release/meilisearch .
EXPOSE  7700/tcp
CMD ["tini", "--","./meilisearch"]`
If interested, I would be happy to pr. Have a good weekend!

@curquiza  I'll do, please assign it to me.

As the change has been reverted (in https://github.com/meilisearch/meilisearch/pull/2032), can we re-open this issue? We would like to run Meilisearch on our Kubernetes cluster. Unfortunately we don't allow running images as a root user. Manually setting the securityContext to overrides this results in a Error: Permission denied (os error 13) error.
This feature would be highly appreciated

Hello everyone, we are having exatcly the same issue in kubernetes.
We dont allow any pod to run as root and so I wanted to ask if there is any solution for this?
Thanks in advance.

did any one find a solution? same problem here


pyItunes fails to load libraries with non-ASCII URL paths in python 2.7

While I'm working on fixing my first issue, I ran into a problem with the most recent build of the library.
Loading my library, I ran into this error:
Python 2.7.10 (default, May 25 2015, 13:06:17)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
>>> import pyItunes
>>> l = pyItunes.Library("/Users/roger/projects/Fun with Data/itunes/iTunes Music Library.xml")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/roger/projects/pyitunes/pyItunes/Library.py", line 36, in __init__
    self.getSongs()
  File "/Users/roger/projects/pyitunes/pyItunes/Library.py", line 77, in getSongs
    s.location = text_type(urlparse.unquote(urlparse.urlparse(attributes.get('Location')).path[1:]))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 40: ordinal not in range(128)

0xcc corresponds to Ì, a "Latin capital letter I with grave". What's weird about this error is that this character doesn't appear at all in my library:
Rogers-iMac:pyitunes roger$ cat "/Users/roger/projects/Fun with Data/itunes/iTunes Music Library.xml" | grep -c Ì
0

The relevant line of code is here:
if ( self.musicPathXML is None or self.musicPathSystem is None ):
    # s.location = text_type(urlparse.unquote(urlparse.urlparse(attributes.get('Location')).path[1:]),"utf8")
    s.location = text_type(urlparse.unquote(urlparse.urlparse(attributes.get('Location')).path[1:]))
else:
    # s.location = text_type(urlparse.unquote(urlparse.urlparse(attributes.get('Location')).path[1:]).replace(self.musicPathXML,self.musicPathSystem),"utf8")
    s.location = text_type(urlparse.unquote(urlparse.urlparse(attributes.get('Location')).path[1:]).replace(self.musicPathXML,self.musicPathSystem))

Commenting out the first s.location line allows me to load the library.

I notice that @codyzu changed this line when adding in some 2/3 compatibility. Will the original line (the one commented out up top) not work with python 3?

Looking at this again. 0xCC is just ASCII, but 0xCC 0x81 refers to an acute accent - ´.
The Library initialization failed on this URL: file://localhost/Users/roger/Music/Albums/Boubacar%20Traore%CC%81%20-%20Mali%20Denhou/11%20Mali%20Tchebaou.mp3 (This should look like this when decoded properly: file://localhost/Users/roger/Music/Albums/Boubacar Traoré - Mali Denhou/11 Mali Tchebaou.mp3)
So this means that I need to find how to get Python 2's urllib.unquote() to work with unicode strings.

Annoyingly, Python 2's urllib.unquote() encodes the above link as '/Users/roger/Music/Albums/Boubacar Traore\xcc\x81 - Mali Denhou/11 Mali Tchebaou.mp3', which then needs to be decoded into proper unicode format - u'/Users/roger/Music/Albums/Boubacar Traore\u0301 - Mali Denhou/11 Mali Tchebaou.mp3', but six.u() fails to do that. I'm having trouble finding an easy solution that would work with Python 2 and 3 combined.

text_type does not work for decoding these strings, because it goes byte by byte (´ is 2 bytes long).

(Ignore all the commits except for 7991560; it took a few tries to rebase my fork's branch to liamks's HEAD so my two pull requests could be created separately)

This should be fixed now in pull request #21.

Just got this using 2.7, conda environment w/ tensorflow. My library definitely has a lot of accents
---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-2-df9cbca52326> in <module>()
     19     if song and song.rating:
     20         if song.rating > 80:
---> 21             print("{n}, {r}".format(n=song.name, r=song.rating))

UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 5: ordinal not in range(128)

I'm simply running the code for using pickle.


Nullable types

This a more thorough implementation of #58 adding the strictNullChecks flag as a default to the project.
Although the output is generated slightly differently, it should be backwards compatible.
The only difference I found was that we were adding both .optional() and .Or(Undefined) to optional properties, which
according to the runtypes documentation will produce the same thing. So I've found a way to revert to .optional() if the null type is not found.

@johngeorgewright Nice one. I guess the "custom ts-morph Project" example from the README should also be changed now, since strictNullChecks is already the default


Fix internal kafka_broker role relative path

Description
I am trying to use the confluent.kafka_broker role as a role dependency, and that does not work - because it results in an incorrect path. This PR corrects this, and I have verified that it solves my use case.
I think it seems odd to refer to a role-internal file using an "external" path, starting with roles/<role-name> - as it really limits what will actually work. The bug seems to have been introduced in this commit, https://github.com/confluentinc/cp-ansible/commit/6d50a7f09d4843bcc1e85821613a560ffa5d95f3#diff-0fec400f04957013fee20d1dfd6d96128a0e5275684c5b0d6713d546b0a67c42, and I assume that the change was done to fix another issue. Let us 🤞 that this issue won't return...
If this PR is merged, the path will be equivalent to a similar reference in https://github.com/confluentinc/cp-ansible/blob/d61b9a2426c151c779e696278d25689edc93e6cc/roles/confluent.zookeeper/handlers/main.yml#L3 making the code more consistent.
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been merged and published in downstream modules
[ ] Any variable changes have been validated to be backwards compatible

@domenicbove Did you intentionally block access to CI-jobs? I can see that the build is failing, but if I click on the 'Details' link, I get to https://jenkins.confluent.io/job/confluentinc-pr/job/cp-ansible/job/PR-442/2/display/redirect - which gives me HTTP 404.

@erikgb Yes unfortunately, the jenkins in VPN access only. But I can tell you right now the failure is because of this: https://github.com/confluentinc/cp-ansible/pull/441


[FBP] example + PEP8

Could you add the example to the example gallery in the doc (https://github.com/silx-kit/silx/blob/master/doc/source/sample_code/index.rst) ?

And maybe use a longer name for the example filename.

waiting for the forward projector to be merged and we will add utils to generate a Shepp-Logan phantom

Now the backprojection example is free from external resources.
It deal with it using the forward projection.
Question :
should we also include an example for the projection ?
I think the current example is enough.


Android: after entering newline: Cursor is offset by a letter after typing a word and hitting space

In Android, when using the default font without any formatting everything seems fine. However when making the font bold… after entering a new line, and typing for example “apple” and hitting space… the word gets shifted by a space and the cursor lands between the last 2 characters: “ apple” with the cursor between the “l” and the “e”.

Seems it may be a Samsung predictive keyboard issue


[5.8] Pass severity information to work command log method to enable filtering

Now it is possible to control the output of job workers with the standard verbosity parameters (-v, -vv etc).
Standard worker messages ("Processing: \App\Job\Foo", "Processed: \App\Job\Foo") are now only visible when using -v or -vv, while errors are always shown.
Using -vv gets back the previous output behavior:

-v for getting "Processed" messages
-vv for both "Processing" and "Processed" messages.

Resolves: https://github.com/laravel/framework/issues/27483

Hmm, I wonder if this should take into account the current environment. IMO I always want to see all messages locally because it lets me know the queue is actually doing something and not just frozen.

I now see that running artisan horizon -vv does not help, because horizon does pass the -vv not down to its workers. Running artisan horizon:work -vv does show the messages as previous, though.

My first thought is that you describe something the log system can already do. It sounds we should change the WorkCommand to use logging instead of writing to console output directly. That way all informational messages can be seen in the console, but only the warning/error messages are written to log files. (And you can configure the console to only show warnings and errors.)

FTR, detailed logging is already possible via the Queue::* facade.
I use something like this to get very detailed per-queue logfiles; it's otherwise too laborious to cross reference jobs in case of problems:
    protected function initializeQueueEvents()
    {
        Queue::before(function (JobProcessing $event) {
            $job = $event->job;
            $logger = $this->getLoggerForQueue($job->getQueue());
            $logger->info(\get_class($event), $this->getJobDetails($job));
        });

        Queue::after(function (JobProcessed $event) {
            $job = $event->job;
            $logger = $this->getLoggerForQueue($job->getQueue());
            $logger->info(\get_class($event), $this->getJobDetails($job));
        });

        Queue::exceptionOccurred(function (JobExceptionOccurred $event) {
            $job = $event->job;
            $logger = $this->getLoggerForQueue($job->getQueue());
            $details = $this->getJobDetails($job);
            $details['exception'] = $event->exception;
            $logger->error(\get_class($event), $details);
        });

        Queue::failing(function (JobFailed $event) {
            $job = $event->job;
            $logger = $this->getLoggerForQueue($job->getQueue());
            $details = $this->getJobDetails($job);
            $details['exception'] = $event->exception;
            $logger->error(\get_class($event), $details);
        });
    }
}

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.


feat: add eth_call, eth_getLogs and eth_estimateGas coverage to conformity tests

Description:

Add eth_call, eth_getLogs and eth_estimateGas test to conformity tests:

[x] eth_call - non existing contract
[x] eth_call - existing contract view function and existing from
[x] eth_call - existing contract tx and existing from
[x] eth_call - existing contract tx, existing from and positive value
[x] eth_call - existing contract view function and non-existing from
[x] eth_call - existing contract tx and non-existing from
[x] eth_call - existing contract tx, non-existing from and positive value
[x] eth_estimateGas - non existing contract
[x] eth_estimateGas - existing contract view function and existing from
[x] eth_estimateGas - existing contract tx and existing from
[x] eth_estimateGas - existing contract tx, existing from and positive value
[x] eth_estimateGas - existing contract view function and non-existing from
[x] eth_estimateGas - existing contract tx and non-existing from
[x] eth_estimateGas - existing contract tx, non-existing from and positive value
[x] eth_getLogs - non-existing contract
[x] eth_getLogs - existing contract
[x] eth_getLogs - existing contract and from/to block

Related issue(s):
Fixes #2754
Depends on:
https://github.com/hashgraph/hedera-mirror-node/pull/9302
Notes for reviewer:

Checklist

[ ] Documented (Code comments, README, etc.)
[ ] Tested (unit, integration, etc.)

LG, one suggestion would be can we update a README explaining what the conformity tests do and cover. We have 4 batches and different endpoints so would be good to give a summary overview to understand what our coverage is and match up to execution-api test it. I feel like we need a TESTING.md file in the root. This can be a follow up or just create a ticket on the backlog and not this as a possible content inclusion

Just created a new task https://github.com/hashgraph/hedera-json-rpc-relay/issues/3054 and put it in the backlog.


Optimize Structuremap usage (performance)

As proposed at https://github.com/roadkillwiki/roadkill/issues/17

@yetanotherchris
this is fixed by https://github.com/roadkillwiki/roadkill/pull/37?

Not fixed quite yet, there's a branch that fixes an IoC problem with the IUnitOfWork


WebApiAssemblyToSwaggerGenerator settings serialization loose TypeMappers

https://github.com/RSuter/NSwag/blob/af109c212bdee3bf5ac59ee53f68e8e6fa392315/src/NSwag.AssemblyLoaderCore/SwaggerGenerators/WebApi/WebApiAssemblyToSwaggerGenerator.cs#L86
https://github.com/RSuter/NSwag/blob/af109c212bdee3bf5ac59ee53f68e8e6fa392315/src/NSwag.AssemblyLoaderCore/SwaggerGenerators/WebApi/WebApiAssemblyToSwaggerGenerator.cs#L102
In the WebApiAssemblyToSwaggerGenerator when you call GenerateForControllersAsync with settings containing TypeMappers, the serialization of settings in json loose those typemappers.
A simple fix is to not serialize the settings to json (I don't know the reason it's done like that).
Here is my usage of WebApiAssemblyToSwaggerGenerator to use Nodatime instead of BCL DateTime.
            var settings = new WebApiAssemblyToSwaggerGeneratorSettings
            {
                AssemblyPaths = options.InputFilePaths.ToArray(),
                IsAspNetCore = options.IsAspNetCore,
                DefaultEnumHandling = options.DefaultEnumHandling,
                DefaultPropertyNameHandling = options.DefaultPropertyNameHandling,
                FlattenInheritanceHierarchy = options.FlattenInheritanceHierarchy,
                Title = options.Title,
                Version = options.Version,
                TypeMappers =
                {
                    new PrimitiveTypeMapper(typeof(LocalDate), s =>
                    {
                        s.Type = JsonObjectType.String;
                        s.Format = "date";
                    }),
                    new PrimitiveTypeMapper(typeof(OffsetDateTime), s =>
                    {
                        s.Type = JsonObjectType.String;
                        s.Format = "date-time";
                    }),
                }
            };

            var generator = new WebApiAssemblyToSwaggerGenerator(settings);
            var controllerClasses = generator.GetControllerClasses();
            var document = await generator.GenerateForControllersAsync(controllerClasses).ConfigureAwait(false);

            var swaggerJson = document.ToJson();
            await File.WriteAllTextAsync(options.OutputFilePath, swaggerJson).ConfigureAwait(false);

WebApiAssemblyToSwaggerGenerator  runs the WebApiToSwaggerGenerator in a second, isolated AppDomain (like a process), you cannot just pass the settings over to this AppDomain. This is why the settings serialized with JSON.
To fix this, we need to pass the type mappers, and other extension classes (e.g. custom PropertyNameGenerator, etc.) as string (just class name) and create an instance in the second appdomain.
I began with this in the "extension-points" branch, but didn't have the time for this yet:
https://github.com/RSuter/NSwag/commit/6d91f61053761333a27ded8e044a288bd8eb1139
At the moment, if you need typemappers, etc then you have to generate the swagger spec via ASP.NET middleware or via C#.

Ok for fullnet but we got the issue with netcore. In that case the serialization is not really needed.
By the way what the reason of running it in an isolated AppDomain?

If you don't need to run it in a separate AppDomain (full .NET) or AssemblyLoadContext (.NET Core) just use WebApiToSwaggerGenerator and everything should work as expected.
AppDomains are required if you need to load and unload DLLs - in the main AppDomain you cannot unload DLLs - and to avoid conflicts with assembly version. Usually if a dll is already loaded, you cannot load the same dll with another version - but NSwag requires this to do, because it loads external assemblies which may have other versions - and this is only possible with appdomains. Also they are required if you want do cleanup all loaded assemblies after genererating a swagger spec..

And BTW: I think we should support nodatime out of the box without typemappers?

Indeed it works perfectly :)
My bad.
Thanks for the very quick answer ;)
Well indeed supporting NodaTime out of the box would be a great idea :)

Yes it's not working without them

We just need to add them here
https://github.com/RSuter/NJsonSchema/blob/master/src/NJsonSchema/Generation/JsonObjectTypeDescription.cs#L79

Well there a lot of types for NodaTime, not only those 2.
http://nodatime.org/2.2.x/userguide/core-types


Use the asdf-schema-helpers package pytest helper

This uses the pytest plugin from asdf-schema-helpers instead of the one bundled with asdf.
Since the plugin is not yet released, we need to install from Github, which means it cannot be listed in the [test] extras and instead needs to be explicitly installed in the tox env.

So adding usedevelop = True seems to work.  The issue here is that plugin checks to see if the extension resolver maps to the same file as the one being checked by the pytest plugin.  If pytest is being run on the repo that's been installed in editable mode, that works fine.  If not installed in editable mode, then they are not the same file, as the installed version gets installed in site-packages.  One could run the tests using pytest --pyargs asdf-transform-schemas after changing dirs out of the repo.  It would be nice for it to work using a mix of these.
Any thoughts?

The other option would be to turn off the check that tries to see if the schema being tested is the one mapped by the resolver.

This has not been touched in two years, so I am closing this.

I think it would be good to look at reviving this PR, because the schema validation that takes place in asdf-schema-helpers is a generalization for use in ASDF adjacent packages that have schemas, and probably should be applied to this package.  Currently using the pytest schema checker plugin in asdf only does very minimal checks to schema validity.
At least this is the reason I recall for wanting to use the (then) new asdf-schema-helpers package to do general schema checking everywhere we have ASDF schemas.  Or is asdf-schema-helpers now defunct?
@eslavich ?


feat: add code search net eval

script is not identical as original code search net, now i'm using:
function docstring to search function code, also very short function docstring (length < 3) has been filtered.
The result is evaluated based on mean reciprocal rank (mrr) since each docstring has 1 matched code.

Nice adding CodeSearchNet would be amazing!


Prefer if examples dont need dp

I reckon it would be easier to use the examples if they didnt need one to learn dp to understand them fully.
Basically, currently one has to kind of learn both rnn and dp together, to understand the examples.  I reckon it would be faster learning curve if the examples in rnn assumed only knowledge of rnn, and core Torch libraries, such as nn etc.

@hughperkins Yeah I could use optim instead. But then one one need to learn optim to understand the examples. Personally, I am not a big fan of optim. I find the all inclusive closure arguments difficult to understand. I also don't like it's reliance on the getParameters() method. I prefer layer-wise calls then one giant call applied to all params/gradParams. But of course those are just personal preferences. I built dp because I was used to pylearn2's way of doing things.
In any case, dp provides a framework for doing experiments, which is something none of the core packages do AFAIK. Every script includes its own implementation of cross-validation. All this variation makes it difficult to separate the wheat from the chaff IMHO. Of course, I built dp so maybe I take it for granted.
Anyway, I agree that we should have optim examples (i.e. non-dp examples). But I think we should still rely on dp to at least provide the data. And yeah, I reckon dp ain't perfect or easy to learn. But its what I know best. Maybe I should try to reproduce some of @karpathy 's code using the rnn package instead of nngraph. That way I could learn optim from his examples, which are pretty good. That could also mean more examples for rnn.

Yeah I could use optim instead.

Yes, or maybe a simple SGD for-loop perhaps?  I assume the examples are mostly there for us to base our own work on, than to provide high performance benchmarks?  (Might be an invalid assumption though?)

But I think we should still rely on dp to at least provide the data.

Yes, seems very reasonable to me.  I was thinking the same thing actually :-)

Maybe I should try to reproduce some of @karpathy 's code using the rnn package instead of nngraph

Funnily enough, I'm exactly targeting trying to do char-rnn using your rnn library instead of the char-rnn library :-)  I think that it's probably a bit more work to get rnn working, but then it seems plausibly more modular, transparent, easier to maintain, perhaps.

Yeah I could use optim instead.

Yes, or maybe a simple SGD for-loop perhaps?  I assume the examples are mostly there for us to base our own work on, than to provide high performance benchmarks?  (Might be an invalid assumption though?)

The recurrent language (the one in the dp repository itself) and recurrent attention models are benchmarked against existing papers. But the encoder-decoder is an unbenchmarked example. So we can have both.

But I think we should still rely on dp to at least provide the data.

Yes, seems very reasonable to me.  I was thinking the same thing actually :-)

Maybe I should try to reproduce some of karpathy 's code using the rnn package instead of nngraph

Funnily enough, I'm exactly targeting trying to do char-rnn using your rnn library instead of the char-rnn library :-)  I think that it's probably a bit more work to get rnn working, but then it seems plausibly more modular, transparent, using standardized building bricks, perhaps.

Yeah. The nn package philosophy is to encapsulate transformations within modules as much as possible. rnn does that. The usual nngraph solutions don't (unless its a fixed length sequence for both training and evaluation).
I would love to see the char-rnn script rewritten with rnn!

Hi,
sorry to get into the conversation, but I am working on a model which looks like char-rnn using rnn and want to use optim as well, so I think my comment is at least a bit relevant :)
The way char-rnn was initially implemented, the model does not predict a sequence, but a single character each time. Therefore, using nn.SequencerCriterion for the output does not work. One cannot use a nn.___Criterion as the behavior is different: models wrapped by rnn.Sequencer require a gradOutput table, not a single gradient vector. @nicholas-leonard, any tips on how to train such a model using rnn?

@jfsantos hi. So I think the char-rnn character-level LM is trained the same way you train a word-level LM. Your model takes as input a sequence of words : w[1], w[2] ... w[T] for which the targets are also
a sequence of words : w[2], w[3] ... w[T+1] where T is the size of your training sequence (rho in our code here). So you would use a Sequencer for that with a SequencerCriterion.
However, when you want to generate sequences, you need to generate one word at a time. So what you do is call sequencer:remember('both') so that it will remember past inputs. You can also do sequencer:evaluate() to reduce memory consumption during sequence generation. Then you just forward a random word w[1] as a table of just one element : {w[1]}.The output will be a table containing one element : the distribution over logP(w[2]|w[1]) (i.e. output of a logsoftmax).  If you take the max of that distribution, you will get the Maximum A Posteriori (MAP) value for w[2]. Then you can feed {w[2]} as input to get w[3], an so forth and so on.
Based on what I could gather from @karpathy  's blog post, I think this is how he goes to generate sequences.

Thank you for your answer, @nicholas-leonard!
I think you are right regarding char-rnn. However, the model I am currently implementing sees an entire sequence before computing an output, and the output is computed based only on the last hidden state. Do you think there's an easy way to use a "scalar" criterion such as nn.MSECriterion or nn.BCECriterion with rnn?
Thanks again.

@jfsantos Yup. Just use stack nn.SelectTable(-1) after the final Sequencer. It will output the last element of the sequence (the last hidden state). You can then feed that through MSE/BCE.

That works, thanks!
If I manage to get my char-rnn-like code ready before @hughperkins does, I'll submit it as an example of using optim instead of dp.

@jfsantos awesome!. Would love to see that added as an example.

@jfsantos Can you pl provide your implementation as an example to me for learning purposes?
It will be really helpful.

@hughperkins The examples folder now contains tons of non-dp examples.