kube-proxy doesn't recover from outdated events

After running a 0.7.2 cluster in GCE over the weekend, service connections seemed flaky.
The following lines are recurring in the kube-proxy log on all minions in my cluster.
W0105 17:28:25.587865    5051 api.go:188] Got error status on WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [578/235]) [1577] Reason: Details: Code:0}
W0105 00:17:07.886554    5042 api.go:124] Got error status on WatchServices channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [348/234]) [1347] Reason: Details: Code:0}
The indices in the error message do not change over time.
kube-proxy seems to be stuck in an endless loop on this condition and seems to fail to get service and endpoint updates.

I brought this up on IRC over the very slow new year holiday, @smarterclayton mentioned this should only really be possible under a high write volume, I presume on the api server or etcd?
We have certainly been pumping a ton of data through service connections, but I'm not sure anything else we're doing in the cluster would qualify as high write volume.

I think this is working as intended.
The kube-proxy "watches" endpoints on the apiserver.
Our internal watching protocol has the apiserver intentionally dropping the connection every few minutes, and clients are supposed to reconnect.
Are you seeing any application-level errors which suggest that the proxy isn't working?
I'd like @lavalamp to confirm my diagnosis, though.  Also, I think we have a related issue discussing how to reduce this warning spew.  Maybe he remembers which issue that was.

My logs have these errors once every ~30 seconds each for WatchEndpoints and WatchServices.
I don't think I could prove they are related, but these errors have been highly correlated with application-level issues with connecting to services. My workaround so far has been to sudo /etc/init.d/kube-proxy restart on all of my minions. Service connection issues go away, and so do these lines of logging. Could be a red herring, but it seems that this failure is preventing kube-proxy from learning about new endpoints.

I saw a bug over the weekend, where kube-proxy stopped seeing new services
as well, and when I bounced the server it looked fine.  It definitely
looked like the watch had failed, and wasn't retrying.  I'll take a glance
at the kube-proxy code and try to see if anything jumps out.
--brendan
On Mon, Jan 5, 2015 at 10:17 AM, Steve Reed<EMAIL_ADDRESS>wrote:

My logs have these errors once every ~30 seconds each for WatchEndpoints
and WatchServices.
I don't think I could prove they are related, but these errors have been
highly correlated with application-level issues with connecting to
services. My workaround so far has been to sudo /etc/init.d/kube-proxy
restart on all of my minions. Service connection issues go away, and so
do these lines of logging. Could be a red herring, but it seems that this
failure is preventing kube-proxy from learning about new endpoints.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68750318
.

I think I know why this happens now.  It's a bug in our use of etcd combined with the addition of events being written to etcd at a much higher rate than previously.
Basically, etcd has a window (1000 events) that it keeps in memory that you can watch on.  When you start a watch at X, if you get no changes to your watched key in 1000 events, the next time you reconnect you'll be outside the window and you'll get the error.  Our watch loops look like "watch X, when I get an event with version Y set X=Y, if I get disconnected watch again at X".
We need a way to update X even when no events are sent within a window of time.

On Jan 5, 2015, at 1:30 PM, Brendan Burns<EMAIL_ADDRESS>wrote:
I saw a bug over the weekend, where kube-proxy stopped seeing new services
as well, and when I bounced the server it looked fine. It definitely
looked like the watch had failed, and wasn't retrying. I'll take a glance
at the kube-proxy code and try to see if anything jumps out.
--brendan
On Mon, Jan 5, 2015 at 10:17 AM, Steve Reed<EMAIL_ADDRESS>wrote:

My logs have these errors once every ~30 seconds each for WatchEndpoints
and WatchServices.
I don't think I could prove they are related, but these errors have been
highly correlated with application-level issues with connecting to
services. My workaround so far has been to sudo /etc/init.d/kube-proxy
restart on all of my minions. Service connection issues go away, and so
do these lines of logging. Could be a red herring, but it seems that this
failure is preventing kube-proxy from learning about new endpoints.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68750318
.

—
Reply to this email directly or view it on GitHub.

smarterclayton's explanation make sense.  This must be a bug.

https://github.com/coreos/fleet/issues/408

https://github.com/coreos/etcd/issues/925

The same issue will probably affect Kubelet, which has its own watch implementation, as well as the scheduler which has a third one.  Not sure about controller-manager.

I think it affects any use of watch - we can mitigate the impact on the server side, but needs a scalable solution.

On Jan 5, 2015, at 2:01 PM, Eric Tune<EMAIL_ADDRESS>wrote:
The same issue will probably affect Kubelet, which has its own watch implementation, as well as the scheduler which has a third one. Not sure about controller-manager.
—
Reply to this email directly or view it on GitHub.

Never mind about scheduler.  That is not watching etcd.

pkg/kubelet/config/etcd.go is still probably also affected though.

Kubernetes at head does not use etcd for kube-proxy.

It is - we just simply pass watch through to etcd.  Anyone who watches something that doesn't change over the period when 1000 writes to etcd will hit this problem.

On Jan 5, 2015, at 2:03 PM, Eric Tune<EMAIL_ADDRESS>wrote:
Never mind about scheduler. That is not watching etcd.
—
Reply to this email directly or view it on GitHub.

@smreed Does your kube-proxy have a -master=$BLAH argument?  I suspect not.  Which cloud provider do you use, or which getting-started-guide did you follow?

N.M.  You said you used  0.7.2 cluster in GCE .

Ah, so, his kube-proxy is talking to the master, but the master is just passing along the etcd error.

Right - and we have no way to pass an updated resource version to the client today, nor a great way to know atomically that we haven't missed a change in the watch stream.  The minimal fix is to treat the error as a reset which requires changes to the root (the master can return an error with resource version zero, but the loop has to handle it).

On Jan 5, 2015, at 2:09 PM, Eric Tune<EMAIL_ADDRESS>wrote:
Ah, so, his kube-proxy is talking to the master, but the master is just passing along the etcd error.
—
Reply to this email directly or view it on GitHub.

@smarterclayton what does "changes to the root" mean?

root = loop, autocomplete fail

I don't think that's right for us, because if we were listening at X, got no events in a 1000, then at X+1014 we stopped listening and tried to reconnect, the reconnect happens after X+1014 (let's say X+1020), we would miss events from X+1014 -> X+1020.
----- Original Message -----

https://github.com/coreos/etcd/blob/master/Documentation/2.0/api.md#waiting-for-a-change
says that in case of 401 we should get the X-Etcd-Index header from the
watch again at (X-Etcd-Index + 1).

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68777215

Yeah, I think the right answer is to do the same thing as at startup.  Do
an initial full-state read, followed by a watch.
--brendan
On Mon, Jan 5, 2015 at 1:45 PM, Clayton Coleman<EMAIL_ADDRESS>wrote:

I don't think that's right for us, because if we were listening at X, got
no events in a 1000, then at X+1014 we stopped listening and tried to
reconnect, the reconnect happens after X+1014 (let's say X+1020), we would
miss events from X+1014 -> X+1020.
----- Original Message -----

https://github.com/coreos/etcd/blob/master/Documentation/2.0/api.md#waiting-for-a-change

says that in case of 401 we should get the X-Etcd-Index header from the
watch again at (X-Etcd-Index + 1).

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68777215
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68787369
.

This is a high-priority bug, in my mind.  If no one else is working on it,
I'll put together a PR.
--brendan
On Mon, Jan 5, 2015 at 1:48 PM, Brendan Burns<EMAIL_ADDRESS>wrote:

Yeah, I think the right answer is to do the same thing as at startup.  Do
an initial full-state read, followed by a watch.
--brendan
On Mon, Jan 5, 2015 at 1:45 PM, Clayton Coleman<EMAIL_ADDRESS>wrote:

I don't think that's right for us, because if we were listening at X, got
no events in a 1000, then at X+1014 we stopped listening and tried to
reconnect, the reconnect happens after X+1014 (let's say X+1020), we would
miss events from X+1014 -> X+1020.
----- Original Message -----

https://github.com/coreos/etcd/blob/master/Documentation/2.0/api.md#waiting-for-a-change

says that in case of 401 we should get the X-Etcd-Index header from
the
watch again at (X-Etcd-Index + 1).

Reply to this email directly or view it on GitHub:

https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68777215
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-68787369
.

@smarterclayton Why it is not correct? First you need to recover the state and you need to know the index of the recovery. That is the first get (maybe a recursive get). The index of the recovery is x-etcd-index. Then you watch from x-etcd-inde + 1.

Sorry, I didn't state the assumption that in many of the watch cases the list/index call on current state is likely extremely expensive (hundreds or thousands of entries), so keeping the watch going even with no write during the watch window is still a huge performance benefit to the client.

On Jan 5, 2015, at 5:19 PM, Xiang Li<EMAIL_ADDRESS>wrote:
@smarterclayton Why it is not correct? First you need to recover the state and you need to know the index of the recovery. That is the first get (maybe a recursive get). The index of the recovery is x-etcd-index. Then you watch from x-etcd-inde + 1.
—
Reply to this email directly or view it on GitHub.

@smarterclayton I think you have to prepare for this kind of failure since the history will never be unlimited.
Probably it is good to partition the entries into different key space. And maybe etcd can support get last changed index of a key space. If the last change is smaller than the last one you have known, you can bypass a recursive get.

Agree history can not be unlimited, and we do design for that case.
A conditional get on a key path prefix would definitely allow us to solve this - we could potentially hide the 401 by issuing a recursive get conditionalized by the last index we saw.
Alternatively, if instead of watch returning 401, if watch could check the keyspace for modifications since that date and simply begin watching, that would also solve our problem elegantly.  Does the store today track that sort of info efficiently (and would it continue to do so under the new store proposal)?

On Jan 5, 2015, at 5:33 PM, Xiang Li<EMAIL_ADDRESS>wrote:
@smarterclayton I think you have to prepare for this kind of failure since the history will never be unlimited.
Probably it is good to partition the entries into different key space. And maybe etcd can support get last changed index of a key space. If the last change is smaller than the last one you have known, you can bypass a recursive get.
—
Reply to this email directly or view it on GitHub.

Not a quick fix, but we could alleviate the frequency of complete reloads by not putting events in the same data store (or same key partition) as the cluster's config info, which should drastically reduce the write rates and help avoid other issues like #2719.

I'm downgrading this to P1 because I think Brendan's change should allow it to recover.
As far as I understand it, there's two possible further problems here.
A) Suppose a watch times out between a client and apiserver, and then client tries to restart it only to discover that the window has now expired. This can be solved by @smarterclayton's heartbeat idea.
B) Suppose a watch returns a result, and client re-watches but the window has expired. This can only happen if the client is extremely slow to re-watch and is a client problem. Possible solution involves getting our etcd client to use etcd's streaming feature. This really shouldn't be happening, though.

Right - hopefully with 0.6.0 we can tweak retention to ridiculous levels (weeks) and this problem goes away for virtually everyone.  I don't know for sure though whether the history traversal will be efficient based on the data structure in the proposed tree.  Something else to raise on the store issue.
----- Original Message -----

I'm downgrading this to P1 because I think Brendan's change should allow it
to recover.
As far as I understand it, there's two possible further problems here.
A) Suppose a watch times out between a client and apiserver, and then client
tries to restart it only to discover that the window has now expired. This
can be solved by @smarterclayton's heartbeat idea.
B) Suppose a watch returns a result, and client re-watches but the window has
expired. This can only happen if the client is extremely slow to re-watch
and is a client problem. Possible solution involves getting our etcd client
to use etcd's streaming feature. This really shouldn't be happening, though.

Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69072810

Running v0.8.0 and still seeing kube-proxy get into this state. The following two messages are being repeated in the logs. The etcd indexes are not changing.
W0112 16:14:27.835652    3169 api.go:198] Got error status on WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/5338]) [11780] Reason: Details:<nil> Code:0}
W0112 16:14:54.260622    3169 api.go:129] Got error status on WatchServices channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/3907]) [11780] Reason: Details:<nil> Code:0}
W0112 16:14:59.510727    3169 api.go:198] Got error status on WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/5338]) [11780] Reason: Details:<nil> Code:0}
W0112 16:15:26.213788    3169 api.go:129] Got error status on WatchServices channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/3907]) [11780] Reason: Details:<nil> Code:0}

Hrmph!
I'll look at this today.
Brendan
On Jan 12, 2015 8:16 AM, "Steve Reed"<EMAIL_ADDRESS>wrote:

Running v0.8.0 and still seeing kube-proxy get into this state. The
following two messages are being repeated in the logs. The etcd indexes are
not changing.
W0112 16:14:27.835652    3169 api.go:198] Got error status on WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/5338]) [11780] Reason: Details: Code:0}
W0112 16:14:54.260622    3169 api.go:129] Got error status on WatchServices channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/3907]) [11780] Reason: Details: Code:0}
W0112 16:14:59.510727    3169 api.go:198] Got error status on WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/5338]) [11780] Reason: Details: Code:0}
W0112 16:15:26.213788    3169 api.go:129] Got error status on WatchServices channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [10781/3907]) [11780] Reason: Details: Code:0}
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69595097
.

Satnam said he saw it too.
On Jan 12, 2015 8:51 AM, "Brendan Burns"<EMAIL_ADDRESS>wrote:

Hrmph!
I'll look at this today.
Brendan
On Jan 12, 2015 8:16 AM, "Steve Reed"<EMAIL_ADDRESS>wrote:

Running v0.8.0 and still seeing kube-proxy get into this state. The
following two messages are being repeated in the logs. The etcd indexes
are
not changing.
W0112 16:14:27.835652 3169 api.go:198] Got error status on
WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:}
ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure
Message:401: The event in requested index is outdated and cleared (the
requested history has been cleared [10781/5338]) [11780] Reason:
Details: Code:0}
W0112 16:14:54.260622 3169 api.go:129] Got error status on WatchServices
channel: &{TypeMeta:{Kind: APIVersion:}
ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure
Message:401: The event in requested index is outdated and cleared (the
requested history has been cleared [10781/3907]) [11780] Reason:
Details: Code:0}
W0112 16:14:59.510727 3169 api.go:198] Got error status on
WatchEndpoints channel: &{TypeMeta:{Kind: APIVersion:}
ListMeta:{SelfLink:/api/v1beta1/endpoints ResourceVersion:} Status:Failure
Message:401: The event in requested index is outdated and cleared (the
requested history has been cleared [10781/5338]) [11780] Reason:
Details: Code:0}
W0112 16:15:26.213788 3169 api.go:129] Got error status on WatchServices
channel: &{TypeMeta:{Kind: APIVersion:}
ListMeta:{SelfLink:/api/v1beta1/services ResourceVersion:} Status:Failure
Message:401: The event in requested index is outdated and cleared (the
requested history has been cleared [10781/3907]) [11780] Reason:
Details: Code:0}
--
Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69595097>

.

--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69602018
.

Hrm. We should make an e2e test that writes a ton of events and then kills a kube-proxy to see if it recovers.

 I haven't quite gotten to the point of managing my own deployment yet. Could try if it would really help... : )
Just want to confirm my comment above was indeed related. I went to each node and ran:
sudo service kube-proxy restart

And then magically all my stress and headaches over the last few days melted away, all my apps were suddenly working!!!

I brought this up on IRC over the very slow new year holiday, @smarterclayton mentioned this should only really be possible under a high write volume, I presume on the api server or etcd?

Regarding workload size I can say that in my case I think I am doing very very very little. Just launching apps, testing how the API and networking works/feels.

@brendandburns it'll take a little work/time but yeah I can do that. I'll report back in a few either way in case something else comes up.

We're working on writing a test to reproduce this bug, so that we can be
sure we fixed this, this time.
--brendan
On Mon, Jan 12, 2015 at 1:47 PM, Steve Reed<EMAIL_ADDRESS>wrote:

@brendandburns https://github.com/brendandburns it'll take a little
work/time but yeah I can do that. I'll report back in a few either way in
case something else comes up.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69652487
.

@brendanburns : )

@brendandburns would it be valid to just manually copy kube-proxy out to my minions and restart? Or would you rather I do a full cluster tear-down?

Nevermind, disks are full on my minions so I'll be tearing it down anyway.

Is it our fault that your disks are full?
--brendan
On Mon, Jan 12, 2015 at 2:07 PM, Steve Reed<EMAIL_ADDRESS>wrote:

Nevermind, disks are full on my minions so I'll be tearing it down anyway.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69655977
.

Ok I'm up and running on a 0.8.0 cluster with the patch from #3410. So far so good.
The full disks are likely due to an issue starting up cadvisor that @vmarmol was helping out with over IRC (no github issue to track AFAIK).

Just triggered this by doing a ton of replication controller resizing, saw the "outdated index" messages in the logs and it looks like kube-proxy recovered and my services are still working fine.
cc @brendandburns @jasonkuhrt

Ok, so you have empirical validation that this seems to fix the problem?
Thanks!
--brendan
On Mon, Jan 12, 2015 at 3:57 PM, Steve Reed<EMAIL_ADDRESS>wrote:

Just triggered this by doing a ton of replication controller resizing, saw
the "outdated index" messages in the logs and it looks like kube-proxy
recovered and my services are still working fine.
cc @brendandburns https://github.com/brendandburns @jasonkuhrt
https://github.com/jasonkuhrt
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69671305
.

Correct. I deployed a few hundred pods to get over the 1000-events-in-etcd threshold, saw the error in the kube-proxy log, but only once.
None of my services are unreachable at this point.
I can see nothing else that indicates that this is still an issue. Thanks for the fix!

Ok, cool, given that, I'm going to cut a 0.8.1 with this cherry-picked in
and get the release out.  We'll still get a test in place too.
--brendan
On Mon, Jan 12, 2015 at 4:14 PM, Steve Reed<EMAIL_ADDRESS>wrote:

Correct. I deployed a few hundred pods to get over the 1000-events-in-etcd
threshold, saw the error in the kube-proxy log, but only once.
None of my services are unreachable at this point.
I can see nothing else that indicates that this is still an issue. Thanks
for the fix!
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-69673094
.

Ah, just when I noticed the bug, I come here and it's fixed. Awesome!!

The error message is logged weirdly - fixing that via https://github.com/GoogleCloudPlatform/kubernetes/pull/3608

I just ran across this the other day.  but I am seeing the error in the kube-controller-manager service.  Here is  a sample of the output.
Jan 18 15:32:34 aboutme-kube-master kube-controller-manager[865]: E0118 15:32:34.057766     865 replication_controller.go:128] unexpected object: &api.Status{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ListMeta:api.ListMeta{SelfLink:"/api/v1beta1/replicationControllers", ResourceVersion:""}, Status:"Failure", Message:"401: The event in requested index is outdated and cleared (the requested history has been cleared [130635/2384]) [131634]", Reason:"", Details:(*api.StatusDetails)(nil), Code:0}
Jan 18 15:32:24 aboutme-kube-master kube-controller-manager[865]: E0118 15:32:24.038972     865 replication_controller.go:128] unexpected object: &api.Status{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ListMeta:api.ListMeta{SelfLink:"/api/v1beta1/replicationControllers", ResourceVersion:""}, Status:"Failure", Message:"401: The event in requested index is outdated and cleared (the requested history has been cleared [130626/2384]) [131625]", Reason:"", Details:(*api.StatusDetails)(nil), Code:0}
Jan 18 15:32:14 aboutme-kube-master kube-controller-manager[865]: E0118 15:32:14.016047     865 replication_controller.go:128] unexpected object: &api.Status{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ListMeta:api.ListMeta{SelfLink:"/api/v1beta1/replicationControllers", ResourceVersion:""}, Status:"Failure", Message:"401: The event in requested index is outdated and cleared (the requested history has been cleared [130614/2384]) [131613]", Reason:"", Details:(*api.StatusDetails)(nil), Code:0}

I can get the behavior to go away temporarily by rebooting the node or restarting the service but it always seems to come back eventually.

Yeah, we may very well have the same problem in the controller-manager
watch code.  I will check it out.
--brendan
On Tue, Jan 20, 2015 at 8:29 AM, Josh Reichardt<EMAIL_ADDRESS>wrote:

I just ran across this the other day. but I am seeing the error in the
kube-controller-manager service. Here is a sample of the output.
Jan 18 15:32:34 aboutme-kube-master kube-controller-manager[865]: E0118 15:32:34.057766     865 replication_controller.go:128] unexpected object: &api.Status{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ListMeta:api.ListMeta{SelfLink:"/api/v1beta1/replicationControllers", ResourceVersion:""}, Status:"Failure", Message:"401: The event in requested index is outdated and cleared (the requested history has been cleared [130635/2384]) [131634]", Reason:"", Details:(*api.StatusDetails)(nil), Code:0}
Jan 18 15:32:24 aboutme-kube-master kube-controller-manager[865]: E0118 15:32:24.038972     865 replication_controller.go:128] unexpected object: &api.Status{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ListMeta:api.ListMeta{SelfLink:"/api/v1beta1/replicationControllers", ResourceVersion:""}, Status:"Failure", Message:"401: The event in requested index is outdated and cleared (the requested history has been cleared [130626/2384]) [131625]", Reason:"", Details:(*api.StatusDetails)(nil), Code:0}
Jan 18 15:32:14 aboutme-kube-master kube-controller-manager[865]: E0118 15:32:14.016047     865 replication_controller.go:128] unexpected object: &api.Status{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ListMeta:api.ListMeta{SelfLink:"/api/v1beta1/replicationControllers", ResourceVersion:""}, Status:"Failure", Message:"401: The event in requested index is outdated and cleared (the requested history has been cleared [130614/2384]) [131613]", Reason:"", Details:(*api.StatusDetails)(nil), Code:0}
I can get the behavior to go away temporarily by rebooting the node or
restarting the service but it always seems to come back eventually.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-70683625
.

Closing this as the original issue is fixed.  Opened:
https://github.com/GoogleCloudPlatform/kubernetes/issues/3829
To track the work in replication controller.

I'm not sure what I did wrong but I still get these kind error message widely across my kubernetes components, including kube-proxy, kube-scheduler, kube-controller-manager, kube-apiserver, kubelet.
When observing the logs, it show ups about every 30 seconds.
The error message is as following:
kube-apiserver:
Jul 21 13:51:21 dsp-01 kube-apiserver[4276]: E0721 13:51:21.878118    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [365267/365129]) [366266]
Jul 21 14:26:06 dsp-01 kube-apiserver[4276]: E0721 14:26:06.803953    4276 reflector.go:183] watch of *api.ResourceQuota ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366150/365129]) [367149]
Jul 21 14:29:22 dsp-01 kube-apiserver[4276]: E0721 14:29:22.056964    4276 reflector.go:183] watch of *api.Secret ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366229/365129]) [367228]
Jul 21 14:34:09 dsp-01 kube-apiserver[4276]: E0721 14:34:09.877753    4276 reflector.go:183] watch of *api.LimitRange ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366347/365129]) [367346]
Jul 21 14:35:01 dsp-01 kube-apiserver[4276]: E0721 14:35:01.630514    4276 reflector.go:183] watch of *api.ServiceAccount ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366368/365129]) [367367]
Jul 21 14:36:38 dsp-01 kube-apiserver[4276]: E0721 14:36:38.057345    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366407/365129]) [367406]
Jul 21 14:44:57 dsp-01 kube-apiserver[4276]: E0721 14:44:57.302124    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366611/366267]) [367610]
Jul 21 15:26:13 dsp-01 kube-apiserver[4276]: E0721 15:26:13.253458    4276 reflector.go:183] watch of *api.Secret ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367622/367229]) [368621]
Jul 21 15:28:18 dsp-01 kube-apiserver[4276]: E0721 15:28:18.727808    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367672/367407]) [368671]
Jul 21 15:30:19 dsp-01 kube-apiserver[4276]: E0721 15:30:19.093392    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367722/367613]) [368721]
Jul 21 15:32:14 dsp-01 kube-apiserver[4276]: E0721 15:32:14.809743    4276 reflector.go:183] watch of *api.LimitRange ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367769/367348]) [368768]

other components are similar, if needed I can provide their error messages later.
I'm running kubernetes 1.0, to avoid some bugs at version 0.19, the accurate version info is
Client Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-15-g2c27b1fa64f4e7", GitCommit:"2c27b1fa64f4e70f04575d1b217494f49332390e", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-15-g2c27b1fa64f4e7", GitCommit:"2c27b1fa64f4e70f04575d1b217494f49332390e", GitTreeState:"clean"}

I yum installed kubernets from here
System info:
[root@dsp-01 ~]# uname -a
Linux dsp-01 3.10.63-11.el7.x86_64 #1 SMP Tue Apr 7 16:47:43 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Tried my best to solve the problem on my own but got no progress, if I should provide any other information or investigate something please let me know.

/sub

Seeing this periodically every five minutes or so is not actually a
problem; the system will just list and watch again. We are working on some
changes that will make this much less frequent. But until then, don't worry
about this unless it's happening like once a second.
On Jul 21, 2015 1:00 AM, "Jay Liang"<EMAIL_ADDRESS>wrote:

I'm not sure what I did wrong but I still get these kind error message
widely across my kubernetes components, including kube-proxy,
kube-scheduler, kube-controller-manager, kube-apiserver, kubelet.
When observing the logs, it show ups about every 30 seconds.
The error message is as following:
kube-apiserver:
Jul 21 13:51:21 dsp-01 kube-apiserver[4276]: E0721 13:51:21.878118    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [365267/365129]) [366266]
Jul 21 14:26:06 dsp-01 kube-apiserver[4276]: E0721 14:26:06.803953    4276 reflector.go:183] watch of *api.ResourceQuota ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366150/365129]) [367149]
Jul 21 14:29:22 dsp-01 kube-apiserver[4276]: E0721 14:29:22.056964    4276 reflector.go:183] watch of *api.Secret ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366229/365129]) [367228]
Jul 21 14:34:09 dsp-01 kube-apiserver[4276]: E0721 14:34:09.877753    4276 reflector.go:183] watch of *api.LimitRange ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366347/365129]) [367346]
Jul 21 14:35:01 dsp-01 kube-apiserver[4276]: E0721 14:35:01.630514    4276 reflector.go:183] watch of *api.ServiceAccount ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366368/365129]) [367367]
Jul 21 14:36:38 dsp-01 kube-apiserver[4276]: E0721 14:36:38.057345    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366407/365129]) [367406]
Jul 21 14:44:57 dsp-01 kube-apiserver[4276]: E0721 14:44:57.302124    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [366611/366267]) [367610]
Jul 21 15:26:13 dsp-01 kube-apiserver[4276]: E0721 15:26:13.253458    4276 reflector.go:183] watch of *api.Secret ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367622/367229]) [368621]
Jul 21 15:28:18 dsp-01 kube-apiserver[4276]: E0721 15:28:18.727808    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367672/367407]) [368671]
Jul 21 15:30:19 dsp-01 kube-apiserver[4276]: E0721 15:30:19.093392    4276 reflector.go:183] watch of *api.Namespace ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367722/367613]) [368721]
Jul 21 15:32:14 dsp-01 kube-apiserver[4276]: E0721 15:32:14.809743    4276 reflector.go:183] watch of *api.LimitRange ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [367769/367348]) [368768]
other components are similar, if needed I can provide their error messages
later.
I'm running kubernetes 1.0, to avoid some bugs at version 0.19, the
accurate version info is
Client Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-15-g2c27b1fa64f4e7", GitCommit:"2c27b1fa64f4e70f04575d1b217494f49332390e", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-15-g2c27b1fa64f4e7", GitCommit:"2c27b1fa64f4e70f04575d1b217494f49332390e", GitTreeState:"clean"}
I yum installed kubernets from here
http://cbs.centos.org/kojifiles/packages/kubernetes/0.19.0/0.4.gita8269e3.el7/
System info:
[root@dsp-01 ~]# uname -a
Linux dsp-01 3.10.63-11.el7.x86_64 #1 SMP Tue Apr 7 16:47:43 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Tried my best to solve the problem on my own but got no progress, if I
should provide any other information or investigate something please let me
know.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-123205771
.

What if i owed a cluster which is completely free of this problem while another suffers?

So seeing this message is actually ok?
If so, I suggest to mark it as warn instead of error
As a user, I'll try to clear out all error messages in case unknown failure afterwards.
Thanks for your reply.

I expect this to be more common in a more heavily loaded cluster. Does that
explain it?
On Jul 25, 2015 6:23 PM, "Emma He"<EMAIL_ADDRESS>wrote:

What if i owed a cluster which is completely free of this problem while
another suffers?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-124925366
.

Yeah, we should change this.
On Jul 25, 2015 11:48 PM, "Jay Liang"<EMAIL_ADDRESS>wrote:

So seeing this message is actually ok?
If so, I suggest to mark it as warn instead of error
As a user, I'll try to clear out all error messages in case unknown
failure afterwards.
Thanks for your reply.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-124951246
.

Nope, exactly the opposite, which make it much odder to me. @lavalamp
(Different authentication configuration,tough.)

Are both clusters running the same version?
On Jul 26, 2015 5:23 PM, "Emma He"<EMAIL_ADDRESS>wrote:

Nope, exactly the opposite, which make it much odder to me. @lavalamp
https://github.com/lavalamp
(Different authentication configuration,tough.)
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/3216#issuecomment-125051485
.

The one suffers v1.0.1, while the healthy one v0.18.2.


Vet clinic database: query multiple tables

In this branch, I have added the following;

[ ] Create a table named owners with the following columns:

id: integer (set it as autoincremented PRIMARY KEY)
full_name: string
age: integer

[ ] Create a table named species with the following columns:

id: integer (set it as autoincremented PRIMARY KEY)
name: string

[ ] Modify animals table:

Make sure that id is set as autoincremented PRIMARY KEY
Remove column species
Add column species_id which is a foreign key referencing species table
Add column owner_id which is a foreign key referencing the owners table
Add all this in the schema.sql file.

[ ] Insert the following data into the owners table:

Sam Smith 34 years old.
Jennifer Orwell 19 years old.
Bob 45 years old.
Melody Pond 77 years old.
Dean Winchester 14 years old.
Jodie Whittaker 38 years old.

[ ] Insert the following data into the species table:

Pokemon
Digimon

[ ] Modify your inserted animals so it includes the species_id value:

If the name ends in "mon" it will be Digimon
All other animals are Pokemon

[ ] Modify your inserted animals to include owner information (owner_id):

Sam Smith owns Agumon.
Jennifer Orwell owns Gabumon and Pikachu.
Bob owns Devimon and Plantmon.
Melody Pond owns Charmander, Squirtle, and Blossom.
Dean Winchester owns Angemon and Boarmon.

[ ] Add these insertions and modify the data.sql file.

[ ] Write queries (using JOIN) to answer the following questions:

What animals belong to Melody Pond?
List of all animals that are pokemon (their type is Pokemon).
List all owners and their animals, remember to include those that don't own any animal.
How many animals are there per species?
List all Digimon owned by Jennifer Orwell.
List all animals owned by Dean Winchester that haven't tried to escape.
Who owns the most animals?

[ ] Add all these written in the queries.sql file.

[ ] Screenshot of the results of your queries.

Hi @TuyishimireEric, Thank you! :hugs:


The influence of different configurations of data-masking/encrypt on show-SQL

Question Description：

The configuration of picture-1  make the configuration of  picture-3 doesn't work.
The configuration of  picture-2 may not.
It has no effect on encrypt/decrypt.
My environment：
springboot + JPA + mysql +sharding-jbdc
Which version of ShardingSphere did you use?
4.0.0-RC1
Expected behavior
just show  sql
Actual behavior
cann't see sql be printed
Reason analyze
Solution：
Just config like picture-2，and wait for the next version
Thank you!

Thank you for suggestion, does anybody want to do a pull request on this?

Hi @terrymanu
I will try it, This is my preliminary idea.
1,Refactoring the constructor of EncryptDataSource  and add java.util.Properties parameters, similar to MasterSlaveDataSource;
2,Refactoring the createDataSource method of EncryptDataSourceFactory  and add java.util. Properties parameters,similar to MasterSlaveDataSourceFactory;
3,Refactoring the constructor of  and add  ShardingProperties parameters;
4,Modify the getSQLUnit method of EncryptPreparedStatement  to print the rewritten SQL log according to configuration information;
5,Modify the getRewriteSQL method of EncryptStatement  to print therewritten SQL log according to configuration information;

Ok, let @tristaZero to review it.

@zhouchangjiang Hi, i think your preliminary idea is great, you can have a try firstly.
Moverover please consider different configuration forms , e.g. java code, yaml, springboot, spring-namespace.

Ok,I get it,Thank you! @tristaZero

@zhouchangjiang  Hi, Is everything ok? Any problems, please contact me.


Simplify ADVI interface (part 2)

This is step 2 in refactoring our ADVI interface before 0.2 release (after #1027), as suggested by @eb8680.
After this PR we can simply
def model():
    ...

guide = AutoMultivariateNormal(model)
svi = SVI(model, guide, Adam({}), Trace_ELBO())

ready to merge.


ISPN-13606 Add embedded jdbc tests into integrationtests/server-integration/third-party-server

https://issues.redhat.com/browse/ISPN-13606

My 2 cents.

The tests are using modules that are doing internal tests, while the idea of the module integrationtests/server-integration is to test integration things. Like: configuring a store and doing calls via cache API
The modules that were added are also using TestNG, while the integration tests are using JUnit. It is possible run it but with workarounds.
The dependencies should be added as a canonical maven dependency instead of using the relative path .addAsLibrary(new File("target/test-libs/infinispan-cachestore-jdbc-tests.jar"))

Closing in favor of #9904


Update requirements.text scipy version

Hi @borhanMorphy ,
Is there a reason for a using a specific version of scipy in requirements.txt. Can we update it to the latest?
I am getting the following error while trying to install scipy==1.4.1.

Hey @ananyahjha93 ,
thank you for reporting that. It appears that there aren't any scipy distributions when using python 3.9 . I just release a new patch 0.1.3. It should solve your problem. Update the package with pip install fastface -U.
Let me know if the problem occurs again.

Thanks @borhanMorphy. This was helpful. I am finishing up the Flash task-a-thon PR for integrating this library. Just adding tests and then it should be good to go.

Thanks @borhanMorphy. This was helpful. I am finishing up the Flash task-a-thon PR for integrating this library. Just adding tests and then it should be good to go.

Oh, thanks! Glad to hear that


Words of compliments

I like your code style, readability, clear separation of concerns.
My suggestion would be to avoid market orders.
Something can go wrong during the process, the market can move adversely
I suggest limit orders here: https://github.com/bmino/binance-triangle-arbitrage/blob/master/src/main/BinanceApi.js#L42-L70
I’m actually figuring out how to analyze all triangular arbitrage pairs on an exchange that has multiple base currencies.

USD
BTC
ETH
EUR
JPY
GBP

Also thinking if there is an opportunity of polygonal (4 trades, 5 trades) arbitrage.
Another view could be:

“if 4-trade arbitrage exists, then 3-trade arbitrage is possible too?”

TRUE or FALSE?

Well thank you!
Neat point on the polygonal trades as that is definitely a thing. Perhaps we can shift this codebase to be shape agnostic in the future.
As per market orders, I actually disagree with ya here. When purchasing higher values you are exhausting multiple levels of an orderbook. So if you are buying an asset, the lowest ask will be exhausted, then the second, third, etc. Market orders do exactly this where limit orders would require multiple orders to be placed per leg in the position. Perhaps you could argue that limit orders are more secure in that the leg would fail quickly if a depth is exhausted before the bot can grab it meaning the position is not the same as calculated

About the limit / market orders: you can specify the limit order above the ticker price.
Example:
ASKS

amount
price

500
0.0011

200
0.0012

1000
0.0013

You can simply create limit order for 1000 at 0.0013
If you were to create market order and market moves against you, then you could end up spending too much.

I still do not know the thing about polygonal trades... If polygonal exist, does the simpler triangular exist too?

Interesting

Interesting indeed. Okay so I was unaware that limit orders placed above the spread are filled in that manner.
I like that it guarantees the calculated quantity and quote. The deal breaker is verifying that the transaction has completed successfully. A market order http call returns (200 code) when the order is filled but a limit order might return when the order is not yet filled. Checking for such a case will use precious time mid-execution. Thoughts on this?

It's better to check, lose 0.5s but avoid market order when it moves in another direction.
Testing will tell.

The ways I know to check:

Listen for order update websocket and filter for trade id
Query trade history endpoint on a loop until order has been filled

This will put a delay for all three steps in the execution (effectively slowing down the first and second execution). Depending on how long this takes it might be unacceptable. This solution was suggested to avoid interacting with a market that has moved, but by introducing more time to verify each step has completed will allow the market to move further


persistent mode (-P) makes fuzzing results worse?

im running honggfuzz on persistent mode (-P) and I am having several logs like `2019-07-23T05:13:10-0700][W][34396] arch_checkWait():249 Persistent mode: pid=11688 exited with status: EXITED, exit code: 1
I am not sure if thats alright or it is damaging the results of the fuzzing? Should I stop using -P ?
I've compiled the source code with hfuzz-clang etc.

Persistent mode requires special fuzzed binary setup - it's described here https://github.com/google/honggfuzz/blob/master/docs/PersistentFuzzing.md


Adding system tests for Bigtable Table API.

Seems like pylint is yelling at you in travis.

Filed #1526 so I remember to re-enable the thing I'm about to pylint: disable once grpcio is sane to install.

Cool. This LGTM once you get Travis to stop grumbling.


Build.ps1 does not build a freshly cloned repo

Problem description
Build.ps1 -ResolveDependency does not build a freshly cloned repo
Verbose logs
Verbose output is too long for GitHub. Pasting only the last portion which ran into the error.

===============================================================================
                        COMPILEDATUMRSOP

-------------------------------------------------------------------------------
  /./build/CompileDatumRsop
  D:\tmp\DscWorkshop\output\RequiredModules\Sampler.DscPipeline\0.1.1\tasks\CompileDatumRsop.build.ps1:39

        Project Name               = ''
        Source Path                = 'source'
        Output Directory           = 'D:\tmp\DscWorkshop\output'
        Release Notes path         = 'D:\tmp\DscWorkshop\output\ReleaseNotes.md'
        Built Module Subdirectory  = 'D:\tmp\DscWorkshop\output\AvoidPSPathOverlap'
        Module Manifest Path (src) = 'C:\Users\janhe\source\.psd1'
VERBOSE: Module version is not determined yet. Evaluating methods to get module version.
VERBOSE: GitVersion is not installed. Trying to use the version from module manifest in path 'C:\Users\janhe\source\.psd1'.
ERROR: Cannot find path 'C:\Users\janhe\source\.psd1' because it does not exist.
At C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Utility\Microsoft.PowerShell.Utility.psm1:544 char:35
+         foreach($resolvedPath in (Resolve-Path @PSBoundParameters))
+                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At D:\tmp\DscWorkshop\output\RequiredModules\Sampler.DscPipeline\0.1.1\tasks\CompileDatumRsop.build.ps1:39 char:1
+ task CompileDatumRsop {
+ ~~~~~~~~~~~~~~~~~~~~~~~
At D:\tmp\DscWorkshop\build.ps1:331 char:13
+             task $workflow $workflowItem
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At D:\tmp\DscWorkshop\build.ps1:331 char:13
+             task $workflow $workflowItem
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Build FAILED. 7 tasks, 1 errors, 0 warnings 00:00:14.2525590
Resolve-Path : Cannot find path 'C:\Users\janhe\source\.psd1' because it does not exist.
At C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Utility\Microsoft.PowerShell.Utility.psm1:544 char:35
+         foreach($resolvedPath in (Resolve-Path @PSBoundParameters))
+                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\janhe\source\.psd1:String) [Resolve-Path], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.ResolvePathCommand

[00:19:30]PS /> $pwd

Path
----
D:\tmp\DscWorkshop

How to reproduce
$VerbosePreference = 'Continue'
git clone git@github.com:dsccommunity/DscWorkshop.git
cd DscWorkshop
.\Build.ps1 -ResolveDependency
Expected behavior
Freshly cloned main branch should always build successfully.
Current behavior
Build.ps1 -ResolveDependency does not build a freshly cloned repo
Suggested solution
None. The build process has become too complex to troubleshoot since migrating to Sampler.
Operating system the target node is running
OsName               : Microsoft Windows 11 Pro
OsOperatingSystemSKU : 48
OsArchitecture       : 64-bit
WindowsVersion       : 2009
WindowsBuildLabEx    : 22000.1.amd64fre.co_release.210604-1628
OsLanguage           : en-US
OsMuiLanguages       : {en-US, de-DE, en-GB}

PowerShell version and build the target node is running
Name                           Value
----                           -----
PSVersion                      5.1.22000.282
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22000.282
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

Module version used
???
CommonTasks2 is not a module and does not exist. The code in use was the latest commit in the main branch:

[00:00:01]PS /> git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
[00:00:00]PS /> git log -n 1 --oneline
274d7ef (HEAD -> main, origin/main, origin/HEAD) Migration to sampler (#121)

Successfully built, thanks @raandree


Massive's db or sets import

As Massive is being phased out, it would be awesome if we could expect to be able to export its db and import it into Flexify.
It's also probably easier to get on with it to support the import of more and more fields as dev goes along than adding support for all imports when the new app becomes full fledged.
If db migration to the new app is too much of a headache, being able to import sets from legacy (Massive) CSV exports would be enough.
Thanks so much for these apps, and for open sourcing it. The philosophy is great, graphs look super promising, timer is awesome and done right, UI overall is sleek and minimalist which is ideal for a workout log app really.

CSV import/export are already supported. Migration is outlined on the README. Its in the settings page.

My apologies, I tried to import and I had to edit entries. But I didn't follow the migration steps. Thanks.

@brandonp2412

CSV import/export are already supported. Migration is outlined on the README. Its in the settings page.

As of now, it seems that one would have to modify the sets.csv file from massive for it to import successfully (plans.csv does import correctly). You'd need to remove the quotations and add one decimal place to each entry's number.
For example:
"id","name","reps","weight","created","unit","hidden","image","sets","minutes","seconds","steps"
"1","Test","0","0","2024-04-24T14:39:39",,"0",,"3","3","30",

Would need to be modified to:
"id","name","reps","weight","created","unit","hidden","image","sets","minutes","seconds","steps"
"1","Test",0.0,0.0,"2024-04-24T14:39:39",,0.0,,3.0,3.0,30.0,

I had this issue when doing my own migration earlier.

I have now upgraded to latest version and I am trying to import sets from the Massive. The above instructions are not working and trying to import the unedited CSV doesn't work either. I just get records with numbers in it.

Please provide a minimal reproducible sample (paste the CSV here) and list the app version this occurs on

I have now upgraded to latest version and I am trying to import sets from the Massive. The above instructions are not working and trying to import the unedited CSV doesn't work either. I just get records with numbers in it.

Could you try removing the quotations from the ID if they are present.
For example:
"id","name","reps","weight","created","unit","hidden","image","sets","minutes","seconds","steps"
1,"Test",0.0,0.0,"2024-04-24T14:39:39",,0.0,,3.0,3.0,30.0,

I just did another test with the latest patch and that seemed to fix an import issue for me.

I have now upgraded to latest version and I am trying to import sets from the Massive. The above instructions are not working and trying to import the unedited CSV doesn't work either. I just get records with numbers in it.

Could you try removing the quotations from the ID if they are present.
For example:
"id","name","reps","weight","created","unit","hidden","image","sets","minutes","seconds","steps"
1,"Test",0.0,0.0,"2024-04-24T14:39:39",,0.0,,3.0,3.0,30.0,

I just did another test with the latest patch and that seemed to fix an import issue for me.

This sample csv worked for 1.0.69. What version are you using?

I am currently running 1.0.69.
This is my edited sets CSV:
"id","name","reps","weight","created","unit","hidden","image","sets","minutes","seconds","steps"
10,"Seated Row",5.00,50.00,"2024-01-08T18:30:30.000Z",,0.00,,3.00,3.00,30.00,
11,"Seated Row",5.00,45.00,"2024-01-08T18:30:17.000Z",,0.00,,3.00,3.00,30.00,

I import this into plans. Nothing gets imported.
This is the original file:
"id","name","reps","weight","created","unit","hidden","image","sets","minutes","seconds","steps"
"10","Seated Row","5","50","2024-01-08T18:30:30.000Z","kg","0","","3","3","30",
"11","Seated Row","5","45","2024-01-08T18:30:17.000Z","kg","0","","3","3","30",

I've tried to import my sets into plans and graphs. Which one should I use?
I also don't use plans. I just track what every movement I can do that session.

I also don't use plans. I just track what every movement I can do that session.

This app doesn't have a free-form input for sets so this style of working out is probably not best for Flexify.

I also don't use plans. I just track what every movement I can do that session.

This app doesn't have a free-form input for sets so this style of working out is probably not best for Flexify.

Checking the README , was led to believe the countary:

Track Gym Progress: Flexify allows you to effortlessly track your gym progress. Log your workouts, sets, reps, and weights with ease

I assumed Flexify would allow the tracking of specific sets and reps and weights as Massive does.
As a user I would really value being able to do what Massive was capable of doing very well: the tracking of individual workouts off plan templates. Where I can add the entity of a workout off a template represented by sets, number of reps and weight for each. And that when adding the tracking of a set it auto complete reps and weight from the latest tracking. It is magically convenient and powerful in Massive.
As a user, I don't care much what time my workout took place in the graphs, what matter most to me is evolution over weeks, months, years. I want to visualize how many reps and weight I performed, how long workouts last, frequency of workouts, over time.
Sorry to dump a set of desire in this thread but I thought it was adequate as the issue is concerning users migrating or hoping to want to migrate from Massive, eventually.
Of course I'm biased and train a particular way, maybe I do not understand how Flexify approach to tracking with just 'Plan' effectively is more hopeful. And of course this is your project and you should take the directions you believe is the best.

Free form adding of sets is something I could easily add but this issue is about importing things via CSV. Make a separate issue if that's a feature you want

I also don't use plans. I just track what every movement I can do that session.

This app doesn't have a free-form input for sets so this style of working out is probably not best for Flexify.

I appreciate that. I was going to try and use plans and if it wasn't viable for me, switch back to massive.
Am I doing this all wrong then? I cannot just import all my sets from Massive into Flexify? Do the sets need to be associated to a plan?

@tomas-samot

Am I doing this all wrong then? I cannot just import all my sets from Massive into Flexify? Do the sets need to be associated to a plan?

Also am I importing my sets.csv into the correct data import type? Should it go into plans or graphs?

I managed to test importing on an empty session and both examples you've given work. They need to be imported as "Graphs"
They don't seem to require a plan associated to import the history. However you will not be able to add new sets until you create a plan afterward.

@Dermanthi
When I try to import the full edited sets.csv into graphs I get this error:
RangeError (index): Invalid value: Only valid value is 0: 1

Are you testing using the sample you provided, or a much larger file?

I'm using the full file. which has 981 lines. Damn, the small file does import. I just tried it now. Is that too many lines?

My own file had 1080 lines, and that imported fine, so I doubt that would be the issue.
There's got to be something, somewhere in the csv that just won't work. Perhaps if you attach the full file, I could take a look.

I found out what the problem is. I was editing the file on a Windows machine and the CSV file was saved with Windows-1252 encoding. I've edit now on my Linux machine with UTF-8 encoding. Imported the whole 981 lines files without issue. Expect maybe a pectoral fly came in at 119.05kg when the max I've done is 110kg (on a machine).

Expect maybe a pectoral fly came in at 119.05kg when the max I've done is 110kg (on a machine).

I'd suggest now clearing the Flexify history and importing a freshly made csv generated from Massive without editing it at all. See if that works. Just make sure you're using the latest version.
Although do take note that Massive has this issue where if sets.csv is already present it creates sets.csv (1) instead of sets (1).csv, which just means you might have to fix the file extension.

Must be the windows vs Linux encoding I've got to make a windows encoded file and figure out the fix for importing it

Must be the windows vs Linux encoding I've got to make a windows encoded file and figure out the fix for importing it

Not necessarily if you want people to just import from Massive directly. I suppose docs and a check to say the file is not UTF-8 encoded please resave in UTF-8 would be enough?

Expect maybe a pectoral fly came in at 119.05kg when the max I've done is 110kg (on a machine).

I'd suggest now clearing the Flexify history and importing a freshly made csv generated from Massive without editing it at all. See if that works. Just make sure you're using the latest version.
Although do take note that Massive has this issue where if sets.csv is already present it creates sets.csv (1) instead of sets (1).csv, which just means you might have to fix the file extension.

This process works now on version 1.0.69. Nice, thank you everyone for helping.
@brandonp2412 Thank you for Massive and Flexify.

Really struggling to replicate this bug but I have seen it before in the past. Here's what I tried:

Created this file and specified windows encoding + CRLF

Sent it to my emulator adb push documents/sets.csv /sdcard/Download
Imported it on my emulator successfully

Doing all of this from windows BTW

Even if I can't replicate it I can just push a release that tries both \n and \r\n line endings and then you could check if it fixes it?

It wasn't the line endings. I managed to make a file that wouldn't read properly. It must be to do with the file encodings like you said. My kotlin code reading the file probably needs to handle parsing different file encodings :cry:

I have changed the way file picking works, and this ticket has grown pretty big beyond the scope of what it should be doing. If someone else runs across this issue we can make a more targetet ticket for it.


Fix small typo

Thank you for this fix.

No worries! We also just posted about this on the official Charm twitter account:
https://twitter.com/charmcli/status/1478477879260459012
It's very cool!

Thank you very much for this. That is very nice of you.

No worries, it's really cool. Also, we have a slack if you ever have any Soft Serve/Charm questions: https://charm.sh/slack

Thanks for the link. I will look into it when I need it.


Drop unused generated clients

How to categorize this PR?

/area scalability dev-productivity
/kind cleanup
What this PR does / why we need it:
This PR drops some unused generated client code (extensions and operations group).
It also drops some unused clients (Garden{Core,SeedManagement,Operations} and API{Registration,Extension}) from kubernetes.Interface.
All of this is dropped in favor of (cached) controller-runtime clients to simplify/unify our code and leverage the clients' caches wherever possible.
Which issue(s) this PR fixes:
Part of https://github.com/gardener/gardener/issues/2414
Special notes for your reviewer:
kubernetes.Interface.Kubernetes() is not dropped in this PR, as there are still a lot of usages.
This can be tackled in a future PR.
The generated operations clients might be needed in the future, when we add some checks in admission plugins (e.g. for Bastions). I decided to drop them nevertheless for the sake of simplification. We can still add them back with a few lines later on if needed.
Release note:

I assume you plan to remove the other clients with subsequent PRs?

That won't be possible so easily, as they are used in admission plugins.
Currently, I'm not sure, if or how we can switch to controller-runtime clients there (tracked in https://github.com/gardener/gardener/issues/2414).
However, what is possible and planned is to remove usages of kubernetes.Interface.Kubernetes() (in all gcm/gardenlet code).

/squash


No Keyword existing for executing JavaScript and getting text of element in appium library

I want to execute JavaScript for various action get text value, swipe, scroll. So is it possible to have keyword like 'Execute JavaScript'.
Also there is no keyword existing for getting text value of element because I want fetch text value of element.
eg:
execute_script("mobile: swipe", {"startX": 0.9, "startY": 0.9, "endX": 0.9, "endY": 0.9, "endY": 3.0})

you can use Swipe

But how to get text of an element ???

try this?

Thank you so much for replying ...
I am using same library but there is no keyword for fetch text value like 'Get Text'.

Indeed there isn't.
I tried the suggested keyword and no joy. Basically, if we have something like this in an Android app:

17:54
Tuesday 1 March 2016

I want to be able to get the text "Tuesday 1 March 2016" (or whatever value it has at the time).
Using "Element Name Should Be" and "Element Value Should Be", I get fail result "Element 'css=#home-time' name should be 'Tuesday 1 March 2016' but it is 'None'."
Using " Element Attribute Should Match, css=#home-time, text, Tues* ", I get TypeError: expected string or buffer.
Can't seem to get actual text values like with the Get Text keyword in, say, Selenium2Library

Raising again in  Getting Elements Text Attribute #98

You can edit your _elements.py and add something like:
def get_element_text(self, locator):
element = self._element_find(locator, True, True)
element_text = element.text
self._info("Element '%s' text: %s " % (locator, element_text))
return element_text
Should work until they add something official

how does one go because the changes I make are not reflected in robot-framework
I added the keyword in _elements.py but robot framework does not recognize


Registered angular-jwt interceptor for X-Access-Token

@ehazlett
I was having a look to see if we can improve the authentication on the UI in some way.  I thought it'd be good to introduce an interceptor that fires whenever $http is called and includes our token.
When doing some research on how folks are doing this in the AngularJS world, JSON Web Tokens (JWT) were recommended.  I've used the angular-jwt package anyway, but tweaked it to add the header in the X-Access-Token format the Shipyard is currently familiar with.  Perhaps support for JWT on the server side would be handy, as that's token authentication stuff that we can re-use.. and there's a go package for it already.
JWT might also give us the ability to better support user/role based actions, as we can store the username, role and an expiry in the token that the UI can decode and make decisions based upon.
Let me know what you think on this. :smile:

Oh yeah -- and I'd say this is just a baby-step in the refactoring of the auth stuff that we have. :yum:

Nice!  I like the idea of this.  I will test this and take a look at the Go JWT package as well.

is this still valid for v3?

This work has been moved to V3.


Clarify new bucket store metrics

[ ] I added CHANGELOG entry for this change.
[x] Change is not relevant to the end user.

Changes
I found the description of the new metrics in the bucket store a bit difficult to understand. Does this change help to clarify it in your opinion?
Verification
N/A

Thanks @prmsrswt for your review. I've applied your suggestions.


Fix faux-HTTP client of SSO SSH endpoint to send \r\n as per spec

The new Apache httpd requires \r\n as a line ending where it
previously allowed \n.

@robotmaxtron gave 👍 in stg PR #2590


[HBASE-24956] ConnectionManager#locateRegionInMeta waits for user region lock indefinitely.

This is just the backport of  branch-2 PR. https://github.com/apache/hbase/pull/2322/
There are 2 differences compared to branch-2.

LockTimeoutException already exists in branch-1 so reusing that.
In TestMetaCache#testUserRegionLockThrowsException, I have set HBASE_CLIENT_RETRIES_NUMBER value to 1 whereas in branch-2, it was set to 0.
In branch-2, they have attempts and retries notion differently. It is calculating numAttempts = numRetries + 1 but in branch-1, numAttempts = numTries.
@bharathv please review branch-1 PR. Thank you !

re-trigged the build

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ branch-1 Compile Tests _

+0 :ok:
mvndep
2m 23s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
8m  2s
branch-1 passed

+1 :green_heart:
compile
1m  3s
branch-1 passed with JDK v1.8.0_262

+1 :green_heart:
compile
1m 11s
branch-1 passed with JDK v1.7.0_272

+1 :green_heart:
checkstyle
2m 14s
branch-1 passed

+1 :green_heart:
shadedjars
3m  3s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m  0s
branch-1 passed with JDK v1.8.0_262

+1 :green_heart:
javadoc
1m  7s
branch-1 passed with JDK v1.7.0_272

+0 :ok:
spotbugs
2m 41s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 16s
branch-1 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 18s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 55s
the patch passed

+1 :green_heart:
compile
1m  1s
the patch passed with JDK v1.8.0_262

+1 :green_heart:
javac
1m  1s
the patch passed

+1 :green_heart:
compile
1m  8s
the patch passed with JDK v1.7.0_272

+1 :green_heart:
javac
1m  8s
the patch passed

+1 :green_heart:
checkstyle
2m  6s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
2m 55s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
4m 43s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2.

+1 :green_heart:
javadoc
0m 52s
the patch passed with JDK v1.8.0_262

+1 :green_heart:
javadoc
1m  5s
the patch passed with JDK v1.7.0_272

+1 :green_heart:
findbugs
4m 59s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
2m 59s
hbase-client in the patch passed.

-1 :x:
unit
69m 29s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 46s
The patch does not generate ASF License warnings.

123m 15s

Reason
Tests

Failed junit tests
hadoop.hbase.regionserver.wal.TestWALReplayCompressed

hadoop.hbase.regionserver.TestHRegion

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2415

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 913980ebc570 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2415/out/precommit/personality/provided.sh

git revision
branch-1 / f0acafc

Default Java
1.7.0_272

Multi-JDK versions
/usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272

unit
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/2/artifact/out/patch-unit-hbase-server.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/2/testReport/

Max. process+thread count
4452 (vs. ulimit of 10000)

modules
C: hbase-client hbase-server U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/2/console

versions
git=1.9.1 maven=3.0.5 findbugs=3.0.1

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop.hbase.regionserver.wal.TestWALReplayCompressed
   hadoop.hbase.regionserver.TestHRegion

These 2 tests failed since it couldn't create new thread.
2020-09-18 09:46:47,445 ERROR [nioEventLoopGroup-6-1] logging.Slf4JLogger(181): Failed to submit a listener notification task. Event loop shut down?
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:717)
	at io.netty.util.concurrent.GlobalEventExecutor.startThread(GlobalEventExecutor.java:222)
	at io.netty.util.concurrent.GlobalEventExecutor.execute(GlobalEventExecutor.java:211)
	at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:760)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:748)

Running the tests locally and both of them succeeded.
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 127.843 s - in org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
[INFO] Running org.apache.hadoop.hbase.regionserver.TestHRegion
[INFO] Tests run: 109, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 139.392 s - in org.apache.hadoop.hbase.regionserver.TestHRegion
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 121, Failures: 0, Errors: 0, Skipped: 0

Cc @bharathv

Ya that looks shady. I kicked off another build, let's see if this is reproducible.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ branch-1 Compile Tests _

+0 :ok:
mvndep
2m 24s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
8m  0s
branch-1 passed

+1 :green_heart:
compile
1m  0s
branch-1 passed with JDK v1.8.0_262

+1 :green_heart:
compile
1m  8s
branch-1 passed with JDK v1.7.0_272

+1 :green_heart:
checkstyle
2m 17s
branch-1 passed

+1 :green_heart:
shadedjars
3m  0s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m  2s
branch-1 passed with JDK v1.8.0_262

+1 :green_heart:
javadoc
1m  6s
branch-1 passed with JDK v1.7.0_272

+0 :ok:
spotbugs
2m 42s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 18s
branch-1 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 18s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 57s
the patch passed

+1 :green_heart:
compile
1m  2s
the patch passed with JDK v1.8.0_262

+1 :green_heart:
javac
1m  2s
the patch passed

+1 :green_heart:
compile
1m  9s
the patch passed with JDK v1.7.0_272

+1 :green_heart:
javac
1m  9s
the patch passed

+1 :green_heart:
checkstyle
2m  7s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
2m 46s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
4m 38s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2.

+1 :green_heart:
javadoc
0m 54s
the patch passed with JDK v1.8.0_262

+1 :green_heart:
javadoc
1m  7s
the patch passed with JDK v1.7.0_272

+1 :green_heart:
findbugs
4m 19s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
2m 44s
hbase-client in the patch passed.

+1 :green_heart:
unit
132m 24s
hbase-server in the patch passed.

+1 :green_heart:
asflicense
0m 59s
The patch does not generate ASF License warnings.

185m 21s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2415

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 13909b755aab 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2415/out/precommit/personality/provided.sh

git revision
branch-1 / f0acafc

Default Java
1.7.0_272

Multi-JDK versions
/usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/3/testReport/

Max. process+thread count
4585 (vs. ulimit of 10000)

modules
C: hbase-client hbase-server U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2415/3/console

versions
git=1.9.1 maven=3.0.5 findbugs=3.0.1

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


Analyze 538 ELO model performance

[ ] Download/format CSV file with history of NFL data
[ ] Ascertain who the favorite was for each game and if the favorite won
[ ] Plot the fraction of games the favorite has won over the history of football
[ ] Do the probabilities generated by the Elo model actually bear out with real football data

Raw data CSV file:

https://projects.fivethirtyeight.com/nfl-api/nfl_elo.csv

Background info on Elo Rating:

https://en.wikipedia.org/wiki/Elo_rating_system


Overflowing text on the Code Schools page for 'Deep Dive Coding'

Bug Report
What is the current behavior?
Text overflows outside the card for Deep Dive Coding
What is the expected behavior?
No overflow should occur
What steps did you take to get this behavior?
Visited https://www.operationcode.org/code_schools
Additional Info
Operating System
macOS Sierra 10.12.6
Browser
Safari 10.1.2
Screenshots

These components are simply too small to render that much text.
I believe that we'll either need to rethink the entire card or limit the name length for the code_schools.yaml on the back-end.
I'd prefer the latter.

Opinions required on this one!

I think a human touch is necessary here for now. IMO schools should be limited to their most recognizable, yet shortest name (<50 chars or something) upon entry in the backend to normalize our data. For instance, the Turing School of Software and Design should just be limited to the Turing School.  Similarly, Deep Dive + Sales Force + blah blah should just be Deep Dive.

Skip to one minute into the recording. I forgot to start the dev server first. :expressionless:
Video of latest changes


Web service works only once

Web service scans the first file successfully, but crashes on the next run.
╰$ docker run -p 3993:3993 malice/zoner web
time="2018-08-06T09:27:32Z" level=info msg="web service listening on port :3993"
time="2018-08-06T09:27:38Z" level=fatal msg="exit status 1" category=av path=/malware/web_341723247 plugin=zoner

Output of go version:
go version go1.10.3 linux/amd64

Output of docker version:
Client:
 Version:      18.05.0-ce-rc1
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   33f00ce
 Built:        Thu Apr 26 01:05:07 2018
 OS/Arch:      linux/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.05.0-ce-rc1
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   33f00ce
  Built:        Thu Apr 26 01:08:54 2018
  OS/Arch:      linux/amd64
  Experimental: false

Output of docker info:
Containers: 23
 Running: 1
 Paused: 0
 Stopped: 22
Images: 39
Server Version: 18.05.0-ce-rc1
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.17.9-200.fc28.x86_64
Operating System: Fedora 28 (Twenty Eight)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 7.685GiB
Name: friday
ID: 5QP4:Z3UQ:TIGC:7P6K:74MB:IAET:BZAK:2GXG:BGNX:BUHU:4B63:X7LJ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

I am surprised this works at all??  What output are you seeing from the plugin when it succeeds?

See #1 also

I see the usual json output, the result is false however.
Okay, I haven't looked at plugins.toml file, just was checking every plugin if it works or not :)

Thank you for testing the plugins it is a very BIG help to me!  I greatly appreciate it 👍

I hope zoner can fix their update server so i can re-enable this plugin soon.

No problem, happy to be helpful :)

@altsab zoner is working again!!!

Yay, cool! But I just checked it, and it processes the first checkup okay, but still fails for the second try..
The reason is the same as before exit status 1

@altsab thank you for testing! I'll look into it 👍


Update troubleshoot-outbound-smtp-connectivity.md

Update troubleshoot-outbound-smtp-connectivity.md to remove references to the 2017 cut-off date to prevent suggesting to spammers that they look for old subscriptions to abuse. Also made the language more concise.

@davegoddard : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@davegoddard， I will submit consolidate the changes into another pull request.  So I will close this one now.
#please-close


[BUG] RuntimeError: No available kernel. Aborting execution.

I recently installed Atmorep in a conda env with Pytorch 2.4 and cuda 12.1.
However, the code doesn't run, has the following error
RuntimeError: No available kernel. Aborting execution.

Anyone having a working conda/pip env, can you please list the packages?

Traceback (most recent call last):
  File "/scratch/atmorep_2024/atmorep/core/evaluate.py", line 77, in <module>
    Evaluator.evaluate( mode, model_id, file_path, options)
  File "/scratch/atmorep_2024/atmorep/core/evaluator.py", line 115, in evaluate
    func( cf, model_id, model_epoch, devices, args)
  File "/scratch/atmorep_2024/atmorep/core/evaluator.py", line 174, in global_forecast
    evaluator.validate( 0, cf.BERT_strategy)
  File "/scratch/atmorep_2024/atmorep/core/trainer.py", line 380, in validate
    preds, atts = self.model( batch_data)
  File "/scratch/env/atmorep/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/scratch/env/atmorep/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/scratch/atmorep_2024/atmorep/core/atmorep_model.py", line 168, in forward
    pred = self.net.forward( xin)
  File "/scratch/atmorep_2024/atmorep/core/atmorep_model.py", line 519, in forward
    fields_embed, att = self.forward_encoder_block( ib, fields_embed)
  File "/scratch/atmorep_2024/atmorep/core/atmorep_model.py", line 553, in forward_encoder_block
    y, att = self.checkpoint( self.encoders[ifield].heads[iblock], *fields_in)
  File "/scratch/atmorep_2024/atmorep/transformer/transformer_base.py", line 26, in checkpoint_wrapper
    return cmodule(*kwargs)
  File "/scratch/env/atmorep/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/scratch/env/atmorep/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/scratch/fp0/mah900atmorep_2024/atmorep/transformer/transformer_attention.py", line 256, in forward
    outs = self.att( qs, ks, vs).transpose( -3, -2).flatten( -2, -1).reshape(s)
RuntimeError: No available kernel. Aborting execution.

The error sounds like a torch problem but if you 2.4 then it should work. Here's a requirements.txt from my env:
requirements.txt

The packages list works, thanks very much.
However, PyTest is causing a caching error
===================================================================================== test session starts ======================================================================================
platform linux -- Python 3.10.16, pytest-8.3.4, pluggy-1.5.0 -- /scratch/env/atmorep/bin/python
cachedir: .pytest_cache
rootdir: /scratch/atmorep_2024
plugins: anyio-4.7.0
collected 0 items

======================================================================================= warnings summary =======================================================================================
../../../env/atmorep/lib/python3.10/site-packages/_pytest/config/__init__.py:1277
  /scratch/env/atmorep/lib/python3.10/site-packages/_pytest/config/__init__.py:1277: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: anyio
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================================== 1 warning in 0.06s ======================================================================================

This is due to multiple pytest.main() call from the same process.
Is the PyTest strictly necessary? or can be ignored.

Hi, no it's not critical. you can set 'with_pytest' : False and it should skip it. This will be soon refactored in a separate testing pipeline so it's just a temporary solution.


slow write BW observed beyond 64KB size

This has been reported by Mark Mark Silberstein<EMAIL_ADDRESS>￼
We finally pinpointed the setup, and it's easily reproducible.

Get the CPU ptr for the buffer in the  mapped BAR
Sequentially pread from file into that buffer in blocks >=64K.

As long as blocks are less than 64K, we get ~1GB/s. For blocks >= 64K we get around 13MB/s

closing as fixed


Repository description and README do not mention use for app bugs of iOS and Android

Where to find the issue

"About" text in repository https://github.com/corona-warn-app/cwa-documentation
README.md

Describe the issue
The "About" text for this repository is set to:
"Project overview, general documentation, and white papers."
This does not mention the use of the repository for handling bugs which affect both iOS and Android operating systems, as described in ISSUE_TEMPLATE/02_bugs.md
* [ ] Bug affects both Android **and** iOS, for specific issues / questions that apply only to one operating system, please raise them in the respective repositories:
    * [Android repository](https://github.com/corona-warn-app/cwa-app-android)
    * [iOS repository](https://github.com/corona-warn-app/cwa-app-ios)

The README.md file contains no mention either of the use of the repository issue list to track not only documentation issues, but also app issues affecting both iOS and Android.
Suggested change

Update the description for this repository which appears under "About" so that it covers "app bugs common to both iOS and Android" (this has to be done by the Open Source Team due to permissions needed).
Update the README.md file of this repository with a new section describing how the repository is used to submit and track app bugs which are common to both iOS and Android operating system environments.
Update the README files of both iOS and Android repositories with an explicit cross-reference to the documentation repository as a place to report and track issues affecting both iOS and Android apps.

This issue is open for ca. 6 months now and there was no comment from the Open-Source-Team at all. Please let us know how you plan to proceed with this issue.


Facebook API version 2.4 is not supported

Facebook published the latest API v.2.4 to every new application, but this API is not supported by current version of Chute PhotoPickerPlus (both Android and iOS version).
The problem is critical as none of new apps can introduce Facebook authorization in its functionality until this issue is resolved.

Can you share some insights to where you read this? I am unable to find any info on this.
Our mobile apps and libraries don't use the facebook sdk's directly, the server contacts the facebook API directly to make the requests. As far as i know their previous version is still functional but deprecated for at least 2 more years. And when we change the API's to communicate with the newest version facebook released any mobile apps using the sdk and pp+ won't have to change anything.

Probably, I submitted issue on wrong location.. Sorry if I'm wrong.
I will try to explain problem in details.
I've created the new Facebook application and by default application use the latest published API (currently v2.4). There is no possibility do change it (like in this case to downgrade it from v2.4 to v2.3).
I've integrated ChutePhotoPickerPlus to my Android and iOS application, and also configured to use my CLIENT_ID/CLIENT_SECRET.
The login dialog shows regularly (user is able to specify his email and password). When user clicks on login button the the following error appears:
Invalid Scopes: offline_access. This message is only shown to developers. Users of your app will ignore these permissions if present. Please read the documentation for valid permissions at: https://developers.facebook.com/docs/facebook-login/permissions
I did a lot of debugging of my application, tried almost every combination of setting on my Facebook development account but nothing gives positive result.
When I try with CLIENT_ID/CLIENT_SECRET which is given in sample project of PhotoPickerPlus:
APP_ID = "4f3c39ff38ecef0c89000003";
APP_SECRET = "c9a8cb57c52f49384ab6117c4f6483a1a5c5a14c4a50d4cef276a9a13286efc9";
It works as expected.
Finally during the debugging process, I've started to monitor URLs which are invoked during the login process and wrote them into a log file - both for my Facebook client and for client from examples. Only difference was in path segment: /v2.4/ - for my version and /v2.3/ for the working version. PhotoPickerPlus, uses Chute SDK, and this component communicate with server. I assume that server generates URL for client, but I'm not sure whether Chute SDK plays important role for this links. That is the reason I've created this issue right here.
So, If I understand correctly, maybe only the getchute.com server should be updated (because the server uses Facebook SDK directly)?

Yes it seems the server will need to be updated if this is causing the issues for new facebook apps OAuth login.
You can also try not setting the facebook app id and app secret for Chute in the admin and just using the ones we provide if that way the login works for your Chute APP id and App secret/


Feature/keyboard handling

Hey!
I am using Iced for a small applicaiton and I wanted to add the abaility to exit by pressing Escape.
But it seems that there isn't a straight forward way to subscribe to keyboard events as far as I can tell.
There is a way to do it in PaneGrids but the handling is a bit more complicated and it seems you can't handle different combinations of moddifiers (For example handle ctrl + d differently from alt + d)
I've implemented a small keyboard handler for container and I updated the stopwatch exmaple to show it I would use this.
If this is something you like, I'd be interested in extending it to other widgets or including mouse events.
Also Iced is really cool!

Hmmm. I just found out that I could have solved this using a subscription.
It doesn't offer the abaility to bubble events as far as I can tell. But it would be enough for my needs

Thanks!
I think we need to be careful with adding too much functionality to simple widgets. In this case, I believe a better approach would be to create a new widget extending a Container by using composition.
But as you mention, using a Subscription is the recommended way to go until some kind of event capturing/bubbling lands. It is not completely clear yet how this feature will shape up. Therefore, we should probably wait until then before introducing more logic that could conflict with it.
There are a bunch of related issues and discussions that we should keep in mind:

Trying to integrate with Kiss3d
Layers
#408
#455


beamsplitter: use generated code for View.java

Note that JNI c++ functions still need to be manually maintained, as well as the setter / getter
methods.
Everything before the following codeline must be manually maintained:
// The remainder of this file is generated by beamsplitter

The diff in View.java is hard to read, but it basically moves all options-related type definitions to the bottom.  That's just a side effect of the codegen.  In theory there no zero API changes caused by this PR.  Even the annotations like @Nullable should still be there.

The diff in View.java is hard to read, but it basically moves all options-related type definitions to the bottom. That's just a side effect of the codegen. In theory there no API changes caused by this PR. Even the annotations like @Nullable should still be there.

So that means that we now have the javadoc only in one place (the cpp file)?


No support for fields.List(tagname='.')

Hi,
Say you are trying to parse the following xml:
<a v="1">
<c d="2"/>
<c d="2"/>
<c d="2"/>
<e f="3"/>
<e f="3"/>
</a>

Using the class definitions:
class c(Model):
  d = fields.Integer()

class e(Model):
  f = fields.Integer()

class a(Model):
  v = fields.Integer()
  cs = fields.List(c, tagname=".", required=False)
  es = fields.List(e, tagname=".", required=False)

This does not work since fields.List(tagname='.') does not appear to be supported. Is there anyway we can implement this? I know the XML schema is terrible but I didn't write it. Unfortunately, I need to parse it :|. Any ideas would be greatly appreciated. I could implement the patch with some direction.

Thanks for reaching out. Unfortunately I am no longer actively maintaining this project, so I'm going to move it into archive mode:
https://rfk.id.au/blog/entry/archiving-open-source-projects/


Add method to validate path

I would like to show an error message to users if the JSONPath they specify is invalid, so it would be useful to have a method here to validate the path.

Added in v0.4.33.
func testSimple17() {
    let json = #"[{"title":"Post 1","timestamp":1},{"title":"Post 2","timestamp":2}]"#

    XCTAssertEqual(json.query(validate: "$"), nil)
    XCTAssertEqual(json.query(validate: ""), "Path must start with $ or @")
    XCTAssertEqual(json.query(validate: "$."), "Path must not end with a \'.\' or \'..\'")
    XCTAssertEqual(json.query(validate: "$.."), "Path must not end with a \'.\' or \'..\'")
    XCTAssertEqual(json.query(validate: "$.store.book[["), "Could not parse token starting at position 12.")
    XCTAssertEqual(json.query(validate: "@.category"), nil)
}

Updated the playground site to use it as well.

It reports that $.users[*].name is an invalid path, but it's valid. users[*].name passes. $.users[*].name and users[*].name are the same thing.

I'm not seeing that. I see "$.users[].name" as valid and "users[].name" failing for "Path must start with $ or @"
func testSimple17() {
        let json = #"[{"title":"Post 1","timestamp":1},{"title":"Post 2","timestamp":2}]"#

        XCTAssertEqual(json.query(validate: "$"), nil)
        XCTAssertEqual(json.query(validate: ""), "Path must start with $ or @")
        XCTAssertEqual(json.query(validate: "$."), "Path must not end with a \'.\' or \'..\'")
        XCTAssertEqual(json.query(validate: "$.."), "Path must not end with a \'.\' or \'..\'")
        XCTAssertEqual(json.query(validate: "$.store.book[["), "Could not parse token starting at position 12.")
        XCTAssertEqual(json.query(validate: "$.users[*].name"), nil)
        XCTAssertEqual(json.query(validate: "users[*].name"), "Path must start with $ or @")
    }

Sorry, my fault. I didn't realize it returned an error message on invalid path.


ci: migrate to Sage

Codecov Report

Merging #243 (fa9bda0) into master (762bfd8) will increase coverage by 0.07%.
The diff coverage is 76.92%.

@@            Coverage Diff             @@
##           master     #243      +/-   ##
==========================================
+ Coverage   44.66%   44.74%   +0.07%
==========================================
  Files          21       21
  Lines        4267     4273       +6
==========================================
+ Hits         1906     1912       +6
  Misses       2249     2249
  Partials      112      112

Impacted Files
Coverage Δ

internal/examples/musicdb/database_gen.go
20.76% <33.33%> (ø)

internal/examples/freightdb/database_gen.go
23.20% <66.66%> (ø)

...nternal/codegen/databasecodegen/readtransaction.go
100.00% <100.00%> (ø)


Line highlightings are not working for non-twoslash code blocks

heh, it's both a bug and enhancement


Table.put() yields huge amount of storage space

Hello guys.
I've been developing an Electron application for quite a long time now, and one of the main concerns we had was that Table.put() is yielding a lot of space in the disk.
I have a logic to update a register inside a loop:
for (let i = 0; i < file.remainingUsers.length; i++) {
  // hidden for brevity

  await db.files.put({
    ...file,
    scores,
    progress,
    remainingUsers: file.remainingUsers.slice(i + 1),
  });
}

My Dexie.js initialization looks as follows:
import Dexie from "dexie";

const db = new Dexie("raw");

db.version(1).stores({
  files:
    "++id, name, path, status, total, remainingUsers, progress, scores, cap, userColumn, leadingAt, limit, calls",
});

export default db;

So, I tried implementing a put logic using the pure IndexedDB API, and result was that I don't get this huge amounts of disk usage as when using Dexie.
Here are two images comparing the results.

Dexie
IndexedDB

In both cases, it is a result of processing 30 rows of my table.
I also tried implementing with Table.update() and I got the same result.
If you want more details, please, let me know.

Dexie's Table.put() does not do anything else than what the native IDBObjectStore.put does unless you have addons like dexie-obsevable that also tracks changes. The native IDB implementation may choose to create new rows for every put operation instead of really replacing it. This is a common pattern in databases - mark the old version as deleted and finally delete it lazily.