gall re-requests updated apps at /=/[desk]/

(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate "build $app now" requests to ford, expecting the second to produce an updated(different) version. This causes a refresh-loop if ford detects the duplicate request, and serves the same response as previously (with the same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be numerical; protocol invariants are underdocumented here, so I'm not sure which vane the responsibility technically belongs to. This would be effective due to a violation of referential transparency wrt what date maps to which revision: currently, something similar is employed earlier in the request flow.
Cc @philipcmonk, you were right that things are relying on the current clay behaviour. A reframing of this issue is "multiple clay revisions can land on same tick"; or at least, one solution is "%next subscriptions never trigger immediately", where "immediately" is "during the referent of the %next".

Broadly speaking we have seen a lot of this class of bug.  The cause is a bad pattern in which reactive behavior is written as "A causes B."  Where A is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall has noticed a change in the source code", i.e., dirty bit is set.  Thus the source change makes us learn that the app is dirty.  This in turn causes the reload, but only once.  You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does.  Should not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin<EMAIL_ADDRESS>wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate "build $app now" requests to ford, expecting the second to produce an updated(different) version. This causes a refresh-loop if ford detects the duplicate request, and serves the same response as previously (with the same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be numerical; protocol invariants are underdocumented here, so I'm not sure which vane the responsibility technically belongs to. This would be effective due to a violation of referential transparency wrt what date maps to which revision: currently, something similar is employed earlier in the request flow.
Cc @philipcmonk, you were right that things are relying on the current clay behaviour. A reframing of this issue is "multiple clay revisions can land on same tick"; or at least, one solution is "%next subscriptions never trigger immediately", where "immediately" is "during the referent of the %next".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

This would stop the error mode from being a /loop/, but if the update is
stale, gall will still never obtain the next version, nor be able to listen
for future changes.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS>wrote:

Broadly speaking we have seen a lot of this class of bug. The cause is a
bad pattern in which reactive behavior is written as "A causes B." Where A
is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall has
noticed a change in the source code", i.e., dirty bit is set. Thus the
source change makes us learn that the app is dirty. This in turn causes the
reload, but only once. You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does. Should
not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate "build
$app now" requests to ford, expecting the second to produce an
updated(different) version. This causes a refresh-loop if ford detects the
duplicate request, and serves the same response as previously (with the
same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be
numerical; protocol invariants are underdocumented here, so I'm not sure
which vane the responsibility technically belongs to. This would be
effective due to a violation of referential transparency wrt what date maps
to which revision: currently, something similar is employed earlier in the
request flow.
Cc @philipcmonk, you were right that things are relying on the current
clay behaviour. A reframing of this issue is "multiple clay revisions can
land on same tick"; or at least, one solution is "%next subscriptions never
trigger immediately", where "immediately" is "during the referent of the
%next".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221131595

What do you mean by "stale"?
If the update doesn't load, keep the dirty bit and try again on the next update.
Sent from my iPhone

On May 23, 2016, at 7:03 PM, Anton Dyudin<EMAIL_ADDRESS>wrote:
This would stop the error mode from being a /loop/, but if the update is
stale, gall will still never obtain the next version, nor be able to listen
for future changes.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS>wrote:

Broadly speaking we have seen a lot of this class of bug. The cause is a
bad pattern in which reactive behavior is written as "A causes B." Where A
is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall has
noticed a change in the source code", i.e., dirty bit is set. Thus the
source change makes us learn that the app is dirty. This in turn causes the
reload, but only once. You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does. Should
not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate "build
$app now" requests to ford, expecting the second to produce an
updated(different) version. This causes a refresh-loop if ford detects the
duplicate request, and serves the same response as previously (with the
same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be
numerical; protocol invariants are underdocumented here, so I'm not sure
which vane the responsibility technically belongs to. This would be
effective due to a violation of referential transparency wrt what date maps
to which revision: currently, something similar is employed earlier in the
request flow.
Cc @philipcmonk, you were right that things are relying on the current
clay behaviour. A reframing of this issue is "multiple clay revisions can
land on same tick"; or at least, one solution is "%next subscriptions never
trigger immediately", where "immediately" is "during the referent of the
%next".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221131595

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

The problematic sequence is:
{without caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo-0"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"

clay: revision 2
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 2 (XX, technically)

gall <- ford : [%dojo@/=home/2 "dependencies are 0vdojo2"]
gall -> ford : "listen on 0vdojo2"
~zod:dojo>

{with caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"

clay: revision 2
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)

ford(cached) : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"
«loop»

{with dirty bit}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"

clay: revision 2
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)

ford(cached) : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall : "loop detected! Not interested in getting %dojo@/=home/1 again"
gall : never receives %dojo@/=home/2
<awaiting hood, please wait a few minutes>

On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS>wrote:

What do you mean by "stale"?
If the update doesn't load, keep the dirty bit and try again on the next
update.
Sent from my iPhone

On May 23, 2016, at 7:03 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:
This would stop the error mode from being a /loop/, but if the update is
stale, gall will still never obtain the next version, nor be able to
listen
for future changes.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

Broadly speaking we have seen a lot of this class of bug. The cause is
a
bad pattern in which reactive behavior is written as "A causes B."
Where A
is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall has
noticed a change in the source code", i.e., dirty bit is set. Thus the
source change makes us learn that the app is dirty. This in turn
causes the
reload, but only once. You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does. Should
not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>
<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>');>> wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate
"build
$app now" requests to ford, expecting the second to produce an
updated(different) version. This causes a refresh-loop if ford detects
the
duplicate request, and serves the same response as previously (with the
same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be
numerical; protocol invariants are underdocumented here, so I'm not
sure
which vane the responsibility technically belongs to. This would be
effective due to a violation of referential transparency wrt what date
maps
to which revision: currently, something similar is employed earlier in
the
request flow.
Cc @philipcmonk, you were right that things are relying on the
current
clay behaviour. A reframing of this issue is "multiple clay revisions
can
land on same tick"; or at least, one solution is "%next subscriptions
never
trigger immediately", where "immediately" is "during the referent of
the
%next".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221131595

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221148826

i.e., "the next update" given only the current one is not a part of the
ford(or clay) dependency API.
On Monday, 23 May 2016, Anton Dyudin<EMAIL_ADDRESS>wrote:

The problematic sequence is:
{without caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo-0"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"

clay: revision 2
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 2 (XX, technically)

gall <- ford : [%dojo@/=home/2 "dependencies are 0vdojo2"]
gall -> ford : "listen on 0vdojo2"
~zod:dojo>

{with caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"

clay: revision 2
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)

ford(cached) : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"
«loop»

{with dirty bit}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"

ford .^ clay : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"

clay: revision 2
ford <- clay : "I have a revision!"

gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)

ford(cached) : ~2016.1.1 is revision 1

gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall : "loop detected! Not interested in getting %dojo@/=home/1 again"
gall : never receives %dojo@/=home/2
<awaiting hood, please wait a few minutes>

On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

What do you mean by "stale"?
If the update doesn't load, keep the dirty bit and try again on the next
update.
Sent from my iPhone

On May 23, 2016, at 7:03 PM, Anton Dyudin<EMAIL_ADDRESS>wrote:
This would stop the error mode from being a /loop/, but if the update is
stale, gall will still never obtain the next version, nor be able to
listen
for future changes.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS>wrote:

Broadly speaking we have seen a lot of this class of bug. The cause
is a
bad pattern in which reactive behavior is written as "A causes B."
Where A
is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall has
noticed a change in the source code", i.e., dirty bit is set. Thus the
source change makes us learn that the app is dirty. This in turn
causes the
reload, but only once. You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does.
Should
not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate
"build
$app now" requests to ford, expecting the second to produce an
updated(different) version. This causes a refresh-loop if ford
detects the
duplicate request, and serves the same response as previously (with
the
same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be
numerical; protocol invariants are underdocumented here, so I'm not
sure
which vane the responsibility technically belongs to. This would be
effective due to a violation of referential transparency wrt what
date maps
to which revision: currently, something similar is employed earlier
in the
request flow.
Cc @philipcmonk, you were right that things are relying on the
current
clay behaviour. A reframing of this issue is "multiple clay revisions
can
land on same tick"; or at least, one solution is "%next subscriptions
never
trigger immediately", where "immediately" is "during the referent of
the
%next".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221131595

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221148826

Gall knows the version number.  Surely it can wait on a new version, rather than on a dependency, once a load fails?  I thought we had that technology.
Sent from my iPhone

On May 23, 2016, at 9:00 PM, Anton Dyudin<EMAIL_ADDRESS>wrote:
i.e., "the next update" given only the current one is not a part of the
ford(or clay) dependency API.
On Monday, 23 May 2016, Anton Dyudin<EMAIL_ADDRESS>wrote:

The problematic sequence is:
{without caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo-0"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
clay: revision 2
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 2 (XX, technically)
gall <- ford : [%dojo@/=home/2 "dependencies are 0vdojo2"]
gall -> ford : "listen on 0vdojo2"
~zod:dojo>

{with caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
clay: revision 2
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)
ford(cached) : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"
«loop»

{with dirty bit}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
clay: revision 2
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)
ford(cached) : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall : "loop detected! Not interested in getting %dojo@/=home/1 again"
gall : never receives %dojo@/=home/2
<awaiting hood, please wait a few minutes>

On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

What do you mean by "stale"?
If the update doesn't load, keep the dirty bit and try again on the next
update.
Sent from my iPhone

On May 23, 2016, at 7:03 PM, Anton Dyudin<EMAIL_ADDRESS>wrote:
This would stop the error mode from being a /loop/, but if the update is
stale, gall will still never obtain the next version, nor be able to
listen
for future changes.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS>wrote:

Broadly speaking we have seen a lot of this class of bug. The cause
is a
bad pattern in which reactive behavior is written as "A causes B."
Where A
is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall has
noticed a change in the source code", i.e., dirty bit is set. Thus the
source change makes us learn that the app is dirty. This in turn
causes the
reload, but only once. You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does.
Should
not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send duplicate
"build
$app now" requests to ford, expecting the second to produce an
updated(different) version. This causes a refresh-loop if ford
detects the
duplicate request, and serves the same response as previously (with
the
same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to be
numerical; protocol invariants are underdocumented here, so I'm not
sure
which vane the responsibility technically belongs to. This would be
effective due to a violation of referential transparency wrt what
date maps
to which revision: currently, something similar is employed earlier
in the
request flow.
Cc @philipcmonk, you were right that things are relying on the
current
clay behaviour. A reframing of this issue is "multiple clay revisions
can
land on same tick"; or at least, one solution is "%next subscriptions
never
trigger immediately", where "immediately" is "during the referent of
the
%next".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221131595

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221148826

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

Gall does not currently know the version number; it could scry this
information from clay, and in general talk to clay bypassing ford, but this
can only be done consistently by listening to every revision of the desk.
"Once a load fails" is a red herring: if the load succeeds, we still want
to continue autoloading future revisions, bringing us back to the above.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS>wrote:

Gall knows the version number. Surely it can wait on a new version, rather
than on a dependency, once a load fails? I thought we had that technology.
Sent from my iPhone

On May 23, 2016, at 9:00 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:
i.e., "the next update" given only the current one is not a part of the
ford(or clay) dependency API.
On Monday, 23 May 2016, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','antechno777@gmail.com');>> wrote:

The problematic sequence is:
{without caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo-0"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
clay: revision 2
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 2 (XX, technically)
gall <- ford : [%dojo@/=home/2 "dependencies are 0vdojo2"]
gall -> ford : "listen on 0vdojo2"
~zod:dojo>

{with caching}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
clay: revision 2
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)
ford(cached) : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1"
«loop»

{with dirty bit}
Event ~2016.1.1

clay: revision 1
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo0"
gall -> ford : "requesting %dojo at ~2016.1.1"
ford .^ clay : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
clay: revision 2
ford <- clay : "I have a revision!"
gall <- ford : "you got a revision for 0vdojo1"
gall -> ford : "requesting %dojo at ~2016.1.1" (XX)
ford(cached) : ~2016.1.1 is revision 1
gall <- ford : [%dojo@/=home/1 "dependencies are 0vdojo1"]
gall -> ford : "listen on 0vdojo1"
gall <- ford : "a revision has already happened on 0vdojo1"
gall : "loop detected! Not interested in getting %dojo@/=home/1
again"
gall : never receives %dojo@/=home/2
<awaiting hood, please wait a few minutes>

On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>
<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>');>> wrote:

What do you mean by "stale"?
If the update doesn't load, keep the dirty bit and try again on the
next
update.
Sent from my iPhone

On May 23, 2016, at 7:03 PM, Anton Dyudin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>>
wrote:
This would stop the error mode from being a /loop/, but if the
update is
stale, gall will still never obtain the next version, nor be able to
listen
for future changes.
On Monday, 23 May 2016, cgyarvin<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

Broadly speaking we have seen a lot of this class of bug. The
cause
is a
bad pattern in which reactive behavior is written as "A causes B."
Where A
is a code change and B is a reload.
It really should be that A causes C, which causes B; C is "%gall
has
noticed a change in the source code", i.e., dirty bit is set.
Thus the
source change makes us learn that the app is dirty. This in turn
causes the
reload, but only once. You could even use a zero timer here.
All our refresh logic needs to work that way. I think kiln does.
Should
not be hard to fix gall to.
Sent from my iPhone

On May 23, 2016, at 4:35 PM, Anton Dyudin <
<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>
<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>');>> wrote:
(possibly duplicate)
If two revisions occur at the same now, %gall will send
duplicate
"build
$app now" requests to ford, expecting the second to produce an
updated(different) version. This causes a refresh-loop if ford
detects the
duplicate request, and serves the same response as previously
(with
the
same dependency-hash metadata).
This could be addressed by ford normalizing dependency beaks to
be
numerical; protocol invariants are underdocumented here, so I'm
not
sure
which vane the responsibility technically belongs to. This would
be
effective due to a violation of referential transparency wrt what
date maps
to which revision: currently, something similar is employed
earlier
in the
request flow.
Cc @philipcmonk, you were right that things are relying on the
current
clay behaviour. A reframing of this issue is "multiple clay
revisions
can
land on same tick"; or at least, one solution is "%next
subscriptions
never
trigger immediately", where "immediately" is "during the referent
of
the
%next".
—
You are receiving this because you are subscribed to this
thread.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221131595

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221148826

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/urbit/arvo/issues/145#issuecomment-221161909

Closing this one as it hasn't been followed up on.

This is probably resolved by "live" subscriptions if those got implemented? Or if clay uses %wait 0 to always place commits on separate times, which I also remember hearing about

Clay never puts two commits at the same time, so this should no longer be an issue
On Fri, Jun 21, 2019 at 9:00 AM, Anton Dyudin <<EMAIL_ADDRESS>> wrote:

This is probably resolved by "live" subscriptions if those got
implemented? Or if clay uses %wait 0 to always place commits on separate
times, which I also remember hearing about
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub (
https://github.com/urbit/arvo/issues/145?email_source=notifications&email_token=ABKLYGGACDRB26EU6VNGBNTP3T3K5A5CNFSM4CET43Y2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYI3ZYA#issuecomment-504478944
) , or mute the thread (
https://github.com/notifications/unsubscribe-auth/ABKLYGGWUVCZRVYLD6M6XRDP3T3K5ANCNFSM4CET43YQ
).


(android) Add daily-connect worker

This PR adds a background worker to the Android app, scheduled to run roughly every day. It simply connects to the LSP, wait for 60s then shuts down. Its purpose is to settle pending actions, for example pending payment that were missed by the inflight-payments worker, and thus avoid force-closings.
Some utility code has been factorised with the inflight-payments worker.

Note, this worker does not replace the channels-watcher that runs every ~3 days, as the channels-watcher does not connect to the peer (on purpose).


Redo .net core tests that changed behaviour due to .net core breaking changes

Commits

https://github.com/Microsoft/msbuild/pull/2957/commits/4c9084bae200ab5525862365902539371f39c542

Search for tests that are Windows only, but disabled for .net core:
[PlatformSpecific(TestPlatforms.Windows)]
[SkipOnTargetFramework(TargetFrameworkMonikers.Netcoreapp)]

Also need to see how bad the errors are for real cases where we were relying on a framework-generated "bad path" error and now won't get one.


Update fendermint to match genesis changes in solidity

Update fendermint to align with the changes in solidity contracts. Some key changes are:

New type changes, i.e. adding GenesisValidator parameter that matches that in the contract.
The validator's power in fendermint is the sum of the validator's collateral + federated power from the parent. See LibStakingPower for references.
Update ipc get genesis from parent to use GenesisValidator.
Update genesis CLI commands to reflect changes on the contract.

This PR is compilable and integration tested, but it's pending updates in unit tests.

I agree with @aakoshh. We do not need to track both collateral and weight.
But the problem is deeper: these should NOT be used in conjunction. There should just be weight, which is either set explicitly (federated) or as a projection of collateral (collateral), either or, both not the sum of both. This confusion came up in this thread and was corrected there, but the implementation hasn't been updated.
The "hybrid" model that uses both both collateral and federated is very unsound. There is no unit enforced for manually-assigned weights. So if the user assigns equal weights of 1, 1, 1, an attacker could come in and deposit 10 FIL and take over the network in a clean sweep.

Actually, I just realised that the child gateway receives individual validator commands in the form of:

set_federated_power(new_power)
deposit(amount)
withdraw(amount)
...

It also stores full ValidatorInfo objects including the power, collateral, etc.
The child gateway then needs to reconcile all this info and recalculate the power locally based on the permissionMode. I think all of this is unnecessarily complex.
Ideally all this acccounting would be encapsulated in the source of truth: subnet actor at the parent. That actor would just emit new weights (computed based on its permissionMode) that propagate down to the child gateway to apply them in the subnet.
Unfortunately all of this is going to require a more profound discussion/architectural change to adjust. Ideally we would've had this discussion before implementing federated validation (at that time we just had collateral, which directly translated into weight), but that ship has sailed. At this stage, we might be better off just biting the bullet and refactoring later.

So I think we're just going to have the bullet and accept this change until we rearchitect to decouple the collateral and static power tracking at the parent, from their translation to a mere weight at the child.
That said, we should implement (2), (3) and (4) from above before we merge this PR.

Remove the GenesisValidator type and simply use ValidatorInfo type; unless I'm missing something, these types are redundant. @aakoshh also pointed it out in his review.
That should allow us to get rid of weirdness like this:

the transformation in finalizeValidatorInfo.
this conditional in genesisValidator().

The subnet actor translates the combination of validator collateral and static power into a weight, depending on the permissionMode.

If collateral, use only collateral.
If federated, use only the static power.
If in the future we want to add a hybrid, we'll add the hybrid. But neither collateral nor federated should be hybrid.

Merged in the interest of landing the fixes in the base branch onto main. Will discuss next steps with the team and @cryptoAtwill on Monday.


Replace zefyr with a forked repository

Replaces zefyr with a forked repository, because zefyr has some issues with beta channel of Flutter.

Related to issue #14


[spark] Add tests for buceted scan with partitioned table

Purpose

This pr adds test for buceted scan with partitioned table. Although we do not support bucketed scan with multi-bucket columns, there is a chance to support multi primary keys if  primary keys contains partition column.
Tests

add test
API and Format

no
Documentation

cc @JingsongLi thank you!

re-trigger test

+1


"project-report" plugin - "htmlDependencyReport" output to System.out

Current Behavior
Output HTML to "build/reports/project/dependencies" folder while also dump the dependency graph to System.out
bash-4.2$ ./gradlew htmlDependencyReport
> Task :htmlDependencyReport

------------------------------------------------------
Root project 'test'
------------------------------------------------------

runtimeClasspath - Runtime classpath of source set 'main'.
+--- com.fasterxml.jackson:jackson-bom:2.17.2
|    +--- com.fasterxml.jackson.core:jackson-core:2.17.2 (c)

Expected Behavior
Should only output HTML to "build/reports/project/dependencies" folder
Context (optional)
No response
Self-contained Reproducer Project

plugins {
  id 'project-report'
  id 'java'
}

dependencies {
  implementation platform('com.fasterxml.jackson:jackson-bom:2.17.2')
}

Gradle version
8.10.0
Build scan URL (optional)
No response
Your Environment (optional)
No response

The issue is in the backlog of the relevant team and is prioritized by them.

Any update on this please?


Example source code in README throws a compiler error

It seems README isn't aligned with the source code. Example cpp code in README throws the following error:
error: ‘shutDownLogging’ is not a member of ‘g3’; did you mean ‘g3::internal::shutDownLogging’?

Thanks for addressing this in: https://github.com/KjellKod/g3log/pull/353


Add .mailmap

This file allows the git short log to merge duplicate contributor entries.
To the users affected, please check your canonical name and e-mail address (based on the first Name <E-Mail Address> pattern in the last line concerning you) and leave a comment if you would like to change it.
@lu-zero @fbossen @ycho @tdaede @negge @shssoichiro @xiphmont @tszymczak

@lu-zero The pattern is Name <Mail> (<Name>) (<Mail>), and the file is processed line by line. In this example, we have N1 <M1> followed by N1 <M1> <M2>. First, all names under M1 are merged into N1; then, M2 is merged into N1 .

Coverage remained the same at 83.348% when pulling 28351178b9b6e4733dbcf3332c2659fff4e7bf57 on rzumer:mailmap into 0004cff7b7abec5a41ba72663f6c9e4a110a63cd on xiph:master.


Where is Length defined?

Hello!
So i am having this error
https://gyazo.com/f152ff48c6458d0bb9fc5d1ea633d1eb
Which happens because length is not defined anywhere

?

There, just edited it.

ChatCommands never actually worked =\

well shit....

Can i go back versions in multiogar, because i need version 1.4.2's playercommands

@CalvinDKHD https://github.com/Megabyte918/MultiOgar-Edited/blob/fd94027fddeb1dfa1362529b921c68685128ee8f/src/modules/PlayerCommand.js

You could've just added the first 7 characters of the commit, no need for links

Lol?

my issue basically is, when you type /login it crashes the server.
And that is because length is not defined.

@ItzLevvie wat xD

@Luka967 GHost edited his post, Levvie didnt do it.

Fixed, just a typo


Add Delegate method notification for did scroll end

Hey @michaelEllisUy
I mean want accurate rolling stop notice, so that I can use this time to load the data network, such interaction can make the UI interface more fluent.
Jack


4.12 - Retry connecting vmedia through a DVD device if available

This commit is a cherry-pick of  e33289816 to 4.12 Branch
Trackers:
4.14 - https://issues.redhat.com/browse/OCPBUGS-14926
4.12 - https://issues.redhat.com/browse/OCPBUGS-19064

/test pep8

/retest-required

/test unit

/override ci/prow/unit

/test pep8

/lgtm


Remove redundant brackets

There are a lot of redundant brackets in the scss, this PR cleans this up a bit. I still left the brackets on some places if it increased the readability of the code like here:
https://github.com/twbs/bootstrap/blob/e9ec5e73929a46852abf9c7d4e5fc03d50f76d3c/scss/_popover.scss#L134

I don't think there's a rule for this, right? In stylelint-scss.

@XhmikosR, no there isn't at the moment. My gut feeling tells me that'll also be a hard one to implement.

LGTM AFAIK, but let's wait to hear from @mdo too

I'm fine to do either way here—having the parentheses helped me with readability and consistency across the project. Obviously a stylistic thing, so if folks don't want them, I'm fine with it. Would be also nice to have a linter for it though as @XhmikosR was suggesting.

We're actually quite inconsistent with this, for example these calculations don't use brackets:
https://github.com/twbs/bootstrap/blob/0a55c7a8f62ff0b7d95d42e3d09db9927f85c439/scss/_variables.scss#L296-L301
https://github.com/twbs/bootstrap/blob/0a55c7a8f62ff0b7d95d42e3d09db9927f85c439/scss/_variables.scss#L487
https://github.com/twbs/bootstrap/blob/0a55c7a8f62ff0b7d95d42e3d09db9927f85c439/scss/_button-group.scss#L70-L71

My main concern here is that if we can't automate the check, it's moot.

There are other things we always need to check ourself like the leading and trailing whitelines of titles in markdown files. I don't really like the idea that you can use whatever style you like here. Linting it would definitely be the best option, but I'm afraid this would be a plugin that is difficult to create, because you can have calc functions, scss variables, multiple value per property,...

Let's go with it, just a 2-3 cases I think the brackets make it clearer.


use last ticked layer to calculate ticker.timeSinceLastTick()

Motivation
ticker.timeSinceLastTick() currently only returns the diff between time.now() and start of the current layer
the purpose of this utility function is to figure out whether our last ticked has be delayed by ticker.sendTickThreshold.
using ticker.lastTickedLayer to yield the current delay since last tick.
Changes
use ticker.lastTickedLayer instead of converting time.Now() to layer number to calculate ticker.timeSinceLastTick()
Test Plan
added a test that failed before the change and passed after the change.
DevOps Notes

[x] This PR does not require configuration changes (e.g., environment variables, GitHub secrets, VM resources)
[x] This PR does not affect public APIs
[x] This PR does not rely on a new version of external services (PoET, elasticsearch, etc.)
[x] This PR does not make changes to log messages (which monitoring infrastructure may rely on)

i am having second thought about this PR.
the comment on sendTickThreshold seems to think that it really did want to calculate the time diff between now and the start of the current layer.
// the limit on how late a notify can be
// an attempt to notify later than sendTickThreshold from the expected tick time will resulted in a missed tick error
const sendTickThreshold = 500 * time.Millisecond

the failure in app_test.go is real. this change stops notifying each tick due to errMissedTickTime. i don't understand the purpose of this threshold. why do we stop notifying in this case? i'd argue we should get rid of this error....
if it really is intended to calculate the delay since the start of the current layer, it does need a better method name than timeSinceLastTick()

I think your doubts are justified, and I was confused earlier. I just looked at the only place in the code where this function is used:
https://github.com/spacemeshos/go-spacemesh/blob/37ea7f6ae6b1208b63a3b9808a9ea907e6d22a71/timesync/ticker.go#L107-L113
What this method is used for is to tell, before the clock initiates a tick, if it's already too late, since too much time has passed since the time it was supposed to be sent. After your change, instead of measuring the time passed from when the tick was supposed to be sent, we're measuring the time since the last tick that was successfully sent (the previous one, not the current one) - so it's always after the threshold.
The behaviour before your change was correct, but I agree that the function name is misleading. I would call it timeSinceLayerStart(), wdyt?

BTW, it's worth noting why this situation would even arise. Go cannot guarantee waking a goroutine that awaits a timer/ticker at exactly the requested time. It can and does only guarantee waking it after the requested time. If there's pressure, the delay can be considerable, especially on machines/VMs with few cores. This protection is there to allow users of the clock to assume that if they receive a tick - it's within a certain time range of the given layer's tick time OR the layer is skipped altogether.
I believe that it's safer and cleaner to pass this responsibility to the caller for several reasons:

Each caller may have a different threshold they care about. Some examples:
a. Blocks must be published within a certain time after the layer starts.
b. Hare starts some cache filling at layer start, but only really cares when the "wakeup delta" passes.
c. ATX builder doesn't care at all - it just needs to know the epoch started.
It should be a Spacemesh developer's habit and responsibility to validate the time at critical points in the code. Even if tick time is guaranteed, Spacemesh has many processes happening in goroutines that may have started later than expected due to load and it should be common to check for timeouts before performing actions like publishing a Hare message or block or when "locking in" a layer's block IDs for voting in the Hare.

thanks for the discussion @noamnelke
closing PR as it's not in any way a priority for now.


Reconnecting frontend (settings page/popup) to refactored code

As discussed yesterday, @stanleymarkman will take the lead on reconnecting the frontend to the refactored code as it stands for now. There are a few roadblocks to fully implementing this issue, namely refactoring the hooks we use to connect to the extension backend regarding domanlist.js as well as storage.js. I will do my best to clear these and make sure we have the correct behavior so this can move forward.
With regards to the concrete work, this issue will entail going through the popup and settings page folders and checking for each place we pass/receive messages to/from the background of the extension in addition to direct calls to the local storage. Furthermore, we will also need to reimplement how the js for the toggles on different UI elements work (namely the main settings page and the domain list page). Direct calls to local storage should be replaced with the storage wrapper I am implementing in storage.js for readability and future easy maintenance.
@stanleymarkman feels free to ask questions and write up what you find here. @kbeliauski if you have any thoughts write them in here too, since this relates to what we are doing with the extension storage.

I'm going to close this issue since we will go ahead with rewritng the front end with React.
@stanleymarkman will take over over there.


Add optional map accessor

I couldn't find a CONTRIBUTING.md file, so I apologize if my  github/syntax educate is incorrect. Please let me know how to fix it!
This should fix issue d3/d3-collection#10.

I’m a bit wary of adding an additional optional argument. Also, I do eventually plan on getting rid of the d3.map and d3.set classes in favor of standard ES collections. For example this hypothetical example:
var rateById = d3.map(data, d => d.id, d => +d.rate);

Can be implemented using a standard Map as:
var rateById = new Map(data.map(d => [d.id, +d.rate]));

I feel you on that. The ES collections replaces the need for it. D3 was ahead of its time! I was honestly just looking for novice level improvements to make to d3. I'm happy that I learned some stuff and will make sure to ask next time to see if the issue is still something that should be worked on.


NRPE move from package to plugin on 24.1

Important notices
Yes. I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
Yes, I have searched the existing issues, open and closed, and I'm convinced that mine is new.
The title contains the plugin to which this issue belongs
Describe the bug
Before upgrading to OPNSENSE 24.1, I did use NRPE via the manually installed package. I had/have a LAN firewall rule pointing port 5666/tcp from the LAN network or address to "This Firewall".
After upgrading - which did remove the manually installed package - and moving to the plugin, external calls to NRPE always generate: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. I do not find relevant log entries on the firewall.
Posting in the forum does not yield results. Hence, there will either be a bug in the NRPE plugin or at least a lack of documentation.
To Reproduce
Steps to reproduce the behavior:

Install NRPE plugin and set it up
ssh to OPNSENSE and verify that check commands work, e. g, by typing /usr/local/libexec/nagios/check_users -w 5 -c 10
ssh to OPNSENSE and verify that /usr/local/etc/nrpe.cfg does contain a configuration matching GUI input
ssh to OPNSENSE and verify that /usr/local/etc/nrpe_commands.cfg does contain the commands matching GUI input
Try combinations of it listening to either localhost, <IP_ADDRESS> or the LAN interface with or without a firewall rule for 5666/tcp
Try adding /usr/local/bin/sudo before commands (i. e. /usr/local/libexec/nagios/check_users or the like) or not adding sudo.
Try adding nrpe user to sudoers like so: https://forum.opnsense.org/index.php?topic=30097.0
Execute NRPE from an allowed host and verify if there is valid feedback or not

Expected behavior
Allowed hosts get meaningful responses to nrpe commands.
Please inform me, if the NRPE plugin is working with OPNSENSE 24.1 in general and if so, what I might be missing. Documentation as to if the plugin should listen to localhost, <IP_ADDRESS> or LAN and if a (which) firewall rule is required would be a big plus to avoid such questions.

Can you post what happens after 2-8? Its just guessing without any input

Thank you very much @mimugmail
My setup is Port 5666, Listen Address <IP_ADDRESS>, Allowed Hosts: Those from which I am querrying plus <IP_ADDRESS> Allow Arguments plus a LAN Firewall rule forwarding 5666/tcp to “This Firewall” (same problem with or without this rule and with Listen Address LAN IP). Please let me know if I shoud try other variants. The LAN ip is <IP_ADDRESS>
The output per step is:

Typing on the OPNSENSE console: /usr/local/libexec/nagios/check_users -w 5 -c 10

yields: USERS OK - 1 users currently logged in |users=1;5;10;0

/usr/local/etc/nrpe.cfg contains:

log_facility=daemon
log_file=/var/log/nrpe.log
debug=0
pid_file=/var/run/nrpe/nrpe.pid
nrpe_user=nagios
nrpe_group=nagios
server_port=5666
server_address=<IP_ADDRESS>
allowed_hosts=<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>
command_timeout=60
connection_timeout=300
dont_blame_nrpe=1
allow_bash_command_substitution=0
include=/usr/local/etc/nrpe_commands.cfg

/usr/local/etc/nrpe_commands.cfg contains:

command[check_users]=/usr/local/bin/sudo /usr/local/libexec/nagios/check_users -w 5 -c 10
command[check_load]=/usr/local/bin/sudo /usr/local/libexec/nagios/check_load -w 6,4,2 -c 12,10,8
(variant without /usr/local/bin/sudo does not work either).
5.+6. Just describe variants tried

Typed (with no feedback): echo "nagios ALL=(ALL) NOPASSWD: /usr/local/libexec/nagios" > /usr/local/etc/sudoers.d/nrpe

From <IP_ADDRESS> (command did work before OPNSENSE 24.1 with the NRPE system package):
/usr/lib/nagios/plugins/check_nrpe -H <IP_ADDRESS> -c check_users

Output:
CHECK_NRPE: Error - Could not connect to <IP_ADDRESS>. Check system logs on <IP_ADDRESS>
nrpe.log contains: [1708418745] Host <IP_ADDRESS> is not allowed to talk to us!
Added <IP_ADDRESS> to allowed hosts (without understanding the background of the error) so that . /usr/local/etc/nrpe.cfg contains:
log_facility=daemon
log_file=/var/log/nrpe.log
debug=0
pid_file=/var/run/nrpe/nrpe.pid
nrpe_user=nagios
nrpe_group=nagios
server_port=5666
server_address=<IP_ADDRESS>
allowed_hosts=<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>
command_timeout=60
connection_timeout=300
dont_blame_nrpe=1
allow_bash_command_substitution=0
include=/usr/local/etc/nrpe_commands.cfg
Then the feedback to /usr/lib/nagios/plugins/check_nrpe -H <IP_ADDRESS> -c check_users
from <IP_ADDRESS> changes to:
NRPE: Unable to read output
with no entries in nrpe.log
I have no clue as to what to try next.

Can you set debug=1 to /usr/local/etc/nrpe.cfg and service nrpe restart via CLI then check again from remote and check for logs? (a save in gui will overwrite the config so stick to CLI)


Programming Language - Python Resources

IMPORTANT: When creating your Pull Request, please use the following format for the title: Category - Resource Name & Section
PR Title samples:

Web development - CSS Portal & Tools
Web development - Leetcode & Challenges

Closes #347
Description:

Complete Resources of Python Learning has been added withe additional Documentation of each and every Topic, Covered all the subtopics of the issue.

Resource URL:

There're many resources attached, Here I'm attaching only some of them..

https://roadmap.sh/python
https://www.geeksforgeeks.org/python-programming-language/
https://www.javatpoint.com/python-oops-concepts

Checklist:
Before submitting your pull request, ensure that you have completed the following tasks:

[x] I have carefully reviewed and adhered to the contributing guidelines before creating this pull request.
[x] I followed the prescribed PR title template. (Check this if you're adding a resource)

fill the bracket with x. Something like this [x]

Thanks, can you please instead add those resources in this folder since the Python already exists before you, add your new resources and sections within this link


Fix Bookmarks component in menu mode

this came up in https://github.com/boundlessgeo/qgis-webappbuilder-plugin/issues/203

Coverage decreased (-0.05%) to 70.798% when pulling ed1f3bc6eab7902ae07644dddc28b826ed57c61c on bookmark-menu into 20d8c33ef779b1a88eaf5a9562bc7f2884aa55c7 on master.


CMake 3.16.3

CI gfxreconstruct build queued with queue ID 44429.

CI gfxreconstruct build # 2567 running.

CI gfxreconstruct build queued with queue ID 44481.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

CI gfxreconstruct build # 2568 running.

Linux testing is passing and you can verify it's using the correct version via the log output from github actions:
cmake version 3.16.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).
-- The CXX compiler identification is GNU 11.3.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info

CI gfxreconstruct build queued with queue ID 44528.

CI gfxreconstruct build # 2569 running.

CI gfxreconstruct build # 2569 passed.


[BEAM-13633] [Playground] Implement method to get a default example for each SDKs

[BEAM-13633]

Implement method to get a default example for each SDKs

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

@KhaninArtur @ilya-kozyrev

R: @pabloem

@pavel-avilov please resolve conflicts and address comment

@pabloem everything is ready for review

@pabloem - What is the next step on this PR?

sorry about the delay. Taking a quick look...

indeed this LGTM! sorry, can you rebase please?

@pabloem Done


Lua script http_response and tracing

Hi! I'm using Lua scripts and http_request object for multiple requests to my backend, because the number of requests will be known after the calculations in script. Everything is good, but I also use jaeger for traicing requests. All backend requests are tracing, but requests from lua script aren't.
My question: Is it possible to enable or configure tracing for http_request from Lua? If not, will be it included into new feature list?
Or maybe there is another way to make calculated number of requests?
I have an endpoing with param as list. I parse its value to lua table(array) and make requests with each value of parsed list.
krakend.json
{
  "version": 3,
  "$schema": "https://www.krakend.io/schema/v2.3/krakend.json",
  "extra_config": {
    "telemetry/opencensus": {
      "sample_rate": 100,
      "reporting_period": 0,
      "exporters": {
        "jaeger": {
          "endpoint": "http://jaeger:14268/api/traces",
          "service_name": "krakend",
          "buffer_max_count": 1000
        }
      }
    }
  },
  "endpoints": [
    {
      "endpoint": "/endpoint1/{codesList}",
      "output_encoding": "json",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/dummyforscript",
          "host": [
            "http://dummy:8193"
          ]
          "extra_config": {
            "modifier/lua-backend": {
              "sources": [
                "./script.lua"
              ],
              "live": true,
              "allow_open_libs": true,
              "post": "some_logic_function( http_response, request, response)",
              "skip_next": true
            }
          }
        },
        {
          "url_pattern": "/realendpoint",
          "host": [
            "http://real-host:8234"
          ]
        }
      ]
    }
  ]
}

Example of my lua script:
local json = require "./scripts/JSON"

function some_logic_function( http_response, request, response )
    local resp = response.load()
    local req = request.load()
    local respData = resp:data()

    local codesLst = req:params('CodesList')

    local lst = {}
    for word in string.gmatch(codesLst, '([^,]+)') do
        table.insert(lst, word)
    end

    local result = luaList.new()
    for k, v in pairs(lst) do
        make_request_and_fill_result(http_response, result, v, params)
    end

    respData:set('multiple', result)
end

function make_request_and_fill_result(http_response, result, code)
    local url = string.format("http://some-host:8081/api/%s", code)

    local response = http_response.new(url)
    local body = json.parse(response:body())
    response:close()

    result:set(result:len(), body.some.value)
end

HI @yarovitchuk ,
Thanks for opening the issue. There is no plan to support this.


[Bug]: Navbar issue

What happened?
I found a BUG 🐞
The navigation bar (navbar) is intended to be fixed at the top of the viewport when scrolling, but currently, there is a visible gap between the top of the navbar and the top of the viewport. This gap allows the content behind the navbar to be visible, creating an unintended space and disrupting the layout and user experience.
Add screenshots

What browsers are you seeing the problem on?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Can you assign me this issue?

This PR fixes this issue: https://github.com/gabrysia694/Gym-Junkies/pull/297


Attach listener not handling two-level parentNode inserting

Expected Behavior
Chart should render as soon as the parent is inserted into the DOM.
Current Behavior
Canvas remains blank.
Possible Solution
Attach listener should use contains to check if the node inserted includes the chart element.
Keeping a reference for every parent node is not feasible.
function createAttachObserver(chart, type, listener) {
  const canvas = chart.canvas;
  const observer = new MutationObserver(entries => {
    entries.forEach(entry => {
      for (let i = 0; i < entry.addedNodes.length; i++) {
        const added = entry.addedNodes[i];
        if (added === canvas || added.contains(canvas)) {
          listener(entry.target);
        }
      }
    });
  });
  observer.observe(document, {childList: true, subtree: true});
  return observer;
}

Steps to Reproduce
https://codepen.io/danielgindi/pen/bGWavLX
Context
Environment

Chart.js version: 3.5.0
Browser name and version: Chrome 92

@kurkle It's like the other side of #9448

Right, the isAttached works, but the attachment is not detected.

@kurkle Please note, that even when applying this fix, there's an odd new behavior - when the element is added to the dom, a vertical bar chart animates like from a 0 on 0 size, not from fullWidth on 0 size.

I had same issue, and I fixed same the solution. It working for my case.


Disable application zooming

I can pinch-zoom the application interface (at least on macOS). This shouldn’t happen, so we should fix that.

Are issues closed when landing relevant changes to master or when a new release is cut and downloadable @tilmx?

@marionebl at the moment, when the the PR is merged with the master. Do you think it would make more sense when a new release is out?

No, I am fine either way. Askes to ensure this can be closed.


build from branch main looks broken

I tried to build from branch main (git hash 9b42be1) following the instructions :
mkdir build
cd build
cmake .. -GNinja -DFETCHCONTENT_QUIET=OFF

This was working for me in past (still works if I check out release 17.0.1) but now this ends up with a cmake error:
FAILED: llvmproject-populate-prefix/src/llvmproject-populate-stamp/llvmproject-populate-patch /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/build/_deps/llvmproject-subbuild/llvmproject-populate-prefix/src/llvmproject-populate-stamp/llvmproject-populate-patch
cd /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/build/_deps/llvmproject-src && git reset --quiet --hard && git clean --quiet --force -dx && git am /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/patches/llvm-project/0001-Fix-llvm-test-CodeGen-ARM-build-attributes.ll-test.patch /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/patches/llvm-project/0002-Add-check-for-building-with-picolibc.patch /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/patches/llvm-project/0003-Run-picolibc-tests-with-qemu.patch /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/patches/llvm-project/0004-xfail-two-remaining-libcxx-with-picolibc-tests.patch && /Applications/CMake.app/Contents/bin/cmake -E touch /Users/scratch/LLVM-ARM/LLVM-embedded-toolchain-for-Arm-macOS/build/_deps/llvmproject-subbuild/llvmproject-populate-prefix/src/llvmproject-populate-stamp/llvmproject-populate-patch
ninja: build stopped: subcommand failed.
CMake Error at /Applications/CMake.app/Contents/share/cmake-3.27/Modules/FetchContent.cmake:1662 (message):
  Build step for llvmproject failed: 1
Call Stack (most recent call first):
  /Applications/CMake.app/Contents/share/cmake-3.27/Modules/FetchContent.cmake:1802:EVAL:2 (__FetchContent_directPopulate)
  /Applications/CMake.app/Contents/share/cmake-3.27/Modules/FetchContent.cmake:1802 (cmake_language)
  /Applications/CMake.app/Contents/share/cmake-3.27/Modules/FetchContent.cmake:2016 (FetchContent_Populate)
  CMakeLists.txt:298 (FetchContent_MakeAvailable)

-- Configuring incomplete, errors occurred!

Hello,
Thank you for reporting the issue!
The error message, apparently, suggests that an internal error occurred in cmake itself. The log suggests that cmake 3.27 was used.
I have cmake 3.27.6 installed via brew on my macOS and the build completed successfully just a few minutes ago. Equally, the internal CI has successful builds for all platforms.
Would you please check if your cmake version can be updated?

Thank you for your fast reply !
I currently run version 3.27.7 of cmake, installed from the package available on cmake.org. I am running macOS  14.1 (Sonoma) on a MacBook Air M2 (2022).
I will then try to install cmake with brew and also try with release candidate of version 3.28.0


Support new Apple silicon targets

More info: https://kotlinlang.org/docs/whatsnew1530.html#apple-silicon-support

Closed via #22


The recommended folder structure is odd

...or at least, its rationale unclear.
Does the shared code folder have to be called SharedCode?
Why doesn't the shared code folder have an __init__.py?
How are the other functions importing from the shared code folder if there's no top-level module defined (i.e. no __init__.py at the parent folder level)?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d955d169-8fb8-c511-8911-3e49f95c5ce6
Version Independent ID: 8a3d927c-51ce-d7d4-e5d1-acd928826db2
Content: Python developer reference for Azure Functions
Content Source: articles/azure-functions/functions-reference-python.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

Hi @robertlagrant Thanks for your feedback. We will review and update as appropriate.

@kulkarnisonia16 can you suggest a more reasonable structure for the shared_code folder examples?

This is still unclear. Can someone provide a working sample with shared code between several python functions?

A recent change in folder requirements should resolve these issues. Please see here. If there are still issues, please feel free to open a new issue. #please-close
@Hazhzeng


feat(candle_demo): 添加candle_demo

Hi! Thanks a lot for the work! Could you please add a README for candle usage?

Hi! Thanks a lot for the work! Could you please add a README for candle usage?

done


Postfix completion not shown with gopls

Make sure you have done the following

[X] I have updated to the latest version of blink.cmp
[X] I have read the README

Bug Description
nvim-cmp：

blink.cmp

Relevant configuration
{ -- saghen/blink.cmp
    'saghen/blink.cmp',
    lazy = false, -- lazy loading handled internally
    -- optional: provides snippets for the snippet source
    dependencies = 'rafamadriz/friendly-snippets',

    -- use a release tag to download pre-built binaries
    version = 'v0.*',
    -- OR build from source, requires nightly: https://rust-lang.github.io/rustup/concepts/channels.html#working-with-nightly-rust
    -- build = 'cargo build --release',
    -- If you use nix, you can build from source using latest nightly rust with:
    -- build = 'nix run .#build-plugin',

    ---@module 'blink.cmp'
    ---@type blink.cmp.Config
    opts = {
      -- 'default' for mappings similar to built-in completion
      -- 'super-tab' for mappings similar to vscode (tab to accept, arrow keys to navigate)
      -- 'enter' for mappings similar to 'super-tab' but with 'enter' to accept
      -- see the "default configuration" section below for full documentation on how to define
      -- your own keymap.
      keymap = { preset = 'enter' },

      -- default list of enabled providers defined so that you can extend it
      -- elsewhere in your config, without redefining it, via `opts_extend`
      sources = {
        completion = {
          enabled_providers = { 'lsp', 'path', 'snippets', 'buffer' },
        },
        providers = {
          lsp = {
            name = 'LSP',
            module = 'blink.cmp.sources.lsp',

            enabled = true, -- Whether or not to enable the provider
            transform_items = function(_, items)
              CompletionItemKind = require('blink.cmp.types').CompletionItemKind
              for _, item in ipairs(items) do

              if item.kind == CompletionItemKind.Function or item.kind == CompletionItemKind.Method then
                item.label = item.label .. "()"
              elseif item.kind == CompletionItemKind.Class or item.kind == CompletionItemKind.Struct then
                item.label = item.label .. "{}"
              end
            end
              return items
            end,
            should_show_items = true, -- Whether or not to show the items
            max_items = nil, -- Maximum number of items to display in the menu
            min_keyword_length = 0, -- Minimum number of characters in the keyword to trigger the provider
            fallback_for = {}, -- If any of these providers return 0 items, it will fallback to this provider
            score_offset = 0, -- Boost/penalize the score of the items
            override = nil, -- Override the source's functions
          },
        }
      },

      completion = {
        accept = { auto_brackets = { enabled = true } } ,
        menu = {
          winblend = 50,
          scrolloff = 1,
          scrollbar = false,
          columns = { { 'kind_icon' }, { 'label', 'label_description', gap = 0 } },

          draw = {
            padding = 0,
            gap = 1,
            treesitter = true,
          }
        },

        documentation = {
          auto_show = true,
          border = "none",
          winblend = 50,
          col = 1,
        },
      },

      signature = {
        enabled = false,
        window = {
          winblend = 50,
        },
      },

      appearance = {
        highlight_ns = vim.api.nvim_create_namespace('blink_cmp'),
        -- Sets the fallback highlight groups to nvim-cmp's highlight groups
        -- Useful for when your theme doesn't support blink.cmp
        -- Will be removed in a future release
        use_nvim_cmp_as_default = true,
        -- Set to 'mono' for 'Nerd Font Mono' or 'normal' for 'Nerd Font'
        -- Adjusts spacing to ensure icons are aligned
        nerd_font_variant = 'mono',
        kind_icons = {
          Text = "  ",
          Method = "  ",
          -- Method = "󰰐",
          -- Function = "󰊕",
          Function = " 󰡱 ",
          -- Function = "󰯻",
          Constructor = "  ",
          -- Field = "󰇽",
          Field = " 󱀿 ",
          -- Field = "󰰱",
          -- Variable = "󰂡",
          Variable = " 󰫧 ",
          -- Variable = "󰰱",
          -- Variable = "󰰫",
          -- Class = "󰠱",
          -- Class = "",
          Class = "  ",
          -- Class = "󰰥",
          Interface = "  ",
          -- Interface = "󰰄",
          Module = " 󰏖 ",
          -- Module = "󰏗",
          Property = " 󰜢 ",
          Unit = "  ",
          Value = " 󰎠 ",
          Enum = "  ",
          Keyword = " 󰌋 ",
          Snippet = "  ",
          Color = " 󰏘 ",
          File = " 󰈙 ",
          Reference = "  ",
          Folder = " 󰉋 ",
          EnumMember = "  ",
          Constant = "  ",
          -- Constant = "󰯲",
          -- Struct = "󰰥",
          -- Struct = "",
          Struct = " 󰙅 ",
          -- Struct = "",
          Event = "  ",
          Operator = " 󰆕 ",
          TypeParameter = " 󰅲 ",
          Codeium = " 󰅴 ",
        },
      },
    },

neovim version
NVIM v0.10.2 Build type: RelWithDebInfo LuaJIT 2.1.1727870382 Run "nvim -V1 -v" for more info
blink.cmp version: branch, tag, or commit
main

I've been unable to reproduce it in my environment 🤔 . Is it still occurring for you?

I've been unable to reproduce it in my environment 🤔 . Is it still occurring for you?

Yes, it's still the same. Could you share your configuration? I’d like to check if it’s a configuration issue. For nvim-cmp, it seems that vsnip needs to be configured to use postfix completion, but I didn’t see any corresponding instructions for blink.cmp.

It seems likely you're missing the capabilities from the installation section of the readme

It seems likely you're missing the capabilities from the installation section of the readme

Thank you, I missed the gopls part of following configuration.
      opts = {
        servers = {
          lua_ls = {},
          gopls = {}
        }
      },


function-literal rule conflict with maximum line length

Every time the code is formatted with ktlint, the code block below alternates between the two states given at the Observed Behavior.
Expected Behavior

I expect this to be accepted without raising an error.
                                          X
someLongNamedVariable.someField?.let {
        field
    ->

Observed Behavior

Below code gives us the error: Newline expected before parameter (standard:function-literal)
                                          X
someLongNamedVariable.someField?.let { field ->

However, below code gives us the error: No newline expected before parameter (standard:function-literal)
                                          X
someLongNamedVariable.someField?.let {
        field
    ->

Steps to Reproduce

Example code provided above.
Your Environment

Version of ktlint used: 1.3.1
Relevant parts of the .editorconfig settings

ktlint_standard_max-line-length = disabled
ktlint_code_style = android_studio

The max_line_length setting is invalid. Use either off or unset as value to disable the max line length.

The max_line_length setting is invalid. Use either off or unset as value to disable the max line length.

In hindsight the "solution" above is really crude. What I meant to say was that if you really want to disable the max-line-length checking, that you need to set below:
max_line_length = unset

Disabling the max-line-length rule with ktlint_standard_max-line-length = disabled only disables that specific rule. But any other rule that uses the property max_line_length will not be affected by that. In this case the function-literal rule is using the max_line_length to decide whether it should report a violation regarding the positioning of the variables of the lambda.
The "mistake" you made is very understandable and you can not be blamed for it. Even in case it would have been documented properly (which it isn't), this misleads others as well. Ktlint should be more fault tolerant.


Unreliable internet connection from pods

Environmental Info:
K3s Version: v1.22.7+k3s1
Node(s) CPU architecture, OS, and Version:
Linux k3s-host-0 5.13.0-30-generic #33~20.04.1-Ubuntu SMP Mon Feb 7 14:25:10 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Cluster Configuration:
1 single host.
Describe the bug:
Internet connection is not reliable inside the pods. It works randomly.
Steps To Reproduce:
Install K3S on Ubuntu 20.04 LTS with the following command: curl -sfL https://get.k3s.io | sh -
In my case, the server is a random bare metal server from hetzner.
Install tcptraceroute on the host, do a traceroute to a random reliable service, and notice it works.
tcptraceroute google.com 80 for example.
Do the same inside a pod:
kubectl run test-pod --rm -i --tty --image alpine -- /bin/sh`
# wait
# Install tcptraceroute, may require multiple tentatives
apk add tcptraceroute
tcptraceroute google.com 80

Notice that it works randomly, but not always.  It's always the last server that does not respond. All intermediary routers do respond.
Expected behaviour:
Internet connection should be reliable.
Actual behaviour:
Internet connection is not reliable and the tcp packets do not seem to be routed correctly.
Additional context / logs:
I installed Docker temporarily and the internet connection works correctly inside the containers. I removed it after testing, don't blame docker.
I also tried the previous version (1.22.6) and the latest version without success.
My installation is very vanilla and I have a similar setup in another VM from another cloud provider that has no issues. I know the problem doesn't appear everywhere, but I have no idea why it appears on my setup and I think it may be present in other similar installations.

😮 I just lost 2 hours trying to figure out why my pods couldn't reach internet anymore.
I reverted back to v1.22.6+k3s1 which instantly reverted the issue.
for what it's worth, I've been experimenting a bit during the issue, and noticed that udp traffic was working correctly.
moreover, with iptables I noticed that TCP connections were established with the internet peers, and the answer came back, but it's as if the container never processed the internet peer answer. masquerading looked good though, and the fact that udp is working would also indicate another issue.
this definitely is a major bug in 1.22.7 😬

The revers didn't work for me. What is your kernel version by any chance ?

cc @manuelbuil perhaps something on the flannel side?

kernel version: 5.11.0-49-generic on an Ubuntu 21.04 node and 5.10.0-9-cloud-amd64 on a Debian 11 node
I'm using Cilium and therefore running my k3s with the following options:
ExecStart=/usr/local/bin/k3s \
    server \
    --flannel-backend=none \
    --disable servicelb \
    --disable traefik \
    --disable local-storage \
    --disable-kube-proxy \

Not sure how this could have an impact. would be interesting to know @fungiboletus what settings you use

Hmm, if you're using your own CNI, perhaps the same issue as this then:

https://github.com/k3s-io/k3s/issues/5188#issuecomment-1059072020

On my side I only use the defaults settings.

@fungiboletus Does disabling the network policy controller have any effect on your issue?

kernel version: 5.11.0-49-generic on an Ubuntu 21.04 node and 5.10.0-9-cloud-amd64 on a Debian 11 node
I'm using Cilium and therefore running my k3s with the following options:
ExecStart=/usr/local/bin/k3s \
    server \
    --flannel-backend=none \
    --disable servicelb \
    --disable traefik \
    --disable local-storage \
    --disable-kube-proxy \

Not sure how this could have an impact. would be interesting to know @fungiboletus what settings you use

As @brandond was saying, please disable the network policy controller. As stated in the docs: "Run K3s with --flannel-backend=none and install your CNI of choice. Most CNI plugins come with their own network policy engine, so it is recommended to set --disable-network-policy as well to avoid conflicts." https://rancher.com/docs/k3s/latest/en/installation/network-options/#custom-cni

@fungiboletus Could you please share the output of the following commands:
ip a
ip r
ip rule
cat /proc/sys/net/ipv4/conf/all/forwarding
cat /proc/sys/net/ipv4/conf/eth0/forwarding (or replace eth0 with whatever is your main interface)

kernel version: 5.11.0-49-generic on an Ubuntu 21.04 node and 5.10.0-9-cloud-amd64 on a Debian 11 node
I'm using Cilium and therefore running my k3s with the following options:
ExecStart=/usr/local/bin/k3s \
    server \
    --flannel-backend=none \
    --disable servicelb \
    --disable traefik \
    --disable local-storage \
    --disable-kube-proxy \

Not sure how this could have an impact. would be interesting to know @fungiboletus what settings you use

@clementnuss As @brandond was saying, please disable the network policy controller. As stated in the docs: "Run K3s with --flannel-backend=none and install your CNI of choice. Most CNI plugins come with their own network policy engine, so it is recommended to set --disable-network-policy as well to avoid conflicts." https://rancher.com/docs/k3s/latest/en/installation/network-options/#custom-cni

This probably isn't an issue with NetPol: I've done the upgrade again to 1.22.7 and I'm again in the failure mode.
you were right, adding --disable-network-policy solved my problem! there is one major catch, though:
all pods have to be restarted, for the solution to be effective (and obviously the k3s service before that)
thanks a lot 👏🏻 🎉

kernel version: 5.11.0-49-generic on an Ubuntu 21.04 node and 5.10.0-9-cloud-amd64 on a Debian 11 node
I'm using Cilium and therefore running my k3s with the following options:
ExecStart=/usr/local/bin/k3s \
    server \
    --flannel-backend=none \
    --disable servicelb \
    --disable traefik \
    --disable local-storage \
    --disable-kube-proxy \

Not sure how this could have an impact. would be interesting to know @fungiboletus what settings you use

@clementnuss As @brandond was saying, please disable the network policy controller. As stated in the docs: "Run K3s with --flannel-backend=none and install your CNI of choice. Most CNI plugins come with their own network policy engine, so it is recommended to set --disable-network-policy as well to avoid conflicts." https://rancher.com/docs/k3s/latest/en/installation/network-options/#custom-cni

~This probably isn't an issue with NetPol: I've done the upgrade again to 1.22.7 and I'm again in the failure mode.~
you were right, adding --disable-network-policy solved my problem! there is one major catch, though: all pods have to be restarted, for the solution to be effective (and obviously the k3s service before that)
thanks a lot 👏🏻 tada

Yay!!! :partying_face:

@manuelbuil  Thanks for following up. I uploaded the outputs on a gist.
It's while running k3s with the --disable-network-policy option, which doesn't fix the issue for me.
I also uploaded another log file where I start an alpine pod and run tcptraceroute. Installing tcptraceroute took 2 tentatives, and the tcptraceroute fails randomly to reach destination as you may see.

@manuelbuil Thanks for following up. I uploaded the outputs on a gist.
It's while running k3s with the --disable-network-policy option, which doesn't fix the issue for me.
I also uploaded another log file where I start an alpine pod and run tcptraceroute. Installing tcptraceroute took 2 tentatives, and the tcptraceroute fails randomly to reach destination as you may see.

Thanks, have you noticed that you have the same ip twice in your main interface but with different mask?
2: enp41s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether a8:a1:59:8b:33:b3 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/32 scope global enp41s0
       valid_lft forever preferred_lft forever
    inet <IP_ADDRESS>/26 brd <IP_ADDRESS> scope global enp41s0
       valid_lft forever preferred_lft forever
    inet6 2a01:4f9:4b:44c9::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::aaa1:59ff:fe8b:33b3/64 scope link
       valid_lft forever preferred_lft forever

As it is the same ip, it should not matter but still.
Could you try to ping <IP_ADDRESS> from the alpine pod? I'd like to see if it works. traceroute is not the most reliable tool ;)
I would also like to see the output of:
kubectl get pods -A -o wide

Thanks. Indeed, I had both a static IP configuration and DHCP enabled. I fixed that and also disabled IPv6 (doesn't need it to debug).
A created a new gist with the updated outputs and the list of pods.
Ping is working.

Thanks. When you say: Internet connection is not reliable and the tcp packets do not seem to be routed correctly., it means tcptraceroute fails, right? Or are you seeing any other problem?

The main problem is that HTTP(s) requests fail or are very slow. tcptraceroute is nice to illustrate the problem, but curl also fail randomly for example.

Ok, so for example curl -I http://www.google.com fails, right?
Can you check if you get any output on:
kubectl get networkpolicy -A

Yes this command fail (I just configured the timeout to not wait 5 minutes like by default):
$ curl --connect-timeout 30 -I http://www.google.com
curl: (28) Failed to connect to www.google.com port 80 after 15202 ms: Operation timed out

I don't have network policy, the command returns No resources found.

I found the issue ! Thanks a lot for the help @manuelbuil The problem was not from k3s but my installation, which was likely the case but I really spend time trying to remove everything else in my system.
My Hetzner server used a default firewall configuration which is simply broken for k3s and probably many other things. It blocks incoming ack tcp packets on ports below 32768. I edited the firewal rule to allow all ports and it's working again.
Broken firewall:

Great!


Reboot don't start service on Ubuntu 14.04

As Ubuntu 14.04 uses upstart, the provided script is a System V init script. The problem is that ubuntu will create:
S20docker-volume-netshare -> ../init.d/docker-volume-netshare

And the provided script uses basename $0 and internally tries to use it as a binary that will not exists.
Maybe there is a relation with #33 /cc @lqb @holgerreif

I've written a simple upstart script and named it /etc/init/docker-volume-netshare.conf
description "Docker Volume Netshare daemon"

start on (filesystem and net-device-up IFACE!=lo)
stop on runlevel [!2345]

respawn

script
  BASE=$UPSTART_JOB
  DKV_NETSHARE=/usr/bin/$BASE
  # This is the pid file created/managed by start-stop-daemon
  DKV_NETSHARE_SSD_PIDFILE=/var/run/$BASE-ssd.pid
  DKV_NETSHARE_LOGFILE=/var/log/$BASE.log
  DKV_NETSHARE_OPTS=
  if [ -f /etc/default/$UPSTART_JOB ]; then
    . /etc/default/$UPSTART_JOB
  fi
  exec $DKV_NETSHARE $DKV_NETSHARE_OPTS
end script

It is working for me


[Question] Working with custom request schemas

I'm looking to use pytriton to run inference for a model. We have existing systems that send requests with the format:
{
    'embedding' : [...]
}

However triton/pytriton requires the format:
{
    "inputs" : [
        {
            "name" : "embedding",
            "shape" : [1, embedding_size],
            "datatype" : "FP32",
            "data" : [...]
        }
    ]
}

Is there a way to change the input schema of pytriton or parse a custom request in the inference function?
Certainly one workaround would be to set up a FastAPI server that receives requests of the first format, transforms them to pytriton format, sends the request to pytriton and returns the response. However this requires adding another server, another hop, deserializing/serializing the request data, etc.
Is there a solution to this within pytriton that avoids this complexity? For example processing the raw request in the infer function.
For instance the vllm example appears to process a different request schema from the triton generate documentation using an infer function that takes a Request as input, but I'm not sure if that's unique to the generate endpoint.

Both Triton and FastAPI are valid options for serving machine learning models, but they serve different purposes:

Triton is ideal for large-scale, production-grade inference with a predefined API structure, though it restricts the ability to define custom endpoints.
FastAPI offers greater flexibility and control over API endpoints, request handling, and business logic, making it suitable for more customized workflows, but it requires more manual setup for performance optimizations.

Each solution comes with its strengths: Triton excels in scalability and performance optimization for inference, while FastAPI shines in its flexibility and ease of customization. Choose Triton if you need performance at scale, or FastAPI if you require fine-grained control over API design.
Triton Inference Server Solution
Here’s the Triton code sample that uses the /generate endpoint to handle embedding generation:
import logging
from typing import Dict, List, AsyncGenerator
import numpy as np
from transformers import AutoTokenizer, AutoModel
import torch
from pytriton.model_config import ModelConfig, Tensor
from pytriton.triton import Triton, TritonConfig
from pytriton.proxy.types import Request

# Load the Hugging Face model and tokenizer
model_name = "sentence-transformers/all-MiniLM-L6-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

# Configure logger
logging.basicConfig(level=logging.INFO)
LOGGER = logging.getLogger("pytriton.embedding_generate_server")

def encode_embeddings(input_texts: List[str]) -> np.ndarray:
    """Generate embeddings using Hugging Face model."""
    inputs = tokenizer(input_texts, return_tensors="pt", padding=True, truncation=True)
    with torch.no_grad():
        outputs = model(**inputs)
    embeddings = outputs.last_hidden_state.mean(dim=1).cpu().numpy()  # Mean pooling
    return embeddings

async def generate_fn(requests: List[Request]) -> AsyncGenerator[List[Dict[str, np.ndarray]], None]:
    """Function to process generate requests and return embeddings."""
    responses = []

    for request in requests:
        raw_embeddings = request.data.get("text_input")

        if raw_embeddings is None:
            raise ValueError("Missing 'text_input' in request.")

        embeddings_list = [item.decode('utf-8') for item in raw_embeddings.tolist()]

        embeddings = encode_embeddings(embeddings_list)

        response = {"embedding": embeddings}
        responses.append(response)

    yield responses

if __name__ == "__main__":
    triton_config = TritonConfig(http_port=8000)

    with Triton(config=triton_config) as triton:
        triton.bind(
            model_name="embedding_model",
            infer_func=generate_fn,
            inputs=[
                Tensor(name="text_input", dtype=np.bytes_, shape=(1,)),
            ],
            outputs=[
                Tensor(name="embedding", dtype=np.float32, shape=(-1, 384)),
            ],
            config=ModelConfig(max_batch_size=1, batching=False)
        )
        triton.serve()

There is no any need to do anything with GPU at inputs level here because you will tokenize then at CPU so there is no any benefit from Triton GPU capabilities here. You have to make sure in Python tak your embeddings will be computed on GPU. The embeddings are just lookups in the model, so it should be fast enough on CPU.
Curl Example for Triton /generate Endpoint:
To send a request to Triton’s /generate endpoint, you can use the following curl command:
curl -X POST http://localhost:8000/v2/models/embedding_model/generate \
    -H "Content-Type: application/json" \
    -d '{
        "text_input": "This is an example sentence.",
        "parameters": {}
    }'

You can't remote the parameters field from the request as it is required by Triton endpoint implementation. If next version will include some other fields, you will also need to work with them so you are dependent on Triton implementation in C++, which you can't easily change.
Expected Triton Response:
{"embedding":[0.46799784898757937,0.3234274387359619,0.29819953441619875,0.45349982380867007,0.1747879683971405,-0.019004419445991517,..., -0.13932937383651734],"model_name":"embedding_model","model_version":"1"}

As you can see response contains the embedding values and metadata about the model used for inference, which you can't change.
Limitations of Triton Inference Server:
While Triton Inference Server is highly optimized for large-scale inference, it has limitations regarding custom endpoints:

Predefined Endpoints: Triton only supports predefined endpoints such as /generate, /infer, etc., which are baked into its C++ backend.
Lack of Flexibility: You cannot easily create custom endpoints with Triton, and the input/output formats are structured around Triton’s expected configuration (e.g., tensor names, shapes, and types).

FastAPI Solution
Here’s the FastAPI implementation that mirrors the behavior of Triton’s /generate endpoint but with more flexibility in handling requests and defining custom logic:
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import List, Optional, Dict
import numpy as np
from transformers import AutoTokenizer, AutoModel
import torch

# Initialize FastAPI
app = FastAPI()

# Load the Hugging Face model and tokenizer
model_name = "sentence-transformers/all-MiniLM-L6-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

class GenerateRequest(BaseModel):
    text_input: str
    parameters: Optional[Dict] = None

class GenerateResponse(BaseModel):
    embedding: List[List[float]]

def encode_embeddings(input_texts: List[str]) -> np.ndarray:
    """Generate embeddings using Hugging Face model."""
    inputs = tokenizer(input_texts, return_tensors="pt", padding=True, truncation=True)
    with torch.no_grad():
        outputs = model(**inputs)
    embeddings = outputs.last_hidden_state.mean(dim=1).cpu().numpy()  # Mean pooling
    return embeddings

@app.post("/v2/models/embedding_model/generate", response_model=GenerateResponse)
async def generate_embedding(request: GenerateRequest):
    if not request.text_input:
        raise HTTPException(status_code=400, detail="Missing 'text_input' in the request.")

    embeddings = encode_embeddings([request.text_input])

    return GenerateResponse(embedding=embeddings.tolist())

This code is much more flexible than Triton, as you can define custom endpoints, request structures, and response formats. You can easily modify the logic to handle different request formats, include additional parameters, or extend the functionality to meet specific use cases. It is also much easier to work with and debug than Triton, as it is written in Python. You can directly itegrate payment gateways, databases, etc. in FastAPI. Non-of this will be possible in Triton so you will need to have additional server for this. You can use PyTorch here also directly and leverage GPU capabilities inside FastAPI without any problems.
Curl Example for FastAPI Endpoint:
To send a request to the FastAPI /generate endpoint, use this curl command:
curl -X POST http://localhost:8000/v2/models/embedding_model/generate \
    -H "Content-Type: application/json" \
    -d '{
        "text_input": "This is an example sentence.",
        "parameters": {}
    }'

Expected FastAPI Response:
{
  "embedding": [
    [
      0.12345, -0.56789, ..., 0.4321  # Embedding values (384-dim array)
    ]
  ]
}

Advantages of FastAPI:

Custom Endpoints: FastAPI allows full control over the endpoints and request structure. You can easily define custom routes, query parameters, and response formats.
Flexibility: You can modify the logic to handle different request formats, include additional parameters, or extend the functionality to meet specific use cases.
Custom Logic: You can define custom logic for handling input and output, unlike Triton, where the format is tightly controlled.


Machine-sku issue

OpenPAI Environment:

OpenPAI version: v0.14.0
OS (e.g. from /etc/os-release): Ubuntu 18.04

The main problem:
During the installation we create layout.yaml where we add information about worker machines. I'm interested in knowing, can we allocate different worker nodes the same machine-type?
For example, I have 1 master and 6 worker nodes (3 of them with Tesla K80 and 3 of them Tesla V100). My question is can I add the same machine-type for these 3 Tesla K80 and another one machine-type for Tesla V100? Will be there something wrong?
Another issue is how to manage an allocation of my job to the certain GPU type?
For example, I'm submitting a job and I want to allocate it to Tesla V100 with 32 Gb, not to Tesla K80 with 24 Gb, how to choose it? Should I create individual cluster (dedicated cluster) for each GPU type?

During the installation we create layout.yaml where we add information about worker machines. I'm interested in knowing, can we allocate different worker nodes the same machine-type?

It's OK, the machine-type isn't actually used in YARN scheduler.

Another issue is how to manage an allocation of my job to the certain GPU type?

For v0.14.0 which uses YARN scheduler, we didn't support different GPU types, using dedicated vc is a workaround.
We suggest you to use the latest v1.0.0 which includes a GPU scheduler and fits your need.

@abuccts, I have some problem with dedicated vc.
I use the command python node_maintain.py dedicated-vc add -m {master_ip} -v {added_vc_name} and it shows me output with error:
- HTTPError: 400 Client Error: Bad Request for url: http://master-ip:9186/api/v1/token

The page http://master-ip:9186/api/v1/token:
- {"code":"NoApiError","message":"API /api/v1/token is not found."}

could you change the following line https://github.com/microsoft/pai/blob/2472b37792092e448b4050c776b35acb3e6ffafb/src/tools/operator_wrapper/restserver_operator.py#L53 to api_path = "/api/v1/authn/basic/login" and try again?

@abuccts, There are the same error and the output on the web page:
- {"code":"NoApiError","message":"API /api/v1/authn/basic/login is not found."}

the api is POST /api/v1/authn/basic/login instead of GET /api/v1/authn/basic/login, the second one does not exist indeed if you only tried to send a http get request in your browser

@abuccts, do you mean this POST?

How to reach it properly?
HTTPError: 400 Client Error: Bad Request for url: http://master-ip:9186/api/v1/authn/basic/login

could you get result from the following command?
curl -X POST -d "username=USERNAME" -d "password=PASSWORD" http://master-ip:9186/api/v1/authn/basic/login

@abuccts, do you mean this POST?

How to reach it properly?
HTTPError: 400 Client Error: Bad Request for url: http://master-ip:9186/api/v1/authn/basic/login

Hi @andrewsen13 - would you mind share the Org name to us? we are adding tags to the issues to clustering issues by user's org.

@abuccts,

could you get result from the following command?
curl -X POST -d "username=USERNAME" -d "password=PASSWORD" http://master-ip:9186/api/v1/authn/basic/login

I tried curl command with my new user which I created using command python node_maintain.py user set -u {admin_username} -p {admin_password} -m {master_ip}, but there was an error: {"code":"NoUserError","message":"User undefined is not found."}.
After this, I tried login and password for rest server which admin and admin-password as default, and there was a good connection, output:
{"user":"admin","token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6ImFkbWluIiwiYWRtaW4iOnRydWUsImlhdCI6MTU5MTAxMzg1NiwiZXhwIjoxNTkxNjE4NjU2fQ.dZxJ3WHPcF0NOojVRKW6V2mJhl661OVEaVuIeX3Pc0o","admin":true,"hasGitHubPAT":false}
However, there is the same error:
- HTTPError: 400 Client Error: Bad Request for url: http://master-ip:9186/api/v1/authn/basic/login

I think there're some misunderstandings.

The account in python node_maintain.py user set -u {admin_username} -p {admin_password} -m {master_ip} is your rest server/webportal account, which is used to login webportal. I don't know which account you used before.

/bin/sh: 1: yarn: not found means hadoop yarn cli instead of yarnpkg in nodejs, pls run the script in dev-box docker. You could use the following command to check it exists:
$ which yarn
/usr/local/hadoop/bin/yarn

@abuccts,

I've just read here  how to create a dedicated VC and there are no information that I should use credentials from the rest server/webportal, only that I have to create a user. By the way, I understood this information and I will use the admin account for rest server in the future. Thanks!
I use a SingleBox deployment using this instruction where I didn't create a dev-box docker image or container. I only installed basic components.
I had an output of which yarn when I installed yarn using npm install -g yarn like this: /usr/local/bin/yarn. However, after your message here I've deleted yarn using npm uninstall -g yarn and now my output which yarn is empty.

How to define yarn to /usr/local/hadoop/bin/yarn?

You could refer to dev-box's Dockerfile to install hadoop
https://github.com/microsoft/pai/blob/2472b37792092e448b4050c776b35acb3e6ffafb/src/dev-box/build/dev-box.dockerfile#L67-L79
then export the binary directory to PATH and check
$ export PATH=$PATH:/usr/local/hadoop/bin
$ which yarn

@abuccts,
Oh my god... /_-
I just didn't add export PATH=$PATH:/usr/local/hadoop/bin, but my configuration of hadoop was the same.
At this moment, I see the path of yarn like this:
 /usr/local/hadoop/bin/yarn

Also, finally, I've created dedicated VC and everything is working fine.
Thank you so much for your help and your time!