hbone: initial echo server/client implementation

Per https://docs.google.com/document/d/1Ofqtxqzk-c_wn0EgAXjaJXDHB9KhDuLe-W3YGG67Y8g/edit

I would really appreciate it if we could have this in istio/pkg repo
instead of istio/istio.
Preferably with a separate go.mod
On Mon, Jun 27, 2022 at 10:07 AM John Howard @.***>
wrote:

Per
https://docs.google.com/document/d/1Ofqtxqzk-c_wn0EgAXjaJXDHB9KhDuLe-W3YGG67Y8g/edit
You can view, comment on, or merge this pull request online at:
https://github.com/istio/istio/pull/39645
Commit Summary

96664a0
https://github.com/istio/istio/pull/39645/commits/96664a0a831bbec8d904893a62242a0668b61746
hbone: initial echo server/client implementation

File Changes
(21 files https://github.com/istio/istio/pull/39645/files)

M pkg/config/protocol/instance.go
https://github.com/istio/istio/pull/39645/files#diff-21283db6b7f65c109d07b20d212b35522f888fceacfdbfa1c5b5b82a6ad78025
(2)
A pkg/hbone/README.md
https://github.com/istio/istio/pull/39645/files#diff-e25697f13fb882efc49d4c89395a4cc965f458f7c2001160fe1d6988f7794afa
(71)
A pkg/hbone/dialer.go
https://github.com/istio/istio/pull/39645/files#diff-cd1929a737ac8d0a2a9fe0143861f844e0f414aa25e1668937f1f911faa7537a
(173)
A pkg/hbone/dialer_test.go
https://github.com/istio/istio/pull/39645/files#diff-d7a237c961968104637e94463dbb3fd4cd339c26d76481106f72a32e42f8bda2
(91)
A pkg/hbone/server.go
https://github.com/istio/istio/pull/39645/files#diff-3dc57dc389b89e1156ea75bebb163573d4e5513460c434b4b6bf7e4b2917211b
(77)
A pkg/hbone/util.go
https://github.com/istio/istio/pull/39645/files#diff-175c2ab06c1a9b382abeaae629acf230d652a3fdc55d68218a212c4a62bc01ad
(112)
M pkg/test/echo/cmd/client/main.go
https://github.com/istio/istio/pull/39645/files#diff-f387c590098171a27e3ca0570906f09e9056d07f359faec76a20cc30227d56b2
(39)
M pkg/test/echo/cmd/server/main.go
https://github.com/istio/istio/pull/39645/files#diff-5160d37a91acd67dec846c9f3f1a6bf217b60844afe4b068cf62d1939acbb665
(13)
M pkg/test/echo/proto/echo.pb.go
https://github.com/istio/istio/pull/39645/files#diff-df0206c1e693524e51bd51b93ce397d8245580647ee379448333811ac3042fc7
(238)
M pkg/test/echo/proto/echo.proto
https://github.com/istio/istio/pull/39645/files#diff-efa4da746fbff1ad3f5958b577a47aefc8af96b54366a0569f3655708a5b197a
(20)
A pkg/test/echo/server/endpoint/hbone.go
https://github.com/istio/istio/pull/39645/files#diff-5bc57d18fd4407914228e8bb431b206cdc802a026fbaee8739b68b5c591423d6
(99)
M pkg/test/echo/server/endpoint/http.go
https://github.com/istio/istio/pull/39645/files#diff-e4304ce552073a178d83dbf2aa18d627adb7c41342327c1bcaf2ae3221e76946
(8)
M pkg/test/echo/server/endpoint/instance.go
https://github.com/istio/istio/pull/39645/files#diff-2db4b8678322246ce21bb3538ab4b861a31b270981ca92844c61201b9263b6f7
(2)
M pkg/test/echo/server/forwarder/config.go
https://github.com/istio/istio/pull/39645/files#diff-ed8ba2e29b5fcbc2dcc6f0a4cb261068bbc2aad67f88e9732a79eb176e788d35
(104)
M pkg/test/echo/server/forwarder/grpc.go
https://github.com/istio/istio/pull/39645/files#diff-a2c77c4473c6fa94dbce1c6583baa7e5f3912977b97dbbcb00f139f4bb5a6f9f
(3)
M pkg/test/echo/server/forwarder/http.go
https://github.com/istio/istio/pull/39645/files#diff-e1b381ca14ea251b66e0a46cdd66f3df2aa7c92322d2291ae5e4fa282b304647
(9)
M pkg/test/echo/server/forwarder/tcp.go
https://github.com/istio/istio/pull/39645/files#diff-24551737683074a5fdb4fb29a4c21bc9290ad3bd8a9884006a9a9f2f970b4bce
(6)
M pkg/test/echo/server/forwarder/tls.go
https://github.com/istio/istio/pull/39645/files#diff-83b0384b5e264e85d625f7fd9ca4ea82b77477e15336c71138957429fc2bf163
(3)
M pkg/test/echo/server/forwarder/util.go
https://github.com/istio/istio/pull/39645/files#diff-7614882b904467a2f1d65ced57b9503f337154a1a74f8fd56611aaa426779d7d
(13)
M pkg/test/echo/server/forwarder/websocket.go
https://github.com/istio/istio/pull/39645/files#diff-c2ed434d786e882a24f7f106d796247e41be019bc2d6bc7d1e20d3c974eb1a06
(4)
M pkg/test/echo/server/instance.go
https://github.com/istio/istio/pull/39645/files#diff-4c45e029e40ea25a0a105ddc3f81bdbc7c3a5f0ce143169521cb6785d555faa4
(1)

Patch Links:

https://github.com/istio/istio/pull/39645.patch
https://github.com/istio/istio/pull/39645.diff

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/39645, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2XTF6KEN2WPQDQ4JXLVRHNWRANCNFSM5Z6TR2XQ
.
You are receiving this because your review was requested.Message ID:
@.***>

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-distroless_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-distroless_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-distroless_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-distroless_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-ipv6_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-ipv6_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-distroless_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-distroless_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-ipv6_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-ipv6_istio

integ-operator-controller_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-operator-controller_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-distroless_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-distroless_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-ipv6_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-ipv6_istio

integ-operator-controller_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-operator-controller_istio

integ-pilot_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot_istio

integ-security_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-security_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-distroless_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-distroless_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-ipv6_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-ipv6_istio

integ-operator-controller_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-operator-controller_istio

integ-pilot_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot_istio

integ-security_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-security_istio

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-distroless_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-distroless_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-ipv6_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-ipv6_istio

integ-operator-controller_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-operator-controller_istio

integ-pilot_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot_istio

integ-security_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-security_istio

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

integ-helm_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-helm_istio

integ-pilot-istiodremote-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-istiodremote-mc_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-pilot_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot_istio

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

integ-helm_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-helm_istio

integ-pilot-istiodremote-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-istiodremote-mc_istio

integ-pilot-istiodremote_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-istiodremote_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry_istio

integ-telemetry-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-telemetry-mc_istio

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

integ-helm_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-helm_istio

integ-pilot-istiodremote-mc_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-istiodremote-mc_istio

integ-pilot-istiodremote_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-istiodremote_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-pilot_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

On Mon, Jun 27, 2022 at 1:55 PM Nathan Mittler @.***>
wrote:

@.**** commented on this pull request.
In pkg/config/protocol/instance.go
https://github.com/istio/istio/pull/39645#discussion_r907720073:

@@ -54,6 +54,8 @@ const (
Redis Instance = "Redis"
// MySQL declares that the port carries MySQL traffic.
MySQL Instance = "MySQL"

// HBONE declares that the port carries HBONE traffic.
HBONE Instance = "HBONE"

Will HBONE always be over h2? If we ever supported h3, would it be
considered a different protocol?

AFAIK we treat H2 and H3 as the same protocol - it really requires
'bidirectional stream support' which is missing in H1.

In pkg/hbone/dialer.go
https://github.com/istio/istio/pull/39645#discussion_r907725471:

"net/http"

"strings"
"sync"
"time"

"golang.org/x/net/http2"
"golang.org/x/net/proxy"

"istio.io/istio/security/pkg/pki/util"
istiolog "istio.io/pkg/log"
+)

+var log = istiolog.RegisterScope("hbone", "", 0)
+
+type Config struct {

ProxyAddress string

Can you clarify what ProxyAddress is and how it's different from address
in DialContext?
In pkg/hbone/dialer.go
https://github.com/istio/istio/pull/39645#discussion_r907728651:

return d.DialContext(context.Background(), network, address)
+}

+func proxyTo(conn io.ReadWriteCloser, req Config, address string) error {

defer conn.Close()
t0 := time.Now()

url := "http://" + req.ProxyAddress
if req.TLS != nil {

url = "https://" + req.ProxyAddress

}
// Setup a pipe. We could just pass conn to http.NewRequest, but this has a few issues:
// * Less visibility into i/o
// * http will call conn.Close, which will close before we want to (finished writing response).
pr, pw := io.Pipe()
r, err := http.NewRequest("CONNECT", url, pr)

nit: http.MethodConnect?
In pkg/hbone/server.go
https://github.com/istio/istio/pull/39645#discussion_r907731066:

+)

+func NewServer() *http.Server {

// Need to set this to allow timeout on the read header
h1 := &http.Transport{

ExpectContinueTimeout: 3 * time.Second,

}
h2, _ := http2.ConfigureTransports(h1)
h2.ReadIdleTimeout = 10 * time.Minute // TODO: much larger to support long-lived connections
h2.AllowHTTP = true
h2.StrictMaxConcurrentStreams = false
h2Server := &http2.Server{}
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

t0 := time.Now()

if r.Method == http.MethodConnect {

	log.WithLabels("host", r.Host).Info("Received CONNECT")

nit: maybe move this to a handler (e.g. onConnect) method for readability?
In pkg/hbone/server.go
https://github.com/istio/istio/pull/39645#discussion_r907732396:

"sync"

"time"

"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
+)

+func NewServer() *http.Server {

// Need to set this to allow timeout on the read header
h1 := &http.Transport{

ExpectContinueTimeout: 3 * time.Second,

}
h2, _ := http2.ConfigureTransports(h1)
h2.ReadIdleTimeout = 10 * time.Minute // TODO: much larger to support long-lived connections
h2.AllowHTTP = true
h2.StrictMaxConcurrentStreams = false

Why this? h2.StrictMaxConcurrentStreams = false
In pkg/hbone/server.go
https://github.com/istio/istio/pull/39645#discussion_r907738467:

"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
+)

+func NewServer() *http.Server {

// Need to set this to allow timeout on the read header
h1 := &http.Transport{

ExpectContinueTimeout: 3 * time.Second,

}
h2, _ := http2.ConfigureTransports(h1)
h2.ReadIdleTimeout = 10 * time.Minute // TODO: much larger to support long-lived connections
h2.AllowHTTP = true
h2.StrictMaxConcurrentStreams = false
h2Server := &http2.Server{}
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

nit: code might be a little easier to read if you name the variables
similar to response, request
In pkg/hbone/util.go
https://github.com/istio/istio/pull/39645#discussion_r907742074:

+package hbone
+
+import (

"io"
"net"
"net/http"
"sync"
"time"

istiolog "istio.io/pkg/log"
+)

+// createBuffer to get a buffer. io.Copy uses 32k.
+// experimental use shows ~20k max read with Firefox.
+var bufferPoolCopy = sync.Pool{New: func() interface{} {
Is there a version of this that uses thread-local storage perhaps? Would
avoid locking.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/39645#pullrequestreview-1020670960,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2UX3OH6LM6ZQ3RJANTVRIIKPANCNFSM5Z6TR2XQ
.
You are receiving this because your review was requested.Message ID:
@.***>

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-pilot_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot_istio

integ-pilot-istiodremote_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote_istio

integ-pilot-istiodremote-mc_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote-mc_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

integ-cni_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-cni_istio

integ-pilot-istiodremote_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote_istio

integ-pilot-istiodremote-mc_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote-mc_istio

integ-pilot_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-multicluster_istio
96664a0a831bbec8d904893a62242a0668b61746
link
true
/test integ-pilot-multicluster_istio

integ-pilot-istiodremote_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote_istio

integ-pilot-istiodremote-mc_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote-mc_istio

integ-pilot_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot_istio

integ-cni_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-cni_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-istiodremote_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote_istio

integ-pilot-istiodremote-mc_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-istiodremote-mc_istio

integ-pilot_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot_istio

integ-cni_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-cni_istio

integ-pilot-multicluster_istio
6444559032bf3ed5f6e02f41417bcc7ccc0acf28
link
true
/test integ-pilot-multicluster_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

Figured it out, to newDialer we now pass config which is later mutated causing issues. IMO https://github.com/istio/istio/pull/39735 is a better fix (assuming it doesn't break anything, i split it out to make it more clear).

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-cni_istio
49f78f0c36f55f09999a2abca3596ba4feba2602
link
unknown
/test integ-cni_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Can we separate the hbone client from the service port name ? I think it's
a very different discussion, better handled in API/doc.
On Mon, Jun 27, 2022 at 11:45 AM Costin Manolache @.***> wrote:

So if a service port is named 'hbone' - it means clients will make tunnel
requests ? Do we want to keep the 'h2 / h2c' distinction for trusted
networks ?
On Mon, Jun 27, 2022 at 11:39 AM John Howard @.***>
wrote:

@.**** commented on this pull request.
In pkg/config/protocol/instance.go
https://github.com/istio/istio/pull/39645#discussion_r907687911:

@@ -54,6 +54,8 @@ const (
Redis Instance = "Redis"
// MySQL declares that the port carries MySQL traffic.
MySQL Instance = "MySQL"

// HBONE declares that the port carries HBONE traffic.
HBONE Instance = "HBONE"

This is not something a user can configure by portname on Service -- but
used for internal. Not by convention, this is enforced.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/39645#discussion_r907687911, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2TEU55CJ6BXLWMXA43VRHYOXANCNFSM5Z6TR2XQ
.
You are receiving this because your review was requested.Message ID:
@.***>

This doesn't impact the service port - "HBONE" is still not a valid service
port, the protocol enum we are adding to is not implicitly adding it as a
valid option for services
On Tue, Oct 11, 2022 at 2:20 AM Costin Manolache @.***>
wrote:

Can we separate the hbone client from the service port name ? I think it's
a very different discussion, better handled in API/doc.
On Mon, Jun 27, 2022 at 11:45 AM Costin Manolache @.***> wrote:

So if a service port is named 'hbone' - it means clients will make tunnel
requests ? Do we want to keep the 'h2 / h2c' distinction for trusted
networks ?
On Mon, Jun 27, 2022 at 11:39 AM John Howard @.***>
wrote:

@.**** commented on this pull request.
In pkg/config/protocol/instance.go
https://github.com/istio/istio/pull/39645#discussion_r907687911:

@@ -54,6 +54,8 @@ const (
Redis Instance = "Redis"
// MySQL declares that the port carries MySQL traffic.
MySQL Instance = "MySQL"

// HBONE declares that the port carries HBONE traffic.
HBONE Instance = "HBONE"

This is not something a user can configure by portname on Service -- but
used for internal. Not by convention, this is enforced.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/39645#discussion_r907687911, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAAUR2TEU55CJ6BXLWMXA43VRHYOXANCNFSM5Z6TR2XQ

.
You are receiving this because your review was requested.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/39645#issuecomment-1274386687, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXORNUHOOZZDPCZOGJLWCUWPPANCNFSM5Z6TR2XQ
.
You are receiving this because you were mentioned.Message ID:
@.***>


Separate bundle test config

Q
A

Bug fix?
no

New feature?
no, improvement

BC breaks?
no

Deprecations?
no

Fixed tickets
-

Related issues/PRs
#2106, #2118

License
MIT

Documentation PR
-

What's in this PR?
Separate the test runner to use the phpunit config of the bundle instead of the global one. Thus fixed the bundles boostrap.php for PHP 7 (see #2118).
Then moved the SYMFONY_DEPRECATIONS_HELPER environment variable into the phpunit config of the bundles, so it can be configured separately.
Why?
For Symfony 3 compatibility it's important that a bundle which is made compatible should not introduce new deprecations with new features. Now you can change a bundle to strict and it will fail in the builds.
Example Usage
Change the SYMFONY_DEPRECATIONS_HELPER in the phpunit.xml.dist of the bundle which got Symfony 3 compatible to strict:
    <php>
        <env name="SYMFONY_DEPRECATIONS_HELPER" value="strict"/>
    </php>

@patkar please rebase (:

Rebased. It's a bit annyoing, that the changelog creates a conflict every time, another PR is merged..

@patkar thanks for contribution


Clarify VP8/VP9/WebM status on Safari

Separated the duplicate portion of note 4 from note 5.
Added note 5 to iOS Safari 14.
Added off-by-default clause to note 5.
Added note 6 for VP9 + WebM + MediaSource support on Safari 14 + Big Sur.

Fixes #5713 as YouTube uses MediaSource/MSE for 4K HDR support on Big Sur.

Thanks!


Indicate that tracker UI sections are resizable

It would be nice to have some kind of UI indication (see example from picture) that the sections in the tracker interface are resizable.
I did not know that. But I noticed that I can drag the "black lines" when creating this issue. So actually quite unimportant thing because the sections already are resizable.

There used to be a different cursor indicating it, but I need to check that it still works after migrating to latest Gioui version (the GUI library I use).

Done. I am pretty sure that I had this cursor indication in the past, but it was nowhere to be found in the code... Anyway, it's there now.


Autofix: Form action should work without JavaScript

I have identified the issue and implemented a solution to ensure that form actions work correctly even when JavaScript is disabled. The problem was in the isAnAction function, which was checking for properties that are only available when JavaScript is enabled. I've updated this function to use a more reliable method of identifying actions.

[!CAUTION]
Disclaimer: This fix was created by Latta AI and you should never merge before you check the correctness of generated code!
The fix provided by Latta AI might not be complete and it can serve as an inspiration.

This bug was fixed for free by Latta AI - https://latta.ai/ourmission
If you no longer want Latta AI to attempt fixing issues on your repository, you can block this account.

Thanks for the contribution, however is fixed in this https://github.com/brisa-build/brisa/pull/597. Currently, your PR is not fixing the bug.


Bump floem, new style methods

update lapce for new style methods. Also removed references to scroll bar color and scroll bar color is only referenced in the top View Style

I've already done this on my branch.


add querybuilder, tft.msearch and tft.analyzer

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


date axis

You could put this in with a separate csv, but it would be a good idea to have an option to set the dates as consecutive days/hours/etc , to save the users time

b5825f636f0b2b71b0867336ecdcb115835380d5


⚠ Fortnite has degraded performance

In 4a41e5d, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1564 ms

Resolved: Fortnite performance has improved in 3090fa8.


Sort out artifacts for multiple macos versions

Not quite sure what I meant here.
We upload the artifact from the oldest macos version. I think that's good enough so closing this.


replaced Twitter futures with Scala futures

removed all Awaits except for one that is used by
MultiCollins/BackgroundProcess
This is one PR which removes Twitter futures and uses Scala Futures/timeouts.  I removed all timeouts that were not connected to multi collins(not used), and for background processing(because changing BackgroundProcess to being asynchronous is not a trivial task).
I have another branch off of this one which removes the finagle library entirely.  I will pr that one separately as I wanted to keep this one focused.

This doesnt have any test coverage, right? I believe not, as @maddalab showed with his 2.1.5 PR. Do you think its worth it to figure out the test framework and get this covered, or is this a one-and-done thing?

lgtm

I tested this today, I am going to go ahead and merge this, feel free to grab me a nick at work if required.


[usb_testutils] Avoid buffer double-free

The control endpoint code had a path that would lead to a double return
of the buffer. Check that the buffer is still active before calling the
DIF to return it.
This case is due to the buffer read function automatically returning the
buffer underneath when all data has been read.
Signed-off-by: Alexander Williams<EMAIL_ADDRESS>

Nice. I have long suspected there was something like this lurking in the code. Always too focused on the hw to look.

Nice. I have long suspected there was something like this lurking in the code. Always too focused on the hw to look.

The code could use a good refactoring, but this one is my fault, heh. The DIFs handle buffers a little differently than the older code that directly accessed the CSRs, and I forgot to handle this path when I did the conversion.


Preview changes to edited markdown files are not working

When I edit a markdown file, and click on the Preview changes, nothing happens. Also, the Commit changes button appears to be inactive. In order to preview or commit changes, I have to deactivate the  GitHub-Defreshed (v 1.4.5) script in Tapermonkey.

Hm, yes, markdown editor seems broken. Bruh. Will fix it somehow :)
Thanks for your report!

Just found a problem. I'll update the script in few hours

Hi! I'm sorry for this delay, but now editors (Markdown, JSON and so on) work as they should (don't forget to update your script to v1.5.0)
And thanks again for the report, it helped me find a mistake due to which the editors didn't work, which in turn is associated with another small problem that I'll solve in the future :)


(Experimental) Add support to NTK RoPE scaling

This adds support for the new NTK RoPE scaling, mentioned in https://github.com/turboderp/exllama/issues/115.
"According to this post, this is a method of rope scaling that result in less perplexity loss and a bigger possible scaling:
https://www.reddit.com/r/LocalLLaMA/comments/14lz7j5/ntkaware_scaled_rope_allows_llama_models_to_have/"
Adds the parameter "a", "alpha", which is used when loading a model with "-a"
Tested on 65B models at 4K context, with 48GB VRAM (2x24) using gs 16,20

Perplexity:
For tulu-30B-GPTQ (non-SuperHOT)

Perplexity at 2048 ctx (no compress_pos_emb, no alpha RoPE): 5.2153
Perplexity at 8192 ctx, compress_pos_emb = 4: 10.0813
Perplexity at 8192 ctx, alpha  = 4: 5.3534
Perplexity at 8192 ctx, compress_pos_emb = 4, alpha  = 4: 15.4406

For Tulu-30B-SuperHOT-8K-4bit-32g:

Perplexity at 8192 ctx, compress_pos_emb = 4: 5.8166
Perplexity at 8192 ctx, alpha  = 4: 7.5073
Perplexity at 8192 ctx, compress_pos_emb = 4, alpha  = 4: 6.0903

Note: For 8K ctx and above, I suggest to keep using SuperHOT.

Update: For now, the value of alpha for NTK RoPE scaling has to be set manually, since when trying to pass the parameter from model_init.py, it doesn't work (It always keeps being 1.0)

Important update: Before, the alpha value wasn't being applied correctly. Now, it does it correctly, and thus, just by setting alpha for NTK RoPE scaling would be enough (without the need to set compress_pos_emb to the same value)

I might refactor this a bit later, but it seems okay. I'll merge it as is for now.

Hi, I'm confused with the current code. It seems like compress_pos_emb is still used beside alpha value especially if we set compress_pos_emb not equal 1 to scale the value of t. But, dynamic scaling seems didn't do that. Is this intentional?

Hi, I'm confused with the current code. It seems like compress_pos_emb is still used beside alpha value especially if we set compress_pos_emb not equal 1 to scale the value of t. But, dynamic scaling seems didn't do that. Is this intentional?

If compress_pos_emb is set to 1, the rotatory embedding base is still set at 10000. (As if nothing changed)
Ideally you just want to set either compression or alpha.
Also, this implementation of NTK is static RoPE scaling. Dynamic NTK Scaling isn't implemented yet on exllama (it depends of the context length when generating)

Oh thank you for the clarification. So the base value is static based on the alpha. correct? Then after that, we could generate with context more than default context size? If I check the graph from this reddit post, with alpha 4, I could generate with context size 5000 with perplexity not exploded just like the yellow line in the graph?

@fahadh4ilyas correct.


Charm in blocked state after restarting underlying machine

Steps to reproduce
(Note I cannot reproduce this locally)

In Openstack, juju deploy postgresql --channel=14/candidate --revision=307
Only 1 unit of Postgresql is running
openstack server reboot --soft <machine-id> --wait
Observe charm now reports
postgresql/0*          blocked   idle   1        <ip>   5432/tcp  failed to set up postgresql_exporter options

Expected behavior
After restarting the underlying machine, I'd expect Postgres to recover.
Actual behavior
As mentioned above, the charm now reports "failed to set up postgresql_exporter options"
Versions

Operating system: Ubuntu 22.04.3 LTS

Juju CLI: 2.9.44-ubuntu-amd64

Juju agent: 2.9.44

Charm revision: 307

LXD: N/A
Log output
Juju debug log:
log.txt
$ sudo snap logs charmed-postgresql
2023-08-29T20:47:14Z systemd[1]: Stopped Service for snap application charmed-postgresql.pgbackrest-service.
2023-08-29T20:47:14Z systemd[1]: Started Service for snap application charmed-postgresql.pgbackrest-service.
2023-08-29T20:47:14Z charmed-postgresql.pgbackrest-service[62281]: ERROR: [037]: server command requires option: tls-server-auth
2023-08-29T20:47:14Z systemd[1]: snap.charmed-postgresql.pgbackrest-service.service: Main process exited, code=exited, status=37/n/a
2023-08-29T20:47:14Z systemd[1]: snap.charmed-postgresql.pgbackrest-service.service: Failed with result 'exit-code'.
2023-08-29T20:47:14Z systemd[1]: snap.charmed-postgresql.pgbackrest-service.service: Scheduled restart job, restart counter is at 5.
2023-08-29T20:47:14Z systemd[1]: Stopped Service for snap application charmed-postgresql.pgbackrest-service.
2023-08-29T20:47:14Z systemd[1]: snap.charmed-postgresql.pgbackrest-service.service: Start request repeated too quickly.
2023-08-29T20:47:14Z systemd[1]: snap.charmed-postgresql.pgbackrest-service.service: Failed with result 'exit-code'.
2023-08-29T20:47:14Z systemd[1]: Failed to start Service for snap application charmed-postgresql.pgbackrest-service.

Hi @kian99! Thanks for the meeting today and for the logs. I tried to reproduce it in other environments without success, as you commented, you don't face it locally. I'm gonna request access to an openstack environment to test it.

Hi @kian99 , I cannot reproduce this using the latest 14/candidate.
After the multipass VM store/start with PG cluster on it, the charm is fully recovered:
:
ubuntu@juju317:~$ juju status
Model   Controller  Cloud/Region         Version  SLA          Timestamp
reboot  lxd_3.1.8   localhost/localhost  3.1.8    unsupported  16:39:55+02:00

App         Version  Status   Scale  Charm       Channel       Rev  Exposed  Message
postgresql  14.11    waiting      3  postgresql  14/candidate  413  no       Data directory not attached

Unit           Workload  Agent      Machine  Public address  Ports     Message
postgresql/0   waiting   executing  0        <IP_ADDRESS>     5432/tcp  (start) Data directory not attached
postgresql/1   waiting   executing  1        <IP_ADDRESS>   5432/tcp  (start) Data directory not attached
postgresql/2*  waiting   idle       2        <IP_ADDRESS>   5432/tcp  awaiting for member to start

Machine  State    Address        Inst id        Base          AZ  Message
0        started  <IP_ADDRESS>    juju-fd0fb0-0<EMAIL_ADDRESS>    Running
1        started  <IP_ADDRESS>  juju-fd0fb0-1<EMAIL_ADDRESS>    Running
2        started  <IP_ADDRESS>  juju-fd0fb0-2<EMAIL_ADDRESS>    Running

...

ubuntu@juju317:~$ juju status
Model   Controller  Cloud/Region         Version  SLA          Timestamp
reboot  lxd_3.1.8   localhost/localhost  3.1.8    unsupported  16:46:52+02:00

App         Version  Status  Scale  Charm       Channel       Rev  Exposed  Message
postgresql  14.11    active      3  postgresql  14/candidate  413  no

Unit           Workload  Agent  Machine  Public address  Ports     Message
postgresql/0   active    idle   0        <IP_ADDRESS>     5432/tcp
postgresql/1   active    idle   1        <IP_ADDRESS>   5432/tcp
postgresql/2*  active    idle   2        <IP_ADDRESS>   5432/tcp  Primary

Machine  State    Address        Inst id        Base          AZ  Message
0        started  <IP_ADDRESS>    juju-fd0fb0-0<EMAIL_ADDRESS>    Running
1        started  <IP_ADDRESS>  juju-fd0fb0-1<EMAIL_ADDRESS>    Running
2        started  <IP_ADDRESS>  juju-fd0fb0-2<EMAIL_ADDRESS>    Running

Is it still reproducible on your side? Tnx!

@taurus-forever I will test this on Monday with an Openstack VM and report back.

Ah sorry I set a reminder and didn't get a chance to check. I will have to carve some time to test. Since there have been a number of releases since, I'm happy to close the issue and I'll re-open if I can reproduce with the latest version.


Generation of PDF fails when using dynamic options in repeating groups

Description of the bug
Adding dynamic options for repeating groups fails when generating PDF.
This is due to our index-indicators are not "initialized" in the pdf-generator. Need to:

Find out how many indexes there are for the given group
"new up" options for each index

Reported in slack (internal link). Related stack trace:
fail: Altinn.App.Api.Controllers.ProcessController[0]
      Process next failed.
      Newtonsoft.Json.JsonException: Unexpected character while parsing path indexer: {
         at Newtonsoft.Json.Linq.JsonPath.JPath.ParseArrayIndexer(Char indexerCloseChar)
         at Newtonsoft.Json.Linq.JsonPath.JPath.ParseIndexer(Char indexerOpenChar, Boolean scan)
         at Newtonsoft.Json.Linq.JsonPath.JPath.ParsePath(List`1 filters, Int32 currentPartStartIndex, Boolean query)
         at Newtonsoft.Json.Linq.JsonPath.JPath.ParseMain()
         at Newtonsoft.Json.Linq.JsonPath.JPath..ctor(String expression)
         at Newtonsoft.Json.Linq.JToken.SelectToken(String path, JsonSelectSettings settings)
         at Altinn.App.Core.Features.Pdf.PdfService.GetComponentKeyValuePairs(JToken component, Object data)
         at Altinn.App.Core.Features.Pdf.PdfService.GetOptionsDictionary(String formLayout, String language, Object data, String instanceId)
         at Altinn.App.Core.Features.Pdf.PdfService.GenerateAndStoreReceiptPDF(Instance instance, String taskId, DataElement dataElement, Type dataElementModelType)
         at Altinn.App.Core.Implementation.DefaultTaskEvents.OnEndProcessTask(String endEvent, Instance instance)

Steps To Reproduce

Add dynamic options for a repeating group
Try submitting app to start pdf generation
See error.

Additional Information
No response

I have a fix for this ready for v7, just need to discuss how we merge this into main. Should be ready for review and testing tomorrow.


Enable server-managed application caching of rdf data

Currently the cache concepts are akin to a fast key/value lookup that replaces the rdf storage. This key/value storage has the following limitations:

not transient
becomes stale
requires long data transform step (e.g. scripts to transform rdf to key/value storage)

The server should really manage an internal cache for efficient retrieval.  The types of data cached should be mostly-immutable data like schema, namespaces, annotations, and BEL specification concepts.
Initially this should be prototyped against namespace data.

Cache plugin added on top of storage. Commits 7316055e17c5f9d46fdc0ec150a3afed143ce908, 215ca5a7b8f37fef7939820d3d43e24408d0a13c, and 68f2dd74030c9c52d168edaf45f5555bfee7ec69.
Default implementation uses Kyoto Cabinet.


Proper way to handle sendMessage result

Hi,
Is there a proper way to handle sendMessage result ?
The use case is simply to add the (just now) sent message to the messages windows, but only if the message has really been sent (or perhaps at least received by the local server).
My first thought was to get and test the msg_id, like this:
var msg_id = client.sendMessage({
                   to: remote_jid,
                    type: 'chat',
                    body: body
                });
if (msg_id) {
   // add the message to the messages pannel here
}

Looking at the source, msg_id is always returned.
Using emmitted 'message:sent' event will not change anything: it's emitted even before the message is really sent:
https://github.com/legastero/stanza.io/blob/master/src/client.js#L368
I can stop verifying msg_id and always add the message in the message list, but at the cost of perhaps adding a message that is not really sent. (the other peer will think it's ok but it's not).
I'm using websocket transport, so i've looked at the transport source code: the message is not directly sent, but added to a send-queue, that's why it's certainly not trivial, but it should be possible to use the callbacks ? or better: emit the 'message:sent' event at the exact right place in the transport, just after the message is really sent ?
Or perhaps there is already a good practice that i've overlooked ?
Thanks,
Tom.

@ThomasChiroux That's a really good question, I have the same problem and hope someone can help or at least clarify why this behaviour is intended.

https://github.com/legastero/stanza/blob/master/docs/Using_Stream_Management.md


update upgrade script to use new CDN

The upgrade script has to deal with redirects due to pointing to former CDN, this reduces HTTP calls by 2
setting as skip-changelog, due to it being a contrib script

The download link on https://gitea.io also still points to https://dl.gitea.io/, should probably update as well.

The download link on https://gitea.io/ also still points to https://dl.gitea.io/, should probably update as well.

good catch, thanks :) I'll send a PR for that now too


Access controller scope from $callbacks method

Hi,
I have defined the callbacks in this way
 vm.treeOptions = {
        accept: accept,
        dropped: dropped,
        beforeDrop: beforeDrop
 };

and for each one of them I have defined a function further in the code.
I need to access one or more variables defined as ``vm.var1for instance from theaccept`, is there a simple way to do?
Thanks a lot for the attention.

Put these properties (variables as you like to call them) inside of your $scope/$rootScope? Then you've made them global to your document ($scope) or your entire app ($rootScope)

Looks like the solution is provided by @Purewebdesign


Refresh rate detection doesn't work when using multiple screens with different refresh rate

The logic here only reruns when the plugin settings or WinitWindows change, but I don't think that will happen just by dragging a window to another screen:
https://github.com/aevyrie/bevy_framepace/blob/4ae0c483d4e813e4f41af02d4a03ac3b5319246e/src/lib.rs#L193
This then causes bevy_framepace to always use the initial screen's refresh rate, even when the window is moved over to another screen with a different refresh rate later, effectively rendering the plugin ineffective on that screen.
If calling detect_frametime every frame is too much work, perhaps the library could be made to work without having an accurate refresh rate value, by measuring the time bevy spends waiting for VSync? In my own experiments, I've managed to measure this by timing the executing time of the prepare_windows system.

The reason for the check is that I found this was taking a significant amount of time, perhaps because a syscall or something is happening. I took another look at this and confirmed the function to get the best video mode was taking upwards of 6ms. Not sure if this was available when I first wrote this, but it now appears you can just use MonitorHandle::refresh_rate_millihertz(), which only takes a few microseconds. I'll switch this over on main.

Fixed by https://github.com/aevyrie/bevy_framepace/commit/b29420d50acf854a32a39cda9a49c53d92b0fc18


SD-1181 Hierarchical FileSystem

Adds a filesystem interpreter that, given a set of mounts, routes filesystem requests to underlying interpreters based on the paths in the requests.

I see no major concerns with this. I would be OK with merging this as is in the interest of moving on to the next steps.

Current coverage is 82.50%

Merging #1048 into master will increase coverage by +0.65% as of df1b482

@@            master   #1048   diff @@
======================================
  Files          139     143     +4
  Stmts         9109    9325   +216
  Branches       295     299     +4
  Methods          0       0
======================================
+ Hit           7456    7694   +238
  Partial          0       0
+ Missed        1653    1631    -22

Review entire Coverage Diff as of df1b482

Powered by Codecov. Updated on successful CI builds.

@mossprescott Thanks for the push to refactor chroot and mounted, much cleaner now.


Error in alerthandler test [development_fixes103]

Error in Test at  /sandbox/code/github/threefoldtech/jumpscaleX_core/JumpscaleCore/tools/alerthandler/RedisAlertHandler.py .. it need to be checked
Wed 29 09:49:44 RedisAlertHandler.py - 459 - test                               : EXCEPTION:
    AssertionError()
--TRACEBACK------------------
<stdin> in <module>
    1
/sandbox/lib/jumpscale/Jumpscale/tools/alerthandler/RedisAlertHandler.py in test
    459    assert self.count() == 2
-----------------------------

JSX>

verified on development_fixes103


Error adding to Teams (via SharePoint AC)

I get the error when uploading the app to the sharepoint catalogue as well as when try to select "add to Teams". API permissions are granted, I've tried deleting and reuploading the app and it still doesn't work. Account is an E licensed Global Admin Account.

Additional comment - I am seeing some customers facing this issue also

Hi @SammyRasha,
Thanks for brining this up. We are looking in to this. We will get back to you shortly.
Thanks.

I have the same issue . Let us know when you will have a solution.

Hi @jscaion ,
Can you please confirm if you are trying for a fresh install of CMP app for the first time.
Thanks.

Fresh install yes, cannot see in under manage apps in TAC

Hi @SammyRasha,
We have noticed this issue too. There seems to be a glitch here. We are working on it. We will let you know shortly. Thanks for your patience.
Thanks.

Hi @SammyRasha, @jscaion,
There seems to be a glitch with "Add to Teams". We are working on a hot fix but meanwhile if you wish you can try the below quick fix.
Quick fix: Instead of "Add to Teams" you can directly upload the attached zip file under "Manage Apps" in Teams Admin center. This zip is a Teams app package file with a slight change in the app version format in manifest file which was causing conflicts.  After the zip file is uploaded you can skip the "Add to Teams" step and rest of the steps are the same as in deployment guide.

Please me know if you see any issues.
TeamsSPFxApp.zip

Hi @SammyRasha, Thanks for confirming.
Hi @jscaion, Iam closing this ticket. If you still face any issues please open new ticket.


Add a POC of rsync

It works, however, it fails when the rsync --server command finished too early (bug reported in #82)


Remote Control Support

Add remote control support along with builtin remote controlling command.
Changes Introduced

addition of new package internal/remote
add remote coral command to root command
addition of environment var parsing at main called SLIDES_REMOTE_SOCKET. Presence of this triggers listening of remote control command at given UNIX socket path. This can then be used by sides remote socket [flags] [command] command to control that slide session.

Use Case

Navigating and Controlling Slide sessions using external processes eg: scripts, to use slides along with other workflows as well.

DEMO

DEMO


BUG: Quicknode Too many Requests Error on Curve TWap

POST https://serene-divine-crater.quiknode.pro/ 429 (Too Many Requests). Why are we making too many requests?
Switch Liquify

New ticket for @jamie088 opened with #1271


Strange warping of raster features

I am trying to render some weather data onto a map of Australia.
I get the data from a local weather service by downloading a jpeg of the current rain radar.
I then use a raster feature to render this forecast over Australia using the bounds of the original forecast query. When I do this however I get strange warping.
It seems as if the forecast is aligned at the top to bottom but the middle is warped somewhat
I was able to fix this by dividing the rendering up into many strips, as I said the tops and bottom of the square always render at the correct position/scale so I just made it so we never stray far from a top or bottom by cutting the image up into 10 strips and rendering those.
I am unsure why this is happening but I suspect it has something to do with strange map projection wizardry that I don't really understand.
Am I doing something wrong here?
Without sliced rendering

With sliced rendering:

Code sample, in f#, but should still be pretty obvious
let raster = new MRaster(bitmapStream, bounds|>LongLatToSherical)
let rasterFeature =new RasterFeature(raster)
let features:MemoryProvider<IFeature> =Providers.MemoryProvider([rasterFeature])
let weatherLayer =new MemoryLayer
    ("RasterLayer",
    DataSource=(features),
    CRS="EPSG:4326",
    Opacity=0.65)
let tiles=[|weatherLayer,osmLayer|]
let map =  new Mapsui.Map(
                    CRS = "EPSG:3857"

                            )
map.Layers.Add tiles

The openstreetmap tile layer is in the SphericalMercator projection "EPSG:3857". Is your raster source also in "EPSG:3857".?

The raster source is a very large section of an epsg:4326 map. As far as i could tell from your documentation this shouldn't effect the rendering only the coordinate system.
Is that not correct?

In Mapsui is it not possible to convert rasters to another projection. Only geometries.

What part of the documentation are you referring too? Perhaps it could be improved.


HB V11.73 recommends Nodejs V12.18.2+

Homebridge V11.73 recommends nodejs V12.18.1 to be install.
Homebridge GUI shows nodejs version as 12.18.1; however, on rpi it is showing v12.18.2. I am confuse as which version is actually installed. I have already submit an issue with Homebridge. I just want to verify with Homebridge-hue plugin as well.

You might need to restart the supervisor after upgrading node, or it will still show the old version as that’s what it’s running in.  Other common cause is multiple installations of node.


Fix JS/CSS asset paths for classic /config page

Fixes https://github.com/prometheus/prometheus/issues/9130
Signed-off-by: Julius Volz<EMAIL_ADDRESS>

Thanks :)


Adding hello world program in fortran

This pull request add the Hello World program in the FORTRAN language.

Thanks! Make sure to give the repository a star and share about it as well! You can tweet about the repository as well using the tweet badge on the Readme file or website!


Can't log in: 500 (Internal Server Error)

I am on a shared host. (Site5 cloud reseller)
Upon trying to log in for the first time I get the following error:
Failed to load resource: the server responded with a status of 500 (Internal Server Error)
Info from hosting support:
od_fcgid: stderr: PHP Fatal error: require_once(): Failed opening required '/home/(MYACCOUNT)/public_html/taskboard/api/../vendor/autoload.php' (include_path='.:/usr/local/php55/lib:/usr/lib/php:/usr/local/lib/php') in /home/(MYURL)/public_html/taskboard/api/api.php on line 3, referer: http://(MYURL)/taskboard/

It looks like the vendor dependencies were not installed. If you only used the instructions on the doc site, they are in need of updating. Take a look at #244 and see if that helps.

Excellent, thank you. Do you heppen to know how this can be accomplished on cpanel?

Unfortunately, I do not as I haven't used it in quite a while. It's possible you may need shell access to your server.
Alternately, you could do a local install, then upload the vendor directory to your shared host (I think this should work).

ok, next problem, I'm getting this:
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/php55/lib/php/extensions/no-debug-non-zts-20121212/phar.co' - /usr/local/php55/lib/php/extensions/no-debug-non-zts-20121212/phar.co: cannot open shared object file: No such file or directory in Unknown on line 0
Warning: This development build of composer is over 60 days old. It is recommended to update it by running "./public_html/taskboard/build/composer.phar self-update" to get the latest version.
Composer could not find a composer.json file in /home/(myAccount)
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section

You should know, I have no idea how to work the command line. Never had to do this before.

Here's the vendor directory temporarily: https://www.dropbox.com/s/tkdftobzlze02uw/vendor.tar.gz?dl=0
Just unpack it and place it in the root TaskBoard folder so you have your_taskboard/vendor

I may go back to having a ready-to-go setup rather than requiring an install of the dependencies.

Was able to get it working thanks to that archive. Thanks so much for your help, the app is great.

Glad you like it, and glad that worked for you.


Participant file, Subset Gen Code, Majority Element Code, LIS code added

In this Pull Request we have done the following additions/edits for the code:

Participant file
Subset Generator Code
Majority Element Code
Longest Increasing Subsequence Code

Are there any updates or should we close the PR?


TEM2015 Test 02: Coastal Relief Model still points to development server

The Coastal Relief Model does not load as it still points to
/ags/rest/services/NationalIntelligence/CoastalReliefModel/ImageServer?f=json&callback=dojo.io.script.jsonp_dojoIoScript1._jsonpCallback

@NatalieCampos Please provide more information. When you say "Coastal Relief Model" are you taking about a service, a web map? What's the id of the portal item? When you say "does not load" what are you referring to? If it's a service are you trying to display the service in the portal viewer?
Tell us how to replicate the issue. This doesn't really tell us anything. Thanks.

id=8df0127db0854fdf9a96d4c9c4b6bbd4
by does not load, I mean that it does not load in the web map

to reproduce click the portal item and add to a new web map

This issue depends on #1323 being resolved.

Assigning to @NatalieCampos to retest after the other issue is resolved.

Verified


Fixup splash screen

Codecov Report

Merging #324 (413515b) into main (9f4171b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #324   +/-   ##
=======================================
  Coverage   75.01%   75.01%
=======================================
  Files          78       78
  Lines        3955     3955
  Branches      677      677
=======================================
  Hits         2967     2967
  Misses        587      587
  Partials      401      401

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


⚠ Punto de Fuga has degraded performance

In 8790b70, Punto de Fuga (http://puntodefugapodcast.com) experienced degraded performance:

HTTP code: 200
Response time: 8737 ms

Resolved: Punto de Fuga performance has improved in 175110d.


Fixed data path on ApiPage

Sorry, the file path were incorrect on ApiPage.

aha all good. i was too lazy to check. my bad :(


Restored bootstrap. Updated app ver to 1.2, updated command help desc of a few commands.

Bootstrap JS lib and CSS have been restored.
MainApp version changed to 1.2.
updated command help description of a few commands.
updated user guide for 'course delete' feature.
addressed comments by me from previous PR.

Coverage increased (+0.04%) to 66.083% when pulling ea5149bdf6794d8f92d7bad7f15f960386b712b9 on m-aslam-mj2:staging3 into 550f328c2d01b8e306e49fdd6fdeb5a3c20464ee on CS2113-AY1819S1-T16-1:master.


Fixing the description of keras.losses.CosineSimilarity in losses.py

Modified the description of keras.losses.CosineSimilarity from -1 to 1 as values closer to 1 indicate greater similarity

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 60.31%. Comparing base (e53e61d) to head (47601c5).

:exclamation:  There is a different number of reports uploaded between BASE (e53e61d) and HEAD (47601c5). Click for more details.
HEAD has 6 uploads less than BASE

Flag
BASE (e53e61d)
HEAD (47601c5)

keras
4
1

keras-torch
1
0

keras-tensorflow
1
0

keras-jax
1
0

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #20597       +/-   ##
===========================================
- Coverage   82.39%   60.31%   -22.08%
===========================================
  Files         516      516
  Lines       48436    48436
  Branches     7564     7564
===========================================
- Hits        39908    29215    -10693
- Misses       6705    17451    +10746
+ Partials     1823     1770       -53

Flag
Coverage Δ

keras
60.31% <ø> (-21.93%)
:arrow_down:

keras-jax
?

keras-numpy
60.31% <ø> (ø)

keras-tensorflow
?

keras-torch
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Suggest Modify Attribute Processor

UPDATED - This is a proposal configuration to merge the add attribute processor and the attribute key processor and add a missing functionality of updating existing keys.
The motivation is to have only one processor that modifies an attribute originated from this discussion.
Current Set of Functionalities
The set of functionalities covered by these processors are:

Adding attributes (the key does not exist) (do nothing when the key does exist)

Use a value from a config (add attribute processor)
Use a value from another attribute (attribute key processor)

Upsert attributes (the key does or does not exist)

Use a value from a config (add attribute processor)
Use a value from another attribute (attribute key processor)

Delete an attribute

The attribute value was copied to a new key and the original attribute isn’t needed anymore. (attribute key processor)

Missing Functionality
What is missing from the set of functionalities is a way to update keys that exist and do nothing when the key does exist.
Proposal
This processor takes a list of actions (insert,update, upsert, delete) and performs the actions in order specified.

Adding attributes (the key does not exist) (do nothing when the key does exist)

Use a value from a config
Use a value from another attribute

Update attributes (the key does  exist) (do nothing when the key does not exist)

Use a value from a config
Use a value from another attribute

Upsert attributes (the key does or does not exist)

Use a value from a config
Use a value from another attribute

Delete an attribute

This allows for complex scenarios like backfilling spans with missing attributes or moving attributes to new keys.
Refer to config.go for the config specs and details.
Example
Example from the processors/attribute/testdata/config.yaml of a rich example
    # The following demonstrates how to backfill spans missing an attribute,
    # insert/update that value to a new key and deleting the old key. This guarantees
    # an attribute `svc.operation` exists in spans and the attribute `operation`
    # doesn't exist.
    attributes/complex:
      actions:
        - key: operation
          value: default
       - key: svc.operation
          from_attribute: operation
          action: upsert
       - key: operation
          action: delete

Follow Up PR's
Once the config is approved, this pr will be completed to add tests for the configuration.
The PR already contains documentation in terms of detailed config.yaml and README.md entry.
A follow up PR will contain the logic of applying the actions in the order specified.
Another PR will remove the deprecated attribute key processor and add attribute processor.
How to extend this processor
In the future, this layout is easy to extend for concepts of defining spans properties to match against  and determine if the attributes processor should be applied or not.
Span Rename
As for span renaming processor, I'm going to reopen the pr and update the configuration. The idea is to have one processor that modifies attribute of a Span such as this proposal and have another processor to modify top level settings of a Span such as the span rename processor.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@7eba056). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #252   +/-   ##
=========================================
  Coverage          ?   73.31%
=========================================
  Files             ?      111
  Lines             ?     6456
  Branches          ?        0
=========================================
  Hits              ?     4733
  Misses            ?     1481
  Partials          ?      242

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7eba056...5b2abe0. Read the comment docs.

@bogdandrutu @tigrannajaryan @flands  this is my updated proposal to merge all the various attribute processors into one. Please look at the sample config.yaml it demonstrates the old configuration compared to the new proposal.

It is strongly suggested to configurers to ensure that they don't set/update/copy the same key/value in one setting of the processor.

@ccaraman can you expand on this? Would it result on undefined behavior?

@bogdandrutu @songy23 Please see the new proposal. The description is updated and there are examples in config.yaml and a readme.md to explain the behavior.
Note: the span rename feature isn't in this processor. It is neater to have one processor for the span attributes and another for top level settings of a span. Re: performance concerns. Currently the objects aren't immutable so no new objects are created when modifying properties of span (ex. attributes, name). And the final conversion to proto happens in the exporter.

@bogdandrutu please have a look. I like this approach.

Nice and clear PR description, good job.


V4 dep updates and backports

Preview: https://deploy-preview-30414--twbs-bootstrap.netlify.com/

@ffoodd @MartijnCuppens @patrickhlauke the #30073 backport is nice for v4-dev, but since the navbar occupies the whole width, the skip links overlap with it. Just pointing it out. I personally don't really like this :/

I agree, it's more awkward in v4-dev.
I see two workarounds (that could match to master as well):

use a colored backplate and make it a sort of sticky-top element (just like Github's one!)
keep the previous pushing design, just left-aligning skip-links (being at the top left is the main goal of this PR) — may consider to make both skip-links visible at once on docs, in that case. This could look weird in v5 since the whole layout is centered.

Depending on what we consider to do, I'll open a PR against master or v4-dev.
Any thoughts?

@mdo @MartijnCuppens thoughts how to proceed with the skip links in this branch?

I removed the docs skip links patch for now and pushed it to https://github.com/twbs/bootstrap/tree/v4-dev-docs-skip-links
@MartijnCuppens mentioned some issues with master too so we better discuss this more and/or revert the patch from master in the meantime.


Ignore pubspec.lock

See https://github.com/dart-lang/wasm/runs/2811566439?check_suite_focus=true#step:8:22

FYI @liamappelbe


Do I need any Swift code if I want to rely on the iOS default share sheet?

Hey,
first of all thank you for sharing this, it's exactly what I was looking for for my project.
I have a very simple share use case: allow to share URLs to my app. Do I need to implement any Swift code even if I just want to rely on iOS provided share sheet?
Any pointer would be appreciated.
Thanks again for your work!

sed life!

Hello, sorry for the very late reply ! I did not received the notification.
You need to write the share sheet in swift yes, you can find a simple example in my github (the link is https://github.com/BreckoEC/share-extension-expo-example)


Wasmer discovery and prototype pt2

@mpapierski please add the feedback from DMs to this issue and close this one.

Current state:

Need to bump wasmi to 0.13.2 (aka "battle tested") as it gives most performance benefit with little work
This requires either bringing pwasm-utils in source, or using a fork.

@RitaMAllenCA the latter is already ticketed: #3496
The former I haven't seen on the board yet - we should indeed create one and then close this.

done #3766


Objectives -> Objectives in AssessmentPlan

Describe the bug
Getting the following error in when validating an AssessmentPlan:
 field required (type=value_error.missing)
objectives -> objectives -> id
  field required (type=value_error.missing)
objectives -> objectives -> control-id
  field required (type=value_error.missing)
objectives -> objectives -> part
  field required (type=value_error.missing)
objectives -> objectives -> objectives
  extra fields not permitted (type=value_error.extra)

trestle has Objectives class with the following line:
objectives: Optional[Union[Objective, Objective]] = None
OSCAL defines it as:
objectives: [0 to ∞]
I believe trestle is expecting an object instead of a list... not 100% familiar with the syntax but perhaps it could be something like:
objectives: Optional[List[Objective]] = None
To Reproduce
Steps to reproduce the behavior:

Create a sample Objectives dictionary
Load it into trestle.oscal.assessment_plan.Objectives.parse_obj(dict)
See error

Expected behavior
Should load without an error.
Additional context
Using v0.2.1

I believe based on my reading of the schema this should be
objectives: Optional[Union[Objective, List[Objective]]] = None

OSCAL AP Schema defines it as:
An array labeled objectives, containing objective objects, or a single objective object [0 to ∞]
A local definition of a control objective. Uses catalog syntax for control objective and assessment actions.
The wording and the [0 to ∞] don't align to me... wouldn't the array be [2 to ∞]?

So it's saying you could have this (in pseudo-objects)
objectives: {objective_object}
or
objectives: [{objective_object}, {objective_object}]
but not
objectives: [{objective_object}]
This approach was prevalent earlier versions of OSCAL across many objects. I'm wondering whether we should flag as the tried to simplify this down so if an array existed it existed.

I believe it is creating Union[Objective, Objective] instead of Union[Objectives, Objective]
So this is related to recent change and is fixable.

Actually the raw file output by modelgen has Union[Objective, Objective] - so that isn't the problem. I will look into it.
Union[Objective, Objective] is odd and that is before fix_any does anything...

Here is a grep in the oscal dir before running fix_any that have this Union[A, A] thing.  Should they be Union[A, List[A]] ?
assessment_plan.py:    objectives: Optional[Union[Objective, Objective]] = None
assessment_results.py:    objectives: Optional[Union[Objective, Objective]] = None
component.py:    annotations: Optional[Union[Annotation, Annotation]] = None
ssp.py:    annotations: Optional[Union[Annotation, Annotation]] = None
target.py:    annotations: Optional[Union[Annotation, Annotation]] = None

I have added a global conversion of all Union[A, A] patterns to Union[A, List[A]] and can PR if that is the correct thing to do.

@fsuits that looks right to me. I'm going to add another PR as there is some inconsistent behaviour still in OSCAL.

If there is one Objective object, and it is placed in a list, would that not get incorrectly validated with that logic?

Yes it would allow A, [A], [A, A]
Is there a way I can disallow [A] using Type notation?

conlist? https://pydantic-docs.helpmanual.io/usage/types/#constrained-types

conlist looks good to me.  ssp.py already imports conint and constr.  I will look into it if the restriction is meant to be enforced:
A or [A,A,...] but not [A} - correct?
That would be Union[A, conlist(A, min_items=2)]  I guess?

For a long term solution we should track here: https://github.com/usnistgov/OSCAL/issues/770


workbox-broadcast-cache-update demo is not working

Browser & Platform:
*E.g. Google Chrome Version 70.0.3538.77 (Official Build) (64-bit)
Issue or Feature Request Description:
workbox-broadcast-cache-update Demo is not working
[Link]:
https://workbox-demos.firebaseapp.com/demo/workbox-broadcast-cache-update/
at responsesAreSame (https://storage.googleapis.com/workbox-cdn/releases/4.0.0-alpha.0/workbox-broadcast-cache-update.dev.js:32:15)
    at BroadcastCacheUpdate.notifyIfUpdated (https://storage.googleapis.com/workbox-cdn/releases/4.0.0-alpha.0/workbox-broadcast-cache-update.dev.js:248:12)
    at self.addEventListener (https://workbox-demos.firebaseapp.com/demo/workbox-broadcast-cache-update/sw.js:25:23)**```

The demos have since been rewritten.


Update flyway-core to 7.5.1

Updates org.flywaydb:flyway-core from 7.3.2 to 7.5.1.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.flywaydb", artifactId = "flyway-core" } ]

labels: library-update, semver-minor

Superseded by #434.

Superseded by #434.


feat(dotnet): improve metadata of NuGet packages

Convert the package.json keywords into PackageTags in the NuGet
metadata. This is applied on the packages hosted in this repository as
well as generated code.
The keywords section from package.json is now forwarded into the
.jsii assembly to support this feature.
At this point, could not find a way to carry these keywords/tags to
Maven Central or PyPI, so only NuGet actually leverages those.
Fixes #170

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


fix: small fixes for Transaction

Motivation

Make Transaction::blob_versioned_hashes an Option so that we can determine if it was not present or just empty. This field has similar semantics to access_list
EIP-1559 txs include deprecated gasPrice field:

We have to account for it when converting to TransactionRequest to not end up with invalid request having both gasPrice and maxFeePerGas.

please link source of screenshot

@prestwich https://ethereum.github.io/execution-apis/api-documentation/ eth_getTransactionByHash section
https://github.com/ethereum/execution-apis/blob/main/src/schemas/transaction.yaml#L124


Adapt to Netty 5.0.0.Alpha5-SNAPSHOT changes

Motivation:
The build of the project is broken. This change is adapting the code to the changes in Netty 5.0.0.Alpha5-SNAPSHOT.
Modifications:

Adapt to the new way of setting channel options
io.netty5.channel.Channel#localAddress now returns SocketAddress
Adapt to the changes in the API for obtaining Decoder results

Result:
The build of the project is green again

@chrisvest PTAL

Thanks!

@chrisvest thanks


Assert k8s errors using v1.24 format in E2E tests

Fixes triggermesh/triggermesh#880

Merging because trivial, and currently resulting in failed tests on main.


YAML to define request and response contract for body transformation

What should we add or change to make your life better?
Could we please have a way to load a YAML (OpenAPI 3) file to declaratively say, for a route, what are the expected datatypes and structure for request and response contracts?
Then, after loading this YAML, we could have (a) middleware classes for the proper YAML validation (comparing payload format with predefined YAML structure) and (b) middleware class for request and response body transformation. Regarding request and response body transformations, they could be done automatically (based on YAML definitions) or manually by adding the needed classes in middleware for transforming the content.
I have seen an issue opened here but it seems more related to URL transformations.
Why is this important to you?
Have a unique package that gathers all necessary classes for adding the YAML contracts for request and response per route, configure classes for validation, and transformation of request and response body.

YAML aside, can you give specific examples about the kinds of body transformations you had in mind?
We don't currently have plans for any request or response body transformations. Such transformations are extremely expensive and scenario specific.
That said, this is the kind of thing that can be plugged into the pipeline as middleware today, I don't think it would require any architectural changes to support.

Triage: Processing & transforming the request/response bodies is not something we are going to build into YARP. That is a feature that is better handled by Ocelot.


Implemented Auth Session functions

Nice.  I'll look into porting that into Godot 2 branch here in a bit.


Reformats alert message to remove localized timestamp and updates tests

Removes localized timestamp.

Current coverage is 68.58%

Merging #421 into master will not affect coverage as of 15a2e8b

@@            master    #421   diff @@
======================================
  Files           63      63
  Stmts         2779    2779
  Branches         0       0
  Methods          0       0
======================================
  Hit           1906    1906
  Partial          0       0
  Missed         873     873

Review entire Coverage Diff as of 15a2e8b

Uncovered Suggestions

+0.61% via goldstone/views.py#30...46
+0.58% via ...stone/drfes/views.py#134...149
+0.40% via ...tone/drfes/models.py#165...175
See 7 more...

Powered by Codecov. Updated on successful CI builds.

:+1:


Use text-colour for headings in footer

This used to be a custom grey variable set to #0b0c0c but was converted to footer-text, which made the headings too grey (see https://github.com/alphagov/govuk_template/pull/201)
Switch back to the darker colour to improve contrast, use $text-colour instead.
cc @gemmaleigh
Before

After

Looks good to me 👍


feat(BooleanControl.js): display messages along toggle

This changes are about:

refactor syntax BooleanControl static properties
add prefix & suffix options
update widget control style to match with new options
update Boolean widget document

Summary
The update allows developers to display messages before (prefix) and after (suffix) the toggle. This design intends to provide a better information and look to users.
In my use case, I have a field to decide if I want to show a post in homepage. If I set "Show in homepage" string in label, the layout of input would be unbalance as the top small part is bigger than the main body. The toggle itself seems lonely as well. So if I can set "Visibility" in label option and "Show in homepage" in prefix option, it would be much sense.
Test plan

Run yarn start
Go to this path collections/kitchenSink/entries/a-big-entry-with-all-the-things
Expect to see the exact Boolean input like this screenshot:

Checklist

[x] I have read the contribution guidelines.

A picture of a cute animal (not mandatory but encouraged)
Be hold! The amazing meowman!

@ngdangtu-vn I see why you would want this feature, but it seems very similar to hint, which exists on all widgets and can be used to explain a field in more detail. Is there a reason why you wouldn't use hint?

Is there a reason why you wouldn't use hint?

I wouldn't say I don't use hint. But my reason is simply:

Not wasting a whole line for only 1 toggle element
Hint text is small enough to be a hint only. Prefix/Suffix will plays nice the the role of important instruction.
In some messages, it looks better with prefix (or|and) suffix as well. I admit, my example was very good. Maybe like this: Show author [x].

I even plan to do take this further to string and number widgets so dev can display fields like this [...]px. Cool right?

Extending this to string and number will be useful! Can you implement that as part of this PR and I will review/merge it all together?

Update photo for recent commits

@martinjagodic Could you review my code and the Widgets' UI? Thanks.

@xaiki
Thanks for your review but I'm not sure if they still need this PR. It kinda a dead PR already.

@xaiki thanks for your reviews
@ngdangtu-vn This PR is not dead, but I understand it sitting here for a long time now. We have quite a backlog and we want to add features based on the triage we decided upon. This one is not essential so it will probably wait a bit longer.

@martinjagodic Ok it's cool, I thought the code won't be able to fit with the new Decap.
@xaiki So I will fix the PR using your advice later 'cause I'm bit busy recently & I need time to reload my memory :))

@martinjagodic if reviews help I'm happy to look at other PRs, just point me to the top of the stack.

@xaiki I will happily assign you review requests. To talk more in-depth about what to do next, please join the contributing channel on our Discord server.

@ngdangtu-vn are you still interested in moving this forward? If yes, please address the comments and solve merge conflicts. Thanks!

Ok, give me around 2 more days

Hold on, I'm on cleanning up repo.

I didn't close this? Hold on I'm reorganizing this... Maybe because I reset branch?

@ngdangtu-vn maybe. I guess you can just reopen and continue from there.

I think I just shot myself to the leg. But it's ok, everything's still under controlled...


Apply patch to PHP 5.6 for running with OpenSSL 0.11

The API of OpenSSL has changed from 0.10 to 0.11. This makes it difficult to run PHP 5 on an updated system. The patch is from https://github.com/php/php-src/pull/2667.
Note that the patch only works for PHP 5.6.31 or higher. While not ideal, being able to install the last (released) PHP5 version will be sufficient for most.

Coverage decreased (-0.04%) to 41.141% when pulling cfb1ae4c6ecf79edc0b025e7ecba5edde64b6abd on jasny:5.6-openssl-0.11 into bceaee1faf158cf2b073be715877e8ba0d71797d on phpbrew:master.

@jasny it didn't work for me:
$ git log -1 --stat
commit cfb1ae4c6ecf79edc0b025e7ecba5edde64b6abd (HEAD -> pull/1041)
Author: Arnold Daniels<EMAIL_ADDRESS>Date:   Fri Sep 20 09:46:13 2019 +0200

    Apply patch to PHP 5.6 for running with OpenSSL 0.11
    The API of OpenSSL has changed from 0.10 to 0.11. This makes it difficult
     to run PHP 5 on an updated system.
    The patch is from https://github.com/php/php-src/pull/2667.

 src/PhpBrew/Patches/PHP56WithOpenSSL11Patch.php | 35 +++++++++++++++++++++++++++++++++++
 src/PhpBrew/Tasks/AfterConfigureTask.php        |  2 ++
 2 files changed, 37 insertions(+)

$ apt policy libssl-dev
libssl-dev:
  Installed: 1.1.1b-1ubuntu2.4
  Candidate: 1.1.1b-1ubuntu2.4
  Version table:
 *** 1.1.1b-1ubuntu2.4 500
        500 http://us.archive.ubuntu.com/ubuntu disco-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.1.1b-1ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu disco/main amd64 Packages

$ phpbrew --debug install 5.6.39 +neutral +openssl
===> phpbrew will now build 5.6.39
---> Parsing variants from command arguments '+neutral +openssl'
===> Loading and resolving variants...
Checking distribution checksum...
Checksum matched: b3db2345f50c010b01fe041b4e0f66c5aa28eb325135136f153e18da01583ad5
===> Distribution file was successfully extracted, skipping...
Source Directory: /home/morozov/.phpbrew/build/php-5.6.39
Writing variant info to /home/morozov/.phpbrew/php/php-5.6.39/phpbrew.variants
===> Checking patches...
Checking patch for replace apache php module name with custom version name
Enabled variants: [openssl, xml, opcache]
Disabled variants: []
Found existing build.log, renaming it to /home/morozov/.phpbrew/build/php-5.6.39/build.log.1570311882
===> Configuring 5.6.39...

Use tail command to see what's going on:
   $ tail -F /home/morozov/.phpbrew/build/php-5.6.39/build.log

./configure '--cache-file=/home/morozov/.phpbrew/cache/config.cache' '--prefix=/home/morozov/.phpbrew/php/php-5.6.39' '--with-config-file-path=/home/morozov/.phpbrew/php/php-5.6.39/etc' '--with-config-file-scan-dir=/home/morozov/.phpbrew/php/php-5.6.39/var/db' '--with-openssl=/usr' '--enable-dom' '--enable-libxml' '--with-libxml-dir=/usr' '--enable-simplexml' '--enable-xml' '--enable-xmlreader' '--enable-xmlwriter' '--with-xsl' '--enable-opcache' >> /home/morozov/.phpbrew/build/php-5.6.39/build.log 2>&1

Uncaught PhpBrew\Exception\SystemCommandException: Configure failed:  in /home/morozov/Projects/phpbrew/src/PhpBrew/Tasks/ConfigureTask.php on line 107

$ tail /home/morozov/.phpbrew/build/php-5.6.39/build.log
checking for xml2-config path... /usr/bin/xml2-config
checking whether libxml build works... yes
checking for OpenSSL support... yes
checking for Kerberos support... no
checking whether to use system default cipher list instead of hardcoded value... no
checking for DSA_get_default_method in -lssl... no
checking for X509_free in -lcrypto... yes
checking for RAND_egd... no
checking for pkg-config... /usr/bin/pkg-config
configure: error: Cannot find OpenSSL's libraries

Are you sure it should be an after-configure task? I'm trying to apply a similar patch in #1044, but it's part of the pre-configure task and requires ./buildconf --force afterward.

@morozov It's tested it, but "works on my machine" :slightly_frowning_face:
What do you see when you run the following?
/usr/bin/pkg-config openssl --modversion

Autoconf should be using pkg-config and not even try to find the lib manually.

@jasny please look at the log I posted above. The build fails before your patch gets applied. How can it work?
Here's the output:

/usr/bin/pkg-config openssl --modversion
1.1.1b

@jasny since #1044 is merged, could you go ahead and use the same approach? You need to move your patch to the pre-configure task and maybe execute the BuildConfTask upon a successful patch.

@morozov I can, but I don't think it will make any difference. The patch doesn't touch the autooconf files and only modifies the c files of the openssl PHP ext.
I can't reproduce the issue you're having by any means. Can you please do some debugging.
In aclocal.m4 (of php-src) you can find
AC_DEFUN([PHP_SETUP_OPENSSL],[

In that function you see the section where pkg-config is used to find the openssl libs. It's commented with
dnl If pkg-config is found try using it
if test "$PHP_OPENSSL_DIR" = "yes" && test -x "$PKG_CONFIG" && $PKG_CONFIG --exists openssl; then
  if $PKG_CONFIG --atleast-version=0.9.6 openssl; then
    found_openssl=yes
    OPENSSL_LIBS=`$PKG_CONFIG --libs openssl`
    OPENSSL_INCS=`$PKG_CONFIG --cflags-only-I openssl`
    OPENSSL_INCDIR=`$PKG_CONFIG --variable=includedir openssl`
  else
    AC_MSG_ERROR([OpenSSL version 0.9.6 or greater required.])
  fi

  if test -n "$OPENSSL_LIBS"; then
    PHP_EVAL_LIBLINE($OPENSSL_LIBS, $1)
  fi
  if test -n "$OPENSSL_INCS"; then
    PHP_EVAL_INCLINE($OPENSSL_INCS)
  fi
fi

This should just work. The part where alternative methods are tried should not be executed.
...
dnl If pkg-config fails for some reason, revert to the old method

Please check why pkg-config isn't able to find openssl when running ./configure.

@jasny thank you for the clarification. Looks like we're experiencing different issues, and I was under the impression the one you're trying to fix is also related to the configuration, not the code.
I'll follow the steps you suggested above and will provide you the additional information as soon as I can.

@jasny, the problem was that when I tried using your build, --with-libdir=lib/x86_64-linux-gnu wasn't appended to the ./configure command. This was recently fixed by #1036. After a rebase, I was able to build PHP:
$ php -v
PHP 5.6.39 (cli) (built: Nov  4 2019 12:36:39)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies

$ php -i | grep OpenSSL
OpenSSL support => enabled
OpenSSL Library Version => OpenSSL 1.1.1c  28 May 2019
OpenSSL Header Version => OpenSSL 1.1.1c  28 May 2019
OpenSSL support => enabled

does not work on mac os with brew installed openssl 1.1

Weird, this PR seems not working, I have to apply the openssl 1.1 patch manually. Here comes the patch output message:
===> Checking patches...
Checking patch for php5.3.29 multi-sapi patch.
Checking patch for php5.3.x on 64bit machine when intl is enabled.
Checking patch for openssl dso linking patch
6 changes patched.
Checking patch for php5.6 with openssl 1.1.x patch.

@danywheeler The patch is not applied when using phpbrew ext install, only when installing PHP 5.6 with the openssl variant.
phpbrew install 5.6.40 +default +openssl --with-openssl="$(brew --prefix openssl)"

Typically +openssl isn't required as it's included in +default, but it may be required if you use some other variant like +minimal.

@markwu Make sure you have the openssl variant (+openssl) and are installing PHP 5.6.31 or later. The match is pretty straight forward.
public function match(Buildable $build, Logger $logger)
{
    return version_compare($build->getVersion(), '5.6') === 0
        && version_compare($build->getVersion(), '5.6.31') >= 0 // patch only works for 5.6.31 and up
        && $build->isEnabledVariant('openssl');
}

Ah ... Okay, I suppose it apply to default. I'll make a test and let you know.

I tried again, the rules triggered, but patches nothing.
I have to download the patch and apply it by myself.
phpbrew command
phpbrew --debug install --jobs "$(nproc)" 5.6.40 +default +openssl +dbs +dba +apxs2 +bz2="$(brew --prefix bzip2)" +zlib="$(brew --prefix zlib)" +curl="$(brew --prefix curl)"

manually patch command
wget https://patch-diff.githubusercontent.com/raw/php/php-src/pull/2667.patch
patch -p 1 < 2667.patch

debug log
===> Checking patches...
Checking patch for replace apache php module name with custom version name
0 changes patched.

Checking patch for php5.3.29 multi-sapi patch.
Checking patch for php5.3.x on 64bit machine when intl is enabled.
Checking patch for openssl dso linking patch
6 changes patched.
Checking patch for php5.6 with openssl 1.1.x patch.
===> Building...
make '-j8' >> /Users/mark/.phpbrew/build/php-5.6.40/build.log 2>&1
Error: Make failed:
The last 5 lines in the log file:
                                                                                ^

/Users/mark/.phpbrew/build/php-5.6.40/ext/openssl/xp_ssl.c:1840:36: warning: address of 'sslsock->s.timeout' will always evaluate to 'true' [-Wpointer-bool-conversion]

                if (began_blocked && &sslsock->s.timeout) {

                                  ~~  ~~~~~~~~~~~^~~~~~~

12 warnings generated.

Please checkout the build log file for more details:
	 tail /Users/mark/.phpbrew/build/php-5.6.40/build.log

I have problem with phpbrew 1.24.1 to build php-5.6.40, that's why I revert to phpbrew 1.23.1 and apply this PR only, to  see if I can figure out the problem.

Can I get more detail log when do the patch job?

@jasny Even with the command you typed, it's still not working, don't know what's the problem, but be sure related to openssl. same error as before.

Okay, I think I found the problem. the match function is not working. I change it to following, evething works!!!
    public function match(Buildable $build, Logger $logger)
    {
        return version_compare($build->getVersion(), '6.0.0') === -1
            && version_compare($build->getVersion(), '5.6.31') >= 0 // patch only works for 5.6.31 and up
            && $build->isEnabledVariant('openssl');
    }

@morozov Should I create a new issue for this?

@markwu please do. Although, I don't think we should express the requirement for PHP 5.6 using 6.0.0 as a boundary.

If so, the only way I can think about is using explode() to find PHP_MAJOR_VERSION.PHP_MINOR_VERSION only, then use the version string to compare.

@markwu indeed. Please go ahead and use your original approach. Given how non-obvious this code is, you can try adding a test that covers the match() method of the patch.


Pass GithubToken as option when building probot serverless app

https://github.com/probot/serverless-lambda/blob/a4688897217d206f0397d81dc4b457123fc302b9/index.js#L8-L13
Problem:
Right now is not possible to create a serverless probot app with a githubToken as option so we obtain an authenticated Github App installation.
Context:
Trying to create a serverless probot authenticated with Github App Installation but not possible on creation with the constructor parameters offered right now.
A workaround is a wrapper appFn used as callback for the .on() where we do the authentication:
	withWebHookContext(app: Application, next: Function): (ctx: Context) => Promise<void> {
		return async(context: Context): Promise<void> => {
			let webhookContext = context as WebhookContext;

                          const github = await app.auth();

		        const { data } = await github.apps.createInstallationToken({
			     installation_id: context.payload.installation.id
		        });

		       webhookContext.gitHubToken = data.token;

		       await next(webhookContext);
		};
	}

app.on(PUSH, decorator.withWebHookContext(app,
		mergeHandler.handle.bind(mergeHandler)));

Solution proposal:
To support process.env.GITHUB_TOKEN environment variable and call createProbot(Options) with githubToken in the Options

If there is an agreement on the issue and proposal solution I will be happy to draft a PR on it.

Could you elaborate in which case you would want a serverless lambda function to run with a static token? Is this for async event handling? The problem is that tokens can expire. Would it make more sense to use a Redis cache for the tokens that can be access across all your functions?

Could you elaborate in which case you would want a serverless lambda function to run with a static token? Is this for async event handling? The problem is that tokens can expire. Would it make more sense to use a Redis cache for the tokens that can be access across all your functions?

As far as I understand the token generated is not static right? On each webhook triggered, withWebHookContext function will be executed so, a new invocation to createInstallationToken() will be executed.
We were getting authentication issues when running the app without this createInstallationToken workaround.
⚠ Important to mention: we are using a very old version of Probot right now (I'm pushing the team to get room to upgrade to the latest):

probot: 7.5.3
@probot/serverless-lambda: 0.5.0

Let's do something, if you agree on that. Once we migrate to latest version of probot and @probot/serverless-lambda I keep on my plate this authentication issue and I will re-check if still is an issue. Maybe the installation token refresh was something introduced in latest versions of the library.
Sounds good to you @gr2m ?

Sounds good, yes! The way tokens are generated changed with the adoption of @octokit/auth-app, which we I think we adopted as part of Probot v9. Before, an installation token was created for each webhook event that was received. Now the token is created when the first request is sent that requires it. And if it expires because the process is running too long, it will be re-created automatically.

This will be possible via https://github.com/probot/adapter-aws-lambda-serverless/pull/59
const { createLambdaFunction, createProbot } = require("@probot/adapter-aws-lambda-serverless");
const appFn = require("./");

module.exports.webhooks = createLambdaFunction(appFn, {
  probot: createProbot({
    defaults: {
      githubToken: process.env.GITHUB_TOKEN
    }
  })
});


PHP 8.0.x compatibility?

This plugin says it requires PHP 8.1 or higher. My server provider only offers up to PHP 8.0.x. Is there a dependency that requires this new version or is it perhaps (fingers crossed) a typo somewhere and this can actually work with PHP 8.0?

Sorry - I seem to have posted this in the wrong repo!


qga: Merge child output streams when available

I added support for merged output streams in qga in https://github.com/qemu/qemu/commit/810f677ab86e649d449233b435197552024b016d. It went in with v8.1 but somehow the docs are slightly off (says 8.0).
Either way, this gives us support for nicer output.
Before:
./target/debug/vmtest -k ~/scratch/bzImage-v6.2 -- "echo one; >&2 echo two; echo three"
=> bzImage-v6.2
===> Booting
===> Setting up VM
===> Running command
one
three
two

After:
./target/debug/vmtest -k ~/scratch/bzImage-v6.2 -- "echo one; >&2 echo two; echo three"
=> bzImage-v6.2
===> Booting
===> Setting up VM
===> Running command
one
two
three

cc @DolceTriade

Blocked on new qapi-rs release


Managing dependencies

I work on a team that has standardized on using VSCode, with varying levels of comfort with the command line. We happen to use pip.
When creating an environment, the Python extension helpfully offers to install dependencies, which is great. Managing dependencies beyond that initial setup is left to the users. My attempt as a user story:
As a VSCode user working on a Python project with file for managing dependencies,
I want VSCode to help ensure my installed dependencies match what I'm supposed to have
so that I'm working with the intended packages/versions.
The dependencies file may change as a result of:

Being modified through the editor
Pulling down updates from version control
Switching branches

One way to go about it: when the dependencies file changes, ask the user whether they'd like their dependencies to be updated. Perhaps there could also be a setting to do this automatically.
I looked through dozens of past issues and didn't see anything that quite matched — apologies if I missed. Thanks!

I want VSCode to help ensure my installed dependencies match what I'm supposed to have
so that I'm working with the intended packages/versions.

@afeld Can you try this? There is a setting python.missingPackage.severity set it to error. And then when you open requirements.txt or pyproject.toml it should tell you if there are missing packages. We could expand that check to report version issues.
Here is what you should see:

Sorry for not responding sooner, I somehow missed the reply.
Yep, I upgraded to the pre-released version of the extension and am seeing the behavior you describe, thank you. Nice-to-haves would be:

Version checking
Something louder, like a notification

Happy to open separate issues for those or whatever would be helpful.

@afeld Please open separate issues.