net/http/httputil: data race in ReverseProxy

What version of Go are you using (go version)?

$ go version
go version go1.11.2 darwin/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/megvii/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/megvii/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.2/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/d_/17dg6drs2ss4yttxylv9bv3r0000gn/T/go-build139806013=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

What did you expect to see?
all going well
What did you see instead?

WARNING: DATA RACE
Read at 0x00c420706ae8 by goroutine 147:
bufio.(*Writer).WriteString()
GOROOT/src/bufio/bufio.go:590 +0xd2
net/http.writeStatusLine()
GOROOT/src/net/http/server.go:1440 +0x75
net/http.(*chunkWriter).writeHeader()
GOROOT/src/net/http/server.go:1410 +0xd5c
net/http.(*chunkWriter).Write()
GOROOT/src/net/http/server.go:359 +0x66a
bufio.(*Writer).Flush()
GOROOT/src/bufio/bufio.go:573 +0x138
net/http.(*response).finishRequest()
GOROOT/src/net/http/server.go:1542 +0x8c
net/http.(*conn).serve()
GOROOT/src/net/http/server.go:1835 +0x831

Previous write at 0x00c420706ae8 by goroutine 188:
bufio.(*Writer).Flush()
GOROOT/src/bufio/bufio.go:585 +0x3c5
net/http.(*expectContinueReader).Read()
GOROOT/src/net/http/server.go:868 +0x2b5
 … // omit some private wrappers for Read()
net/http.transferBodyReader.Read()
GOROOT/src/net/http/transfer.go:60 +0x75
io.(*LimitedReader).Read()
GOROOT/src/io/io.go:446 +0xca
bufio.(*Writer).ReadFrom()
GOROOT/src/bufio/bufio.go:703 +0x13b
io.copyBuffer()
GOROOT/src/io/io.go:386 +0x417
io.Copy()
GOROOT/src/io/io.go:362 +0x74
net/http.(*transferWriter).WriteBody()
GOROOT/src/net/http/transfer.go:337 +0x98a
net/http.(*Request).write()
GOROOT/src/net/http/request.go:622 +0x797
net/http.(*persistConn).writeLoop()
GOROOT/src/net/http/transport.go:1825 +0x2e0

Goroutine 147 (running) created at:
net/http.(*Server).Serve()
GOROOT/src/net/http/server.go:2795 +0x364
net/http.(*Server).ListenAndServe()
GOROOT/src/net/http/server.go:2711 +0xc4
main.main()
service/ossproxy/main.go:64 +0x3ac

Goroutine 188 (running) created at:
net/http.(*Transport).dialConn()
GOROOT/src/net/http/transport.go:1238 +0xb4a
net/http.(*Transport).getConn.func4()
GOROOT/src/net/http/transport.go:957 +0x9f

I found that in serveHTTP() reverse proxy start a default standard transport.RoundTrip() .
After RoundTrip() there is also a running goroutine writeloop() .  This is by design:

From net/http/client.go type RoundTripper interface :
RoundTrip must always close the body, including on errors,
but depending on the implementation may do so in a separate
goroutine even after RoundTrip returns. This means that
callers wanting to reuse the body for subsequent requests
must arrange to wait for the Close call before doing so.

But this goroutine may write to conn.bufw, which will make race condition with net/http.(*Server).Serve().
Sometimes it leads to panic. In net/http.(*Server).Serve() conn is closed, but write loop wants to write something to conn.bufw  .
It is hard to reproduce this issue. It appears about every several hours in my service.
I am trying to find a simple way to reproduce it.

What's your code look like?
The ReverseProxy code has to run synchronously in your ServeHTTP Handler. You can't start it in a new goroutine and then return from your ServeHTTP.

@bradfitz  There is no synchronous code in my ServerHTTP.
I means ReverseProxy.ServeHttp() run  transport.RoundTrip(), and transport.RoundTrip() run a "writeloop()" synchronously, which may lead to race condition.
I've simplified my code:

package main

import (
        "net/http/httputil"

        "github.com/aws/aws-sdk-go/aws/signer/v4"

        "..."
)

type proxy struct {
	p            *httputil.ReverseProxy
    	signer       *v4.Signer
}

func (p *proxy) getDirector(target *url.URL) func(req *http.Request) {
	return func(req *http.Request) {
		req.URL.Scheme = target.Scheme
		req.URL.Host = target.Host

		// Req.Host should be "" or req.URL.Host. Because v4.Signer will prefer req.host to req.URL.host.
		req.Host = target.Host
		if _, ok := req.Header["User-Agent"]; !ok {
			req.Header.Set("User-Agent", "")
		}

		if req.Header.Get("Content-Length") == "0" {
			req.Header.Del("Content-Length")
		}

		_, err := p.signer.Sign(req, nil, "s3", "", time.Now())
		if err != nil {
			log.Errorln(err)
		}
	}
}

func newProxy() (p *proxy, err error) {
	p = &proxy{}

	// httputil.ReverseProxy
	target, err := url.Parse(*flagRgw)
	if err != nil {
		return nil, err
	}
	p.p = &httputil.ReverseProxy{Director: p.getDirector(target)}
}

func (p *proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
        // omit some unimportant codes

        p.p.ServeHTTP(w,r)
}

func main() {
        server := &http.Server{
	        Addr:    80,
        	Handler: p,
        }
        server.ListenAndServe()
}

@bradfitz I reproduce the data race or panic in ReverseProxy.
I construct a broken server. The server will always send 100-continue and an incorrect header to proxy. In ReverseProxy，transport.RoundTrip(outreq) will receive an error and don't keep the connection to client. After the connection is closed, (*conn).finalFlush will assign (*conn).bufw = nil.  At the same time, writeLoop probably doesn't exit and may writeBody. The body here is an *expectContinueReader. When writeLoop first reads it, it writes 100-continue to client. But (*expectContinueReader).resp.conn.bufw == nil, so it leads to panic. If writeBody happens before assign，it leads to data race.
Here is my code:
package main

import (
	"io"
	"io/ioutil"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
	"time"
)

type zeroReader struct{}

func (zeroReader) Read(p []byte) (int, error) { return len(p), nil }

var infinite io.Reader = &zeroReader{}

func brokenBackend() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		conn, bfw, err := w.(http.Hijacker).Hijack()
		if err != nil {
			log.Fatalf("hijack failed: %v", err)
		}
		// Tell client net/http.(*persistConn).writeLoop() to continue writeBody.
		bfw.WriteString("HTTP/1.1 100 Continue\r\n\r\n")

		// Send client a wrong format header to make client net/http.(*persistConn).readLoop() readResponse failed.
		// Then client net/http.(*persistConn).roundTrip() will return an error and probably doesn't wait
		// net/http.(*persistConn).writeLoop() to exit.
		bfw.WriteString("BROKEN-HEADER\r\n\r\n")
		bfw.Flush()

		// Keep this connection for 1s so that the client doesn't think the connection is down immediately.
		<-time.AfterFunc(time.Second, func() { conn.Close() }).C
	})
	http.ListenAndServe("localhost:8081", nil)
}

// Proxy just forwards requests to the broken backend.
func proxy() {
	target, err := url.Parse("http://localhost:8081")
	if err != nil {
		log.Fatalf("parse target url failed: %v", err)
	}
	p := httputil.NewSingleHostReverseProxy(target)
	p.ErrorLog = log.New(ioutil.Discard, "", 0)
	http.ListenAndServe("localhost:8080", p)
}

func main() {
	go brokenBackend()
	go proxy()
	// Wait backend and proxy server to start
	time.Sleep(time.Second)

	for range time.Tick(time.Millisecond * 100) {
		req, err := http.NewRequest("PUT", "http://localhost:8080", io.LimitReader(infinite, 65536))
		if err != nil {
			log.Fatalf("create request failed: %v", err)
		}
		req.Header.Set("Expect", "100-continue")
		_, err = http.DefaultClient.Do(req)
		if err != nil {
			log.Fatalf("do request failed: %v", err)
		}
	}
}

Run the code with race detector. After a few seconds, data race is occurred. In very few cases，it leads to panic.

Go 1.14.5 has fixed a similar bug to this issue, but it doesn't work for @gozssky 's reproducible code.
Has any work been done on this? Is there a work around that I can use?

@inksink as the original issue is over a year old now would you be able to include the output from the data race detector when running Go 1.14.5 or 1.15.beta1. Thanks

@davecheney I run the same code as mentioned above.  Here is the output:
[root@centos7 ~]# go/bin/go version
go version go1.14.5 linux/amd64
[root@centos7 ~]# go/bin/go run -race main.go
==================
WARNING: DATA RACE
Read at 0x00c000cae828 by goroutine 63:
  bufio.(*Writer).Available()
      /root/go/src/bufio/bufio.go:608 +0xd4
  bufio.(*Writer).WriteString()
      /root/go/src/bufio/bufio.go:689 +0x2f
  net/http.writeStatusLine()
      /root/go/src/net/http/server.go:1505 +0x75
  net/http.(*chunkWriter).writeHeader()
      /root/go/src/net/http/server.go:1475 +0xd49
  net/http.(*chunkWriter).close()
      /root/go/src/net/http/server.go:402 +0x176
  net/http.(*response).finishRequest()
      /root/go/src/net/http/server.go:1621 +0xc4
  net/http.(*conn).serve()
      /root/go/src/net/http/server.go:1929 +0x88a

Previous write at 0x00c000cae828 by goroutine 511:
  bufio.(*Writer).Flush()
      /root/go/src/bufio/bufio.go:603 +0x3d4
  net/http.(*expectContinueReader).Read()
      /root/go/src/net/http/server.go:904 +0x2b6
  io.copyBuffer()
      /root/go/src/io/io.go:405 +0x143
  io.Copy()
      /root/go/src/io/io.go:364 +0x78
  net/http.(*transferWriter).doBodyCopy()
      /root/go/src/net/http/transfer.go:400 +0x2f
  net/http.(*transferWriter).writeBody()
      /root/go/src/net/http/transfer.go:348 +0x753
  net/http.(*Request).write()
      /root/go/src/net/http/request.go:685 +0x859
  net/http.(*persistConn).writeLoop()
      /root/go/src/net/http/transport.go:2280 +0x331

Goroutine 63 (running) created at:
  net/http.(*Server).Serve()
      /root/go/src/net/http/server.go:2962 +0x5b6
  net/http.(*Server).ListenAndServe()
      /root/go/src/net/http/server.go:2859 +0x102
  net/http.ListenAndServe()
      /root/go/src/net/http/server.go:3115 +0x2d1
  main.proxy()
      /root/main.go:48 +0x20d

Goroutine 511 (running) created at:
  net/http.(*Transport).dialConn()
      /root/go/src/net/http/transport.go:1648 +0xc33
  net/http.(*Transport).dialConnFor()
      /root/go/src/net/http/transport.go:1365 +0x14f
==================
==================
WARNING: DATA RACE
Write at 0x00c000e11000 by goroutine 63:
  runtime.slicestringcopy()
      /root/go/src/runtime/slice.go:232 +0x0
  bufio.(*Writer).WriteString()
      /root/go/src/bufio/bufio.go:699 +0x2ad
  net/http.writeStatusLine()
      /root/go/src/net/http/server.go:1505 +0x75
  net/http.(*chunkWriter).writeHeader()
      /root/go/src/net/http/server.go:1475 +0xd49
  net/http.(*chunkWriter).close()
      /root/go/src/net/http/server.go:402 +0x176
  net/http.(*response).finishRequest()
      /root/go/src/net/http/server.go:1621 +0xc4
  net/http.(*conn).serve()
      /root/go/src/net/http/server.go:1929 +0x88a

Previous write at 0x00c000e11000 by goroutine 511:
  runtime.slicestringcopy()
      /root/go/src/runtime/slice.go:232 +0x0
  bufio.(*Writer).WriteString()
      /root/go/src/bufio/bufio.go:699 +0x2ad
  net/http.(*expectContinueReader).Read()
      /root/go/src/net/http/server.go:903 +0x27a
  io.copyBuffer()
      /root/go/src/io/io.go:405 +0x143
  io.Copy()
      /root/go/src/io/io.go:364 +0x78
  net/http.(*transferWriter).doBodyCopy()
      /root/go/src/net/http/transfer.go:400 +0x2f
  net/http.(*transferWriter).writeBody()
      /root/go/src/net/http/transfer.go:348 +0x753
  net/http.(*Request).write()
      /root/go/src/net/http/request.go:685 +0x859
  net/http.(*persistConn).writeLoop()
      /root/go/src/net/http/transport.go:2280 +0x331

Goroutine 63 (running) created at:
  net/http.(*Server).Serve()
      /root/go/src/net/http/server.go:2962 +0x5b6
  net/http.(*Server).ListenAndServe()
      /root/go/src/net/http/server.go:2859 +0x102
  net/http.ListenAndServe()
      /root/go/src/net/http/server.go:3115 +0x2d1
  main.proxy()
      /root/main.go:48 +0x20d

Goroutine 511 (running) created at:
  net/http.(*Transport).dialConn()
      /root/go/src/net/http/transport.go:1648 +0xc33
  net/http.(*Transport).dialConnFor()
      /root/go/src/net/http/transport.go:1365 +0x14f
==================
==================
WARNING: DATA RACE
Write at 0x00c000e11018 by goroutine 63:
  runtime.slicestringcopy()
      /root/go/src/runtime/slice.go:232 +0x0
  bufio.(*Writer).WriteString()
      /root/go/src/bufio/bufio.go:699 +0x2ad
  net/http.writeStatusLine()
      /root/go/src/net/http/server.go:1513 +0x1be
  net/http.(*chunkWriter).writeHeader()
      /root/go/src/net/http/server.go:1475 +0xd49
  net/http.(*chunkWriter).close()
      /root/go/src/net/http/server.go:402 +0x176
  net/http.(*response).finishRequest()
      /root/go/src/net/http/server.go:1621 +0xc4
  net/http.(*conn).serve()
      /root/go/src/net/http/server.go:1929 +0x88a

Previous write at 0x00c000e11018 by goroutine 511:
  runtime.slicestringcopy()
      /root/go/src/runtime/slice.go:232 +0x0
  bufio.(*Writer).WriteString()
      /root/go/src/bufio/bufio.go:699 +0x2ad
  net/http.(*expectContinueReader).Read()
      /root/go/src/net/http/server.go:903 +0x27a
  io.copyBuffer()
      /root/go/src/io/io.go:405 +0x143
  io.Copy()
      /root/go/src/io/io.go:364 +0x78
  net/http.(*transferWriter).doBodyCopy()
      /root/go/src/net/http/transfer.go:400 +0x2f
  net/http.(*transferWriter).writeBody()
      /root/go/src/net/http/transfer.go:348 +0x753
  net/http.(*Request).write()
      /root/go/src/net/http/request.go:685 +0x859
  net/http.(*persistConn).writeLoop()
      /root/go/src/net/http/transport.go:2280 +0x331

Goroutine 63 (running) created at:
  net/http.(*Server).Serve()
      /root/go/src/net/http/server.go:2962 +0x5b6
  net/http.(*Server).ListenAndServe()
      /root/go/src/net/http/server.go:2859 +0x102
  net/http.ListenAndServe()
      /root/go/src/net/http/server.go:3115 +0x2d1
  main.proxy()
      /root/main.go:48 +0x20d

Goroutine 511 (finished) created at:
  net/http.(*Transport).dialConn()
      /root/go/src/net/http/transport.go:1648 +0xc33
  net/http.(*Transport).dialConnFor()
      /root/go/src/net/http/transport.go:1365 +0x14f
==================

@davecheney Is there any progress on this issue?

Not yet, it is still labeled waiting for info as it has yet to be reproduced independently.

@bradfitz I reproduce the data race or panic in ReverseProxy.
I construct a broken server. The server will always send 100-continue and an incorrect header to proxy. In ReverseProxy，transport.RoundTrip(outreq) will receive an error and don't keep the connection to client. After the connection is closed, (*conn).finalFlush will assign (*conn).bufw = nil. At the same time, writeLoop probably doesn't exit and may writeBody. The body here is an *expectContinueReader. When writeLoop first reads it, it writes 100-continue to client. But (*expectContinueReader).resp.conn.bufw == nil, so it leads to panic. If writeBody happens before assign，it leads to data race.
Here is my code:
package main

import (
	"io"
	"io/ioutil"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
	"time"
)

type zeroReader struct{}

func (zeroReader) Read(p []byte) (int, error) { return len(p), nil }

var infinite io.Reader = &zeroReader{}

func brokenBackend() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		conn, bfw, err := w.(http.Hijacker).Hijack()
		if err != nil {
			log.Fatalf("hijack failed: %v", err)
		}
		// Tell client net/http.(*persistConn).writeLoop() to continue writeBody.
		bfw.WriteString("HTTP/1.1 100 Continue\r\n\r\n")

		// Send client a wrong format header to make client net/http.(*persistConn).readLoop() readResponse failed.
		// Then client net/http.(*persistConn).roundTrip() will return an error and probably doesn't wait
		// net/http.(*persistConn).writeLoop() to exit.
		bfw.WriteString("BROKEN-HEADER\r\n\r\n")
		bfw.Flush()

		// Keep this connection for 1s so that the client doesn't think the connection is down immediately.
		<-time.AfterFunc(time.Second, func() { conn.Close() }).C
	})
	http.ListenAndServe("localhost:8081", nil)
}

// Proxy just forwards requests to the broken backend.
func proxy() {
	target, err := url.Parse("http://localhost:8081")
	if err != nil {
		log.Fatalf("parse target url failed: %v", err)
	}
	p := httputil.NewSingleHostReverseProxy(target)
	p.ErrorLog = log.New(ioutil.Discard, "", 0)
	http.ListenAndServe("localhost:8080", p)
}

func main() {
	go brokenBackend()
	go proxy()
	// Wait backend and proxy server to start
	time.Sleep(time.Second)

	for range time.Tick(time.Millisecond * 100) {
		req, err := http.NewRequest("PUT", "http://localhost:8080", io.LimitReader(infinite, 65536))
		if err != nil {
			log.Fatalf("create request failed: %v", err)
		}
		req.Header.Set("Expect", "100-continue")
		_, err = http.DefaultClient.Do(req)
		if err != nil {
			log.Fatalf("do request failed: %v", err)
		}
	}
}

Run the code with race detector. After a few seconds, data race is occurred. In very few cases，it leads to panic.

@davecheney Here is the code. It doesn't  contain any dependency.

Slightly smaller reproduction
package main

import (
	"io"
	"io/ioutil"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
	"time"
)

type zeroReader struct{}

func (zeroReader) Read(p []byte) (int, error) { return len(p), nil }

var infinite io.Reader = &zeroReader{}

func brokenBackend() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		conn, bfw, err := w.(http.Hijacker).Hijack()
		if err != nil {
			log.Fatalf("hijack failed: %v", err)
		}
		// Tell client net/http.(*persistConn).writeLoop() to continue writeBody.
		bfw.WriteString("HTTP/1.1 100 Continue\r\n\r\n")
		bfw.Flush()

		conn.Close()
	})
	http.ListenAndServe("localhost:8081", nil)
}

// Proxy just forwards requests to the broken backend.
func proxy() {
	target, err := url.Parse("http://localhost:8081")
	if err != nil {
		log.Fatalf("parse target url failed: %v", err)
	}
	p := httputil.NewSingleHostReverseProxy(target)
	p.ErrorLog = log.New(ioutil.Discard, "", 0)
	http.ListenAndServe("localhost:8080", p)
}

func main() {
	go brokenBackend()
	go proxy()
	// Wait backend and proxy server to start
	time.Sleep(time.Second)

	for {
		req, err := http.NewRequest("PUT", "http://localhost:8080", io.LimitReader(infinite, 65536))
		if err != nil {
			log.Fatalf("create request failed: %v", err)
		}
		req.Header.Set("Expect", "100-continue")
		_, err = http.DefaultClient.Do(req)
		if err != nil {
			log.Fatalf("do request failed: %v", err)
		}
	}
}

The data race in https://github.com/golang/go/issues/29321#issuecomment-660952505 was fixed (so far as I can tell) by CL 251858


Update normalize 3.0.3 > 5.0.0

@acburst @Dogfalo Please look into it and do the needful

https://github.com/Dogfalo/materialize/pull/4611


To change font weight(bold, italic) when using  tag

Description
Currently <Text /> don't have any possible ways to change font weight when rendering a text in <Canvas> tag.
Is there any ways to render font in bold in RNSkia?

At the moment, you need to use bold typography to display bold text.

Can we expect any other ways to render bold text in the future, other than using bold typography?

You will always need to appropriate typography to display the text you want to display.
What will change in the future is that we are looking to offer support for the paragraph module which will be able to pick the appropriate typography based on text style automatically.
I'm closing this issue for now as there doesn't seem to be a bug related to the text component. Please feel free to reopen is needed.

At the moment, you need to use bold typography to display bold text.

Can you please provide an example how can we make a  bold with following example? Couldn't find how to make it bold.

Are we able to set "font-variation-setting" some way, have to use a fixed width for each Text character drawn on a Canvas area in a loop. When using monospace font this is not an issue but for non-monospace fonts need to make additional setting.

const font = useFont(
    require('@assets/fonts/Roboto-Medium.ttf'),
    32
  );

  <Text
      opacity={fromOpacity}
      x={fromX}
      y={fromY}
      text={char}
      color={color.toString()}
      font={font}
    />


fix awkward usage of the word "don't"

Before submitting
Please complete the following checklist when submitting a PR:

[ ] Ensure that your tutorial executes correctly, and conforms to the
guidelines specified in the README.

[ ] Add a thumbnail link to your tutorial in beginner.rst, or if a
QML implementation, in implementations.rst.

[ ] All QML tutorials conform to
PEP8 standards.
To auto format files, simply pip install black, and then
run black -l 100 path/to/file.py.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.

Title:
Summary:
Relevant references:
Possible Drawbacks:
Related GitHub Issues:

love what you do. please let me know how i can best help to advance your project. thanks!

Hi @yishairasowsky! Thank you for your contribution 🙂
In this particular case, however, I believe 'don't' conveys the meaning of the original sentence better. But we appreciate the PR! If you are looking for ways to contribute to the QML repo and the PennyLane project, you can check out our Contributions Guide, or submit a QML demonstration.
Alternatively, look out for issues that are tagged good first issue, these are great starting points for contributions to PennyLane.


Docs: Take whatever line endings and turn them into LF in generation

Use io.open and read/write line breaks as LF
reading: newline=None to read any kind of line break as LF
writing: newline='' to not translate line breaks to OS line separator
This fixes the template render errors in Cygwin. However, some checks fail for unknown reason (maybe more files with Windows line endings?)
Also remove deprecated.py as we don't need it anymore.

works on linux without flaws.

The failing checks in Cygwin come from inconsistent line endings in the temporary files.
There are CRLF as well as LF line endings in /tmp/indoc.txt etc.
This occurs in at least
CYGWIN_NT-10.0 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64
But was fine in (LF line breaks only)
CYGWIN_NT-10.0 2.4.1(0.293/5/3) 2016-01-24 11:26 x86_64
build.sh now has sed -e "s;\r$;;" in two places to remove the carriage returns, which circumvents the issues. Looks like a bug in Cygwin however.


Remove unneeded comment

[Current coverage][cc-pull] is 85.22%

Merging [#14][cc-pull] into [develop][cc-base-branch] will decrease coverage by 0.17%

@@            develop        #14   diff @@
==========================================
  Files            47         47
  Lines          2342       2342
  Methods           0          0
  Messages          0          0
  Branches        376        376
==========================================
- Hits           2000       1996     -4
- Misses          220        222     +2
- Partials        122        124     +2

Powered by Codecov. Last updated by [80d798a...0cab3f4][cc-compare]
[cc-base-branch]: https://codecov.io/gh/be-hase/relumin/branch/develop?src=pr
[cc-compare]: https://codecov.io/gh/be-hase/relumin/compare/80d798a9ff514fbccd2adbea8ceb16ff999c012d...0cab3f41076bbd826b72fbe697643dc718244217
[cc-pull]: https://codecov.io/gh/be-hase/relumin/pull/14?src=pr


make the description for gitpod setup precise.

I'm also missing the point a bit here. If people are interested in how it works exactly, they would just set it up themselves. The point of the Gitpod setup is to keep things as simple as possible.


Automate docs release process

The doc release process which is documented here should be automated as it involves standard edits.
Process is here: https://github.com/devfile/docs/blob/master/README.md

Devfile doc pages for each devfile spec version are maintained under subdirectories, could keep this open to automate the current steps to publish doc pages for new spec releases.
Description needs updating with up to date criteria.


Unable to find installation plugin pemfiles

Hi there,
I have used certbot for linux a lot, In linux I create the cert and use it in nginx , as well is in postfix and dovcot. NOW I would like to do this with iis and hmailserver. IIS works great, the cert gets created and the domain gets secured https, however, I can't seem to create pem files to be placed into hmailserver. wacs keeps moaning Unable to find installation plugin pemfiles.
WHen I run --installation iis,script it basically works (script has error but gets called) but if I add pemfiles, it dies.
Before you post
Make sure to search through the issue list to check if others have reported the same bug already. If you find an existing issue, make your voice heard there. Your case may provide valueable extra clues towards a solution and will increase its priority.
Describe the bug
A clear and concise description of what the bug is.
I try to use installation plugin pemfiles and get error
To Reproduce

Run with command line ...

PS C:\Program Files\win-acme> .\wacs.exe --source manual --host mailsdf.float.i.ng  --certificatestore My --installation iis,pemfiles,script  --installationsiteid 1 --script "Scripts\ImportRDSFull.ps1" --scriptparameters "{CertThumbprint}"   --accepttos   --emailaddress<EMAIL_ADDRESS>--pemfilespath c:\certs --validation route53  --route53accesskeyid   MYKEY   --route53secretaccesskey MYSECRET  --verbose

Pick menu options A, B, C
See error
Unable to find installation plugin pemfiles

Expected behavior
A clear and concise description of what you expected to happen.
Log
Please add any available log output. You may copy it from the console window, get it from the Windows Event Viewer, or the log files on disk, which can typically be found in a sub folder of %programdata%\win-acme.
Platform:

OS: [e.g. Windows 2016, English]

Version: [e.g. 2.1.0, 64-bit, pluggable]

win-acme.v<IP_ADDRESS>2.x64.pluggable
windows 10 hyperv
Additional context
Add any other context about the problem here, for example possible network issues (firewalls, proxies, NAT) that might play a role.
Here is the whole output
PS C:\Program Files\win-acme> .\wacs.exe --source manual --host mailsdf.float.i.ng  --certificatestore My --installation iis,pemfiles,script  --installationsiteid 1 --script "Scripts\ImportRDSFull.ps1" --scriptparameters "{CertThumbprint}"   --accepttos   --emailaddress<EMAIL_ADDRESS>--pemfilespath c:\certs --validation route53  --route53accesskeyid   MYKEY   --route53secretaccesskey MYSECRET  --verbose
[VERB] Verbose mode logging enabled
[VERB] ExePath: C:\Program Files\win-acme\wacs.exe
[VERB] ResourcePath: C:\Program Files\win-acme
[VERB] PluginPath: C:\Program Files\win-acme
[VERB] Loaded validation plugin Route53 from C:\Program Files\win-acme\PKISharp.WACS.Plugins.ValidationPlugins.Route53.dll
[VERB] Looking for settings.json in C:\Program Files\win-acme
[DBUG] Config folder: C:\ProgramData\win-acme\acme-v02.api.letsencrypt.org
[DBUG] Log path: C:\ProgramData\win-acme\acme-v02.api.letsencrypt.org\Log
[DBUG] Cache path: C:\ProgramData\win-acme\acme-v02.api.letsencrypt.org\Certificates
[DBUG] secrets.json not found
[VERB] Arguments: --source manual --host mailsdf.float.i.ng --certificatestore My --installation iis,pemfiles,script --installationsiteid 1 --script Scripts\ImportRDSFull.ps1 --scriptparameters {CertThumbprint} --accepttos --emailaddress<EMAIL_ADDRESS>--pemfilespath c:\certs --validation route53 --route53accesskeyid MYKEY  --route53secretaccesskey ******** --verbose
[DBUG] Renewal period: 55 days
[VERB] Sending e-mails False
[INFO] A simple Windows ACMEv2 client (WACS)
[INFO] Software version <IP_ADDRESS>2 (release, pluggable, standalone, 64-bit)
[INFO] Connecting to https://acme-v02.api.letsencrypt.org/...
[VERB] SecurityProtocol setting: SystemDefault
[DBUG] Send GET request to https://acme-v02.api.letsencrypt.org/directory
[VERB] Request completed with status OK
[DBUG] Connection OK!
[DBUG] IIS version 10.0
[DBUG] Running with administrator credentials
[WARN] Scheduled task not configured yet
[INFO] Please report issues at https://github.com/win-acme/win-acme
[VERB] Unicode display test: Chinese/語言 Russian/язык Arab/لغة
[INFO] Running in mode: Unattended
[VERB] Adding <IP_ADDRESS> as DNS server
[VERB] Adding <IP_ADDRESS> as DNS server
[VERB] Adding <IP_ADDRESS> as DNS server
[VERB] Parsed value for --host: mailsdf.float.i.ng
[VERB] No value provided for --commonname
[INFO] Source generated using plugin Manual: mailsdf.float.i.ng
[VERB] No value provided for --route53iamrole
[VERB] Parsed value for --route53accesskeyid: MYKEY
[VERB] Parsed value for --route53secretaccesskey: ********
[VERB] Flag --ocsp-must-staple not present
[VERB] Flag --reuse-privatekey not present
[VERB] Parsed value for --certificatestore: My
[VERB] Flag --keepexisting not present
[VERB] No value provided for --acl-fullcontrol
[VERB] No value provided for --sslport
[VERB] No value provided for --sslipaddress
[VERB] Parsed value for --installationsiteid: 1
[EROR] Unable to find installation plugin pemfiles. Choose another plugin using the --installation switch or change the default in settings.json
[EROR] Installation plugin could not be selected
[VERB] Exiting with status code -1

I see that the bug is with the docs, They say to add pemfiles as plugin when in reality it needs to be flagged with --store CertificateStore,pemfiles
Now the command below creates pem files as well as IIS certs. The only issue now is that it doesn't seem to be scheduling the renewal.  I'll figure this out too
`
.\wacs.exe --installation iis,script --store CertificateStore,pemfiles  --source manual --host mailsdf.float.i.ng  --certificatestore My   --installationsiteid 1 --script "Scripts\ImportRDSFull.ps1" --scriptparameters "{CertThumbprint}"   --accepttos   --emailaddress<EMAIL_ADDRESS>--pemfilespath c:\certs --validation route53  --route53accesskeyid   mykey   --route53secretaccesskey private  --verbose`

I see that the bug is with the docs, They say to add pemfiles as plugin when in reality it needs to be flagged with --store CertificateStore,pemfiles

There are different types of plugins, --installation plugins are one kind, --store plugins another kind. Store plugins can feed information to installation plugins like paths. But if you point out the piece of documentation that confused you I'd be happy to take a look and see if it can be improved.


Volume control 13.1 Ventura Apple studio M1

Before opening the issue, have you...?

[X] Searched for existing issues

Discussion
I have 2 LG 35WN65C-B Monitors.
The app sees both.
I can adjust individual monitors from the pull-down.
When I use the keyboard to adjust the overall volume,
It only affects the one that is set to Main Monitor in the OS.
IF I drop it to one slider for all it doesn’t affect the Apple Cinema Display.
Thanks

Are you using multiple audio outputs at the same time? Most users probably use one of their external displays for audio output. If you are using one of your displays, you should simply disable volume control for the other one (Preferences/Displays/Show advanced settings). You can also override the audio device name that is associated to the display. You can also try changing Screen to control to Change volume for all screens for keyboard controls.


⚠ Bofrost.at has degraded performance

In 3798ade, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1451 ms

Resolved: Bofrost.at performance has improved in 8670ca5 after 7 minutes.


Add input_fee_ppk to /keys

New fee definition is added to /keysets as input_fee_ppk property. I suggest to add it to /keys response as well. Reason is, that currently only /keys API is sufficient to be called when making new wallet instance. Now, when fees are introduced, two similar APIs needs to be called.

I think this is a bit of an edge case that only applies to the very first initialization of the wallet. Wallets should fetch the /keysets and then fetch /keys only for the keysets it has not seen before. While its true if the wallet has never been created before and has no keys it will need to fetch all keys, its simpler to follow the one flow of checking /keysets for new keysets, state changes and fee_changes and then fetching keys for unknown keys

Thanks for explanation, seems logical. It is likely then some work to ne done in cashu-ts.


[DeleteFilesV1] Migration of DeleteFilesV1 to node10

Task name: DeleteFilesV1
Description: Updated dependencies; Updated execution handler; Replaced deprecated methods MochaDone => Mocha.Done; Removed typings; Fixed typo path => curPath in removeFolder method in tests;
Documentation changes required: (Y/N) N
Added unit tests: (Y/N) N
Attached related issue: (Y/N) Y
Checklist:

[X] Task version was bumped - please check instruction how to do it
[X] Checked that applied changes work as expected

Node 6 backup branch

Can you also delete the entries in the Tests-Legacy folder related to DeleteFiles?

Can you also delete the entries in the Tests-Legacy folder related to DeleteFiles?

Legacy tests removed, thank you!


Missing features

After using the package on a real project, found missing features :

When using php artisan preset ttall, our base blade files are not updated like with ttall-auth and we need to initialize Livewire, turbolinks, and paginations by hand
Missing composer's script for Larastan
Document those changes on the Readme.md file

Done on branch 2.x


[23.11] gitlab: 16.10.6 -> 16.11.4

Manual Backport of #319486.
Description of changes

https://gitlab.com/gitlab-org/gitlab/-/blob/v16.11.4-ee/CHANGELOG.md
Fixes CVE-2024-1495
Fixes CVE-2024-1963
Fixes CVE-2024-4201
Fixes CVE-2024-5469

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 319563 run on x86_64-linux 1

6 packages built:

gitaly
gitlab
gitlab-ee
gitlab-pages
gitlab-shell
gitlab-workhorse


Template rules (implement #45)

I made support for template rules. The rule can be parameterized by other rules / expressions. All references inside template rule with name of template parameter refer to parameter, superimposing visibility scope of the rule with a similar name.
At the moment parameters can't be templates, i.e. such construction is forbidden:
Apply<X, Y> = X<Y>;

Perhaps, further I will make its support.

Thanks for the work, but I’ll close this now. Templates are something I’d like to have a look at after 1.0.0.


[Documentation] hipExtMallocWithFlags coherence mixup

https://rocm.docs.amd.com/en/latest/conceptual/gpu-memory.html#coherence
There appear to be a number of typos in this section.
The hipHostMallocDefault flag should be hipDeviceMallocDefault and have coarse-grained coherence.
The coherence for hipDeviceMallocFinegrained should be fine-grained.

Internal ROCDOC-292 has been created to track the progress of this ticket.

Fixed in PR #2819

https://github.com/ROCm/ROCm/pull/2819


Custom image from Portal

It looks like currently we can deploy VMSS with custom image by portal.
Please also update the document! thank you very much for this improvement :)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c7ce0660-0573-ca8f-462f-78e7593fea64
Version Independent ID: db5b936f-60ab-bc78-cc90-b8834cce009a
Content: Reference a custom image in an Azure scale set template
Content Source: articles/virtual-machine-scale-sets/virtual-machine-scale-sets-mvss-custom-image.md
Service: virtual-machine-scale-sets
GitHub Login: @mayanknayar
Microsoft Alias: manayar

#please-close I've created a work item to have a topic created about using custom images for VMSS creation. Thank you :)


lighttdp is not working

Hi, thank you for this great project!
I have installed the Phoniebox on a Raspberry 3B+ using the latest master branch, following the installation instructions in INSTALL-stretch.md step by step (not the one line install command), using headless installation via ssh.
Now I encountered a problem with lighttpd and am unable to find a solution on my own. I would be very glad if you could help me!
I have comleted and rechecked all the steps up to:
sudo service lighttpd force-reload
This line always produces the output:
Job for lighttpd.service failed because the control process exited with error code. See "systemctl status lighttpd.service" and "journalctl -xe" for details.
pi@phoniebox:~ $ journalctl -xe
-- Unit lighttpd.service has failed.
-- The result is failed.
Oct 30 07:51:28 phoniebox systemd[1]: lighttpd.service: Unit entered failed state.
Oct 30 07:51:28 phoniebox systemd[1]: lighttpd.service: Failed with result 'exit-code'.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Service hold-off time over, scheduling restart.
Oct 30 07:51:29 phoniebox systemd[1]: Stopped Lighttpd Daemon.
-- Subject: Unit lighttpd.service has finished shutting down
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- Unit lighttpd.service has finished shutting down.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Start request repeated too quickly.
Oct 30 07:51:29 phoniebox systemd[1]: Failed to start Lighttpd Daemon.
-- Subject: Unit lighttpd.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- Unit lighttpd.service has failed.
-- The result is failed.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Unit entered failed state.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Failed with result 'exit-code'.
pi@phoniebox:~ $ systemctl status lighttpd.service
● lighttpd.service - Lighttpd Daemon
Loaded: loaded (/lib/systemd/system/lighttpd.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2018-10-30 07:51:29 UTC; 9min ago
Process: 1408 ExecStartPre=/usr/sbin/lighttpd -tt -f /etc/lighttpd/lighttpd.conf (code=exited, status=255)
Oct 30 07:51:28 phoniebox systemd[1]: lighttpd.service: Failed with result 'exit-code'.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Service hold-off time over, scheduling restart.
Oct 30 07:51:29 phoniebox systemd[1]: Stopped Lighttpd Daemon.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Start request repeated too quickly.
Oct 30 07:51:29 phoniebox systemd[1]: Failed to start Lighttpd Daemon.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Unit entered failed state.
Oct 30 07:51:29 phoniebox systemd[1]: lighttpd.service: Failed with result 'exit-code'.

Can you post here please the lighttpd.conf? maybe an permissions error I found online as a possible reason. What says "ls -lisa /etc/lighttpd/lighttpd.conf"?

pi@phoniebox:~ $ ls -lisa /etc/lighttpd/lighttpd.conf
181538 4 -rw-r--r-- 1 root root 1107 Oct 19 21:31 /etc/lighttpd/lighttpd.conf
pi@phoniebox:~ $ cat /etc/lighttpd/lighttpd.conf
server.modules = (
"mod_access",
"mod_alias",
"mod_compress",
"mod_redirect",
)
server.document-root        = "/var/www/html"
Changed by Maike
server.document-root        = "/home/pi/RPi-Jukebox-RFID/htdocs"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80
index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )
default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"
Thank you for helping!

Hi @Moostante
please try to reinstall using the latest one-line-install script. The current version is 1.1.8-rc2
The install doc for manual install is behind the latest code. Sorry about that.


Bug Report: Compiling Issue

Everytime i try to Compile the game, I have this error message:

I've downloaded everything i've needed, Including the 22gb of pain, but everytime it gives me this response.
Even before writing this, I've been looking at the closed issues on github but everything I've tried hasn't worked.
I have the most recent version of KadeEngine Github code, and FYI: This is my first time modding the game.
Please let me know what I'm doing wrong if you have a solution!

There's a new version of the source code available, try that one and update us.

There's a new version of the source code available, try that one and update us.

I updated it, and i still have the same problem

Could it be something about the paths.hx ? It says "Unknown Identifier: paths " on the error message.
Also I made sure to install the new linc luajit and hxcpp debugger required for the new update

Don't know what your custom source code looks like but, try capitalizing paths.

did you try replacing the character.hx file with the new one?

I downloaded the new update so yes, I have the new character.hx file

Don't know what your custom source code looks like but, try capitalizing paths.

Sorry, I don't actually know what you mean by that; could you clarify?

Change paths to Paths

nope, still didn't work

Show us that line (along with a couple others around it) from your Character.hx.

aight i had mine below the christmas-parents

Change the frames ='s paths to Paths, it's a class (and file inside the source folder) and has to be capitalized for its functions to work (such as getSparrowAtlas).

also, Is the positions of animations and Offsets right?

Dunno, it's your code. Also, might as well move this to the discussions page, since it's related to custom code.

Hey! It worked! Thank you so much! I Just needed to make the Path capital!


Need to clarify Planned Maintenance feature would not affect underlay update.

As title, per discussing with feature owner, this Planned Maintenance would not take effect on underlay update part.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9ae4303e-dc21-d795-49ae-3798906fcd53
Version Independent ID: e503495c-bb1c-2956-cc81-79f9d7188b67
Content: Use Planned Maintenance for your Azure Kubernetes Service (AKS) cluster (preview) - Azure Kubernetes Service
Content Source: articles/aks/planned-maintenance.md
Service: container-service
GitHub Login: @qpetraroia
Microsoft Alias: qpetraroia

@gsxmax
Thank you for your feedback. We will investigate and update the thread further.

@gsxmax,
This lacks clarity in the feedback. If this has already been discussed with the feature owner and addressed, please confirm. If not, please provide more details.


PCA, hierarchical clustering and k nearest neighbors

Hi @davidsandberg,
First of all congratulations for your excellent results. The last model you've uploaded is simply amazing. Thanks for sharing it!!
I wanted to share with you how I am using the features generated by the model to build datasets, cluster them and cross-check new images.
First I am using PCA to reduce from 1792 features to 128, as those seem to be sufficient to compare images accurately and all subsequent calculations are faster:
from sklearn.decomposition import PCA
pcafacenet_ = PCA(n_components=128)
pcafacenet_.fit(embfacenet_)
embfacenet = pcafacenet_.transform(embfacenet_)
This PCA object can be persisted using sklearn.externals.joblib.dump and reloaded later, so the first 2 lines can be skipped in the next executions.
Then for clustering I use:
from scipy.spatial.distance import pdist
import fastcluster as fastcluster
import scipy.cluster.hierarchy as hcluster
Y = pdist(embfacenet_, 'euclidean')
Z = fastcluster.single(Y)
labels = hcluster.fcluster(Z, threshold, criterion="distance")
Finally, for comparing a new image with the dataset, I use:
nnfacenet_ = NearestNeighbors(n_neighbors=nn)
nnfacenet_.fit(embfacenet_)
dist, kneighbors = nnfacenet_.kneighbors(newembeddings)
Does this make sense to you?
I know that you are also investing quite some effort in training a classifier. What is the advantage of doing that compared to using k nearest neighbors to look for similar faces?

I'm doing something similar and finding my results to be not great. ie the top nearest neighbor matches are false positives. The PCA doesn't seem to help much. Which models & datasets are you having success with?

Hi @dleonsanchez ,
I don't quite understand the intention behind clustering. Is it to clean a noisy dataset or to use the cluster centers as the canonical features? Could you give more details? Thanks.
Hi @haydenth ,
I was also planning to implement the "Detailed setting in testing" part in the centerloss paper(A Discriminative Feature...), but it seems that you've already done it and did not get good results.
Did you do PCA on the whole training dataset or on the validation dataset (9 of 10 folds)? Did you do horizontal-flip and cosine distance as well? Thanks.

I did PCA only on the validation dataset and on my own validation datasets. Did not try other distance functions yet.

@haydenth I am testing with LFW dataset. PCA does not help in the sense of increasing accuracy, the advantage from my point of view is that we can reduce the number of embeddings (to 128 in my example) without losing accuracy, and then the files where I store the embeddings are much smaller and all calculations (loading the file into a pandas object, knn, clustering, etc) are much faster.
@ugtony, for me clustering helps when the use case includes processing faces with unknown identities. If we want to tag them you can either go one by one and, displaying their nearest neighbours, see if each one of them has the same identity as another picture, or we can cluster them (with hierarchical clustering in my example) and then display them in clusters so that tagging them becomes a much easier exercise.
But I am not too sure about this, so that's also why I was asking in the first place.

I have similar issue. I am building classifier similar to openface's. But accuracy is extremely low - around 1-5%.

@haydenth,
I did horizontal-flip+PCA(trained by casia faces)+cosine with the msceleb1M-pretrained model, and the performance wasn't improved much as well.

Hi @dleonsanchez, @ugtony!
I added a 128d bottleneck before the classification layer and the results look good.
I'm still running training on casia to tune weight decay etc but with the same hyper parameters I get similar performance to the 1792d embeddings.
This will simplify the flow since it will make the PCA unnesseccary.
Will publish new models when training on the Ms-Celeb-1M dataset is done.
        # Build the inference graph
        prelogits, _ = network.inference(image_batch, args.keep_probability,
            phase_train=phase_train_placeholder, weight_decay=args.weight_decay)
        bottleneck = slim.fully_connected(prelogits, args.embedding_size, activation_fn=None,
                weights_initializer=tf.truncated_normal_initializer(stddev=0.1),
                weights_regularizer=slim.l2_regularizer(args.weight_decay),
                normalizer_fn=slim.batch_norm,
                normalizer_params=batch_norm_params,
                scope='Bottleneck', reuse=False)
        logits = slim.fully_connected(bottleneck, len(train_set), activation_fn=None,
                weights_initializer=tf.truncated_normal_initializer(stddev=0.1),
                weights_regularizer=slim.l2_regularizer(args.weight_decay),
                scope='Logits', reuse=False)

        embeddings = tf.nn.l2_normalize(bottleneck, 1, 1e-10, name='embeddings')

Observing the PCA and the bottleneck-fc-layer experimental resluts(done before on vggface). I agree to davidsandberg's opinion that PCA would be unnecessary.  Good to know that the bottleneck layer is to be added to the master branch.

FYI, I have included PCA transformations at the end of validate_on_lfw.py:
for i in [32, 64, 128, 256, 512, 1024, 1792]:
pcafacenet = PCA(n_components=i)
pcafacenet.fit(emb_array)
pcaemb_array = pcafacenet.transform(emb_array)
print('Number of features: {}'.format(i))
tpr, fpr, accuracy, val, val_std, far = lfw.evaluate(pcaemb_array, args.seed, actual_issame, nrof_folds=args.lfw_nrof_folds)
print('Accuracy: %1.3f+-%1.3f' % (np.mean(accuracy), np.std(accuracy)))
print('Validation rate: %2.5f+-%2.5f @ FAR=%2.5f' % (val, val_std, far))
print ('')
These are the results I get:
Number of features: 32
Accuracy: 0.959+-0.007
Validation rate: 0.66702+-0.01962 @ FAR=0.00103
Number of features: 64
Accuracy: 0.987+-0.003
Validation rate: 0.91714+-0.01633 @ FAR=0.00103
Number of features: 128
Accuracy: 0.992+-0.003
Validation rate: 0.97604+-0.00678 @ FAR=0.00068
Number of features: 256
Accuracy: 0.992+-0.003
Validation rate: 0.97500+-0.00732 @ FAR=0.00100
Number of features: 512
Accuracy: 0.992+-0.003
Validation rate: 0.97401+-0.00706 @ FAR=0.00100
Number of features: 1024
Accuracy: 0.993+-0.003
Validation rate: 0.97404+-0.00785 @ FAR=0.00100
Number of features: 1792
Accuracy: 0.993+-0.003
Validation rate: 0.97436+-0.00755 @ FAR=0.00100
Slightly less accuracy with 512, 256 and 128 features. Below that, it is significant. Interesting the slightly better validation rate with 128 too.

Hi @dleonsanchez!
Nice results! Thanks for that!!
It basically confirms the results from the FaceNet paper regarding the required dimensionality of the embedding.
I have added a pretrained model with a 128 wide bottleneck layer. And as you say the performance is very similar to the original 1792 embedding.


System hang occurs during SMMU initialization

@semihalf-bernacki-grzegorz  confirmed that this issue can be worked around by the patch smmu.patch.txt from @ndhillonm. In the patch, the timeout-related change is needed for preventing system hang. However, we're still having no idea why adding a __sync_synchronize() call can work around this issue.  Can we get a BSA expert to check the patch?
The following is the log showing where system hang occurs.
FS4:\EFI\BOOT\bsa\> Bsa.efi

 BSA Architecture Compliance Suite
    Version 1.0

 Starting tests with Print level is  3

 Creating Platform Information Tables
 PE_INFO: Number of PE detected       :   24
 GIC_INFO: Number of GICD             :    1
 GIC_INFO: Number of ITS              :    1
 TIMER_INFO: Number of system timers  :    0
 WATCHDOG_INFO: Number of Watchdogs   :    1
 PCIE_INFO: Number of ECAM regions    :    7
 PCIE_INFO: Number of RCiEP           :   13
 PCIE_INFO: Number of RCEC            :    0
 PCIE_INFO: Number of EP              :   41
 PCIE_INFO: Number of RP              :   25
 PCIE_INFO: Number of iEP_EP          :    0
 PCIE_INFO: Number of iEP_RP          :    0
 SMMU_INFO: Number of SMMU CTRL       :    1

Thanks @semihalf-bernacki-grzegorz and @sunnywang-arm,
We will review the patch and update soon.

The first issue is an error in below function. While loop is based on 'timeout' which is not getting decremented. I think this should be a trivial fix that we can agree on.
static void smmu_cmdq_poll_until_consumed(smmu_dev_t *smmu)
{
uint32_t timeout = SMMU_CMDQ_POLL_TIMEOUT;
smmu_cmd_queue_t *cmdq = &smmu->cmdq;
smmu_queue_t queue = {
.log2nent = smmu->cmdq.queue.log2nent,
.prod = val_mmio_read((uint64_t)smmu->cmdq.prod_reg),
.cons = val_mmio_read((uint64_t)smmu->cmdq.cons_reg)
};
while (timeout > 0) {
    if (smmu_queue_empty(&queue))
        break;
    queue.cons = val_mmio_read((uint64_t)cmdq->cons_reg);
}

if (!timeout) {
    val_print(ACS_PRINT_ERR, "\n       CMDQ poll timeout at 0x%08x", queue.prod);
    val_print(ACS_PRINT_ERR, "\n       prod_reg = 0x%08x,",

val_mmio_read((uint64_t)smmu->cmdq.prod_reg));
val_print(ACS_PRINT_ERR, "\n       cons_reg = 0x%08x",
val_mmio_read((uint64_t)smmu->cmdq.cons_reg));
val_print(ACS_PRINT_ERR, "\n       gerror   = 0x%08x     ",
val_mmio_read(smmu->base + SMMU_GERROR_OFFSET));
}
}

Second issue is the implementation of 'smmu_cmdq_write_cmd' function.
If you look at the SMMU spec,
3.5.2 Queue entry visibility semantics
Any producer (whether the SMMU or software) must ensure that if an update to the PROD index value is observable
by the consumer, all new queue entries are observable by the consumer
This clearly implies a need for a barrier so that queue entries are visible when PROD register is updated. This can also be seen in implementation of 'arm_smmu_cmdq_issue_cmdlist' function in Linux kernel SMMUv3 driver.

Hi @semihalf-bernacki-grzegorz,
Thanks for the patch.
We have integrated the timeout change, and for barrier will use DMB instruction.
Can you help us in verifying the fix with below binary
https://github.com/chetan-rathore/bsa-acs/blob/main/prebuilt_images/Bsa_ES_Fix.efi
Changes for this issue + #11 -> https://github.com/chetan-rathore/bsa-acs/commit/bd24ef61c168aea663958b96ec852992bb5a4757
timeout change + some debug prints -> https://github.com/chetan-rathore/bsa-acs/commit/a6e8f3bdcd183eb6062c4d8b10d71ff1bc9563b4
Thanks,
ACS Team

Hi @chetan-rathore, here is the log from Bsa_ES_Fix.efi:
Bsa_ES_Fix.log

Thanks @semihalf-bernacki-grzegorz,
We will merge the changes into the mainline.

Hi @semihalf-bernacki-grzegorz @sunnywang-arm ,
The changes are updated as part of #15

Closing it as the fix got merged and verified. Thanks, @semihalf-bernacki-grzegor, @ndhillonm, and @chetan-rathore.


Jun 2021 hackday: add optional Tooltips for layers

PR Overview
Affects: WildCam Lab's Map Explorer (i.e. Gorongosa, Darien, and Kenya)
Context: ever since Wildwatch Kenya Lab, we realised that the "extra data layers" on the map (the ones that represent biomes/habitats, park boundaries, etc) actually have a lot of important information that can be surfaced.
This PR adds an optional Tooltip feature. By mousing over certain Features on those Extra Data Layers, a Tooltip will appear displaying all the information we have on that map feature.

A basic "bindTooltip" call has been added. This creates a default tooltip.
NOTE: for an extra data layer to have a tooltip, it needs to have its layer.style.interactive value set to true.
Required secondary functionality: now every time a layer is added/removed (via the built-in layer controls), the Data Layer (aka Camera Layer) is ALWAYS brought to front.

Otherwise, we wouldn't be able to click on the cameras. Any layer with layer.style.interactive=true will block interactions to layers beneath it.

Status
Technically ready, but functionally a WIP. This is practically complete but I'm thinking of adding one or two more flourishes, if I can spare the dev time:

the tooltip should appear close to the mouse, instead of centred on the feature, which can be unreliable at different levels of zoom.
when the tooltip appears on a Feature, that Feature should have some sort of highlight, e.g. bold borders.

If I can't spare the dev time, I'm going to have to get this merged as-is by Friday so I can deploy the latest Kenya map updates.

Update
I'm going to go ahead and merge this first before this branch goes stale; the planned enhancements will be taken care of in a separate PR.
TODO:

Improve UI for tooltips. i.e. "centre tooltip on mouse", not "centre on Feature"
Optional stylings for tooltip-ed Feature, i.e. bold borders
Add new maps for Kenya (this include correcting the incorrectly named "Kenya" layer)


Font Family Correction

The font family used is not alined with the actual font family of youtube plz fix this

Hey @Rishabh32700 , Can you please assign this task to me under Hacktoberfest tag . I would love to work on this.

hey @Rishabh32700 , can you please assign this task to me under Hacktoberfest Tag!

hey @Rishabh32700 , can you please assign this task to me under Hacktoberfest Tag! ,NST

/assigned


Enable dhcp6c auto SLA-LEN correct

Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I have searched the existing issues and I am convinced that mine is new.

dhcp6c relies on the user inputting the correct SLA-LEN so that it matches the prefix given by the ISP, if this does not occur then there are no IPv6 addresses applied to the tracking interfaces. We also 'guestimate' the prefix provided.
dhpc6c should be able to correct this for us, and use the supplied prefix to configure the interfaces, saving the user the hassle of having to find out what the ISP is providing.
This is a discussion, though I already have a dhcp6c version that does this. I have also added a new flag to the command line, 'a' which is 'auto' if this flag is set then dhcp6c will auto correct and configure the tracking interfaces automatically, disregarding what the user has set the SLA-LEN to, if 'a' is not set then dhcp6c will behave as it always did.
There's a PR for this in dhcp6c

Great idea!
This will only work though if the delegated prefix is shorter than the requested one, not the other way round, correct? Let's say you request a /56 and configure 8-bit subnet IDs (like a1, a2, b1, b2) for the tracking LAN interfaces. But the ISP only delegates a /60. The only thing we can do now is fail gracefully. Or am I missing something?

Well I tested with asking for a 60, my primary was set to dish out /56, that worked, so then I set my primary to give out /60 and asked for a /56. that worked too. Just thinking how I can test this to breaking point. I guess If add a couple more VLANs, I have three on the test already, so that would be 5 and set the primary to give out a /62 and see what happens... I'll go play.

You probably used shorter subnet IDs then (1 hex digit)?
If you use 8-bit subnet IDs (2 hex digits) and get a /60 delegation, there is just no way to merge prefix (60) + subnet ID (8) to a /64.

Or I could remove the second catch I put in where the prefix is smaller than requested, in which case dhcp6c will barf as it always did.

@maurice-w - why would you use a 2 digit prefix ID anyway, what benefit is there over using 1,2,3,4.... etc

I have an idea... might be a good one or not, but that's what discussions are for. If we make the 'a' flag an option in the interfaces settings section for dhcp6c,  if not set then it will allow you to set the IPv6 PrefixIDs for the tracking interfaces, if it is set then you have no choice and it will assign the prefixes in sequential order to the max allowed by the received prefix.
Thoughts?

OK, that works.. Added a little check that counts the number of prefixes given out and I override the SLA_ID, just going sequential. SO for example I have six VLANs, only first four are getting an address, the other two there's no address applied. That's probably the best the can be hoped for.

Indeed, that's what discussions are for!
Subnet IDs are widely used to represent existing information like VLAN IDs. That's actually more common than simple sequential numbering.
I don't think we can or should automate everything. If the delegated prefix length doesn't match the requested one but we can still fit the user-configured subnet IDs, by all means, let's do it. But if they don't fit, gracefully fail and let the user know.
My opinion about automatically assigning subnet IDs: No. Just no. That's fine for basic plug'n'play consumer routers, but not for something like OPNsense.

 You have the option, auto or not. If we leave auto off by default  then that's one way to keep your IDs, or Auto is on by default so it will work out of the box.
I could put a check in there to see if the ID supplied is valid for the prefix obtained and log a message if it isn't.
As a certain robot once said, more input needed.

I could put a check in there to see if the ID supplied is valid for the prefix obtained and log a message if it isn't.

Definitely a good idea! Maybe even an error message on the dashboard?
Whether we should try to attract more "idiots" is a more fundamental question. I'd say, if you have to rely on your firewall to auto-create an address plan for you, OPNsense is out of your league. Or is that to harsh?

A bit harsh, we all need to start somewhere. Some days I still do a 'doh' moment with IPv6 and I would guess there are many who are completely lost with IPv6.

Note sure how we could flag a warning in the GUI, perhaps I'd need to make dhcp6c write a file to temp. I'll leave that to those with better brains in that department to think on. TBH, I quiet like the idea of assigning all available prefixes and then skipping the interfaces our excess to the prefix size, Looks like this:

Not sure it does run 'OOTB' with any prefix, What's the default SLA-LEN, 8 ?
I take your point about a removed interface re-assigning prefixes, however the idea is not to force users to go the auto route, just to make sure it auto configures. Once that is done then the information is all there to allow the user to disable auto configure as they see fit. Need the thoughts of @fichtner and @AdSchellevis etc.

Not sure it does run 'OOTB' with any prefix, What's the default SLA-LEN, 8 ?

0, but doesn't really matter with SLA-LEN autocorrect in place (which I very much agree with). I didn't mean to say "works OOTB right now", but "doesn't require dynamic subnet ID assignment to work OOTB".
I just don't see the advantage of dynamic subnet ID assignment at runtime over 'assisted' static assignment at the time of configuration. Even as an optional setting. Agreed, more input required or we'll start going in circles. :)


WIP: mc_pos_control: triplet handling for mc

currently the handling of the triplet setpoint in mc_pos_control_main is rather confusing. In addition, the coupling of vtol to mc makes it also hard to distinguish what is important for vtol and what not. Following the discussion from here #6681, the triplet handling for simple point to point motions are implemented too complicated. Setting the desired current setpoint as the desired position is enough. There should not be any additional slewrate applied to the position. It is completely fine to have a position setpoint step. The velocity limit is applied  already at the output of the position controller.
For the handling of waypoints, I can see the benefit of incorporating the previous and next setpoint, but I would completely split the implementation depending on vtol and normal mc (maybe that is already the case, but it is not obvious to me). Also, instead of having this position setpoint adjustment, I suggest to apply normal position control perpendicular to the line between previous and current setpoint and velocity feedforward in the velocity controller. the velocity feedforward vector can be computed from bezier curve which also incorporates max acceleration. I know how to do trajectory based on bezier curve for mc, but not for vtol and that is another reason why I would split the implementation entirely. obviously with vtol I mean when in fixedwing mode.

I don't think I follow what you're saying about VTOL.
Currently VTOL is running both MC (pos + att) and FW (pos + att) at all times and the VTOL controller is responsible for engaging/disengaging MC or FW actuator outputs and handling the transition between VTOL modes. In MC mode a VTOL is a regular multicopter, and likewise for FW.
MC pos is only aware of the existence of VTOL so that things are properly reset right before transitioning from FW to MC.

I guess what confused me is this line https://github.com/PX4/Firmware/blob/master/src/modules/mc_pos_control/mc_pos_control_main.cpp#L1417

Centralize any VTOL related init in the same spot? https://github.com/PX4/Firmware/blob/master/src/modules/mc_pos_control/mc_pos_control_main.cpp#L2330

I have tested the next scenario, changing the simulator altitude to negative value and seen critical bad behaviors.
I have done that by changing the line:
static const double alt_zurich =
On some cases, the drone start to change altitude rapidly after its seems stabilized
On some cases its takeoff, and doesn't climb, just stay close to ground (for example, for -50m)
-different altitude seem to behave different, 200 seems to be ok for some reason

I also noticed that on landing command POSITION_TARGET_LOCAL_NED.z is set to global altitude of home position. On the previous behavior it wasn't.
What If landing altitude is not the takeoff alt?

@matanhavi: thanks for your testing. indeed, it needs more improvements than just the slewrate in positiont setpoint. by the way, if you have time you are more than welcomed to fix whatever you see is wrong during your testing.
about the second issue:

...POSITION_TARGET_LOCAL_NED.z is set to global altitude of home position
currently the altitude for landing is only used to decrease the velocity in z direction, but not to detect the landing. it is true that the vehicle will also fly slower in the z direction if you do a takeoff in the mountains and then fly downwards toward the valley. This problem could be resolved if the vehicle would have a reliable source of height detection relative to ground.

@dagar:
Can you explain to me the meaning of rotary wing? what is this flag doing in mc pos control? is that only needed for transition?

Rotary wing = MC
Always true for a regular multicopter, always false for a regular plane.
For VTOL it's true in regular hover, false in fixed wing, and true during
transitions.
Outside of VTOL you might have it to stop the MC position controller from
running/publishing if someone started it on a plane.
On Mon, Mar 13, 2017 at 12:59 PM, Dennis Mannhart<EMAIL_ADDRESS>wrote:

@dagar https://github.com/dagar:
Can you explain to me the meaning of rotary wing? what is this flag doing
in mc pos control? is that only needed for transition?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PX4/Firmware/pull/6717#issuecomment-286171999, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFK6HtMiMRqZ-0BRJTrHpEbUGlqc2_Dks5rlXYMgaJpZM4MSMyl
.

@Stifael can you better explain what you expect the bezier curve to do?
Is it improvement of current position logic?
or a "fix", something is "wrong" in current logic?
or it might be "option"? if it is, it should not fully replace current logic.
about POSITION_TARGET_LOCAL_NED.z is set to global altitude of home position, I know that the z sp isn't the position where the motor will stop. I did mention that the value is a bit different that previous behavior, and it need to be noticed.
I also recommend you to test any logic related to position sp around the equator, Greenwich, and +-180deg.
and because its mathematical logic, check MCU %, make sure they doesn't rise too much. @dagar, are there any cpu/ram test between versions?

It will replace the auto logic, but im working on a simpler version to at least fix some of the obvious bugs.
The purpose of the bezier is to replace position setpoint slewrate, which has some fundamental problems due to its feedforward implementation. There is also no reason why you need to generate setpoints on 2d. The bezier curve is one method to compute trajectory from which position, velocity and acceleration can be derived from depending on current state. However, this logic should in fact be implemented a layer up in the navigator, which requires even more changes. Thats why I started to put it in the mc for now.

@matanhavi: about altitude
If battery is low, the vehicle cannot exceed a certain limit. What was the battery status in sitl?


[FLINK-3292]Fix for Bug in flink-jdbc. Not all JDBC drivers supported

Hello,
Here is the fix for issue FLINK-3292. Kindly review & merge.
Thanks & regards,
Subhobrata

is there any way to check which createStatement method is supported? what happens if you use the wrong statement?

Hello zentol,
I made changes according to your post. Kindly review now.

Hello zentol,
To answer your initial question,
is there any way to check which createStatement method is supported? what happens if you use the wrong statement?

I went into the Jdbc driver code & found that both methods are supported, however, with the ones defaulted in the code,
ResultSet.TYPE_SCROLL_INSENSITIVE
ResultSet.CONCUR_READ_ONLY
I got this error,
java.sql.SQLException: Result set type is TYPE_FORWARD_ONLY

I think both variants should be supported & it is up to the api user to specify which variant they should go for. Hence, this PR. Kindly let me know if the changes in code are fine now.

In that case i would suggest the following:

change the default values to TYPE_FORWARD_ONLY and CONCUR_READ_ONLY to be in line with createStatement()
-allow the user to set either one of these

Hello zentol,
Can you check the code changes now? The changes look exactly similar to what you suggest. Can you let me know if this can be merged? or do I provide a new PR?

if you can revert the changes to the imports it's good to merge IMO.
you don't have to create a new PR. you can squash the commits if you wish, otherwise I'll do it when merging.

Hello zentol,
I have squashed all changes. Can you please merge now?

Hello zentol,
I dont see any updates from your side. Can you kindly let me know if this PR is good to be merged so that I can close the issue?

looks good. +1 to merge.

Thank for the contribution and review!
Will merge this...


[CT-1448] Designer-designed default color scheme for all DAG nodes

Describe the feature
For Python models, it was requested (and then contributed by a community member at the Coalesce hackathon!)
I shouldn't be anywhere near deciding on colors for nodes in the DAG, it'd be goofy. designers should come up with something that looks good and is accessible given their expertise in the area. opening this issue for tracking
TODO: enumerate all the things to have default colors for
Describe alternatives you've considered
N/A
Additional context
customer node colors were added in v1.3
Who will this benefit?
all users of dbt docs. Cloud IDE users in the lineage tab once we reuse the same color scheme there as well
Are you interested in contributing this feature?
maybe! we can simply follow this PR: https://github.com/dbt-labs/dbt-docs/pull/324

@lostmygithubaccount I think we have this already in https://github.com/dbt-labs/dbt-docs/issues/325


Content Discussion: Moving off sam deploy

Again, thanks for all your continued work on this gem + the support elements.
Precursor - I think this should be pure content as opposed to part of the boilerplate. It's an unnecessary complexity and more sugar on top.
One great piece that helps towards that end is using AutoPublishAlias: live for blue/green deployments.
Currently, running bin/_deploy uses sam deploy which waits for the auto-built CodeDeploy to finish its job. In a Canary or a Linear traffic swap situation, the shortest increment of 1 minute but really should be shifting traffic in 3 or 5 minute increments resulting in a 6+ minute deploy cycle.
This means we're paying for Github Actions or CodeBuild to wait while CodeDeploy itself is free. Trivial for small applications, but adds up in business environments.
Personally I'm not there yet. I don't fully understand CodeDeploy, AutoPublishAlias is magical, so happy to learn/discuss together if this makes sense.

Turns out it wasn't so bad, I just needed to follow what other CodeBuild SAM users are doing.
General Steps:

sam package output package.yml or any template.yml is spit out as an artifact from the CodeBuild stage.
CodePipeline consumes the template and orchestrates CloudFormation to process that file.

Step 2 offloads/acts-as the sam deploy to CloudFormation directly. Now that CodePipeline/CloudFormation is handling the AutoPublishAlias: live I can let it take as long as it needs.
Let me know if there's a section to be had. Happy to help where it makes sense.

Hey @nitsujri Thanks, I was out the week prior and I'm working thru some old emails. Always appreciate your feedback, some thoughts.
First, let me share that I am not a fan of CodePipeline. For me the complexity of managing that service for deploys is not outweighed by the simplicity of systems like GitHub Actions or CircleCI. That said, I've not yet had the need for an advanced canary deploy option offered by SAM & CodePipeline. If you do, that is fine and why I think a linked blog post from the Lamby site on said topic (or maybe guide) would be helpful. So totally open to doing that.
Second, here is some background on AutoPublishAlias: live as to why it is in the starter and mentioned in that Cold Start issue you linked. I have found that when deploying Rails applications that take a descent amount of traffic that the deploy alias will soften the downstream blow of users hitting that application. This happens even when you do not use provisioned concurrency. In fact, the CI/CD system will finish the sam deploy even before traffic is fully rolled over. Often times it takes a minute or two to switch the alias. When you combine that with how it helps soften rollouts in a clean way with less errors... that's all the deploy method I need.
Again, for me, canaries are kind of moot. We suss out their need in staging. Not to say they are not useful for some and I love the ability. But not the engineering to get there yet because... Code(Deploy|Pipeline) is not super easy. Hence, guide help please.

For me the complexity of managing that service for deploys is not outweighed by the simplicity of systems like GitHub Actions or CircleCI.

100% agreed. Having fully gone through the CodePipeline + CodeBuild, I think in the name of focus + ease of upstart, adding this content section is not in the best interest of the project.
I'll close this for now.


Upload all JS files, not just the maps

Sentry takes advantage of the minified JS files in addition to the maps, as noted in their documentation:

You will want to upload all dist files (i.e. the minified/shipped JS), the referenced sourcemaps, and the files that those sourcemaps point to.

With more colour later on:

Note: You dont have to upload the source files (ref’d by sourcemaps), but without them the grouping algorithm will not be as strong, and the UI will not show any contextual source.

I made the smallest possible change to allow the functionality and can confirm that it's working on my Heroku builds. You can test using my forked buildpack. 😄

Hmm, didn't know about that... Looks interesting!
However (and nitpicking a little):

would it be possible to make this configurable using an environment variable?
...and to white-list the extensions (*.js *.js.map) rather than to use a wildcard that would also match .json files?
Thanks!


Refactor/center remaining capacity formatting

Refactoring Code Base to isolate utility functions and output Remaining seats data in more structured format
This PR Deals with better organizing utility function to utils.py that isolates utility functions from main functions.
Also, Works around representing remaining seats of centers in better format along with the center name and center code

It's a good step but it would be better if there was issue created before the PR so that contributors could discussion about it. Making things more transparent and organized.
Otherwise, we would end up with either conflicts or duplication that will require more manual work.


Documentation : improve installation information in README

In the README file

[x] remove the "chevron" prompt so that lines can be copy-pasted
[x] add the git clone command
[ ] decide whether python=3.10 is needed. We seem to be CICD-ing for python-version: ['3.9', '3.10', '3.11', '3.12'] : as a matter of fact, info on what systems the CICD test the code is in order.
[x] mention that the cxx-compiler is optional, but for mac
[ ] add some information about the lephare-data repo : what happens  1/by default (no LEPHAREDIR) 2/ if LEPHAREDIR is set 3/ if one wants the gitlab legacy setup
[x] add comment about the unit tests, always good to be able to run them in developer mode
[ ] explain what documentation is available, and for developers how to build and check it locally

Modifying the README.md, I realize that the index.rst has been copied from the README.md with the toctree added, and thus does not capture the changes in README.md. It would be better for the README.md to be the basis for index.rst. One way to do that is to remove index.rst, create toctree with only the toctree block, and then in the docs Makefile add :
rm index.rst
pandoc --from=markdown --to=rst --output=readme.rst ../README.md
cat readme.rst toctree.rst > index.rst

Would that work on mac?

In order to build the documentation, I needed to install a few packages that were not there :
pip install sphinx-autoapi
pip install nbsphinx
pip install sphinx_copybutton
pip install sphinx_rtd_theme

which were in the requirements.txt, but there was no doc reminding me to first do pip install -r requirements.txt  before trying to build the doc.

In order to build the documentation, I needed to install a few packages that were not there :
pip install sphinx-autoapi
pip install nbsphinx
pip install sphinx_copybutton
pip install sphinx_rtd_theme

which were in the requirements.txt, but there was no doc reminding me to first do pip install -r requirements.txt before trying to build the doc.

committed

Modifying the README.md, I realize that the index.rst has been copied from the README.md with the toctree added, and thus does not capture the changes in README.md. It would be better for the README.md to be the basis for index.rst. One way to do that is to remove index.rst, create toctree with only the toctree block, and then in the docs Makefile add :
rm index.rst
pandoc --from=markdown --to=rst --output=readme.rst ../README.md
cat readme.rst toctree.rst > index.rst

where toctree.rst is just :

.. toctree::
   :hidden:

   Home page <self>
   Notebooks <notebooks>
   Legacy installation <legacy_install>
   Keywords <keywords>
   C API Reference <doxygen_output/c_lib/library_root>
   Python API Reference <autoapi/index>

Would that work on mac @raphaelshirley ?

committed

This has been captured in the current doc.


Use pflag instead of flag

As far as I can tell, we currently use the standard Go flag package. This has as a result that all options which actually have to be a StringArray or StringSlice, are currently just 'comma separated lists', which are just very long Strings.
This brings problems when trying to create an easy readable configuration, because the lists can't be listed as arrays. In for example the ES-plugin, this works:
es:
  server-urls: "server1:9200, server2:9200"

But this doesn't:
es:
  server-urls:
    - "server1:9200"
    - "server2:9200"

pflag is a drop-in replacement for the standard flag package (and it's already used by Cobra and Viper) and would provide us to do both, leaving compatibility with the current way (i.e. the single String) intact whilst also providing the second option of declaring lists/arrays.
I see the Golang flags are eventually already parsed to pflags in pkg/config/config.go on line 38, but can't we find a way to also be able to pass *pflag.FlagSet instead of only *flag.FlagSet?
That's probably going to require a lot of refactoring and I'm not even sure if it's possible (and viable) at the moment. Any comments, remarks and/or idea's?

If the change is backward compatible +1 from my side.
Would you be interested in submitting a PR?

The reason we chose to use stdlib flag pkg was to avoid external dependencies in much of the code base, since only main() functions are currently aware of pflag. However, now we have many places in the code that accept Viper directly, so depending on pflag wouldn't be such a stretch either.
Having said that, it's a pretty large change, I would assume, I am quite skeptical it's worth all  the churn for a single type of property.

Having said that, it's a pretty large change, I would assume, I am quite skeptical it's worth all the churn for a single type of property.

That's exactly what I was thinking too, and it's the reason I opened this issue; discussion on the matter. I do think it's worth the effort. Pflag has some types flag doesn't have, e.g. IP's, so we could use those instead of parsing them ourselves.

Would you be interested in submitting a PR?

I am, but I currently don't have the time. I'll take another look at it in two weeks, should have more time by then. Any suggestions on where to start?
Probably also best to make a list of components that need to be changed, so we can keep track of them.

Hey, I would like to help if no one has planned to take this up, but I am new to this community and might need guidance at some points.

I am going to close this, there are too many changes for very little benefit.
./cmd/tracegen/main.go:18:	"flag"
./cmd/flags/service.go:18:	"flag"
./cmd/flags/flags.go:18:	"flag"
./cmd/flags/admin.go:19:	"flag"
./cmd/ingester/app/flags.go:18:	"flag"
./cmd/collector/app/builder/builder_flags.go:18:	"flag"
./cmd/agent/app/flags_test.go:18:	"flag"
./cmd/agent/app/reporter/flags_test.go:18:	"flag"
./cmd/agent/app/reporter/grpc/flags_test.go:18:	"flag"
./cmd/agent/app/reporter/grpc/flags.go:18:	"flag"
./cmd/agent/app/reporter/flags.go:18:	"flag"
./cmd/agent/app/reporter/tchannel/flags_test.go:18:	"flag"
./cmd/agent/app/reporter/tchannel/flags.go:18:	"flag"
./cmd/agent/app/flags.go:18:	"flag"
./cmd/agent/app/builder_test.go:19:	"flag"
./cmd/docs/command.go:18:	"flag"
./cmd/query/app/flags.go:18:	"flag"
./plugin/sampling/strategystore/factory.go:18:	"flag"
./plugin/sampling/strategystore/static/options.go:18:	"flag"
./plugin/sampling/strategystore/static/factory.go:18:	"flag"
./plugin/sampling/strategystore/adaptive/options.go:18:	"flag"
./plugin/sampling/strategystore/adaptive/factory.go:18:	"flag"
./plugin/sampling/strategystore/factory_test.go:19:	"flag"
./plugin/configurable.go:18:	"flag"
./plugin/storage/memory/options.go:18:	"flag"
./plugin/storage/memory/factory.go:18:	"flag"
./plugin/storage/grpc/options.go:18:	"flag"
./plugin/storage/grpc/factory.go:18:	"flag"
./plugin/storage/grpc/README.md:40:        "flag"
./plugin/storage/factory.go:18:	"flag"
./plugin/storage/kafka/options.go:18:	"flag"
./plugin/storage/kafka/factory.go:19:	"flag"
./plugin/storage/cassandra/options.go:18:	"flag"
./plugin/storage/cassandra/factory.go:18:	"flag"
./plugin/storage/es/options.go:18:	"flag"
./plugin/storage/es/factory.go:19:	"flag"
./plugin/storage/badger/options.go:18:	"flag"
./plugin/storage/badger/factory.go:19:	"flag"
./plugin/storage/factory_test.go:19:	"flag"
./internal/tracegen/config.go:18:	"flag"
./examples/memstore-plugin/main.go:18:	"flag"
./pkg/metrics/builder.go:20:	"flag"
./pkg/metrics/builder_test.go:19:	"flag"
./pkg/config/config.go:18:	"flag"
./pkg/config/config_test.go:18:	"flag"
./pkg/kafka/auth/options.go:18:	"flag"


Bump to v3.0.1

coverage: 89.491% (+0.02%) from 89.47%
when pulling fff7765dca1e263d449a4a5568e1b804025546a2 on bump_v301
into dfcf36bfec02b3c7342d01d2eb816533a5dc428c on master.


Allow trailing comma for multiline parameters and literals

As trailing coma is one of finished proposals for ECMAScript 2017 specification, it should be allowed in coding standards.
What does it do?
It allows this:
function (
  param1,
  param2,
)

[
  item1,
  item2,
]

{
  prop1,
  prop2,
}

But it doesn't force you to use it, so this will pass to:
function (
  param1,
  param2
)

But whis will not pass:
function(param1, param2,)

Why is it good idea? You don't need to change 2 lines to add 1 param (cleaner diffs) and when you have multiple literals it is much easier to sort them aplhabeticaly 🙃
More here: https://jeffmo.github.io/es-trailing-function-commas/

Looks good, @jzavisek will be happy.

I'll publish this as a minor release (the rule is fixable and previous code will continue to pass lint checks) once I'm back from holidays.
Thanks!

@arnostpleskot, @Alaneor  👏

Released in 5.1.0 🎉


vim 8.2

doesn't work cal funcref()

let funcref = ...
noremap <Plug> .. :cal <SID>funcref()<CR>

de graded

idemenu exe
nmap -> quickhl
fuzzy callback in fml -> no echo 'jump to xx'

https://github.com/serna37/vim/commit/2c45992bda754170d2e09c924a804ee8401ba594


MBL-1016: Create Paginator for pagination in SwiftUI/Combine

📲 What
This adds a new class, Paginator, inspired by our existing paginate method. Paginator is designed to fill the same functionality, but in a way that integrates neatly with SwiftUI and Combine-based code.
🤔 Why
When we move over to SwiftUI and Combine, we'll need an easy way to page through long lists of API results.
🛠 How
Originally I wanted to create a 1-for-1 re-write of paginate in Combine.  However, I spent a significant amount of time trying to duplicate it, and was unable to make any traction. Writing stateless functional code is significantly more challenging (to me at least), and I don't think the benefits of its elegance outweigh the difficult I had modifying and understanding it. Throw in a few more variables, and I think Paginator will be easier to modify and understand in the long run.
paginate came with a great deal of tests, which I used to understand its behavior, and that became my inspiration for writing Paginator.
Note that paginate includes some niche features - you can pass in a customer concater for example - that I chose not to implement. If we need those, we can implement them in the future. Most of those niche features are only used on one or two screens.
👀 See
I created PaginationExampleView and PaginationExampleViewModel as examples (and my own test bed!) for how Paginator could be used in the app. You can try it yourself by throwing in this somewhere in the app: presentViewController(UIHostingController(rootView: PaginationExampleView(), animated: true))
https://github.com/kickstarter/ios-oss/assets/146007185/14e06c76-3356-4b4a-8a4a-9b47b19e97c3

          1 Warning

:warning:
Big PR

  Generated by :no_entry_sign: Danger


fix: always use available context

:tada: This PR is included in version 1.9.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Using main.dart

How can I configure flutter_flavorizr to set the target for iOS as main.dart and not main_.dart?

Closing this as a fix has been provided here:https://github.com/AngeloAvv/flutter_flavorizr/issues/120


reword boilerplate description text wording

Description
We need to update our boilerplate text for describing pycsw.
Environment
NA
Steps to Reproduce
NA
Additional Information
Areas to update:

[ ] docs
[ ] website
[ ] packaging
[ ] social media (Google+, Wikipedia)
[ ] other?

cc @ricardogsilva
Slight changes proposed.  The only change is to the 5th sentence which elaborates more on the various APIs we support:
pycsw is an OGC CSW server implementation written in Python.

pycsw fully implements the OpenGIS Catalogue Service Implementation Specification (Catalogue Service for the Web). Initial development started in 2010 (more formally announced in 2011). The project is certified OGC Compliant, and is an OGC Reference Implementation.

pycsw allows for the publishing and discovery of geospatial metadata. Existing repositories of geospatial metadata can also be exposed via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU), providing a standards-based metadata and catalogue component of spatial data infrastructures.

pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X).

+1

Thanks.
@kalxas:

can you update the 'About Community' section on Google+?
who's the best person to ask to update this in Debian packaging?

I will handle both

Google+ done
Debian:
https://anonscm.debian.org/cgit/pkg-grass/pycsw.git/commit/?id=c059555b40af84ed8ca786aa8332385bfaa9ce37

what else in packaging? Pypi?

PyPI will get automagically updated when we publish the release.

ok, then we are done.


Fixes to integration-test harness + fix a double-deployment bug

The generated app will now be built using the local maven repository set by
the system property "maven.repo.local".
The integration tests are now configured to build the generated app
with the same maven local repo as the integration tests use themselves.
Avoid calling buildAndDeploy multiple times by only dispatching to
the super class a single time. This caused integration tests to intermittently fail
from the generated war file being deployed multiple times.

jenkins, retest this

+1


Consumer stop consuming after several messages consumed

Kafka-node: 4.1.3
const client = new kafka.KafkaClient(config.kafka.client);
const consumer = new Consumer(client, [{topic: config.kafka.topic}], {
    groupId: config.kafka.groupId, autoCommit: true,
});
consumer.on('message', async (message) => {
    // do something
});
consumer.on('error', (err) => {
    // do something
});

The consumer works ok at initial time, but it stops after several messages consumed. Sometimes it will take a few days to stop, and sometimes only one day. This has been bothering me for a long time. Please help me out for this.

Why you are using async in the callback upon receiving message event -- callback is inherently async? Try after removing async.
Also, try using ConsumerGroup like below:
            consumerGroups[groupId] = {};
            consumerGroups[groupId] = new kafka.ConsumerGroup(consumerOptions, topic);
            consumerGroups[groupId].on('error', (error) => {
                    // do something
            });
            consumerGroups[groupId].on('message', (message) => {
                    // do something
            });

@sharmaadarsh563 I am wrong with async in the callback used and have removed it. Using ConsumerGroup is a good idea, thanks a million.


Ministry user processes learning disability bursary

[placeholder ticket]
Context: PD related
Receiving disability services

No longer a requirement. Absorbed by #1053.


Correct ModifyColumn SQL syntax.

The generated SQL for ModifyColumn was:
ALTER TABLE "tablename" MODIFY "columname" type
But should have been:
ALTER TABLE "tablename" ALTER COLUMN "columname" TYPE type
since Modify does not seem to be entirely compatible with all Engines
Make sure these boxes checked before submitting your pull request.

[] Do only one thing
[] No API-breaking changes
[] New code/logic commented & tested
[] Write good commit message, try to squash your commits into a single one
[] Run ./build.sh in gh-pages branch for document changes

For significant changes like big bug fixes, new features, please open an issue to make a agreement on an implementation design/plan first before starting it.
Thank you.
What did this pull request do?

Hi @perrito666
I have written a test for ModifyColumn in branch https://github.com/jinzhu/gorm/tree/perrito666-master, seems your changes haven't pass the test, could you look at it?

Ok the issue is actually with Sqlite, it does not fully support ALTER TABLE it does not support specifically changing a column type (when reverting my change you will notice the error persists) https://sqlite.org/lang_altertable.html is there a way to tell the test to only run when using certain dialects?

Hi @perrito666
You could use https://github.com/jinzhu/gorm/blob/master/create_test.go#L97 , could you take care of it?

Yes I’ll get it sorted and up in the PR tomorrow morning it’s a bit late
here, thank you for the test and sorry I did not include one with the PR
originally.
On Tue, 19 Sep 2017 at 22:30 Jinzhu<EMAIL_ADDRESS>wrote:

Hi @perrito666 https://github.com/perrito666
You could use
https://github.com/jinzhu/gorm/blob/master/create_test.go#L97 , could you
take care of it?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jinzhu/gorm/pull/1614#issuecomment-330719085, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AArtT8u4S70fOYFRS1Y_xrtTlZso02qxks5skGqvgaJpZM4PZkwF
.

There you go, I made it to check explicitly the dialects that fully support ALTER TABLE with ALTER COLUMN let me know if I should fix something else on the PR, cheers.


Update to tokio 0.3

I'm so disconnected from the async ecosystem nowadays it probably doesn't make sense for this crate to live under me any more, if you'd like should I transfer it to you? (the repo, that is)

Sure, works for me.

Oh I think you'll need to delete your own fork, even renaming it I think doesn't work.

Done! I'm surprised this PR didn't get auto-closed.

👍


Add failing test.

just tested it and there's still a problem. when marking a todo as complete it marks the next one too, unless it's the last one in the list

Closing for now, feel free to reopen when ready


AsyncHttpClient using DefaultChannelId directly

gatling-http module dependency: asynchttpclient , has a netty class 'DefaultChannelId' directly copied into their library!
This should not be there, as this library should need to use netty as a dependency. I've seen that latest alpha versions of this library: org.asynchttpclient:async-http-client:2.1.0-alphaXX don't have this netty class copied into their src.
Thanks!

AHC 2.0 has a backport of Netty's DefaultChannelId because it needs it for the DNC codec backport. This isn't supposed to conflict with Netty as DefaultChannelId only exists in Netty 4.1 and AHC 2.0 and Gatling 2 target Netty 4.0.
AHC 2.1 and Gatling 3 will target Netty 4.1 and the DefaultChannelId has of course been removed.


[SPARK-45827][SQL] Variant fixes with codegen and vectorized reader disabled

What changes were proposed in this pull request?
Fix two issues with the new Variant type:

In InterpretedUnsafeProjection, define element size to be 8, since Variant has variable length, so it is categorized as a reference type, which always has size 8. This only manifests as an issue when codegen is disabled and an array or struct contains Variant values.

Define and use a ParquetGroupConverter for Variant. The previous tests used the vectorized reader, so this issue didn't manifest.

Why are the changes needed?
Fixes crashes when Variant is used.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Added a unit test that writes and reads an array of Variant values with codegen and the vectorized reader disabled. Reverting either of the two fixes causes the test to fail.
Was this patch authored or co-authored using generative AI tooling?
No.

thanks, merging to master!


Finder ->append failing silently

an issue you encountered yourself already here https://github.com/php-stubs/woocommerce-stubs/blob/master/finder.php
Symfony ->append( '/path/to/file.php' ) should work fine
but when doing this the generator fails silently.
Instead one has to workaround with:
->append(
        \StubsGenerator\Finder::create()
            ->in(['source/woocommerce'])
            ->files()
            ->depth('< 1')
            ->path('woocommerce.php')
    )

which is a pain if you have an array of files you want to append

Hello @kkmuffme!
This repo is for solving a fixed set of problmes upstream.
Please see https://github.com/symfony/finder/blob/5.4/Finder.php#L170 and solve it yourself.


Dedup thrashing spellcheck calls

In certain scenarios such as when you move the selection over a word quickly, Electron will call out to the spellcheck handler over and over for the same word.
TODO:

[x] Code Review

:gem: