bytes: IndexAny is slower than a naive implementation in some cases

What version of Go are you using (go version)?

$ go version
go version go1.20.3 windows/amd64

What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Kroum\AppData\Local\go-build
set GOENV=C:\Users\Kroum\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=D:\MyPrograms\golibs\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=D:\MyPrograms\golibs
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.3
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=D:\MyDocuments\Programming\go\test\bench\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=D:\Temp\System\Kroum\go-build2837998563=/tmp/go-build -gno-record-gcc-switches

What did you do?
I consider the following naive implementation of        bytes.IndexAny
func IndexAny(s []byte, chars string) int {
	min := -1
	for _, c := range chars {
		if i := bytes.IndexRune(s, c); uint(i) < uint(min) {
			min = i
		}
	}
	return min
}

I benchmark it for 10 delimiters on a data where the first delimiter appears at position 1000, like this
package indexany

import (
	"bytes"
	"strings"
	"testing"
)

func IndexAny(s []byte, chars string) int {
	min := -1
	for _, c := range chars {
		if i := bytes.IndexRune(s, c); uint(i) < uint(min) {
			min = i
		}
	}
	return min
}

const (
	t             = "🚆"
	firstchar     = 1000
	numdelimiters = 10
)

var (
	data       []byte
	separators string
)

// create the data and the separators fot the benchmark
func init() {
	// data = "xxx...🚆"
	data = bytes.Repeat([]byte("x"), firstchar)
	data = append(data, t...)
	// separators = "yyy...🚆"
	separators = strings.Repeat("y", numdelimiters)
	separators = separators + t
}

func BenchmarkNaiveIndexAny(b *testing.B) {
	for i := 0; i < b.N; i++ {
		IndexAny(data, separators)
	}
}

func BenchmarkBytesIndexAny(b *testing.B) {
	for i := 0; i < b.N; i++ {
		bytes.IndexAny(data, separators)
	}
}

What did you expect to see?
That bytes.IndexAny is faster then this "naive" IndexAny.
What did you see instead?
$ go test -benchmem -bench=. .
goos: windows
goarch: amd64
pkg: kpym.github.com/gobench/indexany
cpu: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
BenchmarkNaiveIndexAny-8         3033126               522.9 ns/op             0 B/op          0 allocs/op
BenchmarkBytesIndexAny-8          223227              7233 ns/op               0 B/op          0 allocs/op
PASS
ok      kpym.github.com/gobench/indexany        4.325s

So this "naive" implementation is more than 10 times faster than the standard bytes.IndexAny.
Am I doing something wrong ? Is this relevant ?
Remark : If we take a lot of delimiters (1000) the standard function is 30% better (on my computer).

CC @ianlancetaylor via https://dev.golang.org/owners

Any algorithm is going to have cases where it is better or worse.  The question is how to get good average performance for typical cases.
How does your implementation fare on the benchmarks that are already in the bytes package?
The current bytes.IndexAny algorithm is somewhat optimized for the case where the characters being searched for (chars) are all ASCII.  It may be that we can do better for cases that is not true.

@ianlancetaylor

Any algorithm is going to have cases where it is better or worse. The question is how to get good average performance for typical cases.

I agree. When I searched for bytes.IndexAny on GitHub and in the standard library, it looks like that it is used in almost all the cases :

with between 2 and 4 characters (delimiters),
with characters that are ASCII (like "\r\n", " \t\r\n" or " ,;\r\n").

How does your implementation fare on the benchmarks that are already in the bytes package?

I didn't find any benchmarks for indexAny in the standard library. Am I missing something?
But in all benchmarks with a small number of separators, even if they are all ASCII, the naive version is (much) faster. Except when the first occurrence is really close to the beginning (less than 4). But in this case, they're very similar. And as the distance from the first occurrence increases, so does the advantage of the naive version.

The current bytes.IndexAny algorithm is somewhat optimized for the case where the characters being searched for (chars) are all ASCII. It may be that we can do better for cases that is not true.

I'm really surprised by what you say, because I discovered this naive version precisely because I wanted to treat only the case of ASCII separators, and I was surprised that in indexAny the separator set is "string" and not []byte. And indexAnyByte (or indexByteAny) is missing!
If we only want to handle ASCII delimiters, the following []byte version is even faster
func IndexAnyByte(s, chars []byte) int {
	min := -1
	for _, c := range chars {
		if i := bytes.IndexByte(s, c); uint(i) < uint(min) {
			min = i
		}
	}
	return min
}

I really don't understand the choice of the string parameter in the bytes.IndexAny (vs. strings.IndexAny).
Why bother with all the Unicode complexity in this case?
Anyway, that's not the point of this issue, and we can't change it now (for backward compatibility reasons).

But in all benchmarks with a small number of separators, even if they are all ASCII, the naive version is (much) faster.

This surprises me.  The current algorithm only looks through the string once.  Do you have an explanation for why it is faster to look through it multiple times?  Is this a memory caching effect?

I got curious so I'm running the benchmarks in the bytes package with both implementations. I'll post the results back here.

Here's what I got:
bytes $ benchstat baseline.txt naive.txt
name                    old time/op  new time/op    delta
IndexAnyASCII/1:1-8     6.04ns ± 1%    8.20ns ± 1%    +35.81%  (p=0.008 n=5+5)
IndexAnyASCII/1:2-8     6.03ns ± 1%   13.19ns ± 0%   +118.85%  (p=0.008 n=5+5)
IndexAnyASCII/1:4-8     6.02ns ± 1%   23.26ns ± 1%   +286.17%  (p=0.008 n=5+5)
IndexAnyASCII/1:8-8     6.02ns ± 0%   43.77ns ± 1%   +627.49%  (p=0.008 n=5+5)
IndexAnyASCII/1:16-8    6.29ns ± 1%   84.67ns ± 1%  +1245.89%  (p=0.008 n=5+5)
IndexAnyASCII/1:32-8    7.56ns ± 2%  177.14ns ± 0%  +2243.25%  (p=0.008 n=5+5)
IndexAnyASCII/1:64-8    6.00ns ± 1%  338.48ns ± 1%  +5544.53%  (p=0.008 n=5+5)
IndexAnyASCII/16:1-8    7.58ns ± 1%    8.47ns ± 1%    +11.83%  (p=0.008 n=5+5)
IndexAnyASCII/16:2-8    19.2ns ± 1%    13.8ns ± 0%    -28.51%  (p=0.008 n=5+5)
IndexAnyASCII/16:4-8    22.6ns ± 1%    24.4ns ± 0%     +8.18%  (p=0.008 n=5+5)
IndexAnyASCII/16:8-8    28.4ns ± 1%    45.7ns ± 1%    +61.00%  (p=0.008 n=5+5)
IndexAnyASCII/16:16-8   41.8ns ± 0%    88.6ns ± 0%   +111.92%  (p=0.008 n=5+5)
IndexAnyASCII/16:32-8   57.6ns ± 0%   175.7ns ± 2%   +204.86%  (p=0.008 n=5+5)
IndexAnyASCII/16:64-8    145ns ± 1%     353ns ± 0%   +143.91%  (p=0.008 n=5+5)
IndexAnyASCII/256:1-8   9.64ns ± 1%   10.44ns ± 1%     +8.39%  (p=0.008 n=5+5)
IndexAnyASCII/256:2-8    182ns ± 1%      18ns ± 1%    -90.27%  (p=0.008 n=5+5)
IndexAnyASCII/256:4-8    188ns ± 1%      32ns ± 1%    -82.74%  (p=0.008 n=5+5)
IndexAnyASCII/256:8-8    198ns ± 0%      62ns ± 2%    -68.74%  (p=0.008 n=5+5)
IndexAnyASCII/256:16-8   207ns ± 4%     130ns ± 1%    -37.37%  (p=0.008 n=5+5)
IndexAnyASCII/256:32-8   216ns ± 1%     249ns ± 1%    +15.24%  (p=0.008 n=5+5)
IndexAnyASCII/256:64-8   306ns ± 2%     483ns ± 2%    +57.72%  (p=0.008 n=5+5)
IndexAnyUTF8/1:1-8      5.67ns ± 1%   12.37ns ± 2%   +118.07%  (p=0.008 n=5+5)
IndexAnyUTF8/1:2-8      5.69ns ± 0%   21.32ns ± 0%   +274.51%  (p=0.008 n=5+5)
IndexAnyUTF8/1:4-8      5.67ns ± 0%   23.92ns ± 1%   +321.79%  (p=0.008 n=5+5)
IndexAnyUTF8/1:8-8      5.66ns ± 1%   44.28ns ± 1%   +682.79%  (p=0.008 n=5+5)
IndexAnyUTF8/1:16-8     5.94ns ± 0%   69.25ns ± 1%  +1065.47%  (p=0.008 n=5+5)
IndexAnyUTF8/1:32-8     7.19ns ± 0%  155.06ns ± 1%  +2055.29%  (p=0.008 n=5+5)
IndexAnyUTF8/1:64-8     5.68ns ± 0%  300.56ns ± 1%  +5195.46%  (p=0.008 n=5+5)
IndexAnyUTF8/16:1-8     57.1ns ± 1%    61.3ns ± 1%     +7.43%  (p=0.008 n=5+5)
IndexAnyUTF8/16:2-8     65.0ns ± 0%   119.9ns ± 1%    +84.53%  (p=0.008 n=5+5)
IndexAnyUTF8/16:4-8     65.1ns ± 1%    76.4ns ± 0%    +17.40%  (p=0.008 n=5+5)
IndexAnyUTF8/16:8-8     65.3ns ± 1%   148.9ns ± 1%   +128.23%  (p=0.008 n=5+5)
IndexAnyUTF8/16:16-8    64.0ns ± 0%    91.8ns ± 0%    +43.45%  (p=0.008 n=5+5)
IndexAnyUTF8/16:32-8    83.8ns ± 0%   277.9ns ± 1%   +231.56%  (p=0.008 n=5+5)
IndexAnyUTF8/16:64-8    66.2ns ± 0%   414.2ns ± 1%   +525.28%  (p=0.008 n=5+5)
IndexAnyUTF8/256:1-8     838ns ± 1%     838ns ± 2%       ~     (p=1.000 n=5+5)
IndexAnyUTF8/256:2-8     973ns ± 1%    1704ns ± 2%    +75.16%  (p=0.008 n=5+5)
IndexAnyUTF8/256:4-8     973ns ± 1%     861ns ± 1%    -11.53%  (p=0.008 n=5+5)
IndexAnyUTF8/256:8-8     971ns ± 0%    1726ns ± 1%    +77.80%  (p=0.008 n=5+5)
IndexAnyUTF8/256:16-8    970ns ± 0%     109ns ± 1%    -88.77%  (p=0.008 n=5+5)
IndexAnyUTF8/256:32-8   1.29µs ± 0%    1.90µs ± 1%    +47.48%  (p=0.008 n=5+5)
IndexAnyUTF8/256:64-8   1.01µs ± 1%    1.29µs ± 1%    +28.61%  (p=0.008 n=5+5)

This is pretty interesting, assuming it's reproducible. Largely the implementation in the bytes package is faster (sometimes dramatically so). But there are a number of cases where the naive version does better. Picking them out:
IndexAnyASCII/16:2-8    19.2ns ± 1%    13.8ns ± 0%    -28.51%  (p=0.008 n=5+5)
IndexAnyASCII/256:2-8    182ns ± 1%      18ns ± 1%    -90.27%  (p=0.008 n=5+5)
IndexAnyASCII/256:4-8    188ns ± 1%      32ns ± 1%    -82.74%  (p=0.008 n=5+5)
IndexAnyASCII/256:8-8    198ns ± 0%      62ns ± 2%    -68.74%  (p=0.008 n=5+5)
IndexAnyASCII/256:16-8   207ns ± 4%     130ns ± 1%    -37.37%  (p=0.008 n=5+5)
IndexAnyUTF8/256:4-8     973ns ± 1%     861ns ± 1%    -11.53%  (p=0.008 n=5+5)
IndexAnyUTF8/256:16-8    970ns ± 0%     109ns ± 1%    -88.77%  (p=0.008 n=5+5)

I don't know what's particularly interesting about these cases.
Benchmark platform:
goos: linux
goarch: amd64
cpu: AMD EPYC 7B12

@ianlancetaylor

This surprises me. The current algorithm only looks through the string once. Do you have an explanation for why it is faster to look through it multiple times? Is this a memory caching effect?

In any case we have a double loop, over the separators and over the data. Let n be the number of separators and m be the first occurrence of a separator in the data.
Theoretically, without taking optimizations into account, especially if n<m, it's in our interest to loop over the delimiters first, because if the first delimiter found is on average at position n/2, we avoid half of the m*n/2 comparisons, while in the other direction we avoid only n/2 of the n*m comparisons.
I don't think this explains the dominance of the naive version in my benchmarks, since I put the "founded" separator in the last position.
A possible explanation for the advantage of the naive version in situations where the number of separators is smaller than the first occurrence is the fact that the bytes.IndexByte (or bytes.IndexRune) function is highly optimized. And in the "naive" version, we take advantage of this optimization by keeping the "short" loop and replacing the "long" loop with this optimization.
Notes:

I think in practice we're more often in the case where the number of delimiters is less than the data. The test in the standard libray contains a lot of extreme cases: very short data with no separators.
If we sort the separators by their probability of occurrence, this will improve performance even more in the "naive" version where we loop over the separators first.

I just realized last night that I was wrong in my previous message and that the benchmarks, both mine and the two in the standard library, are no good. They all suffer from the same problem: it's not normal to consider only cases where the separator is absent from the data or it's in the last position.
Let's consider that the separator in position i < n is in position j <m in the data and that the other separators are not present in the data. If we loop on the separators first, we'll have to make i*m comparisons, whereas if we loop on the data first, we can stop after n*j comparisons.
The situation is actually much more complicated, as the position of all the separators comes into play. We can try to make theoretical predictions by approximating the behavior of j by a Poisson distribution and that of i by a uniform distribution, for example, but it seems to me much simpler to design realistic benchmarks than to theorize the question.
I think a realistic benchmark could be

a simple CSV example where there are fields of different lengths (including empty ones) and where we scan for separators one after the other,
a template where we look for "special" characters.

In all cases, we'll limit ourselves to 2, 3 or 4 separators (which seem to be the use cases on GitHub) and to data which is scanned to find all the separators one after the other. I'll try to design a more realistic benchmark and see how it goes.

Using the following benchmark
func BenchmarkIndexAnyCSV(b *testing.B) {
	// A csv dataset with 7 columns (of diffrent lengths) and 10 rows.
	// Some of the fields (gender) are empty.
	// We will check with this data repetaed 1, 10 and 100 times.
	// This data is fake and was generated with https://www.convertcsv.com/generate-test-data.htm.
	var csvdata = []byte(`seq,name,adress,email,birthdate,gender,phone
	1,"May Moody",adress,gofpuwvi@wolamme.gp,08/03/2014,F,(313) 321-8728
	2,"Adele Martinez",adress,nohvahac@du.ai,02/09/1993,,(785) 834-8091
	3,"Zachary Brown",adress,bo@covih.yt,07/17/1901,M,(308) 643-7802
	4,"Lucy Lindsey",adress,sa@oh.fi,04/22/2023,F,(375) 395-3248
	5,"Lucy Oliver",adress,gemjivefa@ha.bd,07/21/1995,M,(422) 714-5921
	6,"Hilda Mullins",adress,kiujma@bok.kh,05/13/1922,,(716) 645-7631
	7,"Mollie Parker",adress,kir@kahamrew.io,03/18/1960,M,(534) 818-5916
	8,"Katie Adkins",adress,konocizah@jibfu.et,02/01/1965,F,(828) 544-2687
	9,"Maurice Carroll",adress,fosmutlo@erpewi.dm,05/01/1958,M,(478) 249-3000
	10,"Lettie Harvey",adress,lircuged@pe.bs,07/04/2008,,(571) 628-2216`)
	// We will check with the first 2, 3 and 4 delimiters of the following list.
	// The 4th is not present in the data.
	var delimiters = ",\n\";"

	for _, k := range []int{1, 10, 100} {
		for _, j := range []int{2, 3, 4} {
			b.Run(fmt.Sprintf("%d:%d", k, j), func(b *testing.B) {
				data := bytes.Repeat(csvdata, k)
				separators := delimiters[:j]
				for i := 0; i < b.N; i++ {
					p, q := 0, 0
					for {
						if q = IndexAny(data[p:], separators); q == -1 {
							break
						}
						p += q + 1
					}
				}
			})
		}
	}
}

I get the following results
$ benchstat standard.txt naive.txt
name                 old time/op  new time/op   delta
IndexAnyCSV/1:2-8    1.42µs ± 2%   1.30µs ± 1%     -8.29%  (p=0.029 n=4+4)
IndexAnyCSV/1:3-8    1.68µs ± 2%   2.34µs ± 1%    +38.98%  (p=0.029 n=4+4)
IndexAnyCSV/1:4-8    1.79µs ± 2%   4.13µs ± 2%   +130.71%  (p=0.029 n=4+4)
IndexAnyCSV/10:2-8   13.9µs ± 4%   14.0µs ± 4%       ~     (p=0.886 n=4+4)
IndexAnyCSV/10:3-8   16.4µs ± 2%   24.1µs ± 1%    +46.48%  (p=0.029 n=4+4)
IndexAnyCSV/10:4-8   17.8µs ± 2%  119.0µs ± 1%   +566.80%  (p=0.029 n=4+4)
IndexAnyCSV/100:2-8   139µs ± 4%    138µs ± 5%       ~     (p=0.686 n=4+4)
IndexAnyCSV/100:3-8   164µs ± 2%    243µs ± 2%    +48.53%  (p=0.029 n=4+4)
IndexAnyCSV/100:4-8   179µs ± 2%  11008µs ± 3%  +6036.16%  (p=0.029 n=4+4)

My conclusion is that the current version is better than the naive version in real-life situations. So there's no need to look any further.
On the other hand, I think we need to add a more realistic benchmark (like this one, for example) to the standard library (for both bytes and strings).

FTR, we try to aggregate more realistic benchmarks in golang.org/x/benchmarks. These run continuously in CI (https://perf.golang.org/dashboard). If you write a benchmark that is executable with go get, we can consider referencing it in bent cmd/bent (https://cs.opensource.google/go/x/benchmarks/+/master:cmd/bent/configs/suites.toml). If you have an even larger, more complex text processing benchmark, it might be worth adding to sweet.
It sounds like there might not be anything left to do in this issue? Closing for now, please let me know if you disagree!

@kpym Thanks for the investigation.

+1, thanks for the investigation!


Plugin manager sorting/filtering capability isn't clearly communicated

The plugin manager can filter plugins by category, enabled state, etc. but it isn't clearly communicated outside of the user documentation. A low-effort fix could be adding filtering examples to the placeholder message.

Fixed in 2.3.2790.


Python extension not able to recognize installed modules immediately

I had installed a python package in vs code terminal and tried to import it after installing. Python extension is showing as import error but I am able to run the code without any issues as the package is already installed in my environment.
After closing and opening vs code python linter is detecting the installed package.
Every time I install a new package I had to reload VS code to remove those annoying import errors.
Is there any solution for this issue.

@3627pranav, thanks for letting us know about this.  Please provide the content of your settings.json, as well as the "Python" output panel.

I'm going to chime in here because I'm experiencing the same issue. I'm by no means a VS Code expert but it seems like the IntelliSense database is not responding to package installs, which is even more frustrating because there doesn't exist a manual command I can run like "Python: Reindex IntelliSense datavase" so the only way to get this to work is by quitting and restarting VS Code, which is tedious.
I've included the contents of my settings.json file: https://gist.github.com/kevtan/b05cbf5d438a1d1b32543095c21a001a. I'm not sure what you mean by a Python output panel.

@kevtan, open the "View" menu and select "Output".  Then in the output panel click on the pull-down menu (probably says "Tasks") and select "Python".  That will switch it to the "Python" output panel.

python output panel data :
User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control'
User belongs to experiment group 'LocalZMQKernel - control'
User belongs to experiment group 'CollectLSRequestTiming - control'
User belongs to experiment group 'CollectNodeLSRequestTiming - experiment'
User belongs to experiment group 'EnableIPyWidgets - experiment'
User belongs to experiment group 'RunByLine - control'
User belongs to experiment group 'CustomEditorSupport - control'

conda --version
pyenv root
python3.7 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python3.6 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python3 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python2 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
py -3.7 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
py -3.6 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
py -3 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
py -2 c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
conda info --json
python c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
Python interpreter path: ~\AppData\Local\Programs\Python\Python38\python.exe
Starting Microsoft Python language server.
~\AppData\Local\Programs\Python\Python38\python.exe c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import jupyter"
~\AppData\Local\Programs\Python\Python38\python.exe c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import jupyter"
~\AppData\Local\Programs\Python\Python38\python.exe c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import notebook"
~\AppData\Local\Programs\Python\Python38\python.exe c:\Users\pranav.vscode\extensions\ms-python.python-2020.6.90262\pythonFiles\pyvsc-run-isolated.py -c "import notebook"
conda --version

My Settings.json file :
{
"editor.suggestSelection": "first",
"vsintellicode.modify.editor.suggestSelection": "automaticallyOverrodeDefaultValue",
"workbench.iconTheme": "material-icon-theme",
"java.configuration.checkProjectSettingsExclusions": false,
"java.semanticHighlighting.enabled": true,
"explorer.confirmDelete": false,
"[html]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
},
"[typescript]": {
"editor.defaultFormatter": "vscode.typescript-language-features"
},
"python.languageServer": "Microsoft",
"[python]": {
}

}

@ericsnowcurrently I have provided required info.

@3627pranav, please verify that by "Python extension is showing as import error" you mean "import-related entries show up in the 'Problems' panel".  In that case, what is the exact text?  Could you provide the text or a screenshot?  It should indicate the source of the entry in parentheses.
Also, do you have the same problem if you use Jedi for your "language server" ("python.languageServer": "Jedi" in your setting.json)?

@jakebailey, known issue?

@ericsnowcurrently  Yes I have the same problem for Jedi also.
I have just installed pysftp package but linter is not recognizing it.
Here is the screenshot of problem :

Yes, this is as designed: https://github.com/microsoft/python-language-server/blob/master/TROUBLESHOOTING.md#completions-do-not-update-andor-import-is-still-being-reported-as-unresolved-after-pip-install
Consider trying Pylance (where we don't have the core problem that forced us to disable this detection).

@jakebailey Thank you. Pylance is working.


Update scope of PKI MM WG to create assessment tool

@vanbroup, there are no objections identified for this change, can you please approve and merge new version of the charter?

Please share the changes for comments to the general mailing list.


Parsers for other type of docs

Hi, I'm impressed with your work. I'm expecting this work will enable us to extract valuable insights from SEC filings. Really appreciate it.
But I'm wondering if you are planning to add more parsers for other types of documents, such as 10-K, 8-K, etc.
It would be very useful if we have more parsers.
Thanks.

Hi, thanks for reaching out! We're glad you found the parser useful.
The difference from the parser's perspective between 8-K, 10-Q, and 10-K is mainly that they have different top-level sections. Other than that, they're mostly the same and the parser should be able to be used already.
Option 1
If you don't care about finding the boundaries between top-level sections (or will use other ways to do so), the easiest way to do it would be to use the sec-parser without the TopSectionManagerFor10Q processing step.
It could look roughly like this:
from sec_parser import Edgar10QParser, TopSectionManagerFor10Q

class EdgarParserFor10Kand8K(Edgar10QParser):
    def get_default_steps(self):
        steps = super().get_default_steps()
        # Filter out the instance of TopSectionManagerFor10Q from the steps
        return [s for s in steps if not isinstance(s, TopSectionManagerFor10Q)]

html = ...
parser = EdgarParserFor10Kand8K()
parser.parse(html)

Option 2
To parse top-level section boundaries, a solution could be to add new section titles to
https://github.com/alphanome-ai/sec-parser/blob/main/sec_parser/semantic_elements/top_section_title_types.py
and then create two more classes Edgar8KParser, Edgar10KParser, each with the respective processing step changed to TopSectionManagerFor10K and TopSectionManagerFor8K
Let us know if this helps 🙌

Thanks for the kind reply. I'll definitely try that!

Hey @jong-inn, I have an updated answer, check it out

[!NOTE]
For most up to date version of this answer, check out https://github.com/Elijas/sec-parser-exploration/blob/main/02_other_sec_form_types.ipynb

How do I parse other form types with sec-parser, such as 10-K, 8-K, S-1, etc.?
As of April 2023, sec-parser supports parsing various SEC form types beyond just 10-Q, such as 10-K, 8-K, S-1, and more. Basically, it supports parsing any structured text forms.
However, currently only parsing 10-Q top level section types is supported.
This gives us two methods to parse other form types with sec-parser:

Ignore the warnings from the parsing step that identifies the 10-Q top level section types
Skip the parsing step that identifies the 10-Q top level section types

Method 1: Ignore warnings from the parsing step that identifies the 10Q top level section types
import warnings
from sec_downloader import Downloader
import sec_parser as sp

dl = Downloader("MyCompanyName"<EMAIL_ADDRESS>html = dl.get_filing_html(ticker="AAPL", form="10-K")
parser = sp.Edgar10QParser()

with warnings.catch_warnings():
    warnings.filterwarnings("ignore", message="Invalid section type for")
    elements: list = parser.parse(html)

tree: sp.SemanticTree = sp.TreeBuilder().build(elements)

demo_output_1: str = sp.render(elements)
demo_output_2: str = sp.render(tree)

Method 2: Skip the parsing steps related to the 10Q top level section types
from sec_downloader import Downloader
import sec_parser as sp
from sec_parser.processing_steps import TopSectionManagerFor10Q, IndividualSemanticElementExtractor, TopSectionTitleCheck

dl = Downloader("MyCompanyName"<EMAIL_ADDRESS>html = dl.get_filing_html(ticker="AAPL", form="10-K")

def without_10q_related_steps():
    all_steps = sp.Edgar10QParser().get_default_steps()

    # Change 1: Remove the TopSectionManagerFor10Q
    steps_without_top_section_manager = [step for step in all_steps if not isinstance(step, TopSectionManagerFor10Q)]

    # Change 2: Replace the IndividualSemanticElementExtractor with a new one that has the top section checks removed
    def get_checks_without_top_section_title_check():
        all_checks = sp.Edgar10QParser().get_default_single_element_checks()
        return [check for check in all_checks if not isinstance(check, TopSectionTitleCheck)]
    return [
        IndividualSemanticElementExtractor(get_checks=get_checks_without_top_section_title_check)
        if isinstance(step, IndividualSemanticElementExtractor)
        else step
        for step in steps_without_top_section_manager
    ]

parser = sp.Edgar10QParser(get_steps=without_10q_related_steps)
elements: list = parser.parse(html)
tree: sp.SemanticTree = sp.TreeBuilder().build(elements)

demo_output_1: str = sp.render(elements)
demo_output_2: str = sp.render(tree)

Feedback
Let us know if it helped or if you have any further questions!
Thanks,
Elijas

Related: https://github.com/alphanome-ai/sec-parser/issues/81
Related: https://github.com/alphanome-ai/sec-parser/issues/82


fix documentation errors in build and add ECC Zoo links for product codes

I noticed there was some errors with build. For the latest documentation to be deployed, all these errors needed to be fixed. The above two errors were causes by the references. I have updated them.
I will also fix all the others errors so that the build passes.
┌ Error: Entry tillich2009quantum is missing the year field(s).
└ @ BibInternal ~/.julia/packages/BibInternal/RsFF9/src/bibtex.jl:93
┌ Error: Entry kovalev1212quantum is missing the year field(s).
└ @ BibInternal ~/.julia/packages/BibInternal/RsFF9/src/bibtex.jl:93
[ Info: SetupBuildDirectory: setting up build directory.
[ Info: Doctest: skipped.
[ Info: ExpandTemplates: expanding markdown templates.
┌ Error: duplicate docs found for 'defining_set' in `@docs` block in src/Classical/cyclic_code.md:85-87
│ ```@docs
│ defining_set
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'set_distance_upper_bound!' in `@docs` block in src/Classical/linear_code.md:84-86
│ ```@docs
│ set_distance_upper_bound!
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: duplicate docs found for 'encode' in `@docs` block in src/Classical/linear_code.md:98-100
│ ```@docs
│ encode
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: unable to get the binding for 'CodingTheory.⊕' in `@docs` block in src/Classical/new_codes_from_old.md:25-27 from expression ':($(Expr(:incomplete, "incomplete: premature end of input")))' in module Main
│ ```@docs
│ CodingTheory.⊕
│ ```
│   exception = `binding` cannot understand expression `$(Expr(:incomplete, "incomplete: premature end of input"))`.
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: unable to get the binding for 'CodingTheory.×' in `@docs` block in src/Classical/new_codes_from_old.md:30-32 from expression ':($(Expr(:incomplete, "incomplete: premature end of input")))' in module Main
│ ```@docs
│ CodingTheory.×
│ ```
│   exception = `binding` cannot understand expression `$(Expr(:incomplete, "incomplete: premature end of input"))`.
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: unable to get the binding for 'CodingTheory.⊗' in `@docs` block in src/Classical/new_codes_from_old.md:35-37 from expression ':($(Expr(:incomplete, "incomplete: premature end of input")))' in module Main
│ ```@docs
│ CodingTheory.⊗
│ ```
│   exception = `binding` cannot understand expression `$(Expr(:incomplete, "incomplete: premature end of input"))`.
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: unable to get the binding for 'CodingTheory./' in `@docs` block in src/Classical/new_codes_from_old.md:44-46 from expression ':($(Expr(:incomplete, "incomplete: premature end of input")))' in module Main
│ ```@docs
│ CodingTheory./
│ ```
│   exception = `binding` cannot understand expression `$(Expr(:incomplete, "incomplete: premature end of input"))`.
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'plot_EXIT_chart' in `@docs` block in src/LDPC/analysis.md:30-32
│ ```@docs
│ plot_EXIT_chart
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'multiplicative_gap' in `@docs` block in src/LDPC/analysis.md:35-37
│ ```@docs
│ multiplicative_gap
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'irregular_LDPC_code' in `@docs` block in src/LDPC/codes.md:39-41
│ ```@docs
│ irregular_LDPC_code
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'shortest_cycles' in `@docs` block in src/LDPC/codes.md:119-121
│ ```@docs
│ shortest_cycles
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'ACE_spectrum' in `@docs` block in src/LDPC/codes.md:124-126
│ ```@docs
│ ACE_spectrum
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'shortest_cycle_ACE' in `@docs` block in src/LDPC/codes.md:128-130
│ ```@docs
│ shortest_cycle_ACE
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'ACE_distribution' in `@docs` block in src/LDPC/codes.md:132-134
│ ```@docs
│ ACE_distribution
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'average_ACE_distribution' in `@docs` block in src/LDPC/codes.md:136-138
│ ```@docs
│ average_ACE_distribution
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'median_ACE_distribution' in `@docs` block in src/LDPC/codes.md:140-142
│ ```@docs
│ median_ACE_distribution
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'mode_ACE_distribution' in `@docs` block in src/LDPC/codes.md:144-146
│ ```@docs
│ mode_ACE_distribution
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'find_MP_schedule' in `@docs` block in src/LDPC/decoders.md:24-26
│ ```@docs
│ find_MP_schedule
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
┌ Error: undefined binding 'decoder_simulation' in `@docs` block in src/LDPC/decoders.md:36-38
│ ```@docs
│ decoder_simulation
│ ```
└ @ Documenter ~/.julia/packages/Documenter/CJeWX/src/utilities/utilities.jl:44
[ Info: CollectCitations
┌ Error: The @cite link text "" does not match required regex
│   ast =
│    @ast MarkdownAST.Link("@cite", "")
│
└ @ DocumenterCitations ~/.julia/packages/DocumenterCitations/OyiEp/src/citation_link.jl:151
┌ Error: Error collecting citations from "Tutorials/Message Passing.md"
...

Please don't merge this yet. I am waiting the results from the build.

I think this is ready for review. I had added the ECC Zoo links for product codes as well. For the false positives errors mentioned in https://github.com/esabo/CodingTheory/pull/40#issue-2534196646, I will look into it later. Thank you!

I'll just review it when the false positives are done because it won't compile before then anyway.

Woohoo, documentation build passed. There is a way to suppress the errors using checkdocs in makedocs.  Using @example block can be helpful than importing .png to message passing.md. In the next PRs, I will do that as well.
I think this is ready for review, Thank you!

There are a lot of deletions but I don't see things added back in?

For some methods, docstrings are not defined, so called them in docs gives error. Namely: decoder_simulation, and find_MP_schedule. For some methods, docstring is defined, but calling it gives undefined binding. I will try to make sure to delete as less as possible. I can added the deleted methods using [[ ]](@ref) is possible.  The assets don't have the images that's why build was throwing error as well. I think @example block can be used in place of manually inputting images.

Thanks for your comment! I hope this is okay. There are no png images in assets as that was causing the documentation error in message passing.md tutorial.  I have made sure to add the methods as See: or See also: so that user can refer to these methods. In the future, a lot more work on documentation can be done to revamp the message passing.md as well.
This PR make sure that we can deploy the docs by suppressing all the errors for methods for which docstrings are not defined! I think it's ready for review, Thank you!


error trying to build example

Thanks for creating this library. I'm new to swift and struggling to get the example working.
When I build the project, I get the following error- "shared" is unavailable: Use view controller based solutions where appropriate instead, pointing to three lines different lines. This is one of them in the DetailViewController- "let splitViewController = UIApplication.shared.delegate?.window!!.rootViewController as! UISplitViewController".
How could I fix this? Thanks in advance.

Can't reproduce it. Some points:

If you're using version 0.5, you must use Xcode 8+
Is this only the example project or do you have other stuff? Are you using Realm? Try updating CocoaPods. Apparently, Realm in combination CocoaPods older than 0.39 can cause this error.
This is likely not the problem, but mentioning it nevertheless,  it's not possible to access the app delegate from an extension.
Finally, do a hard clean (delete derived data) and pod update:

rm -rf "${HOME}/Library/Caches/CocoaPods"
rm -rf "`pwd`/Pods/"
pod search SwiftCharts

Let me know how it goes! If it still doesn't work please provide more details in order to be able to reproduce it.

@timothyrchew have the same issue
quick solution - comment out that rows of code. The example would work

@timothyrchew @DKalachniuk could you provide your specs? Xcode version? Are you using an extension?


exit status

Situation
when the program experiences a minor error, and continues on listing other files, the program will exit like it succeeded.
# expected behaviour
$ ls adf .
ls: adf: No such file or directory
.:
Makefile      README.md     ft_ls         includes      libkm         make_settings src
$ echo $?
1

# current, wrong, behaviour
./ft_ls adf .
adf: No such file or directory
Makefile        README.md       ft_ls   includes        libkm   make_settings   src
$ echo $?
0

how to fix?
Either keep track of the global error state of the program or don`t change minor_error back to success after resolving the issue.
I opt the first solution will be better, because with the current situation its easier to immediately fix minor errors.

fixed in #10


Export the slashing database

Each validator client can export a slashing database to ensure a safe migration of the validator account keys. Please provide a way to export the slashing database for the following clients:

[ ] Lighthouse
[ ] Nimbus
[ ] Prysm
[ ] Teku
[ ] Lodestar

For this we currently have a button in the validator client settings discussed.

Pressing this button opens up a Path Selector to save the database.


How to use in multiple threads?

Database implements no Clone, Send, nor Sync. The documentation however mentions:

Multiple Database objects can be kept around, as leveldb synchronises internally.

Thanks for pointing this out. On further reading of the leveldb docs, this isn't even true - only one pointer can be kept around.
Will fix this. (or do you want to take a go? Basically Send and Sync need to be implemented for Database)

Revolved by https://github.com/skade/leveldb/commit/7b79aebae9bcd7c10cf61d207c62dc93cbb69740, see the commit for a usage example in form of a test.

Wow, that was quick! Thank you very much.


Avoid adding build time dependencies to production/test code

The magetasks could bring a lot of the equipment that base application do not use. More over, those deps could influence the versions of dependencies that are actually used by the base application.
On the other hand, when other repos include knative.dev/kn-plugin-event only the production code is included. Because of this, this issue isn't as pressing as it may seem.

/remove-lifecycle stale

https://github.com/knative/hack/issues/85 should allow this to be possible


Plugin system?

A couple times now we've mentioned that we eventually want Arch to have a plugin system, but nobody really knows what it should do. We can discuss and track that here.
What would you want a plugin system to let you do?

Some things I'll list here may not necessarily be something I'd want. I'm just trying to get the brain juices flowing. I'll add to this over time as I think of stuff...

Add custom watch and build tasks (for compiling css and other assets, for example).


feat(7411) - Replace helper to get namespaced name

The implementation has been updated to retrieve only the namespaced name, eliminating the use of the helper function that previously resolved both Identifier and namespaced name which is useless.

Thanks @a-tarasyuk!


Hello Python - Laboratory Blog

https://laboratory-blog.github.io/2018/11/14/Python-hello_python/
Welcome to here

great posts.


Allow to customize roles inspect query

Closes: #89

Codecov Report

Merging #93 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #93   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           9      9
  Lines         885    885
=====================================
  Hits          885    885

Impacted Files
Coverage Δ

ldap2pg/config.py
100% <ø> (ø)
:arrow_up:

ldap2pg/script.py
100% <ø> (ø)
:arrow_up:

ldap2pg/manager.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9932ce...25b9c1a. Read the comment docs.


dev: refine replace linter dev

The refine vs refine' linter is in mathlib, so this PR with the script to automatically substitute refine' with refine is no longer needed.


New credential challenge for separate KC client.

This is being separated because do not want to provide IDIR login and this instances will be accessible by non-gov members. We want to provide a KC username/password login, and a VC login.

We also need to share details that connects our KC client to the VCauthn service.

We also need to share details that connects our KC client to the VCauthn service.

You need a new KC client registered with vc-authn, correct?
The proof configuration looks good, however I noticed that this configuration AND the existing one do not have a subject identifier specified: was this on purpose, or an oversight? I can't remember if we decided to not specify one last time and why, so I'd like to confirm.

@Jsyro, The new configuration has been posted.


Deprecate docstrings as a separate syntactic category

Differentiating between docstrings and regular comments as part of the syntax is just too finicky, so it's better to do this after parsing.

Removed in 0f7a8b936194804b8026590cf045442a01979c38.


[SPARK-32860][Docs][SQL] Updating documentation

What changes were proposed in this pull request?

Updating javadocs for Encoders bean to include map (which was recently added)
Why are the changes needed?

The previous documentation wasn't updated with the new feature
Does this PR introduce any user-facing change?

Yes. The documentation was not updated with feature SPARK-16706
How was this patch tested?

No tests added because it was updated documentation

Can one of the admins verify this patch?

Can one of the admins verify this patch?

I'm confused why this is breaking when only comments were changed. Any ideas?


make ParentchainApi generic to support Asset Hub

This is WIP. just wanted to know how far away we are. And we're not exactly near. reachable, but with effort
challenges:

supertrait must use extrinsic type (and block type) which introduces yet another type argument and may prevent some super-generic treatment

closed in favor of #1652


Level Order Traversal

Add C++ traversal code of BST in the C++ file

hey! @Faraz1243 can you assign this task to me?


SPIKE - Review of Python SDK Endpoints

As per JSSDK 3.0  architecture discussions : https://github.com/casper-ecosystem/casper-js-sdk/discussions/332

https://github.com/casper-ecosystem/casper-js-sdk/discussions/332#discussioncomment-7096817


modals: Make settings page selectors more specific.

The #settings_page .right.show selector was breaking the Emoji style inputs in Display settings in mobile responsive view.
Fixes #7624.

LGTM (Travis: https://travis-ci.org/zulip/zulip/builds/313055983)

Looks good! This can be merged @showell.

Merged!  Thanks @synicalsyntax @brockwhittaker


Helm installation method needs documentation

SSIA

PR not merged yet - so waiting for that PR to merge to close this issue.

The above PR had merged, and I believe the Helm documentation is in pretty good shape.  Are there additional expectations on Helm documentation?
The only other piece of work that is underway is a collaboration with @gyliu513 to autogenerate the configuration options table.  We are making progress on this work steadily and will have a PR shortly.  For folks that want a preview, I've attached the current tarball of the generator code + example values.yaml file.  Note the generator does not properly handle lists which is why there is not yet a PR for this work.
tablegen.tar.gz

seems resolved.


Update

Update

@manmatha-roy are you trying to provide a patch or new feature? If so, can you explain what it is?


[TF 2.0] Different results of binary cross entropy loss of the same architecture

System information

Tensorflow 2.0

Describe the current behavior
I have created model from tf.keras.models for binary classification problem. It is dense with sigmoid activation in the last layer and binary cross entropy loss. I got different loss values whether my last layer was:
last_layer = Dense(1, activation='sigmoid')(previous_layer)
or
last_layer = Dense(1)(previous_layer)
last_layer = sigmoid(last_layer)

If loss=BinaryCrossentropy(from_logits=False):
-In the first case BCE do apply sigmoid before computing loss.
-In the second case BCE do not apply sigmoid before computing loss.
Describe the expected behavior
I believe that from user point of view it would consistent if both examples gives exactly the same output- there is no difference in architecture.
It would be better if sigmoid is applied, no matter what are the initialization arguments of BinaryCrossentropy.
Code to reproduce the issue
from tensorflow.python.keras.layers import Input, Dense
from tensorflow.python.keras.activations import sigmoid
from tensorflow.python.keras.models import Model
from tensorflow.python.keras.losses import BinaryCrossentropy
import numpy as np

x_train = np.array([[1000]])
y_train = [0]

inp = Input(x_train.shape[1])
out = Dense(1, trainable=False, use_bias=False)(inp)
model = Model(inp, out)
model.get_layer('dense').set_weights([np.array([[1]])])
model.compile(optimizer='adam', loss=BinaryCrossentropy(from_logits=True))

print('\nFrom logits=True, no activation')
model.fit(x_train, y_train, epochs=1)
print('Prediction:')
print(model.predict(x_train))

inp = Input(x_train.shape[1])
out = Dense(1, trainable=False, use_bias=False)(inp)
out = sigmoid(out)
model = Model(inp, out)
model.get_layer('dense_1').set_weights([np.array([[1]])])
model.compile(optimizer='adam', loss=BinaryCrossentropy(from_logits=False))

print('\nFrom logits=False, separate sigmoid')
model.fit(x_train, y_train, epochs=1)
print('Prediction:')
print(model.predict(x_train))

inp = Input(x_train.shape[1])
out = Dense(1, trainable=False, use_bias=False, activation='sigmoid')(inp)
model = Model(inp, out)
model.get_layer('dense_2').set_weights([np.array([[1]])])
model.compile(optimizer='adam', loss=BinaryCrossentropy(from_logits=False))

print('\nFrom logits=False, sigmoid in dense')
model.fit(x_train, y_train, epochs=1)
print('Prediction:')
print(model.predict(x_train))

Output tf 2.0:
From logits=True, no activation
1/1 [==============================] - 0s 17ms/sample - loss: 1000.0000
Prediction:
[[1000.]]

From logits=False, separate sigmoid
Train on 1 samples
1/1 [==============================] - 0s 15ms/sample - loss: 1000.0000
Prediction:
[[1.]]

From logits=False, sigmoid in dense
Train on 1 samples
1/1 [==============================] - 0s 17ms/sample - loss: 15.3332
Prediction:
[[1.]]

output tf 1.14
From logits=True, no activation
1/1 [==============================] - 0s 39ms/sample - loss: 1000.0000
Prediction:
[[1000.]]

From logits=False, separate sigmoid
1/1 [==============================] - 0s 44ms/sample - loss: 1000.0000
Prediction:
[[1.]]

From logits=False, sigmoid in dense
1/1 [==============================] - 0s 57ms/sample - loss: 1000.0000
Prediction:
[[1.]]

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Can you please provide the github gist of the code for TF1.14  @jacekblaz

@gowthamkpr
https://colab.research.google.com/gist/jacekblaz/1b860965969dca734a7618c2009bf018/untitled.ipynb

Thank you for the issue, i will look into this. I believe that some change was made when from_logits=False and last layer is sigmoid to use the logits from the previous layer. I am suspecting there is some issue there.

I could not repro this on the latest nightly, are you still seeing this?

@gowthamkpr I believe you are encountering the issue in TF 2.0b1- there is different output for the same architecture in your gist. This issue is not about error ValueError caused by numpy.
@pavithrasv I have tried to reproduce this error in current nightly as well as in 2.0.0-rc1 and in both it seems to be fixed- results are the same: 1000.0
2.0.0-rc1
2.0-b1
nightly
So at least the same architecture written in different style provide the same results. However I am not sure if that result is correct. Logits has values between 0 and 1, maximum BinaryCrossentropy for those value is 15.424949. I believe BinaryCrossentropy(from_logits=False) should compute logits before applying BinaryCrossentropy itself. Therefore if input=1000, target=0 and BinaryCrossentropy(from_logits=False) the result should be max 15.424949.
I think documentation, code, behavior and math behind it are inconsistent. Look at this comment deeply hidden in backend

This one deserve more attention guys, looks like a bug to me @dynamicwebpaige @pavithrasv and @karmel
I am unable to reproduce my results with image segmentation model when changing only the training loop style to work with tf.function
working style:
model.compile(optimizer=optimizer, loss=['categorical_crossentropy'])
model.train_on_batch(inputs, outputs])
broken style:
categorical_loss = tf.keras.losses.CategoricalCrossentropy()
train_step() # which is tf.function
in my case the loss of the broken style start from ~15.385 and in the working style from ~5

@jacekblaz I found out what's happening here. We have two different functions - one that computes cross-entropy loss when we have the logits information, one that computes cross-entropy loss with probabilities. For the inputs you have provided label=0 and predictions=1000, the loss value is undefined. Each of these functions handle this undefined case differently and hence you see 1000 for one and 15.33 for the other.

We recommend that you use logits as inputs always for numerical stability. All of the TF docs and examples will be updated to reflect the same.


Intensity-based probe filtering for each donor separately

Hi, I use abagen to perform the pre-processing steps for each of the six donors individually, one donor at a time (specifying donors=['XX'] in the main function). How exactly does the intensity-based probe filtering work here, since it normally uses the 50% threshold across all tissue samples across donors (which is not given in the described scenario)?
Thanks!

Hi @daedhalus ! :wave:
In this case the thresholding would only consider samples for the specified donor, so you would be thresholding based on a different number of samples per donor. Instead of this, you may want to consider using return_donors=True with donors='all'. If you do this, the output of the get_expression_data() function will be a list of region-by-gene matrices, with one matrix per donor.
Let me know if that helps or if you have any other questions!

That helps tremendously, thank you very much!


Unexpected toast while opening the app on NU Wifi

While opening the app on NU Student Wifi, there is an unexpected error toast which says Could not retrieve a menu.
Also, downloading the Mess Menu XLSX file from Settings Menu is unsuccessful on Wifi, but works perfectly when on mobile data.

Okay, this was due to Cloudflare SSL installed on my website https://arachnis.xyz.
As the university has a single public IP <IP_ADDRESS>, if anyone does something unusual through the network, cloudflare graylists the complete network and hence throws a CAPTCHA everytime we open any website secured by it.
So, to resolve this, I must make changes to my SSL settings.


Specify Image Registry

Right now there is just an image value for the image to pull:
https://github.com/aws/karpenter/blob/main/charts/karpenter/values.yaml#L91
Ideally it should get broken up into registry, image, and then tag like most other helm charts. Our use case it to utilize ECR pullthrough cache and only need to specify a registry for that to work.

See: https://github.com/aws/karpenter/pull/1737


Remove obsolete validation from DatabricksSqlStatementApiHealthCheck

Codecov Report

Merging #255 (0456b28) into main (0a85cce) will decrease coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #255      +/-   ##
==========================================
- Coverage   70.90%   70.86%   -0.05%
==========================================
  Files         215      215
  Lines        3884     3878       -6
  Branches      458      457       -1
==========================================
- Hits         2754     2748       -6
  Misses        979      979
  Partials      151      151

Flag
Coverage Δ

databricks
76.48% <ø> (-0.35%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

...althChecks/DatabricksSqlStatementApiHealthCheck.cs
100.00% <ø> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Create lychee output file on panic (fixes #82)

Tested and working in https://github.com/mre/docs/runs/5378428826?check_suite_focus=true


add celery beat to worker

Hi,
first: This tutorial in context of Django,Celery and Docker ist one of the best ones available. Thanks so much for it.
In my project I have not to deal with triggering tasks by the UI - myy use case ist more to trigger tasks by celery beat (every minute, every hour, once per day,...). Is my understanding correct, to start the beat-worker inside the worker container via the script, where the worker process gets started?
Thanks a lot for your work.
Regards
olikra

Thank you! Yes, you are correct, you need to start beat service in the container.

Thx...


[fix][broker] Close transactionBuffer after MessageDeduplication#checkStatus failed

Motivation
Currently, we didn't close the TransactionBuffer after MessageDeduplication#checkStatus failed. see: https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L1600
it will lead to memory leak.
In TopicTransactionBuffer, it will create a snapshotWrite in SingleSnapshotAbortedTxnProcessorImpl, and the write will held by SystemTopicClientBase#writers.
If we don't close the TransactionBuffer after MessageDeduplication#checkStatus failed, the number of TransactionBufferSnapshotWriter held by SystemTopicClientBase#writers will keep increasing, and may lead to OOM in the finally.
In https://github.com/apache/pulsar/pull/15015 , it reuses the TransactionBufferSnapshotWriter, which means that the topics of a same namespace will use one single TransactionBufferSnapshotWriter instance, it greatly reduces the amount of memory leaks, but if we don't close the TransactionBuffer, there are still memory leak risks.
Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository:  https://github.com/tjiuming/pulsar/pull/17

Codecov Report

Merging #19157 (8dcb28f) into master (c862356) will decrease coverage by 16.18%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             master   #19157       +/-   ##
=============================================
- Coverage     46.50%   30.32%   -16.19%
+ Complexity    10297     5960     -4337
=============================================
  Files           687      633       -54
  Lines         67425    59874     -7551
  Branches       7237     6237     -1000
=============================================
- Hits          31357    18156    -13201
- Misses        32544    39190     +6646
+ Partials       3524     2528      -996

Flag
Coverage Δ

unittests
30.32% <0.00%> (-16.19%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.


Compilation without modules fails.

I tried to disable the compilation of modules by passing HOLOSCAN_BUILD_MODULES=OFF but the configuration failed because CMake couldn't find holoviz. Looks like these modules are hard coded on multiple locations inside the code (1, 2).
CMake Error at CMakeLists.txt:196 (install):
  install TARGETS given target "holoviz" which does not exist.

Further errors are encountered after fixing the problems above.
Also, support for partial module installations would be useful for deployments that don't require all features. For example, would be cool if I could install Holoviz with TensorRT for inference without being forced to install ONNX Runtime.

After further discussion we've determined that the HOLOSCAN_BUILD_MODULES option was removed in Holoscan SDK v0.6.0.


Store additional information in TLorentzVector

I asked earlier about constructing the Lorentz vector objects directly from numpy arrays. That works really well! But I'd also like to be able to add additional information to the objects, i.e. in some cases we have other jet characteristics like b-tagging scores. Is there some method to append additional information (which is stored in another array) to each object?

Yes. A TLorentzVectorArray contains a Table, so you should be able to attach or remove any other arrays to it, as long as they fit (same number of elements, and if jagged, same number in each subarray).
a = TLorentzVectorArray.from_ptetaphim(pt, eta, phi, mass)
a["isolation"] = isolation

and so on.

OK, that's ridiculously easy.

OK, while this obviously isn't the recommended way to do things, I noticed that when I looped over the  jets the iterator seemed to return a regular TLorentzVector without the isolation attached. Is there some way to get both together? I noticed that the following seems to work:
for vec, iso in zip(vecarray, vecarray.content['isolation']):
    print(vec, iso)

but it's a bit clunky.

To get that to work, I should have made the TLorentzVector object (singular) a mixin of Table.Row, instead of a separate class. At some point, I should do that so that you can get isolation from vec["isolation"].
I'll reopen this to remind myself to go back and do that.


Update fingerprints.yaml

Tuya 4 switch update... having a black and white model... black model working, white model not working so name changes and added it

Deployed (2024-07-15T03:12:28.190211458)


Convert mistral built ins

Certain Mistral built-in actions, like std.noop and std.http do not exist in Orquesta, so we convert them to their core counterparts which work in all workflow engines.

Codecov Report

Merging #16 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #16      +/-   ##
==========================================
+ Coverage   98.16%   98.19%   +0.02%
==========================================
  Files          11       11
  Lines         436      442       +6
==========================================
+ Hits          428      434       +6
  Misses          8        8

Impacted Files
Coverage Δ

orquestaconvert/workflows/base.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c70569a...46538ea. Read the comment docs.

This is more involved than I originally expected so it may take me a bit to circle back around to it.

OK, I think I'll merge this PR and create and issue to address the deep conversion of those actions.


doc/go1.16: document new behavior of asn1.Unmarshal on invalid argument

For: https://github.com/golang/go/pull/41485

@FiloSottile
Could you please review it?
I think #41485 needs to be documented.


std_stamped_msgs: 0.0.2-0 in 'melodic/distribution.yaml' [bloom]

Increasing version of package(s) in repository std_stamped_msgs to 0.0.2-0:

upstream repository: https://github.com/hakuturu583/std_stamped_msgs.git
release repository: https://github.com/hakuturu583/std_stamped_msgs-release.git
distro file: melodic/distribution.yaml
bloom version: 0.6.7
previous version for package: null

I'm going to close this PR out, since we won't merge this unless it has a different name.  Please resubmit a package with a different name and we'd be happy to merge, thanks.


The SMARTS should set the _SmilesStart property to handle inverted ch…

…irality on the first atom.

Reference Issue
None but #760, #1489 are related
What does this implement/fix? Explain your changes.
Chirality on the first atom should invert in SMARTS like it does in SMILES. It seems there was intention for this in the code (logic in Canon::chiralAtomNeedsTagInversion) but the parser was not setting the property.
from rdkit import Chem

smi = "[C@H](C)(O)N"
qry = Chem.MolFromSmarts(smi)
mol = Chem.MolFromSmiles(smi);
print(mol.HasSubstructMatch(qry, useChirality=True))
# was False now True, the SMILES matches it's self as a SMARTS

I conda installed bison/flex but still seemed to be an older version than was currently used. May be worth someone else regenerating smarts.?pp.cmake with the minor changes to the smarts.yy.
Any other comments?
I have many failures when I run ctest (SegFault/Exceptions) so not sure if they breaks anything else.
I would also consider the following questionable behaviour:
smi = "[C@H](C)(O)N"
qry = Chem.MolFromSmarts("[C@!H0](C)(O)N")
mol = Chem.MolFromSmiles(smi);
print(mol.HasSubstructMatch(qry, useChirality=True))
# False

smi = "[C@H](C)(O)N"
qry = Chem.MolFromSmarts("[C@h](C)(O)N")
mol = Chem.MolFromSmiles(smi);
print(mol.HasSubstructMatch(qry, useChirality=True))
# False

smi = "[C@H](C)(O)N"
qry = Chem.MolFromSmarts("[C@](C)(O)N")
mol = Chem.MolFromSmiles(smi);
print(mol.HasSubstructMatch(qry, useChirality=True))
# False

You could handle this this by updating Canon::atomHasFourthValence() / Canon::hasSingleHQuery() but I think the correct logic for this is normally:
bool chiralAtomNeedsTagInversion(const RDKit::ROMol &mol,
                                 const RDKit::Atom *atom, bool isAtomFirst,
                                 size_t numClosures) {
  PRECONDITION(atom, "bad atom");
//  return atom->getDegree() == 3 &&
//         ((isAtomFirst && atom->getNumExplicitHs() == 1) ||
//          (!details::atomHasFourthValence(atom) && numClosures == 1 &&
//           !details::isUnsaturated(atom, mol)));
  return atom->getDegree() == 3 && isAtomFirst;
}

However that would break a lot of existing code, include sulfoxides like this: [S@](=O)(C)CC invert from what RDKit currently says they are. Daylight did not support these so there is no right answer to what that means.
The numClosures logic doesn't look correct, but it's late and I need to look deeper. Tracking the ring closures (which it looks like the parser does) means this shouldn't be needed.

Pinning an issue I opened some time ago as well, as it is the same #3096
@johnmay I think I had an example with mix of ring closures and first atom chirality, but I would need to dig a little deeper to find them again.

Pinning an issue I opened some time ago as well, as it is the same #3096
@johnmay I think I had an example with mix of ring closures and first atom chirality, but I would need to dig a little deeper to find them again.

So this changes the carbon case.
>>> from rdkit import Chem
>>> Chem.MolToSmarts(Chem.MolFromSmarts('[S@@](=O)(-C)-CC'))
'O=[S@@](-C)-CC'
>>> Chem.MolToSmarts(Chem.MolFromSmarts('[C@@H](-O)(-C)-CC'))
'O-[C@&H1](-C)-CC'

This patch I am just making the SMARTS <=> SMILES consistent. My opinion is the S=O should invert like the carbon, however not all toolkits agree on this and Daylight did not support it.
>>> qry = Chem.MolToSmarts(Chem.MolFromSmarts('[S@@](=O)(-C)-CC'))
>>> Chem.MolFromSmiles('[S@@](=O)(-C)-CC').HasSubstructMatch(Chem.MolFromSmarts(qry), useChirality=True)
True

@ricrogz I think you should be aware of this one too. It relates to some of the discussion we had around #2570
Here's the behavior if this patch is applied:

mol (SMILES)
query (SMARTS)
current behavior
post-patch behavior

C[C@](F)(Cl)O
[C@](F)(Cl)O
False
False

[H][C@](F)(Cl)O
[C@](F)(Cl)O
False
False

[H][C@](F)(Cl)O
[C@H](F)(Cl)O
False
True

I went back and looked at the SMARTS documentation and found this example in the section on Atomic Primitives:

C[C@H](F)O
match chirality (H-F-O anticlockwise viewed from C)

I'm not sure how I overlooked this before, but it is clear guidance on how that particular case should be handled and indicates that the changes here are correct when Hs are present in the query.
We should still figure out whether or not the first two examples should be changed, but that's something different.

I conda installed bison/flex but still seemed to be an older version than was currently used. May be worth someone else regenerating smarts.?pp.cmake with the minor changes to the smarts.yy.

I re-ran bison/flex on my box. I will do a PR against your branch with the updated .cmake files.

Thanks for the PR and I'll add the test. I did try that but got the same error on the CI that now happens. Not sure what the issue is there.
2023-09-29T07:32:18.8069247Z In file included from /home/vsts/work/1/s/Code/GraphMol/SmilesParse/SmilesParse.cpp:43:
2023-09-29T07:32:18.8071005Z smarts.yy:75: error: "YY_DECL" redefined [-Werror]
2023-09-29T07:32:18.8071690Z In file included from /home/vsts/work/1/s/Code/GraphMol/SmilesParse/SmilesParse.cpp:38:
2023-09-29T07:32:18.8072257Z smiles.yy:78: note: this is the location of the previous definition

@ricrogz I think you should be aware of this one too. It relates to some of the discussion we had around #2570
Here's the behavior if this patch is applied:

mol (SMILES)
query (SMARTS)
current behavior
post-patch behavior

C[C@](F)(Cl)O
[C@](F)(Cl)O
False
False

[H][C@](F)(Cl)O
[C@](F)(Cl)O
False
False

[H][C@](F)(Cl)O
[C@H](F)(Cl)O
False
True

I went back and looked at the SMARTS documentation and found this example in the section on Atomic Primitives:
C[C@H](F)O	match chirality (H-F-O anticlockwise viewed from C)
I'm not sure how I overlooked this before, but it is clear guidance on how that particular case should be handled and indicates that the changes here are correct when Hs are present in the query.

We should still figure out whether or not the first two examples should be changed, but that's something different.

Yep that is a longer debate/discussion due to the larger change in behaviour. For reference this is what CDK and Arthor do:

[C@](F)(Cl)O SMA
[C@](F)(Cl)O SMA
[C@H](F)(Cl)O SMA

C[C@](F)(Cl)O
Y
Y
N

[H][C@](F)(Cl)O
Y
Y
Y

[H][C@](F)(Cl)O
Y
Y
Y

Our normal contact for testing the Daylight toolkit is unavailable ATM so can't confirm what the OG behaviour was. Been trying to convince @rogersayle to get our own license for such things.
In RDKIT this does work like that when the chiral atom isn't first:
>>> Chem.MolFromSmiles("F[C@H](Cl)O").HasSubstructMatch(Chem.MolFromSmarts("F[C@](Cl)O"), useChirality=True);
True
>>> Chem.MolFromSmiles("F[C@H](Cl)O").HasSubstructMatch(Chem.MolFromSmarts("F[C@!H0](Cl)O"), useChirality=True);
True

But not if there is a single ring closure:
>>> Chem.MolFromSmiles("F[C@H]1CCCCO1").HasSubstructMatch(Chem.MolFromSmarts("F[C@]1CCCCO1"), useChirality=True);
False

Unless it's the first atom:
>>> Chem.MolFromSmiles("[C@H]1(F)CCCCO1").HasSubstructMatch(Chem.MolFromSmarts("[C@]1(F)CCCCO1"), useChirality=True);
True

The biggest issue with making things more consistent here is you invert the meaning on sulfoxides at the first position (C[S@](=O)CC is unchanged but [S@](C)(=O)CC flips). I think those are rare enough (i.e. S in first position) it's a reasonable albeit major change.

@johnmay looks like merging my patch broke the CI. I will try and figure that out/fix it an do an alternate PR
sorry about that

It's not directly connected to this PR, but I want to raise one of the points @ricrogz and I discussed in #2570 again since we're talking about what Daylight does/did: there's still a question about how chiral queries with 3 neighbors should match 4-coordinate chiral atoms.
What is the correct behavior here:
Chem.MolFromSmiles('C[C@@](F)(Cl)O').HasSubstructMatch(Chem.MolFromSmarts('[C@@](F)(Cl)O'),useChirality=True)

The current behavior is that the above does not match, but this does:
Chem.MolFromSmiles('[C@@](F)(Cl)(O)C').HasSubstructMatch(Chem.MolFromSmarts('[C@@](F)(Cl)O'),useChirality=True)

i.e. the fourth ligand is implicitly always "at the end"
I'm curious if this is comparable to what Daylight does/did, if the three-coordinate chiral query always matches a four-coordinate chiral atom (one could make the argument that it should), or if something else happens.
Note that this is NOT associated with the C being at the front of the SMARTS; we get (and should get) the same behavior here:
Chem.MolFromSmiles('C[C@@](F)(Cl)O').HasSubstructMatch(Chem.MolFromSmarts('F[C@@](Cl)O'),useChirality=True)

@ricrogz I think you should be aware of this one too. It relates to some of the discussion we had around #2570

Sorry that it has taken me this long to comment back, I've been off a few days, and I'll probably be still half off for a few more days.
I haven't yet been able to get into the details in this PR, but I think I remember some of the discussion in #2570, and if I remember it correctly, and judging by the case of Daylight's hint on C[C@H](F)O, I think this change makes sense.
My only concern is to merge such a breaking change this late in the release cycle, mostly because I'm worried about checking the effect of this change in downstream code.

My only concern is to merge such a breaking change this late in the release cycle, mostly because I'm worried about checking the effect of this change in downstream code.

I will defer to you on this one if you feel strongly about it, but I would argue that it's worth including given that:

this ought to only affect a small number of cases (places where the first atom in the SMARTS is the chiral atom)
the current behavior is wrong, so any behavior changes should be from wrong->correct. The exception to this would be if someone has built a test into their own code to detect these situations and fix them.

I will defer to you on this one if you feel strongly about it, but I would argue that it's worth including given that

I think you should merge it. I agree that this is now doing the right thing, so any issues cause by this change should be fixed. I just wished this had been merged earlier in the release cycle.

Thanks @johnmay !


32 improve accessibility score

Hello @MrSunshyne,
I have fixed the main issue which was causing the score to be low. Right now it is 96% on accessibility which is a pretty good score.
We have only a color contrast issue on the website(dark mode + light mode). We will have to define all color again so that it meet the aspect ratio required to be validated by lighthouse. We will work on this issue together so that we can score 100% on the website.

A quick lighthouse tests shows pretty good scores !
before

after

thanks for the PR @kushul, merging


xfce4 not mounting on latest update (crouton also not updating)

I'm getting the following output when I'm running sudo startxfce4 after the latest ChromeOS update:
/usr/bin/startxfce4: Starting X server

X.Org X Server 1.15.1
Release Date: 2014-04-13
X Protocol Version 11, Revision 0
Build Operating System: Linux 3.2.0-76-generic x86_64 Ubuntu
Current Operating System: Linux localhost 3.8.11 #1 SMP Tue Apr 14 18:57:27 PDT 2015 x86_64
Kernel command line: cros_secure console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 2506752 verity payload=PARTUUID=790795bd-c947-c44b-9bef-fff36180246f/PARTNROFF=1 hashtree=PARTUUID=790795bd-c947-c44b-9bef-fff36180246f/PARTNROFF=1 hashstart=2506752 alg=sha1 root_hexdigest=a066bff2dc004585fe2b5c44be0c33ce6314a3a4 salt=d9d62c9699d89416b72843e002fe7a0e273b0d9678a826be2a1517e0e06658ae" noinitrd vt.global_cursor_default=0 kern_guid=790795bd-c947-c44b-9bef-fff36180246f add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic iTCO_vendor_support.vendorsupport=3
Build Date: 12 February 2015  02:49:29PM
xorg-server 2:1.15.1-0ubuntu2.7 (For technical support please see http://www.ubuntu.com/support)
Current version of pixman: 0.30.2
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.1.log", Time: Mon Apr 27 13:19:33 2015
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
(EE)
(EE) Backtrace:
(EE) 0: /usr/bin/X (xorg_backtrace+0x48) [0x7f9d284a2848]
(EE) 1: /usr/bin/X (0x7f9d282f9000+0x1ad539) [0x7f9d284a6539]
(EE) 2: /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f9d273f6000+0x10340) [0x7f9d27406340]
(EE) 3: /usr/bin/X (0x7f9d282f9000+0xb57a6) [0x7f9d283ae7a6]
(EE) 4: /usr/bin/X (xf86BusProbe+0x9) [0x7f9d28382099]
(EE) 5: /usr/bin/X (InitOutput+0x74d) [0x7f9d283906fd]
(EE) 6: /usr/bin/X (0x7f9d282f9000+0x59bab) [0x7f9d28352bab]
(EE) 7: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0xf5) [0x7f9d25e36ec5]
(EE) 8: /usr/bin/X (0x7f9d282f9000+0x451ee) [0x7f9d2833e1ee]
(EE)
(EE) Segmentation fault at address 0x0
(EE)
Fatal server error:
(EE) Caught signal 11 (Segmentation fault). Server aborting
(EE)
(EE)
Please consult the The X.Org Foundation support
         at http://wiki.x.org
 for help.
(EE) Please also check the log file at "/var/log/Xorg.1.log" for additional information.
(EE)
(EE) Server terminated with error (1). Closing log file.
/usr/bin/xinit: giving up
/usr/bin/xinit: unable to connect to X server: Connection refused
/usr/bin/xinit: server error
Unmounting /mnt/stateful_partition/crouton/chroots/trusty...

Any help is much appreciated, I'm not quite understanding why I cannot update.

If you follow this guide it should work.

I downloaded a fresh crouton installer and in the update ran into this:
...
Get:17 http://archive.ubuntu.com trusty-security/main Sources [78.8 kB]
Get:18 http://archive.ubuntu.com trusty-security/restricted Sources [2061 B]
Get:19 http://archive.ubuntu.com trusty-security/universe Sources [20.9 kB]
Get:20 http://archive.ubuntu.com trusty-security/multiverse Sources [1922 B]
Get:21 http://archive.ubuntu.com trusty-security/main amd64 Packages [259 kB]
Get:22 http://archive.ubuntu.com trusty-security/restricted amd64 Packages [8875 B]
Get:23 http://archive.ubuntu.com trusty-security/universe amd64 Packages [99.6 kB]
Get:24 http://archive.ubuntu.com trusty-security/multiverse amd64 Packages [3451 B]
Get:25 http://archive.ubuntu.com trusty-security/main i386 Packages [248 kB]
Get:26 http://archive.ubuntu.com trusty-security/restricted i386 Packages [8846 B]
Get:27 http://archive.ubuntu.com trusty-security/universe i386 Packages [99.7 kB]
Get:28 http://archive.ubuntu.com trusty-security/multiverse i386 Packages [3643 B]
Hit http://archive.ubuntu.com trusty-security/main Translation-en
Hit http://archive.ubuntu.com trusty-security/multiverse Translation-en
Hit http://archive.ubuntu.com trusty-security/restricted Translation-en
Hit http://archive.ubuntu.com trusty-security/universe Translation-en
Fetched 2858 kB in 27s (102 kB/s)
W: Failed to fetch http://ppa.launchpad.net/ozcanesen/terra-terminal/ubuntu/dists/trusty/main/binary-amd64/Packages  404  Not Found

W: Failed to fetch http://ppa.launchpad.net/ozcanesen/terra-terminal/ubuntu/dists/trusty/main/binary-i386/Packages  404  Not Found

E: Some index files failed to download. They have been ignored, or old ones used instead.
Failed to complete chroot setup.
Unmounting /mnt/stateful_partition/crouton/chroots/trusty...

These were the same errors I was running into previously.
Any help/suggestions are very much appreciated.

Looks like you may have some broken ppa's.
Take a look at Maurice's instructions for fixing them here - https://github.com/dnschneid/crouton/issues/1623#issuecomment-94247737
I believe something similar should work for you.

Thanks so much for the help Dennis! I guess I should update more often.

No problem, glad it's working.
I usually update crouton and my chroots for each Chrome OS update and sometimes more often, depending what's going on with crouton. PPA's can be a pain though... :/


ui enhancements

nat report: different background for translated columns -> #1398
rule, natrule, changes report: make report window for each device scrollable + fix headline
certification: fill the whole row with background colors for upcoming / overdue recerts
code cleanup: inherit natrule report from rule report

fix #1444 : global tenant was saved but got lost in the fetch of the ldap connections


[Serd-1694] service client

Adding ServiceClient() method which takes a service id, optional base path (default is None), and optional swagger endpoint (default is '/endpoints'), then creates a client from the api spec and endpoints.
A lot of the work is being done in the helper functions and Endpoint class for the original APIClient, but small changes were needed in a few cases (mostly in regards to the url and making requests). These changes are overridden by the ServiceClient and ServiceEndpoint class.

I have a few fixes to make, one moment

Okay, ready for review when you all are @patr1ckm @elsander

Also, don't forget to update the changelog!

I think I covered all the previous comments. Let me know if I missed anything.

LGTM!


Add utility to get application storage paths

Utility to get path of specific kind of application data storage.

Create branch: issue-26-add-utility-to-get-application-storage-paths from main.


Add Request Response limits to documentation

Description:

there are new changes added from the WUM[1] and previously supported configs as well. All should be included in the documentation.
[1] https://github.com/wso2/product-microgateway/issues/1515
Steps to reproduce:
Content Positioning in Documentation:

Link:

Heading (& Step):

Any other reference:

Optional Fields
Related Issues:

https://github.com/wso2/product-microgateway/issues/1515
Suggested Labels:

Suggested Assignees:

@chashikajw


Equipping spells

I have a spell that has an icon set to a pumpkin. When I shift-clicked it to put it in my inventory it was equipped and is unremovable. I didn't test this for other equipable items like armor.

Oh good catch! Just need to add pumpkins to the wearables list in materials - feel free to PR if you’re up for it!

Submitted


Please fix this bug

What happened?
Can you please fix the store because I really want to store to work!!!
To reproduce the behavior?
Can you please make it work
What browser are you seeing the problem on?
Safari

Can you please fix this bug on the store please!!!

Thank you for creating an issue. We will normally respond within 24 hours.

which part of the store is not working for you as expected?

Duplicate of #710


add syn_full feature to serde_derive

This allows users of serde_derive to transitively enable the "syn/full" feature.
While this fixes a problem I'm having I haven't put much thought into its general desirability.
Background
I'd like code like this to compile:
#[derive(serde::Deserialize)]
struct Foo<T: strum::EnumCount> {
    foo: heapless::Vec<u8, T::COUNT>,
}

However Rust currently requires us to write it like so (link)
#![feature(generic_const_exprs)]

#[derive(serde::Deserialize)]
struct Foo<T: strum::EnumCount>
where
    [(); { T::COUNT }]:,
{
    foo: heapless::Vec<u8, { T::COUNT }>,
}

The weird where clause requires the "syn/full" feature

As a work around you can make your crate depend on syn while enabling that feature (without actually using it. As long as it's in Cargo.toml, everything will work out)

@dimpolo I think if you put syn in build-dependencies then it should get unified with proc-macro deps.

@dimpolo I think if you put syn in build-dependencies then it should get unified with proc-macro deps.

Yes! That works! Thanks for the help! I'll close this PR


#CPF-1651 : action in label doesn't effect the checkbox

The issue arose when a labelFormatter was used that contained a link to open a new window.
The click on the link, toggled the checkbox.
(solution)This PR separates the checkbox from the label, so an action taken on the label is seperate from the action on the checkbox.

We are going to do this another way.


Combine lists based a key

I have been struggling with implementing a solution in the point-free functional style for  a certain common use-case.
I have lists of two json structures of given format
 [ {
       project-id: 123,
       github-project: 'my-project1',
       description: 'sample'
       story-id: 4
     },
    {
       project-id: 456,
       github-project: 'my-project2',
       story-id: 5
     }
]

Another list
 [ {
       story-id: 4,
       description: 'story1'
     },
    {
       story-id: 5,
      description: 'story2'
     }
]

Expected Result
 [ {
       project-id: 123,
       github-project: 'my-project1',
       description: 'sample'
       story : {
         id: 4,
         description: 'story1'
       }
     },
    {
       project-id: 456,
       github-project: 'my-project2',
       story: {
          id: 5,
          description: 'story2'
       }
     }
]

Is there an easy way to do this in ramda.js ?
Thanks,
Ravi

This would be very useful addition to innerJoin.

zipWith((x, y) => mergeRight(x, y))(arr1, arr2)


Improve Security vulnerabilities check

Currently, we do something like the following:
	    if($exchangeCU -le [HealthChecker.ExchangeCULevel]::CU12)
	    {
	        if($exchangeCU -eq [HealthChecker.ExchangeCULevel]::CU12)
	        {
	            #CVE-2019-0817
                Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.6 -CVEName "CVE-2019-0817"
                #CVE-2018-0858
                Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.6 -CVEName "CVE-2019-0858"
	            #ADV190018
                Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.7 -CVEName "ADV190018"
	        }
	        #CVE-2019-0686
	        Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.5 -CVEName "CVE-2019-0686"
	        #CVE-2019-0724
	        Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.5 -CVEName "CVE-2019-0724"
            #CVE-2019-1084
	        Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.8 -CVEName "CVE-2019-1084"
	        #CVE-2019-1136
	        Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.8 -CVEName "CVE-2019-1136"
	        #CVE-2019-1137
	        Test-VulnerabilitiesByBuildNumbersAndDisplay -ExchangeBuildRevision $buildRevision -SecurityFixedBuild 1713.8 -CVEName "CVE-2019-1137"
	    }

Would be best to reduce the number of lines in this area of code by passing an array of CVENames for the same build number instead of a single CVEName for each similar build number.

@lusassl-msft  do you think you can do this one as well?

Sure thing! Will give it a try.

Checked into Dev2.37.0 & Dev3.0 branches


Tooltip partially hidden within Card

The tooltip within a card is bounded by overflow: hidden of the card; therefore when the content of the tooltip is too long or close to the edhe of the card, it is particlaly hidden.
Example: https://play.tailwindcss.com/Z8Hq3XM8pn

The only possible temporary workaround is by applying overflow-visible class to the card. I'm also hoping that this issue will be fixed in the future 😅.

The only possible temporary workaround is by applying overflow-visible class to the card. I'm also hoping that this issue will be fixed in the future 😅.

Thanks @NikarashiHatsu ... It's a trade off of showing the card photo corners as rectangle because they are overflow... 😢 So would need to add rounded-tr rounded-tl to the img tag as well...

Yea, it's kind of annoying to have put some extra classes 😅. I thought putting z-50 along with tooltip will fix the issue, sadly it won't.

Hi, I have the same problem when usingthe tooltip in a sidebar is there any other workaround to have the tooltip always showing

This is still present AFAIK, I think overflow-clip is a better solution for this. I was getting some odd (and horrible to track down) additional spacing causing the entire page to overflow with multiple collapses in a column using overflow-visible.


图片增加滤镜选项

https://github.com/nhn/tui.image-editor/blob/master/apps/image-editor/src/js/component/filter.js

http://fabricjs.com/image-filters

https://github.com/nihaojob/vue-fabric-editor/pull/152


add math helper?

Feature Request:
Could you add the "math" helper?
This StackOverflow seemed to have the same idea as I was thinking
https://stackoverflow.com/questions/22103989/adding-offset-to-index-when-looping-through-items-in-handlebars
relevant post:
Actual answer: https://stackoverflow.com/a/46317662/1549191
Register a math handlebar and perform all mathematical operations.
app.engine('handlebars', exphbs({
  helpers:{
    // Function to do basic mathematical operation in handlebar
    math: function(lvalue, operator, rvalue) {lvalue = parseFloat(lvalue);
        rvalue = parseFloat(rvalue);
        return {
            "+": lvalue + rvalue,
            "-": lvalue - rvalue,
            "*": lvalue * rvalue,
            "/": lvalue / rvalue,
            "%": lvalue % rvalue
        }[operator];
    }
}}));
app.set('view engine', 'handlebars');

Then you can directly perform operation in your view.

    {{#each myArray}}
        {{math @index "+" 1}}
    {{/each}}

Would the existing math support in the handlebars-helpers package not suffice?
Your example can be done with:
 {{#each myArray}} {{add @index 1}} {{/each}}

See the "math" section of the readme of the package https://github.com/Budibase/handlebars-helpers
(The above link is already provided in my readme.)

More complex math is done with embedded function calls:
{{ add 10 (multiply @index 5) }}

Amazing, thank you.  I am completely new to handlebars, and I appreciate your direct help.  Your plugin has been a lifesaver for a novice trying to back up some old info.


Destructuring array causes no-unused-vars errors

var arr = ['hello', 'world'];

var [a, b] = arr;

console.log(b); // I just need `b`;

'a' is defined but never used.
I'm using v6.0.0-beta.5

Seems correct to me. You can omit a and just use var [, b] = arr;.

@erikdesjardins  You're right. Thanks. 😄

[, b] is the scariest js I have ever seen, its valid?

Dang, I guess so

Looks like it's documented (under Ignoring some returned values) but it's the first time I noticed it.


Consumer services

By any chance, are there sources available of the predefined consumer services that do ship with VSTS/TFS?

It seems no one answers in this repo :D


Support more http methods mock

Is your feature request related to a problem? Please describe.
Related #10
We need to add the following HTTP method mock methods to the Mock structure.

HEAD
CONNECT
OPTIONS
TRACE

See #48


[3.x] Backport Windows subsystem build option.

Partial backport of #41332, building with enabled console might still be useful for debugging purpose (console redirection might not work with some third-party libraries, if they are using different CRT).

Add windows_subsystem build option (defaults to gui).

Thanks!


When generate is set without package, it does not place generated output into Sources

If I put the following in my config.yml file it will place generated output into the output directory which was the root folder, and not in the Sources folder.
generate:
  - entities
  - enums
  - paths

However if the "package" option is added it will place the files in the correct folder.

This was done by design to address inconsistencies raised as part of #127, although the previous behaviour is not the same as what you are describing.
The current intention is that the --output path is the directory where generated outputs are written to. When using package generation, this becomes the Package directory (with Package.swift at the top level), but when you don't generate as a package, the sources are just written into that directory by default since we don't generate anything other than source code.
What would be the motivation for nesting everything inside a Sources directory regardless compared to passing --output ./OriginalOutputDir/Sources/?
Thanks!

Ok, it was just surprising behavior to me.  I originally generated the package, but after having to fix it several times, I decided to remove it from generation, and suddenly the generated code started appearing in the root folder.  If you would, it would be nice to have that noted in the docs by removing the option, it changes where the generated source code goes, and one should update the output dir accordingly.
I did change the --output after I figured out what was causing it.


Datagrid - Multiline Text

Select one ...  (check one with "x")
[ ] bug
[X] feature request

Expected behavior
Currently, multi-line text is supported(-ish) for text inside rows. We don't have a spec for multi-line text in column headers, though. So:

Confirm the current design for multi-line text in rows is the one we want. See screenshot attached.
Create spec for multi-line text in headers.

Any update on this enhancement ? When is this planning to be released?
We have a use case of Multi-line text in rows.

Multi-lines work fine both in rows and in headers. This issue was here just to make sure that the UX side of the team was fine with the current design (aligned top, line-height, ...).
Multi-line in headers have been fixed a while back too, there is no issue with them.

@yenmade since this seems to work based on the demo available here: https://vmware.github.io/clarity/documentation/datagrid I am closing this issue. Please reopen with specific issues if needed or if we have a difference in the design required. For now, this works as the screenshot shows.


Alphafold raises an error when it is predicting protein structure

When I test some proteins like 1EC5 and 3PBJ downloaded from RCSB PDB, the program reported a wrong message like this:
Traceback (most recent call last):
File "run_alphafold.py", line 302, in
app.run(main)
File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "run_alphafold.py", line 276, in main
predict_structure(
File "run_alphafold.py", line 177, in predict_structure
relaxed_pdb_str, _, _ = amber_relaxer.process(prot=unrelaxed_protein)
File "/home/workspace/alphafold/alphafold/relax/relax.py", line 58, in process
out = amber_minimize.run_pipeline(
File "/home/workspace/alphafold/alphafold/relax/amber_minimize.py", line 460, in run_pipeline
_check_residues_are_well_defined(prot)
File "/home/workspace/alphafold/alphafold/relax/amber_minimize.py", line 140, in _check_residues_are_well_defined
raise ValueError("Amber minimization can only be performed on proteins with"
ValueError: Amber minimization can only be performed on proteins with well-defined residues. This protein contains at least one residue with no atoms.
Could anyone give me some help? Thanks a lot~

These proteins contain "X" residues in their FASTA sequences which are not supported in the relax stage. You should be able to find unrelaxed predictions for these proteins in your output directory. If you want predictions for all residues then you have to modify the fasta sequence by replacing X with one of the standard amino acids. You can find the list here: https://github.com/deepmind/alphafold/blob/b88f8dacef5d94e4d3d49613d08523feb20caec1/alphafold/common/residue_constants.py#L532-L535

These proteins contain "X" residues in their FASTA sequences which are not supported in the relax stage. You should be able to find unrelaxed predictions for these proteins in your output directory. If you want predictions for all residues then you have to modify the fasta sequence by replacing X with one of the standard amino acids. You can find the list here:
https://github.com/deepmind/alphafold/blob/b88f8dacef5d94e4d3d49613d08523feb20caec1/alphafold/common/residue_constants.py#L532-L535

Thank u for your reply~


Fix expanded multi-level grouping collapsing on row selection

Fixes #2258 and #2249.
Description
The groupsIndex did get messed up while sorting. This leads to the wrong groups being collapsed/expanded on subsequent getRenderState() calls. The fix is reindexing the groups after sorting.
Impacted Areas in Application
DataManager

Would you say these comments are accurate? https://github.com/material-table-core/core/pull/74/commits/af792accedbd8800a4508ee6cb3a0c37d23be8e0#diff-0055c7491633e8cf7acb5411e22b0a97c4d165c436e583b07420e39faaaa565bR990-R995
Just trying to make the codebase a little easier to digest.
If you think that is good enough I will go ahead and publish a new version. Thanks again @FalkF !

Would you say these comments are accurate? https://github.com/material-table-core/core/pull/74/commits/af792accedbd8800a4508ee6cb3a0c37d23be8e0#diff-0055c7491633e8cf7acb5411e22b0a97c4d165c436e583b07420e39faaaa565bR990-R995
Just trying to make the codebase a little easier to digest.
If you think that is good enough I will go ahead and publish a new version. Thanks again @FalkF !

Yeah those comments make sense, thanks for taking a look so fast 💯

Yeah those comments make sense, thanks for taking a look so fast 💯

@FalkF v2.3.8 has been published!

@FalkF v2.3.8 has been published!