max_threads to execute query

Describe the situation

The current max_threads which is used to execute a streaming query is not optimal and quite lots of times a waste of lots idle threads. The max threads required for a streaming query is tired to number of shards / partitions (united) of the query.
Please note, we need be careful to calculate the max_threads to at least as big as total shards, otherwise for join etc, the query may be not able to progress.
How to reproduce

Which Proton server version to use
Which interface to use, if matters
Non-default settings, if any
CREATE TABLE statements for all tables involved
Queries to run that lead to slow performance

Expected performance

Additional context

You can customize the number of threads with a custom configuration file. Follow these instructions.
https://github.com/timeplus-io/proton/wiki/Proton-Server-Setup-Guide(with-configuration-guide)#switching-to-config-file-for-detailed-control-part-4

Yes,and the default setting shall be optimal. And we are fixing the default max_threads


Remove options package

Removes the options package which was deprecated in https://github.com/ponylang/ponyc/commit/ae7d070a0d2cb3fc24c9c796 on December 11, 2018.

Marked as do not merge while I wait to understand the proper way to add the release notes to ponylang-main picks up said notes.

Hi @rhagenson,
The changelog - changed label was added to this pull request; all PRs with a changelog label need to have release notes included as part of the PR. If you haven't added release notes already, please do.
Release notes are added by creating a uniquely named file in the .release-notes directory. We suggest you call the file 3844.md to match the number of this pull request.
The basic format of the release notes (using markdown) should be:
## Title

End user description of changes, why it's important,
problems it solves etc.

If a breaking change, make sure to include 1 or more
examples what code would look like prior to this change
and how to update it to work after this change.

Thanks.

This ended up being pretty straightforward. We did not have any remaining uses of options in stdlib or examples. I added a before and after rewrite of the example program that was in options documentation using cli for release notes, but I would be surprised to learn anyone has not made the switch to cli yet.

Thanks @rhagenson


Visual Studio App Center is scheduled for retirement on March 31, 2025

What's the Product's Name?
Visual Studio App Center
Describe the product in a single sentence.
Integrated developer services for building, testing, releasing, and monitoring mobile and desktop apps
When was the product launched? Provide a specific date, if possible.
Since 2017
When was the product discontinued? Provide a specific date, if possible.
March 31, 2025
What type of product was it? App, Service, OS, or Hardware?
Service

Added, thanks for the contribution!


[bug] Cannot match Rust parameter

⏯ Playground Link

Playground link with relevant code
💻 Code

Code:
fn x(a: &mut String) {

}

Pattern:
rule:
  any:
    - pattern: &mut String
fix:
  String

🙁 Actual behavior
Doesn't match the parameter

🙂 Expected behavior
It should match the parameter and output
fn x(a: String) {

}

Hi, thanks for trying out ast-grep.
ast-grep is based on syntax, so

Pattern code must be valid code that tree-sitter can parse.

https://ast-grep.github.io/guide/pattern-syntax.html#pattern-matching

You can construct a pattern using its object style API. https://ast-grep.github.io/reference/rule.html#pattern

https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1c3QiLCJxdWVyeSI6IiRQOiAmbXV0IFN0cmluZyIsInJld3JpdGUiOiIkUDogU3RyaW5nIiwiY29uZmlnIjoiIyBZQU1MIFJ1bGUgaXMgbW9yZSBwb3dlcmZ1bCFcbiMgaHR0cHM6Ly9hc3QtZ3JlcC5naXRodWIuaW8vZ3VpZGUvcnVsZS1jb25maWcuaHRtbCNydWxlXG5ydWxlOlxuICBwYXR0ZXJuOlxuICAgIHNlbGVjdG9yOiBwYXJhbWV0ZXJcbiAgICBjb250ZXh0OiAnZm4gbigkQTogJm11dCBTdHJpbmcpIHt9J1xuZml4OlxuICAnJEE6IFN0cmluZyciLCJzb3VyY2UiOiJmbiB4KGE6ICZtdXQgU3RyaW5nKSB7XHJcbiAgICBcclxufSJ9

I have also added it as a FAQ in ast-grep/ast-grep.github.io@b1b196797bbeafc373b91e924c1fffa4d6634a0b

Cool, so by default the pattern has to parse to a statement like AST? And using the selector field I can pull out the specific part I want to match?
Tool looks awesome BTW!

so by default the pattern has to parse to a statement like AST? And using the selector field I can pull out the specific part I want to match?

Yes! It is! Thanks for the summary I will incorporate it in the doc!


could not launch any new docker container after 1850 containers launched

Output of docker version:
Client:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 22:00:43 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 22:00:43 2016
 OS/Arch:      linux/amd64

Output of docker info:
Containers: 1918
 Running: 1853
 Paused: 0
 Stopped: 65
Images: 2
Server Version: 1.11.2
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 3858
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 4.4.0-21-generic
Operating System: Ubuntu 16.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 24
Total Memory: 188.7 GiB
Name: x3650m4n08
ID: WL45:4OZ5:5ANE:LX2U:HRYA:DEKQ:DNE6:2HY6:CK3E:HJ7I:MAM5:JY6P
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.):
Steps to reproduce the issue:

docker pull yeasy/hyperledger-peer:noops
Run the test script as shown below, the script sets the related limits to be bigger numbers and tries to launch 10000 docker containers.

#!/bin/bash

# Prerequisites setup

echo "root soft nofile 1048576" > /etc/security/limits.conf
echo "root hard nofile 1048576" >> /etc/security/limits.conf
echo "root soft nproc 10485760" >> /etc/security/limits.conf
echo "root hard nproc 10485760" >> /etc/security/limits.conf
echo "root soft stack 32768" >> /etc/security/limits.conf
echo "root hard stack 32768" >> /etc/security/limits.conf
echo "kernel.threads-max = 2091845" >> /etc/sysctl.conf
sysctl -p
echo 210000 > /proc/sys/kernel/pty/max
mkdir -p /etc/systemd/system/docker.service.d/

tee /etc/systemd/system/docker.service.d/tasksmax.conf <<-'EOF'
[Service]
TasksMax=infinity
EOF
systemctl daemon-reload
systemctl restart docker

for i in `seq 1 10`;
do
    brctl addbr docker$i
    ifconfig docker$i 172.$i.254.254/16 up
    docker network create --driver=bridge --ip-range=172.$i.0.0/16 --subnet=172.$i.0.0/16 -o "com.docker.network.bridge.name=docker$i" docker$i
done

for i in `seq 1 10`; do
    for j in `seq 1 1000`; do
        let "z=($i-1)*1000+$j"
        echo $z
        docker run --restart=unless-stopped --net=docker$i -d -e CORE_PEER_ID=vp0 -e CORE_VM_ENDPOINT=http://172.$i.254.254:2375 -e CORE_PEER_ADDRESSAUTODETECT=true -e CORE_NOOPS_BLOCK_TIMEOUT=10 yeasy/hyperledger-peer:noops > /dev/null
    done
done

When launches about 1850 containers, will get "pthread_create failed" error and could not launch any new containers.

Describe the results you received:
The output from the script:
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
docker: Error response from daemon: rpc error: code = 2 desc = "oci runtime error: exit status 2: runtime/cgo: pthread_create failed: Resource temporarily unavailable\nruntime/cgo: pthread_create failed: Resource temporarily unavailable\nSIGABRT: abort\nPC=0x7f9834314418 m=0\n\ngoroutine 0 [idle]:\n\ngoroutine 1 [runnable, locked to thread]:\nruntime.lock(0xc82006c058)\n\t/usr/local/go/src/runtime/lock_futex.go:42 fp=0xc82005fe38 sp=0xc82005fe30\nruntime.chanrecv(0x120dee0, 0xc82006c000, 0x0, 0xc82005ff01, 0x420000)\n\t/usr/local/go/src/runtime/chan.go:401 +0x148 fp=0xc82005fef8 sp=0xc82005fe38\nruntime.chanrecv1(0x120dee0, 0xc82006c000, 0x0)\n\t/usr/local/go/src/runtime/chan.go:349 +0x2b fp=0xc82005ff28 sp=0xc82005fef8\nruntime.gcenable()\n\t/usr/local/go/src/runtime/mgc.go:207 +0x76 fp=0xc82005ff50 sp=0xc82005ff28\nruntime.main()\n\t/usr/local/go/src/runtime/proc.go:73 +0x138 fp=0xc82005ffa0 sp=0xc82005ff50\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82005ffa8 sp=0xc82005ffa0\n\ngoroutine 17 [syscall, locked to thread]:\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1\n\nrax    0x0\nrbx    0x7f98346a3700\nrcx    0x7f9834314418\nrdx    0x6\nrdi    0xa20\nrsi    0xa20\nrbp    0x1ad238a\nrsp    0x7fff3bd65218\nr8     0x7f98346a4770\nr9     0x7f98352e5880\nr10    0x8\nr11    0x202\nr12    0x2897320\nr13    0x1a79070\nr14    0x0\nr15    0x8\nrip    0x7f9834314418\nrflags 0x202\ncs     0x33\nfs     0x0\ngs     0x0\n".
1855
docker: Error response from daemon: rpc error: code = 2 desc = "containerd: container not started".
1856

Describe the results you expected:
As indicated in https://github.com/docker/docker/issues/22978, I could launch more than 9000 containers with the same docker image(different arch of course) and similar hardware configuration on ppc64le platform, I expect to be able to launch similar number of docker containers on x86 also.
Additional information you deem important (e.g. issue happens only occasionally):

With the busybox docker image, I could launch 2735 containers, which is still a very small number. I was able to launch 13000+ busybox containers on a ppc64le platform with similar hardware configuration.

I seem to be running a hard limit of 285 on my setup. I am fairly new to docker so I don't have a base line comparison. However, I loaded your script from above and reached the exact limit of 285.

281
282
283
284
285
286
docker: Error response from daemon: rpc error: code = 2 desc = "oci runtime error: could not synchronise with container process: could not create session key: disk quota exceeded".
287
docker: Error response from daemon: rpc error: code = 2 desc = "oci runtime error: could not synchronise with container process: could not create session key: disk quota exceeded".
288
docker: Error response from daemon: rpc error: code = 2 desc = "oci runtime error: could not synchronise with container process: could not create session key: disk quota exceeded".
289
docker: Error response from daemon: rpc error: code = 2 desc = "oci runtime

I have a pretty heavy duty system, would you happen to have any other configuration recommendations?

betauser@UTraffic100:/opt/apcon/sbin$ sudo docker info
Containers: 968
Running: 285
Paused: 0
Stopped: 683
Images: 13
Server Version: 1.11.2
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 1976
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host null
Kernel Version: 3.16.0-30-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 125.9 GiB
Name: UTraffic100
ID: Y2U3:57NC:NMBU:BYAE:PHRK:PRMF:ODUC:LR3I:KSOI:UKTT:XMLP:HVYD
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support

HI,
The error message ""disk quota exceeded"." indicates that you are running out of disk space,  besides disk space, this blockchain container also consumes memory, my observation is that each 1000 container will need 10GB memory.
I am running my test on a x86 server with 12 CPU cores, 192GB memory and several Tera bytes disk.

Thanks, I did check storage, and there is plenty of storage availabe (3.2TB), I have 20 cores and  128 GB of RAM
none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/dbc8e90dbf56f5cfaa53b59790440c43b09ce11a1e5b9eb575b153edfd950b81 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/0dee5c2390b9601b67c8879958266530a929391ee96c7e02b7361dc0bab4bf75 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/42492a68bed226d52d892b2449cad599d919fb574fa867025ff16b2edf8ef77d none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/4b06a607645a4295860b56a564d3d999fc14d8bbe22c9298aafd9c506496daaf none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/76b3a1d2ffba668406b3c4bb2f277046b938f1671d3aabad31efd9558a4672db none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/ee737f19e38f91ed1d9268503011c1429e86da25d34dbda16b6db02583ef3a19 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/ce7619f3fc9f9add2bf36af0180ebc5f2e65ccc39448d032030d8d18ef60ba0f none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/532bc17ebb42e5fffe743a1f4869ce01197512c94cf63e190ed72819d8d33aca none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/7e116914f5c501cb3d5a148c5e0958af240d3ce7df8c57759d0c6ef3b7f88199 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/d7a830f7ba31c98e272fc8a9c2225da56098f3733d129ae082bd3a4950692bfd none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/d8047986a015d9a74395e96ca7550100cc6a1c4b84a4d24ed1e8c1ce8000129c none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/42f2261e96f3b222e82fdd11eb757875fd6814c53fe45dbed9272e27a488c133 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/afd233345c62fcc737d22911c4972b9f17bb20a36fa6545ad6dbfd749c216dcb none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/619814960c2e5a0a0ce7b85405aa1a8b55033adf3e66375903e4a364a96f2663 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/2647f0a268dc3dcfe86be02fa5572684793a2bf5307b008f899a4bf2f57484c6 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/329f665a46a910941110508e4aba724e99d74ba15fb723d8d581bfe7955f1365 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/8663bd58bf07875383bb41b7c6bbbc5621788a77ce87322692d9bb0236037c3b none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/a85773a3b4d526362da3ef5a3c4dd0db590876e75e347dea0ad55adc29c902de none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/c7667992b56672eda193491ef4cdf21878fe89abdb67d8074bd9791bc4a8ce63 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/6800639178ff9c1a3688c59b124467fdc51ec671cad3b7ed3fa10e43e5125746 none            3.5T  116G  3.2T   4% /var/lib/docker/aufs/mnt/b70a7524e5bacaa911f63f215abbd4a5a66a97ee51f6fbfc7c9487e386bfca7d

is the a volume quote limit imposed per volume that is independent to actual storage?

or, does the user has file system quota? The command "quota " should be able to give some hints.

or, does the user has file system quota? The command "quota <username>" should be able to give some hints.

Interestingly, I updated to Ubuntu 16.04 and now I can get over 1000 containers. I'm also seeing the same limit you are seeing.

@morphyno your issue is related to this https://github.com/docker/docker/pull/23182 - a few old distros/kernel versions have very low session key limits. You can use sysctl -w kernel.keys.root_maxkeys=1000000 before running docker to fix this.

I meet the same problem, interestingly I can run about 3300~3400, but never exceed 3400 containers.
Here is docker log:
ERRO[3040] containerd: start container                   error=oci runtime error: exit status 2: runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0x1221377 m=4

goroutine 0 [idle]:

goroutine 1 [chan receive, locked to thread]:
text/template/parse.(*Tree).peekNonSpace(0xc8200c0540, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:112 +0x169
text/template/parse.(*Tree).itemList(0xc8200c0540, 0xc8202fa8d0, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:328 +0xd4
text/template/parse.(*Tree).parseControl(0xc8200c0540, 0x0, 0x180b360, 0x5, 0x0, 0xa, 0xc820204eb0, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:452 +0x163
text/template/parse.(*Tree).rangeControl(0xc8200c0540, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:494 +0x5c
text/template/parse.(*Tree).action(0xc8200c0540, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:368 +0x4c3
text/template/parse.(*Tree).textOrAction(0xc8200c0540, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:347 +0x8d
text/template/parse.(*Tree).itemList(0xc8200c0540, 0xc8202fa750, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:329 +0x103
text/template/parse.(*Tree).parseControl(0xc8200c0540, 0xc8202f8f00, 0x180b360, 0x5, 0x0, 0x4, 0xc820204dc0, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:452 +0x163
text/template/parse.(*Tree).rangeControl(0xc8200c0540, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:494 +0x5c
text/template/parse.(*Tree).action(0xc8200c0540, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:368 +0x4c3
text/template/parse.(*Tree).textOrAction(0xc8200c0540, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:347 +0x8d
text/template/parse.(*Tree).parse(0xc8200c0540, 0xc8202fa660, 0x0, 0x0)
	/usr/local/go/src/text/template/parse/parse.go:292 +0x684
text/template/parse.(*Tree).Parse(0xc8200c0540, 0x1b6df80, 0x172, 0x0, 0x0, 0x0, 0x0, 0xc8202fa660, 0xc8202fc250, 0x2, ...)
	/usr/local/go/src/text/template/parse/parse.go:231 +0x269
text/template/parse.Parse(0x181f7a0, 0x9, 0x1b6df80, 0x172, 0x0, 0x0, 0x0, 0x0, 0xc8202fc250, 0x2, ...)
	/usr/local/go/src/text/template/parse/parse.go:54 +0x14c
text/template.(*Template).Parse(0xc8202f8f80, 0x1b6df80, 0x172, 0xc8201b7bd0, 0x0, 0x0)
	/usr/local/go/src/text/template/template.go:195 +0x2ab
html/template.(*Template).Parse(0xc8202fa630, 0x1b6df80, 0x172, 0x0, 0x0, 0x0)
	/usr/local/go/src/html/template/template.go:158 +0xb8
net/rpc.init()
	/usr/local/go/src/net/rpc/debug.go:39 +0xf8
github.com/ugorji/go/codec.init()
	/go/src/github.com/ugorji/go/codec/time.go:193 +0xa7
github.com/coreos/etcd/client.init()
	/go/src/github.com/coreos/etcd/client/srv.go:65 +0x8d
github.com/docker/libkv/store/etcd.init()
	/go/src/github.com/docker/libkv/store/etcd/etcd.go:593 +0x6a
github.com/docker/libnetwork/datastore.init()
	/go/src/github.com/docker/libnetwork/datastore/mock_store.go:129 +0x74
github.com/docker/libnetwork/config.init()
	/go/src/github.com/docker/libnetwork/config/config.go:251 +0x59
github.com/docker/libnetwork.init()
	/go/src/github.com/docker/libnetwork/store.go:463 +0x6f
github.com/docker/docker/api/server/router/network.init()
	/go/src/github.com/docker/docker/api/server/router/network/network_routes.go:277 +0x47
main.init()
	/go/src/github.com/docker/docker/docker/flags.go:30 +0xb0

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 6 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 7 [runnable]:
text/template/parse.lexRightDelim(0xc82020ec80, 0x1b405c0)
	/usr/local/go/src/text/template/parse/lex.go:272 +0xc1
text/template/parse.(*lexer).run(0xc82020ec80)
	/usr/local/go/src/text/template/parse/lex.go:206 +0x52
created by text/template/parse.lex
	/usr/local/go/src/text/template/parse/lex.go:199 +0x15d

rax    0x0
rbx    0x22e3208
rcx    0x1221377
rdx    0x6
rdi    0xe3b
rsi    0x4dda
rbp    0x1b91a75
rsp    0x7fcd654c8838
r8     0xa
r9     0x7fcd654c9700
r10    0x8
r11    0x202
r12    0x7fcd580008c0
r13    0x1b389a8
r14    0x0
r15    0x8
rip    0x1221377
rflags 0x202
cs     0x33
fs     0x0
gs     0x0
 id=2ae84ccac4a3d8680f385c9e72e78a0ad58e185a15e1752fde98998c59599d34

docker version: latest master branch.  (can't get in touch of my server, will add HASH later)
OS:   ubuntu 14.04
how to run docker daemon: manually start dockerd on command line, didn't use systemd
I can see there's some related issues: https://github.com/docker/docker/issues/21298 https://github.com/docker/docker/issues/9868 https://github.com/docker/docker/issues/22690,
the suggestion is to set TasksMax=infinity in systemd config.  But I didn't use systemd at all, and I can see @ligc already sets this.
I've tried several days, but get no luck on this.

@morphyno, the Linux bridge could only attach 1024 ports,  if we want to attach more ports to the Linux bridge, we will have to update the Linux kernel code and recompile the kernel, using multiple Linux bridges is a simple workaround for this issue.

@WeiZhang555,  for the 3300-3400 containers in your environment, are they busybox containers or blockchain containers yeasy/hyperledger-peer:noops?

@ligc  Here's configurations:
# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 514074
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1000000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 514074
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
# sysctl -n kernel.threads-max
1028148
# cat /proc/sys/kernel/pty/max
210000

I've tried Docker version 1.9.0, build 76d6bc9, and Docker version 1.11.0, build bb0e2b2, experimental, they got same result.
I'm tuning the system configurations, and will share to you if I make any progress.

@ligc quick updates, I found the solution of my problem.  docker exhausts pid numbers of my system, only need to set a bigger limit for pid_max: echo 655360 > /proc/sys/kernel/pid_max . Original number is 32768 which is too small.
Hope this can be helpful to your problem too.

HI,
Yes, I set the /proc/sys/kernel/pid_max 4194304 and then I could launch 10000 blockchain containers on my x86 server now. Thanks a lot. This issue could be closed.

@ligc @WeiZhang555 do you think this is important enough to include in the check-config.sh script? Similar to https://github.com/docker/docker/blob/master/contrib/check-config.sh#L290-L292 ?

@thaJeztah I am not sure if it is important enough to include in the check-config.sh script, this is kind of stress test configuration, running several thousands containers on a single server may not be necessary in most of the customers production systems. Thanks.

Thanks for checking, @ligc

@thaJeztah I think it's important, adding a simple check will save users like us a lot of time, at least it gives us some hints about what wrong with it.
I'll do it.

@ligc, can you point to me about the kernel config changes required to exceed 1024 interfaces for 1 bridge?

@morphyno I did not attach more than 1024 containers to a single Linux bridge during my test, I was using multiple bridges, attach 1000 containers to each bridge.


An advanced profiler

Do you have any idea about better profiler, like:

Isn't this example a little over-engineered, some of these profiles (such as the L2 data access) sound like it would require a degree in computer science to even understand (nonetheless figuring out what in Godot would cause such to go up and down)?
By extension, the vast majority of developers would really not have a use for some of those fields since they would not understand them.

Hey,
This feature request is for an ADVANCED profiler, there is a basic profiler somewhere?
After looking a bit on the debugger options, I could only find

The "basic profiler" could be the performance and videomem tab?
Also for me the info tab Clicked control/Clicked Control Type/Live Edit Root doesn't make sense, what is the purpose of those three textbox?
What I'm looking for is a way to profile methods, to find bottlenecks on code

How about adding something simple like frame debugger in Unity. It does steps through the rendering and we could know how the engine renders and identify problems like overdraw etc.
Ref(if needed): http://blogs.unity3d.com/2014/07/29/frame-debugger-in-unity-5-0/

this is done now :)
for frame debugger you generally need hardware support, this is quite complex as far as I understand, but you can use external tools for this, there are plenty of opengl debuggers.


Updated CLI documentation to mention the issue with underscores and click 7

When you use Flask in combination with Click 7.x, command containing underscores are no longer found, but instead exposed using dashes, so the example in the docs create_user only works as create-user.
I've extended the docs to mention this, including an example and a workaround.
The issue is mentioned here as well:
https://github.com/pallets/flask/issues/2931
https://github.com/pallets/click/issues/1123
This is the commit in Click that changes the behaviour:
https://github.com/pallets/click/commit/5d1df0e042b2f8b0dee8f6dd9ce74edce331a950

Thanks for working on this. I updated the docs to use dashes instead in #3269.


Base typography styles match neither spec styles or microsite styles

The base typography styles shown on the Styles page do match the typography style of the microsite or the Material Design Spec. Further, other styles, such as those used for the table of contents, are not described on the page and do not seem available. Are there plans to have the styles match one of the two, preferably the Material Design Spec?

The styles currently shown there were crafted with input from the MD team at a time where little typography guidance for the web was available. I think it's worth us reevaluating how far the spec has come here and see if there's more we can do to align.


core / log views - use built-in selectpicker event to handle async init

Hi!
Can we try using built-in event to handle async selectpicker init (should be faster and more reliable)?
sorry, i’m also not very clear about class check part. if this is about https://github.com/snapappointments/bootstrap-select/issues/2756 , do we really need to care about the select class if we assign the selectpicker not to the class, but to the id?
Thanks!
upd. one moment..

@AdSchellevis hm. When re-init was removed from page load, the interference appeared of auto-init (by 'selectpicker' class) and select mode application  on page loading. I removed the selectpicker class to eliminate auto-init.

@kulikov-a there are more cases where auto-init is problematic, removing the class isn't an issue. If this works as it should, I'll test on my end and merge. At a first glance this looks good.

Thanks!


Custom components in Field are passed props.input and props.meta objects

When a custom component is passed to Field, it receives props.input, props.meta objects followed by any custom props declared in Field. This seems contrary to the docs, according to which 'props' is a flattened object (e.g. onChange should be available as props.onChange rather than props.input.onChange)
Example:
const SimpleInput = (props) => {
  console.log('SimpleInput props = ', props);
  return <input type="text" onChange={e => props.input.onChange(e)} value={props.input.value}/>
}

...

<Field name="meow" component={SimpleInput} placeholder="My cat's name"/>

Log from above:
{
  "input":{"name":"meow2","onBlur":"()","onChange":"()", ... "value":""},
  "meta":{"active":false,"asyncValidating":false,"autofilled":false,"dirty":false,"invalid":false,"pristine":true,"submitting":false,"submitFailed":false,"touched":false,"valid":true,"visited":false},
  "placeholder":"My other cat Name"
}

Given the mismatch between implementation and docs, would be great to know which is correct (intended behaviour). Thanks a lot!

Because this issue saw almost no activity for almost a year, I'm closing it.

Hi.  Sorry to zombify this issue.  I can open another if preferred, but hopefully we can just reopen this one.
The FAQ section of the docs is misleading.
https://redux-form.com/7.2.3/docs/faq/customcomponent.md/

The minimum interface needed for a custom component to work with redux-form is to make sure that value and onChange are passed properly. These are pretty standard prop names, so it's possible that your component will work right out of the box.

Sure, props.onChange and props.value are standard.  However, props.input.value and props.input.onChange are not standard.  The example immediately below this paragraph clarifies everything, but I think the misleading paragraph should be removed or improved.


Swap use of lockfree

lockfree allocates in send and deallocates in recv
Maybe use something like ringbuf instead

#98


ODH 2.12 Release Tracker

Links to tracker issues for components planning updates in this.  Please update you manifests before May 10th.
ODH Release slated for May 13th.

[ ]  ODH Operator
[ ]  ODH Dashboard
[ ]  Workbench
[ ]  Data Science Pipelines
[x]  Model Serving
[x]  AI Explainability
[ ]  Distributed Workloads

[ ] Codeflare
[ ] Kuberay
[ ] Kueue
[ ] Training Operator (incubation)

[ ] Model Registry

For model serving:

kserve: https://github.com/opendatahub-io/kserve/releases/tag/v<IP_ADDRESS>
modelmesh-serving: https://github.com/opendatahub-io/modelmesh-serving/releases/tag/v<IP_ADDRESS>-rc0
odh-model-controller: https://github.com/opendatahub-io/odh-model-controller/releases/tag/v<IP_ADDRESS>

AI explainability:
Release notes: https://github.com/trustyai-explainability/trustyai-service-operator/releases/tag/v1.20.0
Release branch: https://github.com/trustyai-explainability/trustyai-service-operator/tree/release/1.20.0

(Also, service's release notes here)

Model Registry v0.2.0:
Tag: https://github.com/opendatahub-io/model-registry-operator/releases/tag/v0.2.0
Branch: https://github.com/opendatahub-io/model-registry-operator/tree/release/v0.2.0

DSP v2.2: https://github.com/opendatahub-io/data-science-pipelines-operator/releases/tag/v2.2.0

Workbench IDE:
Notebooks
Tags: v1.18.0
Release-Note: https://github.com/opendatahub-io/notebooks/releases/tag/v1.18.0
Notebook-controller:
Tags: v1.7.0-9
Release-Note: https://github.com/opendatahub-io/kubeflow/releases/tag/v1.7.0-9
Please point the operator to these versions for the workbenches.
Thank you for your patience.

Dashboard

Release notes: https://github.com/opendatahub-io/odh-dashboard/releases/tag/v2.22.2-odh
Release branch: https://github.com/opendatahub-io/odh-dashboard/tree/v2.22.2-odh-release

Please use the release branch in the Operator build. We have moved away from incubation and now are off main fully.

Hi @KPostOffice @dimakis Can you confirm we can continue with previous versions of manifests for DW?

new release for CodeFlare Operator https://github.com/opendatahub-io/codeflare-operator/releases/tag/v1.4.1-odh

The rest are fine


Unexpected "return" error with<EMAIL_ADDRESS>

Importing Astro SEO and then compiling with<EMAIL_ADDRESS>throws the following error:
6:03:57 AM [vite] Error when evaluating SSR module /src/pages/index.astro:
Error: Transform failed with 1 error:
/src/SEO.astro:120:0: error: Unexpected "return"

Right now, my best guess is that this is a bug in the new compiler, since I can't identify an unexpected return at line 120 or anywhere else.

Openend an issue upstream here: https://github.com/snowpackjs/astro/issues/1719


Use global import for Tabs and TabItem

Signed-off-by: Billy Tat<EMAIL_ADDRESS>

Was this a swizzle of the default components? Like talked about here?

Yup. Swizzled MDXComponents from @docusaurus/theme-classic. The only additions I added were the tab-related lines (12-13 and 30-31) in src/theme/MDXComponents/index.js.


add an example app to test stream

this example is a simple dummy stream to check if terminate is actually
called

looking at the log in console you will see that the stream is restarted (init is called but the message in the terminate callback is never displayed:
INFO REPORT==== 24-May-2017::16:32:24 ===
stream 1 got info: {data,nofin,["id: ","75","\ndata: ","Tick","\n\n"]}
=INFO REPORT==== 24-May-2017::16:32:25 ===
stream 1 got info: {data,nofin,["id: ","76","\ndata: ","Tick","\n\n"]}
=INFO REPORT==== 24-May-2017::16:32:29 ===
init stream: 1
req: #{body_length => 0,has_body => false,
       headers =>
           #{<<"accept">> => <<"text/event-stream">>,
             <<"accept-encoding">> => <<"gzip, deflate, sdch, br">>,
             <<"accept-language">> => <<"fr,en-US;q=0.8,en;q=0.6">>,
             <<"cache-control">> => <<"no-cache">>,
             <<"connection">> => <<"keep-alive">>,<<"dnt">> => <<"1">>,
             <<"host">> => <<"localhost:8080">>,
             <<"last-event-id">> => <<"76">>,
             <<"referer">> => <<"http://localhost:8080/">>,
             <<"user-agent">> =>
                 <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36">>},
       host => <<"localhost">>,method => <<"GET">>,path => <<"/eventsource">>,
       peer => {{127,0,0,1},55097},
       pid => <0.457.0>,port => 8080,qs => <<>>,ref => http,
       scheme => <<"http">>,streamid => 1,version => 'HTTP/1.1'}

=INFO REPORT==== 24-May-2017::16:32:29 ===
stream 1 got info: {headers,200,
                            #{<<"content-type">> => <<"text/event-stream">>,
                              <<"date">> =>
                                  <<"Wed, 24 May 2017 14:32:29 GMT">>,
                              <<"server">> => <<"Cowboy">>}}
=INFO REPORT==== 24-May-2017::16:32:30 ===
stream 1 got info: {data,nofin,["id: ","77","\ndata: ","Tick","\n\n"]}
=INFO REPORT==== 24-May-2017::16:32:31 ===

Fixed in most recent commit. Thanks!


Update README.md to remove stable badge for docs

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (cff09ad) 91.89% compared to head (12d92c0) 91.89%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master       #7   +/-   ##
=======================================
  Coverage   91.89%   91.89%
=======================================
  Files           6        6
  Lines          74       74
=======================================
  Hits           68       68
  Misses          6        6

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Question about the swin-tiny config

Dear Author, Thanks for your great work! I read the paper you have do the experiments about the swin-tiny with and
w/o ImageNet-21k pretraining and distillation. Can you share the config and code here? Thx~

Thanks for your attention to our work!
We use the same hyper-parameter as TinyViT to train Swin-T with and w/o ImageNet-21k pretraining and distillation.
You only need to change the model in the config.
For example, pretrain Swin-T on ImageNet-21k with distillation. Reference: tiny_vit_21m_22k_distill.yaml.
MODEL:
  NAME: swin_tiny_patch4_window7_224
  TYPE: swin

  DROP_PATH_RATE: 0.1

  SWIN:
    EMBED_DIM: 96
    DEPTHS: [ 2, 2, 6, 2 ]
    NUM_HEADS: [ 3, 6, 12, 24 ]
    WINDOW_SIZE: 7

TRAIN:
  EPOCHS: 90
  BASE_LR: 2.5e-4
  WARMUP_EPOCHS: 5
  WEIGHT_DECAY: 0.01

DATA:
  DATASET: imagenet22k

AUG:
  MIXUP: 0.0
  CUTMIX: 0.0


zerocode custom-archetype

GIVEN the archetype/resources are available
WHEN I run the following/similar maven command
mvn archetype:generate -DarchetypeGroupId=<custom-archetype group id e.g.>
    -DarchetypeArtifactId=<custom-archetype artifactid>
    -DarchetypeVersion=<custom-archetype version>
    -DgroupId=<new project Group Id>
    -DartifactId=<new project artifact Id>

THEN I will generate the Zerocode maven project with an example test scenario.
Description
The output of the above command will produce
//TODO - structure of the project with a list of Java and resources files.
//TODO - attach a sample maven project which is expected as output(.zip file)
- src/main/java
  - com.myproject
    - utils
      - MyUtils.java

- src/test/java
  - com.myproject
     - tests
       - MyGetApiTest.java
       - MyPostApiTest.java
       - MyPutApiTest.java
     - MyApiSuite.java   <----- To run the entire test suite

- src/test/resources
  - tests
     - get_api_200.json
     - post_api_200.json
     - put_api_200.json
  - hostconfig_ci.properties
  - hostconfig_sit.properties

Implementation Help

Step by step approach - https://dzone.com/articles/creating-a-custom-maven-archetype

Running examples - https://www.baeldung.com/maven-archetype

@authorjapps I would like to contribute.

@waqassiddiqi - Great to have you helping!
In addition to the links @authorjapps posted earlier, below are some helpful extra bits if required:
In the archetype-metadata.xml file, if you set "filtered" to true for a file/folder structure then the engine will parse any token starting with a $, if you want to create the archetype with $ symbols which I suspect you will due to the token utilities containing a $ (i.e. ${SYSTEM.PROP:username} then you can do the following:

Here I have set a variable named $MONGO_CONNECTION_URI and given it the value ${MONGO_CONNECTION_URI}, this means that when I reference the variable $MONGO_CONNECTION_URI anywhere in the code it will be interpreted and replaced by the value ${MONGO_CONNECTION_URI} in the generated project. Without this approach, the engine would try to parse ${MONGO_CONNECTION_URI} as a variable which wouldn't work as I have no variables in the project with that name -> Admittedly this isn't the clearest concept to grasp so do reach out if you have any questions.
If you want to name the files using one of the maven archetype properties you can encase the file with __propertyName__, two underscores before and after the property name, in my example below I created a custom property called entityName and I used the value of that to give the file names meaning:

When the project is generated, the property name is interpreted and transformed to become "ItemApplication.java` if the entityName property was set to Item.
Finally, you can use IF statements inside the maven archetype, in the example below I have another custom property called includeGetById, I check if the value is equal to true and then if it is, I include a line of Java code, this means that if the condition fails, the generated code would not include this line, if the condition equated to true then the line would appear in the generated code, this is useful for complex scenarios.

If you need anything else please reach out, good luck!

Hello @authorjapps  I'd like to contribute. Is it ok to submit a pull request if it's assigned already?

Wiki page created to capture the basic CLI. Feel free to add/update when you find anything relevant please!
Wiki link: here

Available in version 1.3.17 and higher releases.


Rya 529 fix boundless

Description
fixed changed URL's for locationtech and boundless
Tests
full build succeeded
Links
Jira
Checklist

[ ] Code Review

This was targeting the 3.x branch.  I'm told we are no longer supporting it, so I am closing it unapplied.


Support for ArubaOS CX

Support for ArubaOS-CX virtual switches.
Brutally tested also on containerlab with the following topology:
name: test-aoscx

mgmt:
  ipv4-subnet: <IP_ADDRESS>/24

topology:
  nodes:
    r1:
      mgmt-ipv4: <IP_ADDRESS>
      kind: vr-veos
      image: vrnetlab/vr-aoscx:20230531220439
      runtime: docker
    r2:
      mgmt-ipv4: <IP_ADDRESS>
      kind: vr-veos
      image: vrnetlab/vr-aoscx:20230531220439
      runtime: docker

  links:
  - endpoints:
    - "r1:eth1"
    - "r2:eth1"

Thanks @ssasso !


Correct iteration order of ColumnNameSet

It fixes #149 .

It is published to 0.8.3.BUILD-SNAPSHOT, target version is 0.8.3.RELEASE.


text when highlighting a comment is not properly visible/discernable

In the above screenshot when a comment is selected as you can see the comment text is not readable

Maybe  will be a better option for comment foreground .

maybe if you approve then I can send a PR

I was about to make an issue on this thing but it already existed lol

Yes, please submit a PR and I will approve. Thank you for your help!


String immutability

Challenge Understand String Immutability has an issue.
User Agent is: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:

// Setup
var myStr = "Jello World";

// Only change code below this line

var myStr = "Jello World";
myStr = "Hello World";
myStr[0] = "H"; // Fix Me

@MiltonRoemer
Thanks for reporting this. But your code is incorrect.
GitHub Issues are for reporting bugs on the website only
Please use the HelpJavaScript chat room for getting challenge related help.
Happy Coding.

Closing as it is for code help.
As directed by @Deadpool37 please use the Help button in the challenge you have an issue with, this will take you to a dedicated chat room for help with your code.
Happy coding :+1:


truegaze can't find config file

Identified as an Android application via a manifest located at: AndroidManifest.xml
Scanning using the "AdobeMobileSdk" plugin
-- Cannot find the "ADBMobileConfig.json" file, skipping

This is not an error - it indicates that this application isn't integrated with Adobe and the test will be skipped. I changed the message to clarify this.


Can't download whole channel saved on waybackmachine (web archive)

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Poland
Provide a description that is worded well enough to be understood
I am trying to download from this page (whole channel<EMAIL_ADDRESS>Whenever I try it just says unsupported website. For example if I try to download from this saved channel only single video it downloads without issue (even tho it's the same website): https://web.archive.org/web/20230617094121/https://www.youtube.com/watch?v=h1Iany8Etas
Is there anything to do to fix it? Why does it happen?
Thanks in advance
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp -vU<EMAIL_ADDRESS>[debug] Command-line config: ['-vU'<EMAIL_ADDRESS>[debug] Encodings: locale cp1250, fs utf-8, pref cp1250, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [4d9231208] (pip)
[debug] Python 3.12.3 (CPython AMD64 64bit) - Windows-11-10.0.22631-SP0 (OpenSSL 3.0.15 3 Sep 2024)
[debug] exe versions: ffmpeg 7.0.2-essentials_build-www.gyan.dev (setts), ffprobe 7.0.2-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, mutagen-1.47.0, requests-2.32.3, sqlite3-3.45.3, urllib3-2.2.2, websockets-13.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Plugin directories: ['C:\\Users\\xdekhckr\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\yt_dlp_plugins']
[debug] Loaded 1830 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[generic] Extracting URL<EMAIL_ADDRESS>[generic] videos: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] videos: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL<EMAIL_ADDRESS>Traceback (most recent call last):
  File "C:\Users\xdekhckr\AppData\Local\Programs\Python\Python312\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1626, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xdekhckr\AppData\Local\Programs\Python\Python312\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1761, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "C:\Users\xdekhckr\AppData\Local\Programs\Python\Python312\Lib\site-packages\yt_dlp\extractor\common.py", line 740, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xdekhckr\AppData\Local\Programs\Python\Python312\Lib\site-packages\yt_dlp\extractor\generic.py", line 2526, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL<EMAIL_ADDRESS>

It's not currently supported


Add git remote connection check code

Slight reworking of https://github.com/ManageIQ/manageiq/pull/20645
This is basically @NickLaMuro's PR, but with the check_connection method modified. Instead of making a Rugged connection on the API endpoint, we just perform an http ping on the repo.
I made these changes to Nick's PR for a couple reasons. First and foremost, the Rugged code was segfaulting with a difficult to diagnose error that indicates that, somewhere in the guts of either rugged or libgit2, it's trying to free something that it didn't allocate. Issue reported here: https://github.com/libgit2/rugged/issues/859. Unable to solve that, we had to find a different way.
Since the original issue was really a generic connectivity issue, and not specifically about the github API failing to return expected data, I felt the original rugged code was too heavy. It's the equivalent of a git ls-remote, which fetches data under the hood, and we don't need to fetch data, we just need to test for connectivity.
I chose net-ping as the solution because it was already a dependency in ManageIQ, because I'm the original author, and because it solves our needs. Although I have not been the maintainer for many years now, the code hasn't changed dramatically, so I have good familiarity with it.
@tinaafitz, @billfitzgerald0120, @d-m-u and I tested this on a vmware host where we temporarily disabled the network in the middle of provisioning a catalog item and it worked as expected, i.e. it would check the connection X number of times, and proceed if the connection was restored.
Original bug: https://bugzilla.redhat.com/show_bug.cgi?id=1835226

Cool idea.  @NickLaMuro Please review.

This set ups the framework, but how do you envision this method to be used? (I don't see anything ultimately calling check_connection in a flow).

This set ups the framework, but how do you envision this method to be used? (I don't see anything ultimately calling check_connection in a flow).

I think @tinaafitz or @billfitzgerald0120 can provide an example.

@Fryguy this is intended to be called from the generic state machine, the automate side:
module ManageIQ
  module Automate
    module Service
      module Generic
        module StateMachines
          module GenericLifecycle
            class CheckConnection
...

def check_connection(service)
                rc = service.check_connection(action = "Provision")
                #  check_connection returns true if connection is successful, false otherwise
                if !rc
                  @handle.root['ae_result'] = 'retry'
                  retry_interval
                  else
                  @handle.root['ae_result'] = 'ok'
                end
              end

so that when running a playbook if the connectivity isn't present we can handle the retry more gracefully.

rc = service.check_connection(action = "Provision")

Then I wonder if it's better to rename the method to be a predicate.  Something like check_connection? valid_connection?, connection_valid?

rc = service.check_connection(action = "Provision")

Then I wonder if it's better to rename the method to be a predicate. Something like check_connection? valid_connection?, connection_valid?

I renamed it to check_connection?.

@djberg96 Can this be ivanchuk/yes (which implies jansa/yes and kasparov/yes)?

@dmetzger57 what say you?

Yes to ivanchuk/yes

@miq-bot add_label ivanchuk/yes

Ivanchuk backport details:
$ git log -1
commit 0ce905f5f9515848587f6cc3225ef9f89881f86c
Author: Keenan Brock<EMAIL_ADDRESS>Date:   Wed Nov 4 11:39:58 2020 -0500

    Merge pull request #20759 from djberg96/better_check_connection

    Add git remote connection check code

    (cherry picked from commit 41f40f81f619562e5334f94543dba8e0da42adb3)

    https://bugzilla.redhat.com/show_bug.cgi?id=1835226

Jansa backport details:
$ git log -1
commit 47b8134287e94cfe3b88be5500468615cabd03bf
Author: Keenan Brock<EMAIL_ADDRESS>Date:   Wed Nov 4 11:39:58 2020 -0500

    Merge pull request #20759 from djberg96/better_check_connection

    Add git remote connection check code

    (cherry picked from commit 41f40f81f619562e5334f94543dba8e0da42adb3)

Kasparov backport details:
$ git log -1
commit bb658d4f210764c81c1437b8e705816875fbcb27
Author: Keenan Brock<EMAIL_ADDRESS>Date:   Wed Nov 4 11:39:58 2020 -0500

    Merge pull request #20759 from djberg96/better_check_connection

    Add git remote connection check code

    (cherry picked from commit 41f40f81f619562e5334f94543dba8e0da42adb3)


C#: Encode GodotProjectDir as Base64 to prevent issues with special characters

Fixes https://github.com/godotengine/godot/issues/73932.

Thanks!

After this PR my build fails:
./modules/mono/build_scripts/build_assemblies.py --godot-output-dir=./bin \   --push-nupkgs-local ~/dev/LocalNugetSource
/home/geo/dev/godot/modules/mono/editor/Godot.NET.Sdk/Godot.SourceGenerators.Sample/Godot.SourceGenerators.Sample.csproj(10,5):
error MSB4186: Invalid static method invocation syntax: "[MSBuild]::ConvertToBase64('$(GodotProjectDir)')".
Method '[MSBuild]::ConvertToBase64' not found. Static method invocation should be of the form: $([FullTypeName]::Method()), e.g. $([System.IO.Path]::Combine(`a`, `b`)).
Check that all parameters are defined, are of the correct type, and are specified in the right order.

dotnet --info:

.NET SDK (reflecting any global.json):
 Version:   6.0.114
 Commit:    346c0065dd

Runtime Environment:
 OS Name:     arch
 OS Version:
 OS Platform: Linux
 RID:         arch-x64
 Base Path:   /usr/share/dotnet/sdk/6.0.114/

Host:
  Version:      7.0.3
  Architecture: x64
  Commit:       0a2bda10e8

.NET SDKs installed:
  6.0.114 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.NETCore.App 6.0.14 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  DOTNET_ROOT       [/usr/share/dotnet]

global.json file:
  Not found

Your version of MSBuild is likely outdated, what is the output of dotnet msbuild --version? I think this was added in 17.3.0.

Indeed, I have a version < 17.3.0.
❯ dotnet msbuild --version
Microsoft (R) Build Engine version 17.0.1+b177f8fa7 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

<IP_ADDRESS>01%

How to I update it? I think it's bundled by my distro.

MSBuild is part of the .NET SDK, can you update it?

I've flag the arch packages as out-of-date.

I think we should look for an alternative that doesn't require a newer MSBuild. Otherwise, we can't include it in 4.0.x as it's a breaking change. Is there a way in MSBuild to check if a method is available?

I agree with @neikeq project build are going to fail with this kind of errors:

Which is not user friendly at all.
And is a breaking change.

Cherry-picked for 4.0.2 (alongside https://github.com/godotengine/godot/pull/74479).


request "display raid frames intead of party frames"/ option to hide the raid frame

Description

before in version 6.39 there was an option to show the party frame as a raid frame, now I can not find this option in the settings.
in some characters I use the vuhDo addon to heal, and the option to hide the raid instead of disable would be very good, could look like the Blizzard raid frame option.
https://imgur.com/a/HEtLh
Version Number

v7.11
Steps to Reproduce

Step 1
Step 2
Step 3
Step 4

Expected Behavior

Actual Behavior

Only KkthnxUI Enabled

Additional Information

Sorry for bad English =/
Error Log

Going to close this as this is added in the latest beta release.


Unable to start ngshare service with db open error

Run ngshare docker image from docker hub or start according to configuration in jupyterhub_config.py, both showed error
message:sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
I create file /tmp/ngshare.db and gives the write permission, and error still.

After adding /srv/ngshare/ngshare.db, it works now.


Video Player is black and I can't save my video. 14 sec video and 20 sec video lost because of this.

Please fix the Video Player. It won't work. It loads infinitely and when I try to go back to editing the vid, it loads infinitely too! Please help!

Hello. Can you provide more context on what happened?
What was in the video?
Localhost, or public?
Were you messing with any code if localhosted?
This would make it way easier to find out what happened. Thanks in advance.

Oops, sorry for closing :/


Update cluster-vsphere chart

prerequisite for this is to have a healthy cluster-api-provider-vsphere-app to be able to test the infra capv resources against the working and properly deployed provider
so creating a sub-task:
### Tasks
- [ ] https://github.com/giantswarm/giantswarm/issues/25783
- [ ] https://github.com/giantswarm/roadmap/issues/2015

draft pr: https://github.com/giantswarm/cluster-vsphere/pull/23

@erkanerol Please review this PR.
Reminder: You can use @jkremser's mc-bootstrap hacks to test.


Support rendering JS before attempting to extract.

Checklist

[x] I'm reporting a feature request
[x] I've verified that I'm running youtube-dl version 2021.01.16
[x] I've searched the bugtracker for similar feature requests including closed ones

Description
Vimeo extractor for embedded pages needs to be called from the page embedding it. Problem is, ydl doesn't render JS, and the page makes the video appear only after rendering the jQuery:
<script type="text/javascript">

	jQuery(document).ready(function() {
		var video_target = 'https://maailmakool.ee/?download_file=16572&order=wc_order_AY21DpMoz3a9Y&email=tiia%40akg.vil.ee&key=0cad466f2e85701a945cf80f43b2f085';
		jQuery.ajax({
			url: video_target,
			success: function(result) {
				if (typeof(result.video_redirect) != 'undefined') {
					window.location.href = result.video_redirect;
				} else {
					jQuery('iframe#video_iframe').attr('src', 'https://player.vimeo.com/video/' + result.video_id + '?texttrack=et');
				}
			}, complete: function(result) {
				if (typeof(result.responseJSON) != 'undefined' && result.responseJSON) {
					if (typeof(result.responseJSON.code) != 'undefined' && result.responseJSON.code == 'wp_die') {
						jQuery('#video_iframe').remove();
						jQuery('#video_title').replaceWith(result.responseJSON.message);
					}
				} else {
					window.location.href = mk_embedder.root;
				}
			}, dataType: 'json'
		});
	});

	</script>
	<iframe id="video_iframe" width="1080" height="608" frameborder="0" allow="fullscreen" allowfullscreen></iframe>

A workaround could only be implemented for Vimeo, but I'd say that implementing this feature would possibly fix (in the future) many other extractors, simply by rendering JS.

Source: #11196

Source: #11196


Exclude readonly/system columns from inclusion in MERGE

We have a temporal table that means we won't be inserting into all columns (this also applies for where data may be optional etc). The SQL MERGE generator uses the INFORMATION_SCHEMA to get columns to be used in the MERGE. This means that columns that aren't contained in our class will be included in the MERGE, causing it to fail.
The MERGE code generator should subset the list of columns to be MERGED based on the class definition to prevent excess columns being included in the SQL and causing faults.

A SQL DB/ SQL Server specific solution, might be to use the sys views eg
select
c.name as ColumnName,
types.name +
    case
        when types.name in ('decimal', 'numeric') then '(' + cast(c.[precision] as varchar(9)) + ',' + + cast(c.scale as varchar(9)) + ')'
        when c.scale > 0 then '(' + cast(c.scale as varchar(1)) + ')'
        when c.max_length = -1 then '(max)'
        when c.max_length <> -1 then '(' + cast(c.max_length as varchar(4)) + ')'
        else ''
    end as ColumnDefinition
from sys.columns c
inner join sys.tables t on t.object_id=c.object_id
inner join sys.types on c.user_type_id=types.user_type_id
where t.name = 'load'
and c.generated_always_type = 0
and c.is_computed = 0

A SQL DB/ SQL Server specific solution, might be to use the sys views eg
select
c.name as ColumnName,
types.name +
    case
        when types.name in ('decimal', 'numeric') then '(' + cast(c.[precision] as varchar(9)) + ',' + + cast(c.scale as varchar(9)) + ')'
        when c.scale > 0 then '(' + cast(c.scale as varchar(1)) + ')'
        when c.max_length = -1 then '(max)'
        when c.max_length <> -1 then '(' + cast(c.max_length as varchar(4)) + ')'
        else ''
    end as ColumnDefinition
from sys.columns c
inner join sys.tables t on t.object_id=c.object_id
inner join sys.types on c.user_type_id=types.user_type_id
where t.name = 'load'
and c.generated_always_type = 0
and c.is_computed = 0


Update make:model to remove namespace from ModelFactory filename

This is so trivial I almost didn't submit it, but ...
artisan make:model Models\Thingy -f (or artisan make:model Models\Thingy -a) creates a Thingy model in the Models directory\namespace and creates a ModelsThingyFactory class. This is inconsistent with the creation of the controller and migration, which use only the un-namespaced class name (ThingyController and 2018_01_01_000001_create_thingy_table.
This PR changes the factory class name from ModelsThingyFactory to ThingyFactory.

This has been tried a couple of times. See #23151 and #23521.

So it has - I must have just searched open cases.
Oh well.


Can't find the title bar since ver 0.6

See the attachment, many thanks.

Same here.

fixed in #25

Really thanks.


please help. I cannot use ctrl+c to copy code in my terminal

ADD ISSUE DESCRIPTION HERE
I need to add the code to github to create the ssh key
Version: 1.85.2
Commit: 8b3775030ed1a69b13e4f4c628c612102e30a681
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
Embedder: codespaces

I just found out that right clicking in the terminal will copy it.

/question

We closed this issue because it is a question about using VS Code rather than an issue or feature request. Please search for help on StackOverflow, where the community has already answered thousands of similar questions. You may find their guide on asking a new question helpful if your question has not already been asked. See also our issue reporting guidelines.
Happy Coding!


Supporting Coins that don't use SHA256D for Addresses

We would like to add support for GRS to your repo.
GRS (Groestlcoin) is unique because it does not use SHA256D for the checksum on extended keys and addresses, but instead it uses double groestl (sha3 candidate).
We are wondering what the best way is to make code changes to submit as a pull request.
In the past, I had modified bitcoinjs to use the correct hash function based on added parameters added to some of the functions.  This solution makes it hard to upgrade bitcoinjs after changes are made.
Another option may be to include a groestlcoinjs library.  My concern with this is that there may be name conflicts with bitcoinjs, as they use the same names for functions.  Perhaps JavaScript has a way around this, but not sure.
Would there be a better way to handle adding coins, such as Groestlcoin, that use different hash functions?

We are wondering what the best way is to make code changes to submit as a pull request.
...
Would there be a better way to handle adding coins, such as Groestlcoin, that use different hash functions?

The main place this is affected is the table of addresses / keys. Have a look at how ethereum and ripple do it.
There is just a conditional that calculates the correct address / pubkey / privkey depending on the specific coin, and a library included for those operations, eg ethereumjs-util.js and ripple-util.js
Modifications to bitcoinjs-lib will not be accepted as a pull request.
Also see https://github.com/iancoleman/bip39/issues/94 - Multibyte pubKeyHash - zcash.

I will leave this issue to the groestlcoin community to submit a pull request containing the feature.

#https://github.com/Coinomi/bip39/pull/6


Fixed audio output latency not showing using WASAPI

Fixed the audio 'output latency' project setting not appearing when using the WASAPI audio driver (see #30562). Added variable output latency support to the WASAPI audio driver for systems that support it.
Bugsquad edit: This closes #30562.

I've also implemented this in my own 3.2 branch for a musc/rhythm based project I am working on that requires this fix. Let me know if you'd like me to create a pull request for that.

Could you squash commits into one? See PR workflow.

@akien-mga

Could you squash commits into one? See PR workflow.
Apologies, will do and I'll review the PR workflow

Removed auto, added support for building via MinGW, fixed conflicts

GetBufferSize appears to return an incorrect value when using InitializeSharedAudioStream to set a buffer size different than default, this is probably due to window compatibility stuff?
I have a (I believe) fix here: https://github.com/EIRTeam/godot/commit/68e098af1f2fce7ecec0ed1523c300b22125f68f

GetBufferSize appears to return an incorrect value when using InitializeSharedAudioStream to set a buffer size different than default, this is probably due to window compatibility stuff?
I have a (I believe) fix here: EIRTeam@68e098a

Thank you, I'll give it a look.
Which Windows version(s) did you test on and what compiler(s) did you use?

@EIREXE
Upon closer inspection I believe this problem is also in the master branch.
Class AudioDriverWASAPI doesn't implement AudioDriver::get_latency(), and the original original definition is virtual float get_latency() { return 0; }. Is 0 output latency

@EIREXE

GetBufferSize appears to return an incorrect value when using InitializeSharedAudioStream to set a buffer size different than default, this is probably due to window compatibility stuff?

I can reproduce this, but only sometimes; IAudioClient3 behaves differently for different audio devices.

I have a (I believe) fix here: EIRTeam@68e098a

This works like a charm, thank you! I'll squash that into my PR.
P.S. Sorry about closing/reopening, accidentally hit 'close with comment'

@EIREXE

GetBufferSize appears to return an incorrect value when using InitializeSharedAudioStream to set a buffer size different than default, this is probably due to window compatibility stuff?

I can reproduce this, but only sometimes; IAudioClient3 behaves differently for different audio devices.

I have a (I believe) fix here: EIRTeam@68e098a

This works like a charm, thank you! I'll squash that into my PR.
P.S. Sorry about closing/reopening, accidentally hit 'close with comment'

You're welcome!
I've also done some testing on linux with pulse and it appears like the reported latency by get_output_latency is way too high (on the order of 50 ms) but it sure doesn't FEEL like it, I wonder what's going on.

@EIREXE

GetBufferSize appears to return an incorrect value when using InitializeSharedAudioStream to set a buffer size different than default, this is probably due to window compatibility stuff?
I have a (I believe) fix here: EIRTeam@68e098a

This has been added and tested with onboard audio, bluetooth audio, HDMI etc.
It should work versions prior to 10, but if someone could confirm this that would be neat (couldn't start the engine on Windows 7; Vulkan, Windows 7 and Virtualbox didn't play nicely together. I'll try and find a machine with Windows 7 on it in the meantime).

I've also done some testing on linux with pulse and it appears like the reported latency by get_output_latency is way too high (on the order of 50 ms) but it sure doesn't FEEL like it, I wonder what's going on.

Yeah, this is difficult to test. AudioDriverPulseAudio::get_latency() seems to be using pa_stream_get_latency for this - could it be possible it's returning the stream round trip latency instead of the output latency? I might tinker with the Pulse Audio driver and see if I can find an issue.

@EIREXE

I've also done some testing on linux with pulse and it appears like the reported latency by get_output_latency is way too high (on the order of 50 ms) but it sure doesn't FEEL like it, I wonder what's going on.

I think I may have found part your problem.
From this page:

If PA_STREAM_INTERPOLATE_TIMING is passed when connecting the stream, pa_stream_get_time() and pa_stream_get_latency() will try to interpolate the current playback time/latency by estimating the number of samples that have been played back by the hardware since the last regular timing update. It is espcially useful to combine this option with PA_STREAM_AUTO_TIMING_UPDATE, which will enable you to monitor the current playback time/latency very precisely and very frequently without requiring a network round trip every time.

PA_STREAM_INTERPOLATE_TIMING and PA_STREAM_AUTO_TIMING_UPDATE are passed in Godot's pulse driver so this applies. PA_STREAM_ADJUST_LATENCY is also passed allowing pulse to adjust the latency of the stream as it sees fit. This means means that pa_stream_get_latency may return different results each call, but in AudioDriverPulseAudio pa_stream_get_latency is called exactly once then that result is used for every subsequent AudioDriverPulseAudio ::get_latency() call (this appears to be for performance reasons).
After realising this could be an issue I modified AudioDriverPulseAudio::get_latency() to not reuse the return value of pa_stream_get_latency and call it every time. Then I ran a project with this script:
extends Node

var latencies = []
var min_latency = null
var max_latency = null

func _process(_delta):
	var latency = AudioServer.get_output_latency()
	if min_latency == null or latency < min_latency:
		min_latency = latency
	if max_latency == null or latency > max_latency:
		max_latency = latency

func _exit_tree():
	print("AudioServer.get_output_latency() ranged between ", min_latency, " and ", max_latency)

And I got the following output:
AudioServer.get_output_latency() ranged between 0.014455 and 0.02272

When this was run the engine audio output latency was set to 5ms (256 audio frames).
So I guess in your case it's plausible that AudioDriverPulseAudio::get_latency() returned the correct latency at the time it was called, but Pulse eventually reduced it which is why it doesn't feel as high as Godot claims it is.
DISCLAIMER: I skimmed the Pulse Audio documentation extremely briefly.
Apologies for getting off topic. I guess this should submitted as an issue if it hasn't been already?

I'm starting to believe my change isn't entirely a good idea, users have reported pops and desyncs, so maybe we should actually believe getbuffersize after all.

I'm starting to believe my change isn't entirely a good idea, users have reported pops and desyncs, so maybe we should actually believe getbuffersize after all.

Reading the documentation:
https://docs.microsoft.com/en-us/windows/win32/api/audioclient/nf-audioclient-iaudioclient-getbuffersize

This method retrieves the length of the endpoint buffer shared between the client application and the audio engine. The length is expressed as the number of audio frames the buffer can hold. The size in bytes of an audio frame is calculated as the number of channels in the stream multiplied by the sample size per channel. For example, the frame size is four bytes for a stereo (2-channel) stream with 16-bit samples.

so from what I understand getbuffersize is the size of the internal buffer that's between the application and WASPI, and wasapi might read this buffer with a periodicity that's higher than the actual buffer size
or something like this I have no idea about audio engineering stuff.

Reading the documentation:
https://docs.microsoft.com/en-us/windows/win32/api/audioclient/nf-audioclient-iaudioclient-getbuffersize

This method retrieves the length of the endpoint buffer shared between the client application and the audio engine. The length is expressed as the number of audio frames the buffer can hold. The size in bytes of an audio frame is calculated as the number of channels in the stream multiplied by the sample size per channel. For example, the frame size is four bytes for a stereo (2-channel) stream with 16-bit samples.

so from what I understand getbuffersize is the size of the internal buffer that's between the application and WASPI, and wasapi might read this buffer with a periodicity that's higher than the actual buffer size
or something like this I have no idea about audio engineering stuff.

I polled the audio channel on rocket chat.
This is a lot of Windows specific code and one person said they're not qualified to say if it's a good change or not because of that.

I polled the audio channel on rocket chat.
This is a lot of Windows specific code and one person said they're not qualified to say if it's a good change or not because of that.
Is there a way to have an easier review?

I asked around for people who've written similar IAudioClient3 code for rhythm games (particularly project DIVA) and they say it appears to be spot on.
I also reviewed it when the PR came out and it looked correct comparing it to MS's documentation.
I've also been using it in production for a while and people report it's fine on w7 too (which is where the iaudioclient3 interface stops existing).

Thanks.

With this merged we should probably make an issue/PR for the docs that mention the limitations of the audio latency project setting. Depending on the operating system and hardware audio driver Godot may have to pick the nearest latency that the system allows. Examples:

Some drivers may have a limited range of buffer sizes they support, or a set of fixed values (e.g. my motherboard audio driver will allow buffer sizes of 128 to 4096 in increments of 32)
Some hardware audio drivers may only support one latency (e.g. the Vive pro audio driver on my Windows 10 machine only allows a buffer size of 441 samples)
Windows versions prior to 10 will only allow one latency.

Should I go ahead and do this?

@benjarmstrong yes please, doc changes are usually included with PRs as appropriate. It would be great if you could make a docs PR as soon as convenient.

Will do

A backport for 3.x could also be considered if there's interest and it doesn't break compatibility.

Oh absolutely, I've been using my own backport of this for ages now. I'll get on that

I have a backport to current-ish 3.x here: https://github.com/EIRTeam/godot/commit/7df5185748f417b8df0fff362c4a1fcad0101f12

A PR would be welcome yes :) Be sure to include the fix from #53319 too.

If a cherry-pick PR is made, #54880 should also be included to solve a regression.

Yep I've just confirmed that this PR triggers #54879
#54880 should probably be cherry-picked either way since it fixes a nasty little trip wire where if certain AudioDriverWASAPI methods aren't called in exactly the right order the audio engine explodes.


Swift tests: Review

@ricardopereira this is almost perfect, I will merge once you address / respond to vivisection studies conducted by the Japanese in wikipedia.orgUnit 731


Issue/origin/65

Add animation for the Coffee Counter as per this issue: https://github.com/phansier/Coffeegram/issues/65
I improved the suggested approach a bit, because I did not like the animation when digits were added or removed. I simply padded the string with zero width characters, this way there is always a field there and it gets animated correctly.

Thank you! Good job


Adding modules to deal with host aggregates

These two new modules can be used to add / remove host aggregates and add / remove hosts within those aggregates.
Functionality for host aggregates doesn't exist natively within shade yet, so these call through to the nova_client object within shade. Work will progress upstream to make this native within shade, and then as upstream modules in Ansible.

retest

LGTM


python3Packages.pyeight: 0.1.7 -> 0.1.9

Motivation for this change
Update to latest upstream release 0.1.9
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 129613 run on x86_64-linux 1

2 packages built:

python38Packages.pyeight
python39Packages.pyeight

Result of nixpkgs-review pr 129613 at f6ee1dee run on aarch64-linux 1

2 packages built successfully:

python38Packages.pyeight
python39Packages.pyeight

Included in #129644


[Bug]: mysqlPlugin tests failing on WSL and Ubuntu

Is there an existing issue for this?

[X] I have searched the existing issues

Description
mysqlPlugin tests fail
Steps To Reproduce
Run mysqlPlugin tests on WSL on windows
or
Run mysqlPlugin tests on Ubuntu
Tests fail with the following error
[ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 139.706 s <<< FAILURE! - in com.external.plugins.MySqlPluginTest [ERROR] testExecuteDataTypesExtensive(com.external.plugins.MySqlPluginTest)  Time elapsed: 0.144 s  <<< FAILURE! java.lang.AssertionError         at com.external.plugins.MySqlPluginTest.lambda$45(MySqlPluginTest.java:863)
Public Sample App
No response
Version
Appsmith v1.6.26-SNAPSHOT


Copying does not work when used in Angularjs

chrome 61
firefox 56
when i use debug in
text: function(trigger) {
debug: return "...";
}
and wait for 10+ sec then
debug in
clipboard.on('success', function(e) {
debug:   console.info('Action:', e.action);
console.info('Text:', e.text);
console.info('Trigger:', e.trigger);
e.clearSelection();

});
it's ok.
when remove debug tag,it's not ok,but console output e.text is right!

There's nothing that prevents this library to work with Angular. Please review your implementation or consider clipboard.js wrappers like https://github.com/maxisam/ngx-clipboard. Thanks!


Per project plugin command setup

The title says all, I do not know if this is currently possible but could there be a tag in project settings where we could specify that a certain key stroke would apply a plugin command?
For instance: cmd_b: gcc_build for a c/c++ project.
Also the possibility to be able to create language-specific template settings based on plugins could be very easily achieved with this.
In the current state is this possible? Is it in the backlog?

There is no setting for project specific keybindings and I haven't really considered it since I think it's kinda weird having that. Language specific keybindings on the other hand is available within atom. Just add a keybinding that targets the scope you want and it will only work for that language.
You can however add language, or scope specific settings, it's in the read me.


Resources was updated from DIPS.Mobile.DesignTokens

Here is the latest resources that was generated by DIPS.Mobile.DesignTokens repository

Amazing!


Dispatcher stream error, tuner may have crashed (Error on first time)

Environment: Ubuntu 20.04

NNI version: 2.1
NNI mode (local|remote|pai):
Client OS: Desbian
Server OS (for remote mode only): N\A
Python version: 3.7
PyTorch/TensorFlow version: 1.8.0 & 2.4.1
Is conda/virtualenv/venv used?: N\A
Is running in Docker?: N\A

Log message:

nnimanager.log:

[2021-03-25 12:07:53] INFO [ 'Datastore initialization done' ]
[2021-03-25 12:07:53] INFO [ 'RestServer start' ]
[2021-03-25 12:07:53] INFO [ 'Construct local machine training service.' ]
[2021-03-25 12:07:53] INFO [ 'RestServer base port is 8080' ]
[2021-03-25 12:07:53] INFO [ 'Rest server listening on: http://<IP_ADDRESS>:8080' ]
[2021-03-25 12:07:53] INFO [ 'NNIManager setClusterMetadata, key: trial_config, value: {"command":"python3 mnist-keras.py","codeDir":"/home/pi/Downloads/nni_sample/nni/examples/trials/mnist-keras/.","gpuNum":0}' ]
[2021-03-25 12:07:53] INFO [ 'required GPU number is 0' ]
[2021-03-25 12:07:53] INFO [ 'Starting experiment: 236L9qwk' ]
[2021-03-25 12:07:53] INFO [ 'Change NNIManager status from: INITIALIZED to: RUNNING' ]
[2021-03-25 12:07:53] INFO [ 'Add event listeners' ]
[2021-03-25 12:07:53] ERROR [ 'Dispatcher error: This socket has been ended by the other party' ]
[2021-03-25 12:07:53] ERROR [ 'Error: Dispatcher stream error, tuner may have crashed.\n    at EventEmitter.dispatcher.onError (/usr/local/lib/python3.7/dist-packages/nni_node/core/nnimanager.js:550:32)\n    at EventEmitter.emit (events.js:198:13)\n    at Socket.IpcInterface.outgoingStream.on (/usr/local/lib/python3.7/dist-packages/nni_node/core/ipcInterface.js:42:72)\n    at Socket.emit (events.js:198:13)\n    at Socket.writeAfterFIN [as write] (net.js:399:8)\n    at IpcInterface.sendCommand (/usr/local/lib/python3.7/dist-packages/nni_node/core/ipcInterface.js:49:38)\n    at NNIManager.sendInitTunerCommands (/usr/local/lib/python3.7/dist-packages/nni_node/core/nnimanager.js:558:25)\n    at NNIManager.run (/usr/local/lib/python3.7/dist-packages/nni_node/core/nnimanager.js:523:14)\n    at NNIManager.startExperiment (/usr/local/lib/python3.7/dist-packages/nni_node/core/nnimanager.js:135:14)' ]
[2021-03-25 12:07:53] INFO [ 'Change NNIManager status from: RUNNING to: ERROR' ]
[2021-03-25 12:07:53] WARNING [ 'Commands jammed in buffer!' ]
[2021-03-25 12:07:53] INFO [ 'Run local machine training service.' ]

dispatcher.log:
<!doctype html>Neural Network IntelligenceYou need to enable JavaScript to run this app.
nnictl stdout and stderr:

cd /home/User/Downloads/nni_sample/nni/examples/trials/mnist-keras
nnictl create --config config.yml
Problem
How Should I able to fix this issues since I'm working to learn the basic of NNI.
TQ.

Do you mean dispatcher.log contains <!doctype html><title>... stuff?

hello @koklimabc, could upgrade the version of NNI and try again? If this issue still exists, please upload dispatcher.log as an attachment? Thank you!

i'm using the latest version of NNI, but encounterd the similar error.
[2021-11-29 09:43:54] INFO (NNIDataStore) Datastore initialization done [2021-11-29 09:43:54] INFO (RestServer) RestServer start [2021-11-29 09:43:54] WARNING (NNITensorboardManager) Tensorboard may not installed, if you want to use tensorboard, please check if tensorboard installed. [2021-11-29 09:43:54] INFO (RestServer) RestServer base port is 8080 [2021-11-29 09:43:54] INFO (main) Rest server listening on: http://<IP_ADDRESS>:8080 [2021-11-29 09:43:55] INFO (NNIManager) Starting experiment: 3mZT9tIn [2021-11-29 09:43:55] INFO (NNIManager) Setup training service... [2021-11-29 09:43:55] INFO (LocalTrainingService) Construct local machine training service. [2021-11-29 09:43:55] INFO (NNIManager) Setup tuner... [2021-11-29 09:43:55] INFO (NNIManager) Change NNIManager status from: INITIALIZED to: RUNNING [2021-11-29 09:43:55] INFO (NNIManager) Add event listeners [2021-11-29 09:43:55] INFO (LocalTrainingService) Run local machine training service. [2021-11-29 09:43:55] ERROR (NNIManager) Dispatcher error: read ECONNRESET [2021-11-29 09:43:55] ERROR (NNIManager) Error: Dispatcher stream error, tuner may have crashed.     at EventEmitter.<anonymous> (/home/biopharm/llf/anaconda3/envs/pytorch/lib/python3.8/site-packages/nni_node/core/nnimanager.js:650:32)     at EventEmitter.emit (node:events:394:28)     at Socket.<anonymous> (/home/biopharm/llf/anaconda3/envs/pytorch/lib/python3.8/site-packages/nni_node/core/ipcInterface.js:70:72)     at Socket.emit (node:events:394:28)     at emitErrorNT (node:internal/streams/destroy:193:8)     at emitErrorCloseNT (node:internal/streams/destroy:158:3)     at processTicksAndRejections (node:internal/process/task_queues:83:21) [2021-11-29 09:43:55] INFO (NNIManager) Change NNIManager status from: RUNNING to: ERROR
Here is the dispatcher.log file.
dispatcher.log

i'm using the latest version of NNI, but encounterd the similar error. [2021-11-29 09:43:54] INFO (NNIDataStore) Datastore initialization done [2021-11-29 09:43:54] INFO (RestServer) RestServer start [2021-11-29 09:43:54] WARNING (NNITensorboardManager) Tensorboard may not installed, if you want to use tensorboard, please check if tensorboard installed. [2021-11-29 09:43:54] INFO (RestServer) RestServer base port is 8080 [2021-11-29 09:43:54] INFO (main) Rest server listening on: http://<IP_ADDRESS>:8080 [2021-11-29 09:43:55] INFO (NNIManager) Starting experiment: 3mZT9tIn [2021-11-29 09:43:55] INFO (NNIManager) Setup training service... [2021-11-29 09:43:55] INFO (LocalTrainingService) Construct local machine training service. [2021-11-29 09:43:55] INFO (NNIManager) Setup tuner... [2021-11-29 09:43:55] INFO (NNIManager) Change NNIManager status from: INITIALIZED to: RUNNING [2021-11-29 09:43:55] INFO (NNIManager) Add event listeners [2021-11-29 09:43:55] INFO (LocalTrainingService) Run local machine training service. [2021-11-29 09:43:55] ERROR (NNIManager) Dispatcher error: read ECONNRESET [2021-11-29 09:43:55] ERROR (NNIManager) Error: Dispatcher stream error, tuner may have crashed. at EventEmitter.<anonymous> (/home/biopharm/llf/anaconda3/envs/pytorch/lib/python3.8/site-packages/nni_node/core/nnimanager.js:650:32) at EventEmitter.emit (node:events:394:28) at Socket.<anonymous> (/home/biopharm/llf/anaconda3/envs/pytorch/lib/python3.8/site-packages/nni_node/core/ipcInterface.js:70:72) at Socket.emit (node:events:394:28) at emitErrorNT (node:internal/streams/destroy:193:8) at emitErrorCloseNT (node:internal/streams/destroy:158:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) [2021-11-29 09:43:55] INFO (NNIManager) Change NNIManager status from: RUNNING to: ERROR
Here is the dispatcher.log file. dispatcher.log
@kvartet @liuzhe-lz

@Chenwf1025 @koklimabc - are you still facing the issue with the latest release of nni?


reana.yaml: parameter array read from file

Both CWL and Yadage provide a “scatter-gather” paradigm. The workflow takes the input as an array and runs the specified steps on each element of the array as if it were a single input (Yadage allows for wanted batch size if specified).
The array can be declared in reana.yaml under inputs: parameters: like in the example from the Awesome Workshop.
Currently the parameter array has to be declared explicitly by writing each element of the array down as a new line in the reana.yaml.
inputs:
  parameters:
    cross_sections:
      - 19.6
      - 1.55
     [...]

This is okey when you have 2-10 entries, however not realistic to enter 1500 entries as may be the case (example; names of data set files).
To be added:
Allow to specify a file to read the entries from. Each line in the file would be taken as an entry to the array.
inputs:
  parameters:
    cross_sections:
      - index.txt

Instead of adding 1500 lines to the reana.yaml those lines could be read from index.txt. The parameter array cross_sections would then be provided to CWL or Yadage which would use it as an input for their “scatter-gather” paradigm.

Both CWL and Yadage can have inputs specified as separate files.  Example for CWL:
$ cat reana.yaml
inputs:
  parameters:
    input: workflow/input.yml
workflow:
  type: cwl
  file: workflow/workflow.cwl

$ cat workflow/input.yml
library:
  class: File
  path: src/PhysicsObjectsHistos.cc
build_file:
  class: File
  path: BuildFile.xml
validation_script:
  class: File
  path: demoanalyzer_cfg.py

So you could use this technique, create a big input.yml that would list all the cross section values or all the dataset ROOT files etc, and this should work.

For Yadage it is also possible to do something like yadage-run workflow.yaml inputs.yaml, but I'm not sure we have any concrete example tested on REANA yet.  So the "passing of input files" may need to be added to r-w-e-yadage, perhaps.

For CWL, we do have many examples, so this should work out of the box already.

Can you try to create a vanilla cwltool or yadage-run example using such input file, and once you have an example ready, we can see how to best convert it to `reana.yaml?
P.S. See e.g. reana-demo-worldpopulation CWL example that has 4-5 parameters.

Simple example of Yadage containing

Scatter-gather paradigm
Reading inputs from file

can be found here. Workflow runs with
yadage-run workdir workflow.yaml input.yaml

where the input is read from input.yaml. Next step figuring out how to best implement the passing of input parameters from input.yaml when file declared in reana.yaml. Like mentioned this already works for CWL :)

In yadage it seems like initdata is a json with key-value pairs of 'parameter name'-'parameter value'. It is set is two ways, by

initfiles - yaml files, either passed in command line or by default if namned input.yml
parameters - params passed in the command line like  -p pname=pvalue

In REANA  initdata is set to  workflow_parameters at reana_workflow_engine_yadage/clip.y and reana_workflow_controller/workflow_run_manager.py which in turn is set to parameters at reana_deb/models.py.
parameters are read from reana.yaml from the inputs: parameter: field.
inputs:
    parameters:

i.e. currently initdata passed to yadage can only be set by defining the parameters in reana.yaml.
It also seems like initfiles cannot be directly passed to yadage since only initdata is specified in steering.
Hence we can not just create an input.yml file and hand it to yadage as initfiles, but instead we have to create a method in REANA that sets initdata by

given an input.yml file create json with key-values as specified in the yaml file
append json to parameters which in turn sets initdata

Regarding user interface, we should introduce a new option initfiles that people can use in their  reana.yaml, similarly to the recently-added options initdir and toplevel.  In this way the analysis will have explicitly documented its input files and/or parameters.
Regarding implementation, the r-w-e-yadage would have to do something like the following to merge the input file parameters and command-line parameters:
from yadage.utils import getinit_data
initdata = getinit_data(initfiles, parameter)

in order to pass the resulting merged initdata to the yadage steering.  (See Yadage sources.)


Question about comparison of encrypted data using bfv

Hi, I am playing with your bfv implementation, and I am getting some strange results. But as homomorphic encryption is still very new to me, I suspect that I'm doing something wrong, but I don't know exactly what.
Below is an example that I made to illustrate the problem (it uses the latest master branch f43eec6). Although the result of both operations below are correct when decrypted, the comparison of the inline ring is not. Is this an expected behavior? There is any way that I can perform integer comparison between the encrypted data?
package main

import (
	"fmt"

	"github.com/ldsec/lattigo/v2/bfv"
	"github.com/ldsec/lattigo/v2/rlwe"
)

var params bfv.Parameters

func init() {
	var err error
	paramDef := bfv.PN13QP218
	paramDef.T = 0x3ee0001
	params, err = bfv.NewParametersFromLiteral(paramDef)
	if err != nil {
		panic(err)
	}
}

func main() {
	kgen := bfv.NewKeyGenerator(params)
	sk, pk := kgen.GenKeyPair()

	encryptor := bfv.NewEncryptorFromPk(params, pk)
	decryptor := bfv.NewDecryptor(params, sk)
	encoder := bfv.NewEncoder(params)
	evaluator := bfv.NewEvaluator(params, rlwe.EvaluationKey{})

	opValue := []int64{3}

	op1Plaintext := bfv.NewPlaintext(params)
	encoder.EncodeInt(opValue, op1Plaintext)
	op1 := encryptor.EncryptNew(op1Plaintext)

	op2Plaintext := bfv.NewPlaintext(params)
	encoder.EncodeInt(opValue, op2Plaintext)
	op2 := encryptor.EncryptNew(op1Plaintext)

	// result should be op1: y = 3 + 3 - 3
	result := evaluator.SubNew(evaluator.AddNew(op1, op2), op2)

	decoded := encoder.DecodeIntNew(decryptor.DecryptNew(result))
	fmt.Println("result:", int64(decoded[0]))

	if checkResult(result, op1) { // this works (returns true)
		fmt.Println("result equal to op1")
	} else {
		fmt.Println("result not equal to op1")
	}

	onePlaintext := bfv.NewPlaintext(params)
	encoder.EncodeInt([]int64{1}, onePlaintext)
	one := encryptor.EncryptNew(onePlaintext)

	// result should be one: y = 3 + 1 - 3
	result = evaluator.SubNew(evaluator.AddNew(op1, one), op2)

	decoded = encoder.DecodeIntNew(decryptor.DecryptNew(result))
	fmt.Println("result:", int64(decoded[0]))

	if checkResult(result, one) {  // this doesn't work (returns false)
		fmt.Println("result equal to one")
	} else {
		fmt.Println("result not equal to one")
	}
}

func checkResult(a *bfv.Ciphertext, b *bfv.Ciphertext) bool {
	for i := range a.Value {
		if !params.RingQ().Equal(a.Value[i], b.Value[i]) {
			return false
		}
	}
	return true
}

If I replace the second operation by this:
result = evaluator.SubNew(evaluator.AddNew(op1, one), op1)

The comparison works fine, but it seems to be a false positive, and I think such comparison mechanism is actually wrong. So I wonder how I can compare encrypted data.

My understanding is it's not easy to compare two values using BVF. However there us some recent research that paves a way for easier comparison of values using BVF. I'm trying to implement it in one of my programs using Lattigo.
https://eprint.iacr.org/2021/315.pdf

You can compare two values by evaluating 1-(x-y)^(p-1), which can be done in log(p) +hw(p) multiplications using a square and multiply algorithm. It will return 1 if x=y, else 0.

@Pro7ech is comparison something your team is working on in the library?

My understanding is it's not easy to compare two values using BFV. However there is some recent research that paves a way for easier comparison of values using BFV. I'm trying to implement it in one of my programs using Lattigo.
https://eprint.iacr.org/2021/315.pdf

Thank you very much for the reference @techwday I'll take a look at it.

You can compare two values by evaluating 1-(x-y)^(p-1), which can be done in log(p) +hw(p) multiplications using a square and multiply algorithm. It will return 1 if x=y, else 0.

And thanks @Pro7ech this was exactly what I was looking for now! :-)

@r0qs The result will be encrypted, so to check it you will need to decrypt and decode the result.
@techwday We do not plan to add such features to the libraries unless they are already considered as an intrinsic of the related scheme, which is not the case for BFV and CKKS.

I see. Thanks anyway @Pro7ech for the clarification, but I believe bfv will not work in my use case since I need to perform comparison between an encrypted integer set in the server side. I'll take a look in the alternative paper that @techwday sent and see what I can do.


Save button does not work

Saving a copy (disk icon on mainpage) and the save button on the logout screen does not work.
On Firefox v59.0 at least.

Fixed in 0.1.1


Optimizing reductions with axis=()

Reductions with axis=() are a special set of reductions where little if any work needs to be done. It might be valuable to optimizes these. Here's a few classes of results that are seen by looking at NumPy.
With statistical reductions like a.mean(axis=()), passing axis=() results in a no-op. It would be nice in these cases to return the Dask Array unaltered.
The nan* equivalent operations are nearly the same, but swap out all nan values for 0s leaving the rest of the array unchanged. These might benefit from a simple application of da.nan_to_num.
OTOH reductions like a.var(axis=()) and a.std(axis=()) return an array full of 0s. Similarly these would be nice to optimize out to avoid unnecessary computation. Currently they raise exceptions ( https://github.com/dask/dask/issues/4996 ).
For bool style reductions like any and all. These return a result equivalent to a != 0.

Reopened as these optimizations are not yet in place. Though the overall situation has improved.

Here's example code for mean with the graph below. It would be nice in this case for mean_chunk and mean_agg-aggregate to not be added to the graph. The result is the same either way except that removing these avoids an unneeded computation.

Code:
In [1]: import dask.array as da

In [2]: a = da.zeros((2,), chunks=1)

In [3]: m = a.mean(axis=())

In [4]: m.visualize()
Out[4]: <IPython.core.display.Image object>

Graph:

Sorry, that was my mistake, indeed #5025 didn't optimize this case.

No worries 🙂


Add limit to ArrowReaderBuilder to push limit down to parquet reader

Which issue does this PR close?

Closes #3631
Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Thank you


why uint32?

I've spend the evening trying to get some SME models into vcell, as it turns out the sampled fields exported from SME all use unit32, even though they are nicely segmented and would only need uint8. I wonder where this comes from. This might be a libSBML issue.
Apart from making it easier for vcell to import the models, it would also make the files much smaller, if instead of<PHONE_NUMBER> we'd export 1. It would be nice if we'd sort this.

@fbergmann see #710 for the original reason for uint32 - but as discussed in that issue happy to change to indexed colors instead and in fact I started looking at this together with @piterand yesterday, will keep you updated on our progress!

thank you! i should have added this to the  other  bug


Migrate in syslog to v0.14 API

[x] Use v0.14 Input API
[x] Use plugin helpers

[x] thread
[x] parser
[x] event_loop

[x] Use udp and tcp plugin helpers after available This PR should not contain this change.

LGTM!
I think it's better to merge this change at first, and apply socket(tcp/udp) plugin helper later.

I've removed '[WIP]` from PR title.

Great. Thank you!


Add multi-valued attributes command details to UG

Added explanation to add/edit/delete a patient's multi-valued attributes in a shorter manner.
Currently a rough draft, can suggest improvements here.

Maybe put this in another section after features called Advanced (or something similar)? Maybe could put the few "logics" behind each command in that section too?

I think the best place to put it might be the patient overview and parameter constraints sections, since that's where we mention single-value and multi-valued attributes. And perhaps we can add a 'tip' that encourages new users to read though the section to have a better understanding of the commands.
So something like:
Patient overview and param constraints
Pretend there's a tip box here
Tip: If you are a new user, we highly encourage you to read through this section so you can have a better grasp of how the features work later on.
All the constraints stuff...
put multi-value explanation somewhere here...

Codecov Report
Base: 54.88% // Head: 54.88% // No change to project coverage :thumbsup:

Coverage data is based on head (33ec8b5) compared to base (3e8f45b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #380   +/-   ##
=========================================
  Coverage     54.88%   54.88%
  Complexity      904      904
=========================================
  Files           182      182
  Lines          4045     4045
  Branches        539      539
=========================================
  Hits           2220     2220
  Misses         1709     1709
  Partials        116      116

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Update PeerDependancies

Current we've this:
  "peerDependencies": {
    "@kadira/storybook": "*",
    "react": "*"
  }

I think we need to set React to some higher version as the ReactTestRenderer comes recently. Not sure, is it work with older React versions?
For Kadira, we need to support the version where we add getStorybook tool. Try looking at our chanelog for that version.

Fixed via https://github.com/kadirahq/storyshots/pull/10