Cluster addons make large clusters unstable (user container's oom_score_adj is misconfigured)

Setup a 100 node cluster with 0 extra pods
Periodically run kubectl get pods | sort -k 5 -u till you notice a youger cluster addon, eg:

monitoring-heapster-v2-jnxgm                       1/1       Running   0          10m
monitoring-influx-grafana-v1-7kqyf                 2/2       Running   0          17m
fluentd-elasticsearch-e2e-test-beeps-minion-1khh   1/1       Running   0          1h
elasticsearch-logging-v1-abm8p                     1/1       Running   0          2h
fluentd-elasticsearch-e2e-test-beeps-minion-m251   1/1       Running   0          31m
fluentd-elasticsearch-e2e-test-beeps-minion-i8ke   1/1       Running   1          7m
NAME                                               READY     REASON    RESTARTS   AGE

At this point you'll probably have some dead nodes, or at least a kubelet restart (related:  https://github.com/GoogleCloudPlatform/kubernetes/issues/7815).
Additional information:

All my dead nodes had at least 1 addon eg:

e2e-test-beeps-minion-0so1   kubernetes.io/hostname=e2e-test-beeps-minion-0so1   NotReady
e2e-test-beeps-minion-0v0x   kubernetes.io/hostname=e2e-test-beeps-minion-21dm   NotReady

cat ~/logs/kube-scheduler.log | grep "Attempting to bind"
I0614 03:23:58.132034      12 factory.go:312] Attempting to bind monitoring-heapster-v2-mtr9x to e2e-test-beeps-minion-0so1
I0614 04:01:28.134103      12 factory.go:312] Attempting to bind elasticsearch-logging-v1-0fjuw to e2e-test-beeps-minion-21dm

A spike in cpu/memory usage often occurs just before the restart (and it spikes fast, in ~30m):

beeps@e2e-test-beeps-minion-21dm:~$ sudo atop -r /var/log/atop/atop_20150614 -b 3:20 -C | grep "PID" -A 3 | head -3
  PID  SYSCPU  USRCPU  VGROW  RGROW  RDDSK  WRDSK ST EXC S  CPU CMD
11230  20.09s   2m29s     0K     0K     4K   1.6G --   - S  29% java
 3414   7.08s   9.21s     0K   248K   820K  1032K --   - S   3% kubelet

beeps@e2e-test-beeps-minion-21dm:~$ sudo atop -r /var/log/atop/atop_20150614 -b 4:00 -C | grep "PID" -A 3 | head -3
  PID  SYSCPU  USRCPU  VGROW  RGROW  RDDSK  WRDSK ST EXC S  CPU CMD
11230  34.74s   5m01s     0K 272.4M 799.8M 647.7M --   - S  93% java
 3414  19.81s  20.57s 16392K -4684K   1.0G  2324K --   - S  11% kubelet

This affects the sd of syncPod response times (and obviously during the outage syncPod doesn't run):

I0614 03:52:47.433688    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 24.630828ms
I0614 03:52:57.639778    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 192.807391ms
I0614 03:53:07.621054    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 167.872516ms
I0614 03:53:17.644948    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 185.237351ms
I0614 03:53:27.494427    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 29.206469ms
I0614 03:53:37.651265    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 175.375625ms
I0614 03:53:47.655602    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 32.494134ms
I0614 03:53:57.655484    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 18.381837ms
I0614 03:54:07.664289    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 24.411221ms
I0614 03:54:17.687201    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 38.721561ms
I0614 03:54:27.692326    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 33.002448ms
I0614 03:54:37.888149    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 221.385ms
I0614 03:54:47.702193    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 29.895014ms
I0614 03:54:57.707271    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 28.48375ms
I0614 03:55:08.134910    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 449.835487ms
I0614 03:55:18.859870    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 920.379739ms
I0614 03:55:28.427633    3414 kubelet.go:1121] (kubelet) syncPod = net time for pod elasticsearch-logging-v1-0fjuw 302.093733ms

Sometimes there are OOMs, these don't affect the kubelet, but the node eventually dies.

beeps@e2e-test-beeps-minion-r1ms:~$ cat /var/log/kern.log | grep "Out of"
Jun 14 03:06:48 e2e-test-beeps-minion-r1ms kernel: [31683.055354] Out of memory: Kill process 16730 (manage_accounts) score 4 or sacrifice child
Jun 14 03:06:48 e2e-test-beeps-minion-r1ms kernel: [31683.251853] Out of memory: Kill process 2343 (manage_addresse) score 4 or sacrifice child
Jun 14 03:06:55 e2e-test-beeps-minion-r1ms kernel: [31684.094101] Out of memory: Kill process 2354 (manage_clock_sy) score 4 or sacrifice child

beeps@e2e-test-beeps-minion-r1ms:~$ cat /var/log/kubelet.log | grep fork
I0614 03:06:07.173569    3392 container.go:363] Failed to update stats for container "/e84a731222ad0fcef8bdc713ca4aaa6a2e8c7df1dcdc38fe342deb05506dcfd6": du command failed on /var/lib/docker/aufs/diff/e84a731222ad0fcef8bdc713ca4aaa6a2e8c7df1dcdc38fe342deb05506dcfd6 with output  - fork/exec /usr/bin/du: cannot allocate memory, continuing to push stats
I0614 03:06:07.025625    3392 container.go:363] Failed to update stats for container "/0f8c0eaf2805abf74565fac3cd0cf781e332160f27d2aaa7cd4db45aa5609d27": du command failed on /var/lib/docker/aufs/diff/0f8c0eaf2805abf74565fac3cd0cf781e332160f27d2aaa7cd4db45aa5609d27 with output  - fork/exec /usr/bin/du: cannot allocate memory, continuing to push stats

... but kubelet is still healthy...

I0614 03:19:20.606510   17152 manager.go:750] (kubelet) dockermanager dockertools GetPods took 1.928699ms
I0614 03:19:20.708280   17152 manager.go:750] (kubelet) dockermanager dockertools GetPods took 1.572725ms
I0614 03:19:20.810046   17152 manager.go:750] (kubelet) dockermanager dockertools GetPods took 1.559437ms
I0614 03:19:20.912091   17152 manager.go:750] (kubelet) dockermanager dockertools GetPods took 1.864913ms
I0614 03:19:21.013862   17152 manager.go:750] (kubelet) dockermanager dockertools GetPods took 1.596476ms

... logs are quiet till I reset the node at 18:18 ...

I0614 18:18:49.292671    2208 server.go:269] Using root directory: /var/lib/kubelet
I0614 18:18:49.292740    2208 manager.go:126] cAdvisor running in container: "/"

Sometimes the kubelet restart without any OOMs, grep nodecontroller logs for evidence:

(test_clean_master)$ cat ~/logs/kcm | grep PodEvi
I0614 23:42:45.843681       5 podevictor.go:110] PodEvictor is evicting Pods on Node: e2e-test-beeps-minion-gjds
I0614 23:50:50.927087       5 podevictor.go:110] PodEvictor is evicting Pods on Node: e2e-test-beeps-minion-c5ok
I0614 23:57:14.960444       5 podevictor.go:110] PodEvictor is evicting Pods on Node: e2e-test-beeps-minion-g6wk
I0615 00:21:55.186406       5 podevictor.go:110] PodEvictor is evicting Pods on Node: e2e-test-beeps-minion-hawh

related (although you don't need any extra pods to reproduce this issue): https://github.com/GoogleCloudPlatform/kubernetes/issues/7815

Couple comments related to the issue:

Why does scheduler try to bind a addons pod to not_ready nodes at the first place?
What is the job with process id 11230? I bet it is elasticssearch one since I observed it is using a lot of cpu constantly before.
Can you share more kernel log on that oom node? Do you know how many pods are scheduled to the node at a given time when oom occurs? And why? no spreading from scheduler  at all?

Why does scheduler try to bind a addons pod to not_ready nodes at the first place?

It doesn't, the pod bounces around NotReadying nodes.

What is the job with process id 11230? I bet it is elasticssearch one since I observed it is using a lot of cpu constantly before.

It's always either heapster or elasticsearch, the java job in the example is elasticsearch.

Can you share more kernel log on that oom node?

This is from another node I just reset.
https://gist.github.com/bprashanth/32c9d22cc39b961a395f
td-agent is the event collector daemon used by fluentd:

Jun 15 03:51:32 e2e-test-beeps-minion-1nm0 kernel: [15718.015838] Out of memory: Kill process 3931 (td-agent) score 19 or sacrifice child
Jun 15 03:51:32 e2e-test-beeps-minion-1nm0 kernel: [15718.023653] Killed process 3931 (td-agent) total-vm:390900kB, anon-rss:34360kB, file-rss:4kB

Do you know how many pods are scheduled to the node at a given time when oom occurs?

2, fluentd and heapster.
$ cat ~/logs/kcm  | grep -i "evicting pods.* e2e-test-beeps-minion-1nm0" -A 5
I0615 03:56:01.550502       5 podevictor.go:110] PodEvictor is evicting Pods on Node: e2e-test-beeps-minion-1nm0
I0615 03:56:01.550534       5 nodecontroller.go:427] Delete all pods from e2e-test-beeps-minion-1nm0
W0615 03:56:01.550549       5 request.go:302] field selector: v1 - pods - spec.host - e2e-test-beeps-minion-1nm0: need to check if this is versioned correctly.
I0615 03:56:01.560933       5 nodecontroller.go:438] Delete pod fluentd-elasticsearch-e2e-test-beeps-minion-1nm0
I0615 03:56:01.582468       5 nodecontroller.go:438] Delete pod monitoring-heapster-v2-9dluc
I0615 03:56:01.600577       5 replication_controller.go:305] Too few "default"/"monitoring-heapster-v2" replicas, need 1, creating 1

All I have are the cluster addons on this cluster
$ kubectl get rc
elasticsearch-logging-v1       elasticsearch-logging   gcr.io/google_containers/elasticsearch:1.4        k8s-app=elasticsearch-logging,version=v1   2
kibana-logging-v1              kibana-logging          gcr.io/google_containers/kibana:1.3               k8s-app=kibana-logging,version=v1          1
kube-dns-v3                    etcd                    gcr.io/google_containers/etcd:2.0.9               k8s-app=kube-dns,version=v3                1
                               kube2sky                gcr.io/google_containers/kube2sky:1.9
                               skydns                  gcr.io/google_containers/skydns:2015-03-11-001
monitoring-heapster-v2         heapster                gcr.io/google_containers/heapster:v0.14.0         k8s-app=heapster,version=v2                1
monitoring-influx-grafana-v1   influxdb                gcr.io/google_containers/heapster_influxdb:v0.3   k8s-app=influxGrafana,version=v1           1
                               grafana                 gcr.io/google_containers/heapster_grafana:v0.7

cc/ @thockin on general cluster issues.
cc/ @saad-ali and @ArtfulCoder  since they are working on heapster scalability issue. Can we have resource limit specified for our addons service? Especially monitoring and logging ones?
@vishh Why we only have one replica of heaster for monitoring?

Trying to figure out what it's logging I found that on most nodes /var/lib/docker/containers just has 2 containers, pause and fluentd, and neither of them log anything. On some nodes there are more, I guess these are the influxdb and es containers. They produce logs:
https://gist.github.com/bprashanth/5f7949b37aabade3023c
So maybe just telling those 2 to stay really quiet might help.

cc/ @satnam6502 on logging side  (https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112149413)

@bprashanth You mentioned above that "Sometimes there are OOMs, these don't affect the kubelet, but the node eventually dies" The two OOM killing examples are manage_accounts and manage_addresses, which all belongs to our /system cgroup, but we haven't oom protected them yet. But do you know why that particular node is changed to NotReady from Ready? You can have such information by doing kubectl describe node. Thanks!

Note that PR #9758 will make Cloud Logging the default after which no addons run for logging (unless Elasticsearch is explicitly set as the logging choice).

(unless Elasticsearch is explicitly set as the logging choice).

aka non-gce clusters right (or can one still use gce from aws)? if that's true it will make this even harder to debug :)

@mr-salty to what extend can people use Cloud Logging from AWS, Azure etc?

to what extend can people use Cloud Logging from AWS, Azure etc?

Hmm, I'm assuming even if they can, they won't. Each of those probabaly has a gcl equivalent that those people are more familiar with, and until someone writes plugins for them they'll probably stick with ES.

We can have multiple replicas if required. The plan is to place resource
limits on heapster pod after we profile the pod as part of scalability
testing.
On Mon, Jun 15, 2015 at 11:00 AM, Prashanth B<EMAIL_ADDRESS>wrote:

to what extend can people use Cloud Logging from AWS, Azure etc?
Hmm, I'm assuming even if they can, they won't. Each of those probabaly
has a gcl equivalent that those people are more familiar with, and until
someone writes plugins for them they'll probably stick with ES.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112154241
.

@satnam6502 only GCP services are officially supported at this time, although users can use configuration file settings to specify project and instance metadata on other platforms on an unsupported basis.

Ok, I don't think logging pods causing issues here by analyzing all NotReady nodes, I will explain it below:
Here are what I understood so far:

I observed the several nodes having 100% cpu usage all the time from pantheon, which indicates a kernel crash since I saw a similar issue with porch.com case before. I checked the serial log, it is kernel crash. But the node is not reboot. @brendandburns mentioned that there is a "reboot_on_panic" for each instance, we should check that one by default for all instances on GCE and GKE.

@bprashanth I noticed that all minions you created are g1-small (1 vCPU, 1.7 GB memory), which is indeed too small for one replica of heapster managing 100 nodes (we can optimizing heapster's resource usage in the future, but not in 1.0 scope). Here are what we can do:

Once we have more stats from #5880 to understand resource requirements from heapster based on the workload and nodes, we should place resource limits on heapster pods.
@erickhan suggested that for GKE / GCE users, we should suggest them to us bigger nodes, instead of g1-small. I made changes for our jenkin-e2e tests to avoid unnecessary overcommit issues last week. We can change our config-default.sh to have a bigger node by default too. I will send pr for that unless someone scream now.
We can determine the number of replicas based on cluster side based on the result from #5880 too.

I found most nodes are not useful because system oom due to heapster. But unfortunately heapster has oom_score_adj -900, which causes other daemons on the node being killed. I haven't figured why heapster misconfigured in @bprashanth cluster since my test cluster doens't have such issue. @vishh ?

According to 3), I realized that because heaspter process is inheritant the oom_score_adj from its parent process docker with -900. Will send PR to fix it now.

Ok, I don't think logging pods causing issues here by analyzing all NotReady nodes

A heapster/es crash is what got us the first NotReady node though?

I noticed that all minions you created are g1-small (1 vCPU, 1.7 GB memory), which is indeed too small for one replica of heapster managing 100 nodes (we can optimizing heapster's resource usage in the future, but not in 1.0 scope).

g1-small is essentially what we've been optimizing performance for, I think we recently bumped it up to n1-standard (but that isn't that much better).
Since this is OOMing with 0 extra pods,  customers can just never have a 100 node cluster of g1-smalls because of the 1 or 2 nodes heapster is running on?
@dchen1107  any idea what are the logs being sent to heapster? It can obviously fit on a g1-small, so why does it go nuts?

Heapster requires memory to store metrics fromm all nodes in the cluster.
If the cluster size os large, then a g1-small is not sufficient to run
heapster.
Inheriting docker daemon's oom_score_adj is a bug. Dawn is fixing it.
We can turn off heapster on a scale cluster and optimize the rest of the
components for g1-small, if required.
On Mon, Jun 15, 2015 at 1:39 PM, Prashanth B<EMAIL_ADDRESS>wrote:

Ok, I don't think logging pods causing issues here by analyzing all
NotReady nodes
A heapster/es crash is what got us the first NotReady node though?
I noticed that all minions you created are g1-small (1 vCPU, 1.7 GB
memory), which is indeed too small for one replica of heapster managing 100
nodes (we can optimizing heapster's resource usage in the future, but not
in 1.0 scope).
g1-small is essentially what we've been optimizing performance for, I
think we recently bumped it up to n1-standard (but that isn't that much
better).
Since this is OOMing with 0 extra pods, customers can just never have a
100 node cluster of g1-smalls because of the 1 or 2 nodes heapster is
running on?
@dchen1107 https://github.com/dchen1107 any idea what are the logs
being sent to heapster? It can obviously fit on a g1-small, so why does it
go nuts?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112203538
.

I  will conduct a test with a 100 node cluster set up with Elasticsearch
logging and pods running synthetic loggers on each node and report back.
On Jun 15, 2015 1:51 PM, "Vish Kannan"<EMAIL_ADDRESS>wrote:

Heapster requires memory to store metrics fromm all nodes in the cluster.
If the cluster size os large, then a g1-small is not sufficient to run
heapster.
Inheriting docker daemon's oom_score_adj is a bug. Dawn is fixing it.
We can turn off heapster on a scale cluster and optimize the rest of the
components for g1-small, if required.
On Mon, Jun 15, 2015 at 1:39 PM, Prashanth B<EMAIL_ADDRESS>wrote:

Ok, I don't think logging pods causing issues here by analyzing all
NotReady nodes
A heapster/es crash is what got us the first NotReady node though?
I noticed that all minions you created are g1-small (1 vCPU, 1.7 GB
memory), which is indeed too small for one replica of heapster managing
100
nodes (we can optimizing heapster's resource usage in the future, but not
in 1.0 scope).
g1-small is essentially what we've been optimizing performance for, I
think we recently bumped it up to n1-standard (but that isn't that much
better).
Since this is OOMing with 0 extra pods, customers can just never have a
100 node cluster of g1-smalls because of the 1 or 2 nodes heapster is
running on?
@dchen1107 https://github.com/dchen1107 any idea what are the logs
being sent to heapster? It can obviously fit on a g1-small, so why does
it
go nuts?
—
Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112203538
.

—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112207593
.

The oom score will help undead nodes, but the cpu and memory pressure from the addons will still affect syncPods latency making 99%ile type benchmarks hard to achieve (on a 100 node cluster 99%ile fails if one node is slow to start a pod: https://github.com/GoogleCloudPlatform/kubernetes/issues/3954).
A solution involving more restrictive limit ranges/cgroups is probably out of scope for 1.0, so this is currently a pointless lament :)

I am curious to know the kind of workloads a user would run on a g1-small
100 node cluster.
Why are we spending on optimizing for g1-small?
On Mon, Jun 15, 2015 at 3:06 PM, Prashanth B<EMAIL_ADDRESS>wrote:

The oom score will help undead nodes, but the cpu and memory pressure from
the addons will still affect syncPods latency making 99%ile type benchmarks
hard to achieve (on a 100 node cluster 99%ile fails if one node is slow to
start a pod: #3954
https://github.com/GoogleCloudPlatform/kubernetes/issues/3954).
A solution involving more restrictive limit ranges/cgroups is probably out
of scope for 1.0, so this is currently a pointless lament :)
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112223553
.

Maybe they want to run 30 pause pods :)
We probably should've started off with a bigger node size, meh. But I'm assuming resource pressure will happen on an n1-standard-1 as well (it's just a little bigger right, so it might take a few hours more)?

I'd expect heapster resource usage to be proportional to the size of the
cluster and not vary over time. We can wait for the scalability test
results before we arrive at any conclusions.
On Mon, Jun 15, 2015 at 3:12 PM, Prashanth B<EMAIL_ADDRESS>wrote:

Maybe they want to run 30 pause pods :)
We probably should've started off with a bigger node size, meh. But I'm
assuming resource pressure will happen on an n1-standard-1 as well (it's
just a little bigger right, so it might take a few hours more)?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/9788#issuecomment-112224953
.

See https://github.com/GoogleCloudPlatform/kubernetes/issues/5880#issuecomment-112233798 for initial scalability test results.

@vishh we use small VMs to keep our test cluster size & cost down. I agree that people probably won't use it that way in the wild.

OTOH, if I was speccing out a kubernetes cluster for production I would pick a 100 node g1-small cluster for the same reasons you just mentioned.

According to 3), I realized that because heaspter process is inheritant the oom_score_adj from its parent process docker with -900. Will send PR to fix it now.

@dchen1107 it sounds like this was the real P0 problem; now that #9824 is merged, can we close this issue?

... cpu and memory pressure from the addons will still affect syncPods latency making 99%ile type benchmarks hard to achieve...

@bprashanth Yeah. But we can split this into two issues: syncPods can get resource-starved; and our own addons are resource hogs. Can we find/file issues for both and close this?

The addons being resource hogs sounds like this issue, so maybe we can just demote it in priority? I'll  fork off a syncPods issue if there isn't one already. We should also surface somewhere that there is a relationship between node and cluster size, just so people aren't surprised.

Can we use this issue for misconfigured oom_score ones? That is a real p0 issue, and I had a fix for it which merged and @brendandburns is going to include it in our 0.19 patch release.
Can we move heapster's resource limits and scalability to  #5880 based on workload and cluster size?
to me, SyncPod one sounds like #9833?

@bprashanth Please filed a separate one for syncPods if you don't think it is dup of #9833. I am closing this one for now.

@bprashanth I don't think we are optimizing for g1-small. For example scalability e2e jenkins tests are using n1-standard-1 that have twice as much memory (3.75G vs 1.7G).

@fgrzadkowski we recently bumped it up (https://github.com/GoogleCloudPlatform/kubernetes/pull/9678, and that was a pr for the normal e2es, scalability just benefitted from it). For example if you check the build output from the jenkins scalability run of build 1228 you'll see we use g1-smalls.
@dchen1107 on a cluster I setup last night 2 nodes died, 1 recovered,
(e2e-test-beeps-minion-9sar, e2e-test-beeps-minion-s8eh). The node that's still down was running elasticsearch and the one that recovered was running heapster. It looks like the one that recovered might've had some memory issues ~8hours ago, but no reported OOMs:
I0616 05:29:22.342313    3075 container.go:363] Failed to update stats for container "/24423cb95446e509cc5e53a13832c66bec3a7f83923656249589da964da9b1e9": du command failed on /var/lib/docker/aufs/diff/24423cb95446e509cc5e53a13832c66bec3a7f83923656249589da964da9b1e9 with output  - fork/exec /usr/bin/du: cannot allocate memory, continuing to push stats
I0616 05:29:22.358187    3075 container.go:363] Failed to update stats for container "/a447483789dbda0099a29eac72255380119843ce5c17690e5fe77aac0bef405b": du command failed on /var/lib/docker/aufs/diff/a447483789dbda0099a29eac72255380119843ce5c17690e5fe77aac0bef405b with output  - fork/exec /usr/bin/du: cannot allocate memory, continuing to push stats
I0616 05:29:22.944631    3075 container.go:363] Failed to update stats for container "/7668f4ce116142e207a4c2e0084e0091619091b79ab5ef7c2985783f215021dd": du command failed on /var/lib/docker/aufs/diff/7668f4ce116142e207a4c2e0084e0091619091b79ab5ef7c2985783f215021dd with output  - fork/exec /usr/bin/du: cannot allocate memory, continuing to push stats

I've haven't recovered the one that's down.

@bprashanth Thanks for conducting the test again.
The only unrecovered node is caused by docker memory leakage. It is a known issue since docker 1.3.0, might even earlier version (https://github.com/docker/docker/issues/9139). Docker 1.7.0-rcX (the one I am currently validating) should have a fix for it. Once you restart docker, the problem should be gone, and the node should be recovered. I saw a similar problem before. On each node, there is a monit healthchecking docker daemon process periodically, in most cases, the docker in such bad state will be restarted by monit.


[FLINK-3393] [core] ExternalProcessRunner wait to finish copying error stream

Thanks for the fix. I'll merge as soon as travis is all green.

I merged it. Could you please close the PR if github doesn't.

Thanks!


More specific documentation and Trady test

Hello there,
I'm a new dotnet developer and I'm trying to make use of your Framework but i see very limited documentation. The only test I've been running is the benchmark and I find some things unclear.
I can't even find a tutorial of how Trady works on youtube there's only literally just what you offer.
Are there any plans on making more documentation?
Thanks for all :)
Greetings.

What specifically would you like to see documentation on?
I started out feeling the same way, but after going through the unit tests, most of my questions were answered.

Test functions utilizes Extension methoids of Ienumerable like, candles.Cci(20). In my case I would like to see some example of calling CCI function directly -
var cciResults = new CommodityChannelIndexByTuple(Input, 14);
I do not know how to create the "Input" object. An example of creating IEnumerable<decimal High, decimal Low, decimal Close> object would be nice. Thanks.


Update to 1.20.1

Please update chunk debug to 1.20.1

https://modrinth.com/mod/chunkdebug/version/M5Z6hDQa
should close this as complete....


SPFx loading old version files and sometimes from different environment

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Developer environment
Windows
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[X] 💥 Microsoft Edge
[X] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

browser version - Edge latest
SPFx version - 1.11
Node.js version
etc

Describe the bug / error

We have developed a custom SPFx component and deployed to different environments - Dev/Stg/UAT/Prod. We have enabled side loading in all the Dev/Stg/UAT environments. We have both the Extension & webpart in this SPFx solution. We observed that the  XYZ-extension-application-customizer_16345316eb49d720a1d078eb095432g9.js is loading from local catalog instead of appcatalog folder in the higher/prod environments where side loading is not enabled

Actual:
https://publiccdn.sharepointonline.com/<>.sharepoint.com/teams/<>/ClientSideAssets/<>/XYZ-extension-application-customizer_<>.js
Expected:
https://publiccdn.sharepointonline.com/<>.sharepoint.com/sites/catalog/ClientSideAssets/<>/XYZ-extension-application-customizer_<>.js

We have also observed that for some users the older version of file is getting loaded related to chunk.js and the functionality is breaking

Also the webpart is observed in loading in the context of the lower environment site where we have almost 10 webpart properties. But, it has only 5 webpart properties in higher/prod environment

If we are deleting the "Cookies and other site data" and "Cached images and files" and doing a fresh request in higher/prod site - everything is working fine.

All this was working properly till Apr 28th. We feel this issue has come up suddenly and guessing any environment issue.  Can you please let us know the possible cause for this issue and the resolution.
Steps to reproduce

Develop SPFx webpart
Deploy in more than one site collection
Open the lower env where the side loading is enabled in one tab
Open the higher/prod env site where the side loading is disabled in other tab
In the environment/site opened in second tab, we see the files are loaded from lower environments and not from respective environment.
IF we reverse the opening order of the sites, it is behaving in opposite way
Only the first opened environment is working fine
If we are deleting the "Cookies and other site data" and "Cached images and files" and doing a fresh request in higher/prod site - everything is working fine.

Expected behavior
Irrespective of how many sites are opened that has the same SPFx webpart - webpart should load the respective files from its own package and not from other site associated package and also not from the older versions of the spfx webpart assoiated files.

Hi Everyone, I have few observations on this issue - We have actually uninstalled the App in the site collection that we are using the associated webpart. Following are the observations

We see that very frequently users are getting the SPFx associated JS file loaded from the catalog only this time. But, from old version of the SPFx webpart
When we do Shift F5 webpart loads perfectly and functionality is fine
When we open a new tab and browse the same page again, Still we are seeing same issue again. Again we have to do the Shift F5 do resolve the issue

We cannot ask users to do this every time they come our page - This is really so painful and creating lot of new concerns on the stability. Can you please help us here in addressing this issue ASAP.
Thanks & Regards,
Bhanu

Hi Everyone, Attaching the two screenshots of the manifest with the correct and wrong references. Wrong reference is getting replaced with correct reference when we do a Shift F5.
Wrong Reference seen -

Correct reference seen

Thanks & Regards,
Bhanu

Hi Everyone,
Also, we observe the below console logs when SPFx loader is trying to load the wrong JS file - A preload for 'https://publiccdn.sharepointonline.com/xxxx.sharepoint.com/sites/catalog/ClientSideAssets/431dde0c-370f-4783-9d11-6a35d9fb4519/yyyyy-web-part_39d739c68144f7310a682a62b1d04eb1.js' is found, but is not used because the request credentials mode does not match. Consider taking a look at crossorigin attribute.
And after few fraction of seconds, we are seeing one more message -
The resource https://publiccdn.sharepointonline.com/xxxx.sharepoint.com/sites/catalog/ClientSideAssets/431dde0c-370f-4783-9d11-6a35d9fb4519/yyyyy-part_39d739c68144f7310a682a62b1d04eb1.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate as value and it is preloaded intentionally.
Hope this might give some more insights to the issue that we are facing.
Thankyou.

Just to elaborate a bit why this was closed. As this issue only happens in the MSIT tenant and is being worked using internal channels, there's no value on this for external audience.

We completely understand this. Thanks a lot @VesaJuvonen  @patmill  for the support in this regard.
Thanks & Regards,
Bhanu


Possible canditates removals from the index

Hi,
I spent some time to create a small web dataset command line tool to start analyzing the index and help others creating a dataset from it.
The first task was to analyze the feeds to check if they were all English sites as per the guidelines. After an initial processing, it determined that some feeds are not English feeds.
To identify the language of a feed, it proceeds that way:

use langdetect on the title and description of the feed
use langdetect on the title and content of the articles of each feed
it reassign a different language for the feed if the articles it contains are in majority of another language

The result of this processing is attached to this issue. The ones that are tagged with a specific language should most likely be of that language, and could be candidate for removal. For the ones with empty languages, this happens when there is not enough data to detect the language (concat(title + description) < 64 characters) for most of its articles, so the current heuristic can't determine the appropriate language for the feed.
Attached: small_wed_non_en_feeds.csv

Thanks this is excellent work and seemingly very high accuracy (i just randomly checked a few).
Can you submit a PR to remove these?

I found my own website’s feed on this list (https://joeldueck.com/feed.atom), and was surprised to see it had no language listed in your CSV file.
My feed is in Atom format, and pursuant to the Atom 1.0 spec it includes the optional xml:lang attribute with the value "en", the ISO 639-1 code for English.

Some other entries in the CSV with no language indicator are http://michaelhoney.com/writing (English), https://elly.town/d/blog/ (English), and https://isohedral.ca/ (English). You might want double-check the code behind your processor.

The idea is to indicate onlly what non-english sites are (which csv does) so that we can remove them from the index.

For the feeds with the empty lang field, here is the relevant information from the issue's description:

For the ones with empty languages, this happens when there is not enough data to detect the language (concat(title + description) < 64 characters) for most of its articles, so the current heuristic can't determine the appropriate language for the feed.

So, this is expected at this stage. Further work will be required down the line to improve the accuracy/coverage of those. It shouldn't be a big deal to get this number down by a lot, I am just missing time at the moment.
@vprelovac yes I will do a PR for the ones tagged tomorrow morning

see linked PR @vprelovac


Always use prompts for code snippets?

Hi there,
this is a followup to @norosa's comment:

b) the sh$ prompt was designed for consistency with the frequent use of cr> prompts in the main docs. we can revisit that decision if you like, but I think that should be done as a separate thing (perhaps with a PR to crate-docs/style)

With kind regards,
Andreas.
Originally posted by @norosa in https://github.com/crate/cratedb-examples/pull/3#discussion_r647563591

Question: Is it allowed to omit prompts in tutorials with code/console snippets altogether?
Please vote on this issue with 👍 or 👎.

sh$ and cr> in are both used by the crate reference doctests to indicate the start of a testable example:
https://github.com/crate/crate/blob/83a29b1e20d7d96eefa1545c5eb18646f594e5b9/blackbox/test_docs.py#L128
I think we should use sh$ and cr> elsewhere in the docs for consistency


CI integration: allow adding [skip ci] to semantic-release commit

We've successfully implemented semantic-release in our Gitlab CI workflow on the master-release branch, passing GL_TOKEN through as a protected variable.
There's a minor inconvenience, when semantic-release pushes the new version to the branch within the CI job, it'll trigger the pipeline again (resulting in one needless CI run).
As far as I know, semantic-release allows making use of the [skip ci] flag in semantic-release commit. I wonder if this is the only solution - then maybe it would make sense allowing to set a flag --skip-ci to python-semantic-release which would add [skip ci] to the version commit (or, even better, using ci.skip flag in the git push, not messing up commit messages). Maybe it would even make sense passing this flag by default, since the version-commit will likely have no new code.
Here's our gitlab-ci.yml, for completeness:
build:
  image: python:alpine
  stage: build
  variables:
    GITLAB_CI : "true"
  script:
    - apk update && apk add git
    # checkout to release branch
    - git checkout -B "$CI_BUILD_REF_NAME" "$CI_BUILD_REF"
    - git config --global user.name "semantic-release (via GitlabCI)"
    - git config --global user.email "semantic-release@gitlab"
    # install python-semantic-release
    - pip install python-semantic-release
    # publish release
    - semantic-release publish
  only:
  - master-release

It'll trigger the pipeline again (resulting in one needless CI run).

Considering this needless is kind of subjective I think. Running the CI job is required to get the a green build for the tag for some combinations of CI and VCS hosting.
I am open to have an option that adds [skip ci] to the commit message, but it needs to be optional and of by default.

For those who would be interested (I had trouble finding this information) in gitlab you could use this to skip the CI :
[semantic_release]
commit_subject = chore(release): Release v{version} [skip ci]

[Skip Ci] Skip CI on commit release

Works also like a charm on Google Cloud Build


Rename androidstudio.js to android-studio.js

All of the styles (expect androidstudio) are named in a uniform way, where words are joined with a - and lowercased.
Eg: atom-one-dark.js
Describe the solution you'd like
The solution is simple, Rename the file androidstudio.js to android-studio.js. Doing this will ensure uniformity among the styles. But also note that it may cause problems for most of the dependants.
Describe alternatives you've considered
This is not a problem. So leaving it as it is, won't cause any problems.
Additional context
As I said before, I understand this renaming endeavor may break things which would cause so many problems for the dependants. So if you don't want to rename the file, just close the issue.
I am not good at writing issue reports or feature requests. So forgive me for all of my errors.

These are built by a script and named automatically based on their highlight js name, when the highlight.js name uses a space a - is added, since no space exists in highlight.js the automated script will not add a dash, the overhead requireed seems too much for the benefit gained. See screenshot below from highlight.js docs list of styles


Apify.events unit test randomly fails

The test case "should send persist state events in regular interval" fails about 50% of time, depending on the system workload. The unit test should be deterministic.

Doesn't fail anymore.


RFE: expand type support in opencoarrays module

Avg response time

Request for Enhancement (RFE)
As pointed out by Malcolm Cohen on the WG5 Fortran standards committee mailing list, most or all of the type-specific specific procedures in src/extensions/opencoarrays.F90 can be collapsed down to a set of procedures in which the first argument is declared unlimited polymorphic (class(*)).  This will both simplify the code and broaden the supported use cases.
As an example, the following program demonstrates that this approach which works at least with GCC 5 and higher:
program main
  implicit none
  integer :: b(10,10)
  call print_storage_info(b)
contains
  subroutine print_storage_info(a)
    class(*), intent(in), target, contiguous :: a(..)
    print *,storage_size(a),size(a)
  end subroutine
end program

The storage_size and size intrinsics should be used to replace the c_sizeofcalls that currently necessitate the type-specific implementations in opencoarrays.F90.

I started looking into this.  One issue is choosing a with which to test it.  It should be a compile that triggers the CMake code that builds of the extensions tests -- any non-gfortran compiler or a gfortran compiler version < 5, but let's focus on non-fortran compilers.  In deciding which ones to test, there are a few use cases to consider:

Extending the capability of an existing Fortran 2008 compiler such as
a. Intel
b. Cray
Accessing Fortran 2015 features in a Fortran 2003 compiler:
a. PGI
b. flang
c. NAG
d. IBM
Arguments in favor of testing with the compilers in category 1 above include ease of implementation and greater likelihood that existing users will want additional parallel features (such as collective subroutines) because they're already using some parallel features (such as coarrays).  Arguments in favor of testing with category 2 compilers are that these vendors might be most interested in adopting OpenCoarrays.  An additional argument in favor of compiler 2b above is that it is open source, which might offer the greatest possibility of influencing the project itself and also might provide an avenue to increasing the interest of 2a.

Another consideration is whether the compiler supports the Fortran 2008 intrinsic function storage_size and the Fortran 2015 C-interoperability features such as assumed rank.  These features greatly simplify the code in the opencoarrays module.  The requisite features include, for example, assumed rank.  At this time, I believe the list of compilers that support these features includes:
I. Intel
II. Cray
III. IBM
IV. GNU
I'm going to first check whether flang supports the mentioned 2008 and 2015 features.  If not, I'll switch to Intel.  I just submitted a flang installation issue that's currently blocking me from evaluating it.

I also submitted an issue on the spack package manager recommended by the flang README.

It doesn't appear there's going to be a fix in the foreseeable future for building flang on macOS using spack. Building flang with spack on Lubuntu Linux is also failing for me, which I'll research and/or report.  In the interim, I've sent the code below to Portland Group's J3 representative.  Any compiler that either can compile the code below or supports the Fortran 2015 feature of assumed-size arguments for c_sizeof will facilitate the enhancement pondered in this issue.  Unfortunately, gfortran appears to be the only compiler that can support the code below and of course the code below isn't needed with gfortran!  Arrgghh...  The Cray and Intel compiler both die with internal compiler errors on the code below even thought they nominal support the necessary features.  I'll submit bug reports and I think I'll proceed with rewriting src/extensions/opencoarrays.F90 anyway.
program main
  use iso_c_binding, only : c_sizeof,c_int32_t
  implicit none
  integer(c_int32_t) :: array(10,1)
  associate(expected=>c_sizeof(array),calculated=>c_sizeof_assumed_rank(array))
    if (expected==calculated) then
      print *,"Test passed"
    else
      print *,"Test failed: expected ",expected,", calculated",calculated
    end if
  end associate
contains
   pure function c_sizeof_assumed_rank(a) result(c_sizeof_a)
     use iso_c_binding, only : c_size_t,c_int32_t
     class(*), intent(in), target, contiguous :: a(..) ! Fortran 2015 assumed-rank dummy argument
     integer(c_size_t) :: c_sizeof_a
     integer, parameter :: bits_per_byte=storage_size(0_c_int32_t)/c_sizeof(0_c_int32_t)
     c_sizeof_a= storage_size(a)*size(a)/bits_per_byte  ! Fortran 2008 storage_size intrinsic function
   end function
end program

The code in my last comment works with the IBM compiler so at least gfortran is not the only compiler that can handle it.

Bug reports have been submitted on the Cray and Intel compilers.  Now I'll move forward with revising the module while we wait for the non-GNU compilers to support the requisite features.

And now I've also reported a bug in the NAG Fortran compiler based on the at the beginning of this thread.  An internal compiler error occurs in the NAG compiler when an unlimited polymorphic dummy argument has the contiguous attribute.  I also submitted a feature request to NAG for the assumed-rank dummy argument.  I'm tempted to write rank-specific versions of the procedures in opencoarrays.F90 and copy all arguments to assure contiguity and then test the opencoarrays module with the NAG compiler.

Intel reports that a fix for the bug related to this issue will be in their 18.0 Update 2 release.

Based on project needs, it appears this enhancement will be most useful with the Intel compiler in order to facilitate the use of OpenCoarrays collective subroutines with non-GCC compilers.  This issue will be addressed after Intel finishes fixing related bugs.   Currently, the required features only work with the Cray compiler (with the -Oipa0 flag) and with gfortran.

Intel Fortran Update 2 (Version <IP_ADDRESS> Build 20180210) now correctly compiles both the code block in the original post on this issue and the more complete example posted on October 18, 2017, in this thread with one minor workaround to switch a constant declaration to an associate block:
program main
  use iso_c_binding, only : c_sizeof,c_int32_t
  implicit none
  integer(c_int32_t) :: array(10,1)
  associate(expected=>c_sizeof(array),calculated=>c_sizeof_assumed_rank(array))
    if (expected==calculated) then
      print *,"Test passed"
    else
      print *,"Test failed: expected ",expected,", calculated",calculated
    end if
  end associate
contains
  pure function c_sizeof_assumed_rank(a) result(c_sizeof_a)
    use iso_c_binding, only : c_size_t,c_int32_t
    class(*), intent(in), target, contiguous :: a(..) ! Fortran 2018 assumed-rank dummy argument
    integer(c_size_t) :: c_sizeof_a
    associate(bits_per_byte=>storage_size(0_c_int32_t)/c_sizeof(0_c_int32_t))
      c_sizeof_a= storage_size(a)*size(a)/bits_per_byte ! Fortran 2008 storage_size intrinsic function
    end associate
  end function
end program

which is a workaround for Intel's compiler not supporting the Fortran 2018 feature that allows the c_sizeof intrinsic function to accept assumed-rank arguments.
With this latest workaround, Intel finally becomes the first compiler for which it is both feasible and useful to work on this issue.  Doing so will allow code compiled with a Fortran 2008 compiler to access some Fortran 2018 features (e.g., collective subroutines) via the opencorrays module (in src/extensions/opencoarrays.F90), which justifies my having just added this issue to the low-hanging fruit project.  The one challenging aspect of the project, however, will be adjusting the opencoarrays module to use the latest gfortran descriptors (which have to be constructed for passing arguments that OpenCoarrays will recognize even when compiled with a different compiler.

@zbeekman I marked this as low-hanging fruit.  How do I make it appear in the To Do column on the low-hanging fruit project kansan?  And what does "awaiting triage" mean?

When you mark an issue as part of a project from the issue's main page (i.e., here https://github.com/sourceryinstitute/OpenCoarrays/issues/454) it gets filed into the awaiting triage section on the project page. You must then find it in the "awaiting triage" section and drag it to the "To Do" column.

@rouson There are automation settings for the columns; I updated them so that issues that are moved to the low-hanging-fruit project will automatically end up in the To Do column. This should help make things more tractable.

This can finally be closed.  A minimal example of this strategy has been implemented on the new armflang branch and testing will continue from there.


"Open definition to the side" opens definition in the same group

Issue Type: Bug
"Open definition to the side" works... unless there are multiple definition candidates. In this case the disambiguation popup shows up, after which the definition is opened in the current group, rather than to the side
VS Code version: Code 1.45.0 (d69a79b73808559a91206d73d7717ff5f798f23c, 2020-05-07T16:10:24.511Z)
OS version: Linux x64 4.15.0-99-generic

Extension
Author (truncated)
Version

Bookmarks
ale
11.1.0

pascal-language-basics
Ans
0.1.8

vscode-eslint
dba
2.1.5

gitlens
eam
10.2.1

shell-format
fox
7.0.1

html-preview-vscode
geo
0.2.5

vscode-graphviz
joa
0.0.6

language-haskell
jus
3.0.3

rainbow-csv
mec
1.7.0

python
ms-
2020.4.76186

remote-ssh
ms-
0.51.0

remote-ssh-edit
ms-
0.51.0

cpptools
ms-
0.27.1

debugger-for-chrome
msj
4.12.6

LiveServer
rit
5.6.1

llvm
RRe
0.0.3

code-settings-sync
Sha
3.4.3

markdown-preview-enhanced
shd
0.5.3

perforce
sle
3.2.0

graphviz-interactive-preview
tin
0.0.8

cmake
twx
0.0.17

vscode-lldb
vad
1.5.0

vscode-preview-server
yui
1.3.0

after which the definition is opened in the current group, rather than to the side

From there, you need to use command+click to open to the side. We don't seed the widget with an open-to-side-by-default mode because there is no visual indication for that. Also, command+click is always opening a location to the side.

@jrieken : Thanks for looking into it, and the suggested workaround.
Wouldn't it make more sense to implicitly open to the side from the disambiguation widget if the lookup was initiated with "Open definition to the side"? Right now the behavior is inconsistent, frustrating (not to mention confusing) - "Open definition to the side" sometime opens to the side, sometime it doesn't.
I find this functionally (open to the side) very valuable and a key part of the development workflow. Is there any chance you'd reconsider and have the "open definition to the side" work consistently, regardless of disambiguation widget or not?

FYI - the Ctrl(command)+click in the disambiguation widget is also slightly broken: #97542


fix: Update subsystem tests with new telemetry texts

Description

Pull-request quality

[x] The title adheres to this guide
[ ] Tests are written and executed locally
[ ] Subsystem tests have been tested (by manually deploying to dev_002)

Fixed in #2541


fix: update to eye v3.12.1

fix: update to eye v3.12.1

:tada: This PR is included in version 4.11.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release
v4.11.1

Your semantic-release bot :package::rocket:


add Prince of Songkla University

student email :
@email.psu.ac.th for outlook
@psu.ac.th for gmail
Department of Computer Engineering, Faculty of Engineering
https://www.coe.psu.ac.th/
Department of Computer Science, Faculty of Science
https://office.cs.psu.ac.th/Eng/

domain is psu.ac.th
https://www.psu.ac.th/

@iceman951 I am sorry, the domain psu.ac.th is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Pack at https://education.github.com/discount_requests/application. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


Open recommended apps if installed

I would try to implement this, is that okay?
I'd need some more information on the UI, if there should be one button that opens the app if installed or separate buttons for download/open. (I know, it was my idea, I would prefer the first one, but what should be the text on the link? "Open/Download"?)

I don't think it's possible with the current architecture - the links are being loaded by markdown in the guide tool.
Also, if I'm not mistaken android now requires a package query in the manifest which I'm not sure I want to add without contacting the developers of the recommended apps.
Maybe hold off on this for now? I've also marked issues with help-wanted that are ready for community contribution, as a lot of the untagged issues here have some background information as to why they haven't been implemented/are in the state they are.


macOS packages have missing .lproj directories which prevents getUserMedia working

I use nw-builder to build an application of mine that uses the WebRTC getUserMedia API to capture images.
The capture part of the program has stopped working in macOS Mojave and above once built with nw-builder. The issue is that the macOS's "application x" would like to use your camera prompt is not displayed when the navigator.deviceMedia.getUserMedia() function is called.
I have determined that the issue occurs because packages built with nw-builder do not include .lproj directories inside of Contents > Resources.
The fix is to simply add a method to copy these .lproj directories from the downloaded binary to the packaged application. The .strings files inside of the .lproj directories contain lots of references to Chromium and nwjs so it would be good if these could be renamed to the selected appName.
Here is a comparison between manually building the same app (as per http://docs.nwjs.io/en/latest/For Users/Package and Distribute/#package-your-app) and using nw-builder:
Manual build steps

Download nw.js 0.43.6 from nwjs.io
Copy custom app.icns and document.icns files
Open the macOS package and copy an app.nw directory to Contents > Resources.
Contents > Resources contains the following files:

Building the same app with nw-builder

Run the following:

var NwBuilder = require('nw-builder');
var nw = new NwBuilder({
  "platforms": ["osx64"],
  "files": [
    "app/**",
    "icons/**",
    "package.json"
  ],
  "version": "0.43.6",
  "winIco": "icons/icon.ico",
  "macIcns": "icons/icon.icns",
  "flavor": "normal"
});

// Log stuff you want
nw.on('log',  console.log);

nw.build().then(function () {
   console.log('all done!');
}).catch(function (error) {
    console.error(error);
});

Contents > Resources of the generated package contains the following:

Fixed by #389


Including the google.protobuf types breaks the code

Hello,
Whenever I add gRPC well-known types like Empty, the protoletariat breaks the python code changing:
from google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2

to
from .google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2

Thanks for the report. I should be able to look into this later tonight or tomorrow morning.

@bartek1912 The PR will be merged shortly and a release automatically cut to PyPI and ghcr.io shortly thereafter. Thanks for reporting this.

Thanks! ;)


Misplaced toolbar

When clicking on a cheese, the view's Toolbar is in a very strange place
http://imgur.com/X9uDfLh

Same here

This only seems to be occurring on 5.0

+1

same here.. its in 5.0

Same with me. Have a One plus One running 5.0

Same here, 5.0


Upgrade to v0_5

Proto buffers need to be upgrade when new definitions are released at https://github.com/googleapis/googleapis
v0_5 release: https://ads-developers.googleblog.com/2018/10/announcing-v05-of-google-ads-api.html

Latest proto definitions now available: https://github.com/googleapis/googleapis/commit/200fbbe59cc7b0077ae525eb5f3d6420c0c53e61


Install TagBot as a GitHub Action

You're receiving this pull request because the now-deprecated Julia TagBot GitHub App is installed for this repository. This pull request installs TagBot as a GitHub Action.
If this PR does not belong in this repository for any reason, feel free to close it.
See this Discourse thread for more details and to ask questions.
Please do not comment on this pull request, because it is unlikely that I (@christopher-dG) will see it.

Codecov Report

Merging #15 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #15   +/-   ##
=======================================
  Coverage   51.54%   51.54%
=======================================
  Files           1        1
  Lines          97       97
=======================================
  Hits           50       50
  Misses         47       47

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a8e066c...ff6d97b. Read the comment docs.


vm ip is diffrent from kubectl get vmi

What happened:
when I create a pysical network with  bridge-test ,
apiVersion: "k8s.cni.cncf.io/v1"kind: NetworkAttachmentDefinitionmetadata:  name: bridge-testspec:  config: '{
      "cniVersion": "0.4.0",
      "name": "bridge-test",
      "type": "bridge",
      "bridge": "br1",
      "ipam": {
        "type": "host-local",
        "subnet": "<IP_ADDRESS>/24",
        "rangeStart": "<IP_ADDRESS>",
        "rangeEnd": "<IP_ADDRESS>",
        "routes": [
          { "dst": "<IP_ADDRESS>/0" }
        ],
        "gateway": "<IP_ADDRESS>"
      }
    }'

and I want to  create a vm with the bridge-test network, with yml
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  labels:
    kubevirt.io/vm: nginx-provisioner
  name: cirros-openstack-vm-special-ip
  namespace: default
spec:
  running: true
  template:
    metadata:
      annotations:
        k8s.v1.cni.cncf.io/networks: '[{"interface":"vm-multi-nic","name":"bridge-test","namespace":"default","ips": ["<IP_ADDRESS>/24"] }]'
        #k8s.v1.cni.cncf.io/networks: bridge-test@eth0
      labels:
        kubevirt.io/vm: nginx-provisioner
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: virtio
            name: datavolumedisk1
          - disk:
              bus: virtio
            name: cloudinitdisk
          interfaces:
          - name: vm-multi-nic
            bridge: {}
        resources:
          requests:
            memory: 1Gi
      networks:
      - name: vm-multi-nic
        multus:
          default: true
          networkName: default/bridge-test
      volumes:
      - dataVolume:
          name: fedora-nginx
        name: datavolumedisk1
      - cloudInitNoCloud:
          networkData: |
            version: 2
            ethernets:
              eth0:
                addresses: [ <IP_ADDRESS>/24 ]
          userData: |
            #!/bin/sh
            # removing instances ensures cloud init will execute again after reboot
            mkdir ~/123
            sudo echo -e "zlinux\nzlinux" | passwd root
            rm -rf /var/lib/cloud/instances
        name: cloudinitdisk
  dataVolumeTemplates:
  - metadata:
      name: fedora-nginx
    spec:
      storage:
        storageClassName: managed-nfs-storage
        accessModes:
        - ReadWriteMany
        resources:
          requests:
            storage: 24Gi
      source: # clone from source dv
        pvc:
          namespace: vm-images
          name: fedora-cloud-base-1 # the dv need prepare

and then  I have create the vm successed, but the ip is diffrent
[root@master mutinic]# kubectl  get vmi
NAME                             AGE   PHASE     IP              NODENAME   READYcirros-openstack-vm-special-ip   15m   Running   <IP_ADDRESS>   master     True[root@master mutinic]# virtctl  console cirros-openstack-vm-special-ip
Successfully connected to cirros-openstack-vm-special-ip console. The escape sequence is ^]

[root@cirros-openstack-vm-special-ip ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 8e:e6:ef:fd:09:3c brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::8ce6:efff:fefd:93c/64 scope link
       valid_lft forever preferred_lft forever
[root@cirros-openstack-vm-special-ip ~]#

why this is happend?
What you expected to happen:
I think I create a vm  with a speical IP, and I have declare the IP  address with the annotations.
the vm get the ip should be same as the declare IP.

maybe I was mistake to use it, I can work around from :

define the interface use static ip
annatition the interface in the virtualmachine yml
se ip use the cloudint

Hi @Erick-shi, I'm not really sure what your desired state is, but please note that you don't need to manually set the k8s.v1.cni.cncf.io/networks annotation on the VMI.
When you define a multus network, KubeVirt (or more precisely the VMI controller) creates the virt-launcher pod with the required k8s.v1.cni.cncf.io/networks annotation to be consumed by Multus.
Please note that virt-controller also copies VMI annotations to the virt-launcher pod - so manually setting the k8s.v1.cni.cncf.io/networks probably confuses the system.
/sig-network
@AlonaKaplan @EdDev

/sig network

@orelmisan
hi, thanks for your reply. I want set a static ip for my vm, for example I want to set my vm ip is <IP_ADDRESS>  and I have set <IP_ADDRESS>  use cloudInit.  I finaly  get the <IP_ADDRESS> , but I use kubectl get vmi the ip address show me <IP_ADDRESS>, l want  the vm ip is same as the kubevirt show me.
[root@master mutinic]# kubectl  get vmi
NAME                             AGE   PHASE     IP              NODENAME   READYcirros-openstack-vm-special-ip   15m   Running   <IP_ADDRESS>   master     True
[root@master mutinic]# virtctl  console cirros-openstack-vm-special-ip
Successfully connected to cirros-openstack-vm-special-ip console. The escape sequence is ^]

[root@cirros-openstack-vm-special-ip ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 8e:e6:ef:fd:09:3c brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::8ce6:efff:fefd:93c/64 scope link
       valid_lft forever preferred_lft forever
[root@cirros-openstack-vm-special-ip ~]#

@orelmisan  may I am mistake. I have test the issue retry,

defile a static ip with  static ip like <IP_ADDRESS>

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: static
spec:
config: '{
    "cniVersion": "0.4.0",
    "name": "bridge-test",
    "type": "bridge",
    "capabilities": { "ips": true },
    "bridge": "br1",
    "ipam": {
      "type": "static",
        "addresses": [
                      {
                        "address": "<IP_ADDRESS>/24",
                        "gateway": "<IP_ADDRESS>"
                      }
                  ]
    }
  }'

defile a virtualmachine yaml

      networks:
      - name: vm-multi-nic
        multus:
          default: true
          networkName: default/static
      - cloudInitNoCloud:
          networkData: |
            version: 2
            ethernets:
              eth0:
                addresses: [ <IP_ADDRESS>/24 ]

and it work for me.  Are my testing steps correct?  And  I have two question,

when I set a static  ip for vm , should  I define a NetworkAttachmentDefinition every time?
I   deliberately manually defined two NetworkAttachmentDefinitions, and their IPs are the same, but the virtual machine can still be created normally. Does kubevirt have no IP detection function?

/reopen

If you're using static IPs in cloud-init, might as well just drop the IPAM stanzas from the NAD. I would do that.
Your approach in https://github.com/kubevirt/kubevirt/issues/10887#issuecomment-1859372436 is correct but IMO it is very cumbersome: you pretty much need a 1:1 association between VM and NAD - each of them to configure a single interface IP.
For instance, we do not support IPAM for Openshift VMs in our downsteam ... For the exact reason you report: a VM restart / migration leads it to have a different IP, which breaks networking.
Thus, we recommend not using IPAM at all.

@maiqueb ok, thanks for your commit.


[2024 June] Develop

Summary
description
Reference

This PR is clean!

  Generated by :no_entry_sign: Danger


Add 'IncludeManagementTools' to MSFT_xWindowsFeature

In a bid to cut down the size of configurations I think it would be a good idea to stick IncludeManagementTools into the WindowsFeature resource. What do you guys think?

@LiamFisher92 - I've got no objection to this feature and it does sound pretty useful (although best practice is to not install management tools to servers at all and of course :grin:).
The only thing that might be a challenge is that Test-TargetResource will need to identify if any "management tools" are missing from a feature or role. So it could be a good idea to look into this.

@PlagueHO Ask and you shall receive. 😃  Still some work to go which is why there isn't a PR yet. Let me know if you see anything wrong!
https://github.com/LiamFisher92/xPSDesiredStateConfiguration/tree/IncludeManagementToolsDev

@LiamFisher92 - awesome work! Nicely done! :grin:
I noticed you submitted a PR for this. I'll give you some initial feed back on it.

@PlagueHO Thanks 😁 It's a shame that IncludeManagementTools isn't included in Get-WindowsFeature. Would have made life a little easier. Trying to figure out the MSFT_ServerManagerRequestGuid stuff was a little... painful. New-CimInstance did not like the class one bit! Which is why I'm using the [CimInstance]::new ctor. I still don't quite understand what it's doing shifting the bits along by 8 (6 times). It doesn't really make much sense to me, but it works so I'm not going to touch it!
I wasn't going to submit a PR, but I thought I'd get it up on Reviewable. Thanks for the feedback! 😃


Create: 0219-contains-duplicate-ii.ts

File(s) Modified: 0219-contains-duplicate-ii.ts
Language(s) Used: typescript
Submission URL: https://leetcode.com/problems/contains-duplicate-ii/submissions/863725143/

@Kgosi-Motlogelwa Thank you for your contribution 👍


Emulator is not having microphone

Hi @shwetajan20
Which version of the emulator are you using?  (Have you set the AppId and Password fields in the emulator?  To use speech, the bot must be registered and using a valid MS AppId and Password.)

Tried with 4.0.15, when i set AppId and password in Emulator its showing
Mic but its not functional. and same appid and pwd when use in
web.config... emulator itself is not functional...
When tried with 3.5.36, mic is visible, but its taking only few words...
On Thu, Jul 5, 2018 at 11:47 PM, Eric Dahlvang<EMAIL_ADDRESS>wrote:

Hi @shwetajan20 https://github.com/shwetajan20
Which version of the emulator are you using? (Have you set the AppId and
Password fields in the emulator? To use speech, the bot must be registered
and using a valid MS AppId and Password.)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/BotFramework-Emulator/issues/672#issuecomment-402809729,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ama4WEQYUQokz7Au1dPztiDURU5hfO47ks5uDlhEgaJpZM4VCCiD
.

--
Thanks & Regards
Shweta Agarwal

Hi @shwetajan20 ,
This is a known issue in v4 with the speech token exchange process (#646) and there is a PR out for a fix #686 .
Thanks

Merged #686, should have it fixed now.


The known vulnerability in the shared library zstd which blacklite depends on. Can you help upgrade to patch versions?

Hi, @wsargent , @shipkit-org , I'd like to report a vulnerability issue in com.tersesystems.blacklite:blacklite-codec-zstd:1.1.0.
Issue Description
I noticed that com.tersesystems.blacklite:blacklite-codec-zstd:1.1.0 directly depends on com.github.luben:zstd-jni:v1.4.5-6 in the pom. However, as shown in the following dependency graph, com.github.luben:zstd-jni:v1.4.5-6 sufferes from the vulnerability which the C library zstd(version:1.4.5) exposed: CVE-2021-24032.
Dependency Graph between Java and Shared Libraries

Suggested Vulnerability Patch Versions
com.github.luben:zstd-jni:v1.4.9-1 (>=v1.4.9-1) has upgraded this vulnerable C library zstd to the patch version 1.4.9.
Java build tools cannot report vulnerable C libraries, which may induce potential security issues to many downstream Java projects. Could you please upgrade this vulnerable dependency?
Thanks for your help~
Best regards,
Helen Parr

Filing https://github.com/tersesystems/blacklite/pull/25

Fixed in https://github.com/tersesystems/blacklite/releases/tag/v1.1.1


Add pop-up to input arguments

Description
Adds a pop-up window to enable the input of the parameters required to execute the transitions in the FSM. For now, this is just a placeholder, front end focused. In the final implementation, the required arguments will be pulled from the controller FSM and there might be transitions that do not require arguments, which will also be taken into account.
https://github.com/user-attachments/assets/e441a258-b691-43f7-b4c3-27e7b0f80789
Fixes #216
Type of change

[ ] Documentation (non-breaking change that adds or improves the documentation)
[x] New feature (non-breaking change which adds functionality)
[ ] Optimization (non-breaking, back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Breaking change (whatever its nature)

Key checklist

[x] All tests pass (eg. python -m pytest)
[ ] The documentation builds and looks OK (eg. python -m sphinx -b html docs docs/build)
[x] Pre-commit hooks run successfully (eg. pre-commit run --all-files)

Further checks

[x] Code is commented, particularly in hard-to-understand areas
[x] Tests added or an issue has been opened to tackle that in the future. (Indicate issue here: # (issue))

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.86%. Comparing base (d2d3635) to head (5b809d3).

Additional details and impacted files
@@                Coverage Diff                @@
##           state_machine     #227      +/-   ##
=================================================
+ Coverage          84.55%   84.86%   +0.30%
=================================================
  Files                 35       35
  Lines                492      502      +10
=================================================
+ Hits                 416      426      +10
  Misses                76       76

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


updated my name!

Can you add my name, tho it looks rich

Hey @tanishq-soni, thanks for suggesting a change but this repo isn't really accepting changes in the form of people adding their names to the site.


support inverse: jsdoc-to-ts

Now that .ts syntax will run natively in Node, it can be a better fit for many projects.
One of my projects has hundreds of .js files with TypeScript annotations in JSdoc but I'd prefer to have .ts files for better IDE support and readability.
Given your expertise converting JSDoc to TS, would you be interested in making a package (or extending this one) to do the reverse?

TS in Node is just another case of feature creep increasing conflict potential (like type annotations being parsed as < operators, so valid TS still parses as valid JavaScript, but executing wrongly... oof).
Personally I started to use this project to finally get rid of TS again, simplyfying build pipelines along the way (browsers support ESM). I just hope this never happens to keep things simple.


[Bug]: Closed LN connection missing transfer from spending (low spending balance)

Describe the bug
I just closed my LN connection the normal way (moving the slider to 0% and accepting the transfer from spending) but no activity was created. I can only assume no onchain transaction exists. I had a very low spending balance, a few hundred sats, but it should still have enough in reserve to transfer funds to savings.
orderId of closed connection: a63d2f7d-d66c-4e3b-b68f-4c1cd7de6b1f
ldk node id: 0297f3b31720d311628fe292921cfe661aa0467ee4b8317e8758827f00f699be8c
bitkit_ldk_logs_1717421562063.zip
Reproduce

With a single LN connection (mine was established via CJIT)
Send most the spending balance funds away so only a few hundred sats remain
Close the channel normally (I did is by moving slider to 0% and proceeding)
The main screen is displayed with 0 spending balance but no transfer to spending funds activity is displayed (even after waiting over an hour and manually refreshing)

Screenshots / Recording
No response
Operating system
Android 13 TKQ1.221114.001
Bitkit version
1e714a948f69789cec42936226bfeb43ac8cf5e6
Log output
No response

I think this is the closing tx:
2024-06-03 13:09:36 INFO (LDK): Broadcasting closing tx with txid 0b16a5e9f29482219dd2add31bdf04addc71cd02406eb075eabfd031239e68a7 (lightning::ln::channelmanager 6611)

This closing tx has just 364 sats coming back to you, below dust limit


Introduce OpenSSF Scorecard GitHub action

Suggested commit message:
Introduce OpenSSF Scorecard GitHub action (#574)

And resolve some of the issues it identified.

See https://securityscorecards.dev

For the output, see this view. Only warnings about non-pinned dependencies remain. Those can be resolved using Renovate, IIUC, so I propose we address that separately.

Oh btw, Monday we'll do some big upgrades to Renovate. Let's see if it picks up on the GHA dependencies again. Otherwise, we'll look into that. Pending that we can setup the pinning that it now flags 😄. So to be continued.


WIP: Pin container base images

What type of PR is this?
/kind cleanup
What this PR does
Pins most docker base images.
Why we need it:
Pinned dependencies reduce several security risks:

They ensure that checking and deployment are all done with the same software, reducing deployment risks, simplifying debugging, and enabling reproducibility.
They can help mitigate compromised dependencies from undermining the security of the project (in the case where you've evaluated the pinned dependency, you are confident it's not compromised, and a later version is released that is compromised).
They are one way to counter dependency confusion (aka substitution) attacks, in which an application uses multiple feeds to acquire software packages (a "hybrid configuration"), and attackers fool the user into using a malicious package via a feed that was not expected for that package.

More information refer to ossf
Which issue(s) this PR fixes:
Partially fixes https://github.com/kubernetes-sigs/security-profiles-operator/issues/653
Does this PR have test?
N/A
Special notes for your reviewer:
Does this PR introduce a user-facing change?
NONE

Codecov Report

Merging #741 (ddf0d52) into main (c2c7fcf) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head ddf0d52 differs from pull request most recent head e713026. Consider uploading reports for the commit e713026 to get more accurate results

@@           Coverage Diff           @@
##             main     #741   +/-   ##
=======================================
  Coverage   53.94%   53.94%
=======================================
  Files          41       41
  Lines        4121     4121
=======================================
  Hits         2223     2223
  Misses       1831     1831
  Partials       67       67

/test pull-security-profiles-operator-test-e2e

fedora-e2e flakeness:
--- FAIL: TestSuite/TestSecurityProfilesOperator/cluster-wide:_Seccomp:_Verify_profile_recording_bpf (233.12s)


OPCT-191: Removing OCP CI image dependency on build layer

Removing dependency on OCP CI registry images (registry.ci.openshift.org) when building the project introduced on https://github.com/redhat-openshift-ecosystem/provider-certification-tool/pull/51 . It would be nice to unblock external contributors to use the tool when need to build it locally.
We don't expect to fall into Docker limitations as that image should be used in a developer environment.
https://issues.redhat.com/browse/OPCT-191
 $ make linux-amd64-container
podman build -t quay.io/ocp-cert/opct:latest -f hack/Containerfile --build-arg=RELEASE_TAG=0.0.0 .
[1/2] STEP 1/5: FROM registry.ci.openshift.org/ocp/builder:rhel-8-golang-1.19-openshift-4.13 AS builder
Trying to pull registry.ci.openshift.org/ocp/builder:rhel-8-golang-1.19-openshift-4.13...
Error: error creating build container: initializing source docker://registry.ci.openshift.org/ocp/builder:rhel-8-golang-1.19-openshift-4.13: reading manifest rhel-8-golang-1.19-openshift-4.13 in registry.ci.openshift.org/ocp/builder: unauthorized: authentication required
make: *** [Makefile:49: linux-amd64-container] Error 125

Richard, re-thinking to avoid external dependency when building the project on dev env. LMK WDYT.
/assign @rvanderp3

Richard, re-thinking to avoid external dependency when building the project on dev env. LMK WDYT. /assign @rvanderp3

makes sense to me

/lgtm
/approve


subs() comparing key hashes

This PR makes subs() compare hashes of keys instead of comparing type and equality of tuple items.
This follows the graph specification: "A key is any hashable value that is not a task", which implies that different typed keys that hashes to the same value are the same key?.
This is motivated by my work on culling where keys containing int and numpy.int are substituted.
assert hash(('mykey', 42)) == hash(('mykey', np.int64(42)))

Finally, I expect this to be slightly faster than the current implementation.

[x] Tests added / passed
[x] Passes black dask / flake8 dask

Thanks @madsbk
cc'ing @eriknw who might have thoughts on this

I wouldn't rely only on hashes, which seems far too risky.  But you're also right that relying on types may be overly restrictive.
How about len({x, y}) == 1?  This checks both hash and equality (as one would expect), and should be pretty fast.

Aha, this is even better:
x in {y}

This follows the graph specification: "A key is any hashable value that is not a task", which implies that different typed keys that hashes to the same value are the same key?.

To clarify, "hashable" does not imply only using the hash value.  Typically, it also implies using equality as well, hence why I don't think we should only use hash values.
In general, I think it's probably okay to relax the type check.  This definitely needs to go in the release notes, because I wouldn't be surprised if this change breaks some code somewhere (unlikely, but possible).

To clarify, "hashable" does not imply only using the hash value. Typically, it also implies using equality as well, hence why I don't think we should only use hash values.

Make sense and I really like x in {y} !

LGTM.  I like it!  Thanks @madsbk

@mrocklin, I think this is ready to be merged.
I don't know what is going on with CI. It complains about black reformat of files I haven't touched. On my machine, both black and flake8 passes.

Black recently updated.  I think that Tom resolved this on the Dask side with #6568 .  Pushing an empty commit to trigger CI


Features: Allow dev to limit some tests to only be tested in certain languages

[Not sure exactly how we'd pair this with the EXTRA_LANGUAGES env/config variable functionality. Would they end up conflicting? I think the individual test parameters would take over, but the same language test would then run multiple times.
e.g, EXTRA_LANGUAGES=Español,官话 and Given I start the interview... in Español would end up creating two tests because that's how EXTRA_LANGUAGES currently works. In the step, though, they'd both be switched to Español. That would mean Español would get run twice, which could be confusing for the developer, especially if the test got an error. Partly because the tags would show the EXTRA_LANGUAGES value.
]

Closed by #735


Hide User Preferences Options to store at the iTwin/iModel scope.

The following options are now hidden by default, and the layer sources are always on the iTwin scope:

A new showUserPreferencesStorageOptions option has been added to restored if neeed.

id want someone from a product chime in on the terminology here. "settings" is kinda misleading imo
@jason-crow @jjbeckman13

Yea I agree, it's been like that since the beginning.  Anyway everyone is confused about those options, hence the reason to hide it.  The whole concept of user preferences is kind of distorted  TBH.

@grigasp Looks like it's breaking e2e tests, can you help with that?

https://github.com/iTwin/viewer-components-react/pull/779 should fix this.


Clean up text rendering

characters are stored in individual textures and rendered one at a time
yuck!
https://github.com/Scaless/HaloTAS/blob/d2588b64f2988f4c872ced79f4a298075aafcc56/HaloTAS/TASDLL/render_text.cpp#L83

GL overlay rendering hash a high likelyhood of being removed eventually in favor of D3D9 rendering in-engine.

TASDLL is being frozen and effort has moved to MCCTAS.


Loading gem fails on missing VirtualAssembly::Semantizer

I added the gem and ran bundle. Then running the Rails console fails:
$ bundle exec rails console
/home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require': cannot load such file -- virtual_assembly/semantizer (LoadError)
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:17:in `require'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/activesupport-6.1.7/lib/active_support/dependencies.rb:332:in `block in require'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/activesupport-6.1.7/lib/active_support/dependencies.rb:299:in `load_dependency'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/activesupport-6.1.7/lib/active_support/dependencies.rb:332:in `require'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/datafoodconsortium-connector-1.0.0.pre.alpha/lib/datafoodconsortium/connector/json_ld_serializer.rb:23:in `<main>'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
...

The same happens if I use plain Ruby:
$ ruby -e "require 'datafoodconsortium/connector'"
<internal:/home/maikel/.rbenv/versions/3.0.3/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- virtual_assembly/semantizer (LoadError)
	from <internal:/home/maikel/.rbenv/versions/3.0.3/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /home/maikel/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/datafoodconsortium-connector-1.0.0.pre.alpha/lib/datafoodconsortium/connector/json_ld_serializer.rb:23:in `<top (required)>'
	from <internal:/home/maikel/.rbenv/versions/3.0.3/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

@mkllnk Did you install virtual_assembly-semantizer? Bundle should have installed it for you but I may have miscounfigured the gemfile.
See https://rubygems.org/gems/virtual_assembly-semantizer.

bundle update datafoodconsortium-connector solved this for me. :+1:


Computed values are sometimes stored as wrong type

Apologies if this is a feature rather than a bug - I am new to Python.
When computing a new value using a lambda that returns a float, the result is stored as a float type rather than a Decimal. This causes problems, such as when I try to use print_table.
Here's a working example:
import agate

# create data
fruit_columns = ['shop','apples','oranges']
fruit_column_types = [agate.Text(), agate.Number(), agate.Number()]
fruits = [
    ['asda',100,250],
    ['sainsburys',200,500]
]

# create table
fruit_table = agate.Table( fruits, fruit_columns, fruit_column_types )

# calculate ratio
ratio_table = fruit_table.compute([
    ('ratio', agate.Formula(
        agate.Number(),
        lambda r: (r['apples']/r['oranges'])
    ))
])

# calculate ratio, but wrap in float()
ratio_table_float = fruit_table.compute([
    ('ratio', agate.Formula(
        agate.Number(),
        lambda r: (float(r['apples']/r['oranges']))
    ))
])

# print contents of ratio columns
print ratio_table.columns[3]
print ratio_table_float.columns[3]

# print types of ratio columns' first rows
print type( ratio_table.columns[3][0] )
print type( ratio_table_float.columns[3][0] )

The output is:
<agate.Column: (Decimal('0.4'), Decimal('0.4'))>
<agate.Column: (0.4, 0.4)>
<class 'decimal.Decimal'>
<type 'float'>

Hi Elliot. The fix for your code is actually pretty easy: you can modify your Formula methods to return decimal.Decimal instead of float. However, the conversion from float should be happening automatically, so there is a bug here. Will take a closer look and see why that isn't happening. Thanks for reporting it!


Removed obsolete 'cql' package

This was an old (2012) python driver for CQL which has been replaced by the DataStax cassandra-driver.
See https://pypi.org/project/cql and https://code.google.com/archive/a/apache-extras.org/p/cassandra-dbapi2
#4

Related: https://issues.apache.org/jira/browse/CASSANDRA-17247


Changing AuthorSequence from uint to int

Docs Build status updates of commit 6622b85:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

academic-services/graph/tutorial-azure-search-org-patents.md
:white_check_mark:Succeeded

academic-services/graph/tutorial-azure-search-reference-parsing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit b7ac71b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

academic-services/graph/tutorial-azure-search-org-patents.md
:white_check_mark:Succeeded

academic-services/graph/tutorial-azure-search-reference-parsing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

The script still does not work. Job herehttps://isrcvradla.azuredatalakeanalytics.net/jobLink/ce17b35a-ccf9-4225-89b2-cc8f650233a9.
Component
CSC
Message
Unsupported MAP Value type: 'int'.
Resolution
Ensure MAP Values are nullable built-in types; this excludes UDTs.
Description
MAP Values must be nullable built-in types.
Details
at token 'MAP_AGG', line 100
near the ###:

INNER JOIN
@authors AS A
ON P.AuthorId == A.AuthorId;
@paperAuthorsAggregated =
SELECT PaperId,
"[" + string.Join(",",  ### MAP_AGG(""" + AuthorName + """, AuthorSequenceNumber).OrderBy(a => a.Value).Select(a => a.Key)) + "]" AS Authors
FROM @paperAuthors
GROUP BY PaperId;
Job URL
https://isrcvradla.azuredatalakeanalytics.net/jobLink/ce17b35a-ccf9-4225-89b2-cc8f650233a9
Thanks,
Charles
From: opbld33<EMAIL_ADDRESS>Sent: Monday, April 8, 2019 4:10 PM
To: MicrosoftDocs/microsoft-academic-services<EMAIL_ADDRESS>Cc: Charles Huang<EMAIL_ADDRESS>Review requested<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/microsoft-academic-services] Changing AuthorSequence from uint to int (#51)
Docs Build status updates of commit b7ac71bhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fmicrosoft-academic-services%2Fpull%2F51%2Fcommits%2Fb7ac71b8b64eff603de2954ce31f1e66082bb6eb&data=02|01|Chiyuan.Huang%40microsoft.com|4829008846484fbb71aa08d6bc7741c6|72f988bf86f141af91ab2d7cd011db47|1|0|636903617721486171&sdata=wDzx9OPmxTqj3AWpCggfSi5wyS86zKhhzhdSOcS%2B%2FhY%3D&reserved=0:
✅ Validation status: passed
File
Status
Preview URL
Details
academic-services/graph/tutorial-azure-search-org-patents.mdhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fmicrosoft-academic-services%2Fblob%2Fmaster%2Facademic-services%2Fgraph%2Ftutorial-azure-search-org-patents.md&data=02|01|Chiyuan.Huang%40microsoft.com|4829008846484fbb71aa08d6bc7741c6|72f988bf86f141af91ab2d7cd011db47|1|0|636903617721496171&sdata=DJAuUaAqSIr2k80aUmt%2BP2hi%2B1mjS1l42YxkmUFUL5k%3D&reserved=0
✅Succeeded
academic-services/graph/tutorial-azure-search-reference-parsing.mdhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fmicrosoft-academic-services%2Fblob%2Fmaster%2Facademic-services%2Fgraph%2Ftutorial-azure-search-reference-parsing.md&data=02|01|Chiyuan.Huang%40microsoft.com|4829008846484fbb71aa08d6bc7741c6|72f988bf86f141af91ab2d7cd011db47|1|0|636903617721496171&sdata=C8aC8VzNAUjk6yQkH340TlKhv7pGhpNs%2BjZ6P6HtQhA%3D&reserved=0
✅Succeeded
For more details, please refer to the build reporthttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopbuildstorageprod.blob.core.windows.net%2Freport%2F2019%255C4%255C8%255C15ce2f94-e18c-759b-ebf4-39c1ae7c3a9e%255CPullRequest%255C201904082308256084-51%255Cworkflow_report.html%3Fsv%3D2015-02-21%26sr%3Db%26sig%3Dx7T5oe3uDIJCH5sMSfxoRGYU2xODCm7%252BAG4n4BErENo%253D%26st%3D2019-04-08T23%253A04%253A29Z%26se%3D2019-05-09T23%253A09%253A29Z%26sp%3Dr&data=02|01|Chiyuan.Huang%40microsoft.com|4829008846484fbb71aa08d6bc7741c6|72f988bf86f141af91ab2d7cd011db47|1|0|636903617721506160&sdata=YTy2vtEb4HR5ofgy2pUzTrpp6HbZE1Ub%2BmmQi25kxSI%3D&reserved=0.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fmicrosoft-academic-services%2Fpull%2F51%23issuecomment-481040302&data=02|01|Chiyuan.Huang%40microsoft.com|4829008846484fbb71aa08d6bc7741c6|72f988bf86f141af91ab2d7cd011db47|1|0|636903617721506160&sdata=DzwpF1T6xu8CfSWYwjahoR3np8t9f%2FoXu0SVJkn%2FgDk%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAqcu2pKCJ5nciHjUu02l3MvkxA3weYIaks5ve8wqgaJpZM4cjROh&data=02|01|Chiyuan.Huang%40microsoft.com|4829008846484fbb71aa08d6bc7741c6|72f988bf86f141af91ab2d7cd011db47|1|0|636903617721516154&sdata=HqFgLezl58%2BeNmVt1nc3sR7ojtiHUqThLF%2FTqI9Ldb8%3D&reserved=0.

Docs Build status updates of commit 58bc07a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

academic-services/graph/tutorial-azure-search-org-patents.md
:white_check_mark:Succeeded

academic-services/graph/tutorial-azure-search-reference-parsing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


kundera with jboss

Hi , i have seen that there is some issues  with running kundera on wildfly(jboss).
is there any way for wildfly to manage the kundera entity manager ?
meaning without: <property name="jboss.as.jpa.managed" value="false" /> so i could inject entity manager factory
also do we have any known issue with running kundera on cassandra and eclipselink(postgres) side by side each as a persistence unit in the same xml ?
Thanks.

Hi @asafbennatan
Question 1
Wildfly doesn't provide container managed entity support for Kundera. You need to manage the persistence context at the application level.
@PersistenceUnit
EntityManagerFactory emf;
EntityManager em = emf.createEntityManager();

Check Kundera with Jboss for more details.
Question 2
You can use any number of persistence-units (from different data sources) in single persistence.xml file.
-Dev


No hs.app in the sd-root after dumping

Everything went very well unti "Section V - Injecting FBI" in step 2:
"Go to "SysNAND Backup/Restore", then select the "Health&Safety Dump" option to dump Health & Safety to hs.app (you can use Up and Down / Left and Right to change the name)"
The process succeeded without any problems but after all Im not able to find the "hs.app" in the root of the sd. I also checked the other folders - no dump.
Thank you for you help!

already solved! found it in the "files9" folder

If it's solved just close the issue already.

Does it work for new 3ds 11.0.0-33E ?

@GuiguiPH what do you mean does it work? If you're asking for A9LH yes, you can on 11.0.0-33.


Add new makefile targets for go mod verification

Which issue this PR addresses:

None. It is a small enhancement.
What this PR does / why we need it:
Run 'go mod' functions to search for tidy, and vendor package changes and then verify it.

Test plan for issue:

Is there any documentation that needs to be updated for this PR?
Yes, in the second commit
How do you know this will function as expected in production?

@microsoft-github-policy-service agree company="Red Hat"

FYI you'll need to move this to a branch off the main repo. Development from forks is broken for the foreseeable

FYI you'll need to move this to a branch off the main repo. Development from forks is broken for the foreseeable

Gotcha, then I will close this PR as duplicate of #3550


Have plans to add a search box? #feature request

“Search box” make it better to use.

Not yet, have a try: https://gohugo.io/tools/search/

Here is a pre-made version with fuse.js:
#116

Here is a pre-made version with fuse.js:

https://notepadium-search.gitlab.io
Source Code: https://github.com/rojenzaman/notepadium-with-search

#116


wonky double gamma HRF

Hi Michael,
I was playing around the with DesignMatrix class and I am a little confused by the HRFs. They don't come out looking quite like the canonical HRF. It looks like perhaps the code is not adjusting for different TRs, although I am having trouble tracking down the source of the issue:
from moss import glm
import pandas as pd
import seaborn as sns
%matplotlib inline
design = pd.DataFrame({'condition':['test'],
'onset': [0],
'duration': [2],
})
#convolve
hrf = glm.GammaDifferenceHRF()
model = glm.DesignMatrix(design = design, tr = 1.0, ntp = 14,
hrf_model = hrf, hpf_cutoff = 128)
sns.tsplot(model.design_matrix.values.flatten())

This is an artifact of the highpass filter.

Did this solve the problem?


Bitmap slice doesn't match debug representation

When obtaining the bitmap as a slice I think there may be a bug in the handling. For example, in this program I've written (below), when I run it, the first assertion passes, but the second fails giving
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `([173], 0, 7)`,
 right: `([45], 0, 7)`', src/main.rs:20:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I think that here the bit in position 0 (or 7 in little-endian) has been erroneously set (it is the only bit which is different in 173 base 2 than in 45 base 2. Especially as the start offset is set as zero.
use arrow2::array::{ListArray, MutableListArray, MutablePrimitiveArray, TryExtend};

fn main() {
    let data = vec![
        Some(vec![Some(1), None, Some(2), Some(4), Some(1), None]),
        None,
        Some(vec![Some(1), None, Some(8), Some(4), Some(1), None]),
        Some(vec![Some(1), None, Some(2), Some(4), Some(1), None]),
        None,
        Some(vec![Some(1), None, Some(2), Some(4), Some(1), None]),
        None,
    ];

    let mut mutable_list = MutableListArray::<i32, MutablePrimitiveArray<i32>>::new();
    mutable_list.try_extend(data.clone()).unwrap();
    let list: ListArray<i32> = mutable_list.into();
    let list = list.arced();
    let bitmap = list.validity().unwrap();
    assert_eq!(format!("{:?}", bitmap), "[0b_0101101]");
    assert_eq!(bitmap.as_slice(), (vec![0b0101101 as u8].as_slice(), 0, 7))
}

The issue here is that 0b_0101101 represents: the bitmap with 7 bits (the last one is undetermined). I.e. the _ stands for undetermined. In this case, it is actually a 1:
assert_eq!(bitmap.as_slice(), (vec![0b10101101 as u8].as_slice(), 0, 7))

The technical reason it is a 1 and not a zero is that up to the point where we have not observed a null value in the iterator, the validity is None. Once we observe the first null, we need to initialize a new validity with 1s up to the current length, something like
let mut validity = vec![1; (self.len() + 7) / 8];
unset_bit(&mut validity, self.len());
self.validity = Some(validity)

@teymour-aldridge , would it be ok to close this issue?


Firefox 57 Quantum

There are "Warnings" in the web console by jquery3x and Gijgo.

I'm browsing through some gijgo examples for the tree, grid and datepicker and can't see these warnings. Could you give me more details how can I find them.

I will send you you the messages if I see the warnings again. I have to try out your last development version yet.

ok. I will close the ticket for now. Could you reopen it if you see the message again.


[EasyCodingStandard] Include section doesn't work

It seems that include doesn't work
include:
	- missing.xxx  # no error

Maybe try includes, it uses Neon Loader integration to Symfony and I there is different keyword.
If that fails, I'll look on it too.

This is not working either.
Btw, it should display error when unexpected key is used in config.file.

Ok, let's improve that. Could you add test with ContainerFactory loading those 2 configs?

[ContainerFatctory->createWithConfig(...)](https://github.com/Symplify/Symplify/blob/master/packages/EasyCodingStandard/src/DependencyInjection/ContainerFactory.php#L17
into packages/EasyCodingStandard/tests/DependencyInjection/ContainerFactoryTest.php

And expect some exception.
I will handle the rest.

https://github.com/dg/Symplify/commit/a2401fb0f13b62cc46b1c0077c283404e725ff50
For me it's really hard to do it, I do not know Symfony.

Thank you :+1:

For me it's really hard to do it, I do not know Symfony.

No problem. Could you send that as PR? I will finish that.

@todo for me: Move it under PackageBuilder and test generated parameters.

YAML/NEON includings resolved by #211


From ACR Arch Board: Consider making ContentPermissions a Flags Enum

See APIView comment.

ContentPermissions was renamed ContentProperties (to avoid confusion with things like RBAC permissions), and additional properties may be added to it.