"write failed: connection reset by peer" with parallel tcp connections

What you expected to happen?
Running iperf over weave network with 24 parallel streams is possible, regardless of effect this will have on the performance.
What happened?

Running iperf with anything over 8 streams results in write failed: Connection reset by peer errors.
Connect two weaves together. Run iperf client on one, run iperf server on the other.
Increase parallel streams until "connection reset by peer" starts to occur.
The number of parallel streams is considered low.
Anything else we need to know?
Observed same behavior on following configurations

two hosts behind different NATs using sleeve
two hosts on same network using fastdp

Everything else connectivity wise works well. Only under load from a lot of parallel communication problems start to occur.
Versions:

$ weave version
weave script 2.0.4
weave 2.0.4
$ docker version
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:06:25 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:06:25 2017
 OS/Arch:      linux/amd64
 Experimental: false
$ uname -a
Linux remotemanager 4.12.7-1.el7.elrepo.x86_64 #1 SMP Mon Aug 14 10:03:52 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

Logs:
On the iperf client
$ docker logs weave
<!-- iperf client command is run here -->

INFO: 2017/09/18 12:52:15.405494 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] sleeve write ip4 <IP_ADDRESS>-><IP_ADDRESS>: write: no buffer space available
INFO: 2017/09/18 12:52:15.405543 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection shutting down due to error: no working forwarders to 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.405649 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection deleted
INFO: 2017/09/18 12:52:15.406459 ->[<IP_ADDRESS>:36783] attempting connection
INFO: 2017/09/18 12:52:15.406659 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.406817 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.406958 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.407027 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.441833 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.441933 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.442017 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.457147 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection ready; using protocol version 2
INFO: 2017/09/18 12:52:15.457246 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] using fastdp
INFO: 2017/09/18 12:52:15.457313 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection added (new peer)
INFO: 2017/09/18 12:52:15.464818 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.465906 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] using sleeve
INFO: 2017/09/18 12:52:15.465942 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection fully established
INFO: 2017/09/18 12:52:15.466242 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2017/09/18 12:52:15.469818 Received packet for unknown destination: 62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)
INFO: 2017/09/18 12:52:15.478847 sleeve ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: Effective MTU verified at 1438
INFO: 2017/09/18 12:52:15.716295 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] sleeve write ip4 <IP_ADDRESS>-><IP_ADDRESS>: write: no buffer space available
INFO: 2017/09/18 12:52:15.716323 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] using fastdp
INFO: 2017/09/18 12:53:15.467358 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:53125-><IP_ADDRESS>:36783: read: connection reset by peer
INFO: 2017/09/18 12:53:15.467437 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection deleted
INFO: 2017/09/18 12:53:15.468231 ->[<IP_ADDRESS>:36783] attempting connection
INFO: 2017/09/18 12:53:15.490359 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection ready; using protocol version 2
INFO: 2017/09/18 12:53:15.490451 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] using fastdp
INFO: 2017/09/18 12:53:15.490489 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection added (new peer)
INFO: 2017/09/18 12:53:15.498736 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] using sleeve
INFO: 2017/09/18 12:53:15.498773 ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: connection fully established
INFO: 2017/09/18 12:53:15.498854 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2017/09/18 12:53:15.511520 sleeve ->[<IP_ADDRESS>:36783|62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)]: Effective MTU verified at 1438
INFO: 2017/09/18 12:55:05.502581 overlay_switch ->[62:c6:5e:35:12:3a(bellatrix.qgg.hud.ac.uk)] fastdp timed out waiting for vxlan heartbeat

On the iperf server
INFO: 2017/09/18 12:50:45.512450 ->[<IP_ADDRESS>:30847|ae:d6:c0:4a:9b:4e(remotemanager)]: connection shutting down due to error: read tcp4 <IP_ADDRESS>:36783-><IP_ADDRESS>:30847: read: connection reset by peer
INFO: 2017/09/18 12:50:45.512592 ->[<IP_ADDRESS>:30847|ae:d6:c0:4a:9b:4e(remotemanager)]: connection deleted
INFO: 2017/09/18 12:50:45.512842 Unable to find connection to relay peer ae:d6:c0:4a:9b:4e
INFO: 2017/09/18 12:50:45.519353 ->[<IP_ADDRESS>:53125] connection accepted
INFO: 2017/09/18 12:50:45.523455 Received packet for unknown destination: ae:d6:c0:4a:9b:4e(remotemanager)
INFO: 2017/09/18 12:50:45.529995 ->[<IP_ADDRESS>:53125|ae:d6:c0:4a:9b:4e(remotemanager)]: connection ready; using protocol version 2
INFO: 2017/09/18 12:50:45.530110 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] using fastdp
INFO: 2017/09/18 12:50:45.530163 ->[<IP_ADDRESS>:53125|ae:d6:c0:4a:9b:4e(remotemanager)]: connection added (new peer)
INFO: 2017/09/18 12:50:45.531478 Received packet for unknown destination: ae:d6:c0:4a:9b:4e(remotemanager)
INFO: 2017/09/18 12:50:45.544219 ->[<IP_ADDRESS>:53125|ae:d6:c0:4a:9b:4e(remotemanager)]: connection fully established
INFO: 2017/09/18 12:50:45.546489 Received packet for unknown destination: ae:d6:c0:4a:9b:4e(remotemanager)
INFO: 2017/09/18 12:50:45.546642 Received packet for unknown destination: ae:d6:c0:4a:9b:4e(remotemanager)
INFO: 2017/09/18 12:50:45.550613 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2017/09/18 12:50:45.558396 sleeve ->[<IP_ADDRESS>:36783|ae:d6:c0:4a:9b:4e(remotemanager)]: Effective MTU verified at 1438
INFO: 2017/09/18 12:51:45.530802 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] fastdp timed out waiting for vxlan heartbeat
INFO: 2017/09/18 12:51:45.530863 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] using sleeve
INFO: 2017/09/18 12:51:45.543546 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] sleeve timed out waiting for UDP heartbeat
INFO: 2017/09/18 12:51:45.543610 ->[<IP_ADDRESS>:53125|ae:d6:c0:4a:9b:4e(remotemanager)]: connection shutting down due to error: no working forwarders to ae:d6:c0:4a:9b:4e(remotemanager)
INFO: 2017/09/18 12:51:45.543652 ->[<IP_ADDRESS>:53125|ae:d6:c0:4a:9b:4e(remotemanager)]: connection deleted
INFO: 2017/09/18 12:51:45.559285 ->[<IP_ADDRESS>:51105] connection accepted
INFO: 2017/09/18 12:51:45.566736 ->[<IP_ADDRESS>:51105|ae:d6:c0:4a:9b:4e(remotemanager)]: connection ready; using protocol version 2
INFO: 2017/09/18 12:51:45.566839 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] using fastdp
INFO: 2017/09/18 12:51:45.566892 ->[<IP_ADDRESS>:51105|ae:d6:c0:4a:9b:4e(remotemanager)]: connection added (new peer)
INFO: 2017/09/18 12:51:45.575862 ->[<IP_ADDRESS>:51105|ae:d6:c0:4a:9b:4e(remotemanager)]: connection fully established
INFO: 2017/09/18 12:51:45.588599 EMSGSIZE on send, expecting PMTU update (IP packet was 60028 bytes, payload was 60020 bytes)
INFO: 2017/09/18 12:51:45.596499 sleeve ->[<IP_ADDRESS>:36783|ae:d6:c0:4a:9b:4e(remotemanager)]: Effective MTU verified at 1438
INFO: 2017/09/18 12:52:45.567407 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] fastdp timed out waiting for vxlan heartbeat
INFO: 2017/09/18 12:52:45.567465 overlay_switch ->[ae:d6:c0:4a:9b:4e(remotemanager)] using sleeve

I think the pertinent log message here is
sleeve write ip4 <IP_ADDRESS>-><IP_ADDRESS>: write: no buffer space available

but it looks like the weave container already raises the kernel params for udp connection limits.
I used higher spec hardware, and the problem goes away. Iperf running no problem over weave with 128 parallel streams. This is not a weave bug, but might be nice to document somewhere that this message probably indicates you need a faster machine...

Solution: https://spyff.github.io/net/2019/07/14/many-tcp-flow-with-iperf/#the-problem-of-small-backlog


Cannot create nfs persistent volume

Rancher versions:
rancher/server or rancher/rancher: v2.0.0
Docker version: (docker version,docker info preferred)
17.03.2-ce
Operating system and kernel: (cat /etc/os-release, uname -r preferred)
Ubuntu 16.04.4 LTS
Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
AWS
Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
single node ranche
Environment Template: (Cattle/Kubernetes/Swarm/Mesos)
Kubernetes
Steps to Reproduce:

Launch the nfs-provisioner from catalog apps using the default settings
Add a volume from workflows-volumes, select nfs-provisioner for Storage Class
Results:
The volume status has always been "Pending"

root@ip-172-31-43-111:~# kubectl get pvc
NAME                   STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
test                   Pending                                                                        nfs-provisioner   36s

root@ip-172-31-43-111:~# kubectl describe pvc test
Name:          test
Namespace:     default
StorageClass:  nfs-provisioner
Status:        Pending
Volume:
Labels:        <none>
Annotations:   field.cattle.io/creatorId=user-qdr85
               volume.beta.kubernetes.io/storage-provisioner=rancher.io/nfs-provisioner
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
Events:
  Type    Reason                Age               From                         Message
  ----    ------                ----              ----                         -------
  Normal  ExternalProvisioning  4s (x11 over 2m)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "rancher.io/nfs-provisioner" or manually created by system administrator

Same issue.
But deploy nfs manually is fine, just practiced.

Tested with rancher-server version- v2.0.0 with nfs-provisioner version-0.1.0
Test passed.


Is there Milestone to release <IP_ADDRESS>?

Hi, I've complete test with <IP_ADDRESS> for my environment and find xWebSite and other have to be update to unreleased version.
Is there any plan date to release next xWebAdministration module?

@kwirkykat ?

Already released and close.

@guitarrapc Thanks


No pip install

I notice install via pip was theoretically on the horizon for this package back when it was created. Any chance of an update to support it?

Hi Dave,
Unfortunately, our team does not currently have the bandwidth to support this update. However, if you're interested, I am happy to share repository privileges, credits, etc and/or make whatever updates are necessary if you'd like to take it on. Please let me know if you're interested.

You might both like to join a Slack channel I just made https://sharpspringcommunity.slack.com/archives/C01J8AADV5X

You might both like to join a Slack channel I just made https://sharpspringcommunity.slack.com/archives/C01J8AADV5X


support customizing Log's time zone through environment variables

Hello, thank you very much for your contribution to this project!
When I was using this project, I found that I specified the time zone through the TZ environment variable, but the output log was still in the UTC time zone!

Thanks, this is a good reminder that we need to start adding more complex time support to the whole stack, and also begin to deprecate utcnow() as it's beginning to be phased out with 3.12.


Do we have to change the DB schema / tables of the blockchain LMDB file for Seraphis?

Following my proposal of preferring single-subject issues, I allow myself to turn the following question no 5 from @dangerousfreedom1984 in his issue #10 into a separate issue:

5. I think it is also important to work in parallel in implementing the necessary structures to store the data in an actual blockchain style so we can have a testnet as soon as possible. If we agree with that structure shown in figures 1 and 2, we have to basically modify the file so the DB schema would account for that, right?

As far as I could see going over the existing DB code it does not care about the actual structure of transactions and other things, they all seem to be stored simply as "blobs", as series of bytes. Seems to me that if we manage to serialize Seraphis transactions, we can very well store them into the same tables that the blockchain DB already has, e.g. txs_pruned or txs_prunable.
In my issue #7 I sketch a possible way how to define something like a "meta transaction class" that can hold legacy as well as Seraphis transactions that could be objects that we store in LMDB databases like the blockchain file.
But of course many detail questions remain. Although @UkoeHB says in #9 that we should continue with pruning and prune Seraphis transactions as well, maybe it's not clear the right way to do it is to continue to use two separate tables txs_pruned plus txs_prunable as mentioned above.
And maybe there will be completely new things to store in the blockchain in connection with Seraphis transactions, that will need new tables, or at least new ways to lookup Seraphis transactions that did not yet play a role until now.
My current take: If we want a permanent blockchain to back something like a proof-of-concept / prototype Seraphis wallet to all learn more and get acquainted with UkoeHB's library we just freely define a new schema, just for that purpose. After that, many such detail questions about the "real" blockchain DB hopefully will have cleared up, and we can attack that more complex problem.


bug: the secret controller cannot handle secret update events in sometime

Issue description
In our environment, there is a bug that the certificate in secret is updated, but the ingress controller does not update the new certificate to APISIX. The reason for this is not known yet.
Since the environment where the bug occurred is no longer present, we cannot provide more information for now. If we reproduce it later, we will provide further information.
Environment

your apisix-ingress-controller version (output of apisix-ingress-controller version --long):
your Kubernetes cluster version (output of kubectl version):
if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a):

Minimal test code / Steps to reproduce
/
Actual result
/
Error log
/
Expected result
No response

Thanks! What's your APISIX Ingress version? And did you use the ApisixTls resource?

@nic-6443 .

What's your APISIX Ingress version?

It's a bit complicated, because the version we use is a branch we maintain for special reasons, using code from: https://github.com/nic-6443/apisix-ingress-controller/tree/patch-1 to support upstream mtls, but the code has been merged to upstream.

And did you use the ApisixTls resource?

Yes

If you're not using this repository but a fork version, I can't easily provide support.

And it's not easy to reproduce.

Yes

We may try to reproduce this problem with the standard version. @nic-6443

Okay, we will try to test with the latest version of the ingress controller and sync up if we get any new clues


Compilation error

Hello PrismTech team,
when trying to compile agentv from scratch on CentOS 6.5, I got the following errors.
Do I need to install Intellji?
Thanks
Francois
[info] Compiling 1 Scala source and 1 Java source to /export/home/flefevre/gitPeps/agentv/agentv-commander/target/scala-2.11/classes...
[warn] While parsing annotations in /export/home/flefevre/.ivy2/cache/com.prismtech.cafe/cafe/jars/cafe-2.2.1-SNAPSHOT.jar(org/omg/dds/type/Extensibility.class), could not find RUNTIME in enum object RetentionPolicy.
[warn] This is likely due to an implementation restriction: an annotation argument cannot refer to a member of the annotated class (SI-7014).
[warn] While parsing annotations in /export/home/flefevre/.ivy2/cache/com.prismtech.cafe/cafe/jars/cafe-2.2.1-SNAPSHOT.jar(org/omg/dds/type/Extensibility.class), could not find TYPE in enum object ElementType.
[warn] This is likely due to an implementation restriction: an annotation argument cannot refer to a member of the annotated class (SI-7014).
[warn] While parsing annotations in /export/home/flefevre/.ivy2/cache/com.prismtech.cafe/cafe/jars/cafe-2.2.1-SNAPSHOT.jar(org/omg/dds/core/policy/QosPolicy.class), could not find EXTENSIBLE_EXTENSIBILITY in enum object Kind.
[warn] This is likely due to an implementation restriction: an annotation argument cannot refer to a member of the annotated class (SI-7014).
[warn] three warnings found
[error] /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java:215: error: package com.intellij.uiDesigner.core does not exist
[error]         contentPane.setLayout(new com.intellij.uiDesigner.core.GridLayoutManager(3, 2, new Insets(10, 10, 10, 10), -1, -1));
[error]                                                               ^
[error] /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java:219: error: package com.intellij.uiDesigner.core does not exist
[error]         runningAppsPanel.setLayout(new com.intellij.uiDesigner.core.GridLayoutManager(3, 1, new Insets(0, 0, 0, 0), -1, -1));
[error]                                                                    ^
[error] /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java:220: error: package com.intellij.uiDesigner.core does not exist
[error]         contentPane.add(runningAppsPanel, new com.intellij.uiDesigner.core.GridConstraints(0, 1, 1, 1, com.intellij.uiDesigner.core.GridConstraints.ANCHOR_CENTER, com.intellij.uiDesigner.core.GridConstraints.FILL_BOTH, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, null, null, null, 0, false));
[error]                                                                           ^
[error] /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java:220: error: package com.intellij.uiDesigner.core does not exist
[error]         contentPane.add(runningAppsPanel, new com.intellij.uiDesigner.core.GridConstraints(0, 1, 1, 1, com.intellij.uiDesigner.core.GridConstraints.ANCHOR_CENTER, com.intellij.uiDesigner.core.GridConstraints.FILL_BOTH, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, null, null, null, 0, false));
[error]                                                                                ^
[error] /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java:220: error: package com.intellij.uiDesigner.core does not exist
[error]         contentPane.add(runningAppsPanel, new com.intellij.uiDesigner.core.GridConstraints(0, 1, 1, 1, com.intellij.uiDesigner.core.GridConstraints.ANCHOR_CENTER, com.intellij.uiDesigner.core.GridConstraints.FILL_BOTH, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, null, null, null, 0, false));
[error]

[error] /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java:293: error: package com.intellij.uiDesigner.core does not exist
[error]         panel4.add(deployButton, new com.intellij.uiDesigner.core.GridConstraints(1, 0, 1, 2, com.intellij.uiDesigner.core.GridConstraints.ANCHOR_CENTER, com.intellij.uiDesigner.core.GridConstraints.FILL_HORIZONTAL, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_SHRINK | com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_CAN_GROW, com.intellij.uiDesigner.core.GridConstraints.SIZEPOLICY_FIXED, null, null, null, 0, false));
[error]                                                                                ^
[error] Note: /export/home/flefevre/gitPeps/agentv/agentv-commander/src/main/scala/com/prismtech/agentv/commander/CommanderWindow.java uses unchecked or unsafe operations.
[error] Note: Recompile with -Xlint:unchecked for details.
[error] 100 errors
[error] (compile:compile) javac returned nonzero exit code
[error] Total time: 25 s, completed 23 févr. 2016 15:11:04
[info] Loading project definition from /export/home/flefevre/gitPeps/agentv/agentv-microsvc-daytime/project

@flefevre, yes you need to install IntelliJ as the commander is currently using the Intellij layout manager.


Would term-stats be a good candidate to appear in tiny-care-terminal?

Do you think we can add term-stats to tiny-care-terminal? (Okay maybe I am biased because I made it but..)

One thing I think which could be a problem is the fact that npm download stats refresh after twenty-four hours, so there is no point in refreshing them. They'll remain the same for the rest of the day, anyway.
Another thing is that it could be a serious demotivator if you think about human psychology.
I wanna know if you'd add it, or whether it would be worth adding.
PS: The list of packages is configurable, of course.

Another thing is that it could be a serious demotivator if you think about human psychology.

Yeah, personally it doesn't seem to me like this would be a good fit.  That would personally contribute to my impostor syndrome.

Technically, the motivation for the tool was to keep an eye on how packages you have authored are performing.
But I get it. Thanks for the response. :)


Run tests with ipfs-api again (WIP)

This is a WIP to make OrbitDB run tests with go-ipfs again (through js-ipfs-api).
This PR will:
Use direct-channel module
Use ipfs-pubsub-room fork
Use DirectChannel
Remove obsolete/unused public API methods
Run tests against both js-ipfs and go-ipfs
Use ipfsd-ctl to create test IPFS instances
Add more test utils

👏🏽👏🏽👏🏽👏🏽👏🏽

I'm going to split this PR in two, because there's really two major things in it atm:

Use ipfsd-ctl to create test IPFS instances
Run tests against both js-ipfs and go-ipfs

The first part is to just update the tests to use ipfsd-ctl and the second part brings in everything we need to run with go-ipfs again.

Closes #273

Using ipfsd-ctl to run the tests is now a separate PR here https://github.com/orbitdb/orbit-db/pull/335

This was now finally finished and in itself became a rather small PR compared to the original. The main work was done in the previous PRs (see commit log).
Going to merge this to master and keep testing it for some time before making a new release (0.20.0).
We now support go-ipfs again! 🎉 🚀 😄


setOrigin on bitmaptext is odd/incorrect

Version

Phaser Version: 3.21.0 - 3.23.0

Operating system: Windows 10

Browser:

Description

When using setOrigin it should set the middle to be at the middle of the bitmaptext, it technically(?) does, but it doesn't seem to set it correctly.
Edit:
If you put it in a container and scale the container to (-1,1) then it correctly flips
Example Test Code

https://labs.phaser.io/view.html?src=src\game objects\bitmaptext\static\text origin.js
Versions from 3.21.0 to newest isn't working correctly, 3.20.1 works as intended
Can be tested further by trying to flip the bitmaptexts with scaling it to (-1,1)
Additional Information

Can confirm this issue. Probably this bug appeared after BitmatText wrapping feature had implemented. As I can see BitmapText.getTextBounds() method has changed, is it possible that this change could caus this odd setOrigin behaviour?

Looking at this further, the origin is actually correct. The text is appearing exactly where it should do in the example linked above, regardless of which version you pick.
The issue is that the values the getTextBounds function returns is correct in 3.20 and wrong in 3.21+. If you look at the example you'll see the text itself never changes place, it's only the bounds rectangle that does.
Will have a look to see why that is happening. If you have an example showing how the text itself is misplaced (not just the bounds) then I'd like to see it, please.

Changing the origin of a BitmapText will now flag it as being dirty, allowing things like getTextBounds to recalculate the values correctly.

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.

It still doesn't seem correct, by adding the code at https://pastebin.com/hf9f5E7u at the end of the create function in the example added at the start (https://labs.phaser.io/edit.html?src=src/game objects\bitmaptext\static\text%20origin.js) you can see that it flips over and around the origin but ending at the position if it flipped over the middle left point

This works fine in 3.50, leaving it as closed.

On Phaser version 3.25.0-beta.0 inserting this code https://pastebin.com/HMuCLHVL doesn't do what it's supposed to do according to me, it should;

create a bitmaptext object with the text "Phaser 3"
set the bitmaptext origin to the middle (0.5,0.5)
set the bitmaptext to be interactive
set so that when you click on it, it should then flip it over the middle Y

But what it actually does is;

create a bitmaptext object with the text "Phaser 3"
set the bitmaptext origin to the middle (0.5,0.5)
set the bitmaptext to be interactive
set so that when you click on it, it flips it over the middle Y (0.5,0.5) on the way to being flipped over the left (0,0)

Did you build it from the master branch? If not, it's not using a version with the fixes in (beta 0 doesn't have them)

i haven't built it (cause frankly i have no idea how) i'm using the newest Pre-release (10 days ago it says)

Try with this build.
dev.zip

Yup, that works as it should. Thanks and good job


KAFKA-14711: kafaka-metadata-quorum.sh does not honor --command-confi…

…g option
https://github.com/apache/kafka/pull/12951 accidentally changed the behavior of the kafaka-metadata-quorum.sh CLI by making it silently ignore a --command-config <filename> properties file that exists. This was an undetected regression in the 3.4.0 release.  This patch fixes the issue such that any such specified file will be used.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

There is no workaround for this issue.  A 3.4.1 release should happen ASAP.

Noting that we do in fact test the use of other security protocols with the KRaft remote controllers in tests/kafkatest/sanity_checks/test_verifiable_producer.py.  But we don't invoke the CLI in that test.  The added unit test here is sufficient to catch the issue, and the lack of a system test failure during the release does not necessarily indicate a system test gap.

test failures are unrelated.  Both failures mentioned pass locally.  Merging to trunk and 3.4


chartjs3: object notation for resolveDatasetElementOptions

the DataController.resolveDataElementOptions  supports and object notation to define the option keys. However, the  DataController.resolveDatasetElementOptions doesn't. Is there a reason why, cause I would need the latter one.
https://github.com/chartjs/Chart.js/blob/ae99039e4d686de0999631828f6385378dc3b2a3/src/core/core.datasetController.js#L839-L911

We were hoping to get rid of the object notation. Can you link the use case?

similar to the LineController use case in https://github.com/sgratzl/chartjs-chart-geo/blob/develop/src/controllers/geo.js. I have an option outlineBackgroundColor but the element needs backgroundColor


RequestMtuAsync(int requestValue) question

How do I use  a call to then method Task RequestMtuAsync(int requestValue); correct ?
We are using a bluetooth device that has a certain amount of free credits / MTUs of 20 Byte.
I we send a command using the write command the ValueUpdated event is fired but with an incorrect result.
We assume that we have to request the available MTU, but we always get back 155 calling the
Task RequestMtuAsync(int requestValue);
method on the device.
Configuration
Version of the Plugin:
2.1.1
Platform:
iPhone 8S , iOS Version 13.3.1
Steps to reproduce

Call RequestMtuAsync(int requestValue);
Expected behavior
Get the correct amount of available MTUs after sending a command
Actual behavior
Always getting 155 back
Any hint is welcome.
Best
Marco

Hi @marcoseraphin,
I'm the friendly issue checker.
It seems like (25.00 %) you haven't used our issue template :cry: I think it is very frustrating for the repository owners, if you ignore them.
If you think it's fine to make an exception, just ignore this message.
But if you think it was a mistake to delete the template, please close the issue and create a new one.
Thanks!


Removed optional bitweaving submodule from submodule file

This was an oversight from #169 Adding the bitweaving index repo to the submodule file will be confusing to users who do not have access to the bitweaving repo.

LGTM!


Fix compilation under gcc10

Fix #2097

Dang, you were faster, I just finished my fix.
I have no idea why this was put into the header in the first place. Don't you think we should rather put this into the cpp file?

For a simple function like this, I think this should be in the header to allow the compiler to optimize away the function call.
But indeed the code was incorrect, a variable shouldn't be defined into the header.
See https://gcc.gnu.org/gcc-10/porting_to.html (under C language issues)

Will create a new release shortly.
Thanks for the quick fix.


Fix slow multiprocessing

Description
This PR refer to #329

[ ] Add regression test
[ ] Replace multiprocessing by torch.multiprocessing
[ ] Add regression test Add joblib manager ?

Regarding "Replace multiprocessing by torch.multiprocessing", I wouldn't replace it, but add a "torch_multiprocessing" option.

@omardrwch okay I'll fix it


Deeper investigation into DotProduct loss

The DotProduct loss used by all the ML components can be summarized to be doing the following currently -

Obtain embedding for batch of input(message / tracker state) and labels(intent / response / action): i and l.
Randomly sample a set of negative embeddings for both input and labels - neg_i, and neg_l
Compute five terms - sim_i_pos_l, sim_i_neg_l, sim_l_neg_i, sim_i_neg_i, sim_l_neg_l.
Here sim denotes a similarity function(cosine or inner product). sim_x_y denotes similarity between embeddings x and y. pos_l is the ground truth label for the corresponding i.
Maximize sim_i_pos_l and minimize all other terms. How is this done inside _softmax_loss?

Treat all the similarity terms as logits.
Apply softmax on these logits.
Use cross entropy as the loss function with labels 1 and 0 for sim_i_pos_l and all others respectively.

What we found from experiments(these were primed by a few hypothesis) -

Softmax over logits works great to maximize the separation between [sim_i_pos_l] and [sim_i_neg_l, sim_l_neg_i](Referred to as positive and negative pairs ahead). However, it does not constrain the original similarity values in an approximate range. That's intuitive to imagine because as long as there is a good separation between positive and negative pairs of similarity, cross entropy over softmax probabilities will be low. As an effect of this, the mean value of positive and negative pairs after the model is trained is around +12 and +6 respectively(validated on a couple of datasets across different configurations). These aren't really ideal values for similarities as it can have adverse effects on some proxy tasks and real world test sets.
Another related observation was that sim_i_neg_i and sim_l_neg_l don't need to be minimized relative to other similarity terms. In fact, doing so actually pushed the similarity values of sim_pos to a higher value since the model struggles to minimize sim_i_neg_i to lower values and hence keeps trying to push sim_pos higher to maximize the separation.
In order to constrain all similarity values to an approximate range, we applied sigmoid on each of them and added the cross entropy loss over sigmoid probabilities to the final total loss. This brought back mean similarity values to a better range of +6 and -0.5 for positive and negative pairs respectively. Along with that, logits for softmax were constructed only from sim_i_pos_l, sim_i_neg_l and sim_l_neg_i. These are the similarity terms which need to be maximized and minimized relative to each other. Logits for sigmoid were constructed from all 5 similarity terms and hence each of them individually constrained to a range. Quantiative results are better for some datasets and at par with master on others
Next set of experiments tried to validate the negative sampling procedure. Instead of randomly sampling negatives, we mine hard negatives by choosing those embeddings which have the highest similarity in the pair. For example, to compute sim_i_neg_l we choose those set of neg_l for which sim_i_neg_l turns out to be maximum. We call this strategy as the sorted sampling strategy. We evaluate this on two levels -

F1 score on test set: On large datasets like Carbon Bot and Sara, the performance is the same as random negative sampling. However, on smaller datasets with low number of examples per intent a lot of configurations benefit. On the HINT3 dataset we achieve almost the SoTA or in some cases slightly better as well. Results can be found in 1 and 2. The boosts here are moderate compared to even random sampling + sigmoid and considerably large compared to current performance on master.
Range of similarity values: To track this, along with the mean similarity value of positive and negative pairs we also calculate the standard deviation. Ideally we want the standard deviation to not be too high. With random negative sampling + sigmoid added to the loss term, the similarity values for the negative pairs are around -0.3(+/-2.37) . When employing sorted sampling strategy + sigmoid added to the loss term, the similarity values for negative pairs are around 0.72(+/-0.42). While the mean value is higher, standard deviation is much lower for hard negative strategy which is a good result. These numbers were on Private 3 dataset with sparse + DIET config. However -

It's harder to train with sorted sampling strategy + increasing batch size as the optimization gets stuck in an oscillating behaviour on some datasets(specially small ones). For this reducing the batch size and keeping it fixed to 32 works the best.
The training time is increased because of the sorting that is done on every pass of a minibatch.

We also found that minimizing sim_i_neg_i is tough. A side effect of trying to minimize it is that it impacts the model's ability to minimize similarities of other negative pairs after a certain extent. (The mean il, li similarity that is achieved after training is ~ -0.5). Removing sim_i_neg_i lets the model achieve a mean value of ~ 6.5 for sim_i_pos_l and ~ -7 for sim_i_neg_l  and sim_l_neg_i which is the ideal range we would want them to be in. However, on the other hand not minimizing sim_neg_ii also impacts test performance. This seems like a classical problem of multi-task loss where the value of one term dominates the overall loss value and goes in a cyclic oscillating behaviour where after a point training loss and training accuracy is saturated and cannot be improved further. Two approaches that we are currently trying to see if they help -

Weigh down the contribution of sim_i_neg_i by multiplying it with a small constant. Tried with a constant value of 0.02 and it does seem to benefit(on sorted sampling)
Separate the loss into main loss (containing main softmax and maybe sigmoid) and auxiliary loss containing sim_i_neg_i sigmoid and others, then we filter the auxiliary gradients if they adhere to a certain condition. The filtering condition needs to be thought through.
Use a smaller number negative samples for sim_i_neg_i.

TODOs from above experiments-

[ ] PR for introducing sigmoid term inside the softmax loss.
[ ] PR for sorted sampling strategy but turned off by default and made configurable. Advise users to lower the batch size when using it.
[ ] Look into sim_neg_ii and ways to mitigate the problem.

@dakshvar22 is this in progress? Are the TODOs dependent on each other? Are they both TODOs that can be picked up by other people or are these notes for you?

@dakshvar22 is this in progress? Are the TODOs dependent on each other? Are they both TODOs that can be picked up by other people or are these notes for you?

@koernerfelicia This is in progress and both me and @Ghostvv are working together on 1st and 3rd TODO. The 2nd TODO will follow once the 1st and 3rd are finished.

@koernerfelicia This is in progress and both me and @Ghostvv are working together on 1st and 3rd TODO. The 2nd TODO will follow once the 1st and 3rd are finished.

1st and 3rd TODOs of this were shipped with https://github.com/RasaHQ/rasa/pull/7616

We don't plan to attempt the second TODO as of now because of competing priorities. Deprioritizing for now and closing the issue.


BBC downloads failing (from within the UK)

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.05.14. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[ x] I've verified and I assure that I'm running youtube-dl 2017.05.14

Before submitting an issue make sure you have:

[x ] At least skimmed through README and most notably FAQ and BUGS sections
[x ] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
$ youtube-dl --verbose -o have-i-got-a-bit-more-news-for-you-series-53-episode-4.mp4 http://www.bbc.co.uk/iplayer/episode/b08r8wp6/have-i-got-a-bit-more-news-for-you-series-53-episode-4
[debug] System config: []
[debug] User config: [u'--prefer-ffmpeg', u'--sub-lang', u'en', u'--write-sub', u'--convert-subtitles', u'srt', u'--merge-output-format', u'mp4']
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'-o', u'have-i-got-a-bit-more-news-for-you-series-53-episode-4.mp4', u'http://www.bbc.co.uk/iplayer/episode/b08r8wp6/have-i-got-a-bit-more-news-for-you-series-53-episode-4']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.05.14
[debug] Python version 2.7.6 - Linux-3.13.0-79-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: avconv 11.6, avprobe 11.6, ffmpeg N-78590-g5590ab4, ffprobe N-78590-g5590ab4
[debug] Proxy map: {}
[bbc.co.uk] b08r8wp6: Downloading video page
[bbc.co.uk] b08r8wp6: Downloading playlist JSON
[bbc.co.uk] b08r8wp4: Downloading media selection XML
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading MPD manifest
[bbc.co.uk] b08r8wp4: Downloading MPD manifest
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading MPD manifest
[bbc.co.uk] b08r8wp4: Downloading MPD manifest
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading m3u8 information
[bbc.co.uk] b08r8wp4: Downloading captions
[bbc.co.uk] b08r8wp4: Downloading captions
[bbc.co.uk] b08r8wp4: Downloading captions
[info] Writing video subtitles to: have-i-got-a-bit-more-news-for-you-series-53-episode-4.en.ttml
[debug] Invoking downloader on u'https://vod-dash-uk-live.bbcfmt.hs.llnwd.net/usp/auth/vod/piff_abr_full_hd/18c194-b08r8wp4/vf_b08r8wp4_b5e319f6-6731-44de-b43b-ed59418f42d8.ism.hlsv2.ism/dash/'
[dashsegments] Total fragments: 661
[download] Destination: have-i-got-a-bit-more-news-for-you-series-53-episode-4.fstream-uk-iptv_streaming_concrete_combined_hd_mf_limelight_uk_dash_https-video=5070000.mp4
[download] 100% of 1.36GiB in 04:15
[debug] Invoking downloader on u'https://vod-dash-uk-live.bbcfmt.hs.llnwd.net/usp/auth/vod/piff_abr_full_hd/18c194-b08r8wp4/vf_b08r8wp4_b5e319f6-6731-44de-b43b-ed59418f42d8.ism.hlsv2.ism/dash/'
[dashsegments] Total fragments: 661
[download] Destination: have-i-got-a-bit-more-news-for-you-series-53-episode-4.mp4.fstream-uk-iptv_streaming_concrete_combined_hd_mf_limelight_uk_dash_https-audio_eng=128000
[download] 100% of 38.40MiB in 02:06
[ffmpeg] Merging formats into "have-i-got-a-bit-more-news-for-you-series-53-episode-4.mp4"
[debug] ffmpeg command line: ffmpeg -y -i file:have-i-got-a-bit-more-news-for-you-series-53-episode-4.fstream-uk-iptv_streaming_concrete_combined_hd_mf_limelight_uk_dash_https-video=5070000.mp4 -i file:have-i-got-a-bit-more-news-for-you-series-53-episode-4.mp4.fstream-uk-iptv_streaming_concrete_combined_hd_mf_limelight_uk_dash_https-audio_eng=128000 -c copy -map 0:v:0 -map 1:a:0 file:have-i-got-a-bit-more-news-for-you-series-53-episode-4.temp.mp4
ERROR: Conversion failed!
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1942, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 479, in run
    self.run_ffmpeg_multiple_files(info['__files_to_merge'], temp_filename, args)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 204, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError: Conversion failed!

Description of your issue, suggested solution and other information
I have been using youtube-dl to download from BBC's iPlayer for a while.  About three days ago it stopped working.  Instead of the expected show-name.mp4 and show-name.en.srt files I get long, complicated filenames with extensions like .part, .part-Frag1, .ytdl, .temp.mp4, and no extension.  I had been running an earlier version of youtube-dl, but I upgraded to the latest version and it didn't help.  A complete example is shown above.  You'll need to be in the UK to get even this far, but presumably if you just try my exact command you should get my exact failure.

This appears to be due either to a -f flag or to some odd interaction amongst -o flags.  I had forgotten I set up a youtube-dl.config file, containing the following:
--sub-lang 'en'
--write-sub
--convert-subtitles 'srt'
--embed-subs
-o '/home/larryy/video/%(title)s [%(resolution)s].%(ext)s'
-f 'bestvideo+bestaudio/best'
--merge-output-format mp4

In some testing, I removed the -f flag from my config file and typed just this on the command line:
$ youtube-dl http://www.bbc.co.uk/iplayer/episode/b08r8wp6/have-i-got-a-bit-more-news-for-you-series-53-episode-4

And it downloaded successfully, to files named "Have I Got a Bit More News for You, Series 53, Episode 4-b08r8wp4.mp4" and "Have I Got a Bit More News for You, Series 53, Episode 4-b08r8wp4.en.srt".
Note that the name does not include the resolution, despite it being specified in the config's -o specification.  And it does include the PID on the end.
I am experimenting with restoring my command-line override of the -o value (I need a controllable, specifiable filename, as I normally run youtube-dl from a script and need to move the file once the download is complete).  That will tell me whether the better behavior is related to removing the config's -f param or the command-line's -o param.  Will update tomorrow sometime.

It's the -f 'bestvideo+bestaudio/best' flag.  I can override the -o flag as always and things work fine.  But if that -f flag is present the download is broken.

yup, the same happened to me (from Czech Republic, I think UK location is a red herring here):
matej@mitmanek: ~$ youtube-dl --verbose 'http://www.bbc.com/news/av/world-us-canada-41687716/george-w-bush-decries-bigotry-and-conspiracy-theories'
[debug] System config: [u'--prefer-free-formats']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'http://www.bbc.com/news/av/world-us-canada-41687716/george-w-bush-decries-bigotry-and-conspiracy-theories']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 2.7.5 - Linux-3.10.0-733.el7.x86_64-x86_64-with-redhat-7.5-Maipo
[debug] exe versions: ffmpeg 2.8.13, ffprobe 2.8.13, rtmpdump 2.4
[debug] Proxy map: {}
[bbc] george-w-bush-decries-bigotry-and-conspiracy-theories: Downloading webpage
[bbc] p05kdqnm: Downloading media selection XML
[bbc] p05kdqnm: Downloading m3u8 information
[bbc] p05kdqnm: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc] p05kdqnm: Downloading MPD manifest
[bbc] p05kdqnm: Downloading m3u8 information
[bbc] p05kdqnm: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc] p05kdqnm: Downloading MPD manifest
[bbc] p05kdqnm: Downloading m3u8 information
[bbc] p05kdqnm: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc] p05kdqnm: Downloading MPD manifest
[bbc] p05kdqnm: Downloading m3u8 information
[bbc] p05kdqnm: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc] p05kdqnm: Downloading MPD manifest
[download] Downloading playlist: George W Bush: 'Bigotry seems emboldened'
[bbc] playlist George W Bush: 'Bigotry seems emboldened': Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://vod-dash-ww-live.bbcfmt.hs.llnwd.net/usp/auth/vod/piff_abr_full_sd/7d5177-p05kdqnm/vf_p05kdqnm_7f5e897c-fadd-4be6-8bdc-16100b1c4646.ism.hlsv2.ism/dash/'
[download] George W Bush decries bigotry and conspiracy theories-p05kdqnm.fstream-nonuk-iptv_streaming_concrete_combined_sd_mf_limelight_world_dash_https-video=1570000.mp4 has already been downloaded
[download] 100% of 12.08MiB
[debug] Invoking downloader on u'https://vod-dash-ww-live.bbcfmt.hs.llnwd.net/usp/auth/vod/piff_abr_full_sd/7d5177-p05kdqnm/vf_p05kdqnm_7f5e897c-fadd-4be6-8bdc-16100b1c4646.ism.hlsv2.ism/dash/'
[download] George W Bush decries bigotry and conspiracy theories-p05kdqnm.fstream-nonuk-iptv_streaming_concrete_combined_sd_mf_limelight_world_dash_https-audio_eng_1=128000.m4a has already been downloaded
[download] 100% of 1.01MiB
[ffmpeg] Merging formats into "George W Bush decries bigotry and conspiracy theories-p05kdqnm.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:George W Bush decries bigotry and conspiracy theories-p05kdqnm.fstream-nonuk-iptv_streaming_concrete_combined_sd_mf_limelight_world_dash_https-video=1570000.mp4' -i 'file:George W Bush decries bigotry and conspiracy theories-p05kdqnm.fstream-nonuk-iptv_streaming_concrete_combined_sd_mf_limelight_world_dash_https-audio_eng_1=128000.m4a' -c copy -map '0:v:0' -map '1:a:0' 'file:George W Bush decries bigotry and conspiracy theories-p05kdqnm.temp.mp4'
ERROR: Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2030, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/postprocessor/ffmpeg.py", line 479, in run
    self.run_ffmpeg_multiple_files(info['__files_to_merge'], temp_filename, args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/postprocessor/ffmpeg.py", line 204, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError: Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument

And yes, if I download particular format (not the automatic one), everything is perfectly working.

Tested http://www.bbc.com/news/av/world-us-canada-41687716/george-w-bush-decries-bigotry-and-conspiracy-theories with

yt-dl-2021.02.10, Linux x86, ffmpeg-4.3, python-3.5: all OK
yt-dl-2021.03.25, Linux MIPS32, ffmpeg-4.1, python-2.7: all OK

Probably fixed ages ago.
with pyth

Yup, I can confirm. This ticket should be closed.


host: add kallsyms parsing tests

Start with a few simple tests that can be extended when needed.

Codecov Report

Merging #1105 into master will increase coverage by 0.02%.
The diff coverage is 89.47%.

@@            Coverage Diff             @@
##           master    #1105      +/-   ##
==========================================
+ Coverage   67.28%   67.31%   +0.02%
==========================================
  Files         119      119
  Lines       20617    20621       +4
==========================================
+ Hits        13873    13881       +8
+ Misses       6085     6081       -4
  Partials      659      659

Impacted Files
Coverage Δ

pkg/host/host_linux.go
51.78% <89.47%> (+1.52%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dfd3394...5ac7da0. Read the comment docs.

@xairy, does https://syzkaller.appspot.com/bug?extid=8d3ef574da750aacb37a look like a possible casualty?
github.com/google/syzkaller/pkg/host [github.com/google/syzkaller/pkg/host.test]
pkg/host/host_test.go:100:17: undefined: parseKallsyms


Add a permissive CORS policy

For web developers to be able to play with the demo service in their
browser, we probably want a permissive CORS policy.

Thank you!


Messenger Service not working on W7

Expected Behavior
When I click on the Messenger Service, it should ask me to connect or display the service interface.
Current Behavior
When I click on the Messenger Service, I get a blank page.
Screenshots (if appropriate):

Your Environment

Franz Version used: 5.0.0-beta.18
Operating System and version: Windows 7 Enterprise SPK 1

Same me

I'm having the same issue with version 4.0.4 on debian linux

May be facebook lock

Also experiencing this. When I load up the Service Developer Tools...it shows a 500 error trying to retrieve messenger.com.

Everybody use win 10 build 1703 or higher. This error back up old version everything will be ok


Add '/' route logic for Express app to docs

WHY are these changes introduced?

there is currently no '/' endpoint in the docs for Express

WHAT is this pull request doing?

add logic that checks if the user is logged in. If they're not logged in, redirects user to login with status code 302

Type of change

[ ] Patch: Bug (non-breaking change which fixes an issue)
[ ] Minor: New feature (non-breaking change which adds functionality)
[x] Major: Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] I have added a changelog entry, prefixed by the type of change noted above
[ ] I have added/updated tests for this change
[ ] I have documented new APIs/updated the documentation for modified APIs (for public APIs)

I have some feedback on this: as-is, the documentation is fundamentally broken for standalone apps using offline sessions for a couple of reasons, and it'd be a pretty huge quality of life improvement for the correct way of handling it to be included in the official guides!
First issue: loadCurrentSession in this scenario is flat-out broken, due to the bug mentioned in this PR https://github.com/Shopify/shopify-node-api/pull/272. As such, we'll use loadOfflineSession.
Second issue: loadOfflineSession doesn't validate the shop parameter to confirm the route request is actually coming from Shopify; as such, we'll need to validate the hmac before proceeding with the route.
Third issue: the built-in Shopify.Utils.validateHmac function doesn't work in this context (as the query string keys are different from those defined in the AuthQuery constructor -- we have hmac, host, session, shop, and timestamp here as opposed to AuthQuery's code, timestamp, state, shop, host, and hmac); as such, a custom validation function is required.
Fourth issue: Due to the fact that beginAuth generates an un-auth'd session (I.E. no accessToken or scopes) the moment the install request is generated but BEFORE oAuth has actually been confirmed, the existence of a session actually does not at all determine whether or not a user should be directed to the /login (I.E. generate oAuth) route. See: https://github.com/Shopify/shopify-node-api/issues/255. I don't believe it would be proper to just discard the un-auth'd temp session generated by beginAuth, as validateAuthCallback requires a session to proceed.
I believe that the way to handle checking for whether a user should be directed to /login (I.E. beginAuth, followed by validateAuthCallback) is whether the loaded session has an accessToken, not just that the session exists. (Since it's entirely possible for a user to abandon the install flow, leaving an un-authorized session in perpetual limbo). Most of the example code uses the ACTIVE_SHOPIFY_SHOPS local object (with the assumption that this is loaded from the database on server start) to approximate 'shop has finished oAuth,' but it seems like accessToken is a more robust measure of this? (A more complete solution ought to also check that scopes match, I believe)
As best I can tell, setting up a minimally-viable default-route endpoint (and redirecting to a login path as necessary) using offline sessions should look something like the following, which I've tested on my own app for both post-install oAuth routing and 'Vendor clicks on the app link from admin/apps' routing:
`  api.get('/', async(req, res) => {
const shop = req.query.shop;
const {hmac, ...cleanQueryKeys} = req.query;
const cleanQueryString = new URLSearchParams(cleanQueryKeys).toString();

const generatedHash = crypto.createHmac('SHA256', Shopify.Context.API_SECRET_KEY).update(cleanQueryString).digest('hex');

const safeCompareResult = Shopify.Utils.safeCompare(generatedHash, hmac);

if (!safeCompareResult) {
console.log('Invalid hmac, aborting');
return res.status(401).json({ succeeded: false, message: 'Not Authorized' }).send();
}
const authSession = await Shopify.Utils.loadOfflineSession(shop);
if (!(authSession && authSession.accessToken)) {
console.log('Shop auth is not yet authorized, redirecting to login');
res.redirect(login?shop=${shop});
} else {
console.log('Shop is authorized, redirecting to standalone app');
return res.redirect(https://example.app.com?redirect=shopify&hmac=${hmac}&shop=${shop});
}
});


Update permissions-and-ports-for-harness-connections.md

fixed instance of CurrentGen.
Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the maintainers.
What Type of PR is This?

[ ] Issue
[ ] Feature
[ x] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Preview environment: https://ci.pr.harness.io/pr-673


conu in a container: don't create pyc files

PYTHONDONTWRITEBYTECODE=yes :wink: :wink:

I actually meant to fix it in https://github.com/fedora-modularity/conu/blob/master/Dockerfile, my bad.


py_trees_ros_tutorials: 1.0.6-1 in 'eloquent/distribution.yaml' [bloom]

Increasing version of package(s) in repository py_trees_ros_tutorials to 1.0.6-1:

upstream repository: https://github.com/splintered-reality/py_trees_ros_tutorials.git
release repository: https://github.com/stonier/py_trees_ros_tutorials-release.git
distro file: eloquent/distribution.yaml
bloom version: 0.9.0
previous version for package: null

@stonier Looks like this needs a rebase (that will also re-kick travis, which seems to be stuck).

Kicked...


Remove source code and native-to-build-host grpc_c.so from ruby linux binary packages

Fixes https://github.com/google/protobuf/issues/4210 for the gRPC ruby package (the google-protobuf package should need a similar fix). Prompted by the suggestion in https://github.com/rake-compiler/rake-compiler/issues/146
Running an artifact build in https://grpc-testing.appspot.com/job/gRPC_build_artifacts/2059/

PTAL I think this is ready for a look.
The artifact build on jenkins has been timing out and I haven't been able to get a build on jenkins, but I built packages from this PR locally and did local testing. I also ran the ruby distrib tests locally (except for the ones based off of ubuntu images).
For comparison, here is the content of the latest (1.10) x86_64-linux gem:

https://gist.github.com/apolcyn/71024dcc99b52dd3b7fdee763e69dd96

and here is the content of the x86_64-linux
gem built from this PR:

https://gist.github.com/apolcyn/806e64e7d6c9d458fb8cc2c086e3584d

... with the source code and src/ruby/lib/grpc/grpc_c.so now removed.

[trickle] No significant performance differences

Test failure in the last round of tests was due to
TIMEOUT: tools/run_tests/helper_scripts/run_ruby_end2end_tests.sh, which is new, but I believe is unrelated to this change. Re-running

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

friendly ping, this should be ready for review

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

friendly ping, I'd love to get this in for before the 1.11 branch cut

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences


Fix generated Atom tags being timezone-dependent

The atom_tag_for function uses a timezone-dependent conversion function __nanoc_to_iso8601_date to generate the Atom tag from created_at. For example:
$ TZ=Europe/Amsterdam ruby -rnanoc -ryaml -e "extend Nanoc::Helpers::Blogging;puts attribute_to_time(YAML::load('2015-03-09 00:00:00')).to_iso8601_date"
2015-03-09
$ TZ=America/Los_Angeles ruby -rnanoc -ryaml -e "extend Nanoc::Helpers::Blogging;puts attribute_to_time(YAML::load('2015-03-09 00:00:00')).to_iso8601_date"
2015-03-08

to_iso8601_time already uses getutc, so I think to_iso8601_date should do the same.
Note: this fix might result in people's tags being changed once when they update nanoc if they were originally running their code in a timezone different from UTC.

This makes sense! Can you make two small changes:

Open a PR against the release-4.2 branch instead, so it ends up in the next bugfix release.
Add a test for this in test/extra/core_ext/test_time.rb.

Happy to help with either!

Any ideas on how to approach the test? AFAIK there is no platform-independent way to configure the current timezone.

I believe you can do allow(Time).to receive(:zone).and_return('CEST'). That’d be in the before block of a context.

I looked into writing specs for this, and it turns out that this change isn’t quite correct—when given a date (rather than a time), the fix fails. Will work on a separate PR that fixes this and adds specs.

Thanks for picking this up, I've been swamped...

#to_iso8601_date deliberately does not convert to UTC because the date is supposed to be timezone-agnostic.
For example, if I were to write 2016-10-22, then I intend that specific day, no matter where I am, and what time it is. Converting this date to UTC first could change that date to 2016-10-21 or 2016-10-23, which is not what’s intended.
Perhaps #attribute_to_time should be updated so that, given a Date rather than a Time, the generated Time object is created from UTC time. Will give that a try!

Replaced by #961.


multi-language tts

I see that it can accept multi-language input (Whisper) but multi-language output (tts) only supports English. Is there any tts product that supports automatic multi-language?

Cartesia has an alternate model which supports more languages, but it's in alpha and has some issues, so we decided to just use their English model.
You can fork the project and change sonic-english to sonic-multilingual on line 79 of /api/api/route.ts if you want to!


Writing Documentation

Hi
You can use this as a reference on how to write using latex
https://www.overleaf.com/learn/latex/

thankyou!

@yasmensarhan27 Thank you for your guidance.


IIUC, scss requires semi-colons, so I have no idea how this worked before?
🧬 QA & Testing

Broken on production.
Working in this PR app.

Yeah, not technically required on the last declaration of a ruleset, but it's def best practice to include ‘em IMHO!

Ah, good to know.

This PR was released!
🚀 Changes included in v6.62.3


Fully Functional

Everything works fine

Scanning - Visualization -Data Handling - Frontend - Backend - Download - Routing
Works fine.