HADOOP-17905. Modify Text.ensureCapacity() to efficiently max out the…

… backing array size
Change-Id: I3cfae85000c5fa7aa86c40c2d7efa282958178bf

Description of PR
Allow org.apache.hadoop.io.Text to expand the underlying byte array to a safe maximum.
How was this patch tested?

Ran unit tests
Ran a MapReduce job where a single mapper processed a text file with a size of 1.8G with no line feeds. Array expansion was verified with extra printouts.

For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 54s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
31m  2s

trunk passed

+1 :green_heart:
compile
21m 17s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
18m 28s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 11s

trunk passed

+1 :green_heart:
mvnsite
1m 38s

trunk passed

+1 :green_heart:
javadoc
1m 10s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 39s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 26s

trunk passed

+1 :green_heart:
shadedclient
15m 47s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 55s

the patch passed

+1 :green_heart:
compile
21m 23s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
21m 23s

the patch passed

+1 :green_heart:
compile
19m 39s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 39s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 14s

the patch passed

+1 :green_heart:
mvnsite
1m 51s

the patch passed

+1 :green_heart:
javadoc
1m 15s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 47s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m 21s

the patch passed

+1 :green_heart:
shadedclient
20m 57s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
19m 26s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 58s

The patch does not generate ASF License warnings.

188m 27s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3423

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 1525d4527b51 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ba11e4dcc69491d8afcb9d046147d740a6d2bda9

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/2/testReport/

Max. process+thread count
3149 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

-1 :x:
@author
0m  0s
/results-author.txt
The patch appears to contain 2 @author tags which the community has agreed to not allow in code contributions.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 43s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 47s

trunk passed

+1 :green_heart:
compile
22m 56s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 30s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
3m 49s

trunk passed

+1 :green_heart:
mvnsite
25m 33s

trunk passed

+1 :green_heart:
javadoc
7m 51s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
7m 51s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
36m 49s

trunk passed

+1 :green_heart:
shadedclient
50m  9s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
23m 29s

the patch passed

+1 :green_heart:
compile
22m 27s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 27s

the patch passed

+1 :green_heart:
compile
19m 35s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 35s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 14736 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
checkstyle
3m 47s

the patch passed

+1 :green_heart:
mvnsite
21m 23s

the patch passed

+1 :green_heart:
shellcheck
0m  1s

No new issues.

+1 :green_heart:
javadoc
7m 42s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
7m 47s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
37m 21s

the patch passed

+1 :green_heart:
shadedclient
50m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
1009m 41s
/patch-unit-root.txt
root in the patch passed.

-1 :x:
asflicense
1m 18s
/results-asflicense.txt
The patch generated 2 ASF License warnings.

1351m 58s

Reason
Tests

Failed junit tests
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure

hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3423

Optional Tests
dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle

uname
Linux bbe67560c19d 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / fc99e97a629e28fc4fd4fd772da918634dd1824a

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/1/testReport/

Max. process+thread count
2220 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/1/console

versions
git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 54s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 43s

trunk passed

+1 :green_heart:
compile
23m  1s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 29s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 59s

trunk passed

+1 :green_heart:
mvnsite
1m 32s

trunk passed

+1 :green_heart:
javadoc
1m  2s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 33s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 32s

trunk passed

+1 :green_heart:
shadedclient
36m  2s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 57s

the patch passed

+1 :green_heart:
compile
22m 19s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 19s

the patch passed

+1 :green_heart:
compile
19m 31s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  0s

the patch passed

+1 :green_heart:
mvnsite
1m 31s

the patch passed

+1 :green_heart:
javadoc
1m  0s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 35s

the patch passed

+1 :green_heart:
shadedclient
35m 29s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 59s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

224m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3423

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux fbe9089dd010 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d82e53f9626373492899ce413ed64cb2ad9bb1a8

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/3/testReport/

Max. process+thread count
1252 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  5s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 50s

trunk passed

+1 :green_heart:
compile
23m  8s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 29s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  0s

trunk passed

+1 :green_heart:
mvnsite
1m 32s

trunk passed

+1 :green_heart:
javadoc
1m  1s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 36s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 23s

trunk passed

+1 :green_heart:
shadedclient
35m 44s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m 13s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 13s

the patch passed

+1 :green_heart:
compile
19m 28s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  0s

the patch passed

+1 :green_heart:
mvnsite
1m 30s

the patch passed

+1 :green_heart:
javadoc
1m  0s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 34s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 34s

the patch passed

+1 :green_heart:
shadedclient
35m 48s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m  0s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 49s

The patch does not generate ASF License warnings.

225m  9s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3423

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux b67dd1b46a84 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d82e53f9626373492899ce413ed64cb2ad9bb1a8

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/4/testReport/

Max. process+thread count
2528 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
36m 14s

trunk passed

+1 :green_heart:
compile
27m 55s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
23m 18s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 10s

trunk passed

+1 :green_heart:
mvnsite
1m 57s

trunk passed

+1 :green_heart:
javadoc
1m 16s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 50s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 49s

trunk passed

+1 :green_heart:
shadedclient
27m 20s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 11s

the patch passed

+1 :green_heart:
compile
27m 12s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
27m 12s

the patch passed

+1 :green_heart:
compile
23m  6s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
23m  6s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  1s

the patch passed

+1 :green_heart:
mvnsite
1m 39s

the patch passed

+1 :green_heart:
javadoc
1m 15s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 47s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m  2s

the patch passed

+1 :green_heart:
shadedclient
26m 58s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 28s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 57s

The patch does not generate ASF License warnings.

230m 55s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3423

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 6b9deed95f06 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4a89581189e9b510823aaa1b3962221a3e3356f0

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/5/testReport/

Max. process+thread count
2014 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3423/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Thank You @pbacsko for the contribution and @goiri for the careful reviews.


Poor result of MEMIT over llama3.2 3B

Thank you for the wonderful jobs and provide the hyper-meters of MEMIT over llama3.2 3B.
However, my edit accuracy under 200 facts is low that 50%, while the MEMIT over gpt2-xl have 99% accuracy under 2000 facts. Do you have any tips, thank you very much?

It's strange, I've been working on fixing MEMIT multi-gpu errors recently, and I'll debug this issue as well.

Thank you for your response. To reduce GPU memory usage, I have adjusted the maximum token sequence length in the layer stat file to 512. I'm not sure if this change has negatively affected the results. Looking forward your response after debug.

Translate to English: Hi, my download speed for wikipedia/20200501.en is very slow. Do you have a precomputed cache of Wikipedia (with the file extension .npz) so that I can debug and help you identify the problem faster?

Sure. Upload the file will cost a few hours.

I upload my npc file in the following link:
https://drive.google.com/file/d/1-7BiEG3hZKFBd_Kz3YTgEttGjAHLJydn/view?usp=sharing
By the way, I guess you can use hf-mirror to load the wikipedia/20200501.en.
link: https://hf-mirror.com

I'm glad to have received your npz file. We are currently debugging the issue, and we will inform you promptly once we have results.


Design some state-of-the-art network architecture for semantic segmentation problem

On the model of the job done for feature detection, we could extend semantic segmentation networks to some well-known architectures. See for example this blog post for a review of common architectures. We could implement the following ones:

U-net : a classic decoder-encoder architecture, more sophisticated than the simple architecture.
dilated convolutions : use dilated convolutions as a improvement with respect to upsampling operation
DeepLab (v3) : state-of-the-art architecture with dilated convolutions and a pyramidal structure.

Solved by PR #77 .


Add upgrade complete announcement event

Related:

https://github.com/giantswarm/roadmap/issues/472
https://github.com/giantswarm/upgrade-schedule-operator/pull/20

Add a Slack message to indicate when a cluster upgrade has completed.
Note: I opted for the ClusterVersionUpdated event over the ClusterUpdated event to be more in-line with the existing "version upgrade scheduled" notification. Please correct me if this isn't the right approach.

By the way, you can take a look at how the conditions are set and the events are emitted here: https://github.com/giantswarm/cluster-operator/blob/4957162804e94e2a5dc265677c48ccf2b8f53124/service/controller/resource/statuscondition/create.go#L164
(apologies if you already saw this)
It's a little complicated but essentially, the updated condition is set once all the nodes are rolled and then the new version is updated in the version history. That is why there are two events here. I think picking the latter as you did makes sense since it is the final step.

Yeah you're right, it'll trigger on creation - https://github.com/giantswarm/cluster-operator/blob/4957162804e94e2a5dc265677c48ccf2b8f53124/service/controller/resource/statuscondition/create.go#L310-L317
I'll change it to the updated event. Is there any concern with this potentially alerting more than desired?

I don't think so. It will create message every time a cluster was successfully updated. (Not just through scheduling, but that is okay)

Yeah, just realised it's already doing it for the "ClusterIsUpdating" so it is already notifying on every non-scheduled update anyway.
Updated. Would you mind walking me through how to test this on a test installation please? :)


FIX #639

I've never written Vader tests so not sure if the assert works on lists

Yea, @blueyed wanna help me with the tests? Not sure how to run it, getting
(5/5) [EXECUTE] neomake#utils#GetSetting
    (5/5) [EXECUTE] (X) Vim(let):E706: Variable type mismatch for: g:neomake_myft_mymaker_serialize
      > /tmp/vgGu3lW/138, line 18

Thanks!


System.OutOfMemoryException thrown in CubismPhysicsRig.cs

System.OutOfMemoryException: 'Array dimensions exceeded supported range.'
Line 126 in CubismPhysicsRig.cs
Appears to not be able to handle cases where there are too many particles. In this case, particles were<PHONE_NUMBER>.

Attempt from Azur Lane. Appears to stem from their Unity 2020 upgrade. Newer Unity versions appear to be incompatible with this extractor.

Hello, I managed to get it work.
https://github.com/Perfare/UnityLive2DExtractor/blob/ab37d53b99cf34641305704e93e2e0b9098f80c4/UnityLive2DExtractor/CubismPhysicsRig.cs#L113
Before this line, add:
reader.ReadAlignedString();

Currently, I'm using a version ported to .NET 8, with https://github.com/zhangjiequan/AssetStudio library seems to work fine. I can publish a fork later.

Thank you for this. I made the change suggested, and while it did fix the exception, for some reason it's not able to extract all of the files like it used to. Maybe the updated AssetStudio version would help.
Your fork would be very much appreciated. It does appear this project might be abandoned.

https://github.com/CCRcmcpe/UnityLive2DExtractor

Thanks a lot. It's working.

Hey, I'm sorry in advance if this isn't the place to ask. I was wondering if a release could be made including this fix as I have also been having issues with using the extractor for Azur Lane. I am not sure how to turn the code into the finished product/.exe.
Any help would be greatly appreciated, thank you!

@kugon21 Easiest way is to install .NET SDK, and run dotnet publish -r win-x64 -o bin\pub in the directory containing the .csproj. Then the results are in bin\pub.

Thank you! I was able to get that working with that, thanks again for the help and the fix.


cmd/server: adjust jaro weighting to maximize total weight in strong single word matches

We should favor strong single word matches to reflect higher overall
match percentages. In short this is useful for matching names like:
Nicolas MADURO

against an SDN entity like:
MADURO MOROS, Nicolas

People often won't specify middle names (or multiple first/last
names), so when positive matches are identified we should keep the
overall match percentages high.
Issue: #141

Codecov Report

Merging #144 into master will decrease coverage by 0.93%.
The diff coverage is 70.83%.

@@            Coverage Diff             @@
##           master     #144      +/-   ##
==========================================
- Coverage   59.53%   58.59%   -0.94%
==========================================
  Files          26       26
  Lines        2696     2717      +21
==========================================
- Hits         1605     1592      -13
- Misses        925      964      +39
+ Partials      166      161       -5


I couldn't install any extention

Issue Type: Bug
Hello,
i couldn't install any extention it gives me this message for example
Failed to install 'jundat95.react-native-snippet'.
I tried to download manuallt also failed .This happened all of sudden .
VS Code version: Code 1.37.0 (036a6b1d3ac84e5ca96a17a44e63a87971f8fcc8, 2019-08-08T02:33:50.993Z)
OS version: Windows_NT x64 10.0.17763

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz (8 x 1800)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwareoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_deferred_display_list: disabled_offskia_renderer: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
11.89GB (8.96GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions: none

Try to backup your extensions first.. go to C:/Users/{YourUsername}/.vscode/extensions, after backup, delete the .vscode folder or extensions folder.
Re-installing vscode again.
After installing vscode, copy back your extensions back to C:/Users/{YourUsername}/.vscode/extensions.

Thank you..

still not working ... ! :///////

#78783 seems to be a DNS issue.
Can you try to access this image? https://ms-python.gallerycdn.vsassets.io/extensions/ms-python/python/2019.8.29288/1565127378610/Microsoft.VisualStudio.Services.Icons.Default
If not, can you try changing your DNS setting, like pointing to <IP_ADDRESS> as suggested here? https://<IP_ADDRESS>/dns/

If you are able to repro it, can you please try with previous release -
https://az764295.vo.msecnd.net/stable/2213894ea0415ee8c85c5eea0d0ff81ecc191529/VSCodeUserSetup-x64-1.36.1.exe
Would like to understand what caused this change.


feat: introduce tls policy v1alpha2

Description
Part of https://github.com/Kuadrant/kuadrant-operator/issues/808
Currently just a straight copy of v1alpha1 in preparation for more api changes in the future

Might not be needed since v1apha1 probably wont have any major api changes, so if we decide to add section name support, this can be done to v1alpha1 instead since there's not too much value in adding v1alpha2 if they are broadly exactly the same

Decided that there's no need for v1apha2 for TLSPolicy. Closing


Confusing nav item placeholder behaviour

When creating/editing nav items the .fake-placeholder behaviour is a bit confusing because focusing the URL input field and then blurring causes the url value to be /. This gives the input a value but it's still shown as a placeholder because it matches the base URL. Once the validation is fixed it also increases confusion as it makes it appear as if it's not possible to link to the base url.

As you can hopefully see in the screen cap, the URL input will display as if it's empty and contains a placeholder even though it actually contains a value.
I see two options here:

Update the .fake-placeholder logic to inspect the underlying value and display accordingly. Most likely it would mean only displaying as a placeholder until the field receives focus and then have it act as an actual value.
Get rid of the .fake-placeholder completely so it looks like it's the default-value that it actually is. We can still keep the current behaviour of deleting the whole content on backspace if the current content matches the configured base URL.

My preference would be on option 2 - it simplifies the code and we don't try to fake a placeholder where we actually have a default value.

I am unable to reproduce it. Could you please elaborate it.

Closing for now, will re-open if it's still an issue once the validation fix and ember-sortable PRs are in.

After reviewing my other PRs I've updated the description and re-opened. I believe the fix for this is more of a design decision than a coding issue at this point.
/cc @JohnONolan

Agreed, option 2 sounds good :+1:

Another thing to think about too (will likely be fixed by option 2), is that if you mess around with the URL field but end up with the value being the same as the placeholder, and then click in/click out of the label field, the label field will register a validation error. (See screencap)


Eliminate busy wait, fixing 100% CPU usage.

Also remove unecessary communicate() call. It just prints "None" to the
log.

Erm, I don't see why this PR was frogotten @tomquisel . But you'll have to squash the commits I think. Then I can merge once somebody else +1's. I don't know much about spark.
By the way, the commit header must name "spark" or something somewhere, as the commit message is unclear. :)

I wish you didn't merge this as I think the commit-quality was pretty poor :-/

I think it was fine. What was the issue? Not squashed? yeah maybe should have required squashing it. But I also think it would have been a bit rude to get back after almost a month of silence then asking for a rebase

Totally not a big deal either way :). I'll squash commits and make the commit messages more clear in the future.

Sure, sorry for being not so responsive on the PR, don't hesitate to ping us @tomquisel :)
@erikbern My main issue is that the commit message doesn't mention "spark" anywhere. It sounds like it's fixing a critical bug in the scheduler or something. :)

yeah that's a good point. i was actually really confused too first :)


Question about using the API

Hi @huchenme!
I'm trying to create an iOS app using the github trending API.
May I release an app that uses this api in the AppStore?

Sure, why not? Let me know when your app is done 🙌

@huchenme ok! thanks!


Integrate w/ NSMIv2 / (now called LIST)

Synchronize list of categories with the list in NSMIv2
https://taxonomy.legal/

We should consider doing this before our Bootcamp, because it will also have virtue of making our category list work across the US instead of being tied to CourtFormsOnline.org

We should consider doing this before our Bootcamp, because it will also have virtue of making our category list work across the US instead of being tied to CourtFormsOnline.org

[ ] Change the category names/values to match what is on https://taxonomy.legal/. Start with just the top level tags.
[ ] Use the metadata tags feature in Docassemble. The metadata block should expose the tags as NSMIv2/LIST category ids. https://docassemble.org/docs/initial.html#metadata (search for "tags")

Should the category label "Income and benefits" be changed to "Public
Benefits"?

For most people the term "income" seems to refer to job income, so it
could be misleading.
"Public Benefits" is used on https://taxonomy.legal/.

On Mon, Feb 1, 2021 at 12:43 PM Quinten Steenhuis<EMAIL_ADDRESS>wrote:

Change the category names/values to match what is on
https://taxonomy.legal/. Start with just the top level tags.
Use the metadata tags feature in Docassemble. The metadata block
should expose the tags as NSMIv2/LIST category ids.
https://docassemble.org/docs/initial.html#metadata (search for "tags")

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/issues/92#issuecomment-771032089,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE2LLYMI4LSFWLLVLKPWQC3S43R3VANCNFSM4UOUJADA
.

Can't test the changes until master branch is fixed.

Let's change all of the categories to exactly match the categories on
taxonomy.legal
On Mon, Feb 1, 2021, 1:57 PM purplesky2016<EMAIL_ADDRESS>wrote:

Should the category label "Income and benefits" be changed to "Public
Benefits"?

For most people the term "income" seems to refer to job income, so it
could be misleading.
"Public Benefits" is used on https://taxonomy.legal/.

On Mon, Feb 1, 2021 at 12:43 PM Quinten Steenhuis <
<EMAIL_ADDRESS>wrote:

Change the category names/values to match what is on
https://taxonomy.legal/. Start with just the top level tags.
Use the metadata tags feature in Docassemble. The metadata block
should expose the tags as NSMIv2/LIST category ids.
https://docassemble.org/docs/initial.html#metadata (search for "tags")

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/issues/92#issuecomment-771032089
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AE2LLYMI4LSFWLLVLKPWQC3S43R3VANCNFSM4UOUJADA
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/issues/92#issuecomment-771080071,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB2KTSNA63VYX64S423ZTPLS432QBANCNFSM4UOUJADA
.

@purplesky2016, sorry I hadn't run into that when testing, https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/c5074b8a8fd4b458248ae56db3e77054dc8a0a54 should have fixed it, it did for me

Thanks @BryceStevenWilley, that worked.
@nonprofittechy - Before checking in my changes, I want to confirm with you a few things:

Following naming convention, I used  "be_00_00_00_00" instead of the original LIST value "BE-00-00-00-00". Is this okay?

To keep consistency, I used - Other: Other.

At first I tried "other" and show if: interview.categories['other'], but got an error that it can't be found.
It turned out that this is hard-coded in interview_generator.py line 463: if self.categories['Other']:

With metadata tags, this is what looks like on My Interviews screen. Clicking a tag you'll get a list of all the interviews that have that tag. I assume this what you have in mind?

Great! Let's stick with the exact LIST capitalization and hyphens, not
underscores.
I'll ask Jonathan ideas about the tags and how we might make them a little
nicer looking on the interview page.
On Wed, Feb 3, 2021, 10:06 AM purplesky2016<EMAIL_ADDRESS>wrote:

Thanks @BryceStevenWilley https://github.com/BryceStevenWilley, that
worked.
@nonprofittechy https://github.com/nonprofittechy - Before checking in
my changes, I want to confirm with you a few things:

Following naming convention, I used "be_00_00_00_00" instead of the
original LIST value "BE-00-00-00-00". Is this okay?
2.
To keep consistency, I used - Other: Other.

At first I tried "other" and show if: interview.categories['other'],
but got an error that it can't be found.

It turned out that this is hard-coded in interview_generator.py
line 463: if self.categories['Other']:

With metadata tags, this is what looks like on My Interviews screen.
Clicking a tag you'll get a list of all the interviews that have that tag.
I assume this what you have in mind?
[image: metadata tags]
https://user-images.githubusercontent.com/20231649/106764674-02ef3480-6606-11eb-95a5-d4cd96548e35.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/issues/92#issuecomment-772577506,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB2KTSIHFBAZ3FT4QFADMALS5FQ7LANCNFSM4UOUJADA
.

After changed them to LIST capitalization and hyphens, now I can't get pass that input screen. Below is the end portion of the "Source" code:
add_logic_tree
add_draft_intro_screen
created_preview_question
add_questions_to_interview
created_attachment
add_review_screen
wrote_interview
increment_sequence

show_interview

Needed definition of interview.title at 0.00005s
Completed processing at 0.00251s

But the titled field is filled.
Then I used a test interview to test checkbox with those LIST capitalization and hyphens, it worked fine. So this is not a DA issue. Could it be related to the generator module? Any idea?

Never mind, my fault. I'm going to push the code for PR.

FYI - I am not sure how accurate the "My interviews" screen is regarding the tags. Here is a screen shot:

The first interview was run on 1/25, prior to the LIST feature implementation, yet it has the same tags as the one ran today. This is misleading.
The two side-by-side lists of tags are probably due to the fact that I first used lower_case for the list value, then used UPPER_VALUE. Suppose we can ignore this oddity.
For interview on 2/3 today, I actually ran a few times, each time choosing different category values, I even used two different pdf forms, yet the tags listed didn't get updated, only time was updated each time.


MacOS 15.1: Connection drops mid-command without throwing

Describe the bug
While running Cilicon on macOS 15.1 we have noticed that the command stream eventually fails without throwing or completing, causing Cilicon to get stuck waiting for a response.
We have hacked together a version of Cilicon that uses Process to run ssh/sshpass and we're not seeing this issue there, which indicates a problem on the client side.
I'm not sure whether this is an issue with Citadel or swift-nio.
Reproducer Sample
Cilicon 2.2.2 on macOS 15.1 with github provisioner (it may take quite a while for this issue to occur)
Expected behavior
Ideally the connection should not drop, but when it does the executeCommand function should throw.
Client (please complete the following information):

OS: macOS 15.1
Client: Citadel
Version of Citadel, if applicable: 0.7.2, 0.9.1

Server (please complete the following information):

OS: macOS 14.X, macOS 15.X
Server: opensshd
Version of Citadel, if applicable: --

@Marcocanc can you help me reproduce this? I couldn't find you on any Discord or Slack servers I'm a member of, but I'd love to get in touch about this.

I just spun up a Discord for Cilicon (will add it to the readme at some point). We can chat there! https://discord.gg/g2yyEMMx

@Marcocanc I'm not sure if this is related to the library, but we're seeing crashes on Sequoia when Cilicon runs under a launchd agent, and the privacy prompt for local network access isn't approved. Since local network access isn't part of TCC, we have to log in to each machine and manually approve and then the crashes stop.
The issue is described here:
https://mjtsai.com/blog/2024/10/02/local-network-privacy-on-sequoia

@temap thanks for the info. It indeed seems to be related to the local network permissions being flaky. I can replicate the behaviour by manually disabling local network access in settings during run. No error is thrown, the stream just stops and doesn't recover.


demo1:RuntimeError

hello.I am alread success run the demo2.But when I run the demo1.the error is
Traceback (most recent call last):
File "./demos/web/websocket-server.py", line 72, in
align = openface.AlignDlib(args.dlibFacePredictor)
File "/home/lyz/openface-master/demos/web/../../openface/align_dlib.py", line 89, in init
self.predictor = dlib.shape_predictor(facePredictor)
RuntimeError: Unable to open /home/lyz/openface-master/demos/web/../../models/dlib/shape_predictor_68_face_landmarks.dat
,who can help me! Thank you very much.

Hi @shakevincent - does the file /home/lyz/openface-master/demos/web/../../models/dlib/shape_predictor_68_face_landmarks.dat exist?

Hi @bamos i am sure the file exist.  I get the model again  the demo1 alread success.thank you very much.have a good day

Hi @shakevincent - I'm not sure what could be causing this because the dlib initialization code is identical between the comparison and web demos and only depends on the string to the dlib model's location on disk. The latest web demo is also working well on my OSX machine and in a Linux Docker container. This could be a setup issue or an inconsistency between our systems I've overlooked.
Can you try using the Docker version?
Can you try the following from a Python 2 interpreter?
import dlib
predictor = dlib.shape_predictor("/home/lyz/openface-master/demos/web/../../models/dlib/shape_predictor_68_face_landmarks.dat")

Are you running the web demo as a different user that doesn't have permissions to the dlib model file?
-Brandon.

Closing due to inactivity.

Hello bamos i had the same error as @shakevincent  , i'm actually doubting the opencv version i am using is incompatible with the one openface was made with i am using opencv 3.2 ,python 2.7 ,am i getting this right ?

i tried the following from a Python 2 interpreter as you previously said :
import dlib
predictor = dlib.shape_predictor("/home/lyz/openface-master/demos/web/../../models/dlib/shape_predictor_68_face_landmarks.dat")

i get this error : RuntimeError: Unable to open /home/lyz/openface-master/demos/web/../../models/dlib/shape_predictor_68_face_landmarks.dat

Hi @deepmore1, does that file exist?

Hi Brandon ,
Thank for your response, i actually worked it out,the fille didn't exist
and then i reinstalled dlib's dependencies and it worked just well now i'm
reading your discussion about the unknown faces detection because it's the
most important aspect regarding the project i'm working on.
i worked before using openface on opencv with c++ for face recognition and
managed to pull out unknown person's identification with a certain
confidence value,in a way that unknown persons are never known and the
known one's recognition is a bit worse .
so how far did you achieve regarding that matter ?
Cheers
2017-04-28 13:49 GMT+01:00 Brandon Amos<EMAIL_ADDRESS>
Hi @deepmore1 https://github.com/deepmore1, does that file exist?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cmusatyalab/openface/issues/76#issuecomment-297989205,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Aa1fzLd967DHzO_cYIfp1nbQHhG5AU1uks5r0eBMgaJpZM4G-lTg
.

I have the same problem:  How did you solved it.
I got this error:
RuntimeError: Unable to open /Users/edgaracosta/openface/openface/demos/../models/dlib/shape_predictor_68_face_landmarks.dat
I have Mac OSX Sierra

I realize that inside.   /models/dlib/
the   fileshape_predictor_68_face_landmarks.dat.
is Missing!!!!

Met the same problem. How is this fixed?

[INFO] loading facial landmark predictor...
Getting Runtime error:
Unable to open 68 face landmarks.dat

I was facing this same problem on my Windows laptop running Python 3.6. All I did to solve this, was add the complete path location inside the single quotes even though my console directory was the same folder that consisted this file.

@anujnadig168  Hi, I am studying Deep learning: Face recognition in Lynda, and face this same problem on my Windows laptop, could you please told me how you solved it?

Hi I faced the same problem in macos . Unable to open /Users/ankurpal/Documents/open_cv_faces/cv4faces/common/shape_predictor_68_face_landmarks.dat
I have given correct path where file (shape_predictor_68_face_landmarks.dat ) exist .Dlib installed properly and file exist in my case . what could be the main problem /?

Getting a new error how to solve this?,
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

Solved by changing the folder location in the .py file to the correct location of shape_predictor_68_face_landmarks.dat
alternatively you can move the shape_predictor_68_face_landmarks.dat file to the default folder

@SidharthRai  Hey may I ask did you solve the problem? I am getting the same error and could not fix it.I am pretty sure the file is in the path. Thank you very much!

for my project, I placed the files in a same directory and executed the command according to my program. Those are called args (arguments).
(i use windows 10)
command.
python detect_drowsiness.py --shape_predictor shape_predictor_68_face_landmarks.dat --alarm alarm.wav --webcam 0

Добрый день. Вы можете подсказать, что это за файл, shape_predictor_68_face_landmarks.dat? Появился на смартфоне. Его создаёт какое-то определённое приложение? Или его создают разные приложения?

##########This may help you##########
1.Find your api.py file path (eg.C:\Python39\Lib\site-packages\face_recognition\api.py) in your python Interpreter path.
2.Change line 21 "pose_predictor_68_point = dlib.shape_predictor(predictor_68_point_model)" to
pose_predictor_68_point = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
3.pyinstaller you_python_file.py
4.copy "shape_predictor_68_face_landmarks.dat" to dist dir。
Note that error is in C:\Python39\Lib\site-packages\face_recognition\api.py on line 21,good luck!


Initial hack at migration guide

@matrixbot test this please

Someone who knows more about the /sync etc should probably take a look too.

@richvdh PTAL

sorry, a few more things... I realise this feedback would have been useful two days ago :(

Done all but the very final one.

lgtm


Add graylog alert: Non-internal IP range used for swarm

Add warning for error occuring when non-internal IPs are used for the swarm workers/managers
We occasionally logged: memberlist: Refuting a suspect message (from: 64a553059e2a) on aws prod
See: https://stackoverflow.com/questions/29156840/consul-cluster-frequently-failed

Adressed by changes from @YuryHrytsuk


Reject high-cost requests instead of creating more OS threads when overloaded

We have been using a file removal semaphore with weight 5,000 (half of Go's default 10,000 maximum OS threads, beyond which Go will crash), in an attempt to avoid crashing when the filesystem/storage layer can't keep up with our requirements.
This change renames that semaphore to diskWaitSem and also uses it for disk-write operations. When the semaphore cannot be acquired for disk-writes, we return HTTP 503 (service unavailable) or gRPC RESOURCE_EXHAUSTED error codes to the client.
Relates to #638

@ulrfa: something for you to try out...


Hiding scrollbars for screenshots

This was mentioned here https://github.com/graphcool/chromeless/issues/92 as part of a "full page screenshot option". But there was very little discussion specifically about hiding scrollbars. Is there currently a way to do this?
I'm trying to take screenshots of how things will look on a mobile device, and the scrollbars change the actual dimensions of the page.
Here's an example:

I'm new to both Lambda and Serverless. But I did a little more poking around to try and find where chrome is actually invoked, because i saw that there was a setting for scrollbars there. It looks like the plugin that actually invokes chrome already has scrollbars disabled as a default: https://github.com/adieuadieu/serverless-chrome/blob/d1eda540b98225fcfb0238ca6d32016ffc7d7cc6/src/config.js#L11
I'm not quite sure how to further debug this at the moment.

One quick hack to remove scrollbars is to set the viewport height longer than the page itself.

It's a good thought, but I imagine there are edge cases where that won't work very well (thinking of infinite scroll, etc).

I don't have scrollbars with the setup I have... I am using Google Puppeteer which is a variation of this repo... not sure if that is the reason why or not.

I'm using Puppeteer w/ Lambda... check the issues in that repo, there are a few how-tos.

That seems like a potential option, but sticking with chromeless seems like it will really save me a lot of work, so I think I'm hoping to stick to this.

This is definitely an issue, or at least should be configurable. The chrome protocol for handling this can easily remove scrollbars. qq: does puppeteer expose this in their API?

It looks like puppeteer disables scroll bars by default.
If I'm understanding everything, it seems like chromeless, depends on serverless, which depends on a plugin to actually start google chrome. At that point it's fired with the exact same --hide-scrollbars argument. Unless I missed something they should both be fired with the same argument, so I'm not sure where this is coming from.

I'm having a bit of trouble testing this in https://github.com/graphcool/chromeless/pull/404, but it looks like we could just add the --hide-scrollbars flag.

--hide-scrollbars was removed from Chromium several years ago: https://bugs.chromium.org/p/chromium/issues/detail?id=350136.

Oh, maybe it was brought back just for headless mode: https://github.com/chromium/chromium/blob/d91428bfde320278621f8a9346d2e84c108b81a9/headless/app/headless_shell_switches.cc#L43.

As a workaround you can use the .evaluate function
.evaluate(() => {                     document.querySelector('html').style.overflow = 'hidden'; // hide scrollbar })
Kinda works this way...


[MXNET-307] Add tutorial tests to the CI

Description

Adding the tutorials to the CI
Adding a CI setting to set container shared memory size --shm-size
Adding a sanity check to tutorials:

Every tutorial needs to be downloadable (except whitelist)
Every tutorial needs to be tested (except whitelist)

Future improvements:

Currently, only sequential run (multiprocessing run is flaky and can leave orphan processes running)

❄ Flakiness:

cold image build -> Marco will improve the image build system

Scala install script (added some randomness which improved the situation)
R install script

mldata.org for MNIST data is flaky, failing the GAN tutorial ->using gluon dataset (fixed)

@marcoabreu , if you would like to review thanks!
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

@marcoabreu addressed your comments

@indhub could you have a look at the updated tutorials as per @marcoabreu comment?


Completely full BitBuffers add an extra byte when converted to byte arrays

The example code below prints out a length of 2 when its expected to be 1:
BitBuffer buffer = BitBuffer.allocate(8);
buffer.putByte((byte) 128, 8);
System.out.println(buffer.asByteArray().length);

Should soon be downloadable from maven central as 1.5.1. Thanks for the report!

@magik6k Awesome, thanks for the quick fix. Didn't even realize you published to maven. You should add some of that information to the README.


Error while Deploying Bot

Operation Outcome

@sujitha2023 - what version of the SDK are you on?  If it is not the latest, could you update it, pull and run npm ci and see if it works?

I'm not able to enable the bot. Can you help me with it?


Guard logic for fraud proofs

Guard should handle the following cases:
Fraudulent Attestation

Use scribe to listen for AttestationAccepted event from LightInbox.submitAttestation()
Call Inbox.verifyAttestation() on summit to evaluate whether attestation is fraudulent. Note that we can avoid spending gas on this by registering AttestationSaved events to determine if the attestation exists beforehand
If invalid, call LightInbox.submitAttestationReport() on summit

Fraudulent Snapshot by a Guard/Notary

Use scribe to observe bad state snapshot on remote chain (SnapshotAccepted event in Inbox.submitSnapshot())
Check whether the snapshot state is valid on the origin chain (StateHub.isValidState())
Register the fraud by calling (StatementInbox.verifyStateWithSnapshot()) on origin chain
Alert about the pending fraud on summit, origin, and all other remote chains by calling StatementInbox.submitStateReportWithSnapshot() (maybe openDispute() ?)

Fraudulent Attestation Fraud Report

Listen for DisputeOpened event from call from guard to LightInbox.submitAttestationReport()
Verify that the fraud accusation is valid by calling verifyAttestationReport() ?

Guard pseudocode:
event = listenForEvents()
switch event.type {

case SnapshotAccepted:
  - for each state in snapshot, guard checks origin.IsValidState() on origin chain.
  - if invalid, guard signs snapshot payload, and calls LightInbox.VerifyStateWithSnapshot() on origin to slash
  the invalid snapshot.
  - finally, guard calls Inbox.SubmitStateReportWithSnapshot() on each destination domain to report the fraud to other chains.

case AttestationAccepted:
  - guard calls Summit.IsValidAttestation() on summit.
  - if invalid, guard calls Inbox.VerifyAttestation() on the summit to slash the fraudulent agent.
  - finally, guard calls SubmitAttestationReport() on the destination chain to make the fraudulent agent known.

case StatusUpdated:
  - guard checks if the status is Fraudulent
  - if Fraudulent, guard calls LightManager.updateAgentStatus() to change status to Slashed.
  - question: _what is the best way to fetch the proof argument for updateAgentStatus()?_

case DisputeOpened:
  - guard parses DisputeOpened.disputeIndex and calls AgentManager.getDispute() to fetch `reportPayload`
  - guard determines where the payload corresponds to a snapshot or attestation
  - if attestation:
    - guard calls Summit.IsValidAttestation() on summit to verify whether the report is valid
    - if invalid, guard calls Inbox.VerifyAttestation() on summit to slash the fraudulent guard.
  - if snapshot:
    - call StatementInbox.verifyStateReport() on origin to verify whether the report is valid
}

Pending questions:

What role do disputes play? i.e. when are they opened / closed?
Given a reportPayload returned by getDispute(), how can we tell if it corresponds to a snapshot or attestation?
Do receipts play any role at the moment?


Add On-boarding page

The app requires onboarding page.
It should be of around 3-5 page. A continuous story should be conveyed using nice interaction and animation.

Added on-boarding page with sample icons and texts.

Added


DSJAS doesn't load when visiting localhost after following install directions. Can you please make an install video?

I saw that on your YouTube channel you had a few videos discussing the features but none showing how to properly install and configure DSJAS. I'm trying to install it on a linux VM and nothing I try works. I've followed the instructions step by step, starting from the beginning each time. I've done a full clean linux install. I know everything is up to date. Might be missing some dependencies but none that are specified in the instructions. A video walkthrough would be an absolute godsend. I don't know what I'm doing wrong and the current documentation is of no help.

Do you have a server up and running? It sounds like your server isn't actually running. Try following the instructions in the order laid out here. It shows you how to install MAMP (I recommend this one if you're a beginner with web servers) and start it out of the right directory.
Be sure to download from the releases page, not from the source code. Unless you want to write code for the project, there's no need to download the source.

I figured out a workaround. Instead of trying to run it in a VM, I got it running it locally using MAMP. Just had to bash my head against the wall for a little longer.


With new commits the produce .tex file does not compile

I see you are still working on this, but this just to warn you, in case you didn't notice, that with the last updates MaTeX is not working anymore on (my?) Windows 10 + MMA v10.4.
Even a simple MaTeX[123] produces a .tex file that does not compile. Here is the produced .tex file as an example:
\documentclass[12pt, border=1pt, multi]{standalone}

\newenvironment{matex}{\ignorespaces}{\ignorespacesafterend}

\standaloneenv{matex}

\newbox\MaTeXbox

\newcommand{\MaTeX}[1]{

	\begin{matex}

		\setbox\MaTeXbox\hbox{%

			\strut%

			\(%

				\displaystyle%

				#1%

			\)}%

		\typeout{MATEXDEPTH:\the\dp\MaTeXbox}%

		\typeout{MATEXHEIGHT:\the\ht\MaTeXbox}%

		\typeout{MATEXWIDTH:\the\wd\MaTeXbox}%

		\unhbox\MaTeXbox%

	\end{matex}

}

\usepackage[utf8]{inputenc}

\usepackage{lmodern,exscale}
\usepackage{amsmath,amssymb}

\begin{document}

\fontsize{12pt}{14.4pt}\selectfont

\MaTeX{123}

\end{document}

Trying to compile this gives me the errors:
C:\Users\test\test.tex:46: Missing $ inserted. [\MaTeX{123}]
C:\Users\test\test.tex:46: Missing $ inserted. [\MaTeX{123}]
C:\Users\test\test.tex:46: Missing $ inserted. [\MaTeX{123}]
C:\Users\test\test.tex:46: LaTeX Error: Bad math environment delimiter. [\MaTeX{123}]

I'm not really that expert in tex and I'm not sure how \strut exactly works, but the problem seems to be caused by it
Reverting MaTeX back to the version 1.4.0 makes it work again just fine, so it doesn't look like a problem in my tex configuration.

Hi Luca,
Thank you for the warning.
MaTeX 1.6.0 is not released yet. I was planning to release it tomorrow.  I'll make sure to resolve this before the final release.
Can you let me know the following:

Are all these empty lines really in the .tex file?  How did you obtain the .tex file?  Did you use the "TeXFileFunction" option?

What TeX system are you using?

I'll get the chance to test on Windows later today. On OS X and Linux everything seems to work fine.

Hi szhorvat,
I actually didn't notice the existence of that option, so I just added the following line just before the temporary tex file is created:
Export["C:\\Users\\test\\test.tex", template[content], "Text", CharacterEncoding -> "UTF-8"];

I used the same options used to create the temporary file to obtain the same output, but maybe I didn't because trying now with MaTeX["123", "TeXFileFunction" -> Print] I get the following output
\documentclass[12pt, border=1pt, multi]{standalone}

\newenvironment{matex}{\ignorespaces}{\ignorespacesafterend}

\standaloneenv{matex}

\newbox\MaTeXbox

\newcommand{\MaTeX}[1]{

	\begin{matex}

		\setbox\MaTeXbox\hbox{%

			\strut%

			\(%

				\displaystyle%

				#1%

			\)}%

		\typeout{MATEXDEPTH:\the\dp\MaTeXbox}%

		\typeout{MATEXHEIGHT:\the\ht\MaTeXbox}%

		\typeout{MATEXWIDTH:\the\wd\MaTeXbox}%

		\unhbox\MaTeXbox%

	\end{matex}

}

\usepackage[utf8]{inputenc}

\usepackage{lmodern,exscale}

\usepackage{amsmath,amssymb}

\begin{document}

\fontsize{12pt}{14.4pt}\selectfont

123

\end{document}

Notice that now the \MaTeX command is gone. This tex file now compiles just fine, but no math environment is used so that the tex file generated by something like MaTeX["123", "TeXFileFunction" -> Print] still doesn't compile.

Thanks for checking this.  Which version of Mathematica are you using?  Can you post {$Version, $ReleaseNumber}?
I know you have 10.4, but I need to know if it's 10.4.0 or 10.4.1 and whether it's 32-bit or 64-bit.
Also, just in case: what language/locale is your Windows set to?  US English?  Italian?
If FileTemplate is buggy then all these details are relevant.
Finally, you said you used 1.4.0 before.  Then you haven't tried 1.5.0 at all?  I added FileTemplate in version 1.5.

Thank you for the great tool!
Version and operative system are {"10.4.0 for Microsoft Windows (64-bit) (February 26, 2016)", 0}.
The language (the one shown in the Time&Language settings) of my Windows is UK English, while the keyboard language is US english (though I don't think the keyboard language has any relevance on this). Windows can be quite funny about the language settings, though, so that there are some random things in the system that still show up in Italian, due to some obscure and hidden language setting that seems to be still set up in italian, and that I didn't really ever bothered to track down.
I hadn't use the 1.5.0 before, but reverting to it I see that the problem was also present in that version.

Thanks again!  One more question:
I develop MaTeX on OS X, so there is no \r\n in template.tex, only \n.  How did the \r get into yours?  Did you edit the template?  Or did you check MaTeX out using git and perhaps git auto-converted the line endings to Windows format?
Either way, it is not acceptable for template.tex not to work with CR-LF line endings, so this will be fixed.
I looked at the source of FileTemplate and it appears to use BinaryReadList to read the file, which explains why it retains the \r characters.  I will report this to Wolfram once I get the chance to test everything on Windows.

Yes I believe, and I checked with an hex editor, that it is git that automatically handles line endings based on the operative system (see e.g. the relevant page on github.com). Interestingly, there is a way to override this behaviour by using a .gitattributes file to enforce a specific line-ending behaviour.

Fixed by<PHONE_NUMBER>bf2a982a6d3e7be8f17b5b62e71886

1.6.0 is released now.  It also has CR/LF and encoding related fixes to error reporting on Windows.


AWS Bedrock support

Could you please add support for AWS Bedrock?

Well, I see it is already in. Cool!


:lady_beetle:  Disk transfer tasks aren't completed although the migration is successful

Ref: https://issues.redhat.com/browse/MTV-1310
Issue: Disk transfer tasks aren't completed although the migration is successful
The individual tasks of the pipeline step do not finish, although the pipeline step is completed
As a workaround, if the UI see that the step is finished, but some individual task did not finish, override the individual task data, and show it as if it is finished.

addressed by #1324


Can i collect tweets together with its user info?How to include cashhag in the search?

When I use .Search to scrape the tweets, can I get the information of the user who post this tweet together with the tweet post in the same row? the information of the user includes 'verified or not', 'number of followers', 'number of followings', etc.
Moreover, im trying to only collect tweets with cashtags. I type this in my terminal:
for i in ‘$BRK’ ‘$BRKB’
do
twint -s $i --since 2018-01-20 --until 2018-01-27 --hashtags --lang en -o test_$i.csv --csv
done

it only output tweets between 2018-01-25 and 2018-01-26, having other days missing, while if i only search for BRK without $, it will not have this problem. I checked in the BRK file and filtered with $, there're tweets containing $BRK all the way since 2018-01-20 until 2018-01-27.
I don't know why this will happen.
Thanks in advance!

It seems that Twitter does not handle requests for cashtags, they simply return a message containing "something went wrong". Maybe The request's URL needs to be tuned.
Also it's not possible to get the information about the users that posted tweets at the same time that you get those. You have to first get the tweets and then, for each user, get the info


[DO NOT MERGE] Final bits of Email Status

Do not merge until https://github.com/alphagov/email-alert-api/pull/555 is
deployed and run in production
This switches the archiving/deleting process to use the new Email status field

Need the schema update from https://github.com/alphagov/email-alert-api/pull/556 here too

@rubenarakelyan this should be sorted now


Wheel that is compatible with numpy 2.0.dev ?

Hi. My package requires another package that pulls in this package. I am unable to test my package using numpy 2.0.dev because of this:
glue/core/data_factories/helpers.py:30: in <module>
    from glue.core.component import CoordinateComponent
glue/core/component.py:5: in <module>
    import shapely
shapely/__init__.py:1: in <module>
    from shapely.lib import GEOSException  # NOQA
E   ImportError: numpy.core.multiarray failed to import

Are you able to provide a dev wheel that is built with numpy 2.0.dev from https://anaconda.org/scientific-python-nightly-wheels/numpy ?
Also see:

https://github.com/numpy/numpy/issues/24300

xref https://github.com/spacetelescope/jdaviz/pull/2572 and https://github.com/glue-viz/glue/issues/2469

@pllim sorry for the slow reply!
At the moment, we haven't set up nightly wheels that are available on a pypi index like https://anaconda.org/scientific-python-nightly-wheels/numpy (I did now ask for access (https://github.com/scientific-python/upload-nightly-action/issues/63), but cannot say anything about a timeline to get that up and running)
The main branch is building its wheels now with numpy 2.0, though, so if you download the necessary artifacts manually from https://github.com/shapely/shapely/actions/runs/7871806219 (the release workflow for currently the last commit on main), this should now work.
(of course, I know for automated testing that is not ideal, but at least it should already allow to use such wheel locally)

See also the overview issue I opened a while ago at https://github.com/shapely/shapely/issues/1972. Will close this one in favor of that (and add a TODO to upload nightly wheels there)

Thank you for your response and links! Good luck!

For future reference if people end up finding this issue: we now have nightly wheels uploaded to the Scientific Python index, so you can pip install nightly wheels, see https://github.com/shapely/shapely/pull/1999 for the installation instructions that are being added to the docs.


Occasional Jerky motion

UR5 running ployscope 3.2.18744. very minimal additional end effector mass, ubuntu 14.04 VM on windows host and ros-indigo. Both with test_move.py and when using the moveit_commander interface to plan and execute trajectories to ee goal positions, the arm occasionaly exhibits the odd jolt during execution, particularly for longer paths, when many joints are moving. Here is an example video:
Video
You see the jolt at roughly 3 seconds. This was performed with the moveit_commander interface.
Here is the output on follow_joint_trajectory/goal, that corresponds with the above video.
header:
  seq: 9
  stamp:
    secs:<PHONE_NUMBER>
    nsecs: 202095307
  frame_id: ''
goal_id:
  stamp:
    secs:<PHONE_NUMBER>
    nsecs: 202096304
  id: /move_group-10-1465575469.202096304
goal:
  trajectory:
    header:
      seq: 0
      stamp:
        secs: 0
        nsecs:         0
      frame_id: /world
    joint_names: ['elbow_joint', 'shoulder_lift_joint', 'shoulder_pan_joint', 'wrist_1_joint', 'wrist_2_joint', 'wrist_3_joint']
    points:
      -
        positions: [-0.8290494124041956, -2.5253475348102015, -0.8362420240985315, 0.7118555307388306, -1.584247891102926, -1.302772347127096]
        velocities: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        accelerations: [0.0, 0.0, -0.3327029212965289, 0.0, 0.0, 0.0]
        effort: []
        time_from_start:
          secs: 0
          nsecs:         0
      -
        positions: [-0.8957262914938168, -2.4424336433522873, -0.8870695181020145, 0.814757601131566, -1.5826221988115585, -1.3017115409826472]
        velocities: [-0.120625527129188, 0.15, -0.0919523130137044, 0.18616072997544747, 0.0029410493153477525, 0.0019191105238146062]
        accelerations: [0.0, 0.0, 3.7659579135098096e-16, 0.0, -7.27300622046582e-16, -7.269083347639247e-16]
        effort: []
        time_from_start:
          secs: 0
          nsecs: 552759276
      -
        positions: [-0.9624031705834379, -2.359519751894373, -0.9378970121054974, 0.9176596715243015, -1.5809965065201912, -1.3006507348381986]
        velocities: [-0.12062552712918757, 0.15, -0.09195231301370407, 0.18616072997544697, 0.0029410493153477443, 0.001919110523814601]
        accelerations: [1.5565959375840504e-15, 0.0, 8.787235131522865e-16, -1.8076597984847034e-15, 6.982713631299419e-16, 7.080785451963737e-16]
        effort: []
        time_from_start:
          secs: 1
          nsecs: 105518552
      -
        positions: [-1.029080049673059, -2.2766058604364585, -0.9887245061089802, 1.020561741917037, -1.5793708142288236, -1.2995899286937498]
        velocities: [-0.12062552712918767, 0.15, -0.09195231301370417, 0.18616072997544697, 0.0029410493153477443, 0.001919110523814802]
        accelerations: [-1.9080853428449648e-15, 0.0, -1.2553193045032663e-15, 1.8076597984847034e-15, -6.982713631299419e-16, 1.8829789567548997e-17]
        effort: []
        time_from_start:
          secs: 1
          nsecs: 658277829
      -
        positions: [-1.0957569287626803, -2.1936919689785443, -1.0395520001124632, 1.1234638123097724, -1.5777451219374563, -1.298529122549301]
        velocities: [-0.120625527129188, 0.15, -0.09195231301370452, 0.18616072997544747, 0.0029410493153477525, 0.0019191105238146062]
        accelerations: [7.280851966118966e-16, 0.0, 0.0, 0.0, 7.27300622046582e-16, -7.269083347639247e-16]
        effort: []
        time_from_start:
          secs: 2
          nsecs: 211037105
      -
        positions: [-1.1624338078523013, -2.11077807752063, -1.0903794941159461, 1.226365882702508, -1.5761194296460888, -1.2974683164048524]
        velocities: [-0.1206255271291878, 0.15, -0.09195231301370432, 0.18616072997544728, 0.0029410493153477525, 0.0019191105238146062]
        accelerations: [0.0, 0.0, 7.280851966118966e-16, -7.029788105218311e-16, -7.27300622046582e-16, 7.269083347639247e-16]
        effort: []
        time_from_start:
          secs: 2
          nsecs: 763796381
      -
        positions: [-1.2291106869419224, -2.027864186062716, -1.1412069881194289, 1.3292679530952431, -1.5744937373547214, -1.2964075102604036]
        velocities: [-0.12062552712918784, 0.14999999999999997, -0.09195231301370418, 0.18616072997544722, 0.002941049315347748, 0.0019191105238148044]
        accelerations: [-1.255319304503268e-16, -5.0212772180130727e-17, -2.510638609006536e-16, 5.021277218013072e-16, 7.123937053056047e-16, -9.807182066431782e-18]
        effort: []
        time_from_start:
          secs: 3
          nsecs: 316555658
      -
        positions: [-1.2957875660315437, -1.9449502946048016, -1.1920344821229119, 1.4321700234879788, -1.5728680450633539, -1.2953467041159548]
        velocities: [-0.12062552712918787, 0.14999999999999997, -0.09195231301370425, 0.18616072997544697, 0.002941049315347744, 0.001919110523814601]
        accelerations: [0.0, 0.0, 0.0, -1.456170393223789e-15, -7.2730062204658e-16, -7.269083347639226e-16]
        effort: []
        time_from_start:
          secs: 3
          nsecs: 869314934
      -
        positions: [-1.362464445121165, -1.8620364031468872, -1.2428619761263948, 1.535072093880714, -1.5712423527719865, -1.2942858979715062]
        velocities: [-0.12062552712918784, 0.14999999999999997, -0.09195231301370438, 0.18616072997544703, 0.002941049315347748, 0.0019191105238146036]
        accelerations: [1.255319304503268e-16, 5.0212772180130727e-17, -4.770213357112419e-16, 1.657021481944314e-15, 7.422075387875573e-16, 7.367155168303555e-16]
        effort: []
        time_from_start:
          secs: 4
          nsecs: 422074211
      -
        positions: [-1.429141324210786, -1.779122511688973, -1.2936894701298778, 1.6379741642734496, -1.569616660480619, -1.2932250918270574]
        velocities: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        accelerations: [0.4364486758786933, -0.542731732991224, 0.3327029212965289, -0.6735689039632393, -0.010641338611542983, -0.006943747869278039]
        effort: []
        time_from_start:
          secs: 4
          nsecs: 974833487
  path_tolerance: []
  goal_tolerance: []
  goal_time_tolerance:
    secs: 0
    nsecs:         0
---

Hmmm, it looks strange - Almost like a few network packages are dropped/buffered and transmitted at the same timr.
I don't have any experience using ROS with WM on a windows host, so I don't know if windows might use all of the NIC's resources for a few miliseconds.
Could you perhaps try it on a non-WM machine? or set up another machine with wireshark that monitors the network and then plot the timestamp when data is sent from the computer to the controller. (It should be 28 bytes transmitted at 125 Hz from the computer to the controller).

Thanks for the advice. I will have a look into it. No experience of wireshark, but immediately I'm inclined to think I'll run into problems due to working on a university network.
Do you have any experience of running ROS with VM on any other host, - mac for instance?
Thanks,
Nick

Working with a Motoman robot, I had severe problems with package losses. I solved this by inserting an extra network card in the lab PC and having a direct network link between the PC and the robot. Our URs are on the same internal network as the Motoman initially was on (A network dedicated to robots and ethernet-connected sensors and actuators), and haven't had any issues with that. But it obviously depends on the network load.
Try connecting directly to the robot with a cable.
I haven't used VM very much, so I don't know if lags occurs on the network. I have always just used dedicated Ubuntu machines, or dual boot on the machines that required windows as well.
If you can't dedicate a full PC to running ubuntu, you could run the driver on a raspberry pi or similar, and then have Rviz/moveit running on your laptop with the laptop configured as the ros-master.

Thanks Thomas. Sorry for the late reply. I've now fully tested on a dedicated Ubuntu machine and on a separate dedicated network. As far as I can tell you were correct; the robot seems to move smoothly when controlled from a physical machine. I haven't been able to analyse the connection with wireshark, but I can say that by pinging the robot from both physical and virtual machines the connection seems almost twice fast from the physical machine, so I wonder if this has anything to do with it. Anyway thanks again for your help!

Hi Thomas,
Thank you so much for all your help. You were correct that it was down to
using a VM! I now have to set up a dedicated Ubuntu machine or a dual boot
system. I would really appreciate some advice on the hardware requirements
for the PC.
I have a free PC available to me which is 64-bit, dual core (3.3GHz) with
4GB of ram, which is currently running windows 7. For various reasons I
ideally need to have a good idea as to whether this hardware is going to
cut it before I go ahead with the Ubuntu instillation. I'm going to be
running matlab programs along side all the ros nodes required to control
the arm with moveit.
What would you suggest? Is it likely that I'm going to need something with
more cores or more ram to avoid having to switch PCs again further down the
line?
Any help you can offer is greatly appreciated!!
Thanks,
Nick
On 14 June 2016 at 12:41, Thomas Timm Andersen<EMAIL_ADDRESS>wrote:

Working with a Motoman robot, I had severe problems with package losses. I
solved this by inserting an extra network card in the lab PC and having a
direct network link between the PC and the robot. Our URs are on the same
internal network as the Motoman initially was on (A network dedicated to
robots and ethernet-connected sensors and actuators), and haven't had any
issues with that. But it obviously depends on the network load.
Try connecting directly to the robot with a cable.
I haven't used VM very much, so I don't know if lags occurs on the
network. I have always just used dedicated Ubuntu machines, or dual boot on
the machines that required windows as well.
If you can't dedicate a full PC to running ubuntu, you could run the
driver on a raspberry pi or similar, and then have Rviz/moveit running on
your laptop with the laptop configured as the ros-master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ThomasTimm/ur_modern_driver/issues/56#issuecomment-225856090,
or mute the thread
https://github.com/notifications/unsubscribe/AKPqNLR3gzOauoLs4tTeLYq8zelzPS0Lks5qLpNXgaJpZM4IzKAC
.

--
*Nick Pestell | *PhD Researcher
nick-pestell.com http://nick-pestell.com/

Well, making suggestions is rather difficult for me, as I don't know your setup or future plans. But unless you are planning on handling large data later on (like images or point clouds), your current setup should suffice.
As an alternative, the driver itself uses very few resources and as such could be running on a cheap raspberry pi or similar. Then you could connect the RPI and your VM machine, as the moveit-part is not so timing critical.

Thanks for the advice!


Limit cpu cores used when generating changelog

Which issue does this PR close?

Closes https://github.com/apache/arrow-datafusion/issues/2487
Rationale for this change

Could not generate change logs without this change
What changes are included in this PR?

Limit CPU cores used by docker to slow the process down. Also added docs.
Are there any user-facing changes?

No

When I hit this previously, I simply moved to a slower internet connection temporarily


Dynamically adding playlist but it only plays only one song on-next-up

const initialState = {
clearPriorAudioLists: false,
spaceBar: true,
params: {},
autoPlay:true,
quietUpdate:true,
songData: [],
defaultPosition: {
bottom: 100,
right: 10
}
}
class AudioPlayer extends React.Component {
constructor() {
super();
this.audio = null
}
state = initialState

componentDidUpdate(prevProps, prevState, snapshot) {
    const myPrevProps = prevProps

    if ((this.props.songData !== myPrevProps.songData)) {
        this.setState({songData: {}, clearPriorAudioLists: true})
        const data = [{
            name: this.props.songData.data.song_title,
            singer: this.props.songData.data.description,
            cover: this.props.songData.data.photo_main,
            musicSrc: this.props.songData.data.audio_file,
        }
        ]
         let dataAddList = []
         dataAddList = this.props.songData.playlist.map(item =>{
             console.log(item)
             return {
                 name: item.song_title,
                 singer: item.description,
                 cover: item.photo_main,
                 musicSrc: item.audio_file
             }
         })
        this.setState(initialState)
        this.setState({songData: data, clearPriorAudioLists: true,quietUpdate: true, spaceBar: true,autoPlay:true})

    }

render() {
    return (
        <div ref={this.focusPlayer}>
            <ReactJkMusicPlayer className="music-player-custom" defaultPosition={this.state.defaultPosition}
                                audioLists={this.state.songData}
                                loadAudioErrorPlayNext={true}
                                responsive={false}
                                autoPlay={this.state.autoPlay}
                                remove={false}
                                spaceBar={this.state.spaceBar}
                                quietUpdate={this.state.quietUpdate}
                                toggleMode={true}
                                mode="full"
                                defaultPlayMode= 'orderLoop'
                                showDestroy={false}
                                showDownload={false}
                                clearPriorAudioLists={this.state.clearPriorAudioLists}
                                showPlayMode={false}
                                getAudioInstance={(audio) => {
                                    this.audio = audio
                                }}
            />
            {}
        </div>
    );

}

}
Description
Steps
/**
 * code
 */

Reproduce Link
Steps to Reproduce
Expected Behavior
Current Behavior

System information
Comments

Need Reproduce Link

Need Reproduce Link

thanks but i got the hang of it i was doing it differently thanks.
On Tue, Jan 5, 2021 at 9:06 AM Jinke Li<EMAIL_ADDRESS>wrote:

Closed #232 https://github.com/lijinke666/react-music-player/issues/232.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lijinke666/react-music-player/issues/232#event-4167791419,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AITNRVEL4LDZSUHCW4XGZSTSYKF4DANCNFSM4UT4QRYA
.

thanks but i got the hang of it i was doing it differently thanks.
On Tue, Jan 5, 2021 at 9:06 AM Jinke Li<EMAIL_ADDRESS>wrote:

Closed #232 https://github.com/lijinke666/react-music-player/issues/232.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lijinke666/react-music-player/issues/232#event-4167791419,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AITNRVEL4LDZSUHCW4XGZSTSYKF4DANCNFSM4UT4QRYA
.


Replace Freemarker template by Groovy markup template

The Freemarker templates could be replaced by a more modern template processor like Groovy markup template that in supported in Spring 4.1.

Note that the 'web-theme' solution would require to be adjusted or changed.


Track Sidekiq failures

A count of failed jobs is displayed in the Sidekiq dashboard, but there's no list to review which items didn't ingest successfully.  Try using https://github.com/mhfs/sidekiq-failures to track those.

@escowles Do we still want this? It seems pretty old and I wonder if it still works. Or did we get to the bottom of it earlier this week? See also #937

Yes, I think we can close this one — it looks like Sidekiq is putting jobs back into the queue and the failures number is just a count of how many failures, not a black hole.


1.0 Update

Can we please get an update for this mod?
Thanks :)

Hi! Sorry for the long wait!
I'll try to do a release within the week or the next one!

@Aragas Is there any news about the mod update?


Find projects that are beginners friendly and add to this awesome list

Check out our Contributing Guide and find awesome projects to add in this list.

i'm adding one


Failure in Telegram setup results in RAM leak

Description:
If you try and fail to configure Telegram access and just leave it there, nchat starts in loop trying to login again and again:
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.862 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.863 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}
"  (tgchat.cpp:2216)
2024-08-05 21:10:06.863 | WARN  | auth error "error {
  code = 401
  message = "Wrong database encryption key"
}

This will go on so long that machine either totally freezes or puts out OOM error and forekills nchat.
How to reproduce it:
Setup Telegram and fail, don't remove config of it
Environment:

Version: nchat v4.86
OS / distro: Arch Linux x86_64, 6.10.2-arch1-1

Additional info
I did not test if failure in Whatsapp does same, nor did I test anything else, this just bugged me for half of year and I wish no one else should suffer from this.

Hi @Lympri - thanks for reporting a bug! Could you share a bit more detailed steps, as I cannot reproduce the problem here. I tried from a clean slate, i.e. using a non-existent nchat config dir, selected Telegram, entered a valid phone number, entered wrong authentication code. Nchat exits directly (i.e. no hang or repeated log entries).
$ nchat -d ~/tmpnchat -e -s
Protocols:
0. Dummy
1. Telegram
2. WhatsAppMd
3. Exit setup
Select protocol (3): 1
Enter phone number (ex. +6511111111): +xxxxxxxxxxxx    (my actual number redacted)
Enter authentication code: 99735
Authentication error: PHONE_CODE_INVALID
Setup failed
$

(I also attempted wrong phone number and wrong password, same results)
Is the above what you did too (wrong authentication code), or something else?
Do you have a single profile, or multiple protocols/profiles set up (like ok WhatsApp + failed Telegram, or other combination)?
Thanks!

(I also attempted wrong phone number and wrong password, same results)
Is the above what you did too (wrong authentication code), or something else?

I cannot login to telegram as it says that I need to update telegram. My telegram-cli should be at newest version
$ pacman -Q --info telegram-cli
Name            : telegram-cli-git
Version         : r1365.7f8f739-1

Do you have a single profile, or multiple protocols/profiles set up (like ok WhatsApp + failed Telegram, or other combination)?

I am not quite sure, I had in profiles folder 2 directories, 1 for WhatsApp (working, but I didn't use it because using it ate all ram)

(I also attempted wrong phone number and wrong password, same results)
Is the above what you did too (wrong authentication code), or something else?

I cannot login to telegram as it says that I need to update telegram. My telegram-cli should be at newest version
$ pacman -Q --info telegram-cli
Name            : telegram-cli-git
Version         : r1365.7f8f739-1

Do you have a single profile, or multiple protocols/profiles set up (like ok WhatsApp + failed Telegram, or other combination)?

I am not quite sure, I had in profiles folder 2 directories, 1 for WhatsApp (working, but I didn't use it because using it ate all ram) and another for Telegram (but it's removed allready).
After that removal WhatsApp has worked like a charm.

I cannot login to telegram as it says that I need to update telegram.

Could you share the error message? The telegram service maintains support for old clients pretty long, so you would need a very old version of nchat to run into that, should definitely not happen in v4.86.

My telegram-cli should be at newest version

nchat does not use telegram-cli. nchat has its telegram client code bundled / built-in.

I am not quite sure, I had in profiles folder 2 directories, 1 for WhatsApp (working, but I didn't use it because using it ate all ram) and another for Telegram (but it's removed allready).

Thanks, this is useful! I will try this scenario.

Was able to reproduce repeated "Wrong database encryption key" log entries now, although in a bit hacky way:

Set up telegram successfully, exit nchat
Remove line starting with local_key= from nchat/profiles/Telegram_+nnnnnnnn/telegram.conf
Start nchat again

Will prepare a fix which avoids the infinite loop in this scenario. I'm still not sure how one can end up in this situation (without manually editing/removing telegram.conf) but will do this fix for now.

The high CPU usage and memory leak on broken Telegram setup should be addressed in above commit. Please let me know if still encountered.


Support of FSA must be optionated, please add config-flag for error field.

I chose redux-act VS redux-actions for this reason.

Sorry but probably no.  Why is the error field bothering you? You can just ignore it and just focus on the payload one if that's what you need.

For reducer:
import { createAction, createReducer } from 'redux-act'

export const REDUCER = 'APP'
const NS = `${REDUCER}__`

export const initialState = {
  isLoading: false,
}

const reducer = createReducer({}, initialState)

export const setLoading = createAction(`${NS}SET`)
reducer.on(setLoading, (state, isLoading) => ({ ...state, isLoading }))

export default reducer

I have a lot of tests like this:
import thunk from 'redux-thunk'
import configureMockStore from 'redux-mock-store'
import { setLoading } from '../app'
import reducer from '..'

const middlewares = [thunk]
const mockStore = configureMockStore(middlewares)

describe('sync ducks', () => {
  it('setLoading()', () => {
    let state = {}
    const store = mockStore(() => state)
    store.dispatch(setLoading(true))
    const actions = store.getActions()
    const expectedActions = [{ type: 'APP__SET', payload: true }];
    expect(actions).toEqual(expectedActions);
    actions.forEach(action => {
      state = reducer(state, action)
    })
    expect(state).toEqual({
      ...state,
      app: { ...state.app, isLoading: true },
    })
  })
})

And now I need to change both expect for error-field value, but I do not use this agreement anywhere.

OK, workaround:
      const actions = store.getActions().map(({ type, payload }) => ({ type, payload }));


add snowmelt timing code

This script downloads SNOTEL data from all currently active SNOTEL sites using the RNRCS package. Data are then used to calculate SM50, the date of snowmelt timing across all sites and each year. This value is subtracted from the median snowmelt date for each site and plotted as the difference in days.

Documentation for the #30DayChartChellenge post on  04/12/2021 under the strip distributions category

Awesome - merge away!


Add ETag header for static responses

Related to https://github.com/simonw/datasette/issues/1645
This PR adds ETag headers for static responses. Adding cache-control headers was previously discussed but nothing done yet, so I've thought adding ETag headers would drastically reduce bandwidth meanwhile. Specially, when quickly deploying to Vercel or Fly, and you don't own a domain, required to add a CDN on front.

📚 Documentation preview 📚: https://datasette--2306.org.readthedocs.build/en/2306/

I've branched from latest main, but black found files to be reformatted.
Oh no! 💥 💔 💥
65 files would be reformatted, 32 files would be left unchanged.

Done, and tests passes. However, there's still a weird runtime error. Checking now.
Traceback (most recent call last):
  File "/home/agustin/tmp/datasette/datasette/app.py", line 1668, in route_path
    response = await view(request, send)
  File "/home/agustin/tmp/datasette/datasette/utils/asgi.py", line 337, in inner_static
    await asgi_send_file(
  File "/home/agustin/tmp/datasette/datasette/utils/asgi.py", line 305, in asgi_send_file
    await send(
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/asgi_csrf.py", line 104, in wrapped_send
    await send(event)
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 514, in send
    output = self.conn.send(event=h11.EndOfMessage())
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/h11/_connection.py", line 512, in send
    data_list = self.send_with_data_passthrough(event)
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/h11/_connection.py", line 545, in send_with_data_passthrough
    writer(event, data_list.append)
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/h11/_writers.py", line 67, in __call__
    self.send_eom(event.headers, write)
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/h11/_writers.py", line 96, in send_eom
    raise LocalProtocolError("Too little data for declared Content-Length")
h11._util.LocalProtocolError: Too little data for declared Content-Length

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/home/agustin/tmp/datasette/datasette/utils/asgi.py", line 477, in __call__
    return await self.asgi(scope, receive, send)
  File "/home/agustin/tmp/datasette/datasette/utils/asgi.py", line 186, in __call__
    await self.app(scope, receive, send)
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/asgi_csrf.py", line 108, in app_wrapped_with_csrf
    await app(scope, receive, wrapped_send)
  File "/home/agustin/tmp/datasette/datasette/app.py", line 1625, in __call__
    return await self.route_path(scope, receive, send, path)
  File "/home/agustin/tmp/datasette/datasette/app.py", line 1686, in route_path
    return await self.handle_exception(request, send, exception)
  File "/home/agustin/tmp/datasette/datasette/app.py", line 1807, in handle_exception
    await response.asgi_send(send)
  File "/home/agustin/tmp/datasette/datasette/utils/asgi.py", line 365, in asgi_send
    await send(
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/asgi_csrf.py", line 104, in wrapped_send
    await send(event)
  File "/home/agustin/tmp/datasette/env/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 520, in send
    raise RuntimeError(msg % message_type)
RuntimeError: Unexpected ASGI message 'http.response.start' sent, after response already completed.

Might be related to how I'm forwarding headers.

Done! fixed.

This is clearly a good idea. Adding it directly to asgi_static() looks like the right place, too. I was initial concerned that this might interfere with how /database.db downloads work - the most performance-sensitive part of the code for this because the file might be 1GB+ and I wouldn't want to have to calculate MD5 against that every time - but those are not served by asgi_static() and in fact it turns out they have ETag support already:
https://github.com/simonw/datasette/blob/261fc8d875c43d5421a6ee1096992fd636c2c278/datasette/views/database.py#L327-L340

Yes. I tried adding it inside asgi_send_file() first, then a related test failed and found out that same part where you added ETag headers for downloading the whole db. Then, I realized it was better to move it outside it and asgi_static() was safe.

I deployed this branch to my own Datasette Cloud instance - here's the before:

And after:

Huge improvement!


Delete mergebot-config.json

Breaking the rules. No JIRA, sorry, but I can't stand those pointless emails anymore. Let's disable mergebot

This repo has @mesosphere-mergebot integration. You can interact with the following commands.
@mesosphere-mergebot integrate
@mesosphere-mergebot ship-it
@mesosphere-mergebot test


Update coda to version 2.5.2

Thank you for the contribution. It will be merged in a different pull request due to some simple recommended alterations. Your contribution is still included (and still credited to you), with the appropriate modifications, when applicable.
Changes:

Squashed commits.


Fix TouchEvent undefined message

Fix TouchEvent undefined check for browsers that not support this functionality

This was already fixed in 0.8.2. Not sure what exactly the benefit of this code over what’s already there would be

Is not fixed at all in Firefox and Safari the console throws an error related to event instanceof window.TouchEvent as Is exposed un issue 383


Domain exceptions instead of Dtos on exceptions

Fixed on latest develop.

We need to create domain exceptions for searcher.