MAPREDUCE-7121. add Abortable so that we can abort output

This patch adds abortable to mapreduce outputs
In the case of using a remote filesystem with mapreduce, upon task attempt failure we ensure the file output stream gets aborted before being closed which prevents partial results from being uploaded.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
37
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
25
Maven dependency ordering for branch

+1
mvninstall
1043
trunk passed

+1
compile
1050
trunk passed

+1
checkstyle
150
trunk passed

+1
mvnsite
122
trunk passed

+1
shadedclient
986
branch has no errors when building and testing our client artifacts.

+1
javadoc
87
trunk passed

0
spotbugs
80
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
191
trunk passed

_ Patch Compile Tests _

0
mvndep
22
Maven dependency ordering for patch

+1
mvninstall
74
the patch passed

+1
compile
1023
the patch passed

-1
javac
1023
root generated 1 new + 1476 unchanged - 0 fixed = 1477 total (was 1476)

-0
checkstyle
147
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

-1
mvnsite
38
hadoop-mapreduce-client-core in the patch failed.

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
635
patch has no errors when building and testing our client artifacts.

+1
javadoc
80
the patch passed

-1
findbugs
122
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
34
hadoop-mapreduce-client-core in the patch failed.

_ Other Tests _

-1
unit
534
hadoop-common in the patch failed.

-1
unit
41
hadoop-mapreduce-client-core in the patch failed.

-1
asflicense
38
The patch generated 2 ASF License warnings.

6522

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

Failed junit tests
hadoop.ipc.TestIPC

Subsystem
Report/Notes

Docker
Client=18.09.8 Server=18.09.8 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux b0653e3166ad 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 7f1b76c

Default Java
1.8.0_212

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/diff-checkstyle-root.txt

mvnsite
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1361 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/1/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
40
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
22
Maven dependency ordering for branch

+1
mvninstall
1033
trunk passed

+1
compile
1013
trunk passed

+1
checkstyle
153
trunk passed

+1
mvnsite
128
trunk passed

+1
shadedclient
956
branch has no errors when building and testing our client artifacts.

+1
javadoc
91
trunk passed

0
spotbugs
76
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
190
trunk passed

_ Patch Compile Tests _

0
mvndep
23
Maven dependency ordering for patch

+1
mvninstall
77
the patch passed

+1
compile
968
the patch passed

-1
javac
968
root generated 1 new + 1476 unchanged - 0 fixed = 1477 total (was 1476)

-0
checkstyle
146
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
113
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
669
patch has no errors when building and testing our client artifacts.

+1
javadoc
95
the patch passed

-1
findbugs
127
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
90
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

+1
unit
532
hadoop-common in the patch passed.

+1
unit
335
hadoop-mapreduce-client-core in the patch passed.

-1
asflicense
48
The patch generated 2 ASF License warnings.

6835

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

FindBugs
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

instanceof will always return false in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:[line 461]

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 264b7516c146 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / c7c7a88

Default Java
1.8.0_212

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/artifact/out/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1645 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/2/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
40
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
22
Maven dependency ordering for branch

+1
mvninstall
1083
trunk passed

+1
compile
1072
trunk passed

+1
checkstyle
146
trunk passed

+1
mvnsite
125
trunk passed

+1
shadedclient
975
branch has no errors when building and testing our client artifacts.

+1
javadoc
104
trunk passed

0
spotbugs
82
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
204
trunk passed

_ Patch Compile Tests _

0
mvndep
24
Maven dependency ordering for patch

+1
mvninstall
78
the patch passed

+1
compile
1020
the patch passed

-1
javac
1020
root generated 1 new + 1476 unchanged - 0 fixed = 1477 total (was 1476)

-0
checkstyle
148
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
122
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
678
patch has no errors when building and testing our client artifacts.

+1
javadoc
98
the patch passed

-1
findbugs
136
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
95
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

-1
unit
532
hadoop-common in the patch failed.

+1
unit
338
hadoop-mapreduce-client-core in the patch passed.

-1
asflicense
52
The patch generated 2 ASF License warnings.

7097

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

FindBugs
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

instanceof will always return false in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:[line 461]

Failed junit tests
hadoop.util.TestReadWriteDiskValidator

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 5753f30a6519 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23 09:17:01 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / e872ceb

Default Java
1.8.0_212

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1561 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/3/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
62
Docker mode activated.

_ Prechecks _

+1
dupname
1
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
22
Maven dependency ordering for branch

+1
mvninstall
1067
trunk passed

+1
compile
1079
trunk passed

+1
checkstyle
147
trunk passed

+1
mvnsite
110
trunk passed

+1
shadedclient
999
branch has no errors when building and testing our client artifacts.

+1
javadoc
80
trunk passed

0
spotbugs
78
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
192
trunk passed

_ Patch Compile Tests _

0
mvndep
28
Maven dependency ordering for patch

+1
mvninstall
74
the patch passed

+1
compile
1022
the patch passed

-1
javac
1022
root generated 1 new + 1476 unchanged - 0 fixed = 1477 total (was 1476)

-0
checkstyle
140
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
106
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
616
patch has no errors when building and testing our client artifacts.

+1
javadoc
82
the patch passed

-1
findbugs
122
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
86
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

+1
unit
546
hadoop-common in the patch passed.

-1
unit
27
hadoop-mapreduce-client-core in the patch failed.

-1
asflicense
40
The patch generated 2 ASF License warnings.

6640

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

FindBugs
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

instanceof will always return false in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:[line 461]

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 59266f8ea938 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 00b5a27

Default Java
1.8.0_222

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1389 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/4/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
67
Docker mode activated.

_ Prechecks _

+1
dupname
1
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
22
Maven dependency ordering for branch

+1
mvninstall
1096
trunk passed

+1
compile
1234
trunk passed

+1
checkstyle
147
trunk passed

+1
mvnsite
108
trunk passed

+1
shadedclient
972
branch has no errors when building and testing our client artifacts.

+1
javadoc
82
trunk passed

0
spotbugs
75
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
199
trunk passed

_ Patch Compile Tests _

0
mvndep
24
Maven dependency ordering for patch

+1
mvninstall
79
the patch passed

+1
compile
1193
the patch passed

-1
javac
1193
root generated 1 new + 1469 unchanged - 0 fixed = 1470 total (was 1469)

-0
checkstyle
141
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
106
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
632
patch has no errors when building and testing our client artifacts.

+1
javadoc
87
the patch passed

-1
findbugs
130
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
81
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

+1
unit
540
hadoop-common in the patch passed.

+1
unit
336
hadoop-mapreduce-client-core in the patch passed.

-1
asflicense
49
The patch generated 2 ASF License warnings.

7331

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

FindBugs
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

instanceof will always return false in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:[line 461]

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 13ad574378fb 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / e356e4f

Default Java
1.8.0_222

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/artifact/out/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1572 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/5/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
42
Docker mode activated.

_ Prechecks _

+1
dupname
1
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
23
Maven dependency ordering for branch

+1
mvninstall
1053
trunk passed

+1
compile
1066
trunk passed

+1
checkstyle
142
trunk passed

+1
mvnsite
103
trunk passed

+1
shadedclient
925
branch has no errors when building and testing our client artifacts.

+1
javadoc
86
trunk passed

0
spotbugs
80
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
199
trunk passed

_ Patch Compile Tests _

0
mvndep
24
Maven dependency ordering for patch

+1
mvninstall
83
the patch passed

+1
compile
1112
the patch passed

-1
javac
1112
root generated 1 new + 1469 unchanged - 0 fixed = 1470 total (was 1469)

-0
checkstyle
147
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
112
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
666
patch has no errors when building and testing our client artifacts.

+1
javadoc
82
the patch passed

-1
findbugs
127
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
83
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

+1
unit
532
hadoop-common in the patch passed.

+1
unit
319
hadoop-mapreduce-client-core in the patch passed.

-1
asflicense
41
The patch generated 2 ASF License warnings.

6971

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

FindBugs
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

instanceof will always return false in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:[line 461]

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux c973e0cd2bd8 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 094d736

Default Java
1.8.0_222

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/artifact/out/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1664 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/6/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
40
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
21
Maven dependency ordering for branch

+1
mvninstall
1090
trunk passed

+1
compile
1095
trunk passed

+1
checkstyle
144
trunk passed

+1
mvnsite
109
trunk passed

+1
shadedclient
946
branch has no errors when building and testing our client artifacts.

+1
javadoc
97
trunk passed

0
spotbugs
73
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
197
trunk passed

_ Patch Compile Tests _

0
mvndep
20
Maven dependency ordering for patch

+1
mvninstall
77
the patch passed

+1
compile
1025
the patch passed

-1
javac
1025
root generated 1 new + 1469 unchanged - 0 fixed = 1470 total (was 1469)

-0
checkstyle
148
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
107
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
641
patch has no errors when building and testing our client artifacts.

+1
javadoc
84
the patch passed

-1
findbugs
125
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
28
hadoop-mapreduce-client-core in the patch failed.

_ Other Tests _

-1
unit
502
hadoop-common in the patch failed.

-1
unit
29
hadoop-mapreduce-client-core in the patch failed.

-1
asflicense
41
The patch generated 2 ASF License warnings.

6554

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

Failed junit tests
hadoop.util.TestDiskChecker

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 2b850bacb076 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 69ddb36

Default Java
1.8.0_222

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1605 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/7/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
69
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
27
Maven dependency ordering for branch

+1
mvninstall
1013
trunk passed

+1
compile
977
trunk passed

+1
checkstyle
157
trunk passed

+1
mvnsite
132
trunk passed

+1
shadedclient
1016
branch has no errors when building and testing our client artifacts.

+1
javadoc
104
trunk passed

0
spotbugs
80
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
199
trunk passed

_ Patch Compile Tests _

0
mvndep
23
Maven dependency ordering for patch

+1
mvninstall
77
the patch passed

+1
compile
925
the patch passed

-1
javac
925
root generated 1 new + 1469 unchanged - 0 fixed = 1470 total (was 1469)

-0
checkstyle
162
root: The patch generated 10 new + 854 unchanged - 2 fixed = 864 total (was 856)

+1
mvnsite
128
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
689
patch has no errors when building and testing our client artifacts.

+1
javadoc
106
the patch passed

-1
findbugs
128
hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1
findbugs
89
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

+1
unit
580
hadoop-common in the patch passed.

+1
unit
333
hadoop-mapreduce-client-core in the patch passed.

-1
asflicense
53
The patch generated 2 ASF License warnings.

6994

Reason
Tests

FindBugs
module:hadoop-common-project/hadoop-common

instanceof will always return true for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:for all non-null values in org.apache.hadoop.io.SequenceFile$Writer.abort(), since all org.apache.hadoop.fs.FSDataOutputStream are instances of org.apache.hadoop.fs.common.Abortable  At SequenceFile.java:[line 881]

FindBugs
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

instanceof will always return false in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:in org.apache.hadoop.mapred.ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, Task$TaskReporter, RawKeyValueIterator, RawComparator, Class, Class), since a org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter<OUTKEY,OUTVALUE> can't be a org.apache.hadoop.fs.common.Abortable  At ReduceTask.java:[line 461]

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/402

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 37a2cf466d46 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 3329257

Default Java
1.8.0_222

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/artifact/out/diff-compile-javac-root.txt

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/artifact/out/diff-checkstyle-root.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/artifact/out/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/testReport/

asflicense
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/artifact/out/patch-asflicense-problems.txt

Max. process+thread count
1608 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/8/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
0
Docker mode activated.

-1
patch
11
https://github.com/apache/hadoop/pull/402 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.

Subsystem
Report/Notes

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-402/9/console

versions
git=2.17.1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.


add pause metric in prometheus for scaledobject

Provide a description of what has been changed
Checklist

[ ] When introducing a new scaler, I agree with the scaling governance policy
[x] I have verified that my change is according to the deprecations & breaking changes policy
[x] Tests have been added
[x] Changelog has been updated and is aligned with our changelog requirements
[ ] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
[x] A PR is opened to update the documentation on (repo) (if applicable)
[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #

Relates to #4430
Relates to https://github.com/kedacore/keda-docs/pull/1111

Hi @zroubalik here is my PR and I made all the fix referenced in #4446

Hi @zroubalik could you please review my PR

/run-e2e sequential

@zroubalik Could you please review my PR when you have the time.
cc @JorTurFer

@zroubalik could you please review my PR again. as i have fixed all suggested changes.
cc @JorTurFer

@geoffrey1330 there are still unresolved comments on this PR

I will go ahead and mark all unresolved comments as resolved.

@geoffrey1330 there are still unresolved comments on this PR

I will go ahead and mark all unresolved comments as resolved.

No, it means that these things should be resolved.

Resolved most of them. will check for the one's I'm yet to resolve.

/run-e2e sequential

/run-e2e sequential Update: You can check the progress here

@geoffrey1330 unfortunately the e2e has failed, have you tried to run it locally?

/run-e2e sequential Update: You can check the progress here

@geoffrey1330 unfortunately the e2e has failed, have you tried to run it locally?

Not yet. Let me try that now.

/run-e2e sequential Update: You can check the progress here

@geoffrey1330 unfortunately the e2e has failed, have you tried to run it locally?

@zroubalik I'm using  make e2e-test-local to test locally any advise on this?

@zroubalik during testing locally, it generated some *.pb.go file. should I go ahead and delete this file or rather leave them?

@zroubalik during testing locally, it generated some *.pb.go file. should I go ahead and delete this file or rather leave them?

Please omit these files from this PR, it is just a change in the version of the tool that generates these files.

@zroubalik during testing locally, it generated some *.pb.go file. should I go ahead and delete this file or rather leave them?

Please omit these files from this PR, it is just a change in the version of the tool that generates these files.

Done

Hi @zroubalik could you please trigger /run-e2e sequential

/run-e2e sequential

/run-e2e sequential

/run-e2e sequential

/run-e2e sequential Update: You can check the progress here

@tomkerkhove Could you please trigger /run-e2e sequential  again

/run-e2e sequential

Hi @zroubalik could you please trigger /run-e2e sequential as I've fixed all failing test

/run-e2e sequential

/run-e2e sequential Update: You can check the progress here

@zroubalik I see the test failed but it passed successfully locally.

Hi @zroubalik could you please trigger /run-e2e sequential as I've fixed all failing test

Hi @JorTurFer could you please trigger /run-e2e sequential the initial trigger passed successfully but I fixed DCO issue and would love to test e2e also on this new change

/run-e2e sequential

Hi @zroubalik could you please review my PR as all e2e test passed successfully.
cc @JorTurFer @tomkerkhove

/run-e2e sequential

/skip-e2e


Emacs packages update: 2019-04-08

Motivation for this change
Update elpa-packages, melpa-packages, melpa-stable-packages.
Things done
Tested all sets with nix-instantiate. Also, running Emacs from this update.
org-packages fail to update due to https://github.com/ttuegel/emacs2nix/issues/49

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

cc @ttuegel

Thanks for doing this! I’m wanting to automate this process but haven’t gotten around to it yet. Usually someone comes around every month or so.

@matthewbauer That would be awesome, if you have any pointers I could possibly maybe help a little 😃

Thanks!
How did you get it to generate? I've failed doing so in my last attempt.
Incidentally, I've just finished a rewrite of emacs2nix, for melpa-unstable (for now). I'll link the PR.

Very nice! I'm wondering whether it is considered poor form to backport these packages to NixOS stable? The latest generation in 19.03 seem to be from 2019-01-03, which is quite old…

Herwig Hochleitner<EMAIL_ADDRESS>writes:

Thanks!
How did you get it to generate? I've failed doing so in my last attempt.

I just followed the instructions in emacs2nix repo. It took quite some
time to run though.
How exactly it failed for you?

Robert Helgesson<EMAIL_ADDRESS>writes:

Very nice! I'm wondering whether it is considered poor form to backport these packages to NixOS stable? The latest generation in 19.03 seem to be from 2019-01-03, which is quite old…

I am not sure but I would rather not do that. The update is potentially non-backward-compatible. e.g., magit popups has been rewritten to transient which broke all my custom-defined magit commands.

How exactly it failed for you?

don't remember exactly, it's been some time. something about unstable vs stable and melpa neglecting their Makefile in favor of a Dockerfile

It took quite some time to run though.

That's mainly because the updater re-runs the elpa builder for producing archive.json and recipes.json. Also, indexing the recipes takes a bit ..

The update is potentially non-backward-compatible

we should probably still backport melpa (non - stable) .. when using that, the expectation of stability takes a back-seat to the expectation of recency.

Could you clarify?

There is no magit in elpaPackages.
The one in melpaStablePackages still has this work-around applied.
The one in melpaPackages is running fine for me. (If I revert this diff, I get an evaluation error.)
I left a comment above—magit no longer uses magit-popup, so it must not be passed in here.

For what it's worth, I cherry-picked this ontop of my 19.03 branch yesterday and have used the resulting Emacs (~100 packages) all day today without any issues. I like the new magit and lsp-java versions 🙂

Is anyone willing to merge?

@rasendubi Done 🙂

Thank you!

Everyone interested is welcome to review/test the next Emacs packages update: https://github.com/NixOS/nixpkgs/pull/62784


Get rid of warnings from test runs

There are a few (deprecation and other) warnings in the pytest output - get rid of them where possible. We should properly silence warnings that come from us testing our own deprecated APIs.
pkg_resources

[x] Need to find a replacement for pkg_resources.working_set (done in #1573):

2023-05-08T14:37:11.9069510Z .tox/py37-data/lib/python3.7/site-packages/pkg_resources/__init__.py:121
2023-05-08T14:37:11.9069995Z   /azp/agent/_work/1/s/.tox/py37-data/lib/python3.7/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
2023-05-08T14:37:11.9071595Z     warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)

[ ] This one is outside of our control, we have to seee if mpl_toolkits is up to date etc.:

2023-05-08T14:37:11.9072063Z .tox/py37-data/lib/python3.7/site-packages/pkg_resources/__init__.py:2870
2023-05-08T14:37:11.9072379Z   /azp/agent/_work/1/s/.tox/py37-data/lib/python3.7/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
2023-05-08T14:37:11.9072963Z   Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
2023-05-08T14:37:11.9073163Z     declare_namespace(pkg)

IO-loop related warnings

[ ] don't use make_current anymore, if it is in our control

2023-05-08T14:37:11.9073307Z tests/io/datasets/test_dm.py::test_get_metadata
2023-05-08T14:37:11.9073437Z tests/io/datasets/test_dm.py::test_load_stack_dd
2023-05-08T14:37:11.9074631Z   /azp/agent/_work/1/s/.tox/py37-data/lib/python3.7/site-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
2023-05-08T14:37:11.9074795Z     self.make_current()

numpy related
I think np.matrix access should always be gated by checking for its existence, silencing the warning if it exists, and skipping the test if np.matrix is no longer a thing:

[x] Fixed in #1430 for the memory dataset tests

2023-05-08T14:37:11.9074942Z tests/io/datasets/test_mem.py::test_get_macrotile[numpy.matrix]
2023-05-08T14:37:11.9075350Z   /azp/agent/_work/1/s/.tox/py37-data/lib/python3.7/site-packages/numpy/matrixlib/defmatrix.py:69: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
2023-05-08T14:37:11.9075625Z     return matrix(data, dtype=dtype, copy=False)

[x] Fixed in #1430

2023-05-08T14:33:45.4961829Z .tox/py310-data/lib/python3.10/site-packages/libertem/common/numba/__init__.py:15
2023-05-08T14:33:45.4962237Z   /azp/agent/_work/1/s/.tox/py310-data/lib/python3.10/site-packages/libertem/common/numba/__init__.py:15: DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)
2023-05-08T14:33:45.4962448Z     np.bool8,

numba

[x] Fixed in #1430

2023-05-08T14:33:45.4962694Z .tox/py310-data/lib/python3.10/site-packages/libertem/io/dataset/base/decode.py:7
2023-05-08T14:33:45.4963398Z   /azp/agent/_work/1/s/.tox/py310-data/lib/python3.10/site-packages/libertem/io/dataset/base/decode.py:7: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
2023-05-08T14:33:45.4963844Z     @numba.jit(inline='always', cache=True)

sparse

[x] is a spurious warning

2023-05-08T14:33:45.5020043Z tests/io/datasets/test_tvips.py: 26 warnings
2023-05-08T14:33:45.5020339Z   /azp/agent/_work/1/s/.tox/py310-data/lib/python3.10/site-packages/sparse/_coo/core.py:261: DeprecationWarning: coords should be an ndarray. This will raise a ValueError in the future.
2023-05-08T14:33:45.5020560Z     warnings.warn(

pytest

[x] PytestRemovedIn8Warning: Passing None has been deprecated: fixed in #1430

2023-05-08T14:37:11.9080225Z tests/io/datasets/test_mib.py::test_not_too_many_files
2023-05-08T14:37:11.9080439Z   /azp/agent/_work/1/s/.tox/py37-data/lib/python3.7/site-packages/_pytest/python.py:194: PytestRemovedIn8Warning: Passing None has been deprecated.
2023-05-08T14:37:11.9080745Z   See https://docs.pytest.org/en/latest/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests for alternatives in common use cases.
2023-05-08T14:37:11.9080878Z     result = testfunction(**testargs)

Our own warnings
These should explicitly be silenced, if the test is about a deprecated feature:

[x] Fixed in #1430

2023-05-08T14:37:11.9081033Z tests/io/datasets/test_raw.py::test_load_legacy_direct
2023-05-08T14:37:11.9081319Z   /azp/agent/_work/1/s/.tox/py37-data/lib/python3.7/site-packages/libertem/io/dataset/raw.py:108: FutureWarning: enable_direct is deprecated; pass `io_backend=DirectBackend()` instead
2023-05-08T14:37:11.9081496Z     FutureWarning

[x] ignore "no plottable channels found" if NoOpUDF is run in tests

2023-05-09T11:00:44.9541476Z tests/test_context.py::test_multi_udf[True-True]
2023-05-09T11:00:44.9541686Z tests/test_context.py::test_multi_udf[True-False]
2023-05-09T11:00:44.9541877Z tests/test_context.py::test_multi_udf_async[True-True]
2023-05-09T11:00:44.9542069Z tests/test_context.py::test_multi_udf_async[True-False]
2023-05-09T11:00:44.9542275Z tests/test_context.py::test_udf_iter[True-True]
2023-05-09T11:00:44.9542457Z tests/test_context.py::test_udf_iter[True-False]
2023-05-09T11:00:44.9542665Z tests/test_context.py::test_udf_iter_async[True-True]
2023-05-09T11:00:44.9545344Z tests/test_context.py::test_udf_iter_async[True-False]
2023-05-09T11:00:44.9545607Z tests/test_context.py::test_plots[True]
2023-05-09T11:00:44.9545993Z   /azp/agent/_work/1/s/.tox/py310/lib/python3.10/site-packages/libertem/api.py:1232: UserWarning: No plottable channels found for UDF #0: NoOpUDF, not plotting.
2023-05-09T11:00:44.9546179Z     warnings.warn(

[x] ignore "Plot is not displayed" warning in test_empty for gms and mpl backends

2023-05-09T11:04:32.1048081Z tests/viz/test_gms.py::test_empty
2023-05-09T11:04:32.1048344Z   /azp/agent/_work/1/s/.tox/py311/lib/python3.11/site-packages/libertem/viz/gms.py:82: UserWarning: Plot is not displayed, not plotting. Call display() to display the plot.
2023-05-09T11:04:32.1048479Z     warnings.warn(
2023-05-09T11:04:32.1048510Z
2023-05-09T11:04:32.1048592Z tests/viz/test_mpl.py::test_empty
2023-05-09T11:04:32.1048838Z   /azp/agent/_work/1/s/.tox/py311/lib/python3.11/site-packages/libertem/viz/mpl.py:94: UserWarning: Plot is not displayed, not plotting. Call display() to display the plot.
2023-05-09T11:04:32.1048982Z     warnings.warn(

[x] Find out and fix where we don't properly handle event loop closing

2023-05-09T11:04:32.1049093Z tests/viz/test_viz_base.py::test_live_RGB
2023-05-09T11:04:32.1049489Z   /azp/agent/_work/1/s/.tox/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:444: DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
2023-05-09T11:04:32.1049664Z   fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
2023-05-09T11:04:32.1049849Z   pytest-asyncio will close the event loop for you, but future versions of the
2023-05-09T11:04:32.1049969Z   library will no longer do so. In order to ensure compatibility with future
2023-05-09T11:04:32.1050077Z   versions, please make sure that:
2023-05-09T11:04:32.1050179Z       1. Any custom "event_loop" fixture properly closes the loop after yielding it
2023-05-09T11:04:32.1050302Z       2. Your code does not modify the event loop in async fixtures or tests

[x] bqplot plotting backend: fix "unrecognized arguments passed to init" related issues

2023-05-09T11:04:32.1046614Z tests/viz/test_bqp.py::test_bqp_smoke
2023-05-09T11:04:32.1046715Z tests/viz/test_bqp.py::test_bqp_asymm
2023-05-09T11:04:32.1046809Z tests/viz/test_bqp.py::test_bqp_asymm
2023-05-09T11:04:32.1046902Z tests/viz/test_bqp.py::test_empty
2023-05-09T11:04:32.1047241Z   /azp/agent/_work/1/s/.tox/py311/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:501: DeprecationWarning: Passing unrecognized arguments to super(Figure).__init__(scales={'x': LinearScale(max=1.0, min=0.0), 'y': LinearScale(max=0.0, min=1.0)}).
2023-05-09T11:04:32.1047603Z   object.__init__() takes exactly one argument (the instance to initialize)
2023-05-09T11:04:32.1047828Z   This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.

[ ] fix warnings related to dataset.adjust_tileshape(...) - probably bad tile shapes passed to memory dataset?

2023-05-09T11:04:32.1023375Z tests/analysis/test_analysis_masks.py::test_subframe_tiles_fast
2023-05-09T11:04:32.1024053Z   /azp/agent/_work/1/s/.tox/py311/lib/python3.11/site-packages/libertem/io/dataset/base/tiling_scheme.py:358: UserWarning: dataset.adjust_tileshape() can only accept tile sig shape or switch tofull frames if dead pixel patching is active. Got original tile shape (128, 16, 16), new tileshape (8, 4, 4) and dataset sig shape (16, 16)
2023-05-09T11:04:32.1024325Z     warnings.warn(err_str)

sparse

The "should be an ndarray" thing was spurious and is fixed upstream: pydata/sparse#580

Looking into this now, it does still happen in the current version. One of the culprits being that we call the sparse.COO constructor on whatever is passed as excluded_pixels, which is documented to already be a sparse.COO instance!
https://github.com/LiberTEM/LiberTEM/blob/4d01be801d0ee327b2ec4f5d3963baa9025a04e4/src/libertem/io/corrections/corrset.py#L88-L110
... and sparse doesn't even like COO(COO), so we need to be a bit more careful. To get an overview over all failures, something like this works: pytest tests/ '-Werror:coords should be an ndarray.:DeprecationWarning'
cc @uellue


Report test

Purpose

This PR adds introduces automatic reporting to our slack channel: #coupler-reports.
The report is composed of selected output from builkite experiments:

[x] build_history graph
[x] flame_graphs extended to show only the contributions of the node in question (minus its child contributions)
[x] the AMIP target resolution
[x] slabplanet with conservation check

To-do

Content

[ ] I have read and checked the items on the review checklist.

Addressed in #267


Trying to connect with web3modal - is it possible?

I am not sure if it is possible but I would like to have web3Modal as my provider in my nuxt 3 project.
So far I have this plugin (plugins/use-wagmi.js):
import { UseWagmiPlugin, configureChains, createConfig, mainnet, sepolia } from 'use-wagmi'
import { w3mConnectors, w3mProvider } from '@web3modal/ethereum'

export default defineNuxtPlugin((nuxtApp) => {
    const config = useRuntimeConfig()
    const projectId = config.public.walletConnectProjectId
    const chains = [mainnet, sepolia]

    const { publicClient, webSocketPublicClient } = configureChains(chains, [w3mProvider({ projectId })])
    const wagmiConfig = createConfig({
        autoConnect: true,
        connectors: w3mConnectors({ projectId, chains }),
        publicClient,
        webSocketPublicClient
    })

    nuxtApp.vueApp.use(UseWagmiPlugin, wagmiConfig)
})

But I am unsure how to proceed since actually web3Modal requires an Ethereum client like so:
const ethereumClient = new EthereumClient(wagmiConfig, chains)
const web3modal = new Web3Modal({ projectId }, ethereumClient)

I should then call
const connect = () => { web3modal.openModal() }
to make the modal appear.
How would you recommend me to continue?

@dataexcess I don't know if this will help you https://docs.walletconnect.com/2.0/web/web3modal/html/wagmi/installation


Update Travis CI for support for codecov.io, optimize build jobs

Travis delays are killing me.  Works fine in my fork...


Update cards.js

Test URLs:

Before: https://main--gw-eds-test-project--herzog31.hlx.live/
After: https://herzog31-patch-1--gw-eds-test-project--herzog31.hlx.live/

[!TIP]
We noticed a change to the cards block. Would you like us to suggest some test cases for this block?

[ ] Suggest test cases

Test comment

[ ] checkbox

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests

gen tests


Expose history api fallback options

Clear and concise description of the problem
We're setting up a project simulating micro-frontend like qiankun in development, for this to work we need to respond index as a special "wrapper" html with qiankun runtime and a startup script pulling actual project, just like:
<script src="/__local_qiankun_bundle__.js"></script>
<script>
  qiankun.registerMicroApps([
    {
      name: 'xxx',
      entry: '/index.html', // The actual index html
      container: '#root',
      activeRule: '/',
    }
  ]);
  qiankun.start();
</script>

(This is only an example, actually we bundle qiankun runtime and entry script with esbuild.)
We then need to modify dev server's route into:

history api fallback to /__qiankun_wrapper__.html which responds content described above
on /__local_qiankun_bundle__.js responds a bundled version of qiankun
/index.html to serve as what vite did not

Without the ability to customize historyApiFallback middleware, it is not possible to setup a server exactly do these staffs, and historyApiFallback is a special middleware that we cannot add another one before or after vite's middleware array.
Suggested solution
Expose server.apiHistoryFallbackOptions to allow user customize this behavior.
Alternative
Another solution is to allow a server.historyApiFallback: false option to intentionally disable history api fallback behavior, users can then add this middleware on demand.
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

The alternative seems to be similar to https://github.com/vitejs/vite/issues/5720, but I'm curious if transformIndexHtml would work for your usecase?
That should allow you to inject the extra scripts into index.html. For /__local_qiankun_bundle__.js, maybe you can do <script type="module" src="./path/to/__local_qiankun_bundle__.js"></script> instead? A Vite plugin could be used if you want the src to point to a virtual path, and the plugin dynamically resolve the virtual path.

3. /index.html to serve as what vite did not

I don't understand what you mean here. Do you mean preventing index.html from serving?

I don't understand what you mean here. Do you mean preventing index.html from serving?

We need index.html to be served as usual, but index.html is not the entry, it is loaded by qiankun's wrapper html. This is why we can't make use of transformIndexHtml because we still need index.html and all of its functionality (including original transformIndexHtml)
The qiankun's wrapper html will execute logic like:
// In this fetch we need vite's dev server to serve original index.html
const html = await fetch('/index.html').then(r => r.text());
const subAppDocument = parsetHtml(html);
subAppDocument.head.children.forEach(v => document.head.appendChild(v));
const container = document.getElementById('root');
container.empty();
container.appendChild(subAPpDocument.body.firstChild);

After some deeper investigation, I believe qiankun won't be able to support vite's dev mode at all, it's core import-html-entry can't work with script[type=module] and mostly can't be upgraded to support module script because it requires new Function to eval code, which doesn't support top level import() at all

FWIW I've been testing out the plugin too and it doesn't seem possible unfortunately. Looks like loading .html files isn't possible with Vite plugins as the .html files are handled directly by the dev server directly.

@bluwy You can refer to my custom plugin as an implement of hijacking .html load for vite in dev mode, and this plugin for build mode, I've tested them and works at least in a simple TodoMVC app

Neat! Thanks for figuring out the plugin flow, that would be really helpful in the future. Do you think this feature request is still needed given the plugin solution?

I for one would love to see a built-in solution for this.
I came up with a very different plugin approach (see below), which seems to mostly work, though not perfectly[1].
Rationale for why this seems important to build into vite core:
Vite seems to take the stance that:

it's a dev and build tool only, not to be used in production
built files are meant to be served statically, therefore, it doesn't come with a production server

However, for static file servers:

in many static file servers it's pretty easy to configure it to return index files when a directory is requested
however, they generally don't fallback to serving index.html when a file is not found and instead return a 404 in those situations

Therefore, it doesn't make much sense that Vite's dev server has this fallback behavior when it targets production environments that don't have it. It would be nice if there were a "correct" (that is, well tested) way to turn off the fallback behavior while keeping the rest of the serving behavior (HTML transformation, serving the right headers, etc).

// express not necessary, but its API does simplify things
const express = require("express");
const { join } = require("path");
const { readFile } = require("fs/promises");

// ADJUST THIS FOR YOUR PROJECT
const PROJECT_ROOT = join(__dirname, "..");

function removeHistoryFallback() {
  return {
    name: "remove-history-fallback",
    configureServer(server) {
      // returned function runs AFTER Vite's middleware is built
      return function () {
        removeViteSpaFallbackMiddleware(server.middlewares);
        server.middlewares.use(transformHtmlMiddleware(server));
        server.middlewares.use(notFoundMiddleware());
      };
    },
  };
}

function removeViteSpaFallbackMiddleware(middlewares) {
  const { stack } = middlewares;
  const index = stack.findIndex(function (layer) {
    const { handle: fn } = layer;
    return fn.name === "viteSpaFallbackMiddleware";
  });
  if (index > -1) {
    stack.splice(index, 1);
  } else {
    throw Error("viteSpaFallbackMiddleware() not found in server middleware");
  }
}

function transformHtmlMiddleware(server) {
  const middleware = express();
  middleware.use(async (req, res, next) => {
    try {
      const rawHtml = await getIndexHtml(req.path);
      const transformedHtml = await server.transformIndexHtml(
        req.url, rawHtml, req.originalUrl
      );

      res.set(server.config.server.headers);
      res.send(transformedHtml);
    } catch (error) {
      return next(error);
    }
  });

  // named function for easier debugging
  return function customViteHtmlTransformMiddleware(req, res, next) {
    middleware(req, res, next);
  };
}

async function getIndexHtml(path) {
  const indexPath = join(PROJECT_ROOT, path, "index.html");
  return readFile(indexPath, "utf-8");
}

function notFoundMiddleware() {
  const middleware = express();
  middleware.use((req, res) => {
    const { method, path } = req;
    res.status(404);
    res.type("html");
    res.send(`<pre>Cannot ${method} ${path}</pre>`);
  });
  return function customNotFoundMiddleware(req, res, next) {
    middleware(req, res, next);
  };
}

module.exports = {
  removeHistoryFallback,
};

[1] The plugin above seems to have broken the ability to use a <script> tag with content as the app entry point instead of linking to the entry point via the src attribute:
<html>
…
  <script type="module">
    import Vue from 'vue';
    import App from './App.vue';

    new Vue({
      render: (h) => h(App),
    }).$mount('#app');
  </script>
…
</html>

@sapphi-red  Thanks for linking here.
I'm having a similar issue. I'd like to nest all my entry points including root index.html in /src/pages. I'm having trouble with both; getting rid of the trailing slash for nested pages and getting the root index.html to work in /src/pages. Tried everything in the docs and still no luck.

Vite 3 can now officially support this with appType: 'mpa', which would disable the spa fallback, so you can add a custom one if needed via configureServer. appType is also used for the preview server too so it should emulate your production setup. Otherwise there's also configurePreviewServer hook for plugins.

Amazing, thanks for sharing appType: 'mpa'. I haven't tried yet, but this feels like it should make my issue go away.


Can't open kfusiond.lib when build project

error	LNK1104	can't open file “..\lib\Debug\kfusiond.lib”	demo	E:\appIns\PCL\kinfu_remake-master\kinfu_remake-master\build\apps\LINK	1

Same problem with you

I meet the same problem too, how to solve it?

@InouTooko I instead used OpenCV. Kinfu is included in contrib, you can build from source to use it.

@InouTooko Thanks :)


Add VLC SAP message support

SAP messages for MPEG-TS UDP video have a /x to the port like 8208/1. Wireshark calls this Media Port Count. The media filter of \d does not match this, so it's not parsed. This pull request adds a grammar regex that matches a line like m=video 8208/1 udp 33

Ah, thanks for this.
I think it's probably better to widen the existing regex to capture it if that's possible. AFAIKT it's only the \d that's causing the problem, and everything else is identical. I'd rather not have more than one m line regex because m-line semantics is kind of wild.
A quick test sdp for this would be very helpful.

do you want the regex to be \d and only the char / to be minimalist?

Please don't add a new grammar rule just to parse an optional field that was missing in the existing grammar rule.

finally got around to integrating those 2 rules together. I ended up with /^(\w*) (\d*)(?:\/(\d*))? ([\w/]*)(?: (.*))?/ which adds an optional match for a port number while excluding the / divider with a non-matching group
No port count makes portCount undefined
'video 8208 udp 33'.match(/^(\w*) (\d*)(?:\/(\d*))? ([\w/]*)(?: (.*))?/)
(6) ["video 8208 udp 33", "video", "8208", undefined, "udp", "33", index: 0, input: "video 8208 udp 33", groups: undefined]

A / and port count after the port is now captured
'video 8208/1 udp 33'.match(/^(\w*) (\d*)(?:/(\d*))? ([\w/])(?: (.))?/)
(6) ["video 8208/1 udp 33", "video", "8208", "1", "udp", "33", index: 0, input: "video 8208/1 udp 33", groups: undefined]
The one issue is with \d* it will capture an empty string if there is a / but no port count defined
'video 8208/ udp 33'.match(/^(\w*) (\d*)(?:\/(\d*))? ([\w/]*)(?: (.*))?/)
(6) ["video 8208/ udp 33", "video", "8208", "", "udp", "33", index: 0, input: "video 8208/ udp 33", groups: undefined]


Development.Tools_v2023-10-23.exe and Development.Tools_v2023-10-08.exe --  error message unable to find SetupS

When app is attempted to be run, SetupS_x64 is not found.
Included programs can be run if Development.Tools is extracted, so whatever script is used to create Development.Tools has a problem.

a quick look makes me believe that there is an error in the Autoit script which i know nothing about

similar to another issue at 5


Hugging Face is in maintenance

Hi guys, thanks for your nice work!  It seems that the server on the hugging face is not OK and we cannot try the online demo anymore. Is there any method to fix this?

The Huggingface Space is just in sleep; you can restart this space by clicking on "restart this space". I have restarted it; welcome to use gim. 🤗


AttributeError: 'NumberStatus' object has no attribute 'id'

Hello guys,
I tried to use this statement:
<EMAIL_ADDRESS>But I received this error:
Traceback (most recent call last):
File "demo.py", line 16, in
print<EMAIL_ADDRESS>File "WebWhatsapp/webwhatsapi/objects/number_status.py", line 24, in repr
id=self.id,
AttributeError: 'NumberStatus' object has no attribute 'id'

Are you printing the result? This method that calls the self.id is only called on print. The id contains what you have passed as argument to the method. I see the problem and will fix it as soon as possible, but until that try to don’t print the object :).

Ok, thank you for your help


Fibaro FGS222 switch state is not getting updated when using the pyhsical switch

Checklist

[x]  I am not using Home Assistant. Or: a developer has told me to come here.
[X] I have checked the troubleshooting section and my problem is not described there.
[X] I have read the changelog and my problem is not mentioned there.

Deploy method
Docker
Z-Wave JS UI version
9.27.2.b0ade31
ZwaveJS version
14.3.2
Describe the bug
Since the last update the state of the FGS222 relay switches is not updated, when I use the physical switch.
(I can still switch it with my home automation domoticz via mqtt.)
I noticed it, since the change if the relay switch triggers other lights,
I would be glad about any suggestions,
Torben
To Reproduce
I press the switch, but in the UI I don't see a change of the binary switch. I have two, it shows up on both,
Expected behavior
Pressing the switch fo FG222 should change the state of the binary switch in the UI
Additional context
No response

This should be fixed in the latest release v14.3.4 (https://github.com/zwave-js/node-zwave-js/pull/7426)
If not, feel free to report back and reopen the issue.

Perfect!
Thanks a lot, both relay switches are reporting their state to the controller after I updated.


Future Possibilities

Hi!
Amazing work :)
I was wondering if it will be possible to use any grasshopper libraries.

Hello, and thank you! Absolutely, and it's almost time to start taking this question seriously. If you have any specifically in mind, please drop your suggestions here.


[Relax][ONNX] Add Local Response Normalization (LRN)

Add Local Response Normalization (LRN) operation to ONNX front-end and FFI of Relax.
CI test was also added

Hello @jwfromm! This PR is ready to be merged


Add Spotless rules for SQL and Markdown

Add Spotless rules for SQL and Markdown
Sometimes in code review we flag things like SQL files not ending with trailing
newlines, but that should really be the job of a linter. Configure Spotless to
format (and check) Markdown files and newly-added SQL files. Existing SQL files
are left alone since they're mostly migrations, and modifying a migration
that's already applied will cause Flyway to abort at application start time.

#2038  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sgrimm and the rest of your teammates on  Graphite

For ease of reviewing, the first commit here is the config change, and the second commit is the file edits from applying the new config.


Defaults to False since no backend info available.

reopened from https://github.com/CQCL/pytket-extensions/pull/473

Please check is this is all copied correctly @Roland-djee


Filter out event note text from logs

Who is this PR for?
Developers who use the logs and get email alerts from our logging service.
What problem does this PR fix?
Event note text is currently printed in the logs as plaintext. This isn't the best security practice because it makes the surface larger for accessing sensitive event note data. We want to shrink that surface and make it as small as possible. If the logging service sends an email alert that includes a log trace of an event note being created/updated, without filtering in place we'll have copies of sensitive event note text in developers' inboxes.
What does this PR do?
Filters out any field named "text" from the logs.
Event note creation log (demo data, master branch)

Event note creation log (demo data, this branch)

@kevinrobinson Good idea, just added a code comment. 👍

@kevinrobinson Don't see anything about indicating particular models using this config. 😞
Here are what the Rails guides have to say about it: http://guides.rubyonrails.org/security.html#logging


Weird error messages when adding object to number

https://flowtype.org/try/#0PQKgBAAgZgNg9gdzCYAoVBGMBqMBvAXwG4g
The weird part is that first two errors are pointing to the same 1 which is mentioned as number and string at the same time and it is also mentioned that I try to add it to itself
3: 1 + {};
   ^ number. This type cannot be added to
3: 1 + {};
   ^ string
3: 1 + {};
       ^ object literal. This type cannot be added to
3: 1 + {};
   ^ string

IMO the output above should look like
3: 1 + {};
   ^ string (coerced from number). This type cannot be added to
3: 1 + {};
       ^ object literal. This type cannot be added to
3: 1 + {};
   ^ string

that's bizarre... it should just be
3: 1 + {};
       ^ object literal. This type cannot be added to
3: 1 + {};
   ^ number

I'll look into it. thanks!


Removing Petroleum Report - Infrastructure and Petroleum Report - Cum…

…ulative Water Production
Petroleum Report - Infrastructure, was removed from reporting requirements in 2018 under the MWOLA Act 2018.
Petroleum Report – Cumulative Water Production is no longer required. This report is lodged with the Petroleum Production Report in the LP as part of petroleum production reporting requirements under the Petroleum and Gas (General Provisions) Regulation 2017 Section 42(3)(d) and (f).

@talebib
I think the intent here is to retain these report types as things that have been submitted in the past and still exist in the system, but are no longer current report types that will be accepted by the lodgement portal.
As such I would retain these as concepts, but remove them from the collections that the lodgement portal uses to populate the Report Type Selection drop-down list.
i.e. instead of removing them entirely, find the list grt:lodgement-portal-reports a skos:Collection ;  and just remove grt:petroleum-report-cumulative-water-production and grt:petroleum-report-infrastructure from that list

Thanks Vance. Understood!


An error occurs when debugging a Python project

OS: Mac (Pro)
Build Version: 20220906.4
Repro Steps:

Create and open a Python project in VS Code.
F5.
Check whether no error occurs.

Expect:
No error occurs.
Actual:
An error occurs.

Python Functions on M1 (ARM64) is, unfortunately, not yet supported.  Such testing will need to be done on an Intel Mac.


Fetching the suggestions by AJAX: where to do it

Sorry for my simple question. I didn't find any wiki or stackOverflow group to ask it...
I'm a beginner in reactJs and I need to display the autosuggestion for cities. The city list is quite big and loaded by ajax from server.
Where can I put the logic of such fetching?
I guess that hook didComonentMount will work only once.
Thanks in advance for your help.

You should use the onSuggestionsFetchRequested prop. Documented here.
Example implementation below and on CodePen.
function getSuggestionValue(suggestion) {
  return suggestion.name;
}

function renderSuggestion(suggestion) {
  return (
    {suggestion.name}
  );
}

class App extends React.Component {
  constructor() {
    super();

    this.state = {
      value: '',
      suggestions: []
    };
  }

  onChange = (event, { newValue, method }) => {
    console.log(newValue)
    this.setState({
      value: newValue
    });
  };

  onSuggestionsFetchRequested = ({ value }) => {
    fetch(`https://swapi.co/api/people/?search=${value}`)
      .then(response => response.json())
      .then(data => this.setState({ suggestions: data.results }))
  }

  onSuggestionsClearRequested = () => {
    this.setState({
      suggestions: []
    });
  };

  render() {
    const { value, suggestions } = this.state;
    const inputProps = {
      placeholder: "Search Star Wars",
      value,
      onChange: this.onChange
    };

    return (
      <Autosuggest
        suggestions={suggestions}
        onSuggestionsFetchRequested={this.onSuggestionsFetchRequested}
        onSuggestionsClearRequested={this.onSuggestionsClearRequested}
        getSuggestionValue={getSuggestionValue}
        renderSuggestion={renderSuggestion}
        inputProps={inputProps} />
    );
  }
}

ReactDOM.render(<App />, document.getElementById('app'));

Yep, thank you!

You should use the onSuggestionsFetchRequested prop. Documented here.
Example implementation below and on CodePen.
const getSuggestionValue = (suggestion) => suggestion.name
const renderSuggestion = (suggestion) => ({suggestion.name})

class Example extends React.Component {
  constructor() {
    super()
    this.state = { value: '', suggestions: [] }
  }

  onChange = (event, { newValue, method }) => {
    this.setState({ value: newValue });
  }

  onSuggestionsFetchRequested = ({ value }) => {
    fetch(`https://swapi.co/api/people/?search=${value}`)
      .then(response => response.json())
      .then(data => this.setState({ suggestions: data.results }))
  }

  onSuggestionsClearRequested = () => {
    this.setState({ suggestions: [] });
  };

  render() {
    const { value, suggestions } = this.state;
    const inputProps = {
      placeholder: "Search Star Wars",
      value,
      onChange: this.onChange
    };

    return (
      <Autosuggest
        suggestions={suggestions}
        onSuggestionsFetchRequested={this.onSuggestionsFetchRequested}
        onSuggestionsClearRequested={this.onSuggestionsClearRequested}
        getSuggestionValue={getSuggestionValue}
        renderSuggestion={renderSuggestion}
        inputProps={inputProps} />
    );
  }
}

Btw those looking to try this. The link has changed to https://swapi.dev/api/people/?search=${value}

You should use the onSuggestionsFetchRequested prop. Documented here.
Example implementation below and on CodePen.
const getSuggestionValue = (suggestion) => suggestion.name
const renderSuggestion = (suggestion) => ({suggestion.name})

class Example extends React.Component {
  constructor() {
    super()
    this.state = { value: '', suggestions: [] }
  }

  onChange = (event, { newValue, method }) => {
    this.setState({ value: newValue });
  }

  onSuggestionsFetchRequested = ({ value }) => {
    fetch(`https://swapi.co/api/people/?search=${value}`)
      .then(response => response.json())
      .then(data => this.setState({ suggestions: data.results }))
  }

  onSuggestionsClearRequested = () => {
    this.setState({ suggestions: [] });
  };

  render() {
    const { value, suggestions } = this.state;
    const inputProps = {
      placeholder: "Search Star Wars",
      value,
      onChange: this.onChange
    };

    return (
      <Autosuggest
        suggestions={suggestions}
        onSuggestionsFetchRequested={this.onSuggestionsFetchRequested}
        onSuggestionsClearRequested={this.onSuggestionsClearRequested}
        getSuggestionValue={getSuggestionValue}
        renderSuggestion={renderSuggestion}
        inputProps={inputProps} />
    );
  }
}

Btw those looking to try this. The link has changed to https://swapi.dev/api/people/?search=${value}


Ip_adapter not work with xformers enabled

Hello,
Firstly @chengzeyi Thanks for this project! This the best optimization framework I ever tested by far.
I have a customized pipeline with ip_adapter plus support (by diffusers main branch). The ip_adapter not works with config.enable_xformers = True, and it works well after xformers disabled.
I’m not very sure but I guess there are some conflicts between memory_efficient_attention and ip_adapter’s attnprocessor.  Any thoughts?
Ps: It’s still fast enough even to disable the xformers, amazing work!
Thanks!

You reminded me that Diffusers official said Torch2.0 does not require config.enable_ Xformers=True

@ldtgodlike  For sure it may not need xformers, because most of attenprocessor from diffusers have 2_0 version which use sdpa.  but from stable-fast looks wrapper xformers for others reason. I have a little bit confused here...

well, when I just try to use diffusers wihtout  stable-fast in torch==2.1.0, diffusers==0.24.0，xformers==0.0.22.post7, the result  looks wrapper when I use pipe.enable_xformers_memory_efficient_attention(). @huo-ju

@huo-ju Hello, may I ask how you use stable-fast to accelerate pipe after .load_ip_adapter. I've also had some problems before while doing this

@ldtgodlike it will be changed to 2_0 version if you have torch >=2.0 installed. https://github.com/huggingface/diffusers/blob/f5942649f522f0ae87bcb659735b9fd894349ca8/src/diffusers/models/attention_processor.py#L164

@fkjkey No magic on my code, just disabled xformer ( config.enable_xformers = False ) after .load_ip_adapter()

@huo-ju I understand this, and what I mean is that pure diffusers cause this problem, not stable fast, unless the author of this repo specifically modifies it to address this issue

@huo-ju Can you share your full script so that I can reproduce that?

@chengzeyi may be it could help you, I'm curious if enabling xforms in stable fast will accelerate
import torch
from diffusers import AutoPipelineForText2Image, EulerDiscreteScheduler,ControlNetModel
from sfast.compilers.diffusion_pipeline_compiler import (
    compile, CompilationConfig)
import numpy as np
import cv2
from PIL import Image

def canny_process(image,width, height):
    np_image = cv2.resize(image,(width, height))
    np_image = cv2.Canny(np_image, 100, 200)
    np_image = np_image[:, :, None]
    np_image = np.concatenate([np_image, np_image, np_image], axis=2)
    #canny_image = Image.fromarray(np_image)
    return Image.fromarray(np_image)

def reference_process(image,width, height):
    np_image = cv2.resize(image,(width, height))
    return Image.fromarray(np_image)

def load_model():
    extra_kwargs = {}
    #extra_kwargs['variant'] = variant

    controlnet = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_canny",
                                                 torch_dtype=torch.float16, variant="fp16",
                                                 name="diffusion_pytorch_model.fp16.safetensors", use_safetensors=True)
    extra_kwargs['controlnet'] = controlnet
    model = AutoPipelineForText2Image.from_pretrained("runwayml/stable-diffusion-v1-5",
                                         torch_dtype=torch.float16,
                                         **extra_kwargs)
    model.scheduler = EulerDiscreteScheduler.from_config(model.scheduler.config)
    model.safety_checker = None
    model.load_ip_adapter("h94/IP-Adapter", subfolder="models",
                          weight_name="ip-adapter_sd15.safetensors")
    model.to(torch.device('cuda'))

    model=compile_model(model)
    return model

def compile_model(model):
    config = CompilationConfig.Default()
    try:
        import xformers
        config.enable_xformers = True
    except ImportError:
        print('xformers not installed, skip')
    try:
        import triton
        config.enable_triton = True
    except ImportError:
        print('Triton not installed, skip')
    config.enable_cuda_graph = True

    model = compile(model, config)

    kwarg_inputs = dict(prompt=["aa"] * 4,
                      negative_prompt=[""] * 4,
                      width=768,
                      height=512,
                      num_inference_steps=20,
                      num_images_per_prompt=1,
                      guidance_scale=7.5,
                      ip_adapter_image=[np.ones((768,512,3))] * 4,
                      image=[np.ones((768,512,3))] * 4,
                      controlnet_conditioning_scale=1.0,
                      )
    for _ in range(1):
        output_image = model(**kwarg_inputs).images[0]

    return model

if __name__ == "__main__":
    control_img='control.png'
    reference_img='reference.png'

    width =768
    height=512

    canny_image = cv2.imread(control_img)[:,:,::-1]
    reference_img = cv2.imread(reference_img)[:, :, ::-1]
    canny_image=canny_process(canny_image, width, height)
    reference_img=reference_process(reference_img, width, height)

    model=load_model()
    model=compile_model(model)
    seed=-1
    batch_size=4
    generator = torch.Generator(device=CUDA_DEVICE).manual_seed(seed)
    prompt="dog"
    negative_prompt=""
    num_inference_steps=20
    guidance_scale=7.5
    controlnet_conditioning_scale=1.0
    images = pipe(prompt=[prompt] * batch_size,
                  negative_prompt=[negative_prompt] * batch_size,
                  width=width,
                  height=height,
                  num_inference_steps=num_inference_steps,
                  num_images_per_prompt=1,
                  guidance_scale=guidance_scale,
                  ip_adapter_image=[reference_img] * batch_size,
                  image=[control_img] * batch_size,
                  generator=generator,
                  controlnet_conditioning_scale=float(controlnet_conditioning_scale),
                  ).images

@chengzeyi Yes, my code is very similar to ldtgodlike's but within a big pipeline. I can make a minimal test script if his code not works.

@huo-ju @ldtgodlike I run this script without any problem. So what's the detail of the exception?

@chengzeyi  It seems that it can't reference the style if config.enable_triton = True
reference_img

control_img

prompt="a bedroom"
if config.enable_triton = True, the result look like using controlnet only

but if Annotate this line of code:
#config.enable_triton = True,
it could reference the style

ipadapter not reference the result in my testing, no error, just has the same result with none-ipadapter pipeline.

@huo-ju @ldtgodlike I run this script without any problem. So what's the detail of the exception?
See https://github.com/chengzeyi/stable-fast/blob/9fd07ce57e0cedd62ee59cd78774a577e4f2967b/community/optimize_sd15_with_controlnet_and_ip_adapter.py

same to me. Is this issue solved ?


Should selected impact areas filter available damage categories in the Impact Area Scenario editor

When creating an aggregated stage damage relationship, an impact area is required for each curve.

When working in the Impact Area Scenario, the Damage Category options are dependent on the selected Stage Damage. Should the Damage Category options also be affected by the selected impact area?
For example, given there are two available Stage Damage Relationships with their respective curves.
Option A
Curve 1 - North Area - Occupancy Type Group -> IND
Curve 2 - North Area - Occupancy Type Group -> COM
Curve 3 - South Area - Occupancy Type Group -> RES
Option B
Curve 1 - North Area - Occupancy Type Group -> AUTOS
Curve 2 - South Area - Occupancy Type Group -> PUB
Curve 3 - South Area - Occupancy Type Group -> COM
When the user selects the impact area "North Area" and "Option A" stage damage, then would the options available for the damage categories be the following?
Occupancy Type Group -> IND
Occupancy Type Group -> COM
Would the "Occupancy Type Group -> RES" not be available because it is associated with the impact area of "South Area" and not "North Area"?
Currently, the available Damage Category options are only affected by the selected Stage Damage. So in the same case where "North Area" impact area and "Option A" stage damage are selected, the following options are displayed.
Occupancy Type Group -> IND
Occupancy Type Group -> COM
Occupancy Type Group -> RES

Great question. I asked Richard and we decided that this would be a good thing. I have implemented this so that it now filters the damage categories based on the impact area that they are associated with.

issue has been resolved


fix getting the correct credential after search

@esune - please take a look.
@JakKowalCzyk - thanks for the PR.

@JakKowalCzyk can you please describe how to reproduce the issue that this is going to fix, and how to reproduce it? Thanks!

@esune Of course.
Firstly, we discovered that sometimes when we search a registered company on the Greenlight, we select one and than we want to generate the graph for it, here is what happens:

The graph shows us that any credentials were not issued and we can register the company - which is not true because we were able to search it and pick it
The graph looks fine and it is marked with green color but when we will click on a credential it will move us to the OrgBook but to completely different company
So what we have discovered is that when You select a company after search and want to generate the graph for it, the Greenlight takes ID of company searched Name ID. We think that this is not correct because when we checked it in the database it turned out that Name ID is not equal to credential/topic ID of given company.
In case it matters to reproduce the case, we discovered it when we were using Greenlight configured to have only 2 bcgov modified agents - Business Relationship and Registered corporation, moreover the Business Relationship was depending on Registered corporation and it was the second step in our flow. Actually it looks like this is the reason why we experienced this problem, anyway we had to make the change in our fork to have Greenlight working properly, maybe it is not necessary to change it in official repo.

Thank you @JakKowalCzyk! That makes sense, thank you for submitting the PR. I added a change to make the behaviour consistent when search results are found.


improvement: Adds timestamp to messages

improvement: Adds timestamp to messages
Includes a timestamp for each message/line that flows through the client.
Allows the user to provide their own timestamp through Options- and if the timestamp is not provided- a default value is used.
Semver: Patch
Ref: LOG-7156

also note that code coverage is being published, you can check it in CircleCI artifacts: https://98-268886384-gh.circle-artifacts.com/0/tmp/artifacts/coverage.html#file2 (you will see there is a new uncovered line in this PR)


Can set autoplay?

Hi,
With this plugin can set autoplay to audio?

Yes, why not? Did you try it? autoplay is not an audio specific option.

I try set autoplay:true but not play automatic.

Either you have also set splash - splash and autoplay by definition contradict each other - or you are testing on a mobile device which does not allow autoplay.
This is an issue tracker, please turn to our forum https://flowplayer.org/forum/ for usage questions before you are certain that it is a bug. If you provide a link to a minimal sample page showing the issue, you should get an answer quickly.