HDFS-17546. Implementing HostsFileReader timeout

Description of PR
In hadoop environments that rely on NAS to house dfs.hosts file, during FS hangs (for any reason) the refreshNodes call would infinitely hang (causing a blocked thread) until the FS returns.
This is due to how Java's Filestream works here:
  try (Reader input =
          new InputStreamReader(
              Files.newInputStream(hostFile.toPath()), StandardCharsets.UTF_8)) {
        allDNs = objectMapper.readValue(input, DatanodeAdminProperties[].class);
      } catch (JsonMappingException jme) {

In order to fix this we introduce a method  (+ new config value) which (if set) encapsulates the original method in a Future.Task. If the Task -- reading the file -- does not complete within dfs.hosts.timeout then we raise IOException which would interrupt (close) the thread and return a failure.
How was this patch tested?
Updated Unit tests to account for new method, ran through local test suite, and planning to run through remote test suite
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 50s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  1s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 28s

trunk passed

+1 :green_heart:
compile
5m 55s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 49s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 30s

trunk passed

+1 :green_heart:
mvnsite
2m 16s

trunk passed

+1 :green_heart:
javadoc
1m 50s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 21s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 52s

trunk passed

+1 :green_heart:
shadedclient
41m 14s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  0s

the patch passed

+1 :green_heart:
compile
5m 52s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
5m 52s

the patch passed

+1 :green_heart:
compile
5m 34s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
5m 34s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m 18s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 18 new + 201 unchanged - 0 fixed = 219 total (was 201)

+1 :green_heart:
mvnsite
2m  2s

the patch passed

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m  9s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 59s

the patch passed

+1 :green_heart:
shadedclient
41m 14s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 22s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
254m  3s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 44s

The patch does not generate ASF License warnings.

441m 50s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl

hadoop.hdfs.util.TestCombinedHostsFileReader

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 3d0aed474eda 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 8ed85cafbbff71c9a82d68db429c6e3a9bd705d1

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/2/testReport/

Max. process+thread count
3046 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m  8s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 26s

trunk passed

+1 :green_heart:
compile
6m  6s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 52s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 30s

trunk passed

+1 :green_heart:
mvnsite
2m 20s

trunk passed

+1 :green_heart:
javadoc
1m 51s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 21s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 50s

trunk passed

+1 :green_heart:
shadedclient
44m  8s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 43s

the patch passed

+1 :green_heart:
compile
9m 13s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
9m 13s

the patch passed

+1 :green_heart:
compile
8m 57s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
8m 57s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m  7s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 3 new + 201 unchanged - 0 fixed = 204 total (was 201)

+1 :green_heart:
mvnsite
3m 10s

the patch passed

+1 :green_heart:
javadoc
2m  9s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 59s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
8m 19s

the patch passed

+1 :green_heart:
shadedclient
4m 47s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
1m 12s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt
hadoop-hdfs-client in the patch failed.

-1 :x:
unit
0m 24s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch failed.

+0 :ok:
asflicense
0m 25s

ASF License check generated no output?

165m 55s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 1d281f9d1bbf 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e28fe7ea11a4d6d354ad5545325a931460a0b1ac

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/3/testReport/

Max. process+thread count
527 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 15s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 51s

trunk passed

+1 :green_heart:
compile
6m 16s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 58s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 33s

trunk passed

+1 :green_heart:
mvnsite
2m 27s

trunk passed

+1 :green_heart:
javadoc
1m 54s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 29s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 55s

trunk passed

+1 :green_heart:
shadedclient
41m 28s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 33s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  3s

the patch passed

+1 :green_heart:
compile
6m  2s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
6m  2s

the patch passed

+1 :green_heart:
compile
5m 44s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
5m 44s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 20s

the patch passed

+1 :green_heart:
mvnsite
2m  6s

the patch passed

+1 :green_heart:
javadoc
1m 35s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 12s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
6m  1s

the patch passed

+1 :green_heart:
shadedclient
41m 45s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 26s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
257m 16s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

448m 12s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestRollingUpgrade

hadoop.hdfs.util.TestCombinedHostsFileReader

hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 99e2ef825f54 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>9940d3d141e535cad7f34db485fe6d

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/4/testReport/

Max. process+thread count
3045 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 45s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 31s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m  8s

trunk passed

+1 :green_heart:
compile
6m  7s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 50s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 30s

trunk passed

+1 :green_heart:
mvnsite
2m 17s

trunk passed

+1 :green_heart:
javadoc
1m 49s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 19s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 50s

trunk passed

+1 :green_heart:
shadedclient
42m 10s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 58s

the patch passed

+1 :green_heart:
compile
5m 57s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
5m 57s

the patch passed

+1 :green_heart:
compile
6m 22s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
6m 22s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 25s

the patch passed

+1 :green_heart:
mvnsite
2m 18s

the patch passed

+1 :green_heart:
javadoc
1m 40s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 18s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
7m 33s

the patch passed

+1 :green_heart:
shadedclient
51m  6s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 31s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
1m 38s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch failed.

+0 :ok:
asflicense
0m 28s

ASF License check generated no output?

202m 36s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 73b73b9def27 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / dc61e2d54d8d4f038dfe19ca6f2e2f6f152968b0

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/5/testReport/

Max. process+thread count
528 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 23s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
37m  7s

trunk passed

+1 :green_heart:
compile
6m  7s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 58s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 30s

trunk passed

+1 :green_heart:
mvnsite
2m 22s

trunk passed

+1 :green_heart:
javadoc
1m 54s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 26s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 58s

trunk passed

+1 :green_heart:
shadedclient
40m 55s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 33s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  1s

the patch passed

+1 :green_heart:
compile
6m  2s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
6m  2s

the patch passed

+1 :green_heart:
compile
5m 47s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
5m 47s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 21s

the patch passed

+1 :green_heart:
mvnsite
2m  5s

the patch passed

+1 :green_heart:
javadoc
1m 36s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 13s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
6m  5s

the patch passed

+1 :green_heart:
shadedclient
42m 16s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 27s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
254m 56s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

446m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux adb74b708fb2 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 915702579cc9799c737f1139f88a91082b6a6ba8

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/6/testReport/

Max. process+thread count
3047 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 47s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 26s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
37m 20s

trunk passed

+1 :green_heart:
compile
6m  5s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 45s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 28s

trunk passed

+1 :green_heart:
mvnsite
2m 17s

trunk passed

+1 :green_heart:
javadoc
1m 53s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 19s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 54s

trunk passed

+1 :green_heart:
shadedclient
41m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 59s

the patch passed

+1 :green_heart:
compile
5m 54s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
5m 54s

the patch passed

+1 :green_heart:
compile
5m 41s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
5m 41s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 18s

the patch passed

+1 :green_heart:
mvnsite
2m  4s

the patch passed

+1 :green_heart:
javadoc
1m 33s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m  9s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 58s

the patch passed

+1 :green_heart:
shadedclient
41m  9s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 21s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
259m  7s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 45s

The patch does not generate ASF License warnings.

448m  9s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 637b5ff97176 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 555c098e487fb818bfee71cf57a37f2fe8cd3ced

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/7/testReport/

Max. process+thread count
3046 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 47s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 11s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
37m 18s

trunk passed

+1 :green_heart:
compile
5m 59s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
5m 50s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
1m 29s

trunk passed

+1 :green_heart:
mvnsite
2m 17s

trunk passed

+1 :green_heart:
javadoc
1m 52s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 22s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
5m 48s

trunk passed

+1 :green_heart:
shadedclient
41m  9s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 58s

the patch passed

+1 :green_heart:
compile
5m 57s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
5m 57s

the patch passed

+1 :green_heart:
compile
5m 41s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
5m 41s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 19s

the patch passed

+1 :green_heart:
mvnsite
2m  0s

the patch passed

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m  7s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
6m  0s

the patch passed

+1 :green_heart:
shadedclient
42m  0s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 26s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
52m  6s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

-1 :x:
asflicense
1m  3s
/results-asflicense.txt
The patch generated 32 ASF License warnings.

242m  1s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot

hadoop.hdfs.server.namenode.TestStripedINodeFile

hadoop.hdfs.TestSafeModeWithStripedFile

hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy

hadoop.hdfs.TestDFSStorageStateRecovery

hadoop.hdfs.TestReadStripedFileWithDecoding

hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletionGc

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 7593863b3fde 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 55716166e92c137a0614a86b995e5d11fed7d46c

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/9/testReport/

Max. process+thread count
2091 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 31s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 19s

trunk passed

+1 :green_heart:
compile
5m 28s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
5m 18s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
1m 27s

trunk passed

+1 :green_heart:
mvnsite
2m 25s

trunk passed

+1 :green_heart:
javadoc
1m 54s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
2m 30s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
5m 53s

trunk passed

+1 :green_heart:
shadedclient
36m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 34s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 59s

the patch passed

+1 :green_heart:
compile
5m 19s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
5m 19s

the patch passed

+1 :green_heart:
compile
5m  9s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
5m  9s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 15s

the patch passed

+1 :green_heart:
mvnsite
2m  4s

the patch passed

+1 :green_heart:
javadoc
1m 31s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
2m 13s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
5m 58s

the patch passed

+1 :green_heart:
shadedclient
36m 18s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 30s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
230m 40s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 47s

The patch does not generate ASF License warnings.

403m 23s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/10/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6873

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint

uname
Linux 38403ea00269 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c3cb6891e9c823719fb346ee9d0376ac1aeea123

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/10/testReport/

Max. process+thread count
3533 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6873/10/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Add xmldocs

Modifed the templates to add xmldocs for each method based upon its enumerable counterpart. Decribes each method, parameters, return type, exceptions and adds additional context for how its safe to use (ie the Sum of floating point number may result in a different result).
Added an empty check to AverageCore to match its Enumerable.Average counterpart.
Added Null and Empty tests.

thank you for the great contribution!
I'll merge and release soon.


Add details about scenarios where AAD Registration (WPJ) is useful and why it is mostly not recommended for VDIs

In Microsoft CSS we have received multiple critical situation cases where the solution is to cleanup the Azure AD registration – aka Workplace Joined (WPJ) – details in VDI environments – persistent and non-persistent.
Reason is users are unknowingly performing WPJ with an account from domain A, while the machine is already domain joined (DJ), or even Azure AD Joined (AADJ), or Hybrid Azure AD Joined (HAADJ) to domain A.
The article is mentioning the solution, but only for another reason and only for non-persistent:
"Note
When using non-persistent VDI, if you want to prevent a device join state ensure the following registry key is set:
HKLM\SOFTWARE\Policies\Microsoft\Windows\WorkplaceJoin: "BlockAADWorkplaceJoin"=dword:00000001"

It is not mentioned why it may be useful for WPJ to be blocked with same key also in Persistent environments.
Blocking WPJ in any environment where the machine is DJ, or AADJ, or HAADJ is actually useful for 2 reasons:

Office will no longer prompt for WPJ during sign in – better user experience
No risk for a user to perform WPJ with an account from same domain where the machine is already joined (DJ, or AADJ, or HAADJ) – which is exactly the root cause for the authentication failures triggering critical severity support cases that we have received in support

There is only one scenario where WPJ makes sense where the machine is DJ, or AADJ, or HAADJ:

user has 2 accounts from tenant A and B
machine is DJ, or AADJ, or HAADJ to tenant A
user will login to Office client apps with account from tenant B and will rely on WPJ for SSO to tenant B

The other scenario where WPJ makes sense is BYOD – where the machine is not DJ, or AADJ, or HAADJ.
Based on customer discussions, it is really unclear what the last line in the “Supported scenarios” table:
Azure AD registered Federated/Managed Windows current/Windows down-level Persistent/Non-Persistent Not Applicable
Please document the above OK and Not OK scenarios for Azure AD Registered – aka Workplace Joined (WPJ). This will actually explain the above “Not Applicable” statement and help avoid the issues reported to Microsoft CSS and customer frustration.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0e8f55d4-539c-7835-010b-34026860f699
Version Independent ID: 4f8db376-bed9-6fd2-2e65-ac11c0952540
Content: Device identity and desktop virtualization - Azure Active Directory
Content Source: articles/active-directory/devices/howto-device-identity-virtual-desktop-infrastructure.md
Service: active-directory
Sub-service: devices
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

@SilviuTanasescu Thank you for sharing the feedback and the details. We are requesting a review from the content team on this to take this forward accordingly .

Forwarding to the PG for thoughts and feedback. Thanks for your ideas. #please-close
@SanDeo-MSFT Thoughts?


Access to Django request from within the plugin

Hi
I'm writing a custom check and I would need some informations contained in Django's request object in my check. I could not find a way to access the request nor to pass it to the plugin.
Is it possible to access the request from the plugin?
Thank you.

Hi :wave:
I'm afraid it's not possible because CheckMixin that is subclassed in MainView doesn't pass the request object down to plugins run_check calls.
So the only way is to rewrite this package flow or (personally I dislike this solution) use some middleware to set currently handled request to some global variable

Thank you for the quick answer @skarzi
Do you think it would be possible to add this feature to the package? I would be willing to help.

Yes, it would be 😉  If you want to contribute this feature I'd be happy to review it. Best, Joe


How to deploy this bot

Hello,
Can you please provide some help with deploying bot written with this botframework.
Not just run the bot on some server, but connecting it to MS botframework so I can access the bot in chats.
Thanks

Sorry if I misunderstood your question. You can deploy this in any server. It will connect with Microsoft bot framework as long as app id and secret is provided

It will actually not. I have to add a certificate or something more. But just deploying server as I'm running it locally it does not work.

Deploying this code as it is in heroku by just changing the end point in BotFramework website worked for me. Can you please check if there is any error messages coming ?

@7sedam7: I've run into that problem as well where you want to expose a server process running locally on your workstation to the greater internet, but only ephemerally, and without the hassle of setting up a web server, configuring firewalls and/or NAT'ing, etc.
For that purpose I'm using ngrok (ngrok.com), which does exactly that and works like a charm for my team and me. You simply start the Rack process and let it background/daemonize (or start another terminal process), and then start the Ngrok process with the right arguments, and you're set. :-)
I hope this helps you on the way.

It seems you all have misunderstood me. I do not have a problem with just deploying a server. My problem is that my deployed thing does not want to connect to an actual chat, I can not seem to use my bot from skype, fb messenger, whatever

You can set those in https://dev.botframework.com dashboard

What are those?
I know how to enable it there, when I deploy a dummy bot with c# .net, it just works, this one i can not get to work from skype/fb messenger/etc

I found out the problem. This rackup runs the tcp, while config on bot framework dashboard wants a https endpoint, besides that all is great :)

Thanks for the update.
Heroku provides https by default, you may have https proxy for other hosting providers.

I mean, to correct my answer. rackup writes up that it's listening on tcp://...... and if I paste that to bot framework dashboard ofc it does not work. tcp is just a protocol of transport layer, and https is technically on session layer, but usually implemented on application layer, so https can in lower layer also use tcp protocol and it usually runs on top of it.

Anyway thanks for a nice gem :)


brew install ibm-cloud-cli failing - Docker.app already exists

When trying to install ibm-cloud-cli using brew, it fails due to Docker.app already existing.  See below for the command and output.
brew install ibm-cloud-cli
Updating Homebrew...
==> Caveats
Cask ibm-cloud-cli installs files under /usr/local. The presence of such
files can cause warnings when running brew doctor, which is considered
to be a bug in Homebrew Cask.
==> Downloading https://public.dhe.ibm.com/cloud/bluemix/cli/bluemix-cli/1.2.3/IBM_Cloud_CLI_1.2.3.pkg
Already downloaded: /Users/charters/Library/Caches/Homebrew/downloads/6dbd843afcd4bacd0706f58e7df4af51f3c26965b2980af4a4cdca9c80defcfa--IBM_Cloud_CLI_1.2.3.pkg
==> Installing dependencies: docker
==> Downloading https://desktop.docker.com/mac/stable/48506/Docker.dmg
Already downloaded: /Users/charters/Library/Caches/Homebrew/downloads/9f0919492cb2c29dd0a102523544ac80a611db2302ce4322143a7bc6165cf17d--Docker.dmg
==> Verifying SHA-256 checksum for Cask 'docker'.
==> Installing Cask docker
==> Purging files for version <IP_ADDRESS>,48506 of Cask docker
Error: It seems there is already an App at '/Applications/Docker.app'.
charters@Grahams-MBP ~ %

@gcharters We don't maintain the homebrew formula. The root cause should be that you've already installed a Docker app. It's a quite common issue in homebrew, and it can be fixed by adding --force (I could not find a better solution).
The dependency declaration of ibm-cloud-cli seems include too much

@gcharters We don't maintain the homebrew formula. The root cause should be that you've already installed a Docker app. It's a quite common issue in homebrew, and it can be fixed by adding --force (I could not find a better solution).
The dependency declaration of ibm-cloud-cli seems include too much


Creating File bundles

This PR needs more tests:

test_bids_examples.py should be extended to count file bundles
test_file_bundle.py should be extended to explicitly test the various scenarios mentioned in #49


Plugin for fetching pages using a headless browser

This is a very first version of plugin
We are looking to improve further by
restricting resources (like css, fonts) as per url filter
Adding resources already downloaded in crawled data. For example while loading a web page on browser images on that web page are also downloaded. We can send these downloaded images to crawldb

@smadha Thanks a lot!! Appreciate your hard work  :)
There was a dependency conflict when running fetcher on top of Apache Spark cluster. It is now fixed.
@thammegowda Everything looks good from my end. Tested it locally & on Spark cluster. Do you want to give it a roll before it's merged?

Build failed
[INFO] ------------------------------------------------------------------------
[INFO] Building fetcher-jbrowser 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
...... ((Message trimmed))...
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ fetcher-jbrowser ---
[INFO]
[INFO] --- maven-bundle-plugin:2.5.0:manifest (default-cli) @ fetcher-jbrowser ---
[WARNING] Manifest edu.usc.irds.sparkler.plugin:fetcher-jbrowser:bundle:0.1-SNAPSHOT : Unused Private-Package instructions, no such package(s) on the class path: [!*]
[ERROR] Manifest edu.usc.irds.sparkler.plugin:fetcher-jbrowser:bundle:0.1-SNAPSHOT : Bundle-Activator not found on the bundle class path nor in imports: edu.usc.irds.sparkler.plugin.FetcherJBrowserActivator
[ERROR] Error(s) found in manifest configuration
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] sparkler-parent .................................... SUCCESS [  0.377 s]
[INFO] sparkler-api ....................................... SUCCESS [  4.851 s]
[INFO] sparkler ........................................... SUCCESS [ 45.358 s]
[INFO] sparkler-plugins ................................... SUCCESS [  0.069 s]
[INFO] urlfilter-regex .................................... SUCCESS [  2.329 s]
[INFO] fetcher-jbrowser ................................... FAILURE [  3.998 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:00 min
[INFO] Finished at: 2016-10-19T14:46:13-07:00
[INFO] Final Memory: 64M/1499M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.felix:maven-bundle-plugin:2.5.0:manifest (default-cli) on project fetcher-jbrowser: Error(s) found in manifest configuration -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :fetcher-jbrowser

@thammegowda Can you please try building Sparkler with the following:
mvn clean install

@karanjeets  If i do mvn clean install
and then try to run
14:59 $ bin/sparkler.sh inject -su http://www.isjavascriptenabled.com/
>>jobId = sjob-1476914418319
15:00 $ bin/sparkler.sh crawl -id sjob-1476914418319

I get
16/10/19 15:01:37 INFO FetchFunction$: FETCHING http://www.isjavascriptenabled.com/
16/10/19 15:01:37 INFO PluginService$: Felix Configuration loaded successfully
Bundle Found: org.apache.felix.framework
16/10/19 15:01:37 WARN FetchFunction$: FETCH-ERROR http://www.isjavascriptenabled.com/
java.util.NoSuchElementException: None.get
	at scala.None$.get(Option.scala:347)
	at scala.None$.get(Option.scala:345)
	at edu.usc.irds.sparkler.pipeline.FetchFunction$.apply(FetchFunction.scala:46)
	at edu.usc.irds.sparkler.pipeline.FetchFunction$.apply(FetchFunction.scala:34)
	at edu.usc.irds.sparkler.pipeline.FairFetcher.next(FairFetcher.scala:52)
	at edu.usc.irds.sparkler.pipeline.FairFetcher.next(FairFetcher.scala:29)
	at scala.collection.Iterator$$anon$12.next(Iterator.scala:444)
	at org.apache.spark.storage.MemoryStore.unrollSafely(MemoryStore.scala:285)
	at org.apache.spark.CacheManager.putInBlockManager(CacheManager.scala:171)
	at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:78)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:89)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
16/10/19 15:01:37 INFO ParseFunction$: PARSING  http://www.isjavascriptenabled.com/

@thammegowda This is strange. It worked for me just fine. Let me try with bin/sparkler.sh and see where things are wrong.

@thammegowda The issue is with the bin/sparkler.sh
It is using the conf directory in the build path. As per the new design in Plugin system, the path to bundles directory is generated at compile time using maven resources plugin. See here
Now, as I think, did you add conf directory in build path to pick post build changes? If yes, I can do something similar with the compiled conf directory.

@karanjeets
I am prepending 'con' to priority for configurations. Is there a way to restore that functionality?

@thammegowda Changes have been made. Please review and merge.

@karanjeets @smadha merged 💯 . This is a fantastic PR 👍


MUMUP-1986&1985 Change the "Add to favorites" & Already in favorites" button label

Add/Added to home instead of Add to home/Already in favorite

:+1:

I'll fix the merge conflicts

Thanks so much!
On Thu, Jul 30, 2015 at 1:52 PM, Tim Levett<EMAIL_ADDRESS>wrote:

I'll fix the merge conflicts
—
Reply to this email directly or view it on GitHub
https://github.com/UW-Madison-DoIT/angularjs-portal/pull/267#issuecomment-126436430
.

Merged manually.


This should fix the view tests on Windows

This should prevent us from getting issue reports or pull requests such as #8810 again and again.
This solution works because now the newlines should automatically be the same ones used in the stub files, and therefore Git cannot mess them up (depending on the core.autocrlf setting).
(This, by the way, is how it is done in the Blade compiler tests, too.)
@GrahamCampbell May I kindly ask you not to lock discussion on tickets like #8812 so quickly? That really hinders discussion and is totally not needed in such cases where nothing is escalating or getting out of hand. :beers:
/cc @jaschweder

@franzliedke perhaps squash those commits? :)

@franzliedke Thanks for looking into this. This looks like a good solution. :)

:rocket: :ship:

@franzliedke thanks for the clarification


[Pact -a] Add feature that generates the current time for creationTime

New feature request for pact -a:
If creationTime is omitted from PublicMeta, calculate the current epoch time and subtract some number of seconds (i.e. 15 seconds) from it.
This would improve the user experience as users won't have to manually calculate the epoch time and adjust it in case their clocks are out of sync with the node's clock.

In pact 3.5


Usage of metadata tagged blocks

(this is a follow up to issue https://github.com/kyamagu/psd-tools/issues/100)
Note that I am using the "psd-tools2" version -- if this is incorrect please let me know and I can switch to the standard "psd-tools" library.
My usage goal is to attach a UUID (36 character string) to a layer in a PSD file, and be able to read it back later.
In the linked issue there was an explanation regarding writing and reading metadata tagged blocks, which I will summarize here:
psd = PSDImage.open('tests/psd_files/colormodes/4x4_8bit_rgb.psd')
layer = psd[0]
metadata = layer.tagged_blocks.get_data(Tag.METADATA_SETTING)

item = psd_tools.psd.tagged_blocks.MetadataSetting(key=b'plug', data=b'0xFF')
metadata.append(item)

psd.save('output.psd')

This code seems to work if the metadata setting block has already been created by some process before (for example the method I have used in the old post), but not if the file is new / blank. In this case, when I call get_data (above, third line) the value of "metadata" is "None", and therefore I can not append to it. In the case that the metadata tagged block was not created before, how are we intended to create it?

@sona1111 You can probably try the following:
metadata = psd_tools.psd.tagged_blocks.MetadataSettings()
item = psd_tools.psd.tagged_blocks.MetadataSetting(key=b'plug', data=b'0xFF')
metadata.append(item)
layer.tagged_blocks.set_data(Tag.METADATA_SETTING, metadata)

This is highly experimental and you should not use this in production.

psd-tools2 is now merged to psd-tools, and psd-tools2 is discontinued

@kyamagu
Thank you, I really appreciate you quickly following up on my question. For your comment about it being "highly experimental" , I respect your caution, but I don't feel this warning is really quantifiable without additional context. I intend to use this functionality in "production", but I also intend to report back with reproducible examples of an issue if I find a bug and/or provide a fix for them myself. Hopefully we can improve the usable scope of the library over some time. ^_^
Thank you again for the assistance, it has been very helpful!


[SPARK-43680][SPARK-43681][SPARK-43682][SPARK-43683][PS] Fix NullOps for Spark Connect

What changes were proposed in this pull request?
This PR proposes to fix NullOps test for pandas API on Spark with Spark Connect.
This includes SPARK-43680, SPARK-43681, SPARK-43682, SPARK-43683 at once, because they are all related similar modifications in single file.
Why are the changes needed?
To support all features for pandas API on Spark with Spark Connect.
Does this PR introduce any user-facing change?
Yes, NullOps.lt,  NullOps.le, NullOps.ge, NullOps.gt are now working as expected on Spark Connect.
How was this patch tested?
Uncomment the UTs, and tested manually.

merged to master


Can't modify logging level in liquibase-maven-plugin

Environment
Liquibase Version: 4.4.3
Liquibase Integration & Version:  Maven
Description
Can't change logging level using liquibase-maven-plugin.
How "logging" property is supposed to work when this line is commented out?
Steps To Reproduce
Set the plugin like this:
<plugin>

   <groupId>org.liquibase</groupId>

   <artifactId>liquibase-maven-plugin</artifactId>

   <configuration>

     <logging>WARNING</logging>

   </configuration>
...
</plugin>

Run whatever command you want and you'll still see "[INFO] ..." logs.
Actual Behavior
Can't change log level.
Expected/Desired Behavior
Log level changeable using the aforementioned property.

Hi @giacgbj Thanks for bringing this issue to our attention.  We will add this to the list of issues that we are processing.

With 4.0 we switch the maven plugin to respect the standard maven logging logic vs. something custom of our own. So you can use the -q, -X or -e flags or the setup in https://maven.apache.org/maven-logging.html
But, I created #2261 to better tell people that are using the logging configuration that it is now deprecated.


powertoys run is searching out something unexpected

Microsoft PowerToys version
0.60.1
Running as admin

[ ] Yes

Area(s) with issue?
PowerToys Run
Steps to reproduce

(I've disabled windows search plugin and folder plugin)
✔ Expected Behavior
"视频" doesn't appear.
❌ Actual Behavior
"视频" is the first result.
Other Software
No response

the english name of "视频" is "videos", I think this is why it appears, but as I said, I've disabled windows search and folder plugins.

And if I search something like "downloads" or "music", nothing unexpected appears

bug report zip:
PowerToysReport_2022-07-27-22-37-14.zip

Unfortunately and fortunately, I cannot reproduce this now🤣.closing this issue


Ask for permission when it's needed

I value my privacy strongly and don't use any of Gear Fit's features that might require viewing SMS messages, manage phone calls, or access contacts.
Could you please ask for these permissions only in the places where they're needed and make the rest of the app work without them?

I'll try improving this behavior in the next version


[Seafile] Update template description to include CSRF Verification

When updating to the Seafile 11, a CSRF Verification Failed error occurs.  Need to add directions to template to fix CSRF verification.
Please see https://www.reddit.com/r/unRAID/comments/1f4ebke/guide_for_installing_the_latest_version_of/ for more details.

Added to seafile-guide.md


PulseDefaults from_dict() method doesn't work with parameterized amplitudes.

Information
Qiskit Terra version: 0.18.3

This issue is closely realted to issue "Unable to load pulse defaults due to amp being a list #4162".
What is the current behavior?
PulseDefaults from_dict() method fails if a Parameter variable is chosen as amplitude parameter in a command sequence.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_18540/3762969180.py in <module>
      5             pulse_library = [],
      6             cmd_def = [
----> 7                 Command.from_dict({
      8                     'name':'cooling',
      9                     'qubits': [2],

~\AppData\Roaming\Python\Python38\site-packages\qiskit\providers\models\pulsedefaults.py in from_dict(cls, data)
    155         in_data = copy.copy(data)
    156         if "sequence" in in_data:
--> 157             in_data["sequence"] = [
    158                 PulseQobjInstruction.from_dict(x) for x in in_data.pop("sequence")
    159             ]

~\AppData\Roaming\Python\Python38\site-packages\qiskit\providers\models\pulsedefaults.py in <listcomp>(.0)
    156         if "sequence" in in_data:
    157             in_data["sequence"] = [
--> 158                 PulseQobjInstruction.from_dict(x) for x in in_data.pop("sequence")
    159             ]
    160         return cls(**in_data)

~\AppData\Roaming\Python\Python38\site-packages\qiskit\qobj\pulse_qobj.py in from_dict(cls, data)
    242             data["discriminators"] = discriminators_obj
    243         if "parameters" in data and "amp" in data["parameters"]:
--> 244             data["parameters"]["amp"] = _to_complex(data["parameters"]["amp"])
    245
    246         return cls(name, t0, **data)

~\AppData\Roaming\Python\Python38\site-packages\qiskit\qobj\pulse_qobj.py in _to_complex(value)
    267         return value
    268
--> 269     raise TypeError(f"{value} is not in a valid complex number format.")
    270
    271

TypeError: cool_amp is not in a valid complex number format.

Steps to reproduce the problem
Call the PulseDefaults.from_dict() method with a parameter type argument in the default_dict.
from qiskit.providers.models import  PulseDefaults, Command
from qiskit.circuit import Parameter

cool_amp = Parameter('cool_amp')

PulseDefaults(
            qubit_freq_est = qubit_freq_est,
            meas_freq_est = meas_freq_est,
            buffer = 10,
            pulse_library = [],
            cmd_def = [
                Command.from_dict({
                    'name':'cooling',
                    'qubits': [2],
                    'sequence':  [
                        PulseQobjInstruction(
                            name = 'parametric_pulse',
                            ch = 'd2',
                            t0 = 0 + min_wait_time,
                            pulse_shape = 'constant',
                            parameters = {'duration': 2, 'amp': cool_amp}).to_dict(),
                        ]})])

What is the expected behavior?
A parameterised amplitude in the instruction_schedule_map in the backend should in my opinion be able to contain a parameterised amplitue.  Such that custom circuit instructions can contain amplitude values that then can be bound to the parameterised amplitude value in the backend.
Suggested solutions
Either don't use the _to_complex function in the from_dict() method in the pulse_qobj module or rewrite the _to_complex function to allow for parameter type variables and to solve issue #4162 also complex numbers in a list format.

The Qobj is being deprecated as an IBM-specific API in favour of the new #5885 backend interface. Is there a particular reason you need to use the Qobj rather than add a parameterized schedule to the backend.defaults().instruction_schedule_map directly?

We are going to support qpy format also for schedule #7300 Probably this is good replacement of current Qobj.

The Qobj is being deprecated as an IBM-specific API in favour of the new #5885 backend interface. Is there a particular reason you need to use the Qobj rather than add a parameterized schedule to the backend.defaults().instruction_schedule_map directly?

I just tested the instruction_schedule_map.add() function and even ScheduleBlock objects with parameterised durations are possible, which is great. I never noticed that and was working with Commands to define the PulseDefaults of a Backend, which does not allow for parameterised durations. According to the API documentation  of PulseDefaults, it still requires Commands that require a sequence of PulseQobjInstructions and there is no deprication warning for those. I am a little confused. The idea is to create the ìnstruction_schedule_map in the backend not via the Commands in the PulseDefaults during the initialization of the Backend, but to instatiate a Backend without ìnstruction_schedule_map entries and add them post initialization by the ìnstruction_schedule_map.add() method?
If Qobj is deprecated, does that mean that the assemble() function will become depracated as well? I have a probject that relies on the PulseQobj created by assembling a Schedule.
A bit background on how this is intended would be much appreciated. Thank you.

The idea is to create the ìnstruction_schedule_map in the backend not via the Commands in the PulseDefaults during the initialization of the Backend, but to instatiate a Backend without ìnstruction_schedule_map entries and add them post initialization by the ìnstruction_schedule_map.add() method?

That is the rough idea. In the new interface you will instantiate the backend as a provider and add the instruction_schedule_map to the Target at construction time.

If Qobj is deprecated, does that mean that the assemble() function will become depracated as well? I have a probject that relies on the PulseQobj created by assembling a Schedule.

Yes, the plan is to deprecate this from official recommended workflow. However, it will still be available in qiskit for the near term and will be then ported to the https://github.com/Qiskit/qiskit-ibmq-provider. You should always be able to copy the code into your project so long as you maintain the appropriate copywrite/licensing.

Given the above comments, I'm going to close this issue as "stale" or "obsolete" now.  If there's more to say on it, please feel free to re-open.


LSVQ datasets

Hello. Thanks for your great work. I am trying to retrain the model with LSVQ datasets. However, the form of LSVQ is disabled, and I can not get the datasets. Would you share your LSVQ datasets with me? Thank you very much.

+1

https://huggingface.co/datasets/teowu/LSVQ-videos


‘Illegal instruction (core dumped)’ problem when running go1_visual_embedding.py or a1_ros_run.py

When I tried to run the command in T_visual and T_gru, they raised Illegal instruction (core dumped) and ended.
(parkour_venv) unitree@nx:~/parkour_deploy$ python go1_visual_embedding.py --logdir Nov02_16-18-16_674k_distill_crawljumpjumpleap_vDelay0.25-0.30_camPitch0.52_depthMax2.0_jumpNov02_11-07-49_leapVel1.5_jumpOffset0.05_oracleResetWhenSkill_fricMax2.0_noTanh_fromOct28_16-16-58
Illegal instruction (core dumped)
(parkour_venv) unitree@nx:~/parkour_deploy$ python a1_ros_run.py --mode upboard --logdir Nov02_16-18-16_674k_distill_crawljumpjumpleap_vDelay0.25-0.30_camPitch0.52_depthMax2.0_jumpNov02_11-07-49_leapVel1.5_jumpOffset0.05_oracleResetWhenSkill_fricMax2.0_noTanh_fromOct28_16-16-58
Illegal instruction (core dumped)
(parkour_venv) unitree@nx:~/parkour_deploy$

The unitree_legged_real command went well. I am presently running in the unitree@nx. I also tried with unitree@unitree-desktop, the same problem occured.
What's the problem and how to deal with it? Thanks!

I thought it was because I didn't go into the develop mode, so the core might be occupied by unitree's process. Then I tried L2+A+A, L2+B , L1+L2+start to go into the normal mode and tried the command again. However, the issue of T_visual and T_gru was not solved.

Hmmm, I haven't experienced this before. It seems like some version problems when running the Python program.
Could you locate which function call that leads to the error? e.g. creating a pytorch tensor, tensor to cuda, etc.

I am not able to locate the core dump source code.
Good news is that I tried to create a new venv and the problem was solved. I finally realsed that the reason might be the wrong numpy version. I removed the version requirement at the first time when I make the venv.
Now it's able to run and render the corresponding information normally.
Thanks a lot for your inspiration!


Add localization rules

What's the Problem?
Japanese translation of Files differs depending on the resource provider.
Solution/Idea
I think we should create TranslationRules.svx file with each language section on files.community website.
Contain this:

Localization rules for each language.
The list of each localizarion reviewers(needs to be confirmed by the native speaker).
Any of the following:

If contributer provides localization resources, Files members will mention to this docs to make sure they follow the rules.
Create an issue template for localization and check the checklist to see if you follow the rules.
List resource providers in each language as reviewers in Docs from previous contributions.

Alternatives
If this is required only in Japanese, activate 3.3.
Priorities
No response
Files Version
No response
Windows Version
No response
Comments
No response

@yaichenbaum How do you think? it's good?

@onein528 I think we can add this link to the documentation to https://www.microsoft.com/en-us/language/styleguides

hmm...
okey. It is difficult and redundant to understand even from a native perspective.
How do you @h1g0 think?

I' m new to this OSS community and not familiar with the detailed rules, but my thoughts are as follows:

Basically, I think it's fine to follow the Microsoft guidelines above.
However, it is preferable to have a page or file that can briefly describe the content and rules discussed in previous PRs and Issues.
I also think that having a page or file where localization reviewers can be listed would simplify the review process and improve the quality when new localization PRs are created.

it is same as what I think.

How about assigning @h1g0 or @onein528 when someone providing ja-JP resources?

@onein528 I'm only able to request reviews from people in the Files org, the best thing to do is keep an eye on open pull requests to see if there is something that you'd like to review.

Sorry, I dont know very well about organization privileges. so please continue by a non-member way to get app id/secret.

we may be use this. you can access this page(your access role is ‘Maintain’)?

mention too?
If that's the case, I found out that I can't achieve the goal, so I'll go to see the PR regularly.


Improve cpu info for non-x86 architectures

Currently, it uses a generic build script.
This script assumes:
const REGISTER_SIZE = 16
const REGISTER_COUNT = 16
const CACHELINE_SIZE = 64
const SIMD_NATIVE_INTEGERS = true

If any of these are violated, dependent libraries (e.g., LoopVectorization) are likely to produce suboptimal code. If these numbers undershoot, that would just mean some performance is left on the table, but it's likely to perform reasonably well.
If these numbers overshoot, performance consequences could be dire. Register spills galore.
I believe some ARM CPUs do not have SIMD Float64, so perhaps this should be handled somehow.
Ideally, we'd use a library like CpuId.jl to query hardware info, like we do for AMD and Intel.

arm64 is available as a Travis test environment, it might be good to set up testing in a branch and see where expectations match reality. It's Arm v8.0-A, SBSA Level 3, so it should support 64-bit floating point SIMD.

Yes, that comment was a little out of date. You're right.
It would be great to start testing it on ARM.
I'll have to figure out how to do feature detection.
Are CPUs that don't support Float64 SIMD actually common? What about raspberry pis?
Would also be good to know if 32 floating point registers are the norm. That would help with a lot of operations like BLAS.
Testing would be good to know that it works and there aren't unexpected errors or crashes, but performance benchmarks would be helpful as well.

I think all AArch64/arm64 with SIMD will support Float64. So I think you'd be okay to make that particular assumption based on the architecture. AArch32 is more complicated since it represents a lot more versions of ARM. Based on the ARM v8 spec I think some AArch32 processors can run SIMD operations on regular registers as well as SIMD registers, which makes that more complicated. I'm pretty sure Raspberry PIs have been AArch32 for a long time. I'm not sure if there's new ones that are AArch64 but the vast majority will be AArch32. That said, AArch64 is going to be a huge new user base, since Amazon is heavily advertising their new Graviton2 processors, and Apple is switching to it for seemingly all of their products.
Not sure about the number of registers. The ARMv8-A spec is here: https://documentation-service.arm.com/static/5f20515cbb903e39c84dc459 but that's only one spec.
For feature detection on Linux, the kernel provides asm/hwcap.h (e.g., https://android.googlesource.com/kernel/msm/+/android-7.1.0_r0.2/arch/arm64/include/uapi/asm/hwcap.h). You can get the feature data using hwcap = ccall(:getauxval, Culong, (Culong,), 16) (AT_HWCAP is 16, apparently) and then & with the feature ID in the header. Not sure what other operating systems provide.

There's also this for feature detection:
using Libdl
llvmlib = Libdl.dlopen(only(filter(lib->occursin(r"LLVM\b", basename(lib)), Libdl.dllist())))
gethostcpufeatures = Libdl.dlsym(llvmlib, :LLVMGetHostCPUFeatures)
features_cstring = ccall(gethostcpufeatures, Cstring, ())
features = filter(ext -> (m = match(r"\d", ext); isnothing(m) ? true : m.offset != 2 ), split(unsafe_string(features_cstring), ','))

But from what I recall of seeing discussions on Julia issues/PRs, this is going to be incomplete with ARM.
I'll still have to see how to translate hwcap into the information I want, the most important of which is the 4 constants from the opening post.

I think you can find cache info from the same kernel function getauxval using a different AT_ value: https://man7.org/linux/man-pages/man3/getauxval.3.html
Not sure about that register information. @PallHaraldsson answered a related question on Quora so he might have some suggestions.
I'll note that the ARMv8 spec linked above does tell you exactly how many registers of what kind it has, so worst-case scenario you could hardcode the numbers by architecture.

I think hard coding the number of registers would be worth it.
Currently, LoopVectorization only uses the number of vector registers and just hopes performance isn't hurt too badly by spilling others (it does spill integer registers in some cases, but I think doing so is worthwhile in those cases).
For cross platform cache information, perhaps we should depend on Hwloc.jl?
Although I'm not opposed to making OS-specific performance optimizations (already using faster SIMD special functions on Linux).

I believe I am struggling with the same issue as others have reported: errors compiling LoopVectorization.
The problem, I believe, is that Hwloc does not report any L3 Cache, which is assumed to exist in line 8 of topology.jl.
Here's what that looks like on my Nano:
[ Info: Precompiling VectorizationBase [3d5dd08c-fd9d-11e8-17fa-ed2836048c2f]
ERROR: LoadError: LoadError: type NullAttr has no field size
Stacktrace:
 [1] getproperty(::Hwloc.NullAttr, ::Symbol) at ./Base.jl:33
 [2] top-level scope at /home/doug/.julia/packages/VectorizationBase/26Yla/src/topology.jl:8
 [3] include(::Function, ::Module, ::String) at ./Base.jl:380
 [4] include at ./Base.jl:368 [inlined]
 [5] include(::String) at /home/doug/.julia/packages/VectorizationBase/26Yla/src/VectorizationBase.jl:1
 [6] top-level scope at /home/doug/.julia/packages/VectorizationBase/26Yla/src/VectorizationBase.jl:270
 [7] include(::Function, ::Module, ::String) at ./Base.jl:380
 [8] include(::Module, ::String) at ./Base.jl:368
 [9] top-level scope at none:2
 [10] eval at ./boot.jl:331 [inlined]
 [11] eval(::Expr) at ./client.jl:467
 [12] top-level scope at ./none:3
in expression starting at /home/doug/.julia/packages/VectorizationBase/26Yla/src/topology.jl:8
in expression starting at /home/doug/.julia/packages/VectorizationBase/26Yla/src/VectorizationBase.jl:270

Digging deeper,
julia> using Hwloc
julia> const TOPOLOGY = Hwloc.topology_load();
julia> const CACHE = TOPOLOGY.children[1].children[1];
        D2: L0 P-1 L2Cache  Cache{size=2097152,depth=2,linesize=64,associativity=16,type=Unified}
            D3: L0 P-1 L1Cache  Cache{size=32768,depth=1,linesize=64,associativity=2,type=Data}
                D4: L0 P0 Core
                    D5: L0 P0 PU
            D3: L1 P-1 L1Cache  Cache{size=32768,depth=1,linesize=64,associativity=2,type=Data}
                D4: L1 P1 Core
                    D5: L1 P1 PU
            D3: L2 P-1 L1Cache  Cache{size=32768,depth=1,linesize=64,associativity=2,type=Data}
                D4: L2 P2 Core
                    D5: L2 P2 PU
            D3: L3 P-1 L1Cache  Cache{size=32768,depth=1,linesize=64,associativity=2,type=Data}
                D4: L3 P3 Core
                    D5: L3 P3 PU

Subsequently (from topology.jl, line 8)
julia> CACHE.children[1].children[1]
                D4: L0 P0 Core
                    D5: L0 P0 PU
julia> CACHE.children[1].children[1].attr
julia>

I'd be happy to help fix this!
-thanks
doug

I shouldn't assume that. Lots of non-x86 CPUs only have 2 levels of cache, including the M1.

I'd be happy to help fix this!

You're welcome to make a PR!
You could set all the corresponding references to a missing cache level to nothing when it isn't there.


Slider looses responsiveness on zooming and reloading page

Steps to reproduce

Drag the slider
Zoom the page to 150%
Reload the page
The slider looses its responsiveness and range becomes distorted

Demo: http://jsfiddle.net/cwhgLcjv/ (fork this example and update the link)
Expected behaviour
It should be responsive despite zooming and on reloading range should not get distorted
Actual behaviour
The slider looses its responsiveness and range becomes distorted

How are you zooming? I've tried on the demo website and everything is fine.

It's working fine now, Thanks!


Band of Blades Inlinerollresult color is same as background

Rolling Marshal as an Officer with Risky + Standard + 2 dice.
https://i.imgur.com/EpxXljo.png
.sheet-rolltemplate-blades .inlinerollresult sets the background: transparent while  .sheet-rolltemplate-blades .sheet-holder.sheet-action uses color: # 333. This makes the font effectively invisible.
Recommend setting color to something lighter like #fdfdfe or adding light background to the template.

@joesinghaus My apologies for not reporting this via the directions on the readme.

(no worries about wrong reporting location)
Hi Cassie, this problem is essentially #6166, which has now gone unfixed for weeks. The background for the roll template is a GIF which quickly moves on to a better contrast, but since the image proxy is broken, it is stuck on a bad frame.

If the image proxy issue is going to essentially be unfixed for O(months), I'm going to ask Tim if he can cook up a static background that works, and I can replace the gifs with that...


Update kubevirt to 1.3.0

We want to test latest version to discover regressions early.

Tested with basic test:
$ basic-test/run -c configs/kubevirt/vm-pvc-k8s-regional.yaml envs/regional-dr-kubevirt.yaml
2024-07-23 15:09:13,818 INFO    [deploy] Deploying application
2024-07-23 15:09:13,818 INFO    [deploy] Deploying application 'vm-pvc'
2024-07-23 15:09:15,925 INFO    [deploy] Waiting for 'placement.cluster.open-cluster-management.io/placement' decisions
2024-07-23 15:09:16,211 INFO    [deploy] Application running on cluster 'dr1'
2024-07-23 15:09:16,389 INFO    [enable-dr] Enable DR
2024-07-23 15:09:16,479 INFO    [enable-dr] Disabling OCM scheduling for 'placement.cluster.open-cluster-management.io/placement'
2024-07-23 15:09:16,666 INFO    [enable-dr] Waiting for 'placement.cluster.open-cluster-management.io/placement' decisions
2024-07-23 15:09:17,045 INFO    [enable-dr] waiting for namespace vm-pvc
2024-07-23 15:09:17,196 INFO    [enable-dr] Waiting until 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' reports status
2024-07-23 15:09:17,793 INFO    [enable-dr] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' Available condition
2024-07-23 15:09:17,997 INFO    [enable-dr] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' PeerReady condition
2024-07-23 15:09:18,196 INFO    [enable-dr] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' first replication
2024-07-23 15:10:47,114 INFO    [enable-dr] DR enabled
2024-07-23 15:10:47,291 INFO    [failover] Fail over application
2024-07-23 15:10:47,399 INFO    [failover] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' Available condition
2024-07-23 15:10:47,604 INFO    [failover] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' PeerReady condition
2024-07-23 15:10:47,813 INFO    [failover] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' first replication
2024-07-23 15:10:48,011 INFO    [failover] Waiting for 'placement.cluster.open-cluster-management.io/placement' decisions
2024-07-23 15:10:48,399 INFO    [failover] Starting failover for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' to cluster 'dr2'
2024-07-23 15:10:48,604 INFO    [failover] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' Available condition
2024-07-23 15:11:17,367 INFO    [failover] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' PeerReady condition
2024-07-23 15:13:47,247 INFO    [failover] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' first replication
2024-07-23 15:14:17,376 INFO    [failover] Application was failed over
2024-07-23 15:14:17,551 INFO    [relocate] Relocate application
2024-07-23 15:14:17,654 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' Available condition
2024-07-23 15:14:17,861 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' PeerReady condition
2024-07-23 15:14:18,068 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' first replication
2024-07-23 15:14:18,258 INFO    [relocate] Waiting for 'placement.cluster.open-cluster-management.io/placement' decisions
2024-07-23 15:14:18,622 INFO    [relocate] Starting relocate for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' to cluster 'dr1'
2024-07-23 15:14:18,822 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' phase 'Relocated'
2024-07-23 15:16:47,346 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' Available condition
2024-07-23 15:16:47,554 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' PeerReady condition
2024-07-23 15:17:17,360 INFO    [relocate] Waiting for 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc' first replication
2024-07-23 15:17:17,456 INFO    [relocate] Application was relocated
2024-07-23 15:17:17,633 INFO    [disable-dr] Disable DR
2024-07-23 15:17:17,724 INFO    [disable-dr] Deleting 'drplacementcontrol.ramendr.openshift.io/vm-pvc-drpc'
2024-07-23 15:17:47,092 INFO    [disable-dr] DR was disabled
2024-07-23 15:17:47,270 INFO    [undeploy] Deleting application
2024-07-23 15:17:47,271 INFO    [undeploy] Undeploying application 'vm-pvc'
2024-07-23 15:17:54,634 INFO    [undeploy] Application was deleted

@nirs there are a lot of place to upgrade versions, which implies the possibility of error. Maybe, we can somehow concentrate all the versions in one place?

@nirs there are a lot of place to upgrade versions, which implies the possibility of error. Maybe, we can somehow concentrate all the versions in one place?

Yes this is needed, not only for kubevirt. I started to work on this in #1428


Plan ways for site to be interactive

Identify weather codes and translate that to specific disasters (alien invasions, etc.). -Ken
Choice of Superhero for that specific disaster. -Ken

Let user save favorites for superhero and villain


[Test] Fix conn config path

Description
Temporary change follow-up https://github.com/astronomer/astro-sdk/pull/935
What is the current behavior?

What is the new behavior?

Does this introduce a breaking change?
Checklist

[ ] Created tests which fail without the change (if possible)
[ ] Extended the README / documentation, if necessary

This is Temporary chnages


Don't create comments on unmodified lines

Codecov Report
Attention: Patch coverage is 94.33962% with 3 lines in your changes missing coverage. Please review.

Project coverage is 95.16%. Comparing base (e801589) to head (34b4024).

Files with missing lines
Patch %
Lines

internal/reporter/github.go
93.61%
2 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1123      +/-   ##
==========================================
+ Coverage   95.12%   95.16%   +0.03%
==========================================
  Files         102      102
  Lines       10201    10233      +32
==========================================
+ Hits         9704     9738      +34
  Misses        346      346
+ Partials      151      149       -2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


when submitting tx, DefaultEventHandlerOptions timeout

When committing event to eventstore, there is timeout error thrown but the event is written to eventstore successfully.
Reason: default is 300s timeout.
**Application Log:**
2021-09-16T13:29:40+08:00 [debug]: URL: "/pbc/order/confirm" ([relay] relayService.js)
2021-09-16T13:29:40+08:00 [debug]: HEADERS: {"authorization":"aaaaaaaaaaaa:aaaaaaaaaaaa","content-md5":"K3ZheN6GsVHC8hFIGNEiIQ==","date":"Thu, 16 Sep 2021 05:29:40 GMT","content-type":"application/json","content-length":"105","host":"uat2-relay11.fdi.hktfpcl.com","connection":"Keep-Alive","user-agent":"Apache-HttpClient/4.5.3 (Java/1.8.0_151)","cookie":"JSESSIONID=28C923C7BCE08D4589BC7E5E8E23415C","accept-encoding":"gzip,deflate"} ([relay] relayService.js)
2021-09-16T13:29:40+08:00 [debug]: PARAMS: [{"actionResponse":"A","comment":"Confirm PO","poId":"6837401441309573120","versionNo":1}] ([relay] relayService.js)
2021-09-16T13:29:40+08:00 [info]: ProxyReq Finish<PHONE_NUMBER>791 ([relay] relayService.js)
2021-09-16T13:29:40+08:00 [debug]: [PERFTEST]{"id":"185cfd00e437b0995f12a9dd3650608c","url":"[object Object]","method":"POST","proxyReqStarts":1631770180789,"proxyReqFinish":1631770180791 ([relay] relayService.js)
2021-09-16T13:29:49+08:00 [info]: ProxyRes Starts<PHONE_NUMBER>730 ([relay] relayService.js)
2021-09-16T13:29:49+08:00 [info]: Message processed with response 0 - '{"id":"185cfd00e437b0995f12a9dd3650608c","proxyReqStarts":1631770180789,"proxyReqFinish":1631770180791,"method":"POST","url":{"url":"/pbc/order/confirm","query":{}},"contentType":"application/json","proxyResStarts":1631770189730,"proxyResFinish":1631770189731,"statusCode":200,"statusMessage":""}' ([relay] relayService.js)
2021-09-16T13:29:49+08:00 [debug]: [PERFTEST]{"id":"185cfd00e437b0995f12a9dd3650608c","url":"/pbc/order/confirm","method":"POST","status":200,"proxyResStarts":1631770189730,"proxyResFinish":1631770189731} ([relay] relayService.js)
2021-09-16T13:29:49+08:00 [info]: ProxyRes Completed<PHONE_NUMBER>732 ([relay] relayService.js)
2021-09-16T13:29:49+08:00 [debug]: [PERFTEST]{"sid":1631770189732-0,"count":1,"redisEventStart":1631770189833} ([sniffer] snifferSubscription.js)
2021-09-16T13:29:49+08:00 [debug]: [PERFTEST]{"id":"185cfd00e437b0995f12a9dd3650608c","sid":1631770189732-0,"redisResFinish":1631770189834} ([sniffer] snifferSubscription.js)
2021-09-16T13:29:49+08:00 [info]: [dispatcher] actionName: command_create, tx_id: f7Yzkw dispatched ([fabric-cqrs] createRepository.js)
2021-09-16T13:29:50+08:00 [debug]: gateway connected: myadmin ([fabric-cqrs] getNetwork.js)
2021-09-16T05:34:51.471Z - warn: [TransactionEventHandler]: strategyFail: commit failure for transaction "46446f1665d6e8fcb42c6e5b51f4598746a9965fc510428f50705741a7268e12": TimeoutError: Event strategy not satisfied within the timeout period. No response received from peers: peer0-etc, peer1-etc
    at TransactionEventHandler.timeoutFail (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:125:23)
    at Timeout._onTimeout (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:112:18)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7) {
  transactionId: '46446f1665d6e8fcb42c6e5b51f4598746a9965fc510428f50705741a7268e12'
}
2021-09-16T05:34:51.472Z - error: [EventService]: unregisterEventListener[peer0-etc.cdi.uat:7051] - #11 - event listener was not found
Error: eventListener not found
    at EventService.unregisterEventListener (/home/app/node_modules/fabric-common/lib/EventService.js:750:11)
    at EventListener.unregisterEventListener (/home/app/node_modules/fabric-common/lib/EventListener.js:135:21)
    at CommitListenerSession.close (/home/app/node_modules/fabric-network/lib/impl/event/commitlistenersession.js:33:27)
    at Object.removeListener (/home/app/node_modules/fabric-network/lib/impl/event/listenersession.js:22:17)
    at NetworkImpl.removeCommitListener (/home/app/node_modules/fabric-network/lib/network.js:197:27)
    at TransactionEventHandler.cancelListening (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:81:22)
    at TransactionEventHandler.strategyFail (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:144:14)
    at TransactionEventHandler.timeoutFail (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:126:14)
    at Timeout.<anonymous> (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:112:18)
    at listOnTimeout (internal/timers.js:554:17)
2021-09-16T13:34:51+08:00 [error]: An uncaught error occurred! ([rl-etc] sniffer.js)
2021-09-16T13:34:51+08:00 [error]: Error: eventListener not found
    at EventService.unregisterEventListener (/home/app/node_modules/fabric-common/lib/EventService.js:750:11)
    at EventListener.unregisterEventListener (/home/app/node_modules/fabric-common/lib/EventListener.js:135:21)
    at CommitListenerSession.close (/home/app/node_modules/fabric-network/lib/impl/event/commitlistenersession.js:33:27)
    at Object.removeListener (/home/app/node_modules/fabric-network/lib/impl/event/listenersession.js:22:17)
    at NetworkImpl.removeCommitListener (/home/app/node_modules/fabric-network/lib/network.js:197:27)
    at TransactionEventHandler.cancelListening (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:81:22)
    at TransactionEventHandler.strategyFail (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:144:14)
    at TransactionEventHandler.timeoutFail (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:126:14)
    at Timeout.<anonymous> (/home/app/node_modules/fabric-network/lib/impl/event/transactioneventhandler.js:112:18)
    at listOnTimeout (internal/timers.js:554:17) ([rl-etc] sniffer.js)
2021-09-16T13:50:48+08:00 [info]: ProxyReq Starts<PHONE_NUMBER>684 ([relay] relayService.js)
2021-09-16T13:50:48+08:00 [debug]: URL: "/pbc/invoices/receive" ([relay] relayService.js)

I assert that is network issue. Close here, and will continue troubleshooting in the deployment repo


Pick a better default folder

Right now we're defaulting to 'New Folder' when creating new rules. It might be clearer, and more obvious what's going on if we were to use a folder called 'Paperwork'. We should probably localise this name as well.

See #450.

Done in f4f6b4d526035e770867a4e7b6c190dd3a099e1e. No localisation at this point.


[DINGO-1372] Add strings for new warning validation

🐕
/cc @zendesk/dingo
Description
Adds strings for new warning about default settings for secure/hidden params.
@zendesk/localization
References
https://zendesk.atlassian.net/browse/DINGO-1372
Before merging this PR

[ ] Fill out the Risks section
[ ] Think about performance and security issues

Risks

[RUNTIME] Can this change affect apps rendering for a user? No
None: just a string

@magalimarquetzen Sure. How's it look now?


Incorrect Request Payload Format

Able to lock/unlock vehicle but start climate not working. Relevant lines from log below.
[hyundai_kia_connect_api.KiaUvoAPIUSA] hyundai_kia_connect_api - Error: unknown error response {"status":{"statusCode":1,"errorType":1,"errorCode":9001,"errorMessage":"Incorrect request payload format"}}

What version?

2.5.3
home-assistant_kia_uvo_2023-01-13T20-42-31.349Z.log

Could you post the YAML you passed in?  The last error states no temp was set. I can handle that in the future.  Other ones aren't clear.

Current master branch handles if you don't pass it the settings needed for a basic call.  However you may have more on the go.

Not sure why temperature isn't being passed in the service call, here's the YAML.
service: kia_uvo.start_climate
data:
  temperature: 71

Log using this attached
home-assistant_kia_uvo_2023-01-14T02-15-04.192Z.log

Looks like you aren't passing nearly enough parameters.  Multiple items more than that I am sure are in your mobile app.  This is mentioned in the service call at the top:

That being said if you update from master in HACS it should work even if you lack the options for your car.

okay looks like using the master branch worked. Thank you!


[BUG]: Type Inference Fails for Separated Method Chaining

What version of drizzle-orm are you using?
0.29.4
What version of drizzle-kit are you using?
0.20.14
Describe the Bug
When using Drizzle ORM to construct queries for PostgreSQL, specifically applying method chaining to build queries in a separated manner, the resulting type inference does not align with expectations. The issue arises when attempting to extend the query with additional method chains (e.g., joins) outside the initial query construction context.
import { eq } from 'drizzle-orm'
import { pgTable, serial, text } from 'drizzle-orm/pg-core'
import { drizzle } from 'drizzle-orm/postgres-js'
import postgres from 'postgres'

export const users = pgTable('users', {
  id: serial('id').primaryKey(),
})

export const cars = pgTable('cars', {
  id: serial('id').primaryKey(),
  hoge: text('hoge').notNull(),
  user_id: serial('id')
    .references(() => users.id)
    .notNull(),
})

const flag = true

const connectionString = process.env.DATABASE_URL || ''
const client = postgres(connectionString)
const db = drizzle(client)

const query = db.select().from(users)

if (flag) {
  query.innerJoin(cars, eq(users.id, cars.user_id))
}

const res = await query.execute()

Expected behavior
Expected Type Structure
const res: {
  users: {
      id: number;
  };
  cars: {
      id: number;
      hoge: string;
      user_id: number;
  };
}[]

Actual Inferred Type
const res: {
    id: number;
}[]

Environment & setup
typescript: 5.3.3

I believe this is a typescript limitation. If you do:
let query = db.select().from(users)

if (flag) {
  query = query.innerJoin(cars, eq(users.id, cars.user_id))
}

Typescript will complain because you are changing the type.
The solution is pretty simple:
const query = db.select().from(users)
let joinedQuery;

if (flag) {
  joinedQuery = query.innerJoin(cars, eq(users.id, cars.user_id))
}

const res = joinedQuery ? await joinedQuery.execute() : await query.execute()

Once you set a type to the variable, you can't change it.
I'll close the issue but feel free to reopen if you thing drizzle does indeed have a bug.


App home screen crashes after a long time of scrolling down

The Issue
When on home default screen of app > trending for everyone, when you scroll down quite a long way (about 8-10 full screen-lengths down) it errors with attached red error screen.
Steps to reproduce

Load app
Hamburger menu > all content
Trending for everyone
Scroll down the screen about  #20 full screen depths - it will trigger the error page with top message 'URL cannot include a space'.

Expected behaviour
It shouldn't error ;-)
Actual behaviour
Standard red error screen as attached below with error message 'URL cannot include a space':
System Configuration

Device: Google Pixel
Android version: 10

Anything Else
Is it possible that the error was triggered by an item pulling in a preview image or content file that tripped the error? Maybe bad file formatting?
It happened after 6 test tries - each time about 1 or 2 full page lengths after where pewdiepie was listed (as at 27th Dec)
Screenshots

Internal Use
Acceptance Criteria

Definition of Done

[ ]  Tested against acceptance criteria
[ ] Tested against the assumptions of the user story
[ ] The project builds without errors
[ ] Unit tests are written and passing
[ ] Tests on devices/browsers listed in the issue have passed
[ ] QA performed & issues resolved
[ ] Refactoring completed
[ ] Any configuration or build changes documented
[ ] Documentation updated
[ ] Peer Code Review performed

Thanks for opening the issue. This was recently fixed on the desktop app but hasn't made it into Android yet.
@akinwale , for the next update, please make sure you are up to date on redux (this fix is currently in a branch that should be merged soon).

Fixed in latest release.


Adding more supports on the adapter

[x] added IsEnableLoadMore indicator on ultimateRV
[x] added more supports on the biSwitcher
[x] added max pages on BiAdAdapterSwitcher

@cymcsg can u also issue new gradle pls. thanks!

I have merged it and released 0.3.14.


Fixing the vulnerabilities of amazon corretto pinot-base-runtime docker image.

As per https://github.com/apache/pinot/issues/13461, This PR tries to fix the vulnerabilities of the Amazon corretto jdk image. Here are some details

There are 0 vulnerabilities in the Amazon corretto 11-al2023-jdk image, however while installing git and several other packages we get around 50 high vulnerability. The base image is Amazon Linux-based, and all the packages are installed from the Amazon repo list. Using the fedora epel repo is cumbersome; hence, in this PR, we are switching to an Alpine Linux-based image.
Using a based image, reduced the size of the image; it is now close to 400MB only.
With this change, there is only now 1 Medium vulnerability because of curl package

docker scout quickview
Abhi-Sharma's-MacBook-Pro :: ~/work/pinot ‹13461-fix-amazoncorretto-vuln*› » docker scout quickview                                                                                (sbx/consumer)
    i New version 1.13.0 available (installed version is 1.11.0) at https://github.com/docker/scout-cli
    ✓ Image stored for indexing
    ✓ Indexed 70 packages

    i Base image was auto-detected. To get more accurate results, build images with max-mode provenance attestations.
      Review docs.docker.com ↗ for more information.

  Target             │  local://apachepinot/pinot-base-runtime:11-amazoncorretto  │    0C     0H     1M     0L
    digest           │  d52984cecc31                                              │
  Base image         │  amazoncorretto:11-alpine                                  │    0C     0H     0M     0L
  Updated base image │  amazoncorretto:22-alpine                                  │    0C     0H     0M     0L
                     │                                                            │

Also, I would like to highlight the PR comment by @gortiz, I think we should follow the route suggested in the comment and also recommend two sets of Pinot images, one with distroless image  and other set with various packages installed.

Personally, I don't like Alpine images given the docker image size is not that important* and the fact that Alpine uses musl instead of glib may have unexpected side effects. But I understand others may think differently.
What I don't get is why we need to remove the Amazon Corretto support. Right now we publish three different flavors (corretto, microsoft and openjdk). We can just create a new one. In case we need to remove one, probably openjdk should be the one to delete (given openjdk docker image is not recommended for production.
* In case we really want to improve that we should focus on other parts of the image including more efficient docker cache usage.

What I don't get is why we need to remove the Amazon Corretto support. Right now we publish three different flavors (corretto, microsoft and openjdk). We can just create a new one. In case we need to remove one, probably openjdk should be the one to delete (given openjdk docker image is not recommended for production.

Minor correction here: the image is still Amazon corretto, jdk. It's just that the OS is now Alpine instead of Amazon Linux. Yes, I agree we need to delete openjdk, given that it's not recommended for production.

Could you then modify this PR to substitute openjdk with your alpine solution?

I'm ok with stopping publishing openjdk. But as said above, I prefer to use images not based on Alpine, which doesn't use glibc. I strongly believe using alpine is a bad habit people has because they just focus on the reported docker image size instead of actually understanding how docker layers work.

I've just found this blogpost which is not super updated, but it is good to show the impact using Alpine has in the past. TBH I don't think it is worth to fall into something like that just to reduce some MBs. It should be other and better options to reduce the vulns in our images.

Amazon Corretto JDK docker images are only available on Amazon Linux and Alpine flavour. They do have the Debian based docker files in the repo, however they don't publish it. Here is the reference
If we want to use Amazon Corretto, we can build these Debian-based Corretto images and publish them on the Apache Pinot Docker hub for further build processes. WDYT @gortiz / @xiangfu0 ?

trivy image apachepinot/pinot:latest-21-amazoncorretto only shows 2 high/critical vulns and both of them are in Java. Therefore I think the current corretto image is good enough and I won't change it.

trivy image apachepinot/pinot:latest-21-amazoncorretto only shows 2 high/critical vulns and both of them are in Java. Therefore I think the current corretto image is good enough and I won't change it.

sure, closing the PR. thanks

Therefore I think the current corretto image is good enough and I won't change it.

I guess wording wasn't great. I wanted to say that I don't think we need to opt out from corretto. As said before, having an alpine image is fine as long as it is not the default image.