HADOOP-19027. S3A: S3AInputStream doesn't recover from HTTP/channel exceptions

Add a new Path IOE subclass, AwsHttpChannelException
Method ErrorTranslation.maybeExtractChannelException() to create this from shaded/unshaded NoHttpResponseException, using string match to avoid classpath problems.
And do this for SdkClientExceptions with OpenSSL error code WFOPENSSL0035. This isn't strictly the right place for this as its not an IOE we are remapping...
ErrorTranslation.maybeExtractIOException() to perform this translation as appropriate.

S3AInputStream.reopen() code now retries, except on EOFException, which is converted to a -1 response to the caller, as is done historically.
New tests for the translation, but not for the input stream logic. The place for that will be TestS3AInputStreamRetry
How was this patch tested?
New unit test cases; so far run the contract tests.
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
46m 39s

trunk passed

+1 :green_heart:
compile
0m 42s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 29s

trunk passed

+1 :green_heart:
mvnsite
0m 42s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 32s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
37m 46s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 36s

the patch passed

+1 :green_heart:
compile
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 2 new + 8 unchanged - 0 fixed = 10 total (was 8)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  7s

the patch passed

+1 :green_heart:
shadedclient
37m 33s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 46s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

138m 22s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 7b15f999737c 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 0776a2e797af70b9847117a05e37427b6323688c

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/1/testReport/

Max. process+thread count
589 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Re-ran the whole test suite as well. Don't see any new failures.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
46m 22s

trunk passed

+1 :green_heart:
compile
0m 42s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 29s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 32s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  7s

trunk passed

+1 :green_heart:
shadedclient
37m 32s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
37m 52s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 34s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 34s

the patch passed

+1 :green_heart:
compile
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 2 new + 8 unchanged - 0 fixed = 10 total (was 8)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

the patch passed

+1 :green_heart:
shadedclient
37m 42s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 50s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

138m  0s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 32e3fe1f71b0 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a22fd68ddc13c137fde88190d50a9b1e6205edbd

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/2/testReport/

Max. process+thread count
532 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
46m 18s

trunk passed

+1 :green_heart:
compile
0m 42s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 31s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
37m 54s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 34s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 34s

the patch passed

+1 :green_heart:
compile
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 27 new + 8 unchanged - 0 fixed = 35 total (was 8)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

the patch passed

+1 :green_heart:
shadedclient
37m 54s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 50s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

138m 35s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux ac9a414217a8 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4537fa23743e237902804d476d694f016d37ae85

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/3/testReport/

Max. process+thread count
529 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

note aws sdk has done something here with Netty
https://github.com/aws/aws-sdk-java-v2/pull/4655

Testing

new tests with mocking and of real tests where failure is shorter file than claimed in openFile(); no actual generation of failures within an ITest.
one presumably unrelated failure (https://issues.apache.org/jira/browse/HADOOP-19032)

[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 48.129 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI
[ERROR] testCreateDirectory(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI)  Time elapsed: 13.4 s  <<< ERROR!
org.apache.hadoop.fs.s3a.AWSS3IOException:
Remove S3 Dir Markers on s3a://stevel-london/Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/test-dir/7/testContextURI/createTest: org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException: [S3Error(Key=Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/test-dir/7/testContextURI/createTest/()&^%$#@!~_+}{><?/, Code=InternalError, Message=We encountered an internal error. Please try again.)] (Service: Amazon S3, Status Code: 200, Request ID: null):MultiObjectDeleteException: InternalError: Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/test-dir/7/testContextURI/createTest/()&^%$#@!~_+}{><?/: We encountered an internal error. Please try again.
: [S3Error(Key=Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/test-dir/7/testContextURI/createTest/()&^%$#@!~_+}{><?/, Code=InternalError, Message=We encountered an internal error. Please try again.)] (Service: Amazon S3, Status Code: 200, Request ID: null)
        at org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException.translateException(MultiObjectDeleteException.java:136)
        at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:347)
        at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
        at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:163)
        at org.apache.hadoop.fs.s3a.impl.DeleteOperation.asyncDeleteAction(DeleteOperation.java:445)
        at org.apache.hadoop.fs.s3a.impl.DeleteOperation.lambda$submitDelete$2(DeleteOperation.java:403)
        at org.apache.hadoop.fs.store.audit.AuditingFunctions.lambda$callableWithinAuditSpan$3(AuditingFunctions.java:119)
        at org.apache.hadoop.fs.s3a.impl.CallableSupplier.get(CallableSupplier.java:88)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
        at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:225)
        at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:225)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException: [S3Error(Key=Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/test-dir/7/testContextURI/createTest/()&^%$#@!~_+}{><?/, Code=InternalError, Message=We encountered an internal error. Please try again.)] (Service: Amazon S3, Status Code: 200, Request ID: null)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:3174)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeysS3(S3AFileSystem.java:3405)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:3475)
        at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.removeKeys(S3AFileSystem.java:2491)
        at org.apache.hadoop.fs.s3a.impl.DeleteOperation.lambda$asyncDeleteAction$8(DeleteOperation.java:447)
        at org.apache.hadoop.fs.s3a.Invoker.lambda$once$0(Invoker.java:165)
        at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
        ... 11 more

as it stands, this PR will

map http/ssl channel problems to EOFException subclass which is retried
map out of range 416 to a different EOF subclass -which is not retried
lazyseek will retry on channel failures
close s3 input stream when it returns -1

not done: any attempt for readFully() to recover from that -1 response.
That will take more work, possibly including clarification in spec, as the base implementation doesn't recover either and we should consider this + abfs semantics.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 22s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m  7s

trunk passed

+1 :green_heart:
compile
0m 27s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 22s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 21s

trunk passed

+1 :green_heart:
mvnsite
0m 27s

trunk passed

+1 :green_heart:
javadoc
0m 17s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 23s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 46s

trunk passed

+1 :green_heart:
shadedclient
20m  6s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 16s

the patch passed

+1 :green_heart:
compile
0m 20s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 20s

the patch passed

+1 :green_heart:
compile
0m 14s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 14s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 10s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 49 new + 8 unchanged - 0 fixed = 57 total (was 8)

+1 :green_heart:
mvnsite
0m 17s

the patch passed

+1 :green_heart:
javadoc
0m  9s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
0m 41s

the patch passed

+1 :green_heart:
shadedclient
20m 32s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 14s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 20s

The patch does not generate ASF License warnings.

83m 12s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux dc2e9893b02e 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2a9f54d68420d3528a7dfe812c8d40875f1dd95d

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/4/testReport/

Max. process+thread count
555 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 28s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

-1 :x:
mvninstall
14m 36s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
0m 29s
/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-aws in trunk failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

+1 :green_heart:
compile
1m 48s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 35s

trunk passed

+1 :green_heart:
mvnsite
0m 54s

trunk passed

+1 :green_heart:
javadoc
0m 28s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m 26s

trunk passed

+1 :green_heart:
shadedclient
41m 35s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 36s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 51 new + 8 unchanged - 0 fixed = 59 total (was 8)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 14s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  5s

the patch passed

+1 :green_heart:
shadedclient
37m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 53s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

114m 23s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 2df7e4706748 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2a517eb5a18ae9fcd0fffa22ed433b523ca6f8bf

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/5/testReport/

Max. process+thread count
564 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
49m 11s

trunk passed

+1 :green_heart:
compile
0m 43s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 30s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 32s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  8s

trunk passed

+1 :green_heart:
shadedclient
38m 26s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 36s

the patch passed

+1 :green_heart:
compile
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 50 new + 8 unchanged - 0 fixed = 58 total (was 8)

+1 :green_heart:
mvnsite
0m 32s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

the patch passed

+1 :green_heart:
shadedclient
38m 21s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 55s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

142m 47s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux e5aeef35f82e 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 92c9005fcc7f69dea8c702dbcb8542b580b2fd1a

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/6/testReport/

Max. process+thread count
533 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  2s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

-1 :x:
mvninstall
11m 17s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
0m 26s
/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-aws in trunk failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.

+1 :green_heart:
compile
2m 13s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 36s

trunk passed

+1 :green_heart:
mvnsite
0m 51s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m 35s

trunk passed

+1 :green_heart:
shadedclient
41m  7s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 37s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 37s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 50 new + 8 unchanged - 0 fixed = 58 total (was 8)

+1 :green_heart:
mvnsite
0m 32s

the patch passed

+1 :green_heart:
javadoc
0m 15s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

the patch passed

+1 :green_heart:
shadedclient
37m 51s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 58s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

110m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 44e837d7add7 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a46b4ee36e1ff0763035499c3d501faadc995b2d

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/7/testReport/

Max. process+thread count
534 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

rebased to force rebuild

@mukund-thakur @ahmarsuhail can i get a review of this...want to merge and ideally get onto 3.4 branch in case there's another RC of that

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
17m 21s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
46m 57s

trunk passed

+1 :green_heart:
compile
0m 43s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 30s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 26s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 32s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
37m 49s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 34s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 34s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 18s

the patch passed

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 14s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 23s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m 11s

the patch passed

+1 :green_heart:
shadedclient
37m 53s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 56s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 33s

The patch does not generate ASF License warnings.

155m 57s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6425

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 92a7115f8c41 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 776ac8b7e9aec690427b4457f28f57da1b6d8f54

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/9/testReport/

Max. process+thread count
583 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6425/9/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

update, one new failure, probably from having 12 threads as it was a timeout in one of the tests
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 609.807 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction
[ERROR] testSeeksWithLruEviction[max-blocks-2](org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction)  Time elapsed: 600.009 s  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 600000 milliseconds
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:837)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:999)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1308)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
        at org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction.testSeeksWithLruEviction(ITestS3APrefetchingLruEviction.java:162)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.lang.Thread.run(Thread.java:750)

Does not happen on a standalone rerun.
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.108 s - in org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0


Warning with Android Studio 3.1

I got a red warning after upgrading.

Configuration 'compile' is obsolete and has been replaced with 'implementation'.
It will be removed at the end of 2018

The specified Android SDK Build Tools version (27.0.0) is ignored, as it is below the minimum supported version (27.0.3) for Android Gradle Plugin 3.1.0.
Android SDK Build Tools 27.0.3 will be used.
To suppress this warning, remove "buildToolsVersion '27.0.0'" from your build.gradle file, as each version of the Android Gradle Plugin now has a default version of the build tools.

Android Studio 3.1
Build #AI-173.4670197, built on March 22, 2018
JRE: 1.8.0_152-release-1024-b01 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Linux 4.14.29-1-MANJARO
Do you plan to publish an update soon?
Thank you.

You have changed the gradle plugin version on build.gradle at this line:
classpath 'com.android.tools.build:gradle:3.0.1'
with
classpath 'com.android.tools.build:gradle:3.1.0'
but for Android Gradle plugin the minimum supported version of Android SDK build tools version is 27.0.3...
As described on setup wiki, you have changed the default requirements, so it isn't a plugin error, but it's yours Gradle misconfiguration...
However, as described in warning message, if you would like to use Android Gradle plugin version 3.1.0, you have to change on root build.gradle the Android SDK build tools version to 27.0.3 and on app/library subdir's build.gradle you have to change all dependencies compile with implementation, as it's deprecated on Android Gradle plugin version 3.0.0.

@BoGnY I didn't touch (manually) build.gradle since I imported the project into Android Studio.
So no, I didn't change anything.
It's either a problem with Android Sutdio or a missing update of Candybar.
I'm pretty sure many users of Candybar have/will have the same message.

Same problem after configuring the android studio in new pc.
got rid of build-tools warning but this "compile" and "implementation" is still there.


feat(STONEINTG-362): adding PAC to hacbs-test repo

Adding hacbs-test repo because of regular building of an image.

/approve
/lgtm

/lgtm


Native comp has been merged to master - switch branches?

Hi again,
with native comp merged to master, I think the COPR could use the master branch + pgtk branch instead.
From what I read, the native-comp branch will be stale going forward. And so far native-comp is not enable by default, so there's still a need to pass that flag explicitly.
Hope that made sense. I am not super clear on how this recipe gets its code together.

We are using a fork that does precisely this.


GeoAPI configuration not written after deploy

2022-12-15T07:59:21.866502390Z Traceback (most recent call last):
2022-12-15T07:59:21.866996392Z   File "/tmp/8dad94a2b95750e/run.py", line 16, in <module>
2022-12-15T07:59:21.867744897Z     with open(geoapi_config_name, "r") as f_geoapi_config:
2022-12-15T07:59:21.867764097Z FileNotFoundError: [Errno 2] No such file or directory: 'pygeoapi.yml'

Reattempting deployment

chevron isn't in the existing environment, must've not installed dependencies properly.


feat(relation-widget): Allow the relation widget to target file collections

Summary
Hello everyone! This PR is a WIP and is meant to facilitate discussion, no hurt feeling if it's rejected.
A few challenges when targeting a file collection:

Beside the collection name, the widget will also need the file name, and the name of a field in that file (which is very likely a list widget.)
The widget needs to know if it's dealing with a file collection, a folder collection, or its own entry.
The widget have to be able to handle a simple array of string and potentially an array of object with different shape (in case of an array produced by a typed list widget).

For the first two, I think we could use dot notation in collection name:
# folder collection
- label: Featured Posts
  name: featuredPosts
  widget: relation
  collection: posts

# file collection
- label: Categories
  name: categories
  widget: relation
  # [collection].[fileName].[fieldName]
  collection: "settings.post.categories"

If collection contains dots, use query as usual; otherwise use loadEntry. Later on, when a user want to target a list widget in the same entry as the relation widget, I think it should be smart and use the new entry props.
Also, I found that query's results are stored in queryHits. I wonder if loadEntry results should be stored in a similar manner?
Please let me know if this is a valid approach!
Test plan
I've written 2 basic tests that covers a simple list &  a list of items with the same object shape. Will need a few more.

closes #800

I fleshed this out a bit more today, adding support for targeting list being nested inside an object! I'm rather fond of overloading collection field, since it allows us to easily differ between file & folder collections. Later on if we decide to allow user to target fields within the same entry, perhaps user can specify some thing like this: collection: "self.fieldName".
I also added a category field to the demo site to test this out visually, and found an issue where loaded options are not being persisted. I think it may have to do with the fact that in a folder-targeting relation widget, the options loaded via query are persisted in the redux store as queryHits[forID], but there's no such thing for loadEntry.
In my custom widget, I persist loaded options in the widget's local state, but it doesn't seem to work here. It's also likely something to do with my code, will look more into it.

Hhhm I'm a bit stuck here. Intuitively I think it's best to replicate the same structure from folder relation, but even if I want to do so, the function loadEntry passed to widget is no longer a redux action and thus can't send the payload to the global store the same way query can.
Another approach I can think of, is to do loadEntry/query upfront in componentDidMount and doesn't rely on value stored in queryHits for the first render.

Hi @d4rekanguok, was meaning to comment on this soon (haven't had the time to test my thoughts yet), but I think you can dump loadEntry and use
if (field.has('file')) {
    hits = hits.filter(hit => hit.slug === field.get('file'));
}

Disadvantage it that the CMS will still retrieve all files in the referenced collection, but I think it is a minor issue considering the alternative.
I would actually not overload the collection field since it introduces a new concept. Having an additional file field might easier to grasp, and using the current valueField to reference the file field might work. Haven't thought of a good way to reference lists yet (maybe categories.*.name).

Thanks @erezrokah, that makes sense! I'll explore the approach you mentioned.

Replaced by https://github.com/netlify/netlify-cms/pull/3754


Updated everything from .Net 6 to .Net 8.

Updated NuGet packages (28) including one with a vunerability. Two NuGet packages remain deprecated:
Microsoft.AspNetCore.Mvc
Microsoft.AspNetCore.Mvc.Core

Missing Docker Containers conversion.


Separate Log streaming for API Analytics and Sync server

Description:
We currently have a single log stream for server and analytic server.

Steps to reproduce:

./dev.sh
Wait for container to start
Switch to api logs

Expected behavior:

Separate log streams for analytics sever

Actual behavior:

Single screen for sync and analytics

Server Setup Information:

Operating System:
Deployment Method:

Additional context

Closing this, as this was fixed in #368


Implement serde feature without interpolate_idents

This removes the user having to write
#![feature(plugin)]
#![plugin(interpolate_idents)]

#[macro_use]
extern crate serde_derive;

when using the "serde" feature.

Thank you very much!


Don't merge! Regression PR perf testing without metadata validation

Docs Build status updates of commit faef0bf:
:clock10: Pending: waiting for processors (10 builds ahead of you)
:warning: Docs Build is busy, currently there are 10 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit faef0bf:
:warning: Validation status: warnings

File
Status
Preview URL
Details

developer/TestingPullRequest.md
:warning:Warning
View
Details

developer/TestingPullRequest.md

[Warning] Metadata with following name(s) are not allowed to be set in Yaml header, or as file level metadata in docfx.json, or as global metadata in docfx.json: content_type. They are generated by Docs platform, so the values set in these 3 places will be ignored. Please remove them from all 3 places to resolve the warning.
[Warning] the value invalid_type is undefined for metadata content_type
Line 7: [Warning] Invalid file link:(~/../nonexisted1.md).
Line 8: [Warning] Invalid file link:(~/../nonexisted2.md).
Line 9: [Warning] Invalid file link:(~/../nonexisted3.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit faef0bf:
:clock10: Pending: waiting for processors (27 builds ahead of you)
:warning: Docs Build is busy, currently there are 27 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit faef0bf:
:warning: Validation status: warnings

File
Status
Preview URL
Details

developer/TestingPullRequest.md
:warning:Warning
View
Details

developer/TestingPullRequest.md

[Warning] Metadata with following name(s) are not allowed to be set in Yaml header, or as file level metadata in docfx.json, or as global metadata in docfx.json: content_type. They are generated by Docs platform, so the values set in these 3 places will be ignored. Please remove them from all 3 places to resolve the warning.
[Warning] the value invalid_type is undefined for metadata content_type
Line 7: [Warning] Invalid file link:(~/../nonexisted1.md).
Line 8: [Warning] Invalid file link:(~/../nonexisted2.md).
Line 9: [Warning] Invalid file link:(~/../nonexisted3.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Formatter removes escape slash in HEREDOC

Code before format
<<-REGEX
/\d/
REGEX

Code after crystal tool format that_file.cr
<<-REGEX
/d/
REGEX

I noticed this when running the formatter on the ICR shard. The specs changed on these lines
https://github.com/crystal-community/icr/blob/0537fc21636fc1de467fa7108d80387eeba06893/spec/integration/icr_spec.cr#L300-L317

This is expected behaviour, see #8619. The formatter transforms any string literal like that.
The reason is that \d is an invalid escape sequence. The parser accepts that and treats it as equivalent to d.
If you want an actual backslash, you need to escape it (\\).
Eventually, invalid escape sequences should probably become an error (even if just to avoid such surprises).

Why p! prints 2 \?
Does p! print internal string representation or exact source code?
% crystal eval 'p! "/\\d/"'
"/\\d/" # => "/\\d/"

% crystal eval 'puts "/\\d/"'
/\d/

p! calls String#inspect which escapes special characters (thus the result can be used as a string literal).
puts just prints the string without any escapes.


Version Packages

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.
Releases
<EMAIL_ADDRESS>Patch Changes

e4b6a97: update in css

size-limit report 📦

Path
Size

dist/super-duper-button.cjs.production.min.js
1.29 KB (0%)

dist/super-duper-button.esm.js
1.31 KB (0%)


Activity triggered User update on user logout

I noticed that after a user logs out, the LogsActivity Traits is triggered the updated event. After some searching I discovered that is caused by the remember_token in the User model, because this will be updated after loggin out.
Now I set the protected static $logAttributesToIgnore = [ 'remember_token']; in the User model to ignore the activity, because the logger will attach a new record with {"attributes":[],"old":[]} properties in the database. But the logAttributesToIgnore is not working, because the record will still be added, but now without properties []
Is there a setting I missed or is it a bug?
For you info, this is the record which is added:
Before ignoring the remember_token

id
log_name
description
subject_id
subject_type
causer_id
causer_type
properties

42
user
updated
1
Modules\Users\Entities\User
1
Modules\Users\Entities\User
{"attributes":[],"old":[]}

After ignoring the remember_token

id
log_name
description
subject_id
subject_type
causer_id
causer_type
properties

68
user
updated
1
Modules\Users\Entities\User
1
Modules\Users\Entities\User
[]

Thanks!

I think that it's related to #448 - like said there I can't reproduce it on my local machine.
I will try to reproduce your given example to solve it now and forever.
The method in question should be shouldLogEvent().
https://github.com/spatie/laravel-activitylog/blob/68eb6e65382f94fe86ce5ff6572159f80d024ba9/src/Traits/LogsActivity.php#L107-L125
If you could provide some debug information out of this method (what everything has as value, what it returns and do on) would be great.
Independent I will try to get some details on my own - could be that I need until tomorrow to debug it.

@dennis1502 I got the solution for you - add this (additional to $logAttributesToIgnore) to your Model.
protected static $ignoreChangedAttributes = ['remember_token'];

I have to dive deeper into this and why there are two properties for this. One is in the LogsActivity and the other in DetectsChanges trait. One controls the attributes listed in the attributes and old arrays and the other if it should create the log or not.
At first this sounds like the same but it doesn't have to be.
Reason:
If you have an attribute that shouldn't create a log, if it's the only changed one, but should be listed if another attribute creates a log.
@freekmurze is this the reason? Or is it a mistake and should be fixed to one property?
I will also take a look in the documentation and unittests if there is something missing - tomorrow.

@Gummibeer Thank you so much, that actually worked for me. I checked that file earlier yesterday, but didn't noticed the ignoreChangedAttributes I could use. Now it works lik a charm.
I really don't know the reason yet, maybe @freekmurze can confirm your idea about it. But for now it works! :D
Many thanks!

I'm facing the same problem and the solution provided here didn't help me solve it.

@Zolak94 I thought your problem is that the causer is empty? Like described in #588

@Gummibeer Yes I do, I also have this problem, on my deploy (live) server whenever a user logs out it's also updated I figured out that remember token is the thing that's changing and triggering that. But your solution with putting protected static $ignoreChangedAttributes = ['remember_token']; in my User.php model doesn't work. On the other hand, everything works normally on my local machine.

@Zolak94 do you add it in addition to protected static $logAttributesToIgnore = [ 'remember_token'];?

@Gummibeer Yes, I did.

And you still get an update record for the user? Is there any other changed attribute in the properties listed?

Yeah.

So where's the bug/problem? If there are other changed properties than the ignored ones - it will create an activity entry.

I'm sorry that I didn't answer fully. Nothing else except remember_token changes when a user logs out.

And this is only happening on a live website, not on my local machine.

It doen't work for me. It still updated at the logout time. It look strange.


Create wiki and faq for charts using the common library

Details
Describe the solution you'd like:

The charts using the common library are pretty much the same, we need to come up with a place to document these things.
Related PRs / Issues
https://github.com/k8s-at-home/charts/pull/456
https://github.com/k8s-at-home/charts/pull/456
https://github.com/k8s-at-home/charts/issues/451
https://github.com/k8s-at-home/charts/issues/38

Now closing this and moving to the discussion board
https://github.com/k8s-at-home/charts/discussions/462

Now closing this and moving to the discussion board
https://github.com/k8s-at-home/charts/discussions/462


Syncing or integrating with existing PostgreSQL database

Are there any recommended approach to sync with PostgreSQL? IOW, how do people using Blast integrate it with their existing PostgreSQL systems?

Basically, send PostgreSQL data to Blast and index it. I haven't heard of Blast use cases yet.

@mosuka Thank you so much for your reply. Hopefully debezium along with Kafka will help for real time sync.

The other more scalable way is to use CDC on the SQL database
The change feed can then be used to update blast.
This is the pattern typically used because you can then update a myriad of systems when the data in the dB mutates
It also means that you respect the transactions in the DB being committed or rollbacked

@joeblew99 Thanks for sharing your ideas. For Postgres, do you know any solution other than debezium ?


feat: add InjectMikroORMs inject decorator

Create an inject decorator for multiple mikro-orms

What is this good for?

What is this good for?

Kind of a follow on from https://github.com/mikro-orm/nestjs/pull/88 where instead of doing:
@Injectable()
export class MyService {

  constructor(@Inject('MikroORMs') private readonly orms: MikroORM[]) { }

}

we can do:
@Injectable()
export class MyService {

  constructor(@InjectMikroORMs() private readonly orms: MikroORM[]) { }

}

which makes it look nicer and less strings.
I can also add it within the README.md too if that help?

Yes, but what is that good for?

Yes, but what is that good for?

Its just a helper and just gives us these benefits:

Consistency with the other helper (even those they are used for context names)
Less risk of typos for the string name

Once again, what is this good for? Why would you as a user want this? I understand what you did there, I am asking for what use case? Its not documented anywhere, neither the explicit @Inject('MikroORMs') nor this new version.

Once again, what is this good for? Why would you as a user want this? I understand what you did there, I am asking for what use case? Its not documented anywhere, neither the explicit @Inject('MikroORMs') nor this new version.

Sorry I understand now, we basically want an easy way to get all registered database so we can do the following:

Get all databases for health checks
Multi-Tenancy database to apply setFilterParams

This would make it easier than having to manually add a new InjectMikroORM for every new database.

@B4nan updated based off your review


thoughts on better neural net

The current neural net uses something really made for handwritten digit classification (MNIST).
We could do stuff like use Alexnet instead, or implement the neural nets described in Mnih/Hinton too. Literature also describes using a sequence of pre and post processing neural nets, where you can fill in gaps in road networks.
Expanding on these vague comments, there is a whole body of literature about how to use CNNs, RNNs, global topology, lidar elevation data, and much more to improve the accuracy of satellite imagery label. We should be able to get above 90% on the pixel level, just using semi-local RGB data, and push past that with multiple neural nets, more data, and other documented improvements in the last 2-3 years since Mnih's thesis.
See the README for a list of readings.

@lacker says to make a simpler neural net that works better with the given amount of IR training data, so the analysis overfits less.
Then expand the layers/data.
Don't just use some complicated neural net unsuited to the task.

Yeah, I think you want to get to a state where "it works, just could be better" before scaling up. That way you can figure out how long you need to train, and keep params in a state where you don't overfit. I would try something real simple, like one convolution layer plus one softmax layer, and see if you can make it "work". By "work" I mean significantly better than random, like 80% accuracy, and not much of an overfitting problem.

I think we're now in a state of "works with the simplest network." A single relu or single convolutional relu with a maxpool.


'grpc/support/port_platform.h' file not found

As mentioned here https://github.com/LiquidPlayer/LiquidCore/issues/102
FirebaseFirestore is not compatible with LiquidCore.

I've build a minimal, complete, and verifiable example that demonstrates it. The project has been created with flutter create and I didn't touched a thing. I just added the plugins in pubspec, added a signing team and set the min platform to 10.0.
https://github.com/long1eu/liquid_core_and_firestore_test
Any help is highly appreciated.

As I mentioned in the other issue:
Please share your project file after pod install. (Alternatively just your HEADER_SEARCH_PATHS).
Inevitably these problems happen because you have an entry in there that somehow includes your Pods folder recursively (either directly or indirectly) which allows these conflicts to occur.
Also, the full error message text from the build log rather than a screenshot would be useful.
I don't really know anything about flutter, so it's possible that your liquid_core_and_firestore_test could serve this purpose, but I have no idea how to use it to show the problem. Could you either give instructions for how to use it to show the problem or commit everything that results from pod install?

Closing as it's not relevant to the flutter plugin.


Clustering example not working

I am seeing this error when running the clustering example.  First I had to alter the dns address as my docker nat was on a different address than 172...
/scripts/lib/functions.sh: line 2: syntax error near unexpected token `$'{\r''   I am worried that the HOSTNAME variable might not be resolved correctly but haven't been able to verify.
I tried updating the domain to localhost as one of the other issues talked about...
If i remove the functions.sh then i get a little further but i am seeing issues with the node names when ej starts..
Application asn1 started on node ejabberd@d2276e573bf4
Any ideas what i might be doing wrong here?

docker: Error response from daemon: driver failed programming external connectivity on endpoint dnsdock (xxxxxxxxxxx): Error starting userland proxy: Bind for 172.17.x.1:53: unexpected error (Failure EADDRNOTAVAIL)
I got above error when I try to run cluster example. It could not bind the DNS IP

Is this related to #100?

Hi, the cluster example is a very old example I've setup with an old docker version. I think now there are better ways to setup a good cluster and my solution should be reconsidered. But currently I have no time to work on this.

Hi @rroemhild I had the same issue running the clustering example relating to EADDRNOTAVAIL.
As you mentioned its an old example of clustering the image. Do you know of any better containerized approaches to clustering ejabberd?
I tested bundling your image and nginx-proxy into a stack docker-compose file and that scales without port conflicts.
But it doesn't seem to handle TCP connection over port 5222 via an XMPP client.
Do you have any feedback if this is the right approach to clustering or configuring nginx?
This is the issue I opened on nginx-proxy relating to that setup:
https://github.com/jwilder/nginx-proxy/issues/979


[ASCII-2075] Wait for providers to stop in metadata runner stop

What does this PR do?

Improve stopping mechanism of the metadata runner component.
The metadata runner currently stops running providers asynchronously to avoid blocking the agent stop.
This PR changes that to wait at most 30 seconds for providers to stop.
That way we can both limit risks of interrupting a provider, and avoid blocking the agent stop indefinitely.
Motivation

Trying to fix https://github.com/DataDog/datadog-agent/issues/24171.
We suspect some metadata providers could be interrupted when the agent stops, while they are still running, causing a corruption.
Additional Notes

Providers should check the context so that they stop as early as possible when it's canceled, but most don't.
Follow-up work will improve that.
This PR also fixes an issue with the use of the WaitGroup where it was incremented in the routine instead of before starting it, so when starting and stopping quickly in tests it could not wait for routines.
Possible Drawbacks / Trade-offs

Could make the agent longer to stop, but stop is cleaner.
Describe how to test/QA your changes

Added unit tests to ensure each case works as expected when we try to stop the runner while providers are running:

runner stop timeout is shorter than provider duration: stop returns after timeout
runner stop timeout is longer than provider duration: stop returns when provider ends

/merge

/merge -c

/trigger-ci --variable RUN_ALL_BUILDS=true --variable RUN_KITCHEN_TESTS=true --variable RUN_E2E_TESTS=on --variable RUN_UNIT_TESTS=on --variable RUN_KMT_TESTS=on

/merge


docs: updated systems requirement for OpenShift 4.15

document updated systems requirement for OpenShift 4.15
fixes #4031

/hold
Till we switch to 4.15 bundles.

/unhold


nixos/activation: add types to options

Motivation for this change
In response to #76184.
Things done
Create a type for the elements of nesting.clone and nesting.children and slap it on.
The new type doesn't do any validation. However, a type error will now be raised if the user forgets that both options expect lists.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS inapplicable
[ ] other Linux distributions inapplicable

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests) inapplicable
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip" inapplicable
[ ] Tested execution of all binary files (usually in ./result/bin/) inapplicable
[ ] Determined the impact on package closure size (by running nix path-info -S before and after) inapplicable
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Killed by the new specialisation mechanism
ec2d28e3233c4c04fe6afe13e828691d099167ae


Cross-Browser testing

For now, major browsers should be supported:

Chrome
Firefox
Edge

/remove-lifecycle stale
/lifecycle frozen


Ajax Form Validation Breaks Oddly With Form in Bootstrap Modal

Hello, I'm stumped on how I can be experiencing such odd behavior, so I'm coming to the masters for guidance.
Here are the specifics on my development environment:

PHP Version 5.5.14
Yii Version 2.0.4-dev
I'm developing locally on OS X, using Apache and MySQL
My issue occurs in FireFox, Chrome and Safari, probably other browsers too

I'm receiving no JS errors in my console or errors in the Yii debugger.
I've been working on having forms in modal windows, and I've been having an issue where the ajax form validation is not working when the form is inside a bootstrap modal window.
To be clear, I'm referring to the ajax form validation that is triggered for each field when the filed looses focus. This is the standard Yii form validation, so I'm testing the form in the modal by clicking into a required field, and then clicking out of the field. This triggers the Yii ajax validation and should display a red error message under the field that states the "'field' cannot be blank."
Again, my issue is that sometimes the ajax validation works and sometimes it does not.
I managed to trap the issue to being related to where the modal code is placed on the page, but it makes no sense to me how the following would affect the ajax validation behavior of the form in the modal.
First I'll show you the code that works properly. The ajax form validation works as expected on the form in the modal window.
<?php
use yii\helpers\Html;
use yii\helpers\Url;
use yii\grid\GridView;
use yii\bootstrap\Modal;
?>

<?= Html::encode($this->title) ?>

    <?= Html::button('Create Category', [
        'class' => 'btn btn-success btn-ajax-modal',
        'value' => Url::to('/category/create'),
		'data-target' => '#modal_category',
	]); ?>

	<?php
	Modal::begin([
		'id' => 'modal_category',
	    'header' => '
Category
',
	]);
	echo '

';
	Modal::end();
	?>

    <?= GridView::widget([
        'dataProvider' => $dataProvider,
        'filterModel' => $searchModel,
        'columns' => [
            ['class' => 'yii\grid\SerialColumn'],
            'id',
            'title',
            ['class' => 'yii\grid\ActionColumn'],
        ],
    ]); ?>

Things to note in the code above:

This is my category/index view, so it has a create button, the bootstrap modal code, and then the Yii gridview of categories.

The create button has the btn-ajax-modal class.  This class is identified by a JS function that uses the other button attributes to display the modal widow and load the form content in the modal.  The button value attribute identifies the model and form that should be loaded.

Here is the JS function that handles showing the modal and loading the form:
$('.btn-ajax-modal').click(function (){
	var elm = $(this),
		target = elm.attr('data-target'),
		ajax_body = elm.attr('value');

    $(target).modal('show')
        .find('.modal-content')
        .load(ajax_body);
});

So, all of the above code works perfectly!
BUT... if I make some very simple changes to the above code, the ajax form validation no longer works, and I have no idea why.
First change:

Move the bootstrap modal code below the gridview code, and the ajax validation no longer works.

<?= Html::encode($this->title) ?>

 <?= Html::button('Create Category', [
     'class' => 'btn btn-success btn-ajax-modal',
     'value' => Url::to('/category/create'),
 	'data-target' => '#modal_category',
 ]); ?>

 <?= GridView::widget([
     'dataProvider' => $dataProvider,
     'filterModel' => $searchModel,
     'columns' => [
         ['class' => 'yii\grid\SerialColumn'],
         'id',
         'title',
         ['class' => 'yii\grid\ActionColumn'],
     ],
 ]); ?>

 <?php //*** SEE MODAL CODE IS NOW BELOW GRIDVIEW CODE
 Modal::begin([
 	'id' => 'modal_category',
     'header' => '
Category
',
 ]);
 echo '

';
 Modal::end();
 ?>

It makes no sense to me why moving the modal code below the gridview code would break the ajax form validation.  The modal still displays and loads the proper form, but the ajax form validation does not trigger when I give and remove focus to a required field.
And it gets even more bizarre ...
Second change:

Instead of moving the modal code below the gridview code, just delete the gridview entirely, including use yii\grid\GridView;.

<?= Html::encode($this->title) ?>

 <?= Html::button('Create Category', [
     'class' => 'btn btn-success btn-ajax-modal',
     'value' => Url::to('/category/create'),
 	'data-target' => '#modal_category',
 ]); ?>

 <?php
 Modal::begin([
 	'id' => 'modal_category',
     'header' => '
Category
',
 ]);
 echo '

';
 Modal::end();
 ?>

This change also causes the ajax form validation for the form in the modal to stop working.
This is extremely odd behavior, and I'm stumped as to what may be going on.
I welcome any suggestions anyone may have.  Please let me know if there are any questions I can answer that may help.

I figured it out.  The form in the modal must have an id attribute for the ajax form validation to work properly.


Feature Request: Address normalization

Any plans in the near future to add address normalization function(s) as well?
example from a dataset I'm working with:
1909 North Hwy 81
1909 N Highway 81
These are identical addresses, and the idea would be to have a new function that would take the output of the parse function and normalize each address component value to canonical values, so that we can apply an equality function that would return True if the above example strings were parsed and normalized.

DataMade has no current plans on implementing this, but we would welcome a PR.

I do like the focus that usaddress has on parsing; normalization, disambiguation and standardization are often handled as different steps.
Normalization is pretty simple, using a dictionary for most cases:
https://github.com/derrek/street-address/blob/master/lib/street_address.rb
It could be a little more complex, if you want to handle edit distance.. (e.g. "Nrth").
Disambiguation and standardization often take actual data from the USPS or from the Tiger census (for US addresses), to cover the corner cases.
Now that I'm done with my lecture... ;-)   @fgregg would a patch be part of the main project or better received as a separate module? Would normalization include fancy learning training sets or a simple dictionary?

The basic dictionary works pretty well for street part, e.g. RD, Drive, etc.
Things get quite a bit more complex (and slower) over in PostGIS-land.
https://github.com/postgis/postgis/tree/svn-trunk/extras/tiger_geocoder/tiger_2011/pagc_normalize
Adding in that many comparisons slows down batching by a bit. There is some rule information which would likely help, but at present I'm just naively doing string replaces.

Looking further into this, and having implemented, I think that "normalization" is really a "data append" process. Anything beyond basic translations gets into some complex rule sets.
There are ambiguities that can be cased, such as:
"FL" -> "Florida" or "FL"->"Floor"

But when it comes to it, you're trying to land on something standard. Then you need a large data set.
You end up tapping into these two:
http://federalgovernmentzipcodes.us/
openaddresses.io

And now you're in data-append land.

I'm not sure I know what you mean by "append?
On Fri, Jun 5, 2015 at 5:19 PM Charles Pritchard<EMAIL_ADDRESS>wrote:

Looking further into this, and having implemented, I think that
"normalization" is really a "data append" process. Anything beyond basic
translations gets into some complex rule sets.
There are ambiguities that can be cased, such as:
"FL" -> "Florida" or "FL"->"Floor"
But when it comes to it, you're trying to land on something standard. Then
you need a large data set.
You end up tapping into these two:
http://federalgovernmentzipcodes.us/openaddresses.io
And now you're in data-append land.
—
Reply to this email directly or view it on GitHub
https://github.com/datamade/usaddress/issues/83#issuecomment-109459646.

You're adding data onto the initial record. "Drive" is a pretty simple one for normalization, but when I get to corner cases, I start to see things like checking if the address is in Hawaii, which may mean a zipcode to HI lookup, if only the zipcode and address are shared. At that point, you are appending data, such as saying that this is the state of "HI".

Ah. Makes sense.
On Fri, Jun 5, 2015 at 5:24 PM Charles Pritchard<EMAIL_ADDRESS>wrote:

You're adding data onto the initial record. "Drive" is a pretty simple one
for normalization, but when I get to corner cases, I start to see things
like checking if the address is in Hawaii, which may mean a zipcode to HI
lookup, if only the zipcode and address are shared. At that point, you are
appending data, such as saying that this is the state of "HI".
—
Reply to this email directly or view it on GitHub
https://github.com/datamade/usaddress/issues/83#issuecomment-109461733.

Unless a PR comes down the pipe, I'm considering this closed.

Is there a connecting project or other related effort?

Not that we know of.
On Tue, Dec 13, 2016 at 4:11 PM, Charles Pritchard<EMAIL_ADDRESS>
wrote:

Is there a connecting project or other related effort?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/datamade/usaddress/issues/83#issuecomment-266878310,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgxbUEtqup-qPgo6szL1uuBu-au-awuks5rHxgWgaJpZM4DxDLm
.

--
773.888.2718

I've got a simple address normalizer I've been working on: https://github.com/crccheck/geodude/blob/master/utils/address.py#L24 It's.... really hacky right now and built with trial and error.

I think that it would be really awesome if there was a downstream address
normalizer that use usaddress. We are very unlikely to do it ourselves.
On Tue, Dec 13, 2016 at 4:29 PM, Chris Chang<EMAIL_ADDRESS>wrote:

I've got a simple address normalizer I've been working on:
https://github.com/crccheck/geodude/blob/master/utils/address.py#L24
It's.... really hacky right now and built with trial and error.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/datamade/usaddress/issues/83#issuecomment-266882667,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgxbbCpy8qOPNyx_tttV9cmypuE8qDwks5rHxw7gaJpZM4DxDLm
.

--
773.888.2718

alternative for address normalization: https://github.com/openvenues/libpostal


Update dockerfile

Added apt-get update and then install locales, otherwise, the docker image would fail at this step.

Thanks!


Move TheDocumentFoundation.LibreOffice <IP_ADDRESS> to TheDocumentFoundati…

…on.LibreOffice.LTS <IP_ADDRESS>
Checklist for Pull Requests

[ ] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.6 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

Error information:
Http Response: OK
Http Reason Phrase: OK
Error occurred while downloading installer from https://downloadarchive.documentfoundation.org/libreoffice/old/<IP_ADDRESS>/win/x86/LibreOffice_<IP_ADDRESS>_Win_x86.msi. 0x80072EE2.
Error information:
Http Response: OK
Error information:
Http Response: OK
Http Reason Phrase: OK

4/1/2024 8:43:11 PM
(Automated response - build 876.)

URL: https://downloadarchive.documentfoundation.org/libreoffice/old/<IP_ADDRESS>/win/x86_64/LibreOffice_<IP_ADDRESS>_Win_x64.msi
Status Code: 200
(Automated message - build 876)

@wingetbot run

This is just moving the existing manifest
[Policy] Reset Labels


Build v1beta1 compatible versions of dependencies

Build compatible versions of dependencies like clusterctl and cert-manager

closing in favor of https://github.com/aws/eks-anywhere-build-tooling/issues/172


Inconsistent Errors and Propagation Across Deployments

Hello,
I've been working with MS support engineers for awhile now trying to get FlexVolumes mounted to several pods to integrate KeyVault and AKS (my K8s cluster).  I've run into several problems that have caused confusion in understanding the underlining problems and their solutions.  One of the biggest issues is how after several hours of waiting for FlexVols to mount it seems to spontaneously work.  However, it'll only work for some pods and not others even when the configuration is basically the same for all of them.  Below is a detailed description of the errors I've received and tried to solve and how I'm currently trying to use this setup:
Steps To Reproduce
Flexvolume Setup:
kind: DaemonSet
metadata:
  labels:
    app: keyvault-flexvolume
  name: keyvault-flexvolume
  namespace: *********
spec:
  template:
    metadata:
      labels:
        app: keyvault-flexvolume
    spec:
      tolerations:
      containers:
      - name: flexvol-driver-installer
        image: "mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.10"
        imagePullPolicy: Always
        resources:
          requests:
            cpu: 50m
            memory: 100Mi
          limits:
            cpu: 50m
            memory: 100Mi
        env:
        - name: TARGET_DIR
          value: "/etc/kubernetes/volumeplugins"
        volumeMounts:
        - mountPath: "/etc/kubernetes/volumeplugins"
          name: volplugins
      volumes:
      - hostPath:
          path: "/etc/kubernetes/volumeplugins"
        name: volplugins
      nodeSelector:
        beta.kubernetes.io/os: linux

Pod Identity Setup:
apiVersion: v1
kind: ServiceAccount
metadata:
  name: aad-pod-id-nmi-service-account
  namespace: ********
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: azureassignedidentities.aadpodidentity.k8s.io
spec:
  group: aadpodidentity.k8s.io
  version: v1
  names:
    kind: AzureAssignedIdentity
    plural: azureassignedidentities
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: azureidentitybindings.aadpodidentity.k8s.io
spec:
  group: aadpodidentity.k8s.io
  version: v1
  names:
    kind: AzureIdentityBinding
    plural: azureidentitybindings
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: azureidentities.aadpodidentity.k8s.io
spec:
  group: aadpodidentity.k8s.io
  version: v1
  names:
    kind: AzureIdentity
    singular: azureidentity
    plural: azureidentities
  scope: Namespaced
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: aad-pod-id-nmi-role
rules:
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
- apiGroups: ["aadpodidentity.k8s.io"]
  resources: ["azureidentitybindings", "azureidentities"]
  verbs: ["get", "list"]
- apiGroups: ["aadpodidentity.k8s.io"]
  resources: ["azureassignedidentities"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: aad-pod-id-nmi-binding
  labels:
    k8s-app: aad-pod-id-nmi-binding
subjects:
- kind: ServiceAccount
  name: aad-pod-id-nmi-service-account
  namespace: **********
roleRef:
  kind: ClusterRole
  name: aad-pod-id-nmi-role
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    kubernetes.io/cluster-service: "true"
    component: nmi
    tier: node
    k8s-app: aad-pod-id
  name: nmi
  namespace: ***********
spec:
  template:
    metadata:
      labels:
        component: nmi
        tier: node
    spec:
      serviceAccountName: aad-pod-id-nmi-service-account
      hostNetwork: true
      volumes:
      - hostPath:
          path: /run/xtables.lock
          type: FileOrCreate
        name: iptableslock
      containers:
      - name: nmi
        image: "mcr.microsoft.com/k8s/aad-pod-identity/nmi"
        imagePullPolicy: Always
        args:
          - nmi
          - "--host-ip=$(HOST_IP)"
          - "--node=$(NODE_NAME)"
        env:
          - name: HOST_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
        securityContext:
          privileged: true
          capabilities:
            add:
            - NET_ADMIN
        volumeMounts:
        - mountPath: /run/xtables.lock
          name: iptableslock
      nodeSelector:
        beta.kubernetes.io/os: linux
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: aad-pod-id-mic-service-account
  namespace: **********
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: aad-pod-id-mic-role
rules:
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["*"]
- apiGroups: [""]
  resources: ["pods", "nodes"]
  verbs: [ "list", "watch" ]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create", "patch"]
- apiGroups: ["aadpodidentity.k8s.io"]
  resources: ["azureidentitybindings", "azureidentities"]
  verbs: ["get", "list", "watch", "post"]
- apiGroups: ["aadpodidentity.k8s.io"]
  resources: ["azureassignedidentities"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: aad-pod-id-mic-binding
  labels:
    k8s-app: aad-pod-id-mic-binding
subjects:
- kind: ServiceAccount
  name: aad-pod-id-mic-service-account
  namespace: *************
roleRef:
  kind: ClusterRole
  name: aad-pod-id-mic-role
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    component: mic
    k8s-app: aad-pod-id
  name: mic
  namespace: ************
spec:
  template:
    metadata:
      labels:
        component: mic
    spec:
      serviceAccountName: aad-pod-id-mic-service-account
      containers:
      - name: mic
        image: "mcr.microsoft.com/k8s/aad-pod-identity/mic:1.3"
        imagePullPolicy: Always
        args:
          - mic
          - "--cloudconfig=/etc/kubernetes/azure.json"
          - "--logtostderr"
        volumeMounts:
        - name: k8s-azure-file
          mountPath: /etc/kubernetes/azure.json
          readOnly: true
      volumes:
      - name: k8s-azure-file
        hostPath:
          path: /etc/kubernetes/azure.json
      nodeSelector:
        beta.kubernetes.io/os: linux
---
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentity
metadata:
 name: **************
spec:
 type: 0
 ResourceID: /subscriptions/****************/resourcegroups/************/providers/Microsoft.ManagedIdentity/userAssignedIdentities/**************
 ClientID: ********************
---
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentityBinding
metadata:
 name: **************
spec:
 AzureIdentity: **************
 Selector: k8s-secrets
subjects:
- kind: ServiceAccount
  name: aad-pod-id-nmi-service-account
  namespace: ******************
roleRef:
  kind: ClusterRole
  name: aad-pod-id-nmi-role
  apiGroup: rbac.authorization.k8s.io

Pod Volumes Setup:
- name: kvsecrets
         flexVolume:
           driver: "azure/kv"
           options:
             usepodidentity: "true"
             keyvaultname: *****************
             keyvaultobjectnames: jason
             keyvaultobjecttypes: secret
             resourcegroup: *****************
             subscriptionid: ********************
             tenantid: **************

Errors Received:

From Virtual Machines in AKS agentpool:

Message
The request content has the following duplicate identity ids '/subscriptions/****************/resourcegroups/****************/providers/Microsoft.ManagedIdentity/userAssignedIdentities*****************'.

From MC Resource Group:

Message
Number of write requests for subscription '********************' exceeded the limit of '1200' for time interval '01:00:00'. Please try again after '303' seconds.

From Pods after 30min:

MountVolume.SetUp failed for volume "kvsecrets" : mount command failed, status: Failure, reason: /etc/kubernetes/volumeplugins/azure~kv/azurekeyvault-flexvolume failed, F0628 11:42:55.791400 1832 main.go:82] [error] : failed to get keyvaultClient: failed to get key vault token: failed to get service principal token: nmi response failed with status code: 403

From Pods after 1hour:

Unable to mount volumes for pod "*******nginxproxy-**********(**********)": timeout expired waiting for volumes to attach or mount for pod "**********"/"***********". list of unmounted volumes=[kvsecrets]. list of unattached volumes=[kvsecrets default-token-********]

Expected behavior
Pods should successfully mount FlexVolume and have needed KeyVault Secrets mounted.
Key Vault FlexVolume version
image: "mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.10
Access mode: service principal or pod identity
I'm using Pod Identity
Kubernetes version
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.8", GitCommit:"a89f8c11a5f4f132503edbc4918c98518fd504e3", GitTreeState:"clean", BuildDate:"2019-04-23T04:41:47Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

Additional context
Please let me know if I can provide more information.

Thanks for reporting this issue @JasonKAls!
From the logs you have shared, it looks like the flexvolume driver was not able mount the volume as pod identity could not get a successful auth token for key vault failed to get service principal token: nmi response failed with status code: 403. This error  The request content has the following duplicate identity ids suggests the MIC is not able to assign the same identity to the same VM. https://github.com/Azure/aad-pod-identity/issues/167
Can you please share the entire pod/deployment yaml with the pod identity label aadpodidbinding: k8s-secrets and the flexvolume definition?
And please share outputs for the following commands so that we can see all the resources using pod identity:
kubectl get azureidentity

kubectl get azureidentitybinding

kubectl get azureassignedidentity

kubectl get pod -o wide

Another thing is please make sure you are using mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.4 as the above image has no tag.
cc @kkmsft @aramase

Thanks for responding, @ritazh!
I'll provide the other details, but the azure get commands for kubectl have never produced anything for me.  Even for the environments and pods where FlexVols are working.  Any suggestions?

Hi @JasonKAls  - the issue https://github.com/Azure/aad-pod-identity/issues/167 was reported in the context that there are already previously assigned identities on the node. Is that the case here - do you have some user assigned identities already assigned on the node for some other operation ?

Hello @kkmsft!
The nodes involved are from an AKS agentpool and only have 1 Managed Identity assigned to it.

@JasonKAls Can you pls verify what is the mechanism that assigned that one managed identity on that agent node? was it pod identity? did you manually assign the identity to that node? Thanks!

by using az identity create

@JasonKAls Thank you for the response. I've a few more questions -

Can you pls confirm the identity was never assigned manually using az vm identity assign?
Did you make any changes the ResourceID in the azureidentity during multiple retry attempts?
Can you pls post the output from az vm identity show --resource-group <rg> --name <vm name>? You can remove the subid from the output and post it here.

Hello @aramase!
Before I answer your questions, you should know @ritazh's suggestion to create Managed IDs per deployment seems to be working accept for one.

So the way we handle Identity creation is I use az identity create. I then need to ask another department to assign the correct permissions to it.  az vm identity assign is denied for me.
I do believe I tried recreating the Managed ID, but not for all of the pods.
I have 2 agentpools attached to my per AKS cluster per environment. I'd be happy to provide you the JSON output if you have a more secure method of submitting it to you, but for the record, I only received output for 1 of the 2 AKS agentpool VMs.  Let me know the best method to post output and I'd be happy to share. Not sure I feel comfortable posting any ID or reference information here.

Thanks again!

@JasonKAIs Can you send it to me over email -<EMAIL_ADDRESS>Please redact the client id/principal id and sensitive information from the output. I just need to verify the resource ID format as it appears in the output.

Done! For future reference for this ticket, JSON output should look like:
  "principalId": null,
  "tenantId": null,
  "type": "UserAssigned",
  "userAssignedIdentities": {
    "/subscriptions/************************/resourceGroups/*****************/providers/Microsoft.ManagedIdentity/userAssignedIdentities/prod********-nginx": {
      "clientId": "*******************",
      "principalId": "****************************"
    },
    "/subscriptions/**************************/resourceGroups/********************/providers/Microsoft.ManagedIdentity/userAssignedIdentities/prod***********-api": {
      "clientId": "*************************",
      "principalId": "**********************************"
    },
    "/subscriptions************************/resourceGroups/*************************/providers/Microsoft.ManagedIdentity/userAssignedIdentities/prod-*****************-web": {
      "clientId": "*********************************",
      "principalId": "***************************************"
    }
  }
}

to show each ID attached to a VM.  I have 3 on this one.

@JasonKAls Thank you for posting the output and also for sending it to me through email. I was able to recreate this issue on my cluster.
The issue was caused because the check to see if an id exists on the node was not case insensitive. The resourceID defined in the AzureIdentity you provided is ResourceID: /subscriptions/****************/resourcegroups/************/providers/Microsoft.ManagedIdentity/userAssignedIdentities/************** and the identity that already existed on the node had the format /subscriptions/************************/resourceGroups/*****************/providers/Microsoft.ManagedIdentity/userAssignedIdentities/prod********. The difference is in the case for the string resourceGroups on the node and resourcegroups in the AzureIdentity definition. The check should have been case insensitive.
The fix for that has been merged (https://github.com/Azure/aad-pod-identity/pull/271) and will be included as part of next release.

Hello @aramase!
That's fantastic news! Thanks for your teams diligent attention to this issue and for providing a temporary solution for me and my team.  I look forward to the next merger!

@JasonKAls 1.5-rc2 is now available. Please try it out and provide us with valuable feedback towards 1.5 release - https://github.com/Azure/aad-pod-identity/releases/tag/1.5-rc2
Closing this issue now. Please reopen if you have any further issues.


workspaceFolders isn't a model element

@mickaelistria,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot


A few shell tweaks to test suite

Just a couple of things I noticed that might make things a bit friendlier if you're keen.

Awesome! Thanks for this work @tejr
The use of mktemp is far cleaner and I hadn't considered the usage of trap to include the shutdown information, very tidy.


Some extra steps to install scripts on MacOS

I just noticed my Mac does not have autoconf and automake installed by default. I thought it is beneficial to list the extra step below for others trying to install the scripts on a Mac system:
foo@Macbar: brew install autoconf automake libtool

Thank you for the note.


Serial port is dead

Hello,
I'm trying to communicate between RPi Zero and Pixhawk 4 over the Pi's uart, but I get the error message "The device /dev/ttyS0 is dead" or /dev/serial0. I've tried looking in the souce code which function is giving that error, but can't find it. The serial port isn't the problem because I've tested it with other devices and it works. I was wondering if the problem is on the Pi's side or the Pixhawk's side.
Thank,
Brandon

Problem was I was trying to use Python3.

On Mon, 17 Jul 2017, Brandon du Toit wrote:

Problem was I was trying to use Python3.

Thanks for the update.  We'll now know more next time we see the same
symptoms!

Problem was I was trying to use Python3.

Exceuse me. So could you please share how did you solve the problem?? did you reinstall mavlink and the dronekit in python2 version?


brackets stopped responding when using bootstrap min css file

Using brackets as text editor. Bootstrap.min css file stopped responding on brackets. After a few seonds when bootstrap css file opens brackets hangs. Have to use task manager to close brackets when it's hangs. The file works fine with other text editors, but not on brackets. Apart from bootstrap.min.css other css files opening normally. Try Debug >Disable the extensions in brackets and tried again.when css file loads the problem started again.
Bootstrap version 3.3.2
Brackets version 1.1
Windows7
https://drive.google.com/open?id=0B7jY3Ri5R7prQ3BWRzNuOWpzN1k&authuser=0

@dinesh1d The Google Drive link doesn't have permissions to view/download the CSS file.  Can you upload it as a public Gist maybe?
Are you editing the bootstrap.min.css file, or just viewing it?  Are you putting your cursor in a specific place within the file, or running any other specific Brackets command (for example, Quick Edit) before the freeze occurs?

@peterflynn You can get it here: https://raw.githubusercontent.com/twbs/bootstrap/v3.3.2/dist/css/bootstrap.min.css
It's quite possible @dinesh1d sees #10879 (I see that too with that Bootstrap file and with extensions disabled), which would be fixed with my PR #10882.

@peterflynn I would upload it on gist. Yes i am editing bootstrap for customizing the page. sometime when using ctrl+f command on brackets, it's freezes after showing first few results. able to scroll page for a few minutes then it's hangs.

bootstrap.css, bootsrap.theme.css, bootstrap.theme.min.css all these file worked fine on brackets. only bootstrap.min.css is causing problems.

yes i too face this same problem with bootstrap min css. when i open BS min css to view/edit Brackets showing the file but after some scroll it hangs & then i have to end the process to close Bracket.
My pc configuration
Window 7 64bit
Intel 950 i7 processor
6gb ram
MSI 460gtx x 2 in SLI
Bracket is a very good editor but please try to solve this problem & make Bracket handle vary large coding files smoothly.. Thank in Advance :)

+1
Same issue here.


Training with custom data gives unexpected results.

I tried to fine-tune the given checkpoints with custom data. I had prepared the required data pipeline after taking reference from the given training data. Although I kept everything in the train.py almost same, I am getting a blank image as output during inference.
I have cross-checked the data preparation and found that the differenceMask I am calculating, comes out be an image of 0s for all cases.
Kindly let me know if there is something I might be missing in the steps. TIA!

Fine-tuning Gan's is something I don't recommend, you need to train from scratch. You can get the data from cp-vton link, and try to replicate the results.

Gan learns a particular distribution since GAN's are trained in an adversarial fashion which helps them to replicate certain underlying patterns in a dataset.
Also, we present a concept rather than a system, so you need to work on your end. We were able to use this concept in 4 different tasks infact.

@nile649 Great work! Thanks for sharing.
I understood your point and trying to outwork something around it. Actually, I was trying to use it for cloth warping which I believe is the primary focus of this paper. Kindly let me know if you know some good opensource project on this which is generalizable enough to get satisfactory results on a custom dataset of different cloth types of both men and women.

In my research, we didn’t find anything which can give a perfect result. My work gives good results, but again it’s not the best, and even current papers from CVPR are pretty much the same in metric comparisons.

Body parsing, Pose estimation, Segmentation network, and then Poly-GAN model or any other model.

I have tried with different warping models from latest SOTA papers as well. But the applications of these papers seems limited to only few cloth types like shirts or t-shirts. Given the diversity of cloths of men and women, I believe no single model is able to do justice in warping. Tried training with a subset of my dataset as well, but still it is not able to learn other different cloth types. Looks like we need more diverse data samples to benchmark the models and a more novel architecture.

It won't, the diversity of clothes also means the diversity of distribution of data. GAN's aren’t meant for such complex tasks.

At least not for now. Also, the output from a GAN is not that crisp as we are sampling from a large distribution, which affects the real experience of a VTO. Anyways, I am working on an alternative approach. Thank you for your inputs. Appreciate it.