Add alignment string position support

Alignment string position support
Types of changes
This PR implements the following changes:

[x] :sparkles: New feature (A non-breaking change which adds functionality).
[ ] :bug: Bug fix (A non-breaking change, which fixes an issue).
[ ] :boom: Breaking change (fix or feature that would cause existing functionality to change).

:clipboard: Additional detail

Mapping positions from/to the input or reference sequence onto the common alignment string is quite useful (e.g. when comparing annotations attached to specific positions of the aligned strings). In addition to seq2ref() and ref2seq(), this PR adds 4 new functions to the public API: seq2agn, ref2agn, agn2seq, agn2ref, which allow to convert between input/reference/alignment positions in any direction.

Technically, keeping track of alignment positions is implemented as an AlignmentAnchor.agnpos field. Updating the pairwise alignment algorithms to support alignment positions was easy, only the common traceback utility macros had to be changed. It also should not affect the performance or memory consumption.

I also did a few minor cleanups (@inbounds were beneficial, refactored position conversion to be more generic and using Julia 1.x approach (avoid generated functions, rely on constant propagation, argument splatting)).

Provide a runnable example of use of your addition. This lets reviewers
and others try out the feature before it is merged or makes it's way to release.

:ballot_box_with_check: Checklist

[x] :art: The changes implemented is consistent with the julia style guide.
[x] :blue_book: I have updated and added relevant docstrings, in a manner consistent with the documentation styleguide.
[ ] :blue_book: I have added or updated relevant user and developer manuals/documentation in docs/src/.
[x] :ok: There are unit tests that cover the code changes I have made.
[x] :ok: The unit tests cover my code changes AND they pass.
[ ] :pencil: I have added an entry to the [UNRELEASED] section of the manually curated CHANGELOG.md file for this repository.
[x] :ok: All changes should be compatible with the latest stable version of Julia.
[x] :thought_balloon: I have commented liberally for any complex pieces of internal code.

The failures due to Registry.toml problems, the testing even didn't start.
Also: is it possible to update CI to test on latest stable Julia and also on nightly (with allow_failures). I guess only LTS (is it 1.0 or 1.1?) Julia should be kept from the previous versions.

Some other notes on the PR:

In this PR I haven't yet updated the manual -- but if the maintainers think it's reasonable to include the alignment position functionality, I'll update it. The terminology (e.g. alignment position) also has to be clarified before committing to update the docs.
The PairwiseAlignment iterator interface could be updated to return the alignment position. But that would break the API, so I don't know what to do about that.
In addition to the alignment position, it would be nice to have API to query the input/reference sequence ranges aligned. Partly it exists as Alignment.firstref/lastref, but so far it's not public API and it doesn't cover input sequence positions.
Not a part of this PR, but something that I noticed: there Alignment, AlignedSequence and PairwiseAlignment types. Judging from the names, PariwiseAlignment and Alignment are expected to be from the same type hierarchy, but in fact these are very different types. Perhaps, it's possible to rename Alignment into something like AlignmentDefinition or AlignmentTransform (and fix the .jl filename and field names of that type accordingly (e.g. into transform -- right now in some parts of the code one may read aln.a.aln.anchors)) to avoid the confusion.

I don't know who would be the best to review, but since you were committing recently, maybe you can take a look, @BenJWard ? Or maybe you can suggest a more suitable reviewer? Thanks in advance!

@BenJWard @jakobnissen If you would have time, could you please review this PR?

I'm not 100% sure what this functionality is useful for, though. Does the concept of an "alignment position" make sense?

For multiple sequence alignments it definitely does, because it's actually the column of MSA.
But it also does make sense for pairwise one. If you want to display the pairwise alignment, you may need to know what is the position of a specific letter from either sequence in the aligned sequence.
My use case was to correctly display the post-translational modifications (phosphorylation, ubiquitination etc) of inidividual amino acids on the pairwise-aligned homologous protein sequences.
There could be regions that are present only on the first sequence or regions that are present only on the second one. So you definitely need an aligned position, which preserves everything.

@jakobnissen  It looks like at the moment CI seems broken. Should this package be updated to use GitHab actions for CI?
I can try submitting the PR for that, but maybe it would be better if someone who has better knowledge of BioJulia ecosystem is doing that.

Yeah... BioJulia is strapped for dev manpower at the moment, so there is a lot of cleanup and housekeeping that needs to be done in all sorts of packages, and just haven't been done. If you make a PR, I'll merge it! You can look at e.g. BioSequences for how to do this.

@alyst Ping me when it's ready to be merged :)

@jakobnissen Thank you! I think the PR is good to go.
I have addressed your suggestions, also I have added tests to the new aln2xxx/xxx2aln() methods that were not covered.
There are still new lines in the PR that are not covered, but they were derived from the existing exceptions that got more specific after alnpos was added.
They were not covered by the tests before. So I think it's fine to merge this PR, and address the coverage later by introducing more test cases.

Thanks for putting in the work!

Docs need updating: https://github.com/BioJulia/BioAlignments.jl/blob/master/docs/src/alignments.md#representing-alignments


Recommend using negative PK values for seed data in tables with auto-increment PKs

After facing an error message like this:
The seed entity for entity type 'Entity' cannot be added because a non-zero value is required for property 'Id'. Consider providing a negative value to avoid collisions with non-seed data.

And reading the following issues:
https://github.com/aspnet/EntityFrameworkCore/issues/11776
https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/issues/536
I would suggest adding a tip or a warning to Data Seeding section with a recommendation of using negative PK values in seed data for tables with auto-increment PKs to avoid possible collisions with non-seed data.

With https://github.com/npgsql/efcore.pg/issues/367 done on the Npgsql side in version 5.0, there shouldn't be a need to seed negative ID values any more.

With https://github.com/npgsql/efcore.pg/issues/367 done on the Npgsql side in version 5.0, there shouldn't be a need to seed negative ID values any more.

(AFAIK PostgreSQL was the only provider with this issue)

(AFAIK PostgreSQL was the only provider with this issue)


Update Composer dependencies (2022-01-05-21-12)

Visual regression test passed!
View the visual regression test report


Show test explorer only when tests have been discovered

Show test explorer only when tests have been discovered
Hide if there are no workspaces opened

Closing in favor of $4317


Add geeks scraper

@irisliu77  update this client and servlet after client interface is approved #33

approved :)


HDDS-2193. Adding container related metrics in SCM.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
40
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
66
Maven dependency ordering for branch

-1
mvninstall
31
hadoop-hdds in trunk failed.

-1
mvninstall
30
hadoop-ozone in trunk failed.

-1
compile
21
hadoop-hdds in trunk failed.

-1
compile
15
hadoop-ozone in trunk failed.

+1
checkstyle
61
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
849
branch has no errors when building and testing our client artifacts.

-1
javadoc
23
hadoop-hdds in trunk failed.

-1
javadoc
20
hadoop-ozone in trunk failed.

0
spotbugs
951
Used deprecated FindBugs config; considering switching to SpotBugs.

-1
findbugs
34
hadoop-hdds in trunk failed.

-1
findbugs
21
hadoop-ozone in trunk failed.

_ Patch Compile Tests _

0
mvndep
32
Maven dependency ordering for patch

-1
mvninstall
36
hadoop-hdds in the patch failed.

-1
mvninstall
29
hadoop-ozone in the patch failed.

-1
compile
25
hadoop-hdds in the patch failed.

-1
compile
20
hadoop-ozone in the patch failed.

-1
javac
25
hadoop-hdds in the patch failed.

-1
javac
20
hadoop-ozone in the patch failed.

+1
checkstyle
62
the patch passed

+1
mvnsite
0
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
720
patch has no errors when building and testing our client artifacts.

-1
javadoc
22
hadoop-hdds in the patch failed.

-1
javadoc
20
hadoop-ozone in the patch failed.

-1
findbugs
31
hadoop-hdds in the patch failed.

-1
findbugs
20
hadoop-ozone in the patch failed.

_ Other Tests _

-1
unit
29
hadoop-hdds in the patch failed.

-1
unit
23
hadoop-ozone in the patch failed.

+1
asflicense
33
The patch does not generate ASF License warnings.

2443

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1534

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 86ecbcbe0081 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / b1e55cf

Default Java
1.8.0_222

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/branch-findbugs-hadoop-ozone.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-compile-hadoop-ozone.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-compile-hadoop-hdds.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-findbugs-hadoop-ozone.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-unit-hadoop-hdds.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/artifact/out/patch-unit-hadoop-ozone.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/testReport/

Max. process+thread count
411 (vs. ulimit of 5500)

modules
C: hadoop-hdds/server-scm hadoop-ozone/integration-test U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/3/console

versions
git=2.7.4 maven=3.3.9

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
46
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
65
Maven dependency ordering for branch

-1
mvninstall
32
hadoop-hdds in trunk failed.

-1
mvninstall
29
hadoop-ozone in trunk failed.

-1
compile
18
hadoop-hdds in trunk failed.

-1
compile
14
hadoop-ozone in trunk failed.

+1
checkstyle
64
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
1034
branch has no errors when building and testing our client artifacts.

-1
javadoc
22
hadoop-hdds in trunk failed.

-1
javadoc
19
hadoop-ozone in trunk failed.

0
spotbugs
1132
Used deprecated FindBugs config; considering switching to SpotBugs.

-1
findbugs
33
hadoop-hdds in trunk failed.

-1
findbugs
19
hadoop-ozone in trunk failed.

_ Patch Compile Tests _

0
mvndep
32
Maven dependency ordering for patch

-1
mvninstall
38
hadoop-hdds in the patch failed.

-1
mvninstall
29
hadoop-ozone in the patch failed.

-1
compile
26
hadoop-hdds in the patch failed.

-1
compile
19
hadoop-ozone in the patch failed.

-1
javac
26
hadoop-hdds in the patch failed.

-1
javac
19
hadoop-ozone in the patch failed.

+1
checkstyle
61
the patch passed

+1
mvnsite
0
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
904
patch has no errors when building and testing our client artifacts.

-1
javadoc
21
hadoop-hdds in the patch failed.

-1
javadoc
17
hadoop-ozone in the patch failed.

-1
findbugs
32
hadoop-hdds in the patch failed.

-1
findbugs
19
hadoop-ozone in the patch failed.

_ Other Tests _

-1
unit
34
hadoop-hdds in the patch failed.

-1
unit
23
hadoop-ozone in the patch failed.

+1
asflicense
34
The patch does not generate ASF License warnings.

2824

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1534

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux c6a0cff0280b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / b1e55cf

Default Java
1.8.0_222

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/branch-findbugs-hadoop-ozone.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-compile-hadoop-ozone.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-compile-hadoop-hdds.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-findbugs-hadoop-ozone.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-unit-hadoop-hdds.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/artifact/out/patch-unit-hadoop-ozone.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/testReport/

Max. process+thread count
412 (vs. ulimit of 5500)

modules
C: hadoop-hdds/server-scm hadoop-ozone/integration-test U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/2/console

versions
git=2.7.4 maven=3.3.9

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
164
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
94
Maven dependency ordering for branch

-1
mvninstall
59
hadoop-hdds in trunk failed.

-1
mvninstall
37
hadoop-ozone in trunk failed.

-1
compile
22
hadoop-hdds in trunk failed.

-1
compile
16
hadoop-ozone in trunk failed.

+1
checkstyle
77
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
1168
branch has no errors when building and testing our client artifacts.

-1
javadoc
22
hadoop-hdds in trunk failed.

-1
javadoc
23
hadoop-ozone in trunk failed.

0
spotbugs
1282
Used deprecated FindBugs config; considering switching to SpotBugs.

-1
findbugs
44
hadoop-hdds in trunk failed.

-1
findbugs
20
hadoop-ozone in trunk failed.

_ Patch Compile Tests _

0
mvndep
34
Maven dependency ordering for patch

-1
mvninstall
40
hadoop-hdds in the patch failed.

-1
mvninstall
30
hadoop-ozone in the patch failed.

-1
compile
24
hadoop-hdds in the patch failed.

-1
compile
19
hadoop-ozone in the patch failed.

-1
javac
24
hadoop-hdds in the patch failed.

-1
javac
19
hadoop-ozone in the patch failed.

+1
checkstyle
65
the patch passed

+1
mvnsite
0
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
899
patch has no errors when building and testing our client artifacts.

-1
javadoc
19
hadoop-hdds in the patch failed.

-1
javadoc
18
hadoop-ozone in the patch failed.

-1
findbugs
29
hadoop-hdds in the patch failed.

-1
findbugs
18
hadoop-ozone in the patch failed.

_ Other Tests _

-1
unit
29
hadoop-hdds in the patch failed.

-1
unit
20
hadoop-ozone in the patch failed.

+1
asflicense
30
The patch does not generate ASF License warnings.

3141

Subsystem
Report/Notes

Docker
Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1534

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux fc007d0fa81f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / b1e55cf

Default Java
1.8.0_222

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/branch-findbugs-hadoop-ozone.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-compile-hadoop-ozone.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-compile-hadoop-hdds.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-findbugs-hadoop-ozone.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-unit-hadoop-hdds.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/artifact/out/patch-unit-hadoop-ozone.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/testReport/

Max. process+thread count
308 (vs. ulimit of 5500)

modules
C: hadoop-hdds/server-scm hadoop-ozone/integration-test U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/1/console

versions
git=2.7.4 maven=3.3.9

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
1800
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

0
mvndep
66
Maven dependency ordering for branch

-1
mvninstall
31
hadoop-hdds in trunk failed.

-1
mvninstall
30
hadoop-ozone in trunk failed.

-1
compile
20
hadoop-hdds in trunk failed.

-1
compile
15
hadoop-ozone in trunk failed.

+1
checkstyle
60
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
860
branch has no errors when building and testing our client artifacts.

-1
javadoc
22
hadoop-hdds in trunk failed.

-1
javadoc
21
hadoop-ozone in trunk failed.

0
spotbugs
958
Used deprecated FindBugs config; considering switching to SpotBugs.

-1
findbugs
30
hadoop-hdds in trunk failed.

-1
findbugs
21
hadoop-ozone in trunk failed.

_ Patch Compile Tests _

0
mvndep
31
Maven dependency ordering for patch

-1
mvninstall
34
hadoop-hdds in the patch failed.

-1
mvninstall
30
hadoop-ozone in the patch failed.

-1
compile
24
hadoop-hdds in the patch failed.

-1
compile
20
hadoop-ozone in the patch failed.

-1
javac
24
hadoop-hdds in the patch failed.

-1
javac
20
hadoop-ozone in the patch failed.

+1
checkstyle
57
the patch passed

+1
mvnsite
0
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
719
patch has no errors when building and testing our client artifacts.

-1
javadoc
21
hadoop-hdds in the patch failed.

-1
javadoc
20
hadoop-ozone in the patch failed.

-1
findbugs
30
hadoop-hdds in the patch failed.

-1
findbugs
21
hadoop-ozone in the patch failed.

_ Other Tests _

-1
unit
29
hadoop-hdds in the patch failed.

-1
unit
24
hadoop-ozone in the patch failed.

+1
asflicense
33
The patch does not generate ASF License warnings.

4201

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1534

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux b3121e605a17 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 2adcc3c

Default Java
1.8.0_222

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/branch-findbugs-hadoop-ozone.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-mvninstall-hadoop-hdds.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-mvninstall-hadoop-ozone.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-compile-hadoop-hdds.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-compile-hadoop-ozone.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-compile-hadoop-hdds.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-compile-hadoop-ozone.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-javadoc-hadoop-hdds.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-javadoc-hadoop-ozone.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-findbugs-hadoop-hdds.txt

findbugs
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-findbugs-hadoop-ozone.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-unit-hadoop-hdds.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/artifact/out/patch-unit-hadoop-ozone.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/testReport/

Max. process+thread count
415 (vs. ulimit of 5500)

modules
C: hadoop-hdds/server-scm hadoop-ozone/integration-test U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1534/4/console

versions
git=2.7.4 maven=3.3.9

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Thank you for the contribution. I have committed this patch to the trunk.


Concatenated Files Too Large

Not sure what is causing this, hoping someone can help me out.
When I concatenate 3 mp3 files that are each about 8.5Mb in size, fetched from AWS S3 bucket, the resultant mp3 file after concatenation and download is a whopping 280Mb, which is about 10x what I would expect.
If I get that large file and then using a program like VLC player I stream to export as a fresh mp3 file, it is reduced to about 30Mb, closer to what I would expect.
FWIW, The Content-Type metadata of the files on the S3 bucket is set to audio/mpeg.
Is there some obvious configuration/setting that can resolve this ?

I assume you use the export method at the end of your Crunker workflow? The problem is that it exports the file in .wav format (even if you specify .mp3) as described here. This is probably causing the increase in size.
The solution would be using a 3rd party to convert AudioBuffer directly to mp3, or optionally the exported .wav to .mp3. Unless you'd like to add export to mp3 method to Crunker - that would be lovely ;)

yeah, that would be the reason. thanks

Export to mp3 is a welcome addition. Not something I plan on tackling but I think it would be a useful and desirable addition.


Problem using File manager since 18.31 update (error in file-manager-lib.pm)

Hello,
Since last update (18.29 -> 18.31 applied manually since automatic update was not working)
File Manager no more works.

4 different servers
all 4 are ubuntu 14.04 lts kept up-to-date

File manager loads and chrome and Firefox consoles report :
"Undefined subroutine &extensions::get_selinux_command_type called at /usr/share/webmin/authentic-theme/extensions/file-manager/file-manager-lib.pm line 113"
All further action in file manager won't work then
Can you help ?

You apparently made git pull instead of using the package from Releases.

The git pull is only suitable for Webmin 18.30+.

Well, I'm not sure how I did...
So I only have to replace authentic-theme dir by the one I download from release package ?

Just install it using recommendations on main page of the repo.

How to install Authentic Theme?
Webmin
To install Autentic Theme from repository, just clone it into your wherever Webmin binary folder (libexec) is. After that, you must rename authentic-theme-master directory to authentic-theme to make the theme work properly. Besides, you could just download the theme directly and install it using Webmin:
Webmin->Webmin Configuration->Webmin Themes->Install themes->From uploaded file
From uploaded file - link


improve zip performance

Which issue does this PR close?
No issue
Rationale for this change
I saw that we do for every range of true check if the truthy values and falsy are scalar, which can be checked once
What changes are included in this PR?

Specialize cases for scalars in zip function
Add extend_scalar function to MutableArrayData to be optimized for single value that is being copy over and over again

Are there any user-facing changes?
No (I think)

there are no benchmarks to zip unfortunately

Perhaps you might be able to contribute a benchmark showing the performance difference, there is a lot going on in this kernel and it isn't immediately clear to me that the dyn dispatch in MutableArrayData won't dominate.

Perhaps you might be able to contribute a benchmark showing the performance difference, there is a lot going on in this kernel and it isn't immediately clear to me that the dyn dispatch in MutableArrayData won't dominate.

I actually starting to avoid this dyn dispatch in here:

https://github.com/apache/arrow-rs/pull/6815

I actually started to create a (WIP) PR to avoid this dyn dispatch:
The dyn dispatch is unavoidable in the general case, as arrow supports arbitrarily nested data structures.
Generally the approach is to have specialized kernel implementations for non-nested types where this yields better performance, and then fallback to type-erased versions where this isn't possible

So I did local benchmarks and it did not improve the perf from what I saw, so closing


22 gpu docker

Closes #22
Finally made it work with the GPU also. Only had to change the base image to an official one from NVIDIA (which also supports cuDNN 8).
I added a description on how to set it up for a different worker and run it on the wiki of the example worker and pointed to the wiki in the README.

Not really related to GPU-containerization, but maybe add config.yml to .dockerignore, to make sure that the worker is using the config mounted in /root/.DANE

It already is there (/config)

Shall I merge?

@Veldhoen reminder that I left some comments. Shall I go ahead and merge?

Sorry for the late reply - yes go ahead!


GUVNOR-2541: Guided Decision Table Editor: Add support to 'Save all'

See https://issues.jboss.org/browse/GUVNOR-2541
Simply adds a "Save all" button to the Guided Decision Table editor.

Part of an ensemble:
https://github.com/droolsjbpm/kie-wb-common/pull/377
https://github.com/droolsjbpm/drools-wb/pull/190

+1


Error when using PMEM_AND_DISK storage level

Hello. I'm trying to cache RDD in my persistent memory and I continuously get this error.

My system is a single NUMA node system and I have one pmem path.
I installed spark version 3.0.0 and hadoop version 3.2.0 so that I can use pmem-spill and pmem-common v1.1.0-spark-3.0.0.
Because of the error message as above, I first thought that the reason of the problem was because of the memkind library. However I tested the memkind library with the test examples provided and checked that memkind_malloc worked fine.
Is there something I'm missing? Thank you.

@rophy1310 Are you trying the release jar file? If so, please try to build from source code following the doc and check whether libmemkind.so is under LD_LIBRARY_PATH.

I followed the developer guide, built the OAP from the source code, and also checked libmemkind.so is under LD_LIBRARY_PATH, but still had the same problem. So I checked the dependency installation script(prepare_oap_env.sh) provided at oap-tools and find out that it is better to use the oap-tools master branch for Ubuntu. After using this version, the problem seems to be solved. So now I am using Spark ver 3.1.1, and Hadoop ver 3.2.0.
But now I'm having another problem: java.lang.NoClassDefFoundError: com/intel/oap/common/unsafe/PersistentMemoryPlatform
I checked the pmem-common/target/classes/com/intel/oap/common/unsafe directory and there was PersistentMemoryPlatform class in there, so I'm not sure why this error occurs.

Did you add pmem-commom to spark.executor.extraClassPath? See the detail: https://github.com/oap-project/pmem-spill#configuration

Yes. I added pmem-common to spark.executor.extraClassPath as the below screenshot.

 It's better to run spark on yarn with these configuration.

I was using standalone mode in spark. The spark command I'm using is the K-means benchmark which is mentioned in the https://oap-project.github.io/pmem-spill/1.1.1/ .
The above problem was solved after changing to yarn mode and running the command as the root user. Thank you.
But now I'm having an error: java.lang.ExceptionInInitializerError: Persistent memory initialize (path: /mnt/pmem0/executor_1:, size: 248) failed. Please check the path permission and initial size. (Please refer to the screenshot below.)

I checked the initialize size and path, but it looked okay to me because the path is /mnt/pmem0 and the size is 256GB. At first, I thought that the size was the problem so I tried it with both values 248(which was the size I could check through command lsblk), 256 but still the problem occurs.
I run the command as root user but could the problem occur because of the path permission?

I fixed the above problem by changing the configuration in spark-defaults.conf.
It is said to put my Optane PMem size in GB so I put the value 256 as below.
...
spark.memory.pmem.initial.path /mnt/pmem0
spark.memory.pmem.initial.size 256
...
But somehow, it doesn't converts to Bytes through conf.getSizeAsBytes("spark.memory.pmem.initial.size",0L) in Blockmanager.scala.
So I converted the value to Bytes as below.
...
spark.memory.pmem.initial.path /mnt/pmem0
spark.memory.pmem.initial.size<PHONE_NUMBER>44
...
And the problem was solved. Now the K-means benchmark works. Thank you very much.

spark.memory.pmem.initial.size 256 -> spark.memory.pmem.initial.size 256g
Add unit will solve this problem.

I tried using spark.memory.pmem.initial.size 256g and it worked.
Thank you very much.


Using keyboard to copy and paste blocks no longer works

This was originally reported on the Makeroid forum here.
Copying and pasting blocks using the CTRL+C and CTRL+V keyboard combinations no longer works on Makeroid. I have tested on ai2.appinventor.mit.edu and it works, however, it doesn't work on ai2-test.appinventor.mit.edu, it shows the  typeblock bar instead.
After doing some tests, I found out that reverting commit ba120fc solves the issue.

@conorshipp please take a look at #1069


Clear cache to avoid OOM with iterative generation

I am running a 3.0bpw mixtral on a 3090, so just enough room to fit, but I noticed increasing vram through a generating loop until eventually hitting OOM.  Don't really need to hold kv cache since each generation is independent of one another, so can just clear it with each call.  Code provided below
config.model_dir = "/mnt/models/"
config.prepare()

model = ExLlamaV2(config)
cache = ExLlamaV2Cache(model, lazy = True)

model.load_autosplit(cache)

tokenizer = ExLlamaV2Tokenizer(config)
generator = ExLlamaV2StreamingGenerator(model, cache, tokenizer)
generator.set_stop_conditions([tokenizer.eos_token_id])
gen_settings = ExLlamaV2Sampler.Settings()

def process_with_language_model(input_text):
    cache = ExLlamaV2Cache(model, lazy = True)
    cache.current_seq_len = 0
    instruction_ids = tokenizer.encode(f"[INST] {input_text} [/INST]", add_bos = True)
    context_ids = instruction_ids if generator.sequence_ids is None \
        else torch.cat([generator.sequence_ids, instruction_ids], dim = -1)
    generator.begin_stream(context_ids, gen_settings)

    output_text = ""
    while True:
        chunk, eos, _ = generator.stream()
        if eos: break
        output_text += chunk
        print(chunk, end="")
        sys.stdout.flush()

    return output_text.strip()

If you keep generating with no token limit you'll eventually fill up the cache, but at that point you should get an overflow error, not OoM. Calling generator.begin_stream will reset the cache.
If repeatedly calling this function leads to responses of less than 32k tokens in total (the default for Mixtral) but you're still getting OoM at some point, there could be a memory leak somewhere, but the cache is static and doesn't grow with the sequence length.
It could be that you're right on the cusp of running out of VRAM and something other than exllamav2 (like your window manager or whatever) is allocating a little too much in the background. You could try setting config.max_seq_len = 20000 or something, right after config.prepare() which could free up some memory overall.
These two lines:
    cache = ExLlamaV2Cache(model, lazy = True)
    cache.current_seq_len = 0

have no effect since the generator is created with the first cache object. With lazy = True no VRAM is allocated, though, instead deferring to the autosplit model loader which isn't called subsequently.

Ah it might be that overflow, I didn't see the OoM just noticed when it stopped generating I was looking at runpod and it showed gpu util at 99% so I assumed it was OoM.  I'll try it again with the generator.begin_stream.  Also I don't need to set the cache seq len to 0 with the lazy mode?

You don't need to create a second instance of the cache at all. There's only one cache used by the generator and it's the one you're creating on line 4 of your example, to pass to the ExLlamaV2StreamingGenerator constructor.
Lazy mode just creates the cache without allocating the cache tensors in VRAM. This is for use with the load_autosplit function, which allocates cache layers as it's loading the model, to keep everything aligned across multiple GPUs without knowing in advance which layers will end up on which device.
To actually reset the cache you would indeed set current_seq_len = 0, but this done implicitly by generator.begin_stream anyway, which starts a generation with the provided token IDs which in turn are inserted into the context starting from position zero.

I have found repeated generation causes VRAM usage to creep up. I'm doing:
    gc.collect()
    torch.cuda.empty_cache()

after generations, which keeps VRAM usage consistent but does cost a bit of perf. But the alternative is having VRAM climb until it hits "shared" memory and then perf goes off a cliff.

This does look like a memory management issue in PyTorch. Manually calling gc.collect() and torch.cuda.empty_cache() aren't supposed to do anything, and it's kind of a mystery why it sometimes works, since all the functionality should automatically be invoked when it's needed.
With the shared memory option in later NVIDIA drivers, it's possible that Torch simply doesn't recognize that it's running out of memory and doesn't properly manage its cache since it sees more VRAM than what's physically available. Another reason to disable that evil feature, I guess?

I found that a big part of my problem was that variable scope in python is pretty broad. I had things like caches = [_.cache for _ in work] and even if that went out of "scope" (in the C++ sense I'm used to thinking in), it was still alive under python rules. That meant I often held references to caches and sampler states when allocating new caches (or unloading a model and loading a new one). So I was getting fragmentation and unnecessarily high peak memory usage.
Just going through the code and whacking temporaries to None goes a long way to keeping reserved memory from growing.


5.3.0 Fatal Error Loading Plugin

From skvevr.log:
checking plugin C:\Program Files (x86)\Steam\steamapps\common\SkyrimVR\Data\SKSE\Plugins\\po3_PapyrusExtender.dll
plugin C:\Program Files (x86)\Steam\steamapps\common\SkyrimVR\Data\SKSE\Plugins\\po3_PapyrusExtender.dll (00000001 powerofthree's Papyrus Extender 00000005) disabled, fatal error occurred while loading plugin (handle 15)

Downgrading to 5.2.0 stopped this fatal error for me. I assume it's a mod conflict since I haven't seen other reports. Is there anything I should look for specifically, or an easy way to share my mod list from Vortex? Also, I always remove the PDB before installing. I'm not sure if keeping it would help here.
Probably relevant: I got the same error with the SPID VR 6.0.3, and downgrading also helped with that. I've opened a separate issue for that here.

I found the Vortex filters for SKSEVR-related mods (executables and extenders), in case this helps to spot potential conflicts:

There's normally an error code you can lookup. Did you install all requirements including VC runtime?

The log snippet I posted has the only related codes I noticed (00000001 and 00000005). I'll try again later to see if I can get anything else out of it.
Regarding requirements, 5.2.0 works fine, so I assume my VC is fine. For 5.3.0 I updated po3's Papyrus Extender to keep versions matched. SKSEVR and the address library are the same.

Oh, it's probably the address library. Vortex hasn't detected any updates since 0.48.0, I guess because versions started receiving "v" prefixes, so I didn't realize mine was pretty outdated.

Yep that's what it was. Thanks for the nudge.

Oh, it's probably the address library. Vortex hasn't detected any updates since 0.48.0, I guess because versions started receiving "v" prefixes, so I didn't realize mine was pretty outdated.

Oh hmm. I may have to fix that convention anyway. My CI ended up with the v version numbers and I just let it go. I didn't realize it'd break Vortex.


[REQUEST] Does DeepSpeed support ONNX export and Triton Inference Server?

Hi team, QQ: does deepspeed support the followings,

Convert HuggingFace BERT/RoBERTa models to int8 precision directly
If yes, can the converted model be exported to ONNX format directly?
If so, can the exported ONNX model loaded correctly using Triton Inference Server?

Please re-open this issue if this is something still important to you.


Merge Commands

Currently there is share_plugin and share_theme, they should be merged into share. Seeing as themes and plugins share a an install, uninstall, enable, and disable` this only makes sense.
The command for spotify-modal would need to be renamed to something like share_song but that's easy enough.

Should probably also like, merge this whole thing into replugged core but ¯_(ツ)_/¯

I think we'd rather merge this and rename the share command

It is now integrated natively into Replugged


feat: New module avm/res/service-networking/traffic-controller

Description
New Module avm/res/service-networking/traffic-controller (Application Gateway for Containers).
Related Module Proposal: https://github.com/Azure/Azure-Verified-Modules/issues/712
Adding a new module

[x] A proposal has been submitted and approved.
[ ] I have included "Closes #{module_proposal_issue_number}" in the PR description.
[ ] I have run brm validate locally to verify the module files.
[x] I have run deployment tests locally to ensure the module is deployable.

Pipeline references

Pipeline

@AlexanderSehr, thanks for the review, your comments are implemented and the tests refreshed.


command instead of cmdlet

Hi Guys!
Noticed a small mistake on this page.
Within the Set up a key encryption key (KEK) section, under Azure PowerShell, below the first cmdlet there is the following statement:

You may instead import a private key using the Azure PowerShell az keyvault key import command.

Since this referring to PowerShell I think the line should read:

You may instead import a private key using the Azure PowerShell Add-AzKeyVaultKey command.

Hope this helps!
Cheers

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 26bcd765-e7de-c91e-2193-a6b7c166ed05
Version Independent ID: 6432a759-9995-e08a-ee31-6077fefca464
Content: Creating and configuring a key vault for Azure Disk Encryption on a Windows VM - Azure Virtual Machines
Content Source: articles/virtual-machines/windows/disk-encryption-key-vault.md
Service: virtual-machines
Sub-service: disks
GitHub Login: @msmbaldwin
Microsoft Alias: mbaldwin

Hello @porkamizeria
I have assigned the issue to the content author for further review.

Hi, @porkamizeria. This paragraph was recently updated to no longer reference either CLI or PowerShell. It now reads:

You can instead import a KEK from your on-premises key management HSM. For more information, see Key Vault Documentation.

Thank you! #please-close


Added CSP_DISABLE env option - to disable strict Content Security Policy

Intent
Added CSP_DISABLE env option - to disable strict Content Security Policy

Per discussion, let's switch this to using a defined set of CSP directives and disable the "disable" option


HPIPMInterface Build Errors

When compiling with HPIPM enabled I receive the following errors:
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp: In member function ‘void ct::optcon::HPIPMInterface<STATE_DIM, CONTROL_DIM>::computeLrInvArray()’:
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp:222:11: error: ‘::d_ocp_qp_ipm_get_ric_Lr’ has not been declared; did you mean ‘d_ocp_qp_ipm_get_iter’?
  222 |         ::d_ocp_qp_ipm_get_ric_Lr(i, &workspace_, Lr.data());
      |           ^~~~~~~~~~~~~~~~~~~~~~~
      |           d_ocp_qp_ipm_get_iter
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp: In member function ‘virtual void ct::optcon::HPIPMInterface<STATE_DIM, CONTROL_DIM>::computeFeedbackMatrices()’:
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp:243:11: error: ‘::d_ocp_qp_ipm_get_ric_Ls’ has not been declared; did you mean ‘d_ocp_qp_ipm_get_iter’?
  243 |         ::d_ocp_qp_ipm_get_ric_Ls(i, &workspace_, Ls.data());
      |           ^~~~~~~~~~~~~~~~~~~~~~~
      |           d_ocp_qp_ipm_get_iter
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp:250:7: error: ‘::d_ocp_qp_ipm_get_ric_P’ has not been declared; did you mean ‘d_ocp_qp_ipm_get_iter’?
  250 |     ::d_ocp_qp_ipm_get_ric_P(1, &workspace_, S1.data());
      |       ^~~~~~~~~~~~~~~~~~~~~~
      |       d_ocp_qp_ipm_get_iter
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp: In member function ‘virtual void ct::optcon::HPIPMInterface<STATE_DIM, CONTROL_DIM>::compute_lv()’:
/usr/include/ct/optcon/solver/lqp/HPIPMInterface-impl.hpp:275:11: error: ‘::d_ocp_qp_ipm_get_ric_lr’ has not been declared; did you mean ‘d_ocp_qp_ipm_get_iter’?
  275 |         ::d_ocp_qp_ipm_get_ric_lr(i, &workspace_, lr.data());
      |           ^~~~~~~~~~~~~~~~~~~~~~~
      |           d_ocp_qp_ipm_get_iter

OS: ArchLinux
HPIPM: 0.1.1

Hi @acxz
On HPIPM, we are currently not on 0.1.1 anymore, but on a specific commit, see https://github.com/ethz-adrl/control-toolbox/blob/a59278f6e5ea4f3d63aaa3c5f4d3efb60690ef65/ct/install_hpipm.sh#L17
Everything should work fine once you check out the correct commit.

Yeah I just figured that out, thx. I installed the hpipm-git package on ArchLinux instead of the hpipm package (which is at 0.1.1) and it built just fine. Thx!


[#797] Update linear interpolation of percentiles

Pull Request Checklist:

[x] This PR addresses an already opened issue (for bug fixes / features)

This PR fixes #797

[x] Tests for the changes have been added (for bug fixes / features)
[x] Documentation has been added / updated (for bug fixes / features)
[x] HISTORY.rst has been updated (with summary of main changes)
[ ] bumpversion (major / minor / patch) has been called on this branch
[ ] Tags have been pushed (git push --tags)
[ ] The relevant author information has been added to .zenodo.json

What kind of change does this PR introduce?
Update _calc_perc and percentile_doy functions to allow more linear interpolation methods.
Does this PR introduce a breaking change?
I'm not sure how impactful it is but _calc_perc now return a numpy masked array.

A remark about the failing tests here.
I think it is quite counter productive to have two places where the same things get tested.
For example, the test from test_indices.TestTGXN10p.test_tg10p_simple and  the one from test_temperature.TestT10p.test_tg10p_simple are both testing that tg10p is properly computed.
IMO, the test of atmo.tg10p should only ensure that the Temp object is properly created with the expected properties.

@bzah good point! Indicators need to be tested, but there's no point to repeat the same tests. Recently, I think we went with this guideline:

Indice tests (test_indices.py) use idealized data and really test the mathematical computation. (like edgecases, presence or not of missing data).
Indicator tests (test_{var kind}.py use "real" data (usually atmos_ds, the ERA5 extraction) and test minimally for values, but add tests for attributes and stuff like that.

I'm getting really good results with my implementation, I still have to write some unit tests, some documentation and clean up a bit but the PR will be reviewable soon.

I ended up completely rewriting numpy functions ::percentile and ::nanpercentile
It would be really appreciated if you can give me your insights on this implementation.
However I will also try to submit it to numpy because:

both function show better performances, especially for nanpercentiles which is up to x50 in some cases.
they are, IMO, much more readable than current implementations.
they add some error correction using floating point epsilon (based on the R implementation)

So I understand if you prefer to wait for the PR on numpy before commenting.
I also wrote a script to compare performance an assert the non regression (up to 14th decimal):
https://gist.github.com/bzah/2a84d050b8a1aed1b40a2ed1526e1f12
In the gist comment you can find a report of some test runs.

@huard

calc_percentiles loops over _quantile for each percentile. Is there any way to avoid that ? Once arr is sorted, I would think finding other percentiles is quick. I'm hopeful the slow-down you measure can be reduced by passing the array of percentiles to _quantile. I suspect numpy folks will make the same comment.

I was expecting discussion on this .
Yes, I agree with you it would improve the performances when there are multiple quantiles, and it's more numpy style anyway. However, I find it much easier to manipulate a scalar to understand the code and the performance are not even that bad even with 100 quantiles.
I will try to see how I can make it work with a np.array.
About the doc, once in numpy the API will probably be slightly different.
Because I'm lazy, I wanted to write docs only on numpy but I can start it here and reuse it I suppose.

@huard  I did something that work but I'm pretty sure it is not optimal.
I tested with a smallish array (100, 180, 30), iterated 10 times and on 1000 percentiles (from 0.1 to 99.9, step 0.1).
I have these results:
nanpercentile: 19.904847238 secs
NEW nan_calc_percentiles: 22.494946018 secs
OLD nan_calc_percentiles: 50.227164944 secs

I suppose I'm not doing it properly, it should perform better than nanpercentile.
I'm open to suggestion if someone has a better idea.
I'm coming back Tuesday, I will try to improve it then.

About the test failure:
The failing test is in test_adjustment.py, I'm not 100% sure but I don't think I broke it with this feature.
Moreover the test works fine locally, is there something I'm missing ?
The error isAttributeError: 'DataArray' object has no attribute 'units' on TestDQM::test_mon_U[True--pr]  *
A part from this error, I think the feature is ready for review.

The failing test is in test_adjustment.py

Not your fault! This test shows a seemingly random bug in dask...

FYI I created the draft PR on numpy: https://github.com/numpy/numpy/pull/19857
I did quite a few changes to fit the needs of numpy API, I don't believe they are necessary in Xclim.

I will remove the fuzz part, it may causes license issues because it is inspired by R and climdex which have GPL licenses.

I did quite a lot of changes in the numpy PR which greatly simplify the code of _quantile.
However, the performance improvement has been dropped because it was not part of the initial issue in numpy.
Thus, the xclim version performs much better but:

it does not handle some cases such as:

when dtype is Fraction (object) or bool
using out parameter to mutate an existing array
using keepdims to reshape the result to the input dimensions
using  overwrite_input to qvoid copying the input array
having other interpolation options

It is very hard to read (thus maintain) _nan_quantile because I did the refactoring only on numpy.

My main issue is that I don't know if the performance improvement will be merged or not on numpy.
And I think the boost is too good to be left behind, especially when I think about bootstrapping which call percentile_doy very often.
So, ideally, I would like keep the xclim implementation and refactor it, but to do it only once I'm done with numpy's version to avoid going back and forth between the two.
Is that okay for you ?

@bzah Is this ready to be merged ?

I have not yet done the refactoring because I'm waiting for a final review on the numpy's PR.
If you would like to include this asap, it can be merged as is and I will refactor in a future PR.
The behavior will not change.

I suggest to merge now, and open a new refactoring issue later in the future.


createServerRenderContext named export doesn't work

Version
4.0.0
This works:
import createServerRenderContext from 'react-router/createServerRenderContext';
This doesn't work:
import { createServerRenderContext } from 'react-router';

Already fixed in #3820. Please check existing issues before submitting new ones.


GPMDP randomly skip 2 songs when I only press next once

OS:
Windows 10 pro 64 bits | i5 6600k | 16go ram | keyboard logitech G910
Issue Descriptions:
When I press the multimedia "next" key it skip/jump 2 songs instead of 1, like the command it interpreted 2 times.
Doesn't have any problems with other music players (groove music and media player classic)
Steps to Reproduce:
it's random but happends very often

I have the same problem however it isn't random. It happens every time.

Same Issue Here;  I would like to add that this is when using the media keys on keyboard.  Moving songs within the App with the mouse does not skip songs.
Windows 10 / i7 7700k / 32gb RAM / Logitech G710

See https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/issues/2107


[REQ] F-Droid Release

Describe the problem that needs solving
Can you add Quacker to F-Droid?
Describe the solution you'd like
This is the inclusion policy that must be met to be able to submit. More info about submitting can be found here.
Here is the issue tracker to request your package to be available on F-Droid. After submitting, they can assist on what is needed to be done (if anything) to get it on F-Droid.

I have started the process of getting the app approved into F-Droid. Check it out here!

The app is approved into the IzzyOnDroid repo!
It should be up at 18:00 UTC

Sorry to necro this thread. Just saw the release on IzzyOnDroid, it's much appreciated.
However, the anti-features section lists "Tracks or reports your activity" as relevant and I am curious to know what kind of information exactly is being tracked/reported to where and for what purpose? Fritter doesn't have this anti-feature listed.

@vyeol https://github.com/TheHCJ/Quacker/discussions/41


Reject positional args to init

Codecov Report

Merging #92 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #92   +/-   ##
=======================================
  Coverage   96.45%   96.45%
=======================================
  Files           2        2
  Lines         282      282
=======================================
  Hits          272      272
  Misses         10       10

Impacted Files
Coverage Δ

loggo/loggo.py
96.42% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 78fc4ec...e5ca22a. Read the comment docs.

no problem by me, @gcarq to merge?


Upgrade Avro to 1.9.2 release

This PR upgrades Avro C to the latest release.

@spektom thanks for the PR!


[WIP] dbus-cxx: remove virtclass-native as outdated

The virtclass-native override has deprecated since 2012 and support has been removed since yocto 2.6.
On one side it could be better to replace it with the new override class-native.
But taking into account that we were fine with the absence of this dependency, I decided to remove it as not needed.

Incorrect PR.


Private registry 2.1.1 pull fail with MANIFEST_UNKNOWN Error

I'm experiencing a MANIFEST_UNKNOWN error when trying to pull an image when running registry 2.1.1. I'm able to successfully pull the image using 2.0.1, but when running 2.1.1, I get the following:
$ docker pull <REDACTED>/hubot
Pulling repository <REDACTED>/hubot
Error: image hubot:latest not found

Here are all the logs (daemon and registry) for both pulls:
https://gist.github.com/davelesser/d7a759459e88322b5690
Here are the specifics of our setup:

Running in AWS ECS with S3 as the storage backend
The ELB receives traffic on port 80 and forwards it to port 6000 on the host
We are not overriding the default ENTRYPOINT or CMD
We are using the default /etc/docker/registry/config.yml, but are overriding with environment variables
The environment variables passed to the container are in the gist linked to above

Please let me know what other information might be helpful to you and I'm happy to provide it.

Thanks for the thorough details @davelesser
We'll look into it.

@davelesser : is this still an issue for you?  I've been trying to clear things up in #892

Given the issues we ran into using 2.1.1, we rolled back to 2.0.1 and have been there ever since. I just stood up some new infrastructure to test 2.1.1 and pointed it at the same S3 bucket and used the exact same configuration as our 2.0.1 registry. So, while there are two URLs for the repos, they're using the same underlying data.
Results: A quick spot check of 2.1.1 shows that while it doesn't error out on all pulls, it still has problems with some tags.

All data was pushed with 2.0.1
All data can be pulled with 2.0.1.
Some pulls fail with 2.1.1.

Are there specific repositories that cannot be pulled or specific tags within a repository (or both?)

Items 1 through 3 are correct.
I haven't come across any repositories which, as a whole, cannot be pulled...just specific tags.

On Oct 5, 2015, at 10:27 AM, Richard Scothern<EMAIL_ADDRESS>wrote:
Thanks for digging back into this @davelesser
Just so I understand this.

All data was pushed with 2.0.1
All data can be pulled with 2.0.1.
Some pulls fail with 2.1.1.

Are there specific repositories that cannot be pulled or specific tags within a repository (or both?)
—
Reply to this email directly or view it on GitHub.

thanks for the info @davelesser .  This is a bit ask, but could you inspect the file layout for the problematic repositories?  If it's on a posix file system tree or find $root -ls would do the trick

@RichardScothern It's backed by S3, so I ran a quick script to list out the keys for the problematic repository:
https://gist.github.com/davelesser/fa11565c964307359c3d
Not quite as easy to read as the output from tree, but it shows the structure and was the simplest way to get the info you requested. If you need more info, or if I screwed something up, please let me know and I'll get you whatever you need..

Thanks for the files @davelesser .  Without registry logs and the specific manifest/tag pairing I'm not sure how to debug.  I cannot reproduce this myself.

@davelesser any update on this?

@davelesser closing for book-keeping. Let us know if you still experience problems.


Added missing currencies to oracle prices tables (breaking balances_usd test)

I couldn't refresh the balances table in dev.. timed out after 1,100 seconds. But I tested it in snowflake and it the change worked. Should fix the test failing issue.


[Bank Feeds] createFeedConnections returns 202 (201 in OpenAPI desc)

Creating a single bank feed connection returns the pending result as status code 202 Accepted not the 201 Created as defined in the OpenAPI description.
https://github.com/XeroAPI/Xero-OpenAPI/blob/master/xero_bankfeeds.yaml#L113
I ave not tried this creating multiple bank feed connections in one request, so there may be other response status codes at play which I am not seeing in other situations, so I can't say a 201 is never returned. Issue #378 hints that it may be a 200 with mixed results, but I can only guess.
Expected Behavior

The pending bank feed connection response should be delivered as a HTTP 201 according to the OpenAPI description, indicating it has been accepted but not yet processed. This is what the generated code expects. If the request contains a number of requests, then the response could contain a number of different resues - pending, created and errors - and I have no idea what the HTTP status code would be in that case.
Current Behavior

The correct PENDING response is returned, with the arguably correct 202 Accepted response.
Possible Solution

The OpenAPI description is fixed to show the payload for a 202 response.
Maybe remove the 201 response from the OpenAPI description if it is never returned.
I have a hunch there are other response status codes in play for responses containing a mix of results. I could discover them by experiment, but that doesn't tell me what the responses were planned to be.

Steps to Reproduce (for bugs)

Make POST request to createFeedConnections operation for a single bank feed connection, for a connection that should be valid.
Observe the result: 202 with the FeedConnections schema payload, but without the pagination property.

Context

Your Environment

Version used:
Browser Name and version:
Operating System and version (desktop or mobile):
Link to your project:

Resolved https://github.com/XeroAPI/Xero-OpenAPI/releases/tag/2.10.1


why no dump/restore munu

System information:

ubuntu
6.3.5

Connection specification:

dbeaver-ce

Describe your question:
my database->tool->(there no dump/restore data menu),
in the morning, it was there

hi @dreamlu ,
what is your database?

now, everyone is like the

hi @dreamlu ,
what is your database?

hi @dreamlu ,
what is your database?

mysql

Probably, you could try to reset UI settings using File - Reset UI settings

It's gone for me as well. Tried "Reset UI settings", didn't work. Also version 6.3.5, but on Windows for MariaDB.

reset ui no effect, uninstall and install new dbeaver also no effect

Furthermore, updating the driver/local client/mysqldump has no effect either.

I've been reinstalling old versions and found out that the tools went missing in version 6.3.3 , so it's not a 6.3.5 issue. I'm now using 6.3.2 which still has the dump/restore/execute script/open dashboard tools menu.

I'm now use 6.3.2 too


Attribute Fixes

Changes

Fix language detection when attribute has multiple arguments.
Fix attributes on alias

Related

This is a follow-up to #177
Fixes #158
Fixes #176

Converting to draft while I verify some other things locally.

Ready for review.

Thanks! Could you add a test in test/corpus/statements.txt?

@tgross35 good call. Tests have been added.

Many thanks!


Add Preview Terms notice

Add text to link to the preview terms for new Geocoding API
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[X] Other, please clarify - update description for documentation

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific langauge SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[X] No refresh required for updates in this PR

Contribution checklist:

[X] I commit to follow the Breaking Change Policy of "no breaking changes"
[X] I have reviewed the documentation for the workflow.
[X] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

have moved the change to the https://github.com/Azure/azure-rest-api-specs/pull/16943


Encryption Stuff

Is this a feature OR bug:
https://trello.com/c/a10DqwHK
Additional comments
Simplify, with a couple commands, the process found here:
https://blogs.msdn.microsoft.com/mvpawardprogram/2014/06/02/sql-server-2014-backup-encryption/
But for all versions. So use SMO: https://msdn.microsoft.com/en-us/library/ms162206.aspx
encryptionstuff.txt

Some of this done, but we still need a way to restore encrypted backups easily / cc @Stuart-Moore

I would also like to request adding backup encryption, at the very least being able to backup and encrypt a database.


Port show_documents and list_views to Elixir

Overview
This PR ports show_documents,js and list_views.js to the Elixir test suite

Testing recommendations
make elixir

Related Issues or Pull Requests
N/A

Checklist

[X] Code is written and works correctly
[X] Changes are covered by tests
[ ] Any new configurable parameters are documented in rel/overlay/etc/default.ini
[ ] A PR for documentation changes has been made in https://github.com/apache/couchdb-documentation

Thanks!


keyboard shortcut for Save prompt

Can we have keyboard shortcuts for Save prompt on Mac - like  Cmd+S for Save, Cmd+N for Don't Save and Cmd+C for cancel?
Also if the automatic focus can go to prompt and one can use  to navigate inside the prompt would be lovley.
Right now, one has to remove his hand from keyboard and move to mouse interact with the prompt.

I'd expect Esc to cancel, Enter to Save, and Tab followed by Enter to be the equivalent of clicking "Don't Save".

I'd expect Esc to cancel, Enter to Save, and Tab followed by Enter to be the equivalent of clicking "Don't Save".

Thanks. I was able to Esc to cancel, Enter to Save -- however Tab doesn't work - so Tab + Enter would effectively becomes Enter i.e. Save and would again open up Command Palette for putting in FileName.

A button highlight and underscore suggestion might make it intuitive for users.

I should get payed by Apple for the macOS support I have given so far...


SystemD Service

Not an issue, I want to know if people want this committed to the repo, it does not stop the process correctly however it does work.
I use the run script as I have updated the script to include a git pull to ensure that I am using the latest of the dev branch.
run.sh
[Unit]
Description=Pokemon Bot
After=network.target

[Service]
Type=Forking
WorkingDirectory=%PATH-TO-PokemonGo-Bot%
ExecStart=/bin/bash ./run.sh
Restart=on-abort

[Install]
WantedBy=multi-user.target

Python
[Unit]
Description=Pokemon Bot
After=network.target

[Service]
Type=Forking
WorkingDirectory=%PATH-TO-PokemonGo-Bot%
ExecStart=/usr/bin/python pokecli.py -cf ./configs/config.json
Restart=on-abort

[Install]
WantedBy=multi-user.target

It's been awhile since I tinkered with systemd but isn't there an option to kill processes on crash? Either that or on start/restart kill any python processes (dirty but does the job). Either way I would create a pull request and have your enhancement added and any niggling issues dealt with there

Please move your feature request/s to our feature hub - http://feathub.com/PokemonGoF/PokemonGo-Bot

https://github.com/PokemonGoF/PokemonGo-Bot/issues/3843#issuecomment-239708680

@Trozz Are you still keen to create a PR for this?

@Trozz @k4n30 We kind of have this (at least a notice). Can we close?

@mjmadsen This isn't really implemented, but I think there are sufficient scripts to probably achieve what it's functionally trying to achieve. Happy to close as OP hasn't responded


GN build parity: shell should build for linux (embedded) platform

Problem
Currently, a top-level build does not produce a chip-shell executable in:
out/debug/linux_x64_gcc
out/debug/linux_x64_clang
out/debug/linux_x64_gcc_mbedtls
Proposed Solution
Add these targets to the platform=linux build.

Is this a parity issue? The only build I see for shell is examples/shell/standalone/Makefile.
(Those other configs actually used to build these, but I removed them because it looked to me like it was only supposed to build with config/standalone/ ).

This was the hunk:
diff --git a/BUILD.gn b/BUILD.gn
index a9ffe380..b68101c3 100644
--- a/BUILD.gn
+++ b/BUILD.gn
@@ -57,8 +57,6 @@ if (current_toolchain != "${dir_pw_toolchain}/dummy:dummy") {
 if (chip_build_tools) {
   deps += [

   "${chip_root}/examples/chip-tool",

   "${chip_root}/examples/shell",
   "${chip_root}/src/qrcodetool",
   "${chip_root}/src/setup_payload",
 ]

If you undo that, it will build in the remaining toolchains again.

@mspang Ok, I agree that calling this a "parity" issue may not be appropriate.
Code-wise, the shell example is valid for both standalone and linux targets.
For other embedded targets, we'll need some HAL UART abstraction which isn't ready yet.


More flexible configuration

Allows repositories that doesn't have a cpanfile to have their dependencies installed.
Replaced dash with Bash, since it's already available in the image and has more features.
Allow the optional configuration of the CPAN client, since it might be required by some distributions.
Small improvements regarding DRY.

Thanks for the PR @glasswalk3r I will have a look at it


Avoid signed comparison warning from Google Test

Will certainly merge this, but can you tell me which compiler (with version), full compiler command line (if not using the Makefile) and Google Test version?

Sure, it's from the Makefile but slightly modified:
g++ -Wall -Wextra -std=c++11 -O0 -g -Igtest-1.7.0/include -o minijson_writer_tests minijson_writer_tests.cpp gtest-1.7.0/lib/.libs/libgtest.a -pthread

Compiler is:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.4.0
Thread model: posix

I wasn't able to reproduce it with clang on Ubuntu but I merged the patch.


For Civiform admin, allow ability to toggle on and off autopopulate feature

Is your feature request related to a problem? Please describe.
When asking for "signature" or related approval fields we should not autopopulate forms for applicants, even if previously submitted for other programs.
Describe the solution you'd like
Ability for CF admin to remove auto populate feature.

Good point -- so the CiviForm admin, when building a question, can check a certain checkbox that bans auto-filling of that question.  So even if we have the applicant's data for that program, we shouldn't auto-fill it.

Work-around: create a new question that is unique to the program. Could pose issues with the reapply process though.


How to deploy on swarm

Checklist

Please read the setup instructions
Please read the FAQ

How to deploy on multiple docker swarm cluster and run with a stack deploy command.

Any updates on this @iamapinan ?

Any updates on this @iamapinan ?

No any answer from developers team.

I could use some hints for this too

Deploying any other way than the official one described is not necessarily the expertise of the developers of a software. This would be better suited for the Gitter Chat or the Reddit community (not sure what the status of that is). But if anyone from the community knows how to do this, leave a comment here, but I will close this for now.


[material-ui][Avatar] Can't override style with classes nor className

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
Link to live example: https://mui.com/material-ui/react-avatar/
Steps:

Add a className which override width and height of the Avatar (test for example)
Inspect HTML
see the classes as "MuiAvatar-root MuiAvatar-circular MuiAvatar-colorDefault test css-19biciz-MuiAvatar-root"

Current behavior 😯
The test class come before the one generated by the "styled". css-19biciz-MuiAvatar-root also declares width and height so the values defined in test are not applied. It does not matter in I put test in the className property of the classes={{root:test}}
Expected behavior 🤔
I would have expect the class given in property would by the las one in the chain so that it can override the default style
Context 🔦
I wanted to reduce the size of the avatar, from 40px to 24px
Your environment 🌎

npx @mui/envinfo
System:
    OS: macOS 12.5
  Binaries:
    Node: 18.17.0 - /usr/local/bin/node
    Yarn: 1.19.1 - /usr/local/bin/yarn
    npm: 9.6.7 - /usr/local/bin/npm
  Browsers:
    Chrome: 116.0.5845.187
    Edge: Not Found
    Safari: 16.3
  npmPackages:
    @emotion/react: 11.11.1 => 11.11.1
    @emotion/styled: 11.11.0 => 11.11.0
    @mui/base:  5.0.0-beta.16
    @mui/core-downloads-tracker:  5.14.10
    @mui/icons-material: 5.14.3 => 5.14.3
    @mui/material: 5.14.10 => 5.14.10
    @mui/private-theming:  5.14.10
    @mui/styled-engine:  5.14.10
    @mui/system:  5.14.10
    @mui/types:  7.2.4
    @mui/utils:  5.14.10
    @mui/x-date-pickers: 6.11.1 => 6.11.1
    @types/react: 18.2.19 => 18.2.19
    react: 18.2.0 => 18.2.0
    react-dom: 18.2.0 => 18.2.0
    typescript: 5.1.6 => 5.1.6

Hey @drichard-nexapp, thanks for the report!
Could you provide a minimal reproduction test case with the latest version. This would help a lot 👷. A live example would be perfect. This codesandbox.io template may be a good starting point. Thank you!

Is it possible to re-open this isse?  I am encountering the same problem https://stackoverflow.com/questions/78980184/react-mui-avatar-div-height-defaults-to-static-value-40px

@blackfrancis did this answer solve the issue for you?


Kernel Features Tracker

I've read the source code and I was disappointed not to find any usage of kernel's primitive to watch for filesystem changed (except for Darwin).
Why don't you use kernel primitives (like inotify on linux, for example)?
IMHO, it'll be a lot more efficient than computing a giant hash table of path to monitor.

It seems it is partially answered in this issue

For inotify, that's events potentially disappearing or being duplicated...

Using inotify is something I've drifted back and forth on.
Here is a brief from the author of the LLFIO (this is from a conversation on Slack):
Also Linux's kernel API is racy and broken (both APIs), the Windows one has many gotchas, the only decent kernel API is BSD's. The BSD kernel API is accurate and performant, but if it gets overloaded due to too much filesystem changes it may report back "I give up" and then you need to fall back on a manual delta calculation.

...

@ned14 in what ways are the Linux kernel APIs broken (I’m assuming the inotify API, right?) — and are there solutions or workarounds?

They drop notifications, report the wrong changes, sometimes don't work at all. And appear to spasm between half working and not working in a bursty way if there is enough change happening on the filesystem. Windows and BSD at least have the decency to explicitly tell you "I give up".

I (am hoping to) disagree with some points there. Specifically, I'm looking for a balance between efficiency and safety, and to illustrate those caveats in the readme.

But, yes, you're also right that manually computing deltas (with the already slightly sub-optimal unordered_map isn't great.
There's somewhere between between the two. It requires more research into the broken parts of inotify.

Solution: Bring inotify up. Write filesystem-event heavy tests that illustrate a difference in output between an inotify and the "baseline" warthog adapter. If the results are bad enough, revert. In any other case, make a section in the readme about the caveats.

Personally, I'd make a delta generation implementation which works off scanning the filesystem and comparing graphs. I'd make that fast, which is doable (certainly 10 million items per second is doable). I'd then have inotify trigger the delta calculation, but also have a fallback timer so it always runs every few seconds anyway.
You might, if you wanted, only compare the subset of the graph which inotify tells you about, but then say every ten seconds do a full scan anyway.
inotify has issues on NFS and CIFS. And god only knows with something like Lustre. Certainly some heuristic which detects when inotify isn't telling you about notifications and restores regular polling should be easy enough to implement.
Good luck with it, making this fast, reliable and race free is challenging even for a domain expert.

Preliminary results with an inotify-only adapter are not pretty. This is a snippet taken while running it with tool/test/dir:
`tell bun .` & ; sleep 5

tool/test/dir

"1666145122927923891":{"where":"122","what":"destroy","kind":"dir"},
"1666145122927966274":{"where":"
","what":"destroy","kind":"file"},
"1666145122927971681":{"where":"161","what":"destroy","kind":"dir"},
"1666145122928012833":{"where":"303","what":"destroy","kind":"file"},
"1666145122928016415":{"where":"303","what":"destroy","kind":"dir"},
"1666145122928058140":{"where":"336","what":"destroy","kind":"file"},
"1666145122928061756":{"where":"336","what":"destroy","kind":"dir"},
"1666145122928105673":{"where":"107","what":"destroy","kind":"file"},
"1666145122928109207":{"where":"107","what":"destroy","kind":"dir"},
"1666145122928154216":{"where":"464","what":"destroy","kind":"file"},
"1666145122928157753":{"where":"464","what":"destroy","kind":"dir"},
"1666145122928203024":{"where":"","what":"destroy","kind":"file"},

That's a lot of misreported and missing errors. It's not sequential, some events are bogus, and it missed all of the create events. inotify also seems to hang after some (large) number of directories:
`tell bun $HOME` & ; sleep 5

-- Configuring done
-- Generating done
-- Build files have been written to: /home/edant/smstore/watcher/build/out
[2/2] Linking CXX executable water.watcher
{"water.watcher.stream":{

top

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 886594 edant     20   0    8508   5636   3376 R  99.7   0.0   1:50.88 water.watcher

This is, somehow, worse than I remember inotify being the last time I wrote up a program like this. The preliminary results are a bit too bad.
I'll look into something I might be missing and create a branch for this after I've ruled out programmer error.
Either way, I'll be working towards a solution along the lines of Ned's answer.

Changed the title to better reflect the ongoing work

Using fanotify is told to be lighter in resources and less racy with directory creation.
The "errors" you are reporting here seems to be well documented: inotify is limited by the number of item it can observe. So I guess it's hanging because you're not checking for errors when crossing this limit (and IIRC, it's very low, something like 16K)
As for missing creation event, it's by design I would say even if it's an awful design (fanotify doesn't have this issue, when monitoring mount or filesystem). So inotify can't be used without mirroring the file hierarchy in your software so it's very hungry in memory.
fanotify on the other end, requires linux kernel > 5.1 to be useful, doesn't need to replicate the file hierarchy in memory (it can be re-generated on the fly with the received events).

I had toyed with fanotify but could not get enough useful information in userspace. Perhaps I had set some of the wrong flags. I will revisit fanotify.
Perhaps we can check the user's id and switch to fanotify in that case. It's a wonderful API. I had a lot of fun with it as su.
For now, though, we're using inotify on Linux. It's a tricky API with a lot of caveats. I've managed to work through most of them, including the issues I posted above.

In the future, I'm looking to see where this implementation's shortcomings are and bolster them with sanity checks.
The same would need to happen with fanotify, albeit with less paranoia. (The docs on inotify say as much.)
A "parallel" warthog could run every little while and check up on the state of the file tree. I think the important problem to think about is how to get it to run in piecemeal while we're waiting for events. I don't want the program to stutter when we should be handling events.

Only platform API left to do is Windows. I'm looking into ways to select fanotify if the user is a superuser (id 0).
Rarely, but it does happen, I've found some misreported event types on Darwin and some missing events on Linux.
I think the best way to build up a heuristic is to have a tree of "open" events (such as files being open, but not written, or locks being opened, but not closed). We could try to balance the tree by manually polling the "open" events every little while.
On Darwin, we could ignore the fsevents API's file type reporting and opt in to std::filesystem. I'm tempted to think that's lightweight enough to use for many, many path. It's likely just a stat under the hood. But we should test it.
(We need performance tests.)

The next branch has system API calls for Windows. (It also has support for concurrent watchers.)
Docs are added to the readme in that branch as well (section "OS APIs").
I want to implement an fanotify adapter on Linux to select when the user is root. (Right now, we always use inotify.)
I may close this issue after fanotify.

Done :)


Enhancement - Support Enum Type

Currently, compiler cannot parse table with custom enum type
Eg.
CREATE TYPE "schema--notification"."ContentType" AS
 ENUM ('TEXT','MARKDOWN','HTML');
CREATE TABLE "schema--notification"."notification" (
    content_type "schema--notification"."ContentType"
);

Expected result
from enum import Enum
from gino import Gino

db = Gino()

class ContentType(Enum):
    HTML = "HTML"
    MARKDOWN = "MARKDOWN"
    TEXT = "TEXT"

class Notification(db.Model):
    __tablename__ = 'notification'
    __table_args__ = dict(schema="schema--notification")

    content_type =db.Column(db.Enum(ContentType))
)

@hieulw , hello ) thanks for interest to the project, yes, I need to add construction " CREATE TYPE "schema--notification"."ContentType" AS
ENUM " to https://github.com/xnuinside/simple-ddl-parser first. I will take it in work.

@hieulw, hi! I open WIP pr and released 0.5.0-a0 for tests https://github.com/xnuinside/omymodels/pull/13 it's contains Enums (see examples in test files) & uuid type. About Schema - one of the requirement for initial purposes of development this library was parse schema and put it to Gino() as argument. Because, when you have a 35 tables for example, and use schema for versioning, so you use 1 schema for all tables in same moments of time - it a lot of unnecessary code to put each schema separate in table_args. I think I will add config that allow set up such things, something like use_schema='per_table|globally'.

@hieulw  hello! I released 0.5.0 version with all changes, example in tests https://github.com/xnuinside/omymodels/blob/main/tests/test_models_simple.py#L117

@xnuinside Great! It helps a lot. Now I'm trying to generate schema ddl from database (since pg_dump doesn't fit my need) and your project converts it to Gino model help my project development a lot faster.

@hieulw, can I close the issue?

@xnuinside Yes, of course!


Wallet Synchronisation incomplete

Since days the Wallet doesn't sync properrly.It stops @ 97%  on Windows 10 64bit pro

Ver.1.3.3

I am also experiencing this on Windows 10 with v1.3.3
It worked fine for a while, but at some point I closed it and reopened, and now it won't download past block 437792 even though it says that data is being transferred and there are multiple connections to the network.
Sorry this is just a +1 essentially but hoping for an answer as well because I can't stake or move my coins right now


Question about spectrogram normalization

Since we are using a variety of pretrained vocoders, we need to be able to generate outputs with different spectrogram normalizations.  I notice that spectrogram normalization in
https://github.com/NVIDIA/radtts/blob/main/data.py#L270
seems somewhat arbitrary.  What was the reason for this choice?

Great catch. That was actually a very rough normalization scheme to achieve roughly standarized inputs. You'll definitely be better off performing this step more diligently (with per-channel normalization parameters)  and keeping track of different standardization parameters for each vocoder setting you want to incorporate into your data.


Devise Omniauth and DeviseTokenAuth Omniauth

Is it possible use two way of authentication (devise and devise_token_auth)?
I'm trying but the devise omniauth uses same api path and the redirection (/omniauth/:provider/callback), and after redirect goes to api path with error (Could not authenticate you from Facebook because "Csrf detected".).
My route configuration:
devise_for :users
  namespace :api do
    scope :v1 do
      mount_devise_token_auth_for 'User', at: 'auth'
  end

In separated apps both works fine.

Try this on your application_controller

protect_from_forgery with: :null_session, if: Proc.new { |c| c.request.format.json? }

I've created a new application to test it.
I need now create two way of authentication.

Sign In/Up with Facebook and Devise (Rails)
Sign In/Up with Facebook and Devise Token Auth (Ionic App)

The step 1 works as well, but the step 2 uses same callback fase, and redirect to root_path like the step 1.
How can I configure different callback to Devise Token Auth?
Now my route is:
  devise_for :users, controllers: {omniauth_callbacks: 'callbacks'}
  namespace :api do
    scope :v1 do
      mount_devise_token_auth_for 'User', at: 'auth'
      as :user do
        resources :index
      end
    end
  end
  resources :home

I started new blank application to try solve it.
I configured devise and now I have problem with devise_token_auth.
In callback fase, both return pass thought the devise callback. I don't know how can I separate it or to identify who start the authentication.

Any progress with this?

Any progress in this? :)

I also need this!

Hi there @dtelaroli    ,
In an effort to cleanup this project and prioritize a bit, we're marking issues that haven't had any activity in a while with a "close-in-7-days" label. If we don't hear from you in about a week, we'll be closing this issue. Obviously feel free to re-open it at any time if it's the right time or this was done in error!
If you are still having the issue (especially if it's a bug report) please refer to our new Issue Template to provide some more details to help us solve it.
Hope all is well.

I did'nt saw this before, but, I give up this plugin because it has a lot of bugs and incompatibilities for me with no help.
Thanks.


Laravel 9.38.0 Shift

This pull request includes updates for the recent minor version release of Laravel as well as updates to your package dependencies. Feel free to commit any additional changes to the shift-ci-v9.38.0 branch.
Before merging, you need to:

Checkout the shift-ci-v9.38.0 branch
Review all pull request comments for additional changes
Run composer update
Thoroughly test your application (no tests?, no CI?)

:warning: The Shift build servers run PHP 8.1. If you are running a different version of PHP you should run composer update locally to ensure you have the proper versions of your dependencies to reflect your PHP environment.

:information_source: Shift detected your application only has the default tests. You may quickly start adding tests by using the Test Generator to automatically create the model factories, test classes, and sample tests cases for your Laravel application.


[HTML/CSS]Drop-down menu style in AdapativeCards is not configurable enough

Platform

.NET HTML
botframework-directlinejs

Version of SDK
"Microsoft.Bot.Builder.Integration.AspNet.Core" Version="4.8.0"
"botframework-directlinejs": Version="0.11.6",
Summary
The select/option method used by directlinejs to represent the drop-down items of AdaptiveCards is not the best choice for styling using CSS. One cannot change the hover style of items uniformly for all web browsers. (option:hover has no effect in most Web browsers)
Could you please use ul/li, for example, instead?
We are frustrated as we must change the colors (hover style, which is currently a blue background and white text color) to make them consistent with our company color scheme, but there seems to be no way to that.
Details
We create adaptive cards containing drop-down menus in our .net core back end:
{
    "type": "Container",
    "items": [
        {
            "type": "Input.ChoiceSet",
            "id": "CountryCode",
            "value": "DE",
            "choices": [
                { "title": "Austria", "value": "AT" },
                { "title": "Belgium", "value": "BE" },
                { "title": "Bulgaria", "value": "BG" },
                ...
            ],
            "separator": true,
            "wrap": true
        }
    ]
}

Our Angular 8 based UI establishes a connection using directline and creates a chat window.The
dropdown items are representted by the directlinejs as select/option:
<select class="ac-input ac-multichoiceInput ac-choiceSetInput-compact" style="width: 100%; min-width: 0px;">
<option disabled="" hidden="" value=""></option>
<option value="AT" aria-label="Austria">Austria</option>
<option value="BE" aria-label="Belgium">Belgium</option>
...
</select>

It shows a blue background on the items as we hover the mouse on them. We have to change the blue background as well as the text color to be consistent with our company color scheme.
There seems to be no way to control the option:hover style using CSS. for example,
.ac-choiceSetInput-compact option:hover {
  background-color: red;
  color: yellow;
}

has no effect in Chrome or Firefox.

@sazarghani - Thank you for the feedback. If you would like to raise an issue with DirectLineJS, the DirectLineJS repo is here: https://github.com/microsoft/BotFramework-DirectLineJS
This doesn't appear to have anything to do with DirectLineJS, however. DirectLineJS doesn't know anything about Adaptive Cards so it's certainly not in charge of rendering them. Web Chat does know about Adaptive Cards, but it delegates the actual rendering to the Adaptive Cards package. You had already posted this issue in the Adaptive Cards repo, so it's very unclear why you thought this issue was more appropriate for Web Chat.
I want to clear up some possible confusion. When you author an Adaptive Card, you do not add drop-down menus to it because a choice set is not a drop-down menu. A choice set is an abstraction that a renderer can choose to render however it likes. It is up to the renderer whether it wants to represent that choice set as a drop-down menu or something else. See my blog post for more information: https://blog.botframework.com/2019/07/02/using-adaptive-cards-with-the-microsoft-bot-framework/
Since Web Chat does indeed render choice sets as drop-down menus, I'll say that it looks like styling the hover events for select/option elements is very tricky. There are Stack Overflow posts that offer some advice:

https://stackoverflow.com/questions/2402146/html-select-selected-option-background-color-css-style
https://stackoverflow.com/questions/10484053/change-select-list-option-background-colour-on-hover

The easiest thing for you to do would be to set the style to "expanded" so that the choice set would be rendered as option buttons instead of a drop-down menu. This might be easier for you to style.
A more difficult solution would be to write your own custom element and include that in the card: https://docs.microsoft.com/en-us/adaptive-cards/sdk/rendering-cards/javascript/extensibility
Since this is largely outside the scope of the Web Chat repo, I'm closing this issue now.


Start error on windows 10

Windows_NT 10.0.10586 npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "start" npm ERR! node v6.1.0 npm ERR! npm  v3.8.6 npm ERR! code ELIFECYCLE npm ERR<EMAIL_ADDRESS>start: electron .`
npm ERR! Exit status<PHONE_NUMBER>
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script 'electron .'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the electron-quick-start package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     electron .
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs electron-quick-start
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls electron-quick-start
npm ERR! There is likely additional logging output above.
I have tried ever thing that I cloud think of: installing electron-prebuilt locally/globally updating Nodejs, and it doesn't seems to work.

Solved it!
vcruntime140.dll and msvcp140.dll were missing for windows you can download them from here
https://www.microsoft.com/en-us/download/details.aspx?id=48145 that should solve the problem.

#55

This should be fixed in Electron v1.1.1 which we've just updated this repo to use 👍