Some ideas for the photostacking feature

Hello Frank, thank you for allowing me to experience this wonderful project.
First of all, please understand that I am not familiar with the documentation format on GitHub.
A7R4, photo stacking The number of times the big button was pressed was the same as the number of shots.
I thought that if you press it once, it will shoot continuously even if you release your hand after that, and then press it again to stop shooting, but that didn't work.
(I don't know if the toggle hold function has already been implemented, but it doesn't work on the A7R4.)
(The same is true for the 9-focus function. While the big button is held down, the shooting proceeds by moving through the 9 zones in turn. If the button is accidentally missed in the middle, it goes back to the beginning and starts shooting from the center zone.)
I would like to suggest the toggle hold function of the Big Button and the setting of the total number of stacking shots as ideas.
(It would be nice if the shooting interval could be set as short as possible, but I think it will be difficult because there is a physical difference in the focus motor for each lens.)

A few things I just added (because of your previous suggestions submitted to my blog):

the spacing for the 9-point mode can now be adjusted right from the app by rotating the remote
the focus stepping pause time now takes the shutter speed into account, it will wait until the shutter has opened and closed
I added button toggle mode, if you tried to do focus-stacking/9-point/shutter-stepping while holding the remote upside-down, then the big button is in toggle mode, instead of hold-down mode

I don't want the button-toggle mode to be a default, and I don't really want it to be a run-time-configurable or compile-time-configurable option, so I feel like this is a good compromise.
From a design point of view, the apps that I added button-toggle-mode to are not ones that should be "unattended", hence why I would not implement this as the default behavior.
Please do a full update, including data-upload (there are new image files). Try it out and let me know what you think.

Frank, Thank you for reviewing my idea.
- Photostacking test with A7R4
https://youtu.be/HRqdWYNYITU

The toggle function of the big button is perfectly performed.
Sometimes the picture is taken while the focus is not completely shifted.
(I was testing at relatively fast shutter speeds, so I wasn't sure if this would affect the photo results.
I'll see if that has any effect on slow shutters later.)
- 9 point test with A7R4
https://youtu.be/pZPldXbgRNE
The toggle function worked perfectly.
It seems to work at a relatively slow speed, but it worked just fine.
If you look at the 2 minute 30 second section of the test video, if the camera loses focus, it has not been able to shoot normally since then.
(An idea came to mind for a moment here. What if a function that shoots like a clockwork is added while gradually narrowing the range toward the center when it is set to a wide section?)
- Shutter step test with A7R4
https://youtu.be/nSeLXqRe8zI
In S mode, I pressed the shutter button halfway to confirm autofocusing, and then started shooting.

The toggle function worked perfectly.
After the shutter speed was changed by 1 step, it did not change any more.
After 35 seconds of the test video, the AF mode could not be activated because the focus mode was locked to MF after the work was completed. This issue was only resolved after powering off the M5Stack.

please try out v1.0.12 , I added a new wait state that waits until the camera completely saves the image from buffer to SD card, which makes the shutter speed change much more reliable

I just installed v1.0.12 and tested it. There is a new counter at the bottom of the shutter step menu, and the problem mentioned in the previous v1.0.11 report has not been improved.
The camera locks with MF until the M5StackC plus is powered off was also the same.
I found something new while testing the slow shutter in the focusstacking menu.

The focus did not shift at regular intervals.
The higher the aperture of the lens, the more it shifts the focus.

For an accurate test, I tested with a telephoto lens, shutter speed of 1 second, and aperture value in the range of f/4 to f/22. Very rarely, a shot was missed or two overlapped shots were taken. However, this did not affect the actual photography work.
It was tested only in the focus stacking menu, but it is expected that there will be no problems with the focus movement section in other menus.

Some testing notes with the latest code version with the A74;
The biggest problem I was having was getting the "9-point" function to even focus before taking the photos. It would move the focus point around just fine and take the photos just fine too.
Side note here; the spacing options work fine, the toggle function works great across all the functions it can be used on.
I was wracking my brain trying to figure out what was going wrong when something @my1sec said stuck out to me:
The 9-Point focus setting doesn't pull focus unless you have AF with Shutter turned on. So for me, I use back button focus as my preferred way of triggering auto-focus, which doesn't work with "9-Point", but does work with all the other focus pull methods, I'm assuming because the other modes shift into 'MF' to focus and therefore negates the problem.
Maybe worth adding this as a note under this function on the Features page?

I got a busy weekend ahead, can't update the code nightly like usual for a bit
The MF getting stuck I think I see the bug on line 161 of ShutterStep.ino , where a true needs to be false for cmd_ManualFocusMode
The instructions for "AF with Shutter" will be added to docs, thanks!
As for the MF focus stepping being not regular, I'm not very clear what to do. The code is allowed to send three different step sizes, 1, 3, or 7. 1 is useless, 3 is a bit useless but the best, and 7 is huge. From my testing, one "size 7" step is equal to about 50 "size 3" step. There is a configurable delay between sending each step command, regardless of step size. The focus stacking function can also send consecutive commands. The code looks like
            case 1:
                step_size = SONYALPHA_FOCUSSTEP_FARTHER_SMALL;
                step_cnt = 1;
                break;
            case 2:
                step_size = SONYALPHA_FOCUSSTEP_FARTHER_MEDIUM;
                step_cnt = 1;
                break;
            case 3:
                step_size = SONYALPHA_FOCUSSTEP_FARTHER_MEDIUM;
                step_cnt = 3;
                break;
            case 4:
                step_size = SONYALPHA_FOCUSSTEP_FARTHER_MEDIUM;
                step_cnt = 6;
                break;
            case 5:
                step_size = SONYALPHA_FOCUSSTEP_FARTHER_LARGE;
                step_cnt = 1;
                break;

The case number is what's displayed on the screen

Frank, I have no doubt that your project will be of great help to me and other thirsty alpha users. Every time I leave a comment here, my gratitude for you grows deeper.
Today, I tried using Alpha Fairy for real landscape photography.
The 9point function and shutter step function did not work on the A7R4, so I couldn't test it. So this time, I only tested the focus stacking function.

The interval at which the focus shifted showed irregular movements, sometimes moving rapidly or slowly. There was also occasional erratic delay in shutter timing. When the delay was too severe, I was able to solve the problem by turning the power back on.
(I guess that the difference in the physical motor for each lens and the unknown interference of the WIFI signal have a complex effect. Various delays also appear occasionally in the Sony app.)

The effect on each shot data and the final merged photo result was negligible.
(In most cases, when shooting focus stacking, the aperture value is set high, so I think that even if the focus distance is not uniformly shifted, the result will be less affected.)

Big button toggle function was perfect.
(During continuous shooting, the branches were swaying in the wind, so I had to stop the stacking shooting and start again from the first focus section. This is entirely a problem with my lack of skill.)
When using the toggle function, it would be nice to add a function to stop continuous shooting when focus is reached in the infinity section.

I thought that the focus stacking function was good enough as it is now, as the irregular focus shift problem that occurs very rarely does not interfere with actual use.

I just fixed the MF getting stuck
I'm looking at my code and the 9-point function explicitly engages AF and waits up to 5 seconds to obtain a lock
I disabled "AF with shutter" on my A1, and the AF activation stopped working. I used Wireshark to sniff this while trying the same thing with Sony's Imaging Edge Remote running on my PC. It also is not able to engage AF. This also means I can't make an effective fix, since Sony themselves can't do it.
I think this is an actual camera bug. Sony wouldn't put an AF button on the GUI if it literally does nothing.
I added a clear error message screen for the 9-point app, if the AF fails to engage, it will say that "AF w shutter is off". I also added the instructions to the github documentation.
NOTE: it does seem like the same command engages AEL though

- [ alpha-fairy ver 1.0.13 ] 9 point test with A7R4
https://youtu.be/LDKPjD07m_g
Fixed the problem of MF locking up when finishing 9point work, but the A7R4 didn't shoot.

That video is super helpful, I was wondering why the green dot for focus lock looked different from mine, mine is always the green dot with some circles around it. I realized that you are using AF-S and AF-C.
My mistake is that in AF-C mode, if I just get the green dot, it actually means the subject tracking has lost the target, so in my code it's labelled as SONYALPHA_FOCUSSTATUS_LOST, but AF-S mode uses the same value for the focus status to indicate lock has been achieved.
To fix it, I added an equivalent enum definition SONYALPHA_FOCUSSTATUS_AFS_FOCUSED and made sure the focus status takes into account whether the user is in AF-S or AF-C
Fix is in the following commit (and is available to be pulled from main branch):
https://github.com/frank26080115/alpha-fairy/commit/ff5b84bd157ffafbd7b3cd3efd70c2d69a85a851
I forgot to change branches before editing the code so there's no pull request to link, oops

- [alpha-fairy ver 1.0.13_2 ] 9 point test with A7R4
https://www.youtube.com/watch?v=7psqoeQXQxQ
A test was conducted with the newly updated code applied. The action was quickly and without stalling, and the A7R4 fired the shot. However, if you look at the video from the 14th second, it seems that something has started to stagnate, and the A7R4 has not worked since. Of course, there was no problem with MF lock and focus cursor movement.

For all of the buffering problem (images not being saved and then getting stuck), please double check your camera settings to use "save to camera only", once for PC mode and once for smartphone mode as well.

For example, 5 cuts to infinity in some cases and 3 cuts in other cases. Even when the same scene was shot with the same camera settings, there were times when the number of shots was different.

yes I know, I can't do much about this, the steps sizes seem to vary depending on both the lens model and also where the focus started from

As alpha-fairy was used for a long time, it was found that the delay in the shooting interval gradually increased as if buffering occurred, and the problem was temporarily solved by rebooting alpha-fairy.

I can't debug this, I don't know if the source of the problem is the LWIP implementation, and if it is, I don't really know how to solve it. Months ago I had an entirely different backend using AsyncTCP instead, hoping it wouldn't allow anything to buffer up and choke, it didn't help and was harder to keep stable.

When using the focus stacking function, I would like to suggest what it would be like if a function to stop shooting when reaching infinity was added.

I will probably work on this next.

- [alpha-fairy ver 1.0.13_2 ] shutter timer test with A7R4
https://www.youtube.com/watch?v=AFI1AOlZy1Y
I tested again today and discovered something new.
Saving failed in toggle mode with shutter timer set.
On the other hand, the normal mode was saved but the focus did not move.


Scoped variables cause an error when sort-order is enabled

.form {
    $foo: #fff;
    $bar: #fff;
}

Produces:
TypeError: a.match is not a function
    at sortLeftovers (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb/lib/options/sort-order.js:244:19)
    at /Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb/lib/options/sort-order.js:334:24
    at Array.sort (native)
    at module.exports.process (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb/lib/options/sort-order.js:325:16)
    at Object.Node.map (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/gonzales-pe/lib/node.js:111:9)
    at /Users/gajus/Documents/dev/gajus/pragmatist/node_modules/gonzales-pe/lib/node.js:117:22
    at Array.forEach (native)
    at Object.Node.map (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/gonzales-pe/lib/node.js:115:22)
    at /Users/gajus/Documents/dev/gajus/pragmatist/node_modules/gonzales-pe/lib/node.js:117:22
    at Array.forEach (native)
    at Object.Node.map (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/gonzales-pe/lib/node.js:115:22)
    at /Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb-core/lib/core.js:132:18
    at Array.forEach (native)
    at Object.processTree (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb-core/lib/core.js:131:19)
    at processString (/Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb-core/lib/core.js:523:22)
    at /Users/gajus/Documents/dev/gajus/pragmatist/node_modules/csscomb-core/lib/core.js:449:38

I did a bit of debugging:
This is sortLeftovers function from sort-order.js that triggers the error.
var sortLeftovers = function(a, b) {
    var prefixes = ['-webkit-', '-moz-', '-ms-', '-o-', ''];
    var prefixesRegExp = /^(-webkit-|-moz-|-ms-|-o-)(.*)$/;

    // Get property name (i.e. `color`, `-o-animation`):
    a = a.node.get(0).get(0).content;
    b = b.node.get(0).get(0).content;

    console.log(a, b);

The output of console.log is:
[ { type: 'ident', content: 'foo', start: { line: 2, column: 6 } } ]
[ { type: 'ident', content: 'bar', start: { line: 3, column: 6 } } ]

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/29843200-scoped-variables-cause-an-error-when-sort-order-is-enabled?utm_campaign=plugin&utm_content=tracker%2F214563&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F214563&utm_medium=issues&utm_source=github).

For me this isn't fixed. Still get this issue when I use variables or at-rules. :/

Windows 10, vscode insider build and stable build
csscomb 4.2.0
config:
{
	"always-semicolon": true,
	"color-case": "lower",
	"block-indent": "\t",
	"color-shorthand": false,
	"element-case": "lower",
	"eof-newline": true,
	"leading-zero": true,
	"quotes": "double",
	"sort-order-fallback": "abc",
	"space-before-colon": "",
	"space-after-colon": " ",
	"space-before-combinator": " ",
	"space-after-combinator": " ",
	"space-between-declarations": "\n",
	"space-before-opening-brace": " ",
	"space-after-opening-brace": "\n",
	"space-after-selector-delimiter": "\n",
	"space-before-selector-delimiter": "",
	"space-before-closing-brace": "\n",
	"strip-spaces": true,
	"unitless-zero": true,
	"vendor-prefix-align": true,
        "sort-order": [
                // ....
        ]
}

For me, csscomb stops working if the scss file has an @-rule or a scoped variable in it:
// doesn't work
#header {
        width: 100%;
        @include blackBg;
}

// doesn't work
#header {
        width: 100%;
        $headerHeight: 64px;
}

// does work
#header {
        width: 100%;
        height: $headerHeight;
}

// does work
#header {
        width: 100%;
        #{$bg}-color: blue
}

In the cases it doesn't work, csscomb throws me an error:
TypeError: a.match is not a function

When I remove my sort-order property from the config, everything works fine. :)

This should be fixed in the latest release now that the sortLeftovers uses
a = a.node.first().first().content;
b = b.node.first().first().content;

still getting this issue

Hi @JiniHendrix!
Can you create a small repro repo that demonstrates the issue for me to investigate.

hey @jdalton here's an example repo https://github.com/JiniHendrix/csscomb-sort-break
thanks!

Thank you @JiniHendrix!
Yep, the issue still persists. Let's fix it together!
Would you be up for creating a failing test case in a PR that I can write a patch against?

ill give it a shot 👍

Rock! The issue is that because left-overs can be a variety of things, one of those being a Variable node which has content as an array (instead of a string like the Ident node).

@jdalton looks like there already is a test for it, and it's passing. may need to refactor the tests.
test/options/sort-order/process/test.js line 413

dont really have the time for it unfortunately, seems like you know how to fix the issue though?

If you're up for the refactor that would be appreciated too : )

Ah okay, sorry. I posted my initial reply before the newer comments popped in. No worries. Thanks for pin pointing the existing test!

Update:
Patch PR started at #609.

hey @jdalton  thanks for fixing the issue. any idea when this will be published?


improvements from fork justledbetter

Adapted changes from
https://github.com/justledbetter/mail2most/commit/b8e8aac8f472af6b39da161921afed5972f09c27
great work to justledbetter
Checklist

[x] mage test passes
[ ] unit tests are included and tested
[x] documentation is added or changed

Amazing! Great JOB!

@justledbetter great work !


Changes in 0.10.0 release

Hi all, I plan to make v0.10.0 release from master, changes are listed below:

anything I missed in the HISTORY.rst?
anything else we'd want to include before release?

Rendered changes: https://github.com/scrapinghub/extruct/blob/v0.10.0-changes/HISTORY.rst (didn't manage to make the links to PRs work)

Thanks for review @Gallaecio , merging and releasing


 Exiting

HI Alan,
I tried to run el-gato on two samples, using this command
for samp in $(cat samples.txt); do
/home/myUser/el_gato/el_gato/el_gato.py --assembly ../assemblies/${samp}_assembly.fasta --out output2019_mlst
done
The run failed with this error:
Output directory 'output2019_mlst' exists and overwrite is turned off. Exiting
Any ideas?
Thanks!
TJ

Hi TJ,
As I said here el_gato wants you to provide a different output directory for each sample. It writes output files with the same filenames each run so you can't write outputs from multiple runs in the same directory
Alan

Thanks! I caught the issue by adding the sample name as subdirectory of the output dir.


Dart encryption

@wilburt  I have managed to run the encryption natively in dart. Please take a look at it.

@wilburt


fix: Failure reproduction code missing actually executed values

Fixes #814
Not yet ready - includes a hack that kinda works in the default use case. A better solution would be to use kwargs stored from the last call only when it is actually needed.
TODO:

[ ] Do not use last_used_kwargs unconditionally
[ ] Tests

Codecov Report

Merging #815 into master will decrease coverage by 0.15%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #815      +/-   ##
==========================================
- Coverage   99.08%   98.93%   -0.16%
==========================================
  Files          62       62
  Lines        4157     4124      -33
  Branches      713      715       +2
==========================================
- Hits         4119     4080      -39
- Misses         18       23       +5
- Partials       20       21       +1

Flag
Coverage Δ

#unittests
98.93% <100.00%> (-0.16%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/schemathesis/models.py
96.16% <100.00%> (-0.05%)
:arrow_down:

src/schemathesis/utils.py
98.16% <0.00%> (-1.84%)
:arrow_down:

src/schemathesis/extra/pytest_plugin.py
98.36% <0.00%> (-1.64%)
:arrow_down:

src/schemathesis/cli/__init__.py
98.72% <0.00%> (-1.28%)
:arrow_down:

src/schemathesis/runner/impl/core.py
98.24% <0.00%> (-0.01%)
:arrow_down:

src/schemathesis/lazy.py
100.00% <0.00%> (ø)

src/schemathesis/hooks.py
100.00% <0.00%> (ø)

src/schemathesis/targets.py
100.00% <0.00%> (ø)

src/schemathesis/stateful.py
100.00% <0.00%> (ø)

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21a8d27...132558f. Read the comment docs.


section="main" full-width class is not used when disable_profile and disable_widgets are true

Since the introduction of baseof.html if you set your post with disable_profile =true and disable_widgets = true it no longer sets the section="main" class="full-width".

@dutchiechris This might sound crazy, but can you tell me where exactly to put disable_profile =true? I've tried adding it to a number of places in baseof.html, and the profile isn't disabled! I feel like I'm missing something obvious... I know my question isn't related to your cool fix, so, sorry about that.

@RyanTG you put them in the frontmatter at the top of your markdown files between the section blocked off by +++.


docs(scully): typo in FAQ

PR Checklist
Please check if your PR fulfills the following requirements:

[ ] The commit message follows our guidelines: https://github.com/scullyio/scully/blob/main/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[x] Other... Please describe: typo in docs

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


MINOR: [JS] Add caution to Vector.toArray doc-comment about how it handles nulls (it doesn't).

This just adds a minor cautionary note to Vector.toArray, the diff is self-explanatory.

Closing because it has been untouched for a while, in case it's still relevant feel free to reopen and move it forward 👍


Backport "HBASE-24628 Region normalizer now respects a rate limit" to branch-2

Implement a rate limiter for the normalizer. Implemented in terms of
MB/sec of affacted region size (the same metrics used to make
normalization decisions). Uses Guava RateLimiter to perform the
resource accounting. RateLimiter works by blocking (uninterruptible
😖) the calling thread. Thus, the whole construction of the normalizer
subsystem needed refactoring. See the provided package-info.java for
an overview of this new structure.
Introduces a new configuration,
hbase.normalizer.throughput.max_bytes_per_sec, for specifying a
limit on the throughput of actions executed by the normalizer. Note
that while this configuration value is in bytes, the minimum honored
valued 1_000_000. Supports values configured using the
human-readable suffixes honored by Configuration.getLongBytes
Signed-off-by: Viraj Jasani<EMAIL_ADDRESS>Signed-off-by: Huaxiang Sun<EMAIL_ADDRESS>Signed-off-by: Michael Stack<EMAIL_ADDRESS>

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 31s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
3m 40s
branch-2 passed

+1 :green_heart:
checkstyle
1m 11s
branch-2 passed

+1 :green_heart:
spotbugs
2m  0s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 13s
the patch passed

-0 :warning:
checkstyle
1m 10s
hbase-server: The patch generated 13 new + 152 unchanged - 7 fixed = 165 total (was 159)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
11m 29s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

+1 :green_heart:
spotbugs
2m  8s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 15s
The patch does not generate ASF License warnings.

33m  0s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2520

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux ad9f2ea95dc7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 627002d279

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

Max. process+thread count
94 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 32s
Docker mode activated.

-0 :warning:
yetus
0m  8s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
3m 38s
branch-2 passed

+1 :green_heart:
compile
0m 57s
branch-2 passed

+1 :green_heart:
shadedjars
5m 56s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 43s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 15s
the patch passed

+1 :green_heart:
compile
0m 57s
the patch passed

+1 :green_heart:
javac
0m 57s
the patch passed

+1 :green_heart:
shadedjars
5m 55s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 36s
hbase-server generated 11 new + 2 unchanged - 25 fixed = 13 total (was 27)

_ Other Tests _

-1 :x:
unit
137m 11s
hbase-server in the patch failed.

162m  7s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2520

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 3733501d6de2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 627002d279

Default Java
1.8.0_232

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-hbase-server.txt

unit
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/testReport/

Max. process+thread count
4441 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  8s
Docker mode activated.

-0 :warning:
yetus
0m  7s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
4m 16s
branch-2 passed

+1 :green_heart:
compile
1m  5s
branch-2 passed

+1 :green_heart:
shadedjars
6m 38s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 48s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 52s
the patch passed

+1 :green_heart:
compile
1m  5s
the patch passed

+1 :green_heart:
javac
1m  5s
the patch passed

+1 :green_heart:
shadedjars
6m 45s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 40s
the patch passed

_ Other Tests _

-1 :x:
unit
137m 23s
hbase-server in the patch failed.

166m  9s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2520

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux cffe61ba22da 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 627002d279

Default Java
2020-01-14

unit
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/testReport/

Max. process+thread count
3959 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


Component: CTA - Keyboard - After the model page opens, focus not set on close icon.

Detailed description
Issue :- After the model page opens, focus not set on close icon.
Expected :- As soon as model page opens, focus should set on the close button of the model page.

@kennylam Moving this work to the next sprint as it is a sev 4.


Missing bin folder with 2.1.4

I recently updated from 1.6.1 and I noticed that on 2.1.4 the bin file that holds the CLI is missing now. Has this moved or removed?
This is what I have for 2.1.4

This is what I had for 1.6.1

Could this be because I'm pulling in via a commit vs a version?
pod 'Sourcery', :git => 'https://github.com/krzysztofzablocki/Sourcery.git', :commit => '8df9735'

Hey @arbyruns ,
Sourcery 2.1.4 was released to cocoapods just now.
However, I could not get CLI-Only version to work. That means that 2.1.4 cannot be integrated as:
pod 'Sourcery', '= 2.1.4', :subspecs => ['CLI-Only']

2.1.4 is only installable as (version can be omitted):
pod 'Sourcery', '= 2.1.4'

Thank you, is CLI-Only being removed going forward?

Not sure, I just need to figure out how to make it work, that's all 👍🏻

@arbyruns 2.1.5 should support CLI-Only pod as before 2.1.4.

Thanks!


add binary classification check

Background

There are some methods on TSM that only make sense for binary classification tasks.
relates to #52

Implement

[ ] a check for how many classes an estimator has. This could be calculated on the fly or it could be added as a class attribute in AdvancedSupervisedModelTrainer._trainer()

Work completed elsewhere


Uninstalled Keptn w/ argo and cluster frozen

@bradmccoydev discovered this.

Install KLT & CRDs w/ Argo
Uninstall KLT via Argo
Cluster is frozen. Pods cannot be deleted or scheduled

Trying to schedule a new pod results an mpod.keptn.sh error.
Reason: Argo doesn't remove the CRDs and global objects (mutating webhooks) and the default failurePolicy is to fail.
@bradmccoydev I hope I've transcribed this correctly. Please update if not.

also a separate conversation but it would be nice to be more granular with the app.kubernetes.io/name: label. As now everything says just klt instead of its specific name.

I created a follow up issue for this, we don't need to reuse this one.


Research sagemaker sdk / client in JS

so we can use it on the backend to start a batch transform job given a bucket URI

We have decided to deploy a docker container for running the inference pipeline.


Cyclonedx 1.2 in xml

Make it possible to produce cyclone 1.2 xml boms

This has been merged into the main library https://github.com/voltone/sbom/pull/2 we are currently working on making it possible to have JSON output for 1.2 https://github.com/voltone/sbom/pull/4


Need CLI command for in-repo-engine app

Ember Engines offers only CLI command to create Ember Engine App. There is no cli command for in-repo-engine command
lib
--blog
    --addon
        --routes
        --components
        --helpers
        --services
        --templates

Always need to create routes,components,helpers,services,templates manully.

@mesadhan you don't can create using ember g on in-repo-engines, just on normal app or normal addon
on in-repo-engine is always manually ; )
can you close this, please?


Playground error message - question

Hi,
I have an issue that appeared with the playground. It may be just my misakes, but the error message displayed there is not insightful and I can't progress further after quiet the time, so I am posting here. I looked for a Stack-overflow tag but I could not find one, Please let me know if there is a better place to ask these things, as I see from the current issues that there arent really such help-me question there.
Alright so here, we go. The code in the playground is as follows:
(function jsonGrammarOnlyExample() {
  // ----------------- Lexer -----------------
  const createToken = chevrotain.createToken;
  const Lexer = chevrotain.Lexer;
  const CstParser = chevrotain.CstParser;

  const LSquare = createToken({name: "LSquare", pattern: /\[/});
  const RSquare = createToken({name: "RSquare", pattern: /]/});
  const Slash = createToken({name: "Slash", pattern: /\//});
  const Comma = createToken({name: "Comma", pattern: /,/});
  const MetaChr = createToken({name: "MetaChr", pattern: /[,!@#$%^&*()"':{}=+-;]+/});
  const StringLiteral = createToken({
    name: "StringLiteral", pattern: /(:?[^\\"\n\r]+|\\(:?[bfnrtv"\\/]|u[0-9a-fA-F]{4}))*/
  });
  const NumberLiteral = createToken({
    name: "NumberLiteral", pattern: /-?(0|[1-9]\d*)(\.\d+)?([eE][+-]?\d+)?/
  });
  const WhiteSpace = createToken({
    name: "WhiteSpace",
    pattern: /\s+/,
    group: Lexer.SKIPPED
  });

  const tokens = [WhiteSpace, NumberLiteral, StringLiteral, LSquare, RSquare, Comma, Slash, MetaChr];

  const yedLexer = new Lexer(tokens, {
    // Less position info tracked, reduces verbosity of the playground output.
    positionTracking: "onlyStart"
  });

  // Labels only affect error messages and Diagrams.
  LSquare.LABEL = "'['";
  RSquare.LABEL = "']'";
  Comma.LABEL = "','";
  Slash.LABEL = "/";

  // ----------------- parser -----------------
  // eventClause
  // : StringLiteral+
  // guardClause
  // : StringLiteral*
  // actionsClause
  // : StringLiteral*
  // edgeClause:
  // [eventClause] [\[ guardClause \]] [/ actionsClause] |

  class Parser extends CstParser {
    constructor() {
      super(tokens, {
        recoveryEnabled: false
      })

      const $ = this;

      $.RULE("yed", () => {
        $.OPTION1(() => {
          $.SUBRULE($.eventClause);
        })
        $.OPTION2(() => {
          $.CONSUME(LSquare);
          $.SUBRULE($.guardClause);
          $.CONSUME(RSquare);
        })
        $.OPTION3(() => {
          $.CONSUME(Slash);
          $.SUBRULE($.actionsClause);
        })
      });

      $.RULE("eventClause", () => {
        $.MANY(() => {
          $.CONSUME(StringLiteral)
        })
      });

      $.RULE("guardClause", () => {
        $.MANY(() => {
          $.CONSUME(StringLiteral)
        })
      });

      $.RULE("actionsClause", () => {
        $.MANY(() => {
          $.CONSUME(StringLiteral)
        })
      });

      // very important to call this after all the rules have been setup.
      // otherwise the parser may not work correctly as it will lack information
      // derived from the self analysis.
      this.performSelfAnalysis();

    }
  }

  // wrapping it all together
  // reuse the same parser instance.
  const parser = new Parser([]);

  // ----------------- Interpreter -----------------
  const BaseCstVisitor = parser.getBaseCstVisitorConstructor()

  class yedInterpreter extends BaseCstVisitor {

    constructor() {
      super(tokens, {recoveryEnabled: false})
      // This helper will detect any missing or redundant methods on this visitor
      this.validateVisitor()
    }

    yed (ctx) {
      return {event: this.visit(ctx.eventClause), guard: this.visit(ctx.guardClause), action: this.visit(ctx.actionsClause)}
    }

    eventClause (ctx){ return ctx.StringLiteral ? ctx.StringLiteral : ""}
    guardClause (ctx){ return ctx.StringLiteral ? ctx.StringLiteral : ""}
    actionsClause (ctx){ return ctx.StringLiteral ? ctx.StringLiteral : ""}
  }

  // for the playground to work the returned object must contain these fields
  return {
    lexer: yedLexer,
    parser: Parser,
    visitor: yedInterpreter,
    defaultRule: "yed"
  };
}())

The input I pass is m m m m  / a.
The error message I get is:
Errors during evaluation of the implementation:
Cannot read property 'PATTERN' of undefined

If that matters, running on Windows in the Brave browser (Chromium 81).
The language I want is fairly simple: a [b] / c for instance shoud be recognized and a, b, c can all be empty strings too. For the STRING_LITERAL, I reused the JSON_LITERAL and removed the starting and end quotes ". When doing that, I get the aforementioned error. If you put the quotes back, the error is no longer but then the string literals are not parsed (because they do not start with a quote).
Any ideas what I can be doing wrong? I don't see which PATTERN Chevrotain is trying and failing at validating.

Hello @brucou I am unable to identify the issue.
The playground is not meant to be used as a development environment
It is just a little toy to demonstrate Chevrotain
So its not something I would invest in exploring/debugging/fixing...

If you prefer coding inside a browser maybe try one of:

https://visualstudio.microsoft.com/services/visual-studio-codespaces/
https://www.gitpod.io

It is my first time using and  evaluating Chevrotain, so the playground and starting from working examples looked like the easiest way to try things out with a simple grammar like the one I have.
Error handling is hard, and often neglected in most open source projects (just like documentation). In the time I took to implement this small example from scratch from the docs and with no prior knowledge, the truth is the playground and its immediate feedback and helpful error messages was quite helpful. I had the ambiguity messages and a link which helped solve the problem. That's awesome! A pity it stopped being useful. It is a pretty good tool for starters.
Anyways, I will try on my local machine then. Maybe the error disappear or will be more helpful. Thanks! I do understand that this is the place to write about issues  with grammars then? Or do you have another channel like stack overflow for that?

Error handling is hard, and often neglected in most open source projects (just like documentation). In the time I took to implement this small example from scratch from the docs and with no prior knowledge, the truth is the playground and its immediate feedback and helpful error messages was quite helpful. I had the ambiguity messages and a link which helped solve the problem. That's awesome! A pity it stopped being useful. It is a pretty good tool for starters.

Maybe there should be a quick start template.
Sort of like a playground but as a npm package with:

Simple Grammar
Script to generate diagrams
Input file
Script to parse the input and serialize it to some JSON output.
???

Basically the flow of the playground but in a real dev env.
The playground is just a simple naive hack not a fully fledged dev env.

I do understand that this is the place to write about issues with grammars then? Or do you have another channel like stack overflow for that?

There is a gitter chat channel linked from the docs website / README.md of the repo.

@bd82

Maybe there should be a quick start template.
That would be awesome!! Yes, the main issues for beginners are:

is that the right syntax?

I think you cover that nicely with the analysis of the grammar. (That's how I got those ambiguity warnings)

is that the right semantics?

this is where you pass inputs and look at the outputs. There are two problems:

when there are no outputs: why??
when there are incorrect outputs: why??

There is a gitter chat channel linked from the docs website / README.md of the repo.

Great, thanks, I missed that information apparently.

@bd82 created this starter pack which does following things.

lexer
parser
visitor
serializer
diagram generation
unit tests

Please let me know if you have any feedback.
https://github.com/datasert/chevrotain-starter/pull/1/files

@brsanthu that's awesome:

I loved the commented code. For instance: // "input" is a setter which will reset the parser's state. I had not realized that the parser was stateful and needed to be reset to parse another string. That is typically the small things where you can spend hours trying to figure what the hell is going on.
[those comments](https://github.com/datasert/chevrotain-starter/blob/21686870756373484edc8d02064c31656c345fb1/src/parser.ts#L52 too are useful
I think the visitor is fairly clear. Not sure what the serializer is about though. Oh I see you need the serialized grammar to creae the diagram,, fair enough.

 So I guess I will have to have some tests that will grow as the grammar will grow. The tests I see are clear and give an idea of how to structure them.
Then there is the stage when one test fail where it is necessary to figure out what is wrong. I don't know if it is possible to see what rules have been analyzed, i.e. trace the parser execution.
Anyways, I will have a deeper look at it tomorrow but it looks great. Thank you so much guys.

Parser converts text to model and serializer converts model to text. To generate diagram, you just need parser.

created this starter pack which does following things.

Thanks @brsanthu I think this can be very useful.
Are you interested in contributing it to this repo:

https://github.com/SAP/chevrotain/tree/master/examples
we can create a new quick-start folder

After a quick look my feedback (assuming contribution to Chevrotain) is:

Be un-opinionated and focus on the chevrotain parts

Only use minimal default tsconfig.
avoid eslint.
avoid prettier (its part of the chevrotain mono repo anyhow).

Add more in-depth usage instructions at the top level readme

I can help with that.

I will attempt to dive into this in more depth soon.

@bd82 would be more than happy to contribute so it can live as part chevrotain repo. Couple of things.

I feel it would be much easier for somebody to get going if that started pack is separate repo than part of this mono repo. Examples are meant to reference but this repo is meant for them to clone completely, change few things and get going. So sap/chevrotain-starter may be better.

About eslint and prettier: many times, users will just get going with what comes with something by default and not try to pay attention details of config, especially peripheral config like linter/styles. So I feel that configuring those for best quality, would help users adopt the best practice.

If you think, having typescript as the lang in the starter, we could think about creating chevrotain-starter-ts and chevrotain-starter-js repos so users can pick the one they want.

About docs, yes, we need to enhance it.

Hi @brsanthu
Sorry for the late response.
MonoRepo vs Separate repo.
I agree a separate repo is easier for consumers, but there is also the concern of testing the quick-start repo. If its part of the mono repo it would be ensured to be updated (e.g on breaking changes).
Is there some way to have the cake and eat it too?

e.g a git clone of a specific sub folder of a repo?
Some kind of nesting of git repos (git sub-modules?)

Dev Flows configurations (linter/formatter).
This is quite problematic.

Some tools choice are opinionated, Who is to say X is better than Y tool?
Secondly I'd rather not maintain the dev-flow tools of a quick start repo, e.g:

In the past TSLint was used, but now its deprecated.
Updating versions of dep deps and breaking changes.

IMHO The scope has to be minimal and focused on Chevrotain flows, not generic dev flows.
Both TypeScript and JavaScript quick starts are a good idea

The Chevrotain repo should move to a new organization in the future.
I've created a quick-start repo there you can contribute to if you want to start with a separate repo flow.

https://github.com/Chevrotain/quick-start

With some luck I'll be able to connect it to the mono-repos CI flow somehow...

@brsanthu I've sent you an invite for write permissions in https://github.com/Chevrotain/quick-start

Got the access. Will push to that repo the changes I have and we can go
from there.
On Thu, Jun 11, 2020 at 5:05 PM Shahar Soel<EMAIL_ADDRESS>wrote:

@brsanthu https://github.com/brsanthu I've sent you an invite for write
permissions in https://github.com/Chevrotain/quick-start
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SAP/chevrotain/issues/1214#issuecomment-642989929,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGNZVF5ZMTHD6GDLSYCXOTRWFWNHANCNFSM4NQJYXTQ
.

Thanks @brsanthu
I've done a quick review, lets continue the discussions on that repo.
I will close this issue for now.
You can also send me a direct message on gitter (link in the root README)


Http status should be 503 when readiness check fails.

After code refactoring,  http response status of readiness health check was ignored!!  It should be ensured that 200 would be returned when readiness health check successes, 503 returned instead when readiness health check fails.

And also it should be compatible with WebFlux.


App name tracing

I want to merge this change because it ports https://github.com/saleor/saleor/pull/8848 to 3.1

Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[ ] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

Task linked: SALEOR-5067 Add App name to GraphQL Span when it's possible


Some Error when import uncalled4

Hi Sam,
I have installed uncalled4 successfully and trying the below codes:
`from uncalled4 import PoreModel, Config, EventDetector, SignalProcessor
But it seems that there are still some problems, maybe tblib is forgotten to be installed. After tblib is installed, it would be successful.
And Also may I know what is the meaning of  two parameters '.PRMS_450BPS' and'.PRMS_70BPS'
`from uncalled4 import PoreModel, Config, EventDetector, SignalProcessor
m = PoreModel("dna_r9.4.1_400bps_6mer") #or "dna_r10.4.1_400bps_9mer", "rna_r9.4.1_70bps_5mer", custom TSV file
c = Config()
c.event_detector = EventDetector.PRMS_450BPS #or EventDetector.PRMS_70BPS
sp = SignalProcessor(m,c)
read = sp.process_signal(signal, normalize=True) #signal should be numpy array/list of un-normalized raw sample values
print(read.events["mean"])    #event current means
print(read.events["stdv"])    #event current stdvs
print(read.events["start"])   #event sample starts
print(read.events["length"])  #event sample lengths
print(read.signal)            #normalized raw signal (using method-of-moments)`

And also when I'm trying the tool for event detector, it seems there are some other problems for kernel.
My numpy signal is like [547 464 463 467 464 487 502 482 485 492 486 487 501 491 488 486 489 501  481 491 494 489 502 496 494 494 495 491 500 497 485 492 502 483 509 494  505 495 485 496 497 502 491 490 507 495 488 488 492 492 501 492 496 495  484 482 487 482 491 490 497 512 495 486 490 493 493 486 491 492 502 499  504 479 488 497 497 500 498 502 494 503 499 506 502 506 489 500 491 509  508 487 491 475 490 491 479 493 499 501]
And the code for this part is:
m = PoreModel("dna_r9.4.1_400bps_6mer") #or "dna_r10.4.1_400bps_9mer", "rna_r9.4.1_70bps_5mer", custom TSV file
c = Config()
c.event_detector = EventDetector.PRMS_450BPS #or EventDetector.PRMS_70BPS
sp = SignalProcessor(m,c)
read = sp.process_signal(numpy_signal, normalize=True) #signal should be numpy array/list of un-normalized raw sample values
print(read.events["mean"])    #event current means
print(read.events["stdv"])    #event current stdvs
print(read.events["start"])   #event sample starts
print(read.events["length"])  #event sample length
print(read.signal)            #normalized raw signal (using method-of-moments)
But it still have some problems the kernel would die.
Segmentation fault (core dumped)

Thank you for trying this, I will add the tblib dependency. Are you able to run the example commands under the example/ directory (first run ./download.sh, then ./run.sh)? Just wondering if it's an issue with that code snippet or if it's more fundamental. If the example script works, try turning your signal into a float array like this: signal = np.array(signal, dtype=float). If that doesn't work, could you add print(...) statements between each line to see where the error happens? Segmentation faults in python can be difficult to debug.
The PRMS_450BPS is a preset set of event detection parameters for sequencing speeds of ~400-450 bases per second. These parameters control the event detector t-test window lengths and thresholds (specifically 'peak_height', 'threshold1', 'threshold2', 'window_length1', 'window_length2' via dir(c.event_detector)), which are set based on the expected number of samples-per-second and the observed signal alignment stay/skip rate when using those parameters.

Thanks Sam,
When I'm trying the example, it seems there are still some errors from samtool, others seems ok.
Running 'uncalled4 align ref/ec_16s.fa raw/rna002_r9_ec_ivt.fast5 --bam-in mm2/rna002_r9_ec_ivt.bam --bam-out out/rna002_r9_ec_ivt.align.bam 2> out/rna002_r9_ec_ivt.align.err' Running 'uncalled4 align ref/ec_16s.fa raw/rna002_r9_ec_native.fast5 --bam-in mm2/rna002_r9_ec_native.bam --bam-out out/rna002_r9_ec_native.align.bam 2> out/rna002_r9_ec_native.align.err' Running 'uncalled4 align ref/dm_chr1.fa raw/dna_r10_dm_ctl.fast5 --bam-in mm2/dna_r10_dm_ctl.bam --bam-out out/dna_r10_dm_ctl.align.bam 2> out/dna_r10_dm_ctl.align.err' Running 'uncalled4 align ref/dm_chr1.fa raw/dna_r9_dm_ctl.fast5 --bam-in mm2/dna_r9_dm_ctl.bam --bam-out out/dna_r9_dm_ctl.align.bam 2> out/dna_r9_dm_ctl.align.err' Running 'uncalled4 align --flowcell FLO-MIN106 --kit SQK-RNA002 ref/ec_16s.fa raw/rna002_r9_ec_ivt.pod5 --bam-in mm2/rna002_r9_ec_ivt.bam --bam-out out/rna002_r9_ec_ivt.pod5.align.bam 2> out/rna002_r9_ec_ivt.pod5.align.err' Running 'uncalled4 align --flowcell FLO-MIN106 --kit SQK-RNA002 ref/ec_16s.fa raw/rna002_r9_ec_ivt.blow5 --bam-in mm2/rna002_r9_ec_ivt.bam --bam-out out/rna002_r9_ec_ivt.blow5.align.bam 2> out/rna002_r9_ec_ivt.blow5.align.err' Running 'samtools index out/rna002_r9_ec_ivt.align.bam 2> out/rna002_r9_ec_ivt.align.index.err' rna002_r9_ec_ivt.align.index: failed with code 127 $ cat out/rna002_r9_ec_ivt.align.index.err ./run.sh: line 20: samtools: command not found
And for those command, the error is from normalization part of read = sp.process_signal(signal, normalize=True), when I try read = sp.process_signal(signal), there would be no error, but it seems it is not the right result. The code I run is like
m = PoreModel("dna_r9.4.1_400bps_6mer") #or "dna_r10.4.1_400bps_9mer", "rna_r9.4.1_70bps_5mer", custom TSV file
c = Config()
c.event_detector = EventDetector.PRMS_450BPS #or EventDetector.PRMS_70BPS
sp = SignalProcessor(m,c)
print("yes")
read = sp.process_signal(signal) #signal should be numpy array/list of un-normalized raw sample values
print(read.events["mean"])    #event current means
print(read.events["stdv"])    #event current stdvs
print(read.events["start"])   #event sample starts
print(read.events["length"])  #event sample lengths
print(read.signal)            #normalized raw signal (using method-of-moments)`
the result is yes [] [] [] [] [547 464 463 ... 488 488 492 492]

Ah I think I know the issue, try adding this line right before initializing "SignalProcessor": c.event_detector.max_mean = 4000
The event detector expects the signal to be approximately picoamp-scaled, so by default max_mean is set to 200, so only events with event.mean <= 200 will be output. The values in your input raw signal are "data acquisition" integers, which will be corrected after normalization, but it conflicts with that default event detector parameter. If you're using the ont_fast5_api to read FAST5s, you can instead get approximately picoamp-scaled signal using the option get_raw_data(scale=True). The scaled values are not very accurate, so normalization is still needed. It's mainly useful to identify major outliers prior to normalization. Apologies that none of this is documented, I hope to make the python interface more user-friendly in the future.
Also, a tip for pasting code/output on multiple lines: try surrounding the code blocks with three "back ticks" (```) above and below the lines of code, that way it won't all be put into one line.
Thanks,
Sam

Thank you very much, it works well now!


hetzner cloud integration

Hi @andrasbacsai hetzner cloud integration will be a nice feature as well :)

Isn't this possible with the upcoming Remote Docker Engine destination?

No, RDE only connects to remote docker engines, but this request is to create a Hetzner server programmatically from Coolify, for example.

I'm using docker-machine to create on-demand cloud server for my docker swarm cluster. It's very easy and supports so many providers. However it's deprecated there're only the forks such as gitlab's. So I'm not sure if using docker-machine would be a smart decision.
What about something like Ansible, Terraform?

I'm closing this issue, because tracking features will be moved to GitHub discussions. Github issues will be only for bugs.
Version 4 is in full focus. v3 related issues will be closed, if it is not a critical bug.
Thanks for your understanding.


fixing issue where AzureActiveDirectoryCredentials.AccessToken is null

Fixes Issue #59.

Would this let me fix the AAD case here and get a client directed flow in e2e https://github.com/Azure/azure-mobile-apps-net-server/blob/master/e2etest/Controllers/Table/AuthenticatedController.cs#L52?

I believe so, yes.

Looks good


Create<EMAIL_ADDRESS>

#2

What is this? P1NNAC7E Pro .... Direct Mail Advertising??!

considered spam.


Fixed RestKit example and fixed iCloud stack error

RestKit example would crash when adding new entry

iCloud stack would crash when initializing it (NSForwarding error)

iCloud stack would crash because its initialized in background thread and you're able to use it before its done initializing -- as a (temp?) fix made it post a notification to know when its done

Changed the example project to follow the same format throughout

Ready @dchavezlive . Pa lante!


⚠ Bofrost.be has degraded performance

In e905844, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 2482 ms

Resolved: Bofrost.be performance has improved in 5e37005.


remove old licenses from packages in monorepo

the license in the top level of the monorepo applies to all packages


AddError: could not parse if fileName begins with "-"

https://goreportcard.com/report/github.com/rogozhka/tgwrap
I see no forbiddance for naming like "-wrap-command.go" in language spec. Am I wrong?
BTW it is useful to sort and group files.

@rogozhka This was probably an issue with our code - I don't think we've seen any repos with files that begin with dashes up until now. It looks like you updated your filenames to remove the dashes from the beginning so I'm going to close this. If someone else runs into it we can revisit. Thanks


add recipe for changes

"unknown repository" appears b/c I trashed my fork after having pull accepted


build cpu dockerfile problem with TF version?

Hi
i build Dockerfile.cpu
and when predict
./predict  --docker-image nima-cpu --base-model-name MobileNet --weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 --image-source /mnt/videos/data/frames/videos/1116l.mp4/001.jpg
Using TensorFlow backend.
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/src/evaluater/predict.py", line 73, in <module>
    main(**args.__dict__)
  File "/src/evaluater/predict.py", line 43, in main
    nima.build()
  File "/src/handlers/model_builder.py", line 35, in build
    self.base_model = BaseCnn(input_shape=(224, 224, 3), weights=self.weights, include_top=False, pooling='avg')
  File "/usr/local/lib/python3.6/dist-packages/keras/applications/mobilenet.py", line 248, in MobileNet
    img_input = Input(shape=input_shape)
  File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 1457, in Input
    input_tensor=tensor)
  File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 1319, in __init__
    name = prefix + '_' + str(K.get_uid(prefix))
  File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 68, in get_uid
    graph = tf.get_default_graph()
AttributeError: module 'tensorflow' has no attribute 'get_default_graph'

nimitdesai92 has a working fork, it just needed to add the libraries from tensorflow. I had no idea how to do this since Im new to docker and had no idea how to change the docker image...
After seeing nimitdesai92 fix keras import fork i realised it was way simpler than i thought...

We upgraded to TF2.0 and fixed the TF version in the Dockerfiles. Please try again and re-open if error persists.


Fix MutableBuffer::into_buffer leaking its extra capacity into the final buffer

The extra capacity that is initially used to efficiently append data to the mutable buffer ends up leaking into the final immutable buffer, where it will linger indefinitely, therefore hogging memory.

Consider this code:
use arrow::{
    array::{Array, ArrayRef, ListArray, PrimitiveArray},
    buffer::OffsetBuffer,
    datatypes::{Field, UInt8Type},
};

fn main() {
    let array0: PrimitiveArray<UInt8Type> = (0..200 * 300)
        .map(|v| (v % 255) as u8)
        .collect::<Vec<_>>()
        .into();
    let array0: ArrayRef = Arc::new(array0);

    let (global, local) = memory_use(|| {
        let concatenated = concatenate(array0.clone());
        dbg!(concatenated.data_type());
        eprintln!("expected: ~{}", how_many_bytes(concatenated.clone()));
        concatenated
    });
    eprintln!("global: {global} bytes");
    eprintln!("local: {local} bytes");

    eprintln!("---");

    let array1 = ListArray::new(
        Field::new_list_field(array0.data_type().clone(), false).into(),
        OffsetBuffer::from_lengths(std::iter::once(array0.len())),
        array0.clone(),
        None,
    );
    let array1: ArrayRef = Arc::new(array1);

    let (global, local) = memory_use(|| {
        let concatenated = concatenate(array1.clone());
        dbg!(concatenated.data_type());
        eprintln!("expected: ~{}", how_many_bytes(concatenated.clone()));
        concatenated
    });
    eprintln!("global: {global} bytes");
    eprintln!("local: {local} bytes");
}

fn concatenate(array: ArrayRef) -> ArrayRef {
    let mut concatenated = array.clone();

    for _ in 0..1000 {
        concatenated = arrow::compute::kernels::concat::concat(&[&*concatenated, &*array]).unwrap();
    }

    concatenated
}

fn how_many_bytes(array: ArrayRef) -> u64 {
    let mut array = array;
    loop {
        match array.data_type() {
            arrow::datatypes::DataType::UInt8 => break,
            arrow::datatypes::DataType::List(_) => {
                let list = array.as_any().downcast_ref::<ListArray>().unwrap();
                array = list.values().clone();
            }
            _ => unreachable!(),
        }
    }

    array.len() as _
}

// --- Memory tracking ---

use std::sync::{
    atomic::{AtomicUsize, Ordering::Relaxed},
    Arc,
};

static LIVE_BYTES_GLOBAL: AtomicUsize = AtomicUsize::new(0);

thread_local! {
    static LIVE_BYTES_IN_THREAD: AtomicUsize = AtomicUsize::new(0);
}

pub struct TrackingAllocator {
    allocator: std::alloc::System,
}

#[global_allocator]
pub static GLOBAL_ALLOCATOR: TrackingAllocator = TrackingAllocator {
    allocator: std::alloc::System,
};

#[allow(unsafe_code)]
// SAFETY:
// We just do book-keeping and then let another allocator do all the actual work.
unsafe impl std::alloc::GlobalAlloc for TrackingAllocator {
    #[allow(clippy::let_and_return)]
    unsafe fn alloc(&self, layout: std::alloc::Layout) -> *mut u8 {
        LIVE_BYTES_IN_THREAD.with(|bytes| bytes.fetch_add(layout.size(), Relaxed));
        LIVE_BYTES_GLOBAL.fetch_add(layout.size(), Relaxed);

        // SAFETY:
        // Just deferring
        unsafe { self.allocator.alloc(layout) }
    }

    unsafe fn dealloc(&self, ptr: *mut u8, layout: std::alloc::Layout) {
        LIVE_BYTES_IN_THREAD.with(|bytes| bytes.fetch_sub(layout.size(), Relaxed));
        LIVE_BYTES_GLOBAL.fetch_sub(layout.size(), Relaxed);

        // SAFETY:
        // Just deferring
        unsafe { self.allocator.dealloc(ptr, layout) };
    }
}

fn live_bytes_local() -> usize {
    LIVE_BYTES_IN_THREAD.with(|bytes| bytes.load(Relaxed))
}

fn live_bytes_global() -> usize {
    LIVE_BYTES_GLOBAL.load(Relaxed)
}

/// Returns `(num_bytes_allocated, num_bytes_allocated_by_this_thread)`.
fn memory_use<R>(run: impl Fn() -> R) -> (usize, usize) {
    let used_bytes_start_local = live_bytes_local();
    let used_bytes_start_global = live_bytes_global();
    let ret = run();
    let bytes_used_local = live_bytes_local() - used_bytes_start_local;
    let bytes_used_global = live_bytes_global() - used_bytes_start_global;
    drop(ret);
    (bytes_used_global, bytes_used_local)
}

HEAD:
[src/main.rs:16:9] concatenated.data_type() = UInt8
expected: ~60060000
global: 60060200 bytes
local: 60060200 bytes
---
[src/main.rs:35:9] concatenated.data_type() = List(
    Field {
        name: "item",
        data_type: UInt8,
        nullable: false,
        dict_id: 0,
        dict_is_ordered: false,
        metadata: {},
    },
)
expected: ~60060000
global: 120004384 bytes
local: 120004384 bytes

This patch:
[src/main.rs:16:9] concatenated.data_type() = UInt8
expected: ~60060000
global: 60060200 bytes
local: 60060200 bytes
---
[src/main.rs:35:9] concatenated.data_type() = List(
    Field {
        name: "item",
        data_type: UInt8,
        nullable: false,
        dict_id: 0,
        dict_is_ordered: false,
        metadata: {},
    },
)
expected: ~60060000
global: 60064416 bytes
local: 60064416 bytes

I am concerned that this change will decrease performance by forcing an extra copy in all situations, though I may not understand the implications

I wouldn't expect this to have a noticable impact on performance for two reasons:

MutableBuffer already reallocates memory and copies data around as it grows in any case (that's the root of this issue to begin with, after all).
I don't think the extra copy is a guarantee -- see my answer to the next question.

Does this force a copy in the case where the capacity is larger that the actual need?

As far as I understand by looking at the code, at the end of the day all of this machinery bottoms out in realloc().
Whether this actually reallocates memory and copies data around should therefore be unspecified: it is left as an implementation detail of whichever GlobalAllocator is currently in use.
I'd expect any decent allocator to do whatever is most efficient depending on the exact situation at hand.

Have you considered making the initial capacity calculations more accurate (I am not sure this is possible) so the delta between capacity and actual is lower

That doesn't seem possible given the nature of MutableBuffer and how it is used across the different layers of the codebase.

This PR currently seems to have some CI failures

I've updated the tests to reflect the new expected capacity values, so they should now pass.
Beware that I am changing these values with absolutely zero knowledge of the context in which these tests were written though: it would be fair to say that I have effectively no idea what I'm doing.

That's it for the theory... now for the practice.
I have found two relevant benchmark suites that relate to this change:

--bench concatenate_kernel
--bench mutable_array

--bench concatenate_kernel
Performance looks roughly on pair with master for this benchmark suite. In fact it even looks like a slight overall improvement, for some reason.
taskset -c 7 cargo bench -p arrow --bench concatenate_kernel --features="test_utils"

concat i32 1024         time:   [453.90 ns 454.31 ns 454.75 ns]
                        change: [-3.7808% -3.5907% -3.4059%] (p = 0.00 < 0.05)
                        Performance has improved.

concat i32 nulls 1024   time:   [652.64 ns 652.88 ns 653.18 ns]
                        change: [+0.0226% +0.2489% +0.4726%] (p = 0.03 < 0.05)
                        Change within noise threshold.

concat 1024 arrays i32 4
                        time:   [109.22 µs 109.38 µs 109.57 µs]
                        change: [-7.2900% -6.8379% -6.4195%] (p = 0.00 < 0.05)
                        Performance has improved.

concat str 1024         time:   [9.3471 µs 9.3609 µs 9.3764 µs]
                        change: [-0.1653% +0.4566% +1.0820%] (p = 0.16 > 0.05)
                        No change in performance detected.

concat str nulls 1024   time:   [5.7129 µs 5.7328 µs 5.7480 µs]
                        change: [-4.0256% -3.3280% -2.6276%] (p = 0.00 < 0.05)
                        Performance has improved.

concat str_dict 1024    time:   [2.4110 µs 2.4143 µs 2.4177 µs]
                        change: [+2.5143% +2.6708% +2.8211%] (p = 0.00 < 0.05)
                        Performance has regressed.

concat str_dict_sparse 1024
                        time:   [4.7248 µs 4.7301 µs 4.7371 µs]
                        change: [+1.7168% +1.8403% +1.9509%] (p = 0.00 < 0.05)
                        Performance has regressed.

concat str nulls 1024 #2
                        time:   [5.7316 µs 5.7364 µs 5.7415 µs]
                        change: [-3.8916% -3.0466% -2.2002%] (p = 0.00 < 0.05)
                        Performance has improved.

concat fixed size lists time:   [136.77 µs 136.97 µs 137.18 µs]
                        change: [-0.5076% -0.1102% +0.2847%] (p = 0.63 > 0.05)
                        No change in performance detected.

--bench mutable_array
This one is a different beast -- it is much slower on this branch:
mutable str 1024        time:   [31.213 ms 31.302 ms 31.420 ms]
                        change: [+73.378% +74.185% +74.973%] (p = 0.00 < 0.05)
                        Performance has regressed.

mutable str nulls 1024  time:   [11.210 ms 11.273 ms 11.376 ms]
                        change: [-13.933% -13.406% -12.652%] (p = 0.00 < 0.05)
                        Performance has improved.

which means my first assumption above was wrong: you can certainly feel the extra copy if the right conditions are met, despite MutableBuffer doing a bunch of its own.
That's with the default system allocator though, which is notoriously not a good idea... Running these benchmarks with mimalloc tells a very different story:
diff --git a/arrow/benches/mutable_array.rs b/arrow/benches/mutable_array.rs
index b04e5cd84..d9b388e68 100644
--- a/arrow/benches/mutable_array.rs
+++ b/arrow/benches/mutable_array.rs
@@ -15,6 +15,11 @@
 // specific language governing permissions and limitations
 // under the License.

+use mimalloc::MiMalloc;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
 #[macro_use]
 extern crate criterion;
 use criterion::Criterion;

Results:
mutable str 1024        time:   [11.755 ms 11.790 ms 11.825 ms]
                        change: [+0.1332% +0.5256% +0.9202%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

mutable str nulls 1024  time:   [5.8176 ms 5.8273 ms 5.8385 ms]
                        change: [-1.0864% -0.7649% -0.4585%] (p = 0.00 < 0.05)
                        Change within noise threshold.

which means my second assumption was correct: it is left to the allocator implementation to do the right thing.

Uh-oh, I missed one!

Adding another API (if it doesn't already exist) to "shrink_to_fit" for Arrays in general

That could be an alternative solution if this one doesn't cut it, yes.
Beyond the fact that it is likely much more work, it does sound like an antipattern though: given that Arrow arrays are immutable once created, is there any situation in which you would ever want all that extra capacity to hang around?
The only reason I can think of is if getting rid of that extra capacity comes at a performance cost, but as the benchmarks show above, this is not necessarily the case...

Beyond the fact that it is likely much more work, it does sound like an antipattern though: given that Arrow arrays are immutable once created, is there any situation in which you would ever want all that extra capacity to hang around?

When the buffer is merely an intermediate that won't live for very long. This is extremely common in query processing, and in fact one of the major motivations for the new binary view types is to avoid copying at the expense of less efficient memory usage.

Memory usage divided by 2, irrelevant of the allocator used.

I believe this is only the case where the bump allocator is being used, which is almost always terrible from a performance standpoint. Provided capacity estimation is done correctly, as most of the kernels go to great pains to do, this shouldn't be a major issue. Perhaps there is a particular codepath you are running into that is not doing this?

Adding another API (if it doesn't already exist) to "shrink_to_fit" for Arrays in general

I think this would be my preference, if only because it is much easier to understand the ramifications of such a change. Such a method should be relatively easy to add to the relevant ArrayBuilders, and therefore not hugely disruptive.

I think this would be my preference, if only because it is much easier to understand the ramifications of such a change. Such a method should be relatively easy to add to the relevant ArrayBuilders, and therefore not hugely disruptive.

I'm fine going down that route, but we would need for shrink_to_fit to be implemented on Array directly in our case, not ArrayBuilders.
We keep a lot of Arrow data in long-lived memory storage, and therefore must be guaranteed that the data has been optimized for space before it gets committed to storage, regardless of how it got built or how it got there.
Is that still okay?

We keep a lot of Arrow data in long-lived memory storage, and therefore must be guaranteed that the data has been optimized for space before it gets committed to storage, regardless of how it got built or how it got there.

I think that would be generally useful, and would also handle the case of sliced arrays. I'd recommend something that returns a new Array, e.g. fn shrink_to_fit(&self) -> ArrayRef. You could also provide a strongly typed non-trait version, much like we do for slice.

:+1: Alright, closing this in favor of this issue then:

https://github.com/apache/arrow-rs/issues/6360

I won't be able to attend to it for a couple weeks though.

Tis has now been fixed in another way:

https://github.com/apache/arrow-rs/pull/6790


Reply async?

Is your feature request related to a problem? Please describe.
Would it be possibly to implement some kind of asynchronous reply functionality?
I'm using a language model to generate responses based on command parameters, but it's not fast enough to generate a reply within the 3 second window, so I either have to create a new message entirely or use the thinking method somehow?
Describe the solution you'd like
A reply async function that allows the bot to think and then posts a reply once the message is ready.
Describe alternatives you've considered
I can probably implement this myself somehow, but it'd be a nice feature nonetheless, though maybe it's easier than I think?

About all library functions exposed are already async unless you opt to explicitly use _sync ones.
In this case, you need to use the thinking() function yes, or a placeholder message for reply() and later edit it when your response is ready.

Thanks a lot for the quick response!


Can't create new database connection on Mac

Description
OS- Mac, Ventura, M1 chip.
I downloaded the correct MySQL version (ARM) from https://dev.mysql.com/downloads/mysql/.
The server is active:

When opening Dbeaver, there is no option to create a  new database connection. The button is grayed out. See screenshot:

DBeaver Version
Version <IP_ADDRESS>309041200
Operating System
Mac, Ventura, M1 chip
Database and driver

Driver for MySQL 8 and later. ID -mysql8.
I can't connect a database but I would connect MySQL

Steps to reproduce

Click on 'create new database connection', but the button is grayed out.

Additional context
No response

Could you try resetting perspective? Normally you should see Navigator  on the left.

Thank you very much Elizabeth, this solved the issue.


Move protoc-bin-vendored into a separate repository

It is very simple
It has no dependencies on rust-protobuf or vice-versa
No need to release it each time rust-protobuf is released

Done.


fix(firebase): remove duplicate headers

:tada: This PR is included in version 1.0.0-alpha.13 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


user Id creation are not unique (Important)

The creation of the user id does not work, in the pilot there were all user_ids equal.
The  error has to be in the class UserIdentification in the Start() function:
@PanMig please check if this receives the right values, when adding a name, class and school
string[] user_ids = inputFields.OfType<InputField>().Select(o => o.ToString()).ToArray();
In the SDK we have to check if this function is working correctly:
@Marc please check if the hashing is working correctly
string hashed_user_id = GoedleUtils.userHash(user_ids.ToString());

Marc,
The user_ids array does not have the right values. It seems that you are getting the title of the input field object (it returns name, class ,school). What you need is to get the value of the TEXT child of the input field. I tried to do it but I cannot really understand this line of code. The question is however, do we really need this complex behaviour because you can get the values through the game manager, that works and then parse them for md5 format ?

Thank you for the hint! I already start changing it.


Our videos got deleted from a youtube account we don't own. I uploaded them back.

All in the title.
Looks like those video were uploaded by the company that recorded them for us, and the company closed their youtube channel.
I had a backup so I uploaded them back on youtube plus our own server and enhanced the metadata a bit. Upload to peertube soon™.

It's been ~two days and it seems not yet in production: https://pyvideo.org/pycon-fr-2016/faut-il-etre-masochiste-pour-utiliser-ipv6-dans-son-code-python.html
Is it normal or is it an issue?

Oh! I'll need to issue a PR against they pyvideo/pyvideo repo to update the data commit. I'll do that now.

https://github.com/pyvideo/pyvideo/pull/290

TIL. Thanks!!


Assigning a sub-reducer result to a property mutates original state

🐛 Bug Report
First of all, thanks for Immer!
I am using a main Redux state with child-states, each with its own reducer. For some actions I need the main reducer to update the whole state (for example initialisation from server data).
const subtreeReducer = produce((subtreeDraft, action) => { … }

const rootReducer = produce((draft, action) => {
  switch (action.type) {
    case "INIT_FROM_SERVER":
      return …
      break

    default:
     draft.subtree = subtreeReducer(draft.subtree, action)
  }
}, {)

I think I understand why this is not working (reassigning a whole property, not changing it), but I am not finding the proper "immer" way to do so, and I think this is a quite common usecase.
Link to repro (highly encouraged)
See Codesandbox: https://codesandbox.io/s/olo1yo9y3z
To Reproduce
See codesandbox
Expected behavior
I would expect the same result with a sub-reducer and without it. But when I am using my sub-reducer, the original state is mutated.
I guess the proper solution would be for the sub-reducer to not call produce, but it makes this reducer harder to test.
Environment

Immer version: 2.1.4
[x] Occurs with setUseProxies(true)
[x] Occurs with setUseProxies(false) (ES5 only)

Note: The codesansbox is like this as I tried to reproduce a problem I am seeing in my app.
I just tested to remove the splice call it my reducer and the problem goes away. So I am really unsure of what is really happening here 🤔

It's an internal bug. Looking into it. 👍

:tada: This issue has been resolved in version 2.1.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

Thanks, that was quick!
So this was not me doing anything wrong, good to know 😄


Blank screen for bitcoin price.

There seems to be an issue when getting bitcoin price, regularly the value is not retuned and the screen updates with no values ie: blank.

same issue for me for month now. Sometimes it works after a reinstall of the LCD software but only for a few hours, afterwards the price is just blank, all other stuff is shown. Looks like an issue with the price feed. At first I thought maybe my firewall is blocking something out but I can't find anything (and it would not work after reboot or intermittently)


Closing channel view doesn't show closing transaction

I force closed a channel and checked the closing channel for details but couldn't find the closing transaction that was broadcasted. I think users should be able to view the closing transaction and the time lock applied to their funds.

I also created an issue for this yesterday! Closing as a duplicate of https://github.com/LN-Zap/zap-desktop/issues/1687


chore: add io to global header nav

Because the nav links to https://observabilitypacks.gatsbyjs.io/instant-observability/ it won't be possible to see the darker background until the pr is merged

Closing this in favour of the refactor


possible to enhance the chromium-browser with an adblocker or other chromium addons?

i wonder if there is a possibility to enhance the chromium instance with addons like ublock?

Not currently, and I'm not planning to implement support for them, as it would be quite difficult and I have currently very limited time for implementing new features. See also https://github.com/ttalvitie/browservice/issues/18#issuecomment-784879027.


Fixtures and tooling update

This PR:

Updates existing fixtures and adds SHA256 suite ones. Proof fixture files are included but not added to the spec to avoid further conflicts with #210.
Updates the fixtures tooling to read and write the fixtures on the document.
Closes #217. Updates the message-generators tooling to return the correct file format.

Notes

This PR depends on #216 (in all the fixtures i am not prepending the length of the ciphersuite's ID encoding when hashing).
The message-generators tooling is "deactivated" on the workflow since it is not up to date with the spec. The generator files are not the ones returned by that tool and should be the correct ones.

Would it be possible to include fixtures for some of the other utility operations, specifically scalars = hash_to_scalar(msg_octets, count). This operation is harder to validate as it's always a dependent part inside existing operations. A simple fixture for hashing an octet string into 1 and 2 scalars would go a long way.

Some of the DST or SEED values contain RO_ in them, while some use R0_, letter O vs number 0. Is this intentional?

R.e., the R0_ in the DST's it's a typo. Thanks for identifying it, would never had picked it up 😅 The correct form is RO_ (not with the zero). The fixtures use the correct DST (with RO_)
R.e., the make_generators procedure, the current one is not up to date with the latest spec. PR #220 updates it. This PR has "deactivated" that tooling for the workflow, i.e., the generators fixtures are not yet created from that tool. Will reactivate it after #220 is merged.

Added hash-to-scalar and MapMessageToScalarAsHash test vectors. @tmarkovski hope those will be useful!

@tmarkovski, @christianpaquin I just pushed some new test vectors, after fixing a typo i made in one of the dst's. Sorry about that. It should be consistent now. Please use the updated ones in your testings!

Happy to report that all test vectors are passing at present! Thanks for updating the test vectors and looking into the my issue.

Test Results
running 54 tests
test hashing::test::test_encode ... ok
test encoding::test::encode_for_hash_test_vector ... ok
test encoding::test::to_octet_string_test ... ok
test hashing::test::hash_to_curve_1_scalar_output_shake ... ok
test hashing::test::hash_to_curve_1_scalar_output_sha ... ok
test hashing::test::hash_to_curve_10_scalar_output_shake ... ok
test hashing::test::map_message_to_scalar_test ... ok
test key::test::get_random_key ... ok
test key::test::gen_key_from_ikm ... ok
test key::test::sk_to_pk_test ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof008_json ... ok
test generators::test::generators_test::case_bls12_381_sha_256_generators_json_bls12381sha256 ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof007_json ... ok
test generators::test::generators_test::case_bls12_381_shake_256_generators_json_bls12381shake256 ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof009_json ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof001_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof001_json ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof004_json ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof005_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof007_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof008_json ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof003_json ... ok
test signature::test::signature_from_octets_fails_incorrect_size ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof006_json ... ok
test signature::test::signature_from_octets_succeeds_slice ... ok
test signature::test::signature_from_octets_succeeds ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof009_json ... ok
test proof::test::test_proof_from_bytes ... ok
test proof::test::proof_suite_1::case_bls12_381_sha_256_proof_proof002_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof002_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof004_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof003_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof005_json ... ok
test proof::test::proof_suite_2::case_bls12_381_shake_256_proof_proof006_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature003_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature002_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature001_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature005_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature004_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature006_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature007_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature005_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature003_json ... ok
test signature::test::signature_to_octets_succeeds ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature002_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature001_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature008_json ... ok
test signature::test::signature_suite_1::case_bls12_381_sha_256_signature_signature009_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature004_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature006_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature007_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature008_json ... ok
test signature::test::signature_suite_2::case_bls12_381_shake_256_signature_signature009_json ... ok
test test::bbs_demo ... ok

test result: ok. 54 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.81s

running 5 tests
test src/lib.rs - Bbs::message (line 65) ... ok
test src/lib.rs - Bbs::message_with (line 89) ... ok
test src/lib.rs - Bbs::message (line 56) ... ok
test src/lib.rs - Bbs::sign (line 115) ... ok
test src/lib.rs - Bbs::verify (line 143) ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.58s

My thoughts and findings:

Most negative tests are hard to validate for the reason they specify. Once my positive tests worked, I only assumed the negative tests are working as well, since they were all failing before.
Tests without header and presentation header might be useful as well
I stumbled on the use of hex and treating messages as encoded scalars, but this has already been addressed I think. Might be useful to add variable length message inputs.
Why does the Sign algorithm accept SK and PK as input, but doesn't validate if the PK is correct. SkToPk is fairly inexpensive operation, it seems that only SK is needed. Is there a security concern similar to what affected ECDSA?

@tmarkovski is the implementation and tests (referred in your message above) public? If so, could you please share a link?

It would be good to add a signing test over 0 messages, and a proof test disclosing no messages, to provide test cases for edge conditions (to validate the handling of empty argument arrays).

@tmarkovski are the implementation and tests (referred in your message above) public? If so, could you please share a link?

I believe it is this @jovfer? https://github.com/trinsic-id/bbs/tree/main

FYI @BasileiosKal, my implementation now validates these test vectors; simply sync with main and npm test.

I've attempted to capture all the comments from this PR in #225 and #7. Because we have three implementations verifying these fixtures I'd like to merge at this point and publish a new draft revision ahead of the IETF meeting.

Multiple approvals, reviewed on last WG call, residual gaps captured in issues, merging

Great. I updated my lib to fetch fixtures from the main branch.


Support filtering the Shows/Fairs based on the current selected tab in the map

I originally had some unique functions for doing the filtering per-tab, but but I realized that the bucket holds the source of truth for this already. Instead the tabs for the map have the ability to declare where to find the map data for shows and for fairs.

Nice one @orta


Install Java 11 on Red Hat OSes

Jenkins 2.361.1 requires Java 11 or newer and breaks with Java 1.8.

I'm surprised you merged this, because it really didn't work as expected. Or at all.

mhm I didnt merge it. Github suggested the "automerge when CI passes" option and I clicked that. but the acceptance tests didnt succed, very odd.

I can confirm that this isn't working, it keeps installing openjdk version "1.8.0_322" and then jenkins start fails:
jenkins[3290]: jenkins: invalid Java version: openjdk version "1.8.0_322"


Add support for new generator-ui5-project

This pull request is to prepare this subgenerator for the upcoming new version of the generator-ui5-project, which will use npm workspaces to run and manage the dependencies of one or more uimodule. The new generator-ui5-project:project will use this subgenerator via the static nestedGenerators = ["wdi5] config, when run via yo easy-ui5 project:project.
A little bit of background why I opted to modify this subgenerator itself instead of passing  --configPath and --specs options to it: The way easy-ui5 runs nestedGenerators doesn't allow to pass any options to these nested generators - at least I didn't find a way. I tried to write into this.options of the easy-ui5 generator from within a subgenerator, but no luck. Happy to get educated on this tho.
Also, I would have loved to read the uimodules (workspaces) from the root package.json instead of the .yo-rc.json, but the package.json simply isn't ready when the wdi5 subgenerator gets executed, as the project subgenerator itself is composed with the uimodule subgenerator (generator-ui5-project:uimodule), which handles this part of the package.json.

One more note: If we would find a way to pass --configPath and --specs to this subgenerator from another subgenerator via the easy-ui5 generator, that wouldn't actually be enough. The destinationPath of the wdi5 subgenerator would have to be adapted in any case, as this makes sure the wdi5: wdio run ... scripts gets added to the correct package.json in the respective worksapce.

Hi @vobu and @petermuessig,
As the PR for the generator-ui5-project is very close to being merged, I'd suggest to merge this PR before to avoid ugly errors for the early adopters.
Best, Nico

I suppose you'll merge this/want this merged along with the easy-ui5 rewrite being merged.
So awaiting your go here :)


chore: unify font

update font styles (size, weight, and line height) to align with s2 design

⚠ potentially controversial: added a new font for the heading-font-family variable. boilerplate uses Roboto, which is already a departure from s2. most projects have multiple fonts, so including an example on how to properly configure those might be useful.
Test URLs:

Before: https://main--aem-boilerplate--adobe.hlx.live/
After: https://unify-font--aem-boilerplate--adobe.hlx.live/

Is S2 compliance really a requirement for the boilerplate?

Is S2 compliance really a requirement for the boilerplate?

Not at all. I'm only pulling values from it in an attempt to set some sort of visual consistency across boilerplate > block collection.


Update pump-app example app with OTA

Problem

The OTA feature is not enabled on the pump-app example app
The pump-app.zap file is not fully spec. compliant

Change overview

Enabled the OTA feature by:

Adding OTA clusters to the zap file
Added handling code to the AppTask.cpp for pump-app main application
Enabled feature in the args.gni file

Cleaned up the pump-app.zap file:

Removed any old unneeded clusters
Added missing clusters
Re-ran the zap_regen_all.py script

Testing

Build the pump-app for cc13x2x7_26x2x7 and verified it could run
Re-ran the zap_regen_all.py script and verified the build process of pump-app

/rebase


Stream API & Serialization

Describe the bug
When using the Stream API, and performing a .Net Object Allocation profile for a process using the GetItemQueryStreamIterator, I see many live object instances created of this class
Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.FieldNameNode
Where I expected no serialization or deserialization to take place.
I am using the below API method to perform the operation

To Reproduce
Perform a query using the GetItemQueryStreamIterator
Expected behavior
No serialization related objects being used
Actual behavior
JsonTextNavigator being used
Environment summary
SDK Version: 3.31.0
OS Version: Windows 11
Additional context
I have also seen the same objects being created using the Typed API before our custom serializer is being called. I do have a *.diagsession from Visual Studio 2022 which I can provide if needed

@neildsh / @jcocchi Could anyone from query take a look at this report?

Hi @RapidDev1, the serialization format used by CosmosDB is not exposed to the customers. It need not be text. Thus, the client sdk would have to transcode from whatever encoding is used on the backend to text.  This would also result in the sdk deserializing the stream. I hope this helps.


kill: Implement full unit test coverage

Relying on vcMock framework introduced in virtcontainers library,
this commit raises the unit test coverage of kill.go file to 100%.
Fixes #403

Coverage increased (+1.6%) to 77.265% when pulling 4b38fc3f1cfbd46759e1c659577d0b66cb088231 on sboeuf/unit_test_kill into f7d80f58b0877571a0eacecab149079a7d7382f0 on master.

lgtm


Implemented Speech velocity

Summary of changes

Users can change the speech velocity.
Made a11y accessible with keyboard.
fixed : https://github.com/joomla/joomla-cms/issues/30598

Testing instructions

Make sure you can change the speech velocity.
check you can access a11y with keyboard.
Make sure you can click links and buttons when screen reader is enabled.

Tested successfully

Tested successfully

Tested successfully


Implementation of creep and shrinkage fib MC2010

Adding equations from the following sections:

<IP_ADDRESS>.3 Creep
<IP_ADDRESS>.4 Shrinkage

Progress is tracked in #49.


Migrate Azure Resource plugin

Hey, I just made a Pull Request!

I raised this pull request to move the Azure Resources plugin from repos https://github.com/vippsas/backstage-azure-resources-backend and https://github.com/vippsas/backstage-azure-resource-frontend to the backstage/community-plugins repo so it is easy to manage, collaborate & improve with the Backstage community and standards.
This move also includes handing over ownership of this plugin to @deepan10.
:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Hello @kajahanne
You have to move both frontend & backend plugin in to /workspaces/azure-resources/plugins/ directory with the default version  0.0.0 and create a changeset.
Here is reference PR https://github.com/backstage/community-plugins/pull/711

Hi @deepan10, can you give this a review please? Once you have approved I'll give it another look over.
Hi @kajahanne, as @deepan10 will be helping own this plugin I'd like them to review it first, just wanting to keep you in the loop as to how the review process will work. 👍

Hello @kajahanne ,
Good to see the plugin (Frontend + Backend). It is good to migrate the backend plugin to align with the New backend system https://backstage.io/docs/backend-system/architecture/index
Also, Can you migrate packages/app and packages/backend so that it gets easier running the plugins locally?

Hi @deepan10 @04kash, looks like the requested changes have been made, can you give it another look, please? 🚀

Hi @kajahanne, looks like one of the commits have not been signed with the DCO, can you look into that, please?