PHOENIX-5728 : ExplainPlan with plan as attributes object

ExplainPlan can provide plan as attributes object for simplified assertion of individual plan attributes
Provide an additional API to get plan object
Explain queries can still continue to provide output as String / list of String, object based approach is just an additional parameter for better comparison
Utilizes object based comparison in BaseStatsCollectorIT as per Jira, the plan is to extend object based comparison to maximum tests with further sub-tasks of PHOENIX-5265
Support HashJoinPlan as part of separate Jira

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 3 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
13m 16s
master passed

+1 :green_heart:
compile
0m 59s
master passed

+1 :green_heart:
checkstyle
2m  5s
master passed

+1 :green_heart:
javadoc
0m 47s
master passed

+0 :ok:
spotbugs
3m  5s
phoenix-core in master has 966 extant spotbugs warnings.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
7m 47s
the patch passed

+1 :green_heart:
compile
0m 59s
the patch passed

+1 :green_heart:
javac
0m 59s
the patch passed

-1 :x:
checkstyle
2m 52s
phoenix-core: The patch generated 508 new + 4831 unchanged - 85 fixed = 5339 total (was 4916)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
javadoc
0m 52s
the patch passed

+1 :green_heart:
spotbugs
3m 27s
the patch passed

_ Other Tests _

-1 :x:
unit
109m 37s
phoenix-core in the patch failed.

+1 :green_heart:
asflicense
0m 26s
The patch does not generate ASF License warnings.

149m 36s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/phoenix/pull/990

Optional Tests
dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile

uname
Linux 14da9d27d30c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev/phoenix-personality.sh

git revision
master / 993ad5e

Default Java
Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08

checkstyle
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt

unit
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt

Test Results
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/1/testReport/

Max. process+thread count
6836 (vs. ulimit of 30000)

modules
C: phoenix-core U: phoenix-core

Console output
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/1/console

versions
git=2.7.4 maven=3.3.9 spotbugs=4.1.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 49s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 3 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
15m 25s
master passed

+1 :green_heart:
compile
1m  4s
master passed

+1 :green_heart:
checkstyle
2m 38s
master passed

+1 :green_heart:
javadoc
1m  0s
master passed

+0 :ok:
spotbugs
3m 56s
phoenix-core in master has 966 extant spotbugs warnings.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
8m 51s
the patch passed

+1 :green_heart:
compile
0m 54s
the patch passed

+1 :green_heart:
javac
0m 54s
the patch passed

-1 :x:
checkstyle
2m 33s
phoenix-core: The patch generated 508 new + 4831 unchanged - 85 fixed = 5339 total (was 4916)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
javadoc
0m 44s
the patch passed

+1 :green_heart:
spotbugs
3m  6s
the patch passed

_ Other Tests _

-1 :x:
unit
100m  3s
phoenix-core in the patch failed.

+1 :green_heart:
asflicense
0m 29s
The patch does not generate ASF License warnings.

144m 24s

Reason
Tests

Failed junit tests
phoenix.end2end.ConcurrentMutationsExtendedIT

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/phoenix/pull/990

Optional Tests
dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile

uname
Linux be2522e1ed7f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev/phoenix-personality.sh

git revision
master / 993ad5e

Default Java
Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08

checkstyle
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt

unit
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt

Test Results
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/2/testReport/

Max. process+thread count
6687 (vs. ulimit of 30000)

modules
C: phoenix-core U: phoenix-core

Console output
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-990/2/console

versions
git=2.7.4 maven=3.3.9 spotbugs=4.1.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

+1
I have skimmed the patch, and I like the general idea.
This also seems to be low risk, as it doesn't change existing functionality.
Does the plan involve removing the existing line-by-line generation and building the String format plan form the new object at some point ?

Thanks for the review @stoty

Does the plan involve removing the existing line-by-line generation and building the String format plan form the new object at some point ?

Indeed I think we can give this a shot in long term for sure. That's also partly the reason why I have kept separate new API which takes object builder as input rather than updating current API which just takes list of String to accumulate entire plan step by step.
Thanks

This also seems to be low risk, as it doesn't change existing functionality.

That's correct, and that's the reason this should not cause any regression.


Attempt to fix CPU spikes issue

Signed-off-by: Ameziane H<EMAIL_ADDRESS>Co-authored-by: Karim TAAM<EMAIL_ADDRESS>

PR description
This PR is an attempt to fix CPU spikes issue (#4838).
A CPU profiling on a synced node showed that most of the time is spent on calculating peer outstanding requests. This is triggered by attemptExecution called for each pending request by reattemptPendingPeerRequests and triggered by either EthPeers.dispatchMessage or EthPeers.disconnect.
EthPeers.dispatchMessage
    reattemptPendingPeerRequests
        attemptExecution
            hasAvailableRequestCapacity
EthPeers.registerDisconnect
    reattemptPendingPeerRequests
        attemptExecution
            hasAvailableRequestCapacity
So each time, a message is dispatched or a peer is disconnected, Besu will iterate over all the pending requests and try to find an available peer for each single request.

By looking to the size of pending requests, we can see that this can be huge (more than 200k), and there is a clear correlation with the CPU spikes. My understanding here is that Besu node is spammed sometimes by huge amount of requests which results of a lot of pending requests.

The idea behind this PR is to avoid unnecessary computation and iterate over available peers instead of iterating over pending requests.
Fixed Issue(s)

Should fix #4838
Documentation

[ ] I thought about documentation and added the doc-change-required label to this PR if
updates are required.

Changelog

[ ] I thought about the changelog and included a changelog update if required.

I have been running two nodes with this PR and didn't see a CPU from the installation of the last commit, but it is very hard to see if CPU spikes would happen without this PR.
Analysing the heap dump on one node that had this issue, we found that the number of pending request is very close to the number of GetPooledTransactionsFromPeerTask instances, and it by checking GC roots, it seems to be related to the way BufferedGetPooledTransactionsFromPeerFetcher works currently.
@matkt Made a modification in the last commit to reuse the same instance of BufferedGetPooledTransactionsFromPeerFetcher, I would like to have @fab-10 opinion on that

The class histogram of one of the nodes facing this issue with more than 42k pending requests

LGTM, just evaluate to change PendingPeerRequest::getPeerToUse to not filter by minimum block height

I suggest to do this in another PR as it in not related to CPU spikes.

LGTM, just evaluate to change PendingPeerRequest::getPeerToUse to not filter by minimum block height

I suggest to do this in another PR as it in not related to CPU spikes.

Fine for me


Update client.js

Identity comparison.

Fixes
Checklist

[ ] I have made a material change to the repo (functionality, testing, spelling, grammar)
[ ] I have read the [Contribution Guide] and my PR follows them.
[ ] I updated my branch with the master branch.
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation about the functionality in the appropriate .md file
[ ] I have added in line documentation to the code I modified

Short description of what this PR does:

If you have questions, please send an email to Sendgrid, or file a Github Issue in this repository.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@leomongeg
We can’t merge your PR without the CLA being signed.
[ ] Sign the CLA before November 3rd
[ ] Tag @mbernier and @thinkingserious to let us know it is signed
Then we will comment with a link to get your shirt!

@leomongeg
We have not been able to merge your Pull Request, but because you are awesome - we wanted to make sure you could still get a SendGrid Hacktoberfest shirt.
Please go fill out our swag form before Nov 5th and we will send the shirt! (We know that you might have tried this before and it didn’t work, sorry about that!)
You have till Nov 5th to fill out this form in order to get the Hacktoberfest shirt!
Thank you for contributing during Hacktoberfest! We hope to see you in the repos soon! Just so you know, we always give away a SendGrid shirt for your first ever non-Hacktoberfest PR that gets merged.

Duplicates #61


Flex() make it easier to reverse child order

Use case
Given that Flutter is also used for web development, flex layout in native web technology have this "comprehensive" property values for the direction, for example in CSS we can do:
flex-direction: row equivalent with direction: Axis.horizontal
flex-direction: column equivalent with direction: Axis.vertical
But, how about these?
flex-direction: row-reverse
flex-direction: column-reverse
Let say I have code like below:
final _isPortrait = MediaQuery.of(context).orientation == Orientation.portrait;

Flex(
  direction: _isPortrait ? Axis.horizontal : Axis.vertical,
  children: <Widget>[
    Expanded(flex: _isPortrait ? 2 : 1, child: MyWidget1()),
    Expanded(flex: _isPortrait ? 1 : 0, child: MyWidget2()),
  ],
),

When orientation is portrait, of course the MyWidget1() will be on the left side as a first child.
However, when orientation is landscape, I want to swap the position of MyWidget1() and MyWidget2(), thus first child's position is now below of the second child. How to do that in Flutter?
Current solution: (Use more ternary)
Flex(
  direction: _isPortrait ? Axis.horizontal : Axis.vertical,
  children: <Widget>[
    Expanded(
      flex: _isPortrait ? 2 : 1,
      child: _isPortrait ? MyWidget1() : MyWidget2(),
    ),
    Expanded(
      flex: _isPortrait ? 1 : 0,
      child: _isPortrait ? MyWidget2() : MyWidget1(),
    ),
  ],
),

How if the child not just two, but three or more? Use additional "over-the-hood" script like _isPortrait   ? widgets : widgets.reversed.toList() is not good in my opinion.
Proposal
Why not add more property to make it cleaner and easier?
Flex(
  direction: _isPortrait ? Axis.horizontal : Axis.vertical,
  reverseOrder: !_isPortrait,
  children: <Widget>[
    Expanded(flex: _isPortrait ? 2 : 1, child: BottomButton1()),
    Expanded(flex: _isPortrait ? 1 : 0, child: BottomButton2()),
  ],
),

I can't wait for this!

Is there any workaround yet reversing the direction of Flex, Row or Column's children?

Is there any workaround yet reversing the direction of Flex, Row or Column's children?

Hi @bonnop. Try declaring children as a list of widgets final list = <Widget>[/* ... */]; and then, if needed, reverse the list: list.reversed 😀

list.reversed.toList() works but it causes issues with the focus traversal.

It would also be great to set the order of the child.


Extract Seeders and Leachers

I think it's written "leechers" and not "leachers": https://en.wikipedia.org/wiki/Glossary_of_BitTorrent_terms#Leech

You are completely right! It's even written that way on the kickass search result page :laughing:


Use initializable library inside other libraries

For example in upgrades library:
https://github.com/OpenZeppelin/cairo-contracts/blob/b764660da224ee757b831ea62bf7a4c63ee898fb/src/openzeppelin/upgrades/library.cairo#L44-L55
Wouldn't be better to import and using here the initializable library instead of rewriting the initialization logic? I think importing libraries from libraries is not against the Extensibility Pattern.

The reason I did not use Initializable was to isolate the initialization of the proxy from the implementation one. If it did, initializing the proxy would automatically initialize the implementation.

I don't get it, let me know what I'm missing:
This Proxy.initializer is meant to be called from the code of the implementation right? And the implementation is declared (but not deployed) so it can't be initialized. Initializing the proxy would use the implementation initializer through a library_call, and this initializer would use the Proxy.initializer from the library to set the proxy_admin. But the name of the storage variable to handle "already initialized" is not relevant right?
I'm suggesting something like this in the Proxy library initializer:
func initializer{syscall_ptr: felt*, pedersen_ptr: HashBuiltin*, range_check_ptr}(
     proxy_admin: felt
 ) {
     Initializable.initialize();

     _set_admin(proxy_admin);
     return ();
 }

But the name of the storage variable to handle "already initialized" is not relevant right

Yes it is. The storage address of a given storage variable is a function of its name, therefore all library calls to Initializable.initialize() would be sharing the Initializable_initialized storage variable which can only be set once. This is why it shouldn't be called from any other library initializer, otherwise it would stall the initialization process (if any initializer also checks for uninitialization).
That being said, I'm not being able to remember why not simply following the extensibility pattern and removing the initialization check completely, since the pattern expects the checks (if any) to be performed in the constructor as a whole, not in each initializer. Resulting in:
// "./openzeppelin/upgrades/library.cairo"

func initializer{syscall_ptr: felt*, pedersen_ptr: HashBuiltin*, range_check_ptr}(
     proxy_admin: felt
 ) {
     _set_admin(proxy_admin);
     return ();
 }

And users (if following the extensibility pattern) shouldn't call this function from another initializer, nor from another function aside the constructor, and just once.

I think importing libraries from libraries is not against the Extensibility Pattern.

All in all, is against the pattern to use initializers anywhere but in constructors, this includes (i.e. excludes :p) library modules. So you can import libraries, but you shouldn't call initializers.

Now makes sense. I wasn't considering that a contract could extend from different libraries simultaneously (of course).
With this said:

And users (if following the extensibility pattern) shouldn't call this function from another initializer, nor from another function aside the constructor, and just once.

In the UUPS upgradeable implementation version we are providing from the Wizard, we have an external initializer (acting as the constructor) which calls the Proxy.initializer inside. If we don't check initialization then the proxy pointing to this implementation could be re-initialized

That being said, I'm not being able to remember why not simply following the extensibility pattern and removing the initialization check completely

This could be the why. I understand why the proxy needs this design, but this goes against the extensibility pattern current definition, maybe we should add an entry for this exception in the docs?

Closing this since it will not be relevant anymore after the ongoing Cairo 1.0 migration. If you think this is a mistake, feel free to open a new issue.


in place for logistic regression

modify logistic regression to do in-place training. Update type to include beta coefficients and training to update these in place

Updated to include in-place training, but want to include standard errors for betas next


fix: update memo input component

Try out this version of Leather — Extension build, Test report

Re #4959 I've migrated memo field over to new input.

Visually it still differs from the others though. @fabric-8 do we have a new-input compatible version of the recipient input?

as it'd be nice to get this one migrated too. Otherwise, can we leave as is until further redesign work is finalised?

@kyranjamie "obvious" solution would be something like this where we align the label/select sizes, but we didn't harmonize those bits yet plus the wallet ux redesign project is starting this week, so maybe fine to hold off for now, merge and tackle later?

@kyranjamie Afterthought: After a short time of considering and looking into it, it might be fine to align things for now as we're really just adjusting input content / label sizes and spacing to align with one another — I did a quick round of adjustment here to unblock, nice small improvement might also be to add a :hover for the address/bns dropdown trigger + select address. LMK what you think!

Fixing #4959 was a little more involved than originally thought but this is a nice refactor as we replace TextInputField. This component was a bit of a mess. It was a named as a generic input component, yet had use-case specific logic in it for the send form.
@fabric-8 to design review.
Btc form

Btc custom fee input:

Btc increase fee form

Nice! Minor thing I noticed: The dropdown popover items are now too small which looks like an accident — Can we render this one the same way as the one for the fee below?

TODO: will address Fab's comment by replacing with new select component


Intelligent default column mapping

I think it'd be nice to map a struct field named
FooBarId to a column foo_bar_id
Would you be open to either a) changing the default mapping scheme or b) opening this up via some flag/api/configuration?

This would be realy usefull. Gorp could ship with a CamelCaseMapper and an UnderscoreMapper, and anyone could implement a default mapper.


Rest API: Download: Diagnosis keys newer then download date are not downloaded

Describe the bug
When you set in rest API "download" a date in the past, diagnosis keys newer then the set date are not downloaded. Only the diagnosis keys of the set date are downloaded.
Expected behaviour
When you download data for date X, all diagnosis keys of date X and newer should be downloaded.
See document: 20201408_EU_Interop_Architecture_v1.3_draft.pdf (chapter: download interface):
"diagnosis keys newer than {date} will be downloaded
Steps to reproduce the issue
Example:
01.09:  100 diagnosis keys uploaded by other countries
02.09:  500 diagnosis keys uploaded by other countries
03.09:  200 diagnosis keys uploaded by other countries
When you download data for 02.09, only the 500 keys are downloaded.
When you download data for 02.09, 500+200 = 700 keys should be downloaded.

Pls review, whether this was already fixed as side effect of other fix

eu.interop.federationgateway.service.DiagnosisKeyBatchService#getFirstBatchTagOfTheDay
--> remove end variable
eu/interop/federationgateway/controller/DownloadController.java:162
--> change check to proof that batch date is gte to requested date
eu.interop.federationgateway.service.DiagnosisKeyBatchService#getFirstBatchTagOfTheDay
--> remove end variable (just get the last entity with batchLink is null
--> if no entity is present generate the first batchTag
eu/interop/federationgateway/service/TransactionalDiagnosisKeyBatchService.java:112
--> move the setBatchLink call outside of the if-block so it will be always set.

Our first analysis shows conflicting interpretation of the specification

"4.2.9 Document Batch Tag" -->   it’s used to “navigate” within the day
"5.3.1" Download API spec mentions  --> diagnosis keys newer than {date} will be downloaded

The proposal to be verified would be:
For /download/date/{date}, if no batchtag is provided, find the first batch not older than date
For /download/date/{date}, if batchtag is provided, find the first batch not older than date having the batchtag

When first connecting to the EFGS, will this pseudo-ish code provide me with all available keysets?
date = today - 14-days
batchTag = null
while(batch = efgs.download(date, batchTag) != null)
    (nextDate, nextBatchTag) = process(batch)
    date = nextDate
    batchTag = nextBatchTag

How would this work when the date is today and you're polling every, say, hour, to see if there are new batches? You can't link to a non-existing batch the API doesn't accept a time..


A few changes

Fix the identity bug.
(The ZMQ identity has bytes outside the ASCII range, in the old code it will crash silently)
Fix notebook restart as a result of above.
Add  interrupt signal handler to interrupt eval
Add basic syntax highlight by adding the [:language_info :name] field in the kernel_info_reply
Change is_complete_reply to use parsely instead (fix the @ deference bug)
Remove cider middlewares from the code and import them instead
Add basic incanter plotting integration (PNG plots), inspired by gorilla-repl
Add stdin channel  for input
Add control channel handler to handle kernel_shutdown request
Add basic completion using the cider completion middleware

Wow! This sounds great!

Wow, thank you!!

Thank you so much, this is awesome. I gave you push access to the repository as well.


Palette buttons need to group into (at least) lists

The first list of buttons should be the cross outs and calc buttons. The next group should be the rest. The layout shouldn't change though.
To be specific, it should be something like

		  <button class="staticMath btn btn-default paletteButton" data-toggle="tooltip" title="Cross out (⌨: shift+delete)" onclick="MathLivePasteFromButton(this); GoogleAnalytics('Cross Out');">$$\enclose{updiagonalstrike downdiagonalstrike}[2px solid red]{\blacksquare}$$</button>
		  <button class="btn btn-default paletteButton" data-toggle="tooltip" title="Calculate (⌨: ctrl+=)" onclick="CalculateAndReplace(this); GoogleAnalytics('Calculate');">Calc</button>
		  <button class="staticMath btn btn-default paletteButton" data-toggle="tooltip" title="Cross out & replace (⌨: shift+ctrl+delete)" onclick="MathLivePasteFromButton(this); GoogleAnalytics('Cross Out & Replace');">$$ \enclose{updiagonalstrike downdiagonalstrike}[2px solid red]{\blacksquare}^{\square} $$</button>

		     ...

But this messes up the layout of the buttons.
We may want to break those into the structure buttons (fractions, etc) and operator buttons, but let's leave that for latter.

sequencing for palette:

editing - cross out/replace/calc
operations & symbols - combining first and third set (anything without a white/black box)
notations (stuff with the boxes)

completed.  Order of buttons might need to change.


ci: check docs spelling

Motivation
Catch spelling mistakes before merging
Description
Run PySpelling in CI
Closes https://github.com/ChainSafe/lodestar/issues/1829

Run scripts/wordlist_sort_check.sh
sort: .wordlist.txt:2: disorder: APIs
Error: The wordlist is not sorted.

https://github.com/ChainSafe/lodestar/actions/runs/5112160804/jobs/9189892314?pr=5527#step:9:6

Yeah I'm not sure why it breaks in CI. Running sort locally does not error. I tried different sorting algorithms but the one CI always disagrees :(

@dapplion got this over the hump.  Please check that the configuration works for our needs

The sort seems to be working correctly for me.
This is the run from https://github.com/ChainSafe/lodestar/pull/5527/commits/ca261a216cfd2722803efa50af62431050bc3a1d
https://github.com/ChainSafe/lodestar/actions/runs/5318818176/jobs/9630659354?pr=5527

big thank you @matthewkeil for helping deliver this!

:tada: This PR is included in v1.10.0 :tada:


Correct return and arg types for Stream::pipe

Stream::pipe both takes a Writable as its destination and returns that Writable so the call to pipe can be chained.
Fixes #906

looks right to me, thanks

@facebook-github-bot shipit


Allow for modifying Skia PlatformRenderInterface behavior.

What does the pull request do?

Allows the user to change behavior of Skia rendering platform. I need this for my project as I want to control creation of GrContext and render target creation.
What is the current behavior?

While I can technically change the used rendering platform I cannot just modify a small part of the current PlatformRenderInterface without rewriting it from scratch.
What is the updated/expected behavior with this PR?

User can now override platform behavior.
How was the solution implemented (if it's not obvious)?

Checklist

[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Avaloniaui.net with user documentation

Breaking changes

Fixed issues

@kekekeks

I'm not sure that we want to make these methods to be the part of the public API. I don't know why PlatformRenderInterface class is public to begin with.
It you want to control GrContext and render target creation, I think we should add some kind of extensibility interface. Something like:
interface ICustomSkiaGpu
{
      GrContext CreateImmediateUiThreadGrContext();
      ICustomSkiaRenderTarget TryCreateRenderTarget(IEnumerable<object> surfaces);
}
interface ICustomSkiaRenderTarget : IDisposable
{
      ICustomSkiaRenderSession BeginRendering();
}
interface ICustomSkiaRenderSession : IDisposable
{
      GrContext GrContext {get;}
      double ScaleFactor {get;}
      SkCanvas Canvas {get;}
}

Also see https://github.com/AvaloniaUI/Avalonia/issues/2365 I think it should be specified by
    .With(new SkiaOptions { CustomGpuFactory = () => new MyCustomGpu() });

@kekekeks I thought about this approach but then I noticed that some parts of Skia are public so I figured I can do it with lower effort :)
I have few questions about your design:

why would we have multiple GrContext as implied by ImmediateUiThreadGrContext ?
who is responsible for calling ICustomSkiaRenderTarget .BeginRendering? Should I implement some proxy CustomRenderTarget that implements IRenderTarget and adapts that interface?

Currently we have 2 OpenGL contexts with shared resources, one is used for UI thread rendering (for RenderTargetBitmap basically), another one is used by render thread for actual rendering the render targets. Those need to be wrapped into different GrContext objects (right now we are reusing a single one, I think, which isn't a good thing but currently doesn't cause issues in most cases since RenderTargetBitmap usage is rare.

who is responsible for calling ICustomSkiaRenderTarget .BeginRendering? Should I implement some proxy CustomRenderTarget that implements IRenderTarget and adapts that interface?

Yes, something like that. See GlRenderTarget

Thanks for info. I also forgot to ask how is ScaleFactor supposed to be used, who provides that?

It represents the DPI of the target surface.

@kekekeks Implemented your feedback, I can cleanup this code later if you are fine with this implementation.
Additionally I am trying to figure what should be the public API of the Avalonia.Skia assembly?
I would need some way of getting SKCanvas to draw to (using the https://github.com/AvaloniaUI/Avalonia/pull/2185), as I have some custom SKSurface that I would like to draw to the SKCanvas. Since to actually implement something drawable from Skia I need to implement IDrawableBitmapImpl which is internal to the Skia assembly.

Isn't it possible to work with an IBitmap implementation if you need access to the SKCanvas(SKSurface)?

You need IDrawableBitmapImpl to actually make it work with Skia. Which is an internal interface.

I think we should introduce a abstract base class for this and also use SKImage by default when it is possible. Only WritableBitmapImpl needs to use SKBitmap if I am not mistaken.
public abstract class BitmapImpl : IBitmapImpl
{
    public abstract SKImage GetSkiaImage();
}

For me I actually have SKSurface and not an SKImage. I could snapshot the surface to an image but that might cause unwanted copy. Since ideally I would like to have gpu -> gpu copy. I will take a look at the API to see what could be abstracted.

When you create your surface on GPU Snapshot should also result on GPU

@kekekeks Are you fine with this implementation? I also did some cleanup on Skia image classes, and made more classes internal so it is clearer what should be used from the outside. If you are fine I will add comments and clean it up a bit.

@kekekeks I trimmed down this PR to just custom gpu changes for now. Let me know if you want more changes.

This needs to be refactored to use new "options" way of configuring platforms - https://github.com/AvaloniaUI/Avalonia/pull/2368

@kekekeks Implemented platform options. Let me know if you need more changes in this PR.

Whoops, not approving. Options are implemented in a wrong way

@kekekeks Can you be a bit more specific? My implementation looks the same as Win32 and X11 ones unless I am missing something.

I've misinterpreted the code because of this: https://github.com/AvaloniaUI/Avalonia/pull/2364/files#diff-82af421f9bf2b29fd90e5f6c8823dcafR16. Sorry, a bit sick today.
Will merge once CI passes (Mono has segfaulted on Avalonia.Visuals tests

@grokys @kekekeks No idea why CI is failing, seems that a lot of other PRs also have this segfault issue on Linux.


Command Bar control text scaling issue

Regression: Control Center footer add/remove buttons in edit mode look bad with text scaling
See item below:
https://microsoft.visualstudio.com/OS/_workitems/edit/26126908?src=WorkItemMention&src-action=artifact_link

Closing because this is by design behavior.


Actions that first require login should continue after a successful login

Describe the bug
If a user is attempting to perform an action that requires a login session (refreshing list of available Xcodes, installing a new Xcode) they will be presented with a login sheet. After successfully logging in though the action that was previously being performed does not resume.
To Reproduce

Start with a logged out session
Tap install on any Xcode that is not currently installed
You should be prompted to sign in
Provide valid sign in credentials

Expected behavior
The install process starts
Actual behaviour
The install process is cancelled/interrupted by the login flow
Version

OS: 11.2.3
Xcodes: 1.0.5 (6)

@aerickson14

Bug

As of v1.8 we no longer require apple accounts to download Xcode so these issues won't show up anymore 🥳


New column format is not DRY

Going through the code, I realized this column format is deprecated:
class MyDatatable(Datatable):
    class Meta:
        columns = [
            ("ID", 'id', helpers.link_to_model),
            ("Blog", 'blog__name', helpers.link_to_model(key=lambda instance: instance.blog)),
            ("Headline", 'headline', helpers.make_xeditable),
            ("Body Text", 'body_text', helpers.itemgetter(slice(0, 30))),
            ("Publication Date", 'pub_date', helpers.format_date('%A, %b %d, %Y')),
            ("Modified Date", 'mod_date'),
            ("Age", 'pub_date', helpers.through_filter(timesince)),
            ("Interaction", 'get_interaction_total', helpers.make_boolean_checkmark),
            ("Comments", 'n_comments', helpers.format("{0:,}")),
            ("Pingbacks", 'n_pingbacks', helpers.format("{0:,}")),
        ]

in favor of:
class MyDatatable(Datatable):
    class Meta:
        columns = ['id', 'blog_name', 'headline', 'body_text', 'pub_date', 'mod_date', 'age',
                   'interaction', 'n_comments', 'n_pingbacks']
        processors = {
            'id': helpers.link_to_model,
            'blog_name': helpers.link_to_model(key=lambda obj: obj.blog),
            'headline': helpers.make_xeditable,
            'body_text': helpers.itemgetter(slice(0, 30)),
            'pub_date': helpers.format_date('%A, %b %d, %Y'),
            'n_comments': helpers.format("{0:,}"),
            'n_pingbacks': helpers.format("{0:,}"),
        }

I like the older format better not only because it is more DRY (you don't have to repeat field names here and there, but also you have all the settings for a field in one place, which makes code easier to read.
It is also possible to have all settings of a column/field in one place by explicitly defining it in the form of: blog_name = columns.TextColumn(... but that is verbose.

In this situation, I opted to follow the django Forms strategy, which I agree can be repetitive in cases like this.  Also, the old format ultimately made the declaration of virtualized columns more awkward than I wanted.  For example, I know the difference between the following, but who else does:

("My Column", 'get_my_value')
("My Column", None, 'get_my_value')

Also, it was annoying in my experience to declare a field with a processor callback where I didn't want to copy-paste the verbose name from the model into the config list.  The following was not valid:

(None, 'blog_name', 'my_processor')

So the new column "field" syntax was made, as you mention at the end.  At that point, it felt redundant to support so many different ways of doing something, which is usually not the python way, particularly when I felt the old syntax was so lacking.
At this point I'm fond of the imitation of django.forms, and I don't know if there's a compromise available anymore.

It is usually good to do what others do :) (although I don't like django's syntax either, if you had a lot of fields, it would become a puzzle to read; esp. if the code was not organized well)


Can't npm install grpc on raspberry pi 2

I tried running npm install grpc and I get this error when the install process says its going to compile with node-gyp
Any help?

We do not currently distribute ARM packages, and I don't know if the gRPC Node package builds on ARM. There should be more error output in above what's shown in your screenshot. If it's failing because you don't have a C compiler, you might be able to get a compiler/linker/etc. package and fix the problem. If it's failing because of a build error, you're probably out of luck.

We have added experimental support for ARM processors (which Raspberry Pi uses) in gRPC 1.4.1. You may be able to install it on your Raspberry Pi. Please tell us if you have any further problems.


Design first Kariba Data Challenge

We should start working out our playbook as we do it.
Should be announced in all the channels, including starting our first newsletter

https://docs.google.com/document/d/11e6450FHaJVnzSkdCsClFFF_F7iznnUT6XYDoR_ADPs/edit?usp=sharing

We'll probably split this into multiple (~2) challenges across April / May based on available datasets


OCaml syntax hilighting bug

(Original issue 746 created by None on 2012-03-10T14:39:35.113871+00:00)
Hello, I would like to report a ocaml highlighter bug.
In the following code: http://pygments.org/demo/35677/
everything after () is treated as a comment (since comments in OCaml start with `(', however, in this case it is a multiplication operator used as a function.

(Original issue 746 last updated on 2012-07-10T08:31:37.774105+00:00)
(Issue automaticaly closed due to status in Bitbucket: resolved)

(Original comment by birkenfeld on 2012-03-10T17:54:11.779931+00:00)
Fixes #746: OCaml allows (*) as an operator, not comment start.
→ <>


Implement autocomplete API for PostgreSQL

This PR finishes the implementation of the autocomplete API (RFC 27) for PostgreSQL.

The new test for autocomplete with a fields argument (which is necessary to provide test coverage of PostgresAutocompleteQueryCompiler.get_fields_vector) is now failing on Elasticsearch:
self.backend.autocomplete("Georg", models.Author, fields=['name'])

(there are also various failures of test_get_mapping / test_get_document which are expected, given the model change I made to add index.AutocompleteField('name') on Author)
Haven't had chance to dig into this in detail, but I think it might be due to this line where an explicit fields mapping takes precedence over the one that tells it to use edgengrams:
https://github.com/wagtail/wagtail/blob/1958bba2476c32069acafd6837514749a4d3ceea/wagtail/search/backends/elasticsearch2.py#L623
@kaedroho, can you advise on whether passing fields to autocomplete is something that we'd want to support, or is that not feasible on Elasticsearch? If not, I guess the best thing to do is leave the fixed version of PostgresAutocompleteQueryCompiler.get_fields_vector in place, but leave it untested (or make that a Postgres-specific test).

It is feasible on elasticsearch, we could index the autocomplete twice. Both in the _edgengrams field but also as each field individually. But I don't think we should support this, I can't think of a single moment I ever needed to do that (usually autocomplete is only added to the title anyway).

Latest changes look good to me, thanks!

Thanks! Haven't added any release note / changelog entry for this, as this is still an undocumented API - we should probably plan for making it official once the dust has settled.


[Snyk] Fix for 1 vulnerabilities

This PR was automatically created by Snyk using the credentials of a real user.![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)
Snyk has created this PR to fix 1 vulnerabilities in the npm dependencies of this project.
Snyk changed the following file(s):

frontend/package.json

Vulnerabilities that will be fixed with an upgrade:

Issue
Score

Missing Release of Resource after Effective Lifetime SNYK-JS-INFLIGHT-6095116
  141

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

Checkmarx One – Scan Summary & Details – bce4e02b-6f39-4a93-ae14-2d29b1ac4427
New Issues

Severity
Issue
Source File / Package
Checkmarx Insight

CVE-2024-37890
Npm-ws-7.4.6
Vulnerable Package

CVE-2024-38355
Npm-socket.io-3.1.2
Vulnerable Package


Missing swipeToClosePercent prop

There is a swipeToOpenPercent prop, which allows you to control how far you need to swipe to open the row, but there is no similar functionality for closing the row. I believe it should either be a separate prop or the existing prop should affect both opening and closing behavior.

Any updates on this issue?

There is an open PR ( #148 ). Should be merged in and released very soon

This has been fixed by #148 and is in the latest release (v0.4.6)


DefaultPropagationMode = "rprivate" is probably wrong for bind-mounts

https://github.com/containerd/nerdctl/blob/5b52e3b2523647c13d41723cb499dbe1b520971a/pkg/mountutil/mountutil_linux.go#L71
sudo docker run -it --rm -v /:/mnt alpine seems to propagate mount events from the host.
DefaultPropagationMode = "rprivate" is probably wrong for bind-mounts.

cc @ktock

https://github.com/moby/moby/pull/36055 seems related.
Docker uses rslave if the source dir of the bind-mount contains /var/lib/docker.
But nerdctl doesn't do this because it doesn't know what directory is the containerd root directory.


ARMv6 packages missing for Go 1.6.3

It appears that no packages were publishes for ARMv6 for Go 1.6.3. Checking the download page; https://golang.org/dl/, and https://storage.googleapis.com/golang, there are packages for Go 1.6.2, but not for 1.6.3
Are there plans to provide packages, or should we build from source?
Relates to https://github.com/docker/docker/pull/24811

@broady Can we build ARMv6 binaries?

We can. He just didn't notice some error earlier and forgot to upload them. cmd/release needs a summary at the end to say which ports built.

They're up there. I'm building 1.7rc2 binaries too.

Thanks for the quick response, really appreciated :heart:

@broady glad to see your fast response to resolve this issue! Well done. 👍


How to organize mixed (C/Acton) modules

Modules are typically written 100% in Acton (just a .act file) or 100% in C, which then involve a .act file that is just stub compiled for getting the types.
Sometimes we have the need for writing some functions in a module in C but the majority in Acton. How? Should we mix C and Acton in modules?
Do we place the C code in a separate module with some specific naming convention? For example, for the time module, it could be:

time.act - majority of things, written in Acton
_time.(c|h|act) C stuff, like now() which is then wrapped in more thing in time.act

Discussed with @nordlander. Mixing within a module will be very tricky. Let's stick to either C or Acton for one module, which means that "mixed modules" actually need to become two modules. We'll go with foo and _foo (C) for now. It could be nice to limit visibility of the C module, somehow making it a private sub-module of the Acton module, but we don't know how that sort of scheme would look, so we're not trying to fix that right now. Thus, _foo (or _time) will be visible to all but really shouldn't be used. Users will need to do the right thing.
This exact scheme is implemented for the time module in #599.


feat(mesh): add sponsorship page

Notable Change

implement sponsorship flow through payment/sponsor/[publisherId] route figma
use reward api to top up points for testing transactions doc

TODO:

debug sendTransaction. Currently, users need to refresh the page before click transaction button.

To see the specific tasks where the Asana app for GitHub is being used, see below:

https://app.asana.com/0/0/1208199866131045

可以考慮在選擇 payment/sponsor 確定有登入錢包後就先拿 user 的 balance 以 block user 餘額不足的 request (看打我們在機的 gql 貴還是打 alchemy 貴)

改為登入狀態下有 address 的 user, 直接在 Page 層 getBalance 0db2cf6


[Chart] The meshery chart need to bump the version on it reame

Current State:
https://artifacthub.io/packages/helm/meshery/meshery
Desired State:
The home page shows our latest version 2.1.2
We can use the script https://github.com/layer5io/meshery/blob/244d7ab685c92484f31e88909a046e583f83b580/Makefile#L143

@zjyingchang Cloud you please take this one? Finish this one before doing the issue https://github.com/layer5io/meshery.io/issues/391

@zjyingchang Cloud you please take this one? Finish this one before doing the issue https://github.com/layer5io/meshery.io/issues/391


Gateway timeout after updating Docker service

Do you want to request a feature or report a bug?
Bug
What did you do?
Our build process automatically deploys the latest build to our test environment. This is performed via docker stack deploy. The site's configuration is handled via Docker labels. After the update, we are unable to access the site anymore. Instead, we receive a "gateway timeout" error in the web browser. If we then stop the container, the swarm automatically restarts it and the site begins to work. We only just recently began experiencing this behavior, and it only happened after we updated to Traefik 1.3.1 (from 1.2.3).

What did you expect to see?
The site should work immediately after deploy.
What did you see instead?
A "gateway timeout" error message in the web browser.
Output of traefik version: (What version of Traefik are you using?)

Note: SSL certificate names have been replace with "somecert" for privacy. The acme email has also been replaced.
defaultEntryPoints = ["http", "https"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
        CertFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.crt"
        KeyFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.key"
      [[entryPoints.https.tls.certificates]]
        CertFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.crt"
        KeyFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.key"
      [[entryPoints.https.tls.certificates]]
        CertFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.crt"
        KeyFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.key"
      [[entryPoints.https.tls.certificates]]
        CertFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.crt"
        KeyFile = "/etc/ssl/certs/deployed/wildcard.somecert.com.key"

[file]
  watch = true

[acme]
  email = "someemail"
  storage = "acme.json"
  storageFile = "acme.json"
#  onDemand = true
  onHostRule = true
  entryPoint = "https"
#  caServer = "https://acme-staging.api.letsencrypt.org/directory"

[docker]
  exposedbydefault = false

Args from docker service inspect traefik:
"--docker",
"--docker.swarmmode",
"--docker.domain=traefik",
"--docker.watch",
"--web"

Traefik labels on the Docker service:
traefik.docker.network=traefik-net-no-tls
traefik.frontend.passHostHeader=true
traefik.frontend.rule=Host:somesite.com
traefik.port=80

If applicable, please paste the log output in debug mode (--debug switch)
level=warning msg="Error forwarding to http://<IP_ADDRESS>:80, err: dial tcp <IP_ADDRESS>:80: getsockopt: connection refused"

Hello @nathantreid .
Did you do any changes in the Traefik configuration?
Can you give some debug logs (by activating logLevel=DEBUG in the Traefik configuration).

No configuration changes were made. I get this error when attempting to load the site:
level=warning msg="Error forwarding to http://<IP_ADDRESS>:80, err: dial tcp <IP_ADDRESS>:80: getsockopt: connection refused"

@nathantreid  Usually, this kind of problems appear when Traefik and the backend are not on the same Docker network and/or when the port to contact on the backend is not expose and/or backend has not to be exposed by Traefik.
Moreover, I saw you activate the  exposedbydefault = false option but I didn't see the label traefik.enable=false on you service. Is it missing?
Can you inspect the Traefik network in the way to see if all your services are present?

I am also getting the same issue on docker without swarm mode. Using traefik version v1.3.4
In my case, the back end container is on 2 networks,test_default created by docker-compose and traefik_webgateway which traefik is started on. Network details (from docker inspect) for the backend container is as follows:
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "bd04272ac7eb24b0a6436b91938d9c0f2dcc756b34bbce2df3d98e465cbf3d28",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "80/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/bd04272ac7eb",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "test_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "efb3e4944a0d",
                        "wordpress"
                    ],
                    "NetworkID": "756f40110595ef0006ec6edfe5a1c4253a80d6bece533a337f9807efd616a9b3",
                    "EndpointID": "6d8956b6402167675ea9b1812a2600cc19d6431a6e2672929abfedb34cfa7df3",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:13:00:04",
                    "DriverOpts": null
                },
                "traefik_webgateway": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "efb3e4944a0d",
                        "wordpress"
                    ],
                    "NetworkID": "1408dff2e4855b823057d5d8c057b889834d42b18dcb60cb60c5a0577d547f0d",
                    "EndpointID": "563d1e8892a51ac792e65434b19538d7434f06d16a653766d700c57814205616",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:12:00:03",
                    "DriverOpts": null
                }
            }
        }

Usually when updating the service or when re-creating it, traefik often picks up the wrong ip ie it uses the ip from test_default instead of the ip from traefik_webgateway and usually ends up giving a gateway timeout and never updates to the correct backend ip.
It kind of happens at random I haven't found a sure shot way to reproduce or work around this issue. After Deleting/Re-Creating the service multiple times traefik eventually (randomly??) picks the ip from the right network. How does traefik decide which network ip to use as of now??

I'll close the issue due to inactivity but feel free to re-open if needed.

Thanks @nmengin - it saved my day :) I had exposedbydefault = false from one of examples but did not remember that (and I'm pretty new too) so I was tearing my hair because of gateway timeout.


Migrate away from rustc-serialize

Since rustc-serialize has been deprecated, it seems like it would be preferable to move to other libraries that provide the functionality currently used.

There were the following issues with your Pull Request

Commit: dcc0f3dcc2527fa16f3fc92079294997164c1c2a

Commits must be in the following format: %{type}(%{scope}): %{description}

Commit: 7bf653f5d16c6de245145b65de2c2e824aa8c6ea

Commits must be in the following format: %{type}(%{scope}): %{description}

Guidelines are available at https://github.com/lettre/lettre/blob/master/CONTRIBUTING.md

This message was auto-generated by https://gitcop.com

There were the following issues with your Pull Request

Commit: dcc0f3dcc2527fa16f3fc92079294997164c1c2a

Commits must be in the following format: %{type}(%{scope}): %{description}

Commit: 7bf653f5d16c6de245145b65de2c2e824aa8c6ea

Commits must be in the following format: %{type}(%{scope}): %{description}

Guidelines are available at https://github.com/lettre/lettre/blob/master/CONTRIBUTING.md

This message was auto-generated by https://gitcop.com


Fix issues 26 and 31

Fixes:

Template parse errors in Angular 2 (https://github.com/dgarlitt/karma-nyan-reporter/issues/26)
Reset numbering of tests on each re-run (https://github.com/dgarlitt/karma-nyan-reporter/issues/31)

Coverage decreased (-0.6%) to 99.443% when pulling bddf4a0f0c3a45258c19e2c8c602f6c06ded2d9a on whyboris:issues26and31 into 68da8261defe424c4ecf7805034f2de778c06d96 on dgarlitt:master.

Coverage remained the same at 100.0% when pulling 8531d20fa17d9cc307e1fa854815c65f8337fbbb on whyboris:issues26and31 into 68da8261defe424c4ecf7805034f2de778c06d96 on dgarlitt:master.

Coverage remained the same at 100.0% when pulling fb7f0dff3388f20ea83454bd27015a7125b92197 on whyboris:issues26and31 into 68da8261defe424c4ecf7805034f2de778c06d96 on dgarlitt:master.


Default Theme: Add a comment that declares the background color of operator tokens as intentional

Since people seem to somewhat regularly report and fix the background color of operator tokens in the default theme, I added a little comment to clarify that it's indeed intentional.
NGL, when I first saw Prism, I also thought it's a bug.

Hey, is there's a reason for this intentional background statement?

The styles are entirely up to the author of the theme. They probably like the light emphasis the background color puts on operators.


Very large VE may push WAL message beyond size limit

WAL limits messages to 1MB.
Very large VE will push PreCommit messages beyond this limit.
This behavior should be captured in a specific UT and documented in the ABCI spec.

Should we consider upper bounding the side of VE to a (common sense) limit, eg 1MB? Comet is not a video streaming service so not sure what use-case could require large VE.

If we decide to bound the size of VE, then it should probably be a consensus param (similar to the block size), with a hard-coded common-sense maximum size, like the block size

What is the behavior is a node tries to push into WAL a message that is > 1MB? Trying to assess how severe this problem may be.

Here is where it is tested
https://github.com/cometbft/cometbft/blob/c3f7ea35c12ff0ec3fc67b9f686a0da566dad73f/internal/consensus/wal.go#L319

What is the behavior if a node tries to push into WAL a message that is > 1MB? Trying to assess how severe this problem may be.

This is actually curious. If it received from a peer, then log and go ahead. If from itself, then panic

This is actually curious. If it received from a peer, then log and go ahead. If from itself, then panic

We write asynchronously messages from peers, we fsync the WAL when we write or own message. The rationale here is probably that peers can send us invalid message, and we shouldn't break because of that, but we shouldn't broadcast (send to ourselves) invalid messages, so we panic.

What is the behavior if a node tries to push into WAL a message that is > 1MB?

This is definitely problematic. While we should consider not writing blocks (split into parts or not) to the WAL, there is no possibility of non writing votes to the WAL. One possibility to consider is to not write vote extensions to the WAL. In particular because I don't see the use case for replaying vote extensions...

This is just to add more context to the problem.
As Daniel said the problem is that with the tendermint/wal/MsgInfo written to the WAL which might be bigger than the allowed msg size in the WAL.
Currently the WAL max message size is maxMsgSizeBytes = maxMsgSize + 24 in wal.go, where maxMsgSize = 1048576 (~1MB) in consensus/reactor
A MsgInfo on a WAL (converted to JSON) could looks like this:
{"time":"2024-04-08T21:05:11.089733Z","msg":{"type":"tendermint/wal/MsgInfo","value":{"msg":{"type":"tendermint/BlockPart","value":{"Height":"2","Round":0,"Part":{"index":0,"bytes":"0A80030A04080B10011211746573742D636861696E2D5237466E4E391802220B0886B8D1B00610B8859A232A480A20798D106E907255899C4DB68E6D7B0A1FB485E1698D28BDC12C1260FB739A108A12240801122049B359831AEEF15FDB179C4833F8750F41C685B68566D57065B61C1DA81066BB322081E6467AA5053DC485A10B1D50EDE4FF7ED5ADE86B3514A1A189047F8D7D59573A20E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B8554220BFC08C9E365339B52B02BA79F366C5E4C26B544D82E94811242CA49B1B16D82E4A20BFC08C9E365339B52B02BA79F366C5E4C26B544D82E94811242CA49B1B16D82E522068ECD6F333119CE43751ECE583B981F23508AEAF4221FF582B1BB33BE42BCEFA5A0800000000000000006220E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B8556A20E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B85572144C1949EDFEC2EB7AE430D50FB16706D84B27793E12001A0022B50108011A480A20798D106E907255899C4DB68E6D7B0A1FB485E1698D28BDC12C1260FB739A108A12240801122049B359831AEEF15FDB179C4833F8750F41C685B68566D57065B61C1DA81066BB2267080212144C1949EDFEC2EB7AE430D50FB16706D84B27793E1A0B0886B8D1B00610B8859A232240EE2CDDA334425AED705FFCA422007ADBF3442FEB5F00CBD2ABB87885D2B7AA6E7111E99CB5E80AEB0F1F5A2C2605E36908D1AD198C604C2539F8264D339AE806","proof":{"total":"1","index":"0","leaf_hash":"Or35gnue5Jn8uNxu0m8pxR8X2/THmec/feyufZDBqGY="}}}},"peer_key":"","receive_time":"0001-01-01T00:00:00Z"}}}```

So after some investigation, there are two methods in the WAL, Write and WriteAsync. In WriteAsync internally it invokes Write. Then in the Write method it invokes Encode.  In the Encode message it will the consensus message is converted to its equivalent proto message (NewValidBlock, BlockPart, VoteMessage, etc).
Then in the encode there's a check for the length of the message to see if it is bigger than maxMsgSizeBytes
	data, err := proto.Marshal(&pv)
	if err != nil {
		panic(fmt.Errorf("encode timed wall message failure: %w", err))
	}

	crc := crc32.Checksum(data, crc32c)
	length := uint32(len(data))
	if length > maxMsgSizeBytes {
		return fmt.Errorf("msg is too big: %d bytes, max: %d bytes", length, maxMsgSizeBytes)
	}

if so, an error is returned. In most places where the error from Write is returned, just an error is logged and the logic continues. But in the WriteSync (as Lasaro) pointed out, it will panic
	case mi = <-cs.internalMsgQueue:
			err := cs.wal.WriteSync(mi) // NOTE: fsync
			if err != nil {
				panic(fmt.Sprintf(
					"failed to write %v msg to consensus WAL due to %v; check your file system and restart the node",
					mi, err,
				))
			}

while in the case of a peer message it would just log the error and continue
	case mi = <-cs.peerMsgQueue:
			if err := cs.wal.Write(mi); err != nil {
				cs.Logger.Error("failed writing to WAL", "err", err)
			}
			// handles proposals, block parts, votes
			// may generate internal events (votes, complete proposals, 2/3 majorities)
			cs.handleMsg(mi)

So we need to ensure that the

Don't panic in WriteSync just return an error (what are the implications?)

We can proceed (namely, broadcast and process a message) if we are not able to sync the WAL. Otherwise, when recovering from a crash, we will "forget" that we sent that message (and that we received all messages preceding it, and therefore being a precondition for its production).

Thanks for the feedback @cason. So if it's possible to change the logic and not write the vote extensions to the WAL and no major implication would occur that might solve this problem as I understand.
But the issue related with the size of vote extensions would still persist from a p2p perspective right ? Does the p2p.max_packet_msg_payload_size parameter ensures the ve size is capped from a P2P message perspective ?

Does the p2p.max_packet_msg_payload_size parameter

No, messages are split into packets. A message can take multiple packets. So this is not a problem. Recall that we ship application snapshots, blocks, etc., which are way larger.

So if it's possible to change the logic and not write the vote extensions to the WAL and no major implication would occur

I am not 100% sure about that. @sergio-mena is the best to ask this question.

Short answer: it MAY be possible, but it won't be a simple solution.
Long answer: As vote extensions are part of the precommit message, if we don't save them to the WAL, the recovered consensus instance will be missing them then we replay all consensus messages upon recovery.
This problem is similar to the one discussed in RFC100 for nodes that are block-syncing, so a similar solution may exist.

A good solution is far from simple. The main point is that vote messages were assumed to be fixed size, but now we add the ability of making them a variable-size message type. This has impacts in many ways, such as vote propagation, timeouts, and persistence.
Can we find a temporarily workaround, such as adding a maximum size for vote extensions?

Thanks @cason, I'm assuming this parameter would be a Consensus Parameter right ?

It could only be hard-coded, for now, based on the maximum WAL message size.

Remember that, in the current design, it is up to the application (typically as part of VerifyVoteExtension) to set (and check) the maximum size of vote extensions

This seems to be an application concern, I'll leave this open for future reference but feel free to close in case there's no immediate action.


PIPENV_ACTIVE gets set when starting tmux

Issue description
When I activate tmux, everytime I try pipenv shell I get Shell for UNKNOWN_VIRTUAL_ENVIRONMENT already activated.  No action taken to avoid nested environments. And the environmental variable PIPENV_ACTIVE gets set
Expected result
pipenv shell completes successfully, PIPENV_ACTIVE not set
Actual result
Before entering tmux, the environmental variable PIPENV_ACTIVE is not set, after starting tmux, and when creating new tmux windows, PIPENV_ACTIVE gets set to 1.  I tried setting PIPENV_ACTIVE=0 and I still can't run pipenv shell, however when I unset PIPENV_ACTIVE then I can run pipenv shell
Steps to replicate
(Not exactly sure)
Run tmux on Mac OS Catalina (10.15.16) with pipenv
tmux config:
setw -g mode-keys vi
unbind C-b
set-option -g prefix C-s
bind-key C-s send-prefix

bind | split-window -h
bind - split-window -v
unbind '"'
unbind %

bind P paste-buffer
bind-key -T copy-mode-vi v send-keys -X begin-selection
bind-key -t vi-copy y copy-pipe "reattach-to-user-namespace pbcopy"

bind r source-file ~/.tmux.conf

set -g mouse on

Please run $ pipenv --support, and paste the results here. Don't put backticks (`) around it! The output already contains Markdown formatting.
$ pipenv --support
Pipenv version: '2020.6.2'
Pipenv location: '/Users/naterichman/Library/Python/3.8/lib/python/site-packages/pipenv'
Python location: '/usr/local/bin/python3'
Python installations found:

3.8.4: /usr/local/bin/python3
3.8.4: /usr/local/bin/python3.8
3.8.4: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3
3.8.4: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
3.7.3: /usr/bin/python3
2.7.16: /usr/bin/python2
2.7.16: /usr/bin/python2.7

PEP 508 Information:
{'implementation_name': 'cpython',
 'implementation_version': '3.8.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT '
                     '2020; root:xnu-6153.141.1~1/RELEASE_X86_64',
 'python_full_version': '3.8.4',
 'python_version': '3.8',
 'sys_platform': 'darwin'}

System environment variables:

COLORFGBG
COLORTERM
COMMAND_MODE
HOME
ITERM_PROFILE
ITERM_SESSION_ID
LANG
LC_TERMINAL
LC_TERMINAL_VERSION
LESS
LOGNAME
LSCOLORS
LaunchInstanceID
OLDPWD
PAGER
PATH
PIPENV_ACTIVE
PIP_DISABLE_PIP_VERSION_CHECK
PIP_PYTHON_PATH
PWD
PYTHONDONTWRITEBYTECODE
SECURITYSESSIONID
SHELL
SHLVL
SSH_AUTH_SOCK
TERM
TERM_PROGRAM
TERM_PROGRAM_VERSION
TERM_SESSION_ID
TMPDIR
TMUX
TMUX_PANE
USER
XPC_FLAGS
XPC_SERVICE_NAME
ZSH
__CF_USER_TEXT_ENCODING
LC_ALL
_
PIP_SHIMS_BASE_MODULE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_ACTIVE: 1

Debug–specific environment variables:

PATH: /Users/naterichman/.local/bin:/Users/naterichman/Library/Python/3.8/bin:/Users/naterichman/bin:/usr/local/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/Users/naterichman/.local/bin:/Users/naterichman/Library/Python/3.8/bin:/Users/naterichman/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin
SHELL: /bin/zsh
LANG: en_US.UTF-8
PWD: /Users/naterichman

Contents of Pipfile ('/Users/naterichman/Pipfile'):
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]

[requires]
python_version = "3.8"

Can you confirm it isn't being set in your shell config?

Yes it wasn't being set in my shell config, however I apparently had some zombie tmux sessions in the background, a restart of my computer seemed to fix the issue.

I have the same problem. I use tmux kill-session -a to kill all session and solve the problem

I have the same problem. I use tmux kill-session -a to kill all session and solve the problem


Choose limit for line length and reforamt the code

PEP8 recommends to limit all lines to a maximum of 79 characters. While I was trying to follow PEP8, new commits break this rule:

https://github.com/niccokunzmann/hanging_threads/blob/daa569e2f6ec06f98ae2d452d77de0f958a5f0d4/hanging_threads.py#L8
https://github.com/niccokunzmann/hanging_threads/blob/daa569e2f6ec06f98ae2d452d77de0f958a5f0d4/hanging_threads.py#L44-L45

We need to choose the line length limit for this project for the code style unification.

@niccokunzmann What are your thoughts?

PEP 8 about comments:

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

We can create a pep8 check, use pep8 or flakes and make travis run tests so we see when pep8 is violated. This also generates easy-to-solve issues for new contributors.

Resolving current PEP8 line lenght problems.

@MarkoMackic Is there any reason you add blank line before closing """?

So we desided to follow PEP 8 recommendations about the line leghts.
I think this issue may be closed. I'll create new issue for automated pep8 check.


Revisit approach to use of toasts in example apps

See @KacperKluka's response here.
There may be some situations where we're currently showing a toast but an alert dialog would be more appropriate. We need to be careful about this, however, as the last thing we want to do is create a scenario where alert dialogs can keep popping up one after the other.
In situations where an alert dialog is inappropriate or otherwise risky (see previous paragraph) then we should look to emit Exception detail to the log - in addition to showing the sparser toast.

The corresponding Jira issue is https://ably.atlassian.net/browse/SDK-1346


Include extrinsics len in PoV size

With WeightsV2 and storage proof tracking we will also need to take the encoded extrinsics length into account. The point being that the actual PoV size is the size of the block header, the extrinsics and the storage proof.
As we also should include the header, we could use the length of an empty encoded header plus the the size of the digests.
CC @ggwpez

We only need to take this into account when building the block and when calculating the resulting weight i think.
So no benchmarking needed, just adding extrinsic.encoded_len() to the Proof weight result.

Actually I think for standalone chains like the relay chain we even have to only track extrinsic.encoded_len() and no storage proof size. Otherwise if we take the size/storage proof weight into account we would underutilize the available block length.

Actually I think for standalone chains like the relay chain we even have to only track extrinsic.encoded_len() and no storage proof size. Otherwise if we take the size/storage proof weight into account we would underutilize the available block length.

The DotSama relay chains have u64::MAX as proof size, so the proof size is not important at all.

Yeah and the block size is already tracked independently here: https://github.com/paritytech/substrate/blob/2bf16d831b777d3c44445d869689a66f0c0e64c6/frame/system/src/extensions/check_weight.rs#L73-L86

The DotSama relay chains have u64::MAX as proof size, so the proof size is not important at all.

That does not help at all. If we are still accounting proof size in the proof size weight, then we cannot use it for tracking block size - and would not know when a block becomes overweight with regards to that part.

To me this does not look like it would apply to inherents?

Inherents are just normal (unsigned) extrinsics and this also applies to them: https://github.com/paritytech/substrate/blob/057f2afd686dceb48c08d8a31f35b338bb5992d3/frame/system/src/extensions/check_weight.rs#L206-L212

So we would need to check size internally even more precisely then.

Not sure what you mean by "internally", but yes. I mean you could do these checks inside create_inherent (inside the runtime) to ensure that the size is not running above the allowed maximum. We should probably design this system you proposed to handle the size usage between inherents. Inherents could expose some kind of minimum size they would need, which in the case for timestamp is just the u64 and for the parachain inherents the encoded size of  the empty inherent data. Then when creating the inherents we pass the left over size the inherent can use.

To me it sounds like, if we do the "proof size weight" proper (name should probably be adjusted then), then we already pretty much have the mechanism in place I was suggesting, but actually better. Because we already do filtering on weight, so in theory if I get benchmarks to work correctly for candidate receipts and we have proof size weights which take into account block size (and only that for standalone chains), then the code that is already there would already have the desired effect: Perfect, just seems to be the right solution to me!
The code we have, queries what is the remaining available weight and then filters out stuff to fit things in. With weights also taking into account size, that code already does all we need.
Shouldn't be to hard to ignore actual proof size for standalone chains?

Shouldn't be to hard to ignore actual proof size for standalone chains?

I think I proposed this at some point, by making the size an option and set it to None. However, currently we set it to u64::Max which is the same as ignoring it?

Let's see whether I understand this correctly:

The second dimension of weight will/does contain the proof size + the block size used.
When we calculate the proof size, we set it u64::Max for standalone chains.
Then we calculate the second weight dimension by adding together with overflow the proof size and the used block size, hence for standalone chains we end up only with the used block size?

Did I get this correctly? If so, I am still unsure why we would want to use u64::Max and not 0? Sounds more correct and safer to me. E.g. if we had a bug in benchmarking and used block size was 0, then we would always be overweight.
So I think, I am absolutely not getting what you actually mean. :grimacing:

Shouldn't be to hard to ignore actual proof size for standalone chains?

I think I proposed this at some point, by making the size an option and set it to None. However, currently we set it to u64::Max which is the same as ignoring it?

Tried this a while back, but it resulted in a large breaking change where all downstream chains would have to change how they construct their RuntimeBlockWeights: https://github.com/paritytech/substrate/pull/13164
It uses a WeightLimit type that can be set to unlimited (=None).

Did I get this correctly? If so, I am still unsure why we would want to use u64::Max and not 0?

The limit is set to u64::MAX, not the initial value. The parachain limit is 5MiB in comparison. Or do you mean we should set the relay limit also to 0?
Sorry I am neither getting what you mean…

Ok, that was my initial understanding - that you are talking about the limit, but given @bkchr comment I wasn't so sure anymore.
The problem is for our use case, just setting the limit to u64::MAX is absolutely not what we want. We just don't want to track proof size (as there is none). Instead we only want to track block size and know when we reached the limit. So the limit should not be u64::MAX, but maximum block size - header size.
If we keep tracking the size for the non existing proof for standalone chains, then the second weight component is totally useless here. While if we tracked correctly, it would be very valuable and useful in maintaining the security of the chain.

I also quite don't get how not adding the proof size in benchmarks (but only counting extrinisic length) for standalone chains should break any parachain.
I am specifically talking about benchmarks. If we benchmark a call for a parachain runtime we get for the second weight component:
accessed state proof size + extrinisic length

While for the relay chain we would get:
extrinsic length

The limits for a parachain would be the PoV size limit. For the relay chain it would be the maximum block size.

Does this make sense? At least for parachain consensus we do have a couple of things which can be large in size and we need a proper way to ensure we are not producing blocks exceeding the block size limit.

At least for parachain consensus we do have a couple of things which can be large in size and we need a proper way to ensure we are not producing blocks exceeding the block size limit.

With "things" do you mean dispatchables? So you have a point in the code where you have X bytes of space left and want to know whether or not to Dispatch something without overflowing that size limit?
The extrinsic length is known by calling encoded_len on it. You can even do this before dispatching and then adding the PoV weight yourself.
I dont get how benchmarking is neccecary. This way is much better than benchmarking, since it is exact.

Yes it would not be benchmarked, the generated code would just call encoded_len. It would just be part of the benchmarking code gen, not benchmarking itself. Anyhow, in my mind it still makes sense to have it there, but we will start by wrapping WeightInfo calls manually for now and see how it goes.

Okay that makes more sense, thanks.

Anyhow, in my mind it still makes sense to have it there, but we will start by wrapping WeightInfo calls manually for now and see how it goes.

I dont think there is a good way around this though. Even if we include it in the WeightInfo output: Dispatchables are not MaxEncodedLen bound. So now we would have to write benchmarks not only in a way that consumes the most resources, but also to hit the maximal possible length. Dont know how this should be done for calls that accept a Vec.
The wrapping approach is superior in my opinion, since you get the exact length. Not some benchmarking approximation.
But it is un-ergonomic… maybe we can get around this with some macros. Would have to see the usage first.

Oh shoot, I remembered wrongly. Indeed we are already using a wrapper function that takes the actual argument and not the u32 parameters.
I think I will just override the second weight component in the wrapper function, since we don't care about proof size anyway. Feels a bit hacky, but let's see how it goes.

I found this issue while thinking about how to manage Weight v2 in frontier and moonbeam.
I think the optimal solution is to have two different CheckWeight implementations for the sovereign chain and for the parachains.
Since the transition to Weight v2 will be mainly to manage the PoV limit for parachains, it seems to me more natural and less breakable to adapt the current susbtrate CheckWeight for the sovereign chain, and to create a new CheckWeight (maybe in cumulus) dedicated to the parachains.
The solution which would consist in integrating the size of the tx in the benchmarks does not suit us for frontier, because in frontier we not have benchmarks (we accounting resources usage dynamically).


sniff test

why is this commented out in config/linkml_stage_mixs_modifications.yaml ?
samp_transport_dur:
  comments:
    - "{float} {unit} or period?"
    - problematic ISO 8601 period validation
    - P10D does not match ^P(?!$)(\\d+Y)?(\\d+M)?(\\d+W)?(\\d+D)?(T(?=\\d+[HMS])(\\d+H)?(\\d+M)?(\\d+S)?)?$
    examples:
      - value: 10 days
        description: converted slot's ISO 8601 period to a measurement value string

Important to review config/linkml_stage_mixs_modifications.yaml including comments... add a report?

time, date, period,duration, interval slots should use consistent rules (all ISO 8601? does YYYY-MM-DD pass?)

lots of door, window, ceiling terms

terms with PVs or aerob...

hand... terms

see also

#23


Dangling indirect reference to try-deno in package dependencies

Building the dashboard fails with :
npm ERR! enoent ENOENT: no such file or directory, chmod '/app/node_modules/deno/try-deno'

Using podman, it builds under root, so it's looking like a rootless issue, with the node image a possible culprit?


Better Dropdown alignment with jQuery plugin behavior

Dropdown now wraps additions in  tags and uses additionPosition="top" by default, just like the jQuery semantic-ui plugin.

There is also some more subtle changes to align with the jQuery version:

default additionLabel is now "Add" without the semicolon, and includes the space between the label and the addition. This allows users to use labels which do not wish to have space between the two.

this overall API differs from the jQuery version which uses unsafe templates: addResult     : 'Add {term}'. We could instead use callbacks, if even more alignment was preferred. addResult={term => Add {term}
or of course, we could use templates too and use dangerouslySetInnerHTML 😨

The addition item now has the addition className added to it--this can be important for themes

You will notice that, because of a React limitation/design I am passing text as an array of elements. Because of the current way the React diffing algorithm works, even though these elements will always be in the same order we must define a key for them. This isn't pretty--but at least for me, having the same styling behavior of the wrapped  is critical to UX.
This was my first experience with enzyme/chai-enzyme so lmk if there is a better way to write the prop tests that contain the array of elements. It's not ideal 😞

Current coverage is 99.48% (diff: 100%)

No coverage report found for master at 24e9490.

Powered by Codecov. Last update 24e9490...7d87e1d

This is awesome, thanks!  Would you consider adding an example showing use of a custom additionLabel?

@levithomason to docs/app/Examples/modules/Dropdown/Usage ?
something like this?
render() {
  // etc

  <Dropdown
    additionLabel="Custom: "
    {...etc}
  />

  // etc

  <Dropdown
    additionLabel={Custom: }
    {...etc}
  />
}

Exactly.  Two examples would be superb.  One showing a text label, and a separate one showing a component example.

Added requested examples. 🎉

Note: I forgot to mention that this is technically a breaking change and I have added such in the commit message for commitizen

BREAKING CHANGE: Dropdown now now wraps additions in  and uses
additionPosition="top" by default

This looks great!  Just one lint issue to fix that is failing CI:

doh! fixed. I incorrectly assumed it would lint as part of the test runner. Might be helpful to create a pre-commit hook that lints the code.

We used a few pre-commit hooks but found them to be problematic.  We just rely on the PR status.  Though, this obviously needs be a little more clear to developers!  Thanks for the feedback.

Could you clarify something regarding the breaking change? I'd like to release this as a feature rather than an API change.  My understanding:

The exposed API hasn't change
The previous additionLabel accepted plain text only
The default additionPosition changed
The additionLabel is now bold

My conclusion based on ^ these points is that old additionLabel code will still work, it will just be bold and default to the top of the Dropdown.  I see this as a non-breaking feature.  What are your thoughts?

I'm actually going to release this as a feature and we can adjust if there are issues 👍

@levithomason okiedokie :shipit:

Released in<EMAIL_ADDRESS>


Flutter engine crashes in production devices (sk_out_of_memory / regexp)

Steps to Reproduce
Our customers report that it happen when starting the app. It happens on 0.2% of our runs, and people says it happen multiple times in a row (e.g. it always crashes when they restart the app).
We are unable to reproduce this problem locally, but we receive tombstones through the Google Play Console directly (it doesn't appear in Crashlytics).
It happens on Samsung devices, but also on LG Nexus, so we don't think it is a device specific problem. And of course, everything worked fine before we upgraded our app to 1.2.1.
Logs
The crash dump, produced by ndk-stack on tombstones:
********** Crash dump: **********
#00 0x000000000004ad34 /system/lib/libc.so (tgkill+12)
#01 0x00000000000484d3 /system/lib/libc.so (pthread_kill+34)
#02 0x000000000001d4c5 /system/lib/libc.so (raise+10)
#03 0x0000000000019001 /system/lib/libc.so (__libc_android_abort+34)
#04 0x0000000000017064 /system/lib/libc.so (abort+4)
#05 0x000000000012d22b /data/app/fr.lescroissants.droid-1/lib/arm/libflutter.so
sk_out_of_memory(unsigned int)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/libcxx/include/memory:3496:5
#06 0x00000000004a8829 /data/app/fr.lescroissants.droid-1/lib/arm/libflutter.so
dart::Assert::Fail(char const*, ...)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/platform/assert.cc:43:3
#07 0x000000000041e165 /data/app/fr.lescroissants.droid-1/lib/arm/libflutter.so
dart::Zone::AllocateExpand(int)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/zone.cc:0:0
#08 0x00000000003fabeb /data/app/fr.lescroissants.droid-1/lib/arm/libflutter.so
dart::Zone::AllocUnsafe(int)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/zone.h:217:14
int* dart::Zone::Alloc<int>(int)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/zone.h:237:0
dart::BacktrackStack::BacktrackStack(dart::Zone*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/regexp_interpreter.cc:131:0
dart::IrregexpInterpreter::IrregexpResult dart::RawMatch<unsigned char>(unsigned char const*, dart::String const&, int*, int, unsigned int, dart::Zone*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/regexp_interpreter.cc:157:0
dart::IrregexpInterpreter::Match(dart::TypedData const&, dart::String const&, int*, int, dart::Zone*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/regexp_interpreter.cc:582:0
dart::ExecRaw(dart::RegExp const&, dart::String const&, int, bool, int*, int, dart::Zone*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/regexp_assembler_bytecode.cc:488:0
dart::BytecodeRegExpMacroAssembler::Interpret(dart::RegExp const&, dart::String const&, dart::Smi const&, bool, dart::Zone*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/vm/regexp_assembler_bytecode.cc:520:0
#09 0x000000000035514d /data/app/fr.lescroissants.droid-1/lib/arm/libflutter.so
dart::ExecuteMatch(dart::Zone*, dart::NativeArguments*, bool)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/lib/regexp.cc:86:10
#10 0x0000000000354ef7 /data/app/fr.lescroissants.droid-1/lib/arm/libflutter.so
dart::DN_HelperRegExp_ExecuteMatch(dart::Isolate*, dart::Thread*, dart::Zone*, dart::NativeArguments*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/lib/regexp.cc:92:10
dart::BootstrapNatives::DN_RegExp_ExecuteMatch(_Dart_NativeArguments*)
/b/rr/tmpmr462B/w/src/out/android_release/../../third_party/dart/runtime/lib/regexp.cc:90:0
#11 0x00000000000025e8 /data/data/fr.lescroissants.droid/app_flutter/vm_snapshot_instr

flutter doctor -v output:
[✓] Flutter (Channel stable, v1.2.1, on Mac OS X 10.14.3 18D109, locale fr-FR)
    • Flutter version 1.2.1 at /Users/stan/dev/flutter
    • Framework revision 8661d8aecd (il y a 7 semaines), 2019-02-14 19:19:53 -0800
    • Engine revision 3757390fa4
    • Dart version 2.1.2 (build 2.1.2-dev.0.0 0a7dcf17eb)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/stan/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Users/stan/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/182.5314842/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[!] iOS toolchain - develop for iOS devices (Xcode 10.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.2, Build version 10E125
    ✗ Verify that all connected devices have been paired with this computer in Xcode.
      If all devices have been paired, libimobiledevice and ideviceinstaller may require updating.
      To update with Brew, run:
        brew update
        brew uninstall --ignore-dependencies libimobiledevice
        brew uninstall --ignore-dependencies usbmuxd
        brew install --HEAD usbmuxd
        brew unlink usbmuxd
        brew link usbmuxd
        brew install --HEAD libimobiledevice
        brew install ideviceinstaller
    • ios-deploy 1.9.4
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.3)
    • Android Studio at /Users/stan/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/182.5314842/Android Studio.app/Contents
    • Flutter plugin version 33.4.1
    • Dart plugin version 182.5215
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[✓] VS Code (version 1.32.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 2.24.0

[!] Connected device
    ! No devices available

! Doctor found issues in 2 categories.

I'm also getting the same issue. We are thinking to go back to native. I think flutter is not stable yet. It'll take time. I don't like cross platform solution any more. I had a high hope on flutter. It also disappoints me. I never want to work on any cross platform solution.

Wow, we are not thinking like that all – the Flutter team did a very good job with the whole developer and production experience, and those crashes only occurs on 0.2% of our users and only since we upgraded to 1.2.1 (was on 1.0 before).
But it is still very painful to see those errors still flowing in, and not being able to do anything about it. If any maintainer could jump in and provide any insights very soon, it would be very helpful!

This looks like an OOM issue when attempting to parse some document with a regular expression. I am not sure what the document or the regular expression is. Unfortunately, without a reduced test case, it is unlikely more progress can be made on this issue.

I'll be closing this since it hasn't seen any updates in a few years and the version of flutter that was used when this issue was reported is quite old.
If anyone still faces this issue with the latest version of flutter, please provide the updated crash log along with the output of flutter doctor -v and a minimal sample as requested in https://github.com/flutter/flutter/issues/30324#issuecomment-643554791 so that it can be properly investigated.
Thank you


Integration fails if more than 891 LEDS are defined.

Hi,
Hope you can help.
Just finished my projector HyperHDR setup with 1172 LEDS. The HyperHDR is working really well but Home Assistant integration doesn't work.
I have spent a few hours trying to track down the issue and found that as soon as I define more than 891 LEDs, the integration crashes and can no longer connect. Anything up to 891 LEDS works flawlessly.
I am using Rapsberry PI 5 with HyperHDR 20 beta.
The error is HA logs are follows:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/streams.py", line 548, in readline
line = await self.readuntil(sep)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/streams.py", line 626, in readuntil
raise exceptions.LimitOverrunError(
asyncio.exceptions.LimitOverrunError: Separator is not found, and chunk exceed the limit
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
resp = await request_handler(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 543, in _handle
resp = await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 114, in impl
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 85, in security_filter_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 80, in ban_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 233, in auth_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 31, in headers_middleware
response = await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 149, in handle
result = await handler(request, **request.match_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 63, in with_admin
return await func(self, request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 177, in post
return await super().post(request, flow_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 72, in wrapper
result = await method(view, request, data, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post
result = await self._flow_mgr.async_configure(flow_id, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 320, in async_configure
result = await self._async_handle_step(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 416, in _async_handle_step
result: FlowResult = await getattr(flow, method)(user_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/hyperhdr/config_flow.py", line 234, in async_step_user
return await self._advance_to_auth_step_if_necessary(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/hyperhdr/config_flow.py", line 141, in _advance_to_auth_step_if_necessary
return await self.async_step_confirm()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/hyperhdr/config_flow.py", line 401, in async_step_confirm
async with self._create_client() as hyperhdr_client:
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 149, in aenter
result = await self.async_client_connect()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 369, in async_client_connect
await self.async_client_disconnect()
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 422, in async_client_disconnect
return await self._async_client_disconnect_internal()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 451, in _async_client_disconnect_internal
await self._await_or_stop_task(receive_task, stop_task=True)
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 588, in _await_or_stop_task
await task
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 568, in _receive_task_loop
while await self._async_receive_once():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 608, in _async_receive_once
resp_json = await self._async_safely_read_command(use_timeout=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/hyperhdr/client.py", line 485, in _async_safely_read_command
resp = await asyncio.wait_for(future_resp, timeout=timeout_secs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/tasks.py", line 452, in wait_for
return await fut
^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/streams.py", line 557, in readline
raise ValueError(e.args[0])
ValueError: Separator is not found, and chunk exceed the limit

I got it to work by hacking at the code. After some hours of debugging, I could see that hyperHDR is sending the full LED device data and it is overflowing the buffer. I made some changes to hyperhdr/client.py (under hyperhdr-py) by finding a post on google regarding how to read a stream and catch partial read exception. I then forced a second read under json.loads exception to get the full buffer.
It is a not a pretty solution, but it got my integration to load and work. Posting it here in case this can help in fixing the code correctly. Technically, this should be posted under https://github.com/mjoshd/hyperhdr-py.
client.py

add new function
async def _readWithoutLimit(self, stream):
sep = b'\n'
seplen = len(sep)
try:
return await stream.readuntil(sep)
except asyncio.IncompleteReadError as e:
return e.partial
except asyncio.LimitOverrunError as e:
return await stream.read(e.consumed)

change line 484 to:
future_resp = self._readWithoutLimit(self._reader)
line 514, remove "Return None" and replace with:
future_resp = self._readWithoutLimit(self._reader)
resp = resp + await asyncio.wait_for(future_resp, timeout=timeout_secs)
I also removed the LOGGER.warning (line 510) so it doesn't appear in my home assistant log.
This is a not "good" code but works until someone with more knowledge than me can fix it correctly. Unfortunately, I know nothing about asyncio functionality.

I'm actually wondering what huge projector you have, for using 1172 leds, that's insane (in a good way)
I'm intersted in how you accomplished that project. If you're willing to share your project i won't mind :D

Closing as Can't fix rather than Won't fix; I would gladly fix it if I had any idea how to do it 'right'.
I know nothing of Python and have been stumbling and fumbling my way through this from the beginning.
I would love if someone who knew how all this stuff works would take over and maintain this project but no offers have arrived.
Perhaps this can be revisited in the future but for now it is being closed.