Frequent Crashes with Android 10 beta

Hi, while testing Android 10 beta, I stumbled over some weird crashes which led to reinstallation of the K9 app.
Expected behavior
a) A tap on the K9 icon should launch K9 and the main screen should be visible
Actual behavior
With Android 10 beta, this is, what I observed:

a tap on the K9 icon launches K9 for a second and then the main window closes down. I can't tell if it crashes or if the GUI is just not rendered. But neither recurring starts, nor completely stopping the app and restarting will help. Then I tried to clear the app cache with no success. After a complete reinstallation, K9 asks for permission to access files and address book. Then I can add accounts and use it as usual.

Now comes the weird part: After resetting all App-Permissions to system Default, the same
"crashes" happened again. This time I chose to delete the application data instead of reinstalling and again, K9 asks for permission to access files and address book, lets me add accounts and works like normal.
So, this seems somehow related to the changes in permission management, but since I don't see any error messages, I can't tell more about it, right now.
Steps to reproduce

Install Android 10 beta
Tap on K9-Icon
"crash" or "disapperaring window"

Environment
K-9 Mail version: 5.600
Android version: 10 on Google Pixel XL (Version 1)
Last Security Updates: 05.05.2019
Account type (IMAP, POP3, WebDAV/Exchange): several imap(s) accounts
Please take some time to retrieve logs and attach them here:
I have logged 4 different times:

After resetting app defaults:
k9-log.16354-1.txt

After first tap on K9-icon / crash
k9-log.16354-2.txt

A second try
k9-log.16354-3.txt

A third one
k9-log.16354-4.txt

The logs don't contain stack traces of the crashes. Can you please record a log without filtering for K-9 Mail's PID. The log entries we're after should have the AndroidRuntime tag.

Here it is. On second view, this doesn't look like a crash, the PID of the k9 process doesn't change while testing.
full-log.after.2nd.k9start.txt

Btw: I don't read this webpage frequently and Github is not IPv6 capable. If you want to contat me directly. just send an email to<EMAIL_ADDRESS>

I'm seeing a similar crash on launch with my Pixel 3a XL on the latest beta. I've attached the adb log; I stopped the log dump after attempting to launch. From what I can tell, there's not much information in it, though.
k9-log.txt

I can't find any crashes of K-9 in the logs. If you're still experiencing crashes with Android 10 (non-beta) please create a new issue.


Add PoS testnet spec and point CLI to it

Description of proposed changes
Adds the chainspec currently used on the PoS testnet. Also changes the CLI to point to the new spec, so that
when you pass --chain test or --chain testnet it gives you to PoS testnet spec instead of the old one.

Practical tips for PR review & merge:

[ ] All GitHub Actions report PASS
[ ] Newly added code/functions have unit tests

[ ] Coverage tools report all newly added lines as covered
[ ] The positive scenario is exercised
[ ] Negative scenarios are exercised, e.g. assert on all possible errors
[ ] Assert on events triggered if applicable
[ ] Assert on changes made to storage if applicable

[ ] Modified behavior/functions - try to make sure above test items are covered
[ ] Integration tests are added if applicable/needed

@atodorov Yup


[Feature] sc-27764 Support BigQuery datasource

PR checklist

[x] Ensure you have added or ran the appropriate tests for your PR.
[x] DCO signed

What type of PR is this?
Featrue
What this PR does / why we need it:

TBD ...

Which issue(s) this PR fixes:
sc-27764
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

profiler LGTM


Fix backwards logic checking for --call_creds=none

Thank you for your pull request. Before we can look at your contribution, we need to ensure all contributors are covered by a Contributor License Agreement.
After the following items are addressed, please respond with a new comment here, and the automated system will re-verify.

User @deadmoose isn't covered by a CLA. They will need to complete the form at https://identity.linuxfoundation.org/projects/cncf

Regards,
CLA GitHub bot


Compile p4est in P4est_jll.jl with BLAS support

Right now, the P4est_jll.jl package provides a p4est library without using the BLAS library:
https://github.com/JuliaPackaging/Yggdrasil/blob/999e6cf6b9054a05a4e76a1d23de93c5651396a2/P/P4est/build_tarballs.jl#L21
Since OpenBLAS is available as a jll package as well, we should try to build p4est against it and update the Yggdrasil recipe (https://github.com/JuliaPackaging/Yggdrasil/blob/master/P/P4est/build_tarballs.jl).

Fixed in JuliaPackaging/Yggdrasil#1749


Lock DOCKER_API_VERSION instead of relying on docker cli version

📚 Context
Since we're using docker CLI as an interface to docker, we should lock our interaction using the DOCKER_API_VERSION, this way we avoid problems with different docker versions from different docker users.
This way we can update the API with more control, when we need some functionality for new API versions.
✔ Solution
Lock the DOCKER_API_VERSION.
https://docs.docker.com/engine/api/
📈 Subtasks

[ ] See where we should lock the version
[ ] Adapt sunodo doctor to check for the API version instead of the CLI version.

I tried using the Docker API directly in the beginning of the project.
It's way harder than using the CLI.

To run everything, we're using mainly docker compose cli, other commands we use to check the status or version, we could try to use the API, it may be easier.
And I found this project, don't know if you already assessed it https://github.com/apocas/dockerode-compose

And I found this project, don't know if you already assessed it https://github.com/apocas/dockerode-compose

I did. It's very immature.


doc: update the doc of https for Chinese version.

here is the commit in English doc: https://github.com/iresty/apisix/commit/225b3d310623ded8e4a4e9914c71b125ede42936

I'd like to do my best!


Remove Jenkins from WCF master.

Also removed a non-existant branch.

Is there anything else I need to do to fully remove the Jenkins jobs from running, delete the jobs in Jenkins or anything like that?

@mmitche Let me know if there is anything else I need to do. Thanks!


websocket mode doesn't work in shadow DOM

I am writing a wrapper web component for Deep Chat. The "basic" REST handler works, but problems arise when I set websocket: true. If url is given, the connection is never initiated, and if handler if given, it is never called. The submit button isn't enabled when I type in the input field, I assume because that only gets enabled once the connection is established. I noticed this only happens when rendering deep-chat inside of the shadow DOM. If I don't create a shadow DOM, it works, but this isn't the recommended way to create custom components.
I created a minimal reproducible example that shows the difference in behavior when Deep Chat is used in the light and shadow DOM. If you set websocket to false, you'll see that the examples behave identically.
https://stackblitz.com/edit/lit-nmk2pm
I've been trying to track down the issue, but I haven't had any luck so far. My best guess is that Deep Chat queries elements in the DOM to add an event listener somewhere, but I haven't been able to find evidence of that. Maybe you will have a better idea of what the problem could be.
As an aside, the reason I'm attempting to switch to the websocket handler is because I want to render multiple responses from a single HTTP response (the actual text response, and a source for the response also returned by the API). I came across this comment that recommends using the websocket handler to achieve this.

Of course as soon as I open the issue, I find it. The websocket connection never gets created because the code is checking for the existence of Deep Chat in the document body.

I'm not entirely sure why you make that check, but if all you want is to make sure Deep Chat is still attached to the DOM, I think this is a valid replacement:
// Current check
if (!document.body.contains(io.deepChat)) return;

// Proposed check
if (!(io.deepChat.getRootNode({composed: true}) instanceof Document)) return;

I'll go ahead and make a PR with that change.

Hi @mchill.
Thankyou very much for doing the investigation and finding the fix for the problem! I have recently got a new job and have been struggling to balance it with my open source endeavours, hence I apologise for getting back so late.
The reason why I had included the document.body.contains(io.deepChat) code was because in MPA (Multi Page Apps) if Deep Chat was configured to connect to a websocket connection and the connection would fail, the component would attempt to automatically reconnect via the retryConnection method. The problem arises when the user attempts to navigate to another page as this method would keep trying to reconnect anyway. Hence, I added the check to make sure that the component is still rendered in the page if it is attempting to reconnect.
I have tested your solution and the intended functionality still works, hence if it also works for your case it is a fantastic solution. Thankyou very much for your efforts!
May I ask, why were you trying to render Deep Chat as a light dom component? What particular use-case does this help you accomplish? Thanks!

I have published your change in deep-chat-dev and deep-chat-react-dev packages version 9.0.172. These packages behave just like the main packages except their names are different. They also contain a few property name changes, such as the request property is changed to connect and stream is moved to the connect object. All of the current/old properties will still keep working as usual except you will see warnings in the console for their deprecation and get a few TypeScript warnings which you can ignore.
My work should become a bit more relaxed next month, hence that is when I plan to release this fix to the main packages. In the meanwhile, please use the dev packages and look out for the next release.
Thanks!

May I ask, why were you trying to render Deep Chat as a light dom component?

I wasn't, it's the other way around - it was broken when nested inside of another shadow DOM. I'm using the Lit framework, which by default wraps each component in a shadow DOM. I just tried rendering to the light DOM while debugging, which led me to the root cause of the bug.
Thanks for the quick reply! I'm already using the dev package for some other fixes, so that's no problem.

Understood, thankyou for the help once again!


Fix Speaker mode gets stuck in the ON state

AudioManager from the application context works without stuck.
MOB-2234

1965600

fixed - 1965600

!squash


Handle symlinks to directories

First, thanks for Shake!  I am replacing some old and crufty agglomeration of Makefiles and OMakefiles and worse, and it's going splendid.
But for this project I need a symlink to a directory to be created.  I had hoped to do something like this:
symlink %> createSymLink' from symlink
Alas, my 'from' is a directory, so shake doesn't like this at all.
The usual workaround for depending on directories, ie depending on a file inside the directory doesn't work: shake will just create a new empty directory where the symlink should be..
Any chance of fixing this, please?  (I would contribute a patch, but I have to properly figure out the open source policy of my new employer first..)

So you want to depend on a directory? See this FAQ: http://shakebuild.com/faq#how-can-i-depend-on-directories - it's generally not a good idea.
Why not have an action, which is always run, along the lines:
action $ unlessM (doesDirectoryExist symlink) $ createSymLink' from symlink

(I am also still figuring out the open source policy of my new employer)

Oh, I want to depend on a symlink as a symlink, not as a directory. (Ie,
don't dereference, just treat it as an opaque thing.)
I'll have a look at the FAQ entry you mentioned, and will try the
workaround you suggested.
On Tue, 8 Nov 2016, 22:33 Neil Mitchell<EMAIL_ADDRESS>wrote:

So you want to depend on a directory? See this FAQ:
http://shakebuild.com/faq#how-can-i-depend-on-directories - it's
generally not a good idea.
Why not have an action, which is always run, along the lines:
action $ unlessM (doesDirectoryExist symlink) $ createSymLink' from symlink
(I am also still figuring out the open source policy of my new employer)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ndmitchell/shake/issues/498#issuecomment-259280911,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADewimXx5wJH9ziXO3S28UBglvN9kaPks5q8Pi8gaJpZM4KsoSc
.

Does allow rules for symlinks at all? It seems it doesn't? I can't find any documentation on this.

A comment (since deleted) asked about symlinks in general. The answer is Shake doesn't really handle them at all - worth calling out in the docs.

I've released safe-0.3.12 with this included.

Commenting on entirely the wrong ticket!

Directories are special, and that's now called out. However, symlinks aren't special, and all the normal stuff should work with them fine, so nothing to say.

Thanks. I also managed to change the way we build so we don't need to track that symlink any more.


Weighted fruchterman reingold

Hi. Feel free to ignore this if you don't like it, but I added another force that also takes into accounts the weights of the nodes.
Of course it can only be used if the weight can be converted into an f32.

Hi!, thanks this looks great I'd love to include it in this project.
Once I pull it in I'll add it into the already existing fruchterman_reingold module, then add it as an option for fdg-macroquad.


BTCe.com nonce issues when order not filled.

2017-06-08 21:48:03 (INFO):	Attempting to BUY 3.1398476313056083 ETH at BTC-E
2017-06-08 21:49:05 (INFO):	BUY order was not (fully) filled, cancelling and creating new order
/Users/freefall/gekkoinstalls/gekko/node_modules/btc-e/btc-e.js:46
params.nonce = Math.round((new Date()).getTime() / 1000);
^
TypeError: Cannot create property 'nonce' on number '1832535087'
at BTCE.makeRequest (/Users/freefall/gekkoinstalls/gekko/node_modules/btc-e/btc-e.js:46:18)
at BTCE.orderList (/Users/freefall/gekkoinstalls/gekko/node_modules/btc-e/btc-e.js:126:8)
at BTCE.bound [as orderList] (/Users/freefall/gekkoinstalls/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Trader.cancelOrder (/Users/freefall/gekkoinstalls/gekko/exchanges/btce.js:146:13)
at Trader.bound [as cancelOrder] (/Users/freefall/gekkoinstalls/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Manager.finish (/Users/freefall/gekkoinstalls/gekko/plugins/trader/portfolioManager.js:291:21)
at bound (/Users/freefall/gekkoinstalls/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Trader.check (/Users/freefall/gekkoinstalls/gekko/exchanges/btce.js:137:7)
at bound (/Users/freefall/gekkoinstalls/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Request._callback (/Users/freefall/gekkoinstalls/gekko/node_modules/btc-e/btc-e.js:82:5)

Thanks! Will look at it asap :)

This does happen 100% of the time?

Yes.

I will pick this up when going over BTCe as part of #737.

This was not fixed in f775c1d yet:

/Users/pear/projects/gekko/node_modules/btc-e/btc-e.js:78
params.nonce = Math.round((new Date()).getTime() / 1000);
^
TypeError: Cannot create property 'nonce' on number '1885514729'
at BTCE.makeRequest (/Users/pear/projects/gekko/node_modules/btc-e/btc-e.js:78:18)
at BTCE.orderList (/Users/pear/projects/gekko/node_modules/btc-e/btc-e.js:127:8)
at BTCE.bound [as orderList] (/Users/pear/projects/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Trader.cancelOrder (/Users/pear/projects/gekko/exchanges/btce.js:198:13)
at Trader.bound [as cancelOrder] (/Users/pear/projects/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Manager.cancelLastOrder (/Users/pear/projects/gekko/plugins/trader/portfolioManager.js:271:17)
at Manager.bound [as cancelLastOrder] (/Users/pear/projects/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Manager.trade (/Users/pear/projects/gekko/plugins/trader/portfolioManager.js:147:17)
at Manager.bound [as trade] (/Users/pear/projects/gekko/node_modules/lodash/dist/lodash.js:729:21)
at Trader.processAdvice (/Users/pear/projects/gekko/plugins/trader/trader.js:51:18)
RECEIVED ERROR IN GEKKO<PHONE_NUMBER>02668
Child process has died.


Fix sporadic urn:acme:error:badNonce responses

The acme->nonce was being released after the HTTP headers' pool was cleared, causing random memory contents to be sent to the ACME server.

Doh! Nice catch.


Fixes 161 search published entries

Fixes #161
Search bar added to the published entries and my entries so that users can search by date or title of notes.
This is needed because it make the user experience much better especially when the total notes grows.
I added a TextInput in the HTML part of the HomeScreen file and the render notes function was changed so that it filtered only the ones based on what is in the search bar. The total notes counter also needed to be updated so there is a variable for that in the code.

This PR shows some of the same changes as PR #164. Does anyone understand why this is happening? @yashb196 @rcAsironman

This PR shows some of the same changes as PR #164. Does anyone understand why this is happening? @yashb196 @rcAsironman

i believe it might be a local pull from the branch that may have shown the same changes as 164

This PR shows some of the same changes as PR #164. Does anyone understand why this is happening? @yashb196 @rcAsironman

@kate-holdener, I believe this is because we branched out of the home-screen-nav branch and not main, but we have that mess solved we are going to branch out of main and re apply our feature to the main branch.
Thank You

@ademDurakovic
Fantastic work on implementing the search functionality! Your code is well-structured and demonstrates a strong grasp of the requirements. I have one key suggestion to ensure everything aligns perfectly with the project goals:
Search Scope Adjustment
The current implementation searches across all entries. However, the requirement specifies that the search should focus solely on published entries. Please adjust the filtering logic to ensure that only published notes are included in the search results.
Your attention to this detail will enhance the functionality and ensure it meets the project specifications. Keep up the great work—your contributions are highly valued!


Single line progress

Right now the single line progress will leave the cursor on the second line, which makes it looks less elegant. Maybe have a single line progress bar version that allows putting the cursor at the back or at the front (or even hide it) while displaying another progress bar, without having to have an extra line.

New discussion here https://github.com/mitsuhiko/indicatif/issues/334

Let's close this in favor of the other issue, which has more discussion.


Add IGListCollectionContext API to get visible indexes.

Changes in this pull request
Add IGListCollectionContext API to get visible indexes:
- (NSArray<NSIndexPath *> *)visiblePathsForSectionController:(IGListSectionController<IGListSectionType> *) sectionController
Issue fixed: #465
Checklist

[x] All tests pass. Demo project builds and runs.
[ ] I added tests, an experiment, or detailed why my change isn't tested.
[x] I added an entry to the CHANGELOG.md for any breaking changes, enhancements, or bug fixes.
[x] I have reviewed the contributing guide

@rnystrom thanks! will fix up and submit tomorrow. 👍

Thanks again @Malecks ! 👍


Client windshaft map split

Move request logic from map-base to windshaft client

Also, these tests can now be moved to test/spec/windshaft/client.spec.js.


Regression on latest master

Current master (db47b26) does not work against the currently deployed 0.4.2 peers

Seems to be false alarm (current master is working now). It's a bit of a hassle since there was no ICMP being reported back, which should generally be the case. Another case could be a routing loop, though there also was no hop limit exceeded ICMP. This would indicate a routing loop in 2 directions (if that is the case). For now I'll close this but keep it in the back of my head. Additionally, it might be useful to get an insight in how many packets are dropped per peer.


Use aliases to convert CluProcessor to BalaurProcessor

Show examples in TestHash.  HomeController still had a CluProcessor and it still works unchanged.  Only the package file needs to be used, so this is just a draft.

Nice!!
On Wed, Sep 20, 2023 at 14:55 Keith Alcock @.***> wrote:

Show examples in TestHash. HomeController still had a CluProcessor and it
still works unchanged. Only the package file needs to be used, so this is
just a draft.
You can view, comment on, or merge this pull request online at:
https://github.com/clulab/processors/pull/759
Commit Summary

7685ce8
https://github.com/clulab/processors/pull/759/commits/7685ce8f39fa978840aec6279b37e40b855518b5
Use aliases to convert CluProcessor to BalaurProcessor

File Changes
(4 files https://github.com/clulab/processors/pull/759/files)

A main/src/main/scala/org/clulab/processors/clu/package.scala
https://github.com/clulab/processors/pull/759/files#diff-faf87a8675939ac6f73c8f1ea061affb5432c22cd79b53d1704984360c9e9b02
(6)
M main/src/test/scala-2.11_2.12/org/clulab/utils/TestHash.scala
https://github.com/clulab/processors/pull/759/files#diff-6d648bde7a0abfd3f97e2fcad5be715b269d202e16d3ec9a99e8b4213d639970
(5)
M main/src/test/scala-2.13/org/clulab/utils/TestHash.scala
https://github.com/clulab/processors/pull/759/files#diff-14ad6a35dcb1ee5edd460f7e39e8b46ff16c2619bfe8359d20a95069ddaae61c
(5)
M main/src/test/scala-3/org/clulab/utils/TestHash.scala
https://github.com/clulab/processors/pull/759/files#diff-e69813c75c0bb0cb65abc17c273e02fef735fcbd5efcb8196726438eaa4c301e
(5)

Patch Links:

https://github.com/clulab/processors/pull/759.patch
https://github.com/clulab/processors/pull/759.diff

—
Reply to this email directly, view it on GitHub
https://github.com/clulab/processors/pull/759, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI75TWWNTI4ZWGSOEYEMZDX3NQ5NANCNFSM6AAAAAA5ASN2NY
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Add section for error handling

This PR adds a section about error handling.
The following statement is written twice inside the Function Evaluation section:

How and when this error is raised is implementation specific

IMHO, this statement applies to all error handling scenarios. So instead of making that statement specific to function evaluation, it could be moved to the section about error handling.
The spec has 34 occurrences of the word "error", but it does not really codify how errors are raised.
The spec does not have a list of error types and it does not have guidelines about when to use these error types.

@sebastien-rosset thanks for your contribution.
I think this is a good opportunity to address error type precedence in this PR.
Also, any changes to the spec should ideally be tracked into a JEP – only for historical purposes.
Would you mind starting a JEP document in the discussions for that? You can find some samples of past discussions tagged with jep-candidate there as well.
The numbering would be defined later on.


Feature request: extracting intermediate values from custom vjp

I'm in a situation where I would like to log some internal properties of a custom vjp (e.g., number of iterations). I can't seem to find an elegant way of doing this.
This is somewhat related to #136 . I feel like any traceable logging mechanism would also allow extracting intermediate values from custom vjp functions.
Maybe a mechanism for attaching aux data to arrays which propagate would provide a useful solution. This is related to #844 and #918 .

I'm also encountering the same challenge. Take implicit differentiation for example: it would be useful to have some sort of hook/callback to monitor the linear solver inside the custom vjp itself. As far as I understand, having side-effects within a custom vjp is not a good idea because of the tracer.


Performance comparisons?

Hi!
I am considering using this library vs Boost.Locale. Do you have any benchmarks comparing the two? I am in particular interested in grapheme cluster iteration. My use-case is a text-editor: https://github.com/arximboldi/ewig. I am using UTF-32 now for simplicity but this wastes memory and is not really correct without normalized graphemes, so I would like to adopt some proper UTF-8 handling.
Cheers!

No, I have not compared...
On Thu, Aug 10, 2017, 04:39 arximboldi<EMAIL_ADDRESS>wrote:

Hi!
I am considering using this library vs Boost.Locale. Do you have any
benchmarks comparing the two? I am in particular interested in grapheme
cluster iteration. My use-case is a text-editor:
https://github.com/arximboldi/ewig. I am using UTF-32 now for simplicity
but this wastes memory and is not really correct without normalized
graphemes, so I would like to adopt some proper UTF-8 handling.
Cheers!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ruoso/u5e/issues/7, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE9K9HtPWtxlF-9xshAvC3O1Xnazy46ks5sWrOjgaJpZM4OzDhn
.

Note that Boost.locale uses icu under the hood, so if you're not using
utf16 you will be constantly paying the conversion cost.
On Thu, Aug 10, 2017, 07:04 Daniel Ruoso<EMAIL_ADDRESS>wrote:

No, I have not compared...
On Thu, Aug 10, 2017, 04:39 arximboldi<EMAIL_ADDRESS>wrote:

Hi!
I am considering using this library vs Boost.Locale. Do you have any
benchmarks comparing the two? I am in particular interested in grapheme
cluster iteration. My use-case is a text-editor:
https://github.com/arximboldi/ewig. I am using UTF-32 now for simplicity
but this wastes memory and is not really correct without normalized
graphemes, so I would like to adopt some proper UTF-8 handling.
Cheers!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ruoso/u5e/issues/7, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE9K9HtPWtxlF-9xshAvC3O1Xnazy46ks5sWrOjgaJpZM4OzDhn
.


Python: Threading in Python

Description
Description
This issue will create documentation to serve as a introductory guide to the important concept of threading in python.
It will cover:

What is threading?
Its utility
Some illustrative examples

Domain
Python_Domain
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

@adityabisoi Issue created. Please assign this to me. Thank you.

\assign

/assign

@monalsingh10! you can not self assign yourself.

/assign

Please keep a check on your actions. Do not do this kind of activity. Ask for issues and be fair. @monalsingh10 highly disappointing!

@TishaJhabak1014 Go ahead

@adityabisoi  isthis open for video then plz assign me


ensure FIPs compliance

We should really try to address this from the start this time in sigstore-go. As it stands sigstore cannot be used on public sector sites and likely in verticals such as banking, telco etc, due to us being non FIPs compliant
https://github.com/sigstore/cosign/issues/94

can it use the OS fips crypto provider instead of rolling its own crypto? or a HSM HW device via PKCS#11?

To clarify, Cosign has never implemented its own crypto.
Cosign uses the Go standard library crypto implementation. Compiling Go's crypto library against BoringSSL is possible (but considered experimental by Go). This may or may not result in a FIPS-compliant Cosign; you'd have to ask a lawyer.
It is currently possible to use PKCS#11 signing with Cosign as well.
See the Cosign issue: https://github.com/sigstore/cosign/issues/94


Crash when using SliverGrid, findChildIndexCallback, a sparsely populated grid and updating the children

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
In the app I'm working on, we have a custom SliverGrid layout. It does a somewhat sparse layout of grid items, with a contiguous series of items and other items interspersed along the way. Some actions performed in the app causes the grid contents to be limited - thus items are removed from the grid.
We also use a findChildIndexCallback with SliverChildBuilderDelegate. It seems like that combination can lead to places in the grid where removing items leads to a crash.

Open https://dartpad.dev/?id=d9f723468f22d4035dfcf90366802151
Run app
Scroll down until "grid item 7" just scrolls into view in the left column
Press the FAB/refresh button in the bottom right

Expected results
No crashes or issues.
Actual results
══╡ EXCEPTION CAUGHT BY RENDERING LIBRARY ╞═════════════════════════════════════════════════════════
The following assertion was thrown during performLayout():
Assertion failed:
file:///opt/flutter/packages/flutter/lib/src/rendering/sliver_multi_box_adaptor.dart:508:12
childCount >= leadingGarbage + trailingGarbage
is not true

Widget creation tracking is currently disabled. Enabling it enables improved error messages. It can
be enabled by passing `--track-widget-creation` to `flutter run` or `flutter test`.

When the exception was thrown, this was the stack

The following RenderObject was being processed when the exception was fired: RenderSliverGrid#255ad relayoutBoundary=up1 NEEDS-LAYOUT NEEDS-COMPOSITING-BITS-UPDATE:
  needs compositing
  creator: SliverGrid ← Viewport ← IgnorePointer-[GlobalKey#e6cb3] ← Semantics ← Listener ←
    _GestureSemantics ← RawGestureDetector-[LabeledGlobalKey<RawGestureDetectorState>#04dfe] ←
    Listener ← _ScrollableScope ← _ScrollSemantics-[GlobalKey#8e450] ←
    NotificationListener<ScrollMetricsNotification> ← RepaintBoundary ← ⋯
  parentData: paintOffset=Offset(0.0, 0.0) (can use size)
  constraints: SliverConstraints(AxisDirection.down, GrowthDirection.forward, ScrollDirection.idle,
    scrollOffset: 6878.2, remainingPaintExtent: 805.0, crossAxisExtent: 826.0, crossAxisDirection:
    AxisDirection.right, viewportMainAxisExtent: 805.0, remainingCacheExtent: 1305.0, cacheOrigin:
    -250.0)
  geometry: SliverGeometry(scrollExtent: 21000.0, paintExtent: 805.0, maxPaintExtent: 21000.0,
    hasVisualOverflow: true, cacheExtent: 1305.0)
  currently live children: 1 to 6
This RenderObject had the following descendants (showing up to depth 5):
    child with index 1: RenderIndexedSemantics#9f23a
      child: RenderRepaintBoundary#89623 NEEDS-PAINT
        child: _RenderColoredBox#da96c NEEDS-PAINT
          child: RenderPositionedBox#60e7f NEEDS-LAYOUT
            child: RenderParagraph#3e200 relayoutBoundary=up1 NEEDS-LAYOUT
    child with index 2: RenderIndexedSemantics#cd9d3
      child: RenderRepaintBoundary#04b11 NEEDS-PAINT
        child: _RenderColoredBox#d3dc9 NEEDS-PAINT
          child: RenderPositionedBox#d0f81 NEEDS-LAYOUT
            child: RenderParagraph#6065a relayoutBoundary=up1 NEEDS-LAYOUT
    child with index 5: RenderIndexedSemantics#5f68d NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
      child: RenderRepaintBoundary#ed0b9 NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
        child: _RenderColoredBox#b737d NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
          child: RenderPositionedBox#2daac NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
            child: RenderParagraph#26a3e NEEDS-LAYOUT NEEDS-PAINT
    child with index 6: RenderIndexedSemantics#4b2f1 NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
      child: RenderRepaintBoundary#93339 NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
        child: _RenderColoredBox#10147 NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
          child: RenderPositionedBox#a9025 NEEDS-LAYOUT NEEDS-PAINT NEEDS-COMPOSITING-BITS-UPDATE
            child: RenderParagraph#23002 NEEDS-LAYOUT NEEDS-PAINT
════════════════════════════════════════════════════════════════════════════════════════════════════

Code sample
Code sample
See https://dartpad.dev/?id=d9f723468f22d4035dfcf90366802151.

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[Paste your output here]

Reproducible using the code sample shared above.
Tested on 3.13, 3.16 & 3.17.0-12.0.pre.39
Labeling for further insight.

code sample (pasted for convenience)
import 'dart:math';
import 'package:flutter/material.dart';
import 'package:flutter/src/rendering/sliver.dart';
import 'package:flutter/src/rendering/sliver_grid.dart';
import 'package:collection/collection.dart';

/// Demo for a Flutter bug.
///
/// ## Instructions
/// 1. Run app
/// 2. Scroll down until "grid item 7" just scrolls into view in the left column
/// 3. Press the FAB/refresh button in the bottom right
///
/// In the console, you'll see something like:
///
/// ══╡ EXCEPTION CAUGHT BY RENDERING LIBRARY ╞═════════════════════════════════════════════════════════
/// The following assertion was thrown during performLayout():
/// Assertion failed:
/// file:///opt/flutter/packages/flutter/lib/src/rendering/sliver_multi_box_adaptor.dart:508:12
/// childCount >= leadingGarbage + trailingGarbage
/// is not true
///
/// If you remove the `findChildIndexCallback` from the `SliverChildBuilderDelegate` the crash
/// is not triggered.

void main() => runApp(const CustomScrollViewExampleApp());

class CustomScrollViewExampleApp extends StatelessWidget {
  const CustomScrollViewExampleApp({super.key});

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      home: CustomScrollViewExample(),
    );
  }
}

class CustomScrollViewExample extends StatefulWidget {
  const CustomScrollViewExample({super.key});

  @override
  State<CustomScrollViewExample> createState() =>
      _CustomScrollViewExampleState();
}

class _CustomScrollViewExampleState extends State<CustomScrollViewExample> {
  late final List<MyCollection> _initialCollections;
  late List<MyCollection> _activeCollections;
  late List<ValueKey<String>> _activeItemKeys;
  var _hideBadgesAndChildren = false;

  @override
  void initState() {
    super.initState();
    _initialCollections = [
      MyCollection(2),
      MyCollection(1),
      MyCollection(0),
      MyCollection(1),
      MyCollection(2),
      MyCollection(0),
      MyCollection(0),
    ];
    _refreshActiveCollections();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Bug demo'),
      ),
      body: _activeCollections.isNotEmpty
          ? CustomScrollView(
              slivers: <Widget>[
                SliverGrid(
                  gridDelegate: MyGridDelegate(_activeCollections),
                  delegate: SliverChildBuilderDelegate(
                    (BuildContext context, int index) {
                      return Container(
                        key: _activeItemKeys[index],
                        alignment: Alignment.center,
                        color: Colors.teal[100 * (index % 9)],
                        child: Text('grid item $index'),
                      );
                    },
                    childCount: _activeCollections.length +
                        _activeCollections.map((c) => c.childCount).sum,
                    // Comment out this findChildIndexCallback and the
                    // associated test passes ...
                    findChildIndexCallback: (key) {
                      final potentialIndex =
                          _activeItemKeys.indexOf(key as ValueKey<String>);
                      if (potentialIndex > 0) {
                        return potentialIndex;
                      }
                      return null;
                    },
                  ),
                ),
              ],
            )
          : Container(),
      floatingActionButton: FloatingActionButton(
          child: const Icon(Icons.refresh_rounded),
          // Toggle children and badges on and off.
          onPressed: () {
            setState(() {
              _hideBadgesAndChildren = !_hideBadgesAndChildren;
              _refreshActiveCollections();
            });
          }),
    );
  }

  void _refreshActiveCollections() {
    _activeCollections = [];
    for (final c in _initialCollections) {
      c.hideChildren = _hideBadgesAndChildren;
      _activeCollections.add(c);
      if (!_hideBadgesAndChildren && _initialCollections.last != c) {
        _activeCollections.add(MyCollection.badge());
      }
    }
    _assignItemKeys();
  }

  /// The generated [_activeItemKeys] are used in the findChildIndexCallback and
  /// when building child widgets.
  ///
  /// Should be called whenever the list of [_activeCollections] changes.
  void _assignItemKeys() {
    _activeItemKeys = [];
    var badgeIndex = 0;
    var collectionIndex = 0;
    for (final collection in _activeCollections) {
      if (collection.isBadge) {
        _activeItemKeys.add(ValueKey('badge $badgeIndex'));
        badgeIndex++;
      } else {
        _activeItemKeys.add(ValueKey('collection $collectionIndex'));
        for (var i = 0; i < collection.childCount; i++) {
          _activeItemKeys
              .add(ValueKey('collection $collectionIndex, child $i'));
        }
        collectionIndex++;
      }
    }
  }
}

class MyGridDelegate implements SliverGridDelegate {
  final List<MyCollection> _collections;

  MyGridDelegate(this._collections);

  @override
  SliverGridLayout getLayout(SliverConstraints constraints) {
    final childWidth = constraints.asBoxConstraints().maxWidth / 3;
    const maxChildHeight = 200.0;
    const collectionHeight = 3000.0;
    return MyGridLayout(
        _collections, childWidth, maxChildHeight, collectionHeight);
  }

  @override
  bool shouldRelayout(covariant MyGridDelegate oldDelegate) {
    final result = oldDelegate._collections != _collections;
    return result;
  }
}

/// Defines the layout that helps reproduce this bug.
///
/// [MyCollection] instances are laid out vertically and their children can
/// start being rendered at half their height. The overall height of the layout
/// is defined by the summed height of the collections.
///
/// [MyCollection.badge] instances render over/between other collections. They
/// don't contribute toward the height.
///
/// Here is a simple diagram that shows a layout of a a collection, its two
/// children, a badge and then the start of another collection.
///
///     +----+
///     |    |
///     |    |
///     |    |----+----+
///     |    |    |    |
///     |    |----+----+
///     |    |
///     |    |
///     | +----------+
///     +-|          |
///     | +----------+
///     |    |
///     ......
class MyGridLayout implements SliverGridLayout {
  final double _childWidth;
  final double _childHeight;
  final double _collectionHeight;

  final List<MyCollection> _collections;

  MyGridLayout(this._collections, this._childWidth, double maxChildHeight,
      this._collectionHeight)
      : _childHeight = maxChildHeight;

  @override
  double computeMaxScrollOffset(int childCount) {
    return _collections.map((c) => c.isBadge ? 0.0 : _collectionHeight).sum;
  }

  @override
  SliverGridGeometry getGeometryForChildIndex(int index) {
    var consumed = 0;
    var scrollOffset = 0.0;
    assert(_collections.isNotEmpty);
    for (final collection in _collections) {
      if (index == consumed++) {
        if (collection.isBadge) {
          return _badgeGeometry(collection, scrollOffset);
        } else {
          return _headerGeometry(collection, scrollOffset);
        }
      }
      if (consumed + collection.childCount > index) {
        final childIndex = index - consumed;
        return _childGeometry(childIndex, scrollOffset, _childHeight);
      }
      consumed += collection.childCount;
      scrollOffset += collection.isBadge ? 0 : _collectionHeight;
    }
    throw StateError("can't get geometry for index $index");
  }

  @override
  int getMaxChildIndexForScrollOffset(double scrollOffset) {
    var consumed = 0.0;
    var index = -1;
    for (final collection in _collections) {
      // Increment for the collection.
      index++;
      consumed += collection.isBadge ? 0 : _collectionHeight / 2;
      // The children can appear starting at halfway down the parent height.
      // First check if the scrollOffset is earlier than that.
      if (scrollOffset <= consumed) {
        return index;
      }

      var consumedByChildren = 0.0;
      for (var availableChildren = collection.childCount;
          availableChildren > 0;
          availableChildren -= 2) {
        consumedByChildren += _childHeight;
        index += min(2, availableChildren);
        if (scrollOffset <= consumed + consumedByChildren) {
          return index;
        }
      }

      final height = collection.isBadge ? 0 : _collectionHeight;
      if (consumedByChildren < height) {
        consumed += height / 2; // - consumedByChildren;
      }
      if (scrollOffset <= consumed) {
        return index;
      }
    }
    return index;
  }

  @override
  int getMinChildIndexForScrollOffset(double scrollOffset) {
    var consumed = 0.0;
    var index = -1;
    for (final collection in _collections) {
      index++;
      consumed += collection.isBadge ? 0 : _collectionHeight;
      if (scrollOffset <= consumed) {
        return index;
      }
    }
    return 0;
  }

  SliverGridGeometry _headerGeometry(
      MyCollection collection, double scrollOffset) {
    return SliverGridGeometry(
        crossAxisOffset: 0,
        crossAxisExtent: _childWidth,
        mainAxisExtent: _collectionHeight,
        scrollOffset: scrollOffset);
  }

  SliverGridGeometry _badgeGeometry(
      MyCollection collection, double scrollOffset) {
    return SliverGridGeometry(
        crossAxisOffset: 50,
        crossAxisExtent: _childWidth * 3 - 100,
        mainAxisExtent: 50,
        scrollOffset: scrollOffset - 25);
  }

  SliverGridGeometry _childGeometry(
      int childIndex, double scrollOffset, double childHeight) {
    final columnIndex = childIndex % 2;
    final rowIndex = childIndex ~/ 2;
    return SliverGridGeometry(
        crossAxisOffset: _childWidth + _childWidth * columnIndex,
        crossAxisExtent: _childWidth,
        mainAxisExtent: childHeight,
        scrollOffset:
            scrollOffset + rowIndex * childHeight + _collectionHeight / 2);
  }
}

/// Represents either a collection with a number of children or a "badge" that
/// is shown between/over other collections.
class MyCollection {
  final int _childCount;
  final bool isBadge;
  bool hideChildren = false;

  MyCollection(this._childCount) : isBadge = false;

  MyCollection.badge()
      : _childCount = 0,
        isBadge = true;

  int get childCount {
    if (isBadge || hideChildren) {
      return 0;
    } else {
      return _childCount;
    }
  }

  @override
  String toString() => isBadge
      ? 'MyCollection.badge()'
      : 'MyCollection(childCount: $childCount)';
}

flutter doctor -v
[!] Flutter (Channel stable, 3.16.0, on macOS 14.1.1 23B81 darwin-arm64, locale en-GB)
    • Flutter version 3.16.0 on channel stable at /Users/nexus/dev/sdks/flutter
    ! Warning: `flutter` on your path resolves to /Users/nexus/dev/sdks/flutters/bin/flutter, which is not inside your current Flutter SDK checkout at /Users/nexus/dev/sdks/flutter. Consider adding /Users/nexus/dev/sdks/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to /Users/nexus/dev/sdks/flutters/bin/dart, which is not inside your current Flutter SDK checkout at /Users/nexus/dev/sdks/flutter. Consider adding /Users/nexus/dev/sdks/flutter/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision db7ef5bf9f (6 days ago), 2023-11-15 11:25:44 -0800
    • Engine revision 74d16627b9
    • Dart version 3.2.0
    • DevTools version 2.28.2
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0-rc1)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0-rc1
    • Java binary at: /Users/nexus/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0)
    • Xcode at /Applications/Xcode-15.0.0-Release.Candidate.app/Contents/Developer
    • Build 15A240d
    • CocoaPods version 1.13.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Users/nexus/Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.5)
    • IntelliJ at /Users/nexus/Applications/IntelliJ IDEA Ultimate.app
    • Flutter plugin version 76.3.4
    • Dart plugin version 232.10072.19

[✓] VS Code (version 1.84.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.76.0

[✓] Connected device (4 available)
    • Nexus (mobile)       • 00008020-001875E83A38002E • ios            • iOS 17.1.1 21B91
    • Dean’s iPad (mobile) • 00008103-000825C811E3401E • ios            • iOS 17.1 21B74
    • macOS (desktop)      • macos                     • darwin-arm64   • macOS 14.1.1 23B81 darwin-arm64
    • Chrome (web)         • chrome                    • web-javascript • Google Chrome 119.0.6045.159

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 3.17.0-12.0.pre.39, on macOS 14.1.1 23B81 darwin-arm64, locale en-GB)
    • Flutter version 3.17.0-12.0.pre.39 on channel master at /Users/nexus/dev/sdks/flutters
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b6ef78e955 (6 hours ago), 2023-11-20 23:37:25 -0500
    • Engine revision 70b1c73412
    • Dart version 3.3.0 (build 3.3.0-152.0.dev)
    • DevTools version 2.30.0-dev.4

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0-rc1)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0-rc1
    • Java binary at: /Users/nexus/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0)
    • Xcode at /Applications/Xcode-15.0.0-Release.Candidate.app/Contents/Developer
    • Build 15A240d
    • CocoaPods version 1.13.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Users/nexus/Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.5)
    • IntelliJ at /Users/nexus/Applications/IntelliJ IDEA Ultimate.app
    • Flutter plugin version 76.3.4
    • Dart plugin version 232.10072.19

[✓] VS Code (version 1.84.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.76.0

[✓] Connected device (4 available)
    • Nexus (mobile)       • 00008020-001875E83A38002E • ios            • iOS 17.1.1 21B91
    • Dean’s iPad (mobile) • 00008103-000825C811E3401E • ios            • iOS 17.1 21B74
    • macOS (desktop)      • macos                     • darwin-arm64   • macOS 14.1.1 23B81 darwin-arm64
    • Chrome (web)         • chrome                    • web-javascript • Google Chrome 119.0.6045.159

[✓] Network resources
    • All expected network resources are available.

• No issues found!

b/312248242 upstream

cc @Piinks @chunhtai

Not sure where the issue is yet, but here is what I have found so far.
When I run the repro, I see grid item 2 is a small box towards the top. If I comment out findChildIndexCallback to see what happens without a crash, and push the button, grid item 2 becomes a different widget, it is a large box further down the list. Additionally, grid item 2 is being assigned different keys when this change happens.
When it crashes instead, the custom grid layout from the customer code is determining that only index 2 needs to be laid out, and that children up to index 1 and after index 4 need to be disposed, but that seems like the wrong indices since it omits index 3. 🤔
Restoring findChildIndexCallback means that the SliverMultiBoxAdaptorElement._childElements are manipulated during performRebuild. Looking at this in the crash/no-crash scenario, it looks like the childCount used in RenderSliverGrid.performLayout is not being updated when the child elements are manipulated. The childCount is what is firing the assertion and causing the crash.
In the no-crash scenario it plays out like this:
// Before pressing button
in RenderSliverGrid.performLayout
  grid layout says indices 4-7 should be laid out, childCount is 4
  Nothing to garbage collect.

// Button pressed!
at the end of SliverMultiBoxAdaptorElement.performRebuild (called 4 times with same result)
  _childElements.length is 4
next in RenderSliverGrid.performLayout
  grid layout says indices 2-2 should be laid out, childCount is 3
  Garbage is collected for up to index 0 and >= index 3 (still seems off in the no crash case)

Compared to the crashing scenario:
// Before pressing button
in RenderSliverGrid.performLayout
  grid layout says indices 4-7 should be laid out, childCount is 4
  Nothing to garbage collect.

// Button pressed!
at the end of SliverMultiBoxAdaptorElement.performRebuild (called 3 times with same result)
  _childElements.length is 3
at the end of SliverMultiBoxAdaptorElement.performRebuild (called once more with new result)
  _childElements.length is 2
next in RenderSliverGrid.performLayout
  grid layout says indices 2-2 should be laid out, childCount is 4
  Garbage is collected for up to index 1 and >= index 4

Other bits:
In RenderSliverGrid.performLayout, the childCount as well as the first and last indices of the prior layout pass are used to determine garbage collection along with the new first and last index provided by the SliverGridLayout (which is a custom one in the OP). It could be that RenderSliverGrid is not doing a good job of supporting children being removed and inserted like in this case.
This is what I have found so far, I will be out from here through the holiday. Passing it on to @chunhtai in case he discovers something while I am out!

I found the root cause, in SliverGrid.performLayout, we assume the children's indices are continuous, but that is not true when you remove child in the middle during a rebuild.
I will put up a fix shortly


introduce AqlItemBlock::_nrEffectiveItems

Scope & Purpose
Introduce _nrEffectiveItems for AqlItemBlock, which keeps track of the maximum modified (i.e. written-to) row number in the AqlItemBlock.
It is updated on every write, and read back when mass operations, such as cleanup or copying, or executed for all rows of the block. The idea is that such operations can be saved for rows that are known to be empty anyway.
This is a minimal backport of https://github.com/arangodb/arangodb/pull/12473, omitting all the renamings done in that PR.

[ ] :hankey: Bugfix
[x] :pizza: New feature
[ ] :hammer: Refactoring
[x] :book: CHANGELOG entry made
[x] :muscle: The behavior in this PR was manually tested
[x] :computer: The behavior change can be verified via automatic tests

Backports:

[x] No backports required
[ ] Backports required for

Testing & Verification

[ ] This change is a trivial rework / code cleanup without any test coverage.
[x] This change is already covered by existing tests, such as shell_server_aql.

http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/11543/

Tests blue

I think you could rename it to maxModifiedIndex

http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/11623/

Tests blue


Update notification in mini player could be improved

OS:
OSX
Issue Descriptions:
The update notification doesn't look look with the smal player. Maybe just have the heading and the buttons, with some text ("see more..." or something) expanding the full player and showing the current full text?

Steps to Reproduce:
Have mini player open when an update notification arrives

This was fixed in the latest release


Start Tetun list

I created a new project as this here is not maintained anymore, including you list:
https://github.com/LDNOOBWV2/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words_V2/


WASM support

Refactored platform detection to make it more robust
Added WASM support

TODO:

[ ] Add tests
[ ] Configure CI to test WASM targets

Fixes  #53

Gave this a shot with a fairly standard template component, it looks like the embedding works but cargo-audit might not have Wasm support. https://github.com/brooksmtownsend/cargo-auditable-wasmcloud
It's also possible I'm not using cargo audit properly 😄

Indeed, cargo audit doesn't support WASM yet. That will also have to be implemented.
Back when I was writing cargo auditable, I've found that the libraries for extracting data did not handle untrusted input particularly well - they could allocate unbounded amounts of memory, and also panicked on certain inputs. Because of that I've written https://github.com/Shnatsel/binfarce, which cannot have such failure modes by design.
I'm not sure how robust the wasmparser crate is. It seems to be doing a whole lot more parsing than what we actually need, presenting a rather big attack surface. I don't know how resilient it is to untrusted input; I don't see any indication that the parser has been fuzzed, for example.
It shouldn't be hard to extend binfarce to also parse WASM custom sections, if wasmparser proves unsuitable.

Well, it seems wasmparser actually was designed for parsing untrusted input - it is dealing with WASM after all. Fuzzing it didn't seem to turn up any issues. There is one unsafe block but there is no way to express that in safe code, and it appears to be correct. I think I'll be able to wire it up to the extraction pipeline.
The part I am not thrilled about is that I can no longer guarantee absence of heap allocations and therefore absence of OOM denial-of-service if I use wasmparser. Adding WASM support to binfarce would avoid that.

I've added the extraction pipeline using wasmparser. You can test it with rust-audit-info in-tree.
IIRC cargo audit will require a bit more work, but it is going to be quite trivial.

Sadly wasmparser pulls in more dependencies than I would like: https://github.com/bytecodealliance/wasm-tools/issues/1528
It doesn't look like they're actually using them when the std feature is enabled, so it might not be active attack surface - but it's weird to lug all this dead weight around, and complicate packaging for distributions.


Error message when type of analysis does not exist/is not set up

I have come across several users who have the problem that they try to run an analysis which is not set up or does not exist (for example model contains no load combinations). It locks the GH script and you have to close down the FD instance. Could we create an error handler for this common user mistake?

@Marco-Pellegrino What do you mean? I can run analysis with calcGmax, calcStab and calcSeis :)
I have raised a bug for CalcImp on FogBugz https://strusoft.fogbugz.com/f/cases/122810/StruXml-Can-t-run-imperfection-analysis. This is a bug in fdscript, but will be fixed in FEM-Design 21.00.006

Interesting.
How do we run CalcSeis if we can not specify anything about the seismic analysis?
How do we run CalcStab if we can not specify any settings?

Stability analysis settings are on the CombItem
If you have a model (.str) with elements, loads and setting you can run the seisemic analysis in C# at least. But I realise now that we can't run the seisemic analysis in grasshopper since the component require a Model (struxml) as well. However with the pipe I think we can make this work :)


Getting rid of zoom in while selecting by removing 'setViewPortOffsets'

(Pulled this out of the cells branch as it didn't really belong there)

Tried to pull out certain commits, but that looks like it might be harder.


Replace deprecated assert functions in test files.

Several of the assert functions we've been using in our tests are technically marked as deprecated, so I went ahead and updated those to the supported functions so that the build won't randomly break once those assert functions are finally removed. The asserts were updated in the following ways:
assert.equal -> assert.strictEqual
assert.notEqual -> assert.notStrictEqual
assert.deepEqual -> assert.deepStrictEqual
assert.notDeepEqual -> assert.notDeepStrictEqual
I also update the usage of a deprecated overload of assert.fail.
Since these asserts are now strict versions, I had to update some tests to be more specific about their expected data values. The only test where this proved to be a little annoying was with outputProcessor.test.ts, since the actual results in those tests have a null object prototype, so all our expected data objects had to be updated to be created with a null prototype as well in order to pass assert.deepStrictEqual.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 42.087%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
26468

Relevant Lines:
56993

💛  - Coveralls


Run in node.js

I try  run glyphhanger in node like this
const cp = require('child_process')

cp.exec('glyphhanger --version')
cp.exec('glyphhanger --subset="*.ttf"')

It's not working and process never ends. Empirically, I found a place where this happens - here. We waiting for input but it doesn't happen. Can you help me?

@g-rodigy not sure if you're still running into this or not. I was able to run Glyphhanger in Node using the spawn method:
const cp = require('child_process');

const glyphhangerProcess = cp.spawn(
  'glyphhanger',
  [
    // Spaces in the whitelist breaks the CLI script
    `--whitelist=${subset.unicodeRange.replaceAll(' ', '')} `,
    `--subset=Inter-${font.name}.woff2`,
    '--output=subsets',
    '--formats=woff2',
  ],
  { stdio: 'inherit' }
);

glyphhangerProcess.on('close', () => {
  console.log('glyphhanger completed');
});

@g-rodigy not sure if you're still running into this or not.
I ran into a similar issue but was able to get it working using the spawn method:
const cp = require('child_process');

const glyphhangerProcess = cp.spawn(
  'glyphhanger',
  [
    // Pass in flags here
    '--subset="*.tff"',
  ],
  { stdio: 'inherit' }
);

glyphhangerProcess.on('close', () => {
  console.log('glyphhanger completed');
});

Any suggestions on how I could create a glyphhanger gulp task in gulpfile.js, perhaps using the above code? Thanks.

@SamuelMiller , it's been a little while since I wrote a gulp task, but I think the majority of the code above could be repurposed for that. I think the main relevant change would be to call the cb() function from the gulp task when the task completes. Maybe something like this?
const cp = require('child_process');

function runGlyphhanger(cb) {
  const glyphhangerProcess = cp.spawn(
    'glyphhanger',
    [
      // Pass in flags here
      '--subset="*.tff"',
    ],
    { stdio: 'inherit' }
  );

  glyphhangerProcess.on('close', () => {
    console.log('glyphhanger completed');
    // Run `cb` to complete the Gulp task
    cb();
  });
}

// This assumes this is the only task in this file and you want to export it for
// use in other Gulp files.
// If that's not true you could change this to a call to `series` or
// `parallel` to run this task and other tasks
// @see https://gulpjs.com/docs/en/getting-started/creating-tasks/
exports.default = runGlyphhanger;

I haven't tested this code but it seems like it should work. If you try it and get an error let me know and I may be able to help (though I'll be out of the office for the rest of the week after today)

Thanks for responding. When I return to the part of my current web project that will be using glyphhanger to optimize my fonts, I'll test out the code and get back to you.

Try execSync:
// example:
const result = execSync('npx glyphhanger http://localhost:3001').toString();
console.log(result)


Error: the fixed layout requires the slimscroll plugin!

I'm a new bye guy with webpack and its stuffs.
I just want to know how to add this pligin via webpack or other way.
i'm using the fixed layout (changed in config/adminlte.php ) and now, i'm just trying to execute npm run dev and it is giving another error like "app.scss doesn't export content".
Could you help me explaing to me how to understand and solve problems arround webpack n'stuffs?

This is currently not supported. You are welcome to submit a PR to add this feature.


Added deadline field in project Model

Solved #646

Please extend the pr to cover the whole issue, front end work as well

resolve conflicts before review

i think you reverted some recent changes, please merge carefully with upstream/develop2


Occasionally managed rke1 cluster re-provisioning fails using terraform provider

Rancher Server Setup

Rancher version: 2.6.2
Installation option (Docker install/Helm Chart):

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): RKE2

Proxy/Cert Details:

Information about the Cluster

Kubernetes version: 1.21.5
Cluster Type (Local/Downstream): local

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider):

Describe the bug
Occasionally managed rke1 cluster re-provisioning fails using terraform provider
provider version 1.21.0
To Reproduce

provision managed rke1 cluster using terraform provider
destroy it
re-provision

Result
Provisioning fails with following error:
module.cluster.rancher2_cluster.this: Creating...
│ Error: Bad response statusCode [409]. Status [409 Conflict]. Body: [baseType=error, code=AlreadyExists, message=clusterregistrationtokens.management.cattle.io "default-token" already exists] from [https://api.rancher.test.com/v3/clusterregistrationtokens]

Expected Result
Managed cluster provisioned sucessfully
Screenshots

Additional context
After destroy the default-token goes away from from ClusterRegistrationTokens along with namespace that was created for the new cluster but it still fails with already exists error.

UPD: was even able to get this error on a very first cluster on a fresh rancher installation

this is still actual issue, please consider reopening it

Relevant bug in the terraform rancher2 repo: https://github.com/rancher/terraform-provider-rancher2/issues/867

@riuvshyn @iTaybb , could you confirm this issue is no longer happening with Rancher TF provider version v1.24.2?

@riuvshyn @iTaybb , wondering if you've had a chance to verify this?

I didn't have an exact way to trigger this bug everytime, it happened once every 25 deployments or so.
But - I've upgraded to the latest version around one month ago, and so far we didn't encounter this issue.


Port over docker build --secret and docker build --ssh (low hanging fruit)

This would be really useful - it's so hard to build code from private repositories using Docker.

Any update on when this may be implemented?

Any workarounds? Not having a lot of luck on google searching for img sshagent

img project is not under active development recently.
I'd suggest using buildctl-daemonless.sh build --ssh instead.
https://github.com/moby/buildkit/blob/master/examples/buildctl-daemonless/buildctl-daemonless.sh


Cannot use file for targer as namp will fail to resolve

Given a list of IPs
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>

and command: docker run -v "$(pwd):/data" --net=host -t ullaakut/cameradar -t /data/ips.txt -d -p 554
nmap reports
> [Nmap Warning] Failed to resolve "<IP_ADDRESS>
".> [Nmap Warning] Failed to resolve "<IP_ADDRESS>
etc...

and only the last IP in the list is actually scanned.
The file is a UTF-8 plaintext file:

I'm running docker desktop on Windows with the provided example commands

You try to separate the ips with comma ,

Hi @zorrobyte
The target list should indeed be newline-separated, so your file format seems good.
On your host machine, where is the file located? Is it in $(pwd)?
Why iis the error talking about a target at IP <IP_ADDRESS> even though that one is not in your target file?

Hi @zorrobyte
The target list should indeed be newline-separated, so your file format seems good.
On your host machine, where is the file located? Is it in $(pwd)? Why iis the error talking about a target at IP <IP_ADDRESS> even though that one is not in your target file?

I was just using pwd to mount the volume containing my file into the container (current dir), it was on my C: root at the time.
<IP_ADDRESS> was just a random IP I changed from my own, so I don't doxx myself.
I'll try comma separating and trying again, will report back in a few days, or sooner

Replicated @Ullaakut :

Adding an additional space doesn't work, nor does adding a comma @vgpastor
This is a plaintext file formatted in UTF-8 I'm running Windows 11 with Docker Desktop in WSL2 mode, I have also tried Hyper-V mode with no change.
Can you attempt repo and share your OS and setup? I can try to replicate using the same.

I've never attempted doing it in Windows, natively or through WSL, so that might be the source of the issue, but I don't see why it would be. I will check that the loader feature didn't get broken but last time I checked it worked fine on Linux and MacOS 🤔
Thanks for the detailed report 🙏

I replicated this on a new Ubuntu install, sadly.
docker run -v "$(pwd):/data" --net=host -t ullaakut/cameradar -t /data/ipList.txt -d -p 554
Successfylly parsed targets file with 59 entriesLoading credentials...ok
  > Loading credentials dictionary from path "/app/dictionaries/credentials.json"
  > Loaded 14 usernames and 51 passwords
Loading routes...ok
  > Loading routes dictionary from path "/app/dictionaries/routes"
  > Loaded 194 routes
Scanning the network...ok
  > [Nmap Warning] Failed to resolve "".
  > [Nmap Warning] Failed to resolve "".
  > [Nmap Warning] Failed to resolve "".
  > Found 0 RTSP streams

Let me know how I can help further


about face tracking

first of all thank you so much for your marvelous work.
second, regarding face tracking why do use it why don't you just extract the landmarks from every frame by the landmark detector
thanks in advance

and how did the file mean_pts3d.npy created
is that the mean of this point across the dataset ?

it is the average of landmarks of all video frames of the target person

Using 3d landmarks obtained by face tracking has several advantages over directly using detected 2D landmarks. It helps disentangle the camera parameters, head pose, and facial movements, which allow explicit control over these parameters while using 2D landmarks can't do it. Besides, it is much easier for networks to learn normalized facial movements (in 3D space) than entangled landmarks, generating more accurate results.
'mean_pts3d.npy' should be the mean 3d landmarks of the target person in the training set. The network learns the displacements instead of the absolute locations.

Hope the above helps.

thank you so much for your reply
please i have another question

here the audio_features: [b, T, ndim]
from the paper i understand that T = 240 is the batch size (number of consecutive frames ) and ndim is 512
i don't get what is b

thank you so much for your reply it really helped
I have another question about sequence length in the code you define sequence length to be 240
in the paper 240 is the batch size (𝑇 = 240 represents the number of consecutive frames sent to
the model at each iteration)  that means it is the batch size ??
If 240 is sequence length then audio features sent during training will be
[b,T,ndim] = 32(batch size) and 240(seq_length) and 512 (ndim)
or do you mean that you will send batches of one each batch contain sample with 240 seq_length
thanks in advance

Of course the latter one. LSTM is a kind of RNN network, and it should take sequential data as input. 240 frames equal to 4 seconds under the 60 FPS setting.
Batch_size means for each forward pass, how many batches of sequential data (240 frames data) are sent.

thank you so much i don't know how to thank you you really helped me !

First, LSTM takes sequential data as input and its output is also sequential, therefore T frames input results in T frames output. Please carefully check the definition of LSTM networks. During training, we use 4 seconds as the length while during the test there is no length limitation.
Secondly, the audio2mouth network learns the displacements.
Thirdly, frame2* is because the APC feature frame is half of the 1/60. Please check the paper for details.

thank you for your replays,
I am wondering do you know any open source algorithm for face tracking as i can't find one to produce the same output of your paper.
thanks in advance

Any parametric monocular face reconstruction method would be an alternative, like FaceScape, DECA, 3DDFA_v2, etc.

Any parametric monocular face reconstruction method would be an alternative, like FaceScape, DECA, 3DDFA_v2, etc.

What the method did you use? could you please upload the code?


Include PackageDownload in targets

I don't like adding the boilerplate PackageDownload line to every csproj. It would be nice to have this in the targets:
  <ItemGroup Condition="'$(AnnotatedReferenceAssemblyVersion)' != ''">
    <PackageDownload Include="Microsoft.NETCore.App.Ref" Version="[$(AnnotatedReferenceAssemblyVersion)]" />
  </ItemGroup>

If control of the package name is desired, we could do this:
  <PropertyGroup Condition="'$(AnnotatedReferenceAssemblyPackageName)' != ''">
    <AnnotatedReferenceAssemblyPackageName>Microsoft.NETCore.App.Ref</AnnotatedReferenceAssemblyPackageName>
  </PropertyGroup>

  <ItemGroup Condition="'$(AnnotatedReferenceAssemblyVersion)' != ''">
    <PackageDownload Include="$(AnnotatedReferenceAssemblyPackageName)" Version="[$(AnnotatedReferenceAssemblyVersion)]" />
  </ItemGroup>

This is likely not possible (it is certainly not possible for PackageReference, but I haven't tried with PackageDownload). NuGet does not include props/targets from packages when evaluating the dependency graph.

This makes sense. The package targets aren't available until after the restore phase and the PackageDownload item needs to be present before the restore phase.


How can I determine all the top level Java packages available in Ghidra at runtime?

Hello,
I've been trying to resolve an issue using the excellent python project pyhidra, but I'm stuck on something.
https://github.com/dod-cyber-crime-center/pyhidra/pull/18#issuecomment-1212435519
How can I dynamically list which top level classes are available in Ghidra at runtime (specifically Headless runtime)?
I tried to statically build a list of all the top level base Ghidra java packages with:
$ find /ghidra/ -name *src.zip -exec sh -c "zipinfo -1 {}" \; | grep -v "/." | sort | uniq -c | grep '/'
      1 SWIG/
      4 agent/
      3 db/
      1 decompiler/
      1 dist/
      2 docking/
      1 foundation/
      1 functioncalls/
      3 generic/
     52 ghidra/
      2 help/
      1 lib/
      1 log/
      1 mdemangler/
      1 org/
      1 pdb/
      1 resource/
      1 resources/
      1 src.META-INF/
      1 src.win32/
      1 src/
      1 util/
      2 utilities/
      1 utility/

But that list isn't perfect, and could be different a runtime.
Perhaps all the Java runtime classes are added here during initialization?https://github.com/NationalSecurityAgency/ghidra/blob/f9463e600d318a74e81462656e4a87f3e598ddec/Ghidra/Framework/Utility/src/main/java/ghidra/GhidraLauncher.java#L87-L105
Is there any way to create that list through the FlatProgramApi?

Seems like the better solution is to refactor the code in the pdb package to ghidra.something.something.pdb.???

It would be easier to change the namespace, but the hope is not directly modify the Ghidra source Java files.

I think I found it. I was somewhat conflating classes with packages, but I found a way to get the top level Java packages.
Found this example
// Java program to demonstrate
// getPackages() method

public class Test {
    public static void main(String[] args)
    {

        // get the Packages using getPackages() method
        Package[] packages = Package.getPackages();

        // print all packages, one by one
        for (int i = 0; i < packages.length; i++) {
            System.out.println(packages[i]);
        }
    }
}

Running this basically gets what I need!
Top level packages list:
{'java', 'decompiler', 'foundation', 'help', 'mdemangler', 'dc3', 'log', 'pdb', 'docking', 'javax', 'agent', 'sun', 'resources', 'com', 'util', 'generic', 'net', 'utility', 'db', 'edu', 'functioncalls', 'SWIG', 'ghidra', 'jdk', 'org', 'utilities'}

Cheers!


[Core] SPARK-5954: Top by key

This PR implements two functions

topByKey(num: Int): RDD[(K, Array[V])] finds the top-k values for each key in a pair RDD. This can be used, e.g., in computing top recommendations.

takeOrderedByKey(num: Int): RDD[(K, Array[V])]  does the opposite of topByKey

The sorted is used here as the toArray method of the PriorityQueue does not return a necessarily sorted array.

Test build #28735 has started for   PR 5075 at commit 0895c17.

This patch merges cleanly.

Test build #28735 has finished for   PR 5075 at commit 0895c17.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28735/
Test FAILed.

Test build #28746 has started for   PR 5075 at commit 70c6e35.

This patch merges cleanly.

Test build #28746 has finished for   PR 5075 at commit 70c6e35.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28746/
Test PASSed.

Looking good, but what about the Java / Python APIs too?

Test build #28815 has started for   PR 5075 at commit 901b0af.

This patch merges cleanly.

Thanks for the contribution.
I think we should first put this in MLlib, and then consider moving it to RDD in the future. The RDD API is fairly crowded, and adding more functions to it will make it harder to learn/read/understand.

Test build #28815 has finished for   PR 5075 at commit 901b0af.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28815/
Test PASSed.

Test build #28891 has started for   PR 5075 at commit 82dded9.

This patch merges cleanly.

@rxin @mengxr per the comments, I created MLPairRDDFunctions.scala and moved the function there in the update.

Test build #28897 has started for   PR 5075 at commit a80e0ec.

This patch merges cleanly.

Test build #28898 has started for   PR 5075 at commit 6f565c0.

This patch merges cleanly.

Test build #28891 has finished for   PR 5075 at commit 82dded9.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class MLPairRDDFunctions[K: ClassTag, V: ClassTag](self: RDD[(K, V)]) extends Serializable

  class CheckpointWriteHandler(

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28891/
Test PASSed.

Test build #28897 has finished for   PR 5075 at commit a80e0ec.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class MLPairRDDFunctions[K: ClassTag, V: ClassTag](self: RDD[(K, V)]) extends Serializable

  class CheckpointWriteHandler(

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28897/
Test PASSed.

Test build #28898 has finished for   PR 5075 at commit 6f565c0.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class MLPairRDDFunctions[K: ClassTag, V: ClassTag](self: RDD[(K, V)]) extends Serializable

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28898/
Test PASSed.

Test build #28906 has started for   PR 5075 at commit 1611c37.

This patch merges cleanly.

Test build #28906 has finished for   PR 5075 at commit 1611c37.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class MLPairRDDFunctions[K: ClassTag, V: ClassTag](self: RDD[(K, V)]) extends Serializable

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28906/
Test PASSed.

LGTM. Merged into master. Thanks!

Thanks!


Update Release.yml

:x: Build portainer-ci 1.0.38 failed (commit https://github.com/ssbkang/portainer/commit/0427e0f434 by @ssbkang)

:x: Build portainer-ci 1.0.39 failed (commit https://github.com/ssbkang/portainer/commit/15de08f8fb by @ssbkang)

:white_check_mark: Build portainer-ci 1.0.40 completed (commit https://github.com/ssbkang/portainer/commit/00e161cee4 by @ssbkang)

:x: Build portainer-ci 1.0.43 failed (commit https://github.com/ssbkang/portainer/commit/85c4196c8f by @ssbkang)


Haddock Documentation (for people who may contribute to this) would be useful.

For those who want to help out with haskellnews, Haddock documentation would be immensely helpful.

PR welcome


@DocumentId breaks JSONEncoder

[REQUIRED] Step 1: Describe your environment

Xcode version: 11.5
Firebase SDK version: Firebase/Core (6.26.0)
Firebase Component: Firestore
Component version: FirebaseFirestore (1.15.0), FirebaseFirestoreSwift (0.3.0)
Installation method: CocoaPods

[REQUIRED] Step 2: Describe the problem
Steps to reproduce:
Try to encode a model, which is using @DocumentId, as JSON using Swift's JSONEncoder. 💥
▿ FirestoreEncodingError
  - encodingIsNotSupported : "DocumentID values can only be encoded with Firestore.Encoder"

Relevant Code:
struct Model: Codable {
  @DocumentId var id: String? = nil
  var foo = "bar"
}
let model = Model()
try! JSONEncoder().encode(model)

@mrackwitz Thanks for filing this issue!
This is actually working as intended as DocumentReferences contain state that is not serializable to JSON. Please also see https://github.com/firebase/firebase-ios-sdk/issues/5438#issuecomment-619262231


Ch3hw9implementation

Do not merge this PR.
This implements requirements (1) and (2) of a modified readme.md file. I will add comments on the review to changes to readme.md and Tests/** changes tomorrow. There were a few clarifications or data errors requiring corrections, which I'll comment on review for justifications for all changes made.

I need to head to bed. I'm going to comment on changes made to readme.md and Tests/** over the weekend.

Thank you very much for this, it looks great. I don't have time to look at it in detail yet, but from skimming it it looked good!

Could you please make a PR for just tests part changes? I would then merge it.

Could you please make a PR for just tests part changes? I would then merge it.

I see you already did 😄 Thanks!


Updated native Pdfium build instructions, options to disable modules

I wanted to update the native PDFium libraries for myself and ended up with a few suggestions that I'd like to share. They are closer to the current build instructions and reduce the amount of customization work and don't require any actual VS IDE work (you'll still need the installation for the compilers).
While poking around the native build instructions I also found a few interesting flags (pdf_enable_xfa and pdf_enable_v8) that I might want to employ to reduce binary size/compilation time/attack surface.
To better illustrate the amount of work required, I'll use screenshots for now. But I'd be happy to create a pull request for this if and when we've hammered out the details.
Adjustments to the BUILD.gn (output DLL, include your pdfiumviewer.cpp):

Adjustments to the pdfiumviewer.cpp (conditional clean-up of V8):

Example build configuration for gn (create this in the step gn args <buildname>):

I haven't set up x64 building yet. This might provide some hints: https://groups.google.com/forum/#!topic/pdfium/aBjlQCtldyA
UPDATE 2: See notes below.
UPDATE 3: See notes below.
The build instructions of the PDFium project again for reference: https://pdfium.googlesource.com/pdfium/
UPDATE 1:
To avoid some error output (the test projects won't build because we change the library output type to shared_library), conditionally disable those projects in BUILD.gn like so:

It would be cleaner to just remove those projects from the BUILD.gn file (as they won't build even when pdf_is_standalone is true at the moment). But for illustrative purposes, I chose to keep the diff small here.
UPDATE 2:
By default, the tools will build x64 on x64 machines, it appears.
UPDATE 3:
For x86 builds on x64 machines, add a line to args.gn (see above) that says:
target_cpu = "x86"
You can also set it to x64 but that is the default on x64 machines anyway and presum. won't work on x86 machines.
To generate VS projects files (useful for checking the configuration), run:
gn gen <directory> --ide=vs
As I said above, VS IDE work is not required. Usually you'd want to just run this to produce the DLL:
ninja -C <directory>

Why would you want to disable V8? From what I understand, PDF documents may contain JavaScript. Are you proposing to maybe create an alternative version of the DLL that does not have V8?

I am not suggesting that you disable it by default. This issue is mostly about build documentation and (maybe) simplifying the process, as well as keeping the build instructions close to those of the native project.
Regarding your question:
I use your library mostly for PDF rendering. I have not encountered any PDFs yet whose basic output (incl. annotations) depends on JavaScript being executed.
Disabling the V8 module doesn't necessarily preclude JavaScript from running. From what I remember, disabling V8 doesn't disable JavaScript support altogether.
Anyway, since I do not require the JavaScript parts, I prefer to not include them as it increases the attack surface for PDF-JavaScript-based exploits. V8 is an obvious target.

Sorry for the slow response and thank you for the build instructions.
I've updated the build instructions on https://github.com/pvginkel/PdfiumViewer/wiki/Building-PDFium to use your build instructions.
I've included two versions of the Pdfium DLL's. The standard ones still include V8. However, I did another attempt at getting Windows XP compatible DLL's and it turns out that, if you use a recent dbghelp.dll version, the only thing preventing Pdfium from running on Windows XP is V8. So, now, in Libraries\Pdfium\Without V8 there are builds of the Pdfium DLL without V8 support. See https://github.com/pvginkel/PdfiumViewer/tree/master/Libraries/Pdfium for more information.

I've created a new project at PdfiumBuild to automate building the PDFium library. This project runs on a build server to create new versions daily. Have a look at that project for where to find the output.


Remove request for animation frame if width is equal to zero

Request animation frame if width is null or undefined, but not if current width is set to zero.

Duplicate of #11


Upgrade artifactory

which requires java 8
which means we have to upgrade jenkins, as the old version couldn't handle the java upgrade
much hilarity ensued updating alternatives
the chef java recipe is a cruel joke... probably best to avoid it

this is running in prod

i think there's still a problem with the java recipe forcing us back to java 7 and messing up /etc/alternatives...

nope -- the java recipe just keeps messing with java
need to manually do:
$ alternatives --config java

There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
 + 1           /usr/lib/jvm/java-1.7.0/bin/java
*  2           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
   3           /usr/lib/jvm/java-1.8.0-openjdk.x86_64/bin/java

and select option 3, which i installed with
alternatives --install /usr/bin/java java /usr/lib/jvm/java-1.8.0-openjdk.x86_64/bin/java 3

oh my this went pretty deep again... brain dumping

windows chef attributes:

  "jenkinsHomes": {
    "y:\\jenkins": {
      "java_path": "\"/cygdrive/c/Program Files/Java/jdk1.8.0_92/bin/java\""
    }
  },
  "java": { "java_home": "c:/Program Files/Java/jdk1.8.0_92/bin" },

jenkins-master: the java version attribute somehow didn't get bumped to 8, was stuck on 7 until i overrode in the UI. the old java version silently crashed artifactory

i installed java 8 manually on windows -- hardcoded the paths in chef


Add length/substring/concat functions for string type

Issue: https://github.com/TuGraph-family/tugraph-db/issues/363

@qishipengqsp
All unit tests are passed.
[20240110 10:54:32.029267 0x00007f9fd7ff2a00 DEBUG test/test_query.cpp:212] test_file: ../../test/resource/cases/demo.test
[20240110 10:54:32.029333 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:214] -----CYPHER-----
[20240110 10:54:32.029358 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:215] RETURN LENGTH('abc1234');
[20240110 10:54:32.034834 0x00007f9fd7ff2a00 DEBUG src/cypher/execution_plan/execution_plan.cpp:1413] Current Pattern Graph:
(EMPTY GRAPH)

[20240110 10:54:32.034884 0x00007f9fd7ff2a00 DEBUG src/cypher/execution_plan/execution_plan.cpp:1404] Execution Plan:
Produce Results
    Project [LENGTH("abc1234")]

[20240110 10:54:32.035070 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:177] -----result-----
[20240110 10:54:32.035100 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:179] [{"LENGTH(\"abc1234\")":7}]
[20240110 10:54:32.035152 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:214] -----CYPHER-----
[20240110 10:54:32.035250 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:215] RETURN SUBSTRING('abc1234', 4, 4);
[20240110 10:54:32.036100 0x00007f9fd7ff2a00 DEBUG src/cypher/execution_plan/execution_plan.cpp:1413] Current Pattern Graph:
(EMPTY GRAPH)

[20240110 10:54:32.036155 0x00007f9fd7ff2a00 DEBUG src/cypher/execution_plan/execution_plan.cpp:1404] Execution Plan:
Produce Results
    Project [SUBSTRING("abc1234",4,4)]

[20240110 10:54:32.036333 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:177] -----result-----
[20240110 10:54:32.036389 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:179] [{"SUBSTRING(\"abc1234\",4,4)":"1234"}]
[20240110 10:54:32.036431 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:214] -----CYPHER-----
[20240110 10:54:32.036504 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:215] RETURN CONCAT('abc', '12', '34');
[20240110 10:54:32.036742 0x00007f9fd7ff2a00 DEBUG src/cypher/execution_plan/execution_plan.cpp:1413] Current Pattern Graph:
(EMPTY GRAPH)

[20240110 10:54:32.036777 0x00007f9fd7ff2a00 DEBUG src/cypher/execution_plan/execution_plan.cpp:1404] Execution Plan:
Produce Results
    Project [CONCAT("abc","12","34")]

[20240110 10:54:32.036869 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:177] -----result-----
[20240110 10:54:32.036930 0x00007f9fd7ff2a00 INFO  test/test_query.cpp:179] [{"CONCAT(\"abc\",\"12\",\"34\")":"abc1234"}]
[20240110 10:54:32.037481 0x00007f9fd7ff2a00 DEBUG src/plugin/python_plugin.cpp:238] Killing all python processes
[       OK ] TestQuery.TestDemo (627 ms)
[----------] 2 tests from TestQuery (672 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (672 ms total)
[  PASSED  ] 2 tests.


Error in browser/ maybe ssr after starting a new project.

@julia-allyce noted an issue she had after starting a new project.  I ran into a similar issue.  See my thoughts below.
julia [6:38 PM]
uhmm I just generated a new project, and ran `donejs develop`

[6:39]
if i go to localhost:8080/production.html i get the error:

[6:39]
Uncaught SyntaxError: Unexpected token < index.js:1

[6:39]
which i assume is from parsing index.stache?

[6:40]
ehh development.html works

kevin [6:50 PM]
I think you probably just need to do `donejs build` before hitting production.html

----- Today January 23rd, 2016 -----
julia [9:44 AM]
Potentially unhandled rejection [13] SyntaxError: Error loading "package.json!npm" at http://localhost:8080/package.json
Unexpected token <

[9:44]
Im getting this really ambiguous error…I can’t figure out where it is coming from

marshall [10:03 AM]
@julia I fixed this by running `npm install` inside the node_modules/<dependency> folder for one of my dependencies, but I can’t remember which.(edited)

[10:07]
I think it’s caused by a missing sub-dependency, so where there would be an expected package.json there is nothing, so the SSR server is returning a response that starts with `<`.  Just a guess.(edited)

After talking more with @julia-allyce, I think I ran into this issue by selecting the wrong npm version during  donejs init.  She appears to be getting the issue by changing npm versions afterwards.  How does one tell Steal that we've changed versions?

@marshallswain  thanks for documenting this

Switching from which version to what version?

If you're using NPM 3.x you want to add "npmAlgorithm": "flat" to the the package.json under "system", if using NPM 2.x remove that property.  It does mean that everyone on the team needs to be using the same version. Maybe we could supporting setting this config in localStorage or something, so individuals could control it.

Has this already been documented?

Yeah, it's in the stealjs docs like here: http://stealjs.com/docs/npm.html#section_Use
If you create a new project with donejs init it will detect what version of npm you are using and add the flag if needed.

Thanks.


AVRO-2600: Swap to new properties in Velocity Engine

https://issues.apache.org/jira/browse/AVRO-2600
Make sure you have checked all steps below.
Jira

[ ] My PR addresses the following Avro Jira issues and references them in the PR title. For example, "AVRO-1234: My Avro PR"

https://issues.apache.org/jira/browse/AVRO-XXX
In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Tests

[ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

[ ] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters (not including Jira issue reference)
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Documentation

[ ] In case of new functionality, my PR adds documentation that describes how to use it.

All the public functions and the classes in the PR contain Javadoc that explain what it does

Hi, @sigand! Thanks for contributing to Apache Avro. Could you please update your PR title and rebase your commit to have the Jira ticket referenced in both the PR title and commits, as per the PR template?

@kojiromike Fixed it

Hello everyone! I still got all the above warnings with Avro version 1.9.2. Do you have any idea about this?

@maisreymom Can you share the errors?


update monitoring.md to accurately describe the history server

What changes were proposed in this pull request?
It looks like the docs were recently updated to reflect the History Server's support for incomplete applications, but they still had wording that suggested only completed applications were viewable.  This fixes that.
My editor also introduced several whitespace removal changes, that I hope are OK, as text files shouldn't have trailing whitespace.  To verify they're purely whitespace changes, add &w=1 to your browser address.  If this isn't acceptable, let me know and I'll update the PR.
I also didn't think this required a JIRA.  Let me know if I should create one.
How was this patch tested?
Not tested

Test build #54459 has started for PR 12045 at commit c543647.

Test build #54459 has finished for PR 12045 at commit c543647.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54459/
Test PASSed.

Merging into master.


Flatly: cleanup theme

Use same background color for dropdowns, cleanup header spacing, fix progress bar spacing.

Thanks! some of the themes has been added without review. I just wanted to grab them before they was deleted.

Sure :) Would you like me to review all of them, and clean them up?

If your willing and have time yes please!


[Frontend][Core] Add plumbing to support audio language models

This adds infrastructure to support multi-modal audio language models.

MultiModalDataBuiltins defines audio as a tuple of a numpy.ndarray and the sample rate. (librosa is used to parse the audio.)
DEFAULT_PLUGINS now includes an AudioPlugin
The OpenAI API frontend tentatively supports "audio_url" content parts, albeit with the same restrictions that currently apply to "image_url" parts (e.g. only one multi-modal chunk per prompt).
A number of methods/types that were only nominally vision-specific are renamed to be more generically multi-modal. For example, SupportsVision is now SupportsMultiModal.

This PR does not include any such audio models, but does include a test that exercises the OpenAI frontend and audio pipeline (see tests/entrypoints/openai/test_audio.py).
FIX #7335
BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

Are you going to work on adding a new audio model after this PR?

This is exciting! Thank you for kicking off the initiative to support audio LMMs!

Are you going to work on adding a new audio model after this PR?

If you're open to it, I'd love to bring https://github.com/fixie-ai/ultravox into vLLM. We're currently serving it via an externally registered model + some monkeypatches to handle in audio via "image_url" but we'd certainly rather it be directly supported!

Going to merge this so I can incorporate it into my #7126 tomorrow.