Json value extraction fails with mixed nested objects

The json extraction code in ParsingUtils.values() doesn't always extract the right values when the dot paths are at a mix of levels within the Json object.  For example, given the following source JSON and matching on "a.b" and "c", the extractor can't find the value of "c" because the parser gets stuck in the nested object "a":
json:
{
   "a" : { "b": "nested" },
   "c" : "top-level"
}

test case:
@Test
public void testMixed() throws Exception {
    List<Object> vals = ParsingUtils.values(parser, "a.b", "c");
    assertEquals(2, vals.size());
    assertEquals("nested", vals.get(0));
    assertEquals("top-level", vals.get(1));
}

test case output:
org.junit.ComparisonFailure:
Expected :top-level
Actual   :(not found)

Note that this test succeeds if the order of "a" and "c" happen to be reversed in the source JSON.
Here's a test case that will reproduce the problem within the existing unit tests:
public class JsonValuePathTest {
    ...

    @Test
    public void testMixedLevels() throws Exception {
        List<Object> vals = ParsingUtils.values(parser, "firstName", "address.state", "state");
        assertEquals(3, vals.size());
        assertEquals("John", vals.get(0));
        assertEquals("NY", vals.get(1));
        assertEquals("CA", vals.get(2));
    }

Fixed through the PR in master and 2.x


Incorrect(?) warning when using SparkDataSet on Databricks with S3

Description
Using a SparkDataSet on Databricks with a filepath that starts with s3:// produces a warning; Using SparkDataSet on Databricks without the /dbfs/ prefix in the filepath is a known source of error. The error comes from here. I don't think this is actually a problem.
Context
The warning is not clear about why this is a problem, using s3 doesn't seem to be a cause of error for my use case. It creates log messages which need to be filtered.
Steps to Reproduce

Use a SparkDataSet on Databricks with a filepath that starts with s3://.

Expected Result
The warning should be more clear about why this is a problem if it is actually a problem or should not be produced.
Actual Result
Confusing warning produced.
Your Environment
Occurs on current master when running on Databricks due to https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets/spark/spark_dataset.py#L286C7-L286C7.

Thanks for the heads up @alamastor. @noklam I see this comes from https://github.com/kedro-org/kedro-plugins/issues/117#issuecomment-1470891953, do you want to add some more context?

I see what's the problem. The assumption here is we assume you are using dbfs on Databricks. Is there a reason why you are not using dbfs when you are using Databricks already?
I think in this case there is no problem and we shouldn't raise the warning. In databricks if you are not using dbfs, the data will be saved to the driver node, which creates weird bug( You may see the data on the notebook you created, but you get FileNotFound on other nodes). Under the hood, Databricks also have some magic path conversion (Spark only).
What we can do here is add an extra condition for the warning.

startswith dbfs (no warning as is)
startswith local path / or relative path (show warning as is)
startswith s3 or equivalent (use CLOUD_PROTOCOL) (new change)

Thanks for the explanation @noklam. @alamastor would you like to send a pull request for this one?

Thanks for the quick response!

Is there a reason why you are not using dbfs when you are using Databricks already?
Using s3 directly seems to be working well, so there's no reason to for us to use dbfs.

I'm happy to do a PR - I'll try to get something up in the next week or so.

@alamastor Awesome! Have a look at https://github.com/kedro-org/kedro-plugins/blob/main/CONTRIBUTING.md and let us know if you need help. We are also reachable on slack.kedro.org

PR for this here :smile:


isSocketAllowedBySecurityPolicy start : device null

Hello,
I've been facing issues with the Android 12, the example provided works perfectly with any other android version, but when running on Samsung devices with Android 12 (And Oppo devices as well), I've been getting stuck at
D/SimplebluePlugin(16646): android call received: "connect" D/SimplebluePlugin(16646): Connecting to 64:A2:F9:F5:8A:33 D/BluetoothUtils(16646): isSocketAllowedBySecurityPolicy start : device null I/BluetoothSocket(16646): connect() for device 64A2F9_3 called by pid: 16646
It connects to the selected device, then disconnects instantly, or just never connects
Any suggestions would be highly appreciated
Sincerely
P.S : All permissions have been provided from the "Settings" page, and the app & package works perfectly with other android version

Hi,
Sorry for the late answer it took some time to find devices with Android 12. :)
So we couldn't replicate the issue (tested in Samsung and Xiaomi devices). @jkirtyan is our expert in that field, so if you can provide more details on what device have you tried and what was on the other end, where you tried to connect, etc.
What I can imagine that the two device's Bluetooth profile or supported profiles are different. I usually facing with similar issue when connecting to embedded systems and there is a profile mismatch.
Tested devices:

Samsung A71 SM-A715F/DS

Close the issue, if you still have any additional to it, please reopen it.


Adds pre-commit checks to the repo -- SGPROJ-1573

Change description

Please review this PR. It passes all checks locally. Feel free to fix anything that will block it from passing on Github, otherwise it should be good to go.

Once you review and approve it, you can merge it to main too (don’t forget to delete the branch). At that point, it’ll be ready to be moved to tf-auto

Type of change

[ ] Bug fix (fixes an issue)
[x] New feature (adds functionality)

Related issues

Fix #1

Checklists
Development

[x] Lint rules pass locally
[ ] Application changes have been tested thoroughly
[x] Automated tests covering modified code pass

Security

[ ] Security impact of change has been considered
[ ] Code follows company security practices and guidelines

Code review

[x] Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
[x] "Ready for review" label attached and reviewers assigned
[ ] Changes have been reviewed by at least one other contributor
[x] Pull request linked to task tracker where applicable

@JoseMotto-sg please review again


Error creating Ubuntu 20 build container

Azure Data Studio Version: 1.25.2

Steps to Reproduce:

Using mcr.microsoft.com/dotnet/sdk:latest as base image
wget -nv -O azuredatastudio.deb "https://go.microsoft.com/fwlink/?linkid=2151506"
dpkg -i azuredatastudio.deb
azuredatastudio --install-extension microsoft.admin-pack --user-data-dir='.'

results in:
/usr/share/azuredatastudio/bin/../azuredatastudio: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory

@MHacker9404 are you able to start azuredatastudio? I am trying to understand whether the error is related to install-extension.

@MHacker9404 are you able to start azuredatastudio? I am trying to understand whether the error is related to install-extension.

Within the ubuntu container that I'm trying to create? Trying to manually run ADS gives me the same exact error as above.
All I'm trying to do is a create a custom image that can be used for my build processes - I don't need to run ADS in an interactive mode in the container.  That's on my desktops...

Within the ubuntu container that I'm trying to create? Trying to manually run ADS gives me the same exact error as above.
All I'm trying to do is a create a custom image that can be used for my build processes - I don't need to run ADS in an interactive mode in the container.  That's on my desktops...

thanks, in that case it is not related to install extension, I will put it under the correct area path.
A couple more questions:

Did this ever work before?
since you said you don't need to run ADS in interactive mode, I am curious how are you planning to use the custom image with azuredatastudio?

thanks, in that case it is not related to install extension, I will put it under the correct area path.
A couple more questions:

Did this ever work before?
since you said you don't need to run ADS in interactive mode, I am curious how are you planning to use the custom image with azuredatastudio?

So this works with the AWS CodeBuild Ubuntu image to build an sqlproj as part of the buildspec.yaml.  But there were intermittent failures on loading the extensions, which is why I'm going down the road with the custom-image, so everything is already installed.  (Plus - it speeds up build times).
The interactive sessions woud be my developers using ADS on their local workstations....I just need the build capability.
I found the dockerfiles that Amazon uses to create their images - so I may start with that, and add to it....

So this works with the AWS CodeBuild Ubuntu image to build an sqlproj as part of the buildspec.yaml.  But there were intermittent failures on loading the extensions, which is why I'm going down the road with the custom-image, so everything is already installed.  (Plus - it speeds up build times).
The interactive sessions woud be my developers using ADS on their local workstations....I just need the build capability.
I found the dockerfiles that Amazon uses to create their images - so I may start with that, and add to it....

If you only need the build capability you might not need ads.
@udeeshagautam @kisantia what is the easiest way to build a .sqlproj on Linux?

If you only need the build capability you might not need ads.
@udeeshagautam @kisantia what is the easiest way to build a .sqlproj on Linux?

I wish - I've tried the MSBuild.Sdk, but having issues with the master.dacpac reference - which I need for a view in my database.
ADS builds the whole thing fine - but the buildspec is not stable enough - so I go the custom image route 😄

I wish - I've tried the MSBuild.Sdk, but having issues with the master.dacpac reference - which I need for a view in my database.
ADS builds the whole thing fine - but the buildspec is not stable enough - so I go the custom image route 😄

@MHacker9404 you may look at how we create our Linux Integration Test images https://github.com/microsoft/azuredatastudio/blob/main/build/azure-pipelines/linux/Dockerfile.  This has some apt install commands and the xvfb config steps to run ADS headless.

@MHacker9404 you may look at how we create our Linux Integration Test images https://github.com/microsoft/azuredatastudio/blob/main/build/azure-pipelines/linux/Dockerfile.  This has some apt install commands and the xvfb config steps to run ADS headless.

Will do - thank you muchly....

Will do - thank you muchly....

@MHacker9404 - if you're looking to build a sqlproj from Linux, you may find this info helpful: https://docs.microsoft.com/sql/azure-data-studio/extensions/sql-database-project-extension-build-from-command-line
It lays out which files to grab from an ADS install to build from the command line - only prerequisite then is the .NET Core SDK.

@MHacker9404 - if you're looking to build a sqlproj from Linux, you may find this info helpful: https://docs.microsoft.com/sql/azure-data-studio/extensions/sql-database-project-extension-build-from-command-line
It lays out which files to grab from an ADS install to build from the command line - only prerequisite then is the .NET Core SDK.

I close this issue based on the above discussion.  I think some of the commands in the linked docker file could make ADS work on the build container, but likely @dzsquared's suggestion to pull out the required build tools makes more sense for CI pipeline.  Please let us know if there's anything else we can assist with here.


text rendering can break with small font size and depending on the font

Godot version
4.0.1
System information
linux
Issue description
See the screenshot and mrp. As you can see just putting the text and using the provided kenney font, some of it is blurry. And this changes depending on like, the control size?

it can also break in the middle of a line if you put the wrong character (→ in this case)

Steps to reproduce
see mrp
Minimal reproduction project
blurryfont.zip

duplicate of #74694

Thanks, seems like it is.


feat: add EVM indexer

Summary
Closes: https://github.com/checkpoint-labs/checkpoint/issues/188
Closes: https://github.com/snapshot-labs/pitches/issues/29
To make indexers extractable it was separated into Indexer class which instance consumer initiaties. This allows better separation (no need for pulling deps for all networks) - everything currently is exported under either evm or starknet object, it can be extracted later (we might need to extract some common things first into other package). We also have specific types for writers for each network.
This could also be useful in the future if we put multiple APIs in single instance of checkpoint, it could accept indexers instead of just single indexer.
Test plan

Test with https://github.com/checkpoint-labs/checkpoint-template/tree/sekhmet/evm

TODO

Make sure all edge cases work.

I'm trying to index another contract on Goerli to test but eth_getLogs request timeout after few minutes here is the error:

I've tried to run couple time but still endup with same error at the same block. The last_fetched_block is stuck at 6109472. Here is the branch: https://github.com/checkpoint-labs/checkpoint-template/tree/fabien/evm-poster
Usually request timeout because there are too many events within the requested block range but I don't see any tx within the requested range, txs starting from the block 6623932: https://goerli.etherscan.io/txs?a=0x000000000000cd17345801aa8147b8d3950260ff&p=11 .
Also in _metadatas table the network identifier is not correct, showing starknet_5

@bonustrack it seems that it's some issue on Infura side that it can never resolve initial request as we are not doing lots of requests and I even lowered the maximum range we are using to 10 blocks. I asked them about it on Discord.

I've updated to last changes and even if network id is fixed, it still update last_fetched_block by chunk of 100. It's also extremely slow, take about 10sec to update 100 blocks, tried with Alchemy and Infura got the same performance, with Ankr (doesn't work / resolve). I've tried to add "disable_checkpoints": true in the config file but it seem like it still query and store checkpoints and it fail at some point with this error:

I've updated to last changes and even if network id is fixed, it still update last_fetched_block by chunk of 100.

It’s expected, I only modified the max step internally when handling requests.
I will checkout the rest but getLogs call is generally slow and takes 1s for each call. So generally we should do as little requests as possible, but this might increase likelihood of timeouts (however it seems that it can timeout regardless of range).
I assume that Anker either doesn’t support getLogs or fails differently to Infura if responses is too big (need to look into it).
I guess the issue will still be that we depend on slow getLogs call. Maybe we need better way of handling it - instead of fetching all logs we fetch only those that match tracked contracts - and if new contracts are added to sources on the fly we come back. Might solve the issue.

I guess the issue will still be that we depend on slow getLogs call. Maybe we need better way of handling it - instead of fetching all logs we fetch only those that match tracked contracts - and if new contracts are added to sources on the fly we come back. Might solve the issue.

Yes, I think this is the only way, if we can target specific contract it will be more efficient we wouldn't need to load each blocks events. It might just be a bit tricky to implement. If we prefetch events and find a new deployment event, we can't continue the prefetch process because we know 1 contract will be missing and will require to reset prefetched events. And the logic to discover new template contract address is part of a writer function, which in theory should run in the right order with the others writer functions. It sound like the solution would be to prefetch events until a new deployment is found then run indexer up to this event then continue the prefetch. But curious to know if you have some others ideas.
I think also that we will not find a perfect block range limit for the method eth_getEvent, I've used this in the past and it usually fail when there is too much events, which is something we can't predict, if there is 100 events within 2 blocks it may fail with 2 block range, while it may work for 10000 block range if there is just 10 event within this range. I imagine this would require some kind of exponential backoff to deal with range.

Yes, I think this is the only way, if we can target specific contract it will be more efficient we wouldn't need to load each blocks events. It might just be a bit tricky to implement. If we prefetch events and find a new deployment event, we can't continue the prefetch process because we know 1 contract will be missing and will require to reset prefetched events. And the logic to discover new template contract address is part of a writer function, which in theory should run in the right order with the others writer functions. It sound like the solution would be to prefetch events until a new deployment is found then run indexer up to this event then continue the prefetch. But curious to know if you have some others ideas.

I think this was considered as a one way of implementing it for Starknet before, but we decided to go other way - I can't find the discussion anymore to see what was the reasoning - ideally we should have consistent logic on all networks - at least to certain degree.

I think also that we will not find a perfect block range limit for the method eth_getEvent, I've used this in the past and it usually fail when there is too much events, which is something we can't predict, if there is 100 events within 2 blocks it may fail with 2 block range, while it may work for 10000 block range if there is just 10 event within this range. This also depend on node provider limits. I imagine this would require some kind of exponential backoff to deal with range.

Infura handles that and tells you new range to try and we handle that:
https://github.com/checkpoint-labs/checkpoint/blob/81dd6bf5aeb9083f447d69711f1e0be0cba7d1c4/src/providers/evm/provider.ts#L267-L271

@Sekhmet I just found the discussion, it's here: https://discord.com/channels/1088202060283007137/1088202061176381492/1113515047277318214
Haven't read it yet, but I believe we can do such change on both Starknet and EVM chains.

I spent some time debugging timeouts and for some reason it very rarely happens when using curl (only been able to reproduce it while running checkpoint in background), but was able to reproduce it using simple script with node-fetch:
https://gist.github.com/Sekhmet/a8f3bb693f51fb7fdf713ce5e6253ff7
This also happens when using address filter and topics filter, so I'm unsure if we can do something about it, I provided those PoCs to Infura for debugging.

If you like I can send you endpoint from others providers to see if this issue is just on Infura side or not. Even if this resolve fast I think syncing all events from a chain is never going to be fast, on the chain like Arbitrum where block time is small it will become even more problematic, we most likely would need to do the change we've discussed previously to sync only relevant contracts.

@bonustrack would be great if we can review this, would be nice to have refactor land on master to avoid conflicts.

What do you mean by "Indexed event parameters probably don't work"?

Just tried, somehow indexing seem to never get started, Checkpoint logs does not appear, same for custom console.log

It seem to works well, when I compared events I can see I have one extra event, 26 instead of 25, not sure why, this is the event details:
{
  "id": "0x3fBc546BC7Fcf1e6eC7dAdfe6eBCf3c3ad2713ed",
  "implementation": "0xC3031A7d3326E47D49BfF9D374d74f364B29CE4D",
  "deployer": "0x556B14CbdA79A36dC33FcD461a04A5BCb5dC2A70",
  "tx_hash": "",
  "created_at":<PHONE_NUMBER>,
  "created_at_block": 5717246
}

@bonustrack it shows last 25 transactions by default, and it's 26th.
https://sepolia.etherscan.io/txs?a=0x4b4f7f64be813ccc66aefc3bfce2baa01188631c
There are more transactions we just index them from specific block.

Ok actually I was looking here and seen only 25 events: https://sepolia.etherscan.io/address/0x4b4f7f64be813ccc66aefc3bfce2baa01188631c#events
Do you say there is actually 147 txs and events on this contract but sync start at a specific block and it's correct if I have 26 proxies on my Checkpoint test instance?

Yes, I didn't want to make it index everything, but we could. I was checking it by looking at all transactions in Etherscan starting at block 5717246 (the one we have configured) and making sure all of those are also present in Checkpoint.


WslRegisterDistribution failed with error: 0x80041002 Error: 0x80041002 (null)

Windows Version
Microsoft Windows [Version 10.0.22631.3737]
WSL Version
<IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
No response
Distro Version
Kali Linux <IP_ADDRESS> (rolling)
Other Software
No response
Repro Steps

Run command "wsl -install" in terminal

Expected Behavior
An installation of WSL of either the default distro Ubuntu or Kali linux.
Actual Behavior

It shows this error.

When I tried to run using microsoft store, it said installing and then I got this error.
I ran sfc /scannow but it shows that there were no problems. Virtualisation is enabled in task manager.
Diagnostic Logs
No response

WslLogs-2024-07-03_15-04-51.zip

How do I install it then?

The easiest way would be to install the latest MSI package

The easiest way would be to install the latest MSI package

Installing using this method works for me Thanks 👏


Any way to improve performance?

Really nice work on this! Can you think of a way to improve performance? Adding the headers makes the list scrolling not very smooth, this is even on a Note 4. The onDraw adds over 70ms per draw, can anything be done you think?

From a quick look there is an easy gain to be made in [HeaderPositionCalculator.hasNewHeader(int position)] (https://github.com/timehop/sticky-headers-recyclerview/blob/master/library/src/main/java/com/timehop/stickyheadersrecyclerview/HeaderPositionCalculator.java#L47) by not calling mAdapter.getHeaderId(position) twice :smiley:

Nice, thanks, do you want to PR it?
On Apr 16, 2015 12:01 PM, "Jamie Clarke"<EMAIL_ADDRESS>wrote:

From a quick look there is an easy gain to be made in HeaderPositionCalculator.hasNewHeader(int
position)
https://github.com/timehop/sticky-headers-recyclerview/blob/master/library/src/main/java/com/timehop/stickyheadersrecyclerview/HeaderPositionCalculator.java#L47
by not calling mAdapter.getHeaderId(position) twice [image: :smiley:]
—
Reply to this email directly or view it on GitHub
https://github.com/timehop/sticky-headers-recyclerview/issues/19#issuecomment-93785293
.

Went a little further then that :smile: https://github.com/timehop/sticky-headers-recyclerview/pull/42

Also there are tons of Rect allocations (tracked for like 5-second scrolling for a list with 100 items and 10 headers):

I might submit a pull request to make methods that return Rect accept outRect parameter if you don't mind.

That would be much appreciated!
On Aug 7, 2015 7:50 AM, "Denis Kniazhev"<EMAIL_ADDRESS>wrote:

Also there are tons of Rect allocations (tracked for like 5-second
scrolling for a list with 100 items and 10 headers):
[image: image]
https://cloud.githubusercontent.com/assets/3080318/9135955/3cf94c0c-3d1b-11e5-916a-b50979eb464c.png
[image: image]
https://cloud.githubusercontent.com/assets/3080318/9135971/65b7f986-3d1b-11e5-8969-e3adc0845956.png
[image: image]
https://cloud.githubusercontent.com/assets/3080318/9135986/80d0416a-3d1b-11e5-8658-32c425cd793f.png
[image: image]
https://cloud.githubusercontent.com/assets/3080318/9136002/92f0ab64-3d1b-11e5-86a4-36f75105363c.png
I might submit a pull request to make methods that return Rect accept
outRect parameter if you don't mind.
—
Reply to this email directly or view it on GitHub
https://github.com/timehop/sticky-headers-recyclerview/issues/19#issuecomment-128691963
.

@jacobtabak Sorry, just got round to it. I have submitted a pull request, please take a look.


refactor: remove unused oracle addresses

Overview
This PR removes all references to the old oracles from the messages and the contract config.
Additionally, to properly remove the old oracle addresses from the config, the migration logic has been updated and
the MsgMigrate has been converted to an enum to support future migration logics.
To migrate the contract from the version v0.4.18 the following migrate message must be send:
{
  "v0_4_18_to_v0_4_19": {}
}

closes: MILKTIA-32
What changes have been made in this PR?

[ ] Removed all the references to the old oracles
[ ] Added a migration logic to remove the old oracles address from the state of the contract
[ ] Refactored the migration logic to support future migrations

Checklist

[ ] Appropriate labels applied
[ ] Targeted PR against correct branch
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Wrote unit and integration
[ ] Updated relevant documentation

Here some screenshots of the contract state before and after the migration.
Contract state on version v0.4.18

Contract state on version v0.4.19


:fire: remove no-bullets variant from the list component.

people should rather use the ordered or unordered list components with the visual counter or bullet points.
if the need something extra they need to style that themself using ul
the special link-list component does just that.
decided during hedwig meeting 14. march 2024

trello.com/c/lxSrFhc2


Introduce prettier into CI pipeline

I realize that this might be bad time since CircleCI is broken but I encountered various other problems while working on this so I wanted to get it out.
This package currently uses prettier to format its code but does not verify that contributors check their format. With this change the CI pipeline will check changed files for their format.
While find enables piping to grep to check for changed files only it does not match its behavior to eslint and prettier. Prettier has its own --ignore-path argument and prettier:files  could be removed in favor of prettier --ignore-path .eslintignore "**/*:{d.ts,js,tsx}".
However this caused performance degradation caused by prettier/prettier#4989. Timing both variants reveals little over 2x the runtime of the original command:
$ time find . -name "*.js" -o -name "*.d.ts" -o -name "*.tsx" | grep -v -f .eslintignore | xargs prettier --write
real    0m26.754s
user    0m34.860s
sys     0m2.356s

$ time prettier --ignore-path .eslintignore --write "**/*.{d.ts,js,tsx}"
real    1m2.349s
user    0m58.672s
sys     0m14.988s

Another issue is that the checked files changed:
--- prettier_files_find.sorted  2018-08-17 09:18:45.409774091 +0200
+++ prettier_files.sorted       2018-08-17 09:18:32.645896784 +0200
@@ -1,5 +1,8 @@
 ./.babelrc.js
 ./docs/.eslintrc.js
+./docs/scripts/buildApi.js
+./docs/scripts/buildIcons.js
+./docs/scripts/buildServiceWorker.js
 ./docs/src/config.js
 ./docs/src/modules/components/AppContent.js
 ./docs/src/modules/components/AppDrawer.js
@@ -329,21 +332,9 @@
 ./packages/eslint-plugin-material-ui/src/rules/docgen-ignore-before-comment.js
 ./packages/eslint-plugin-material-ui/src/rules/docgen-ignore-before-comment.test.js
 ./packages/material-ui-codemod/src/v0.15.0/import-path.js
-./packages/material-ui-codemod/src/v0.15.0/import-path.test/actual.js
-./packages/material-ui-codemod/src/v0.15.0/import-path.test/expected.js
-./packages/material-ui-codemod/src/v0.15.0/import-path.test.js
 ./packages/material-ui-codemod/src/v1.0.0/color-imports.js
-./packages/material-ui-codemod/src/v1.0.0/color-imports.test/actual.js
-./packages/material-ui-codemod/src/v1.0.0/color-imports.test/expected.js
-./packages/material-ui-codemod/src/v1.0.0/color-imports.test.js
 ./packages/material-ui-codemod/src/v1.0.0/import-path.js
-./packages/material-ui-codemod/src/v1.0.0/import-path.test/actual.js
-./packages/material-ui-codemod/src/v1.0.0/import-path.test/expected.js
-./packages/material-ui-codemod/src/v1.0.0/import-path.test.js
 ./packages/material-ui-codemod/src/v1.0.0/svg-icon-imports.js
-./packages/material-ui-codemod/src/v1.0.0/svg-icon-imports.test/actual.js
-./packages/material-ui-codemod/src/v1.0.0/svg-icon-imports.test/expected.js
-./packages/material-ui-codemod/src/v1.0.0/svg-icon-imports.test.js
 ./packages/material-ui-docs/scripts/copy-files.js
 ./packages/material-ui-docs/src/index.js
 ./packages/material-ui-docs/src/MarkdownElement/index.js
@@ -356,6 +347,8 @@
 ./packages/material-ui-docs/src/svgIcons/LightbulbFull.js
 ./packages/material-ui-docs/src/svgIcons/LightbulbOutline.js
 ./packages/material-ui-docs/src/svgIcons/Twitter.js
+./packages/material-ui-icons/builder.js
+./packages/material-ui-icons/builder.test.js
 ./packages/material-ui-icons/.eslintrc.js
 ./packages/material-ui-icons/renameFilters/default.js
 ./packages/material-ui-icons/renameFilters/material-design-icons.js
@@ -1262,6 +1255,7 @@
 ./pages/guides/api.js
 ./pages/guides/composition.js
 ./pages/guides/csp.js
+./pages/guides/flow.js
 ./pages/guides/interoperability.js
 ./pages/guides/migration-v0x.js
 ./pages/guides/minimizing-bundle-size.js

Since using --ignore-path considers the same files as eslint we should switch at some point to --ignore-path.
I have another iteration that uses --ignore-path at least in ci since the slower runtime is negligible since we only run on the changed files.
For an explanation as to why prettier only runs on changed files and not on all see the commit message of  7d42ca0.

does not verify that contributors check their format.

We do, we have an eslint plugin for it. Now, TypeScript handling is different. We don't run eslint on TypeScript, but I'm wondering if we shouldn't.

So how do I trigger this? yarn test did not throw which does run lint if
I remember correctly.
For typescript we should use tslint I think.
On Fri, Aug 17, 2018, 19:34 Olivier Tassinari<EMAIL_ADDRESS>wrote:

does not verify that contributors check their format.
We do, we have an eslint plugin for it. Now, TypeScript handling is
different. We don't run eslint on TypeScript, but I'm wondering if we
shouldn't.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mui-org/material-ui/pull/12564#issuecomment-413936492,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALuPzwyrgXOy_pDuFY4uKsgJFjdNbEMhks5uRv6XgaJpZM4WBwp2
.

CircleCI is broken

It's definitely broken: https://discuss.circleci.com/t/saving-cache-stopped-working-warning-skipping-this-step-disabled-in-configuration/24423.

how do I trigger this?

yarn test triggers it, but we use yarn lint directly in the CI, same behavior. I don't think that it's a coincidence the only files that changed are the TypeScript ones. I believe we would have way more diffs if prettier  was actually broken for the JavaScript files in the CI.

Ok, let's try this workflow without eslint-plugin-prettier. I will add it back if I feel the need to. Thank you for this new CI check :)

It saves ~20s in the lint duration.


ptx: special char breaks it with "thread 'main' panicked at 'assertion failed: end <= s.len()', src/uu/ptx/src/ptx.rs:291:5"

With foo.txt containing:
it’s disabled

The char ’ is key here
ptx -G foo.txt
thread 'main' panicked at 'assertion failed: end <= s.len()', src/uu/ptx/src/ptx.rs:291:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I have an idea on how to fix this: use iterators instead of Vec<char>.
Trimming whitespaces, for example, could be done simply with skip_while.
Trimming broken words is more complicated, as we need to check for whitespace beyond the edge of the string. We need our own type that is like std::str::Chars but also keep the character beyond the beginning/end of the iterator.
For the before/tail chunk, we can leverage std::str::Chars being DoubleEndedIterator (utf-8 can be walked backward too, I just discovered).
@sylvestre I've been making test cases. Should I add Unicode cases too or skip it until this is fixed? (adding it would cause failures even for things unrelated to ptx - I worry that might be annoying).

@wishawa are you working on your idea? You have more context than me, so if you are taking care of that I will work on something else.

@wishawa are you working on your idea? You have more context than me, so if you are taking care of that I will work on something else.

No. I'm still working on the tests (and am slow at that lol). Feel free to go ahead and work on this one😃

It is still happening.
As I don't think @wishawa is still working on it, others can work on it!


WD 05 SCO

Is there a reason why the process observable's ID is specified as:
    ('id', IDProperty(_type)),

instead of
    ('id', IDProperty(_type, spec_version='2.1')),

Is there a reason why the process observable's ID is specified as:
    ('id', IDProperty(_type)),

instead of
    ('id', IDProperty(_type, spec_version='2.1')),

No reason, that's just a relic of an old way I used to separate the Process SCO from the other SCOs when it came time to creating a deterministic ID (since Process doesn't use deterministic IDs). I've added the spec_version back in. Thanks for catching that.

Codecov Report

Merging #285 into stix2.1 will decrease coverage by 1.01%.
The diff coverage is 74.21%.

@@            Coverage Diff             @@
##           stix2.1    #285      +/-   ##
==========================================
- Coverage    99.01%     98%   -1.02%
==========================================
  Files          121     123       +2
  Lines        12924   13403     +479
==========================================
+ Hits         12797   13135     +338
- Misses         127     268     +141

Impacted Files
Coverage Δ

stix2/test/v21/test_datastore_filters.py
100% <ø> (ø)
:arrow_up:

stix2/v21/common.py
100% <ø> (ø)
:arrow_up:

stix2/v20/common.py
100% <ø> (ø)
:arrow_up:

stix2/test/v21/test_core.py
100% <ø> (ø)
:arrow_up:

stix2/test/v21/test_utils.py
100% <ø> (ø)
:arrow_up:

stix2/v20/sdo.py
100% <ø> (ø)
:arrow_up:

stix2/test/v20/test_utils.py
100% <ø> (ø)
:arrow_up:

stix2/test/v21/test_observed_data.py
100% <100%> (ø)
:arrow_up:

stix2/v21/sro.py
94.73% <100%> (+0.14%)
:arrow_up:

stix2/test/v21/test_properties.py
100% <100%> (ø)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2a649b...9c7128d. Read the comment docs.


Overall review of SDK docs v5.2.0

The latest version of the SDK v5.1.4 has now had a partial re-write and also contains new guides.
Therefore, after all of the new updates/additions/changes a full overall review, including the content flow, grammar, punctuation, links, etc. is required.

Duplicate of #1035


Not possible to stop Hass.io through hassio command line or web UI

Home Assistant release with the issue:

0.92.2
Operating environment (HassOS/Generic):

Ubuntu 18.04.02 LTS on x86
Supervisor logs:

May 05 10:22:36 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:36 INFO (SyncWorker_0) [hassio.docker.interface] Stop homeassistant application
May 05 10:22:36 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:36 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API error: Received message 8:1000 is not str
May 05 10:22:36 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:36 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API connection is closed
May 05 10:22:41 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:41 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
May 05 10:22:41 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:41 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
May 05 10:22:41 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:41 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
May 05 10:22:46 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:46 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
May 05 10:22:46 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:46 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
May 05 10:22:46 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:46 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
May 05 10:22:51 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:51 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
May 05 10:22:51 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:51 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
May 05 10:22:51 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:51 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
May 05 10:22:56 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:56 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
May 05 10:22:56 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:56 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
May 05 10:22:56 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:22:56 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
May 05 10:23:01 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:01 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
May 05 10:23:01 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:01 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
May 05 10:23:01 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:01 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
May 05 10:23:06 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:06 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
May 05 10:23:06 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:06 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
May 05 10:23:06 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:06 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
May 05 10:23:07 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:07 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
May 05 10:23:07 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:07 INFO (SyncWorker_10) [hassio.docker.interface] Start homeassistant/qemux86-64-homeassistant
May 05 10:23:11 rcoleman-linux hassio-supervisor[3868]: 19-05-05 17:23:11 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize

Description of problem:
When I try to stop my Hass.io instance via hassio ha stop or via the web UI, it goes down, but then the supervisor watchdog kicks in exactly 30s later and starts it up again.  I think the watchdog is doing exactly what it's supposed to do, but the checks that would normally prevent it from restarting are failing.  It seems like it has to be in here somewhere, and specifically something in the first conditional should prevent the restart by returning:
    async def _watchdog_homeassistant_docker(self):
        """Check running state of Docker and start if they is close."""
        # if Home Assistant is active
        if (
            not await self.sys_homeassistant.is_fails()
            or not self.sys_homeassistant.watchdog
            or self.sys_homeassistant.error_state
        ):
            return

        # if Home Assistant is running
        if (
            self.sys_homeassistant.in_progress
            or await self.sys_homeassistant.is_running()
        ):
            return

        _LOGGER.warning("Watchdog found a problem with Home Assistant Docker!")
        try:
            await self.sys_homeassistant.start()
        except HomeAssistantError:
            _LOGGER.error("Watchdog Home Assistant reanimation fails!")

I don't see anything to indicate that the homeassistant docker container exited with an error code, but that could be it.  I don't know how to get that kind of historical information from docker.
The errors in the log are just appdaemon trying to connect while the system is going down, so I don't think they're related.  I don't know what the rejected "8:1000" error means.

That is because your home assistant instance doesn't exit with 0. They crash on shutdown and that make the watchdog restart it.
Hmm, I try to find a solution

Thanks, Pascal.  As I mentioned in that, I stopped AppDaemon and that stopped the spam, but it still restarted automatically after 30s and there were no logs between receiving the "stop" command and the "Watchdog found a problem with Home Assistant Docker!".  I can set the global log level to "debug" tonight to see if I can catch something going wrong during shutdown, but just printing the container exit code may be a good start.

I have the same issue:
19-10-30 09:42:55 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
19-10-30 09:42:55 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
19-10-30 09:42:55 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
19-10-30 09:42:58 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
19-10-30 09:42:58 INFO (SyncWorker_78) [hassio.docker.interface] Start homeassistant/qemux86-64-homeassistant
19-10-30 09:42:59 ERROR (MainThread) [hassio.homeassistant] Error on call http://<IP_ADDRESS>:8123/api/states/binary_sensor.snapshots_stale: Cannot connect to host <IP_ADDRESS>:8123 ssl:None [Connection refused]
19-10-30 09:42:59 ERROR (MainThread) [hassio.api.proxy] Error on API for request states/binary_sensor.snapshots_stale
19-10-30 09:43:00 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
19-10-30 09:43:00 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_appdaemon3
19-10-30 09:43:00 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].


lxc copy x:c1 y:c1 makes many assumptions about network topology, firewalls etc.

With lxd 2.0.0-0ubuntu3~ubuntu15.1
This works fine:
nated-machine# lxc exec remotemachine:c57 bash
... however it doesn't appear possible to copy a container from a machine behind a firewall and/or nat and a machine elsewhere:
nated-machine# lxc copy c1 remotemachine:c2
... since the remote machine expects to be able to contact the IP address of the local machine to 'pull' the container over.
It's also not possible to copy containers over ssh at all (I assume this was a design decision, but it is a total pain for system administration - ssh is the universal transport for most sysadmins, and having to fart around with firewalls to open up specific TCP ports is cumbersome and arguably less secure too).

Linked #1898

That's going to be dealt with #553 which will have us implement push mode too, we'll then be able to:

Have the target pull from the source
Or have the source push to the target
Or have the client pull from the source and push to the target

Closing as duplicate


[SPARK-13784][ML] Persistence for RandomForestClassifier, RandomForestRegressor

What changes were proposed in this pull request?
Main change: Added save/load for RandomForestClassifier, RandomForestRegressor (implementation details below)
Modified numTrees method (deprecation)

Goal: Use default implementations of unit tests which assume Estimators and Models share the same set of Params.
What this PR does: Moves method numTrees outside of trait TreeEnsembleModel.  Adds it to GBT and RF Models.  Deprecates it in RF Models in favor of new method getNumTrees.  In Spark 2.1, we can have RF Models include Param numTrees.

Minor items

Fixes bugs in GBTClassificationModel, GBTRegressionModel fromOld methods where they assign the wrong old UID.

Implementation details

Split DecisionTreeModelReadWrite.loadTreeNodes into 2 methods in order to reuse some code for ensembles.
Added EnsembleModelReadWrite object with save/load implementations usable for RFs and GBTs

These store all trees' nodes in a single DataFrame, and all trees' metadata in a second DataFrame.

Split trait RandomForestParams into parts in order to add more Estimator Params to RF models
Split DefaultParamsWriter.saveMetadata into two methods to allow ensembles to store sub-models' metadata in a single DataFrame.  Same for DefaultParamsReader.loadMetadata

How was this patch tested?
Adds standard unit tests for RF save/load

Test build #54727 has started for PR 12118 at commit 607ed4e.

Test build #54727 has finished for PR 12118 at commit 607ed4e.

This patch fails Scala style tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54727/
Test FAILed.

Test build #54728 has started for PR 12118 at commit b13b11f.

@GayathriMurali @hhbyyh  Could you please help review this PR?  Thanks!

Test build #54730 has started for PR 12118 at commit cd07aff.

Test build #54730 has finished for PR 12118 at commit cd07aff.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54730/
Test FAILed.

That seems like a spurious or unrelated failure.  I'll retest

Test build #2727 has started for PR 12118 at commit cd07aff.

Test build #54728 has finished for PR 12118 at commit b13b11f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54728/
Test PASSed.

Test build #2727 has finished for PR 12118 at commit cd07aff.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

@jkbradley Thanks for this. This looks great and clarifies a lot of things I was trying to do. I had one minor comment, except that it looks fine to me.

@GayathriMurali Thanks for taking a look!  @hhbyyh It would be great if you could take a look too since you're familiar with some of this code.

Sorry for the delay. I'll start now.

just some minor comments. LGTM.

Test build #54828 has started for PR 12118 at commit afed67b.

Test build #54828 has finished for PR 12118 at commit afed67b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54828/
Test FAILed.

Merged build finished. Test FAILed.

Test build #54836 has started for PR 12118 at commit e0306a9.

Test build #54836 has finished for PR 12118 at commit e0306a9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54836/
Test PASSed.

@GayathriMurali Thanks for the initial commits, and @hhbyyh thanks for the review!  I'll go ahead and merge this with master since you gave a LGTM pending minor changes.


Ensure pip modules are caching between builds

There is some suspicion that pip modules aren't being reserved between runtime builds. This needs to be investigated.

Output from a first run pip install:
  - Running pip install :
    Collecting colorama==0.3.7 (from -r requirements.txt (line 1))
      Downloading colorama-0.3.7-py2.py3-none-any.whl
    Collecting Django==1.10.5 (from -r requirements.txt (line 2))
      Downloading Django-1.10.5-py2.py3-none-any.whl (6.8MB)
        100% |████████████████████████████████| 6.8MB 166kB/s
    Collecting django-webpack-loader==0.4.1 (from -r requirements.txt (line 3))
      Downloading django_webpack_loader-0.4.1-py2.py3-none-any.whl
    Collecting djangorestframework==3.5.3 (from -r requirements.txt (line 4))
      Downloading djangorestframework-3.5.3-py2.py3-none-any.whl (709kB)
        100% |████████████████████████████████| 716kB 397kB/s
    Collecting psycopg2==2.6.2 (from -r requirements.txt (line 5))
      Downloading psycopg2-2.6.2.tar.gz (376kB)
        100% |████████████████████████████████| 378kB 553kB/s
    Collecting pymssql==2.1.3 (from -r requirements.txt (line 6))
      Downloading pymssql-2.1.3.tar.gz (897kB)
        100% |████████████████████████████████| 901kB 788kB/s
    Collecting pypyodbc==<IP_ADDRESS> (from -r requirements.txt (line 7))
      Downloading pypyodbc-<IP_ADDRESS>.zip
      Requested pypyodbc==<IP_ADDRESS> from https://pypi.python.org/packages/5a/04/623370ba35a1e48fd64cdf247bec19495cc3a7f2f2f602a0d48aed80d612/pypyodbc-<IP_ADDRESS>.zip#md5=4752bfb7d3574d99b13b6836f755f346 (from -r   requirements.txt (line 7)), but installing version None
    Collecting python-dateutil==2.6.0 (from -r requirements.txt (line 8))
      Downloading python_dateutil-2.6.0-py2.py3-none-any.whl (194kB)
        100% |████████████████████████████████| 194kB 1.9MB/s
    Collecting six==1.10.0 (from -r requirements.txt (line 9))
      Downloading six-1.10.0-py2.py3-none-any.whl
    Collecting termcolor==1.1.0 (from -r requirements.txt (line 10))
      Downloading termcolor-1.1.0.tar.gz
    Collecting virtualenv==15.1.0 (from -r requirements.txt (line 11))
      Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
        100% |████████████████████████████████| 1.8MB 402kB/s
    Collecting setuptools (from pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Downloading setuptools-34.3.2-py2.py3-none-any.whl (389kB)
        100% |████████████████████████████████| 399kB 524kB/s
    Collecting appdirs>=1.4.0 (from setuptools->pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Downloading appdirs-1.4.3-py2.py3-none-any.whl
    Collecting packaging>=16.8 (from setuptools->pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Downloading packaging-16.8-py2.py3-none-any.whl
    Collecting pyparsing (from packaging>=16.8->setuptools->pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
        100% |████████████████████████████████| 61kB 568kB/s
    Installing collected packages: colorama, Django, django-webpack-loader, djangorestframework, psycopg2, pymssql, appdirs, pyparsing, six, packaging, setuptools, pypyodbc, python-dateutil, termcolor,   virtualenv
      Running setup.py install for psycopg2 ... done
      Running setup.py install for pymssql ... done
      Running setup.py install for pypyodbc ... done
      Running setup.py install for termcolor ... done
    Successfully installed Django-1.10.5 appdirs-1.4.3 colorama-0.3.7 django-webpack-loader-0.4.1 djangorestframework-3.5.3 packaging-16.8 psycopg2-2.6.2 pymssql-2.1.3 pyparsing-2.2.0 pypyodbc-1.3.3 python-dateutil-2.6.0 setuptools-34.3.2 six-1.10.0 termcolor-1.1.0 virtualenv-15.1.0

Output from pip install after the first:
  - Running pip install :
    Collecting colorama==0.3.7 (from -r requirements.txt (line 1))
      Using cached colorama-0.3.7-py2.py3-none-any.whl
    Collecting Django==1.10.5 (from -r requirements.txt (line 2))
      Using cached Django-1.10.5-py2.py3-none-any.whl
    Collecting django-webpack-loader==0.4.1 (from -r requirements.txt (line 3))
      Using cached django_webpack_loader-0.4.1-py2.py3-none-any.whl
    Collecting djangorestframework==3.5.3 (from -r requirements.txt (line 4))
      Using cached djangorestframework-3.5.3-py2.py3-none-any.whl
    Collecting psycopg2==2.6.2 (from -r requirements.txt (line 5))
      Using cached psycopg2-2.6.2.tar.gz
    Collecting pymssql==2.1.3 (from -r requirements.txt (line 6))
      Using cached pymssql-2.1.3.tar.gz
    Collecting pypyodbc==<IP_ADDRESS> (from -r requirements.txt (line 7))
      Using cached pypyodbc-<IP_ADDRESS>.zip
      Requested pypyodbc==<IP_ADDRESS> from https://pypi.python.org/packages/5a/04/623370ba35a1e48fd64cdf247bec19495cc3a7f2f2f602a0d48aed80d612/pypyodbc-<IP_ADDRESS>.zip#md5=4752bfb7d3574d99b13b6836f755f346 (from -r   requirements.txt (line 7)), but installing version None
    Collecting python-dateutil==2.6.0 (from -r requirements.txt (line 8))
      Using cached python_dateutil-2.6.0-py2.py3-none-any.whl
    Collecting six==1.10.0 (from -r requirements.txt (line 9))
      Using cached six-1.10.0-py2.py3-none-any.whl
    Collecting termcolor==1.1.0 (from -r requirements.txt (line 10))
      Using cached termcolor-1.1.0.tar.gz
    Collecting virtualenv==15.1.0 (from -r requirements.txt (line 11))
      Using cached virtualenv-15.1.0-py2.py3-none-any.whl
    Collecting setuptools (from pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Using cached setuptools-34.3.2-py2.py3-none-any.whl
    Collecting appdirs>=1.4.0 (from setuptools->pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Using cached appdirs-1.4.3-py2.py3-none-any.whl
    Collecting packaging>=16.8 (from setuptools->pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Using cached packaging-16.8-py2.py3-none-any.whl
    Collecting pyparsing (from packaging>=16.8->setuptools->pypyodbc==<IP_ADDRESS>->-r requirements.txt (line 7))
      Using cached pyparsing-2.2.0-py2.py3-none-any.whl
    Installing collected packages: colorama, Django, django-webpack-loader, djangorestframework, psycopg2, pymssql, appdirs, pyparsing, six, packaging, setuptools, pypyodbc, python-dateutil, termcolor,   virtualenv
      Running setup.py install for psycopg2 ... done
      Running setup.py install for pymssql ... done
      Running setup.py install for pypyodbc ... done
      Running setup.py install for termcolor ... done
    Successfully installed Django-1.10.5 appdirs-1.4.3 colorama-0.3.7 django-webpack-loader-0.4.1 djangorestframework-3.5.3 packaging-16.8 psycopg2-2.6.2 pymssql-2.1.3 pyparsing-2.2.0 pypyodbc-1.3.3   python-dateutil-2.6.0 setuptools-34.3.2 six-1.10.0 termcolor-1.1.0 virtualenv-15.1.0

please provide the output from nanobox build-runtime -v

I think #43 will handle this, so I just wanted to mention it here so things can be updated or closed as needed.

any update on this?


Trinity.Core: Fix assembly blacklist in AssemblyPath

Discussion: shall we blacklist assemblies from test frameworks (e.g. xunit, nunit, ...) ?

@leasunhy,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

@leasunhy nope, as long as the test frameworks do not call into Trinity directly.


False error "Viewport Texture must be set to use it."

Godot version
4.0 beta 1 mono
System information
Windows 10
Issue description
Scenes with viewport shows this errors even though viewport textures are set and working in materials:
ERROR: Viewport Texture must be set to use it.
at: (scene/main/viewport.cpp:114)
ERROR: Viewport Texture must be set to use it.
at: (scene/main/viewport.cpp:109)
Steps to reproduce
Create simple scene with viewport and material that is using it as texture.
Minimal reproduction project
No response

I guess the ViewportTexture is obtained by instantiating it instead of getting it from an existing Viewport. Some properties need to be set before it can be used. The error message is really not very detailed either.

I keep getting this too (in GDscript), but otherwise my SubViewport works fine.

I got this same situation/error. The viewport texture does not work when exporting (it does work when executing from the editor)

I found the solution here: https://github.com/godotengine/godot/issues/55471
I set the subviewport update mode to always. It seems there's a bug with "update only when visible" option and it has not been fixed

In RC2 this error seems to be happening even with the update mode set to always...

Yes, I'm having the same issue, no matter what configuration the SubViewport uses, it will act as if there's no Viewport set on debug or on the editor once the scene is reloaded.

V4.0. table. official [92bee43ad] Still the same

Godot version
4.0.1 RC2 mono
System information
Windows 10
Issue description
Scenes with viewport as a 3D texture shows errors on game startup. Mesh materials from viewport (2D) textures do not work and show up purple ingame, even though they show up in the editor.
E 0:00:00:0832   get_height: Viewport Texture must be set to use it.
<C++ Error>    Condition "!vp" is true. Returning: 0
<C++ Source>   scene/main/viewport.cpp:116 @ get_height()
These appear as errors, even though they work except for the mesh material option.
Steps to reproduce

Create simple scene with viewport and 2D control node
Disable 3D in viewport
Use viewport as texture for material

Minimal reproduction project
ViewportTexture66247.zip

As a workaround, set the viewport texture in the parent node of both the SubViewport and the MeshInstance3D in code:
Example in C#
public override void _Ready()
{
      RenderingServer.FramePostDraw += OnFramePostDraw;
}

private void OnFramePostDraw()
{
      SubViewport subViewport = GetNode<SubViewport>("SubViewport");
      var viewportTexture = subViewport.GetTexture();

      MeshInstance3D meshInstance = GetNode<MeshInstance3D>("Mesh");

      StandardMaterial3D mat = new()
      {
          AlbedoTexture = viewportTexture
      };

      meshInstance.SetSurfaceOverrideMaterial(0, mat);
}

What seems to be happening is that this depends on the order of initialization. For me, the mesh was initialized first and it holds references to the uninitialized viewport. That issue resolves itself after a frame or so.
As mentioned, one workaround is to remove the viewport texture in the scene tree and create it programmatically.

What seems to be happening is that this depends on the order of initialization.

This has little to do with the order of instantiation. It mainly depends on whether ViewportTexture::setup_local_to_scene() is called and when it is called.
Currently, even if a resource has local_to_scene enabled, but its parent resource does not have local_to_scene enabled, the resource will not call setup_local_to_scene().

I followed along with the 3D https://docs.godotengine.org/en/stable/tutorials/animation/playing_videos.html#displaying-a-video-on-a-3d-surface on 4.0.3.rc1 ... my video plays well but errors when running from editor gives:
E 0:00:00:0580   get_width: Viewport Texture must be set to use it.
  <C++ Error>    Condition "!vp" is true. Returning: 0
  <C++ Source>   scene/main/viewport.cpp:111 @ get_width()

E 0:00:00:0580   get_height: Viewport Texture must be set to use it.
  <C++ Error>    Condition "!vp" is true. Returning: 0
  <C++ Source>   scene/main/viewport.cpp:116 @ get_height()

E 0:00:00:0623   setup_local_to_scene: Condition "proxy.is_valid()" is true.
  <C++ Source>   scene/main/viewport.cpp:89 @ setup_local_to_scene()

Fixed by #75751?

Fixed by #75751?

That's likely. Please test in 4.1 dev 2 or the upcoming 4.0.3 RC 2.

4.1 dev 2 still has the issue with get_height() and get_width()

Project.zip

Also, reopening the project greets you with Viewport Texture must be set to use it., so it is not fixed at all

This is still an issue in 4.02 stable. Having multiple errors, even though it technically works.

@RichardR01 I think it has been fixed in 4.0.3
https://github.com/godotengine/godot/pull/75751#issuecomment-1545520308

I upgraded to version 4.03 rc2. I am still having problems. Most of the errors are gone, but I still get 2 when I run the scene. Also the editor view isn't being updated, as it always shows white on my object. The object also doesn't get drawn in runtime, unless I add the viewport texture to the emission channel. It still technically works, but I think it is worse now that it was. The editor visual got updated when I reloaded godot. But the 2 errors till persist. This really needs to be tested thoroughly.
E 0:00:01:0410   get_height: Viewport Texture must be set to use it.
<C++ Source>   scene/main/viewport.cpp:131 @ get_height()
E 0:00:01:0410   get_width: Viewport Texture must be set to use it.
<C++ Source>   scene/main/viewport.cpp:121 @ get_width()

Most of the errors are gone, but I still get 2 when I run the scene.

This happens during the scene instantiation phase. Resources are configured before nodes. The get_width()/get_height() of the texture is called when setting the texture for a material. See https://github.com/godotengine/godot/issues/66247#issuecomment-1548775331.

Also the editor view isn't being updated, as it always shows white on my object.

This is a regression from #64388, see #77207.

I am stilling getting the following error when loading up the scene file in the editor.
scene/main/viewport.cpp:185 - Condition "proxy.is_valid()" is true.

This is caused by multiple calls to setup_local_to_scene() when the target viewport has not changed. As far as I know, toggling resource_local_to_scene causes setup_local_to_scene() to be called.

same as the above and it really just kills my project since it depends on subviewports in a way that i cant even move forward without it, so yeah i hope this gets fixed soon

@DigitalN8m4r3 in https://github.com/godotengine/godot/issues/66247#issuecomment-1483786200 there seems to be a workaround. If that works there is no killing. And what version you are working with?

Getting the same error on 4.1 stable

I hasty un-merged this in our fork. https://github.com/V-Sekai/godot/tree/viewport-path-update-fix.
The screen is pink and after the fix it works as it did in 4.0.
In 4.1 and 4.2 the screen is pink and the desktop copy of the vr viewport does not work. It's a blocker for us, so this is rush job fix.

Same Error 4.1.1

Error is gone for me in 4.2-dev5

Still present for me in 4.2 dev5

regarding the workaround
This works in 4.1.1. Gerre's a GDScript approximation:
extends MeshInstance3D

@export var viewport:SubViewport # Could instead get a child with an @onready

func _ready() -> void:
	RenderingServer.frame_post_draw.connect(on_draw.bind())

func on_draw() -> void:
	var tex = viewport.get_texture()
	var mat = StandardMaterial3D.new()
	mat.albedo_texture = tex
	set_surface_override_material(0, mat)
	RenderingServer.frame_post_draw.disconnect(on_draw.bind())

Still present in 4.2 beta 1

I built Godot from source using master today and I still get the error, so even in 4.2 Beta 3+

It all works fine in game though, so I guess it is a timing issue?
I will try some of the workarounds, as the error is distracting when I'm looking for real problems.

Are there plans to backport the fix to godot lts?

Are there plans to backport the fix to godot lts?

The issue isn't fuilly resolved in 4.x yet, so that needs to be done first before it can be backported to 3.x.

The issue isn't fuilly resolved in 4.x yet as per the above comments, so that needs to be done first before it can be backported to 3.x.

Thank you! I confirm that a similar issue exists in 3.5.3.

I found the solution here: #55471
I set the SubViewport update mode to always to fix the problem. It seems there's a bug with "update only when visible" option and it has not been fixed yet

My SubViewport update mode is set to Always, and I still get the error message when trying to duplicate the texture. I get no error when debugging in the editor, but it happens in the exported program (in my case, the Quest 3).

I found a solution while attempting to bypass that check, and it seems to be working fine. I want to recreate Prey's screens, and this is how it's turning out:
https://twitter.com/GalacticLake/status/1769937516889616637
Before, I was getting pink screens every time I started the project. This is my solution:

It seems the trick lies in creating an exported variable that stores a packed scene of what the subviewport will display. Inside the _ready function, assign it as a child of the subviewport, then make the necessary configurations, such as assigning the material and calling setup_local_to_scene(). Disabling the process mode isn't necessary; that's part of my multi-screen system. With this method, I no longer encounter errors because it's created and configured after the nodes are ready.
In the editor, the screens won't be visible, but they will appear in the game; that's the only issue.

A demo to test the latest status of the Viewport

https://github.com/MASSHUU12/godot-control-3d/issues/3

I was still running into this issue with Godot 4.3 Dev 5.
Based on some of the earlier explanations of the issue, it definitely seems to be an order of initialization issue.
I had my SubViewport as a child to my MeshInstance3D previously, but I instead moved the SubViewport out as a sibling and placed it above the MeshInstance3D in the scene hierarchy which fixed the issue (no more errors).
Basically, based on how I understand Godot to work, children's Ready methods are called before their parent, but the opposite is true for EnterTree. I don't know exactly when the check occurs, but it's probably the latter. So by making the SubViewport a sibling and higher up in the hierarchy I believe it is initializing before the MeshInstance who wants to use its RenderTexture.

FWIW, the error, Viewport Texture must be set to use it. occurs in

v4.2.stable.official [46dc27791] (4.2.1 Stable release)
and
v4.3.dev.custom_build [9d6bdbc56] (latest build from Master at the moment)

When running the following Godot Demo projects:
Procedural Materials
GUI in 3D
So both of those demo projects can be used to test fixes for this issue in current and future versions.

I'm using 4.2.2, my project is working fine but I have those issues, should we just ignore those warnings and errors for the time being?

4.3 stable, still happening :(

I am using the 4.3 stable version, but i get the same error:
This is the structure of my nodes:

And i am defining the HUD as my texture on "MeshInstance3D" but i get the same errors:

How can i set a texture of a viewport? I can't see the option

i see this has been open for awhile. is there trouble getting a dev to fix this issue? what is blocking this for so long

Please see the thread above for an explanation, but please don't bump issues without adding significant new information


[Request] Otherside doesnt replace blocks below Y = 0 yet.

Worlds have now been updated to have heights all the way down to -64, but the otherside doesn't seem to be configured to handle this yet. The otherside will still generate regular Deepslate caves and resources below this point that are not replaced by the block replacement configuration.

I think all the code has been updated to look below y=0, however I never added replacement rules for any of the "deep" blocks. I'll try and take a look at this when I can.

I made modifications to the materials customization for a custom otherside (including all the deep blocks), but blocks that are being replaced above ground such as lava, gravel etc are not being replaced below 0. It definitely still has something to do with the boundaries it looks for. :/

This will be fixed in the next dev build and release 10.8.4
There was in fact a default of 0 - 255, which I have removed.
I also added replacement for deepslate and tuff ... it'd be fun to do ancient cities but I didn't have the energy for that XD
If you happen to make rules for that yourself and want to share, it'd definitely be appreciated!

I didn't even think about Ancient cities, good call!

Verified that this is working now so closing issue


Missing support for deprecated annotations

We need to add support for recognizing the Deprecated AvailabilityStatus so that we can include a Deprecated: ... comment according to Go's documentation style guide.
Along with this change, we should incorporate changes into seed (i.e. via a seed publish command) so that generators start revealing what features they've implemented. That way we can host a feature matrix of all the generators and quickly recognize where the gaps are. We imagine the structure will be something as simple as the following:
Features:
  properties:
    supportsAvailability: boolean
    ...

closing and moving https://linear.app/fern-api/issue/FER-656/[feature]-support-deprecated-annotations


Add TypeScript definition file

Thanks for your contribution! Please see my comments.

Hi, thank you, I'll add and test those today.

Thank you very much for your contribution!


Add API method to query all events / rpcs.

I'm in the process of migrating our application from Crossbar over to deepstream, however one thing I missed was a visual debugging tool similar to WAMPShark which would let me tap into existing subscriptions and show me what was going on. I looked around on the web for a while and didn't find one, so I decided to go ahead and write my own, which I call "peepstream" (https://github.com/viridia/peepstream), which I plan to make freely available once it's done.
However, one stumbling block I ran into is that deepstream, unlike WAMP, doesn't provide a means to query for all existing events or registered rpcs.
This seems like it would be relatively simple to add, how about it?

Ah, I figured out that when you listen for events, it gives you all of the existing events immediately. So that solves my problem for events at least. Let's hope that rpcs has something similar :)


October 12th, 2020 Community Meeting

Please add a topic in this thread and add a link to the Github issue associated with the topic.
Please make sure you give folks enough time to review/discuss the topic offline on Github before coming into the meeting
(optional) Paste the image of an animal :smile:

Meeting minutes:

#325 reminder for community to revisit the logo issue. It seems we are very close for a final logo decision.
We did sort of a standup. Nothing relevant, there are some open proposal (e.g. #432 and #419 ), please provide feedback if possible.
We discussed the need of the tekton version bump. This is already part of the v1.0 milestone.


⚠ Bofrost.lu has degraded performance

In 8f98412, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 877 ms

Resolved: Bofrost.lu performance has improved in 9bec587 after 8 minutes.


Auto-delete old Google Cloud instance templates

Is your feature request related to a problem? Please describe.
Every few months, we run out of Google Cloud instance templates:

ERROR: (gcloud.compute.instance-templates.create-with-container) Could not fetch resource:

Quota 'INSTANCE_TEMPLATES' exceeded.  Limit: 300.0 globally.

Describe the solution you'd like
Instead, we should auto-delete templates that are more than 30 days old, before creating a new template.
Describe alternatives you've considered
Just keep on manually deleting them.
Additional context
We expect this to next become a problem some time in June, so I've scheduled it for the last sprint in May.

This task doesn't contribute directly to NU5 validation or client work, so I'm removing it from the sprint.
In the meantime we can just delete these when there are too many of them.

We might want to fix this soon because we are near the quota


Upgrade to version 0.3 of the steamcmd base-image

Weird. Any idea what that could cause? I think this didn't happen with Debian 8.
Running close-cleanly [5/16]
rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "exec: \"kill\": executable file not found in $PATH"
[FAIL] close-cleanly

Yes, really weird... I'll check it this night. It didn't find the kill command O_o

I think that I found the problem... and the problem is the test. It doesn't work before.
Probably we can't use exec with kill...

Do we have to kill it at all?


 U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6162/1/console

versions
git=2.34.1 maven=3.9.8

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ master Compile Tests _

-1 :x:
spotless
0m 44s

branch has 1 errors when running spotless:check, run spotless:apply to fix.

_ Patch Compile Tests _

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
shellcheck
0m  1s

No new issues.

+1 :green_heart:
spotless
0m 40s

patch has no errors when running spotless:check.

_ Other Tests _

+1 :green_heart:
asflicense
0m 11s

The patch does not generate ASF License warnings.

3m 29s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6162/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/6162

Optional Tests
dupname asflicense codespell detsecrets shellcheck shelldocs spotless

uname
Linux 12f8037fa4c4 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master /<PHONE_NUMBER>2a6bce6f7d6c2bdcb853061a0b3f74

spotless
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6162/1/artifact/yetus-general-check/output/branch-spotless.txt

Max. process+thread count
47 (vs. ulimit of 30000)

modules
C: . U: .

 U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6162/2/console

versions
git=2.34.1 maven=3.9.8

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 52s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
shelldocs
0m  0s

Shelldocs was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
spotless
1m  4s

branch has no errors when running spotless:check.

_ Patch Compile Tests _

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
shellcheck
0m  1s

No new issues.

+1 :green_heart:
spotless
1m  7s

patch has no errors when running spotless:check.

_ Other Tests _

+1 :green_heart:
asflicense
0m 17s

The patch does not generate ASF License warnings.

4m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6162/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/6162

Optional Tests
dupname asflicense codespell detsecrets shellcheck shelldocs spotless

uname
Linux a99609787637 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master /<PHONE_NUMBER>2a6bce6f7d6c2bdcb853061a0b3f74

Max. process+thread count
46 (vs. ulimit of 30000)

modules
C: . U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6162/2/console

versions
git=2.34.1 maven=3.9.8 shellcheck=0.8.0

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Spotless failure is due to some error in build. Re-triggered build!


Preprare v0.5.2

Getting ready to release.

@jbusecke, are the pre-commit hooks passing locally?

doubt it. These GH PRs with markdown tend to trigger whitespace errors.

Yeah this is really the downside of making PRs in the browser interface...Let me try to fix this now.


[C++] Segmentation fault in hash-join/swiss-join

Describe the bug, including details regarding any error messages, version, and platform.
Continuing from https://github.com/apache/arrow/issues/32570
Here is the crash I see in my code occasionally. Unfortunately, I do not have a small or even a reliable test case to reproduce.
@zanmato1984 fixed several crashes related to hash join before, I suspect this one may be another case from that family.
Visit<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:332:9)> at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:133 [0x2e57685]
DecodeSelected at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:330 [0x2e57685]
FlushBuildColumn at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1672 [0x2e5f9f6]
Flush at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1716 [0x2e60178]
AppendAndOutput<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:612:9), (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993:9)> at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:570 [0x2e60dbc]
Append<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993:9)> at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:610 [0x2e60dbc]
OnNextBatch at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993 [0x2e60dbc]
ProbeSingleBatch at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2144 [0x2e653e7]
OnProbeSideBatch at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:818 [0x2e0766d]
InputReceived at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:891 [0x2e06491]
OutputBatchCallback at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:1004 [0x2e0a3af]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:947 [0x2e0a3af]
__invoke_impl<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:947:5) &, long, arrow::compute::ExecBatch> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x2e0a1fb]
__invoke_r<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:947:5) &, long, arrow::compute::ExecBatch> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:114 [0x2e0a1fb]
_M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:290 [0x2e0a1fb]
operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:591 [0x2e60eb5]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993 [0x2e60eb5]
AppendAndOutput<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:612:9), (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993:9)> at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:571 [0x2e60eb5]
Append<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993:9)> at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:610 [0x2e60eb5]
OnNextBatch at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:1993 [0x2e60eb5]
ProbeSingleBatch at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2144 [0x2e653e7]
OnProbeSideBatch at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:818 [0x2e0766d]
InputReceived at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:891 [0x2e06491]
OutputBatchCallback at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:1004 [0x2e0a3af]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:947 [0x2e0a3af]
__invoke_impl<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:947:5) &, long, arrow::compute::ExecBatch> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x2e0a1fb]
__invoke_r<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:947:5) &, long, arrow::compute::ExecBatch> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:114 [0x2e0a1fb]
_M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:290 [0x2e0a1fb]
operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:591 [0x2e618ef]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2039 [0x2e618ef]
Flush<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2039:5)> at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join_internal.h:626 [0x2e618ef]
OnFinished at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2039 [0x2e618ef]
OnScanHashTableFinished at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2425 [0x2e683ff]
StartScanHashTable at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2323 [0x2e68ff0]
ProbingFinished at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/swiss_join.cc:2153 [0x2e655f9]
OnQueuedBatchesProbed at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:876 [0x2e0a61b]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:968 [0x2e0a61b]
__invoke_impl<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:967:9) &, unsigned long> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x2e0a61b]
__invoke_r<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/hash_join_node.cc:967:9) &, unsigned long> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:114 [0x2e0a61b]
_M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:290 [0x2e0a61b]
operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:591 [0x2e695f5]
OnTaskGroupFinished at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/task_util.cc:252 [0x2e695f5]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/task_util.cc:371 [0x2e6a313]
__invoke_impl<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/task_util.cc:371:5) &, unsigned long> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x2e6a313]
__invoke_r<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/task_util.cc:371:5) &, unsigned long> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:114 [0x2e6a313]
_M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:290 [0x2e6a313]
operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:591 [0x2e3279f]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/query_context.cc:82 [0x2e3279f]
__invoke_impl<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/query_context.cc:80:40) &> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x2e3279f]
__invoke_r<arrow::Status, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/acero/query_context.cc:80:40) &> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:114 [0x2e3279f]
_M_invoke at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:290 [0x2e3279f]
operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:591 [0x2e3355e]
operator()<std::function<arrow::Status ()> &, arrow::Status, arrow::Future<arrow::internal::Empty> > at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/util/future.h:150 [0x2e3355e]
__invoke_impl<void, arrow::detail::ContinueFuture &, arrow::Future<arrow::internal::Empty> &, std::function<arrow::Status ()> &> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x2e3355e]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/util/functional.h:140 [0x3376417]
WorkerLoop at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/util/thread_pool.cc:457 [0x3376417]
operator() at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/util/thread_pool.cc:618 [0x3376417]
__invoke_impl<void, (lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/util/thread_pool.cc:616:23)> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:61 [0x3376417]
__invoke<(lambda at /tmp/source-root/.conan2/p/b/arrow19d6b0dc5db3a/b/src/cpp/src/arrow/util/thread_pool.cc:616:23)> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/invoke.h:96 [0x3376417]
_M_invoke<0UL> at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_thread.h:292 [0x3376417]
operator() at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_thread.h:299 [0x3376417]
_M_run at /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_thread.h:244 [0x3376417]

Component(s)
C++

Thanks for reporting.
For me, it's hard to see the cause from this bare stack. It'll help to have a cpp test case, not necessarily small though, to reproduce the crash, even randomly.
Alternatively, if you are able to build your own arrow, it would also help to enable ASAN by adding cmake options -DARROW_USE_ASAN=ON -DARROW_JEMALLOC=OFF -DARROW_MIMALLOC=OFF. This will catch the issue at a higher rate and output extensive context of the crash.

Enabling ASAN makes this crash go away.
Enabling TSAN results in some reports which I describe in #40068, #40069.
Building in debug mode results in this assertion firing.
Lowering arrow::dataset::ScanOptions::batch_size to 16 also fixes the crash (and lowering to 1024 does not).

Enabling ASAN makes this crash go away. Enabling TSAN results in some reports which I describe in #40068, #40069. Building in debug mode results in this assertion firing.
Lowering arrow::dataset::ScanOptions::batch_size to 16 also fixes the crash (and lowering to 1024 does not).

Thanks for the experiments. Though I can only guess what was happening, I think we are making progress.
First I think the errors reported by TSAN don't seem to be related to this crash. But the fired assertion does. It indicates that an arrow-managed stack-like temp buffer is overflowed and possibly causing subsequent unexpected behaviors. It also explains why lowering batch_size makes crash go away - less temp space is required for smaller batch data. Though I can't explain why ASAN makes the crash go away, except that it slows down the program significantly so the chance of crash is reduced.
To verify if the fired assertion is the root cause, could you try something similar to #40007 and see if it resolves the issue?

I agree that TSAN crashes seem unrelated, I've reported them for completeness only. Then again, you never know with races.
The patch in #40007 seems to resolve my case, thank you! Looks like that PR will be merged and a new "investigate why this is even needed" issue will be open. If it's the case I suggest linking this issue there, and for now I'm closing it.


HTML Text Classification

Is your feature request related to a problem? Please describe.
Every page on the world wide web (WWW) is an HTML page, which offers the potential to use tags as metadata. Today, data labelers are forced to rely on text when performing text classification.
Describe the solution you'd like
As a data labeler, I would like to perform text classification on HTML files, similar to HTML NER Tagging.
Describe alternatives you've considered
I'm assuming that NER models are similar to text classification models
Additional context
None

Hi, @slavakurilyak !
You can easily do an HTML classification by applying the following config
<View>
  <Choices name="choice" toName="text">
    <Choice value="Class A"></Choice>
    <Choice value="Class B"></Choice>
  </Choices>
  <HyperText name="text" value="$text"></HyperText>
</View>

Do you suggest it should be added to the templates list?

@niklub Yes, please do!
Tracking clicks and views on templates and playground will allow the Heartex Labs to better understand the popularity of this template.


Release 0.1.2

Release Note
Cherrypicked smaller builder images changes

Manually confirmed integration tests and unit tests pass on a fresh cluster.
Automated e2e tests fail because at this branch the tests seem to pull from a static location that's updated past these changes. e2e-tests.sh also tries to create a cluster with v1.9.6-gke.1 which is unsupported today (my tests ran using the latest supported GKE version).


Fixes merging PATH and PYTHONPATH in env variables

For #4166
Was working for me as i have a custom .env file with some stuff in there.

Closed accidentally

Doesn't this conflict with the kernelEnv and custeEditVars merging of path and python path? It's now replacing on merge and appending later.

It will merge and append yes thats correct.
Previously it was doing the same thing, but didn't append as the value was empty.
Thus:

Assume user has PATH = ABC
& Custom variable with PATH=HELLO
Now we have PATH=ABC,HELLO

@DonJayamanne Take these two lines as an example.
        this.envVarsService.mergeVariables(customEditVars, mergedVars); // custom vars win over all.
        if (customEditVars[this.platformService.pathVariableName]) {
            this.envVarsService.appendPath(mergedVars, customEditVars[this.platformService.pathVariableName]!);
        }

Still a small issue with the merging order it looks like to me.

Agreed, fixed


Added enum support

Do you think that there's a chance that we can get your changes and mine into the base repo? Support for generators and data sources is particularly useful.

@Bathlamos Would you open a PR for the enum support at https://github.com/IBM/prisma-schema-transformer ? I should be able to get into it by Friday.

@ExiaSR Sure thing, here it is: https://github.com/IBM/prisma-schema-transformer/pull/10
I didn't notice that the changes in this repo had been merged already. Thanks for commenting!


Collection (and other utilities?) documentation

I'm unable to find anything real about collection in the docs page. For example, on the select page, there's these properties itemToValue and itemToString but no mention of what they're for.
By experimentation, I see that itemToValue is necessary for giving Zag a way to uniquely identify the item (selection wont work without it). Looking at the source code, it seems like itemToString is maybe used in type-ahead or search or something like that?

Good catch, @vincerubinetti.
I've updated the docs to mention the collection, especially how to use it when you have custom objects.


LibGfx: Support AVIF images with missing pixi property

This makes it so that a certain error is ignored by the AVIF decoder.
Technically these images are in violation of the AVIF spec, but other browsers also accept them and ImageMagick generates them like that.
(At least the version I just got from apt install imagemagick on Ubuntu 24 does, but I believe this is fixed in newer versions)
An example of a website that is fixed by this would be https://crossuniverse.net/cards/

The change itself LGTM, but a test case needs to be added to TestImageDecoder.cpp to test the image that's been added.

Done.


Fix LineBucket::addGeometry() empty coordinates.

This fix avoids using a negative number to index the coordinates inside the "len" lambda in LineBucket::addGeometry().  It does an early return similar to the one on line ~90 for invalid geometry.

Can we add a unit test for this?

Are there docs for how to run the unit tests?  When I run the mbgl-test-runner project, it says:
[==========] Running 0 tests from 0 test suites.
[==========] 0 tests from 0 test suites ran. (1 ms total)
[  PASSED  ] 0 tests.
I tried filtering for Buckets.LineBucket, but that didn't help.

That should work. What platform are you on? Could be that we don't have it set up properly for Windows.

That should work. What platform are you on? Could be that we don't have it set up properly for Windows.

I'm on Windows 11 with GLFW as the platform.


Access the joint forces generated by frictionloss

Hello,
I'm a researcher trying to use MuJoCo for generating the parameters to control of a physical robot arm. We found that the Mujoco simulated arm movement best reflects the actual arm movement when we add frictionloss at each of the joints. I'm trying to figure out where in the code / if the effects of frictionloss on each joint are stored in the MjData are MjContact structs somewhere, so that I can use them to try to cancel out the friction effects during control.
I saw in the documentation https://mujoco.readthedocs.io/en/latest/computation.html?highlight=friction#friction-loss that it's used as a constraint, I'm hoping that doesn't mean it's not explicitly calculated and stored somewhere. I printed out a few of the friction related variables from the relevant structs (e.g. data.contacts.friction), but it wasn't clear to me how to process the output (e.g., // tangent1, 2, spin, roll1, 2).
I'm hoping someone can provide insight on how I can retrieve the resulting friction forces applied to each joint so that I can negate them in my controller.
Thanks!

fricitonloss is indeed a constraint, but unrelated to contacts. For example joint friction is not affected by the force through the joint, it is a simple dry friction force that opposes motion with a force possibly as large as, but no larger, than the specified frictionloss value.
You can find the actual force values by iterating through the efc_force vector. The ones you want have efc_type == mjCNSTR_FRICTION_DOF, and the associated joint id will be in efc_id.

mujoco.mjtConstraint.mjCNSTR_CONTACT_FRICTIONLESS


LSM: implement move table optimization for compaction

When the table can be moved directly between levels without reading values from the iterators, do so and skip the rest of the compaction process.
This is currently a prototype. The benchmark ends up hitting this case quite frequently and not really doing any Grid disk reads.
Pre-merge checklist
Performance:

[x] Compare zig benchmark on linux before and after this pr.# benchmark results before
1223 batches in 371.97 s
load offered = 1000000 tx/s
load accepted = 26883 tx/s
batch latency p00 = 5 ms
batch latency p10 = 37 ms
batch latency p20 = 42 ms
batch latency p30 = 46 ms
batch latency p40 = 47 ms
batch latency p50 = 47 ms
batch latency p60 = 47 ms
batch latency p70 = 52 ms
batch latency p80 = 57 ms
batch latency p90 = 58 ms
batch latency p100 = 21847 ms
transfer latency p00 = 5 ms
transfer latency p10 = 4123 ms
transfer latency p20 = 18843 ms
transfer latency p30 = 41475 ms
transfer latency p40 = 73113 ms
transfer latency p50 = 118810 ms
transfer latency p60 = 172811 ms
transfer latency p70 = 220026 ms
transfer latency p80 = 272064 ms
transfer latency p90 = 321065 ms
transfer latency p100 = 361967 ms

# benchmark results after
1223 batches in 318.06 s
load offered = 1000000 tx/s
load accepted = 31440 tx/s
batch latency p00 = 5 ms
batch latency p10 = 37 ms
batch latency p20 = 42 ms
batch latency p30 = 42 ms
batch latency p40 = 47 ms
batch latency p50 = 47 ms
batch latency p60 = 47 ms
batch latency p70 = 48 ms
batch latency p80 = 52 ms
batch latency p90 = 57 ms
batch latency p100 = 18951 ms
transfer latency p00 = 5 ms
transfer latency p10 = 4206 ms
transfer latency p20 = 19490 ms
transfer latency p30 = 41898 ms
transfer latency p40 = 72858 ms
transfer latency p50 = 112216 ms
transfer latency p60 = 153890 ms
transfer latency p70 = 195212 ms
transfer latency p80 = 234889 ms
transfer latency p90 = 272316 ms
transfer latency p100 = 308050 ms

Nice work @kprotty.
What is the batch size used per transfers batch here?

LGTM. @jorangreef did you want to give it another look before merge?

This should be good to go now. Could you give it a final look over again? @sentientwaffle