refactor node view

Here's a re factor and consolidation of the infrastructure node view dashboard. This gets rid of a lot of the redundant panels that existed in the old version as well as the panels describing the characteristics of the system (total CPU cores, total memory, and the always wrong total disk size). IMO, this information is not all that useful to monitoring node performance and just takes up space. The new Quickview section contains useful information at a glance, and the following sections dig deeper into their respective categories. On the interface panels, Rx and Tx directions are combined into a single panel, with Tx values showing up as a negative value. This allows more information to be displayed with less space.
Additionally, all attempts to calculate disk size based on the size of the mounted file systems have been thrown out for two reasons:

It's nearly impossible to build a query that can figure out hardware disk size based on file systems in all environments
The information is not useful.
It is more useful to measure how much of each partition is being used. Thus, I replaced all attempts to determine how much of the disk is used with how much of each file system is used. This fixes https://bugzilla.redhat.com/show_bug.cgi?id=1843667

First of all, this looks really good!
Couple comments inline.

Here's a re factor and consolidation of the infrastructure node view dashboard. This gets rid of a lot of the redundant panels that existed in the old version as well as the panels describing the characteristics of the system (total CPU cores, total memory, and the always wrong total disk size). IMO, this information is not all that useful to monitoring node performance and just takes up space.

Yea I can get behind that logic.

The new Quickview section contains useful information at a glance, and the following sections dig deeper into their respective categories.

Yes! I had always wanted all those gauges moved up since I think seeing that information together provides value.

On the interface panels, Rx and Tx directions are combined into a single panel, with Tx values showing up as a negative value. This allows more information to be displayed with less space.

Nice!

Additionally, all attempts to calculate disk size based on the size of the mounted file systems have been thrown out for two reasons:
1. It's nearly impossible to build a query that can figure out hardware disk size based on file systems in all environments

2. The information is not useful.
   It is more useful to measure how much of each partition is being used. Thus, I replaced all attempts to determine how much of the disk is used with how much of each file system is used. This fixes https://bugzilla.redhat.com/show_bug.cgi?id=1843667

Cool, that makes sense. The panels look so much better. The only question I have is the use of legends on some panels and no legends on others. Do you have some sort of workflow that determines when to enable the legend vs hiding it? I do like the cleaner overall look of the no-legend approach.
This looks to be much improved though!

This all looks pretty good to me! I also noticed the legend inconsistency and would prefer to see them on every panel that has more than one line ("Data" and "Aggr. Usage by type" need them for sure. "Hugepages" probably does not need it.). I'd also suggest try moving them to the bottom instead of the right and see if they are still readable; doing that turns whitespace into graph space.

Fixed hugepages querry error @csibbitt pointed out, added panel showing percentage of hugepages used, and included keys in all panels that include more than one metric. Panel sizes have been adjusted so that the first three keys are viewable on a standard 13.5in monitor, and they are now included below the graphs. This makes each graph take up more space on the screen, but they values are now selectable.

Hugepages graph scale should be positive values only.
This is looking really good, though! Nice work, Paul.

Yea this looks great, nice work! I looked again and don't see anything else to comment on.


Add docs test to validate presence and length of summary field in front-matter

The summary field in a page's front-matter is used to generate a meta description that is important for SEO. We should add a PR check to make sure pages have summary and then they are between 50 and 160 characters.

cc @yingzhuchin, another test that we could build into Netlify.

Here are pages in v21.1 without a summary:

admin-ui-cluster-overview-page.md

admin-ui-statements-page.md

cockroachdb-architecture.md

Here are all the pages that have summaries with fewer than 50 characters:

bulk-update-data.md

cancel-job.md

cockroach-import.md

cockroach-nodelocal-upload.md

cockroach-quit.md

cockroach-start-single-node.md

cockroach-userfile-delete.md

cockroach-userfile-upload.md

common-errors.md

connection-pooling.md

cost-based-optimizer.md

date.md

dbeaver.md

debug-and-error-logs.md

delete.md

deploy-cockroachdb-on-digital-ocean-insecure.md

deploy-cockroachdb-on-digital-ocean.md

enterprise-licensing.md

enum.md

file-an-issue.md

flyway.md

import.md

intellij-idea.md

known-limitations.md

licensing-faqs.md

liquibase.md

monitor-cockroachdb-with-prometheus.md

multilinestring.md

multipoint.md

multipolygon.md

order-by.md

polygon.md

query-spatial-data.md

recommended-production-settings.md

release-savepoint.md

remove-nodes.md

savepoint.md

schema-design-database.md

schema-design-indexes.md

schema-design-table.md

schema-design-update.md

selection-queries.md

sql-grammar.md

sql-playground.md

sql-statements.md

support-resources.md

time.md

update.md

use-cloud-storage-for-bulk-operations.md

And here are pages that have summaries with over 160 characters:

create-changefeed.md
orchestrate-cockroachdb-with-kubernetes-multi-cluster.md

I think we should fix the above before merging any new tests to CI. Otherwise we're incurring build minutes on each PR without taking any action. What do you think, @jseldess?

@yingzhuchin, just opened a PR to address the meta description issues: https://github.com/cockroachdb/docs/pull/10251. Please review. Note that the following two pages without meta descriptions will be handled separately by @taroface:
admin-ui-cluster-overview-page.md
admin-ui-statements-page.md


[TRA1][W14-B4] Chen Zhi Liang

Remove an unused import from favorite command class

This PR is not necessary as the PR for this activity should be sent from your organization as a group. I have seen your commits in the group PR. So i will close this.


Templates not found

I have just brought up compose after cloning the repo, and the application seems to be working - I logged into the web UI at :8080 and added an ASN, but clicking on "Import from PeeringDB" under internet exchanges results in a redirect to :8080/internet-exchanges/peeringdb-import/ and the following error:
<class 'peering.models.AutonomousSystem.DoesNotExist'>
AutonomousSystem matching query does not exist.
Python version: 3.9.1
Peering Manager version: v1.3.0

Similarly, clicking on "Add a Peering Session" under an AS's Direct Peering Sessions tab results in a redirect to :8080/direct-peering-sessions/add/?autonomous_system=&return_url= and error
<class 'django.template.exceptions.TemplateDoesNotExist'>
peering/session/direct/add_edit.html
Python version: 3.9.1
Peering Manager version: v1.3.0

Any help would be greatly appreciated!

Both issues are not related to the Docker images themselves.

Has been addressed in this commit, will be available in next release.

Linked to this issue, will be available in next release.

Both issues are not related to the Docker images themselves.

Has been addressed in this commit, will be available in next release.

Linked to this issue, will be available in next release.


Git and Less

Hi,
when I enter the command "git log" I get the message "error: cannot run less: No such file or directory". Git is installed and Less is installed too.
When I enter the command "less --help" I get the message "WARNING: terminal is not fully functional". When I set the environment variable TERM to xterm, then the command works. I found the solution for this here.
https://blog.rtwilson.com/how-to-fix-warning-terminal-is-not-fully-functional-error-on-windows-with-cygwinmsysgit/
I found another article, that the pager for git has to be configured. I tried this, but without success. The article is here.
https://itenium.be/blog/dev-setup/git-core-pager-less-pager/
Is there`s a way to use a pager for git on IBM i?
Regards
Frank

Do you have /QOpenSys/pkgs/bin on your PATH? If it is, git should be able to run it just fine. Since you say you can run less --help, that would seem to me that your PATH should be set correctly, though if you ran it unqualified.
As to the TERM setting, this is usually set by your terminal emulator and passed through SSH. Are you running from SSH or QSH/QP2TERM? If SSH, what client are you using (PuTTY, TeraTerm, ssh, etc)?

I ran the "git log" from QSH. Now I tried it from my pc with "Git Bash" and it worked. Is it possible to make it work from QSH? On our own system of course I can use "Git Bash", but thats not possible on customer systems. On customer systems I have e.G. Microsoft Teams or Teamviewer and then I can start QSH, but I cannot start "Git Bash". Its to much to explain to the customer he has to install "Git Bash" on his pc.
Generally speaking, I have the need to run .sh scripts with git commands in batch jobs invoked by QSH. I use QSH, because the program who`s calling the script can retrieve the return code set by the script.  Prior to calling QSH I set the following environment variables.
QIBM_MULTI_THREADED : 'Y'
QIBM_QSH_CMD_OUTPUT : 'NONE'
QIBM_QSH_CMD_ESCAPE_MSG : 'Y'
QIBM_CCSID : '1208'
PASE_LANG : 'EN_US'
QIBM_PASE_CCSID : '1208'
PATH : '/QOpenSys/pkgs/bin:/usr/bin:.:/QOpenSys/usr/bin'
From your point of view, is there`s a problem to use QSH and scripts with git commands?

From a 5250 environment (either QSH or QP2TERM) you will not get a proper Unix terminal behavior that some applications expect. Many programs work fine, but programs like pagers won't work properly. I would suggest setting git's pager to cat, eg. git config --global core.pager cat or setting PAGER=cat, then relying on page up/down within QSH for paging.

From your point of view, is there`s a problem to use QSH and scripts with git commands?

No, in general there is no issue.

Thank you for this information. This helped.


Support for Visual Studio for Mac

(This issue should be labeled with "question")
In Visual Studio for Mac, I can add the NuGet package. But no warnings are generated (neither during build nor during code changes in the IDE).
Is this is a problem of StyleCopAnalyzers?
Or does Visual Studio for Mac not (yet) support Roslyn analyzers via NuGet (and if so, does someone know where to find more information about that)?

@johannes-schmitt Using VS4Mac 7.3 and the beta version of the analyzers worked for me to get the warnings/errors that I needed. I'm not sure if the beta is required, but that's what I had. You make need to edit the project to specify the Code Analysis rule set.
<CodeAnalysisRuleSet>../../StyleCop.Analyzers.ruleset</CodeAnalysisRuleSet>

@johannes-schmitt Using VS4Mac 7.3 and the beta version of the analyzers worked for me to get the warnings/errors that I needed. I'm not sure if the beta is required, but that's what I had. You make need to edit the project to specify the Code Analysis rule set.
<CodeAnalysisRuleSet>../../StyleCop.Analyzers.ruleset</CodeAnalysisRuleSet>

This worked for me as well - at least to get build warnings/errors to show up on rebuild of a particular project. I'm using VS for Mac 8.4.8 from the stable channel.
Using this ruleset file as a template I can set behavior for specific SA rules
Unfortunately, I don't see any of the nice roslyn squiggles I'm expecting to see at coding-time. Still trying to figure that out.


Add AwsSystemsManagerParameterStoreSettingsSource

What I'm adding
In a spirit similar to #175, this PR adds support for AWS Systems Manager Parameter Store.  The name is a bit of a mouthful, but I errored on the side of verbosity and named the source AwsSystemsManagerParameterStoreSettingsSource.  Something like AwsSsmParameterStoreSettingsSource might be better.
For Pydantic v1, we have been using https://github.com/developmentseed/pydantic-ssm-settings/ on a number of projects, however having the functionality included in Pydantic Settings core seems preferred.
How I did it
I more or less copied the pattern of AzureKeyVaultSettingsSource.
Uncertainties
The testing mocks out the AWS SSM Client, however typically I use something like moto to mock out the AWS services when testing.  I wasn't sure about this project's appetite for the added test dependency.
Additionally, I was a bit uncertain about typing the SSMClient.  Projects like boto3-stubs provide such types, however I wasn't sure about the project's appetite for adding this dependency.

Thanks @alukach for this PR.
First of all, I'm not sure we will support this settings source.
I'll check with the team and get back to you first.

@hramezani Sounds good, I'll stand by and wait for the green light before committing any more time to this.  Given @samuelcolvin's endorsement of supporting AWS SecretsManager, I figured AWS Parameter Store wouldn't be much of a stretch.  Once this is merged, I intend also to add functionality similar to Secrets Manager.


Availability: Optimize EnableTcpConnectionEndpointRediscovery with nonblocking cache

EnableTcpConnectionEndpointRediscovery  causes an Address cache refresh when the TCP connection is closed.
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/70b1b4a71216d4437229ae7a4f35b5a686c4950a/Microsoft.Azure.Cosmos/src/CosmosClientOptions.cs#L494
This uses the direct ConnectionStateListner:
https://msdata.visualstudio.com/_git/CosmosDB?path=/Product/Microsoft.Azure.Documents/SharedFiles/ConnectionStateListener.cs
The ConnectionStateListner calls this UpdateAsync on the GlboalAddressResolver. The problem here is it does a TryRemove which will remove all the addresses from the cache.
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/70b1b4a71216d4437229ae7a4f35b5a686c4950a/Microsoft.Azure.Cosmos/src/Routing/GlobalAddressResolver.cs#L150
This is bad because it will block all requests to that partition until the address are retrieved from the gateway which can take multiple seconds if there are networking/gateway issues. If the new request fails then it has to be retried until there is a success. Ideally only the one address should have been marked as unhealthy and it trigger the cache refresh. That way the other 3 replicas can continue to process requests.
This can be done by following a similar design as this:
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/70b1b4a71216d4437229ae7a4f35b5a686c4950a/Microsoft.Azure.Cosmos/src/Routing/GatewayAddressCache.cs#L199

One concern with this approach is it could cause throttling because of the number of refreshes that would occur at the same time.


Delete All

Give a command to Jukebox, and with confirmation, it will find and delete all of a user's previous posts from the server.

Not possible due to api limitation


Make ConnectionConfiguration initialiser visible for objective-c

@lukasz-szyszkowski can you please delete the underlying branch if it's no longer needed.

@lukasz-szyszkowski can you please delete the underlying branch if it's no longer needed.

done


Add some unit tests

This demo serves as test project, but would be ideal to have dev setup to run automated tests.

Closing here. This issue belongs to the AppServices submodule


[Bug] Cannot start the module

Current Behavior
Hello,
I don't know what's happening but I cannot start the module anymore.
It's installed successfully but it cannot launch, I've tried uninstall reinstall but maybe it's not sufficient
I also tried to restore my whole home assistant from a 1 week backup
Expected Behavior
No response
Steps To Reproduce
No response
Environment
System Information

version
core-2024.4.2

installation_type
Home Assistant OS

dev
false

hassio
true

docker
true

user
root

virtualenv
false

python_version
3.12.2

os_name
Linux

os_version
6.6.20-haos

arch
aarch64

timezone
Europe/Paris

config_dir
/config

Home Assistant Community Store

GitHub API
ok

GitHub Content
ok

GitHub Web
ok

GitHub API Calls Remaining
4999

Installed Version
1.34.0

Stage
running

Available Repositories
1400

Downloaded Repositories
15

Home Assistant Cloud

logged_in
true

subscription_expiration
17 avril 2024 à 02:00

relayer_connected
true

relayer_region
eu-central-1

remote_enabled
true

remote_connected
true

alexa_enabled
true

google_enabled
false

remote_server
eu-central-1-12.ui.nabu.casa

certificate_status
ready

instance_id
4a71c214125a4030a54f7381e8137246

can_reach_cert_server
ok

can_reach_cloud_auth
ok

can_reach_cloud
ok

Home Assistant Supervisor

host_os
Home Assistant OS 12.1

update_channel
stable

supervisor_version
supervisor-2024.04.0

agent_version
1.6.0

docker_version
24.0.7

disk_total
28.0 GB

disk_used
9.2 GB

healthy
true

supported
true

board
green

supervisor_api
ok

version_api
ok

installed_addons
File editor (5.8.0), RPC Shutdown (2.4), Terminal & SSH (9.10.0), Filebrowser (2.23.0_13), Firefox (1.1.0)

Dashboards

dashboards
2

resources
9

views
5

mode
storage

Recorder

oldest_recorder_run
4 avril 2024 à 15:45

current_recorder_run
11 avril 2024 à 13:02

estimated_db_size
156.98 MiB

database_engine
sqlite

database_version
3.44.2

Spotify

api_endpoint_reachable
ok

Container creation
Installed through Mincka's Home Assistant Add-ons
Container log
[cont-env    ] GTK_THEME: terminated successfully.
[cont-env    ] GTK_THEME: loading...
[cont-env    ] HOME: loading...
[cont-env    ] QT_STYLE_OVERRIDE: executing...
[cont-env    ] QT_STYLE_OVERRIDE: terminated successfully.
[cont-env    ] QT_STYLE_OVERRIDE: loading...
[cont-env    ] TAKE_CONFIG_OWNERSHIP: loading...
[cont-env    ] XDG_CACHE_HOME: loading...
[cont-env    ] XDG_CONFIG_HOME: loading...
[cont-env    ] XDG_DATA_HOME: loading...
[cont-env    ] XDG_RUNTIME_DIR: loading...
[cont-env    ] XDG_STATE_HOME: loading...
[cont-env    ] container environment variables initialized.
[cont-secrets] loading container secrets...
[cont-secrets] container secrets loaded.
[cont-init   ] executing container initialization scripts...
[cont-init   ] 10-certs.sh: executing...
[cont-init   ] 10-certs.sh: terminated successfully.
[cont-init   ] 10-check-app-niceness.sh: executing...
[cont-init   ] 10-check-app-niceness.sh: terminated successfully.
[cont-init   ] 10-clean-logmonitor-states.sh: executing...
[cont-init   ] 10-clean-logmonitor-states.sh: terminated successfully.
[cont-init   ] 10-clean-tmp-dir.sh: executing...
[cont-init   ] 10-clean-tmp-dir.sh: terminated successfully.
[cont-init   ] 10-fontconfig-cache-dir.sh: executing...
[cont-init   ] 10-fontconfig-cache-dir.sh: terminated successfully.
[cont-init   ] 10-init-users.sh: executing...
[cont-init   ] 10-init-users.sh: terminated successfully.
[cont-init   ] 10-nginx.sh: executing...
[cont-init   ] 10-nginx.sh: terminated successfully.
[cont-init   ] 10-openbox.sh: executing...
[cont-init   ] 10-openbox.sh: terminated successfully.
[cont-init   ] 10-pkgs-mirror.sh: executing...
[cont-init   ] 10-pkgs-mirror.sh: terminated successfully.
[cont-init   ] 10-set-tmp-dir-perms.sh: executing...
[cont-init   ] 10-set-tmp-dir-perms.sh: terminated successfully.
[cont-init   ] 10-vnc-password.sh: executing...
[cont-init   ] 10-vnc-password.sh: terminated successfully.
[cont-init   ] 10-web-data.sh: executing...
[cont-init   ] 10-web-data.sh: terminated successfully.
[cont-init   ] 10-x11-unix.sh: executing...
[cont-init   ] 10-x11-unix.sh: terminated successfully.
[cont-init   ] 10-xdg-runtime-dir.sh: executing...
[cont-init   ] 10-xdg-runtime-dir.sh: terminated successfully.
[cont-init   ] 15-cjk-font.sh: executing...
[cont-init   ] 15-cjk-font.sh: terminated successfully.
[cont-init   ] 15-install-pkgs.sh: executing...
[cont-init   ] 15-install-pkgs.sh: terminated successfully.
[cont-init   ] 55-check-snd.sh: executing...
[cont-init   ] 55-check-snd.sh: sound device group 29.
[cont-init   ] 55-check-snd.sh: terminated successfully.
[cont-init   ] 55-firefox.sh: executing...
[cont-init   ] 55-firefox.sh: generating machine-id...
[cont-init   ] 55-firefox.sh: terminated successfully.
[cont-init   ] 56-firefox-set-prefs-from-env.sh: executing...
[cont-init   ] 56-firefox-set-prefs-from-env.sh: terminated successfully.
[cont-init   ] 85-take-config-ownership.sh: executing...
[cont-init   ] 85-take-config-ownership.sh: terminated successfully.
[cont-init   ] 89-info.sh: executing...
    ╭――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╮
    │                                                                      │
    │ Application:           Firefox                                       │
    │ Application Version:   119.0-r0                                      │
    │ Docker Image Version:  23.11.3                                       │
    │ Docker Image Platform: linux/arm64                                   │
    │                                                                      │
    ╰――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╯
[cont-init   ] 89-info.sh: terminated successfully.
[cont-init   ] all container initialization scripts executed.
[init        ] giving control to process supervisor.
[supervisor  ] loading services...
[supervisor  ] loading service 'default'...
[supervisor  ] loading service 'app'...
[supervisor  ] loading service 'gui'...
[supervisor  ] loading service 'nginx'...
[supervisor  ] loading service 'xvnc'...
[supervisor  ] loading service 'certsmonitor'...
[supervisor  ] service 'certsmonitor' is disabled.
[supervisor  ] loading service 'openbox'...
[supervisor  ] loading service 'logrotate'...
[supervisor  ] loading service 'logmonitor'...
[supervisor  ] service 'logmonitor' is disabled.
[supervisor  ] all services loaded.
[supervisor  ] starting services...
[supervisor  ] starting service 'xvnc'...
[xvnc        ] Xvnc TigerVNC 1.13.1 - built Nov 17 2023 22:52:03
[xvnc        ] Copyright (C) 1999-2022 TigerVNC Team and many others (see README.rst)
[xvnc        ] See https://www.tigervnc.org for information on TigerVNC.
[xvnc        ] Underlying X server release 12014000
[xvnc        ] Thu Apr 11 13:05:35 2024
[xvnc        ]  vncext:      VNC extension running!
[xvnc        ]  vncext:      Listening for VNC connections on /tmp/vnc.sock (mode 0660)
[xvnc        ]  vncext:      Listening for VNC connections on all interface(s), port 5900
[xvnc        ]  vncext:      created VNC server for screen 0
[supervisor  ] starting service 'nginx'...
[nginx       ] Listening for HTTP connections on port 5800.
[supervisor  ] starting service 'openbox'...
[supervisor  ] starting service 'app'...
[app         ] Mozilla Firefox 119.0
[supervisor  ] all services started.

Container inspect
No response
Anything else?
No response

It looks like my issue is not affecting Firefox Docker but all Addons in general..


Support MySQL and PostgreSQL

MySQL and PostgreSQL are common database options, but currently not officially supported.
To support these, we'd need to update the Travis configuration to run the current test suite against those databases as well.

:shipit:


Mistake in the docs for resumable uploads

The docs give following possibilities for the conflict behavior:
<EMAIL_ADDRESS>"rename | fail | overwrite"
However, the overwrite option does not exists and should be changed to replace.

Link to the docs

Hi @JValck, I've logged an internal bug for us to go update the documentation.  Thanks for noticing this!
-Jeremy

The documentation is now updated.


Do not prefer empty MAC address

Motivation:
When comparing MAC addresses searching for the best MAC address, if locally-administered address (e.g., from a Docker container) is compared against an empty MAC address, the empty MAC address will be marked as preferred. In cases this is the only available MAC address, this leaves Netty using a random machine ID instead of using a perfectly valid machine ID from the locally-adminstered address.
Modifications:
This commit modifies the MAC address logic so that the empty MAC address is not preferred over a locally-administered address. This commit also simplifies the comparison logic here.
Result:
Empty MAC addresses will not be preferred over locally-administered addresses thus permitting the default machine ID to be the locally-adminstered MAC address if it is the only available MAC address.

This problem was introduced by #6147.

@normanmaurer Thanks for reviewing, I think this is ready for you to take another look.

@Scottmitch PTAL and merge once satisfied

Thanks @normanmaurer. I've squashed in preparation for integration.

@jasontedor thanks... @Scottmitch will take care of the rest 👍

4.1 (42c0359)
@jasobrown - Sorry for the breakage ... thanks for the fix & unit tests!

Thank you @Scottmich (and no worries, breakage happens, especially when there are no unit tests).

@jasontedor doh, that hurts ... but yes you are right, no unit test == bad 👎

@normanmaurer Sorry, I was not trying to make a hurtful comment, but rather an empathetic one.

@jasontedor haha all good :) Truth hurts, no worries ;)

@jasontedor - No harm done ... unit tests FTW.

@normanmaurer @jasontedor
Hi i am getting same error when deployed on virtual linux container I am using Vertx framework latest version which uses Netty 4.1.8 final.
I just wanted to know is this warning can create problems for me and will my code work with this warning?

I just wanted to know is this warning can create problems for me and will my code work with this warning?

It's mostly annoying but harmless.


Fix Delete All Test Case

fix: ui changed - now needs model name to confirm delete

:tada: This PR is included in version 0.331.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Rust Integration: Perform Max Flow in Rust instead of Python

The overarching goal is to improve performance by using Rust instead of Python to perform intense computations, while maintaining the Python API.
We implement Python-Rust interop using pyo3. We use the pathfinding crate for the Rust-side implementation of the Edmonds-Karp max flow algorithm.
Through initial profiling, we have identified the call to nx.max_flow to be the overwhelming bottleneck (51-74% of total time spent). This PR replaces the calls to nx.max_flow with Rust-side bindings, which initial profiling shows a total speedup of 1.6-2.6x.
Note: profiling infrastructure implemented in https://github.com/ohjuny/lowtime-exp
Build and Run
Assume you have some project that is using lowtime, and you have a local clone of this lowtime branch.
$ ls
project/    lowtime/

Then run the following to set up a virtual environment with this version of lowtime.
$ cd project/
$ python -m venv .env          # create virtual environment
$ source .env/bin/activate     # activate virtual environment
$ cd ../lowtime/
$ pip install -e .             # (editable) install lowtime
$ pip install maturin          # install maturin, required to build lowtime_rust
$ cd ./lowtime/lowtime_rust
$ maturin develop -r           # compile rust code using --release flag
$ cd ../../../project/
$ #work on project, lowtime is now installed

Could you also quickly document how to build and run this?

Not 100% knowledgeable on Python packaging and distribution, but here's what I tried:

Started with src layout that you linked before. I got confused about how to reconcile the 2 different pyproject.tomls that used different build-backends.
Found an example (deltalake-python) that had mixed Python and Rust. The organization looked clean, and used a single maturin build-backend for both Rust and Python files.
Read through maturin README for more info as well. Main takeaway was that (1) the deltalake-python organization works and (2) it is good convention to prepend _ to the Rust library if the intention is that it is internal.

I'm trying to fix the last ci.yaml pyright test, where it seems to say that _lowtime_rust does not exist. When I test locally, just running "pip install lowtime" seems to be sufficient in building both Rust and Python libraries, so I'm not sure why the automated test is failing. Will have to look deeper into this, but let me know if you spot anything.

Have a pyi file? lowtime/_lowtime_rs.pyi.
from typing import List, Tuple

class PhillipsDessouky:
    node_ids: List[int]

    def __init__(
        self,
        node_ids: List[int],
        source_node_id: int,
        sink_node_id: int,
        edges_raw: List[Tuple[Tuple[int, int], float]]
    ) -> None: ...

    def max_flow(self) -> List[Tuple[Tuple[int, int], float]]: ...

Have a pyi file? lowtime/_lowtime_rs.pyi.
from __future__ import annotations

class PhillipsDessouky:
    node_ids: list[int]

    def __init__(
        self,
        node_ids: list[int],
        source_node_id: int,
        sink_node_id: int,
        edges_raw: list[tuple[tuple[int, int], float]]
    ) -> None: ...

    def max_flow(self) -> list[tuple[tuple[int, int], float]]: ...

DIdn't seem to work :( Will look further into this.


[BUG] csi v0.25.0 can not manage exist mount pod which has no hashval annotation

What happened:
In version v0.25.0, a new mount pod annotation hashval was added for passing fusefd to mount pod. I check the pod driver code, found it will throw error, if mount pod has no hashval annotation(create by old version csi).
https://github.com/juicedata/juicefs-csi-driver/blob/c545bcd8d343d10eca9cd49959edb1dac52b596a/pkg/controller/pod_driver.go#L152-L158
And worse, csi uses err group to handle mount pod. since exist mount pod checkAnnotations throw a error, this driver will cancel other mount pods' reconcile.
https://github.com/juicedata/juicefs-csi-driver/blob/c545bcd8d343d10eca9cd49959edb1dac52b596a/pkg/controller/reconciler.go#L147-L153
What you expected to happen:
remove hashval annotaintion check, not throw an error for exist mount pod.
How to reproduce it (as minimally and precisely as possible):

have mount pod create by 0.24.x
upgrade csi to v0.25.0

The mount pod must have hashVal label no matter what version CSI is. We think pod without hashVal label is not mount pod.
Besides, mount pods which can smooth upgraded must meet the following conditions:

CSI version >=v0.25.0
created by CSI with v0.25.0 version and above.
image must be ce-v1.2.1 or ee-5.1.0 and above.

See more details in https://juicefs.com/docs/zh/csi/upgrade-juicefs-client#smooth-upgrade

@zwwhdls sorry for my mistake, hashval not add in verion v0.25.0, but it clear that mount pod hashval was added after v0.13.2
in version v0.13.2, we can see mount pod has no hashval annotation, can pod name has no hash suffix.
https://github.com/juicedata/juicefs-csi-driver/blob/91b1fe08d35d68047b1a7385d6fad424dc980c3c/pkg/juicefs/mount/pod_mount.go#L420
we can see the exist mount create by only v0.13.2, it has no hash suffix, and no hashval annotation.
juicefs-svr40176in5112-vacation-ai-pv                      1/1     Running             0          350d    app.kubernetes.io/name=juicefs-mount,app.paas.ctrip.com/appid=100010351,cdos.ctrip.com/region=SHA,paas.ctrip.com/bu=SYS,paas.ctrip.com/env=PROD,paas.ctrip.com/orgid=58,paas.ctrip.com/type=app,topology.kubernetes.io/region=SHA,topology.kubernetes.io/zone=SHARB


fix: emit a warning for credential-alias shadowing

What does this PR try to resolve?
If a credential-alias shadows a built-in provider the user could be confused about which provider is being used.
How should we review this PR?
See the test to see what the warning looks like.
r? @epage who listed this as a concern on the FCP in #8933

@bors r+

:pushpin: Commit 8f18f2bdc809f5a3bbdcb29069708f45f17775be has been approved by epage
It is now in the queue for this repository.

:hourglass: Testing commit 8f18f2bdc809f5a3bbdcb29069708f45f17775be with merge d5336f813df39d476e61fc46daabb1446350660a...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing d5336f813df39d476e61fc46daabb1446350660a to master...


Export Java.Do module

Create a Java.Do module.
This module defines new type classes and helper functions to work with JWT types shaped like functors, applicative or monads.
This PR relates to https://github.com/typelead/eta/issues/243

hey @rahulmutt,
I try to condense here the discussion we maintained through gists and the work you sent me to deal with  Functions of any arity.
I took me long but I was able to make it work with both my validation and OSGi "functional java libraries" so this should be, at least, a good start for the module.
Please accept my apologies if the formatting of the code or the PR is not the appropiate. Feel free to shape it as you need.
Please let me know if you have any suggestions.
Thanks in advance.

Looks good, thanks! If the examples you've worked out are available publicly, please do share on Gitter.

they are still not available but I can try and groom and upload them

No problem. Take your time.


Small Typo

Small Typo
Set-AzureRmSiteRecoveryVaultSettings -ASRVault
Thanks!

Hi @CharbelNemnom, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@CharbelNemnom : Thanks for your contribution to the Azure documentation! The author, @bsiva, has been notified to review your proposed change.

@CharbelNemnom : Thanks for the suggestion, but this is in fact not a typo. We have two different parameters to set the vault context depending on how the vault was created. A recovery services vault (which is the combined backup + site recovery vault) that is either created via the new Azure portal or via PowerShell using the New-AzureRmRecoveryServicesVault cmdlet will take the parameter -ARSVault as is shown in this example. A Site Recovery vault which was created using the Classic portal will take the paramenter -ASRVault. Thanks for pointing this out however. We've received feedback from others as well that this is confusing and are working on making the parameters more distinct or unifying them so as to avoid the confusion

Thank You @bsiva
Yes updating the parameters will help since it's confusing.
BR,
-Charbel

@CharbelNemnom @bsiva - is it okay to close this pull request?

Yes please. Thank You!


Investigate if we can remove "transitionend" event listener

The bug described in https://bugzilla.mozilla.org/show_bug.cgi?id=1866773 is because at some point a blur event is triggered by:
https://github.com/mozilla/pdf.js/blob/4bf7ff20270ec5dd6458a136c79fdf1cc590a1c6/web/app.js#L687-L695
This code has been added a while back:
https://github.com/mozilla/pdf.js/pull/1614
If I understand correctly the idea was to resize to main container once the transition for the sidebar resize is done.
The transitionend event is triggered for whatever transition we've in the main container, so for example when we're searching a word in the pdf, the event is triggered (because of the spinner).
I'd be in favor to remove this listener because it doesn't make sense to trigger a resize for whatever transition and we should figure out how to resize correctly the main container without such an event.
@Snuffleupagus, any ideas, opinions ?

If I understand correctly the idea was to resize to main container once the transition for the sidebar resize is done.

Yes, it's intended to trigger rendering in the viewer (for the "named" zoom value) when the sidebar has been opened/closed.
We obviously need to keep that working, and another possibility would be to use an already existing ResizeObserver to trigger re-rendering. However, the only annoying thing would be that the ResizeObserver also fires while the sidebar is opening/closing and we really don't want to trigger re-rendering for all those "intermediate" events. We could perhaps use a timeout to avoid triggering re-sizing until things have settled down; I'll try to do some experiments later this week.

The transitionend event is triggered for whatever transition we've in the main container, so for example when we're searching a word in the pdf, the event is triggered (because of the spinner).

Using the latest Nightly version, on Windows 11, I cannot actually reproduce this; maybe it's different on other OSes?

Yes Emilio fixed the bug so it isn't a concern anymore.
But, for me at least, the bug highlights this use of transitionend and I think we shouldn't use it, so it's why I filed this issue.


Sometime, we can get stuck in the New Offer page

When creating offer, it happen that we can get stuck in the New Offer page.
It is then impossible to go back to the main Offers Page.
To reproduce:

Create a new Offer, with any assets.
Click Create Offer Button.
Instead of Saving the offer, click outside or click the x button to close the modal dialog.

The issue with that is that we will lost the offer in the wallet because it is not saved. But we could have shared it externally (on Dexie for example 😄). We then need to import it again if we want to see it on the wallet.
Really looking forward for this amazing wallet that really is blazing fast.
Version 0.8.1 - Mac OS Sonoma

Thanks, I've definitely noticed things getting stuck on the offer page. I'll look into it

Is this fixed @Ganbin?


Last couple entries on yearly and global leaderboard show zero score while it should be a score between 0-1

Fixed.


fix(#268): Removes overflow auto and sets to inherit so selects in card can go over the div boundary

Fixes #268

@phantomjinx Do you know why you hard-set the value to auto on the css? I checked all the camel pages to see if the change breaks something but it'd be great to know if you remember what was it so I can double check.
In any case, using inherit should be extra safe instead of removing the rule altogether

Ok I just talked with @phantomjinx and there was one thing I broke with the change. Horizontal overflows on smaller screens don't work (because the Page component itself has it set to hidden). I added that as well.


Add 'toPandas' test for Indexes

A test for the 'to_pandas' exists in the indexes.py as doctest, but for 'toPandas' is missing.
I added this because I think it would be better to test this part for compatibility with Spark.

Codecov Report

Merging #688 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #688   +/-   ##
=======================================
  Coverage   93.54%   93.54%
=======================================
  Files          32       32
  Lines        5455     5455
=======================================
  Hits         5103     5103
  Misses        352      352

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1b8bfb...05a7caa. Read the comment docs.

Codecov Report

Merging #688 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #688   +/-   ##
=======================================
  Coverage   93.54%   93.54%
=======================================
  Files          32       32
  Lines        5455     5455
=======================================
  Hits         5103     5103
  Misses        352      352

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1b8bfb...05a7caa. Read the comment docs.


Support phpcs 3.3

PHPCS 3.3.0 made a few changes which break some of these sniffs:
https://github.com/squizlabs/PHP_CodeSniffer/releases

The T_ARRAY_HINT token has been deprecated and will be removed in version 4.

The T_RETURN_TYPE token has been deprecated and will be removed in version 4.

Fixed by #64 for phpcs 3.3 and by #65 for phpcs 3.2.


chore: Adds 4 extra files for tests

This is related to https://github.com/trustification/trustify/issues/1036#issue-2680107467
to make easy for us to select those files when testing upload.

nitpick - one might consider a better PR title then the ambiguous current one.


Eltwise Binary tests don't truly pass.

Add, subtract, and multiply all do not pass.
The assertion at the end of these tests is:
assert [compare_with_golden_pcc(golden=fo, calculated=co, pcc=0.99) for fo, co in zip(fw_out, co_out)]

However, any list of size > 0 will evaluate to true:
>>> bool([False])
True

What we want to assert for these tests is
assert all([compare_with_golden_pcc(golden=fo, calculated=co, pcc=0.99) for fo, co in zip(fw_out, co_out)])

Which ensure all list elements are True
From debugging, it looks like only the first row of the output is valid data, the rest are zeros.
CC: @nvukobratTT

Thanks for the notice, we should update this for all tests we're currently running..
I'll have one more eng from MCW coming to the Forge in a few days, so this looks like a good issues for a starter :))

Ofc, let me know if you would like to take a look beforehand :))

Ofc, let me know if you would like to take a look beforehand :))

Yeah I’m taking a look right now, it seems like this did work recently, bisecting…

@nvukobratTT I'm not so sure what the issue was. Uplifting to the latest MLIR has fixed the issue though. It may have been entirely my fault and I had edited something on my branch without remembering.

Merged


Python - newest SDK/pyrealsense2 does not allow connection via USB hub

Required Info

Camera Model
T265

Firmware Version
<IP_ADDRESS>9

Operating System & Version
Win10

Kernel Version (Linux Only)
(e.g. 4.14.13)

Platform
PC

SDK Version
<IP_ADDRESS>4

Language
python

Segment
{Robot/Smartphone/VR/AR/others }

Issue Description
Hi!
I just upgraded to the newest Realsense SDK and pyrealsense2.
Off topic question: do I need to update both or is it enough to just update pyrealsense2?
I have two T265 cameras connected via USB hub which with the version I had before worked fine (pyrealsense <IP_ADDRESS>5, SDK <IP_ADDRESS>2) but when I now plugged them in via USB hub they won't work.
They work fine when plugging in directly in the laptop.
I kind of need the hub because I have a lot of other stuff connected as well.
Thanks in advance,
Beate

Sorry I didn't mean to close it...


Added points on collaborations

Hey there,
I added two points on how to work with other existing projects that inhabit the same space of features/tools etc.
Cheers!

Thanks for that, Bastien! There's some overlap here with other features, I'll edit the file a bit to take that into account.

Great, thanks a lot! 👍


[Débito Técnico]: Teste em HTML

Tipo de Débito Técnico
Código Duplicado
Pontuação
1
Link do Repositório
google.com
Descrição do Débito Técnico
No response
Soluções Propostas ou Considerações
No response
Notas Adicionais
Teste

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx


Reactify

Upon clean install i get:
events.js:72
throw er; // Unhandled 'error' event
^
Error: Cannot find module 'reactify'
My project apparently needs reactify instead of the id-builder using it's own reactify
I don't know if it works this way but maybe you can solve it by requiring reactify at the start of  browserify.js and then pass the variable containing reactify into the b.transform instead of b.transform("reactify").
Kind off like this:
reactify = require("reactify")
b.transform(reactify)

Hi.
I'm currently working on id-builder, so expect a new release soon. It will remove unneeded dependencies so the deps will install and run with io.js, swap out 6to5 for Babel (which has renamed) and after that I'm going to check out JSX support.
JSX Support currently works with Browserify I think, but if I'm going to use ES6 with Babel, I'm going to use Babel instead to compile the JSX, since it's already compiling the source files anyway. Leaves everything cleaner for Browserify.

How are you experiencing id-builder @hawkerboy7 ? Is it faster/nicer then id-project ?

It's a lot faster indeed @Industrial11 I do have some problems getting every functionality working though

id-builder now demands (when the option is turned on) all .js files to be ES6 and compiled with Babel.js so that should cover JSX. It's baked in. This also happens before the Browserify step meaning Browserify never has to worry about this, so that should be a speedup in watchify recompile times I hope.


Any plans to fix builds for linux?

Hello! First of all thank you for the parser! 😊 I'm wondering if there is any update on Linux builds? I saw there were errors in the GitHub actions for the build of version 0.17, and it seems unresolved. Do you have any understanding of what the problem might be? 🤔
6.049 ERROR: Could not find a version that satisfies the requirement demoparser2==0.17.0 (from versions: 0.15.2, 0.15.3, 0.16.0, 0.16.1)
6.049 ERROR: No matching distribution found for demoparser2==0.17.0

The build should now be fine. The original problem was that in 0.17.0 I added parsing of voice data, but I never managed to build that on linux so voice parsing is on hold for now.
0.16.0 Should be fine, the only missing feature should be the voice data.

0.18.0 works fine for linux


Support embedded resources that have no links

Add support for embedded resources that contain no "self" link or no "_links" property at all, as permitted by the HAL spec. Fixes issue #29.

Thanks for the patch! And um, thanks for waiting :+1:


Update jq command in channel config docs

I've found that quotes are needed around the jq target path.
Signed-off-by: David Enyeart<EMAIL_ADDRESS>

@Mergifyio backport release-2.5


Link on front-page is broken

The link http://noumlaut.com/graphviz-cookbook does not work.

Shouldn't that be http://heyrod.com/projects/gv-cookbook.html?

Closing due to lack of activity.


Add plugin to bower

Please, add your plugin to bower.

Bower can't manage github repos directly ?
/cc @ebrehault @fredericbonifas

Looks like you can use a github repo in your bower.json directly: http://stackoverflow.com/questions/20196707/can-i-add-a-git-repository-to-my-bower-json


Generate k8s client types for CRDs

Ambassador uses a number of custom CRDs. It would be desirable if generated Go client files could be made available for easy creation, listing and watching of Ambassador CRDs from Go programs and/or other operators.
Ideal solution would be to add client code generation with https://github.com/kubernetes/code-generator

+1

Great timing, since we're actually working on this. :wink:

As of Ambassador 1.6.0, you can look in pkg/api/getambassador.io/v2 to find Go types for all of our CRDs. 🙂


Export variable from Resource script lost when duplicating resource in exported build

This issue happened under the situation of the script is binding on a Resource(or subtype of Resource) and export the project with script compiled.

please make a small example project showing the problem, otherwise it's difficult to figure out from what you describe

@reduz I show you this problem.
script_problem.zip
Run it in the editor it say "Has script".

Then, export it with script compiled.

And run the exported application, it say "No script".

Still exist in 3.1
HasScript3.zip

Fixed the title, the issue is a bit more specific than just losing a bound script. What happens is that the export variable is lost when duplicating the resource in the exported build only.
Here's an updated project that demonstrates it (valid in current master):
HasScript3.zip
In editor:
res_data script: [GDScript:1062]
res_data has script_data: True
res_dup script: [GDScript:1071]
res_dup has script_data: True

Once exported:
res_data script: [GDScript:1029]
res_data has script_data: True
res_dup script: [GDScript:1038]
res_dup has script_data: Null

Still reproducible in current master branch (966c68ba).

Still valid in 3.2.3

I guess this should be fixed by #41925


[Bug] Only support subquery in binary predicate in case statement

Search before asking

[ ] I had searched in the issues and found no similar issues.

Version
master
What's Wrong?
sql below report error " Only support subquery in binary predicate in case statement"
select case when t1.a in(select 1 from t1 ) and t2.a in(select 1 from t1  )  then 'a1'  else 'a2' end
from  t1 cross join  t2
group by t1.a ,t2.a
What You Expected?
sql can be executed successufly
How to Reproduce?
No response
Anything Else?
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

非常感谢您的帮助。
At 2024-03-15 16:23:33, "zzwwhh" @.***> wrote:
CREATETABLEtest_varchar (
str varchar(100),
len  int
)
DUPLICATE KEY(str)
DISTRIBUTED BY HASH(str) BUCKETS 10
PROPERTIES("replication_num"="1");
insert into test_varchar select'abcdef',100;
insert into test_varchar select'test',256;
set enable_fallback_to_original_planner = false;
select  case when len in(select len from test_varchar  ) then 1 else 1 end c1  from test_varchar t2  group by len;
ERROR 1105 (HY000): errCode = 2, detailMessage = Only support subquery in binary predicate in case statement.
select/+SET_VAR(disable_nereids_rules='TRANSPOSE_LOGICAL_SEMI_JOIN_AGG_PROJECT, TRANSPOSE_LOGICAL_SEMI_JOIN_AGG')/  case when len in(select len from test_varchar  ) then 1 else 1 end c1  from test_varchar t2  group by len;
can get right result
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


fix(jest-config):  allow reporters in project config

Summary
jest currently warns that reporters as a unknown option, when configured at the project level config. But the project level configuration still applies the value.

Reporters is useful in project config for custom outputDirectory. Personally used with NX monorepo
#13576

Test plan
existing tests pass
Note: this is in reference #13576
Where options are validated as invalid for the project level potentially incorrectly. reporters Option throws warning about, but still works on the project level and appear to be used in most cases.

yarn build
yarn test

For Awareness from https://github.com/jestjs/jest/issues/13576
@mhornbacher @SimenB @tscislo @andrew-productiv @dmeents

https://github.com/jestjs/jest/releases/tag/v30.0.0-alpha.3


What's wrong? Add torrent with hash.

Hi guys!
I try to add a torrent using hash:
var torrentHash = "9FB3221627DC4D437A8FDB6829258EB5C83A2FE5";

 client.add(torrentHash, function(torrent){
      console.log(torrent)
});

But console.log don't execute nothing.
I see on the net the petition is pending but does nothing more.

help?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/11547528-what-s-wrong-add-torrent-with-hash?utm_campaign=plugin&utm_content=tracker%2F298985&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F298985&utm_medium=issues&utm_source=github).

The torrent must be explicitly seeded either using the hybrid client or online using HTTPS://instant.io. This issue has been raised multiple times and is discussed in the readme.

Well... My code:
client.add("magnet:?xt=urn:btih:390C83FC95B05A4259C0BF1DB95DBFCB4E08D85E",
		function(torrent){
		    // Got torrent metadata!
		    console.log('Torrent info hash:', torrent.infoHash)
		}
	);

But callback never execute.
With localStorage.debug = '*'
Print this output:


(Upgrade) get upstream from MM 4.9.3

get upstream from MM 4.9.3
@dennis00010011b please fix e2e and integration tests

@vbaranov
Token balance displayed only for account where it was added
Should be displayed for each account in network
Tested with localhost , Sokol 0x229483ce51cbe485c93496c5c19ebb422596acc8

@dennis00010011b e2e tests for Firefox look unstable. Could we improve them?

@vbaranov
There is issue for Firefox https://github.com/poanetwork/metamask-extension/issues/167
That's why e2e failed
I can skip this test for current PR

@dennis00010011b yes, let’s skip for this PR.


After installed qor-example product section not working in the admin backend

←[35m(Error 1146: Table 'qor_example.products' doesn't exist)←[0m
←[33m[2016-05-05 13:34:58]←[0m ←[31;1m ←[0m
[WARNING] Qor SEO: The parameter you passed is not a Struct
←[35m(Error 1146: Table 'qor_example.products' doesn't exist)←[0m
←[33m[2016-05-05 13:34:58]←[0m ←[31;1m ←[0m
[GIN] 2016/05/05 - 13:34:58 |←[97;42m 200 ←[0m|        62.5ms | ::1 |←[97;44m  ←
[0m GET     /
When i run main.go it shows these below errors
←[35m(Error 1146: Table 'qor_example.products' doesn't exist)←[0m
←[33m[2016-05-05 13:34:58]←[0m ←[31;1m ←[0m
[WARNING] Qor SEO: The parameter you passed is not a Struct
[GIN] 2016/05/05 - 13:34:58 |←[97;42m 200 ←[0m|     62.4987ms | ::1 |←[97;44m  ←
[0m GET     /
←[35m(Error 1146: Table 'qor_example.products' doesn't exist)←[0m
←[33m[2016-05-05 13:34:58]←[0m ←[31;1m ←[0m
Finish [GET] /admin/widgets/inline-edit Took 31.24ms

should been fixed, thank you for your report


Feature main cycle change

Pull Request Testing

[X] Describe testing already performed for these changes: hanged PROJECT ASSIGNMENT to CYCLE ASSIGNMENT for consistency.  Please note, the changes didn't kick off immediate "Action" runs.

[ ] Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

[ ] Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]

[ ] Do these changes include sufficient testing updates? [Yes or No]

[ ] Will this PR result in changes to the test suite? [Yes or No]
If yes, describe the new output and/or changes to the existing output:

[ ] Please complete this pull request review by [Fill in date].

Pull Request Checklist
See the METplus Workflow for details.

[ ] Review the source issue metadata (required labels, projects, and milestone).
[ ] Complete the PR definition above.
[x] Ensure the PR title matches the feature or bugfix branch name.
[x] Define the PR metadata, as permissions allow.
Select: Reviewer(s)
Select: Organization level software support Project or Repository level development cycle Project
Select: Milestone as the version that will include these changes
[ ] After submitting the PR, select Development issue with the original issue number.
[ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
[ ] Close the linked issue and delete your feature or bugfix branch from GitHub.

Ian is looking into the trivy failure, which has been going on for a while. You’re good to ignore it and merge this PR.

On May 3, 2023, at 10:41 AM, jprestop @.***> wrote:
@jprestop approved this pull request.
I have reviewed the changes and approve this pull request. Thanks for your work on this task @lisagoodrich https://github.com/lisagoodrich. @mollybsmith-noaa https://github.com/mollybsmith-noaa I see that the "Trivy Repo Scan / Analyze (pull_request)" is failing, but I don't know what that is or why that would be. Can you please confirm if it is ok to squash and merge this PR?
—
Reply to this email directly, view it on GitHub https://github.com/dtcenter/METexpress/pull/125#pullrequestreview-1411316495, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHBWGZPZQA3SLKIMXMBZL7TXEKDCDANCNFSM6AAAAAAXR5KWP4.
You are receiving this because you were mentioned.


perf: optimize syncing from L1

Optimize syncing from L1 by pipelining ckb/ckb-indexer requests and the actual syncing.
Generally, syncing should be a bit faster.
This is particularly helpful when ckb/ckb-indexer request-response time is bit high, cc #507.
(This also updates opentelemetry-jaeger and enables batch auto split to avoid message too large errors when exporting trace spans to jaeger.)

Can you update this PR?


WIP New Migration Logic

UPDATE: this has turned into a pretty comprehensive change to the logic and implementation behind migrations.
Description
With the migration refactor findMigrations() now returns migrations with their version as their index key. latestAll() currently uses array_merge() to create one final array from all namespace migrations, but array_merge() renumbers numeric (i.e. integer, even as a technical string) keys so that the target version fails to locate in checkMigrations().
This change uses the array union operator instead of array_merge() as it does no renumbering.
Checklist:

[X] Securely signed commits
[X] Component(s) with PHPdocs
[x] Unit testing, with >80% coverage
[x] User guide updated
[X] Conforms to style guide

@lonnieezell @jim-parry This might be a good one to expedite before people (like me!) start converting projects to the new migrations.

Good catch. Thanks!

Holy Redis test failure! I saw a recent Redis merge come through - perhaps related? Looks like migrations tests all passed though.

One more change to fix #2139 - migrations harvested across all namespaces need to be sorted before calling end() to determine the "latest".

Once again, all test failures were due to Redis->delete() - applicable tests succeeded.

Well bollocks - I didn't know how to do an upstream recursive merge on the website so I did it locally but forgot this computer wasn't signed. I'll see if I can fix it, but it truly is me. ^^

Okay, redone with GPG. @lonnieezell think this one is all set - still need to look at the migration logic but hopefully this takes care of the bugs.

I have a lot more coming - this has grown significantly, hoping to have the new version posted tonight.

Looking forward to it. Thanks so much for tackling this one!

@lonnieezell I still have to update the commands, tests, and docs but I changed the Runner so much I wanted to get it up for your review before I build around it. Changes started small and then kept growing and growing... See what you think, I'm open to anything.

This is real close but I'm hitting a snag. setPath() allows loading migrations from a source that otherwise is not detected by findMigrations(). This is convenient but it means that while regressing through batches the runner is not able to locate these migrations. Possible solutions:

Nix setPath(), find some way to namespace the test migrations so they are located
Save migration paths in the history, and have regress() check for the actual path instead of looking for it in the return of findMigrations()
Have addHistory() record an empty namespace when setPath() is used (currently indicates "App", which is usually wrong) and skip these when regressing batches.

I'm leaning towards 2, but would like some guidance. @lonnieezell ?

I'm fine with option 2.
I guess ideally option 1 would be better, but might be trickier to manage.

I tried to make option 1 work. Support migrations were moved to their own directory and namespace, "DatabaseTestMigrations/Database/Migrations", but now CIDatabaseTestCase isn't detecting them with findMigrations() so all the live DB tests are failing. I tried adding the namespaces manually to Config/MockServices but no bones.
I need to back-burner this project for a bit but feel free to pick it up @lonnieezell, I've allowed "edits from maintainers".

Looks like I'm not able to push my changes directly back into this repo. Good news is that I did get it figured out how to do it. I'm thinking MockServices isn't used anymore. Ended up using Services::autoloader()->addNamespace() in CIDatabaseTestCase for the database namespace. The other one I only loaded during the migration runner test.
I've got 3 tests that are failing currently, but I'll have to get to those tomorrow. Close!

That’s great news! I didn’t know that autoloader function existed, very handy.
I haven’t used the “allow edits from maintainers” before, but let me see if I can add you to my fork...

Okay I added you and @jim-parry to my fork in case you ever need to modify a PR.

@lonnieezell I should be available to work on this again today or tomorrow. Do you have a solution already? or progress you'd like to share? Feel free to push anything to the fork, or I can take it up from here with your addNamespace() hint.

@MGatner I just pushed all of my changes up in to the migration-union branch.  Can you pull that down and give it a spin. I think everything is working now.


JointPositionController not working for nested models

Environment

OS Version: macOS Monterey
Source or binary build? Source: gz-sim7, 31f7d5b384408eb88db3fcc85cc5529eb926f069

Description

Expected behavior: The JointPositionController should control joints of nested models.
Actual behavior: The nested joints do not respond to commands.

Steps to reproduce
test_nested_model.sdf.zip

Run the Gazebo using the example world:

gz sim -v4 -r test_nested_model.sdf

In a separate terminal issue commands to move the rotors:

# Move rotor in model1
gz topic -t "/rotor1_cmd" -m gz.msgs.Double -p "data: 1"

# Move rotor in model2 (nested)
gz topic -t "/rotor2_cmd" -m gz.msgs.Double -p "data: 1"

# Move rotor in model3
gz topic -t "/rotor3_cmd" -m gz.msgs.Double -p "data: 1"

Output
The single yellow block is model1, the teal block is model3 and the yellow block attached to the teal block is model2 which is nested in model3.  The rotors in the top level models (model1 and model2) both move. The rotor in the nested model (model2) does not respond.

When stepping through the joint controller plugin the JointPositionController::PreUpdate returns at L268 because jointPosComp->Data() is always empty for the nested joints.
https://github.com/gazebosim/gz-sim/blob/31f7d5b384408eb88db3fcc85cc5529eb926f069/src/systems/joint_position_controller/JointPositionController.cc#L267-L268
Server log:
gz sim -v4 -s -r test_nested_model.sdf
[Msg] Gazebo Sim Server v7.1.0
[Msg] Loading SDF world file[/Users/rhys/Code/ardupilot/simulation/ardupilot_gazebo/worlds/test_nested_model.sdf].
[Msg] Serving entity system service on [/entity/system/add]
[Dbg] [Physics.cc:869] Loaded [gz::physics::dartsim::Plugin] from library [/Volumes/MacPro2_DV1/Code/osrf/gz_garden_ws/install/lib/gz-physics-6/engine-plugins/libgz-physics-dartsim-plugin.dylib]
[Dbg] [SystemManager.cc:71] Loaded system [gz::sim::systems::Physics] for entity [1]
[Msg] Create service on [/world/test_nested_model/create]
[Msg] Remove service on [/world/test_nested_model/remove]
[Msg] Pose service on [/world/test_nested_model/set_pose]
[Msg] Pose service on [/world/test_nested_model/set_pose_vector]
[Msg] Light configuration service on [/world/test_nested_model/light_config]
[Msg] Physics service on [/world/test_nested_model/set_physics]
[Msg] SphericalCoordinates service on [/world/test_nested_model/set_spherical_coordinates]
[Msg] Enable collision service on [/world/test_nested_model/enable_collision]
[Msg] Disable collision service on [/world/test_nested_model/disable_collision]
[Msg] Material service on [/world/test_nested_model/visual_config]
[Msg] Material service on [/world/test_nested_model/wheel_slip]
[Dbg] [SystemManager.cc:71] Loaded system [gz::sim::systems::UserCommands] for entity [1]
[Dbg] [SystemManager.cc:71] Loaded system [gz::sim::systems::SceneBroadcaster] for entity [1]
[Dbg] [JointPositionController.cc:205] [JointPositionController] system parameters:
[Dbg] [JointPositionController.cc:206] p_gain: [2]
[Dbg] [JointPositionController.cc:207] i_gain: [0.1]
[Dbg] [JointPositionController.cc:208] d_gain: [0.01]
[Dbg] [JointPositionController.cc:209] i_max: [1]
[Dbg] [JointPositionController.cc:210] i_min: [-1]
[Dbg] [JointPositionController.cc:211] cmd_max: [1000]
[Dbg] [JointPositionController.cc:212] cmd_min: [-1000]
[Dbg] [JointPositionController.cc:213] cmd_offset: [0]
[Dbg] [JointPositionController.cc:214] Topic: [rotor1_cmd]
[Dbg] [JointPositionController.cc:215] initial_position: [0]
[Dbg] [SystemManager.cc:71] Loaded system [gz::sim::systems::JointPositionController] for entity [8]
[Dbg] [JointPositionController.cc:205] [JointPositionController] system parameters:
[Dbg] [JointPositionController.cc:206] p_gain: [2]
[Dbg] [JointPositionController.cc:207] i_gain: [0.1]
[Dbg] [JointPositionController.cc:208] d_gain: [0.01]
[Dbg] [JointPositionController.cc:209] i_max: [1]
[Dbg] [JointPositionController.cc:210] i_min: [-1]
[Dbg] [JointPositionController.cc:211] cmd_max: [1000]
[Dbg] [JointPositionController.cc:212] cmd_min: [-1000]
[Dbg] [JointPositionController.cc:213] cmd_offset: [0]
[Dbg] [JointPositionController.cc:214] Topic: [rotor3_cmd]
[Dbg] [JointPositionController.cc:215] initial_position: [0]
[Dbg] [SystemManager.cc:71] Loaded system [gz::sim::systems::JointPositionController] for entity [16]
[Dbg] [JointPositionController.cc:205] [JointPositionController] system parameters:
[Dbg] [JointPositionController.cc:206] p_gain: [2]
[Dbg] [JointPositionController.cc:207] i_gain: [0.1]
[Dbg] [JointPositionController.cc:208] d_gain: [0.01]
[Dbg] [JointPositionController.cc:209] i_max: [1]
[Dbg] [JointPositionController.cc:210] i_min: [-1]
[Dbg] [JointPositionController.cc:211] cmd_max: [1000]
[Dbg] [JointPositionController.cc:212] cmd_min: [-1000]
[Dbg] [JointPositionController.cc:213] cmd_offset: [0]
[Dbg] [JointPositionController.cc:214] Topic: [rotor2_cmd]
[Dbg] [JointPositionController.cc:215] initial_position: [0]
[Dbg] [SystemManager.cc:71] Loaded system [gz::sim::systems::JointPositionController] for entity [25]
[Msg] Loaded level [3]
[Msg] Serving world controls on [/world/test_nested_model/control], [/world/test_nested_model/control/state] and [/world/test_nested_model/playback/control]
[Msg] Serving GUI information on [/world/test_nested_model/gui/info]
[Msg] World [test_nested_model] initialized with [1ms] physics profile.
[Msg] Serving world SDF generation service on [/world/test_nested_model/generate_world_sdf]
[Msg] Serving world names on [/gazebo/worlds]
[Msg] Resource path add service on [/gazebo/resource_paths/add].
[Msg] Resource path get service on [/gazebo/resource_paths/get].
[Msg] Resource path resolve service on [/gazebo/resource_paths/resolve].
[Msg] Resource paths published on [/gazebo/resource_paths].
[Msg] Server control service on [/server_control].
[Msg] Found no publishers on /stats, adding root stats topic
[Msg] Found no publishers on /clock, adding root clock topic
[Dbg] [SimulationRunner.cc:508] Creating PostUpdate worker threads: 2
[Dbg] [SimulationRunner.cc:519] Creating postupdate worker thread (0)
[Err] [SDFFeatures.cc:1077] Asked to create a joint between links [base2_link] as parent and [rotor2_link] as child, but the child link already has a parent joint of type [RevoluteJoint].
[Msg] Serving scene information on [/world/test_nested_model/scene/info]
[Msg] Serving graph information on [/world/test_nested_model/scene/graph]
[Msg] Serving full state on [/world/test_nested_model/state]
[Msg] Serving full state (async) on [/world/test_nested_model/state_async]
[Msg] Publishing scene information on [/world/test_nested_model/scene/info]
[Msg] Publishing entity deletions on [/world/test_nested_model/scene/deletion]
[Msg] Publishing state changes on [/world/test_nested_model/state]
[Msg] Publishing pose messages on [/world/test_nested_model/pose/info]
[Msg] Publishing dynamic pose messages on [/world/test_nested_model/dynamic_pose/info]
[Dbg] [EntityComponentManager.cc:1619] Updated state thread iterators: 32 threads processing around 2 entities each.

The error:
[Err] [SDFFeatures.cc:1077] Asked to create a joint between links [base2_link] as parent and [rotor2_link] as child, but the child link already has a parent joint of type [RevoluteJoint].

Suggests that the physics engine may be attempting to build the nested model twice?
Related
The same issue is observed when composing models using the <include> element.

Investigations
Work in progress trying to tack down the origin of the issue. Notes below.

The issue, as far as the gz-sim physics plugin is concerned, appears to be that the nested joints are not in the Physics plugin's entityJointMap.
The cause of this occurs at a lower level in the dart physics engine, somewhere in gz::physics::dartsim::EntityManagementFeatures where the nested joint is not found in the call to GetJoint. Possibly because this does not recurse down to nested models or for some other reason?

Notes

gz::sim::systems:: PhysicsPrivate:: CreateModelEntities

There are two places where nested models are constructed. The first for models that are direct children of the world:
https://github.com/gazebosim/gz-sim/blob/31f7d5b384408eb88db3fcc85cc5529eb926f069/src/systems/physics/Physics.cc#L1123-L1124
and the second for models that are not (and so must be nested):
https://github.com/gazebosim/gz-sim/blob/31f7d5b384408eb88db3fcc85cc5529eb926f069/src/systems/physics/Physics.cc#L1175
The first call will construct models recursively, so that when the second call is made the nested model is already constructed.

gz::sim::systems::PhysicsPrivate::CreateJointEntities

For the non-nested joints, the function CreateJointEntities hits the test if they already exist, and proceeds to insert them into the entityJointMap:
https://github.com/gazebosim/gz-sim/blob/31f7d5b384408eb88db3fcc85cc5529eb926f069/src/systems/physics/Physics.cc#L1680-L1689
For the nested joints this test fails (which is odd because we see below there is an error message reporting the joint already exists). In any case the function continues until it hits:
https://github.com/gazebosim/gz-sim/blob/31f7d5b384408eb88db3fcc85cc5529eb926f069/src/systems/physics/Physics.cc#L1723-L1734
where the test for a valid physics pointer to a joint fails and the the inserting of the joint into the entityJointMap is skipped. To see why the attempt to create the joint failed we need to look into ConstructSdfJoint.

gz::physics::dartsim::SDFFeatures::ConstructSdfJoint

The attempt to create the nested joint fails here:
gz-physics6/dartsim/src/SDFFeatures.cc#L1074-L1083
    auto childsParentJoint = _child->getParentJoint();
    if (childsParentJoint->getType() != "FreeJoint")
    {
      gzerr << "Asked to create a joint between links "
             << "[" << _parent->getName() << "] as parent and ["
             << _child->getName() << "] as child, but the child link already "
             << "has a parent joint of type [" << childsParentJoint->getType()
             << "].\n";
      return this->GenerateInvalidId();
    }

because the joint has already been associated with a link, but for some reason is not recorded in the dart physics plugin as an existing link when queried using the entity management feature GetJointFromModel. So that's where to look next.

Thanks for sleuthing @srmainwaring. I'm wondering if this is a regression from #1560 or something that existed all along.

I'm wondering if this is a regression from #1560 or something that existed all along

I think it was around before then. I rolled back gz-sim to the commit before #1560 and the issue remained.
I'm slowly zeroing in on possible causes - I suspect it may either be a recursion thing - where the dart model only reports its immediate child joints, or there is a double entry of the nested joint into the Base::AddJoint and one of them is not connected up to anything. It's a little daunting navigating through the type lists but I'm getting there...

@azeey - the issue arises in the function ConstructSingleAxisJoint in SDFFeatures  where the joints from the nested model are reassigned to the container model. In the notes above, I've including a link to a fork containing some debug code for Base.hh to print the contents of the EntityStore as these are difficult to inspect in a debugger directly.
It's not clear to me what the correct fix is - any change is going to have knock on effects elsewhere. Options seem to be:

Don't reassign joints to the parent entity for nested models.
Continue to reassign them and update the search functions to resolve joints for nested models as though they were not reassigned.

I don't think Option 2 is a good choice - it would be very confusing and almost impossible to maintain (tracing the model state is quite difficult as it is). That leaves option 1, but I'm not familiar with DART to know how the object model will cope with this type of nesting.

I've started a fix for this. The branches will probably change as I tidy things up and remove debug code but currently the changes are here:

https://github.com/srmainwaring/gz-physics/tree/srmainwaring/6/1844-joint-pos-nested-models
https://github.com/srmainwaring/gz-sim/tree/srmainwaring/7/joint-pos-controller

The main changes are broken down as follows:
gz-physics

[ ] Update EntityManagementFeatures::GetJoint to search recursively up to the top-level containing model for a named joint (or joint by index), as the joints are moved to the container body nodes as the skeleton is assembled. gz-physics: b9b3ffd

gz-sim

[ ]  Update the JointPositionController to search using both scoped and unscoped joint names.  gz-sim: 61bca3d

Outstanding issues

[ ] ECM Entity Id is not correctly mapped to Physics Id when the container and nested model contain joints with the same unscoped name.
[ ] Joints between two nested models are not resolved correctly.

@azeey this should still be open as it requires https://github.com/gazebosim/gz-sim/pull/1851 to be merged as well.


Grammar and end links in article Add a pie chart layer

Per prior arrangement, if this PR needs approval, or if the commits need to be pulled into the private repo, only @eriklindeman should be notified. Thanks.

@deniseatmicrosoft

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


feat: use gh repo set-default if set for GitHub

When working in a fork of a GitHub repo, depending on how the remotes are configured locally, completion of issues and pull requests may not work because the fork is being used for the  lookup.
This patch checks the value of gh repo set-default --view and uses that instead of the git info discovered in the cmp_git.utils.git_git_info() function. If the default repo has not previously been configured, the command to view it will fail no change is made to the git info.

Caveats, I don't use octo.nvim so have only tested this outside of that context.

LGTM. Thank you for the PR


Feature | leverage debug method

What?

It should be a simple method callable from the leverage cli to open a shell using the busybox (or similar) docker image but keeping the docker parameters used for the task that needs to be debugged.
That functionality needs to be cross-repo (terraform, ansible and helsman)
e.g. if the problem happens when running leverage plan, the debug functionality should simulate that execution but using shell as the docker image entrypoint.

Why?

The idea is to simply testing and development being able to enter the container and throw debugging commands like:

netstat ,
dig ,
nslookup,
nc/netcat,
curl,
ip,
and some others.

As agreed with @diego-ojeda-binbash and @angelofenoglio we'll not move forward with this functionality since a leverage user could simply use the leverage terraform shell built in command and then run for example $ apk add curl inside the container shell to quickly and easily get any debugging tool he/she will need for the specific scenario.
Ref:

https://leverage.binbash.com.ar/user-guide/leverage-cli/reference/terraform/#shell
https://leverage.binbash.com.ar/user-guide/leverage-cli/shell/


Zarr Support

Zarr Support
Adds basic Read Support for the v 2 Zarr file format. Some features have been omitted for now (groups, filters, compression and some of the "non cpp native" data types, most of which thus also don't have a respective daphne supported type.
This Pr simply uses the stl for i/o, but once #736 has been merged, another Pr will follow that adds an async zarr reader.
This Pr also fixes a couple bugs in the rechunk and dice variants and cleans a couple of other things up in the tensor implementations.
As mentioned in #694 and like #736 this Pr also again switches to c++20, here mostly for std::endian, but also uses it a lot elsewhere for convenience e.g. using enum, make_shared<T[]> :) etc..
One possible discussion point
This pr includes 2 zarr "files" for testing, which total ~8MB and ~1k files. I don't particularly like dumping a bunch of binary files into a repo. There are already a couple of "io test files" there, but these would be ~ as large as the rest combined and it is just a lot of files. Should we avoid this or is this fine with everybody? The alternative would be to add a writer implementation or a dependency to generate some tmp zarr files on the fly.

Seems like the compiler on the CI machine again does not like the c++20 features....

Jun 03 15:33:30 /__w/daphne/daphne/src/parser/metadata/ZarrFileMetaDataParser.cpp:85:15: error: expected nested-name-specifier before 'enum'
Jun 03 15:33:30    85 |         using enum ZarrDatatype;

@pdamme Thx for having a look and fixing my numerous typos :)

ZarrFileMetaData.cpp, operator<< for ZarrDatatype prints "double" for ZarrDatatype::FP32. Is that intended or should it be "float" or "single"?

y, thats a mistake.

ReadZarrTest.cpp: In many of the test cases you memcmp the data in the reference tensor and the read tensor and compare the tensors through ==. Isn't the memcmp redundant as the object comparison includes comparing the data?

The == tests more than just the actual memory of the elements, but yes that memcmp is redundant in a couple of the tests, as well as some other checks also done by the == call. I removed the duplicates.

When I build your code with g++-13.1.0 on Ubuntu 20.04.6, there are still several warnings, most of them from the MLIR code and related to operator==. I assume these warnings are not related to your changes, but to the switch to C++20, so we will probably handle them in the commit that switches to C++20. But before the merge, we need to double-check that they are gone from this PR.

I just tested it with g++-13.2.0 on 24.04 and y it indeed spits out some new warnings, but as far as i can tell these come from mlir and llvm code, which i think are caused by a "missing" const on a operator==(), as well as one warning from antlr and spdlog each. Not sure what you want to do about these e.g. update their version or silence them.
Btw on Ubuntu 24.04 building protobuf fails due to missing libnsl. I guess we should add libnsl-dev to the dependency list for ubuntu 24.04.

Regarding the two Zarr "files" for the test cases: We should at least make a conscious decision here. It would be great if we could avoid putting so many little binary files that sum up to a few MBs into the repository. As far as I understand, the reason why there are so many chunk files is that you use a tensor with 10x10x10 = 1000 chunks (with 10x10x10 = 1000 fp64 elements = 8 KB each). Couldn't we simply decrease the data size by having less and/or smaller chunks, e.g., by 2x2x2 = 8 chunks of 10x10x10 elements each? Shouldn't that suffice to test the Zarr reader?

Y, that sounds reasonable.

my 2 cents:

We should find an alternative to having 1000+ files added in test/
Yes, most of the warnings come from LLVM/MLIR. Now that Ubuntu24 has landed in DAPHNE Docker Land we'll have to deal with it somehow :face_with_spiral_eyes:
Would be nice to have this hooked up to the generic read kernel (e.g., "if (zarr file) -> readZarr()")

my 2 cents:

We should find an alternative to having 1000+ files added in test/
Yes, most of the warnings come from LLVM/MLIR. Now that Ubuntu24 has landed in DAPHNE Docker Land we'll have to deal with it somehow 😵💫
Would be nice to have this hooked up to the generic read kernel (e.g., "if (zarr file) -> readZarr()")

We could just use a little python helper to automatically generate the test files on-the-fly.
Regarding the overall integration. we already have a "running" prototype that does the wiring, but requires some more polishing before opening a PR.


chore: Handle no neuroglancer config file

Description
Sets the neuroglancer config to None for cases where there is no neuroglancer_config path specified in the tomogram metadata.

We'll want the equivalent change for the v2 db ingestion as well!


pari: 2.13.4 -> 2.15.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/pari/versions.
meta.description for pari is: "Computer algebra system for high-performance number theory computations"
meta.homepage for pari is: "http://pari.math.u-bordeaux.fr"
meta.changelog for pari is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2.15.0 with grep in /nix/store/rq9i7yn5gbww2a5jd9c8ns3xhja4cq8k-pari-2.15.0
found 2.15.0 in filename of file in /nix/store/rq9i7yn5gbww2a5jd9c8ns3xhja4cq8k-pari-2.15.0

Rebuild report (if merged into master) (click to expand)

21 total rebuild path(s)

21 package rebuild(s)

First fifty rebuilds by attrpath
eclib
giac
giac-with-xcas
gp2c
lcalc
pari
python310Packages.cypari2
python310Packages.cysignals
python310Packages.fpylll
python310Packages.paranoid-crypto
python310Packages.pplpy
python310Packages.primecountpy
python39Packages.cypari2
python39Packages.cysignals
python39Packages.fpylll
python39Packages.paranoid-crypto
python39Packages.pplpy
python39Packages.primecountpy
sage
sageWithDoc
sympow

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/rq9i7yn5gbww2a5jd9c8ns3xhja4cq8k-pari-2.15.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A pari https://github.com/r-ryantm/nixpkgs/archive/1479b7ae6494f1fa29799ebc1c2f82110765d701.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/rq9i7yn5gbww2a5jd9c8ns3xhja4cq8k-pari-2.15.0
ls -la /nix/store/rq9i7yn5gbww2a5jd9c8ns3xhja4cq8k-pari-2.15.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

6 packages failed to build:

giac
giac-with-xcas
python310Packages.cypari2
python39Packages.cypari2
sage
sageWithDoc

15 packages built:

eclib
gp2c
lcalc
pari
python310Packages.cysignals
python310Packages.fpylll
python310Packages.paranoid-crypto
python310Packages.pplpy
python310Packages.primecountpy
python39Packages.cysignals
python39Packages.fpylll
python39Packages.paranoid-crypto
python39Packages.pplpy
python39Packages.primecountpy
sympow

Maintainer pings
cc @ertes @AndersonTorres @timokau @omasanori @7c6f434c @collares for testing.

Not yet: https://trac.sagemath.org/ticket/34537

@ofborg build pari sageWithDoc

Sage intermittent test failure is known: https://trac.sagemath.org/ticket/32773

@jiegec PARI 2.15.1 seems to build fine on aarch64-darwin, even after removing the legacy fixes. I'd appreciate if you could check that basic functionality also works. Thanks!

@jiegec PARI 2.15.1 seems to build fine on aarch64-darwin, even after removing the legacy fixes. I'd appreciate if you could check that basic functionality also works. Thanks!

PARI 2.15.1 works on aarch64-darwin, thanks for you hard work.

@ofborg build giac

@ofborg build giac

giac's testsuite doesn't work on aarch64-darwin, so I marked it as broken on this platform. This was previously masked by the fact that PARI (a giac dependency) was broken too, but giac failures on aarch64-darwin are not a regression and aren't Nix-specific: https://trac.sagemath.org/ticket/34537#comment:65.
cc @symphorien

Filed random Sage test failure as https://trac.sagemath.org/ticket/34803. Will merge later today if no one objects.

thank you for the giac related head-up


How to train in lower bitrates ( < 0.14)?

What is needed to change in the code to train with a different bitrate?

Assuming you are working in the low regime, which is the default, change the value for low in the target_rate_map dict in default_config.py to whatever bitrate you want.


ARROW-14192: [C++][Dataset] Backpressure broken on ordered scans

While scanning we do our best to readahead multiple files so we will read files 1, 2, 3, and 4 all at the same time.  This helps to maintain bandwidth when some files hit a snag (sometimes happens on AWS).  However, when doing an ordered scan, this can cause backpressure to explode when there is a slow consumer.
The sequencer (placed at the end of the pipeline) can get into a situation where it pulls aggressively from files 2, 3, and 4 while waiting for the next chunk from file 1.  Since the sequencer is consuming the batches the backpressure mechanism thinks they are being consumed.  However, the actual consumer is leaving the batches piling up at the sequencer.
This PR introduces one possible solution (and it may be the only possible solution) which is to sequence the batches at merge time (early in the pipeline).  The sequencer won't need to pull aggressively and backpressure will be maintained.  This pretty significantly reduces (but does not eliminate) the amount of file readahead we do in ordered scans.  We can worry about that if it ends up being a bottleneck at some point but for now I think it is better we do not explode RAM.
This builds on ARROW-13611 and will remain in draft until that PR has merged.

Rebased and addressed PR comments.

@westonpace It seems that "C++ / AMD64 Ubuntu 20.04 C++ ASAN UBSAN" is failed by this.


Threshold Signatures

Hi,
I've been digging into the code for the past week but I couldn't find if you're actually using the threshold signature scheme.
I couldn't find the distribution of the partial signatures and the methods to compute it is actually empty:
void compute() override {}
Am I missing something?

"Partial" signatures are distributed via votes.
Please check the paper -- at the time of writing the paper, there wasn't a fast and reliable threshold signature library available.
So we "crippled" our evaluation to use a set of secp256k1 signatures without actually saving the extra O(n) cost. While it is still correct, it should be improved by offering a version that supports something like BLS. (See the templates were left for this purpose)
The scheme of signature is almost a drop-in replacement to HotStuff, as the protocol treats is as a black box.
It's encouraged to contribute to this repo by adding the additional support of using some practical threshold signature library.

Oh sorry, I didn't mean partial signatures but partial keys instead. But your answer includes that too.
So you do distribute the complete set of votes instead of actually sending a single signature and you also verify the complete set of signatures on reception at each replica too for this purpose.

That's correct if you use HotStuffSecp256k1, which is the only one available for now. There should be more options here https://github.com/hot-stuff/libhotstuff/blob/master/include/hotstuff/hotstuff.h#L310

I am working on preparing code to integrate BLS signatures. However I'm a bit confused about the difference between "PartCert" and "QuorumCert" and SigSecp256k1.
I believe SigSecp is a single signature and QuorumCert is the complete list of signatures that is necessary (at least k out of n).
However, what is PartCert?
I couldn't find either where PartCert would store any signatures.

While I am on it.
For this I'll need two more things:
a) I need a place to generate all signatures centrally and distribute them including the master public key.
b) I need a place to store the master public key.
Where would the best place for this be?
void HotStuffBase::start to generate and ReplicaConfig to store a pointer to it?

In general it seems everything is very tied in with the existing signature scheme. It's loaded from file, it needs the certificate etc.
As far as I can see it is not very "blackbox" atm at all.

More comments.
/*** end HotStuff protocol logic ***/
void HotStuffCore::on_init(uint32_t nfaulty) {

    config.nmajority = config.nreplicas - nfaulty;
    b0->qc = create_quorum_cert(b0->get_hash());
    //b0->qc->compute();
    b0->self_qc = b0->qc->clone();
    b0->qc_ref = b0;
    hqc = std::make_pair(b0, b0->qc->clone());
}

Why is there a compute on the quorum cert if we don't even have a quorum at that point yet?


⚠ Bofrost.de has degraded performance

In e94eedc, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1005 ms

Resolved: Bofrost.de performance has improved in 0a7ec11 after 6 minutes.