feat(evm)!: updated imports to use ethermint fork

removed claims
Description

Closes #XXX
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow-up issues.
PR review checkboxes:
I have...

[ ]  Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ]  Targeted PR against the correct branch
[ ]  included the correct type prefix in the PR title
[ ]  Linked to the GitHub issue with discussion and accepted design
[ ]  Targets only one GitHub issue
[ ]  Wrote unit and integration tests
[ ]  Wrote relevant migration scripts if necessary
[ ]  All CI checks have passed
[ ]  Added relevant godoc comments
[ ]  Updated the scripts for local run, e.g genesis_config_commands.sh if the PR changes parameters
[ ]  Add an issue in the e2e-tests repo if necessary

SDK Checklist

[ ] Import/Export Genesis
[ ] Registered Invariants
[ ] Registered Events
[ ] Updated openapi.yaml
[ ] No usage of go map
[ ] No usage of time.Now()
[ ] Used fixed point arithmetic and not float arithmetic
[ ] Avoid panicking in Begin/End block as much as possible
[ ] No unexpected math Overflow
[ ] Used sendCoin and not SendCoins
[ ] Out-of-block compute is bounded
[ ] No serialized ID at the end of store keys
[ ] UInt to byte conversion should use BigEndian

Full security checklist here
----;
For Reviewer:

[ ]  Confirmed the correct type prefix in the PR title
[ ]  Reviewers assigned
[ ]  Confirmed all author checklist items have been addressed

---;
After reviewer approval:

[ ]  In case the PR targets the main branch, PR should not be squash merge in order to keep meaningful git history.
[ ]  In case the PR targets a release branch, PR must be rebased.

@mtsitrin we probably need to add the ante handler AuthZ protection same like in the hub

@trinitys7 I think the e2e tests fails on
https://github.com/dymensionxyz/e2e-tests/blob/e32a4393d5962a06c9d04a29bda177b9ab0d3ef1/tests/setup.go#L133-L134

@mtsitrin I will create a PR to fix them.

@mtsitrin once we merge the ethermint pr will need to add the messages to the ante handler


🎉 Add siunitx package to latex, JATS and MyST

This implements the majority of the siunitx package in latex:
https://texdoc.org/serve/siunitx/0
This also creates alt-text and keeps other information about numbers and units in the mdast node.

I think you're right that the angle-bracket syntax is the most MyST-like. In my view, sometimes the problem is just verbose, rather than the markup. I think the recent MEP for links is a useful one; links are already implemented in Markdown, and a notion of a scheme applies well to our use case. However, in this case, we don't have any prior art to improve upon, so going with something ReST-like is probably the right call?
With an si-unit parser, the verbose {si}`5 <\kilo\gram\metre\per\square\second>` becomes {si}`5 <kg.m/s^2>`, which is very readable.

Love it. Ok, I will go on this path and let you know where I end up!! (Will also open an issue to track this!)


Update README.md

#9 Add license link to README.md

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Nice work @204070 👍  Just a small suggestion. Next time make sure you are giving a nice title to the pull request, describing the changes you did. In this case, something like added license badge to readme. This will help you and other developers to get a better understanding of the contents of the pull request.


Missing app

One of my apps with package name com.universe.messenger isn't showing up in app menu or search.

What is the app's name? I haven't encountered this happening for any app myself and would like to see whether I can reproduce the issue.
It is also possible that it simply shows a different name in this launcher than it did in your other one, so it might be good to check for that.

That is a modified version of whatsapp.
I tried opening all the apps, but none of them was it, the app itself is fully visible and accessible from the settings, but in the launcher it's not in the list or the search.
Here is the link
https://www.deltalabsstudio.com/2024/12/updatedelta-wa-v662.html?m=1
The app is safe as far as I know, never encountered any issues in 4 years of using it.

Forgot to add, it was showing just fine before the update.

Forgot to add, it was showing just fine before the update.

This explains it. The update changes how apps are found. This was done to support apps with multiple home screen icons for different things. I'm guessing the modified WhatsApp doesn't properly define its launch intents. I can experiment with creating a backup option where the app is shown and used with the old method  if the new method doesn't find it.

I'm not sure how or why, but reinstalling the launcher and then restarting the phone resolved the issue.

Okay, that's really weird. I wonder if it's just something with Android or some issue with updating in that case.

I guess we'll never know. 😁


[MM-28150] e2e: add MM-T924, MM-T928, MM-T929, MM-T930

Summary
Adds automated test cases for the first half of the ticket:

MM-T924
MM-T928
MM-T929
MM-T930

PS: It got quite big already, I thought it would be better to split into two PRs.
Ticket Link
https://mattermost.atlassian.net/browse/MM-28150

/update-branch

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.


Add Proper Docker Run Command

-t opens a TTY
-i keeps STDIN open even if not connected
Both are required to run via the command line.

Thanks for your contribution


[api breaking] Rename .options to .given.

This would make us<EMAIL_ADDRESS>Renames .options to .given to avoid conflicting with things mixed into a broadway.App instance.

+1 Thanks for fixing #60, @indexzero. I know it wasn't a big deal or priority, but it confused me when attempting setup the CORS for a specific route. I know that it would have bitten someone else along the way.


[Logbook] - Subgraph integration

Since I have already added a sample on https://github.com/thematters/subgraphs#usages, I think we can close this issue.
cc @devformatters


Common Language Runtime detected an invalid program

Hi,
I have a problem with the code below, which throws a "Common Language Runtime detected an invalid program" error, when ever I compile the code.
The interesting thing is when I remove the "Clicked="Btn_Clicked"" from the button element, everything works perfect.
Any ideas why this would be throwing this error?
What I am trying to do is to have some informational information displayed that the user can swipe between. Also I want to have a next and back button that will move the carousel between the different views.
Any help would be greatly appreciated.
`

<ContentPage.Content>
<cv:CarouselViewControl >
cv:CarouselViewControl.ItemsSource
<x:Array Type="{x:Type Grid}">
                <Grid>
                    <Label>test</Label>
                </Grid>

                <Grid>
                    <Button Text="Test Button" Clicked="Btn_Clicked"></Button>
                </Grid>

            </x:Array>
        </cv:CarouselViewControl.ItemsSource>
    </cv:CarouselViewControl>
</ContentPage.Content>

`

I'm also having this issue.
It's because there's an eventbinding to a button inside the carousel.
When you would remove the eventbinding, your app would actually build.
eg change:
<Button Text="Test Button" Clicked="Btn_Clicked"></Button>
to
<Button Text="Test Button"></Button>
and the problem disappears.
Off course this gets me nowhere, since there's no other way to bind the event.
I also tried the event binding through c#, but somehow the button is just NULL.
I'll try to make an issue with demo, and let's hope the problem will be solved by the developers of this package.

I've uploaded an example to my webserver.
This app is not building. Remove the Clicked event handler and the app will build.
It would be greatly appreciated if this issue gets solved.
Cheers,
Pieterjan

I am not able to find out why this happens. It should be an internal Xamarin.Forms issue.
As a workaround, try using a Command binding:
<Button Text="Click me!" Command="{Binding OnClicked}"/>


UPX unpacker: "Invalid block found" while unpacking UPXed ELF file

Hi,
I'm trying to unpack x64 ELF file with SHA256 hash f723f0a1e6910023b3f9a3bd2f0ae35f751a00c86c08819d2dc7b33b5bb2bc7f by using the retdec-unpacker, but I only get:
[UPX] Detected NRV2E unpacking stub based on signature & metadata.
[UPX] Started unpacking of file 'SAMPLE'.
[UPX] Unfiltering filter 0x0 with parameter 0.
[UPX] Unpacking block at file offset 0x1c4.
[UPX] Unfiltering filter 0x49 with parameter 67.
[UPX] Unpacking block at file offset 0x9a85f.
[ERROR] [UPX] Invalid block found.

Could you, please modify the unpacked if it is unpackable?
IMHO @thinkcz was able to unpack it by using the latest UPX for Linux, but I'm getting compressed data violation with upx 3.95.
Thank you!

IMHO you should not wast time on PE packers in this great project. This is just adding fat.
The same with handling malformed PE files you will lose too much time and gain very little.

The same with handling malformed PE files you will lose too much time and gain very little.

This is off-topic, but please know that we need to be able to handle both valid and malformed PE files as we have our use cases for that. At the very least, retdec-fileinfo should not crash. Preferably though, retdec-fileinfo should detect that the input file is malformed.


EXSWHTEC-350 - Implement tests for hipModuleLaunchCooperativeKernel

Negative parameters tests and positive tests for hipModuleLaunchCooperativeKernel and hipModuleLaunchCooperativeKernelMultiDevice.

Depends-on: #22

PR has been merged into develop branch via commit 303836a7fda08afe8e57f01212dd8f5afc12002b.


refactor: linter warnings eliminated

See the containing epic for more context and the scope of packages that are being affected: https://github.com/hyperledger/cactus/issues/1497

Eliminate the need for the following ESLint ignore rules:
packages/cactus-verifier/src/main/typescript/

packages/cactus-cmd-socketio-server/**
packages/cactus-plugin-ledger-connector-sawtooth-socketio/**
packages/cactus-plugin-ledger-connector-go-ethereum-socketio/**
packages/cactus-plugin-ledger-connector-*-socketio/**

For reference/guidance on how to do it: The work for this was already done in #719 but then got deleted.

Plugins refactored


Pre-compiled extension does not load on Windows 11 (The specified module could not be found)

Same issue as https://github.com/asg017/sqlite-vec/issues/13
I'm on Windows 11 64 bit, Python 3.12, SQLite 3.45 and sqlite-vec = "^0.0.1a36"
I wrote this function to make the result more explicit:
def load_sqlite_vec(conn: sqlite3.Connection) -> sqlite3.Connection:
    sqlite_version = conn.execute("select sqlite_version()").fetchone()
    print(f"SQLite version: {sqlite_version}")
    import sys

    print(f"Python version: {sys.version}")
    # Load the SQLite vector extension
    conn.enable_load_extension(True)
    # sqlite_vec.load(conn)
    sqlite_vec_path = sqlite_vec.loadable_path()
    print(f"Loading SQLite vector extension from {sqlite_vec_path}")
    if os.path.exists(sqlite_vec_path + ".dll"):
        try:
            conn.load_extension(sqlite_vec_path)
        except sqlite3.OperationalError as e:
            print(f"Error loading extension: {e}")
        try:
            conn.load_extension(sqlite_vec_path + ".dll")
        except sqlite3.OperationalError as e:
            print(f"Error loading extension: {e}")
    else:
        print(f"Could not find SQLite vector extension at {sqlite_vec_path}")
    conn.enable_load_extension(False)
    return conn

Outputs:
SQLite version: ('3.45.3',)
Python version: 3.12.4 (tags/v3.12.4:8e8a4ba, Jun  6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]
Loading SQLite vector extension from .venv\Lib\site-packages\sqlite_vec\vec0
Error loading extension: The specified module could not be found.

Error loading extension: The specified module could not be found.

The file vec0.dll is present, as is evident from os.path.exists succeeding.

Thank you for the detailed report!
So looks like you're trying the windows version of the sqlite-vec pip package, which is compiled with mingw.
Could you try this specific vec0.dll? This one was compiled with msvc, but I have had trouble running it https://github.com/asg017/sqlite-vec/releases/download/v0.0.1-alpha.37/vec0.dll
import sqlite3
db = sqlite3.connect(":memory:")
db.enable_load_extension(True)
db.load_extension("./vec0.dll") #  make sure it's from the link above!

Also, just curious - how did you install your version of Python? On my machine the mingw one works fine, but I dont think im on a MSC python build

I tested this file and it seems to produce the same error.
def load_sqlite_vec(conn: sqlite3.Connection) -> sqlite3.Connection:
    sqlite_version = conn.execute("select sqlite_version()").fetchone()
    print(f"SQLite version: {sqlite_version}")
    import sys

    print(f"Python version: {sys.version}")
    # Load the SQLite vector extension
    conn.enable_load_extension(True)
    # sqlite_vec.load(conn)
    # sqlite_vec_path = sqlite_vec.loadable_path()
    sqlite_vec_path = "./vec0"
    print(f"Loading SQLite vector extension from {sqlite_vec_path}")
    if os.path.exists(sqlite_vec_path + ".dll"):
        md5, sha256 = calculate_hashes(sqlite_vec_path + ".dll")
        print(f"MD5: {md5}, SHA-256: {sha256}")
        try:
            conn.load_extension(sqlite_vec_path)
        except sqlite3.OperationalError as e:
            print(f"Error loading extension: {e}")
        try:
            conn.load_extension(sqlite_vec_path + ".dll")
        except sqlite3.OperationalError as e:
            print(f"Error loading extension: {e}")
    else:
        print(f"Could not find SQLite vector extension at {sqlite_vec_path}")
    conn.enable_load_extension(False)
    return conn

I updated the function to use the local vec0(.dll) and to print the sha256 and md5 hash so you can verify it is the same file.
SQLite version: ('3.45.3',)
Python version: 3.12.4 (tags/v3.12.4:8e8a4ba, Jun  6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]
Loading SQLite vector extension from ./vec0
MD5: 382449844d4b9cbb1b7e0d017945db62, SHA-256: d179b834501aa8af22cc866cbd2e838093a594a4bf06260009169ee2cf0fc986
Error loading extension: The specified module could not be found.

Error loading extension: The specified module could not be found.

As you can see it still fails to find the module, whether I import ./vec0.dll or ./vec0

I forgot to say, I installed it simply using poetry add sqlite-vec

So, I tried something else: since you said the version in the PyPI package was for mingw I tried using that dll with mingw python.
C:\msys64\mingw64\bin\python.exe
Python 3.11.9 (main, Apr 12 2024, 09:55:31)  [GCC 13.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> conn = sqlite3.connect(":memory:")
>>> conn.enable_load_extension(True)
>>> conn.load_extension("./vec0")
>>> conn.enable_load_extension(False)
>>> print(conn.execute("select vec_version()").fetchone())
('v0.0.1-alpha.36',)

This worked. However I'd like to use it with windows' python, since that's what my project uses. Any ideas? Should I try to compile it myself using MSVC? Are there any compilation instructions for this?

Another update:
I tried mingw python with the MSVC compiled DLL you provided:

Could you try this specific vec0.dll? This one was compiled with msvc, but I have had trouble running it https://github.com/asg017/sqlite-vec/releases/download/v0.0.1-alpha.37/vec0.dll

And it also works:
C:\msys64\mingw64\bin\python.exe .\main.py
SQLite version: ('3.45.2',)
Python version: 3.11.9 (main, Apr 12 2024, 09:55:31)  [GCC 13.2.0 64 bit (AMD64)]
Loading SQLite vector extension from ./vec0
MD5: 382449844d4b9cbb1b7e0d017945db62, SHA-256: d179b834501aa8af22cc866cbd2e838093a594a4bf06260009169ee2cf0fc986
('v0.0.1-alpha.37',)

Another update:
The DLL seems to work with some python MSC versions. Specifically, Python 3.11 and 3.12 by Anaconda seem to work:
Q:\projects\vectest via 🐍 v3.12.4
❯ ~\.conda\envs\textrain\python.exe .\main.py
SQLite version: ('3.41.2',)
Python version: 3.12.0 | packaged by Anaconda, Inc. | (main, Oct  2 2023, 17:20:38) [MSC v.1916 64 bit (AMD64)]
Loading SQLite vector extension from ./vec0
MD5: 382449844d4b9cbb1b7e0d017945db62, SHA-256: d179b834501aa8af22cc866cbd2e838093a594a4bf06260009169ee2cf0fc986
('v0.0.1-alpha.37',)

❯ ~\.conda\envs\imagen\python.exe .\main.py
SQLite version: ('3.41.2',)
Python version: 3.11.5 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:26:23) [MSC v.1916 64 bit (AMD64)]
Loading SQLite vector extension from ./vec0
MD5: 382449844d4b9cbb1b7e0d017945db62, SHA-256: d179b834501aa8af22cc866cbd2e838093a594a4bf06260009169ee2cf0fc986
('v0.0.1-alpha.37',)

Q:\projects\vectest via 🐍 v3.12.4
❯ ~\AppData\Local\Microsoft\WindowsApps\python.exe .\main.py
SQLite version: ('3.45.3',)
Python version: 3.12.4 (tags/v3.12.4:8e8a4ba, Jun  6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]
Loading SQLite vector extension from ./vec0
MD5: 382449844d4b9cbb1b7e0d017945db62, SHA-256: d179b834501aa8af22cc866cbd2e838093a594a4bf06260009169ee2cf0fc986
Error loading extension: The specified module could not be found.

Error loading extension: The specified module could not be found.

The windows store version of python doesn't work. Why?
The MSC version differs (1916 vs 1940)

I noticed that the working versions have SQLite 3.41.2, whereas Windows Store python has SQLite 3.45.3.
I installed Windows Store python 3.10 which has SQLite 3.40.1 and it still throws the same error:

Conclusion
Upon further testing with other SQLite extensions, the same error appears, and only with Windows Store Python versions, Conda and Mingw seem to work just fine with the DLLs.
I have to conclude this is an issue with Microsoft Store Python, but I can't find any specific information about it.
Except this:
https://docs.python.org/3.11/library/sqlite3.html#sqlite3.Connection.enable_load_extension
It seems extension support can be disabled at compile time, but I'm not sure if this is the case here. No exception is raised with enable_load_extension, only the error you see when trying to load extensions.

I've switched to python.org python and everything seems to work for me now .
I'm closing this issue since the problem doesn't seem specific to SQLite-vec.

Closing

Thank you again @reasv for your thorough notes!

Thank you, no problem. I hope you'll find out why this is happening eventually.


Option to hide pink borders around input controls

Is there any way for hiding pink border around input controls, or to make them tinner (like in vscode).
Visual Studio

VS Code

unfortunately, not without affecting other controls. You can fork the repo and introduce changes if you wish to customize.


installShellCompletion: add sanity check

Description of changes
When implementing https://github.com/NixOS/nixpkgs/pull/288279 I first made a typo (i.e. calling <($out/bin/espflash completion bash) instead of <($out/bin/espflash completions bash)) and I didn't notice it until I manually checked the generated completion files - any errors when using the <(...) syntax are apparently silently ignored.
I tried to add some error handing to it but it looked a bit tricky to fix without unindended breakage. But maybe having this simple sanity check would be nice.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/3976

For cross compilation, this breaks the typical pattern of
postInstall = '''
  installShellCompletion --cmd foo \
    --bash <($out/bin/foo completion bash)
    …
'';

Looking around a bit,
$ rg -lU 'installShellCompletion(?s).*<\(\$out/bin' | wc -l
329

is definitely more than a handful. Out of a random sample of 10, only 1 guarded against this correctly with canExecute. (I've also seen stdenv.hostPlatform == stdenv.buildPlatform while looking into this, not optimal either.)
300 is definitely more instances than I'd like to fix manually. Wouldn't it be better to disable this check on cross compilation?


Ripser output intervals to file

Is it possible to save all intervals computed by Ripser to an external file?
I am trying to edit the code to do this, but some intervals are not inserted in the file.
I was using fopen() and fprintf (eg. fprintf(ripser_out, "%lu %.8f %.8f\n",dim,diameter,death);) but for some random data set the output on screen is not the same as the one created in output file.
I have been working on this for more than 1 week but no progress.
It would be good to have an output file to be used within other programs. So a good output could have the format: dim birth death for intervals as [a,b) and dim birth -1 for [a, ).
I have  no idea why my code does not work. Maybe because some limitation with fopen.
I appreciate any help.
Thanks in advance.

If you're willing to step outside of c++, I created a short script to convert the output into JSON using Python.
You can find the gist here
https://gist.github.com/sauln/da65fa1d699c9857bb9ba845973e6745

Thanks. I'd like to adapt the cpp code. It is easy to use some grep or sed to manipulate the original output and produce those format to export on a file, using bash for example. But I'd like to produce that from Ripser so I can only run it and get the output to use in other programs.
Anyway, thanks.

I get that. If you get it to work, I would appreciate the addition also.

After aksing help on StackOverflow, I got a solution and now I extended the ripser.cpp to ripser_otf.cpp (otf=OutputToFile). So, how can I propose the new version?

You want to submit your changes to this repo?
You need to fork this repo, clone it to your computer, make the changes onto that copy of the repo, ad then push back up to your fork. Once that is done, github will ask if you want to submit a pull request.
Check out https://help.github.com/articles/fork-a-repo/

OK, thanks. I did a fork but I never used Git so I'll try to create a pull request, as necessary. That compare part is strange. Lets see.
Regards.

Git can definitely be confusing, even after using it for a long time.
We’re you able to commit your new changes and push the commits up to github?

Too complicated. I have no idea how to use this.
If you wish, please, try to open from here:
https://github.com/tmelorc/ripser/blob/e7a9e35cb3ec5497889b0e67c9ee64ddb8e526fa/ripser_otf.cpp

That pull request looks good. I added the ability to input a filename as an argument.
I made a pull request to your branch. If you accept that request, then (I'm pretty sure) it will automatically update your original pull request to incorporate those changes.

Well, I did. I'm not sure if it right, but let's see. I'll try to get your version to my computer, using git again. Hopefully it will work. In case of problem, I have backup.

Looks like it worked correctly.  Now that you've merged the pull request, all you should have to do is
git pull origin master

If I do this, the new version from git would be downloaded to my computer?

Yes.  How did you originally get the code down to your computer? You should have a local copy of the repo, and running git pull will pull the new updates from origin master where origin is your fork in the cloud and master is the branch or your fork.

Usually I download the file directly from the git, using the raw file. I'll try git pull.

OK, now it works. I got the new version. I've just compiled it and run. It works. But the output is not default. The option --output you created is necessary to have the output on file.
What do you think to make it default? Then if we pass --output option we can choose the output file name.
Thanks for your help.

as it is now, if you don't pass a file name, then it won't print out a file. I think that'd be better for a default use case, rather than always create a new file. You might want something different yourself though.

Good. After two weeks trying to print that, finally it woks.
Regards.

It looks like every time you are commiting, it updates the pull request. You could close the request, create a new branch, and then issue the pull request from that branch.
That way you can keep pushing new commits without it effecting the essential pull request.

Well, sorry. This git is really very complicated.
I closed the pull request. I have no idea how to use new branch. So I think I'll stop committing for a while, since I was only putting some examples more. Nothing important.
Thanks for helping.
Regards.

No worries. It is pretty complicated and there is a ton to learn. It gets pretty complicated fast.
You can create a branch using ‘git checkout -b BRANCHNAME’ and then commiting changes. You would then push to github using ‘git push origin BRANCHNAME’.

What should be a good BRANCHNAME? Some name with my user name, or with Ripser + my_user name? Or ripser_tests, or...? This name will be used only by me or should be relevant to others?

Usually you want it to capture the intent of your work.  Something like “output-examples” if that’s what you’re working on. No need for your name since all the commits will have that already also. Same with the library name, it would be redundant.

Hi all,
Is this issue still open? Since I see the @tmelorc 's pull request still open.

Since the licence was changed some time ago, it would be good to have this feature in the original program (I'm not sure if forked material is allowed anymore).

I'm closing this issue since now I have some better knowledge how Github works and maybe I can deal with a better way to propose some changes. Thanks.


Subscriber data missing in event mw.mail_subscribe

In file userfiles\modules\newsletter\functions\subscriber_functions.php line 233 event_trigger('mw.mail_subscribe', $subscriber_data); passes the $subscriber_data array, but this array does not include 'name' or 'email', etc, it only includes 'rel_type'. 'rel_id', and 'option_group'; surely array $subscriber_data needs merging with $data?

Now fixed in latest newsletter module.


implement flag r feature in 'build' command

examples:
!build -r
!build -r @user1 @user2

flag r toggles the skill level gap, if on: ignore skill levels

#17 merged

#17 merged


s3 failing for certain region / bucket name combinations

needs testing for all regions with and without dots in names
switch endpoints as per previous issue around dots in bucket names discusses

deploying to "new" s3 buckets seems OK, old is having the issue


[Feature] Distributed capabilities

Description
Kicks off the work of distributing tensordict content across devices.
Allows for sync and async coordination.
See tests and docstrings for examples
cc @albertbou92

Codecov Report
Base: 89.16% // Head: 88.28% // Decreases project coverage by -0.89% :warning:

Coverage data is based on head (7566800) compared to base (8e88cba).
Patch coverage: 41.80% of modified lines in pull request are covered.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #230      +/-   ##
==========================================
- Coverage   89.16%   88.28%   -0.89%
==========================================
  Files          27       28       +1
  Lines        9314     9489     +175
==========================================
+ Hits         8305     8377      +72
- Misses       1009     1112     +103

Flag
Coverage Δ

linux-cpu
86.19% <41.80%> (-0.95%)
:arrow_down:

linux-gpu
88.05% <41.80%> (-1.05%)
:arrow_down:

linux-macos-cpu
?

linux-stable-cpu
86.32% <41.80%> (-0.85%)
:arrow_down:

linux-stable-gpu
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

tensordict/tensordict.py
81.83% <13.25%> (-2.33%)
:arrow_down:

test/test_fx.py
97.43% <60.00%> (-2.57%)
:arrow_down:

test/test_distributed.py
67.04% <67.04%> (ø)

tensordict/memmap.py
85.75% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

FYI convergage decreases because we don't test coverage across procs.
That's something we should do


KeyError: 'graph_vs_graph_logos'

Thanks for the nice looking tool! I'm trying to locally run the simple_conga_pipeline.ipynb. I get the KeyError: 'graph_vs_graph_logos' error here:
tag = conga.tags.GRAPH_VS_GRAPH_LOGOS
pngfile = adata.uns['conga_results'][tag]

I previously ran conga.plotting.make_graph_vs_graph_logos(), but I guess it did not populate the dict?
print(conga.tags.GRAPH_VS_GRAPH_LOGOS, '#', adata.uns['conga_results'].keys())
# graph_vs_graph_logos # dict_keys(['graph_vs_graph', 'graph_vs_graph_help'])

Hi Chris,
Thanks for trying conga out! Quick question: is this running on your own data or on the same dataset from the notebook? I'm wondering whether conga found any graph-vs-graph clusters at all. In the output after the make_graph_vs_graph_logos() block, do you see lines like
"making cluster logos: 0 4 tmp_hs_pbmc_graph_vs_graph_logos.png"
(like in the original notebook on github)?
Either way, sounds like we should add a check for this.
Take care,
Phil

This error is due to no conga clusters passing the min_cluster_size threshold so nothing is being stored in adata.uns['conga_results']['graph_vs_graph_logos']. You could try decreasing the threshold to 2, the minimum, and see if you have any luck.

Thanks Stefan, great suggestion!

Thank you both for the quick and helpful responses! To clarify, I am using my own data (not the test data). There is no output from the make_graph_vs_graph_logos()- the cell runs without error but does not print anything (unlike in the tutorial notebook). Running make_graph_vs_graph_logos() with min_cluster_size = 2 leads to the same error.
Here is conga stats:
OrderedDict([('num_cells_w_gex', 3865),
             ('num_features_start', 36601),
             ('num_cells_w_tcr', 2589),
             ('min_genes_per_cell', 200),
             ('max_genes_per_cell', 2500),
             ('max_percent_mito', 0.1),
             ('num_filt_max_genes_per_cell', 98),
             ('num_filt_max_percent_mito', 4),
             ('num_antibody_features', 0),
             ('num_TR_genes', 91),
             ('num_TR_genes_in_hvg_set', 87),
             ('num_highly_variable_genes', 1579),
             ('num_cells_after_filtering', 2487),
             ('num_clonotypes', 1972),
             ('max_clonotype_size', 127),
             ('num_singleton_clonotypes', 1879)])

The conga score plot above is not looking colourful:

I'm guessing something failed/performed poorly upstream? I can dig into it

Huh, no, that's not very colorful at all! It's certainly possible that conga is not finding much correlation between GEX and TCR in the dataset. But it would be nice to rule out a technical issue, since the software is still relatively new. What does the graph-vs-graph results dataframe look like (shape, range of conga scores)? This would be shown in the block after the code:
results = conga.correlations.run_graph_vs_graph(
    adata, all_nbrs, outfile_prefix=outfile_prefix)

or accessible from adata in adata.uns['conga_results']['graph_vs_graph']
Also, I'm curious whether the other analyses are returning anything... To rule out some failure in neighbor-finding or the like.

Thanks again! adata.uns['conga_results']['graph_vs_graph'] contains:
  | conga_score | num_neighbors_tcr | cluster_size | overlap | overlap_corrected | mait_fraction | clone_index | nbr_frac | graph_overlap_type | gex_cluster | tcr_cluster | va | ja | cdr3a | vb | jb | cdr3b
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
0.570129 | 19 | 324 | 10 | 10 | 0.0 | 884 | 0.01 | gex_cluster_vs_tcr_nbr | 2 | 7 | TRAV26-1*01 | TRAJ48*01 | CIVCGFGNEKLTF | TRBV10-3*01 | TRBJ2-3*01 | CAISARAAGEDTQYF
0.973314 | 19 | 345 | 10 | 10 | 0.0 | 705 | 0.01 | gex_cluster_vs_tcr_nbr | 0 | 6 | TRAV22*01 | TRAJ42*01 | CAVDYGGSQGNLIF | TRBV19*01 | TRBJ2-5*01 | CASRGQGRGPETQYF

I'm guessing there should be more rows? Maybe there isn't much correlation between GEX and TCR for this data, but I'm surprised because it's just 'vanilla' PBMCs. I'm getting plenty of other plots like UMAPs and phylogenies, although I can't say that I have yet digested their meaning. I could upload the full notebook if helpful and you have time to look at it?

Sure, I'd be happy to take a look. If you'd rather keep it semi-private, you could also send me a dropbox link (pbradley@fredhutch.org). I can forward to Stefan and see if either of us has any ideas. I agree that PBMCs usually have some "positive controls" for conga, like MAITs or CD4 vs CD8 TCR sequence differences, but we haven't looked at all that many datasets... Thanks for exploring this with us!


Missing value for atKey

Lead: @sachins-geekyants
Describe the bug
while fetching value for atKey, data was missing.
Form @ minorgettingplayed7 , I sent some files  to @ significantredpanda.
We are able to fetch the atKeys on receiver's end but value is coming as "".
This is one of the key that is failing.
cached:@significantredpanda:file_transfer_143d3c8a-7092-4bb8-a6e3-f846efce17b4.mospherepro@minorgettingplayed7
error message
flutter: SEVERE|2022-01-25 14:30:39.657492|AtClientImpl|Decryption failed. encryption value is null for file_transfer_143d3c8a-7092-4bb8-a6e3-f846efce17b4.mospherepro
To Reproduce
Steps to reproduce the behavior:

Onboard with @ significantredpanda
Try to get received file transfer data and check the AtValue.value

Expected behavior
AtValue.value should give the actual data instead of "".
Were you using an @application when the bug was found?

@ mosphere pro

Had call with @sachins-geekyants  and below are the initial analysis from the secondary server logs. Looks like when running TTR job if empty data is received, the keys are updated with empty data.
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    | FINER|2022-01-25 08:00:05.163903|OutboundClient|writing to outbound connect
ion: lookup:file_transfer_143d3c8a-7092-4bb8-a6e3-f846efce17b4.mospherepro@minorgettingplayed7
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    |
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    |
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    | FINER|2022-01-25 08:00:10.265871|OutboundClient|lookup result after format:

bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    |
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    | FINEST|2022-01-25 08:00:10.266120|AtRefreshJob|lookup value of file_transfe
r_143d3c8a-7092-4bb8-a6e3-f846efce17b4.mospherepro@minorgettingplayed7 is
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    |
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    | FINEST|2022-01-25 08:00:10.266980|HiveKeystore|hive key:cached:@significant
redpanda:file_transfer_143d3c8a-7092-4bb8-a6e3-f846efce17b4.mospherepro@minorgettingplayed7
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    |
bd07fcc6-0147-52fb-b1e2-5050f27e88d6_secondary.1.o2jgq3kqzkb0@hornet-12    | FINEST|2022-01-25 08:00:10.267044|HiveKeystore|hive value:AtData{data: , me
taData: {createdBy: null, updatedBy: null, createdAt: 2022-01-21 08:35:17.659Z, updatedAt: 2022-01-25 08:00:10.266976Z, availableAt: null, expiresAt: 20
22-02-24 08:00:10.266Z, refreshAt: null, status: active, version: 1, ttl:<PHONE_NUMBER>, ttb: null, ttr: -1, ccd: false, isBinary: null, isEncrypted: null,
 dataSignature: null}```

@kalluriramkumar we are still facing this issue, do we have any update on this ?

Hi Sachin, we will take this issue into current sprint(PR-30).

FINEST|2022-02-16 12:04:00.006821|AtRefreshJob|Scheduled Refresh Job started

FINER|2022-02-16 12:04:15.722007|OutboundClient|handShakeResult: @murali@

FINER|2022-02-16 12:04:15.722333|AtRefreshJob|connect result: true

FINER|2022-02-16 12:04:15.723894|OutboundClient|writing to outbound connection: lookup:key1@sitaram

FINER|2022-02-16 12:04:15.776405|OutboundClient|lookup result after format: data:value1_1

FINEST|2022-02-16 12:04:21.779640|AtRefreshJob|key1@sitaram cached value is same as looked-up value. Not updating the cached key

FINER|2022-02-16 12:04:24.388587|OutboundClientManager|retrieved outbound client from pool to @sitaram

FINER|2022-02-16 12:04:24.392218|OutboundClient|writing to outbound connection: lookup:key2@sitaram

FINER|2022-02-16 12:04:29.496313|OutboundClient|lookup result after format:

FINEST|2022-02-16 12:04:34.339313|AtRefreshJob|value not found for key2@sitaram. Failed updating the cached key

FINEST|2022-02-16 12:04:38.276458|AtRefreshJob|scheduled Refresh Job completed

Logs after the fix was done.

Following PR fixes this issue.

fix published in at_client version 3.0.10

I am running @ mospherePro with at_client:3.0.13
I can still reproduce this issue.
The following key was sent by significantredpanda to dying36dragonfly on 9th march 2022. On receiver's end data is coming as "".
data:{“key”:“cached:@dying36dragonfly:file_transfer_6284349d-7921-40d5-9c08-cca9e786f99a.mospherepro@significantredpanda”,“data”:“”,“metaData”:{“createdBy”:null,“updatedBy”:null,“createdAt”:“2022-03-11 07:56:29.485Z”,“updatedAt”:“2022-03-11 07:56:29.485Z”,“availableAt”:null,“expiresAt”:“2022-04-10 07:56:29.485Z”,“refreshAt”:null,“status”:“active”,“version”:0,“ttl”:2592000000,“ttb”:null,“ttr”:-1,“ccd”:false,“isBinary”:null,“isEncrypted”:null,“dataSignature”:null}}

I am running @ mospherePro with at_client:3.0.13
I can still reproduce this issue.
The following key was sent by significantredpanda to dying36dragonfly on 9th march 2022. On receiver's end data is coming as "".
data:{“key”:“cached:@dying36dragonfly:file_transfer_6284349d-7921-40d5-9c08-cca9e786f99a.mospherepro@significantredpanda”,“data”:“”,“metaData”:{“createdBy”:null,“updatedBy”:null,“createdAt”:“2022-03-11 07:56:29.485Z”,“updatedAt”:“2022-03-11 07:56:29.485Z”,“availableAt”:null,“expiresAt”:“2022-04-10 07:56:29.485Z”,“refreshAt”:null,“status”:“active”,“version”:0,“ttl”:2592000000,“ttb”:null,“ttr”:-1,“ccd”:false,“isBinary”:null,“isEncrypted”:null,“dataSignature”:null}}

The secondary server is restarted on 11th March and unable to fetch the earlier logs. Requested to @sachins-geekyants to share the files today and captured the id's. Will keep an eye on the server logs.

Can still see this issue for ID: cached:@dying36dragonfly:file_transfer_2b3b7446-7003-40f0-a78c-8f2ad1e00514.mospherepro@significantredpanda

Can still see this issue for ID: cached:@dying36dragonfly:file_transfer_2b3b7446-7003-40f0-a78c-8f2ad1e00514.mospherepro@significantredpanda
had a discussion with Sitaram, he pointed

data: [{"id":"7","name":"secondaryServerVersion","value":"3.0.10"}]
For some reason the secondary server is running on 3.0.10 version
we have the fix on 3.0.12 version. Hence you are still seeing the issue

@sachins-geekyants : Even if the server is upgraded to a new version, the key will not have the old value because, as far as i remember app does not store the key (setting sharedWith to dying36dragonfly in significantredpanda atSign)

We can not recover the old value but if the server is upgraded to 3.0.12 version, we should not be seeing this issue?

We can not recover the old value but if the server is upgraded to 3.0.12 version, we should not be seeing this issue?

@sachins-geekyants : Correct.

Similar issue was observed on different atsign as well - 5distinctiveaquabike
data:{“key”:“cached:@5distinctiveaquabike:file_transfer_0731d300-06e4-4d60-94d9-8e5ed92e2577.mospherepro@jetskiracingpetite”,“data”:“”,“metaData”:{“createdBy”:null,“updatedBy”:null,“createdAt”:“2022-03-31 09:01:11.246Z”,“updatedAt”:“2022-03-31 09:01:11.247Z”,“availableAt”:null,“expiresAt”:“2022-04-30 09:01:11.247Z”,“refreshAt”:null,“status”:“active”,“version”:0,“ttl”:2592000000,“ttb”:null,“ttr”:-1,“ccd”:false,“isBinary”:null,“isEncrypted”:null,“dataSignature”:null,“sharedKeyEnc”:null,“pubKeyCS”:null}}

Assigning myself to this for PR56 as I will be adding multiple tests for the refresh job and ensuring that it works correctly, and continues to work correctly in the future.

Can no longer reproduce


Interop with tide::Error (http_types::Error)

First off, thanks for the great library!
I'm using tide which will convert StdErrror to a tide::Error, but all of the constructors for tide::Error, will wrap the error in anyhow, using anyhow::Error::new(e) this fails because
 .map_err(|e| anyhow::Error::new(e))
   |                                             ^ the trait `std::error::Error` is not implemented for `anyhow::Error`
...
         E: StdError + Send + Sync + 'static,
            -------- required by this bound in `anyhow::error::<impl anyhow::Error>::new`

I tried creating a helper method to leverage the conversion that you've been advising in similar via an explicit into() call at method invocation to_tide_error(e.into()):
    pub fn to_tide_error(e: Box<dyn std::error::Error + Send + Sync + 'static>) -> tide::Error {
        tide::Error::from(e)
    }

this is yielding Error
error[E0277]: the size for values of type `dyn std::error::Error + std::marker::Send + std::marker::Sync` cannot be known at compilation time
  --> src/api.rs:60:27
   |
60 |         tide::Error::from(e)
   |                           ^ doesn't have a size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `dyn std::error::Error + std::marker::Send + std::marker::Sync`
   = note: to learn more, visit <https://doc.rust-lang.org/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait>
   = note: required because of the requirements on the impl of `std::error::Error` for `std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>`
   = note: required because of the requirements on the impl of `std::convert::From<std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>` for `http_types::error::Error`
   = note: required by `std::convert::From::from`

I eventually found: https://github.com/dtolnay/anyhow/issues/63#issuecomment-591011454 where you advise using thiserror.
It was quite a bit of digging to find that solution. I'm kinda bummed that I have to use another library to use a library that leverages anyhow.
I don't understand the coherence issues, but I've seen them referenced in other issues. I'm raising this issue mostly out of concern around the sizing issue that I don't completely understand making that From impls in anyhow less effective in practical usage.
Thanks again for anyhow. Sorry for what's basically a repeat question, but I hope that putting this together could help address this with some code or documentation changes that make these issues happen less and save everyone time in the long run.

It would be reasonable for tide::Error to have a constructor that takes Box<dyn Error + Send + Sync + 'static> so that you can call it with tide::Error::some_constructor_name(your_error.into()), relying on the implementation of Into<Box<dyn Error + Send + Sync + 'static>> for anyhow::Error.

I'm raising this issue mostly out of concern around the sizing issue that I don't completely understand

That error is just saying Box<dyn Error + Send + Sync + 'static> does not implement Error. The std::error::Error impl for Box in the standard library only applies if T is not a trait object.

I believe have a similar question:
An external library I'm using returns anyhow::Error. Underneath it's a io::Error. I want to use thiserror to wrap this error in my own type.
Something like this works:
#[derive(Debug, Display, Error)]
pub enum MyError {
    /// missing source
    MissingSource(#[from] anyhow::Error),
}

But I'm trying to figure out how I can make it #[from] std::io::Error since anyhow::Error could be any type of error.
One solution I could see is downcasting the generic error and add a second MyError::Uknown type to fall back to, but is that the conventional solution here, since that seems to be a relatively verbose solution that requires me to map the errors vs just appending the ? operator.
I'd rather have it automatically fall back to the unknown error using the technique described in thiserror:
#[derive(Debug, Display, Error)]
pub enum MyError {
    /// missing source
    MissingSource(#[from] std::io::Error),
    // ^ use this if `anyhow::Error` downcasts to `io::Error`

    #[error(transparent)]
    Other(#[from] anyhow::Error)
    // ^ use this if no other error matches
}

Apologies if my question is completely different, I'm happy to open a new issue if that helps.

@dtolnay thanks for your fast advice. with your help, I was able to find a constructor with a suitable signature tide::Error::from_str which passes anything that meets those trait bounds to ``anyow::Error::msg```. It looses the backtrace, but  moves me forward.

https://github.com/http-rs/http-types/issues/88

I think that issue is different. That is about converting http_types::Error into a different error type, not converting a different error type into http_types::Error.

Can we use https://docs.rs/tide/0.9.0/tide/struct.Error.html#method.new directly with anyhow::Error?


Changed max length of title to 26

dev
JIRA

Main JIRA ticket

Code reviewers

[ ] @ita-social-projects/lv-734-02-streetcode

[ ] @ita-social-projects/lv-734-03-streetcode

[ ] @ita-social-projects/lv-734-04-streetcode

[ ] @github_username

Summary of issue
ToDo
Summary of change
ToDo
Testing approach
ToDo
CHECK LIST

[ ]  СI passed
[ ]  Сode coverage >=95%
[ ]  PR is reviewed manually again (to make sure you have 100% ready code)
[ ]  All reviewers agreed to merge the PR
[ ]  I've checked new feature as logged in and logged out user if needed
[ ]  PR meets all conventions

BUG TICKER https://github.com/ita-social-projects/StreetCode/issues/386


No title bar for Non-Maximised windows

I would like to know if it is possible to add this feature (no title bar) for non-maximised windows?
This is a problem when using an extension like ShellTile, as seen in the image bellow.

The plugin works perfectly if the window is maximised.

@JasoonS, you should give a try to Undecorate extension, this is what it does.
https://extensions.gnome.org/extension/1208/undecorate/

@GuilloOme Thank you very much! That works perfectly. A tiny bit annoying that you need to manually 'undecorate' each window, but I'm happy.


Make unravelling of oil bus consistent with new refineries

I.e. separate DE oil primary from DE oil. This is a partner to PyPSA-Eur PR https://github.com/PyPSA/pypsa-eur/pull/1192/files.

looks good
the model ran through without problems, but i still want to update the exporter before merging.
open points:

[ ] should decentral heating oil be considered a Liquid?
[ ] update how prices are computed in the exporter @JulianGeis

Decentral heating oil is a liquid.


Support key material format common to Kubernetes (PEMs) and add upstream TLS support

Type of change
Select the type of your PR

Enhancement / new feature

Description

Added support for TLS key material in PEM format, with or without key encryption.  The PEM format is common when working with Kubernetes so supporting PEM will simplify integration with Kubernetes.  (Certificate and private key PEMs are configured separately).

                                virtualClusters:
                                  demo:
                                    targetCluster:
                                      bootstrap_servers: kafka.example:1234
                                    tls:
                                       key:
                                         certificateFile: /tmp/cert
                                         privateKeyFile: /tmp/key
                                         keyPassword:
                                           password: keypassword

Added support for the Kafka-style PEM keystores as implemented by KIP-651.  The motivation is to accept key material in the same way as Kafka to smooth integration with Kafka use-cases.  (Certificate and private key PEMs are concatenated are in a single file).

                                virtualClusters:
                                  demo:
                                    targetCluster:
                                      bootstrap_servers: kafka.example:1234
                                    tls:
                                       key:
                                        storeType: PEM
                                         storeFile: /tmp/my.pem
                                         keyPassword:
                                           password: keypassword

Added support for PKCS12 format keystore.

                                virtualClusters:
                                  demo:
                                    targetCluster:
                                      bootstrap_servers: kafka.example:1234
                                    tls:
                                       key:
                                        storeType: PKCS12
                                         storeFile: /tmp/my.p12
                                         keyStorePassword:
                                           password: keypassword

Added support for upstream TLS with optional truststore and insecure mode.

                               virtualClusters:
                                  demo:
                                    targetCluster:
                                      bootstrap_servers: kafka.example:1234
                                      tls: {}
                                    clusterNetworkAddressConfigProvider:
                                      type: SniRouting
                                      config:
                                        bootstrapAddress: cluster1:9192
                                        brokerAddressPattern: broker-$(nodeId):$(portNumber)

Added support for keystore passwords and key passwords to come from files.  Inline passwords are often considered unacceptable in production environments.

The model adopted should be fit for future extension - TLS client auth, TLS protocol/cipher suite restrictions should fit right in without rework.
Additional Context
Move Kroxylicious towards production like TLS support.

Once this PR is review and comments address I'll open a separate PR with documentation for this feature.

I'm also a cautious as there seems to be a lot of new File(xxx) without any validation that the path is valid. I think we generally catch or let the exceptions propagate as appropriate but if feels a bit "off". Should we be more cautions about handling paths?

Yep maybe we should check for existence (we could also validate that it points at what it claims it does, like a valid PEM) while instantiating the records. It would be nice to know at configuration time that there's a problem.

I'm wary of the file IO happening on the eventLoop.

Definitely agree. I think the virtualcluster should be responsible for creating the SSLContexts and caching them on startup.  It should periodic check to see if it needs to regenerate the SSLContext so that Kroxylicious handles cases such as key rotation.

I'm also a cautious as there seems to be a lot of new File(xxx)

We could have the configuration model in terms of FIle objects, rather than Strings.  That would clean up the code.  I expect at Jackson level, we'd need a com.fasterxml.jackson.databind.ser.std.FileSerializer plugged into the mapper so that config serialisation works.  Sundrio builders don't special case the File object, so uses of the fluent API would need to do withNewBlah(new File) but I think that'd be okay.  WDYT?

without any validation that the path is valid.  I think we generally catch or let the exceptions propagate as appropriate but if feels a bit "off". Should we be more cautions about handling paths?

On startup, I think it would be reasonable for Kroxylicious to fail to startup.
At runtime, I think Kroxylicious should tolerate a FileNotFoundException and just retry the SSLContext creation later.  I say this because I am thinking about a Kubernetes projected Secret object expressed in the pod's filesystem.  If someone is replacing a secret containing a certificate or password, it is possible that the projected file could disappear momentarily.

On startup, I think it would be reasonable for Kroxylicious to fail to startup.
At runtime, I think Kroxylicious should tolerate a FileNotFoundException and just retry the SSLContext creation later.

Sounds good to me

One of the key aims of this PR was to add support for the direct use of key material commonly generated by Kube/OpenShift operators.
The example I had in mind was the OpenShift service serving certificate feature of OpenShift.  It generates private key material in the PKCS#1 format and this is not configurable.  Unfortunately, the JCA is unable to parse PKCS#1 formatted PEMs.   (There is an RFE against https://issues.redhat.com/browse/RFE-1803).
So, where does leave this PR?  The KeyPair feature does work with PKCS#8 formatted private keys (with or without key passwords), so the feature still has utility.    Furthermore, the feature will work with PKCS#1 formatted private keys if Bouncycastle is available on the classpath.  This is because Netty detects BC at runtime, and uses it in preference to it own code, and Bouncycastle has support for PKCS#1.

@robobario @SamBarker could you give this one last look over, I think it is now good to merge.
I'll open a separate PR for the documentation (the scope needs to be slightly wider than this PR)..


demo.py fails

pip version: 20.3.3
python version: 3.7.9
step to reproduce:
git clone https://github.com/yhenon/pyimgsaliency
cd pyimgsaliency
python setup.py install
cd pyimgsaliency
python demo.py

get the following error
C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\pyimgsaliency-0.1-py3.7.egg\pyimgsaliency\saliency.py:83: FutureWarning: skimage.measure.label's indexing starts from 0. In future version it will start from 1. To disable this warning, explicitely set the `start_label` parameter to 1.
  segments_slic = slic(img_rgb, n_segments=250, compactness=10, sigma=1, enforce_connectivity=False)
Traceback (most recent call last):
  File ".\demo.py", line 8, in <module>
    rbd = psal.get_saliency_rbd(filename).astype('uint8')
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\pyimgsaliency-0.1-py3.7.egg\pyimgsaliency\saliency.py", line 92, in get_saliency_rbd
    (vertices,edges) = make_graph(grid)
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\pyimgsaliency-0.1-py3.7.egg\pyimgsaliency\saliency.py", line 62, in make_graph
    vertices[x/num_vertices]] for x in edges]
  File "C:\Users\avh\AppData\Local\Programs\Python\Python37\lib\site-packages\pyimgsaliency-0.1-py3.7.egg\pyimgsaliency\saliency.py", line 62, in <listcomp>
    vertices[x/num_vertices]] for x in edges]
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

okay, I figured it out
codebase is written for python2, and I have installed python3
so in 62 insaliency.py you have to replace / with //
and replace every xrange with range


Kodi installed via RetroPie

Hi,
On my RPi2 I installed Kodi via RetroPie and installed Hyperion through the sh script. I used the test command hyperion-remote --priority 50 --color red --duration 5000, however nothing happens. Is my installation currently supported? Is there a way to debug the installation?

still need help? or can be closed?

Closed due to age. Please open a new one, if problems still exists.
Thank you


Failure to run on Windows and Linux

When i follow the instructions to the letter for both Windows and Linux, I get the same error:
Error: While importing 'autoexpress', an ImportError was raised:
Traceback (most recent call last):
File "/home/smashlootrepeat/AI/AutoExpress/my_env/lib/python3.12/site-packages/flask/cli.py", line 245, in locate_app
import(module_name)
File "/home/smashlootrepeat/AI/AutoExpress/autoexpress/init.py", line 10, in
from autoexpress.views import autoexpress
File "/home/smashlootrepeat/AI/AutoExpress/autoexpress/views.py", line 7, in
from autoexpress.modules import (
File "/home/smashlootrepeat/AI/AutoExpress/autoexpress/modules/image_parser.py", line 2, in
from sd_parsers import ParserManager
ModuleNotFoundError: No module named 'sd_parsers'
I manually install sd_parsers to my venv and get:
Error: While importing 'autoexpress', an ImportError was raised:
Traceback (most recent call last):
File "/home/smashlootrepeat/AI/AutoExpress/my_env/lib/python3.12/site-packages/flask/cli.py", line 245, in locate_app
import(module_name)
File "/home/smashlootrepeat/AI/AutoExpress/autoexpress/init.py", line 10, in
from autoexpress.views import autoexpress
File "/home/smashlootrepeat/AI/AutoExpress/autoexpress/views.py", line 7, in
from autoexpress.modules import (
File "/home/smashlootrepeat/AI/AutoExpress/autoexpress/modules/image_parser.py", line 3, in
from sd_parsers import Generators
ImportError: cannot import name 'Generators' from 'sd_parsers' (/home/smashlootrepeat/AI/AutoExpress/my_env/lib/python3.12/site-packages/sd_parsers/init.py)

Fixed it, changed Line 2 in modules/image_parser.py to: from sd_parsers.data import Generators

Changed the image parser module as per the suggestion. Thanks!


Arrow 0.11.1

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin please rerender

Hi! This is the friendly automated conda-forge-webservice.
I tried to re-render for you, but it looks like there was nothing to do.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

So I've only changed the condition to >=0.11.0 without bumping arrow-cpp's version.

+1. go ahead and merge once the build passes

Ok, will do.


Can not run CARLA Simulattor while running the following script for improvised traffic manager

Version : 9.0.14
OS: Windows
Problem: The main problem is when I am trying to run my code, CARLA is crashing and it is not showing any results which I wanted to see.
Expect to happen: I am expecting to see the result of my script which I am trying to simulate for running the traffic light states based on traffic density near the traffic lights. For example, if no of vehicles are more than thresold value (6 or more cars) then the traffic lights gould turn green for a specified time duration of 10 seconds to optimize the traffic flow. In simple terms, I am overriding traffic manager by working according to traffic density near to the traffic lights.
Any help would be greatful!!
Below is my whole script:
import carla
import random
import time

# Connect to the client and retrieve the world object
client = carla.Client('localhost', 2000)
client.set_timeout(10.0)  # Set a timeout for client commands
world = client.get_world()

# Set up the simulator in synchronous mode
settings = world.get_settings()
settings.synchronous_mode = True  # Enables synchronous mode
settings.fixed_delta_seconds = 0.02
world.apply_settings(settings)

# Set up the TM in synchronous mode
traffic_manager = client.get_trafficmanager()
traffic_manager.set_synchronous_mode(True)

# Set a seed so behaviour can be repeated if necessary
traffic_manager.set_random_device_seed(0)
random.seed(0)

actors = world.get_actors()

def get_traffic_density(traffic_light, radius=50):
    vehicles = world.get_actors().filter('vehicle.*')
    density = 0
    for vehicle in vehicles:
        if vehicle.get_location().distance(traffic_light.get_location()) < radius:
            density += 1
    return density

def set_traffic_light_state(traffic_light, state, duration):
    traffic_light.set_state(state)
    traffic_light.set_green_time(duration if state == carla.TrafficLightState.Green else 0)
    traffic_light.set_yellow_time(3.0)
    traffic_light.set_red_time(duration if state == carla.TrafficLightState.Red else 0)

def control_traffic_lights(traffic_lights, check_interval=20, density_threshold=5, green_duration=10):
    while True:
        world.tick()  # Advance the simulation by one tick
        for traffic_light in traffic_lights:
            density = get_traffic_density(traffic_light, radius=50)
            if density > density_threshold:
                set_traffic_light_state(traffic_light, carla.TrafficLightState.Green, green_duration)
                print(
                    f"Traffic light at {traffic_light.get_location()} turned green for {green_duration} seconds due "
                    f"to high traffic density.")
                time.sleep(green_duration + 3.0)  # Include the yellow light duration
                set_traffic_light_state(traffic_light, carla.TrafficLightState.Red, green_duration)
            else:
                set_traffic_light_state(traffic_light, carla.TrafficLightState.Red, green_duration)
                world.tick()
        time.sleep(check_interval)

traffic_lights = world.get_actors().filter('traffic.traffic_light')

# We will also set up the spectator
spectator = world.get_spectator()

spawn_points = world.get_map().get_spawn_points()

# Select some models from the blueprint library
models = ['dodge', 'audi', 'model3', 'mini', 'mustang', 'lincoln', 'prius', 'nissan', 'crown', 'impala']
blueprints = []
for vehicle in world.get_blueprint_library().filter('*vehicle*'):
    if any(model in vehicle.id for model in models):
        blueprints.append(vehicle)

# Assign vehicle counts to each route
route_vehicle_counts = [10, 10, 20, 10]
total_vehicles = sum(route_vehicle_counts)

# Set a max number of vehicles and prepare a list for those we spawn
max_vehicles = total_vehicles
max_vehicles = min([max_vehicles, len(spawn_points)])
vehicles = []

# Take a random sample of the spawn points and spawn some vehicles
for i, spawn_point in enumerate(random.sample(spawn_points, max_vehicles)):
    temp = world.try_spawn_actor(random.choice(blueprints), spawn_point)
    if temp is not None:
        vehicles.append(temp)

# Routes with their respective spawn points and indices
routes = [
    ([32], [129, 28, 137, 101, 57, 58, 154, 147]),
    ([149], [21, 105, 134, 52, 86, 120, 4, 121]),
    ([147], [72, 146, 126, 99, 108, 68, 24]),
    ([106], [85, 1, 104, 67, 140, 10, 143])
]

# Prepare route locations
route_locations = []
for spawn_point, indices in routes:
    route = [spawn_points[spawn_point[0]].location]
    for ind in indices:
        route.append(spawn_points[ind].location)
    route_locations.append(route)

# Assign vehicles to routes
assigned_vehicles = {
    0: [],  # Vehicles for route 1
    1: [],  # Vehicles for route 2
    2: [],  # Vehicles for route 3
    3: []  # Vehicles for route 4
}

vehicle_idx = 0
for route_idx, count in enumerate(route_vehicle_counts):
    for _ in range(count):
        if vehicle_idx < len(vehicles):
            assigned_vehicles[route_idx].append(vehicles[vehicle_idx])
            vehicle_idx += 1

# Assign routes to vehicles
for route_idx, vehicle_list in assigned_vehicles.items():
    for vehicle in vehicle_list:
        traffic_manager.set_path(vehicle, route_locations[route_idx])
        vehicle.set_autopilot(True)
        # Randomly set the probability that a vehicle will ignore traffic lights
        traffic_manager.ignore_lights_percentage(vehicle, random.randint(0, 50))

# Set delay to create gap between spawn times
spawn_delay = 20
counter = spawn_delay

try:
    control_traffic_lights(traffic_lights, check_interval=20, density_threshold=5, green_duration=10)
except KeyboardInterrupt:
    print("Traffic control interrupted")

Thanks for contributing to CARLA!
If you are reporting an issue, please use the following outline:
CARLA version:
Platform/OS:
Problem you have experienced:
What you expected to happen:
Steps to reproduce:
Other information (documentation you consulted, workarounds you tried):
If you are asking a question please make sure your question was not asked before
by searching among the existing issues and checking the CARLA forum https://github.com/carla-simulator/carla/discussions. Also make sure you have read ourdocumentation and FAQ at http://carla.readthedocs.io.
If your question is about creating content (assets, levels) you'll most likely
have all the info you need in the Unreal Engine's documentation
https://docs.unrealengine.com
Please, keep the threads focused and single-themed. Make them easy to find
for everyone.
-->

Cloud you share the complete crash log (client and server side)? Your script is running without crashes in Ubuntu with CARLA 0.9.15.
Also, it seems there is a misunderstanding in your code with simulation time and real time when running in sync mode. Using time.sleep within you main loop does not affect to the simulation time. See me comments below:
def control_traffic_lights(traffic_lights, check_interval=20, density_threshold=5, green_duration=10):
    while True:
        world.tick()  # Advance the simulation by one tick
        for traffic_light in traffic_lights:
            density = get_traffic_density(traffic_light, radius=50)
            if density > density_threshold:
                set_traffic_light_state(traffic_light, carla.TrafficLightState.Green, green_duration)
                print(
                    f"Traffic light at {traffic_light.get_location()} turned green for {green_duration} seconds due "
                    f"to high traffic density.")

                # WARNING: This will freeze the server for the (green duration + 3.0) interval. The does not affect to the simulation time which will remain at the same time-step!!!
                time.sleep(green_duration + 3.0)  # Include the yellow light duration
                set_traffic_light_state(traffic_light, carla.TrafficLightState.Red, green_duration)
            else:
                set_traffic_light_state(traffic_light, carla.TrafficLightState.Red, green_duration)
                world.tick()

        # WARNING: This will freeze the server for the check_interval duration. The does not affect to the simulation time which will remain at the same time-step!!!
        time.sleep(check_interval)

For more information please check the documentation: https://carla.readthedocs.io/en/latest/adv_synchrony_timestep/

Thank you for your insights however, the problem is when I am running the script, it is running fine with outcomes of when the green light is changing and with location but the CARLA in the back is not opening then I have to shut down the engine by manually with task manager :>end task for CARLA to reopen again!
If it is running with Ubanu, please tell me whether it is functioning fine with overriding traffic light states according to traffic density. (I mean is it changing automatically when there are more vehicles)
Thank you again and I will share the screenshot where I am crashing the CARLA until tomorrow!

This was the issue that I was telling you about! As you can see the script is running but the CARLA is crashing!

@mev5644 From what I can see your script is running as expected in your end. Please, read carefully the documentation about synchronous mode (https://carla.readthedocs.io/en/latest/adv_synchrony_timestep/) as your issue is just a misunderstanding of this concept.
On one side, the time.sleep(x) lines are not working as you think. During this time you are not calling world.tick() so the simulation time does not advance! This time.sleep(x) is just freezing the simulation completely for that interval. You should instead call world.tick() repeatedly until this time passes in simulation time. You can check the simulation time in different ways:
timestamp = world.get_snapshot().timestamp

timestamp.elapsed_seconds  # Simulated seconds elapsed since the beginning of the current episode.
timestamp.delta_seconds  # Simulated seconds elapsed since the previous frame. In your case 0.02 as you fixed_delta_seconds is set to that value.

On the other side, when closing the script you are leaving the server in synchronous mode. As no client is ticking the world, the server freezes and therefore the process is set to Not responding by the OS. To avoid this you should reconfigure the server to asynchronous mode again before closing the script. This is done always for our scripts. For instance: https://github.com/carla-simulator/carla/blob/dev/PythonAPI/examples/manual_control.py#L1294

Hi Joel,
Thank you for your feedback and I updated some settings as you said however, I still need your help on the simulation as whenI am trying to run the script in Pycharm, it is running and showing at which locations the green lights are turning green according to the traffic density but on the other side, in CARLA environment the signals are not turning green and it is staying red as it is!!
I can share some screenshots for your reference.
I hope you will get my question correctly..

@mev5644 Take into account that the duration times your are setting in set_traffic_light_state are being overwritten by your main loop in control_traffic_lights. The issue probably comes due to your code changing traffic light states every tick based on your traffic density metric.

@joel-mb Understood, thanks. However, if I am not defining the set_traffic_light_state function in my script then it won't call the control_traffic_lights as it will not defined the function itself that is why I have to insert that function, however, my approach is to set the traffic light state to Green as sson as the traffic density is increasing and that is why I am slo separating 4 different routes using waypoints, which follows the vehicles and the traffic controller (My API) should work according to logic where traffic flow goes easily. That is also my thesis topic and I am really stuck into the situation as it is not working as I expected.
I did some changes in my script and can share with you as well, could you please help me to figure out what mistake which I am still not be able to find out, I will be greateful from your help and your support!
import carla
import random
import time
import pandas as pd

# Connect to the client and retrieve the world object
client = carla.Client('localhost', 2000)
client.set_timeout(10.0)  # Set a timeout for client commands
client.reload_world()

world = client.get_world()

# Set up the simulator in synchronous mode
settings = world.get_settings()
settings.synchronous_mode = True  # Enables synchronous mode
# settings.fixed_delta_seconds = 0.01
world.apply_settings(settings)

# traffic_lights = world.get_actors().filter('traffic.traffic_light')

# Set up the TM in synchronous mode
traffic_manager = client.get_trafficmanager()
traffic_manager.set_synchronous_mode(True)

# # Set a seed so behaviour can be repeated if necessary
# traffic_manager.set_random_device_seed(0)
# random.seed(0)

actors = world.get_actors()

def get_traffic_lights(world):
    actors = world.get_actors()
    traffic_lights = actors.filter('traffic.traffic_light')
    return traffic_lights

def get_traffic_density(traffic_light, radius=50):
    vehicles = world.get_actors().filter('vehicle.*')
    density = 0
    for vehicle in vehicles:
        if vehicle.get_location().distance(traffic_light.get_location()) < radius:
            density += 1
    return density

# def savedata():
#     density_data = []
#     start_time = time.time()
#     for traffic_light in traffic_lights:
#         density = get_traffic_density(traffic_light)
#         elapse_time = time.time() - start_time
#         density_data.append({'Time': elapse_time, 'Traffic Light': traffic_light.id, 'Density': density})
#
#     df = pd.DataFrame(density_data)
#     df.to_excel('Traffic_light_density_data.xlsx', index=False)

# def set_traffic_light_state(traffic_light, state, green_duration=20):
#     # traffic_light.set_state(state)
#     # traffic_light.set_green_time(duration if state == carla.TrafficLightState.Green else 0)
#     # traffic_light.set_green_time(duration)
#     # traffic_light.set_yellow_time(3.0)
#     # traffic_light.set_red_time(duration if state == carla.TrafficLightState.Red else 0)
#     traffic_light.set_state(state)
#     if state == carla.TrafficLightState.Green:
#         traffic_light.set_green_time(green_duration)
#         traffic_light.set_yellow_time(3.0)
#         traffic_light.set_red_time(1.0)
#     else:
#         traffic_light.set_green_time(0)
#         traffic_light.set_yellow_time(0)
#         traffic_light.set_red_time(green_duration + 3.0)
def set_traffic_light_state(traffic_light, state, duration=20):
    traffic_light.set_state(state)
    # traffic_light.set_green_time(duration if state == carla.TrafficLightState.Green else 0)
    traffic_light.set_green_time(duration)

def savedata(density_data):
    df = pd.DataFrame(density_data)
    df.to_excel('Traffic_light_density_data.xlsx', index=False)

def control_traffic_lights(world, check_interval=20, density_threshold=5, green_duration=20):
    traffic_lights = get_traffic_lights(world)
    density_data = []
    start_time = time.time()

    try:
        while True:
            world.tick()
            for traffic_light in traffic_lights:
                density = get_traffic_density(traffic_light, radius=50)
                elapsed_time = time.time() - start_time
                density_data.append({'Time': elapsed_time, 'Traffic Light': traffic_light.id, 'Density': density})

                if density > density_threshold:
                    # if (traffic_light.)
                    set_traffic_light_state(traffic_light, carla.TrafficLightState.Green, green_duration)
                    print(
                        f"Traffic light at {traffic_light.get_location()} turned green for {green_duration} seconds due "
                        f"to high traffic density.")
                    # world.debug.draw_point(traffic_light.get_location(), 0.1)
                    # # time.sleep(green_duration + 1.0)  # Include the yellow light duration
                    # set_traffic_light_state(traffic_light, carla.TrafficLightState.Red, green_duration)
                else:
                    set_traffic_light_state(traffic_light, carla.TrafficLightState.Red, 10)
                    print(
                        f"Traffic light at {traffic_light.get_location()} turned red")
            # time.sleep(check_interval)
    except KeyboardInterrupt:
        print("Terminating the control loop")
        savedata(density_data)
        for i in vehicles:
            i.destroy()
            # world.tick()
        # time.sleep(0.02)
    # time.sleep(check_interval)

# We will also set up the spectator
spectator = world.get_spectator()

spawn_points = world.get_map().get_spawn_points()

# Select some models from the blueprint library
models = ['dodge', 'audi', 'model3', 'mini', 'mustang', 'lincoln', 'prius', 'nissan', 'crown', 'impala']
blueprints = []
# blueprints = world.get_blueprint_library().filter('vehicle.*')
for vehicle in world.get_blueprint_library().filter('vehicle.*'):
    if any(model in vehicle.id for model in models):
        blueprints.append(vehicle)

# Routes with their respective spawn points and indices
routes = [
    ([32], [129, 28, 137, 101, 57, 58, 154, 147]),
    ([149], [21, 105, 134, 52, 86, 120, 4, 121]),
    ([147], [72, 146, 126, 99, 108, 68, 24]),
    ([106], [85, 1, 104, 67, 140, 10, 143])
]

# Assign vehicle counts to each route
route_vehicle_counts = [30, 25, 20, 15]
total_vehicles = sum(route_vehicle_counts)

# Set a max number of vehicles and prepare a list for those we spawn
# max_vehicles = total_vehicles
max_vehicles = min([total_vehicles, len(spawn_points)])
vehicles = []

# Take a random sample of the spawn points and spawn some vehicles
for i, spawn_point in enumerate(random.sample(spawn_points, max_vehicles)):
    temp = world.try_spawn_actor(random.choice(blueprints), spawn_point)
    if temp is not None:
        vehicles.append(temp)

route_1 = []
route_2 = []
route_3 = []
route_4 = []

# Prepare route locations
# route_locations = []
count = 0
for spawn_point, indices in routes:
    if count == 0:
        route_1.append(spawn_points[spawn_point[0]].location)
        # route = [spawn_points[spawn_point[0]].location]
        for ind in indices:
            route_1.append(spawn_points[ind].location)
    if count == 1:
        route_2.append(spawn_points[spawn_point[0]].location)
        # route = [spawn_points[spawn_point[0]].location]
        for ind in indices:
            route_2.append(spawn_points[ind].location)
    if count == 2:
        route_3.append(spawn_points[spawn_point[0]].location)
        # route = [spawn_points[spawn_point[0]].location]
        for ind in indices:
            route_3.append(spawn_points[ind].location)
    if count == 3:
        route_4.append(spawn_points[spawn_point[0]].location)
        # route = [spawn_points[spawn_point[0]].location]
        for ind in indices:
            route_4.append(spawn_points[ind].location)
    count += 1
    # route_locations.append(route)

# Assign vehicles to routes
assigned_vehicles = {
    0: [],  # Vehicles for route 1
    1: [],  # Vehicles for route 2
    2: [],  # Vehicles for route 3
    3: []  # Vehicles for route 4
}

vehicle_idx = 0
for route_idx, count in enumerate(route_vehicle_counts):
    for _ in range(count):
        if vehicle_idx < len(vehicles):
            assigned_vehicles[route_idx].append(vehicles[vehicle_idx])
            vehicle_idx += 1
    vehicle_idx = 0

print(route_1)

# Assign routes to vehicles
for route_idx, vehicle_list in assigned_vehicles.items():
    if route_idx == 0:
        for vehicle in vehicle_list:
            traffic_manager.set_path(vehicle, route_1)
            vehicle.set_autopilot(True)
            # Randomly set the probability that a vehicle will ignore traffic lights
            # traffic_manager.ignore_lights_percentage(vehicle, random.randint(0, 50))
    elif route_idx == 1:
        for vehicle in vehicle_list:
            traffic_manager.set_path(vehicle, route_2)
            vehicle.set_autopilot(True)
    elif route_idx == 2:
        for vehicle in vehicle_list:
            traffic_manager.set_path(vehicle, route_3)
            vehicle.set_autopilot(True)
    elif route_idx == 3:
        for vehicle in vehicle_list:
            traffic_manager.set_path(vehicle, route_4)
            vehicle.set_autopilot(True)
# while True:
#     world.tick()  # Advance the simulation by one tick

# if __name__ == '__main__':
#     main()
# try:
#     control_traffic_lights(traffic_lights, density_threshold=5, green_duration=20)
# except KeyboardInterrupt:
#     for i in vehicles:
#         i.destroy()

control_traffic_lights(world)

@mev5644 From what I see your code has the same issue as before. In your control_traffic_lights method you are overwriting every tick the traffic light state set in the previous iteration. You should see traffic lights in green state but not for the duration you are setting. Your traffic lights will be set to red as soon as your density check threshold becomes false.
Moreover, I think your code is setting to green more than one traffic light for the same controller/intersection. I highly doubt this is intentional as this will probably produce problems in the traffic manager.

Yes, you are indeed right about this structure of my coding. But you see, my goal is to first install the traffic manager then it should call the traffic lights after that I am giving instructions to the traffic manager by overriding traffic signals to the green with predefined timing (let's say 20 seconds) to simulate the traffic manager functionality based on number of vehicles (density of vehicles) and for that reason I also add different routes using way points which vehicles are following on autopilot mode as you can see from my code.
However, if you have any small solution? or any other helping hand for me then I would be grateful for that.
Thank you again for your guidance.

Are you able to solve the problem? I want to use the TM with multiple vehicles and also follow the traffic lights. Mine is not following the traffic light.
Thanks

Hi, @SExpert12 ! You can see this Python API example of handling TM with multiple vehicles that follow traffic lights!
[https://carla.readthedocs.io/en/latest/tuto_G_traffic_manager/)

Hi @mev5644,
I am using the same code. But vehicle is stopping inside the intersection area now as the lights turn to red or yellow.

`max_vehicles = 100
max_vehicles = min([max_vehicles, len(spawn_points)])
vehicles = []
Route 1
spawn_point_1 = spawn_points[32]
Create route 1 from the chosen spawn points
route_1_indices = [129, 28, 137, 101, 57, 58, 154, 147]
route_1 = []
for ind in route_1_indices:
route_1.append(spawn_points[ind].location)
spawn_point_1 = spawn_points[32]
Create route 1 from the chosen spawn points
route_1_indices = [129, 28, 137, 101, 57, 58, 154, 147]
route_1 = []
Route 2
spawn_point_2 = spawn_points[149]
Create route 2 from the chosen spawn points
route_2_indices = [21, 105, 134, 52, 86, 120, 4, 121]
route_2 = []
for ind in route_2_indices:
route_2.append(spawn_points[ind].location)
Route 3
spawn_point_3 = spawn_points[147]
Create route 3 from the chosen spawn points
route_3_indices = [72, 146, 126, 99, 108, 68, 24]
route_3 = []
for ind in route_3_indices:
route_3.append(spawn_points[ind].location)
Route 4
spawn_point_4 = spawn_points[106]
Create route 4 from the chosen spawn points
route_4_indices = [85, 1, 104, 67, 140, 10, 143]
route_4 = []
for ind in route_4_indices:
route_4.append(spawn_points[ind].location)
Variables to alternate between spawn points and routes
spawn_point_index = 0
route_index = 0
Main simulation loop
while True:
world.tick()
update_waiting_times()
n_vehicles = len(world.get_actors().filter('*vehicle*'))
vehicle_bp = random.choice(blueprints)

# Spawn vehicle only after delay
if counter == spawn_delay and n_vehicles < max_vehicles:
    # Select the spawn point based on the index
    if spawn_point_index == 0:
        spawn_point = spawn_point_1
    elif spawn_point_index == 1:
        spawn_point = spawn_point_2
    elif spawn_point_index == 2:
        spawn_point = spawn_point_3
    else:
        spawn_point = spawn_point_4

    # Try to spawn the vehicle at the selected spawn point
    vehicle = world.try_spawn_actor(vehicle_bp, spawn_point)

    if vehicle:  # If vehicle is successfully spawned
        vehicle.set_autopilot(True)  # Give TM control over vehicle

        # Set parameters of TM vehicle control, we don't want lane changes
        traffic_manager.update_vehicle_lights(vehicle, True)
        traffic_manager.random_left_lanechange_percentage(vehicle, 0)
        traffic_manager.random_right_lanechange_percentage(vehicle, 0)
        traffic_manager.auto_lane_change(vehicle, False)

        # Select the route based on the index
        if route_index == 0:
            traffic_manager.set_path(vehicle, route_1)
        elif route_index == 1:
            traffic_manager.set_path(vehicle, route_2)
        elif route_index == 2:
            traffic_manager.set_path(vehicle, route_3)
        else:
            traffic_manager.set_path(vehicle, route_4)

        # Update indices for spawn points and routes
        spawn_point_index = (spawn_point_index + 1) % 4
        route_index = (route_index + 1) % 4

        vehicles.append(vehicle)

    counter -= 1
elif counter > 0:
    counter -= 1
elif counter == 0:
    counter = spawn_delay

`
Try this with your implemented code and check out, it should work.

Hi,
I am able to make the vehicles to follow the traffic lights but now I am facing two new problem.

When vehicle stops in the intersection area also at a times when the light turn to red which some times hamper the other traffic.
I want to allow straight-going and left turn traffic to cross the intersection simultaneously but only straight going vehicles only cross the intersection and left-turn vehicles stays in the intersection area. The problem is I think at a time only one side light is green and second light for left-turn is not green one.
How to solve this problem now?

Thank for help


Add OpenAI configurations and update dependencies in GitHub Actions

Summary by CodeRabbit

New Features

Introduced support for OpenAI services in the GitHub Actions workflow through new environment variables.
Enhanced testing capabilities for OpenAI functionalities with structured tests to validate service interactions.

Bug Fixes

Updated dependencies for improved performance and functionality, ensuring compatibility with the latest features.

Refactor

Added a new abstract class to facilitate conditional test execution based on the environment setup.

@coderabbit review


Running slackbot server from now -- Removed all deployments and bot still running?

Hello,
Now-pro user here.
I'm having a bit of an issue with running a slackbot from now. As you might have guessed from the title of this message, even after deleting all deployments, bots still run. There's no way to kill them. Is this the expected behavior?
This behavior happened twice in a row now after creating a slackbot app from scratch, deploying it, and then removing the deployment.
Happy to clarify any other areas of confusion.
Thanks

Hold that thought...
I'm looking back again now and it looks like the bot is powered down. Is there a 3-5 minute shutoff cycle or something?
Thanks for the clarification.

Yeah... I think that's what happened. Disregard!


[Incubator][VC]Revise README.md

Revise README.md to describe virtualcluster in more details.

/lgtm
/approve


Chapter 8:  Page 252 & 253

it is actually quite difficult to follow through the code in the book if there are so many mistakes.
un= trades.filter(col("symbol") == "K").select('event_ ts', 'price', 'symbol', 'bid', 'offer', 'ind_cd'). union(quotes.filter(col("symbol") == "K").select('event_ ts', 'price', 'symbol', 'bid', 'offer', 'ind_cd'))
I am sure there is no row with the symbol = 'K' and there is no such column named 'bid', 'offer', 'ind_cd'
Is the book actually proof-read?

There are some rows with symbol "K".
However this is completely wrong code, it looks like is filtering the df, and later in the chapter is grouping by this column.
Obviously you cannot make an union using samples in the book as column names are different, unless you will match the structure.

yes, please make it correct, else readers have difficulty following through it, hope you understand. And someone needs to read-proof to reduce such mistakes, please.


Adopt to yiisoft/db last changes

Q
A

Is bugfix?
❌

New feature?
❌

Breaks BC?
❌

Fixed issues
-

Closing since it can't be applied anymore.


Adding logs to instance provider create

Fixes #N/A
Description
Adding error logs for instance provider create call.
How was this change tested?
manually tested
Does this change impact docs?

[ ] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[x] No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

1 of 2 (50.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 82.873%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/providers/instance/instance.go
1
2
50.0%

Totals

Change from base Build<PHONE_NUMBER>0:
-0.01%

Covered Lines:
5642

Relevant Lines:
6808

💛  - Coveralls


e2e: cctx status is not Reverted

After https://github.com/zeta-chain/node/pull/2483, the upgrade tests have been failing at a high rate. It seems the light upgrade tests fail a bit more, but the regular upgrade tests also fail sometimes:
erc20      | [ERROR]
	Error Trace:     /go/delivery/zeta-node/e2e/utils/require.go:22
                                 /go/delivery/zeta-node/e2e/e2etests/test_erc20_deposit_refund.go:85
	Error:      	Not equal:
	            	expected: 5
	            	actual  : 6
	Messages:   	cctx status is not "Reverted" cctx index 0x7850b6987192a8e7a287405cf374ee4a598d0d20d51c8214fa16c526bc89c35a

erc20      | [ERROR]Test failed

I was able to to run the light upgrade tests on the prior commit 6 times without failure.
Running into some other errors too:
bitcoin    | ⏳running - testing Bitcoin ERC20 cross-chain swap
erc20      | ✅ completed in 1m36.259911982s - withdraw ERC20 from ZEVM
erc20      | ⏳running - withdraw ERC20 from ZEVM in multiple withdrawals
zeta       | ✅ completed in 1m41.307115252s - withdraw ZETA from ZEVM to Ethereum
zeta       | ⏳running - evm->evm message passing (sending ZETA only)
ether      | ✅ completed in 1m56.343918893s - withdraw Ether from ZEVM
ether      | ⏳running - tests sending ETH on ZEVM and check context data using ContextApp
ether      | ✅ completed in 9.079634056s - tests sending ETH on ZEVM and check context data using ContextApp
ether      | ⏳running - deposit ZRC20 into ZEVM and call a contract
bitcoin    | [ERROR]
	Error Trace:	/go/delivery/zeta-node/e2e/utils/require.go:22
	            				/go/delivery/zeta-node/e2e/e2etests/test_crosschain_swap.go:123
	Error:      	Not equal:
	            	expected: 3
	            	actual  : 5
	Messages:   	cctx status is not "OutboundMined" cctx index 0xc8d986cf36fa336619b08f475a05a36eac8aad3b300dc4e532cdc2196648adb2

bitcoin    | [ERROR]Test failed

The priority fee is not used yet so there shouldn't be any logic influencing the cctx workflow. Having some look into it.


Switch to cquery instead of query

@aiuto, thanks so much for comaintanence on this project. It's really put a burst of wind into my sails! I can't tell you how excited I am to have you help.
Thanks for the review!
Fixes #305

You'll have to give me a day or two. This looks like something I am going to have to actually read carefully

@aiuto friendly ping

I whacked on the settings yesterday.  I think you will be able to merge this.

@aiuto thanks for the review!


Add b2sum support to hashsum

GNU coreutils now has b2sum, so it needs to be added to hashsum.

Hey, I want to contribute to this. Can I be assigned to this task?

Sure, nobody is working on it currently.
Please make sure that you have tests too!

This was resolved in the referenced PR above.


Docker Cleanup

sudo should only be used on docker commands if needed
COPY requires the directories to exist. I think this complicates the Dockerfile with no real benefit.

Let me know if you have any questions.

I haven't seen any Dockerfiles with caching like this. Curious if it would be better to do this caching outside the Dockefile would be best?
Maybe something like? https://github.com/jpetazzo/squid-in-a-can This would help maintainers and maybe make contributors feel more familiar?

sudo doesn't work on a mac with boot2docker. This a why using just a Dockerfile and not a shell script to run docker is a great idea.
COPY requires the directories to exist

@nicgrayson

It's clear now, thank you for clarification. Will incorporate your commit with skipping sudo;
This command COPY .docker/apt-proxy* /tmp/apt-proxy doesn't require directory/file to exist, so should work without existing apt-proxy file.

@nicgrayson Merged your patch that skips sudo for docker. I guess this PR could be closed for now?

@sherzberg Thanks for the point. Need to think about that. But probably the changes would not be in the scope of the current PR.


Jwilaby/tech debt buydown

Fixes ngrok and buys down tech debt
Fixes the app settings form inputs
Fixes the incorrect ngrok reporting in the log panel

Pull Request Test Coverage Report for Build 1976

106 of 125 (84.8%)  changed or added relevant lines in 5 files are covered.
203 unchanged lines in 12 files lost coverage.
Overall coverage decreased (-1.1%) to 49.258%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/app/main/src/emulator.ts
0
1
0.0%

packages/app/main/src/ngrokService.ts
23
28
82.14%

packages/app/main/src/ngrok.ts
64
77
83.12%

Files with Coverage Reduction
New Missed Lines
%

packages/app/main/src/constants.ts
1
0.0%

packages/app/main/src/telemetry/telemetryService.ts
1
81.48%

packages/app/client/src/ui/editor/appSettingsEditor/appSettingsEditorConainer.ts
1
88.89%

packages/app/main/src/utils/getDirectories.ts
1
25.0%

packages/app/client/src/ui/editor/appSettingsEditor/appSettingsEditor.tsx
1
83.87%

packages/app/main/src/utils/sendNotificationToClient.ts
3
0.0%

packages/app/main/src/utils/ensureStoragePath.ts
5
25.0%

packages/app/main/src/utils/getBotsFromDisk.ts
5
7.14%

packages/app/main/src/ngrokService.ts
6
70.43%

packages/app/main/src/extensions.ts
21
0.0%

Totals

Change from base Build 1974:
-1.1%

Covered Lines:
4409

Relevant Lines:
8341

💛  - Coveralls

@justinwilaby let's merge this after a manual test with @tomlm 's Teams bot


do not select textareas without formatters

This is actually deliberate behaviour. I considered removing the link for textareas without formatters, but then I decided it doesn't hurt to be able to preview the contents. There might still be raw HTML formatting that you want to preview.
I'm open to a counterargument though, if you feel it's more beneficial to remove the link from such textareas.

This is actually deliberate behaviour. I considered removing the link for textareas without formatters, but then I decided it doesn't hurt to be able to preview the contents. There might still be raw HTML formatting that you want to preview.
I'm open to a counterargument though, if you feel it's more beneficial to remove the link from such textareas.

if there's any rationalization behind it all, that's fine by me.  i just removed it for a particular project because certain pages contain quite a few textareas and the number of links became slightly overwhelming, so i thought i would share it in the event that this was a case unforseen.  i'm not at all opposed to the functionality remaining intact, because your argument is very valid.

Haha. Well let's put it this way ... it took me a moment to recall that I had deliberately left it in there. Thanks for pointing it out.


Product, Product Link and Utility Links blocks need UI enhancements

All three blocks in the Store have acceptable add/edit forms, but these could do with some tidying.
Both the Product and Product Link blocks may benefit from some tabs, or at least two columns.
(I dislike long blocks that have to scroll to show options).
Some of the option labels could also be improved.

the product block has some code commented out (line 128 of form.php) that is not commented out properly so some PHP code is showing on the edit screen.
Can that be simply deleted or do you need it for future reference?

That can be deleted I think. I commented it out as I didn't think they'd ever be a need to list product groups against products that way. Pretty easy to add back in.

Other question: those screens are not that big. Wouldn't making the edit screen wider make it look better and less cramped rather than using tabs for a small number of options?

@mnakalay Making them wider would totally help. I'm not a big fan of tabs, especially when there might only be two of them.

Addressed by #43


I want to set placeholderImage,but that func didn't work

the version 3.2.0
I already wrote " import AlamofireImage" into the VC,
but "imageView.af_setImage(withURL: url, placeholderImage: placeholderImage)" didn't work
but the way, I can't find this func in your project.

headView.af_setImage(withURL: kImageUrl("xxxx"), placeholderImage: kProductDefault, imageTransition: nil)

Sorry, we use our GitHub project for bug reports and feature requests. In the future, you should open questions like this on Stack Overflow and tag alamofireimage.
Cheers. 🍻

From our Contribution Guidelines
Asking Questions
We don't use GitHub as a support forum. For any usage questions that are not specific to the project itself, please ask on Stack Overflow instead. By doing so, you'll be more likely to quickly solve your problem, and you'll allow anyone else with the same question to find the answer. This also allows maintainers to focus on improving the project for others.

get it,thanks a lot...