Robot controller error due to Vulkan crash

Hi there,
I have just installed the benchbot successfully on a machine with RTX2080 8GB, 32GB ram, i7-9700K CPU.
But when i tried to run
benchbot_run --robot carter --env miniroom:1 --task semantic_slam:passive:ground_truth
I keep getting a robot controller error. (small snippet below and the full log in the file attached.)
I'm wondering if you guys ever encountered this.
Thanks!
Chen.

...
Supervisor is now available @ 'http://<IP_ADDRESS>:10000' ...
Waiting until a robot controller is found @ 'http://benchbot_robot:10000' ...
Found
Sending environment data & robot config to controller ...
Ready
################################################################################
####################### BENCHBOT ROBOT CONTROLLER ERROR ########################
################################################################################
ERROR: The BenchBot Robot Controller container has exited unexpectedly. This
should not happen under normal operating conditions. Please see the complete
log below for a dump of the crash output:
Robot controller is now available @ 'http://<IP_ADDRESS>:10000' ...
Waiting to receive valid config data...
<IP_ADDRESS> - - [2021-03-22 15:04:04] "GET // HTTP/1.1" 200 152 0.000542
<IP_ADDRESS> - - [2021-03-22 15:04:05] "POST //configure HTTP/1.1" 200 137 0.066839
Starting the requested real robot ROS stack ...
THE PROCESS STARTED BY THE FOLLOWING COMMAND HAS CRASHED:
sed -i "0,/"pose":/{s/("pose": )(.)/\1[0.7, 0, 0, -0.7, 1.2, 1.5, 0.3]/}" /benchbot/isaac_sdk/apps/carter/carter_sim/bridge_config/carter_full_config.json && perl -0777 -i -pe 's/"static_mesh".?]/"static_mesh":[{"name": "bottle"}, {"name": "cup"}, {"name": "knife"}, {"name": "bowl"}, {"name": "wine glass"}, {"name": "fork"}, {"name": "spoon"}, {"name": "banana"}, {"name": "apple"}, {"name": "orange"}, {"name": "cake"}, {"name": "potted plant"}, {"name": "mouse"}, {"name": "keyboard"}, {"name": "laptop"}, {"name": "cell phone"}, {"name": "book"}, {"name": "clock"}, {"name": "chair"}, {"name": "table"}, {"name": "couch"}, {"name": "bed"}, {"name": "toilet"}, {"name": "tv"}, {"name": "microwave"}, {"name": "toaster"}, {"name": "refrigerator"}, {"name": "oven"}, {"name": "sink"}, {"name": "person"}]/s' /benchbot/isaac_sdk/apps/carter/carter_sim/bridge_config/carter_full_config.json && cd "/benchbot/addons/benchbot_addons/benchbot-addons/envs_isaac_develop/environments" && .sim_package/IsaacSimProject.sh "/Game/AI_vol3_03_base/Maps/AI_vol3_scene_03" -isaac_sim_config_json= "/benchbot/isaac_sdk/apps/carter/carter_sim/bridge_config/carter_full.json" -windowed -ResX=960 -ResY=540 -vulkan -game
...
log.txt

Thanks for reporting @cheneeheng .
We've seen this arbitrary Segmentation fault (core dumped) issue occur before when running on a machines with non-standard graphics configuration. Unfortunately it comes from Isaac Sim and doesn't give us much detail regarding the cause (everything that says "Error" in that log is part of a normal working run...).
Can we confirm how you are using the machine specified:

Are you directly on the machine, and does it have a physical screen attached?
Are you connecting via SSH with window forwarding?
Are you using other remote software like remote desktop or alternatives?

Hi @btalb,
aiks that does not sound good.

But what do you mean by non-standard ?
Setup other than the ones mentioned in the prerequisite ?
What did you do the last time you encountered this issue?

As for your questions:
1. Are you directly on the machine, and does it have a physical screen attached?
Yes and yes.
2. Are you connecting via SSH with window forwarding?
No.
3. Are you using other remote software like remote desktop or alternatives?
No. Although the plan is to do so once everything is running.
Thanks !

The core of the issue is Vulkan only seems to be happy when it is using a discrete GPU to render to a physical screen.
The reason I ask all of those questions is that we have had these issues when using configurations that tamper with that relationship. For example:

I have a  working system next to me (see log_success.txt),
but when I instead SSH into it with window forwarding I get the same SegFault (see log_failure.txt)

I can see Vulkan is the cause as I see the following extra lines in the failure when I diff the logs:
[2021.03.23-20.25.55:954][  0]LogRHI: Warning: Failed to find entry point for vkDestroySurfaceKHR
[2021.03.23-20.25.55:954][  0]LogRHI: Warning: Failed to find entry point for vkGetPhysicalDeviceSurfaceSupportKHR
[2021.03.23-20.25.55:954][  0]LogRHI: Warning: Failed to find entry point for vkGetPhysicalDeviceSurfaceCapabilitiesKHR
[2021.03.23-20.25.55:954][  0]LogRHI: Warning: Failed to find entry point for vkGetPhysicalDeviceSurfaceFormatsKHR
[2021.03.23-20.25.55:954][  0]LogRHI: Warning: Failed to find entry point for vkGetPhysicalDeviceSurfacePresentModesKHR
[2021.03.23-20.25.55:959][  0]LogLinux: Warning: MessageBox: Failed to find all required Vulkan entry points! Try updating your driver.: No Vulkan entry points found!:

I can also see those lines in the log you provided me. We need to dig a little deeper though to try and figure out why Vulkan is throwing those errors for the simulator:

Run a barebones Vulkan command to show a spinning cube:

https://user-images.githubusercontent.com/3508780/112221489-38fa7d80-8c73-11eb-918a-2b35820bdae0.mp4
Here's the command you need:
docker run -e DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix --rm --gpus all -it benchbot/backend:base /bin/bash -c 'vkcube'

If that fails, can you show me the output of the following diagnostic command for Vulkan:

docker run -e DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix --rm --gpus all -it benchbot/backend:base /bin/bash -c 'vulkaninfo'

Thanks; I wish these things were simpler...

I shall try them out once i get back to the lab on friday.
I regularly work with programs using CUDA, so this is not the worst I have seen :smiley:

Hi,
I encountered the same error log as @cheneeheng today when I tried to run
benchbot_run --robot carter --env miniroom:1 --task semantic_slam:passive:ground_truth
Last week when I run this command, there's no error but just no simulator window coming out after this:

Supervisor is now available @ 'http://<IP_ADDRESS>:10000' ...
Waiting until a robot controller is found @ 'http://benchbot_robot:10000' ...
Found
Sending environment data & robot config to controller ...
Ready

I tried this command as suggested:
docker run -e DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix --rm --gpus all -it benchbot/backend:base /bin/bash -c 'vkcube
unfortunately I get this error

'Cannot find a compatible Vulkan installable client driver (ICD)'

I am using a x2goclient via SSH to a remote machine with RTX 3080. I wonder if it has anything to do with the remote desktop that the simulator window doesn't appear?
Thanks in advance!

Thanks for the information @tyou1 .
The behaviour you experienced last week with the window showing up and disappearing was a bug which was hiding the error log. We weren't correctly bubbling the crash log up to benchbot_run's stdout. That's been fixed in v2.1 (benchbot_run --version), so you should always see the crash log now when the simulator crashes.
What's important to understand with any remote access systems is how they actually perform the rendering. I don't know much about x2go, but here's some examples I know of where a laptop is the client and the GPU machine is the server:

SSH with X forwarding: this passes the X rendering commands to your client machine (i.e. if I SSH into my GPU server from my laptop, my laptop does the rendering)
VNC servers & clients: this forwards whatever the server renders on its screen to your client, which just shows an image of what's been rendered by the server
Remote desktop systems with virtual screens: these often create virtual X servers & forward their contents to the client. Where things get tricky is the nvidia driver can only be tied to a single X server & can't be swapped without killing the entire X server. So when you turn your computer on & sign in, your GPU is locked away in that default X server and can't be used by the virtual X server created for remote desktops. TL;DR: remote desktop software almost never provides hardware-accelerated rendering, and my guess is x2go falls in this bucket

How does this tie in with BenchBot? NVIDIA's Isaac Simulator relies on hardware-accelerated rendering powered by Vulkan. If the system doing the rendering doesn't meet those requirements, then we get a crash from the simulator (a crash that should be much more verbose & explicit.... but a crash nonetheless).
So from those requirements, there's only a couple of solutions I would expect to work for cases where your GPU is on a remote machine:

VNC: setup a VNC server on your GPU machine & connect to it via a VNC client. Make sure the VNC server is attached to the same X server as your GPU. This is the easiest option to use BenchBot on a remote machine, and is extremely simple if the remote machine has a physical screen attached (@david2611 uses a solution like this one all the time).
SSH with tweaked X forwarding: you essentially tell benchbot_run to render the simulator on your GPU machine's screen, not your laptop's. The downside is you won't see the simulator remotely on your laptop, but it will run successfully (I use this solution all the time). To tell benchbot_run where to render you simply adjust the DISPLAY environment variable. For example, terminals opened on my GPU machine show this:ben@gpu-machine:~$ echo $DISPLAY
:1

Then when I'm SSHing from home, I manually set the DISPLAY target to :1 via:ben@home-machine:~$ ssh -X ben@gpu-machine
ben@gpu-machine:~$ echo $DISPLAY
localhost:10.0
ben@gpu-machine:~$ export DISPLAY=:1
ben@gpu-machine:~$ benchbot_run ...

Hope this helps. I know it's not an ideal solution, but hardware-accelerated rendering under Linux with Vulkan support is something that's traditionally caused enough challenges by itself. Crisp solutions for remote use on top of this unfortunately aren't quite there yet.
We're always interested in better solutions though. If anyone knows of better ways to enable remote hardware-accelerated rendering, especially on headless machines, we'd love to hear them. Unfortunately, it's not something I have time to dig too far into at the moment.

@btalb the vulkaninfo command is returning this error:
No protocol specified
WARNING: [Loader Message] Code 0 : loader_icd_scan: Can not find 'ICD' object in ICD JSON file /usr/share/vulkan/icd.d/nvidia_layers.json.  Skipping ICD JSON
error: XDG_RUNTIME_DIR not set in the environment.
No protocol specified
XCB failed to connect to the X server due to error:1.
ERROR at /build/vulkan-tools-<IP_ADDRESS>~rc1-1lunarg18.04/vulkaninfo/vulkaninfo.h:847: AppCreateXcbSurface failed to establish connection
Vulkan may not be correctly installed. Have you seen this before ?

That's excellent @cheneeheng, great to hear!
I'm not sure the relation of that series of errors (the first error I've never seen before, even before xhost local:root was added into the scripts). Did a reboot fix that error?
It's a little odd that line is causing issues with containers as it is running after all of the containers have started, so shouldn't effect them. But maybe there is some asynchronous behaviour causing race conditions. Thanks for pointing that out though, that's a really good find.
I'll close this issue here, but feel free to open a new issue with any outdated commands you find in the documentation / tutorials. I'm always keen to fix those when they're found. Unfortunately, I'm a little documentation blind by this point ;)

Reboot (x3) did not fix the error. Only the xhost command did.

Hi
May I ask where is this line xhost -local:root > /dev/nullis = true you mentioned that you commented out @cheneeheng? Unfortunately, I still have this crash log after I switch to Remmina to connect to the remote machine (RDP). So, I wonder if there is something else besides the remote access problem that causes the crash log.
Or is it only works with VNC server&client that the simulator window appears successfully? May I ask what specific VNC server & client that @david2611 use to run benchbot smoothly ?
Thanks a lot! :)

@tyou1
Here is the line : https://github.com/qcr/benchbot/blob/783e9ca27866ae64c891262aa5f46a7a53ee37c7/bin/benchbot_run#L347
You could try running xhost +local:root on terminal and run the vkcube command to check if the access rights is the problem. Also make sure the env DISPLAY is correctly set when you do this :smiley:

Hi @tyou1 , good question.
Only VNC will work as RDP generally creates a virtual X server which won't have the hardware accelerated rendering.
@david2611 uses NoMachine, just make sure it's not using a virtual screen.
There's plenty of simple VNC options out there also like:

TigerVNC (I've had succcess with this many years ago)
TightVNC
RealVNC
Xvnc
etc.

The crucial thing is just to make sure it is mirroring a physical screen, and not creating a virtual one.

Remmina also should be fine as a VNC client to conect to a server.


Use Media.xcassets for better images

Description
Use Media.xcassets for (most) images.
Notes
This PR does not change the UI. It changes the code base so that we start adopting Asset Catalogs, which allows for a smaller footprint (App Slicing) while we move toward providing 1x and 2x and 3x images for a more beautiful UI.
This also, helpfully, shows us which icons edX is missing for an optimally lovely UI. See Media.xcassets and tap through to see what's missing. The app will work fine without these sizes, but it'll look better with them.
How to test this PR
Run the test suite to verify that none of the screenshots have changed. This will confirm that the UI has not changed.
Reviewers
If you've been tagged for review, please "Start a review" with the Github GUI.

[ ] Code review: @BenjiLee

Thanks for the pull request, @jaimeohm! I've created OSPR-1544 to keep track of it in JIRA. JIRA is a place for product owners to prioritize feature reviews by the engineering development teams.
Feel free to add as much of the following information to the ticket:

supporting documentation
edx-code email threads
timeline information ("this must be merged by XX date", and why that is)
partner information ("this is a course on edx.org")
any other information that can help Product understand the context for the PR

All technical communication about the code itself will still be done via the GitHub pull request interface. As a reminder, our process documentation is here.
If you like, you can add yourself to the AUTHORS      file for this repo, though that isn't required.     Please see the CONTRIBUTING     file for more information.

@jaimeohm We removed .xcassets intentionally because they are not white-label friendly.
You can find more details here.

@saeedbashir Hmmm.. I looked at that JIRA issue and I don't see the reason they're not white-label friendly. I'm finding they're improving the quality of the two white-label projects I currently have in-progress. Could you tell me what wasn't working for you? Perhaps something has changed.

@saeedbashir Hmmm.. I looked at that JIRA issue and I don't see the reason they're not white-label friendly. I'm finding they're improving the quality of the two white-label projects I currently have in-progress. Could you tell me what wasn't working for you? Perhaps something has changed. cc @danialzahid94 @BenjiLee

@saeedbashir Hmmm.. I looked at that JIRA issue and I don't see the reason they're not white-label friendly. I'm finding they're improving the quality of the two white-label projects I currently have in-progress. Could you tell me what wasn't working for you? Perhaps something has changed. @BenjiLee @danialzahid94

@jaimeohm Having all resources centralized at one locations makes it simpler and more efficient to run a single script to replace current files with the required resources. This makes white labeling exponentially easy.

@danialzahid94 Gotcha. Thank you for sharing your reasoning.


[hmac/dv] Verify save & restore

Tentative list of TODOs:

[ ] Extend testplan and coverplan to test and cover save & restore feature.
[ ] Enable save & restores in tests and make necessary extensions to the scoreboard as well as burst_wr_msg so this gets correctly checked and fully covered.
[ ] Add additional directed test that covers context switching between multiple message streams (not sure if this has to be an UVM test, maybe a SW test would suffice?).

@moidx on track according to reports we got today. @andreaskurth agree, TLT in cryptolib is passing now and we made progress in block-level DV. @moidx more TLTs to follow over the next days.

The current issue can be closed once this PR is merged

https://github.com/lowRISC/opentitan/pull/23116

as the following issue is overtaking the remaining DV work for M5:

https://github.com/lowRISC/opentitan/issues/23562

FYI @vogelpi / @johannheyszl

Moving this issue to M5, but lets continue to prioritize this to land it by the end of the week if possible.

Closed as https://github.com/lowRISC/opentitan/pull/23116 has been merged


Investigate places affected by OSX App Translocation or snap upgrade

OSX since High Sierra introduced the App Translocation feature. Due to this feature, launching applications that aren't moved after downloaded or unarchived will be intercepted by the system and the system will make the application think it launches from a random location. That random location will change after the computer reboots. We should investigate if we are storing any local paths and fix them to be using relative paths or add validations before using them because they might be invalid after the user reboots the computer.
Snap may have the same issue if the user upgrades from there. The path of the snap app look like "/snap/storage-explorer/some_identifier/......". Each identifier maps to a version of the snap installed. If the user upgrades, the path will change and the referenced file may no longer exist.

[ ] Tab session restoration


[Improvement]: Test Double approach doesn't work as expected for function calls with path parameters

Description
Trying to create a test double for githubRestClient in this example. Since the example include the function call with path parameters(self.githubRestClient->/users/[owner]/repos;), I have used the following approach with appropriate return values.
final http:Client githubRestClient = check initRestClient();
function initRestClient() returns http:Client|error => check new ("https://api.github.com", {auth: {token: authToken}});

public client class MockHttpClient {

    remote function get(string path, map<string|string[]>? headers = (), http:TargetType targetType = http:Response) returns http:Response|anydata|http:ClientError {
    }

    resource function get [string... path](map<string|string[]>? headers = (), http:TargetType targetType = http:Response, *http:QueryParams params) returns http:Response|anydata {
    }

    remote function post (string path, http:RequestMessage message, map<string|string[]>? headers = (), string? mediaType = (), http:TargetType targetType = http:Response) returns http:Response|anydata|http:ClientError {
    }
}

@test:Mock {
    functionName: "initRestClient"
}
function initMockRestClient() returns http:Client|error {
    return test:mock(http:Client, new MockHttpClient());
}

When running the bal test command, it returns error: there are test failures.
Describe your problem(s)
No response
Describe your solution(s)
No response
Related area
-> Test Framework
Related issue(s) (optional)
No response
Suggested label(s) (optional)
No response
Suggested assignee(s) (optional)
No response

Update:

The initial issue in the above-defined test double was a return type mismatch. Identified an issue in the use of runtime generated main method for multi-module packages. Created an issue https://github.com/ballerina-platform/ballerina-lang/issues/40934

Next, if we are expecting multiple responses for each path param we need to handle each case within the mock double. Check the following example.

public client class MockHttpClient {

    isolated resource function get [http:PathParamType... path](map<string|string[]>? headers = (), http:TargetType targetType = http:Response,
            *http:QueryParams params) returns http:Response|anydata|http:ClientError {
        GitHubRepository repo = {
            id: 1,
            name: "Test-Repo",
            'fork: false,
            created_at: "2020-10-10T10:10:10Z",
            updated_at: "2020-10-10T10:10:10Z",
            language: "Ballerina",
            has_issues: false,
            forks_count: 5,
            open_issues_count: 9,
            visibility: "Public",
            forks: 7,
            open_issues: 3,
            watchers: 55,
            default_branch: "master"
        };
        if path.toString().startsWith("/user") {
            GitHubUser user = {
                id: 1233,
                login: "balUser",
                url: "https://api.github.com/users/balUser",
                created_at: "2020-10-10T10:10:10Z",
                updated_at: "2020-10-10T10:10:10Z",
                avatar_url: "https://avatars0.githubusercontent.com/u/1233?v=4",
                'type: "User",
                blog: "https://ballerina.io",
                company: "WSO2",
                name: "Ballerina User",
                location: "Sri Lanka"
            };
            return user;
        }
        if path.toString().startsWith("/users") && path.toString().endsWith("/repos") {
            return [repo];
        }
        if path.toString().startsWith("/repos") {
            if path.toString().endsWith("/issues") {
                Issue issue = {
                    number: 123
                };
                return issue;
            }
            return repo;
        }
        return error("error while mocking the response");
    }

    remote function post(string path, http:RequestMessage message, map<string|string[]>? headers = (), string? mediaType = (), http:TargetType targetType = http:Response)
            returns http:Response|anydata|http:ClientError {
        GitHubRepository expectedResult = {
            id: 1,
            name: "Test-Repo",
            'fork: false,
            created_at: "2020-10-10T10:10:10Z",
            updated_at: "2020-10-10T10:10:10Z",
            language: "Ballerina",
            has_issues: false,
            forks_count: 5,
            open_issues_count: 9,
            visibility: "Public",
            forks: 7,
            open_issues: 3,
            watchers: 55,
            default_branch: "master"
        };
        return expectedResult;
    }

}

Based on above, it is  error handling that is not working as expected and will be fixed with https://github.com/ballerina-platform/ballerina-lang/issues/40934


Username and Password in plaintext (Post Data)

Hi everyone;
I was profiling Serene and I've noticed that Usrname and Password are displayed in plaintext. Is there a way to prevent this or switching to HTTPS Protocol will be enough?

Excuse my paranoia:) I'm new to web development  and I wanna make sure the app is secure enough.

Yes you should enable HTTPS if you care about sniffing.

Thanks volkan. I'll switch to HTTPS when the website is ready for production. Have a good day.


The preferred write region is missing in the screenshot.

The preferred write region is missing in the screenshot.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 05af42e7-9d86-b794-e133-efbd46665b9c
Version Independent ID: 0af5eead-7620-ea77-efad-c9ec595fae4f
Content: Connect a MongoDB application to Azure Cosmos DB
Content Source: articles/cosmos-db/connect-mongodb-account.md
Service: cosmos-db
Sub-service: cosmosdb-mongo
GitHub Login: @timsander1
Microsoft Alias: tisande

@MaxMelcher We are checking this and will get back to you.

@MaxMelcher Please elaborate your feedback. I am seeing the screenshots same as in Azure Portal for this document

https://docs.microsoft.com/en-us/azure/cosmos-db/connect-mongodb-account?source=docs

When you say the preferred write region is missing, please provide a screenshot for comparison.

@MaxMelcher Please get back to us with more information regarding this issue.

@MaxMelcher At this time we are going ahead and closing this issue. Please get back to us with added information by raising a new issue to work on.


Cleanup misspelling and some format things

s/projest/project
s/Smetimes/Sometimes
add relative to the authorDateRelative command
remove the from a few commands (since the rest of the commands list doesn't use the)
reword short form to be easier to read

Hope this is helpful!

Thanks for your contribution @chaseadamsio. Great job :+1:.


Doc 'type' is optional in mget

In elasticsearch, Doc 'type' is optional in mget request. According to elastic documentation, If you don’t set the type and have many documents sharing the same _id, you will end up getting only the first matching document.
See https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html#mget-type

Coverage increased (+0.01%) to 54.505% when pulling f8fc99bccd46f23e2591fe663672ad48935199ef on logzio:master into f34fa45e34c9230d58069f8f3e9b86cabfabf603 on searchbox-io:master.

Hi @kramer ,
Can you please take a look at this PR?
This is a minor change in order to allow calling _mget without providing the document 'type', as possible and supported in Elasticsearch.
Thanks

Hi @ferhatsb,
Can you please take a look at this PR?
This is a minor change in order to allow calling _mget without providing the document 'type', as possible and supported in Elasticsearch.
Thanks

thanks!


session can be undefined

I have this error a lot

Humm I had it with another tool of mine which had previous 0.2.0 chromecast-player. I was unable to reproduce the bug with the 0.2.0.

Hmm, I think the callback should get invoked with an 'session not found'-error in that case.

Sure, I dont know well the internal of your library, it was easiest to me to show with an example what error I had and which trick saved me at the moment. But I'm not sure this is still relevant as I cannot reproduce it on 0.2.0.
I had one program with < 0.2.0 and one with 0.2.0. The first with old chromecast-player put it on a weird state if I launched 2 commands in a row, like described in #5 And the second show me this log each time I was using it when I though the 1st one crashed.
GET /javascripts/app.js 200 2.297 ms - 1391

/home/jsteunou/Git/castnow-webui/node_modules/chromecast-player/api.js:119
    var sessionId = session.mediaSessionId;
                           ^
TypeError: Cannot read property 'mediaSessionId' of undefined
    at /home/g-rom/Git/castnow-webui/node_modules/chromecast-player/api.js:119:28
    at /home/g-rom/Git/castnow-webui/node_modules/chromecast-player/api.js:109:5
    at /home/g-rom/Git/castnow-webui/node_modules/chromecast-player/api.js:63:7
    at RequestResponseController.onmessage (/home/g-rom/Git/castnow-webui/node_modules/chromecast-player/node_modules/castv2-client/lib/controllers/request-response.js:27:7)
    at RequestResponseController.emit (events.js:117:20)
    at Channel.onmessage (/home/g-rom/Git/castnow-webui/node_modules/chromecast-player/node_modules/castv2-client/lib/controllers/controller.js:16:10)
    at Channel.emit (events.js:98:17)
    at Client.onmessage (/home/g-rom/Git/castnow-webui/node_modules/chromecast-player/node_modules/castv2-client/node_modules/castv2/lib/channel.js:23:10)
    at Client.emit (events.js:117:20)
    at PacketStreamWrapper.onpacket (/home/g-rom/Git/castnow-webui/node_modules/chromecast-player/node_modules/castv2-client/node_modules/castv2/lib/client.js:79:10)
Program node ./bin/www exited with code 8

Okay, I can't really tell you why this error occurs at the moment, but if you change your PR to
if (!session) return cb(new Error('session not found')); I'll gladly merge it.

and done ;)

thanks :)


home-manager: 0-unstable-2024-05-23 -> 0-unstable-2024-05-30

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for home-manager is: A Nix-based user environment configurator
meta.homepage for home-manager is: https://nix-community.github.io/home-manager/
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0-unstable-2024-05-30 in filename of file in /nix/store/sdk4skbj5g9mp08jf118hnl7zigxj9zw-home-manager-0-unstable-2024-05-30

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

home-manager

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/sdk4skbj5g9mp08jf118hnl7zigxj9zw-home-manager-0-unstable-2024-05-30 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A home-manager https://github.com/r-ryantm/nixpkgs/archive/8f3d1f9d4ac56acb932fa92ce68c59102a58bcdf.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/8f3d1f9d4ac56acb932fa92ce68c59102a58bcdf#home-manager

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/sdk4skbj5g9mp08jf118hnl7zigxj9zw-home-manager-0-unstable-2024-05-30
ls -la /nix/store/sdk4skbj5g9mp08jf118hnl7zigxj9zw-home-manager-0-unstable-2024-05-30/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

1 package built:

home-manager

Maintainer pings
cc @AndersonTorres for testing.

@NixOS/nixpkgs-merge-bot merge

@NixOS/nixpkgs-merge-bot merge


Add settings for debugging compilations

F5 debugging is possible by creating the file to be compiled under the "sample-for-debug" directory and entering that file when debugging.

@uekann What VSCode extension or software do I need for debugging? I cannot debug even though I have lanza.lldb-vscode and llvm installed. VSCode says "Configured debug type 'lldb' is not supported".

I'm using the extension "vadimcn.vscode-lldb".
I've added it to extensions.json and committed it.

Thanks! I succeeded in debugging.


When I turned on comments, because the img tag was dynamically added, the invert did not take effect

Without using JS, CSS can be solved perfectly?
If anyone else encounters similar problems, welcome to discuss together

:root {
  --bg-color: white;
  --text-color: black;
}

@media (prefers-color-scheme: light) {
  :root {
    --bg-color: white;
    --text-color: black;
  }
}

@media (prefers-color-scheme: dark) {
  :root {
    --bg-color: black;
    --text-color: white;
  }

  a {
    color: yellow;
  }
}

// bg color is also needed in html in order to
// block body's background propagation
// see: https://stackoverflow.com/a/61265706
html,
body {
  background: var(--bg-color);
}

body {
  color: var(--text-color);
  font-family: monospace;
  font-size: 16px;
  line-height: 1.4;
  margin: 0;
  min-height: 100%;
  overflow-wrap: break-word;
}

here is my new version without invert

It can also solve the problem of YouTube videos being invented

Sorry for the late response. Got inspired by your version and pushed a new commit to solve this issue. Thanks!


feat(awsim_sensor_kit_launch): awsim load concatenate node parameters

Description
PR https://github.com/autowarefoundation/autoware.universe/pull/8300 introduces a new concatenate algorithm and also removes the default parameter, thus the launcher would need to load the parameter from the parameter file.

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[ ] I've confirmed the contribution guidelines.
[ ] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Thank you. I will confirm e2e_simulator.

@SakodaShintaro
I see, I understand your concern.
I think you can test with the timeout sec: 0.05 or even lower in the simulation environment.

@SakodaShintaro
You are right :)
The parameters are moved in this PR https://github.com/autowarefoundation/autoware_individual_params/pull/54.
Could you help test this PR with the individual PR together on the awsim :)
Thanks for your support!

@SakodaShintaro
Sorry that the changes are not necessary, so I reverted the changes.

@vividf
I merged the two PRs

https://github.com/autowarefoundation/autoware.universe/pull/8300
https://github.com/tier4/awsim_sensor_kit_launch/pull/25

into autoware:main on my machine and ran e2e_simulator with AWSIM, which has 3 lidars (top, left, and right).
However, Autoware could not engage due to LiDAR flickering.
https://github.com/user-attachments/assets/b57ab1ff-c59b-47e1-9008-5bd40707cbd7
The multi LiDAR AWSIM is available to TIER IV employees only. link
Could you please confirm this?


1480 | Running Sum of 1d Array | Java

Running Sum of 1d Array
What will change
Create a solution in Java for this problem
Type of Issue
Please add/delete options that are not relevant.

[x] Adding New Code

Programming Language
Please add/delete options that are not relevant.

[x] Java

Self Check

Ask for issue assignment and wait to get assigned to it before making Pull Request.
Add your file in the proper folder
Clean Code and Documentation for better readability
Add Title and Description of the program in the file

:star2: Star it :fork_and_knife:Fork it :handshake: Contribute to it!
Discord server ✉ - https://discord.gg/CXksGdn
Happy Coding,

Assign that to me please

May I be assigned this issue please?


Port "cpustates" output processor to WA3

Related to: https://github.com/ARM-software/workload-automation/issues/160

Related to: https://github.com/ARM-software/workload-automation/issues/416


SimpleRegressor KeyError: 'poisson' when running boston dataset

Describe the bug
KeyError: 'poisson' when running boston dataset with the SimpleRegressor. When testing the boston dataset with the SimpleRegressor there's a KeyError. Removing poisson  from the "DecisionTreeRegressor" in the param_grid remedies the issue. I think #24 will fix this though? I'm also logging this so I can implement the __str__ and __repr__ methods in for the SimpleRegressor class in #52.
Go to the python shell and run the following:
from sklearn.datasets import load_boston
from simple_learn.regressors import SimpleRegressor
boston = load_boston()
true_x = boston.data
true_y = boston.target
clf = SimpleRegressor()
clf.fit(true_x, true_y)
Expected behavior
Expected the classifier to fit.
Desktop (please complete the following information):

OS: Windows 10

Additional context
>>> from sklearn.datasets import load_boston
>>> from simple_learn.regressors import SimpleRegressor
>>> boston = load_boston()
>>> true_x = boston.data
>>> true_y = boston.target
>>> clf = SimpleRegressor()
>>> clf.fit(true_x, true_y)
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\externals\loky\process_executor.py", line 431, in _process_worker
    r = call_item()
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\externals\loky\process_executor.py", line 285, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 262, in __call__
    return [func(*args, **kwargs)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 262, in <listcomp>
    return [func(*args, **kwargs)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_validation.py", line 531, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\tree\_classes.py", line 1242, in fit
    super().fit(
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\tree\_classes.py", line 336, in fit
    criterion = CRITERIA_REG[self.criterion](self.n_outputs_,
KeyError: 'poisson'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Aidan\simple_learn\simple_learn\regressors\simple_regressor.py", line 115, in fit
    grid_clf.fit(train_x, train_y)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\utils\validation.py", line 72, in inner_f
    return f(**kwargs)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_search.py", line 736, in fit
    self._run_search(evaluate_candidates)
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_search.py", line 1188, in _run_search
    evaluate_candidates(ParameterGrid(self.param_grid))
  File "C:\Users\Aidan\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_search.py", line 708, in evaluate_candidates

When removing poisson from the DecisionTreeRegressor in param_grid. This is the output.
>>> from sklearn.datasets import load_boston
>>> from simple_learn.regressors import SimpleRegressor
>>> boston = load_boston()
>>> true_x = boston.data
>>> true_y = boston.target
>>> clf = SimpleRegressor()
>>> clf.fit(true_x, true_y)
>>> clf
<simple_learn.regressors.simple_regressor.SimpleRegressor object at 0x000001E0145966A0>
>>> print(clf)
{
    "Type": "GradientBoostingRegressor",
    "Training Duration": "0.019023418426513672s",
    "GridSearch Duration": "47.94800543785095s",
    "Parameters": {
        "criterion": "mse",
        "learning_rate": 0.1,
        "loss": "ls",
        "max_depth": 3,
        "max_features": "sqrt",
        "n_estimators": 50
    },
    "Metrics": {
        "Training Score": 4.627489012405178,
        "Mean Absolute Error": 1.6081269523286306,
        "Mean Square Error": 4.232804375535621,
        "R-Squared": 0.9498599072505728
    }
}

Hi @Turtle24, good catch. Although it is not a complete error, this does point out the issue that the try/except block does not catch all errors.
https://github.com/skekre98/simple_learn/blob/39f1674ebee90f2bc5df36b4c933a2021addeae4/simple_learn/regressors/simple_regressor.py#L114-L119
I'd suggest using BaseException in the except statement: docs. It would end up looking something like this:
try:
    # gridsearch
except BaseException as error:
    self.failed_models.append(model_name)
    self.logger.warning(f"{model_name} failed, Error : {error}")
    continue

We would like to let users know that a certain model has failed and why because it would let them know more about their data. Would you be interested in taking up this issue?

Hi @Turtle24, good catch. Although it is not a complete error, this does point out the issue that the try/except block does not catch all errors.
https://github.com/skekre98/simple_learn/blob/39f1674ebee90f2bc5df36b4c933a2021addeae4/simple_learn/regressors/simple_regressor.py#L114-L119
I'd suggest using BaseException in the except statement: docs. It would end up looking something like this:
try:
    # gridsearch
except BaseException as error:
    self.failed_models.append(model_name)
    self.logger.warning(f"{model_name} failed, Error : {error}")
    continue

We would like to let users know that a certain model has failed and why because it would let them know more about their data. Would you be interested in taking up this issue?

That makes a lot sense @skekre98. Yeah I'll gladly take it on. I should be finished with #52 quite soon. So I'll be able to jump over to this. I just removed poisson so I could test the SimpleRegressor methods.

That makes a lot sense @skekre98. Yeah I'll gladly take it on. I should be finished with #52 quite soon. So I'll be able to jump over to this. I just removed poisson so I could test the SimpleRegressor methods.

That makes a lot sense @skekre98. Yeah I'll gladly take it on. I should be finished with #52 quite soon. So I'll be able to jump over to this. I just removed poisson so I could test the SimpleRegressor methods.

Awesome :)

That makes a lot sense @skekre98. Yeah I'll gladly take it on. I should be finished with #52 quite soon. So I'll be able to jump over to this. I just removed poisson so I could test the SimpleRegressor methods.

Awesome :)

@skekre98 I added the BaseException as you suggested and it seems to be working. What do you think?
 try:
      grid_clf.fit(train_x, train_y)
except BaseException as error:
      self.failed_models.append(name)
      self.logger.warning(f"{name} failed, Error : {error}")
      continue

Poisson fails still but the next best model gets fitted.
>>> from sklearn.datasets import load_boston
>>> from simple_learn.regressors import SimpleRegressor
>>> boston = load_boston()
>>> true_x = boston.data
>>> true_y = boston.target
>>> clf = SimpleRegressor()
>>> clf.fit(true_x, true_y)
DecisionTreeRegressor failed, Error : 'poisson'
>>> clf
<simple_learn.regressors.simple_regressor.SimpleRegressor object at 0x0000022E4DF466A0>
>>> print(clf)
{
    "Type": "GradientBoostingRegressor",
    "Training Duration": "0.03603243827819824s",
    "GridSearch Duration": "46.36451077461243s",
    "Parameters": {
        "criterion": "mse",
        "learning_rate": 0.1,
        "loss": "ls",
        "max_depth": 3,
        "max_features": "sqrt",
        "n_estimators": 100
    },
    "Metrics": {
        "Training Score": 4.538821867258774,
        "Mean Absolute Error": 1.213331369076752,
        "Mean Square Error": 2.47434032484902,
        "R-Squared": 0.9706899628769452
    }
}
>>>

@skekre98 I added the BaseException as you suggested and it seems to be working. What do you think?
 try:
      grid_clf.fit(train_x, train_y)
except BaseException as error:
      self.failed_models.append(name)
      self.logger.warning(f"{name} failed, Error : {error}")
      continue

Poisson fails still but the next best model gets fitted.
>>> from sklearn.datasets import load_boston
>>> from simple_learn.regressors import SimpleRegressor
>>> boston = load_boston()
>>> true_x = boston.data
>>> true_y = boston.target
>>> clf = SimpleRegressor()
>>> clf.fit(true_x, true_y)
DecisionTreeRegressor failed, Error : 'poisson'
>>> clf
<simple_learn.regressors.simple_regressor.SimpleRegressor object at 0x0000022E4DF466A0>
>>> print(clf)
{
    "Type": "GradientBoostingRegressor",
    "Training Duration": "0.03603243827819824s",
    "GridSearch Duration": "46.36451077461243s",
    "Parameters": {
        "criterion": "mse",
        "learning_rate": 0.1,
        "loss": "ls",
        "max_depth": 3,
        "max_features": "sqrt",
        "n_estimators": 100
    },
    "Metrics": {
        "Training Score": 4.538821867258774,
        "Mean Absolute Error": 1.213331369076752,
        "Mean Square Error": 2.47434032484902,
        "R-Squared": 0.9706899628769452
    }
}
>>>

@skekre98 I added the BaseException as you suggested and it seems to be working. What do you think?
 try:
      grid_clf.fit(train_x, train_y)
except BaseException as error:
      self.failed_models.append(name)
      self.logger.warning(f"{name} failed, Error : {error}")
      continue

Poisson fails still but the next best model gets fitted.
>>> from sklearn.datasets import load_boston
>>> from simple_learn.regressors import SimpleRegressor
>>> boston = load_boston()
>>> true_x = boston.data
>>> true_y = boston.target
>>> clf = SimpleRegressor()
>>> clf.fit(true_x, true_y)
DecisionTreeRegressor failed, Error : 'poisson'
>>> clf
<simple_learn.regressors.simple_regressor.SimpleRegressor object at 0x0000022E4DF466A0>
>>> print(clf)
{
    "Type": "GradientBoostingRegressor",
    "Training Duration": "0.03603243827819824s",
    "GridSearch Duration": "46.36451077461243s",
    "Parameters": {
        "criterion": "mse",
        "learning_rate": 0.1,
        "loss": "ls",
        "max_depth": 3,
        "max_features": "sqrt",
        "n_estimators": 100
    },
    "Metrics": {
        "Training Score": 4.538821867258774,
        "Mean Absolute Error": 1.213331369076752,
        "Mean Square Error": 2.47434032484902,
        "R-Squared": 0.9706899628769452
    }
}
>>>

Awesome, looks good to me! On another note, shouldn't >>> clf be returning the same thing as >>> print(clf)?

@skekre98 I added the BaseException as you suggested and it seems to be working. What do you think?
 try:
      grid_clf.fit(train_x, train_y)
except BaseException as error:
      self.failed_models.append(name)
      self.logger.warning(f"{name} failed, Error : {error}")
      continue

Poisson fails still but the next best model gets fitted.
>>> from sklearn.datasets import load_boston
>>> from simple_learn.regressors import SimpleRegressor
>>> boston = load_boston()
>>> true_x = boston.data
>>> true_y = boston.target
>>> clf = SimpleRegressor()
>>> clf.fit(true_x, true_y)
DecisionTreeRegressor failed, Error : 'poisson'
>>> clf
<simple_learn.regressors.simple_regressor.SimpleRegressor object at 0x0000022E4DF466A0>
>>> print(clf)
{
    "Type": "GradientBoostingRegressor",
    "Training Duration": "0.03603243827819824s",
    "GridSearch Duration": "46.36451077461243s",
    "Parameters": {
        "criterion": "mse",
        "learning_rate": 0.1,
        "loss": "ls",
        "max_depth": 3,
        "max_features": "sqrt",
        "n_estimators": 100
    },
    "Metrics": {
        "Training Score": 4.538821867258774,
        "Mean Absolute Error": 1.213331369076752,
        "Mean Square Error": 2.47434032484902,
        "R-Squared": 0.9706899628769452
    }
}
>>>

Awesome, looks good to me! On another note, shouldn't >>> clf be returning the same thing as >>> print(clf)?

@skekre98 The repo I did this on was a commit behind because of the PR not being merged yet so you're absolutely right. I'll double check it and make sure it's working as intended.

@skekre98 The repo I did this on was a commit behind because of the PR not being merged yet so you're absolutely right. I'll double check it and make sure it's working as intended.


Use CONNECT method to contact Crawlera proxy

Scrapy can connect fine with Crawlera using CONNECT method.
I'm not 100% sure we need to have this on by default though. This could be switchable with a new CRAWLERA_USE_HTTP_CONNECT setting of some sort.

Codecov Report

Merging #44 into master will decrease coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master      #44     +/-   ##
=========================================
- Coverage   93.75%   93.65%   -0.1%
=========================================
  Files           2        2
  Lines         128      126      -2
=========================================
- Hits          120      118      -2
  Misses          8        8

Impacted Files
Coverage Δ

scrapy_crawlera/middleware.py
93.54% <ø> (-0.11%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26b82e3...4e11552. Read the comment docs.

I think we should make the use of CONNECT configurable, perhaps through a key in _settings (https://github.com/scrapy-plugins/scrapy-crawlera/blob/master/scrapy_crawlera/middleware.py#L24)?

@eLRuLL - Sorry for accidentally requesting review there. Github recommended you as a reviewer, but I misread the UI to mean that you had recommendations. :)

no problem @cathalgarvey I accept it btw 😄


malformed IPPool panic removal

Description

Remove panic when getting key of type IPPool when CIDR parse fails.
fixes https://github.com/projectcalico/libcalico-go/issues/820
Todos

[x] Tests

Release Note
None required

Signed-off-by: derek mcquay<EMAIL_ADDRESS>

You'll actually need this change in the appropriate Calico v2.x branch of libcalico-go.  As it stands, I don't think this will actually impact any code that is actually executed in the master branch (we have a lot of old code that we really should have removed by now).  I think it's fine having this change in master too, but just wanted to highlight this.
I believe an additional change is required in the KeyFromDefaultPath() method in the model/ippool.go file.  At the moment we are ignoring errors returned from the ParseCIDR call.  This code is executed when we list IPPools, and is used in the master codebase in the v2->v3 upgrade code.

@robbrockbank I asked @bcreane about which branch I should create this PR against and he mentioned that pushing to master and then cherry-picking the changes to the needed branches. Does that still seem like a valid idea given what these changes touch?


Feature Request: Support for private extensions

I would like there to be a feature where it is possible to download private extensions for the development environment in Visual Studio Code, it would help tech companies to improve resources privately for their developers in the company

you can manually send the .vsix file

The upcoming feature covered by the #208184 testplan may be relevant.

Dup of https://github.com/microsoft/vscode/issues/21839


Import failed?

[ ] I have read and agree to the contributing guidelines.

Describe the bug
A clear and concise description of what the bug is.
Tried to install it didn't see griptape nodes went back to the manager and got this error

To Reproduce
A minimal, reproducible code example.
try install from manager no worky for me
Expected behavior
A clear and concise description of what you expected to happen.
to have the griptape nodes
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS] Win10
Version [e.g. 0.5.1] 22H2

Additional context
Add any other context about the problem here.

Hello @serget2 - I've just pushed a new release that I believe will resolve this issue. It turns out there was still a requirement for OPENAI_API_KEY to be in your environment or the install would fail - I've taken care of that, and we shouldn't see the same install issue.
Please let me know if this resolves it for you!


model.pth fails to download correctly

Stats show
2024-02-29 08:18:10.509 | INFO     | xtts_webui:<module>:57 - Start loading model v2.0.2
[XTTS] Downloading config.json...
  0%| 0.00/68.0 [00:00<?, ?iB/s]
100%| 68.0/68.0 [00:00<00:00, 67.9kiB/s]
[XTTS] Downloading model.pth...
  0%| 0.00/68.0 [00:00<?, ?iB/s]
100%| 68.0/68.0 [00:00<?, ?iB/s]
[XTTS] Downloading vocab.json...
  0%| 0.00/68.0 [00:00<?, ?iB/s]
100%| 68.0/68.0 [00:00<00:00, 19.8kiB/s]

But the model.pth is only 1kb.
Then the model load fails and the script quits.

Ignore this one.  It was due to Hugging Face being down for maintenance.  Downloads work fine again now they are back up.

Ok, if you have any problems, post in this issue or open a new one.
Glad it's solved.


option to migrate despite errors

Something like the authentication type being WindowsAuthentication should not block the migration. For production it is not an option to disable this without a service window. Using something else like AAD auth in the web app would also require some code changes which one possibly will want to do on the migrated app.
Would be great if it is an option to "continue at own risk"

The ability to package (and migrate) a site even with a failed configuration check is now possible using the PowerShell scripts option (use "-MigrateSitesWithIssues" flag on Get-SitePackage.ps1 script) - the scripts are a lightweight option that involves simply dropping a directory of scripts on the machine (or at least a machine-accessible location) in order to try it out.
Main info page: https://github.com/Azure/App-Service-Migration-Assistant/wiki/PowerShell-Scripts
Place to download: https://azure.microsoft.com/en-us/services/app-service/migration-assistant/


Local Storage RTL Storage Shared Between Apps

Describe the bug
When having more than one window open, the current selected RTL/LTR direction is stored between sessions.
What is the expected behavior?
This info should not be stored in local storage.
What are the steps to reproduce?
Open two windows side-by-side, go to a component and toggle the RTL addon.
Screenshots or links to minimum reproduction example

Environment

Anything else to add?

Use session storage instead of local storage


Create Design Document for Inference Server v0.1 RSNA 2021

Create a design document for Inference Server v0.1 RSNA 2021.
The document should include (at least):

Choice of programming language
Choice of framework(s)
Mechanism for inference request handling

Detail data input and output formats

Mechanism for deployment
Mechanism for MAP selection/inclusion
Mechanism for error handling
Mechanism for logging

Choice of programming language: Python


Allow compilation with gcc6 and clang; fixes various low-level issues (MLDB-2190)

Note: test failure is spurious (already failing in master since trivial change, azure_blob_storage_test)


NAPS dataset: distinguishing 'search' from 'test' IO examples

Hello,
The NAPS paper says, "Each full solution is accompanied with 2-10 inputs/outputs each split into two groups. The first group can be used in search or can be included into the problem specification
as part of the model input. The second group can be used for the evaluation at the test time."
However in the NAPS dataset, it seems that there is a single list of IO examples with the key 'tests'. How may I distinguish 'search' examples from 'test' examples?
Thanks.

Added split tests pipe in https://github.com/nearai/program_synthesis/commit/9e64e067ff307294fe0a7aa36ba89314e98180fe
Use it to deterministically split tests upon reading the dataset.


Implement What's New page

Suggested new feature or improvement
The Introducing Dev Home page should become a What's New page once we send out our next release.
Scenario
We're constantly adding new features and what's new should be showcased within Dev Home.
Additional details
No response

@cinnamon-msft this issue still a thing?

Yeah, we have it designed and spec'd. Just waiting for implementation. :)

/dup https://github.com/microsoft/devhome/issues/2159


Bump the HSTS max-age to 2 years

Final bit of https://github.com/wellcomecollection/wellcomecollection.org/issues/7348

Can't work out the choice of Saint Kitts and Nevis for this one...

I couldn't find an emoji for "hs", but I could for "st".


Agent API loadbalancer obstructs network errors

Environmental Info:
K3s Version: v1.20.6+k3s1
Node(s) CPU architecture, OS, and Version: x86_64, Linux, kernel 5.4.81
Cluster Configuration: 3 servers, 201 agents
Describe the bug:

We spend quite some time trying to troubleshoot why k3s agent can't connect to the API server. Only error logs it was producing were:
level=error msg="Failed to retrieve agent config: CA cert validation failed: Get \"https://<IP_ADDRESS>:34605/cacerts\": read tcp <IP_ADDRESS>:42924-><IP_ADDRESS>:34605: read: connection reset by peer"

In a while we enable debug logs and turned out that the actual problem was related to dns infrastructure:
level=debug msg="Dial error from load balancer: dial tcp: lookup api.example.com: no such host"

I suggest to change log level for dial error to "error" to avoid confusion as "CA cert validation failed" error obfuscates network issues https://github.com/k3s-io/k3s/blob/ac507e530d134fbfa15cff90d66c0cb8c71801a2/pkg/agent/loadbalancer/loadbalancer.go#L132

it's still relevant, fix is tiny, can make a PR if it's accepted

The reason this is only logged at Debug level is that it can be VERY noisy when a cluster member is down, to the point of obscuring other more meaningful messages. There may be some way to log initial connection errors at a higher log level, but it hasn't been a priority to date. We'd welcome a PR that resolves the issue without creating excessive log spam during cluster maintenance.


Feature: OG Images for workspaces

As an owner of a public workspace, I want an open graph (OG) image for my workspace so that I can share it on socials.

Closed by #2939


Tooling and documentation: JSON schema for profile editing

Codecov Report

Merging #187 (203da80) into main (861cadd) will decrease coverage by 0.23%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #187      +/-   ##
==========================================
- Coverage   62.71%   62.49%   -0.23%
==========================================
  Files          34       34
  Lines        1813     1813
  Branches      401      401
==========================================
- Hits         1137     1133       -4
+ Misses        585      577       -8
- Partials       91      103      +12

Impacted Files
Coverage Δ

qgis_deployment_toolbelt/__about__.py
100.00% <100.00%> (ø)

qgis_deployment_toolbelt/utils/win32utils.py
40.00% <0.00%> (-40.00%)
:arrow_down:

qgis_deployment_toolbelt/profiles/shortcuts.py
51.61% <0.00%> (-19.36%)
:arrow_down:

...loyment_toolbelt/jobs/job_environment_variables.py
91.37% <0.00%> (-8.63%)
:arrow_down:

qgis_deployment_toolbelt/commands/cli_upgrade.py
53.62% <0.00%> (-7.25%)
:arrow_down:

qgis_deployment_toolbelt/cli.py
83.56% <0.00%> (-2.74%)
:arrow_down:

qgis_deployment_toolbelt/plugins/plugin.py
75.00% <0.00%> (+1.51%)
:arrow_up:

qgis_deployment_toolbelt/constants.py
72.00% <0.00%> (+4.00%)
:arrow_up:

...deployment_toolbelt/jobs/job_plugins_downloader.py
66.98% <0.00%> (+47.16%)
:arrow_up:


Cleanup post FinderBase API introduction

Introducing the FinderBase<CandidateType> implementation introduces a lot of things that need to be cleaned up, deprecated, etc. This ticket will act as a followup to #123634 but be purely focused on cleanup, allowing #123634 to focus on cleanly introducing the new API.
Cleanup:

[ ] Deprecate finders.dart#Finder

References to Finder should be updated to WidgetFinder

[ ] Deprecate finders.dart#ChainedFinder in lieu of ChainedFinderMixin
[ ] Deprecate finders.dart#MatchFinder in lieu of MatchFinderMixin
[ ] Deprecate the methods in _LegacyFinderMixin and migrate to the FinderBase API internally

This was handled in the original PR instead of with a followup. Closing.


How to skip importing files via HeadlessAnalyzer if AutoImporter doesn't find a loader.

Hello,
I would like to import all execution file to Ghidra from specific folder and
I know that some files don't match a file loader of Ghidra.
(I got an error in HeadlessAnalyzer that ERROR REPORT: Import failed for file: No load spec found for import file:)
Could someone tell me an option to skip an import and continue importing next file if AutoImported fails to import?
Thank you !

If performing a recursive import of a directory's content, the import should report such an import error but continue importing/processing the remaining files.

If performing a recursive import of a directory's content, the import should report such an import error but continue importing/processing the remaining files.

I've confirmed that even though Ghidra logs errors on files it cannot import, it does indeed continue pass the errors and import the rest of the files in the directory.


add protocol-radius and protocol-radius-test.

Checklist:

[x] Meaningful commit message - please not Update build-constraints.yml
[x] Some time passed since Hackage upload
[ ] stack build --resolver nightly --haddock --test --bench --no-run-benchmarks
-- Executed. but an error occurs. error of cassava package?
AesonException "Error in $.packages.cassava.constraints.flags['bytestring--lt-0_10_4']: Invalid flag name: \"bytestring--lt-0_10_4\""

Hey @khibino, run a stack upgrade to get rid of that error! Merging 👍


Notifications don't work

macOS Monterey 12.6.1
Fastmate 1.9.0 and 1.9.1
Hi I can't get notifications for Fastmate to appear. I've deleted the app, deleted it from 'Notifications & Focus', reinstalled it, and granted permissions when asked on launch but they simply won't appear.
This started happening after getting a new MacBook and restoring a Time Machine backup to it.
Apart from the app itself, what files does Fastmate install on my system? Deleting absolutely everything seems like the logical place to start.
Thanks

@mikescott Fastmate installs nothing except the binary from the zip file. Fastmail leaves cookies to keep you logged in/your settings via Fastmate though, but it's nothing that Fastmate has any control over really. It's the same cookies as it would set in any other browser.
As for notifications not working - make sure you've turned them on within the Fastmail settings inside Fastmate. The switch to the upper right:

Does that help?


Unique validator ignores custom message when using targetAttribute

What steps will reproduce the problem?
Add a unique validator for two attributes and define a custom Error message:
    /**
     * @inheritdoc
     */
    public function rules()
    {
        return [
            [['rate_id', 'vehicle_id'], 'required'],
            [['rate_id', 'vehicle_id'], 'integer'],
            [['price'], 'number'],
            [['rate_id'], 'exist', 'skipOnError' => true, 'targetClass' => Rate::className(), 'targetAttribute' => ['rate_id' => 'id']],
            [['vehicle_id'], 'exist', 'skipOnError' => true, 'targetClass' => Vehicle::className(), 'targetAttribute' => ['vehicle_id' => 'id']],
            ['vehicle_id', 'unique', 'targetAttribute' => ['vehicle_id', 'rate_id'], 'message' => 'Dieses Fahrzeug wird in diesem Tarif schon verwendet.'],
        ];
    }

What is the expected result?
The validation message should be  'Dieses Fahrzeug wird in diesem Tarif schon verwendet.'
What do you get instead?
The validation message is 'Die Kombination "1"-"1" für Fahrzeug and Rate ID wird bereits verwendet.'
Additional info

Q
A

Yii version
2.0.10

PHP version
7.0.8

Operating system
Linux Mint 18

https://github.com/yiisoft/yii2/blob/master/framework/validators/UniqueValidator.php#L96

This should be fixed in 2.0.10. Are you sure you're using 2.0.10?

Sorry, my bad. After updating it works fine.

Hello. I have this issue with version 2.0.14.
['worker_id', 'unique', 'targetAttribute' => ['team_id', 'worker_id'], 'message' => Yii::t('app', 'This Worker is assigned to another Team.')],
I get the generic message error:  "The combination of fields has already been taken."
Also, the message appears in both fields, and must be only in worker_id, as I understand.
Thanks

Please use the forum / slack channel or other to ask such questions, thank you.

Sorry @bizley , Why I must do that? I think this is a bug. Thanks

Hmm, sorry, maybe I was too fast with this. Anyway - your description indicates that it behaves differently than expected but first of all - use the current version (2.0.43) and if it still happens create a new issue describing the error. Thank you.

Ok. Thanks for the clarification. Unfortunately I can update the Yii version because I have a very close deadline. Thanks anyway.


Support for color definition format beginning with "0x"

In the ColorUtils file, I don't see a regex to match colors like this:
0xFFFFFF
Can we add this?

@redmunds Where is the ColorEditorTemplete.html located?

ColorEditor template is in:
src/extensions/default/InlineColorEditor/ColorEditorTemplate.html

@redmunds Like this
0x
 sorry if that is wrong I don't understand what your saying

@PlanetVaster That's basically it, but it needs a unique title which is the tooltip.

So like
0x

Yes, Then COLOR_EDITOR_0X_BUTTON_TIP string needs to be defined in src/nls/root/strings.js

What is the status of this?

Guess i arrived here late... is this issue still open for claiming?

@KrithikaNarayanan I think this hasn't been fixed / added yet. Feel free to work on it if you'd like to!
See the article How to hack on Brackets to get started with the Brackets codebase. If you have any questions, feel free to post them here! We also have a Slack channel for day-to-day communication. (You can join by sending a mail to<EMAIL_ADDRESS>with the subject line slack registration request specifying the email addresses you would like to register).

Hello. I found myself interested in taking a crack at this issue if that could be approved.  :)
@zaggino

Hi @nyteksf , you don't need anyone's approval to start working on an issue that's not being worked on. Just open a PR (unfinished is absolutely fine) that you're working on this and mention the link here. That will let others will know that the issue is being worked on.

Hello again. I believe I'm finished with this one. Please do feel free to let me know if any changes need to be made, and I'll be happy to get right on that for you. Thank you.
@zaggino @ficristo

@nyteksf  thank you for filing the issue.
As a side note: in future, in your PRs, could you add the issue number also in the comment?
In this way GitHub cans generate an automatic link to your PRs in the issue itself.

@ficristo Noob question here, I suppose; being that this is my second PR ever. Could you please elaborate for me just a little about what was meant so I can do that for you all the next time? I already try to always add the issue number to the title and every commit message. Do you mean to also add it to the literal comments I am making much like this one? My bad. Thanks for your help.

Don't worry about asking, you are welcome.
Yes, I meant exactly comments like this. Right now, for example, I don't know if you have already opened a PR, I can only see commits that were done in you fork.
So if I add the PR number like this #13154 now GitHub will show a link in the PR to this issue.
Adding the issue number in the title is fine, but will not generate a link.

#13154 - @ficristo Aha. I see now. And thank you very much.


Updated podspec to use ObjectMapper 3.x

Thank you but this is a duplicate of an earlier PR which I merged


Version graph arrow direction incompatible with label

Describe the bug
label "New Model Version Of" is not compatible with the arrow direction when looking at a versioning graph using the Show Merge Graph option.
Expected behavior
Either reverse the arrow head or (preferred) change label to "has new version"
Screenshots

so the label is taken from the VersionLinkType which exists, so the JSON from the API cant change the label so a label change would have to be coded into the UI which is probably to be considered as messy,
therefore the option really is to chnage the direction of the arrow.
I suspect the JSON being supplied by the API is the cause of this issue rather than the UI, but if someone in UI can check then we may want to move this to mdm-core.

@olliefreeman There's a side issue which may also just be a presentation thing, or may be a bug in the database importers - the orange branch has Version 1.0.0 on it, but is coming off finalised model 1.1.0.


Adding [assembly: OwinStartup(typeof(Startup))].

This won't work without the OwinStartupAttribute!

Hi @noopman, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@dstrockis - can you review when you have a chance?  Thanks.

@BryanLa please don't merge yet, I would need to update the quick start code and test it.

@dstrockis - sounds good

@dstrockis  - just checking back on this one ...

@dstrockis - another ping ...

@dstrockis - ping

Good to go @BryanLa sorry for the delay

@dstrockis - thanks!
@noopman - thanks for the contribution!


hi-jack cancel errors for webRT (bug 1133963)

See the sadness in action! http://screencast.com/t/tem5nKue4SP
@muffinresearch r?
Spartacus patch: https://github.com/mozilla/spartacus/pull/208

This makes me feel like this:

But if we have to r+


Shared eslint config

https://app.asana.com/0/1201807839809179/1208561434802767/f

Upgrade to eslint 9
use shared DDG config
fix some lint errors

Ok this is ready for review now. I recommend looking commit-by-commit for better context


Lock down grafana version

If we're going to keep grafana, then at least lock down the package version.  At the moment we specify a version (because the api keeps changing), but then apt keeps updating it to the latest.  Either pin the pkg, build from source, or better yet just remove grafana.

I quite like the graphics it produces visually but if I recall you were having issues taking the elements and displaying them in a custom view?
I'm personally willing to explore other options if you would like to drop it.

Use apt-mark to hold the version

Added bit of code but commented out, to hold grafana package, as it causes a puppet loop as grafana module can't cope with held packages.
Not worth the effort, grafana will probably disappear and this is an annoyance.


Follow up on libbeat interface cleanup

Ref - https://github.com/elastic/libbeat/pull/28

no public function interface is changed, closing this.


option to prevent defaulting to first tab

Ionic version:  (check one with "x")
[x] 3.x
[x] 4.x
I'm submitting a ...  (check one with "x")
[x] feature request
Current behavior:
According to the docs "by default, the first tab will be selected upon navigation to the Tabs page". I would like an option to prevent this so I can decide which tab to load in an async operation.
While I can use select() to load the correct tab after async, that's after the first potentially incorrect tab has already loaded and you see a healthy flash between the two pages.
Expected behavior:
n/a
Steps to reproduce:
n/a
Related code:
n/a
Other information:
n/a
Ionic info: (run ionic info from a terminal/cmd prompt and paste output below):
n/a

Hello! Thank you for the feature request. I'm going to move this issue over to our internal list of feature requests for evaluation. We are continually prioritizing all requests that we receive with outstanding issues. We are extremely grateful for your feedback, but it may not always be our next priority. I'll copy the issue back to this repository when we have begun implementation. Thanks!

This issue was moved to ionic-team/ionic-feature-requests#314


Question 4 -Lab 2

I have used this code for question 3 b
mean(downtown$yearbuilt >= "1980"& downtown$land_use == "Commercial", na.rm = TRUE)

and It gave me NAN.
Is it because I used downtown? Or is there a problem with the code.
Thanks!

It also gives me NaN - let's try breaking it down into smaller pieces to troubleshoot.
First, the expression passed to function mean() should be of class "logical" and therefore be TRUE and FALSE in order for mean() to work as intended.
> downtown$yearbuilt >= "1980"& downtown$land_use == "Commercial"
logical(0)

So we know the trouble isn't with mean().  What if we look at these individually? That is, each relational statement?
> head(downtown$yearbuilt >= "1980")
[1] FALSE FALSE FALSE FALSE FALSE FALSE

So far so good!
> downtown$land_use == "Commercial"
logical(0)

So therein lies the issue.  Let's check the variables to see what's happening.
> names(downtown)
 [1] "tax_id"        "stnum"         "stname"
 [4] "zip"           "frontfeet"     "depth"
 [7] "sqft"          "acres"         "nhood"
[10] "ward"          "censustract"   "landuse"
[13] "yearbuilt"     "units"         "vacantbuild"
[16] "watersvc"      "assessedland"  "assessedval"
[19] "citytaxable"   "starc"         "taxtrust"
[22] "amtdelinqt"    "taxyrsdelinqt" "overduewater"
[25] "totint"

Can you spot what may have been the source of the trouble?  If so, try it again and let me know if that helps!

Yes
Thank you so much.
I can see that it was the variable landuse not Land_use


Make upgradable

It became pretty clear, that the applet will need to be upgradable. At the moment this will be obtained by simply leaving the card with default Global Platform keys. However in the future we could use a key derived from the master wallet for upgrades (this means requiring a PIN to accept upgrading).
Upgrading removes the stored keys, PIN(s) etc. This means the procedure will not be as straightforward and will require backups. We must make sure upgrades are released only when strictly needed for introducing additional features. This also means that when we do a release we must be fairly sure that there are no bugs, especially security-related ones. This is currently achieved through extensive testing of all possible error conditions and malformed inputs.

There is an option you could consider: in general, you can use two applets to allow upgradability. One applet is the "back-end", as such, and it stores key material and PINs. The front-end, which is upgradable, can then just refer to the "back-end" applet to use the keys and PINs. Upgrades can occur without loss of data in this circumstance.
Otherwise, you can use chip-based keys (NXP for instance offers its PUF, which is a key derived from small differences in manufacture) to export and import backups only able to be decrypted by the chip in question.


add code snippets to contribute pictograms

add code snippets in the how to save section, in the pictograms > contribute  page
Closes #
{{short description}}
Changelog
New

{{new thing}}

Changed

{{change thing}}

Removed

{{removed thing}}

error


Added/updated german localization

I added the Location stuff to the german localization....

Fixed it, thanks for reviewing it!
Bye, Frido.

Thanks!


convert image to pencil sketch

Script Title -
what will change - This script will convert given image to pencil sketch using openCV
Instructions

Create a new folder for your script and file/folder name should be appropriate.
Create a README.md in your folder for program Instructions
add requirements.txt if needed

Please add/delete options that are not relevant.

[x] Adding New Code

Programming Language

[x] Python

:star2: Star it :fork_and_knife:Fork it :handshake: Contribute to it!
Discord server  - https://discord.gg/FXyh2S3
Happy Coding,

Interested in this Task ! Can I Proceed

I had opened issue.
Can I work on this?

please use flake8 for linting python code.

sure!! Thanks
@codePerfectPlus

please use flake8 for linting python code.


Adding respawn and respawn limit

As requested by #2

Sorry about the closing and re-opening of the pr, for some reason travis didn't want to immediately run for this pr, had to flip the switch a couple times before it picked it up.

closes #2


This bot gets you instantly banned by Discord

Title says it all.
It's only a 1-day banned though

What...?

Closed due to no evidence shown for a very odd claim

Closed due to no evidence shown for a very odd claim

What do you mean old claim ?
I literally cloned the repository, launched the .py, and got banned 10 seconds later :

Ah I see what you did, don't use your personal account token to run this, as using a self-bot and not an official bot account as provided by the README tutorial is against Discord ToS. Be more careful next time with what kind of token you use.

Ah I see what you did, don't use your personal account token to run this, as using a self-bot and not an official bot account as provided by the README tutorial is against Discord ToS. Be more careful next time with what kind of token you use.

I read the README tutorial, watched the video, created an application etc I did exactly like you did in the video

Well your email says you selfbotted and I do not show you how to get your personal account token in any video in the README.....


Add support for new text-embedding-3 models

text-embedding-3-large and text-embedding-3-small aren't supported in js-tiktoken.
All we have to do is to add them to the config, like it's in the original tiktoken repo.

@dqbd could you please approve this so it gets merged? Thanks!

Closed in favour of #112 to add support for WASM-based tiktoken as well, thank you for the contribution though @lukasugar!


Add fields option to rules to allow for runtime fields and others

Description

Checklist

[x] I have reviewed the contributing guidelines.
[ ] I have included unit tests for my changes or additions.
[x] I have successfully run make test-docker with my changes.
[x] I have manually tested all relevant modes of the change in this PR.
[x] I have updated the documentation.
[x] I have updated the changelog.

Questions or Comments

Sorry, you didn't miss anything.  I did not include unit tests for this change.  Would you like me to write some?

Yes, please. See base_test.py and loaders_test.py which already have unit tests that should be covering those existing functions. I think get_hits() may be called indirectly via run_query() in the unit tests. load_options() should be called directly in the existing unit tests.

I have added tests, please let me know if there are any additional tests you think need to be added.


asyncCustomerChatUX is now deprecated

In Step 5, asyncCustomerChatUX has now been deprecated and has been replace by startChatContactAPI, so the instructions don't work perfectly. I'm working through remediation steps now, but if anyone else has something complete, it'd be great to see a PR.

Thanks for mentioning this: https://github.com/amazon-connect/amazon-connect-chat-ui-examples/tree/master/cloudformationTemplates/asyncCustomerChatUX
It is noted here and we'll try to update this demo.

Is there any alternative demo reference I can follow, as a part of this demo is deprecated?


Question GO:0042790,nucleolar large rRNA transcription by RNA polymerase I

Should this term exist?
GO:0042790,nucleolar large rRNA transcription by RNA polymerase I
It isn't a different process from
GO:0006360  transcription by RNA polymerase I

These two terms are not actually synonymous as there are some species where RNAP I is involved in transcription of a select set of protein coding genes (has been observed in some Trypanosomes, if I remember correctly).
I was wondering if the more specific term "nucleolar large rRNA transcription by RNA polymerase I (GO:0042790)" was actually being used, and was surprised by the fact that there is significant annotation to this term. Here are a couple tables.
Here's a table by evidence code of all the terms that map up to "transcription by RNA polymerase I (GO:0006360)

Here's a more detailed table of just the two terms in question that includes Contributor:

OK can close this.


Doesn't work with journals

Currently, It only works on single page. It is possible to make the back button function assuming today's date, on journals. I can make a pull request.

Thank you for the suggestion, I agree with you and have implemented it in 1.1.0.


Tracking Issue for -Zgitoxide

Summary
RFC: None
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#gitoxide
Issues: https://github.com/rust-lang/cargo/labels/Z-gitoxide
Original issue:

shallow clones #1171
slow on file:// #9167
slow on fetch #11014
high CPU usage #9215
high memory usage #10583

With the ‘gitoxide’ unstable feature, all of the specified git operations will be performed by the gitoxide crate instead of git2. This allows git2 to be replaced in full.
Steps
Unstable feature sub-categories:

[x] fetch: all fetches are done with gitxide   (#11448)
[x] shallow_index: the crates index will be a shallow clone (#11840)
[x] shallow_deps: git dependencies will be a shallow clone (#11840)
[ ] checkout: plain checkouts with gitoxide (including submodule handling)
[ ] package: status checks, untracked files, excluded files

Further tasks

[x] short-hash handling can be done by gix_hash::Prefix.
[ ] optimize dependencies to only include those that cargo actually uses.
[ ] When specifying a dependency with rev but nothing found, cargo with shallow fetch should fetch full history as a fallback.

Unresolved Issues

[x] work with https://rsproxy.cn (the proxy seems to work different making naive negotiations fail due to multi-round negotiation)
[ ] all features as listed in this gitoxide tracking issue to be able to replace git2 in full
[ ] support for the file:// protocol without relying on the git binary
[ ] support of the ssh:// protocol without relying on the ssh binary
[ ] double-check and validate that authentication works similarly to git2 or if unfeasible, assure users have a migration path. How to deal with this correctly is still to be determined.
[x] transitive dependency tempfile is pinned to ~3.3.0 temporarily (see<EMAIL_ADDRESS>[ ] https://github.com/rust-lang/cargo/labels/A-git might contain more potential blockers

take a look at all of these and either fix them, resolve them, or store them in gitoxide itself

Future Extensions
Some features not in libgit2 are good candidates for future extensions

sparse checkout #10150
split index #11165
git-lfs integration #9692

About tracking issues
Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
Implementation history

#11448
#11800
#11830
#11831
#11840

transitive dependency tempfile is pinned to ~3.3.0 temporarily (see<EMAIL_ADDRESS>
This surprises me and the links don't quite make the motivation for this clear.
From my experience

Using anything but ^ for general dependencies like tempfile can be ecosystem-breaking
Generally workarounds like this should be handled in Cargo.lock

Hmm… agreed! gix-tempile previously depends on 3.4.0, so alternatively it could just depend on ^3.3.0 and we handle it in rust-lang/rust. I cannot guarantee but believe bootstrap team will fix it in the near future :).

I would agree that using ^3.3.0 would be the more appropriate requirement. I would generally avoid ~ in most situations.

I cannot guarantee but believe bootstrap team will fix it in the near future

I do not think anyone is looking at fixing this particular issue. The fix from https://github.com/rust-lang/rust/pull/106610 didn't seem to work for me, though I'm not sure why.
If cargo moves to a separate workspace, then this particular problem might not be an issue.

I don't know if there is a better solution to this problem than the one we opted for, but by sharing the status quo maybe we will come up with something.

gix-tempfile v4.1.0 refers tempfile ^v3.4
gix-tempfile v4.1.1 refers tempfile ~v3.3 and is the latest version that everybody gets right now.

When publishing gix-tempfile v4.1.1 my CI broke (due to cargo deny) and for some reason I didn't even notice it. Had I known that there is a RUSTSEC advisory out aganst tempfile 3.3 I would not have pinned to it so lightheartedly.
Since then, I have removed the pin of tempfile in such a way that the next release of gix-tempfile will be a major version change to fix my CI. gix-tempfile v4.1.1 will still cause other peoples CI to fail for the same reason it failed for me, and it's pulled in by default until it is yanked by all available versions of gix.
Soon I hope to be able to publish a new major version of gix which then depends on gix-tempfile v5.0, providing an upgrade path out of this situation.
Thus far, doing this was quite some churn and already caused breakage in other people's CI, and there is probably some fallout that I don't know about. The main culprit here was me being unaware of the security advisory which is sure way to break people who run cargo deny. I am sure there are plenty of other ways as well even without that.
It might be worth investigating a better process to handle such issues as I have a feeling the probability for this to happen is quite high given the large amount of additional dependencies that gitoxide (still) pulls in combined with the large amount of platforms cargo is being compiled for.

When publishing gix-tempfile v4.1.1 my CI broke (due to cargo deny) and for some reason I didn't even notice it. Had I known that there is a RUSTSEC advisory out aganst tempfile 3.3 I would not have pinned to it so lightheartedly.

This is not the only reason to avoid pinning in version requirements.  See my previous link where another crate author pinned things and caused problems.

Since then, I have removed the pin of tempfile in such a way that the next release of gix-tempfile will be a major version change to fix my CI.

Why did removing the ~ require a major version change?   It seems like a ^3.3 could be in a 4.1.2

This is not the only reason to avoid pinning in version requirements. See my previous link where another crate author pinned things and caused problems.

Even though I didn't acknowledge it I don't think I made the claim that what happened here is all that can happen. Am I missing another point?

Why did removing the ~ require a major version change? It seems like a ^3.3 could be in a 4.1.2

As of now, cargo seems to require the dependency fix/hack to function. I made my CI fix (removing the pin) breaking to assure that the next release will be a major one that cargo update will not automatically update to, which would undo the fix. Now, gix up to v0.40 will use the pinned version, and gix v0.41 which I am prepping now will put things back to normal.
Funnily enough I am also preparing a PR that upgrades cargo to gix v0.41 to get a fix for this issue, so I guess that one will be blocked until the underlying cargo issue is fixed. 'Kerfuffle' is my favorite word for this, right after 'conundrum' 😅.

Even though I didn't acknowledge it I don't think I made the claim that what happened here is all that can happen. Am I missing another point?

I was just wanting to confirm since I had read your comment as the RUSTSEC issue being the only thing that would have had you re-evaluate.

As of now, cargo seems to require the dependency fix/hack to function. I made my CI fix (removing the pin) breaking to assure that the next release will be a major one that cargo update will not automatically update to, which would undo the fix

Maybe others feel different but I would not consider this to be a part of your project's compatibility guarantees and personally feel making it a breaking change exacerbates the problem as the max version (default for the resolver) in the 4.x.y series does not play well with other crates.

Funnily enough I am also preparing a PR that upgrades cargo to gix v0.41 to get a fix for https://github.com/rust-lang/cargo/issues/11821, so I guess that one will be blocked until the underlying cargo issue is fixed. 'Kerfuffle' is my favorite word for this, right after 'conundrum' 😅.

Why is this necessarily blocked?  Couldn't gix-tempfile use ^3.3 instead of ^3.4?  Then cargo could use 3.3 and everyone else can avoid the tainted version and use 3.4.

I was just wanting to confirm since I had read your comment as the RUSTSEC issue being the only thing that would have had you re-evaluate.

Right! Even though I ran into issues like the one in your link myself and know that pinning is usually going to bite, in this case I didn't think much about the possible repercussions as fixing cargo seemed like an overwhelming priority along with a lack of alternatives.

Maybe others feel different but I would not consider this to be a part of your project's compatibility guarantees and personally feel making it a breaking change exacerbates the problem as the max version (default for the resolver) in the 4.x.y series does not play well with other crates.

Definitely, it breaks the stability guarantee of the project as per STABILITY.md (even though it doesn't actually break anybody), and the fix for the v4.x. line of gix-tempfile will happen by yanking v4.1.1 once cargo is in the clear. It is a hack to protect another hack which seemed inevitable.

Why is this necessarily blocked? Couldn't gix-tempfile use ^3.3 instead of ^3.4? Then cargo could use 3.3 and everyone else can avoid the tainted version and use 3.4.

How would cargo be able to force itself to use gix-tempfile v4.1.1? Refer to the crate and pin it maybe using ~4.1.1? Right now the assumption is that it is served whatever gix comes with, and that will currently top out at 4.1.1 by normal resolution rules.
Probably I was missing the obvious solution of pinning gix-tempfile in cargo, as now what happened is said exacerbation of the problem. gix v0.41 now refers gix-tempfile v5.0 which is good technically but wrong from cargo, which now cannot pin a lower version anymore without conflicts (or at least, it wouldn't force the whole dependency tree to use the same version). But if that were true, couldn't cargo have pinned tempfile all by itself in the first place?
So it looks like all that can be done to fix this issue is to have one PR which disables the tests in question, and another that re-enables them and upgrades to gix v0.41 that contains the actual fix and removes the pinned tempfile crate. And then we hope cargo and tempfile v3.4 will soon work again so the hacks can be removed as good as possible.

Definitely, it breaks the stability guarantee of the project as per STABILITY.md (even though it doesn't actually break anybody

I did not see anything that said you are responsible for bugs  in your dependencies.  tempfile didn't see a reason to make it a breaking version, so what isn't clear to me is why it needs to be one for gix-tempfile.
Also, unsure if we've already had this conversation but I would recommend reconsidering MSRV changes being breaking changes.  I know that was a topic for a while during the early days of MSRV discussions but I feel like things have mostly settled towards not doing that.  Take the recent libc MSRV discussion.  That had a lot of voices with a lot of different (strong) opinions but only 1 person advocated for a general policy.  The only other mention of it I found was Josh bringing it as part of brainsxtorming for breaking the gordian knot for libc.  Updating MSRV is like rustc updating the kernel, glibc, or android NDK.
semver is a tool to help and if it is used to cover too much ground it can make it difficult to make any progress or can cause extra churn from too many breaking releases and then hurts more than helps.  I originally wrote this in talking about MSRV but I suspect it ties into our tempfile conversation

How would cargo be able to force itself to use gix-tempfile v4.1.1? Refer to the crate and pin it maybe using ~4.1.1? Right now the assumption is that it is served whatever gix comes with, and that will currently top out at 4.1.1 by normal resolution rules.

Unsure if this is what you are getting at but something I had overlooked is the challenge of (some) monorepo development: intra-repo version requirements.  I'm assuming on every release all dependents get their version requirement updated to latest.  This makes   development in the repo easier but increases churn for end-users and offers users fewer workarounds, like in this case.  I say this not to say what is the right approach for gix but to acknowledge it.  gix is of the scale it could be hard to manage.  In some of my projects, I do similar though in ones like clap where its smaller and the pieces are less entangled, I don't.  One future possibility to help with this is my idea to stablize the #[stable] attribute and have clippy or rustc produce warnings when you use an API item from newer than the minimal version of your version requirement.  While it wouldn't guarantee that version reqs within the repo are always valid, it helps.

. But if that were true, couldn't cargo have pinned tempfile all by itself in the first place?

Maybe?  I don't have the context for what precipitated this.  The tracking issue links to a tag which links to a comment from Chris Denton that says nothing about tempfile and I've expanded commetns around it and I don't see anything.  I did a search for tempfile on the page and only found this comment from ehuss.

semver is a tool to help and if it is used to cover too much ground it can make it difficult to make any progress or can cause extra churn from too many breaking releases and then hurts more than helps. I originally wrote this in talking about MSRV but I suspect it ties into our tempfile conversation

Making MSRV changes non-breaking definitely can be discussed - while gix is in pre-release mode I find it easy to declare it a breaking change and prevent anybody downstream from being broken by a patch release. For stable releases, I'd love this to change as well even though I see value in the downstream being able to protect itself from breakage without having to pin their gix dependency down to the patch level. gitoxide's MSRV is effectively dictated by the windows dependency, and that makes it even harder to 'do the right thing'.
Wasn't there an RFC that would allow cargo to choose only versions that have a compatible MSRV? With that, the problem would be solved I think.
This is not to discard the suggestion of making MSRV changes non-breaking, it's just to explain who one could choose one over the other to prevent downstream breakage. To me, that is still paramount, and ideally there is a better way of doing things once gix goes 1.0 to maintain that major version number for 6 months or more.

[..] One future possibility to help with this is my idea to stablize the #[stable] attribute and have clippy or rustc produce warnings when you use an API item from newer than the minimal version of your version requirement. While it wouldn't guarantee that version reqs within the repo are always valid, it helps.

On the question on managing complex workspaces with plenty of interconnected crates, having #[stable] would definitely be another tool in the belt. It would probably be particularly useful for stable crates (i.e. X.Y with X>0).

Maybe? I don't have the context for what precipitated this. The tracking issue links to a tag which links to a comment from Chris Denton that says nothing about tempfile and I've expanded commetns around it and I don't see anything. I did a search for tempfile on the page and only found https://github.com/rust-lang/rust/pull/108665#issuecomment-1452337751.

Seeing this makes me fear I made all this up due to some huge misunderstanding 😅. The issue feels quite illusive, too, and I guess it's time for some sharing my take-aways:

don't ever hack dependencies, even if it would fix cargo
to avoid the previous point, try to find other solutions and involve more people if you have to

This whole topic did costs me a couple of hours and certainly had higher costs throughout the ecosystem due to gix-tempfile being tainted for a day without alternative. I will wait for the tempfile and cargo issues to be resolved (in the hopes of being pinged when that happens) to yank the tainted version and allow upgrades to more recent gix versions (> v0.40) to happen.

Wasn't there an RFC that would allow cargo to choose only versions that have a compatible MSRV? With that, the problem would be solved I think.

It was discussed as a possibility as part of package.rust-version.  We still need to work out the UX once we have package.rust-version in the index.  This will help a lot, along with a list of other ideas for improving the experience summarized in a post I made in the libc thread that is now in the hidden section, making it difficult to get to...

Seeing this makes me fear I made all this up due to some huge misunderstanding 😅. The issue feels quite illusive, too, and I guess it's time for some sharing my take-aways:

Seems like we are on the same page now.  Unsure what to do about whats been done but hopefully we won't hit this again.  I'll go ahead and close these comments out as resolved.

Is it possible to enable this through environment variables somehow? I wasn't able to figure out any syntax that didn't run into this error:
> CARGO_UNSTABLE_GITOXIDE=true cargo build
error: missing field `fetch`

(it may also be useful for this usecase to support an explicit CARGO_UNSTABLE_GITOXIDE=all feature alias that acts like -Zgitoxide)

I think I did that with the following, on stable:
export CARGO_REGISTRIES_CRATES_IO_PROTOCOL=git
export __CARGO_USE_GITOXIDE_INSTEAD_OF_GIT2=1 # nobody should be using this on stable, but it's there if truly needed.

CARGO_UNSTABLE_GITOXIDE=fetch cargo build should work, and so should CARGO_UNSTABLE_GITOXIDE=fetch,shallow cargo build for shallow clones. Please note that I wasn't aware of these environment variables before reading this, so I just assume whatever -Z gitoxide=<value> supports can be used as CARGO_UNSTABLE_GITOXIDE=<value>.

CARGO_UNSTABLE_GITOXIDE=fetch cargo build should work

It doesn't, it gives the same error message. The CARGO_UNSTABLE environment variable corresponds to the [unstable] table in .cargo/config.toml not the -Z flag.
By actually trying to set it in .cargo/config.toml it gave some more useful error messages, and I managed to get it working with
[unstable.gitoxide]
fetch = true
shallow_index = true
shallow_deps = true
checkout = true
internal_use_git2 = false

and
> CARGO_UNSTABLE_GITOXIDE_FETCH=true CARGO_UNSTABLE_GITOXIDE_SHALLOW_INDEX=true CARGO_UNSTABLE_GITOXIDE_SHALLOW_DEPS=true CARGO_UNSTABLE_GITOXIDE_CHECKOUT=true CARGO_UNSTABLE_GITOXIDE_INTERNAL_USE_GIT2=false cargo build

I do think it would be good to override the toml parser for this variable too so that you don't have to specify every single field.

How is shallow cloning dependencies different than specifying the refspec (e.g. +HEAD:refs/remotes/origin/HEAD) that cargo already does? I think it only really helps if when fetching a specific branch or tag?

A refspec reduces the heads that you're requesting, so you don't get other unneeded branches being pulled down. But you still request the entire history from that head back to initial commit(s) of the repo. A shallow clone allows you to limit how much of the history you request, presumably for cargo's usecase it only needs to pull down the current commit of the requested head and can ignore the history entirely (though technically this is visible to build.rs scripts running from git dependencies).