[Investigation] Read string data using CLFN (and pass out string pointer separately)

Current Approach - Read String using LabVIEW's MoveBlock functionality (before closing reference)

Here is the VI that reads a cstring from a memory location:

The MoveBlock function is exported by the LabVIEW executable app:

After we're done reading the data, we close the reference:

Different Approach - Use CLFN to Read String, but also pass string reference (so we can close reference)
A slightly different approach to reading string data from Rust is to have the CLFN read the string data (instead of just getting the address of the data and reading it with the Memory Manager). This is nice, because we can rely on LabVIEW's built in feature for reading a cstring, however, it means that we must also pass out the string's address, separately, so that we can ask Rust to deallocate/free the string after LabVIEW has read the data.
That looks something like this:

Notes:

there's an additional parameter string_ptr: *mut c_void, which is how we will obtain the string pointer (in note 2 below).
right before we return, we will write the return string's address to the string_ptr parameter that was passed in --> unsafe { *(string_ptr as *mut *mut c_char) = raw_string; }

// return a toml string from a Document
#[allow(dead_code)]
#[no_mangle]
pub extern "C" fn toml_edit_doc_to_string (
    doc: *mut c_void,
    string_ptr: *mut c_void,
) -> *mut c_char {
    let doc = unsafe { &mut *(doc as *mut Document) };

    let toml_str = match Document::to_string(doc) {
        toml_str => toml_str,
    };

    let raw_string = match CString::new(toml_str).unwrap().into_raw() {
        ptr if ptr.is_null() => {
            println!("Unable to allocate memory for string");
            return CString::new("").unwrap().into_raw();
        },
        ptr => ptr,
    };

    // write raw_string's address as the value stored in string_ptr
    unsafe { *(string_ptr as *mut *mut c_char) = raw_string; }
    return raw_string;
}

When we call this from LabVIEW, we can read the return string using the built in CLFN's C String Pointer format for strings.
And, we will ALSO get the address of the string as an integer that we can pass back to Rust to deallocate/free, using the cstring_free_memory() function shown below:
// exported function that frees the memory allocated for a string
// this *must* be called for every string returned from a function in this library
#[no_mangle]
pub extern "C" fn cstring_free_memory(s: *mut c_char) {
    unsafe {
        if s.is_null() {
            return;
        }
        CString::from_raw(s)
    };
}

This is far from ready yet but my goal with the labVIEW interop crate (https://github.com/WiresmithTech/Rust-LabVIEW-Interop) would be that you can pass the string handle into rust and it could handle resizing and perhaps encoding differences.
Right now though I've not started on it so there is some work to be done first!

@JamesMc86 Yes, it would be nice to have an easy way to work with a LabVIEW string handle in Rust.  I think you're right that doing the string resizing in Rust would make things nice and simple in LabVIEW.  Question: how does one access the LabVIEW memory manager on the Rust side of things? Would Rust call into LabVIEW.exe and access its exported functions similar to how we can do this with a call library function node?

Yep that's it. I've not got as far as trying it but I think you can dynamically load the functions assuming it is running inside LabVIEW. I've looked in principle but not got as far as trying it
James McNally
Director/Systems Engineer
Wiresmith Technology
t: 01635 597171

From: Jim Kring @.>
Sent: Thursday, June 22, 2023 2:46:25 PM
To: JKISoftware/toml-edit-labview @.>
Cc: James McNally @.>; Mention @.>
Subject: Re: [JKISoftware/toml-edit-labview] [Investigation] Read string data using CLFN (and pass out string pointer separately) (Issue #3)
@JamesMc86https://github.com/JamesMc86 Yes, it would be nice to have an easy way to work with a LabVIEW string handle in Rust. I think you're right that doing the string resizing in Rust would make things nice and simple in LabVIEW. Question: how does one access the LabVIEW memory manager on the Rust side of things? Would Rust call into LabVIEW.exe and access its exported functions similar to how we can do this with a call library function node?
—
Reply to this email directly, view it on GitHubhttps://github.com/JKISoftware/toml-edit-labview/issues/3#issuecomment-1602669729, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABBTHYHND7GFDEG6LWH2KFTXMREDDANCNFSM6AAAAAAZPJPOCY.
You are receiving this because you were mentioned.Message ID: @.***>

I see. It looks like this is the way that would work...
https://docs.rs/libloading/latest/libloading/

@JamesMc86 I was able to get this far...

// exported function that calls into LabVIEW.exe's memory manager
#[allow(dead_code)]
#[no_mangle]
pub extern "C" fn labview_get_string_size (
    labview_path: *const c_char,
    string_to_size: *const c_char,
) -> u32 {

    let labview_path = unsafe { CStr::from_ptr(labview_path).to_string_lossy().into_owned() };

    let labview_lib = match unsafe { libloading::Library::new(labview_path.clone().as_str()) } {
        Ok(lib) => lib,
        Err(_) => {
            println!("Unable to load LabVIEW.exe");
            return 0;
        }
    };
    // the DSGetHandleSize function is exported from LabVIEW.exe and takes a handle as an argument
    let DSGetHandleSize: libloading::Symbol<unsafe extern fn(*const c_char) -> u32> = match unsafe { labview_lib.get(b"DSGetHandleSize") } {
        // will get back Result<Symbol<'_, _>, Error>
        Ok(func) => func,
        Err(_) => {
            println!("Unable to find DSGetHandleSize function in LabVIEW.exe");
            return 0;
        }
    };

    let handle_size = unsafe { DSGetHandleSize(string_to_size) };

    return handle_size
}

Looks good. I think there is a way that you can do it without the path to LabVIEW but will have to remember where I've seen it. Bit because it is already in memory you can grab it from there
James McNally
Director/Systems Engineer
Wiresmith Technology
t: 01635 597171

From: Jim Kring @.>
Sent: Friday, June 23, 2023 4:56:16 AM
To: JKISoftware/toml-edit-labview @.>
Cc: James McNally @.>; Mention @.>
Subject: Re: [JKISoftware/toml-edit-labview] [Investigation] Read string data using CLFN (and pass out string pointer separately) (Issue #3)
@JamesMc86https://github.com/JamesMc86 I was able to get this far...
[image]https://user-images.githubusercontent.com/381432/248149319-613cafa9-0e5e-495c-b0f7-b30067dc0d32.png
// exported function that calls into LabVIEW.exe's memory manager
#[allow(dead_code)]
#[no_mangle]
pub extern "C" fn labview_get_string_size (
labview_path: *const c_char,
string_to_size: *const c_char,
) -> u32 {
let labview_path = unsafe { CStr::from_ptr(labview_path).to_string_lossy().into_owned() };

let labview_lib = match unsafe { libloading::Library::new(labview_path.clone().as_str()) } {
    Ok(lib) => lib,
    Err(_) => {
        println!("Unable to load LabVIEW.exe");
        return 0;
    }
};
// the DSGetHandleSize function is exported from LabVIEW.exe and takes a handle as an argument
let DSGetHandleSize: libloading::Symbol<unsafe extern fn(*const c_char) -> u32> = match unsafe { labview_lib.get(b"DSGetHandleSize") } {
    // will get back Result<Symbol<'_, _>, Error>
    Ok(func) => func,
    Err(_) => {
        println!("Unable to find DSGetHandleSize function in LabVIEW.exe");
        return 0;
    }
};

let handle_size = unsafe { DSGetHandleSize(string_to_size) };

return handle_size

}
—
Reply to this email directly, view it on GitHubhttps://github.com/JKISoftware/toml-edit-labview/issues/3#issuecomment-1603647611, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABBTHYDPXWXF3LUOH5O6POLXMUHWBANCNFSM6AAAAAAZPJPOCY.
You are receiving this because you were mentioned.Message ID: @.***>

Yes, I was thinking that it must be possible to get the name from memory. I
guess another consideration is that we would want this to work in a built
EXE, which means that the application/executable name (and path) would be
different that for the LabVIEW IDE.

I tried just using the name "LabVIEW.exe" instead of the full path and it worked fine:
let labview_lib = unsafe { libloading::Library::new("LabVIEW.exe")

And, here's a little gem that gets the main executable's path, which should address the built/stand-alone executable use case:
    use std::env

    let exe_path = match env::current_exe() {
        Ok(exe_path) => exe_path,
        Err(e) => {
            println!("failed to get current exe path");
            return 1;
        },
    };

    let labview_lib = match unsafe { libloading::Library::new(exe_path) } {
        Ok(lib) => lib,
        Err(_) => {
            println!("Unable to load LabVIEW.exe");
            return 42;
        }
    };

When I've gon digging for the interop crate this is the nicest solution I found and seems to be working.

Use dlopen2 crate to create an API container (to handle lifetimes etc for us)
Use ctor to initialise a static with this container when the DLL is loaded.

Initial testing looks good - unsure about error handling in the constructor. Right now it would panic but unclear if there is an expected way to handle this.
Code is in https://github.com/WiresmithTech/Rust-LabVIEW-Interop/blob/main/labview-interop/src/labview.rs


Warn on duplicate artifact creation

With a workspace such as in this repository: https://github.com/Mark-Simulacrum/cargo-issue-5524, running Cargo check on stable results in the following metadata being generated.
I believe that Cargo should try to detect the situation where duplicate artifacts will be generated and stop compilation; however, I'm not sure it's possible to do this well. I think even a heuristic that checks for lib/bin entries with the same name across the workspace would be sufficient for most of the benefits, though.
It's worth noting that I think this has only become an issue (at least for cargo check) since https://github.com/rust-lang/rust/pull/49289, which made rustc --emit=metadata emit metadata for bin targets. It might be enough to make change to rustc to disambiguate binary and lib metadata in the filename. In a way, this might be a regression, but I'm not sure.
$ find target -type f -name '*rmeta' | xargs ls -trl # stable 1.26
-rw-r--r-- 2 mark mark 1058 May 11 21:35 target/debug/libfoo.rmeta
-rw-r--r-- 2 mark mark 1058 May 11 21:35 target/debug/deps/libfoo-a9f370a27c2df2eb.rmeta

However, on beta, the following occurs:
$ find target -type f -name '*rmeta' | xargs ls -trl # beta 1.27
-rw-r--r-- 1 mark mark 1221 May 11 21:36 target/debug/deps/libfoo-11803c32bdc69a09.rmeta
-rw-r--r-- 2 mark mark    0 May 11 21:36 target/debug/libfoo.rmeta
-rw-r--r-- 2 mark mark    0 May 11 21:36 target/debug/deps/libfoo-f0bc89da5bcecba3.rmeta

cc @alexcrichton -- do you think this ought to be marked as a regression? It's confusing behavior that has only become a problem since 1.27, but I don't know whether it really warrants regression-like tracking.

Yeah sort of like rlibs I think we could just avoid uplinking these altogether

I think if someone were to detail how to "avoid uplinking these altogether" it might expedite a fix here. 🙂

@dwijnand It's not clear (to me) how safe it is to change these behaviors.  Like #6131 where someone was manually running rustc to link against rlibs created by cargo, the location of the rlibs is not easy to determine (generally need to slurp the json output to get the correct deps location).  If someone has some tool which looks at the rmeta files, removing them might make things more difficult for them.  I think rls-analysis does the correct thing, but it would be good to double-check if it would be impacted as it is the primary user.
In fact, I would imagine rls-analysis is the only user, so if it still works it should be fine.  It would be good to know if any other tool expects those files to exist.
The decision on what to hardlink is done here.   The code relevant to rmeta and bin files is here.  Essentially link_stem should return None if unit.mode.is_check().  Also, the comment about "non-lib targets" in calc_outputs should be removed since it is no longer true (although a new comment should be added that it uses the term lib even for binary targets, which is somewhat confusing).

@nrc I was under the impression that rls read the rmeta files somewhere, but I don't see that anywhere.  Are they used anywhere by rls?  I believe rustdoc reads them, but does anything else?

Oh, and as a side note, it might be nice if calc_outputs also checked if any of the paths it computes are duplicated and warn or error, since that would likely indicate a bug in cargo.


Update MSDN links, Part 11

@NextTurn : Thanks for your contribution! The author, @anuragm, has been notified to review your proposed change.

We're reviewing this pull request.

Hi @NextTurn - Could you remove versioning in your relative links, or indicate why version information might be necessary for a relative link that you use? We're reviewing your other pending PRs (titled "Update MSDN links"). When version information is removed or confirmed as necessary, we'll move each commit to the private repo so we can test the links on staging before merging the content updates. Thanks.

There is no version information in this PR.

Thank you @NextTurn. Thanks again for your contribution.
Private repo PR 71717 replaces this pull request. We moved the commit to the private repo so we can validate formatting changes by using automated validation and staging. We'll close this PR now.


fixes for dynamic resizing

Description
Fixes some quirks that came with the DynamicResizing patch:

display issues in firefox with overflow in #content

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Verification Process

[ ] local build and check
[ ] public build and check

Release Notes

Fixed an issue where Firefox was not able to correctly display status pages devices, functions, rooms

Contributing checklist

[x] My code follows the code style of this project.
[x] I have read the CONTRIBUTING and LICENSE document.
[x] I fully agree to distribute my changes under Apache 2.0 license.

Why are you integrating your fixes in 0024-WebUI-Filter-Operations and not in the DynamicResizing WebUI patch?

Why are you integrating your fixes in 0024-WebUI-Filter-Operations and not in the DynamicResizing WebUI patch?

Since in this patch we touch the exact same files that are needed for the fix and there are mostly the same lines affected. I think this is one of the edge cases where you can not separate all changes into only one patch but have to touch others as well.
Please advice.

@jens-maus: An alternative approach could be to revert changes on the tables from both relevant patch files 0024-WebUI-Filter-Operations and 0039-WebUI-Fix-Style-Glitches and bring all fixes in 0154-WebUI-DynamicResizing. What do you think?

This looks good now! I tested it with Firefox, Chrome and Safari and all three show the expected behaviour. Will merge this PR accordingly!


Canonical URL not working with V2 theme

Issue Summary
Canonical URL not used in post's rel="canonical" element.
To Reproduce

Specify canonical URL in Post Settings → Meta Data
Publish
View source for newly published post

Results
Expected: rel="canonical" element in <head> uses specified canonical URL
Actual: canonical element still uses post's permalink
Technical details
Ghost 2.25.9, Node 8.10.0
Additional information

I see this is expected behavior for themes using the V1 API, but I'm using V2 in my theme/package.json
Associated forum post
I don't know if this matters but FWIW the site I'm working on is currently set to private

Hi @magnolialogic! I tried reproducing this issue but with no success. Tried both default Casper and custom themes and canonical URL set from post settings menu appeared when theme was configured with API v2:
    "ghost-api": "v2"

and was defaulting to the permalink when this attribute was not present of version was set to v1 (as expected).
Also tried putting the site into private mode and it didn't effect the behavior.
Can you please provide more details about the environment you are using: theme, theme configuration, db version, Ghost instance configuration etc.? Might also help if you post a sample data you are setting in Canonical URL for the post (I have tried both local reference and external one when testing).


[Bug] Input delay

Platform

Device: Desktop
OS: Windows
Browser: Edge
Browser Version: 120.0.2210.61
Better xCloud Version: 2.0.5

Describe the bug
I've been using Better xCloud to strech my resolution, but after 2.0 I've got some input delay when playing.
I tried re-installing, clearing cache, deactivating every setting, go back to older versions, but nothing seems to work. Even reloading the page doesn't fix it.

Does it still happen when you disable Better xCloud?

No.

Ok, so I decided to test the new update, and saw that my ping is above 120, so I think it's more a ping problem rather than input.
Do you want me to close this thread?


Query Aurora instances per cluster

What does this PR do?
Updates Aurora autodiscovery to query database instances per cluster.
Motivation
DescribeDBInstances has a limit of 100 instances per call. As a first pass at handling large AWS RDS installations this will allow the agent to autodiscover up to 100 instances per cluster, rather than 100 instances total.
Describe how you validated your changes

Mocked tests have been updated but full confirmation can only be done in a deployment to a customer with a large number of Aurora instances being autodiscovered.
Possible Drawbacks / Trade-offs
Additional Notes

/merge

/remove

/merge


Refactor/code spring-javaformat

centralize and add Java type constants
update generic type usage for Enum classes
integrate and configure Spring Java Format
run spring-javaformat:apply to format this project
add Spring Java Format validation workflow

Isn't the default tab indentation in Spring Java Format excessive? Most indentations tend to use a single tab. It's possible to adjust the 'indentation-style' setting.

The community currently needs the workflow to run integration tests the most. For example, complete an install within a workflow, then pull the test code and run it for testing. Afterward, we want to include the documentation in the report so it can be downloaded for review.

Isn't the default tab indentation in Spring Java Format excessive? Most indentations tend to use a single tab. It's possible to adjust the 'indentation-style' setting.

By default tabs are used for indenting formatted code.
see  https://github.com/spring-io/spring-javaformat?tab=readme-ov-file#indenting-with-spaces


Create printnightmare-cve-2021-1675

Here's a few queries to detect usage of cve-2021-1675 to gain privilege escalation.

Rebuilding to fit template.


Running out of disk space leads to strange behaviour

CrateDB version
5.1.3
CrateDB setup information
Small single-node CrateDB Cloud "cluster" (CR0) with 4 GiB disk.
Problem description
It's easily possibly to exhaust the disk to 100%, even with the high watermark limiting usage to 95%.
When that happens, cluster is in a funny state -> sys.health shows green, however the master is lost, and no operations can be performed on the cluster.
Steps to Reproduce
Create a small cluster with 4 GiB (or less) of disk.
Run a few COPY FROM jobs that insert a lot of data.
See how disk is completely used up.
Actual Result
See above.
Expected Result
Would expect:

The high watermark to be obeyed
sys.health to not return GREEN

Thanks for the report.
There are some spots where disk space isn't checked before allocating shards.
Some of these are already addressed in 5.3 (E.g. https://github.com/crate/crate/commit/7db0f2e7f313d39eebfa95456fc4759b9a917d01) but there could be more.
Did you import to a partitioned table, or a regular table?

Did you import to a partitioned table, or a regular table?

Regular!

I had a closer look at this and it seems this is at least somewhat expected.
By default the master node collects information about shard sizes and disk usages ever 30 seconds and then runs the disk threshold checks. If the flood check fails it then triggers an update that sets the tables to read-only.
This implies that there's a window of a bit more than 30 seconds where you can run out of disk space without the flood protection, if you manage to write enough data to go from "everything is good" to full.
For 4GB disks it might be appropriate to lower the thresholds to account for that.
I'm not sure if there's a practical way to improve the behavior without having a negative performance impact.


Bind update_property in EditorProperty subclasses

The bind for update_property in EditorProperty is a virtual method:
https://github.com/godotengine/godot/blob/91f7df2b4c178b7d897bd34615a0217b77394343/editor/editor_inspector.cpp#L894-L896
But subclasses defined by the engine doesn't bind the actual implementations.
I need them to be exposed for my own engine module so I though it would be ok to have this added upstream.

I'm not sure I understand why you need it exposed specifically in implementation classes?
update_property is available in the EditorProperty public API and can thus be used in all derived classes, also from the scripting API.

I need it because when I call it with undoredo it says the method doesn't exist in the class as it is only vitrtual and I'm calling the method of the types my PR touches.
Whith these changes it works as expected.

I'm closing this as I think it's better not to modify internal components of the engine for custom module usecases. I'll make a workarround until the PR to use callables with Umdoredo is merged.


Assignment4 complete

Assignment 4 repo at https://github.com/cu-ecen-aeld/assignment-4-betai97 - test cases failing

Assignment 4 repo at https://github.com/cu-ecen-aeld/assignment-4-betai97 - test cases failing

I think it's passing: https://github.com/cu-ecen-aeld/assignment-4-betai97/actions/runs/6295167392


Implemented this library using java language

Hi, plainheart. Thank you very much for developing such a useful toolkit. Recently I wanted to use bing translate to implement some translation functions, but unfortunately there is no similar toolkit in the java ecosystem, so I imitated this project. A java version of the tool was created. If you want, can you link it to Related projects in the README?

Cool. Please open a PR to add your repo to the related project list.

Cool. Please open a PR to add your repo to the related project list.

I submitted this PR : https://github.com/plainheart/bing-translate-api/pull/30


CPU Spike when Titling List

(extension version): 1.4.7
(browser version): Google Chrome/Canary Microsoft Edge <IP_ADDRESS>
(operating system): Windows 10 x64
(problem description): CPU usage spike for browser process when waiting for the delayed text to appear for list title.
(actual behavior): I have roughly 15 lists with 7-10 sites on each list, anytime I create a new list and go to give it a name, better onetab slows down dramatically. It slows down so much that I type out the name of the list, then after roughly 10 seconds it appears, as in no letters appear for at least 10 seconds. I opened windows task manager to check resources and during the 10-second delay, the CPU usage of my browsers process jumps from 2% to 20% then back down once the letters appear.
(expected behavior): Smooth typing when titling list, and no delay or lag.
重现步骤 (reproduce step):

I have the same issue on Firefox, it's really annoying! Makes me not want to Better Onetab as it's borderline unworkable. It's so slow to load each page of lists, it takes a few seconds.
 I don't have even that many tab groups! Like, 40 I think, with maybe 200 tabs in total spread between them.
And the naming delay is painful. I literally have to type it out in notepad, copy and paste it across! It's so slow.
I love this addon, there's some great work been put in here. But this problem undermines it, drastically.

Same here. Great functionality but sadly this is too much for my computer, it's super slow. Onetab lacks many of the features here but I can't use this extension if it takes half a minute to rename a list, 10 seconds to open it, and a lot of delay in general. @joshbrenner mentioned having ~40 lists with 200 tabs, I have 15 lists with 100 tabs and rhis happens to me too.
Good luck guys.


BUG:  links is bugged

Describe the bug
So I made a website using the fonctionnality of pages but when I click on the link, it bug the app resulting in an API 404 error code but when I use the URL bar, it works normally.
(In the code it's : French History)
To Reproduce
Steps to reproduce the behavior:

Go to my site : buss://history.fr
Click on French History in the link's bar
See the error
Go to dingle.it to reload the whole page or else it's impossible to go to the main page (don't know why)
Then go to buss://history.fr/french and see the page (and also links that points to the main page bug)

Expected behavior
It should show the page, not an 404 error
Screenshots
With the link :

With the URL :

Desktop

OS: Windows 10
Version 1.3.0 compiled version

Yeah lol

My bad


This is an amazing work, when will the code be released?

@wcy1122

Any updates?) keen to compare it with Masked Scene Contrast on my datasets 😶🌫

Sorry for the delay. Chengyao has already fully transferred to the VLM domain.


Scala Common Enrich: add adapter to pre-process Marketo webhooks

http://developers.marketo.com/documentation/webhooks/
https://docs.marketo.com/display/public/DOCS/Create+a+Webhook

Missing milestone.


安装了Xtuner但是无法识别到

(trainer) [root@VM-0-12-centos tzz]# pip install -U 'xtuner[deepspeed]'
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: xtuner[deepspeed] in /root/usr/lib/python3.10/site-packages (0.1.13)
Requirement already satisfied: bitsandbytes>=0.40.0.post4 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.41.3.post2)
Requirement already satisfied: datasets>=2.16.0 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (2.16.1)
Requirement already satisfied: einops in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.4.1)
Requirement already satisfied: lagent>=0.1.2 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.2.1)
Requirement already satisfied: mmengine>=0.10.1 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.10.3)
Requirement already satisfied: openpyxl in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (3.1.2)
Requirement already satisfied: peft>=0.4.0 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.6.0)
Requirement already satisfied: scipy in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (1.11.4)
Requirement already satisfied: SentencePiece in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.1.99)
Requirement already satisfied: tiktoken in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.5.2)
Requirement already satisfied: torch in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (2.0.0)
Requirement already satisfied: transformers!=4.34.1,!=4.35.0,!=4.35.1,!=4.35.2,>=4.32.1 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (4.37.2)
Requirement already satisfied: transformers-stream-generator in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.0.4)
Requirement already satisfied: deepspeed>=0.12.3 in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (0.13.1)
Requirement already satisfied: mpi4py-mpich in /root/usr/lib/python3.10/site-packages (from xtuner[deepspeed]) (3.1.2)
Requirement already satisfied: filelock in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (3.13.1)
Requirement already satisfied: numpy>=1.17 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (1.26.1)
Requirement already satisfied: pyarrow>=8.0.0 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (14.0.1)
Requirement already satisfied: pyarrow-hotfix in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (0.6)
Requirement already satisfied: dill<0.3.8,>=0.3.0 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (0.3.7)
Requirement already satisfied: pandas in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (2.1.4)
Requirement already satisfied: requests>=2.19.0 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (2.31.0)
Requirement already satisfied: tqdm>=4.62.1 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (4.66.1)
Requirement already satisfied: xxhash in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (3.4.1)
Requirement already satisfied: multiprocess in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (0.70.15)
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /root/usr/lib/python3.10/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets>=2.16.0->xtuner[deepspeed]) (2023.10.0)
Requirement already satisfied: aiohttp in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (3.9.1)
Requirement already satisfied: huggingface-hub>=0.19.4 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (0.19.4)
Requirement already satisfied: packaging in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /root/usr/lib/python3.10/site-packages (from datasets>=2.16.0->xtuner[deepspeed]) (6.0.1)
Requirement already satisfied: hjson in /root/usr/lib/python3.10/site-packages (from deepspeed>=0.12.3->xtuner[deepspeed]) (3.1.0)
Requirement already satisfied: ninja in /root/usr/lib/python3.10/site-packages (from deepspeed>=0.12.3->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: psutil in /root/usr/lib/python3.10/site-packages (from deepspeed>=0.12.3->xtuner[deepspeed]) (5.9.5)
Requirement already satisfied: py-cpuinfo in /root/usr/lib/python3.10/site-packages (from deepspeed>=0.12.3->xtuner[deepspeed]) (9.0.0)
Requirement already satisfied: pydantic in /root/usr/lib/python3.10/site-packages (from deepspeed>=0.12.3->xtuner[deepspeed]) (1.9.2)
Requirement already satisfied: pynvml in /root/usr/lib/python3.10/site-packages (from deepspeed>=0.12.3->xtuner[deepspeed]) (11.5.0)
Requirement already satisfied: arxiv in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (2.1.0)
Requirement already satisfied: distro in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (1.9.0)
Requirement already satisfied: func-timeout in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (4.3.5)
Requirement already satisfied: google-search-results in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (2.4.2)
Requirement already satisfied: griffe in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (0.40.0)
Requirement already satisfied: json5 in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (0.9.14)
Requirement already satisfied: jsonschema in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (4.20.0)
Requirement already satisfied: jupyter in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (1.0.0)
Requirement already satisfied: jupyter-client in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (8.6.0)
Requirement already satisfied: phx-class-registry in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (4.1.0)
Requirement already satisfied: pillow in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (9.5.0)
Requirement already satisfied: python-pptx in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (0.6.23)
Requirement already satisfied: timeout-decorator in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (0.5.0)
Requirement already satisfied: typing-extensions in /root/usr/lib/python3.10/site-packages (from lagent>=0.1.2->xtuner[deepspeed]) (4.9.0)
Requirement already satisfied: addict in /root/usr/lib/python3.10/site-packages (from mmengine>=0.10.1->xtuner[deepspeed]) (2.4.0)
Requirement already satisfied: matplotlib in /root/usr/lib/python3.10/site-packages (from mmengine>=0.10.1->xtuner[deepspeed]) (3.8.2)
Requirement already satisfied: rich in /root/usr/lib/python3.10/site-packages (from mmengine>=0.10.1->xtuner[deepspeed]) (13.7.0)
Requirement already satisfied: termcolor in /root/usr/lib/python3.10/site-packages (from mmengine>=0.10.1->xtuner[deepspeed]) (2.4.0)
Requirement already satisfied: yapf in /root/usr/lib/python3.10/site-packages (from mmengine>=0.10.1->xtuner[deepspeed]) (0.40.2)
Requirement already satisfied: opencv-python>=3 in /root/usr/lib/python3.10/site-packages (from mmengine>=0.10.1->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: accelerate>=0.21.0 in /root/usr/lib/python3.10/site-packages (from peft>=0.4.0->xtuner[deepspeed]) (0.21.0)
Requirement already satisfied: safetensors in /root/usr/lib/python3.10/site-packages (from peft>=0.4.0->xtuner[deepspeed]) (0.4.2)
Requirement already satisfied: sympy in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (1.12)
Requirement already satisfied: networkx in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (3.2.1)
Requirement already satisfied: jinja2 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (3.1.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (11.7.99)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (11.7.101)
Requirement already satisfied: nvidia-cudnn-cu11==<IP_ADDRESS> in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-cublas-cu11==<IP_ADDRESS> in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-cufft-cu11==<IP_ADDRESS> in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-curand-cu11==<IP_ADDRESS> in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-cusolver-cu11==<IP_ADDRESS> in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-cusparse-cu11==<IP_ADDRESS> in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (2.14.3)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (11.7.91)
Requirement already satisfied: triton==2.0.0 in /root/usr/lib/python3.10/site-packages (from torch->xtuner[deepspeed]) (2.0.0)
Requirement already satisfied: setuptools in /root/usr/lib/python3.10/site-packages (from nvidia-cublas-cu11==<IP_ADDRESS>->torch->xtuner[deepspeed]) (65.5.0)
Requirement already satisfied: wheel in /root/usr/lib/python3.10/site-packages (from nvidia-cublas-cu11==<IP_ADDRESS>->torch->xtuner[deepspeed]) (0.42.0)
Requirement already satisfied: cmake in /root/usr/lib/python3.10/site-packages (from triton==2.0.0->torch->xtuner[deepspeed]) (3.28.0)
Requirement already satisfied: lit in /root/usr/lib/python3.10/site-packages (from triton==2.0.0->torch->xtuner[deepspeed]) (17.0.6)
Requirement already satisfied: regex!=2019.12.17 in /root/usr/lib/python3.10/site-packages (from transformers!=4.34.1,!=4.35.0,!=4.35.1,!=4.35.2,>=4.32.1->xtuner[deepspeed]) (2023.10.3)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /root/usr/lib/python3.10/site-packages (from transformers!=4.34.1,!=4.35.0,!=4.35.1,!=4.35.2,>=4.32.1->xtuner[deepspeed]) (0.15.1)
Requirement already satisfied: et-xmlfile in /root/usr/lib/python3.10/site-packages (from openpyxl->xtuner[deepspeed]) (1.1.0)
Requirement already satisfied: attrs>=17.3.0 in /root/usr/lib/python3.10/site-packages (from aiohttp->datasets>=2.16.0->xtuner[deepspeed]) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /root/usr/lib/python3.10/site-packages (from aiohttp->datasets>=2.16.0->xtuner[deepspeed]) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /root/usr/lib/python3.10/site-packages (from aiohttp->datasets>=2.16.0->xtuner[deepspeed]) (1.9.4)
Requirement already satisfied: frozenlist>=1.1.1 in /root/usr/lib/python3.10/site-packages (from aiohttp->datasets>=2.16.0->xtuner[deepspeed]) (1.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /root/usr/lib/python3.10/site-packages (from aiohttp->datasets>=2.16.0->xtuner[deepspeed]) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /root/usr/lib/python3.10/site-packages (from aiohttp->datasets>=2.16.0->xtuner[deepspeed]) (4.0.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /root/usr/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.16.0->xtuner[deepspeed]) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /root/usr/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.16.0->xtuner[deepspeed]) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /root/usr/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.16.0->xtuner[deepspeed]) (1.26.18)
Requirement already satisfied: certifi>=2017.4.17 in /root/usr/lib/python3.10/site-packages (from requests>=2.19.0->datasets>=2.16.0->xtuner[deepspeed]) (2023.11.17)
Requirement already satisfied: feedparser==6.0.10 in /root/usr/lib/python3.10/site-packages (from arxiv->lagent>=0.1.2->xtuner[deepspeed]) (6.0.10)
Requirement already satisfied: sgmllib3k in /root/usr/lib/python3.10/site-packages (from feedparser==6.0.10->arxiv->lagent>=0.1.2->xtuner[deepspeed]) (1.0.0)
Requirement already satisfied: colorama>=0.4 in /root/usr/lib/python3.10/site-packages (from griffe->lagent>=0.1.2->xtuner[deepspeed]) (0.4.6)
Requirement already satisfied: MarkupSafe>=2.0 in /root/usr/lib/python3.10/site-packages (from jinja2->torch->xtuner[deepspeed]) (2.1.3)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /root/usr/lib/python3.10/site-packages (from jsonschema->lagent>=0.1.2->xtuner[deepspeed]) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /root/usr/lib/python3.10/site-packages (from jsonschema->lagent>=0.1.2->xtuner[deepspeed]) (0.32.0)
Requirement already satisfied: rpds-py>=0.7.1 in /root/usr/lib/python3.10/site-packages (from jsonschema->lagent>=0.1.2->xtuner[deepspeed]) (0.15.2)
Requirement already satisfied: notebook in /root/usr/lib/python3.10/site-packages (from jupyter->lagent>=0.1.2->xtuner[deepspeed]) (7.0.7)
Requirement already satisfied: qtconsole in /root/usr/lib/python3.10/site-packages (from jupyter->lagent>=0.1.2->xtuner[deepspeed]) (5.5.1)
Requirement already satisfied: jupyter-console in /root/usr/lib/python3.10/site-packages (from jupyter->lagent>=0.1.2->xtuner[deepspeed]) (6.6.3)
Requirement already satisfied: nbconvert in /root/usr/lib/python3.10/site-packages (from jupyter->lagent>=0.1.2->xtuner[deepspeed]) (7.14.2)
Requirement already satisfied: ipykernel in /root/usr/lib/python3.10/site-packages (from jupyter->lagent>=0.1.2->xtuner[deepspeed]) (6.29.0)
Requirement already satisfied: ipywidgets in /root/usr/lib/python3.10/site-packages (from jupyter->lagent>=0.1.2->xtuner[deepspeed]) (8.1.1)
Requirement already satisfied: jupyter-core!=5.0.*,>=4.12 in /root/usr/lib/python3.10/site-packages (from jupyter-client->lagent>=0.1.2->xtuner[deepspeed]) (5.7.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /root/usr/lib/python3.10/site-packages (from jupyter-client->lagent>=0.1.2->xtuner[deepspeed]) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /root/usr/lib/python3.10/site-packages (from jupyter-client->lagent>=0.1.2->xtuner[deepspeed]) (25.1.2)
Requirement already satisfied: tornado>=6.2 in /root/usr/lib/python3.10/site-packages (from jupyter-client->lagent>=0.1.2->xtuner[deepspeed]) (6.4)
Requirement already satisfied: traitlets>=5.3 in /root/usr/lib/python3.10/site-packages (from jupyter-client->lagent>=0.1.2->xtuner[deepspeed]) (5.14.1)
Requirement already satisfied: contourpy>=1.0.1 in /root/usr/lib/python3.10/site-packages (from matplotlib->mmengine>=0.10.1->xtuner[deepspeed]) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /root/usr/lib/python3.10/site-packages (from matplotlib->mmengine>=0.10.1->xtuner[deepspeed]) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /root/usr/lib/python3.10/site-packages (from matplotlib->mmengine>=0.10.1->xtuner[deepspeed]) (4.47.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /root/usr/lib/python3.10/site-packages (from matplotlib->mmengine>=0.10.1->xtuner[deepspeed]) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /root/usr/lib/python3.10/site-packages (from matplotlib->mmengine>=0.10.1->xtuner[deepspeed]) (3.1.1)
Requirement already satisfied: pytz>=2020.1 in /root/usr/lib/python3.10/site-packages (from pandas->datasets>=2.16.0->xtuner[deepspeed]) (2023.3.post1)
Requirement already satisfied: tzdata>=2022.1 in /root/usr/lib/python3.10/site-packages (from pandas->datasets>=2.16.0->xtuner[deepspeed]) (2023.3)
Requirement already satisfied: lxml>=3.1.0 in /root/usr/lib/python3.10/site-packages (from python-pptx->lagent>=0.1.2->xtuner[deepspeed]) (4.9.4)
Requirement already satisfied: XlsxWriter>=0.5.7 in /root/usr/lib/python3.10/site-packages (from python-pptx->lagent>=0.1.2->xtuner[deepspeed]) (3.1.9)
Requirement already satisfied: markdown-it-py>=2.2.0 in /root/usr/lib/python3.10/site-packages (from rich->mmengine>=0.10.1->xtuner[deepspeed]) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /root/usr/lib/python3.10/site-packages (from rich->mmengine>=0.10.1->xtuner[deepspeed]) (2.17.2)
Requirement already satisfied: mpmath>=0.19 in /root/usr/lib/python3.10/site-packages (from sympy->torch->xtuner[deepspeed]) (1.3.0)
Requirement already satisfied: importlib-metadata>=6.6.0 in /root/usr/lib/python3.10/site-packages (from yapf->mmengine>=0.10.1->xtuner[deepspeed]) (6.11.0)
Requirement already satisfied: platformdirs>=3.5.1 in /root/usr/lib/python3.10/site-packages (from yapf->mmengine>=0.10.1->xtuner[deepspeed]) (4.1.0)
Requirement already satisfied: tomli>=2.0.1 in /root/usr/lib/python3.10/site-packages (from yapf->mmengine>=0.10.1->xtuner[deepspeed]) (2.0.1)
Requirement already satisfied: zipp>=0.5 in /root/usr/lib/python3.10/site-packages (from importlib-metadata>=6.6.0->yapf->mmengine>=0.10.1->xtuner[deepspeed]) (3.17.0)
Requirement already satisfied: mdurl~=0.1 in /root/usr/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->mmengine>=0.10.1->xtuner[deepspeed]) (0.1.2)
Requirement already satisfied: six>=1.5 in /root/usr/lib/python3.10/site-packages (from python-dateutil>=2.8.2->jupyter-client->lagent>=0.1.2->xtuner[deepspeed]) (1.16.0)
Requirement already satisfied: comm>=0.1.1 in /root/usr/lib/python3.10/site-packages (from ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.2.1)
Requirement already satisfied: debugpy>=1.6.5 in /root/usr/lib/python3.10/site-packages (from ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.8.0)
Requirement already satisfied: ipython>=7.23.1 in /root/usr/lib/python3.10/site-packages (from ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (8.21.0)
Requirement already satisfied: matplotlib-inline>=0.1 in /root/usr/lib/python3.10/site-packages (from ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.1.6)
Requirement already satisfied: nest-asyncio in /root/usr/lib/python3.10/site-packages (from ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.6.0)
Requirement already satisfied: widgetsnbextension~=4.0.9 in /root/usr/lib/python3.10/site-packages (from ipywidgets->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (4.0.9)
Requirement already satisfied: jupyterlab-widgets~=3.0.9 in /root/usr/lib/python3.10/site-packages (from ipywidgets->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (3.0.9)
Requirement already satisfied: prompt-toolkit>=3.0.30 in /root/usr/lib/python3.10/site-packages (from jupyter-console->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (3.0.43)
Requirement already satisfied: beautifulsoup4 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (4.12.2)
Requirement already satisfied: bleach!=5.0.0 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (6.1.0)
Requirement already satisfied: defusedxml in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.7.1)
Requirement already satisfied: jupyterlab-pygments in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.3.0)
Requirement already satisfied: mistune<4,>=2.0.3 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (3.0.2)
Requirement already satisfied: nbclient>=0.5.0 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.9.0)
Requirement already satisfied: nbformat>=5.7 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (5.9.2)
Requirement already satisfied: pandocfilters>=1.4.1 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.5.1)
Requirement already satisfied: tinycss2 in /root/usr/lib/python3.10/site-packages (from nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.2.1)
Requirement already satisfied: jupyter-server<3,>=2.4.0 in /root/usr/lib/python3.10/site-packages (from notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.12.5)
Requirement already satisfied: jupyterlab-server<3,>=2.22.1 in /root/usr/lib/python3.10/site-packages (from notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.25.2)
Requirement already satisfied: jupyterlab<5,>=4.0.2 in /root/usr/lib/python3.10/site-packages (from notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (4.0.12)
Requirement already satisfied: notebook-shim<0.3,>=0.2 in /root/usr/lib/python3.10/site-packages (from notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.2.3)
Requirement already satisfied: qtpy>=2.4.0 in /root/usr/lib/python3.10/site-packages (from qtconsole->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.4.1)
Requirement already satisfied: webencodings in /root/usr/lib/python3.10/site-packages (from bleach!=5.0.0->nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.5.1)
Requirement already satisfied: decorator in /root/usr/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (5.1.1)
Requirement already satisfied: jedi>=0.16 in /root/usr/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.19.1)
Requirement already satisfied: stack-data in /root/usr/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.6.3)
Requirement already satisfied: exceptiongroup in /root/usr/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.2.0)
Requirement already satisfied: pexpect>4.3 in /root/usr/lib/python3.10/site-packages (from ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (4.9.0)
Requirement already satisfied: anyio>=3.1.0 in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (3.7.1)
Requirement already satisfied: argon2-cffi in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (23.1.0)
Requirement already satisfied: jupyter-events>=0.9.0 in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.9.0)
Requirement already satisfied: jupyter-server-terminals in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.5.2)
Requirement already satisfied: overrides in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (7.4.0)
Requirement already satisfied: prometheus-client in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.19.0)
Requirement already satisfied: send2trash>=1.8.2 in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.8.2)
Requirement already satisfied: terminado>=0.8.3 in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.18.0)
Requirement already satisfied: websocket-client in /root/usr/lib/python3.10/site-packages (from jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.7.0)
Requirement already satisfied: async-lru>=1.0.0 in /root/usr/lib/python3.10/site-packages (from jupyterlab<5,>=4.0.2->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.0.4)
Requirement already satisfied: jupyter-lsp>=2.0.0 in /root/usr/lib/python3.10/site-packages (from jupyterlab<5,>=4.0.2->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.2.2)
Requirement already satisfied: babel>=2.10 in /root/usr/lib/python3.10/site-packages (from jupyterlab-server<3,>=2.22.1->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.14.0)
Requirement already satisfied: fastjsonschema in /root/usr/lib/python3.10/site-packages (from nbformat>=5.7->nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.19.1)
Requirement already satisfied: wcwidth in /root/usr/lib/python3.10/site-packages (from prompt-toolkit>=3.0.30->jupyter-console->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.2.12)
Requirement already satisfied: soupsieve>1.2 in /root/usr/lib/python3.10/site-packages (from beautifulsoup4->nbconvert->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.5)
Requirement already satisfied: sniffio>=1.1 in /root/usr/lib/python3.10/site-packages (from anyio>=3.1.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.3.0)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /root/usr/lib/python3.10/site-packages (from jedi>=0.16->ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.8.3)
Requirement already satisfied: python-json-logger>=2.0.4 in /root/usr/lib/python3.10/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.0.7)
Requirement already satisfied: rfc3339-validator in /root/usr/lib/python3.10/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.1.4)
Requirement already satisfied: rfc3986-validator>=0.1.1 in /root/usr/lib/python3.10/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.1.1)
Requirement already satisfied: ptyprocess>=0.5 in /root/usr/lib/python3.10/site-packages (from pexpect>4.3->ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.7.0)
Requirement already satisfied: argon2-cffi-bindings in /root/usr/lib/python3.10/site-packages (from argon2-cffi->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (21.2.0)
Requirement already satisfied: executing>=1.2.0 in /root/usr/lib/python3.10/site-packages (from stack-data->ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.0.1)
Requirement already satisfied: asttokens>=2.1.0 in /root/usr/lib/python3.10/site-packages (from stack-data->ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.4.1)
Requirement already satisfied: pure-eval in /root/usr/lib/python3.10/site-packages (from stack-data->ipython>=7.23.1->ipykernel->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (0.2.2)
Requirement already satisfied: fqdn in /root/usr/lib/python3.10/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.5.1)
Requirement already satisfied: isoduration in /root/usr/lib/python3.10/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (20.11.0)
Requirement already satisfied: jsonpointer>1.13 in /root/usr/lib/python3.10/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.4)
Requirement already satisfied: uri-template in /root/usr/lib/python3.10/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.3.0)
Requirement already satisfied: webcolors>=1.11 in /root/usr/lib/python3.10/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.13)
Requirement already satisfied: cffi>=1.0.1 in /root/usr/lib/python3.10/site-packages (from argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.16.0)
Requirement already satisfied: pycparser in /root/usr/lib/python3.10/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (2.21)
Requirement already satisfied: arrow>=0.15.0 in /root/usr/lib/python3.10/site-packages (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (1.3.0)
Requirement already satisfied: types-python-dateutil>=2.8.10 in /root/usr/lib/python3.10/site-packages (from arrow>=0.15.0->isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->lagent>=0.1.2->xtuner[deepspeed]) (<IP_ADDRESS>40106)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
(trainer) [root@VM-0-12-centos tzz]# xtuner list-cfg
Traceback (most recent call last):
  File "/root/usr/lib/python3.10/site-packages/xtuner/tools/list_cfg.py", line 4, in <module>
    from xtuner.configs import cfgs_name_path
ModuleNotFoundError: No module named 'xtuner'

pip所安装的位置有问题 显示安装到了 /root/usr/lib/python3.10/site-packages
而非你的anaconda环境
需要确保pip是你anaconda python环境所对应的pip命令

重装miniconda解决了问题！！


 For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method. (Parameter 'name')'

Why is that?

See https://github.com/AngleSharp/AngleSharp/commit/24708a2038baf481cc96b635e95c21cbaa14c68d for an idea for an adjustment.
The exception you're seeing is a caught one


share shorten url to whatsapp, gmail, linkedin without copying it

i have add functionality that user can able to share shorten url to whatsapp, gmail, linkedin without copying it .
solves #74

Hey @rahul1841, I am merging this PR.
One suggestion, Try to  use code formatter like prettier to format the code.