Apply Changes from Source Control is not working as expected

Copying from @ramyaramdasan in https://github.com/ServiceNow/sncicd_githubworkflow/issues/4.
Apply Changes from Source Control is not working as expected. On Flow Designer and Azure Task I have used ‘Apply Changes from Source Control’ and it updated the ServiceNow instance configured as Target Instance with code from the branch assigned to the branch input. For example, both these actions had TEST as Target Instance and DEV as the branch name. However, the equivalent GitHub Action doesn’t seem to operate the same way.
Based on sample workflow provided in the repo, I configured SNOW_SOURCE_INSTANCE with the TEST instance domain name and it didn’t get updated with code from source control. When we are applying changed from Source Control to a ServiceNow Instance, shouldn't the latter be called SNOW_TARGET_INSTANCE? Also, I didn’t see an input option for source control branch.

@ramyaramdasan
Quick update for you. So yes, we confirmed that the default behavior of the "branch_name" optional parameter for the API, if provided, is to switch the branch and apply the changes from Git remote.
One of our engineers looked into code and noted that the original developer was actually trying to be more user-friendly and pass in the branch name from whichever branch triggered the build. So for example, if you have a feature branch with a pull request, that feature branch would be the value passed in. If you're triggering a CD build via push to master, then the master branch would be the value passed in. In this way, the branch_name value was hidden from the end user, because usually you want to run your build to test the branch you are merging, or deploying from.
There is, however, a bug. The branch_name value is being passed in as the body of the request, instead of as a query parameter as designed in the API.
https://github.com/ServiceNow/sncicd-apply-changes/blob/master/src/App.ts#L107
That means it's not actually working, so the API will assume there is no branch_name value passed in, and Apply Remote Changes for whichever branch is currently active on that instance.
Our SN team needs to loop together on how we'd approach solving this.

verified with my test pipeline that directly supplying parameter with branch name changed the branch on instance.
- name: ServiceNow CI/CD Apply Changes
uses<EMAIL_ADDRESS>env:
snowUsername: ${{ secrets.SN_USERNAME }}
snowPassword: ${{ secrets.SN_PASSWORD }}
snowSourceInstance: ${{ secrets.SN_DEV_INSTANCE }}
appSysID: ${{ secrets.SN_APP_SYSID }}
branch: 'master'
next to figure out how to have dynamic branch name passed in

@chiarnglin,
I use the below line of code to extract branch name(the one into which the push happened) and pass it as an input - ${{ steps.extract_branch.outputs.branch }}to an action in the flow. I use it in our hotfix deployment workflow to allow developers to create their own branch. I need to extract the branch names to use it in subsequent actions for merging and deleting the branch.
steps:
- uses: actions/checkout@v2
- name: Branch name
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF##*/})"
id: extract_branch
Thanks
Ramya

verified with my test pipeline that directly supplying parameter with branch name changed the branch on instance.

  - name: ServiceNow CI/CD Apply Changes
    uses<EMAIL_ADDRESS>    env:
      snowUsername: ${{ secrets.SN_USERNAME }}
      snowPassword: ${{ secrets.SN_PASSWORD }}
      snowSourceInstance: ${{ secrets.SN_DEV_INSTANCE }}
      appSysID: ${{ secrets.SN_APP_SYSID }}
      branch: 'master'

next to figure out how to have dynamic branch name passed in

@chiarnglin , I verified this too. It does solve a major concern of wrong version of application getting published from the wrong branch. The branch switching done from the pipeline seems to handle everything internally.
I updated my pipeline to use 'dev' as branch for sncicd-apply-changes and triggered the pipeline by committing work to 'dev' branch from our DEV instance. I had my TEST instance pointing to 'main' branch. It didn't switch the branch on the instance, I also checked the sys_repo_config  to confirm the current branch which continued to be 'main'. While we have the ATF run on the Client Runner open on TEST instance, looks like it would still refer to the 'main' branch and not test the features that may be added to the dev branch.
Its not a showstopper, but I just thought of sharing the observations.

@ramyaramdasan
I tried in my pipeline your scenario with a little different flow. On a PR from dev branch to main I would send an apply-changes trigger to the testing instance but switching it to the dev branch. This way I get the latest version for running ATF. Once complete (failure or not) I would re-apply the main branch to reset back to what it should be.
If the test completed successfully the PR can be merged. Another apply-changes to the test instance is than required to get to the latest stage of the main branch.

@phifogg,
What I observed is that, the Actions in pipeline is taking care of picking the correct branch for applying changes and also publish the application from that branch. But for ATF to run the new code using the Scheduled Client Runner that I opened for TEST instance, which was pointing main, it didn't switch to dev. So, in a case where we also pushed a new ATF Test corresponding to the feature that was developed under dev branch, are you saying that the instance will be having it, though we don't see a branch switch happening in the instance.
The DPP Instances have been under maintenance past many days, so I didn't really get a chance to test it. I will do so when I get my access back.
Also, when you said - "Once complete (failure or not) I would re-apply the main branch to reset back to what it should be.", are you manually doing it or is there a way to do it from pipeline?
Thanks
Ramya

@ramyaramdasan
Are your ATF tests part of the same scoped application or separate? In my use case I kept the tests with the application. This way the new 'dev' tests will also be pulled from source control before executing. If your tests are in a separate scope you will need also to switch branches with another apply-from-sources action on that scope. I would though recommend to keep tests and application together.
For the re-applying main branch: I had this automated in the pipeline. Just another apply-from-sources with the main branch as target.

@phifogg
Yes, they are in the same scope and do get deployed along with rest of the code. Probably the only point unclear to me is -
Is apply-from-sources  different from sncicd-apply-changes? If they are same, once it is executed, do you see the Studio on that instance pointing to the branch for which you applied sncicd-apply-changes? We don't see that happening in our instances.

@ramyaramdasan
Yes, these actions are the same. Apply From Source is the name inside ServiceNow, sncicd-apply-changes is the name of the action on github.
Indeed, in my instance I did see the branch change in Studio. We can connect on a call if you want to see yours.

@phifogg
Oh that will be great. But unfortunately I don't have any instances to walk you through the issue. Once I have them, we can surely connect.

Sorry about the slow instance upgrades (there's some bug with the data center automation), and thank you Daniel for helping Ramya check in on this!

hi @phifogg,
The branch name when extracted dynamically and added as an input to the Action, the step fails
steps:
- uses: actions/checkout@v2
- name: Branch name
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF##*/})"
id: extract_branch
   - name: ServiceNow CI/CD Apply Changes
     uses<EMAIL_ADDRESS>     env:
      snowUsername: ${{ secrets.SN_USERNAME }}
      snowPassword: ${{ secrets.SN_PASSWORD }}
      snowSourceInstance: ${{ secrets.SN_TEST_INSTANCE }}
      appSysID: ${{ secrets.SN_APP_SYSID }}
      branch: '${{ steps.extract_branch.outputs.branch }}'

d

HI @ramyaramdasan
I have checked your setup in an example app on my side. You can see it here: https://github.com/phifogg/x_snc_jap
There has to be something wrong with your branch name. It does work just fine for me, a push to my 'instances/dev' branch will trigger the pipeline and push it to my test environment (by changing its branch) and reverting that as well.
I extract the branch name using this command:
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
id: extract_branch
Your version looks slightly different, if I use it the resulting branch name is just 'dev' - missing the 'instances/' part.

Hi @phifogg
Thanks for quickly checking. I did try with a few other branches with names - bug, hotfix, etc. But still it didn't work. For another workflow where I hard-coded the branch name 'dev' as input to this action, it indeed works. It switches branch automatically on the instance where its applying the changes.
I will try the command you provided above and check if that works. I will let you know the outcome.
Thanks
ramya

Hi @phifogg,
I took a closer look at your repo (https://github.com/phifogg/x_snc_jap) and workflow executions. You were using<EMAIL_ADDRESS>while I was trying<EMAIL_ADDRESS>I checked the releases and there was no 1.2.0 available. The branch extraction worked fine on 1.0.0 version, so something may have changed in 1.2.1 causing the step to fail.

@ramyaramdasan
I just shipped v2.0.0 for all the GitHub Actions, do you want to check to see if this is still a problem? (Or maybe if this is a really long and old Issue, we can create new ones if there are new problems!)

@chiarnglin,
Thanks for this note. I will definitely try the new versions on the workflows tied to Applications on the DPP Instances.
2.0.0 seems to be associated with Quebec/ Rome release. Are they Paris compatible? I wonder if I should update all the workflows currently in use on our live instances also with 2.0.0. Please suggest.
Thanks
Ramya

The APIs are backwards compatible, so the GitHub Actions should be as well... but if that's not the case and you find a bug definitely let us know!!

Cool! I shall keep you posted if any issues. Thanks much!

Closing issue as no more feedback - assuming it works. If not please raise another issue. Thanks.


Upgrade angular-drag-and-drop-lists

Updating the angular-drag-and-drop library should improve performance and introduce a simpler placeholder positioning algorithm.
One case where that matters is on 'dragenter', an optimization was made here https://github.com/marceljuenemann/angular-drag-and-drop-lists/blob/master/angular-drag-and-drop-lists.js#L292
Although it's a major change, it shouldn't break anything according to their release notes.

Thanks!


Error building extension 'MultiScaleDeformableAttention' when running sample from web site.

Bug
When running the sample app, I get these errors:
Could not load the custom kernel for multi-scale deformable attention: Error building extension 'MultiScaleDeformableAttention': [1/2] /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output ms_deform_attn_cuda.cuda.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/TH -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/THC -isystem /usr/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu -o ms_deform_attn_cuda.cuda.o
FAILED: ms_deform_attn_cuda.cuda.o
/usr/bin/nvcc --generate-dependencies-with-compile --dependency-output ms_deform_attn_cuda.cuda.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/TH -isystem /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/torch/include/THC -isystem /usr/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu -o ms_deform_attn_cuda.cuda.o
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu:19:9: warning: #pragma once in main file
   19 | #pragma once
      |         ^~~~
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu:19:9: warning: #pragma once in main file
   19 | #pragma once
      |         ^~~~
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(261): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_im2col_cuda(cudaStream_t, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(69): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(762): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(872): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(331): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(436): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(544): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_im2col_cuda.cuh(649): warning #177-D: variable "q_col" was declared but never referenced
          detected during instantiation of "void ms_deformable_col2im_cuda(cudaStream_t, const scalar_t *, const scalar_t *, const int64_t *, const int64_t *, const scalar_t *, const scalar_t *, int, int, int, int, int, int, int, scalar_t *, scalar_t *, scalar_t *) [with scalar_t=double]"
/mnt/programming/CurrentDevelopment/DoclingTest/venv/lib/python3.10/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu(140): here

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.

...
Steps to reproduce
pip install docling
Then create a script with:
from docling.document_converter import DocumentConverter

source = "https://arxiv.org/pdf/2408.09869"  # PDF path or URL
converter = DocumentConverter()
result = converter.convert(source)
print(result.document.export_to_markdown())  # output: "### Docling Technical Report[...]"

Running this code produces the error above.
...
Docling version
Docling version: 2.12.0
Docling Core version: 2.10.0
Docling IBM Models version: 3.1.0
Docling Parse version: 3.0.0
...
Python version
Python 3.10.12
...

Do you get the error when installing Docling, i.e. pip install docling or when running it?
The error you posted looks like a compilation issue which, in case, would happen in the install phase.

It was happening when I ran the sample script:
from docling.document_converter import DocumentConverter

source = "https://arxiv.org/pdf/2408.09869"  # document per local path or URL
converter = DocumentConverter()
result = converter.convert(source)
print(result.document.export_to_markdown())  # output: "## Docling Technical Report[...]"

Yeah, I didn't get that either. It seemed to be trying to compile something in C++, if I recall correctly, before running the script.
At some point it ceased doing that. It now gives me this message:
/doclingtest/venv/lib/python3.10/site-packages/torch/cuda/init.py:129: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
return torch._C._cuda_getDeviceCount() > 0
and then it goes on to run, but without using my GPU (RTX 3050.). I have CUDA 12.6 installed and CUDA Toolkit 12.6 as well.

We never encounter (yet) such an error. I'm a bit suspicious about the CUDA and Pytorch versions. I would recommend making sure torch and cuda are compatible, e.g. using the install methods listed on https://pytorch.org/.

I got the same error,but i still can convert it to MarkDown.And this error may disappear after you update gcc version.
/lib/python3.9/site-packages/torch/include/c10/util/C++17.h:13:2: error: #error "You're trying to build PyTorch with a too old version of GCC. We need GCC 9 or later."

I am having the same issue after updating the package today.

We never encounter (yet) such an error. I'm a bit suspicious about the CUDA and Pytorch versions. I would recommend making sure torch and cuda are compatible, e.g. using the install methods listed on https://pytorch.org/.

Uninstalled and reinstalled cuda-toolkit and that seemed to have fixed it. Thanks.

I am having the same issue ,too


[style]: Developer profile card width

What's the style idea?
While scrolling through the website, I came across this styling issue on mobile screens, the card which shows developer infos are going out of the edge. The bug is related to the width of the card.
Please assign me this issue so that I can solve this styling bug. Also, I'm in GSSoC'23.
Add screenshots

Code of Conduct

[X] I agree to follow this project's Code of Conduct

yes. I also, faced this issue.
assigned to you

Completed. Thanks for your contributions 🎉 ❤


Packer 1.5+ support

Changed clean_ami_name to clean_resource_name.
clean_ami_name was deprecated in 1.5.0: https://github.com/hashicorp/packer/blob/master/CHANGELOG.md#backwards-incompatibilities-1

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement
Learn more about why HashiCorp requires a CLA and what the CLA includes
Have you signed the CLA already but the status is still pending? Recheck it.

Tests passed! Merging now, thank you!

https://github.com/hashicorp/terraform-aws-nomad/releases/tag/v0.6.1


Store and retrieve secret_key_base from disk file

follow up of https://github.com/kubic-project/velum/pull/110

Is this ready for review?


Fix CA problems with DistinguishedName order

The CertificateAuthority functionality in MoCoCrW has the
problem that distinguished names are defined as ordered in
the standard but this is not reflected in the DistinguishedName
objects in the library.
This lead to the problem that the CertificateAuthority objects
can't issue certificates with a CA certificate that uses a different
attribute order in the subject than the libraries default order.
This patch fixes the issue by directly copying the CAs subject
and the subject in the CSRs directly into the issuer and subject
fields of the newly issued certificate.
This fixes #95.
On the way, improve one the const correctness of some of the OpenSSL
methods as OpenSSL has improved since we began developing this library.

Looks good.


raid overlay is not working

Hope you got this figured out!

Thx dude
On Thu, May 2, 2019, 6:57 AM Adrienne Walker<EMAIL_ADDRESS>wrote:

Hope you got this figured out!
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/quisquous/cactbot/issues/406#issuecomment-488539112,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALLGBHSJNX4KAL46F4ZT2VTPTJKCJANCNFSM4HJZ7AHA
.


Where is the express?

I like that you are trying to make the footprint as small as possible. However, you no longer include express ;-). Don't you think you should either re-add express or change the name of the project?

Nooo :) I am including it, you have it as deps in package.json

lol I missed that so I guess just not using it?

Ahhh no I missed user, is that still getting used lol?

Sorry got lost I thought we had reverted back to straight node... bad day lol

I just update that to rc.3 :) you can pull changes. Yea, i am still using it. There is no much stuff in node / express, just one route.... :(

I expanded it a little on my fork to include some stuff from the hero example on the ng2 site. I also created another app that includes NgMaterial 2 but is based off this one. Feel free to pull in whatever you want.


Nave bar links property.

Hi,
In default.json file, in section links, how can I add some <a> tag property?

Hi,
Unfortunately it's not available to set <a> properties with default.json and it's a bit complicated to implement such functionality. The only you can is to fix a base template here: https://github.com/savuir/blo/blob/786d8988ba77652b3260f8bf9ebcb2ae87bdf441/blo/templates/base.html#L39
Glad to see blo looks good in Persian.

Thanks,
So Isn’t any way by jinja or programming in python to solve it?
Your welcome, That's because of the simplicity of blo.
But I'm new in source codes by python ;-)


make and readme to make it quicker to get going.

It's got enough for now.
Makefile, Readme, screen shots. Its a start :)
hope the conventions I use are ok.
sorry but I did it unverified at the time.

Thank you for PR! It's a great start and  will help move things forward ... but with time everything can be tuned or change, my code and texts as others' :)


Bump dependencies for framework and identity-apps

$subject
Onboards following two fixes.

https://github.com/wso2/carbon-identity-framework/pull/4327
https://github.com/wso2/identity-apps/pull/3613

Related issue

https://github.com/wso2/product-is/issues/15140

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3430822433

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3430822433
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3430822433


is possible to store kinesis video stream in S3 or somewhere? as like chunk or some other technique...please share

Hi @hameedriswan7 ,
With a configurable data retention of the KVS stream, your video data will be store during the retention window. You can use GetMedia to retrieve the video through https://github.com/aws/amazon-kinesis-video-streams-parser-library#kinesisvideoexample
If you still want to store the file in S3, you can do it with the file.

Thanks @zhiyua-git for the info. Resolving this for now.


[PERMISSIONS-SUPPORT] :sparkles: Add support for maintain and triage repositories permissions

Add support for

maintain_repositories
triage_repositories

lists, to be able to give the team permissions to repositories, according to https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization
and
https://registry.terraform.io/providers/integrations/github/latest/docs/resources/team_repository

Tests seem to fail because PRs coming from forked repos won't be able to access secrets in CI runs. Will trigger a test run manually once we have the change request applied.

Good catch. I addressed the comment.

I've checked out the branch and executed the tests. LGTM!

lgtm!

@PavelSusloparov will merge later today, edit the CHANGELOG and release.


Correct task return types

task.defer(), task.delay(), and task.spawn() return a thread.

Thanks!


Extra space in tableview

Hey,
I run your example but in the search view, there is extra space above the tableview. See first cell.

I have this same issue, any update???


Row Metadata

Refactor a few of the new row controlling properties into something similar to ColumnMetadata.

This is part of the functionality coming with the switch to https://github.com/GriddleGriddle/griddle-render/


[GHSA-f7cm-ccfp-3q4r] Django Incorrectly Validates URLs

Updates

Affected products
References
Source code location

Comments
add patch for v1.7: https://github.com/django/django/commit/bf650a2ee78c6d1f4544a875dcc777cf27fe93e9

Hey @MarkLee131, I'm going to reject this one as we don't have a 1.7.x affected product and the patch is largely the same as others which we already have.


Getting crash: contextual chat screen

Hi, Please have a look at the attached screenshot. Im getting crash when I click on Back or Done button. What could be the reason?

@manjunathgudisi can you share the crash logs?

I have fixed the crash. What is the purpose of PickerView in this screen. How does this help me.

@manjunathgudisi Can I know how did you fix the crash?

@jimmy0804
I have used the below code ALChatViewController.m file
-(void)updateContextInView
{
//added this code to fix the crash
    **if (self.pickerConvIdsArray.count == 0) {
        [self reloadView];
        return;
    }**

    [self.mTableView setUserInteractionEnabled:YES];
    [self.sendMessageTextView setHidden:NO];
}

https://github.com/AppLozic/Applozic-iOS-SDK/issues/384


[tests-only] Bump core commit 20201016

Gets the following core PRs used in testing:
https://github.com/owncloud/core/pull/37796 [Tests-only] adjust tests for EOS storage (allows the future possibility to run tests with EOS storage - which will also need a lot of CI-fu to set-up in drone...)
https://github.com/owncloud/core/pull/38013 [tests-only] test trashbin related scenarios when shares are received in subfolder

@ishank011 this is ready for review.

Rebased just now to get fresh CI - a couple of things were merged in master recently.
@ishank011 This is fine ready to merge when CI is green.


img with onload attribute may cause xss problem.

Version
1.1.0
Test Environment
Chrome
Current Behavior

onload is filtered out, but 'xss' still alert.
<img src="/image.jpg" onload="alert('xss')"/>

Expected Behavior

@LiangCY For using the latest version, this issue does not occur so I will close this issue. If you have problem, please reopen or make a new issue.


Issues importing Kismet pcapdump files

When importing a pcapdump file captured using Kismet into offline_capture, I get the following error and SniffAir crashes:

[am_13thfloor]# offline_capture /root/Documents/Kismet_files/Kismet-20171017-12-33-03-1.pcapdump
|[*] Importing /root/Documents//Kismet_files/Kismet-20171017-12-33-03-1.pcapdump
|/---| /
[+] Completed
[+] Cleaning Up Duplicates
Traceback (most recent call last):
File "SniffAir.py", line 600, in
main_menu()
File "SniffAir.py", line 133, in main_menu
choice()
File "SniffAir.py", line 113, in choice
exec_menu(choice)
File "SniffAir.py", line 151, in exec_menu
menu_actionscmd
File "SniffAir.py", line 219, in Workspace
choice()
File "SniffAir.py", line 113, in choice
exec_menu(choice)
File "SniffAir.py", line 151, in exec_menu
menu_actionscmd
File "SniffAir.py", line 390, in Help
choice()
File "SniffAir.py", line 113, in choice
exec_menu(choice)
File "SniffAir.py", line 151, in exec_menu
menu_actionscmd
File "SniffAir.py", line 297, in Offline_Capture
d.clean_up()
File "lib/Queries.py", line 51, in clean_up
qr = dp.read_sql('select * from '+ tb +'', con)
File "/usr/local/lib/python2.7/dist-packages/pandas/io/sql.py", line 400, in read_sql
chunksize=chunksize)
File "/usr/local/lib/python2.7/dist-packages/pandas/io/sql.py", line 1453, in read_query
data = self._fetchall_as_list(cursor)
File "/usr/local/lib/python2.7/dist-packages/pandas/io/sql.py", line 1462, in _fetchall_as_list
result = cur.fetchall()
sqlite3.OperationalError: Could not decode to UTF-8 column 'ESSID' with text '??$?H`l'
root@beast:/opt/SniffAir#

When I review the am_13thfloor.db file, I find the following entries:
INSERT INTO accessPoints VALUES(NULL,'??$?Hl','18:64:72:58:ff:71','Aruba Networks','52','-64','WPA2','TKIP','PSK'); INSERT INTO accessPoints VALUES('??$?Hl','18:64:72:58:ff:71','Aruba Networks','52',-64,'WPA2','TKIP','PSK');
Testing with a different pcapdump file provides the same results, SniffAir crashes:

[default]# workspace create test
[+]  Workspace test created
[test]# offline_capture /root/Documents//Kismet_files/Kismet-20171017-12-40-16-1.pcapdump
|[*] Importing /root/Documents/Kismet_files/Kismet-20171017-12-40-16-1.pcapdump

[+] Completed
[+] Cleaning Up Duplicates
Traceback (most recent call last):
File "SniffAir.py", line 687, in
main_menu()
File "SniffAir.py", line 137, in main_menu
choice()
File "SniffAir.py", line 117, in choice
exec_menu(choice)
File "SniffAir.py", line 155, in exec_menu
menu_actionscmd
File "SniffAir.py", line 225, in Workspace
choice()
File "SniffAir.py", line 117, in choice
exec_menu(choice)
File "SniffAir.py", line 155, in exec_menu
menu_actionscmd
File "SniffAir.py", line 304, in Offline_Capture
d.clean_up()
File "lib/Queries.py", line 51, in clean_up
qr = dp.read_sql('select * from '+ tb +'', con)
File "/usr/local/lib/python2.7/dist-packages/pandas/io/sql.py", line 400, in read_sql
chunksize=chunksize)
File "/usr/local/lib/python2.7/dist-packages/pandas/io/sql.py", line 1453, in read_query
data = self._fetchall_as_list(cursor)
File "/usr/local/lib/python2.7/dist-packages/pandas/io/sql.py", line 1462, in _fetchall_as_list
result = cur.fetchall()
sqlite3.OperationalError: Could not decode to UTF-8 column 'ESSID' with text 'm5'
root@beast:/opt/SniffAir#

Reviewing the following StackOverflow post and similar ones refer to  "conn.text_factory = str"
https://stackoverflow.com/questions/22751363/sqlite3-operationalerror-could-not-decode-to-utf-8-column
However, I do not see this value defined in SniffAir.py

The error is occurring after the importing process. It looks like the non UTF-8 characters are causing problems with sqlite3. Are both errors related to the same ESSID name or different ones?

The errors occur with different ESSIDs, both of which reside in different pcapdump files from Kismet.

We were able recreate the issue and will have a solution shortly.

Thanks for the hard work!

Issue resolved in latest update.


[macOS] Software updates week 7 2021

Description

Xamarin.iOS - <IP_ADDRESS>
Xamarin.Mac - <IP_ADDRESS>

Related issue: #1816
Check list

[x] Related issue / work item is attached
[ ] Tests are written (if applicable)
[ ] Documentation is updated (if applicable)
[ ] Changes are tested and related VM images are successfully generated

/azp run macos1015, macos110


[BUG] The kube-vip is unable to handle service annotation update.

Describe the bug

If we add the annotation kube-vip.io/ignore-service-security: "true" to the existing service, kube-vip is unable to handle the modification.
To Reproduce
Steps to reproduce the behavior:

Create a load balancer type service and configure a pingable IP or <IP_ADDRESS> if there is a DHCP server on the field spec.loadBalancerIP in the Harvester cluster. The kube-vip will add the IP, and add the IPTables rule to limit the port access, for example, the IP will not be pingable.
Add the annotation kube-vip.io/ignore-service-security: "true" to the service. The kube-vip should have deleted the IPTables rules but it did not. Execute the shell command iptables-save | grep kube-vip to confirm it.

Expected behavior

The kube-vip should handle the annotation kube-vip.io/ignore-service-security: "true" change.
Support bundle

Environment

Harvester ISO version:  v1.2.0-rc5
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): Baremetal with Dell PowerEdge R630

Additional context
Add any other context about the problem here.

The issue is a follow-up to support the feature upstream. (See the comment)


Faild \cite{} and \ref

I just wrote a paper via templete IEEE Trans. And the \cite{} and \ref are useless, and there is shown as a ?

Thanks for your report. This can indeed be a problem.
To handle citations and references, the expected workflow is to reuse auxiliary files produced by a normal LaTeX build.
However, TeXpresso uses Tectonic packages from 2022. In TeXlive 2024, some important packages have been updated which break compatibility.
A workaround to get working citations is to build your document with tectonic and keep intermediate files, e.g. tectonic -X compile -k myfile.tex.
This is the best I can suggest at the moment. I would like to reintroduce TeXlive compatibility, but I have no time for this at the moment and I keep delaying this work. Hopefully in Q1 or Q2 2025.


helmfile 0.82.0 brew upgrade issue

relates to https://github.com/Homebrew/homebrew-core/pull/43728
https://jenkins.brew.sh/job/Homebrew Core Pull Requests/47572/version=mojave/console
error
	fatal: unable to access 'https://git.apache.org/thrift.git/': Failed to connect to git.apache.org port 443: Operation timed out
go: error loading module requirements

Hey! Thanks for reporting.
thrift moved to github for hosting the go pkg. Maybe the old host finally stopped working?
Would you mind adding replace git.apache.org/thrift.git => github.com/apache/thrift v0.12.0 to go.mod of helmfile and see if it builds correctly?

Also see for some background:
https://github.com/mozilla/sops/issues/443#issuecomment-474209549
https://stackoverflow.com/questions/49678711/go-get-thrift-package-failed-with-repo-not-found

Would you mind adding replace git.apache.org/thrift.git => github.com/apache/thrift v0.12.0 to go.mod of helmfile and see if it builds correctly?

that is totally what I am going to do.

As homebrew works now, I filed a PR to fix the apache-thrift git location change.


mitmproxy crash on connection

Issue
I've wanted to dump session keys for connection on port 1883 (mqtt)  using transparent proxy to use them in Wire-shark to decrypt connection.  Mitmproxy crashed.
Steps to reproduce the problem:

add iptable rule: sudo iptables -t nat -A PREROUTING -i "$device" -p tcp --dport 8883 -j REDIRECT --to-port 1883
on default ubuntu 16.04 package mitmproxy v0.15
run: mitmproxy SSLKEYLOGFILE="$PWD/.mitmproxy/sslkeylogfile.txt" mitmproxy -T --host -p 1883 -e --cert ./cacertfile.pem
on latest v4.0.4
run: SSLKEYLOGFILE="$PWD/sslkeylogfile.txt" ./mitmproxy --mode transparent -p 1883 --certs ~/workspace/docker/rabbitmq-config/cacertfile.pem --ssl-insecure
create (can be mqtt) host waiting for connection, using cacertfile.pem i.e. with eclipse-mosquitto or with RabbitMQ with MQTT enabled in my case.
make connection attempt to host
crash happens with logs below

System information
version: mitmproxy 0.15
Log:
v0.15:
`Exception in thread SlaveThread (<IP_ADDRESS>:1883):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
File "/usr/lib/python2.7/dist-packages/libmproxy/controller.py", line 83, in run
File "/usr/lib/python2.7/dist-packages/netlib/tcp.py", line 818, in serve_forever
File "/usr/lib/python2.7/socket.py", line 206, in accept
error: [Errno 24] Too many open files
Event log
<IP_ADDRESS>:50490: clientconnect
<IP_ADDRESS>:50492: clientconnect
<IP_ADDRESS>:50494: clientconnect
<IP_ADDRESS>:50496: clientconnect
<IP_ADDRESS>:50498: clientconnect
<IP_ADDRESS>:50500: clientconnect
<IP_ADDRESS>:50502: clientconnect
<IP_ADDRESS>:50504: clientconnect
<IP_ADDRESS>:50506: clientconnect
<IP_ADDRESS>:50508: clientconnect
<IP_ADDRESS>:50510: clientconnect
<IP_ADDRESS>:50512: clientconnect
<IP_ADDRESS>:50514: clientconnect
<IP_ADDRESS>:50516: clientconnect`
v4.0.4
Exception in thread ServerThread (('::', 1883, 0, 0)):
Traceback (most recent call last):
File "threading.py", line 916, in _bootstrap_inner
File "mitmproxy/master.py", line 37, in run
File "mitmproxy/net/tcp.py", line 607, in serve_forever
File "socket.py", line 205, in accept
OSError: [Errno 24] Too many open files

Pretty sure you have a redirect loop here where you redirect traffic outgoing from mitmproxy back to mitmproxy again.

@pholat hi! :)
Can you please check what mhils wrote above? If it is already irrelevant, I will close the issue.

We just merged a brand new protocol implementation - if you can repeat the problem with the latest version from git-master, please re-open this issue.


Add DI extension method namespace to Program.cs in templates

Adding a using for Microsoft.Extensions.DependencyInjection so that you can use common DI extension methods like AddSingleton.

Also rename configure parameter to services like it is in ASP.NET Core.


Re-init builds and add link to nist

required by https://github.com/opentelekomcloud-docs/docsportal/issues/57

yes, as discussed in https://github.com/opentelekomcloud-docs/docsportal/issues/57 first there must be responsible people identified.


Menu

the code is same as like of vampcat just added the ships and changed there positions

Can one of the admins please verify this patch?

DON'T USE GITHUB WEB INTERFACE TO UPLOAD FILES! Also, you just basically copied vampcat's code, changing only from displaying asteroids to displaying ships moving with the same move patter. You even broke the asteroids during that.

There is a subtle, yet important line between "similar" and "same", and you crossed it by quite far.


Feature : Support baichuan2 model and efficient multi-lora adapters in a single batch #182

Feature #182
(A more efficient version for #1533 )
Because I need to use baichuan2-13B with more than one lora adapters at the same time, I tried to implement these features by myself. It can work well for my situation now. And this feature was mentioned in #182. Welcome to give me some comments, and I'll try my best to modify them.
Add Features

Support Baichuan2-13B

Support multi-lora adapters in a single batch inference
I use peft to implement multi-lora adapters. And in this situation, because we want to use more than one lora adapters, we can't merge the lora weights into the base model. So there will be some extra computation which will increase the latency. If there is only one lora adapter that you want to use, just do not use this feature.
I also tested the benchmark of my modification by using the baichuan2-13B, running on one A100 GPU . The results are like this：

benchmark_latency.py
origin vllm:
 python3 benchmark_latency.py --model $MODEL --trust-remote-code --input-len 512
result :

Avg latency: 4.033462169269721 seconds

vllm with lora:
python3 benchmark_latency.py --model $MODEL --trust-remote-code --input-len 512 --lora-paths $LORA_PATH --adapter-names adapter1
result:

Avg latency: 4.7929534030457335 seconds

benchmark_throughput.py
origin vllm:
python benchmark_throughput.py --backend vllm --dataset ./ShareGPT_V3_unfiltered_cleaned_split.json --model $MODEL --trust-remote-code
result:

Throughput: 2.56 requests/s, 1224.07 tokens/s

vllm with lora:
python3 benchmark_throughput.py  --dataset ./ShareGPT_V3_unfiltered_cleaned_split.json --model $MODEL --trust-remote-code --lora-paths $LORA_PATH --adapter-names adapter1
result:

Throughput: 1.87 requests/s, 895.59 tokens/s

benchmark_serving.py
origin vllm:
python3 benchmark_serving.py --backend vllm --dataset ./ShareGPT_V3_unfiltered_cleaned_split.json --request-rate 1 --tokenizer $MODEL --trust-remote-code --num_prompts=100
result:

Total time: 93.24 s
Throughput: 1.07 requests/s
Average latency: 4.80 s
Average latency per token: 0.02 s
Average latency per output token: 0.10 s

vllm with lora:
result:

Total time: 93.66 s
Throughput: 1.07 requests/s
Average latency: 3.64 s
Average latency per token: 0.02 s
Average latency per output token: 0.09 s

The performance decreased a little bit because of the extra computation. I'm still wokring on an more efficient version.
Now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation.
If this feature is accepted, I'm willing to support more models for lora adapter.
Changes of files

requirements.txt

Add peft for lora adapters

tests/kernels/test_blora.py

Test scripts for multi-lora computation

tests/kernels/test_normhead.py

Test scripts for NormHead layer which is used in baichuan2-13B

vllm/engine/arg_utils.py

Add two args which are used to initialize lora adapters when load the model

vllm/engine/async_llm_engine.py

Check the lora config args are valid.

vllm/engine/llm_engine.py

Check the lora config args are valid. And pass the lora config to workers.

vllm/entrypoints/llm.py

Add lora config parameters and pass them to llm_engine

vllm/model_executor/lora_utils.py

Create lora adapters and replace the target module in the base model

vllm/model_executor/model_loader.py

Support baichuan2 and add lora adapters when initialize model.

vllm/model_executor/models/init.py

Support baichuan2

vllm/model_executor/models/baichuan.py

Support baichuan2 and schedule the lora information after each iteration according to the metadata.
Impelement the method to load lora weights in parallel.

vllm/model_executor/parallel_utils/layers.py

Implement the lora module with ColumnParallelLinear and Row ParallelLinear

vllm/sampling_params.py

Add lora_id parameter to specify the lora adapter you want to use for this prompt.

vllm/worker/worker.py

pass the lora config to initialize the model

examples/baichuan_lora.py

example for use multi-lora adapters in baichuan model with offline inference.

examples/baichuan_lora_api.py

example for use multi-lora adapters in baichuan model with online serving.
Thanks and looking forward to your comments!

Thank you for the contribution. It seems that multi-lora adapters are orthogonal to Baichuan2. Can you open separate PRs?
Additionally, it would be great to reconcile the Baichuan2 PR with #1092 (which seems to be the newer iteration of #1022).

Thank you for the contribution. It seems that multi-lora adapters are orthogonal to Baichuan2. Can you open separate PRs?
Additionally, it would be great to reconcile the Baichuan2 PR with #1092 (which seems to be the newer iteration of #1022).

Thank you for your advices! No problem, I'll check and fix it.

I did not read this, but a paper has just been released that implements this efficiently in LightLLM. How does your implementation compare to theirs? Might it be worth considering looking at their implementation for inspiration on optimizations?
Paper: https://arxiv.org/abs/2311.03285

I did not read this, but a paper has just been released that implements this efficiently in LightLLM. How does your implementation compare to theirs? Might it be worth considering looking at their implementation for inspiration on optimizations?
Paper: https://arxiv.org/abs/2311.03285

Thank you for your comment and paper! I haven't read this paper; I'll try to read it and do more optimization.

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error:
File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel
raise ValueError(f"No module named {name} " +
ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight',
...
'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/moel_executor/models/baichuan.py. The implement of mlp layer between  vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/model_executor/models/baichuan.py. The implement of mlp layer between vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

OK, thank you very much for your patient response. I believe that the method of loading the checkpoint should become an important and highly valuable contribution to VLLM. Your work is very meaningful, and I want to express my gratitude once again

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/model_executor/models/baichuan.py. The implement of mlp layer between vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

Dear friend, I'd like to consult you on a matter: Have you tested your Lora loading method for parallel execution on multiple GPU cards? I am currently using two cards for parallel loading (original model + LORA), but I've noticed that only the original model is being loaded. From the actual results, it seems that LORA is not successfully loaded (the same code works fine on a single card). Could you please let me know if your method supports multi-GPU loading? If multi-GPU loading for LORA is not currently supported and I wish to modify it myself, could you guide me on where to make these modifications? I am eagerly anticipating your guidance and greatly appreciate your assistance.

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/model_executor/models/baichuan.py. The implement of mlp layer between vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

Dear friend, I'd like to consult you on a matter: Have you tested your Lora loading method for parallel execution on multiple GPU cards? I am currently using two cards for parallel loading (original model + LORA), but I've noticed that only the original model is being loaded. From the actual results, it seems that LORA is not successfully loaded (the same code works fine on a single card). Could you please let me know if your method supports multi-GPU loading? If multi-GPU loading for LORA is not currently supported and I wish to modify it myself, could you guide me on where to make these modifications? I am eagerly anticipating your guidance and greatly appreciate your assistance.

I have tested it on two A100s, it should have worked well for two cards. I'm afriad I can't give you more guidance without specific information. How about print some logs about the weights after loading model and LORA and check whether their weights are loaded succussfully?

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/model_executor/models/baichuan.py. The implement of mlp layer between vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

Dear friend, I'd like to consult you on a matter: Have you tested your Lora loading method for parallel execution on multiple GPU cards? I am currently using two cards for parallel loading (original model + LORA), but I've noticed that only the original model is being loaded. From the actual results, it seems that LORA is not successfully loaded (the same code works fine on a single card). Could you please let me know if your method supports multi-GPU loading? If multi-GPU loading for LORA is not currently supported and I wish to modify it myself, could you guide me on where to make these modifications? I am eagerly anticipating your guidance and greatly appreciate your assistance.

I have tested it on two A100s, it should have worked well for two cards. I'm afriad I can't give you more guidance without specific information. How about print some logs about the weights after loading model and LORA and check whether their weights are loaded succussfully?

Thank you very much for your response. The printed information regarding the occurrence of this issue is as follows:
...
DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65808 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>65808 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock
DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65808 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>65808 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock
DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>65088 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>65088 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>67440 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>67440 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>67440 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>67440 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>65088 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>65088 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>70752 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>70752 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>70752 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>70752 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>72864 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>72864 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>72864 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
DEBUG:filelock:Lock<PHONE_NUMBER>72864 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock
2023-11-12 19:49:33,560	WARNING utils.py:581 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set RAY_USE_MULTIPROCESSING_CPU_COUNT=1 as an env var before starting Ray. Set the env var: RAY_DISABLE_DOCKER_CPU_WARNING=1 to mute this warning.
2023-11-12 19:49:33,706	INFO worker.py:1673 -- Started a local Ray instance.
INFO 11-12 19:49:34 llm_engine.py:71] Initializing an LLM engine with config: model='/root/autodl-tmp/LLaMA-Efficient-Tuning-main/model/Baichuan2-chat', tokenizer='/root/autodl-tmp/LLaMA-Efficient-Tuning-main/model/Baichuan2-chat', tokenizer_mode=slow, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=2, quantization=None, seed=123)
WARNING 11-12 19:49:34 tokenizer.py:66] Using a slow tokenizer. This might cause a significant slowdown. Consider using a fast tokenizer instead.
INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmp4mq68iui
INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmp4mq68iui/_remote_module_non_scriptable.py
INFO 11-12 19:50:09 llm_engine.py:210] # GPU blocks: 1058, # CPU blocks: 655
...
It seems that there is an issue with Ray. By the way, I want to mention that I checked the issue list for VLLM, and a response to one of the issues indicates that Ray does not support parallel loading. So, I'm quite surprised that you were able to run it successfully on two A100 GPUs.
in #182 ,nxphi47 mentioned:

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/model_executor/models/baichuan.py. The implement of mlp layer between vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

Dear friend, I'd like to consult you on a matter: Have you tested your Lora loading method for parallel execution on multiple GPU cards? I am currently using two cards for parallel loading (original model + LORA), but I've noticed that only the original model is being loaded. From the actual results, it seems that LORA is not successfully loaded (the same code works fine on a single card). Could you please let me know if your method supports multi-GPU loading? If multi-GPU loading for LORA is not currently supported and I wish to modify it myself, could you guide me on where to make these modifications? I am eagerly anticipating your guidance and greatly appreciate your assistance.

I have tested it on two A100s, it should have worked well for two cards. I'm afriad I can't give you more guidance without specific information. How about print some logs about the weights after loading model and LORA and check whether their weights are loaded succussfully?

Thank you very much for your response. The printed information regarding the occurrence of this issue is as follows: ... DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65808 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65808 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65808 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65808 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>67440 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>67440 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>67440 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>67440 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>70752 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>70752 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>70752 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>70752 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>72864 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>72864 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>72864 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>72864 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock 2023-11-12 19:49:33,560 WARNING utils.py:581 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set RAY_USE_MULTIPROCESSING_CPU_COUNT=1 as an env var before starting Ray. Set the env var: RAY_DISABLE_DOCKER_CPU_WARNING=1 to mute this warning. 2023-11-12 19:49:33,706 INFO worker.py:1673 -- Started a local Ray instance. INFO 11-12 19:49:34 llm_engine.py:71] Initializing an LLM engine with config: model='/root/autodl-tmp/LLaMA-Efficient-Tuning-main/model/Baichuan2-chat', tokenizer='/root/autodl-tmp/LLaMA-Efficient-Tuning-main/model/Baichuan2-chat', tokenizer_mode=slow, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=2, quantization=None, seed=123) WARNING 11-12 19:49:34 tokenizer.py:66] Using a slow tokenizer. This might cause a significant slowdown. Consider using a fast tokenizer instead. INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmp4mq68iui INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmp4mq68iui/_remote_module_non_scriptable.py INFO 11-12 19:50:09 llm_engine.py:210] # GPU blocks: 1058, # CPU blocks: 655 ... It seems that there is an issue with Ray. By the way, I want to mention that I checked the issue list for VLLM, and a response to one of the issues indicates that Ray does not support parallel loading. So, I'm quite surprised that you were able to run it successfully on two A100 GPUs. in #182 ,nxphi47 mentioned:

Yes，actually I didn't use ray to do parallel loading. I guess the weights of model and lora adapter are loaded locally in two machines. They just load the different parts of the weight. I don't your gpus are on one machine or two. If you use two machines, I guess you need to make sure that the environment of two machines are exactlly the same. And the paths of the base model files and lora adapter files in two machines are the same.

I have completely downloaded the code #1560. When I run python3.10 examples/offline_inference.py, there are no issues, and the model provides perfect answers. However, when I run python3.10 examples/baichuan_lora.py, the system throws an error: File "/root/autodl-tmp/vllm-baichuan2-1560/vllm/model_executor/models/baichuan.py", line 452, in load_lora_weights_parallel raise ValueError(f"No module named {name} " + ValueError: No module named model.layers.0.mlp.gate_proj.lora_A.adapter_1.weight in the base model: odict_keys(['model.embed_tokens.weight', 'model.layers.0.self_attn.W_pack.weight', 'model.layers.0.self_attn.W_pack.lora_A.adapter_1.weight', ... 'model.layers.39.self_attn.o_proj.lora_A.adapter_1.weight', 'model.layers.39.self_attn.o_proj.lora_B.adapter_1.weight', 'model.layers.39.mlp.gate_up_proj.weight', 'model.layers.39.mlp.down_proj.weight', 'model.layers.39.mlp.down_proj.lora_A.adapter_1.weight', 'model.layers.39.mlp.down_proj.lora_B.adapter_1.weight', 'model.layers.39.input_layernorm.weight', 'model.layers.39.post_attention_layernorm.weight', 'model.norm.weight', 'lm_head.weight'],
It seems like Lora is not reading the weights, but I have correctly set the checkpoint path in the Lora-path. Could you tell me what could be the problem, and how can it be resolved?
I am eagerly looking forward to and greatly appreciate your patient response.

Well, just as I mentioned in my comment, by now I just implemented the method of loading lora weights for Baichuan model while the target modules of adapter are limited to W_pack and o_proj which we used in our situation. It seems like you want to apply adapter to mlp layers, then you can try to modify the load_lora_weights_parallel method in vllm/model_executor/models/baichuan.py. The implement of mlp layer between vllm and huggingface is different, you'd better understand how vllm implement mlp layer and how the weights are loaded, then modify the function carefully : )

Dear friend, I'd like to consult you on a matter: Have you tested your Lora loading method for parallel execution on multiple GPU cards? I am currently using two cards for parallel loading (original model + LORA), but I've noticed that only the original model is being loaded. From the actual results, it seems that LORA is not successfully loaded (the same code works fine on a single card). Could you please let me know if your method supports multi-GPU loading? If multi-GPU loading for LORA is not currently supported and I wish to modify it myself, could you guide me on where to make these modifications? I am eagerly anticipating your guidance and greatly appreciate your assistance.

I have tested it on two A100s, it should have worked well for two cards. I'm afriad I can't give you more guidance without specific information. How about print some logs about the weights after loading model and LORA and check whether their weights are loaded succussfully?

Thank you very much for your response. The printed information regarding the occurrence of this issue is as follows: ... DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65808 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65808 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65808 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65808 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/node_ip_address.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>67440 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>67440 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>67440 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>67440 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>65088 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>65088 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>70752 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>70752 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>70752 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>70752 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to acquire lock<PHONE_NUMBER>72864 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>72864 acquired on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Attempting to release lock<PHONE_NUMBER>72864 on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock DEBUG:filelock:Lock<PHONE_NUMBER>72864 released on /tmp/ray/session_2023-11-12_19-49-31_972308_6561/ports_by_node.json.lock 2023-11-12 19:49:33,560 WARNING utils.py:581 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set RAY_USE_MULTIPROCESSING_CPU_COUNT=1 as an env var before starting Ray. Set the env var: RAY_DISABLE_DOCKER_CPU_WARNING=1 to mute this warning. 2023-11-12 19:49:33,706 INFO worker.py:1673 -- Started a local Ray instance. INFO 11-12 19:49:34 llm_engine.py:71] Initializing an LLM engine with config: model='/root/autodl-tmp/LLaMA-Efficient-Tuning-main/model/Baichuan2-chat', tokenizer='/root/autodl-tmp/LLaMA-Efficient-Tuning-main/model/Baichuan2-chat', tokenizer_mode=slow, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=2, quantization=None, seed=123) WARNING 11-12 19:49:34 tokenizer.py:66] Using a slow tokenizer. This might cause a significant slowdown. Consider using a fast tokenizer instead. INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmp4mq68iui INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmp4mq68iui/_remote_module_non_scriptable.py INFO 11-12 19:50:09 llm_engine.py:210] # GPU blocks: 1058, # CPU blocks: 655 ... It seems that there is an issue with Ray. By the way, I want to mention that I checked the issue list for VLLM, and a response to one of the issues indicates that Ray does not support parallel loading. So, I'm quite surprised that you were able to run it successfully on two A100 GPUs. in #182 ,nxphi47 mentioned:

Yes，actually I didn't use ray to do parallel loading. I guess the weights of model and lora adapter are loaded locally in two machines. They just load the different parts of the weight. I don't your gpus are on one machine or two. If you use two machines, I guess you need to make sure that the environment of two machines are exactlly the same. And the paths of the base model files and lora adapter files in two machines are the same.

Thank you very much for your patient reply. I performed the loading on a single server. Although I did not trace the execution process of the code, I believe that the weights are loaded using the load_lora_weights_parallel function in \vllm-baichuan2-1560\vllm\model_executor\models\baichuan.py. You mentioned that you did not use Ray's loading method. Could you provide more detailed guidance or tell me which tool you used for loading? Thanks again, and I look forward to your response.

Yes you're right. That function does load the weight parallelly. If you read it, you'll find that it just loads a part of the weights according to the tp_rank number which is used to distinguish different gpu devices. So that's why I said I did't use ray to do parallel loading. Ray is just used for distributed inference. The weights are just loaded locally and independently in difference processes. Frankly I didn't test it with ray in single machine because I don't have two gpus in one machine, I just tested it with two machines.

I roughly understand your point. Your approach involves using two independent machines, each with only one GPU. You execute the code separately on each machine, loading the weights of LORA. Since tp_rank is 0 at this point, parallel loading doesn't actually occur, thus avoiding the issue. May I ask, during the inference, do you also perform it independently? Since the models are loaded onto different machines, parallel inference is not possible. It's a bit regrettable as this feature is highly practical. I've seen multiple requests on the forum about loading and inferring with LORA, but the author currently lacks the time and resources for further development, only continuing with research.

Yes you're right. That function does load the weight parallelly. If you read it, you'll find that it just loads a part of the weights according to the tp_rank number which is used to distinguish different gpu devices. So that's why I said I did't use ray to do parallel loading. Ray is just used for distributed inference. The weights are just loaded locally and independently in difference processes. Frankly I didn't test it with ray in single machine because I don't have two gpus in one machine, I just tested it with two machines.

I roughly understand your point. Your approach involves using two independent machines, each with only one GPU. You execute the code separately on each machine, loading the weights of LORA. Since tp_rank is 0 at this point, parallel loading doesn't actually occur, thus avoiding the issue. May I ask, during the inference, do you also perform it independently? Since the models are loaded onto different machines, parallel inference is not possible. It's a bit regrettable as this feature is highly practical. I've seen multiple requests on the forum about loading and inferring with LORA, but the author currently lacks the time and resources for further development, only continuing with research.

I guess I didn't describe it clearly. The tp_ranks are still 0 and 1. I just started a process on one machine and Ray will start another process on the other machine. And then, the two processes will initialize the model and lora adapters independently according to the tp_rank.
So the inference process will be executed parallelly on the two machines.

got your point. for the process, how can I understand about it?
---Original---
From: @.>
Date: Mon, Nov 13, 2023 21:46 PM
To: @.>;
Cc<EMAIL_ADDRESS>Subject: Re: [vllm-project/vllm] Feature : Support baichuan2 model andefficient multi-lora adapters in a single batch #182 (PR #1560)
Yes you're right. That function does load the weight parallelly. If you read it, you'll find that it just loads a part of the weights according to the tp_rank number which is used to distinguish different gpu devices. So that's why I said I did't use ray to do parallel loading. Ray is just used for distributed inference. The weights are just loaded locally and independently in difference processes. Frankly I didn't test it with ray in single machine because I don't have two gpus in one machine, I just tested it with two machines.
I roughly understand your point. Your approach involves using two independent machines, each with only one GPU. You execute the code separately on each machine, loading the weights of LORA. Since tp_rank is 0 at this point, parallel loading doesn't actually occur, thus avoiding the issue. May I ask, during the inference, do you also perform it independently? Since the models are loaded onto different machines, parallel inference is not possible. It's a bit regrettable as this feature is highly practical. I've seen multiple requests on the forum about loading and inferring with LORA, but the author currently lacks the time and resources for further development, only continuing with research.
I guess I didn't describe it clearly. The tp_ranks are still 0 and 1. I just started a process on one machine and Ray will start another process on the other machine. And then, the two processes will initialize the model and lora adapters independently according to the tp_rank.
So the inference process will be executed parallelly on the two machines.
—
Reply to this email directly, view it on GitHub, or unsubscribe.


Firefox not supported for webcontainer

Describe the bug
When I open the link in Firefox browser, I get the following error.
WebContainers currently work in Chrome and Chromium-based browsers. We’re hoping to add support for more browsers as they implement the necessary Web Platform features.
Link to the blitz that caused the error
https://stackblitz.com/github/webpack/webpack.js.org/tree/master/examples/module-federation?terminal=start&terminal=
Steps to reproduce

Go to the link.
See the right hand side of screen.

Expected behavior
I should see a terminal.
Parity with Local

[ ] I have run the project in my local machine and I could not reproduce the issue.

Screenshots
No response
Platform
Browser name  = Firefox
Full version  = 95.0
Major version = 95
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0
performance.memory = undefined
Hash = 4bc445b0bf2f24de41719bcf76515e3a7b66a071

Additional context
No response

Closed in favor of #437


Rocket setup

This pull request adds the basic setup of the Rocket page.
Change made :

Add store.js and rocketSlice.js
Add style for header

Hello @akbarsazish
Your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉

Highlights

Project met all technical requirements ✔
Great implementation of all the requirements in this milestone. ✔
You have done an amazing job ✔
Your project is complete! There is nothing else to say other than... it's time to merge it. ✔

Congratulations! 🎉
CHEERS AND HAPPY CODING!👏👏👏


Project activity

With the last release in December 2019 and 10 pending PR's, is this project supported by the repo owner?

I need to find time to address the outstanding PR's and plan the next set of work. The past 6 months have been very busy and I have not been able to spend time on this project but I am hoping to sort this soon.

With the latest stable version of Nomad being 0.11 (and 0.12 upcoming), without https://github.com/jrasell/levant/pull/327 merged this amazing software is not usable. It is very sad we or other people who stay with the latest Nomad version could not use Levant.

The project has now been moved into the HashiCorp repository which allows me to work on this during my normal working day, dependant on priorities and workload.


Preferably use quarto actions to publish when they work

The rendering is currently done with a call to 'quarto render' via bash, the quarto publish action not working with conda. An issue has been filed about this. It seems that the compilation is a bit faster by this way, so we would have to gain.

I don’t think not using their official actions is really an issue : ATM their actions are just barebone (they’re just calls to "render/publish") and lack a lot of options.
I don’t understand what do you mean by "compilation is a bit faster", actions can’t speedup compilations, they’re just javascript scripts executed inside a VM.
We could preload the conda env as a pre-render project script, but it will need another action script for that: not worth the hassle, IMHO.

I don’t think not using their official actions is really an issue : ATM their actions are just barebone (they’re just calls to "render/publish") and lack a lot of options.

Ok, but in the end, we could expect them having more comprehensive actions, and it is also a way to support them. Anyway, ok to use our own actions.

I don’t understand what do you mean by "compilation is a bit faster", actions can’t speedup compilations, they’re just javascript scripts executed inside a VM.

It was probably due to the use of pip in place of conda.

We could preload the conda env as a pre-render project script, but it will need another action script for that: not worth the hassle, IMHO.

Ok, agree with that. But an example with the possibility of caching the conda-env would be nice. What do you think?

I don’t think not using their official actions is really an issue : ATM their actions are just barebone (they’re just calls to "render/publish") and lack a lot of options.

Ok, but in the end, we could expect them having more comprehensive actions, and it is also a way to support them. Anyway, ok to use our own actions.

I don’t understand what do you mean by "compilation is a bit faster", actions can’t speedup compilations, they’re just javascript scripts executed inside a VM.

It was probably due to the use of pip in place of conda.

Anything is faster than conda 😬
For the CI, I use Micromamba as a drop-in replacement for conda (and compatible with it). It’s several orders of magnitudes faster.

We could preload the conda env as a pre-render project script, but it will need another action script for that: not worth the hassle, IMHO.

Ok, agree with that. But an example with the possibility of caching the conda-env would be nice. What do you think?

It is already in the CI:

Ok thanks !


[SQL] add support for backup storage redundancy param to create/restore/update db commands

Description

Enabling customer to set Backup Storage Redundancy (StorageAccountType in REST API) via --backup-storage-redundancy parameter by adding parameter to the following cmds:
az sql db create
az sql db update
az sql db restore
az sql db copy
az sql db replica create
az sql db ltr-backup restore
Testing Guide

Test all of the above commands with additional --backup-storage-redundancy parameters
History Notes

[SQL] BREAKING CHANGE: az sql db create: Add --backup-storage-redundancy parameter; add warning for unspecified bsr/bsr == Geo.
Same for remaining cmds.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

SQL

/azp run


[RESTEASY-1692] ResourceInfoInjectionTest freeze with Undertow 1.4.18…

ResourceInfoInjectionTest freeze with newest Undertow.

ResourceInfoInjectionTest uses ResourceInfoInjectionResource end-point.
ResourceInfoInjectionResource end-point uses ServletInputStream and custom ReadListener class. ReadListener calls read() without calling isReady().
But according to this Stuart's comment, calling read() without calling isReady is illegal: https://issues.jboss.org/browse/JBEAP-12223?focusedCommentId=13438437&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13438437]

RESTEasy jira: https://issues.jboss.org/browse/RESTEASY-1692
3.0 PR: https://github.com/resteasy/Resteasy/pull/1218

This PR cover now both legacy-integration-tests and integration-tests variants of the test


Release tag 4.0.0 is missing

The release tag is missing for now. Is this because 4.0.0 is not really released?

fixed


Testing should also be on expected failure

When we test we shouldn't only do sunshine testing but also test on expected errors.

Anything specific we are not testing here? Specific file?

We are currently covered for this.


Should "unit test mode" ignore false positive intents when negative intent explicitly defined?

Currently, if the user explicitly defines the expected intent to be None, the configured true negative intent is set to None, and the result is not a negative intent, when running in "unit test mode", a false positive result will not be generated. Given that the user explicitly declared the expected intent to be the true negative intent, I expect we want false positive results here.
E.g., test settings:
trueNegativeIntent: None

Expected utterance:
{
  "text": "what time is it",
  "intent": "None"
}

Actual utterance:
{
  "text": "what time is it",
  "intent": "Some"
}

I believe we want to generate a false positive intent here since the user explicitly defined the intent. In unit test mode, we should explicitly check for a null intent in the expected utterance, which would imply that the user does not care about the intent (they are only unit testing for, e.g., entities).

We should be careful here. It will be difficult in the current code to differentiate between:
{
  "text": "hello Jane",
  "intent": null
}

And:
{
  "text": "hello"
}

Depends on #321


Task - Finalize the Find Skills page

Description
The newer version of the Explore Skills modal combines the previous Explore Skills ui with the search bar, and includes them directly in the My Skills page.
Images
Updated My Skills page wireframe

In newer designs the content of the Explore Skills page has been combined into a single modal, and added to the My Skills page. I'm repurposing this issue from Finalize the Find Skills Page to Combine Explore Skills modal and Search Bar.

In newer designs the content of the Explore Skills page has been combined into a single modal, and added to the My Skills page. I'm repurposing this issue from Finalize the Find Skills Page to Combine Explore Skills modal and Search Bar.


Missing Lib

Hi,
i am running into following error:
RosDataProviderInterface.h:22:10: fatal error: pose_graph_tools/PoseGraph.h: is missing
what library do i need to solve it?

Hi
I have the same issue. How do you solve it?


Spyder crashed during last session

Description of your problem
What steps will reproduce the problem?

when I open it ，it tells me the crashed news,
I use cmd.exe and cin  'spyder --show-console'
It tells me ' Spyder is already running. If you want to open a new
instance, please pass to it the --new-instance option'

What is the expected output? What do you see instead?
Please provide any additional information below
Versions and main components

Spyder Version:<IP_ADDRESS>
Python Version:2.7
Qt Version:
PyQt Version:
Operating system:win10

Dependencies
Please go to the menu entry Help > Optional Dependencies (or
Help > Dependencies), press the button Copy to clipboard
and paste the contents below:

This problem was fixed in Spyder 3.1, so please update to our latest version (3.1.3 at the moment).

I find a txt named SPYDER CRASH ，it tell me news:
Traceback (most recent call last):
File "C:\Users\26983\Anaconda2\lib\site-packages\spyder\app\mainwindow.py", line 2989, in main
mainwindow = run_spyder(app, options, args)
File "C:\Users\26983\Anaconda2\lib\site-packages\spyder\app\mainwindow.py", line 2891, in run_spyder
main = MainWindow(options)
File "C:\Users\26983\Anaconda2\lib\site-packages\spyder\app\mainwindow.py", line 494, in init
socket.IPPROTO_TCP)
File "C:\Users\26983\Anaconda2\lib\socket.py", line 191, in init
_sock = _realsocket(family, type, proto)
error: [Errno 10106]