[Installation]: Failed building editable for vllm

Your current environment
The output of `python collect_env.py`

Collecting environment information...

/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-11.3/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      52 bits physical, 57 bits virtual
Byte Order:                         Little Endian
CPU(s):                             128
On-line CPU(s) list:                0-127
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
CPU family:                         6
Model:                              106
Thread(s) per core:                 2
Core(s) per socket:                 32
Socket(s):                          2
Stepping:                           6
CPU max MHz:                        3400.0000
CPU min MHz:                        800.0000
BogoMIPS:                           5200.00
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr
L1d cache:                          3 MiB (64 instances)
L1i cache:                          2 MiB (64 instances)
L2 cache:                           80 MiB (64 instances)
L3 cache:                           96 MiB (2 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-31,64-95
NUMA node1 CPU(s):                  32-63,96-127
Vulnerability Gather data sampling: Vulnerable: No microcode
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] No relevant packages
[conda] No relevant packagesROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: N/A
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      PXB     PXB     PXB     SYS     SYS     0-31,64-95      0               N/A
GPU1    PXB      X      PXB     PXB     SYS     SYS     0-31,64-95      0               N/A
GPU2    PXB     PXB      X      PIX     SYS     SYS     0-31,64-95      0               N/A
GPU3    PXB     PXB     PIX      X      SYS     SYS     0-31,64-95      0               N/A
GPU4    SYS     SYS     SYS     SYS      X      PIX     32-63,96-127    1               N/A
GPU5    SYS     SYS     SYS     SYS     PIX      X      32-63,96-127    1               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

How you are installing vllm
git clone https://github.com/vllm-project/vllm.git
cd vllm
pip install -e .

'''
Building wheels for collected packages: vllm
Building editable for vllm (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building editable for vllm (pyproject.toml) did not run successfully.
│ exit code: 1
╰─
CMake Error at /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:814 (message):
Compiling the CUDA compiler identification source file
"CMakeCUDACompilerId.cu" failed.
    Compiler: /usr/bin/nvcc

    Build flags:

    Id flags: --keep;--keep-dir;tmp -v

    The output was:

    255

    #$ _NVVM_BRANCH_=nvvm

    #$ _SPACE_=

    #$ _CUDART_=cudart

    #$ _HERE_=/usr/lib/nvidia-cuda-toolkit/bin

    #$ _THERE_=/usr/lib/nvidia-cuda-toolkit/bin

    #$ _TARGET_SIZE_=

    #$ _TARGET_DIR_=

    #$ _TARGET_SIZE_=64

    #$ NVVMIR_LIBRARY_DIR=/usr/lib/nvidia-cuda-toolkit/libdevice

    #$
    PATH=/usr/lib/nvidia-cuda-toolkit/bin:/tmp/pip-build-env-l6d1bzk0/overlay/bin:/tmp/pip-build-env-l6d1bzk0/normal/bin:/usr/local/cuda/bin:/data2/fanbingbing/.conda/envs/vllm-embedding/bin:/usr/local/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

    #$ LIBRARIES= -L/usr/lib/x86_64-linux-gnu/stubs -L/usr/lib/x86_64-linux-gnu

    #$ rm tmp/a_dlink.reg.c

    #$ gcc -D__CUDA_ARCH__=520 -D__CUDA_ARCH_LIST__=520 -E -x c++

    "tmp/CMakeCUDACompilerId.cudafe1.gpu" "tmp/CMakeCUDACompilerId.cpp1.ii" -o
    "tmp/CMakeCUDACompilerId.ptx"

    #$ ptxas -arch=sm_52 -m64 "tmp/CMakeCUDACompilerId.ptx" -o
    "tmp/CMakeCUDACompilerId.sm_52.cubin"

    ptxas tmp/CMakeCUDACompilerId.ptx, line 9; fatal : Unsupported .version
    7.5; current version is '6.3'

    ptxas fatal : Ptx assembly aborted due to errors

    # --error 0xff --

  Call Stack (most recent call first):
    /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
    /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
    /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCUDACompiler.cmake:131 (CMAKE_DETERMINE_COMPILER_ID)
    /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:47 (enable_language)
    /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:87 (include)
    /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
    CMakeLists.txt:67 (find_package)

  -- Configuring incomplete, errors occurred!
  Traceback (most recent call last):
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 155, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 357, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 280, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 307, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-l6d1bzk0/over
      _build_ext.run(self)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "<string>", line 192, in build_extensions
    File "<string>", line 175, in configure
    File "/data2/fanbingbing/.conda/envs/vllm-embedding/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/data2/fanbingbing/Segregation/LLaMA-embedding/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/tmp6odosk48.build-lib/vllm', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=/tmp/tmpy8shcns0.build-temp', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/data2/fanbingbing/.conda/envs/vllm-embedding/bin/python', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=128']' returned non-zero exit status 1.
  /tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `vllm` in editable mode.

          The following steps are recommended to help debug this problem:

          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.

          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:

              https://github.com/pypa/setuptools/issues

          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************

  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/data2/fanbingbing/.conda/envs/vllm-embedding/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/data2/fanbingbing/.conda/envs/vllm-embedding/lib/python3.11/site-packages/pip/_vendor/py
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 401, in <module>
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 104, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 184, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 155, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 357, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 280, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 307, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 91, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-l6d1bzk0/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "<string>", line 192, in build_extensions
    File "<string>", line 175, in configure
    File "/data2/fanbingbing/.conda/envs/vllm-embedding/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/data2/fanbingbing/Segregation/LLaMA-embedding/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/tmp6odosk48.build-lib/vllm', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=/tmp/tmpy8shcns0.build-temp', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/data2/fanbingbing/.conda/envs/vllm-embedding/bin/python', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=128']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building editable for vllm
Failed to build vllm
ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects
'''

Hi @Fanb1ing we don't support NVIDIA GPUs with sm_52 since it is quite old at this point. I think the oldest that the community has tried is Pascal aka sm_60 https://github.com/vllm-project/vllm/issues/963

Thanks very much for your help.
However, I am still confused that the GPU I used are NVIDIA GeForce RTX 4090 and NVIDIA GeForce RTX 3090, their corresponding compute capability are 8.9 and 8.6. Where did the "sm_52" problem arise?

By the way, I have tried "pip install . " to install in a new conda environment with python=3.9. It didn't work. Here is the error:
`Building wheels for collected packages: vllm
Building wheel for vllm (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for vllm (pyproject.toml) did not run successfully.
│ exit code: 1
?─> [299 lines of output]
running bdist_wheel
running build
running build_py
copying vllm/sampling_params.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/config.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/logger.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/init.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/utils.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/_custom_ops.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/outputs.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/envs.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/block.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/sequence.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/pooling_params.py -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/attention/selector.py -> build/lib.linux-x86_64-cpython-39/vllm/attention
copying vllm/attention/layer.py -> build/lib.linux-x86_64-cpython-39/vllm/attention
copying vllm/attention/init.py -> build/lib.linux-x86_64-cpython-39/vllm/attention
copying vllm/core/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/init.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/evictor_v1.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/scheduler.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/embedding_model_block_manager.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/block_manager_v1.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/policy.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/evictor_v2.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/core/block_manager_v2.py -> build/lib.linux-x86_64-cpython-39/vllm/core
copying vllm/worker/neuron_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/worker_base.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/cpu_model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/init.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/embedding_model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/cpu_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/neuron_model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/worker.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/worker/cache_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
copying vllm/model_executor/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
copying vllm/model_executor/pooling_metadata.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
copying vllm/model_executor/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
copying vllm/model_executor/sampling_metadata.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
copying vllm/usage/usage_lib.py -> build/lib.linux-x86_64-cpython-39/vllm/usage
copying vllm/usage/init.py -> build/lib.linux-x86_64-cpython-39/vllm/usage
copying vllm/spec_decode/util.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/ngram_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/metrics.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/init.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/top1_proposer.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/spec_decode_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/batch_expansion.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/spec_decode/multi_step_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
copying vllm/lora/punica.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/request.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/init.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/layers.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/worker_manager.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/fully_sharded_layers.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/lora.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/lora/models.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
copying vllm/distributed/communication_op.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
copying vllm/distributed/init.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
copying vllm/distributed/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
copying vllm/distributed/parallel_state.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
copying vllm/executor/multiproc_gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/distributed_gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/ray_gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/neuron_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/init.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/cpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/executor_base.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/ray_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/executor/multiproc_worker_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
copying vllm/logging/init.py -> build/lib.linux-x86_64-cpython-39/vllm/logging
copying vllm/logging/formatter.py -> build/lib.linux-x86_64-cpython-39/vllm/logging
copying vllm/entrypoints/api_server.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints
copying vllm/entrypoints/init.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints
copying vllm/entrypoints/llm.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints
copying vllm/engine/metrics.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
copying vllm/engine/init.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
copying vllm/engine/async_llm_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
copying vllm/engine/arg_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
copying vllm/engine/llm_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
copying vllm/transformers_utils/config.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
copying vllm/transformers_utils/init.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
copying vllm/transformers_utils/detokenizer.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
copying vllm/transformers_utils/tokenizer.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
copying vllm/attention/ops/prefix_prefill.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
copying vllm/attention/ops/paged_attn.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
copying vllm/attention/ops/init.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
copying vllm/attention/ops/triton_flash_attention.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
copying vllm/attention/backends/xformers.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/attention/backends/torch_sdpa.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/attention/backends/rocm_flash_attn.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/attention/backends/flash_attn.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/attention/backends/init.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/attention/backends/flashinfer.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/attention/backends/abstract.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
copying vllm/core/block/cpu_gpu_block_allocator.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/core/block/prefix_caching_block.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/core/block/block_table.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/core/block/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/core/block/init.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/core/block/naive_block.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/core/block/common.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
copying vllm/model_executor/guided_decoding/outlines_logits_processors.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
copying vllm/model_executor/guided_decoding/lm_format_enforcer_decoding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
copying vllm/model_executor/guided_decoding/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
copying vllm/model_executor/guided_decoding/outlines_decoding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
copying vllm/model_executor/layers/vocab_parallel_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/logits_processor.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/rotary_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/activation.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/sampler.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/rejection_sampler.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/layernorm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/pooler.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/layers/linear.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
copying vllm/model_executor/model_loader/weight_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
copying vllm/model_executor/model_loader/loader.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
copying vllm/model_executor/model_loader/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
copying vllm/model_executor/model_loader/neuron.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
copying vllm/model_executor/model_loader/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
copying vllm/model_executor/model_loader/tensorizer.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
copying vllm/model_executor/models/qwen2_moe.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/qwen2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/stablelm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/phi.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/llava.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/internlm2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/olmo.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/bloom.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/orion.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/qwen.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/mpt.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/mixtral.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/jais.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/deepseek.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/gemma.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/xverse.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/gpt_bigcode.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/dbrx.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/gpt2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/gpt_neox.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/mixtral_quant.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/decilm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/arctic.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/baichuan.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/falcon.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/llama_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/vlm_base.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/gpt_j.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/commandr.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/minicpm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/starcoder2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/opt.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/llama.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/models/chatglm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
copying vllm/model_executor/layers/fused_moe/fused_moe.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe
copying vllm/model_executor/layers/fused_moe/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe
copying vllm/model_executor/layers/ops/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
copying vllm/model_executor/layers/ops/sample.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
copying vllm/model_executor/layers/ops/rand.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
copying vllm/model_executor/layers/quantization/schema.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/gptq_marlin.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/fp8.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/squeezellm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/gptq_marlin_24.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/base_config.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/aqlm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/awq.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/marlin.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/gptq.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/deepspeedfp.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
copying vllm/model_executor/layers/quantization/utils/marlin_perms.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
copying vllm/model_executor/layers/quantization/utils/init.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
copying vllm/model_executor/layers/quantization/utils/format_24.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
copying vllm/model_executor/layers/quantization/utils/marlin_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
copying vllm/model_executor/layers/quantization/utils/marlin_24_perms.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
copying vllm/model_executor/layers/quantization/utils/quant_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
copying vllm/distributed/device_communicators/init.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
copying vllm/distributed/device_communicators/custom_all_reduce.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
copying vllm/distributed/device_communicators/pynccl.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
copying vllm/distributed/device_communicators/pynccl_wrapper.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
copying vllm/entrypoints/openai/serving_chat.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/serving_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/api_server.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/init.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/run_batch.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/serving_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/serving_completion.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/protocol.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/entrypoints/openai/cli_args.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
copying vllm/engine/output_processor/util.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
copying vllm/engine/output_processor/multi_step.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
copying vllm/engine/output_processor/single_step.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
copying vllm/engine/output_processor/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
copying vllm/engine/output_processor/init.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
copying vllm/engine/output_processor/stop_checker.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
copying vllm/transformers_utils/configs/mpt.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/configs/jais.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/configs/dbrx.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/configs/init.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/configs/arctic.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/configs/falcon.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/configs/chatglm.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
copying vllm/transformers_utils/tokenizer_group/init.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
copying vllm/transformers_utils/tokenizer_group/tokenizer_group.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
copying vllm/transformers_utils/tokenizer_group/ray_tokenizer_group.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
copying vllm/transformers_utils/tokenizer_group/base_tokenizer_group.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
copying vllm/transformers_utils/tokenizers/init.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizers
copying vllm/transformers_utils/tokenizers/baichuan.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizers
copying vllm/py.typed -> build/lib.linux-x86_64-cpython-39/vllm
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=2048,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=4096,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=16,N=1344,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_A100-SXM4-40GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=16,N=2688,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=NVIDIA_H100_80GB_HBM3,dtype=float8.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=16,N=2688,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=NVIDIA_A100-SXM4-40GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=16,N=1344,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=16,N=1344,device_name=NVIDIA_A100-SXM4-40GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_H100_80GB_HBM3,dtype=float8.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=2048,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=4096,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
copying vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
running build_ext
CMake Error at CMakeLists.txt:3 (project):
Running
     '/tmp/pip-build-env-_st73erw/overlay/bin/ninja' '--version'

    failed with:

     no such file or directory

  -- Configuring incomplete, errors occurred!
  Traceback (most recent call last):
    File "/data2/fanbingbing/.local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/data2/fanbingbing/.local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/data2/fanbingbing/.local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 410, in build_wheel
      return self._build_with_temp_dir(
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 401, in <module>
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 104, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
      return run_commands(dist)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 91, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-rgurjuq7/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
      self.build_extensions()
    File "<string>", line 192, in build_extensions
    File "<string>", line 175, in configure
    File "/data2/fanbingbing/.conda/envs/vllm-embedding/lib/python3.9/subprocess.py", line 373, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/data2/fanbingbing/Segregation/LLaMA-embedding/vllm', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data2/fanbingbing/Segregation/LLaMA-embedding/vllm/build/lib.linux-x86_64-cpython-39/vllm', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=build/temp.linux-x86_64-cpython-39', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/data2/fanbingbing/.conda/envs/vllm-embedding/bin/python', '-DNVCC_THREADS=1', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=128']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for vllm
Failed to build vllm
ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects`

I assumed an old GPU because of this section of your original log
    #$ gcc -D__CUDA_ARCH__=520 -D__CUDA_ARCH_LIST__=520 -E -x c++

    "tmp/CMakeCUDACompilerId.cudafe1.gpu" "tmp/CMakeCUDACompilerId.cpp1.ii" -o
    "tmp/CMakeCUDACompilerId.ptx"

    #$ ptxas -arch=sm_52 -m64 "tmp/CMakeCUDACompilerId.ptx" -o
    "tmp/CMakeCUDACompilerId.sm_52.cubin"

    ptxas tmp/CMakeCUDACompilerId.ptx, line 9; fatal : Unsupported .version
    7.5; current version is '6.3'

    ptxas fatal : Ptx assembly aborted due to errors

Seeing your system setup with CUDA 12.4 and RTX 3090/4090 now seems like something that should be easily supported. I'm not sure what is going wrong here, but given it seems the build process is finding CUDA 11.x binaries, I would carefully check your paths to see if you have multiple versions of CUDA installed and check the version of nvcc + other CUDA utils

 '/tmp/pip-build-env-_st73erw/overlay/bin/ninja' '--version'

failed with:

 no such file or directory

The cmake command somehow remembers the old build directory /tmp/pip-build-env-_st73erw , while the current build directory is /tmp/pip-build-env-rgurjuq7 .
You have a dirty build, and need to clean up previous build before you build again.

Really Really Thanks to you all !!!
Today I tried again in a new environment and re-download git code to avoid dirty build problem.
I actually found that there are many different version of CUDA on the server.

I set the CUDA-12.1 into PATH，by
export CUDA_HOME=/usr/local/cuda-12.1
export PATH="${CUDA_HOME}/bin:$PATH"

and Then I ran  pip install .
I got a new error and it shows "nvcc fatal   : Unsupported gpu architecture 'compute_89'".  I am so sorry that I can't figure out what wrong with the version.  T^T
 Building wheel for vllm (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for vllm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [390 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/sampling_params.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/config.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/logger.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/utils.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/_custom_ops.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/outputs.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/envs.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/block.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/sequence.py -> build/lib.linux-x86_64-cpython-39/vllm
      copying vllm/pooling_params.py -> build/lib.linux-x86_64-cpython-39/vllm
      creating build/lib.linux-x86_64-cpython-39/vllm/attention
      copying vllm/attention/selector.py -> build/lib.linux-x86_64-cpython-39/vllm/attention
      copying vllm/attention/layer.py -> build/lib.linux-x86_64-cpython-39/vllm/attention
      copying vllm/attention/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/attention
      creating build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/evictor_v1.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/scheduler.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/embedding_model_block_manager.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/block_manager_v1.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/policy.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/evictor_v2.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      copying vllm/core/block_manager_v2.py -> build/lib.linux-x86_64-cpython-39/vllm/core
      creating build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/neuron_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/worker_base.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/cpu_model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/embedding_model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/cpu_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/neuron_model_runner.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/worker.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      copying vllm/worker/cache_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/worker
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor
      copying vllm/model_executor/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
      copying vllm/model_executor/pooling_metadata.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
      copying vllm/model_executor/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
      copying vllm/model_executor/sampling_metadata.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor
      creating build/lib.linux-x86_64-cpython-39/vllm/usage
      copying vllm/usage/usage_lib.py -> build/lib.linux-x86_64-cpython-39/vllm/usage
      copying vllm/usage/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/usage
      creating build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/util.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/ngram_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/metrics.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/top1_proposer.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/spec_decode_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/batch_expansion.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      copying vllm/spec_decode/multi_step_worker.py -> build/lib.linux-x86_64-cpython-39/vllm/spec_decode
      creating build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/punica.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/request.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/layers.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/worker_manager.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/fully_sharded_layers.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/lora.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      copying vllm/lora/models.py -> build/lib.linux-x86_64-cpython-39/vllm/lora
      creating build/lib.linux-x86_64-cpython-39/vllm/distributed
      copying vllm/distributed/communication_op.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
      copying vllm/distributed/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
      copying vllm/distributed/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
      copying vllm/distributed/parallel_state.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed
      creating build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/multiproc_gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/distributed_gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/ray_gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/neuron_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/gpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/cpu_executor.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/executor_base.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/ray_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      copying vllm/executor/multiproc_worker_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/executor
      creating build/lib.linux-x86_64-cpython-39/vllm/logging
      copying vllm/logging/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/logging
      copying vllm/logging/formatter.py -> build/lib.linux-x86_64-cpython-39/vllm/logging
      creating build/lib.linux-x86_64-cpython-39/vllm/entrypoints
      copying vllm/entrypoints/api_server.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints
      copying vllm/entrypoints/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints
      copying vllm/entrypoints/llm.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints
      creating build/lib.linux-x86_64-cpython-39/vllm/engine
      copying vllm/engine/metrics.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
      copying vllm/engine/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
      copying vllm/engine/async_llm_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
      copying vllm/engine/arg_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
      copying vllm/engine/llm_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/engine
      creating build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
      copying vllm/transformers_utils/config.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
      copying vllm/transformers_utils/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
      copying vllm/transformers_utils/detokenizer.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
      copying vllm/transformers_utils/tokenizer.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils
      creating build/lib.linux-x86_64-cpython-39/vllm/attention/ops
      copying vllm/attention/ops/prefix_prefill.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
      copying vllm/attention/ops/paged_attn.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
      copying vllm/attention/ops/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
      copying vllm/attention/ops/triton_flash_attention.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/ops
      creating build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/xformers.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/torch_sdpa.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/rocm_flash_attn.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/flash_attn.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/flashinfer.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      copying vllm/attention/backends/abstract.py -> build/lib.linux-x86_64-cpython-39/vllm/attention/backends
      creating build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/cpu_gpu_block_allocator.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/prefix_caching_block.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/block_table.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/naive_block.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      copying vllm/core/block/common.py -> build/lib.linux-x86_64-cpython-39/vllm/core/block
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
      copying vllm/model_executor/guided_decoding/outlines_logits_processors.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
      copying vllm/model_executor/guided_decoding/lm_format_enforcer_decoding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
      copying vllm/model_executor/guided_decoding/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
      copying vllm/model_executor/guided_decoding/outlines_decoding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/guided_decoding
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/vocab_parallel_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/logits_processor.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/rotary_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/activation.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/sampler.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/rejection_sampler.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/layernorm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/pooler.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      copying vllm/model_executor/layers/linear.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      copying vllm/model_executor/model_loader/weight_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      copying vllm/model_executor/model_loader/loader.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      copying vllm/model_executor/model_loader/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      copying vllm/model_executor/model_loader/neuron.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      copying vllm/model_executor/model_loader/utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      copying vllm/model_executor/model_loader/tensorizer.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/model_loader
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/qwen2_moe.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/qwen2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/stablelm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/phi.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/llava.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/internlm2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/olmo.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/bloom.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/orion.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/qwen.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/mpt.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/mixtral.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/jais.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/deepseek.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/gemma.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/xverse.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/gpt_bigcode.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/dbrx.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/gpt2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/gpt_neox.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/mixtral_quant.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/decilm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/arctic.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/baichuan.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/falcon.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/llama_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/vlm_base.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/gpt_j.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/commandr.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/minicpm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/starcoder2.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/opt.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/llama.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      copying vllm/model_executor/models/chatglm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/models
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe
      copying vllm/model_executor/layers/fused_moe/fused_moe.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe
      copying vllm/model_executor/layers/fused_moe/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
      copying vllm/model_executor/layers/ops/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
      copying vllm/model_executor/layers/ops/sample.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
      copying vllm/model_executor/layers/ops/rand.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/ops
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/schema.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/gptq_marlin.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/fp8.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/squeezellm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/gptq_marlin_24.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/base_config.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/aqlm.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/awq.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/marlin.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/gptq.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      copying vllm/model_executor/layers/quantization/deepspeedfp.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      copying vllm/model_executor/layers/quantization/utils/marlin_perms.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      copying vllm/model_executor/layers/quantization/utils/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      copying vllm/model_executor/layers/quantization/utils/format_24.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      copying vllm/model_executor/layers/quantization/utils/marlin_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      copying vllm/model_executor/layers/quantization/utils/marlin_24_perms.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      copying vllm/model_executor/layers/quantization/utils/quant_utils.py -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/quantization/utils
      creating build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
      copying vllm/distributed/device_communicators/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
      copying vllm/distributed/device_communicators/custom_all_reduce.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
      copying vllm/distributed/device_communicators/pynccl.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
      copying vllm/distributed/device_communicators/pynccl_wrapper.py -> build/lib.linux-x86_64-cpython-39/vllm/distributed/device_communicators
      creating build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/serving_chat.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/serving_engine.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/api_server.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/run_batch.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/serving_embedding.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/serving_completion.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/protocol.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      copying vllm/entrypoints/openai/cli_args.py -> build/lib.linux-x86_64-cpython-39/vllm/entrypoints/openai
      creating build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      copying vllm/engine/output_processor/util.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      copying vllm/engine/output_processor/multi_step.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      copying vllm/engine/output_processor/single_step.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      copying vllm/engine/output_processor/interfaces.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      copying vllm/engine/output_processor/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      copying vllm/engine/output_processor/stop_checker.py -> build/lib.linux-x86_64-cpython-39/vllm/engine/output_processor
      creating build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/mpt.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/jais.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/dbrx.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/arctic.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/falcon.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      copying vllm/transformers_utils/configs/chatglm.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/configs
      creating build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
      copying vllm/transformers_utils/tokenizer_group/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
      copying vllm/transformers_utils/tokenizer_group/tokenizer_group.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
      copying vllm/transformers_utils/tokenizer_group/ray_tokenizer_group.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
      copying vllm/transformers_utils/tokenizer_group/base_tokenizer_group.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizer_group
      creating build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizers
      copying vllm/transformers_utils/tokenizers/__init__.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizers
      copying vllm/transformers_utils/tokenizers/baichuan.py -> build/lib.linux-x86_64-cpython-39/vllm/transformers_utils/tokenizers
      copying vllm/py.typed -> build/lib.linux-x86_64-cpython-39/vllm
      creating build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=2048,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=4096,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=16,N=1344,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_A100-SXM4-40GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=16,N=2688,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=NVIDIA_H100_80GB_HBM3,dtype=float8.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=16,N=2688,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=NVIDIA_A100-SXM4-40GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=16,N=1344,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=16,N=1344,device_name=NVIDIA_A100-SXM4-40GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=3584,device_name=NVIDIA_H100_80GB_HBM3,dtype=float8.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=2048,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=4096,device_name=NVIDIA_A100-SXM4-80GB.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      copying vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=NVIDIA_H100_80GB_HBM3.json -> build/lib.linux-x86_64-cpython-39/vllm/model_executor/layers/fused_moe/configs
      running build_ext
      -- The CXX compiler identification is GNU 11.4.0
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Build type: RelWithDebInfo
      -- Target device: cuda
      -- Found Python: /usr/local/anaconda3/bin/python (found version "3.9.12") found components: Interpreter Development.Module
      -- Found python matching: /usr/local/anaconda3/bin/python.
      -- Found CUDA: /usr/local/cuda-12.1 (found version "12.1")
      -- The CUDA compiler identification is NVIDIA 11.5.119
      -- Detecting CUDA compiler ABI info
      -- Detecting CUDA compiler ABI info - done
      -- Check for working CUDA compiler: /usr/bin/nvcc - skipped
      -- Detecting CUDA compile features
      -- Detecting CUDA compile features - done
      -- Found CUDAToolkit: /usr/include (found version "11.5.119")
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Caffe2: CUDA detected: 12.1
      -- Caffe2: CUDA nvcc is: /usr/local/cuda-12.1/bin/nvcc
      -- Caffe2: CUDA toolkit directory: /usr/local/cuda-12.1
      -- Caffe2: Header version is: 12.1
      -- /usr/local/cuda-12.1/lib64/libnvrtc.so shorthash is d540eb83
      -- USE_CUDNN is set to 0. Compiling without cuDNN support
      -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
      -- Autodetected CUDA architecture(s):  8.6 8.6 8.9 8.9 8.9 8.9
      -- Added CUDA NVCC flags for: -gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89
      CMake Warning at /tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
        static library kineto_LIBRARY-NOTFOUND not found.
      Call Stack (most recent call first):
        /tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
        CMakeLists.txt:67 (find_package)

      -- Found Torch: /tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/torch/lib/libtorch.so
      -- CUDA supported arches: 7.0;7.5;8.0;8.6;8.9;9.0
      -- CUDA target arches: 86-real;89-real
      -- CMake Version: 3.29.3
      -- CUTLASS 3.5.0
      -- CUDART: /usr/local/cuda-12.1/lib64/libcudart.so
      -- CUDA Driver: /usr/local/cuda-12.1/lib64/stubs/libcuda.so
      -- NVRTC: /usr/local/cuda-12.1/lib64/libnvrtc.so
      -- Default Install Location: install
      -- Found Python3: /data2/fanbingbing/.conda/envs/embedding-0522/bin/python3.9 (found suitable version "3.9.19", minimum required is "3.5") found components: Interpreter
      -- CUDA Compilation Architectures: 70;72;75;80;86;87;89;90;90a
      -- Enable caching of reference results in conv unit tests
      -- Enable rigorous conv problem sizes in conv unit tests
      -- Using NVCC flags: --expt-relaxed-constexpr;-DCUTLASS_TEST_LEVEL=0;-DCUTLASS_TEST_ENABLE_CACHED_RESULTS=1;-DCUTLASS_CONV_UNIT_TEST_RIGOROUS_SIZE_ENABLED=1;-DCUTLASS_DEBUG_TRACE_LEVEL=0;-Xcompiler=-Wconversion;-Xcompiler=-fno-strict-aliasing;-lineinfo
      -- CUTLASS Revision: 5f6d10c1
      -- Configuring cublas ...
      -- cuBLAS Disabled.
      -- Configuring cuBLAS ... done.
      -- Completed generation of library instances. See /data2/fanbingbing/Segregation/LLaMA-embedding/vllm/build/temp.linux-x86_64-cpython-39/_deps/cutlass-build/tools/library/library_instance_generation.log for more information.
      -- Punica target arches: 86-real;89-real
      -- Enabling C extension.
      -- Enabling moe extension.

      Traceback (most recent call last):
        File "/data2/fanbingbing/.local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/data2/fanbingbing/.local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/data2/fanbingbing/.local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 410, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 401, in <module>
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 91, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-5vq6sgdl/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
        File "<string>", line 202, in build_extensions
        File "/usr/local/anaconda3/lib/python3.9/subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', '_moe_C', '-j', '128']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for vllm
Failed to build vllm
ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects

  -- Found CUDAToolkit: /usr/include (found version "11.5.119")

Your cuda environment is too complicated. It somehow finds the 11.5 version.

@Fanb1ing I think this bug is related to this commit: [Kernel] Add w8a8 CUTLASS kernels
It build success when I reset the main branch to previous commit [Misc] remove old comments
I guess Cutlass may have some conflict with cuda version or gpu driver version, which needs more analysis. @youkaichao Do you have time to look at this issue?

I don't know how cutlass kernel affects this. You need to clean up the environment until it can finds the right cuda.

My test environment only have one cuda version.
The base docker image is nvcr.io/nvidia/cuda:11.8.0-devel-centos7
nvidia-smi
+-----------------------------------------------------------------------------+
 NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0

and
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

Thank to all you help.
I carefully checked different version of CUDA. It seems that the version of CXX compiler and CUDA compiler are still old, which may be caused by multiple versions downloaded on the server.

Unfortunately, I can't easily delete the old version because it will affect other users.
This web may provide a possible solution which I will try in future: https://blog.kovalevskyi.com/multiple-version-of-cuda-libraries-on-the-same-machine-b9502d50ae77

My test environment only have one cuda version. The base docker image is nvcr.io/nvidia/cuda:11.8.0-devel-centos7
nvidia-smi
+-----------------------------------------------------------------------------+
 NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0

and
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

My test environment only have one cuda version. The base docker image is nvcr.io/nvidia/cuda:11.8.0-devel-centos7
nvidia-smi
+-----------------------------------------------------------------------------+
 NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0

and
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

@youkaichao I solved this issue by upgrading to CUDA version 12.4. Therefore, my guess seems correct: some of the new features in the main branch may not be compatible with CUDA 11.8.

It looks like the arch is set to be compute_90a here https://github.com/vllm-project/vllm/commit/2060e93659f1f63a3d2a76aee61559ccb1fe732e#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20aR206 .
Does cuda 11.8 support compute_90a?

I had this issue as well trying to build on a GCP VM with L4 on cuda 11.8. I had to revert before the above mentioned commit to v0.4.2.

Yeah it probably doesn't make sense to try to compile that using 11.8 since it'll fail. Can that be fixed by conditioning on the cuda version?

Closing as the problem has been solved for OP.


[WIP] Cleanly stop() loop when awaiting promise without fiber

This changeset makes sure we cleanly stop() the loop when awaiting a promise without a fiber. This is especially important for test suites and other use cases where you would mix await() and Loop::run() calls. The await() call may currently suspend the Loop::run() call and leave it in an unpredictable state, possibly leaving socket resources or timers behind.
Builds on top of #15 and #32
Refs #27 and #50

This is going to be a fun one to test out before we can ship it to production 😅

Interestingly enough I'm running into this while making the https://github.com/reactphp/socket/pull/302 passes before opening it up for review. Will check our non-test environments and report back.

Deployed it in a side project to see how it goes, after inspecting the code it shouldn't be an issue but I want to see it running for a while without causing issues first 😅

Hello, hows the testing going @WyriHaximus, is it ready for merging? It would helped us with fixing test suites for react/async: ^4.0.

It would helped us with fixing test suites for react/async: ^4.0.

@tomas-novotny I'm curious, can you give some details what specific use case this would help with? I consider this change a WIP that could potentially have a significant impact (see #27 for context), so I would like to get a better understanding how this would affect the larger ecosystem.

@clue I tried to come up with a simplified version of our tests that were failing without this change, but I was unable to do so.
On further investigation, I found that ^4.0 behaves a little differently than ^3.0, but actually better, so the problem was our wrong test.
So after fixing a few assertions and calling cancelTimer everything was fixed and now it passes our tests.

The testSyncPublishWithAsyncConsume always fails.

These PR changes fix these tests. I have not been able to fix them any other way.
Is there something I am doing wrong or some change that will make it work without these PR changes?

@tomas-novotny Thank you for looking into this and providing reproducible test scripts! Definitely a good start, though I would love to see if we could eventually trim this down to remove any external dependencies to get to the core of this.
The gist here seems to be that your test suite mixes await() and Loop::run() calls which usually should be avoided. This PR here ensures that await() cleanly stops the loop so a following Loop::run() works without issues, but the recommendation to not mix both styles still seems reasonable in either case. We're working on making the Loop opaque in a future ReactPHP v3 (https://github.com/orgs/reactphp/discussions/481), so I wonder what impact this would have on your test cases.

@clue Thank you for your hints how to fix my test suites.
I replaced Loop:run() with awaiting for a deferred promise and it works fine even without this PR changes.
+$deferred = new Deferred();

// failsafe to stop loop
$timers[] = $loop->addTimer(5, static function () use ($loop): void {
-    $loop->stop();
+    $deferred->reject();
});

// periodic check if all messages have been consumed
$timers[] = $loop->addPeriodicTimer(1, static function (TimerInterface $timer) use ($loop, $messages, &$consumedMessages): void {
    if (count($messages) !== count($consumedMessages)) {
        return;
    }

    $loop->cancelTimer($timer);
-   $loop->stop();
+   $deferred->resolve();
});

// finish consuming
await($consumer);

// run loop
-$loop->run();
+await($deferred->promise());


Updated VRTCAL support flags

Thanks for improving the documentation 😃
Please give a short description and check the matching checkboxes to help us review this as quick as possible.
🏷 Type of documentation

[ ] new bid adapter
[ ] update bid adapter
[ ] new feature
[ ] text edit only (wording, typos)
[ ] bugfix (code examples)
[ ] new examples

📋 Checklist

[ ] Related pull requests in prebid.js or server are linked
[ ] For new adapters check submitting your adapter docs

Hello bretg, our update was only to update our officially documented support of gdpr,etc as all the logical decisioning for the gdpr, copppa, etc passed values is handled downstream on our server side. The only reason we saw the need to update the dev docs is because a supply partners mentioned that they read that we don't support things like gdpr traffic,etc.. Thus when we realized that our dev docs were not up to date.

I don't see any changes to your PBJS adapter, nor do I see any reference in the existing adapter to coppa, schain, gdpr, or usp.

Apparently, we were specifically considering our prebid-server adapter...Our team is looking into any special additional needs of our prebid JS now..

Putting this on hold until there's the OP gets back to us.

If only your Prebid Server adapter supports these fields, then you're going to need to split your docs... copy this md file and call one vrtcal(pbs) and the other vrtcal(pbjs) with the appropriate differences.

If only your Prebid Server adapter supports these fields, then you're going to need to split your docs... copy this md file and call one vrtcal(pbs) and the other vrtcal(pbjs) with the appropriate differences.

Both of our adapters support the new data. The PBJS adapter updates have been merged and we've updated the test cases for our Prebid Server adapter (which have been approved) as it already had native support for the data fields, just no test cases...Please proceed with this merge. thank you.

The update for both our PrebidJS and Prebid Server adapters have been merged. Please let us know if anything further is needed for this support flags documentation merge to take place.


Migrate to pganalyze/pg_query_go

It appears that @lfittl moved the pg_query_go repo to a new home in https://github.com/pganalyze/pg_query_go. Since sqlc builds much of its postgres capabilities on this library, we should aim to stay up to date with its development. This would also enable support for postgres 13 additions.

@johanbrandhorst Thats right - note that the v2 release has considerable differences in the parse tree format (its now auto generated from Protocol Buffers), so be aware that this might be a small undertaking to update.
Happy to provide input if you or @kyleconroy have questions :)


SFTP downloads not working

Title of Bug
SFTP downloads not working
Details
SFTP downloads not working. Returns a 404 error.
Steps to reproduce bug

Try downloading an SFTP file

Resolved in #185 .